tweet-preprocessing

In this study, we propose a geolocation method based on named entities that are present in user tweets. We try to predict the US states where Twitter users reside in. We present a base method, compare it with a common approach in literature, and also examine some design decisions about our method.

General-purpose regex-based tweet tokenizer that employs pattern matching with a single regular expression

This code can be used to automate the tweet cleansing process.