Name: GeoWAC
Description: match the ground-truth geographic
  distribution of each language
Provider: University of Canterbury
Paper Title: Geographically-Balanced Gigaword Corpora for 50 Language Varieties
  
Tasks: dialect identification
Venue Name: International Conference on Language Resources and Evaluation
Authors: Jonathan Dunn,B. Adams
Abstract: While text corpora have been steadily increasing in overall size, even very large corpora are not designed to represent global population demographics. For example, recent work has shown that existing English gigaword corpora over-represent inner-circle varieties from the US and the UK. To correct implicit geographic and demographic biases, this paper uses country-level population demographics to guide the construction of gigaword web corpora. The resulting corpora explicitly match the ground-truth geographic distribution of each language, thus equally representing language users from around the world. This is important because it ensures that speakers of under-resourced language varieties (i.e., Indian English or Algerian French) are represented, both in the corpora themselves but also in derivative resources like word embeddings.