Name: Multilingual LAMA
Description: multilingual version of lama.  The underlying idea of LAMA is to query knowledge from pretrained LMs using templates without any finetuning
Provider: LMU Munich
Paper Title: Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
Tasks: lm probing
Venue Name: nan
Authors: Nora Kassner, Philipp Dufter, Hinrich Schutze
Abstract: Recently, it has been found that monolingual English language models can be used as
  knowledge bases. Instead of structural knowledge base queries, masked sentences such as
  “Paris is the capital of [MASK]” are used as
  probes. We translate the established benchmarks TREx and GoogleRE into 53 languages.
  Working with mBERT, we investigate three
  questions. (i) Can mBERT be used as a multilingual knowledge base? Most prior work only
  considers English. Extending research to multiple languages is important for diversity and
  accessibility. (ii) Is mBERT’s performance
  as knowledge base language-independent or
  does it vary from language to language? (iii)
  A multilingual model is trained on more text,
  e.g., mBERT is trained on 104 Wikipedias.
  Can mBERT leverage this for better performance? We find that using mBERT as a knowledge base yields varying performance across
  languages and pooling predictions across languages improves performance. Conversely,
  mBERT exhibits a language bias; e.g., when
  queried in Italian, it tends to predict Italy as
  the country of origin.
  