Words or Characters? Fine-Grained Gating for Reading Comprehension

A mode for reading comprehension created by Zhilin Yang, Bhuwan Dhingra, Ye Yuan, Junjie Hu, William W. Cohen, Ruslan Salakhutdinov in CMU

• This paper should be read more carefully!
• Goal: The authors present a fine-grained gating mechanism to dynamically combine word-level and character-level representations based on properties of the word and model the interaction between questions and paragraphs for reading comprehension.
• Method: The authors compute a vector gate as a linear projection of the token features followed by a sigmoid avtivation. Then multiplicatively apply the gate to the character-level and word-level representations. Each dimension of the gate controls how much information is flowed from the word-level and character-level representations respectively. The gate is determined by named entity tags, part-of-speech tags, document frequencies, and word-level representations as the features for token properties. The gating mechanism can be generally used to model multiple levels of structure in language, including words, characters, phrases, sentences and paragraphs.
  
• Datasets: Children’s Book Test dataset.
• Tasks: children’s book test dataset and social media tag prediction task.
• Experiments: Their approach can improve the performance on reading comprehension task.
• advantage: Character-level representations are used to alleviate the difficulties of modeling out-of-vocabulary(OOV) tokens.