Japanese Word Similarity Dataset
Data
We built a Japanese word similarity dataset including rare words.
We target verb, adjective, noun and adverb.
Data construction
We constructed our dataset following the Stanford Rare Word Similarity Dataset
(RW) proposed by Luong et al.
(2013).
We extracted pairs of Japanese verbs (including sahen verb) and adjectives
(both i-adjective and na-adjective) from Kodaira et al.
(2016)'s Evaluation dataset for Japanese lexical
simplification.
We employed a crowdsourcing service (Lancers) to
recruite 10 annotators to assign 11 levels of similarity for word pairs.
0 (most dissimilar) - 10 (most similar)
Entry
The sample of the dataset is as follows:
word1, word2, mean(remove_extreme_annotator), sub1, sub2, ..., sub9, sub10, mean, ------------, -------------, ------------, ------------, ------------, ------------, ------------, ------------, ------------, 排除する, 無視する, 4.6, 5, 3, ..., 5, 6, 4.8, 排除する, 除外する, 6.6, 7, 6, ..., 5, 7, 6.8, mean(remove_extreme_annotator) : average of the similarity scores assigned by annotators(the annotator attached an extreme value are removed)
mean : average of the similarity scores assigned by annotators
sub* : the similarity score for each annotator
Helper script in src
The src directory contains a helper script to calculate Spearman's rank
correlation coefficient used in our LREC paper.
Specifically, we learned word vectors from Japanese Wikipedia to
calculatethe rank correlation coefficient between the similarity of
word pairs and mean of annotated scores.
License
Our work is licensed under Creative Commons Attribution-ShareAlike 3.0
Unported (CC BY-SA 3.0).
Citation
If you use our dataset, please cite our LREC paper.
- Yuya Sakaizawa and Mamoru Komachi. Construction of a Japanese Word
Similarity Dataset. In 11th edition of the Language Resources and Evaluation
Conference (LREC 2018), pp.948-951. May 2018. - Yuya Sakaizawa and Mamoru Komachi. Construction of a Japanese Word Similarity Dataset. In arXiv e-prints, 1703.05916 (5 pages). March 2017.
References
- Tomonori Kodaira, Tomoyuki Kajiwara, Mamoru Komachi. Controlled and
Balanced Dataset for Japanese Lexical Simplification. ACL 2016 Student
Research Workshop, pp.1-7. August 2016. - Minh-thang Luong, Richard Socher, Christopher D. Manning. Better Word Representations with Recursive Neural Networks for Morphology. CoNLL 2013, pp.104-113. August 2013.
Tokyo Metropolitan University
Yuya Sakaizawa
e-mail: ksf.doingmorewithless-at-gmail.com