Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets · HackerPapers discussions

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets — HN submission

Tue, 16 Apr 2024 09:40:28 GMT

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2022)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets — HN submission

Thu, 11 Apr 2024 16:08:51 GMT

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets — mentioned by Veedrac

Sun, 04 Apr 2021 16:21:43 GMT

Heh, I guess this is a difference in perspectives. ML has some really shoddy datasets, like really bad, worse than you think I mean. > The macro-averaged results show that the ratio of correct samples (“C”) ranges from 24% to 87%, with a large variance across the five audited datasets. Particularly severe problems were found in CCAligned and WikiMatrix, with 44 of the 65 languages that we audited for CCAligned containing under 50% correct sentences, and 19 of the 20 in WikiMatrix. In total, 15 of the 205 language specific samples (7.3%) contained not a single correct sentence. Quality at a…

Paper and discussion history. A mention is not an endorsement.

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets — HN submission

Mon, 29 Mar 2021 20:04:09 GMT

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.