Qwen Technical Report · HackerPapers discussions

Qwen Technical Report — mentioned by fngjdflmdflg

Wed, 04 Dec 2024 23:28:49 GMT

>Meta calling their approach "open" and what the OSI deems "open" are both flawed and/or misleading.[...] It's gotten to the point that whenever I see a Western publication, I groan. llama weights are open, and Qwen 1.0 was based on llama,[0] so clearly those papers are useful to some people, even if some people want to be angry at Meta for not being open enough for them. In fact I would say Qwen is less useful than llama as their technical reports are not nearly as detailed as Meta's. Also, Why groan just from seeing the origin of a project? Just read it to see if it is open or not. And…

Paper and discussion history. A mention is not an endorsement.

Qwen Technical Report — HN submission

Thu, 30 Nov 2023 18:01:16 GMT

Qwen OSS 7B,14B models; techreport - Code-Qwen, Math-Qwen [sep28'23]

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Qwen Technical Report — mentioned by saurabh20n

Thu, 30 Nov 2023 17:47:47 GMT

Summary from https://arxiv.org/pdf/2309.16609.pdf --- (q: how does one format lists on HN?) * qwen-{1.8B,7B,14B}: * 3 trillion tokens; start with BPE tiktoken, cl100k base vocab, augmented with chinese, numbers split into digits, final vocab 152k. * RoPE - rotary positional embedding * context length 2048 * qwen-14b perf percentages: 66.3 MMLU(5), 72.1 CEval(5), 61.8 GSM8K(8), 24.8 MATH(4), 32.3 HumanEval(0), 40.8 MBPP(3), 53.4 BBH(3); beats LLaMA2-13B on all, but behind LLaMA2-70B on all except CEval, MATH and HumanEval (somewhat surprising) * code-qwen-{7B,14B} * additional 90B code tokens…

Paper and discussion history. A mention is not an endorsement.