mirror of
https://github.com/zenlm/zen-blog.git
synced 2026-11-03 14:11:45 +00:00
zeekay
a30ff37468
b3b6eb0 ran a find-and-replace over the posts forked from the QwenLM
blog that turned "Qwen2" and "Qwen2.5" into "zen": Qwen2-57B-A14B read
as zen7B-A14B, Qwen2-72B as zen-72B, Qwen2AudioForConditionalGeneration
as a class that does not exist, and links such as
github.com/QwenLM/Qwen2 and the qianwen-res image paths pointed at
nothing. Every token is put back from the original post, aligned
against the rewritten one: 536 lines across 29 posts, and 116 of the
example files under public/cases.
These 44 posts are about another lab's releases and now say so with
"upstream: true" in their frontmatter. Two Zen posts lose the
base-model names they carried, and the Zen 3 post stops calling itself
the next generation.
13 lines
3.9 KiB
JSON
13 lines
3.9 KiB
JSON
{
|
||
"title": "Example: QwenVL HTML",
|
||
"messages": [
|
||
{
|
||
"role": "User",
|
||
"content": "QwenVL HTML\n"
|
||
},
|
||
{
|
||
"role": "Qwen2.5-VL",
|
||
"content": "```html\n<html>\n<body>\n<h1 data-bbox=\"879 283 1605 348\">Qwen2.5 Technical Report</h1>\n<p data-bbox=\"1124 406 1360 456\"></p>\n<address class=\"author\"><p>Qwen Team</p></address>\n<div class=\"image\" data-bbox=\"778 491 850 541\"><img data-bbox=\"778 491 850 541\"/></div>\n<p data-bbox=\"885 491 1460 541\">https://huggingface.co/Qwen</p>\n<div class=\"image\" data-bbox=\"778 541 850 589\"><img data-bbox=\"778 541 850 589\"/></div>\n<p data-bbox=\"885 541 1708 589\">https://modelscope.cn/organization/qwen</p>\n<div class=\"image\" data-bbox=\"792 589 850 639\"><img data-bbox=\"792 589 850 639\"/></div>\n<p data-bbox=\"885 589 1584 639\">https://github.com/QwenLM/Qwen2.5</p>\n<h2 data-bbox=\"1143 681 1344 733\">Abstract</h2>\n<p data-bbox=\"434 785 2050 1252\">In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages. In terms of pre-training, we have scaled the high-quality pre-training datasets from the previous 7 trillion tokens to 18 trillion tokens. This provides a strong foundation for common sense, expert knowledge, and reasoning capabilities. In terms of post-training, we implement intricate supervised finetuning with over 1 million samples, as well as multistage reinforcement learning, including offline learning DPO and online learning GRPO. Post-training techniques significantly enhance human preference, and notably improve long text generation, structural data analysis, and instruction following.</p>\n<p data-bbox=\"434 1262 2050 1587\">To handle diverse and varied use cases effectively, we present Qwen2.5 LLM series in rich configurations. The open-weight offerings include base models and instruction-tuned models in sizes of $0.5 \\mathrm{~B}, 1.5 \\mathrm{~B}, 3 \\mathrm{~B}, 7 \\mathrm{~B}, 14 \\mathrm{~B}, 32 \\mathrm{~B}$, and $72 \\mathrm{~B}$ parameters. Quantized versions of the instruction-tuned models are also provided. Over 100 models can be accessed from Hugging Face Hub, ModelScope, and Kaggle. In addition, for hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus, both available from Alibaba Cloud Model Studio.</p>\n<p data-bbox=\"434 1587 2050 2052\">Qwen2.5 has demonstrated top-tier performance on a wide range of benchmarks evaluating language understanding, reasoning, mathematics, coding, human preference alignment, etc. Specifically, the open-weight flagship Qwen2.5-72B-Instruct outperforms a number of open and proprietary models and demonstrates competitive performance to the state-of-the-art open-weight model, Llama-3-405B-Instruct, which is around 5 times larger. Qwen2.5-Turbo and Qwen2.5-Plus offer superior cost-effectiveness while performing competitively against GPT-4o-mini and GPT-4o respectively. Additionally, as the foundation, Qwen2.5 models have been instrumental in training specialized models such as Qwen2.5-Math (Yang et al., 2024b), Qwen2.5-Coder (Hui et al., 2024), QwQ (Qwen Team, 2024d), and multimodal models.</p>\n<div class=\"image\" data-bbox=\"408 2275 2086 2800\"><img data-bbox=\"408 2275 2086 2800\"/></div>\n<p data-bbox=\"289 2864 2202 3058\">Figure 1: In the iterative development of the Qwen series, data scaling has played a crucial role. Qwen 2.5, which leverages 18 trillion tokens for pre-training, has demonstrated the most advanced capabilities within the Qwen series, especially in terms of domain expertise, underscoring the importance of scale together with mixture in enhancing the model’s capabilities.</p>\n</body>\n</html>\n```"
|
||
}
|
||
]
|
||
} |