From f16eb4f219b2789aa8d9722fd5777fe0c2eaea3f Mon Sep 17 00:00:00 2001 From: kero Date: Wed, 24 Jun 2026 16:06:47 +0800 Subject: [PATCH 1/3] update appendix --- ..._a_tools_and_frameworks_quick_reference.md | 8 +++--- ...ndix_b_compliance_and_release_checklist.md | 10 +++---- ..._cost_estimation_and_resource_templates.md | 4 +-- ...ppendix_d_paper_to_implementation_guide.md | 2 +- docs/en/appendix_g_datagallery_note.md | 24 ++++++++--------- docs/en/appendix_h_mindspore_note.md | 8 +++--- docs/en/index.md | 4 +-- .../p15_dataagent_semantic_nl2sql_agent.md | 4 +-- ..._a_tools_and_frameworks_quick_reference.md | 10 +++---- ...ndix_b_compliance_and_release_checklist.md | 12 ++++----- ..._cost_estimation_and_resource_templates.md | 6 ++--- ...ppendix_d_paper_to_implementation_guide.md | 2 +- docs/zh/appendix_g_datagallery_note.md | 26 +++++++++---------- docs/zh/appendix_h_mindspore_note.md | 8 +++--- docs/zh/index.md | 2 +- .../p15_dataagent_semantic_nl2sql_agent.md | 4 +-- mkdocs.yml | 8 +++--- 17 files changed, 69 insertions(+), 73 deletions(-) diff --git a/docs/en/appendix_a_tools_and_frameworks_quick_reference.md b/docs/en/appendix_a_tools_and_frameworks_quick_reference.md index 8eae7633..8ea413e3 100644 --- a/docs/en/appendix_a_tools_and_frameworks_quick_reference.md +++ b/docs/en/appendix_a_tools_and_frameworks_quick_reference.md @@ -253,7 +253,7 @@ The later parts of the book connect naturally to tool choices: - Chapters 44-45: pre-training and post-training recipes need batch processing, version governance, experiment tracking, and data cards. - Chapters 46-48: reasoning, multimodal, and generative scenarios need trajectory records, evaluation slices, storage layering, and inference services. - Chapters 38-43: specialized datasets need fact checking, sample schemas, build pipelines, evaluation protocols, compliance audits, and reproducibility boundaries. -- Appendices A-C translate those capabilities into operational checklists and templates for project managers, teaching assistants, platform teams, and maintainers. +- Appendices A-H translate those capabilities into operational checklists and templates for project managers, teaching assistants, platform teams, and maintainers. This reminds readers that appendices are not secondary extras. They translate engineering capabilities from the main text into operational language. @@ -292,8 +292,8 @@ Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Ra Pushkarna M, Zaldivar A, Kjartansson O, Cicconi P, Chen V, Efrat A, Zou Y, Mueller J, Taly A, Ehyaei A, Karkkainen K, Marathe A, Han X, Mittal A, Schuster T, Yarmand M, Sohn H, Dwarakanath N C, McCann B (2022) Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp 1776-1826. https://doi.org/10.1145/3531146.3533231. -DVC Contributors (2026) Data Version Control Documentation. Available at: https://dvc.org/doc. +DVC Contributors (2026) Data Version Control Documentation. https://dvc.org/doc. -MLflow Authors (2026) MLflow Documentation. Available at: https://mlflow.org/docs/latest/. +MLflow Authors (2026) MLflow Documentation. https://mlflow.org/docs/latest/. -Hugging Face (2026) Hugging Face Datasets Documentation. Available at: https://huggingface.co/docs/datasets. +Hugging Face (2026) Hugging Face Datasets Documentation. https://huggingface.co/docs/datasets. diff --git a/docs/en/appendix_b_compliance_and_release_checklist.md b/docs/en/appendix_b_compliance_and_release_checklist.md index ee8d875e..d36a56a5 100644 --- a/docs/en/appendix_b_compliance_and_release_checklist.md +++ b/docs/en/appendix_b_compliance_and_release_checklist.md @@ -311,14 +311,14 @@ Third, long-term risk is reduced not by one approval form but by source records, ## References -National People's Congress of the People's Republic of China (2016) Cybersecurity Law of the People's Republic of China. Available at: https://www.gov.cn/xinwen/2016-11/07/content_5129723.htm. +National People's Congress of the People's Republic of China (2016) Cybersecurity Law of the People's Republic of China. https://www.gov.cn/xinwen/2016-11/07/content_5129723.htm. -National People's Congress of the People's Republic of China (2021a) Data Security Law of the People's Republic of China. Available at: https://www.gov.cn/xinwen/2021-06/11/content_5616919.htm. +National People's Congress of the People's Republic of China (2021a) Data Security Law of the People's Republic of China. https://www.gov.cn/xinwen/2021-06/11/content_5616919.htm. -National People's Congress of the People's Republic of China (2021b) Personal Information Protection Law of the People's Republic of China. Available at: https://www.gov.cn/xinwen/2021-08/20/content_5632486.htm. +National People's Congress of the People's Republic of China (2021b) Personal Information Protection Law of the People's Republic of China. https://www.gov.cn/xinwen/2021-08/20/content_5632486.htm. -National Institute of Standards and Technology (2023) AI Risk Management Framework (AI RMF 1.0). Available at: https://www.nist.gov/itl/ai-risk-management-framework. +National Institute of Standards and Technology (2023) AI Risk Management Framework (AI RMF 1.0). https://www.nist.gov/itl/ai-risk-management-framework. -European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Available at: https://eur-lex.europa.eu/eli/reg/2024/1689/oj. +European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). https://eur-lex.europa.eu/eli/reg/2024/1689/oj. Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji I D, Gebru T (2019) Model Cards for Model Reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp 220-229. https://doi.org/10.1145/3287560.3287596. diff --git a/docs/en/appendix_c_cost_estimation_and_resource_templates.md b/docs/en/appendix_c_cost_estimation_and_resource_templates.md index 1ef93d8c..f2087395 100644 --- a/docs/en/appendix_c_cost_estimation_and_resource_templates.md +++ b/docs/en/appendix_c_cost_estimation_and_resource_templates.md @@ -319,6 +319,6 @@ Narayanan D, Shoeybi M, Casper J, LeGresley P, Patwary M, Catanzaro B (2021) Eff Kwon W, Li Z, Zhuang S, Sheng Y, Zheng L, Yu C H, Gonzalez J E, Zhang H, Stoica I (2023) Efficient Memory Management for Large Language Model Serving with PagedAttention. In: Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, pp 611-626. https://doi.org/10.1145/3600006.3613165. -Kubernetes Authors (2026) Kubernetes Documentation. Available at: https://kubernetes.io/docs/. +Kubernetes Authors (2026) Kubernetes Documentation. https://kubernetes.io/docs/. -vLLM Project (2026) vLLM Documentation. Available at: https://docs.vllm.ai/. +vLLM Project (2026) vLLM Documentation. https://docs.vllm.ai/. diff --git a/docs/en/appendix_d_paper_to_implementation_guide.md b/docs/en/appendix_d_paper_to_implementation_guide.md index b5ee42c1..9862cf51 100644 --- a/docs/en/appendix_d_paper_to_implementation_guide.md +++ b/docs/en/appendix_d_paper_to_implementation_guide.md @@ -178,7 +178,7 @@ For a chapter-level reproduction repository, consider this structure: | `data/` | Data snapshot, index, and version note | | `src/` | Core implementation | | `configs/` | Parameters, paths, run configuration | -| `scripts/` | One-command run scripts | +| `scripts/` | Reproducible run scripts | | `eval/` | Evaluation scripts and slice reports | | `docs/` | Documentation, boundary notes, FAQ | | `reports/` | Result charts, acceptance screenshots, postmortems | diff --git a/docs/en/appendix_g_datagallery_note.md b/docs/en/appendix_g_datagallery_note.md index e3240a83..dd6988bc 100644 --- a/docs/en/appendix_g_datagallery_note.md +++ b/docs/en/appendix_g_datagallery_note.md @@ -1,18 +1,18 @@ -# Appendix G: DataGallery Open-source Ecosystem Overview +# Appendix G: DataGallery Open-source Ecosystem and Reproduction Notes ## G.1 Purpose of This Appendix -This appendix explains where DataGallery sits in Project 15 and in the agentic data engineering practices discussed in this book. The public open-source entry for DataGallery is hosted on GitCode at [https://gitcode.com/datagallery](https://gitcode.com/datagallery). This appendix is not an installation manual for DataGallery or DataAgent, nor does it replace the README, example configurations, dependency notes, or release records in the corresponding repositories. When reproducing an experiment or integrating the project into an engineering system, readers should treat the public repository, a specific tag or commit, and the project documentation as the source of truth (DataGallery Contributors 2026a; DataGallery Contributors 2026b). +This appendix explains where DataGallery sits in Project 15 and in the agentic data engineering practices discussed in this book. The public open-source entry for DataGallery is hosted on GitCode at [https://gitcode.com/datagallery](https://gitcode.com/datagallery). This appendix is not an installation manual for DataGallery or DataAgent, nor does it replace the README, example configurations, dependency notes, or release records in the corresponding repositories. When reproducing an experiment or integrating the project into an engineering system, readers should treat the public repository, a specific tag or commit, and the project documentation as the source of truth. -In this book, DataGallery is best understood as a set of open-source engineering entry points around Data + AI practice, rather than as the name of one isolated tool. Project 15 uses DataAgent to build an enterprise semantic BI assistant. Its core question is not "how to call a model to generate SQL," but how to organize business questions, the semantic layer, an NL2SQL sub-agent, tool calls, workspace assets, runtime traces, and service interfaces into a reviewable data engineering system. DataGallery provides the open-source home for this kind of work, while DataAgent is the project entry most directly referenced in this book. +In this book, DataGallery is best understood as a set of open-source engineering entry points around Data + AI practice, rather than as the name of one isolated tool. Project 15 uses DataAgent to build an enterprise semantic BI assistant. Its core question is not "how to call a model to generate SQL," but how to organize business questions, the semantic layer, an NL2SQL sub-agent, tool calls, workspace assets, runtime traces, and service interfaces into a reviewable data engineering system. DataGallery provides the public organization and project entry, while DataAgent is the engineering project most directly referenced in this book. -This appendix therefore focuses on the relationship between DataGallery and the book's data engineering methods: how it helps readers connect the chapters on agents, tool use, semantic layers, DataOps, reproduction, and governance to runnable, auditable, and iterative open-source projects. For concrete APIs, startup commands, configuration fields, and dependency versions, this appendix gives usage principles rather than a step-by-step tutorial. +This appendix therefore focuses on the relationship between DataGallery and the book's data engineering methods: how the chapters on agents, tool use, semantic layers, DataOps, reproduction, and governance can be connected to runnable, auditable, and iterative open-source projects. For concrete APIs, startup commands, configuration fields, and dependency versions, this appendix gives usage principles rather than a step-by-step tutorial. ## G.2 Relationship to Project 15 Project 15 uses DataAgent as the practical object for an enterprise semantic BI assistant because it places agent orchestration, semantic-layer enhancement, NL2SQL, tool use, workspace assets, and service interfaces in one engineering chain. DataGallery provides the higher-level open-source organization entry, so DataAgent can be understood not as an isolated repository, but as part of an ecosystem for data agents, data applications, and reproducible data engineering. -This distinction matters. DataAgent is the concrete project: readers need to inspect its YAML configuration, Semantic Service, NL2SQL sub-agent, execution tools, workspace, and A2A interface. DataGallery is the ecosystem entry: readers should use it to confirm project ownership, public repositories, license status, maintenance state, related projects, and future migration clues. For a book chapter, the main text should explain the project chain; an appendix is the right place to explain the open-source ecosystem and reproduction boundary. +This boundary should remain explicit. DataAgent is the concrete project: reproduction requires checking its YAML configuration, Semantic Service, NL2SQL sub-agent, execution tools, workspace, and A2A interface. DataGallery is the ecosystem entry: readers can use it to confirm project ownership, public repositories, license status, maintenance state, related projects, and future migration clues. For a book chapter, the main text should explain the project chain; an appendix is the right place to explain the open-source ecosystem and reproduction boundary. As a reading path, Project 15 and this appendix should be read together. Project 15 answers "how can DataAgent be used to build an enterprise semantic BI assistant?" This appendix answers "how is that project positioned, reproduced, and governed inside the DataGallery open-source ecosystem?" Together they connect the case to the ecosystem. @@ -20,13 +20,13 @@ As a reading path, Project 15 and this appendix should be read together. Project From a data engineering perspective, DataGallery's value is not to hide a complex system behind a black box. Its value is to give reproducible projects a public organizational boundary. A mature data engineering case usually needs to answer seven questions: where to obtain the project, whether the code is open source, what license applies, whether example data and configuration are reproducible, whether outputs are persisted, whether failure samples and logs can be reviewed, and how later version changes are tracked. As a public entry, DataGallery helps readers check these questions in one place. -DataGallery also reminds readers that the key assets of an open-source data-agent project are not only model-calling code. For a system such as DataAgent, the long-lived assets include semantic-layer schemas, tool configurations, database connections, execution permissions, workspace directories, runtime traces, test cases, evaluation scripts, and pre-launch gates. Only when these assets are organized can an agent move from demo capability to engineering capability. +This also means that the key assets of an open-source data-agent project are not only model-calling code. For a system such as DataAgent, the long-lived assets include semantic-layer schemas, tool configurations, database connections, execution permissions, workspace directories, runtime traces, test cases, evaluation scripts, and pre-launch gates. Only when these assets are organized can an agent move from demo capability to engineering capability. In team collaboration, DataGallery can also serve as a cross-role communication entry. Algorithm engineers care about models, prompts, and tool choices. Data engineers care about schemas, samples, sources, quality gates, and result assets. Platform engineers care about environments, permissions, service interfaces, and runtime audit. Business users care about metric definitions, query boundaries, and result interpretation. An open repository and organization entry let these roles collaborate around the same versions, documents, and issue records instead of scattered temporary notes. ## G.4 Technical Map of the DataGallery Ecosystem -The DataGallery organization profile describes the ecosystem as a Data + AI open-source organization that reconstructs the data engineering chain through an agentic paradigm (DataGallery Contributors 2026a). In engineering terms, it can be read as a layered system rather than a single executable package. Its technical map contains four pillars. +From the public organization page and project documentation, DataGallery can be read as a Data + AI open-source practice that organizes data engineering through an agentic paradigm. In engineering terms, it can be read as a layered system rather than a single executable package. Its technical map contains four pillars. The first pillar is **DataAgent**, which is the currently open-source execution engine. It carries NL2SQL, data analysis, feature engineering, tool calling, workspace asset persistence, and service exposure. This is the part most directly used by Project 15. @@ -38,9 +38,9 @@ The fourth pillar is an **evaluation framework** for data-intelligence tasks. Fo These four pillars correspond closely to the structure of this book. The semantic layer connects to the chapters on metadata, data catalogs, RAG, and data products. DataAgent connects to the chapters on tool use, multi-turn interaction, agent architecture, and project delivery. Self-evolution connects to DataOps and feedback loops. Evaluation connects to data quality, benchmark construction, acceptance gates, and reproducibility. -## G.5 DataAgent as the First Open-source Entry +## G.5 DataAgent as the Current Primary Open-source Entry -DataAgent is the open-source project that currently gives readers the most concrete way to reproduce DataGallery's engineering ideas. Its public README presents it as an enterprise Data + AI agent platform with Python 3.11+, Apache License 2.0, LangGraph integration, openJiuwen integration, GaussVector-oriented semantic retrieval support, and a versioned open-source package entry (DataGallery Contributors 2026b). +DataAgent is currently a suitable open-source project for reproducing DataGallery's engineering ideas. The public project documentation presents it as an enterprise Data + AI agent platform with Python 3.11+, Apache License 2.0, LangGraph integration, openJiuwen integration, GaussVector-oriented semantic retrieval support, and a versioned open-source package entry. From the repository structure and documentation, DataAgent can be understood through five engineering layers. @@ -94,8 +94,8 @@ Third, turn the reproduction chain into an engineering checklist. Record the ver Fourth, continuously synchronize with repository changes. After an open-source project changes, compare configuration fields, service interfaces, dependency versions, and example paths, then update the companion reproduction notes or teaching materials. -## References +## Open-source Entries -DataGallery Contributors (2026a) DataGallery organization page. Available at: https://gitcode.com/datagallery. +- DataGallery organization entry: [https://gitcode.com/datagallery](https://gitcode.com/datagallery). -DataGallery Contributors (2026b) DataAgent source repository. Available at: https://gitcode.com/datagallery/DataAgent. +- DataAgent project repository: [https://gitcode.com/datagallery/dataagent](https://gitcode.com/datagallery/dataagent). diff --git a/docs/en/appendix_h_mindspore_note.md b/docs/en/appendix_h_mindspore_note.md index 5ff7a8d1..b755f9a1 100644 --- a/docs/en/appendix_h_mindspore_note.md +++ b/docs/en/appendix_h_mindspore_note.md @@ -250,10 +250,10 @@ This acknowledgment explains the collaboration background and resource sources f ## References -MindFace Contributors (2026) MindFace source repository. Available at: https://github.com/mindspore-lab/mindface. +MindFace Contributors (2026) MindFace source repository. https://github.com/mindspore-lab/mindface. -MindSpore Contributors (2026a) MindSpore Documentation. Available at: https://www.mindspore.cn/view/en. +MindSpore Contributors (2026a) MindSpore Documentation. https://www.mindspore.cn/view/en. -MindSpore Contributors (2026b) MindSpore source repository. Available at: https://github.com/mindspore-ai/mindspore. +MindSpore Contributors (2026b) MindSpore source repository. https://github.com/mindspore-ai/mindspore. -MindSpore Contributors (2026c) Automatic Differentiation, MindSpore Tutorials. Available at: https://www.mindspore.cn/tutorials/en/r2.9.0/beginner/autograd.html. +MindSpore Contributors (2026c) Automatic Differentiation, MindSpore Tutorials. https://www.mindspore.cn/tutorials/en/r2.9.0/beginner/autograd.html. diff --git a/docs/en/index.md b/docs/en/index.md index cb274b15..53f617ce 100644 --- a/docs/en/index.md +++ b/docs/en/index.md @@ -30,7 +30,7 @@ The current Chinese mainline uses the 2026 Springer-size publication structure. - [Appendix D: From Paper to Implementation Guide](appendix_d_paper_to_implementation_guide.md) - [Appendix E: Common Data-Engineering Bug Debugging Manual](appendix_e_common_bug_debugging_manual.md) - [Appendix F: Terminology and Chinese-English Mapping](appendix_f_terminology_and_chinese_english_mapping.md) -- [Appendix G: DataGallery Open-source Ecosystem Overview](appendix_g_datagallery_note.md) +- [Appendix G: DataGallery Open-source Ecosystem and Reproduction Notes](appendix_g_datagallery_note.md) - [Appendix H: MindSpore Technical Appendix and Acknowledgments](appendix_h_mindspore_note.md) ## Part-by-Part Contents @@ -190,5 +190,5 @@ This part connects acquisition, cleaning, synthesis, RAG, agents, DataOps, priva - [Appendix D: From Paper to Implementation Guide](appendix_d_paper_to_implementation_guide.md) - [Appendix E: Common Data-Engineering Bug Debugging Manual](appendix_e_common_bug_debugging_manual.md) - [Appendix F: Terminology and Chinese-English Mapping](appendix_f_terminology_and_chinese_english_mapping.md) -- [Appendix G: DataGallery Open-source Ecosystem Overview](appendix_g_datagallery_note.md) +- [Appendix G: DataGallery Open-source Ecosystem and Reproduction Notes](appendix_g_datagallery_note.md) - [Appendix H: MindSpore Technical Appendix and Acknowledgments](appendix_h_mindspore_note.md) diff --git a/docs/en/part14/p15_dataagent_semantic_nl2sql_agent.md b/docs/en/part14/p15_dataagent_semantic_nl2sql_agent.md index 49408f04..c771d37d 100644 --- a/docs/en/part14/p15_dataagent_semantic_nl2sql_agent.md +++ b/docs/en/part14/p15_dataagent_semantic_nl2sql_agent.md @@ -238,7 +238,7 @@ This chapter depends on DataAgent, Semantic Service, a value-match service, and ### 5.2 Install the Project -DataAgent is the currently open-source execution engine in the DataGallery open-source ecosystem (DataGallery Contributors, 2026a). The DataGallery open-source entry is [https://gitcode.com/datagallery](https://gitcode.com/datagallery), and the DataAgent source repository is [https://gitcode.com/datagallery/dataagent](https://gitcode.com/datagallery/dataagent) (DataGallery Contributors, 2026b). For DataGallery's broader technical map, reproduction boundaries, and project-governance usage, see [Appendix G: DataGallery Open-source Ecosystem Overview](../appendix_g_datagallery_note.md). +DataAgent is one of the primary execution engines in the DataGallery open-source ecosystem. The DataGallery open-source entry is [https://gitcode.com/datagallery](https://gitcode.com/datagallery), and the DataAgent source repository is [https://gitcode.com/datagallery/dataagent](https://gitcode.com/datagallery/dataagent). For DataGallery's broader technical map, reproduction boundaries, and project-governance usage, see [Appendix G: DataGallery Open-source Ecosystem and Reproduction Notes](../appendix_g_datagallery_note.md). First pin the version, then install dependencies from the repository root: @@ -931,5 +931,3 @@ As part of Part 14, this chapter validates earlier methods at the project level. 3. Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761. 4. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. 5. dbt Labs. (2026). dbt Documentation. https://docs.getdbt.com/. -6. DataGallery Contributors. (2026a). DataGallery organization page. https://gitcode.com/datagallery. -7. DataGallery Contributors. (2026b). DataAgent source repository. https://gitcode.com/datagallery/dataagent. diff --git a/docs/zh/appendix_a_tools_and_frameworks_quick_reference.md b/docs/zh/appendix_a_tools_and_frameworks_quick_reference.md index 7a10426a..ebaab57a 100644 --- a/docs/zh/appendix_a_tools_and_frameworks_quick_reference.md +++ b/docs/zh/appendix_a_tools_and_frameworks_quick_reference.md @@ -251,9 +251,9 @@ Agent 工具调用数据与普通问答数据最大的不同，是其中的中 - 面向预训练与后训练配方，优先强调批处理、版本治理、实验追踪和数据卡片。 - 面向推理、多模态与生成场景，优先强调轨迹记录、评测切片、存储分层和推理服务。 - 面向专项数据集案例，优先强调事实核验、样本 schema、构建流水线、评测协议、合规审计和复现边界。 -- 面向附录 A-H，重点承担“把这些工具能力翻译成可执行清单和模板”的职责。 +- 面向附录 A-H，重点承担“把这些工具能力转化为可执行清单和模板”的职责。 -这样的映射有一个很重要的好处：它提醒读者，附录并不是正文之外的次要补充，而是把正文里出现过的工程能力重新整理成能直接被项目经理、助教、平台同学和维护者调用的操作层语言。 +这样的映射提醒读者，附录并不是正文之外的次要补充，而是把正文里出现过的工程能力重新整理成能被项目经理、助教、平台工程师和维护者直接调用的操作层语言。 ## A.9 本附录建议长期维护的速查字段 @@ -293,8 +293,8 @@ Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Ra Pushkarna M, Zaldivar A, Kjartansson O (2022) Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp 1776-1826. -DVC Contributors (2026) Data Version Control Documentation. Available at: https://dvc.org/doc. +DVC Contributors (2026) Data Version Control Documentation. https://dvc.org/doc. -MLflow Authors (2026) MLflow Documentation. Available at: https://mlflow.org/docs/latest/. +MLflow Authors (2026) MLflow Documentation. https://mlflow.org/docs/latest/. -Hugging Face (2026) Hugging Face Datasets Documentation. Available at: https://huggingface.co/docs/datasets. +Hugging Face (2026) Hugging Face Datasets Documentation. https://huggingface.co/docs/datasets. diff --git a/docs/zh/appendix_b_compliance_and_release_checklist.md b/docs/zh/appendix_b_compliance_and_release_checklist.md index 68031ff6..964af227 100644 --- a/docs/zh/appendix_b_compliance_and_release_checklist.md +++ b/docs/zh/appendix_b_compliance_and_release_checklist.md @@ -244,7 +244,7 @@ | public_notice | 对外公告链接或说明 | | preventive_action | 后续预防措施 | -把事故台账制度化的意义，不在于“显得正式”，而在于防止同一类问题在不同学期、不同项目、不同责任人之间重复发生。 +事故台账制度化的意义不在于增加流程形式，而在于防止同一类问题跨学期、跨项目、跨责任人重复发生。 ### B.8.3 高风险红旗信号 @@ -309,14 +309,14 @@ ## 参考文献 -National People's Congress of the People's Republic of China (2016) Cybersecurity Law of the People's Republic of China. Available at: http://www.npc.gov.cn/zgrdw/npc/xinwen/2016-11/07/content_2001605.htm. +National People's Congress of the People's Republic of China (2016) Cybersecurity Law of the People's Republic of China. http://www.npc.gov.cn/zgrdw/npc/xinwen/2016-11/07/content_2001605.htm. -National People's Congress of the People's Republic of China (2021a) Data Security Law of the People's Republic of China. Available at: https://www.cac.gov.cn/2021-06/11/c_1624994566919140.htm. +National People's Congress of the People's Republic of China (2021a) Data Security Law of the People's Republic of China. https://www.cac.gov.cn/2021-06/11/c_1624994566919140.htm. -National People's Congress of the People's Republic of China (2021b) Personal Information Protection Law of the People's Republic of China. Available at: http://www.npc.gov.cn/npc/c2/c30834/202108/t20210820_313088.html. +National People's Congress of the People's Republic of China (2021b) Personal Information Protection Law of the People's Republic of China. http://www.npc.gov.cn/npc/c2/c30834/202108/t20210820_313088.html. -National Institute of Standards and Technology (2023) AI Risk Management Framework (AI RMF 1.0). Available at: https://www.nist.gov/itl/ai-risk-management-framework. +National Institute of Standards and Technology (2023) AI Risk Management Framework (AI RMF 1.0). https://www.nist.gov/itl/ai-risk-management-framework. -European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Available at: https://eur-lex.europa.eu/eli/reg/2024/1689/oj. +European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). https://eur-lex.europa.eu/eli/reg/2024/1689/oj. Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji I D, Gebru T (2019) Model Cards for Model Reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp 220-229. https://doi.org/10.1145/3287560.3287596. diff --git a/docs/zh/appendix_c_cost_estimation_and_resource_templates.md b/docs/zh/appendix_c_cost_estimation_and_resource_templates.md index 5eb1751b..ba0db331 100644 --- a/docs/zh/appendix_c_cost_estimation_and_resource_templates.md +++ b/docs/zh/appendix_c_cost_estimation_and_resource_templates.md @@ -8,7 +8,7 @@ ## C.2 成本为什么要按生命周期拆开 -很多预算争议的根源，不是总额高低，而是不同角色盯的成本对象不同。研究同学可能只盯训练 GPU，平台同学更关心存储和回流，课程负责人更关心镜像和助教工时，管理层则关心季度投入与交付节奏。若没有统一拆分，讨论会不断失焦。 +很多预算争议的根源，不是总额高低，而是不同角色关注的成本对象不同。研究人员可能只关注训练 GPU，平台工程师更关心存储和回流，课程负责人更关心镜像和助教工时，管理层则关心季度投入与交付节奏。若没有统一拆分，讨论会不断失焦。 一个更适合数据工程项目的拆法，是按生命周期拆成六类成本： @@ -319,6 +319,6 @@ Narayanan D, Shoeybi M, Casper J, LeGresley P, Patwary M, Catanzaro B (2021) Eff Kwon W, Li Z, Zhuang S, Sheng Y, Zheng L, Yu C H, Gonzalez J E, Zhang H, Stoica I (2023) Efficient Memory Management for Large Language Model Serving with PagedAttention. In: Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, pp 611-626. https://doi.org/10.1145/3600006.3613165. -Kubernetes Authors (2026) Kubernetes Documentation. Available at: https://kubernetes.io/docs/. +Kubernetes Authors (2026) Kubernetes Documentation. https://kubernetes.io/docs/. -vLLM Project (2026) vLLM Documentation. Available at: https://docs.vllm.ai/. +vLLM Project (2026) vLLM Documentation. https://docs.vllm.ai/. diff --git a/docs/zh/appendix_d_paper_to_implementation_guide.md b/docs/zh/appendix_d_paper_to_implementation_guide.md index cb350e0d..11d31b0e 100644 --- a/docs/zh/appendix_d_paper_to_implementation_guide.md +++ b/docs/zh/appendix_d_paper_to_implementation_guide.md @@ -178,7 +178,7 @@ | `data/` | 数据快照、索引和版本说明 | | `src/` | 核心实现 | | `configs/` | 参数、路径、运行配置 | -| `scripts/` | 一键运行脚本 | +| `scripts/` | 可重复运行脚本 | | `eval/` | 评测脚本和切片报告 | | `docs/` | 说明文档、边界说明、FAQ | | `reports/` | 结果图表、验收截图、复盘记录 | diff --git a/docs/zh/appendix_g_datagallery_note.md b/docs/zh/appendix_g_datagallery_note.md index c696257a..79c559e6 100644 --- a/docs/zh/appendix_g_datagallery_note.md +++ b/docs/zh/appendix_g_datagallery_note.md @@ -1,18 +1,18 @@ -# 附录G：DataGallery 开源生态简介 +# 附录G：DataGallery 开源生态与复现说明 ## G.1 附录定位 -本附录用于说明 DataGallery 在本书项目十五及相关 Agent 数据工程实践中的位置。DataGallery 的公开开源入口位于 GitCode：[https://gitcode.com/datagallery](https://gitcode.com/datagallery)。本附录不是 DataGallery 或 DataAgent 的安装手册，也不替代对应仓库中的 README、示例配置、依赖说明和发布记录；读者在复现实验或接入工程系统时，应以公开仓库、具体 tag、commit 和项目文档为准（DataGallery Contributors 2026a; DataGallery Contributors 2026b）。 +本附录用于说明 DataGallery 在本书项目十五及相关 Agent 数据工程实践中的位置。DataGallery 的公开开源入口位于 GitCode：[https://gitcode.com/datagallery](https://gitcode.com/datagallery)。本附录不是 DataGallery 或 DataAgent 的安装手册，也不替代对应仓库中的 README、示例配置、依赖说明和发布记录；读者在复现实验或接入工程系统时，应以公开仓库、具体 tag、commit 和项目文档为准。 -在本书语境中，DataGallery 更适合被理解为围绕 Data + AI 实践组织的一组开源工程入口，而不是单一工具名称。项目十五使用 DataAgent 构建企业级语义问数助手，其核心问题并不是“如何调用一个模型生成 SQL”，而是如何把业务问题、语义层、NL2SQL 子 Agent、工具调用、workspace 资产、运行轨迹和服务接口组织成可复查的数据工程系统。DataGallery 为这类工作提供开源承载位置，DataAgent 则是本书最直接引用的项目入口。 +在本书语境中，DataGallery 更适合被理解为围绕 Data + AI 实践组织的一组开源工程入口，而不是单一工具名称。项目十五使用 DataAgent 构建企业级语义问数助手，其核心问题并不是“如何调用一个模型生成 SQL”，而是如何把业务问题、语义层、NL2SQL 子 Agent、工具调用、workspace 资产、运行轨迹和服务接口组织成可复查的数据工程系统。DataGallery 提供公开组织和项目入口，DataAgent 则是本书最直接引用的工程项目。 -因此，本附录重点讨论 DataGallery 与数据工程方法之间的关系：它如何帮助读者把书中关于 Agent、Tool-Use、语义层、DataOps、复现和治理的内容，落到可运行、可审计、可迭代的开源项目中。对于具体 API、启动命令、配置文件字段和依赖版本，本附录只给出使用原则，不展开逐项教程。 +因此，本附录重点讨论 DataGallery 与数据工程方法之间的关系：如何把书中关于 Agent、Tool-Use、语义层、DataOps、复现和治理的内容，落到可运行、可审计、可迭代的开源项目中。对于具体 API、启动命令、配置文件字段和依赖版本，本附录只给出使用原则，不展开逐项教程。 ## G.2 与项目十五的关系 项目十五选择 DataAgent 作为企业语义问数助手的实践对象，是因为它把 Agent 编排、语义层增强、NL2SQL、工具调用、workspace 资产和服务化接口放在同一条工程链路中。DataGallery 则提供了更上层的开源组织入口，使 DataAgent 不只是孤立仓库，而是可以被放入数据智能体、数据应用和数据工程复现的生态视角中理解。 -这一区分很重要。DataAgent 是具体项目：读者需要关注它的 YAML 配置、Semantic Service、NL2SQL 子 Agent、执行工具、workspace 和 A2A 接口。DataGallery 是生态入口：读者应通过它确认项目归属、公开仓库、许可证、维护状态、相关项目和后续迁移线索。对于出版物来说，章节正文适合讲清项目链路，附录则适合说明开源生态和复现边界。 +这一边界需要明确。DataAgent 是具体项目：复现时需要检查它的 YAML 配置、Semantic Service、NL2SQL 子 Agent、执行工具、workspace 和 A2A 接口。DataGallery 是生态入口：读者可通过它确认项目归属、公开仓库、许可证、维护状态、相关项目和后续迁移线索。对于出版物来说，章节正文适合讲清项目链路，附录则适合说明开源生态和复现边界。 在阅读路径上，建议将项目十五和本附录连读。项目十五回答“如何用 DataAgent 构建一个企业语义问数助手”；本附录回答“这个项目在 DataGallery 开源生态中如何被定位、复现和治理”。两者共同构成从案例到生态的说明。 @@ -20,13 +20,13 @@ 从数据工程角度看，DataGallery 的价值不在于把复杂系统包装成一个黑盒，而在于为可复现项目提供公开组织边界。一个成熟的数据工程案例通常需要回答七类问题：项目从哪里获取，代码是否开源，许可证是什么，示例数据和配置是否可复现，运行结果是否能落盘，失败样本和日志是否能回看，后续版本变化如何追踪。DataGallery 作为公开入口，可以帮助读者把这些问题集中到同一处核对。 -DataGallery 也提醒读者，开源数据智能体项目的核心资产不只是模型调用代码。对 DataAgent 这类系统而言，真正需要长期维护的是语义层 schema、工具配置、数据库连接、执行权限、workspace 目录、运行轨迹、测试样例、评估脚本和上线前门禁。只有这些资产被组织起来，Agent 才能从演示能力变成工程能力。 +这一点也意味着，开源数据智能体项目的核心资产不只是模型调用代码。对 DataAgent 这类系统而言，需要长期维护的是语义层 schema、工具配置、数据库连接、执行权限、workspace 目录、运行轨迹、测试样例、评估脚本和上线前门禁。只有这些资产被组织起来，Agent 才能从演示能力变成工程能力。 -在团队协作中，DataGallery 还可以作为跨角色沟通入口。算法同学关注模型、提示词和工具选择；数据工程同学关注 schema、样本、数据源、质量门禁和结果资产；平台同学关注环境、权限、服务接口和运行审计；业务同学关注指标口径、查询边界和结果解释。开源仓库和组织入口使这些角色能够围绕同一套版本、文档和 issue 记录协作，而不是依赖分散的临时说明。 +在团队协作中，DataGallery 还可以作为跨角色沟通入口。算法工程师关注模型、提示词和工具选择；数据工程师关注 schema、样本、数据源、质量门禁和结果资产；平台工程师关注环境、权限、服务接口和运行审计；业务人员关注指标口径、查询边界和结果解释。开源仓库和组织入口使这些角色能够围绕同一套版本、文档和 issue 记录协作，而不是依赖分散的临时说明。 ## G.4 DataGallery 生态的技术版图 -DataGallery 组织介绍将其定位为一个面向 Data + AI 的开源组织，目标是用 Agent 范式重构数据工程全链路（DataGallery Contributors 2026a）。从工程角度看，它不是一个单独可执行包，而更像一个分层生态。其技术版图可以概括为四个支柱。 +从公开组织页和项目文档看，DataGallery 面向 Data + AI 开源实践，强调用 Agent 范式组织数据工程链路。从工程角度看，它不是一个单独可执行包，而更像一个分层生态。其技术版图可以概括为四个支柱。 第一个支柱是 **DataAgent**，也就是当前已经开源的执行引擎。它承载 NL2SQL、数据分析、特征工程、工具调用、workspace 资产落盘和服务化暴露能力，是项目十五最直接使用的部分。 @@ -38,9 +38,9 @@ DataGallery 组织介绍将其定位为一个面向 Data + AI 的开源组织， 这四个支柱与本书结构高度对应。语义层连接元数据、数据目录、RAG 和数据产品章节；DataAgent 连接工具使用、多轮交互、Agent 架构和项目交付章节；自演进连接 DataOps 和反馈闭环；评测框架连接数据质量、benchmark 构建、验收门禁和可复现性。 -## G.5 DataAgent 作为当前首个开源入口 +## G.5 DataAgent 作为当前主要开源入口 -DataAgent 是目前最能让读者复现 DataGallery 工程思想的开源项目。其公开 README 将它描述为面向企业 Data + AI 场景的智能体平台，要求 Python 3.11+，采用 Apache License 2.0，集成 LangGraph、openJiuwen，并优先支持围绕 GaussVector 的语义检索增强和版本化开源包入口（DataGallery Contributors 2026b）。 +DataAgent 是当前较适合读者复现 DataGallery 工程思想的开源项目。公开项目文档显示，它面向企业 Data + AI 场景，要求 Python 3.11+，采用 Apache License 2.0，集成 LangGraph、openJiuwen，并支持围绕 GaussVector 的语义检索增强和版本化开源包入口。 结合仓库结构和文档，DataAgent 可以拆成五层理解。 @@ -94,8 +94,8 @@ DataGallery 与本书多处内容有自然连接。第六篇讨论推理与 Agen 第四，持续同步仓库变化。开源项目更新后，应比较配置字段、服务接口、依赖版本和示例路径的变化，并更新本书配套复现说明或教学材料。 -## 参考文献 +## 开源入口 -DataGallery Contributors (2026a) DataGallery organization page. Available at: https://gitcode.com/datagallery. +- DataGallery 组织入口：[https://gitcode.com/datagallery](https://gitcode.com/datagallery)。 -DataGallery Contributors (2026b) DataAgent source repository. Available at: https://gitcode.com/datagallery/DataAgent. +- DataAgent 项目仓库：[https://gitcode.com/datagallery/dataagent](https://gitcode.com/datagallery/dataagent)。 diff --git a/docs/zh/appendix_h_mindspore_note.md b/docs/zh/appendix_h_mindspore_note.md index 758aea74..de7f1663 100644 --- a/docs/zh/appendix_h_mindspore_note.md +++ b/docs/zh/appendix_h_mindspore_note.md @@ -250,10 +250,10 @@ MindSpore 的分布式训练能力通常可以从多个并行维度理解。数 ## 参考文献 -MindFace Contributors (2026) MindFace source repository. Available at: https://github.com/mindspore-lab/mindface. +MindFace Contributors (2026) MindFace source repository. https://github.com/mindspore-lab/mindface. -MindSpore Contributors (2026a) MindSpore Documentation. Available at: https://www.mindspore.cn/view/en. +MindSpore Contributors (2026a) MindSpore Documentation. https://www.mindspore.cn/view/en. -MindSpore Contributors (2026b) MindSpore source repository. Available at: https://github.com/mindspore-ai/mindspore. +MindSpore Contributors (2026b) MindSpore source repository. https://github.com/mindspore-ai/mindspore. -MindSpore Contributors (2026c) Automatic Differentiation, MindSpore Tutorials. Available at: https://www.mindspore.cn/tutorials/en/r2.9.0/beginner/autograd.html. +MindSpore Contributors (2026c) Automatic Differentiation, MindSpore Tutorials. https://www.mindspore.cn/tutorials/en/r2.9.0/beginner/autograd.html. diff --git a/docs/zh/index.md b/docs/zh/index.md index ac78e42b..20985c12 100644 --- a/docs/zh/index.md +++ b/docs/zh/index.md @@ -182,5 +182,5 @@ - [附录D：论文到工程化转换指南](appendix_d_paper_to_implementation_guide.md) - [附录E：常见数据工程 Bug 调试手册](appendix_e_common_bug_debugging_manual.md) - [附录F：术语表与中英文对照](appendix_f_terminology_and_chinese_english_mapping.md) -- [附录G：DataGallery 开源生态简介](appendix_g_datagallery_note.md) +- [附录G：DataGallery 开源生态与复现说明](appendix_g_datagallery_note.md) - [附录H：MindSpore 技术附录与致谢](appendix_h_mindspore_note.md) diff --git a/docs/zh/part14/p15_dataagent_semantic_nl2sql_agent.md b/docs/zh/part14/p15_dataagent_semantic_nl2sql_agent.md index 84fbfe30..de300ebd 100644 --- a/docs/zh/part14/p15_dataagent_semantic_nl2sql_agent.md +++ b/docs/zh/part14/p15_dataagent_semantic_nl2sql_agent.md @@ -237,7 +237,7 @@ DataAgent 的运行状态、消息轨迹、工具返回和 workspace 文件为 ### 5.2 安装项目 -DataAgent 是 DataGallery 开源生态中当前已经开源的执行引擎（DataGallery Contributors, 2026a）。DataGallery 开源入口见 [https://gitcode.com/datagallery](https://gitcode.com/datagallery)，DataAgent 项目仓库见 [https://gitcode.com/datagallery/dataagent](https://gitcode.com/datagallery/dataagent)（DataGallery Contributors, 2026b）。关于 DataGallery 的整体技术版图、复现边界和项目治理方式，可参见[附录G：DataGallery 开源生态简介](../appendix_g_datagallery_note.md)。 +DataAgent 是 DataGallery 开源生态中的主要执行引擎之一。DataGallery 开源入口见 [https://gitcode.com/datagallery](https://gitcode.com/datagallery)，DataAgent 项目仓库见 [https://gitcode.com/datagallery/dataagent](https://gitcode.com/datagallery/dataagent)。关于 DataGallery 的整体技术版图、复现边界和项目治理方式，可参见[附录G：DataGallery 开源生态与复现说明](../appendix_g_datagallery_note.md)。 建议先固定版本，再在仓库根目录安装依赖： @@ -899,5 +899,3 @@ NL2SQL -> CSV -> 图表 -> Markdown 报告 -> 业务交付 3. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761. 4. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. 5. dbt Labs. (2026). dbt Documentation. https://docs.getdbt.com/. -6. DataGallery Contributors. (2026a). DataGallery organization page. https://gitcode.com/datagallery. -7. DataGallery Contributors. (2026b). DataAgent source repository. https://gitcode.com/datagallery/dataagent. diff --git a/mkdocs.yml b/mkdocs.yml index c952926b..d998f24b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -175,8 +175,8 @@ plugins: - 附录D：论文到工程化转换指南: appendix_d_paper_to_implementation_guide.md - 附录E：常见数据工程 Bug 调试手册: appendix_e_common_bug_debugging_manual.md - 附录F：术语表与中英文对照: appendix_f_terminology_and_chinese_english_mapping.md - - 附录G：DataGallery 开源生态简介: appendix_g_datagallery_note.md - - 附录H：MindSpore 简介与致谢: appendix_h_mindspore_note.md + - 附录G：DataGallery 开源生态与复现说明: appendix_g_datagallery_note.md + - 附录H：MindSpore 技术附录与致谢: appendix_h_mindspore_note.md - 卷末：后记与使用说明: afterword.md - locale: en name: English @@ -305,8 +305,8 @@ plugins: - "Appendix D: From Paper to Implementation Guide": appendix_d_paper_to_implementation_guide.md - "Appendix E: Common Data-Engineering Bug Debugging Manual": appendix_e_common_bug_debugging_manual.md - "Appendix F: Terminology and Chinese-English Mapping": appendix_f_terminology_and_chinese_english_mapping.md - - "Appendix G: DataGallery Open-source Ecosystem Overview": appendix_g_datagallery_note.md - - "Appendix H: MindSpore Overview and Acknowledgments": appendix_h_mindspore_note.md + - "Appendix G: DataGallery Open-source Ecosystem and Reproduction Notes": appendix_g_datagallery_note.md + - "Appendix H: MindSpore Technical Appendix and Acknowledgments": appendix_h_mindspore_note.md - Afterword: afterword.md - locale: ja name: 日本語 From fa05bfe46e5b27a26af3ac9f21786d97b9d63fa8 Mon Sep 17 00:00:00 2001 From: kero Date: Wed, 24 Jun 2026 22:23:45 +0800 Subject: [PATCH 2/3] update table format --- docs/en/abbreviations.md | 5 + docs/en/part1/ch01_data_change.md | 6 - docs/en/part1/ch02_quality_framework.md | 1 - docs/en/part1/ch03_data_stack.md | 3 - docs/en/part10/ch31_agent_architecture.md | 28 +- .../ch32_auto_collection_parsing_cleaning.md | 16 +- .../ch33_labeling_synthesis_evaluation.md | 25 +- docs/en/part10/ch34_dataops_agent.md | 25 +- .../ch35_security_permission_collaboration.md | 23 +- ...h36_compliance_framework_and_governance.md | 9 + ...ing_and_privacy_preserving_technologies.md | 1 + .../ch38_text_corpora_transparent_ledger.md | 11 + ...ge_text_candidate_pool_data_engineering.md | 5 + ..._visual_document_table_data_engineering.md | 15 +- ..._visual_reasoning_tool_data_engineering.md | 4 + ...eech_audio_interaction_data_engineering.md | 12 +- ...ning_trace_compression_data_engineering.md | 13 +- docs/en/part13/ch44_pretrain_recipes.md | 12 +- docs/en/part13/ch45_posttrain_recipes.md | 12 +- docs/en/part13/ch46_rl_reasoning_data.md | 12 +- docs/en/part13/ch47_vlm_data_recipes.md | 9 +- docs/en/part13/ch48_t2i_t2v.md | 9 +- docs/en/part14/p01_mini_c4.md | 16 +- docs/en/part14/p02_legal_sft.md | 13 +- docs/en/part14/p03_llava_instruct.md | 28 +- docs/en/part14/p04_synthetic_textbook.md | 13 +- docs/en/part14/p05_mm_rag.md | 13 +- docs/en/part14/p06_prm.md | 13 +- docs/en/part14/p07_agent_tooluse.md | 13 +- docs/en/part14/p08_dataops.md | 13 +- docs/en/part14/p09_privacy_pipeline.md | 13 +- docs/en/part14/p10_flywheel.md | 13 +- docs/en/part14/p11_mini_deepseek.md | 42 +- docs/en/part14/p12_r1_reasoning_flywheel.md | 12 +- .../p13_multimodal_instruction_factory.md | 45 +- docs/en/part14/p14_video_generation.md | 39 +- .../p15_dataagent_semantic_nl2sql_agent.md | 31 +- docs/en/part2/ch04_data_sources.md | 2 - docs/en/part2/ch05_cleaning_dedup.md | 9 +- docs/en/part2/ch06_tokenization_loading.md | 2 - docs/en/part2/ch07_data_operations.md | 2 +- docs/en/part3/ch08_multimodal_image.md | 2 - docs/en/part3/ch09_recaptioning_ocr.md | 6 +- docs/en/part3/ch10_video_audio.md | 7 +- docs/en/part3/ch11_cross_modal_alignment.md | 4 - docs/en/part4/ch12_sft.md | 6 +- docs/en/part4/ch13_preference.md | 6 +- docs/en/part4/ch14_qa.md | 6 +- docs/en/part5/ch15_data_synthesis.md | 4 +- docs/en/part5/ch16_distillation.md | 6 +- docs/en/part5/ch17_quality.md | 7 +- docs/en/part6/ch18_cot.md | 4 +- docs/en/part6/ch19_tool.md | 5 +- docs/en/part6/ch20_agent.md | 4 +- docs/en/part7/ch21_rag_pipeline.md | 24 +- .../ch22_multimodal_rag_visual_retrieval.md | 24 +- .../ch23_online_feedback_knowledge_update.md | 36 +- docs/en/part8/ch24_dataops_flywheel_team.md | 90 +- ...h25_data_versioning_experiment_tracking.md | 29 +- .../part8/ch26_data_platform_observability.md | 27 +- ...27_data_catalog_and_metadata_governance.md | 15 +- ..._data_productization_and_data_contracts.md | 12 +- .../en/part9/ch29_data_valuation_and_reuse.md | 27 +- ...rnal_data_market_and_sharing_governance.md | 9 +- .../springer_alt_text_inventory.csv | 627 +- .../springer_alt_text_inventory.json | 12226 +++++++--------- .../springer_alt_text_inventory.xlsx | Bin 75697 -> 94898 bytes scripts/export_springer_submission_package.py | 57 +- 68 files changed, 5992 insertions(+), 7866 deletions(-) diff --git a/docs/en/abbreviations.md b/docs/en/abbreviations.md index e08eea9f..5d706768 100644 --- a/docs/en/abbreviations.md +++ b/docs/en/abbreviations.md @@ -4,6 +4,7 @@ This page lists high-frequency technical abbreviations used throughout the book ## General Abbreviations +*Table FM-1: General Abbreviations.* | Abbreviation | Full Name | Description | Main Locations | | --- | --- | --- | --- | | A100 | NVIDIA A100 GPU | NVIDIA A100 accelerator | Part 1, Part 10, Part 11 | @@ -37,6 +38,7 @@ This page lists high-frequency technical abbreviations used throughout the book ## Data Engineering and Platforms +*Table FM-2: Data Engineering and Platforms.* | Abbreviation | Full Name | Description | Main Locations | | --- | --- | --- | --- | | DataOps | Data Operations | Data operations and data-engineering operations system | Part 2, Part 8, Part 10 | @@ -58,6 +60,7 @@ This page lists high-frequency technical abbreviations used throughout the book ## Training, Alignment, and Reasoning +*Table FM-3: Training, Alignment, and Reasoning.* | Abbreviation | Full Name | Description | Main Locations | | --- | --- | --- | --- | | CoT | Chain-of-Thought | Chain-of-thought reasoning | Part 6, Part 10, Part 11 | @@ -76,6 +79,7 @@ This page lists high-frequency technical abbreviations used throughout the book ## Multimodality and Vision +*Table FM-4: Multimodality and Vision.* | Abbreviation | Full Name | Description | Main Locations | | --- | --- | --- | --- | | BBox | Bounding Box | Bounding box | Part 3, Part 10, Part 11 | @@ -99,6 +103,7 @@ This page lists high-frequency technical abbreviations used throughout the book ## Evaluation, Compliance, and Governance +*Table FM-5: Evaluation, Compliance, and Governance.* | Abbreviation | Full Name | Description | Main Locations | | --- | --- | --- | --- | | AGI-Eval | AGI Evaluation | Evaluation benchmark for general-intelligence capabilities | Part 11 | diff --git a/docs/en/part1/ch01_data_change.md b/docs/en/part1/ch01_data_change.md index 7a7abd17..5bac3a4c 100644 --- a/docs/en/part1/ch01_data_change.md +++ b/docs/en/part1/ch01_data_change.md @@ -90,7 +90,6 @@ In 2022, a DeepMind paper titled "Training Compute-Optimal Large Language Models The DeepMind team conducted rigorously controlled compute-optimal experiments. Their results showed that the 70B-parameter Chinchilla model, trained on approximately 1.4T tokens, outperformed the previously larger 280B-parameter Gopher model (Rae et al. 2021) on a wide range of evaluations. The contrast between the two model families in terms of parameter count and training data is presented in Table 1-1. *Table 1-1: Comparison of Data Resources Between DeepMind's Old-Paradigm and New-Paradigm Models. Source: compiled from publicly available information in Rae et al. (2021) and Hoffmann et al. (2022).* - | Model (Organization) | Parameter Count $N$ | Training Token Count $D$ | Estimated Training Compute (relative) | Inference-Side Characteristics | | :--- | :--- | :--- | :--- | :--- | | **Gopher** (Rae et al. 2021) | 280B | 300B tokens (~0.3T) | Equal controlled variable | Larger parameter count; higher inference deployment cost | @@ -116,7 +115,6 @@ When training data has higher information density, less noise, and clearer task The research trajectory described above reveals that, under the LLM data engineering paradigm, the true constraint on the capability frontier of a model is not a single dimension but the combined trade-off among **scale, quality, and diversity**. Within a limited budget and limited time, all three cannot be simultaneously maximized; pushing any one to an extreme typically incurs costs in the other two or in engineering overhead. Table 1-2 presents a cost-constraint matrix for all three dimensions, showing data processing methods, direct benefits, and primary constraints. *Table 1-2: Cost-Constraint Matrix for Scale, Quality, and Diversity in LLM Data Engineering. Source: compiled by the authors based on public research trajectories and engineering practice.* - | Core Dimension | Primary Data Processing Methods | Direct Benefits | Primary Constraints | | :--- | :--- | :--- | :--- | | **Scale** | Large-scale collection via Common Crawl, proprietary crawlers, code-repository mirrors, and licensed corpora, followed by a first-pass filter using MinHash LSH, language identification, and basic quality filtering. | Provides broad world knowledge and multi-domain language patterns; a necessary condition for the model to operate in the effective range of Scaling Laws. | Storage, network, and preprocessing costs grow rapidly; if scale expansion lacks quality gates, low-value tokens translate directly into wasted training compute. | @@ -130,7 +128,6 @@ Because it is impossible to simultaneously push scale, quality, and diversity to For engineering teams with long experience in recommendation systems, search ranking, or industrial computer vision, transitioning to LLM training often involves significant methodological friction. Traditional data warehouses and machine learning pipelines primarily handle structured tables, log features, and finite label spaces, whereas LLM training involves unstructured text, code, documents, multimodal long sequences, and open-ended generation objectives. Much of the traditional ETL experience remains valuable, but it cannot directly substitute for the LLM-specific work of data cleaning, deduplication, contamination detection, mixing, version management, and training I/O optimization. Table 1-3 highlights the differences between the two data paradigms in terms of core data types, physical volume, and quality-control challenges. *Table 1-3: Traditional Machine Learning Data Pipelines vs. LLM-Native Data Systems. Source: compiled by the authors based on engineering differences between traditional data platforms and LLM data pipelines.* - | Comparison Dimension | Traditional ML Data Pipeline (e.g., recommendation systems) | LLM-Native Data System | | :--- | :--- | :--- | | **Core data type** | Primarily user behavior tables, business event tables, sensor logs, and wide feature tables; relatively stable structure. | Primarily web text, code, papers, PDFs, image-text pairs, audio/video, and interaction logs; diverse formats with unstable boundaries. | @@ -157,7 +154,6 @@ The data flywheel refers to a continuously self-reinforcing data loop: after a m The prerequisite for this flywheel to operate at high speed is the existence of **clear, executable data handoff SLAs (service-level agreements)** between every pair of roles. Without them, any ambiguous interface—for example, "the product side says it will pass feedback data to the data team, but the format and field definitions are unspecified"—will stall the flywheel at its weakest link. Table 1-4 defines the data responsibilities, upstream/downstream deliverables, and key SLA metrics for the six core roles. *Table 1-4: Core Role and Data Interface Responsibility Definitions for Six LLM Project Roles. Source: compiled by the authors based on LLM project collaboration interfaces and data governance practices.* - | Role | Core Data Responsibilities | Data Inputs from Upstream | Data Deliverables to Downstream | Key SLA Metrics | | :--- | :--- | :--- | :--- | :--- | | **Platform Architect / MLOps** | Build and operate the underlying compute scheduling, distributed file systems (e.g., Lustre / HDFS), and training cluster stability | Data package paths, format specifications, and size estimates submitted by data engineers | Stable GPU/TPU training cluster access interface; DataLoader optimization recommendations | Training stability, I/O wait time, and GPU utilization targets should be defined against the project baseline | @@ -205,7 +201,6 @@ The modern **LLM data engineer** has differentiated from the intersection of tra Table 1-5 compares the capability boundaries of LLM data engineers and traditional ML data engineers across dimensions including core technology stack, data-volume experience, and quality assessment ability. *Table 1-5: Capability Boundary Comparison Between LLM Data Engineers and Traditional ML Data Engineers. Source: compiled by the authors based on role capability boundaries and toolchain evolution.* - | Capability Dimension | Traditional ML Data Engineer | LLM Data Engineer | | :--- | :--- | :--- | | **Core technology stack** | SQL / Pandas / Spark ETL / BI dashboards | Ray Data / datatrove / MinHash / KenLM / LakeFS | @@ -256,7 +251,6 @@ The remaining chapters of this book cover pretraining data, multimodal data, ali **Path C: Full-Stack LLM Data Expert.** Readers who need to lead data engineering decisions may read in the following order: "Part I foundational framework → Parts II and III data acquisition and processing → Parts IV–VI alignment and reasoning data → Part VII application-level data engineering → Parts VIII, IX, and XI platform and governance → Parts XIII and XIV hands-on projects." This path emphasizes end-to-end capabilities spanning data sourcing, quality assessment, platform interfaces, and compliance auditing. As shown in Table 1-6, different reader types exhibit markedly different reading priorities across the parts. *Table 1-6: Chapter Priority Recommendations by Reader Type (1 = Low, 5 = High). Source: compiled by the authors; scores are reading-path recommendations rather than measured evaluations.* - | Part | Platform / MLOps Engineer | Transitioning ML Engineer | Full-Stack LLM Data Expert | | :--- | :---: | :---: | :---: | | Part I (This Part): Paradigm and Overview | 5 | 5 | 5 | diff --git a/docs/en/part1/ch02_quality_framework.md b/docs/en/part1/ch02_quality_framework.md index e12c2fee..de51e758 100644 --- a/docs/en/part1/ch02_quality_framework.md +++ b/docs/en/part1/ch02_quality_framework.md @@ -61,7 +61,6 @@ This contract document is not a static file but a versioned document that evolve Quality is by no means a static standard; it presents entirely different core requirements at different stages as the data lifecycle progresses. Applying a fixed standard to measure data across the entire lifecycle will inevitably produce serious misjudgments. As shown in Table 2-1, the core quality requirements and detection metrics differ significantly across the four stages of pre-training, instruction fine-tuning, preference alignment, and RAG deployment. *Table 2-1: LLM Data Four-Stage Quality Objective Evolution Matrix. Source: compiled by the authors; scale ranges and metric definitions reflect common engineering practice and must be recalibrated for the project dataset in production.* - | Training Stage | Typical Data Scale | Core Quality Requirements | Primary Detection Metrics | Typical Defects and Risks | Primary Processing Tools | | :--- | :--- | :--- | :--- | :--- | :--- | | **Pre-training** | Hundreds of billions to tens of trillions of tokens | High diversity, low duplication rate, broad knowledge coverage | N-gram duplication rate, PPL distribution, domain proportion, language distribution | Insufficient deduplication ("parrot" effect); benchmark data leaking in (inflated evaluation scores); excessive low-quality SEO content | MinHash / SimHash; fastText language identification; KenLM; Quality Classifier | diff --git a/docs/en/part1/ch03_data_stack.md b/docs/en/part1/ch03_data_stack.md index 975e1c6f..b55ee73e 100644 --- a/docs/en/part1/ch03_data_stack.md +++ b/docs/en/part1/ch03_data_stack.md @@ -120,7 +120,6 @@ The processing orchestration layer solves the core engineering problem: how to r Two mainstream industrial choices are **Apache Spark** (Zaharia et al. 2016) and **Ray Data** (Moritz et al. 2018). They differ fundamentally in design philosophy and use cases. *Table 3-1: Core feature comparison of Apache Spark vs. Ray Data. Source: compiled by the authors based on public documentation of open-source frameworks and LLM data-processing practice.* - | Dimension | Apache Spark | Ray Data | | :--- | :--- | :--- | | **Core language runtime** | Scala/Java core; Python through PySpark, with JVM-Python serialization overhead | Python-native, no JVM overhead, seamless with PyTorch and Hugging Face | @@ -181,7 +180,6 @@ An LLM data stack must manage three very different kinds of data. Each has disti The three mainstream lakehouse formats each have their own appropriate scenarios. *Table 3-2: Lakehouse table format selection comparison: Apache Iceberg vs. Delta Lake vs. Apache Hudi. Source: compiled by the authors based on public documentation of open-source projects and lakehouse architecture practice.* - | Feature | Apache Iceberg | Delta Lake | Apache Hudi | | :--- | :--- | :--- | :--- | | **Core maintainer** | Netflix to Apache Foundation | Databricks, core commercial open source | Uber to Apache Foundation | @@ -300,7 +298,6 @@ The recommended large-team pattern centers on a **unified data platform**. At th An important lesson is that large-team data platforms should be built in **three stages**, not all at once. Stage 1, taking one to three months, should connect the core path: storage ingestion, basic cleaning operators, and version management, so data can flow in a controlled way. Stage 2, taking three to six months, should build observability: quality dashboards, experiment tracking, and alerting systems, turning the platform into a transparent system. Stage 3, after six months, should add more complex multi-tenant isolation, cross-project lineage insight, and resource quota management. Entering Stage 3 too early can make platform complexity exceed real needs and reduce core data-flow efficiency. *Table 3-3: Quick selection matrix for data stacks across three team types. Source: compiled by the authors; setup cycles are empirical planning ranges, and actual cycles depend on team experience, permission workflows, and data-source complexity.* - | Team size | Recommended storage | Recommended compute | Recommended orchestration | Recommended version management | Estimated build cycle | | :--- | :--- | :--- | :--- | :--- | :--- | | 1-5 people, startup | S3/MinIO + Parquet | DuckDB / Polars | Prefect / Dagster | DVC | 1-2 weeks | diff --git a/docs/en/part10/ch31_agent_architecture.md b/docs/en/part10/ch31_agent_architecture.md index 6df9e32b..6c727b20 100644 --- a/docs/en/part10/ch31_agent_architecture.md +++ b/docs/en/part10/ch31_agent_architecture.md @@ -62,8 +62,7 @@ Four high-frequency task categories in LLM data engineering are naturally suited ### 31.1.2 Benefit and Risk Matrix -*Table 31-1: Benefit and risk assessment for agentizing high-frequency data engineering tasks* - +*Table 31-1: Benefit and risk assessment for agentizing high-frequency data engineering tasks.* | Task type | Average manual time | Time after agentization | Risk level | Suggested automation level | | --- | --- | --- | --- | --- | | Parser exception detection and attribution | 30 min | 2 min | Low | Semi-automatic: agent suggests, human confirms | @@ -123,8 +122,7 @@ Task boundaries must be constrained through layering. Each layer owns one type o *Figure 31-1: Six-layer architecture for data engineering agents.* -*Table 31-2: Responsibility boundaries and failure modes of six layers* - +*Table 31-2: Responsibility boundaries and failure modes of six layers.* | Layer | Core responsibility | Boundary that must not be crossed | Typical failure mode | | --- | --- | --- | --- | | Planner | Generate task plans, split substeps, select tools | Must not execute tools directly | Bad plan not detected before execution | @@ -151,8 +149,7 @@ Planner design principles: The Tool Executor maps abstract plan steps to concrete tool calls. It maintains a **Tool Registry** where each tool declares capability boundaries, parameter schema, risk level, required permission, and rollback support. -*Table 31-3: Example agent tool registry* - +*Table 31-3: Example agent tool registry.* | Tool | Capability | Risk | Required permission | Rollback support | | --- | --- | --- | --- | --- | | `data_profiler` | Generate data quality report | Low | read | N/A | @@ -268,8 +265,7 @@ Hard constraints: ### 31.2.9 Failure Isolation and Degradation -*Table 31-4: Degradation strategies for six-layer failures* - +*Table 31-4: Degradation strategies for six-layer failures.* | Failed layer | Failure type | Degradation strategy | Recovery condition | | --- | --- | --- | --- | | Planner | Plan generation timeout or unreasonable plan | Use templated plan based on known successful plans | Resume after three successful manual plans | @@ -285,8 +281,7 @@ The key principle is that degradation must increase human involvement. When auto DataAgent can be understood as an Agentic Data Engineering framework for enterprise data tasks. It does not put every capability into one large model. Instead, configuration, tools, sub-agents, semantic layer, and workspace turn natural-language tasks into manageable engineering units. -*Table 31-5: Mapping DataAgent to the six-layer architecture* - +*Table 31-5: Mapping DataAgent to the six-layer architecture.* | Architecture layer | DataAgent capability | Meaning for data engineering agents | | --- | --- | --- | | Planner | FlexAgent, ReAct main agent, `SCENARIO`, task prompts | Convert business questions into executable steps and decide when to call tools or sub-agents | @@ -310,8 +305,7 @@ DataAgent has three roles in this part: Not all data engineering tasks are suitable for fully automatic execution. This chapter proposes a four-level automation model, where each level corresponds to different agent permissions and human involvement. -*Table 31-6: Four-level automation matrix* - +*Table 31-6: Four-level automation matrix.* | Level | Name | Agent role | Human role | Typical tasks | Hard constraint | | --- | --- | --- | --- | --- | --- | | L1 | Recommendation | Analyze and suggest | Decide and execute | Alert attribution, quality report interpretation | Agent performs no writes | @@ -349,8 +343,7 @@ A minimum viable Data Engineering Agent should complete this loop independently: ### 31.4.2 Technology Choices -*Table 31-7: Technology choices for MVP components* - +*Table 31-7: Technology choices for MVP components.* | Component | Minimal implementation | Recommended implementation | | --- | --- | --- | | Planner | Rule-based step templates | LLM plus few-shot prompts and tool descriptions | @@ -376,8 +369,7 @@ Business question This usually maps to L1-L2. The agent may read metadata, generate SQL, execute read-only queries, and save results, but it should not rewrite schemas, publish metric definitions, or trigger downstream production pipelines. -*Table 31-8: Configuration gates for a DataAgent semantic query MVP* - +*Table 31-8: Configuration gates for a DataAgent semantic query MVP.* | Configuration surface | Typical content | MVP gate | | --- | --- | --- | | `MODEL` | chat model, temperature, base URL, API key | Config loads; secrets are not stored in repository or written config | @@ -406,6 +398,7 @@ Input quality report for `orders`: *Planner output* +*Table 31-9: Planner output.* | Step | Tool | Target | Estimated rows | Risk | | --- | --- | --- | --- | --- | | Step 1 | `field_fixer` | Normalize `order_date` | ~1500 | Low | @@ -463,8 +456,7 @@ The architecture assumes every layer can fail and prevents one failure from beco ### Production Agent Maturity Model -*Table 31-9: Data engineering agent maturity model* - +*Table 31-10: Data engineering agent maturity model.* | Maturity | Characteristics | Automation | Human intervention | Typical timeline | | --- | --- | --- | --- | --- | | L0: Manual | All tasks done by humans | 0% | 100% | Baseline | diff --git a/docs/en/part10/ch32_auto_collection_parsing_cleaning.md b/docs/en/part10/ch32_auto_collection_parsing_cleaning.md index fe31f71a..9e9d86d4 100644 --- a/docs/en/part10/ch32_auto_collection_parsing_cleaning.md +++ b/docs/en/part10/ch32_auto_collection_parsing_cleaning.md @@ -73,8 +73,7 @@ The first responsibility of a collection agent is **automatic task generation**. 2. **Strategy generation.** It generates pagination, frequency control, concurrency, and retry strategy. 3. **Provenance capture.** Every collection records URL, collection time, HTTP headers, and file hash. This matters for compliance audit and copyright tracing. -*Table 32-1: Collection failure categories and retry strategies* - +*Table 32-1: Collection failure categories and retry strategies.* | Failure type | Detection | Retry strategy | Max retries | After limit | | --- | --- | --- | --- | --- | | Network timeout | HTTP status or exception | Exponential backoff: 1s, 2s, 4s, 8s | 5 | Mark temporarily unavailable; retry after 1 hour | @@ -89,8 +88,7 @@ Retry strategy must be polite. Aggressive retry can worsen access blocks or crea When an agent manages hundreds of sources, scheduling is no longer simple cron execution. It becomes a constrained optimization problem: maximize throughput and freshness while respecting access limits. -*Table 32-2: Collection scheduling constraints* - +*Table 32-2: Collection scheduling constraints.* | Constraint | Meaning | Example | | --- | --- | --- | | Frequency limit | Request-rate limit of target website or API | At most 10 requests per second | @@ -200,8 +198,7 @@ Sandbox validation is the last defense before rule release. The sandbox should s - **Full validation.** Run against all matching data, not only samples, to catch edge cases. - **Diff visualization.** Show before/after comparisons and highlight modified fields and modification volume. -*Table 32-3: Sandbox validation dimensions and pass conditions* - +*Table 32-3: Sandbox validation dimensions and pass conditions.* | Dimension | Check | Pass condition | If failed | | --- | --- | --- | --- | | Rule match scope | How many rows are affected | Affected rows within expected range, deviation < 20% | Adjust scope | @@ -215,8 +212,7 @@ Sandbox validation is the last defense before rule release. The sandbox should s The hardest part of quality judgment is not deciding right or wrong. It is knowing how uncertain the agent is. A well-designed agent should request human help under uncertainty rather than forcing a possibly wrong decision. -*Table 32-4: Quality uncertainty routing* - +*Table 32-4: Quality uncertainty routing.* | Quality dimension | High certainty: auto-pass | Medium certainty: mark and sample review | Low certainty: human review | | --- | --- | --- | --- | | Format correctness | Regex/constraints match 100% | Match rate 80%-99% | Match rate < 80% | @@ -253,8 +249,7 @@ Thresholds should be configurable by business need and data characteristics: ### 32.4.4 Priority and SLA Management for Human Review -*Table 32-5: Human review priority and SLA management* - +*Table 32-5: Human review priority and SLA management.* | Priority | Trigger | SLA | Timeout action | | --- | --- | --- | --- | | P0 | Affects key downstream pipeline, such as model training export | Within 1 hour | Escalate and pause related pipelines | @@ -274,6 +269,7 @@ A legal AI team transforms its manual collection process into an agent-driven ad ### Key Metric Changes +*Table 32-6: Key Metric Changes.* | Metric | Before: manual | After: agent-assisted | Change | | --- | --- | --- | --- | | New source onboarding time | 3-5 days | 4-8 hours | -80% | diff --git a/docs/en/part10/ch33_labeling_synthesis_evaluation.md b/docs/en/part10/ch33_labeling_synthesis_evaluation.md index f175b5a0..68a52929 100644 --- a/docs/en/part10/ch33_labeling_synthesis_evaluation.md +++ b/docs/en/part10/ch33_labeling_synthesis_evaluation.md @@ -63,8 +63,7 @@ The labeling-assistance agent is an intelligent co-pilot for annotators. It does Inter-annotator agreement is a core quality metric. Agents can monitor each annotator's label distribution and deviation from baselines. -*Table 33-1: Annotator consistency monitoring metrics* - +*Table 33-1: Annotator consistency monitoring metrics.* | Metric | Calculation | Alert threshold | Suggested action | | --- | --- | --- | --- | | Individual-group consistency | Cohen's Kappa / Fleiss' Kappa | < 0.7 | Notify annotator and push calibration samples | @@ -76,8 +75,7 @@ Inter-annotator agreement is a core quality metric. Agents can monitor each anno Annotation quality cannot be pursued without cost constraints. The key principle is: **perfect labels have infinite cost; engineering needs a Pareto balance among cost, quality, and speed.** -*Table 33-2: Annotation cost structure comparison* - +*Table 33-2: Annotation cost structure comparison.* | Cost item | Manual mode | Agent-assisted mode | Savings source | | --- | --- | --- | --- | | Annotator hourly wage | $15-25/hr | $15-25/hr | No change | @@ -99,8 +97,7 @@ When deadlines are tight, agents can support data-driven tradeoffs: Agents can assign the right samples to the right annotators. -*Table 33-3: Annotator-profile dimensions for assignment* - +*Table 33-3: Annotator-profile dimensions for assignment.* | Annotator profile dimension | Collection method | Assignment logic | | --- | --- | --- | | Domain proficiency | Historical accuracy by domain | Prefer samples in strong domains | @@ -131,8 +128,7 @@ The core loop is a controlled generate-verify-filter pipeline. **Verifier calls.** Synthetic data must pass multiple checks before entering storage. -*Table 33-4: Multi-layer validation for synthetic data* - +*Table 33-4: Multi-layer validation for synthetic data.* | Layer | Check | Tool or method | | --- | --- | --- | | Format | Conforms to schema | Structured validation | @@ -175,8 +171,7 @@ Synthetic data is generated by a model, but its legal and IP risks are not simpl Difficulty control prevents synthetic data from collapsing into trivial or impossible samples. -*Table 33-5: Difficulty-control dimensions for synthetic data* - +*Table 33-5: Difficulty-control dimensions for synthetic data.* | Dimension | Evaluation method | L1: basic | L3: medium | L5: difficult | | --- | --- | --- | --- | --- | | Reasoning steps | Minimum steps needed to answer | 1-2 | 3-4 | 5+ | @@ -224,6 +219,7 @@ The red-team agent's output should be a continuously updated **security defect d Red-team testing belongs in continuous integration. +*Table 33-6: Frequency and Triggers for Red-Team Testing.* | Trigger | Scope | Depth | | --- | --- | --- | | Every model release | Full red-team set | Deep | @@ -238,8 +234,7 @@ When a new vulnerability type appears, the agent should generate variants and ex Evaluation agents should slice results along multiple dimensions to locate weaknesses. -*Table 33-6: Evaluation slicing dimensions* - +*Table 33-7: Evaluation slicing dimensions.* | Slice dimension | Examples | Use | | --- | --- | --- | | Capability | Reasoning, knowledge, generation, safety | Locate capability gaps | @@ -252,8 +247,7 @@ Evaluation agents should slice results along multiple dimensions to locate weakn Evaluation sets need continuous health monitoring. -*Table 33-7: Evaluation set health metrics* - +*Table 33-8: Evaluation set health metrics.* | Metric | Calculation | Healthy threshold | If unhealthy | | --- | --- | --- | --- | | Discriminative power | Score variance across models | Variance > 0.05 | Add harder questions if variance is too small | @@ -274,8 +268,7 @@ When agents participate in both generation and evaluation, self-evaluation risk ### 33.4.2 Human-Agent Consistency Matrix -*Table 33-8: Human-agent evaluation consistency matrix* - +*Table 33-9: Human-agent evaluation consistency matrix.* | Evaluation dimension | Agent-only scoring accuracy | Human scoring accuracy | Agent-human consistency | Suggested use | | --- | --- | --- | --- | --- | | Factual accuracy | 82% | 95% | 0.78 | Agent prescreen plus human review | diff --git a/docs/en/part10/ch34_dataops_agent.md b/docs/en/part10/ch34_dataops_agent.md index 146ccb0c..53b97fd5 100644 --- a/docs/en/part10/ch34_dataops_agent.md +++ b/docs/en/part10/ch34_dataops_agent.md @@ -64,8 +64,7 @@ Data sources: The agent aligns timelines and uses causal signals to generate candidates. -*Table 34-1: Root cause candidate types and detection logic* - +*Table 34-1: Root cause candidate types and detection logic.* | Root cause type | Pattern | Detection logic | Confidence | | --- | --- | --- | --- | | Upstream schema change | Change precedes first alert and affects all downstream tasks | Cross-check change records and alert timeline | 0.90 | @@ -88,6 +87,7 @@ Calibration methods: *Confidence-to-action mapping* +*Table 34-2: Confidence-to-action mapping.* | Raw confidence | Calibrated confidence | Automation action | | --- | --- | --- | | > 0.90 | Usually consistent | Push root cause and repair suggestion | @@ -117,8 +117,7 @@ Aggregation strategies: 2. **Time-window aggregation.** Merge repeated alerts of the same type within a window, such as five minutes, and attach count and trend. 3. **Dynamic priority adjustment.** Adjust priority based on business impact and data volume. Low-priority 3 a.m. alerts can wait until working hours. -*Table 34-2: Example alert aggregation effect* - +*Table 34-3: Example alert aggregation effect.* | Raw alerts | Aggregated alerts | Aggregation rate | Daily alerts handled by engineers | | --- | --- | --- | --- | | 200+ | 15-20 | ~90% | From 50+ to 12 | @@ -145,8 +144,7 @@ Rollback is high risk and requires approval. *Figure 34-2: Rollback approval workflow.* -*Table 34-3: Approval matrix for rollback and repair* - +*Table 34-4: Approval matrix for rollback and repair.* | Operation | Risk | Approval | Rollback plan | | --- | --- | --- | --- | | Field-level data repair | Low | Agent automatic plus post-hoc audit | Keep original values | @@ -162,8 +160,7 @@ Beyond rollback and repair, a DataOps Agent can attempt limited self-healing aft Self-healing is bounded. The agent may only execute operations in a predefined safe operation set. Anything outside that set requires human approval. -*Table 34-4: Self-healing permissions by data classification* - +*Table 34-5: Self-healing permissions by data classification.* | Self-healing operation | L0 public data | L1 internal data | L2 sensitive data | L3 confidential data | | --- | --- | --- | --- | --- | | Job rerun | Automatic | Automatic | Automatic, once | Approval | @@ -192,8 +189,7 @@ Rollback plans must include: ### 34.2.5 Decision Support Matrix for Data Rollback -*Table 34-5: Rollback versus repair decision support* - +*Table 34-6: Rollback versus repair decision support.* | Factor | Rollback option | Repair option | Agent analysis | | --- | --- | --- | --- | | Time cost | Snapshot restore plus downstream reruns | Script development plus validation | Estimate both using historical data | @@ -219,8 +215,7 @@ Data platform cost is often ignored because billing is monthly. By the time the ### 34.3.2 Automatic Optimization Suggestions -*Table 34-6: Cost anomaly detection and optimization suggestions* - +*Table 34-7: Cost anomaly detection and optimization suggestions.* | Cost anomaly | Detection | Suggestion | Estimated saving | | --- | --- | --- | --- | | Intermediate table growth | Table size growth > daily mean by 5% | Set TTL and archive old partitions | 20-40% storage | @@ -243,8 +238,7 @@ DataOps Agents should warn before resource exhaustion. Optimization requires knowing who spends what. Agents allocate platform cost by team, project, task type, and data source. -*Table 34-7: Cost attribution dimensions* - +*Table 34-8: Cost attribution dimensions.* | Attribution dimension | Cost types | Attribution method | | --- | --- | --- | | Team | Storage, compute, labeling labor | Resource owner tags | @@ -273,6 +267,7 @@ Postmortems are central to the DataOps flywheel, but engineers often skip them a *Postmortem draft template* +*Table 34-9: Postmortem draft template.* | Element | Auto-fill source | | --- | --- | | Event timeline | Alert times, operation logs, approval records | @@ -345,6 +340,7 @@ The rollout is gradual: ### Continuous MTTR Improvement +*Table 34-10: Continuous MTTR Improvement.* | Stage | MTTR | Agent automation | Engineer satisfaction | | --- | --- | --- | --- | | Manual baseline | 45 min | 0% | 3.2/5, alert fatigue | @@ -389,6 +385,7 @@ Technical path: *Reactive versus predictive operations* +*Table 34-11: Reactive versus predictive operations.* | Dimension | Reactive operations | Predictive operations | | --- | --- | --- | | Trigger | After failure | Before failure | diff --git a/docs/en/part10/ch35_security_permission_collaboration.md b/docs/en/part10/ch35_security_permission_collaboration.md index 907de5be..67ac220b 100644 --- a/docs/en/part10/ch35_security_permission_collaboration.md +++ b/docs/en/part10/ch35_security_permission_collaboration.md @@ -49,8 +49,7 @@ Security investigation reveals the attack path: The core principle is **least privilege**: at any moment, the agent should hold only the minimum permissions needed for the current task. This differs from traditional data engineering permissions. Engineers often have broad permissions for emergencies; agents should receive task-scoped permissions that are granted and revoked dynamically. -*Table 35-1: Data classification and agent permission matrix* - +*Table 35-1: Data classification and agent permission matrix.* | Data level | Definition | Example | Agent read | Agent write | Approval | | --- | --- | --- | --- | --- | --- | | L0 public | No sensitive information; freely shareable | Public datasets, open-source code | Automatic | Automatic within bounded scope | Post-hoc audit | @@ -62,8 +61,7 @@ The core principle is **least privilege**: at any moment, the agent should hold Tool calls must be controlled by allowlists. The allowlist defines not only which tools are callable, but also under what conditions. -*Table 35-2: Agent tool allowlist and call conditions* - +*Table 35-2: Agent tool allowlist and call conditions.* | Tool | Default state | Automatic-call condition | Approval required when | | --- | --- | --- | --- | | Data read | Open | Most scenarios | Reading L3 data | @@ -87,8 +85,7 @@ Tool calls must be controlled by allowlists. The allowlist defines not only whic Data engineering agents process many untrusted sources, so their prompt injection surface is wider than that of general agents. -*Table 35-3: Prompt injection attack vectors for agents* - +*Table 35-3: Prompt injection attack vectors for agents.* | Attack vector | Injection channel | Risk | Real-world analogy | | --- | --- | --- | --- | | Web content injection | Hidden instruction text in crawled pages | Medium | White text on white background | @@ -184,8 +181,7 @@ Handling process: Agent audit logs differ from ordinary application logs. They must record not only what happened, but why it happened and who approved it. -*Table 35-4: Agent audit log field specification* - +*Table 35-4: Agent audit log field specification.* | Field | Meaning | Example | | --- | --- | --- | | `event_id` | Unique event ID | `evt_20240601_001` | @@ -215,8 +211,7 @@ When an automatic operation causes data problems, responsibility must be explici In regulated industries, audit logs must serve as legal evidence. -*Table 35-5: Compliance audit requirements for agent logs* - +*Table 35-5: Compliance audit requirements for agent logs.* | Compliance need | Log requirement | Example | | --- | --- | --- | | Source traceability | Every training item traces to original source | Data A came from page 3 of PDF X collected on 2024-03-15 | @@ -241,6 +236,7 @@ Security incidents require predefined response, not improvised decisions. *Incident response SLA* +*Table 35-6: Incident response SLA.* | Stage | Target time | Owner | | --- | --- | --- | | Incident confirmation to agent pause | Within 15 minutes | On-call security engineer | @@ -256,8 +252,7 @@ Security incidents require predefined response, not improvised decisions. The collaboration question is which tasks agents can perform, which require human review, and which require multi-person approval. Use four dimensions. -*Table 35-6: Human-AI task allocation matrix* - +*Table 35-7: Human-AI task allocation matrix.* | Decision dimension | Agent autonomous decision | Human review | Multi-person approval | | --- | --- | --- | --- | | Operation risk | Low, affects a few rows | Medium, affects hundreds to thousands | High, affects tens of thousands or cross-table | @@ -316,6 +311,7 @@ The collaboration question is which tasks agents can perform, which require huma ### Remediation Results +*Table 35-8: Remediation Results.* | Metric | Before | After | | --- | --- | --- | | Rule release approval coverage | 60%, some low-risk rules auto-passed | 100%, every rule receives at least single review | @@ -345,8 +341,7 @@ Security feedback loop: Trust is quantifiable. -*Table 35-7: Human-AI collaboration trust metrics* - +*Table 35-9: Human-AI collaboration trust metrics.* | Trust metric | Measurement | Healthy value | Improvement action | | --- | --- | --- | --- | | Approval rejection rate | Share of agent suggestions rejected in Human Gate | 5-15% | > 20% means poor agent quality; < 2% means rubber-stamping | diff --git a/docs/en/part11/ch36_compliance_framework_and_governance.md b/docs/en/part11/ch36_compliance_framework_and_governance.md index 3e0d5f46..e049911e 100644 --- a/docs/en/part11/ch36_compliance_framework_and_governance.md +++ b/docs/en/part11/ch36_compliance_framework_and_governance.md @@ -119,6 +119,7 @@ When these are separated, a familiar misalignment appears: the business asks for ### 36.1.4 Traditional Flow vs. Shift-Left Governance +*Table 36-1: Traditional Flow vs. Shift-Left Governance.* | Stage | Traditional Mode | Shift-Left Governance Mode | | :--- | :--- | :--- | | Requirements | Focus on business functions; data boundaries are rarely explicit | Define data types, purpose, authorization basis, and output boundaries | @@ -158,6 +159,7 @@ Not all data should be governed with the same intensity. Treating all data as hi This chapter uses a three-level baseline: +*Table 36-2: Data Classification Architecture.* | Security Level | Definition and Examples | Processing Requirements | Masking and Encryption Strategy | | :--- | :--- | :--- | :--- | | **L3 high sensitivity (C3)** | Sensitive personal information such as biometrics, medical health, precise location; core trade secrets such as unpublished financials | Separate consent; cannot enter analytics domains without masking; legal may veto | Strong storage encryption such as AES-256; full display masking; “usable but not visible” through privacy computing | @@ -226,6 +228,7 @@ When the data level is high, the action is strong, and the impact scope is broad ### 36.2.5 Example Risk Matrix +*Table 36-3: Example Risk Matrix.* | Data Level | Purpose | Processing Action | Risk Level | Default Controls | | :--- | :--- | :--- | :--- | :--- | | C1 | Stability monitoring | Aggregate query | Low | Standard access control | @@ -241,6 +244,7 @@ When the data level is high, the action is strong, and the impact scope is broad Without a clear accountability chain, even good rules distort during execution. Governance must define who proposes the need, who judges legality, who provides technical controls, who is responsible for compliant use, and who audits execution. +*Table 36-4: Accountability Chain: RACI Matrix.* | Role | Main Responsibilities | RACI | | :--- | :--- | :--- | | Legal / Compliance | Interpret regulations, define red lines, approve high-risk scenarios | Accountable | @@ -304,6 +308,7 @@ Without RoPA, a team cannot explain which data was used by whom and for what rea #### Minimal RoPA Form +*Table 36-5: Minimal RoPA Form.* | Field | Description | | :--- | :--- | | project_id | Unique project identifier | @@ -356,6 +361,7 @@ DPIA is not about writing a long report. It asks: #### DPIA Risk Scoring Example +*Table 36-6: DPIA Risk Scoring Example.* | Dimension | Score Description | | :--- | :--- | | Data sensitivity | C1=1, C2=2, C3=3 | @@ -404,6 +410,7 @@ A mature audit system should record: #### Audit Log Design Example +*Table 36-7: Audit Log Design Example.* | Field | Description | | :--- | :--- | | event_time | Event time | @@ -606,6 +613,7 @@ Privacy-preserving methods for prompt tuning, text generation, and LLM services ### 36.4.7 High-Risk Scenario Summary +*Table 36-8: High-Risk Scenario Summary.* | Scenario | Main Risks | Core Controls | | :--- | :--- | :--- | | Health care | Health-data leakage, purpose drift | Independent encryption zone, fine-grained permissions, strong audit | @@ -930,6 +938,7 @@ approval_rules: Governance templates are not paper designs. The table below maps common deliverables in a privacy-governance pipeline to the governance capabilities they represent. +*Table 36-9: Governance Deliverable Mapping.* | Deliverable | Governance Meaning | | :--- | :--- | | `compliance_scope.json` | Defines compliance scope | diff --git a/docs/en/part11/ch37_federated_learning_and_privacy_preserving_technologies.md b/docs/en/part11/ch37_federated_learning_and_privacy_preserving_technologies.md index e684255c..f8ee0311 100644 --- a/docs/en/part11/ch37_federated_learning_and_privacy_preserving_technologies.md +++ b/docs/en/part11/ch37_federated_learning_and_privacy_preserving_technologies.md @@ -100,6 +100,7 @@ The industry has evolved five main technical families for privacy-preserving com ### 37.2.1 Core Technologies and Comparison +*Table 37-1: Core Technologies and Comparison.* | Technology Family | Core Principle | Protection Target | Applicable Phase | Implementation Cost and Main Bottlenecks | | :--- | :--- | :--- | :--- | :--- | | **Federated Learning (FL)** | Data stays local while models move; nodes train locally and exchange gradients or parameters | Raw training data does not directly leave its domain | Model training and fine-tuning | High communication cost; gradient leakage risk; sensitive to node heterogeneity | diff --git a/docs/en/part12/ch38_text_corpora_transparent_ledger.md b/docs/en/part12/ch38_text_corpora_transparent_ledger.md index e24c3c0b..c40d6039 100644 --- a/docs/en/part12/ch38_text_corpora_transparent_ledger.md +++ b/docs/en/part12/ch38_text_corpora_transparent_ledger.md @@ -71,6 +71,7 @@ FineWeb is publicly available as a full dataset, configurations split by Common *Table 38-1 Public FineWeb Forms and Engineering Uses* +*Table 38-1: Public FineWeb Forms and Engineering Uses.* | Form | Public Description | Engineering Use | Usage Notes | | --- | ---: | --- | --- | | FineWeb full dataset | The initial paper reports 15T tokens; the official dataset card continues listing later dumps | Large-scale English Web pre-training, data ablation, filtering-strategy research | The data continues to update; cite the dataset-card access time and scale convention | @@ -115,6 +116,7 @@ The official FineWeb dataset card states that samples include `language`, `langu *Table 38-2 FineWeb-like Web Document Record Schema* +*Table 38-2: FineWeb-like Web Document Record Schema.* | Field Group | Typical Fields | Source or Generation Method | Engineering Use | | --- | --- | --- | --- | | Provenance fields | `url`, `dump`, `warc_record_id`, `fetch_time` | WARC metadata and reader supplements | Trace original Web pages, locate crawls, respond to removals | @@ -190,6 +192,7 @@ The main processing pipeline can be abstracted in the following order. Class nam *Table 38-3 Key Modules in the FineWeb Main Processing Pipeline* +*Table 38-3: Key Modules in the FineWeb Main Processing Pipeline.* | Order | DataTrove Module | Input | Output | Role | | ---: | --- | --- | --- | --- | | 1 | `WarcReader` | Common Crawl WARC segments | Raw HTML document stream | Reads Web snapshots from `s3://commoncrawl/crawl-data/.../warc/` | @@ -256,6 +259,7 @@ FineWeb's evaluation protocol can be summarized in Table 38-4. *Table 38-4 FineWeb Data-ablation Evaluation Protocol* +*Table 38-4: FineWeb Data-ablation Evaluation Protocol.* | Control Item | FineWeb Paper Practice | Data-engineering Meaning | | --- | --- | --- | | Model scale | Ablation model has 1.82B parameters and Llama architecture | Prevents model-scale changes from hiding data differences | @@ -290,6 +294,7 @@ FineWeb's experience can be converted into an error-attribution table for Web pr *Table 38-5 Common Failures and Repair Actions for FineWeb-like Web Corpora* +*Table 38-5: Common Failures and Repair Actions for FineWeb-like Web Corpora.* | Error Type | Symptom | Possible Root Cause | Data-engineering Repair Action | | --- | --- | --- | --- | | Page-template residue | Model repeats menus, footers, or cookie text | Direct WET usage or poor main-text extraction | Return to WARC, re-extract with Trafilatura or similar tools, and sample-check template residue | @@ -369,6 +374,7 @@ Dolma is not a single static file, but a corpus asset with version evolution. Th *Table 38-6 Public Dolma Versions and Uses* +*Table 38-6: Public Dolma Versions and Uses.* | Version | Release Date | Compressed Size | Dataset-card Description | Engineering Use | | --- | --- | ---: | --- | --- | | `v1` | 2023-08-18 | 6.0 TB | First Dolma release | Trace the earliest public corpus form | @@ -386,6 +392,7 @@ Dolma covers Web, code, papers, social media, books, and encyclopedic sources. T *Table 38-7 Dolma v1.6 Source Statistics* +*Table 38-7: Dolma v1.6 Source Statistics.* | Source | Document Type | UTF-8 Bytes | Documents | Unicode Words | Llama Tokens | | --- | --- | ---: | ---: | ---: | ---: | | Common Crawl | web pages | 9,022 GB | 3,370M | 1,775B | 2,281B | @@ -451,6 +458,7 @@ Transparent corpora do not end with packaging and uploading `text`. At least thr *Table 38-8 Dolma-like Transparent Corpus Record Schema* +*Table 38-8: Dolma-like Transparent Corpus Record Schema.* | Layer | Typical Fields | Source or Generation Method | Engineering Use | | --- | --- | --- | --- | | Document level | `id`, `source`, `text`, `text_hash` | Data reader and hash computation | Locate samples, deduplicate, read during training | @@ -502,6 +510,7 @@ Dolma Toolkit documentation summarizes data organization as four actions: tag, d *Table 38-9 Dolma Toolkit Processing Actions and Evidence Outputs* +*Table 38-9: Dolma Toolkit Processing Actions and Evidence Outputs.* | Order | Action | Official Documentation Description | Evidence Output | Main Risk | | ---: | --- | --- | --- | --- | | 1 | Taggers | Assign language, toxicity, perplexity, and other attribute tags to document spans | Document quality labels and risk labels | Tagger-version changes alter filtering results | @@ -541,6 +550,7 @@ When $\Delta_s$ changes clearly on code tasks, scientific QA, or long-context ta *Table 38-10 Dolma-like Transparent Corpus Evaluation and Diagnosis Table* +*Table 38-10: Dolma-like Transparent Corpus Evaluation and Diagnosis Table.* | Evaluation Question | Required Records | Metric or Evidence | Possible Action | | --- | --- | --- | --- | | Which source drives a capability | Source mix, sample proportion, seen tokens | Source ablation, task-score difference $\Delta_s$ | Adjust source weight or add data | @@ -553,6 +563,7 @@ When $\Delta_s$ changes clearly on code tasks, scientific QA, or long-context ta *Table 38-11 Common Failures and Repair Actions for Dolma-like Transparent Corpora* +*Table 38-11: Common Failures and Repair Actions for Dolma-like Transparent Corpora.* | Failure Mode | Symptom | Possible Root Cause | Governance Action | | --- | --- | --- | --- | | Source-mix drift | A task category suddenly regresses in a new version | Sample proportion or filtering rules changed | Compare manifest diff, roll back, or resample by strata | diff --git a/docs/en/part12/ch39_image_text_candidate_pool_data_engineering.md b/docs/en/part12/ch39_image_text_candidate_pool_data_engineering.md index e5d62757..5ba1f076 100644 --- a/docs/en/part12/ch39_image_text_candidate_pool_data_engineering.md +++ b/docs/en/part12/ch39_image_text_candidate_pool_data_engineering.md @@ -56,6 +56,7 @@ The LAION-5B paper reports 5.85B CLIP-filtered image-text pairs, including appro *Table 39-1 Public Subset Structure of LAION-5B* +*Table 39-1: Public Subset Structure of LAION-5B.* | Subset | Scale | Text-language Form | Engineering Meaning | Typical Use | | --- | ---: | --- | --- | --- | | LAION-2B-en | 2.32B | English | Higher language-identification confidence, filtered with English CLIP | English CLIP, image-text retrieval, English T2I data candidates | @@ -79,6 +80,7 @@ Image-text candidate pools should also be modeled by channel. The Parquet metada *Table 39-2 Image-text Candidate Record Schema* +*Table 39-2: Image-text Candidate Record Schema.* | Channel | Typical Fields | Source or Generation Method | Engineering Use | | --- | --- | --- | --- | | Text channel | `text`, `language`, `text_length`, `text_hash` | Alt text, language identification, hash | Text filtering, language bucketing, contamination detection | @@ -119,6 +121,7 @@ LAION-5B construction can be divided into six stages: extract candidates from Co *Table 39-3 LAION-5B Construction Flow* +*Table 39-3: LAION-5B Construction Flow.* | Stage | Input | Processing Action | Output | Corresponding Channel | | ---: | --- | --- | --- | --- | | 1 | Common Crawl WAT metadata | Parse HTML IMG tags and retain image candidates with alt text | `` candidate pairs | Text channel, visual channel | @@ -200,6 +203,7 @@ Controllable speech data must validate semantics, style, and audio quality simul *Table 39-4 Quality-evaluation Metrics for Image-text Candidate Pools* +*Table 39-4: Quality-evaluation Metrics for Image-text Candidate Pools.* | Channel | Core Question | Automatic Metrics | Human-review Focus | Handling of Failed Samples | | --- | --- | --- | --- | --- | | Text channel | Is the caption usable? | Language confidence, length, template hits, repetition rate | Whether it is advertising, filename, or SEO text | Text-rule filtering, source downweighting | @@ -226,6 +230,7 @@ Risks in image-text data are easier for the public to perceive and harder to ful *Table 39-5 Risk-control Checklist for LAION-5B-like Image-text Data* +*Table 39-5: Risk-control Checklist for LAION-5B-like Image-text Data.* | Risk Type | Trigger Scenario | Control Measures | Audit Evidence | | --- | --- | --- | --- | | URL decay | Images cannot be downloaded or content changes when rerun | Preserve hash, download time, failure logs, and snapshot strategy | Download manifest | diff --git a/docs/en/part12/ch40_visual_document_table_data_engineering.md b/docs/en/part12/ch40_visual_document_table_data_engineering.md index 7f39cba7..86ecbe6f 100644 --- a/docs/en/part12/ch40_visual_document_table_data_engineering.md +++ b/docs/en/part12/ch40_visual_document_table_data_engineering.md @@ -69,8 +69,7 @@ This is why arithmetic self-consistency must be a first-class object. In high-ri StructBill-CN contains **2,300 high-resolution bill images** across **six business schemas**, all from two public medical datasets: CHIP-2022 and SIBR-Med. The mixture intentionally includes wired-grid tables, text-dense records, and borderless tables so the model cannot simply memorize one layout. -*Table 40-1: StructBill-CN composition and characteristics* - +*Table 40-1: StructBill-CN composition and characteristics.* | Source Subset | Document Type | Count | Table Form | | --- | --- | ---: | --- | | CHIP-2022 | Inpatient invoice | 680 | Wired grid | @@ -249,8 +248,7 @@ expense_schema = Schema( #### 40.3.4 Field Types, Annotation Rules, and Metrics -*Table 40-2: Field type, annotation rule, and metric mapping* - +*Table 40-2: Field type, annotation rule, and metric mapping.* | Field Type | Representative Fields | Annotation Rule | Main Metric | | --- | --- | --- | --- | | Text attribute | `Hospital_Name`, `Item_Name` | Semantic ownership first; tolerate minor OCR noise in long text | ANLS / Entity-Level F1 | @@ -401,8 +399,7 @@ Reproducible evaluation requires a fixed test split fingerprint, fixed schema ve #### 40.5.3 Error Attribution and Repair Actions -*Table 40-3: Common errors and repair actions* - +*Table 40-3: Common errors and repair actions.* | Error Type | Symptom | Root Cause | Data-Engineering Repair | | --- | --- | --- | --- | | Numeric hallucination | Amount or quantity is fabricated or copied wrongly | Token-level approximation, missing logic constraints | Bind P x Q = A and sum = T; use Doc-ACR as a quality gate; create numeric negatives | @@ -447,7 +444,6 @@ Regarding the benchmark data, all images in StructBill-CN **come exclusively fro Regarding production extension, when this chapter's methodology is applied to real private bills or medical records, field-level masking must be performed at the earliest stage of the pipeline. The table below provides concrete masking rules for each sensitive-field type in medical-expense documents. *Table 40-4: Field-level de-identification rules for medical-expense documents (for production extension)* - | Sensitive Field Type | Example | Masking Rule | Notes | |---|---|---|---| | Patient name | Zhang XX | Replace entirely with placeholder `` or irreversible hash | Core PHI field; must be fully removed | @@ -519,6 +515,7 @@ https://huggingface.co/datasets/champion666/SparseTable_Bench_Dataset In terms of scale, dataset documentation typically summarizes STB as "approximately 11,000 table images"; by split-level counts, the precise sample count is 10,983. To avoid ambiguity, this chapter uses the numbers from the split table as the authoritative reference. +*Table 40-5: Case B.2: Dataset Overview and Task Boundaries.* | Split | Image Count | Annotation Format | Primary Purpose | |---|---:|---|---| | STB-Train | 8,000 | HTML + cell bbox | Multi-task supervised training | @@ -565,6 +562,7 @@ The `[EMPTY_CELL]` token here is not ordinary text; it is a placeholder expressi From a data engineering perspective, the sample schema of STB includes at least the following fields and validation rules. +*Table 40-6: Case B.3: Sample Schema: Synchronized Representation of HTML, Text, and Bounding Boxes.* | Object | Typical Fields | Purpose | Key Quality Checks | |---|---|---|---| | Image | `image_id`, image file, width/height | Serves as visual input and bbox coordinate reference | Image opens successfully; resolution is consistent with bbox coordinate system; no corrupted pages | @@ -620,6 +618,7 @@ These structural errors are further reflected in evaluation results. In sparse t Errors in sparse tables can be classified into five types. +*Table 40-7: Case B.5: How Empty Cells and Sparse Layouts Induce Structural Errors.* | Error Type | Manifestation | Primary Cause | Observation Method in STB | |---|---|---|---| | Missing empty position | Empty `` not generated; column count decreases | Empty cells lack visual text anchors | `[EMPTY_CELL]` recall, TEDS-S, row-column expansion check | @@ -661,6 +660,7 @@ These two metrics are appropriate for cross-model comparison, but must not be in In extremely sparse tables or tables with many empty cells, lower TEDS/TEDS-S scores usually originate from structural prediction errors. Numerous empty positions weaken visible text anchors. A model may fill nonexistent content into empty cells, skip empty columns, or assign neighboring-column content to the wrong column position. Once these errors enter the HTML output, the number of `` nodes, node order, and row-column expansion relationships change, ultimately reducing TEDS or TEDS-S. TEDS-S further focuses on structural topology and empty-cell positions, making it especially useful for exposing such row-column alignment errors. +*Table 40-8: Case B.7: Evaluation Protocol: TEDS, TEDS-S, and Error Interpretation.* | Metric Pattern | Possible Interpretation | Conclusion That Should Not Be Drawn | Supplementary Check | |---|---|---|---| | TEDS high, TEDS-S high | Structure and text are broadly stable | Does not imply bboxes are necessarily correct | Cell bbox IoU, row-column geometric alignment | @@ -697,6 +697,7 @@ Fifth, error cases should be traced back to data objects. A single failure can b For reproducibility, using STB should not stop at the coarse-grained procedure of "load data, train model, report score." A more rigorous approach is to decompose each experiment into four auditable stages. Stage one is data loading verification: randomly sample training, validation, standard test, and pressure test samples to confirm that the image, HTML, cell list, and bounding boxes can all be associated through a single sample ID. Stage two is schema rendering verification: expand the HTML into a two-dimensional grid and overlay bounding boxes on the original image to confirm that empty cells, merged cells, and non-empty text are visually interpretable. Stage three is model input-output verification: clarify whether the model receives the original image, a cropped image, or a patch-based image, and whether it outputs pure HTML, HTML with coordinate tokens, or multi-task HTML and bbox results. Stage four is evaluation and attribution verification: compute Standard-Test and STB-Mask-Stress scores separately, then sample and review failures by the four error categories of empty-position miss, column shift, text error, and spatial drift. +*Table 40-9: Case B.8: Data Engineering Practice: Using STB for Training and Reproduction.* | Reproduction Stage | Input Objects | Output Objects | Key Checks | |---|---|---|---| | Data loading | Image, HTML, cells, bbox | Unified sample record | ID alignment, field completeness, correct split assignment | diff --git a/docs/en/part12/ch41_visual_reasoning_tool_data_engineering.md b/docs/en/part12/ch41_visual_reasoning_tool_data_engineering.md index 116a9057..8c5e8df4 100644 --- a/docs/en/part12/ch41_visual_reasoning_tool_data_engineering.md +++ b/docs/en/part12/ch41_visual_reasoning_tool_data_engineering.md @@ -123,6 +123,7 @@ The example is one integrated science infographic. Its internal regions belong t #### Case A.3.2 Full Question Chain +*Table 41-1: Case A.3.2 Full Question Chain.* | ID | Type | Question | Answer | Evidence Source | Dependency | | --- | --- | --- | --- | --- | --- | | Q1 | Maximum lookup | Which U.S. county has the highest historical shark-attack count? | Volusia, FL | Subchart A | None | @@ -278,6 +279,7 @@ The basic idea of MedImage-ToolVQA can therefore be summarized as follows: trans MedImage-ToolVQA targets medical image multiple-choice QA. Samples are built on region-level information from BiomedParse (Zhao et al. 2025), including original image, target region, mask, bbox, target description, question, candidate options, correct answer, and local observation images returned by tools. The final training data has **24,992 records**. +*Table 41-2: Case B.3: Data Objects and Scale.* | Metric | Value | Data-Engineering Meaning | | --- | ---: | --- | | Total records | 24,992 | Medical image tool-use samples for training and evaluation | @@ -478,6 +480,7 @@ MedImage-ToolVQA uses three visual tools: `Zoom-in`, `BiomedParse`, and `SAM2`. `SAM2` is a general bbox-prompted segmentation tool (Ravi et al. 2025). It does not rely on medical semantics; instead, it generates a finer mask from a geometric prompt. For samples that already have a candidate box but need a clearer boundary, `SAM2` can provide supplementary observation. Its main risk is strong dependence on bbox quality: if the bbox covers background or adjacent structures, the segmentation result will also be affected. +*Table 41-3: Case B.6: Three Tools and Their Boundaries.* | Tool | Main Input | Return | Best For | Risks to Control | | --- | --- | --- | --- | --- | | `Zoom-in` | Image index, bbox coordinates | Local crop image | Region too small, whole-image resolution insufficient, detail review needed | Bbox drift, over-cropping, context loss | @@ -689,6 +692,7 @@ The fifth failure is observation not consumed. The model calls a tool and the to The sixth failure is over-calling. The model calls a tool even when the whole image is sufficient, or it calls multiple tools in sequence without gaining new information. Over-calling increases inference cost and can introduce latency and error accumulation in a real system. The training set therefore needs to preserve a certain proportion of direct visual reasoning samples, and evaluation should distinguish necessary tool use from formalized tool use. +*Table 41-4: Case B.9: Common Failure Modes.* | Failure Mode | Symptom | Risk | Governance Method | | --- | --- | --- | --- | | Text-answerable question | Can answer without image | Model ignores visual input | No-image check, rewrite or filter | diff --git a/docs/en/part12/ch42_speech_audio_interaction_data_engineering.md b/docs/en/part12/ch42_speech_audio_interaction_data_engineering.md index 2dbeaaf2..10fef6be 100644 --- a/docs/en/part12/ch42_speech_audio_interaction_data_engineering.md +++ b/docs/en/part12/ch42_speech_audio_interaction_data_engineering.md @@ -46,8 +46,7 @@ VoiceStyleControl is composed of two task types: speech-to-speech dialogue gener VoiceStyleControl contains 154,906 samples in total. Of these, S2SEmoControl contains 20,117 samples (approximately 13.0% of the total), targeting style-controllable speech-to-speech dialogue generation; TTSSpeakerControl contains 134,789 samples (approximately 87.0% of the total), targeting controllable text-to-speech generation. The former is closer to a real voice assistant scenario, where the model must understand the user's spoken request and generate a spoken assistant response; the latter focuses more directly on training the model to generate target speech from a style text, acoustic condition, and emotional style. -**Table 42-1: VoiceStyleControl Sample Scale and Emotion Distribution** - +*Table 42-1: VoiceStyleControl Sample Scale and Emotion Distribution.* | Emotion | S2SEmoControl | TTSSpeakerControl | Total | Total ratio | |---|---:|---:|---:|---:| | happy | 4,050 | 38,500 | 42,550 | 27.5% | @@ -128,8 +127,7 @@ TTSSpeakerControl concentrates the control capability in a text-to-speech form. Combining samples from both S2S and TTS, the fields in VoiceStyleControl can be organized into six layers: task identifier, text content, acoustic conditions, emotion conditions, speech supervision, and basic audio configuration. S2S samples contain both user-side and assistant-side fields and therefore distinguish query-side from answer-side; TTS samples generate only assistant-side speech and therefore have a more concentrated set of fields. `language` fixes the language, and `sample_rate` fixes the audio sampling configuration; these foundational fields are the underlying contract for training loading and evaluation reproducibility and must not be inferred implicitly from path names or directory conventions alone. -**Table 42-2: Field Descriptions for Speaker, Emotion, and Sampling Labels** - +*Table 42-2: Field Descriptions for Speaker, Emotion, and Sampling Labels.* | Label layer | Field | Values / examples | Distribution or engineering requirements | |---|---|---|---| | Query-side speaker | `query_gender` | `female` / `male`, e.g., `female` | Calculated separately for the query side. | @@ -370,8 +368,7 @@ Quality assessment for controllable voice interaction data must simultaneously c Quality gates should be divided into "hard failures" and "soft risks." Missing paths, incorrect sampling rates, corrupted audio, unreadable tokens, and severe ASR reverse-transcription inconsistency typically constitute hard failures and should be blocked immediately. Slightly weak emotion intensity, average naturalness, or borderline acoustic condition perception can enter a soft-risk queue, where the decision to re-synthesize, downweight, or manually review is made based on task criticality. Treating every issue as a disqualifying veto wastes remediable samples; allowing every issue to pass dilutes the control signal with noise. -**Table 42-3: Quality Assessment Metrics** - +*Table 42-3: Quality Assessment Metrics.* | Assessment dimension | Core question | Automated metrics | Key points for human review | Handling of failures | |---|---|---|---|---| | Semantic consistency | Does the answer address the user's intent? Is TTS content read out correctly? | ASR reverse-transcription CER/WER, semantic similarity, intent hit rate | Non-responsive answers, omission of key information, hazardous suggestions | Rewrite text, re-synthesize, remove | @@ -409,8 +406,7 @@ Evaluation results should also be written back to the data version, not stored o Voice identity is a highly sensitive data asset. A person's voice contains cues about age, gender, regional background, emotional state, health condition, and personal identity; in speaker verification systems, voice can even function as an authentication credential. Once controllable voice data incorporates voice cloning, authorization, revocation, usage restriction, and auditing must be embedded in the data lifecycle — not appended as disclaimer footnotes at model release time. -**Table 42-4: Privacy and Misuse Risk Control Checklist** - +*Table 42-4: Privacy and Misuse Risk Control Checklist.* | Risk type | Triggering scenario | Control measures | Audit evidence | |---|---|---|---| | Voice identity authorization | Reference audio originates from real speakers or identifiable voices | Pre-collection consent, purpose limitation, revocability, authorization version number | Authorization timestamp, revocation records | diff --git a/docs/en/part12/ch43_reasoning_trace_compression_data_engineering.md b/docs/en/part12/ch43_reasoning_trace_compression_data_engineering.md index 5d977515..7d7f6954 100644 --- a/docs/en/part12/ch43_reasoning_trace_compression_data_engineering.md +++ b/docs/en/part12/ch43_reasoning_trace_compression_data_engineering.md @@ -42,6 +42,7 @@ The final training split of the [Latent-Switch-69K dataset](https://huggingface. In terms of difficulty distribution, the dataset does not pursue perfect uniformity. Medium-difficulty samples constitute the majority at 45,650 samples (65.5%); hard samples account for 17,428 (25.0%); and easy samples account for 6,667 (9.5%). This distribution has a clear rationale for latent-switch training. Medium-difficulty questions typically require genuine reasoning rather than templated question-answering, yet they are not so complex as to destabilize the distillation process. Hard samples provide longer, more complex reasoning chains, exposing the model to higher-budget implicit planning scenarios. Easy samples help the model retain the ability to produce short answers and direct verifications, preventing all samples from being cast as long-reasoning tasks. +*Table 43-1: Dataset Overview: Scale, Difficulty, and Domain Composition.* | Statistic | Value | Share / Notes | | --- | ---: | --- | | Total examples | 69,745 | 100.0% | @@ -363,6 +364,7 @@ Here \(\mathcal{S}_{\mathrm{prompt}}\) denotes positions in the user prompt and `teacher_kl_mask` is used for teacher-distribution supervision. Each sample also generates a teacher-reference conversation: it contains no student latent placeholders; instead, it merges the original question and the distilled solution intuition as teacher input and provides a distributional reference at the shortened ` ... ` and answer positions. The benefit is that the teacher does not need to simulate continuous latent placeholders; it supervises only the token distribution quality of explicit reasoning and the answer. +*Table 43-2: Supervision Masks: Which Tokens Contribute to the Loss.* | Span | Example tokens | CE label | Primary mask | Engineering significance | | --- | --- | --- | --- | --- | | Prompt and assistant prefix | user question `<\|im_start\|>assistant` | `-100` | `prompt_mask` | Serves as conditioning, not as output target | @@ -387,6 +389,7 @@ Another detail worth noting is the weight applied to explicit CoT. Latent-Switch Quality control for Latent-Switch-69K is not merely about filtering dirty text. Because the dataset simultaneously contains compressed reasoning, latent budgets, and multiple masks, risks are distributed across multiple layers. +*Table 43-3: Quality Control: Five Categories of Risk in Compression, Boundaries, and Bias.* | Risk type | Typical symptom | Impact | Remediation | | --- | --- | --- | --- | | Over-compression | Compressed CoT contains only a conclusion with no visible verification chain | Model fails to learn the transition from latent planning to explicit verification | Add verification sufficiency checks; reject samples with missing steps | @@ -464,8 +467,8 @@ Latent-Switch-69K illustrates the core problem of reasoning-trace data engineeri ## References -1. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022. arXiv:2201.11903. -2. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K. (2023). Let's Verify Step by Step. arXiv:2305.20050. -3. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. -4. DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. -5. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., & Steinhardt, J. (2021). Measuring Mathematical Problem Solving With the MATH Dataset. NeurIPS 2021. arXiv:2103.03874. +Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le Q V, Zhou D (2022) Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022. arXiv:2201.11903. +Lightman H, Kosaraju V, Burda Y, Edwards H, Baker B, Lee T, Leike J, Schulman J, Sutskever I, Cobbe K (2023) Let's Verify Step by Step. arXiv:2305.20050. +Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan K, Cao Y (2023) ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. +DeepSeek-AI (2025) DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. +Hendrycks D, Burns C, Kadavath S, Arora A, Basart S, Tang E, Song D, Steinhardt J (2021) Measuring Mathematical Problem Solving With the MATH Dataset. NeurIPS 2021. arXiv:2103.03874. diff --git a/docs/en/part13/ch44_pretrain_recipes.md b/docs/en/part13/ch44_pretrain_recipes.md index 124213be..37054517 100644 --- a/docs/en/part13/ch44_pretrain_recipes.md +++ b/docs/en/part13/ch44_pretrain_recipes.md @@ -56,8 +56,7 @@ As shown in Figure 44-1, the data recipe funnel narrows progressively from top t ![Figure 44-3: Hierarchical Map of Pretraining Data Sources](../../images/part13/ch44_03_pretrain_data_source_map.png)
Figure 44-3: Hierarchical Map of Pretraining Data Sources (redrawn from the foundational figure in Chapter 4)
-**Table 44-1: Data Transparency Spectrum of Leading Open-Source LLMs (6 rows × 5 columns)** - +*Table 44-1: Data Transparency Spectrum of Leading Open-Source LLMs (6 rows × 5 columns).* | Model Family | Source Category Disclosure | Mixture Ratio Disclosure | Cleaning Rules / Code Disclosure | Pretraining Data Downloadable | Overall Transparency Rating | |---|---|---|---|---|---| | DeepSeek-V3 | Detailed high-level categories | Macro ratios disclosed | High-level strategy descriptions only | No | Grey-box (partially transparent) | @@ -73,8 +72,7 @@ After filtering out promotional language in technical reports, the critical ques *(Note: Figures in the table follow the annotation convention: [D] = explicitly disclosed in the report; [I] = reasonable inference based on model behavior and known information; [E] = community or author estimate.)* -**Table 44-2: Pretraining Data Composition Comparison for Leading Open-Source LLMs (6 rows × 8 columns)** - +*Table 44-2: Pretraining Data Composition Comparison for Leading Open-Source LLMs (6 rows × 8 columns).* | Data Category | Subcategory / Characteristics | Quality Requirements | DeepSeek-V3 (14.8T) | Qwen2.5 (18T) | Qwen3 (inferred) | Llama-3.1/3.3 (inferred) | OLMo-2 (inferred) | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **General Web Text** | High-quality web pages | Very high; deduplicated, marketing content removed | ~50% [I] | ~55% [I] | ~50% [E] | ~50% [I], used in phased manner; high-frequency front-loaded | High proportion; strict deduplication, multi-tier filtering | @@ -145,8 +143,7 @@ As a fully open-source project, OLMo-2's code data sources are more transparent, OLMo-2 emphasizes multi-language coverage and project structure preservation in its code data processing: each sample attempts to retain the original project's directory and file hierarchy so the model can learn modular design, dependency relationships, and naming conventions. Deduplication is strictly enforced to prevent the model from memorizing templated code patterns, while in mathematical derivation and algorithm implementation code, complete logic and comments are preserved to assist the model in learning higher-order logical reasoning. -**Table 44-3: Code Data Sources and Scale for Leading Models (4 rows × 6 columns)** - +*Table 44-3: Code Data Sources and Scale for Leading Models (4 rows × 6 columns).* | Model Family | Estimated Total Code Scale | GitHub Source Code Share | Interactive Notebooks / Q&A Share | Cross-File Parsing (Repo-level) | Format Preservation Strategy | | :--- | :--- | :--- | :--- | :--- | :--- | | **DeepSeek-V3** | ~2.5T tokens [E] | ~70% [E] | ~30% [E] | Strong support, topological sorting [D] | Indentation and directory tree preserved | @@ -203,8 +200,7 @@ In practice, all three strategies—natural long-form text, synthetically genera This multi-strategy approach to long-text construction embodies the principle of "balancing quantity and quality, structure and coverage" in large model training. It provides a practical pathway for achieving stable and efficient long-context learning, and offers a replicable reference framework for downstream model transfer and data engineering design. -**Table 44-4: Long-Context Data Strategies for Leading Models (4 rows × 6 columns)** - +*Table 44-4: Long-Context Data Strategies for Leading Models (4 rows × 6 columns).* | Model Family | Maximum Context Window | Long-Text Data Sources | Short-Document Packing Strategy | RoPE Scaling and Fine-Tuning Phase | Performance Penalty Control | |---|---:|---|---|---|---| | DeepSeek-V3 | 128K [D] | Long-form books / repo-level code | Cross-document packing with isolation | RoPE base frequency extended in final annealing phase [D] | YaRN (Peng et al. 2023); minimal precision loss | diff --git a/docs/en/part13/ch45_posttrain_recipes.md b/docs/en/part13/ch45_posttrain_recipes.md index 74cfff2c..9147dd5a 100644 --- a/docs/en/part13/ch45_posttrain_recipes.md +++ b/docs/en/part13/ch45_posttrain_recipes.md @@ -86,8 +86,7 @@ When reading the table below, please note the annotation conventions: * **[I]**: Scale figures reasonably inferred based on publicly available training pipelines, split ratios, or contextual information. * **[E]**: Estimates derived for pedagogical illustration or engineering approximation; should not be treated as official disclosed figures. -**Table 45-1: Post-Training Data Transparency and Scale of Mainstream Open-Source Models** - +*Table 45-1: Post-Training Data Transparency and Scale of Mainstream Open-Source Models.* | Model / Project | Post-Training Stages | Data Openness | SFT Data Scale | Preference / Reward Data Scale | Key Data Sources | Reproducibility Value | | --- | --- | --- | --- | --- | --- | --- | | **Tülu-3** | SFT / DPO / RLVR | High | 939K [D] | DPO mixture scale requires item-by-item verification [D] | SFT-Mix, DPO mix, RLVR verifier | Fully open-source recipe reference system; suitable for reproduction and transfer | @@ -131,8 +130,7 @@ Another notable approach is Magpie. It minimizes dependency on manual seeds by d **Engineering Advantage:** Substantially reduces human intervention, and the resulting data distribution more closely approximates the distribution of real, natural user queries across the long tail. When discussing the post-training of open-source models such as Qwen2.5, Magpie can serve as a reference method for "how large-scale synthetic instructions can enhance diversity," rather than being attributed as an official recipe component of a specific model without a direct source. **Primary Risk:** Model self-generation can amplify the model's inherent biases and hallucinations. The data pipeline requires distribution filtering and safety filtering mechanisms. -**Table 45-2: Engineering Comparison of the Three SFT Synthesis Schools** - +*Table 45-2: Engineering Comparison of the Three SFT Synthesis Schools.* | School | Seed Dependency | Generation Method | Suitable Tasks | Primary Risk | Quality Control Focus | Representative Material | Relationship to This Chapter | | --- | --- | --- | --- | --- | --- | --- | --- | | **Self-Instruct** | Medium | Seed-inspired expansion | General instruction breadth coverage | Templatization, homogeneity | ROUGE deduplication, diversity evaluation, answerability checks | Self-Instruct paper | Foundational synthesis approach | @@ -189,8 +187,7 @@ Reinforcement Learning with Verifiable Rewards (RLVR) advances the source of pre * **Suitable tasks:** Mathematical problem-solving, code generation, format compliance, and tool invocation. Math problems can be verified against the final answer, code problems can run unit tests, structured outputs can undergo JSON/XML regex or schema validation, and tool calls can check API return status codes. * **Engineering challenge:** Writing verifiers that cover a sufficiently wide range of tasks without loopholes. This section introduces the data shape concept of RLVR to lay the groundwork for subsequent discussion; the R1-style reasoning flywheel will be detailed in Ch46. -**Table 45-3: Different Data Requirements Across Preference Paradigms** - +*Table 45-3: Different Data Requirements Across Preference Paradigms.* | Paradigm | Core Data Shape | Reward Source | Suitable Tasks | Data Engineering Challenges | Interface with Ch46 | | --- | --- | --- | --- | --- | --- | | **RLHF** | prompt + multiple candidates + human/AI preference | Human annotation / trained RM | General assistant behavior, complex value alignment | High annotation cost, inter-annotator consistency difficult to guarantee | Provides industrial background for multi-round iteration | @@ -262,6 +259,7 @@ The value of multi-round iteration lies in the continuous absorption of boundary This re-integration is still, in essence, reconstructing preference signals or reward signals; it is not simply continuing to expand the SFT instruction pool. It should be managed separately from the sample-generation tasks in Section 45.3, so that "creating samples" and "creating signals" are not conflated. +*Table 45-4: Case Study B: Interpreting Llama-3's Multi-Round RLHF Iteration.* | Iteration Step | Data Input | Processing Action | Data Output | Annotation Convention | | --- | --- | --- | --- | --- | | Post-SFT initial sampling | Hard prompts, production failure cases | Multi-candidate generation | Candidate responses | Mark as `[I]` when sampling scale lacks a source | @@ -312,6 +310,7 @@ In post-training, contamination means not only repeated problem statements, but 3. **Implicit filtering contamination:** During rejection sampling, using the pass rate on external evaluation sets as a filtering signal is equivalent to leaking test set metrics into the model. 4. **Feedback loop contamination:** In online feedback systems, directly re-integrating user-submitted test set prompts into daily training tasks causes severe data leakage. +*Table 45-5: Data Contamination in the Post-Training Phase.* | Contamination Type | Occurrence Location | Typical Symptoms | Inspection Method | Remediation | | --- | --- | --- | --- | --- | | SFT contamination | Instruction synthesis, data mixing | Model exhibits unusual familiarity with benchmark questions | n-gram / embedding near-duplicate detection | Remove contaminated samples, rebuild split | @@ -352,6 +351,7 @@ Post-training involves not only technical route selection, but also cost, organi **Engineering costs that must not be overlooked:** Advancing a full-chain post-training effort requires teams to reserve budget across several cost categories. First is the **human preference annotation cost**: building a high-consistency RM training set typically requires an annotation team with domain expertise. Second is the **synthetic data inference cost**: whether using Evol-Instruct or multi-path sampling, both consume substantial GPU time. Once the alignment stage begins, **multi-candidate sampling costs** and **Reward Model / Verifier maintenance and retraining costs** continue to accumulate. Finally, the **data audit and contamination detection cost** that runs throughout is an essential investment for ensuring the final model's trustworthiness. +*Table 45-6: Implementation Risks, Costs, and Applicability Boundaries.* | Cost Item | Primary Source | Easily Underestimated Component | Downgrade Strategy | | --- | --- | --- | --- | | Human preference annotation | Chosen/rejected pairs, rankings, attribute scoring | Annotation consistency training and review | Start with a small high-consistency set | diff --git a/docs/en/part13/ch46_rl_reasoning_data.md b/docs/en/part13/ch46_rl_reasoning_data.md index 206e3ef9..6e9a40de 100644 --- a/docs/en/part13/ch46_rl_reasoning_data.md +++ b/docs/en/part13/ch46_rl_reasoning_data.md @@ -196,6 +196,7 @@ The greatest advantage of rule-based rewards is reproducibility, but they also i The advantage of model-based rewards is coverage of more complex human preferences, but they must be constrained to appropriate positions. Allowing a judge model to directly decide all rewards risks the training system inheriting the judge's tastes—for example, preferring longer explanations, more polite phrasing, or more confident tone. For reasoning tasks, a judge is better suited as a supplementary evaluator: checking whether the reasoning process is self-consistent, whether conditions were missed, or whether obvious hallucinations are present—not substituting for verifiable answers. Judge prompts must also be saved in data records; otherwise, when evaluation standards change, different data batches cannot be compared. +*Table 46-1: Rule-Based Rewards and Model-Based Rewards.* | Reward Type | Applicable Tasks | Advantages | Risks | Key Data Records | | --- | --- | --- | --- | --- | | Rule-based reward | Math, code, structured output | Stable, reproducible, low cost | Limited coverage; rules may have loopholes | Verifier version, test cases, failure reason | @@ -241,6 +242,7 @@ For data filtering, a lightweight language detector can calculate the language r Reasoning data relies more heavily on metadata than ordinary SFT data. The recommended minimum fields are: +*Table 46-2: Trajectory Storage and Version Control.* | Field | Meaning | | --- | --- | | `task_id` | Original problem ID | @@ -317,8 +319,7 @@ The shared value of OpenThoughts and Sky-T1 is that they transform reasoning dat However, open-source data cannot be equated directly with business data. The problem types, languages, difficulty levels, and answer styles of datasets like OpenThoughts have their own distributions, and Sky-T1's training objectives may not align with enterprise scenarios. Before using them in a project, three checks are required: whether the target language is consistent, whether the target task types are covered, and whether evaluation set contamination risk is present. Only after passing these checks is open-source data suitable as cold-start data or control-experiment data. -**Table 46-1: Comparison of Dataset Exposure Across Major Reasoning Models** - +*Table 46-3: Comparison of Long-CoT Data Characteristics.* | Model/Dataset | Core Driving Stages | Reasoning Trajectory Source | Open-Source/Downloadable | Distinctive Strategy | Annotation | | --- | --- | --- | --- | --- | --- | | DeepSeek-R1 | Cold start + RL + Rejection sampling + SFT | Proprietary multi-path sampling and rule-based verification | Model open; training data not fully open | 600K reasoning data + 200K non-reasoning data | [D] | @@ -327,8 +328,7 @@ However, open-source data cannot be equated directly with business data. The pro | OpenThoughts-114K | SFT / open-source reproduction | Community synthesis and curation | Dataset open | 114K-scale Long-CoT data | [D] | | Sky-T1 | Small-scale Long-CoT SFT | QwQ distillation and curation | Model, data, and code open | Low-cost reproduction of reasoning capability | [D] | -**Table 46-2: Comparison of Long-CoT Data Characteristics** - +*Table 46-4: Comparison of Long-CoT Data Characteristics.* | Dimension | DeepSeek-R1 | QwQ-32B | Kimi k1.5 | OpenThoughts / Sky-T1 | | --- | --- | --- | --- | --- | | Primary tasks | Math, code, general alignment | Math, code, reasoning | Long-context, multimodal, reasoning | Math, code, science, logic | @@ -371,6 +371,7 @@ Distilled reasoning data also needs to record the relationship among the teacher An actionable data catalog is as follows: +*Table 46-5: Case A: Anatomy of the OpenThoughts-114K Dataset.* | Subset | Entry Condition | Use | | --- | --- | --- | | `curated_long_cot` | Complete reasoning, verifiable answer, clear license | Cold-start SFT | @@ -441,6 +442,7 @@ The goal of rejection sampling is to select high-quality trajectories from a lar Rejection sampling should retain failed samples. Failed samples are not garbage; they help the team analyze common model errors and can serve as future PRM or hard-case data. It is recommended to classify samples into four categories: +*Table 46-6: Case C: Rejection Sampling in Practice.* | Type | Meaning | Downstream Use | | --- | --- | --- | | pass_good_trace | Correct answer, clear reasoning | Enter second-round SFT | @@ -476,6 +478,7 @@ This explicit decision record allows the team to reuse old trajectories when adj Chapter 46 defines the theory and paradigm, while Part XIV Project 12 turns the paradigm into a runnable project. To avoid mixing the chapter explanation with the project implementation, the core objects are mapped as follows. +*Table 46-7: Interface Mapping to Part XIV Project 12.* | Chapter object | P12 asset | Interface meaning | | --- | --- | --- | | Cold-start SFT | `data/processed/cold_start_5k.jsonl`, `cold_start_summary.json` | Organizes Long-CoT seeds into first-round trainable samples | @@ -548,6 +551,7 @@ For small teams, starting with full RL training is not recommended. A more reali The focus of this path is to first validate the data production closed loop, then consider scaling the model and task volume. +*Table 46-8: Cost, Risk, and Applicability Boundaries.* | Stage | Cost Source | Small-Team Downgrade Option | | --- | --- | --- | | Cold-start SFT | Data cleaning, training | Use an open-source Long-CoT subset | diff --git a/docs/en/part13/ch47_vlm_data_recipes.md b/docs/en/part13/ch47_vlm_data_recipes.md index beb667e4..f16ecd71 100644 --- a/docs/en/part13/ch47_vlm_data_recipes.md +++ b/docs/en/part13/ch47_vlm_data_recipes.md @@ -74,8 +74,7 @@ As with pure-text foundation models, the data barriers between multimodal models *(Note: Annotation convention in the table: [D] = explicitly disclosed in report; [I] = inferred; [E] = estimated.)* -**Table 47-1: Cross-Comparison of Mainstream VLM Data Compositions (4 rows × 8 columns)** - +*Table 47-1: Cross-Comparison of Mainstream VLM Data Compositions (4 rows × 8 columns).* | Model Family | Pre-training Pair Scale | Pre-training Cleaning Strategy | Interleaved Document Ratio | SFT Multimodal Instruction Volume | Video Data Scale | OCR/Doc Specialization | Hi-Res Resolution Support | | :------------------------ | :---------------------- | :-------------------------------- | :------------------------- | :-------------------------------- | :---------------------------- | :--------------------------- | :------------------------ | | **Qwen2.5-VL** | ~2B+ Pairs [I] | Proprietary image filtering + rewrite | Very high (~30%) [I] | ~5M+ [E] | Very high, variable-length clips [D] | Strong, multilingual OCR | Native Resolution | @@ -127,8 +126,7 @@ Represented by Qwen2-VL and Qwen2.5-VL. These models abandon rigid tiling logic; This recipe preserves the most complete global and local information, entirely eliminating semantic discontinuity at patch boundaries, and is the highest-precision approach. However, its data engineering complexity is also the highest: training data must be precisely bucketed and packed by token count (rather than image count) to prevent extreme length variance within each batch causing OOM errors. Qwen2.5-VL specifically developed a "token-aware" data packer that constrains the total vision token count per batch within a fixed interval, sacrificing approximately 15% of training throughput in exchange for a near-zero OOM rate [I]. -**Table 47-2: Native Resolution vs. Dynamic Hi-Res Data Processing Differences (2 rows × 6 columns)** - +*Table 47-2: Native Resolution vs. Dynamic Hi-Res Data Processing Differences (2 rows × 6 columns).* | Resolution Approach | Representative Model | Image Data Preprocessing Action | Visual Encoder Modification | LLM-side Token Sequence Characteristics | Strengths and Weaknesses | | :-------------------------- | :------------------- | :------------------------------------------- | :-------------------------------- | :-------------------------------------- | :------------------------------------------------------------------ | | **Native Res.** | Qwen2.5-VL | Retain original image, dynamically unfold by patch size | Remove fixed positional embedding | 2D absolute coordinate mapping (M-RoPE) | Highest precision, no boundary discontinuity / Very high engineering complexity, prone to memory fragmentation | @@ -153,8 +151,7 @@ As shown in Figure 47-4, multimodal instruction synthesis has long surpassed the 3. **Knowledge recombination with a capable LLM**: Feed the text enriched with bounding box and OCR information into a capable language model. Since the image has already been converted to precise text, even a pure-text GPT-4 (rather than GPT-4V) can complete this step—instructing it to generate complex reasoning questions such as: "Based on the invoice total in the upper right corner of the image and the line items on the left, calculate the tax rate." 4. **Quality filtering and deduplication**: Use self-consistency (majority-vote across multiple samples) or LLM-as-Judge to score the quality of synthesized outputs, filtering out samples with severe hallucinations or logical incoherence; also perform semantic-level instruction deduplication to prevent a single image from spawning large numbers of homogeneous question-answer pairs. -**Table 47-3: Comparison of Multimodal Instruction Data Synthesis Methods (3 rows × 5 columns)** - +*Table 47-3: Comparison of Multimodal Instruction Data Synthesis Methods (3 rows × 5 columns).* | Synthesis Approach | Core Dependent Models | Typical Application Scenarios | Cost Estimate | Noise and Hallucination Risk | | :---------------------------------------- | :--------------------------------------- | :----------------------------------------- | :------------------------------- | :------------------------------------------------------------- | | **GPT-4V Distillation** | External closed-source VLM API | Complex logical reasoning, long summarization | Very high (API quota dependent) | Relatively low, but constrained by teacher model's inherent biases | diff --git a/docs/en/part13/ch48_t2i_t2v.md b/docs/en/part13/ch48_t2i_t2v.md index dd8d38e4..569e675c 100644 --- a/docs/en/part13/ch48_t2i_t2v.md +++ b/docs/en/part13/ch48_t2i_t2v.md @@ -92,8 +92,7 @@ Wan2.2 (Wan Team 2025) places greater emphasis on aesthetic direction for high-q Comparing these models together reveals three relatively clear approaches. The first is the image recaptioning approach represented by DALL·E 3 and SD3, focused on making training text more granular and more closely aligned with user prompts. The second is the video engineering approach represented by HunyuanVideo and Open-Sora, focused on unifying motion, shot language, quality, OCR, watermarks, and structured captions into a single pipeline. The third is the industry model approach represented by FLUX and Wan2.2—publicly disclosed effects are strong, data chain details are limited, but one can observe trends toward high aesthetics, stringent safety constraints, and orientation toward creative use cases. For the purposes of this chapter, Table 48-1 serves primarily to consolidate these differences; what readers truly need to understand is this: generative model data pipelines have evolved from "data collection" to "supervisory signal production." Data no longer passively enters training sets; after filtering, rewriting, stratification, and routing, it actively shapes the generative capabilities of models. -**Table 48-1: Comparative Overview of Mainstream T2I/T2V Model Data Pipelines** - +*Table 48-1: Comparative Overview of Mainstream T2I/T2V Model Data Pipelines.* | Model / Project | Modality | Data Ingestion & Disclosure Level | Filtering & Governance Focus | Caption / Annotation Strategy | Implications for Data Engineering | |---|---|---|---|---|---| | DALL·E 3 (Betker et al. 2023; OpenAI 2023) | T2I | Data pool not fully disclosed; public materials emphasize high-quality image-text supervision | Safety filtering, personal information reduction, inference-side prompt rewriting | Highly descriptive captions; user prompts expanded at inference time | Demonstrates that highly descriptive captions are critical for prompt following | @@ -139,8 +138,7 @@ T2I recaptioning commonly takes one of three forms: SD3's 50/50 mixing strategy is particularly worth emulating. Replacing all original captions wholesale with VLM-generated captions may cause concept forgetting and stylistic uniformity; retaining some original captions preserves the rough diversity of the data distribution. In engineering practice, this can be extended to random-length mixing: the same image is sometimes paired with a short caption, sometimes with a dense caption, and sometimes with a long caption assembled from structured fields. This prevents the model from adapting exclusively to one text length. -**Table 48-2: Comparison of Caption Rewriting Approaches (DALL·E 3 / SD3 / CogVLM2 Distillation)** - +*Table 48-2: Comparison of Caption Rewriting Approaches (DALL·E 3 / SD3 / CogVLM2 Distillation).* | Approach | Training-side caption generator | Original caption retention strategy | Output form | Inference-side prompt rewriting | Advantages | Risks and costs | |---|---|---|---|---|---|---| | DALL·E 3 (Betker et al. 2023) | Specially trained highly descriptive captioner | Not fully disclosed in public materials | High-density natural-language caption | Yes, user prompt expanded with GPT-4 | Marked improvement in prompt following; short prompts converted into executable descriptions | System is closed; captioner training details and data thresholds cannot be reproduced | @@ -233,8 +231,7 @@ T2V data pipelines therefore typically include a layer of **spatiotemporal align It is important to emphasize that spatiotemporal alignment does not mean this chapter re-examines long-video timeline design. That subject is handled in Chapter 10. This chapter addresses a narrower question: **when training a T2V model, how to make captions more faithfully correspond to the action sequence and spatial changes within a clip.** The more precise this alignment, the better the model learns fine-grained processes such as "the character first turns around, then steps forward, and finally leaves the frame," rather than outputting vaguely animated sequences. -**Table 48-3: Spatiotemporal Alignment Strategies for Video Captioning** - +*Table 48-3: Spatiotemporal Alignment Strategies for Video Captioning.* | Strategy | Method | Representative Projects | Advantages | Limitations | Applicable Position | |---|---|---|---|---|---| | Single-shot single caption | One overall description per single-shot clip | HunyuanVideo, Open-Sora | High throughput; suitable for large-scale pretraining | Intra-shot details and action order easily compressed | Large-scale pretraining main data | diff --git a/docs/en/part14/p01_mini_c4.md b/docs/en/part14/p01_mini_c4.md index 2cfa584c..c5e8e428 100644 --- a/docs/en/part14/p01_mini_c4.md +++ b/docs/en/part14/p01_mini_c4.md @@ -229,13 +229,12 @@ Therefore, the goal of the body text extraction phase is not "to capture as many *Figure P01-2: Parsing Path from WARC to Body Text.* +*Table P01-1: Component and Reason for Choice Reference Table.* | Component | Selection | Reason for Choice | |---|---|---| | WARC reading | `warcio` | Standard WARC reading library with streaming support, avoiding the memory pressure of loading large files all at once | | Body text extraction | `trafilatura` | More stable extraction of main content areas; compared to simple HTML parsing approaches, offers better cleanup of navigation bars, footers, and template areas | -*Table P01-1: Component and Reason for Choice Reference Table* - ### 5.3 The Engineering Value of Stream Processing WARC files are typically large and contain many responses that are not needed. @@ -698,6 +697,7 @@ Its significance lies in making the dataset not merely a collection of scattered The final retention funnel obtained by this project is as follows: +*Table P01-2: Stage and Typical Interception Reasons Reference Table.* | Stage | Record Count | Retention Rate (based on extracted) | Typical Interception Reasons | |---|---:|---:|---| | Extracted | 3028 | 100.0% | HTML parsing failure, empty content | @@ -705,8 +705,6 @@ The final retention funnel obtained by this project is as follows: | Dedup | 2305 | 76.12% | Mirror sites, template pages, reposts | | Final | 526 | 17.37% | Directory pages, high perplexity, language mixing | -*Table P01-2: Stage and Typical Interception Reasons Reference Table* - ### 12.2 What These Numbers Really Indicate Looking only at the final result, 526 samples may seem like a small number. @@ -1073,8 +1071,8 @@ As part of Chapter 14, this chapter corresponds to the project-level empirical v ## References -1. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. -2. Hugging Face. (2026). Datasets Documentation. https://huggingface.co/docs/datasets/. -3. Ray Project. (2026). Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. -4. MLflow Authors. (2026). MLflow Documentation. https://mlflow.org/docs/latest/. -5. Great Expectations Contributors. (2026). Great Expectations Documentation. https://docs.greatexpectations.io/. +Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P J (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. +Hugging Face (2026) Datasets Documentation. https://huggingface.co/docs/datasets/. +Ray Project (2026) Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. +MLflow Authors (2026) MLflow Documentation. https://mlflow.org/docs/latest/. +Great Expectations Contributors (2026) Great Expectations Documentation. https://docs.greatexpectations.io/. diff --git a/docs/en/part14/p02_legal_sft.md b/docs/en/part14/p02_legal_sft.md index 59280ddd..adec4e1e 100644 --- a/docs/en/part14/p02_legal_sft.md +++ b/docs/en/part14/p02_legal_sft.md @@ -296,6 +296,7 @@ This not only degrades sample readability, but also causes downstream Self-Instr ### 7.2 Component Selection +*Table P02-1: Components and Selection Rationale.* | Component | Choice | Function | Rationale | | --- | --- | --- | --- | | PDF parsing | `pdfplumber` | Read page text and coordinates | Supports bounding-box-based header/footer trimming; well-suited for institutional PDFs | @@ -303,8 +304,6 @@ This not only degrades sample readability, but also causes downstream Self-Instr | Generative model | `DeepSeek-V3` | Instruction synthesis and reasoning expansion | Balances reasoning quality and cost; suitable for large-scale synthesis | | Orchestration logic | `Python` | Batch processing, sampling, post-processing | Facilitates rapid construction of a minimal reproducible pipeline | -*Table P02-1: Components and Selection Rationale* - ### 7.3 Trimming Headers and Footers In legal PDFs, the most typical repetitive noise comes from headers and footers — for example, the regulation title repeating at the top of every page, and page numbers or publication information in the footer. If these are not removed during parsing, they are repeatedly ingested into training data as if they were body text. @@ -1218,8 +1217,8 @@ As part of Part 14, this chapter corresponds to the project-level validation of ## References -1. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. *JMLR*, 21(140), 1–67. -2. Hugging Face. (2026). Datasets Documentation. https://huggingface.co/docs/datasets/. -3. Ray Project. (2026). Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. -4. MLflow Authors. (2026). MLflow Documentation. https://mlflow.org/docs/latest/. -5. Great Expectations Contributors. (2026). Great Expectations Documentation. https://docs.greatexpectations.io/. +Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P J (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. +Hugging Face (2026) Datasets Documentation. https://huggingface.co/docs/datasets/. +Ray Project (2026) Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. +MLflow Authors (2026) MLflow Documentation. https://mlflow.org/docs/latest/. +Great Expectations Contributors (2026) Great Expectations Documentation. https://docs.greatexpectations.io/. diff --git a/docs/en/part14/p03_llava_instruct.md b/docs/en/part14/p03_llava_instruct.md index 75d4e3c0..87f00539 100644 --- a/docs/en/part14/p03_llava_instruct.md +++ b/docs/en/part14/p03_llava_instruct.md @@ -59,6 +59,7 @@ The main text retains only the key implementation excerpts that illustrate desig Acceptance metrics include image-text consistency, task type coverage, OCR evidence usability, conversation turn distribution, format pass rate, and manual sampling quality. If the project enters a production, course, or public reproducibility environment, version numbers, dependency environments, random seeds, sample spot-check results, and failure sample review records should also be documented. +*Table P03-1: LLaVA Multimodal Instruction Data Factory Publication Acceptance Table.* | Acceptance Dimension | Metric / Evidence | Publication Review Criteria | | --- | --- | --- | | Task boundary | Coverage records for LLaVA-style conversation templates, image descriptions, OCR, chart reading, bbox grounding, and multi-image comparison | State that this project is the classic LLaVA workflow baseline and does not include Qwen-VL factory-scale extension capabilities in project conclusions | @@ -66,8 +67,6 @@ Acceptance metrics include image-text consistency, task type coverage, OCR evide | Training delivery | train/val/smoke splits, manifest, schema checks, and project inspection report | Training records must be stably consumable by downstream scripts, and report figures must match artifact counts | | Manual review | Sampling ratio, reviewer roles, failure sample handling status, and regeneration records | Grounding, OCR, and chart-type samples must not rely solely on automated rules for approval | -*Table P03-1: LLaVA Multimodal Instruction Data Factory Publication Acceptance Table* - ## Cost, Risk, and Compliance Boundaries Costs arise primarily from vision model calls, OCR, and manual review; risks concentrate on image copyright, hallucinated descriptions, sensitive visual content, and task distribution skew. When external data, personal information, copyrighted content, or third-party services are involved, source documentation, permission status, desensitization strategies, call logs, and manual review records should be retained. @@ -306,6 +305,7 @@ Therefore, this project does not remain at COCO natural images but further deriv Table P03-2 summarizes the relationship between different asset types and their task mappings. +*Table P03-2: Asset Types and Primary Risks Reference Table.* | Asset Type | Typical Source | Compatible Tasks | Primary Risks | | --- | --- | --- | --- | | `general_image` | COCO natural images, general scene photographs | Image description, counting, visual QA, local grounding | Hallucinated descriptions, missed objects, category confusion | @@ -313,8 +313,6 @@ Table P03-2 summarizes the relationship between different asset types and their | `chart_image` | Bar charts, line charts, report screenshots, dashboards | Chart reading, trend summarization, value comparison | Trend reversal, category relationship errors, missed values | | `interleaved_pair` | Multi-image pairs, cross-page samples, comparative screenshots | Multi-image comparison, shared feature summarization, difference identification | Order confusion, cross-image interference, pairing imbalance | -*Table P03-2: Asset Types and Primary Risks Reference Table* - --- ## 7. Data Schema: Structuring Multimodal Seeds @@ -825,6 +823,7 @@ This indicates: Table P03-3 summarizes the relationships among task types, coverage capability, and engineering value. +*Table P03-3: Task Types and Engineering Value Reference Table.* | Task Type | Primary Input | Primary Output | Coverage Capability | Engineering Value | | --- | --- | --- | --- | --- | | `image_description` | General images | Scene description | Whole-image understanding | Builds visual subject and scene expression capability | @@ -835,8 +834,6 @@ Table P03-3 summarizes the relationships among task types, coverage capability, | `region_grounding` | Images + bounding boxes | Coordinate answers | Object alignment | Builds region-level supervision and grounding capability | | `multi_image_comparison` | Multi-image input | Comparative summary | Cross-image reasoning | Builds sequential awareness, difference identification, and information aggregation capability | -*Table P03-3: Task Types and Engineering Value Reference Table* - ### 17.3 Why a 100% Pass Rate Should Not Be Over-interpreted These numbers look impressive, but a more reasonable interpretation is: the project conducted a small-scale, highly constrained data factory validation in a controlled environment, making quality easier to maintain. @@ -862,6 +859,7 @@ These figures are modest, but they already reflect the cost structure of a small Table P03-4 summarizes the current project's costs, time investment, and manual effort. +*Table P03-4: Project Items and Notes Reference Table.* | Item | Current Result | Notes | | --- | ---: | --- | | Total assets | 87 | Three asset categories, balanced at 29 each | @@ -872,8 +870,6 @@ Table P03-4 summarizes the current project's costs, time investment, and manual | Manual review cost | 267 CNY | Demonstrates that multimodal QA is not a free step | | Project checks | 11 / 11 PASS | Code, data, and report closure established | -*Table P03-4: Project Items and Notes Reference Table* - ### 18.1 Why Manual Review Cost Deserves Separate Attention Because in multimodal scenarios, manual QA is not an optional final step but the key source of the entire project's credibility. Especially for grounding, OCR, and chart-type tasks, the risk increases significantly without any manual spot-checking. @@ -927,6 +923,7 @@ These failure samples can be classified into at least the following types: Table P03-5 summarizes typical failure sample types and priority repair directions. +*Table P03-5: Failure Types and Priority Repair Directions Reference Table.* | Failure Type | Typical Manifestation | Most Likely Source | Priority Repair Direction | | --- | --- | --- | --- | | Visual hallucination | Response describes objects or relationships not present in the image | Open-ended generation over-diverging, re-captioning too expansive | Tighten prompt, add constraints on salient objects | @@ -935,8 +932,6 @@ Table P03-5 summarizes typical failure sample types and priority repair directio | Grounding offset | Coordinates fall on adjacent target or box exceeds boundary | Bounding box conversion, normalization, or image version inconsistency | Back-render boxes for verification, check dimensions and clamping | | Multi-image confusion | Information from two images conflated into a single conclusion | Insufficient sequence control, unstable payload organization | Strengthen sequence identification, control multi-image sample complexity | -*Table P03-5: Failure Types and Priority Repair Directions Reference Table* - After aggregating these failure samples into a "failure attribution table," it can directly support the next round of template tightening, QA revision, and spot-check strategy adjustment. ### 19.3 Why Failure Attribution Needs to Be Refined to Error Types @@ -1069,6 +1064,7 @@ As sample scale grows, pure manual spot-checking quickly becomes a bottleneck. T Table P03-6 summarizes the categories and roles of this project's deliverables. +*Table P03-6: Categories and Roles Reference Table.* | Category | Representative Files | Role | | --- | --- | --- | | Assets and intermediate layer | `asset_manifest.jsonl`, `llava_alignment.jsonl`, `llava_interleaved.jsonl` | Records asset provenance, task derivation, and intermediate sample state | @@ -1076,8 +1072,6 @@ Table P03-6 summarizes the categories and roles of this project's deliverables. | Training delivery layer | `final_llava_dataset.jsonl`, `train.jsonl`, `val.jsonl`, `smoke_test.jsonl`, `training_manifest.json` | Provides entry points for training, validation, and connectivity checks | | Reports and verification layer | `p3_metrics.json`, `p3_report.md`, `p3_test_results.json`, `p3_test_report.md` | Records metrics, conclusions, and project-level inspection results | -*Table P03-6: Categories and Roles Reference Table* - --- ## 24. Conclusion @@ -1136,8 +1130,8 @@ As part of Part 14, this chapter corresponds to the project-level validation of ## References -1. Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023). Visual Instruction Tuning. NeurIPS 2023. https://doi.org/10.52202/075280-1516. -2. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common Objects in Context. ECCV 2014. https://doi.org/10.1007/978-3-319-10602-1_48. -3. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., et al. (2021). Learning Transferable Visual Models From Natural Language Supervision. ICML 2021. -4. Mathew, M., Karatzas, D., & Jawahar, C. V. (2021). DocVQA: A Dataset for VQA on Document Images. WACV 2021. https://doi.org/10.1109/wacv48630.2021.00225. -5. Masry, A., Long, D. X., Tan, J. Q., Joty, S., & Hoque, E. (2022). ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. ACL 2022. https://doi.org/10.18653/v1/2022.findings-acl.177. +Liu H, Li C, Wu Q, Lee Y J (2023) Visual Instruction Tuning. NeurIPS 2023. https://doi.org/10.52202/075280-1516. +Lin, T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick C L (2014) Microsoft COCO: Common Objects in Context. ECCV 2014. https://doi.org/10.1007/978-3-319-10602-1_48. +Radford A, Kim J W, Hallacy C, Ramesh A, Goh G, et al. (2021) Learning Transferable Visual Models From Natural Language Supervision. ICML 2021. +Mathew M, Karatzas D, Jawahar C V (2021) DocVQA: A Dataset for VQA on Document Images. WACV 2021. https://doi.org/10.1109/wacv48630.2021.00225. +Masry A, Long D X, Tan J Q, Joty S, Hoque E (2022) ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. ACL 2022. https://doi.org/10.18653/v1/2022.findings-acl.177. diff --git a/docs/en/part14/p04_synthetic_textbook.md b/docs/en/part14/p04_synthetic_textbook.md index 9c16bd9a..020843d3 100644 --- a/docs/en/part14/p04_synthetic_textbook.md +++ b/docs/en/part14/p04_synthetic_textbook.md @@ -57,6 +57,7 @@ The main text retains only the key implementation snippets that illustrate desig Acceptance metrics include validation pass rate, difficulty distribution, topic coverage, solution consistency, proportion of erroneous samples, and structural completeness of volumes. If the project enters production, curriculum, or public reproducibility environments, version numbers, dependency environments, random seeds, sample spot-check results, and failure sample post-mortem records should also be logged. +*Table P04-1: Synthetic Textbook Factory Publication Acceptance Table.* | Acceptance Dimension | Metric / Evidence | Publication Review Criteria | | --- | --- | --- | | Data closed loop | Seed problems, evolved problems, PoT solutions, sandbox logs, and textbook volumes are all traceable | Spot-checked samples must be traceable back to original seeds and validation records | @@ -65,8 +66,6 @@ Acceptance metrics include validation pass rate, difficulty distribution, topic | Code isolation | Executable validation scripts, non-executable display code, sandbox logs, and dependency manifests | Code snippets in published text must not be executed as production scripts by default; companion scripts must be validated in an isolated environment | | Decision boundaries | Code safety, copyright contamination, and expert review boundaries are maintained | Before public delivery, confirm problem source authorization, non-executable code isolation, and failure sample rework records | -*Table P04-1: Synthetic Textbook Factory Publication Acceptance Table* - ## Cost, Risk, and Compliance Boundaries Costs arise primarily from generation API calls, sandbox execution, and erroneous sample review; risks are concentrated in pseudo-correct answers, code safety, problem duplication, and copyright contamination. When external data, personal information, copyrighted content, or third-party services are involved, source descriptions, permission status, desensitization strategies, call records, and manual review records should be retained. @@ -948,8 +947,8 @@ As part of Part 14, this chapter corresponds to the project-level implementation ## References -1. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. -2. Hugging Face. (2026). Datasets Documentation. https://huggingface.co/docs/datasets/. -3. Ray Project. (2026). Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. -4. MLflow Authors. (2026). MLflow Documentation. https://mlflow.org/docs/latest/. -5. Great Expectations Contributors. (2026). Great Expectations Documentation. https://docs.greatexpectations.io/. +Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P J (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. +Hugging Face (2026) Datasets Documentation. https://huggingface.co/docs/datasets/. +Ray Project (2026) Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. +MLflow Authors (2026) MLflow Documentation. https://mlflow.org/docs/latest/. +Great Expectations Contributors (2026) Great Expectations Documentation. https://docs.greatexpectations.io/. diff --git a/docs/en/part14/p05_mm_rag.md b/docs/en/part14/p05_mm_rag.md index 6087f1cc..32d413ea 100644 --- a/docs/en/part14/p05_mm_rag.md +++ b/docs/en/part14/p05_mm_rag.md @@ -59,6 +59,7 @@ The body text retains only the key implementation snippets that illuminate desig Acceptance metrics include retrieval hit rate, citation accuracy, answer keyword accuracy, latency, page processing cost, noise-page suppression effectiveness, and evidence back-link completeness. If the project enters production, a course environment, or a public reproducibility experiment, the version number, dependency environment, random seeds, sample inspection results, and failure sample post-mortems should also be recorded. +*Table P05-1: Multimodal RAG Publication Acceptance Table.* | Acceptance Dimension | Metric / Evidence | Publication Verification Criteria | | --- | --- | --- | | Evidence Retrieval | Top-K hit rate, table-of-contents page false recall rate, cross-page recall coverage, and evidence page stability | Each answer should be traceable to a page image, page number, retrieval score, and match rationale | @@ -66,8 +67,6 @@ Acceptance metrics include retrieval hit rate, citation accuracy, answer keyword | Answer Quality | Citation accuracy, numerical consistency, trend judgment accuracy, and abstention coverage | Financial figures, chart trends, footnote limitations, and conclusions must be human-verifiable | | Cost Boundaries | Page rendering cost, visual model call count, and average latency | Delivery must clarify the difference between small-sample demonstration and production full-library scale | -*Table P05-1: Multimodal RAG Publication Acceptance Table* - ## Cost, Risk, and Compliance Boundaries Costs arise primarily from page rendering, multimodal indexing, and visual model calls; risks are concentrated in financial figure misreading, chart hallucination, table-of-contents page false recall, and copyright boundaries. When external data, personal information, copyrighted content, or third-party services are involved, source descriptions, permission status, desensitization strategies, call records, and manual review records should be retained. @@ -1209,8 +1208,8 @@ As part of Part Fourteen, this chapter corresponds to the project-level deployme ## References -1. Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023). Visual Instruction Tuning. NeurIPS 2023. https://doi.org/10.52202/075280-1516. -2. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common Objects in Context. ECCV 2014. https://doi.org/10.1007/978-3-319-10602-1_48. -3. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., et al. (2021). Learning Transferable Visual Models From Natural Language Supervision. ICML 2021. -4. Mathew, M., Karatzas, D., & Jawahar, C. V. (2021). DocVQA: A Dataset for VQA on Document Images. WACV 2021. https://doi.org/10.1109/wacv48630.2021.00225. -5. Masry, A., Long, D. X., Tan, J. Q., Joty, S., & Hoque, E. (2022). ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. ACL 2022. https://doi.org/10.18653/v1/2022.findings-acl.177. +Liu H, Li C, Wu Q, Lee Y J (2023) Visual Instruction Tuning. NeurIPS 2023. https://doi.org/10.52202/075280-1516. +Lin, T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick C L (2014) Microsoft COCO: Common Objects in Context. ECCV 2014. https://doi.org/10.1007/978-3-319-10602-1_48. +Radford A, Kim J W, Hallacy C, Ramesh A, Goh G, et al. (2021) Learning Transferable Visual Models From Natural Language Supervision. ICML 2021. +Mathew M, Karatzas D, Jawahar C V (2021) DocVQA: A Dataset for VQA on Document Images. WACV 2021. https://doi.org/10.1109/wacv48630.2021.00225. +Masry A, Long D X, Tan J Q, Joty S, Hoque E (2022) ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. ACL 2022. https://doi.org/10.18653/v1/2022.findings-acl.177. diff --git a/docs/en/part14/p06_prm.md b/docs/en/part14/p06_prm.md index 5692d4ed..44ded33e 100644 --- a/docs/en/part14/p06_prm.md +++ b/docs/en/part14/p06_prm.md @@ -57,14 +57,13 @@ The main text retains only the key implementation excerpts that illustrate desig Acceptance metrics include step annotation consistency, validation pass rate, positive-to-negative sample ratio, repair coverage, PRM discriminability, and spot-check error rate. If the project enters production, a curriculum, or a public reproduction environment, version numbers, dependency environments, random seeds, sample spot-check results, and failure sample post-mortem records should also be documented. +*Table P06-1: Process Supervision Data Publication Acceptance Table.* | Acceptance Dimension | Metric / Evidence | Publication Review Criteria | | --- | --- | --- | | Process Labels | Step annotation consistency, reward bucket distribution, and validation pass rate | Verify that positive, negative, and repair trajectories each have independent evidence upon spot-check | | Training Interface | PRM sample field completeness rate, training split records, and manifest | Each data batch should be traceable back to task, trajectory, validator, and label source | | Noise Control | Negative sample contamination rate, ambiguous repair trajectory samples, and manual review conclusions | Do not treat samples not covered by the validator directly as high-quality supervision signals | -*Table P06-1: Process Supervision Data Publication Acceptance Table* - ## Cost, Risk, and Compliance Boundaries Costs arise primarily from multi-path reasoning generation and validation; risks are concentrated in incorrect trajectories being labeled as positive examples, insufficient validator coverage, and process label noise. When external data, personal information, copyrighted content, or third-party services are involved, source descriptions, permission status, desensitization strategies, call records, and manual review records must be retained. @@ -1138,8 +1137,8 @@ As part of Part Fourteen, this chapter corresponds to the project-level validati ## References -1. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022. arXiv:2201.11903. -2. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K. (2023). Let's Verify Step by Step. arXiv:2305.20050. -3. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. -4. DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. -5. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., & Steinhardt, J. (2021). Measuring Mathematical Problem Solving With the MATH Dataset. NeurIPS 2021. arXiv:2103.03874. +Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le Q V, Zhou D (2022) Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022. arXiv:2201.11903. +Lightman H, Kosaraju V, Burda Y, Edwards H, Baker B, Lee T, Leike J, Schulman J, Sutskever I, Cobbe K (2023) Let's Verify Step by Step. arXiv:2305.20050. +Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan K, Cao Y (2023) ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. +DeepSeek-AI (2025) DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. +Hendrycks D, Burns C, Kadavath S, Arora A, Basart S, Tang E, Song D, Steinhardt J (2021) Measuring Mathematical Problem Solving With the MATH Dataset. NeurIPS 2021. arXiv:2103.03874. diff --git a/docs/en/part14/p07_agent_tooluse.md b/docs/en/part14/p07_agent_tooluse.md index f037cb86..0fe37074 100644 --- a/docs/en/part14/p07_agent_tooluse.md +++ b/docs/en/part14/p07_agent_tooluse.md @@ -57,14 +57,13 @@ The main text retains only the key implementation snippets that illuminate desig Acceptance metrics include tool-call validity rate, parameter completeness, trajectory success rate, recovery path coverage, block scenario proportion, and format compliance rate. If the project enters production, course, or public reproduction experimental environments, version numbers, dependency environments, random seeds, sample audit results, and failure post-mortem records should also be documented. +*Table P07-1: Agent Tool-Use Publication Acceptance Checklist.* | Acceptance Dimension | Metric / Evidence | Publication Review Criteria | | --- | --- | --- | | Tool Contract | Schema field completeness rate, parameter validity rate, and environment execution records | Each tool-call type should have defined inputs, outputs, errors, and permission boundaries | | Behavior Trajectory | Proportions of success, recovery, and block trajectories, and task success rate | Do not show only success paths; retain failure-recovery and safety-block samples | | Safety Boundary | Unsafe block coverage, permission denial records, and manual sampling conclusions | Public samples must not induce real-world high-risk tool execution | -*Table P07-1: Agent Tool-Use Publication Acceptance Checklist* - ## Cost, Risk, and Compliance Boundaries Costs arise primarily from trajectory generation, execution simulation, and manual sampling; risks concentrate around misconfigured tool permissions, non-executable trajectories, and post-training misuse. When external data, personal information, copyrighted content, or third-party services are involved, source documentation, permission status, anonymization strategy, call records, and manual review records should be retained. Risk identification may reference the NIST AI RMF (NIST 2023) and OWASP LLM Application Risk List (OWASP Foundation 2025); observability of execution trajectories and call logs may follow OpenTelemetry recording practices (OpenTelemetry Authors 2026). @@ -1163,8 +1162,8 @@ As part of Part Fourteen, this chapter corresponds to the project-level validati ## References -1. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. -2. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761. -3. NIST. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology. -4. OWASP Foundation. (2025). OWASP Top 10 for Large Language Model Applications. https://genai.owasp.org/llm-top-10/. -5. OpenTelemetry Authors. (2026). OpenTelemetry Documentation. https://opentelemetry.io/docs/. +Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan K, Cao Y (2023) ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. +Schick T, Dwivedi-Yu J, Dessì R, Raileanu R, Lomeli M, Hambro E, Zettlemoyer L, Cancedda N, Scialom T (2023) Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761. +NIST (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology. +OWASP Foundation (2025) OWASP Top 10 for Large Language Model Applications. https://genai.owasp.org/llm-top-10/. +OpenTelemetry Authors (2026) OpenTelemetry Documentation. https://opentelemetry.io/docs/. diff --git a/docs/en/part14/p08_dataops.md b/docs/en/part14/p08_dataops.md index 55436238..fb8a270f 100644 --- a/docs/en/part14/p08_dataops.md +++ b/docs/en/part14/p08_dataops.md @@ -83,8 +83,7 @@ Acceptance metrics include version traceability, experiment-record completeness, If the project enters production, coursework, or public reproduction, reports should also record version numbers, dependency environments, random seeds, sampled inspection results, and failed-sample reviews. -*Table P08-1: Publication acceptance table for the DataOps platform* - +*Table P08-1: Publication acceptance table for the DataOps platform.* | Acceptance dimension | Metric or evidence | Publication review focus | | --- | --- | --- | | Governance objects | Completeness of tenants, projects, roles, versions, experiments, and events | Platform explanation should prioritize object relationships, not only UI pages or scripts | @@ -1590,8 +1589,8 @@ Readers can combine this case with the data recipes in Part 13, the platform-gov ## References -1. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. -2. Hugging Face. (2026). Datasets Documentation. https://huggingface.co/docs/datasets/. -3. Ray Project. (2026). Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. -4. MLflow Authors. (2026). MLflow Documentation. https://mlflow.org/docs/latest/. -5. Great Expectations Contributors. (2026). Great Expectations Documentation. https://docs.greatexpectations.io/. +Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P J (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. +Hugging Face (2026) Datasets Documentation. https://huggingface.co/docs/datasets/. +Ray Project (2026) Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. +MLflow Authors (2026) MLflow Documentation. https://mlflow.org/docs/latest/. +Great Expectations Contributors (2026) Great Expectations Documentation. https://docs.greatexpectations.io/. diff --git a/docs/en/part14/p09_privacy_pipeline.md b/docs/en/part14/p09_privacy_pipeline.md index 78967458..4588cff2 100644 --- a/docs/en/part14/p09_privacy_pipeline.md +++ b/docs/en/part14/p09_privacy_pipeline.md @@ -59,6 +59,7 @@ The main text retains only the key implementation fragments needed to illustrate Acceptance metrics include PII detection rate, false-positive/false-negative samples, redacted field coverage, audit chain integrity, policy hit rate, and inspection pass rate. If the project enters a production, course, or publicly reproducible experimental environment, the version number, dependency environment, random seed, sample spot-check results, and failed-sample postmortem records should also be logged. +*Table P09-1: Privacy Pipeline Publication Acceptance Table.* | Acceptance Dimension | Metrics / Evidence | Publication Review Criteria | | --- | --- | --- | | Detection and handling | PII detection rate, false-positive/false-negative samples, and redacted field coverage | Each sensitive field category should have a rule source, processing action, and verification sample | @@ -66,8 +67,6 @@ Acceptance metrics include PII detection rate, false-positive/false-negative sam | Release gate | Preflight results, version freeze, rollback path, and responsible owner | Align item by item with Appendix B; confirm that release, takedown, rollback, and postmortem evidence are complete | | Compliance boundaries | Cross-border transfer, over-redaction, permission misconfiguration, and retention period risks | Do not present rule-based prototypes as conclusions of full regulatory compliance | -*Table P09-1: Privacy Pipeline Publication Acceptance Table* - ## Cost, Risk, and Compliance Boundaries Costs arise primarily from detection rule maintenance, manual review, and compliance record-keeping; risks concentrate around under-redaction, over-redaction, cross-border data transfers, and permission misconfiguration. When external data, personal information, copyrighted content, or third-party services are involved, source documentation, permission status, redaction policy, invocation records, and manual review records should be retained. @@ -1126,8 +1125,8 @@ As part of Part XIV, this chapter corresponds to the project-level validation of ## References -1. European Union. (2016). Regulation (EU) 2016/679: General Data Protection Regulation. https://eur-lex.europa.eu/eli/reg/2016/679/oj. -2. NIST. (2020). NIST Privacy Framework: A Tool for Improving Privacy through Enterprise Risk Management, Version 1.0. https://doi.org/10.6028/nist.cswp.10. -3. Dwork, C., & Roth, A. (2014). The Algorithmic Foundations of Differential Privacy. *Foundations and Trends in Theoretical Computer Science*. https://doi.org/10.1561/9781601988195. -4. Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., et al. (2021). Advances and Open Problems in Federated Learning. *Foundations and Trends in Machine Learning*. https://doi.org/10.1561/2200000083. -5. OWASP Foundation. (2025). OWASP Top 10 for Large Language Model Applications. https://genai.owasp.org/llm-top-10/. +European Union (2016) Regulation (EU) 2016/679: General Data Protection Regulation. https://eur-lex.europa.eu/eli/reg/2016/679/oj. +NIST (2020) NIST Privacy Framework: A Tool for Improving Privacy through Enterprise Risk Management, Version 1.0. https://doi.org/10.6028/nist.cswp.10. +Dwork C, Roth A (2014) The Algorithmic Foundations of Differential Privacy. Foundations and Trends in Theoretical Computer Science. https://doi.org/10.1561/9781601988195. +Kairouz P, McMahan H B, Avent B, Bellet A, Bennis M, Bhagoji A N, et al. (2021) Advances and Open Problems in Federated Learning. Foundations and Trends in Machine Learning. https://doi.org/10.1561/2200000083. +OWASP Foundation (2025) OWASP Top 10 for Large Language Model Applications. https://genai.owasp.org/llm-top-10/. diff --git a/docs/en/part14/p10_flywheel.md b/docs/en/part14/p10_flywheel.md index 17a7a15a..54365cae 100644 --- a/docs/en/part14/p10_flywheel.md +++ b/docs/en/part14/p10_flywheel.md @@ -83,8 +83,7 @@ Acceptance metrics include asset coverage, feedback-loop rate, version iteration For production, course, or public reproduction settings, reports should also record version numbers, dependency environments, random seeds, sampled inspection results, and failed-sample review records. -*Table P10-1: Publication acceptance table for the LLM data flywheel* - +*Table P10-1: Publication acceptance table for the LLM data flywheel.* | Acceptance dimension | Metric or evidence | Publication review focus | | --- | --- | --- | | Asset integration | Upstream project coverage, registry completeness, and interface mapping records | Every upstream asset should explain its source, owner, version, and reuse path | @@ -1565,8 +1564,8 @@ Readers can combine this case with the data recipes in Part 13, the platform-gov ## References -1. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. -2. Hugging Face. (2026). Datasets Documentation. https://huggingface.co/docs/datasets/. -3. Ray Project. (2026). Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. -4. MLflow Authors. (2026). MLflow Documentation. https://mlflow.org/docs/latest/. -5. Great Expectations Contributors. (2026). Great Expectations Documentation. https://docs.greatexpectations.io/. +Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P J (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140), 1-67. +Hugging Face (2026) Datasets Documentation. https://huggingface.co/docs/datasets/. +Ray Project (2026) Ray Data Documentation. https://docs.ray.io/en/latest/data/data.html. +MLflow Authors (2026) MLflow Documentation. https://mlflow.org/docs/latest/. +Great Expectations Contributors (2026) Great Expectations Documentation. https://docs.greatexpectations.io/. diff --git a/docs/en/part14/p11_mini_deepseek.md b/docs/en/part14/p11_mini_deepseek.md index c2585a5b..ca2dbc59 100644 --- a/docs/en/part14/p11_mini_deepseek.md +++ b/docs/en/part14/p11_mini_deepseek.md @@ -40,8 +40,7 @@ The body of the chapter retains only the key implementation fragments that illus Acceptance metrics include token distribution, corpus-mix deviation, packing efficiency, training loss trend, throughput, GPU memory/cost, and failed-sample review. If the project enters production, a course, or a public reproduction experiment environment, the version number, dependency environment, random seed, sample spot-check results, and failed-sample retrospective records should also be logged. -*Table P11-1: Mini-DeepSeek Pre-Training Reproduction Publication Acceptance Table* - +*Table P11-1: Mini-DeepSeek Pre-Training Reproduction Publication Acceptance Table.* | Acceptance Dimension | Metric / Evidence | Publication Review Criterion | | --- | --- | --- | | Recipe reproduction | Corpus-mix deviation, cross-source deduplication records, and tokenizer training logs | A reduced-scale experiment must state the scale difference from the original recipe and the boundaries of non-comparability | @@ -88,6 +87,7 @@ The four core components of the pipeline are: Table P11-2 maps each architectural component to its code entry point, stage artifact, and review fields. Project chapters need to retain tables of this kind because they connect "the engineering narrative visible to the reader" with the scripts in `code/zh/project_11_mini_deepseek`, preventing the chapter from remaining at the level of conceptual introduction alone. +*Table P11-2: Mini-DeepSeek Data Pipeline Stage Artifacts and Code Entry Points.* | Stage | Code Entry Point | Primary Input | Primary Output | Review Fields | | --- | --- | --- | --- | --- | | Multi-source sampling | `mix_sampler.py` | `RECIPE`, target document count, Hugging Face data sources | `./data/mixed_1b_raw` | `source`, sample count, recipe weight deviation | @@ -97,8 +97,6 @@ Table P11-2 maps each architectural component to its code entry point, stage art | End-to-end run | `run_pipeline.sh` | Stage scripts and local environment | Complete data directory | Logs, failed stages, artifact integrity | | Unit tests | `tests/test_pipeline.py` | Recipe, MinHash, packing constants | Test report | Weights sum to 1, MinHash similarity, `SEQ_LEN` | -*Table P11-2: Mini-DeepSeek Data Pipeline Stage Artifacts and Code Entry Points* - The most easily overlooked column in Table P11-2 is "Review Fields." For example, `mixed_1b_raw` is simply a Hugging Face Dataset directory on its own and says nothing about whether the recipe is correct; one must additionally verify that the sample count for each `source` is consistent with the target weights. Similarly, `mixed_1b_dedup` cannot be validated merely by checking whether the directory exists—the duplicate sample ratio and threshold must also be recorded. For the tokenizer, the existence of the file does not indicate training success; special tokens, vocabulary size, Chinese/code compression ratio, and rare-character coverage must also be checked. ## Step-by-Step Implementation @@ -236,6 +234,7 @@ This command sequentially generates `mixed_1b_raw`, `mixed_1b_dedup`, `mini_deep Table P11-3 lists the minimal audit information that should be recorded before and after a run. +*Table P11-3: Mini-DeepSeek Minimal Reproduction Experiment Record Items.* | Category | Record Item | Purpose | | --- | --- | --- | | Data version | Data source repo id, split, config name, sampling time | Explains sample distribution changes | @@ -246,14 +245,13 @@ Table P11-3 lists the minimal audit information that should be recorded before a | Environment information | Python, datasets, tokenizers, datasketch versions | Diagnoses artifact differences | | Run results | Sample count, token count, directory size, failure logs | Determines deliverability | -*Table P11-3: Mini-DeepSeek Minimal Reproduction Experiment Record Items* - ## Data Quality and Recipe Acceptance The acceptance focus for Mini-DeepSeek is not the final model score but whether the recipe is interpretable, verifiable, and extensible. A common pitfall is reporting only "1B tokens were ultimately obtained" without stating which domains those tokens came from, how many cross-source duplicates were removed, or whether the tokenizer is biased toward a particular language or code domain. For pre-training data, quantity is merely the outcome; the recipe is the core. Table P11-4 gives the recipe-level acceptance criteria. +*Table P11-4: Mini-DeepSeek Recipe-Level Acceptance Checklist.* | Acceptance Item | Recommended Check | Non-conformance Indicator | | --- | --- | --- | | Weight consistency | Deviation between each data source's sample count and its `RECIPE` weight | A source is over-sampled or underestimated due to streaming interruption | @@ -264,8 +262,6 @@ Table P11-4 gives the recipe-level acceptance criteria. | Packing integrity | All `input_ids` have length `4096` | Padding or length inconsistency occurs during training | | Randomness | Fixed shuffle seed and sampling strategy | Sample order or distribution across multiple runs is inexplicable | -*Table P11-4: Mini-DeepSeek Recipe-Level Acceptance Checklist* - In practice, begin by sampling 100 records from each source for manual inspection to verify that the text type matches expectations; then spot-check near-duplicate pairs from the MinHash-deleted samples to assess whether the threshold is too strict. If the duplication rate is anomalously high, it may indicate genuine overlap between data sources, or it may indicate that 5-grams are overly sensitive to short texts. If the duplication rate is anomalously low, check whether the text field was selected correctly—some datasets use `content` rather than `text`. ## Fine-Grained Inspection of Tokenizer and Packing @@ -274,6 +270,7 @@ The tokenizer training stage most easily produces the problem of "file creation It is recommended to compute the metrics in Table P11-5 after training is complete. +*Table P11-5: Tokenizer and Packing Quality Metrics.* | Metric | Computation Method | Interpretation | | --- | --- | --- | | Chinese tokens/char | Token count of Chinese web page samples divided by character count | Assesses Chinese compression efficiency | @@ -283,8 +280,6 @@ It is recommended to compute the metrics in Table P11-5 after training is comple | `<|endoftext|>` presence | Whether separator tokens exist in packed data | Assesses whether document boundaries are preserved | | OOV behavior | `<|unk|>` usage rate | Assesses whether ByteLevel coverage is normal | -*Table P11-5: Tokenizer and Packing Quality Metrics* - The packing stage also requires inspection. `pack_shuffle.py` concatenates sample tokens and truncates to an integer multiple of `SEQ_LEN`, outputting fixed-length `input_ids`. This improves training throughput but means original document boundaries are no longer directly visible. Therefore, the insertion and counting of `<|endoftext|>` is critically important. If the separator is forgotten, the model learns adjacent documents as continuous text; if separators are overly dense, short documents will dominate the context structure. ## Test Coverage and Code Isolation @@ -293,6 +288,7 @@ The packing stage also requires inspection. `pack_shuffle.py` concatenates sampl Table P11-6 maps tests to gaps. +*Table P11-6: Mini-DeepSeek Test Coverage and Acceptance Gaps.* | Test Item | Covered Content | Still Requires Manual or Integration Acceptance | | --- | --- | --- | | `test_recipe_weights` | Recipe weights sum to 1 | Actual sample count per source | @@ -302,8 +298,6 @@ Table P11-6 maps tests to gaps. | `test_tokenizer_pack` | `SEQ_LEN` constant | Length of each packed sample | | `test_end_to_end_mock` | Test entry point exists | Real end-to-end small-sample run | -*Table P11-6: Mini-DeepSeek Test Coverage and Acceptance Gaps* - Published manuscripts should clearly distinguish "teaching example code" from "production-ready code." The code in this chapter illustrates the basic organization of multi-source sampling, MinHash, BPE, and packing, but real large-scale pre-training additionally requires distributed execution, data-source failure retries, download caching, contamination detection, sensitive-content filtering, license whitelisting, and training framework integration. ## Common Failures and Diagnostic Paths @@ -322,6 +316,7 @@ The goal of P11 is not to report a complete large-model training result but to d Table P11-7 provides a training smoke test record template. +*Table P11-7: Mini-DeepSeek Training Smoke Test Record Template.* | Record Item | Example | Review Significance | | --- | --- | --- | | Data directory | `./data/mixed_1b_final_packed` | Confirms that training reads from the final packed data | @@ -334,8 +329,6 @@ Table P11-7 provides a training smoke test record template. | Tokens/s | Tokens per second | Estimates cost of subsequent scale-up | | Sample decoding | Random decode of 10 packed samples | Checks for garbled content, boundary issues, and repetition | -*Table P11-7: Mini-DeepSeek Training Smoke Test Record Template* - The smoke test report should not simply state "training runs successfully." More useful content includes: read throughput, loss over the first several steps, decoded results from several packed samples, and a record of how failed samples were handled. If the loss fails to decrease over an extended period, this may indicate a model configuration problem or the presence of large amounts of duplicated, garbled, or meaningless tokens in the data. If throughput is below expectation, the `.arrow` shards may be too small, data loading workers may be insufficient, or disk I/O may be the bottleneck. ## Data Contamination and Benchmark Leakage Inspection @@ -344,6 +337,7 @@ Pre-training reproduction projects must pay attention to benchmark contamination Table P11-8 presents a minimal contamination inspection plan. +*Table P11-8: Pre-Training Data Contamination Inspection Plan.* | Inspection Target | Method | Handling | | --- | --- | --- | | Exact duplicates | Hash or normalized string matching against benchmark prompts | Directly remove matched samples | @@ -353,14 +347,13 @@ Table P11-8 presents a minimal contamination inspection plan. | Forum reposts | Check for benchmark problem reposts on web pages or blogs | Remove or flag as contamination risk | | Training report | Save match counts, deletion strategy, and sample ids | Supports subsequent benchmark claims | -*Table P11-8: Pre-Training Data Contamination Inspection Plan* - Contamination inspection should be completed before packing, because once the data enters a packed dataset, original document boundaries and source fields become harder to trace. If a teaching project does not implement complete contamination filtering, the report should explicitly state "not used for public benchmark claims" to prevent readers from mistaking smoke-test scores for publishable results. ## Deliverable Directory and Release Package Structure A deliverable P11 project should contain not only the final data directory but also the recipe, logs, tests, and audit materials. Table P11-9 gives the recommended directory structure. +*Table P11-9: Mini-DeepSeek Project Deliverable Directory.* | Path | Content | Included in Public Release | | --- | --- | --- | | `data/mixed_1b_raw/` | Raw mixed data sampled according to the recipe | Depends on data licenses | @@ -374,14 +367,13 @@ A deliverable P11 project should contain not only the final data directory but a | `tests/` | Unit tests and small-sample tests | May be made public | | `LICENSES.md` | Data source license documentation | Must be made public | -*Table P11-9: Mini-DeepSeek Project Deliverable Directory* - If data sources do not permit redistribution, the recipe, scripts, tokenizer, and report templates may still be published, but raw or packed samples cannot be released directly. In this case, a reproducibility guide should be provided so that readers with the requisite authorization can rebuild the data locally. ## Scaling from 1B Tokens to Larger Scales The reduced-scale pipeline of P11 helps readers understand the recipe, but scaling to 10B, 70B, or larger requires systematic refactoring. Table P11-10 lists the refactoring checklist for moving from the teaching implementation to a production-scale implementation. +*Table P11-10: Mini-DeepSeek Refactoring Checklist from Teaching Implementation to Production Scale.* | Module | Teaching Implementation | Production-Scale Refactoring | | --- | --- | --- | | Data ingestion | Hugging Face streaming + local Dataset | Connect to object storage, data lake, or distributed cache | @@ -392,8 +384,6 @@ The reduced-scale pipeline of P11 helps readers understand the recipe, but scali | Auditing | Manual reports | Metadata service, lineage tracking, deletion-request replay | | Testing | Unit tests + mock e2e | Small-sample real e2e, data-diff regression, contamination scan | -*Table P11-10: Mini-DeepSeek Refactoring Checklist from Teaching Implementation to Production Scale* - The most important point when scaling is not to mistake "the script runs" for "the system is scalable." Large-scale pre-training data systems must handle failure retries, checkpoint-based resumption, version freezing, sample deletion, license changes, in-training data mixing strategy adjustments, and multi-team collaboration. The value of P11 lies in presenting the minimal form of these problems rather than claiming to replace a complete industrial system. ## Data Dashboard and Continuous Monitoring @@ -402,6 +392,7 @@ Once pre-training data engineering enters continuous iteration, one-off run logs Table P11-11 gives the recommended dashboard metrics. +*Table P11-11: Mini-DeepSeek Data Dashboard Metrics.* | Dashboard Metric | Statistics Source | Observation Purpose | | --- | --- | --- | | Source sample count | `mixed_1b_raw` | Check for recipe proportion drift | @@ -415,8 +406,6 @@ Table P11-11 gives the recommended dashboard metrics. | Packed shard size | `mixed_1b_final_packed` | Check training read balance | | Contamination match count | Contamination scan | Record benchmark leakage risk | -*Table P11-11: Mini-DeepSeek Data Dashboard Metrics* - The core value of the dashboard is batch comparison. For example, if The Stack v2 sample count is unchanged in a given run but the token count rises significantly, it may indicate that code samples have become longer or that filtering conditions have changed. If the deduplication rate for OpenWebMath suddenly increases in a given run, it may indicate that templated content in math web pages has increased. Without a batch dashboard, these changes often go undetected until a training loss or benchmark anomaly is observed. ## Deletion Requests and Sample Withdrawal Mechanism @@ -425,6 +414,7 @@ Multi-source pre-training data must have a sample withdrawal capability. Even in Table P11-12 gives the processing path for deletion requests. +*Table P11-12: Pre-Training Sample Deletion Request Processing Path.* | Step | Action | Artifact | | --- | --- | --- | | Receive request | Record the requester, URL, sample characteristics, and timestamp | Deletion request ticket | @@ -434,8 +424,6 @@ Table P11-12 gives the processing path for deletion requests. | Rebuild packed data | Re-run packing and shuffle | Revised packed dataset | | Update report | Update recipe proportions, token counts, and deletion notes | Release note | -*Table P11-12: Pre-Training Sample Deletion Request Processing Path* - Real production systems typically store document-level hashes, URLs, source, license, and a packed-shard reverse index. Teaching implementations do not necessarily require a complete index, but readers should understand that the earlier lineage is discarded, the higher the subsequent withdrawal cost. Retaining this point in the published text prevents readers from mistakenly believing that `.arrow` training data can circulate independently of its origin. ## Domain Transfer: From General Recipe to Domain-Specific Models @@ -444,6 +432,7 @@ The Mini-DeepSeek recipe is dominated by web pages, code, mathematics, academic Table P11-13 gives adjustment directions for domain transfer. +*Table P11-13: Domain Transfer Considerations for the Mini-DeepSeek Recipe.* | Domain | Data to Add | Additional Risks | Acceptance Focus | | --- | --- | --- | --- | | Legal | Regulations, case law, contracts, compliance Q&A | Regional and temporal differences | Statute version, citation accuracy | @@ -452,14 +441,13 @@ Table P11-13 gives adjustment directions for domain transfer. | Industrial | Equipment manuals, fault records, process documents | Internal confidentiality and terminology ambiguity | Permissions, terminology glossary, fault classification | | Education | Textbooks, exercises, solutions, course notes | Copyright and answer leakage | Copyright licenses, question-bank contamination | -*Table P11-13: Domain Transfer Considerations for the Mini-DeepSeek Recipe* - When transferring to a domain, it is recommended to retain the general corpus as a foundation and gradually increase the domain corpus proportion. If the domain data proportion is raised too rapidly, the model may acquire domain terminology capability while losing general language and coding capability. A safer approach is to design a multi-round curriculum: maintain a dominant share of general corpus in the early stages, then increase domain- and task-relevant data proportions in the mid-to-late stages, monitored by domain validation sets. ## Relationship with P01 Mini-C4 P01 focuses on single-source web page cleaning; P11 focuses on a multi-source pre-training recipe. These are not redundant; they represent an upgrade from "cleaning one type of corpus" to "organizing multiple types of corpora." Table P11-14 summarizes the differences. +*Table P11-14: Differences Between P01 Mini-C4 and P11 Mini-DeepSeek.* | Dimension | P01 Mini-C4 | P11 Mini-DeepSeek | | --- | --- | --- | | Data sources | Single source or a small number of web sources | Web pages, code, mathematics, academic papers, Chinese text | @@ -470,8 +458,6 @@ P01 focuses on single-source web page cleaning; P11 focuses on a multi-source pr | Acceptance focus | Text quality and cleaning rules | Recipe, compression ratio, contamination, smoke test | | Scaling direction | Larger web corpus | Larger multi-source pre-training system | -*Table P11-14: Differences Between P01 Mini-C4 and P11 Mini-DeepSeek* - Understanding this relationship helps readers connect the projects in Part 14. P01 is the starting point of data cleaning; P11 organizes multiple cleaned sources into a pre-training recipe. Without the quality filtering of P01, the multi-source recipe of P11 would absorb large amounts of noise; without the recipe organization of P11, the single-source cleaning of P01 is insufficient to support modern general-purpose model training. ## Results Presentation and Analysis diff --git a/docs/en/part14/p12_r1_reasoning_flywheel.md b/docs/en/part14/p12_r1_reasoning_flywheel.md index e91fdbc5..b5129ba6 100644 --- a/docs/en/part14/p12_r1_reasoning_flywheel.md +++ b/docs/en/part14/p12_r1_reasoning_flywheel.md @@ -43,14 +43,13 @@ The main text retains only the key implementation excerpts that illuminate desig Acceptance metrics include reasoning accuracy, candidate retention rate, verification coverage, long-chain length distribution, recirculated sample quality, and cost per sample. If the project enters production, a course, or a public reproducibility experiment environment, the version number, dependency environment, random seeds, sample spot-check results, and failed-sample post-mortem records should also be logged. +*Table P12-1: Publication Acceptance Table for the Pedagogical R1 Reasoning Data Flywheel.* | Acceptance Dimension | Metric / Evidence | Publication Review Criterion | | --- | --- | --- | | Candidate generation | Number of multi-path samples, long-chain length distribution, and task-source coverage | Describe the differences between mock, vLLM, and real model sampling | | Filtering and recirculation | Verifier coverage, candidate retention rate, recirculated sample quality, and format pass rate | Each recirculated sample should be traceable to its original task, candidate trace, and verification result | | Risk control | Post-mortem of self-reinforcing errors, verifier bias, and overly long trace noise | Do not equate a rejection-sampling pass with an improvement in reasoning capability | -*Table P12-1: Publication Acceptance Table for the Pedagogical R1 Reasoning Data Flywheel* - ## Cost, Risk, and Compliance Boundaries Costs arise primarily from long-chain generation, multi-candidate sampling, and verification. Risks concentrate on self-reinforcing errors, verifier bias, and overly long trace noise. When external data, personal information, copyrighted content, or third-party services are involved, source descriptions, permission status, desensitization strategies, call logs, and manual review records should be retained. @@ -111,6 +110,7 @@ The sixth component is **training and evaluation**. `train_lora.py` provides a m The main artifacts are as follows: +*Table P12-2: Stage and Description Reference Table.* | Stage | Default Artifact | Description | | --- | --- | --- | | Cold-start extraction | `data/processed/cold_start_5k.jsonl` | First-round SFT samples | @@ -122,8 +122,6 @@ The main artifacts are as follows: | Training record | `data/training/training_manifest.json` | Training data composition | | Evaluation results | `data/reports/eval_results_gsm8k_math.json` | GSM8K/MATH comparison results | -*Table P12-2: Stage and Description Reference Table* - --- ## 3. Step-by-Step Implementation: From Cold-Start Samples to a Recirculable SFT Dataset @@ -458,6 +456,7 @@ It should be emphasized that LoRA and the evaluation script in this project are The final output of this project is not a single score table but a set of reviewable data assets. The minimum acceptable results should include: +*Table P12-3: Artifact and Checkpoint Reference Table.* | Artifact | Checkpoint | | --- | --- | | `cold_start_5k.jsonl` | Fields are complete; `messages` can be used directly for SFT | @@ -469,12 +468,11 @@ The final output of this project is not a single score table but a set of review | `training_manifest.json` | Merge scale and domain distribution are recorded | | `eval_results_gsm8k_math.json` | Base and LoRA evaluation results can be compared | -*Table P12-3: Artifact and Checkpoint Reference Table* - From an engineering perspective, acceptance can be divided into three tiers. The first tier is pipeline acceptance: `pytest -q` passes, and mock mode can complete cold start, sampling, verification, rejection sampling, merging, training, and evaluation. The second tier is real-sampling acceptance: the vLLM service can be called by `sample_traces.py`, sampling results enter `sampled_traces`, and can be processed by the verifier. The third tier is performance acceptance: after LoRA training, there is a stable gain over the base model on GSM8K/MATH. The current project prioritizes the first two tiers; the third tier requires larger-scale data and multiple rounds of hyperparameter tuning. In terms of cost, the primary expenses come from multi-path sampling and training. If resources are constrained, the following fallback strategies can be applied: +*Table P12-4: Resource Bottleneck and Fallback Strategy Reference Table.* | Resource Bottleneck | Fallback Strategy | | --- | --- | | Insufficient VRAM | Reduce `max_model_len`, `max_num_seqs`, or concurrent prompts | @@ -484,8 +482,6 @@ In terms of cost, the primary expenses come from multi-path sampling and trainin | Slow LoRA training | First run a smoke train with `--max-train-samples 1024` | | Long evaluation time | Start with `--max-examples 100`, then expand the evaluation scale | -*Table P12-4: Resource Bottleneck and Fallback Strategy Reference Table* - ## Chapter Summary This chapter uses the "Pedagogical R1 Reasoning Data Flywheel" as a case study to demonstrate the engineering organization of a closed-loop reasoning data pipeline covering Long-CoT cold start, rejection sampling, and recirculated SFT. The primary value of this case lies in placing task definition, data boundaries, architectural decisions, sample schemas, metric acceptance, and reproducibility resources in a single chain, transforming the project from a sequence of operational steps into a reviewable case study. diff --git a/docs/en/part14/p13_multimodal_instruction_factory.md b/docs/en/part14/p13_multimodal_instruction_factory.md index f916864a..8ca988c5 100644 --- a/docs/en/part14/p13_multimodal_instruction_factory.md +++ b/docs/en/part14/p13_multimodal_instruction_factory.md @@ -42,6 +42,7 @@ The chapter keeps only implementation fragments that explain design trade-offs. Acceptance metrics include task coverage, image-text consistency, OCR usability, format pass rate, safety-filtering rate, and manual spot-check quality. If the project enters production, a course environment, or a public reproduction environment, it should also record version numbers, dependency environment, random seeds, sample spot-check results, and failure-sample review records. +*Table P13-1: Publication acceptance table for the multimodal instruction factory.* | Acceptance dimension | Metric / evidence | Publication review rule | | --- | --- | --- | | Task coverage | Ratio of description, OCR, chart, grounding, and multi-turn QA tasks | Task types must correspond to data sources, model capability, and downstream training goals | @@ -49,8 +50,6 @@ Acceptance metrics include task coverage, image-text consistency, OCR usability, | Multilingual expansion | Ratio of Chinese, English, and translated samples; cross-language terminology consistency; format-preservation rate | Multilingual samples must not be judged by quantity only; semantic consistency, visual reference, and proper-name translation require sampling review | | Copyright safety | Image authorization, sensitive-content interception, redistribution boundary | Public examples should prefer authorized or owned assets; external images require separate registration | -*Table P13-1: Publication acceptance table for the multimodal instruction factory.* - ## Cost, Risk, and Compliance Boundaries Cost mainly comes from vision-language models, OCR, and manual review. Risk concentrates in image authorization, sensitive content, hallucinated descriptions, and task homogenization. When external data, personal information, copyrighted material, or third-party services are involved, retain source notes, permission status, masking strategy, call records, and manual-review records. @@ -92,6 +91,7 @@ The factory is divided into five components, shown in Figure 13-1. Table P13-2 maps architecture components to code entry points and key artifacts. Unlike P03, P13 does not walk through LLaVA image-text preparation again. Its focus is how a modern multimodal instruction factory organizes seed selection, templates, generation, filtering, expansion, packaging, and acceptance into a reviewable chain. +*Table P13-2: Stage artifacts and code entry points for the multimodal instruction factory.* | Stage | Code entry | Main input | Main output | Key review point | | --- | --- | --- | --- | --- | | Seed selection | `seed_selector.py` | LAION metadata or private visual-asset manifest | Seed list | Resolution, aspect ratio, original caption length, authorization status | @@ -103,8 +103,6 @@ Table P13-2 maps architecture components to code entry points and key artifacts. | Unified packaging | `pack_multi_image_video.py` | Scored records | `mm_sft_final.jsonl` | Qwen format, image/video paths, conversation fields | | Unit tests | `tests/test_factory.py` | Template, judge, expansion, packaging functions | Test report | Basic contracts and example-output completeness | -*Table P13-2: Stage artifacts and code entry points for the multimodal instruction factory.* - The key function of Table P13-2 is to split "generation" out of a single model call. In real projects, VLM generation is only the middle of the pipeline. Before it, controlled seeds and task templates are required; after it, consistency checks, score filtering, multilingual review, and format packaging are required. If only the generation script is kept, the chapter becomes a demo. If stage artifacts and review fields are kept, the chapter has the engineering depth expected of a project chapter. ## Step-by-Step Implementation @@ -329,6 +327,7 @@ pack_to_qwen_format(expanded, "./data/mm_sft_final.jsonl") This code describes the factory's minimal closed loop, but it is not yet a production script. Production runs need four additional controls. First, model calls must record model path, temperature, top-p, max tokens, and concurrency. Second, seeds must record source, authorization, and download status. Third, judge output must retain the scoring prompt, threshold, and human calibration set. Fourth, before packaging, the pipeline must check image links, conversation format, and sample deduplication. +*Table P13-3: Runtime records for the multimodal instruction factory.* | Category | Record item | Purpose | | --- | --- | --- | | Asset version | Image source, URL, authorization, download time | Proves sample traceability | @@ -339,12 +338,11 @@ This code describes the factory's minimal closed loop, but it is not yet a produ | Packaging version | Output format, field schema, target training framework | Ensures training scripts can read the data | | Spot-check record | Human samples, failure samples, revision notes | Supports release gates | -*Table P13-3: Runtime records for the multimodal instruction factory.* - ## Data Schema and Sample Contract A minimal multimodal instruction record cannot contain only `image`, `instruction`, and `response`. The project chapter should emphasize that training formats can be narrow, but engineering intermediate states must be wider. Otherwise, once hallucination, format errors, or copyright problems appear, the data team cannot trace a sample back to its image, template, model call, or filtering step. +*Table P13-4: Intermediate-state sample schema for the multimodal instruction factory.* | Field | Example | Meaning | | --- | --- | --- | | `sample_id` | `p13_laion_000001` | Stable primary key across logs | @@ -360,14 +358,13 @@ A minimal multimodal instruction record cannot contain only `image`, `instructio | `language` | `en`, `zh` | Distinguishes multilingual samples | | `audit_trace` | Batch, timestamp, script version | Supports review and takedown | -*Table P13-4: Intermediate-state sample schema for the multimodal instruction factory.* - The final `mm_sft_final.jsonl` file in Qwen format can be narrower than the intermediate state, but the intermediate state should not be discarded. Training files serve the training framework; audit files serve quality and release. They can be joined by `sample_id`. ## Quality Filtering: From Length Thresholds to Calibrated Rubrics The demonstration `llm_judge.py` uses response length as a proxy: answers above a certain word count receive 4.5, while shorter answers receive 3.0. This is acceptable for teaching, but not for a real release gate. A real LLM-as-Judge setup should include at least four scoring dimensions: image-text consistency, answer completeness, task following, and safety/compliance. +*Table P13-5: LLM-as-Judge scoring rubric for multimodal instruction samples.* | Scoring dimension | 5-point behavior | Low-score risk | | --- | --- | --- | | Image-text consistency | Describes only content supported by visual evidence | Hallucinates subjects, actions, or text | @@ -377,14 +374,13 @@ The demonstration `llm_judge.py` uses response length as a proxy: answers above | Safety and compliance | Avoids sensitive identity inference and improper content | Privacy, bias, or dangerous guidance | | Language quality | Clear expression without severe repetition | Mechanical repetition, garbling, or abnormal language mixing | -*Table P13-5: LLM-as-Judge scoring rubric for multimodal instruction samples.* - Self-consistency complements judge blind spots. For complex reasoning questions, the model can generate multiple answers, then compare whether conclusions and key evidence agree. If different samples conflict on subjects, text, or spatial relationships, the record should not enter the training set even if one answer is long and fluent. The current `self_consistency.py` is a simplified interface and teaching implementation; real projects should plug in multi-sample generation and consistency metrics. ## Multilingual Expansion and Cross-Language Acceptance Multilingual expansion is not simply copying an English instruction and adding an `instruction_zh` field. In multimodal tasks, cross-language errors often occur in visual references and proper-name translation. For example, "the sign on the left" may be translated as "the sign on the right," or brands, place names, and units may be localized incorrectly. P13 should treat Chinese and English as two sample sets that both require spot checks, not as a cheap way to double the count. +*Table P13-6: Multilingual expansion acceptance items.* | Acceptance item | Check method | Common issue | | --- | --- | --- | | Reference consistency | Compare image against left/right, top/bottom, foreground/background | Direction words mistranslated | @@ -393,14 +389,13 @@ Multilingual expansion is not simply copying an English instruction and adding a | Format preservation | Check tables, lists, JSON, Markdown | Translation breaks structure | | Safety boundary | Check whether sensitive content bypasses filtering in another language | English filtering works but Chinese filtering fails | -*Table P13-6: Multilingual expansion acceptance items.* - If the project targets Chinese-model training, do not only translate English samples into Chinese. Keep a portion of native Chinese templates and native Chinese judge prompts. Translated samples are useful for scale, but native Chinese samples better reflect real Chinese user questions. ## Test Coverage and Code Notes `tests/test_factory.py` covers template existence, random prompt return type, judge filtering, Chinese expansion, and JSONL packaging. These tests prevent basic interface breakage, but they do not prove the factory is releasable. In particular, `generate_with_qwen_vl.py` is a teaching example. Before real vLLM or Qwen-VL integration, it needs input variables, exception handling, model-call result parsing, and failed-sample records. The chapter presents it to explain the generation-stage interface, not to claim production completeness. +*Table P13-7: Test coverage and acceptance gaps for the multimodal instruction factory.* | Test item | Covered | Still needed | | --- | --- | --- | | Template test | Three template types exist; prompt returns a string | Template repetition rate, task ratio | @@ -409,8 +404,6 @@ If the project targets Chinese-model training, do not only translate English sam | Packaging test | JSONL file can be written | Conversation-field spot check | | End-to-end mock | Test entry exists | Small-sample real model run | -*Table P13-7: Test coverage and acceptance gaps for the multimodal instruction factory.* - ## Common Faults and Troubleshooting Paths Seed-stage issues usually involve dead image links, missing image-size fields, or low-quality original captions. First count filtering reasons rather than only final seed count. If many images are removed by aspect-ratio filters, confirm field units and source schema. @@ -425,6 +418,7 @@ Packaging-stage issues involve mismatch among image URLs, `` markers, and Multimodal instruction factories often look good on automated metrics while failing under human reading. High-scoring samples may be fluent but unfaithful to the image; multilingual samples may be grammatical but wrong on direction, count, or OCR text. Manual spot checks are therefore mandatory before release. +*Table P13-8: Manual review strata for the multimodal instruction factory.* | Review layer | Sample source | Review focus | | --- | --- | --- | | High-score samples | Highest judge-score batch | Whether the judge over-rewards long text | @@ -435,12 +429,11 @@ Multimodal instruction factories often look good on automated metrics while fail | Chinese samples | Multilingual expansion results | Terminology, direction, proper names | | Multi-image / video samples | Packer extensions | Reference order and placeholders | -*Table P13-8: Manual review strata for the multimodal instruction factory.* - Manual review should use dual review plus arbitration. The first reviewer checks image-text consistency and task following. The second checks language quality and safety boundaries. Conflicts enter an arbitration pool, which is then used to revise judge prompts, templates, and thresholds. Manual review is not a one-time quality check; it is part of factory iteration. Release gates should include at least four checks. First, sample sources must be traceable, and external images must not be represented only by naked URLs. Second, training files must be readable by the target framework, not merely valid JSON. Third, there must be an agreement report between judge and human review. Fourth, if multilingual samples are released, Chinese and English quality must be reported separately. +*Table P13-9: Release-gate checklist for the multimodal instruction factory.* | Gate | Required evidence | Action on failure | | --- | --- | --- | | Source gate | URL, license, download status, deletion-request handling | Remove unauthorized or untraceable samples | @@ -450,12 +443,11 @@ Release gates should include at least four checks. First, sample sources must be | Safety gate | Sensitive content, privacy, identity-inference checks | Delete samples and update filtering rules | | Version gate | Model version, template version, run batch | Freeze versions before release | -*Table P13-9: Release-gate checklist for the multimodal instruction factory.* - ## Multi-Image and Video Extension Path The presence of `pack_multi_image_video.py` indicates that this project targets more than single-image SFT. Modern VLM training increasingly depends on interleaved images, multi-image comparison, and short video clips. The core issue is not concatenating several `` tags, but making the instruction clearly point to each visual input and making the answer explicitly express comparison, ordering, temporal change, or cross-image relation. +*Table P13-10: Comparison of multimodal instruction types.* | Type | Input organization | Instruction focus | Common error | | --- | --- | --- | --- | | Single image | One `` | Description, OCR, local reasoning | Hallucinated object or text | @@ -464,14 +456,13 @@ The presence of `pack_multi_image_video.py` indicates that this project targets | Short video | Multiple frames or `