中文预训练语料去重:从 MinHash 到段落级指纹
整理一次中文网页语料去重的实践:为什么文档级 MinHash 不够用,段落级指纹怎么做,以及如何验证去重没有误伤高质量内容。
阅读全文整理一次中文网页语料去重的实践:为什么文档级 MinHash 不够用,段落级指纹怎么做,以及如何验证去重没有误伤高质量内容。
阅读全文Notes on building a small, fixed question set for a RAG system so it measures correctness instead of quietly agreeing with whatever the current index returns.
A working note on the question this site keeps returning to: what evidence should technical work leave behind, and how to design pipelines so the evidence is a byproduct, not a chore.
A repeatable loop for measuring retrieval quality, citation grounding, and refusal behavior in a small knowledge-base assistant — built so each run leaves behind debuggable evidence.