文本切分五个层级：
Level 1: Character Splitting - 简单的字符长度切分
Level 2: Recursive Character Text Splitting - 通过分隔符切分，然后递归合并
Level 3: Document Specific Splitting - 针对不同文档格式切分 (PDF, Python, Markdown)
Level 4: Semantic Splitting - 语义切分
Level 5: Agentic Splitting-使用代理实现自动切分
为什么需要Recursive Character Text Splitting
按字符长度切分的问题在于，根本没有考虑到文档的结构。只是按固定数量的字符分割，这样很容易造成文档被随意切分开了，导致最后的效果不好。
递归字符文本分割器可以帮助解决这个问题。有了它，切分的过程可描述为：我们先指定一系列分隔符，这些分隔符用于分割文档，然后将分割好的小块合并，达到指定的长度。
它是切分器中最基础的做法，也是快速搭建应用流程的首选。如果在不知道选择哪个切分器的时候，这是一个不错的选择。
我们看看在langchain中的默认分隔符["\n\n", "\n", " ", ""]：
看看下面例子，在一般情况下，我们希望的切分情况，如何能实现它？
text = """One of the most important things I didn't understand about the world when I was a child is the degree to which the returns for performance are superlinear.Teachers and coaches implicitly told us the returns were linear. "You get out," I heard a thousand times, "what you put in." They meant well, but this is rarely true. If your product is only half as good as your competitor's, you don't get half as many customers. You get no customers, and you go out of business.It's obviously true that the returns for performance are superlinear in business. Some think this is a flaw of capitalism, and that if we changed the rules it would stop being true. But superlinear returns for performance are a feature of the world, not an artifact of rules we've invented. We see the same pattern in fame, power, military victories, knowledge, and even benefit to humanity. In all of these, the rich get richer. [1]"""
可以很直观地将它切分成三个段落：
如何实现
加载依赖方法
from langchain.text_splitter import RecursiveCharacterTextSplitter
构建切分器
text_splitter = RecursiveCharacterTextSplitter(chunk_size=65, chunk_overlap=0)text_splitter.create_documents([text])# chunk_size 每个片段的字符数# chunk_overlap 片段之间的重叠字符数
text_splitter = RecursiveCharacterTextSplitter(chunk_size=450, chunk_overlap=0)text_splitter.create_documents([text])
总共生成3段，这就是我们预期的切分效果。
仅仅使用字符个数作为小块内的切分依据，似乎不是一个好的方案；相同语义下，中英文、数字等字符个数差别太大了。下篇文章我们将介绍如何优化小块内的切分和计数方案。
