> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-zh.cn/llms.txt
> Use this file to discover all available pages before exploring further.

# 拆分 Markdown - 文本分割器集成

> 使用 LangChain Python 与拆分 Markdown 的文本分割器集成。

许多聊天或问答应用程序在嵌入和向量存储之前涉及对输入文档进行分块。

[Pinecone 的这份笔记](https://www.pinecone.io/learn/chunking-strategies/) 提供了一些有用的提示：

```wrap theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
当整个段落或文档被嵌入时，嵌入过程会考虑整体上下文以及文本内句子和短语之间的关系。这可能会导致更全面的向量表示，捕捉到文本的更广泛含义和主题。
```

如前所述，分块通常旨在将具有共同上下文的文本保持在一起。考虑到这一点，我们可能希望特别尊重文档本身的结构。例如，Markdown 文件是按标题组织的。在特定的标题组内创建分块是一个直观的想法。为了解决这个挑战，我们可以使用 [MarkdownHeaderTextSplitter](https://reference.langchain.com/python/langchain-text-splitters/markdown/MarkdownHeaderTextSplitter)。这将根据指定的一组标题拆分 Markdown 文件。

例如，如果我们想拆分以下 Markdown：

```markdown theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
md = '# Foo\n\n ## Bar\n\nHi this is Jim  \nHi this is Joe\n\n ## Baz\n\n Hi this is Molly'
```

我们可以指定要拆分的标题：

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
[("#", "Header 1"),("##", "Header 2")]
```

内容会根据共同的标题进行分组或拆分：

```
{'content': 'Hi this is Jim  \nHi this is Joe', 'metadata': {'Header 1': 'Foo', 'Header 2': 'Bar'}}
{'content': 'Hi this is Molly', 'metadata': {'Header 1': 'Foo', 'Header 2': 'Baz'}}
```

让我们看看下面的示例。

### 基本用法：

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
pip install -qU langchain-text-splitters
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain_text_splitters import MarkdownHeaderTextSplitter
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
markdown_document = "# Foo\n\n    ## Bar\n\nHi this is Jim\n\nHi this is Joe\n\n ### Boo \n\n Hi this is Lance \n\n ## Baz\n\n Hi this is Molly"

headers_to_split_on = [
    ("#", "Header 1"),
    ("##", "Header 2"),
    ("###", "Header 3"),
]

markdown_splitter = MarkdownHeaderTextSplitter(headers_to_split_on)
md_header_splits = markdown_splitter.split_text(markdown_document)
md_header_splits
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
[Document(metadata={'Header 1': 'Foo', 'Header 2': 'Bar'}, page_content='Hi this is Jim  \nHi this is Joe'),
 Document(metadata={'Header 1': 'Foo', 'Header 2': 'Bar', 'Header 3': 'Boo'}, page_content='Hi this is Lance'),
 Document(metadata={'Header 1': 'Foo', 'Header 2': 'Baz'}, page_content='Hi this is Molly')]
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
type(md_header_splits[0])
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
langchain_core.documents.base.Document
```

默认情况下，`MarkdownHeaderTextSplitter` 会从输出分块的内容中剥离被拆分的标题。可以通过设置 `strip_headers = False` 来禁用此功能。

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
markdown_splitter = MarkdownHeaderTextSplitter(headers_to_split_on, strip_headers=False)
md_header_splits = markdown_splitter.split_text(markdown_document)
md_header_splits
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
[Document(metadata={'Header 1': 'Foo', 'Header 2': 'Bar'}, page_content='# Foo  \n## Bar  \nHi this is Jim  \nHi this is Joe'),
 Document(metadata={'Header 1': 'Foo', 'Header 2': 'Bar', 'Header 3': 'Boo'}, page_content='### Boo  \nHi this is Lance'),
 Document(metadata={'Header 1': 'Foo', 'Header 2': 'Baz'}, page_content='## Baz  \nHi this is Molly')]
```

<Note>
  **默认的 `MarkdownHeaderTextSplitter` 会剥离空格和新行。若要保留 Markdown 文档的原始格式，请查看 [`ExperimentalMarkdownSyntaxTextSplitter`](https://reference.langchain.com/python/langchain-text-splitters/markdown/ExperimentalMarkdownSyntaxTextSplitter)。**
</Note>

### 如何将 Markdown 行作为单独文档返回

默认情况下，`MarkdownHeaderTextSplitter` 根据 `headers_to_split_on` 中指定的标题聚合行。我们可以通过指定 `return_each_line` 来禁用此功能：

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
markdown_splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on,
    return_each_line=True,
)
md_header_splits = markdown_splitter.split_text(markdown_document)
md_header_splits
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
[Document(metadata={'Header 1': 'Foo', 'Header 2': 'Bar'}, page_content='Hi this is Jim'),
 Document(metadata={'Header 1': 'Foo', 'Header 2': 'Bar'}, page_content='Hi this is Joe'),
 Document(metadata={'Header 1': 'Foo', 'Header 2': 'Bar', 'Header 3': 'Boo'}, page_content='Hi this is Lance'),
 Document(metadata={'Header 1': 'Foo', 'Header 2': 'Baz'}, page_content='Hi this is Molly')]
```

请注意，此处每个文档的 `metadata` 中都保留了标题信息。

### 如何限制分块大小：

在每个 Markdown 组内，我们可以应用任何我们想要的文本分割器，例如 `RecursiveCharacterTextSplitter`，它允许进一步控制分块大小。

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
markdown_document = "# Intro \n\n    ## History \n\n Markdown[9] is a lightweight markup language for creating formatted text using a plain-text editor. John Gruber created Markdown in 2004 as a markup language that is appealing to human readers in its source code form.[9] \n\n Markdown is widely used in blogging, instant messaging, online forums, collaborative software, documentation pages, and readme files. \n\n ## Rise and divergence \n\n As Markdown popularity grew rapidly, many Markdown implementations appeared, driven mostly by the need for \n\n additional features such as tables, footnotes, definition lists,[note 1] and Markdown inside HTML blocks. \n\n #### Standardization \n\n From 2012, a group of people, including Jeff Atwood and John MacFarlane, launched what Atwood characterised as a standardisation effort. \n\n ## Implementations \n\n Implementations of Markdown are available for over a dozen programming languages."

headers_to_split_on = [
    ("#", "Header 1"),
    ("##", "Header 2"),
]

# MD splits
markdown_splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on=headers_to_split_on, strip_headers=False
)
md_header_splits = markdown_splitter.split_text(markdown_document)

# Char-level splits
from langchain_text_splitters import RecursiveCharacterTextSplitter

chunk_size = 250
chunk_overlap = 30
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=chunk_size, chunk_overlap=chunk_overlap
)

# Split
splits = text_splitter.split_documents(md_header_splits)
splits
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
[Document(metadata={'Header 1': 'Intro', 'Header 2': 'History'}, page_content='# Intro  \n## History  \nMarkdown[9] is a lightweight markup language for creating formatted text using a plain-text editor. John Gruber created Markdown in 2004 as a markup language that is appealing to human readers in its source code form.[9]'),
 Document(metadata={'Header 1': 'Intro', 'Header 2': 'History'}, page_content='Markdown is widely used in blogging, instant messaging, online forums, collaborative software, documentation pages, and readme files.'),
 Document(metadata={'Header 1': 'Intro', 'Header 2': 'Rise and divergence'}, page_content='## Rise and divergence  \nAs Markdown popularity grew rapidly, many Markdown implementations appeared, driven mostly by the need for  \nadditional features such as tables, footnotes, definition lists,[note 1] and Markdown inside HTML blocks.'),
 Document(metadata={'Header 1': 'Intro', 'Header 2': 'Rise and divergence'}, page_content='#### Standardization  \nFrom 2012, a group of people, including Jeff Atwood and John MacFarlane, launched what Atwood characterised as a standardisation effort.'),
 Document(metadata={'Header 1': 'Intro', 'Header 2': 'Implementations'}, page_content='## Implementations  \nImplementations of Markdown are available for over a dozen programming languages.')]
```

## 故障排除：chunk\_overlap 似乎未生效

* 基于标题的拆分后（例如，`MarkdownHeaderTextSplitter`），请使用 **`split_documents(docs)`**（而不是 `split_text`），以便重叠应用于**每个部分内部**，并且每部分的元数据（标题）保留在分块上。
* 仅当**单个部分**超过 `chunk_size` 并被拆分为多个分块时，才会出现重叠。
* 重叠**不会跨越**部分/文档边界（例如，`# H1` → `## H2`）。
* 如果标题变成很小的第一个分块，请考虑将 `strip_headers` 设置为 `True`，这样标题行就不会成为独立的分块。
* 如果您的文本缺少换行符/空格，请在 `separators` 中保留一个后备 `""`，以便分割器仍然可以拆分并应用重叠。

***

<div className="source-links">
  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/i18n\zh-CN\oss\python\integrations\splitters\markdown_header_metadata_splitter.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>

  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>
</div>