在文档（尤其是法律文档）中，我们经常遇到的一些非常具体的问题之一是需要建立文档中不同条款的层级结构。这是因为某些条款有时会引用其他条款，以获取完整的含义和上下文。
为了获得完整的上下文，您需要递归地导航并检索任何提到的条款（甚至是脚注！），通过文档层级图找到提到的条款，查看是否提到了其他条款，并重复这一过程。递归检索可以应用于法律文档以外的其他文档元素，例如页码、多模态数据（如图像）、指向其他文档或外部数据的超链接等。
在涉及基于图的遍历和检索时，法律文档因其结构化的特性一直让我感到着迷。法律文档还要求一个非常具体的结构化检索步骤，即“定义页”，其中每个重要术语都有一个具体的变量或固定定义，这些定义可能会根据文档的性质而发生变化。
在本 Notebook 和示例中，我们希望展示如何轻松创建法律文档中条款的文档层级图、法律文档中的定义图，以及一个多图多智能体的递归检索过程，该过程通过智能导航文档层级图和定义图来获取针对问题的完整上下文。
在这个示例中，我们选取了一份由马来西亚中央银行发布的简单监管文件，特别是这份文件（https://www.bnm.gov.my/documents/20124/938039/pd_compliance_Oct2015+%28Reissuance_May2016%29.pdf/6da52ca8-734a-d2fb-e214-093430ea8fb6?t=1592219195301），它规定了各种组织的合规规则。
我们想要回答的问题是：董事会和首席合规官（CCO）如何管理控制职能？
回答这个问题的理想流程如下：
• 从定义页中获取 CCO 的定义。
• 检索第 6.3 条和第 7.2 条的信息。
• 检测到第 7.2 条中提到“参见第 7.3 条和第 7.4 条”。
• 遍历并检索第 7.3 条和第 7.4 条。
• 检测到提到第 9.1 条并检索第 9.1 条。
作为快速基准测试，当我们使用嵌入了智能体推理能力的 GPT-4o 运行该问题时，我们得到了以下答案：
The management of control functions by the Boardand the ChiefCompliance
Officer(CCO) involves several key responsibilities as outlined in the
compliance document:
Responsibilities of the Board:
1.Oversight of ComplianceRiskManagement:TheBoardis responsible for
overseeing the management of compliance risk within the financial institution.This includes approving the compliance policy, establishing the compliance function,and appointing the CCO.
2.ApprovalandSupportfor the ComplianceFunction:TheBoard must
ensure that the compliance functionand the CCO have the appropriate standing,
authority,and independence to carry out their duties effectively.TheBoard
must also ensure that the compliance functionis adequately resourced.
3.Engagementwith the CCO:TheBoard should engage with the CCO regularly to
discuss issues faced by the compliance functionandensure the CCO has direct
access to the Board.
4.AnnualEvaluation:TheBoard must evaluate the effectiveness of the
institution’s management of compliance risk at least annually, considering
assessments from senior management,internal audit,and the CCO.
Responsibilities of the ChiefComplianceOfficer(CCO):
1.CoordinationandManagement of ComplianceRisk:The CCO is responsible for
coordinating the identification, management,and monitoring of compliance risk
across the institution.This includes ensuring compliance monitoring and
testing are consistent across the organization.
2.ReportingandAdvising:The CCO must report regularly to senior management
on the findings and analysis of compliance risks and must ensure that reports
are readily available to internal audit and regulatory authorities.The CCO
also advises the Boardand senior management on legal and regulatory
requirements, keeping them informed of developments and their implications.
3.Independence:The CCO must maintain independence from business lines to
effectively carry out the role of a control function.This includes ensuring
that there isno conflict of interest in their responsibilities, reporting
lines,or remuneration.
4.ResourcesandTraining:The CCO must ensure that the compliance function
is sufficiently resourced with officers who have the necessary qualifications
and experience.The CCO is also responsible for ensuring that adequate training
is provided to officers on relevant legal and regulatory requirements.
InteractionwithOtherControlFunctions:
The compliance function should be organized in a way that allows effective
management of compliance risk, considering the size, nature,and complexity of
the financial institution’s operations.Where responsibilities are shared
between a dedicated compliance unit and other control functions,
clear documentation and coordination are required to ensure that the CCO can
perform their duties effectively without impairing independence or focus.
By fulfilling these responsibilities, the Boardand the CCO can ensure that
the institution's compliance risks are managed effectively, supporting the
overall safety and soundness of the financial institution.这个答案有趣但不完整，因为它：
• 似乎特别关注“合规”这一关键词，并强调了与其他条款相关的所有合规义务，而不是专门针对董事会和 CCO。例如，第 8.2 条中提到的跨司法管辖区的协调是在“合规”上下文中提出的，而第 7.8 条中提到的薪酬独立性也是在“合规”上下文中提出的。
• 关键是，它似乎遗漏了第 7.3 条和第 7.4 条中明确提到的董事会和 CCO 之间关于 CCO 共享控制职能时需要董事会批准的关键义务。它还遗漏了第 9.1 条关于审计与合规职能分离的内容。
这可以理解，因为递归检索条款、页码和脚注并不是典型基于语义相似性检索（RAG）流程的明确部分。
多图多智能体工作流开发总结
为了让大家了解每个智能体的作用，我们在附录中包含了每个智能体的代码片段。
图的创建
在本 Notebook 中，我们首先通过 Reducto 的文档摄取引擎解析文档结构。文档结构将每一页分解为不同的元素，例如章节标题、列表项或页脚。
这些元素根据它们出现的顺序及其隐含的层级结构组合在一起，例如章节标题是列表项的父节点。然后，我们分析文档中的链接，以识别提取元素之间的连接，这些连接可以在词汇图中建模。
然后我们使用我们的 SDK（https://whyhow-ai.github.io/whyhow-sdk-docs/examples/create_graph_from_triples/） 将块和三元组导入 WhyHow 的知识图工作室中，以创建词汇图。
我们还为文档构建了一个法律定义图。法律文档中独特的一点是，每个文档都有一个定义部分，用于定义特定术语的具体解释方式。这可能因文档、用例和客户而异。在本例中，文档在第 4-5 页包含定义。这些文本被提取后传递给 GPT-4o，并通过提示提取法律术语及其定义（逐字返回），然后将输出转换为 CSV 文件，并使用 SDK 和预定义架构作为单独的图上传。定义智能体在需要时调用定义图，以通过特定相关定义增强上下文。在本例中，定义智能体在初始条款被检索后被调用。
然后我们将 WhyHow 中的节点导入 Notebook，并使用 LlamaIndex 对节点信息进行本地索引，同时保留 WhyHow 生成的嵌入。我们结合使用了 LlamaIndex 的向量检索、BM25 和关键词检索。在法律文档用例中，查询和检索过程需要术语的精准性，BM25 有助于在高度重复的文本中识别关键术语，而关键词检索则确保了重要术语即使出现频率较低，也能被检索到。
我们使用 LangGraph 结合 WhyHow SDK 和 GPT-4o 构建了一个围绕词汇图的多智能体工作流。本质上，当传递查询时，系统首先通过初始搜索智能体搜索相关的向量块。在本例中，向量块是条款或子条款。随后调用定义智能体以通过定义部分增强条款的相关定义。路由智能体接着检测是否需要引用额外的链接部分或页脚，如果需要，则检索相应部分并加以考虑。如果后续检索到的条款（如本例中所示）提到更多条款，则递归检索智能体将递归地执行此操作。
最初检索到的条款是第 6.3 条和第 7.2 条。定义图被引用以检查定义部分是否有任何额外的上下文需要包括。关于“CCO”和“高级管理层”的附加定义被包括在内。
在第 6.3 条中，有以下子条款 6.3.f：
• “如果 CCO 同时负责其他控制职能3，董事会应确保这种职责的组合不会损害健全的整体控制环境。”
根据检索到的第一个条款的信息，路由智能体帮助检测材料中是否提到了条款或页脚。在本例中，第一个条款关联了一个页脚（脚注 3）。路由智能体接着触发页脚解析智能体。页脚解析智能体识别相关页脚并返回以下页脚：
• “参见第 7.3 条和第 7.4 条。”
此时需要进行另一次遍历，递归检索智能体被用来遍历词汇图并检索第 7.3 条和第 7.4 条中的块/条款。
来自第 7.3 条和第 7.4 条的新信息被组合在一起。第 7.4 条 (b) 提到了第 9.1 条：
• “合规职能的职责不得与内部审计共享，CCO 也不得承担内部审计职责，因为此类做法将使第 9.1 条中描述的独立审查流程无效。”
第 7.4 条包含了第 9.1 条的链接，路由智能体检测到这一点，并指示递归检索智能体在下一次传递中检索它。回答智能体跟踪所有传入信息以总结并最终制定返回给用户的最终答案。
最终构建的答案反映了我们的理想流程，包含定义页、第 6.3 条和第 7.2 条、脚注 3、第 7.3 条、第 7.4 条和第 9.1 条的所有相关信息，并通过智能遍历准确总结返回给用户。
为了确保结果不是偶然的，我们将最终查询运行了 3 次，均成功检索到相关信息。
To manage control functions effectively, the Boardand the ChiefCompliance
Officer(CCO) have distinct responsibilities that they must exercise:
Responsibilities of the Board:
-Approve critical decisions regarding the CCO, including appointment,
remuneration,and termination (Section6.3(a)).
-Ensure the CCO has sufficient stature to engage effectively with senior
management (Section6.3(b)).
-Regularly engage with the CCO to discuss compliance issues and consider
interactions without senior management present (Section6.3(c)).
-Provide the CCO with unimpeded access to communicate with the board directly
(Section6.3(d)).
-Support the CCO with adequate resources to perform duties effectively,
including competent staff (Section6.3(e)).
-Satisfy themselves that combined responsibilities,if any,donot compromise
the control environment (Section6.3(f)).
Responsibilities of the CCO:
-Coordinate the identification and management of institution-wide compliance
risks (Section7.2(b)).
-Ensure consistent conduct of compliance monitoring and testing across the
organization (Section7.2(b)).
-Maintain independence and sufficient focus on compliance duties, even when
tasked with additional control functions (Section7.3).
SharedResponsibilities&Coordination:
-The board must approve any sharing of compliance function responsibilities
between the compliance unit and other control functions (Section7.4(a)).
-Function responsibilities, including timely communication of issues, should
be well-definedand documented (Section7.2(a)).
-Effective arrangements for coordination among control functions should be in
place to facilitate the CCO’s responsibilities (Section7.2(d)).
-Compliance responsibilities must not compromise the separation of the
internal audit function(Section9.1).
The board should ensure comprehensive oversight,and the CCO should maintain
effective coordination and communication across the organization to manage
control functions efficiently.通过这次练习，我们开发了一个系统，展示了以下内容：
• 一个多图系统，其中每个图代表 RAG 系统中不同的流程和目标。
• 使用 Reducto、WhyHow 和 LlamaIndex 创建支持 RAG 的自动词汇图。
• 一个多智能体系统，允许基于文档意图进行智能遍历，以结构化方式返回每个部分和子部分的信息。
• 一个由 LangGraph 管理的多图多智能体系统。
WhyHow.AI 的知识图工作室平台（https://github.com/whyhow-ai/knowledge-graph-studio）是构建模块化智能知识图（结合 LLM、开发者和非技术领域专家工作流）的简单方法。
附录：智能体代码片段
定义智能体
• 检索查询中提到的术语定义。
def definitions_search(query_prompt: str, client:Optional[WhyHow]=None)->Dict[str, str]:
"""
Search for definitions of terms in a question prompt and return them as a dictionary.
"""
if client isNone:
client =WhyHow(api_key=WHYHOW_API_KEY, base_url=WHYHOW_API_URL)
definitions_response = client.graphs.query_unstructured(
graph_id=definitions_graph.graph_id,
query=query_prompt,
)
response_text = definitions_response.answer
term_def_pairs = response_text.split('\n')
definitions_dict ={}
for pair in term_def_pairs:
if':'in pair:
term, definition = pair.split(':',1)
definitions_dict[term.strip()]= definition.strip()
return definitions_dict
query_prompt ="""Return me definitions for the terms in this query: "How can the Boardand the CCO manage control functions?" Ensure the term-definition pairs are separated by newlines, properly capitalised"""
definitions_dict = definitions_search(query_prompt)
def print_prompt_definitions_dict(definitions_dict):
prompt ="Relevant Definitions:\n"
for term, definition in definitions_dict.items():
prompt += f"{term}: {definition}\n"
return prompt
print(print_prompt_definitions_dict(definitions_dict))路由智能体
• 决定流程是停止还是继续。还会识别包含页脚信息或链接到其他节点的相关节点，供递归检索智能体检索。
def router_agent(state:AgentState)->AgentState:
# decide if process should should stop or continue
starter_prompt_footer = f"""
You are an intelligent agent overseeing a multi-agent retrieval process of graph nodes from a document. These nodes are to answer the query:
```{state['query']}```
Below this request is a list of nodes that were automatically retrieved.
You must determine if the list of nodes is enough to answer the query. If there isn't enough information, you must identify any relevant footer information in the nodes.
A node can footer information asking to look in another section/part of the document, which will require a separate natural language search.
Example: If the footer says "see paragraph x", a search query e.g. "Return paragraph x to answer the query '{state['query']}'" should be made.
If there are no further nodes worth analyzing, return an empty response. ONLY RETURN QUERIES FOR FOOTERS THAT ARE RELEVANT TO ANSWERING THE QUERY
Else, if any relevant nodes require a footer search, specify the node_id and the search query.
Nodes are identified by node_id and must be quoted in backticks.
"""
starter_prompt_link = f"""
You are an intelligent agent overseeing a multi-agent retrieval process of graph nodes from a document. These nodes are to answer the query:
```{state['query']}```
Below this request is a list of nodes that were automatically retrieved.
You must determine if the list of nodes is enough to answer the query. If there isn't enough information, you must identify any linked nodes that could be worth exploring.
If there are no further nodes worth analyzing, return an empty response.
Return a list of node_ids. ONLY RETURN NODE_IDS for NODES THAT ARE RELEVANT TO ANSWERING THE QUERY. Nodes are identified by node_id and must be quoted in backticks.
"""
# collect latest nodes, and all nodes
last_fetched_nodes_flattened:Dict[str,MultiAgentSearchLocalNode]={}
all_nodes_flattened:Dict[str,MultiAgentSearchLocalNode]= {}监督智能体
• 监控上下文窗口，如果上下文过多，则修剪节点。
• 还会跟踪搜索失败情况，如果页脚搜索或链接节点未找到相关信息，且重复失败次数过多，则会提前结束检索流程。
def supervisor_agent(state:AgentState)->AgentState:
# Look for search failures. This might be an instance where multiple searches were made for certain parts of the document, but no relevant information was found.
# This means that the search has to be ended prematurely to prevent infinite loops.
printout =""
for node in state["previous_nodes"]:
printout += node.print_node_prompt()
for node in state["last_fetched_context_nodes"]:
printout += node.print_node_prompt()
prompt = f"""
You are a supervisor agent overseeing the multi-agent retrieval process of graph nodes from a document. The nodes are to answer the query:
```{state['query']}```
Below is a list of nodes that were automatically retrieved, followed by a list of errors. If there are many similar, repeated errors in the retrieval process , where no further linked or relevant nodes could be retrieved, return END to end the process. Else return CONTINUE.
Return only a single word, either END or CONTINUE.
"""
completion = openai_client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role":"system","content": prompt},
{"role":"user","content": printout},
{"role":"user","content": state['search_failures']},
],
)递归智能体
• 检索由路由智能体标记的信息。
• 要么获取新的链接节点，要么对文档中的页脚进行关键词搜索（例如，如果页脚要求参考第 7.3 条和第 7.4 条而没有链接，则改为搜索）。此步骤中，LLM 调用会修剪冗余节点。
def recursive_retrieval(state:AgentState)->AgentState:
current_nodes = state["last_fetched_context_nodes"]
for current_node in current_nodes:
state["previous_nodes"].append(current_node)
new_current_nodes =[]
# look up the nodes to fetch by id
for node_id in state["node_links_to_fetch"]:
# sometimes GPT returns node ids with or without backticks
if node_id[0]=="`":
node_id = node_id[1:-1]
if node_id in local_nodes_map:
new_current_nodes.append(local_nodes_map[node_id])
else:
state["search_failures"].append(f"Failed to fetch node with id: {node_id}")
for node_id, search_query in state["node_footers_to_fetch"].items():
# fetch nodes by keyword and bm25 search
footer_retrieved_nodes = retrieve_with_keywords_bm25(search_query)
# LLM prunes nodes that are not relevant
footer_retrieved_nodes, _ = prune_nodes(search_query, footer_retrieved_nodes)
for node in footer_retrieved_nodes:
new_current_nodes.append(node)
# if no nodes fetched, log failure
if len(footer_retrieved_nodes)==0:
state["search_failures"].append(
f"Failed to fetch nodes for query: {search_query}"
)
state["last_fetched_context_nodes"]= new_current_nodes
state["pass_count"]+=1
state["node_footers_to_fetch"]={}
state["node_links_to_fetch"]=[]
return state回答智能体
• 根据检索过程中收集的所有信息以及文档定义，最终确定查询的答案。
• 还会尝试链接到来源，例如段落编号。
def answering_agent(state: AgentState) -> AgentState:
# answer the query
prompt = f"""
You are an answering agent. You will be given a list of document nodes that were automatically retrieved by the system. These nodes are to answer the query:
```{state['query']}```
Give references to sections/paragraphs if possible, but do not output full node ids with backticks and the hash.
"""通过多图多智能体系统检索的答案变体
答案 #2：
To manage control functions effectively, both the Boardand the Chief
ComplianceOfficer(CCO) must collaborate and establish a structured
framework.Key measures include:
Board’s Responsibilities:
-TheBoard must approve the appointment, remuneration,and termination of the
CCO, ensuring they have the necessary stature to engage effectively with
senior management (S 6.3(a),(b)).
-TheBoard should engage regularly with the CCO to discuss compliance issues
andensure the CCO has direct access to the Board(S 6.3(c),(d)).
-TheBoard must also assure that the CCO is granted adequate resources and
support, including competent officers, to fulfill their duties (S 6.3(e)).
CCO’s RoleandIndependence:
-The CCO should oversee the institution-wide compliance risk management and
ensure that responsibilities are clearly documented and communicated in the
compliance policy (S 7.2(a),(b)).
-If the CCO is involved in other control functions, they must ensure their
independence and ability to focus on compliance responsibilities remain
uncompromised (S 7.3).
CoordinationwithOtherFunctions:
-Proper coordination between the CCO and other control functions is essential.
This includes robust information flows and mechanisms for seeking compliance
advice (S 7.2(d)).
-TheBoard must approve the sharing of compliance responsibilities to ensure
there isno dilution of focus or effectiveness (S 7.4(a)).
SeparationfromInternalAudit:
-Compliance functions should remain distinct from the internal audit to
maintain the efficacy of independent reviews (S 7.4(b), S 9.1).
TrainingandCapacityBuilding:
-The compliance functionis responsible for training and providing guidance
to other officers on legal and regulatory requirements to manage compliance
risk effectively (S 8.11).
By implementing these steps, the Boardand the CCO can effectively manage
control functions, maintaining robust compliance and governance frameworks.答案 #3：
The management of control functions by the Boardand the ChiefCompliance
Officer(CCO) involves several key responsibilities and structures:
Role of the BoardinManaging CCO Functions:
-The board must approve the appointment, remuneration,and termination of the
CCO (Section6.3(a)).
-The board should ensure that the CCO has sufficient stature to interact
effectively with the CEO and senior management (Section6.3(b)).
-Regular engagement between the board and the CCO is important to discuss
compliance issues directly (Section6.3(c)).
-The CCO must have direct, unimpeded access to the board (Section6.3(d)).
-There must be adequate resources and support for the CCO to perform his
duties effectively (Section6.3(e)).
SharedResponsibilitiesandIndependence:
-Where compliance functions are shared, the board must approve this
arrangement,and responsibilities should be clearly definedand documented
in the compliance policy (Section7.2).
-The CCO should not assume responsibilities forinternal audit,asthis can
compromise independent review processes (Sections7.4,9.1).
-The CCO must ensure that their independence and ability to focus on
compliance are not impaired by additional responsibilities (Section7.3).
ResponsibilitiesWithin the Organization:
-Complianceis the responsibility of all officers within the institution.
Business lines manage compliance risk through their managerial controls,
while the compliance function ensures that these controls are adequate
(Section1.2).
-Theinternal audit function provides independent assurance on the quality
and effectiveness of the institution’s controls, including those concerning
compliance (Section1.2(c)).
CoordinationAcrossControlFunctions:
-Arrangementsfor coordination among control functions and the CCO must
promote a consistent approach to managing compliance risk,with adequate
information flows and avenues for advice (Section7.2(d)).
By following these guidelines, the Boardand the CCO can manage the
compliance control functions effectively, ensuring that compliance risks
are appropriately identified, managed,and mitigated across the organization.