两个关键问题限制了 RAG 的发展：
新型 RAG 算法之间缺乏全面和公平的比较。
像 LlamaIndex 和 LangChain 这样的开源工具使用了高级抽象，这导致了透明度的缺失，并限制了开发新算法和评估指标的能力。
不同 RAG 库和框架的比较。公平比较指的是在评估过程中对所有基本组件进行对齐，包括随机种子、生成器、检索器和指令。数据收集器指的是能够收集或生成训练和测试数据的能力，无论是通过从现有的原始数据集中抽样，还是通过使用LLM构建标记数据。
RAGLAB提供了一个模块化的架构，允许用户轻松地替换和扩展算法的各个组成部分，包括检索器（retriever）、生成器（generator）和指令（instruction）。
RAGLAB 框架的架构和组件
检索器（Retriever）：集成了基于BERT的模型，如Contriever和ColBERT，提供了统一的查询接口和客户端-服务器架构，以及检索缓存机制。
语料库（Corpus）：提供预处理的Wikipedia语料库，包括2018年和2023年的版本，以及对应的索引和嵌入。
生成器（Generator）：集成了Huggingface Transformers和VLLM，支持量化和低秩适应（LoRA）技术，允许使用大型模型。
指令实验室（Instruction Lab）：包含系统指令、任务指令和算法指令，允许用户自定义和组合指令。
训练器（Trainer）：集成了Accelerate和DeepSpeed库，支持模型的微调，包括LoRA和量化LoRA技术。
数据集和度量（Dataset and Metric）：收集了10个广泛使用的基准数据集，覆盖五种不同的任务类型，并提供了灵活的数据适配机制和多种评估指标
Naive RAG# read process insruction"### Instruction:\n {task_instrucion} \n## Input:\n\n{query}\n\n Now, based on the following passagesand your knowledge, please answer the question more succinctly and professionally. ### BackgroundKnowledge:\n {passages} \n\n### Response:\n"RRR# rewrite process instruction"Provide a better search query for Wikipedia to answer the given question, end the query with '**'. \n\nQuestion: Ezzard Charles was a world champion in which sport? \n\n Query: Ezzard Charleschampion** \n\n Question: What is the correct name of laughing gas? \n\n Query: laughing gasname** \n\n Question: {query} \n\n Query: " # read process insruction"### Instruction:\n {task_instrucion} \n## Input:\n\n{query}\n\n Now, based on the following passagesand your knowledge, please answer the question more succinctly and professionally. ### BackgroundKnowledge:\n {passages} \n\n### Response:\n"ITER-RETGEN# read process insruction"### Instruction:\n {task_instrucion} \n## Input:\n\n{query}\n\n Now, based on the following passagesand your knowledge, please answer the question more succinctly and professionally. ### BackgroundKnowledge:\n {passages} \n\n### Response:\n"Self ASK# follow up question instruction"Question: When does monsoon season end in the state the area code 575 is located? Are follow upquestions needed here: Yes. Follow up: Which state is the area code 575 located in? Intermediateanswer: The area code 575 is located in New Mexico. Follow up: When does monsoon season end inNew Mexico? Intermediate answer: Monsoon season in New Mexico typically ends in mid-September. So the final answer is: mid-September. \n{query} Are follow up questions needed here:" # read process insruction"### Instruction:\n {task_instrucion} \n## Input:\n\n{query}\n\n Now, based on the following passagesand your knowledge, please answer the question more succinctly and professionally. ### BackgroundKnowledge:\n {passages} \n\n### Response:\n"Active RAG# read process insruction"### Instruction:\n {task_instrucion} \n## Input:\n\n{query}\n\n Now, based on the following passagesand your knowledge, please answer the question more succinctly and professionally. ### BackgroundKnowledge:\n {passages} \n\n### Response:\n"Self-RAG# read process instruction"### Instruction:\n{task_instruction}\n\n## Input:\n\n{query}\n\n### Response:\n"
RAGLAB 系统用户评估问卷
RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generationhttps://arxiv.org/pdf/2408.11381https://github.com/fate-ubw/RAGLab
