基于Llama3的RAG
https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_rag_agent_llama3_local.ipynb
展示了如何从头开始使用LangGraph和Llama3-8b构建可靠的本地代理。 将3种先进RAG技术（自适应RAG、纠正性RAG和自我RAG）的思想结合起来，形成LangGraph中的单一控制流程。 路由：自适应RAG。将问题路由到不同的检索方法。 后备方案：纠正性RAG。如果文档与查询不相关，则回退到网页搜索。 自我纠错：自我RAG。修正包含幻觉或未回答问题的答案。
Llama3微调
LLaMA-Factory微调，已经支持LLaMA-3
https://github.com/hiyouga/LLaMA-Factory
Colab笔记本中将Llama-3微调速度提高2倍
https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing
基于Llama3的function calling/Agent
目前官方的说法是Llama3不支持function calling
一种方法采用Function Calling数据微调后的Llama 3
https://huggingface.co/Trelis/Meta-Llama-3-8B-Instruct-function-calling
prompt模板：
<|begin_of_text|><|start_header_id|>function_metadata<|end_header_id|>You have access to the following functions. Use them if required:[{"type": "function","function": {"name": "get_stock_price","description": "Get the stock price of an array of stocks","parameters": {"type": "object","properties": {"names": {"type": "array","items": {"type": "string"},"description": "An array of stocks"}},"required": ["names"]}}},{"type": "function","function": {"name": "get_big_stocks","description": "Get the names of the largest N stocks by market cap","parameters": {"type": "object","properties": {"number": {"type": "integer","description": "The number of largest stocks to get the names of, e.g. 25"},"region": {"type": "string","description": "The region to consider, can be \"US\" or \"World\"."}},"required": ["number"]}}}]<|eot_id|><|start_header_id|>user<|end_header_id|>Get the names of the five largest stocks by market cap<|eot_id|><|start_header_id|>assistant<|end_header_id|>Generated Response:{"name": "get_big_stocks","arguments": {"number": 5,"region": "US"}}<|eot_id|>
另外一种就是第三方项目中有一个调用示例，可以尝试下：
项目地址
https://github.com/themrzmaster/llama-cpp-python/blob/4a4b67399b9e5ee872e2b87068de75e2908d5b11/llama_cpp/llama_chat_format.py#L2866
调用示例
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="s")response = client.chat.completions.create(model="m",messages = [{"role": "system","content": "You are a digital waiter Pay attention to the user requests and use the tools to help you."},{"role": "user","content": "search for burguer and a maximum price of 10 dollars. at the sime time, look for merchant named 'Burger King'"},],tools=[{"type": "function","function": {"name": "search_merchant","description": "Search for merchants in the catalog based on the term","parameters": {"type": "object","properties": {"name": {"type": "string","description": "name to be searched for finding merchants.",}},"required": ["name"],},},},{"type": "function","function": {"name": "search_item","description": "Search for items in the catalog based on various criteria.","parameters": {"type": "object","properties": {"term": {"type": "string","description": "Term to be searched for finding items, with removed accents.",},"item_price_to": {"type": "integer","description": "Maximum price the user is willing to pay for an item, if specified.",},"merchant_delivery_fee_to": {"type": "integer","description": "Maximum delivery fee the user is willing to pay, if specified.",},"merchant_payment_types": {"type": "string","description": "Type of payment the user prefers, if specified.","enum": ["Credit Card","Debit Card","Other",],},},"required": ["term"],},},}],tool_choice="auto")
Llama3实操技术选型推荐
https://twitter.com/9hills/status/1781757051838050630
中文RAG选择 CommandR+;Agent/FunctionCalling 使用 Llama3-70B 或 CommandR+;中文文案写作用Qwen-72B;特定任务的小参数微调base模型用 Llama3-8B 或 Mistral-7B;大参数微调base 模型用Yi-34B;代码生成用 Llama3-70B或deepseek-coder-33B;特定类编程（比如nl2sql），小参数llama3-8b，大参数deepzeek-coder-33b;生产环境：vLLM、TensorRT-LLM;本地环境：Ollama.
