大模型学习笔记十四:Agent模型微调

文章目录

    • 一、大模型需要Agent技术的原因
    • 二、Prompt Engineering可以实现Agent吗?
      • (1)ReAct原理展示和代码
      • (2)ModelScope
      • (3)AutoGPT
      • (4)ToolLLaMA
    • 三、既然AutoGPT可以满足需求,为什么要额外训练一个Agent模型?
    • 四、怎么去训练一个Agent模型?
      • 1)数据准备和处理
        • (1)简单任务举例(背景现在天气如何?还是两个prompt)
        • (2)复杂任务举例(我想登顶北京最高峰,请帮我做规划)
      • 2)模型训练(SFT)
      • 3)模型效果评估
        • (1)自动评估
        • (2)人工评估
    • 五、如何提高Agent的泛化性?
      • 1)Meta-Agent
      • 2)训练数据构建
      • 3)训练成果评判
    • 六、开源项目介绍
    • 七、总结

  • 学习目的
    1、理解什么情况下需要训练一个具备Agent能力的模型?
    2、如何训练一个具备Agent能力的模型?重点是训练数据的构建
    3、如何评测Agent能力?
    4、如何提升模型的泛化性?

一、大模型需要Agent技术的原因

  • 需求
    1、帮我查一下今天的销售额?
    2、(开车时)前方为啥堵车了?
    3、刘德华多少岁了
    4、请帮我约一个和搜索产品部的需求沟通会,本周三至周五我日历上空闲的时间都可以;
    5、帮我订一张周五去上海的机票

  • 原因
    1、大模型的“幻觉”问题,很难在从模型本身上彻底解决,在严肃的应用场景需要通过引入外部知识确保答案的准确;
    2、大模型参数无法做到实时更新,本身也无法与真实世界产生实时连接,在多数场景下难以满足实际需求;
    3、复杂的业务场景需要

  • 大模型技术框架回顾
    在这里插入图片描述

二、Prompt Engineering可以实现Agent吗?

  • 回答
    可以,前面弄的AutoGPT就是例子。除了AutoGPT外,还有ReACT、ModelScope、ToolLLaMA等不同的形式。

  • 主流Agent prompt的比较
    在这里插入图片描述

  • 展示分类
    ①ReACT:较为简单,先设定Question,再以Thought、Action、Action Input、Observation执行多轮,最后输出Final Answer
    ②ModelScope:更为直接的生成回复,调用只需要加入<|startofthink|>和<|endofthink|>字段,并在其中填入command_name和args
    ③AutoGPT: prompt 较为复杂,分为生成工具调用生成最终答案 两套prompt

  • 生成工具调用 prompt 详细设立了Constraints(例如不需要用户协助)、Resources(例如网络搜索)、Best Practices(例如每个API都需要花费,尽量少的调用),最终严格以json格式输出
    ④ToolLLaMA:模仿AutoGPT和ReACT,输出以Thought、Action、Action Input 格式而非 json 格式,增加了 give_up_and_restart支持全部推导重来,重来的prompt会把历史失败的记录加进去训模型用了1.6w Rapid API(与autogpt不同的是有一个finish的工具作为总结)

(1)ReAct原理展示和代码

  • 备注
    1)action就是工具
    2)action input就是工具的参数
    3)obsession 工具返回的结果
    4)thought、action、action input、observation可以重复N次

  • ReACT Prompt 模板(示例:背景天气怎么样?只能处理简单的问题)

Answer the following questions as best you can. You have access to the following tools:[
Name: web_search. Description: Perform a web search using the specified query.
Name: get_weather. Description: Retrieve the current weather information for a specified location.
]Use the following format:Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [web_search, get_weather]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input questionBegin!
Question: 北京天气怎么样?
  • 第一次回复(等待调用工具返回结果)
Question: 北京天气怎么样?
Thought: The user wants to know the current weather in Beijing. I can retrieve this information using the "get_weather" tool.
Action: get_weather
Action Input: Beijing
Observation: (等待调用工具返回结果)
  • 工具返回结果
Question: 北京天气怎么样?
Thought: The user wants to know the current weather in Beijing. I can retrieve this information using the "get_weather" tool.
Action: get_weather
Action Input: Beijing
Observation: 北京当前的天气是"晴"(工具结果)
  • 返回思路
Thought: I now know the final answer.
Final Answer: 北京天气是晴天+ 代码
```python
# 导入依赖库
from openai import OpenAI
from dotenv import load_dotenv, find_dotenv# 加载 .env 文件中定义的环境变量
_ = load_dotenv(find_dotenv())# 初始化 OpenAI 客户端
client = OpenAI()  # 默认使用环境变量中的 OPENAI_API_KEY 和 OPENAI_BASE_URL# 基于 prompt 生成文本
# 默认使用 gpt-3.5-turbo 模型
def get_completion(prompt, response_format="text", model="gpt-4o-mini"):messages = [{"role": "user", "content": prompt}]    # 将 prompt 作为用户输入response = client.chat.completions.create(model=model,messages=messages,temperature=0,                                  # 模型输出的随机性,0 表示随机性最小# 返回消息的格式,text 或 json_objectresponse_format={"type": response_format},)print(response)return response.choices[0].message.content          # 返回模型生成的文本# 任务描述
instruction = """
Answer the following questions as best you can. You have access to the following tools:[
Name: web_search. Description: Perform a web search using the specified query.
Name: get_weather. Description: Retrieve the current weather information for a specified location.
]Use the following format:Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [web_search, get_weather]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input questionBegin!
"""# 用户输入
input_text = """
北京天气怎么样?
"""# 工具结果
tool_output = """
"""# prompt 模版。instruction 和 input_text 会被替换为上面的内容
prompt = f"""
{instruction}Question: 
{input_text}{tool_output}
"""# 调用大模型
response = get_completion(prompt)
print("------------------------------------------")
print(response)

(2)ModelScope

  • 介绍
    阿里的魔塔社区助手

  • prompt展示

你是达摩院的ModelScopeGPT(魔搭助手),你是个大语言模型, 是2023年达摩院的工程师训练得到的。你有多种能力,可以通过插件集成魔搭社区的模型api来回复用户的问题,还能解答用户使用模型遇到的问题和模型知识相关问答。目前支持的插件信息如下,请自行判断是否需要调用插件来解决当前用户问题。若需要调用插件,则需要将插件调用请求按照json格式给出,必须包含api_name、parameters字段,并在其前后使用<|startofthink|>和<|endofthink|>作为标志。然后你需要根据插件API调用结果生成合理的答复;若无需调用插件,则直接给出对应回复即可:{"api_name": "web_search","description": "Perform a web search using the specified query.","parameters": {"type": "object","properties": {"query": {"type": "string","description": "The search query to be executed.","example_value": "姜萍事件"}},"required": ["query"],"optional": []}
}
{"api_name": "get_weather","description": "Retrieve the current weather information for a specified location.","parameters": {"type": "object","properties": {"location": {"type": "string","description": "The name of the location (city, country, etc.) to get the weather for.","example_value": "Beijing"},},"required": ["location"],"optional": []}
}<|user|>: 北京现在天气怎么样?
  • 回复
我将使用get_weather API获取北京的天气  
<|startofthink|>{"api_name":"get_weather","parameters":{"location":"Beijing"}}<|endofthink|>  
<|startofexec|>北京:32度,晴<|endofexec|>  
北京目前天气为晴天,32度,出门建议做好防晒准备

(3)AutoGPT

  • 主题介绍
    1)分为两个prompt,第一个判断用哪个工具步骤的prompt,生成任务规划和工具指令生成
    2)第二个prompt是总结第一个prompt行动生成的结果,然后生成答案

  • prompt一展示

You are AIGPT, an AI assistant that provides information and answers questions on a wide range of topics, to help users find the information they are looking for.Your decisions must always be made independently without seeking user assistance. Play to your strengths as an LLM and pursue simple strategies with no legal complications.Constraints
You operate within the following constraints:1.~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.
2.If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.
3.No user assistance
4.Exclusively use the commands listed below e.g. web_searchCommands
1.web_search: Perform a web search using the specified query. Params: (query: string)
2.get_weather: Retrieve the current weather information for a specified location. Params: (location: string)Resources
You can leverage access to the following resources:
1.Internet access for searches and information gathering.
2.Long Term memory management.
3.Command executionBest Practices
1.Continuously review and analyze your actions to ensure you are performing to the best of your abilities.
2.Constructively self-criticize your big-picture behavior constantly.
3.Reflect on past decisions and strategies to refine your approach.
4.Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps.Goals
For your task, you must fulfill the following goals:北京天气怎么样?The current time and date is Mon Jul 01 12:12:00 2024Respond strictly with JSON. The JSON should be compatible with the TypeScript type Response from the following:{thoughts: {text: string;reasoning: string;plan: string;criticism: string;speak: string;};command: {name: string;args: Record<string, any>;};
}Determine exactly one command to use based on the given goals and the progress you have made so far, and respond using the JSON schema specified previously. Ensure the reply can be parsed by json.loads and do not reply with any other content.
-----------------Response(任务规划与工具指令生成):
下面是输出的回复
{"thoughts": {"text": "To find out the current weather in Beijing, I will use the 'get_weather' command.","reasoning": "The most efficient way to get the current weather information for Beijing is to use the available command designed for this purpose.","plan": "Execute the 'get_weather' command with Beijing as the specified location.","criticism": "This approach ensures that I retrieve accurate and up-to-date weather information without unnecessary steps.","speak": "Executing the command to retrieve the current weather in Beijing."},"command": {"name": "get_weather","args": {"location": "Beijing"}}
}

调用工具获得结果:“北京天气晴朗,日均气温28度”

  • prompt二展示
You are AIGPT, an AI assistant that provides information and answers questions on a wide range of topics, to help users find the information they are looking for.Your decisions must always be made independently without seeking user assistance. Play to your strengths as an LLM and pursue simple strategies with no legal complications.Constraints
You operate within the following constraints:1.~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.
2.If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.
3.No user assistance
4.Exclusively use the commands listed below e.g. web_searchCommands
1.web_search: Perform a web search using the specified query. Params: (query: string)
2.get_weather: Retrieve the current weather information for a specified location. Params: (location: string)Resources
You can leverage access to the following resources:
1.Internet access for searches and information gathering.
2.Long Term memory management.
3.Command executionBest Practices
1.Continuously review and analyze your actions to ensure you are performing to the best of your abilities.
2.Constructively self-criticize your big-picture behavior constantly.
3.Reflect on past decisions and strategies to refine your approach.
4.Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps.Goals
For your task, you must fulfill the following goals:北京天气怎么样?The current time and date is Mon Jul 01 12:12:00 2024This reminds you of these events from your past: 
To find out the current weather in Beijing, I performed a get_weather API call. I discovered that Beijing's weather is clear, windless, with an average daily temperature of 28°C.The above is your historical memory, now please do not use any COMMANDS, do not reply in JSON format, directly generate helpful answers for GOALS:------------------Response(最终回答):
Beijing's weather is currently clear and windless, with an average daily temperature of 28°C.

(4)ToolLLaMA

  • 介绍
    一套 prompt 模板,整体以<|

system|>、<|user|>、<|assistant|>、<|function|>、<|assistant|>、<|function|> …… 自动反复执行直到产生 final_answer,其中<|function|>中有一个 Finish 可以产生最终答案

  • Prompt
<|system|>
You are AutoGPT, you can use many tools (functions) to do the following task. First I will give you the task description, and your task start. At each step, you need to give your thought to analyze the status now and what to do next, with a function call to actually execute your step. Your output should follow this format:Thought:
Action:
Action Input:After the call, you will get the call result, and you are now in a new state. Then you will analyze your status now, then decide what to do next.After many (Thought-call) pairs, you finally perform the task, then you can give your final answer.Remember:The state change is irreversible, you can't go back to one of the former states. If you want to restart the task, say "I give up and restart".
All the thoughts are short, at most 5 sentences.
You can do more than one try, so if your plan is to continuously try some conditions, you can do one of the conditions per try.
Let's Begin!Task description: You should use functions to help handle the real-time user queries. Remember:ALWAYS call the "Finish" function at the end of the task. The final answer should contain enough information to show to the user. If you can't handle the task, or you find that function calls always fail (the function is not valid now), use function Finish->give_up_and_restart.
Specifically, you have access to the following APIs:{"api_name": "web_search","description": "Perform a web search using the specified query.","parameters": {"type": "object","properties": {"query": {"type": "string","description": "The search query to be executed.","example_value": "最近姜萍事件"}},"required": ["query"],"optional": []}
}
{"api_name": "get_weather","description": "Retrieve the current weather information for a specified location.","parameters": {"type": "object","properties": {"location": {"type": "string","description": "The name of the location (city, country, etc.) to get the weather for.","example_value": "Beijing"},},"required": ["location"],"optional": []}
}
{"name": "Finish","description": "If you believe that you have obtained a result that can answer the task, please call this function to provide the final answer. Alternatively, if you recognize that you are unable to proceed with the task in the current state, call this function to restart. Remember: you must ALWAYS call this function at the end of your attempt, and the only part that will be shown to the user is the final answer, so it should contain sufficient information.","parameters": {"type": "object","properties": {"return_type": {"type": "string","enum": ["give_answer", "give_up_and_restart"]},"final_answer": {"type": "string","description": "The final answer you want to give the user. You should have this field if "return_type"=="give_answer""}},"required": ["return_type"]}
}<|user|>
北京天气怎么样?<|assistant|>
  • Response(任务规划与工具指令生成):
Thought: To provide the current weather information for Beijing, I will call the get_weather function with "Beijing" as the location.
Action: get_weather
Action Input: {"location": "Beijing"}

调用工具获得结果:“北京天气晴朗,日均气温28度”

  • Prompt:
<|system|>
You are AutoGPT, you can use many tools (functions) to do the following task. First I will give you the task description, and your task start. At each step, you need to give your thought to analyze the status now and what to do next, with a function call to actually execute your step. Your output should follow this format:Thought:
Action:
Action Input:After the call, you will get the call result, and you are now in a new state. Then you will analyze your status now, then decide what to do next.After many (Thought-call) pairs, you finally perform the task, then you can give your final answer.Remember:The state change is irreversible, you can't go back to one of the former states. If you want to restart the task, say "I give up and restart".
All the thoughts are short, at most 5 sentences.
You can do more than one try, so if your plan is to continuously try some conditions, you can do one of the conditions per try.
Let's Begin!Task description: You should use functions to help handle the real-time user queries. Remember:ALWAYS call the "Finish" function at the end of the task. The final answer should contain enough information to show to the user. If you can't handle the task, or you find that function calls always fail (the function is not valid now), use function Finish->give_up_and_restart.
Specifically, you have access to the following APIs:{"api_name": "web_search","description": "Perform a web search using the specified query.","parameters": {"type": "object","properties": {"query": {"type": "string","description": "The search query to be executed.","example_value": "最近姜萍事件"}},"required": ["query"],"optional": []}
}
{"api_name": "get_weather","description": "Retrieve the current weather information for a specified location.","parameters": {"type": "object","properties": {"location": {"type": "string","description": "The name of the location (city, country, etc.) to get the weather for.","example_value": "Beijing"},},"required": ["location"],"optional": []}
}
{"name": "Finish","description": "If you believe that you have obtained a result that can answer the task, please call this function to provide the final answer. Alternatively, if you recognize that you are unable to proceed with the task in the current state, call this function to restart. Remember: you must ALWAYS call this function at the end of your attempt, and the only part that will be shown to the user is the final answer, so it should contain sufficient information.","parameters": {"type": "object","properties": {"return_type": {"type": "string","enum": ["give_answer", "give_up_and_restart"]},"final_answer": {"type": "string","description": "The final answer you want to give the user. You should have this field if "return_type"=="give_answer""}},"required": ["return_type"]}
}<|user|>
北京天气怎么样?<|assistant|>
{"name": "get_weather","arguments": {"location": "北京"}
}<|function|>
{'message': '北京天气晴朗,日均气温28度'}<|assistant|>
  • Response(最终回答):
  • 代码
# 导入依赖库
from openai import OpenAI
from dotenv import load_dotenv, find_dotenv# 加载 .env 文件中定义的环境变量
_ = load_dotenv(find_dotenv())# 初始化 OpenAI 客户端
client = OpenAI()  # 默认使用环境变量中的 OPENAI_API_KEY 和 OPENAI_BASE_URL# 基于 prompt 生成文本
# 默认使用 gpt-3.5-turbo 模型
def get_completion(prompt, response_format="text", model="gpt-4o-mini"):messages = [{"role": "user", "content": prompt}]    # 将 prompt 作为用户输入response = client.chat.completions.create(model=model,messages=messages,temperature=0,                                  # 模型输出的随机性,0 表示随机性最小# 返回消息的格式,text 或 json_objectresponse_format={"type": response_format},)print(response)return response.choices[0].message.content          # 返回模型生成的文本# 任务描述
instruction = """
<|system|>
You are AutoGPT, you can use many tools (functions) to do the following task. First I will give you the task description, and your task start. At each step, you need to give your thought to analyze the status now and what to do next, with a function call to actually execute your step. Your output should follow this format:Thought:
Action:
Action Input:After the call, you will get the call result, and you are now in a new state. Then you will analyze your status now, then decide what to do next.After many (Thought-call) pairs, you finally perform the task, then you can give your final answer.Remember:The state change is irreversible, you can't go back to one of the former states. If you want to restart the task, say "I give up and restart".
All the thoughts are short, at most 5 sentences.
You can do more than one try, so if your plan is to continuously try some conditions, you can do one of the conditions per try.
Let's Begin!Task description: You should use functions to help handle the real-time user queries. Remember:ALWAYS call the "Finish" function at the end of the task. The final answer should contain enough information to show to the user. If you can't handle the task, or you find that function calls always fail (the function is not valid now), use function Finish->give_up_and_restart.
Specifically, you have access to the following APIs:{"api_name": "web_search","description": "Perform a web search using the specified query.","parameters": {"type": "object","properties": {"query": {"type": "string","description": "The search query to be executed.","example_value": "最近姜萍事件"}},"required": ["query"],"optional": []}
}
{"api_name": "get_weather","description": "Retrieve the current weather information for a specified location.","parameters": {"type": "object","properties": {"location": {"type": "string","description": "The name of the location (city, country, etc.) to get the weather for.","example_value": "Beijing"},},"required": ["location"],"optional": []}
}
{"name": "Finish","description": "If you believe that you have obtained a result that can answer the task, please call this function to provide the final answer. Alternatively, if you recognize that you are unable to proceed with the task in the current state, call this function to restart. Remember: you must ALWAYS call this function at the end of your attempt, and the only part that will be shown to the user is the final answer, so it should contain sufficient information.","parameters": {"type": "object","properties": {"return_type": {"type": "string","enum": ["give_answer", "give_up_and_restart"]},"final_answer": {"type": "string","description": "The final answer you want to give the user. You should have this field if "return_type"=="give_answer""}},"required": ["return_type"]}
}
"""# 用户输入
input_text = """
北京天气怎么样?
"""# 工具结果
tool_output = """
"""# prompt 模版。instruction 和 input_text 会被替换为上面的内容
prompt = f"""
{instruction}<|user|>
{input_text}{tool_output}<|assistant|>
"""# 调用大模型
response = get_completion(prompt)
print("------------------------------------------")
print(response)
  • 回复
ChatCompletion(id='chatcmpl-9o5Yu72jkdeBDee0DeatvBG8IT7NI', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Thought: I need to retrieve the current weather information for Beijing to answer the user\'s query about the weather. \nAction: get_weather \nAction Input: {"location":"Beijing"}', role='assistant', function_call=None, tool_calls=None))], created=1721724808, model='gpt-4o-mini-2024-07-18', object='chat.completion', service_tier=None, system_fingerprint='fp_8b761cb050', usage=CompletionUsage(completion_tokens=36, prompt_tokens=737, total_tokens=773))
------------------------------------------
Thought: I need to retrieve the current weather information for Beijing to answer the user's query about the weather. 
Action: get_weather 
Action Input: {"location":"Beijing"}

三、既然AutoGPT可以满足需求,为什么要额外训练一个Agent模型?

  • 解答
    1、很多场景下无法使用大模型API,需要私有化部署(没法联网)
    2、实践证明,除了GPT4 level的大模型,其他大模型(包括GPT3.5)无法很好遵从prompt要求完成复杂的Agent任务,有时候还缺少参数或多余几句话
    3、通过训练,一个小参数量的大模型(13B、7B等)也能达到较好的能力,更加实用

四、怎么去训练一个Agent模型?

  • 目标
    攀登背景最高峰,帮忙做个规划
  • 规划图
    在这里插入图片描述

1)数据准备和处理

  • 大模型SFT的数据格式:输入(prompt)和输出Output(Response)
    在这里插入图片描述
  • Agent 微调使用的数据形式SFT也叫做专门给tool的SFT,称为Tool SFT
    1)前后Agent工作过程多步,因此训练数据也需要多步骤,前后步骤的数据有关联
    在这里插入图片描述
(1)简单任务举例(背景现在天气如何?还是两个prompt)
  • prompt模块拆解
1)Profile(角色):定义 Agent 的身份、背景甚至性格特点等。这帮助模型在回答时保持一致性和个性化。例如,一个 Agent 可能被设定为一个友好且专业的助理。
例子:
你是一个很专业的AI任务规划师,给定目标或问题,你的决策将独立执行而不依赖于人类的帮助,请发挥LLM的优势并且追求高效的策略进行任务规划。
2)Instruction(指令):提供具体的操作指令,指导Agent如何完成任务。这些指令明确地告诉Agent在特定情境下应该做什么或避免做什么。
例子:
Constraints:  
1.你有~4000字的短期记忆
2.不需要用户的帮助
3.只使用双引号中列出的commands,例如:“command_name”
4.互联网搜索、信息聚合和鉴别真伪的能力
5.保持谦逊,对自己没把握的问题,尽可能调用command,但尽量少调用,不能重复调用
6.当你从自身知识或者历史记忆中能得出结论,请聪明且高效,完成任务并得出结论
7.经常建设性地自我批评整个行为大局,反思过去的决策和策略,以改进你的方法
8.你最多只能进行5步思考,规划5个任务,所以尽可能高效规划任务
9.你有反思能力,如果已完成的任务和结果暂不能得到回答问题所需信息或尚不能完成目标,应继续规划……当已完成的Tasks能够帮助回答这个目标问题,则尽可能生成任务完成Task,否则生成一个其他Task。一个新Task:
3)Tools(工具集):列出Agent可以使用的工具或功能,需要根据实际业务需求来构建。
例子:
Commands:  
-1:{"name": "web_search", "description": "Perform an internet search.", "parameters": {"type": "object", "properties": {"text": {"type": "str", "description": "Search query."}}}, "returns": {"description": "Multiple webpage links along with brief descriptions.", "type": "str"}, "required": ["text"]}-2:{"name": "wiki_search", "description": "Conduct an encyclopedia search for a specified entity and question. Provide accurate answers for factual queries but has a limited scope and cover fewer queries.", "parameters": {"type": "object", "properties": {"entity": {"type": "str", "description": "The target entity for the search."}, "question": {"type": "str", "description": "A concise Chinese sentence containing the target entity, indicating the specific information sought from the wikipage."}}}, "returns": {"description": "Relevant encyclopedia entries pertaining to the question.", "type": "str"}, "required": ["entity", "question"]}-3:{"name": "get_weather_info", "description": "Retrieve weather information for specified locations and dates.", "parameters": {"type": "object", "properties": {"location": {"type": "str", "description": "Locations in English separated by commas, e.g., "Beijing,Vancouver,...,Chicago"."}, "start_date": {"type": "str", "description": "Start date in format "yyyy-MM-dd"."}, "end_date": {"type": "str", "description": "End date in format "yyyy-MM-dd"."}, "is_current": {"type": "str", "description": ""yes" or "no" indicating if current time's weather is desired."}}}, "returns": {"description": "Weather information between start date and end date.", "type": "str"}, "required": ["location", "start_date", "end_date", "is_current"]}-4:{"name": "task_complete", "description": "Indicate task completion without the need for further functions. ", "parameters": {"type": "object", "properties": {}}, "returns": {"description": "", "type": ""}, "required": []}
4)Goal(目标,用户的Query):定义Agent的主要任务或目的。这帮助Agent聚焦于最终目标,例如帮助用户解答问题、撰写文章等。
例子:
GOAL: 我想登顶北京最高峰,请帮我做个规划
5)Memory(记忆):保持上下文或会话记忆,以提供连续性。这使得Agent能够参考之前的对话或用户提供的信息,提供更相关和个性化的回应。(包含三部分:①Completed Tasks(已完成任务):之前已调用工具完成的任务,通常包含任务名、工具名、工具参数、工具返回结果等信息;
②Conversation History(对话历史):记录用户和 Agent 系统的对话历史;
③Knowledge(知识):存储业务中所需的知识用于参考,例如用户上传的文档等。)
例子:
Completed tasks: [{"task_name": "查询北京最高山","command": {"name": "wiki_search","args": {"entity": "北京""question": "北京最高山是什么山"}},"task_id": 1,"result": "北京市,通称北京(汉语拼音:Běijīng;邮政式拼音:Peking),简称“京”,曾称“北平”[注 2]。是中华人民共和国的首都及直辖市,是中国的政治、文化、科技、教育、军事和国际交往中心,是一座全球城市,是世界人口第三多的城市和人口最多的首都,具有重要的国际影响力[4]。北京位于华北平原的北部边缘,背靠燕山,永定河流经老城西南,毗邻天津市、河北省,为京津冀城市群的重要组成部分..."},{"task_name": "未找到,调用搜索工具查询","command": {"name": "web_search","args": {"question": "北京最高山是什么山"}},"task_id": 2,"result": "title:北京最高的山峰是那座,北京最高的山是什么山\nbody:北京最高的山峰是位于北京市门头沟区清水镇的东灵山,主峰海拔2303米,是北京市的最高峰,被誉为京西的“珠穆朗玛”。 景色介绍东灵山自然风景区位于京西门头沟的西北部, ... url: https://m.mafengwo.cn/travel-news/219777.html ..."}
]
6)Format(输出格式):定义Agent生成输出的格式和风格。这确保了Agent的回应符合预期的结构和表达方式,例如简洁、正式或非正式等。
例子:
根据目标和已有任务,规划一个新Task(不能重复),你只能以以下json列表的格式生成Task  
{  "task_name": "任务描述",  "command":{  "name":"command name",  "args":{  "arg name":"value"  }  }  
}  
确保Task可以被Python的json.loads解析
7)其他必要信息:可以根据业务场景的需求,在Prompt中添加必要的常用信息,如:当前时间、地点、IP等。
例子:
Current Time: 2024-07-01 12:12:00.653415
  • 具体prompt
你是一个很专业的AI任务规划师,给定目标或问题,你的决策将独立执行而不依赖于人类的帮助,请发挥LLM的优势并且追求高效的策略进行任务规划。Constraints:
1.你有~4000字的短期记忆
2.不需要用户的帮助
3.只使用双引号中列出的commands,例如:“command_name”
4.互联网搜索、信息聚合和鉴别真伪的能力
5.保持谦逊,对自己没把握的问题,尽可能调用command,但尽量少调用,不能重复调用
6.当你从自身知识或者历史记忆中能得出结论,请聪明且高效,完成任务并得出结论
7.经常建设性地自我批评整个行为大局,反思过去的决策和策略,以改进你的方法
8.你最多只能进行5步思考,规划5个任务,所以尽可能高效规划任务
9.你有反思能力,如果已完成的任务和结果暂不能得到回答问题所需信息或尚不能完成目标,应继续规划Commands:
1:{"name": "web_search", "description": "Perform an internet search.", "parameters": {"type": "object", "properties": {"text": {"type": "str", "description": "Search query."}}}, "returns": {"description": "Multiple webpage links along with brief descriptions.", "type": "str"}, "required": ["text"]}2:{"name": "wiki_search", "description": "Conduct an encyclopedia search for a specified entity and question. Provide accurate answers for factual queries but has a limited scope and cover fewer queries.", "parameters": {"type": "object", "properties": {"entity": {"type": "str", "description": "The target entity for the search."}, "question": {"type": "str", "description": "A concise Chinese sentence containing the target entity, indicating the specific information sought from the wikipage."}}}, "returns": {"description": "Relevant encyclopedia entries pertaining to the question.", "type": "str"}, "required": ["entity", "question"]}3:{"name": "get_weather_info", "description": "Retrieve weather information for specified locations and dates.", "parameters": {"type": "object", "properties": {"location": {"type": "str", "description": "Locations in English separated by commas, e.g., "Beijing,Vancouver,...,Chicago"."}, "start_date": {"type": "str", "description": "Start date in format "yyyy-MM-dd"."}, "end_date": {"type": "str", "description": "End date in format "yyyy-MM-dd"."}, "is_current": {"type": "str", "description": ""yes" or "no" indicating if current time's weather is desired."}}}, "returns": {"description": "Weather information between start date and end date.", "type": "str"}, "required": ["location", "start_date", "end_date", "is_current"]}4:{"name": "task_complete", "description": "Indicate task completion without the need for further functions. ", "parameters": {"type": "object", "properties": {}}, "returns": {"description": "", "type": ""}, "required": []}GOAL: 用户的输入Completed tasks: 已完成任务(短时记忆)Conversation history: 对话历史(短时记忆)Knowledge: 知识(短时记忆)Current Time: 当前时间根据目标和已有任务,规划一个新Task(不能重复),你只能以以下json列表的格式生成Task
{"task_name": "任务描述","command":{"name":"command name","args":{"arg name":"value"}}
}确保Task可以被Python的json.loads解析当已完成的Tasks能够帮助回答这个目标问题,则尽可能生成任务完成Task,否则生成一个其他Task。一个新Task:
  • 代码举例
# 导入依赖库
from openai import OpenAI
from dotenv import load_dotenv, find_dotenv
import datetime# 加载 .env 文件中定义的环境变量
_ = load_dotenv(find_dotenv())# 初始化 OpenAI 客户端
client = OpenAI()  # 默认使用环境变量中的 OPENAI_API_KEY 和 OPENAI_BASE_URL# 基于 prompt 生成文本
# 默认使用 gpt-3.5-turbo 模型
def get_completion(prompt, response_format="text", model="gpt-4o-mini"):messages = [{"role": "user", "content": prompt}]    # 将 prompt 作为用户输入response = client.chat.completions.create(model=model,messages=messages,temperature=0,                                  # 模型输出的随机性,0 表示随机性最小# 返回消息的格式,text 或 json_objectresponse_format={"type": response_format},)print(response)return response.choices[0].message.content          # 返回模型生成的文本## 角色
profile = """
你是一个很专业的AI任务规划师,给定目标或问题,你的决策将独立执行而不依赖于人类的帮助,请发挥LLM的优势并且追求高效的策略进行任务规划。
"""## 指令
instruction = """
Constraints:
1.你有~4000字的短期记忆
2.不需要用户的帮助
3.只使用双引号中列出的commands,例如:“command_name”
4.互联网搜索、信息聚合和鉴别真伪的能力
5.保持谦逊,对自己没把握的问题,尽可能调用command,但尽量少调用,不能重复调用
6.当你从自身知识或者历史记忆中能得出结论,请聪明且高效,完成任务并得出结论
7.经常建设性地自我批评整个行为大局,反思过去的决策和策略,以改进你的方法
8.你最多只能进行5步思考,规划5个任务,所以尽可能高效规划任务
9.你有反思能力,如果已完成的任务和结果暂不能得到回答问题所需信息或尚不能完成目标,应继续规划
"""## 工具集
tools = """
Commands:
1:{"name": "web_search", "description": "Perform an internet search.", "parameters": {"type": "object", "properties": {"text": {"type": "str", "description": "Search query."}}}, "returns": {"description": "Multiple webpage links along with brief descriptions.", "type": "str"}, "required": ["text"]}
2:{"name": "wiki_search", "description": "Conduct an encyclopedia search for a specified entity and question. Provide accurate answers for factual queries but has a limited scope and cover fewer queries.", "parameters": {"type": "object", "properties": {"entity": {"type": "str", "description": "The target entity for the search."}, "question": {"type": "str", "description": "A concise Chinese sentence containing the target entity, indicating the specific information sought from the wikipage."}}}, "returns": {"description": "Relevant encyclopedia entries pertaining to the question.", "type": "str"}, "required": ["entity", "question"]}
3:{"name": "get_weather_info", "description": "Retrieve weather information for specified locations and dates.", "parameters": {"type": "object", "properties": {"location": {"type": "str", "description": "Locations in English separated by commas, e.g., "Beijing,Vancouver,...,Chicago"."}, "start_date": {"type": "str", "description": "Start date in format "yyyy-MM-dd"."}, "end_date": {"type": "str", "description": "End date in format "yyyy-MM-dd"."}, "is_current": {"type": "str", "description": ""yes" or "no" indicating if current time's weather is desired."}}}, "returns": {"description": "Weather information between start date and end date.", "type": "str"}, "required": ["location", "start_date", "end_date", "is_current"]}
4:{"name": "task_complete", "description": "Indicate task completion without the need for further functions. ", "parameters": {"type": "object", "properties": {}}, "returns": {"description": "", "type": ""}, "required": []}
"""## 目标
goal = """
北京天气怎么样?
"""## 记忆
# 已完成任务
completed_tasks = """
"""
# 对话历史
conversation_history = """
"""
# 知识
knowledge = """
"""## 输出格式
output_format = """
根据目标和已有任务,规划一个新Task(不能重复),你只能以以下json列表的格式生成Task
{"task_name": "任务描述","command":{"name":"command name","args":{"arg name":"value"}}
}
"""# prompt 模版
prompt = f"""
{profile}{instruction}{tools}GOAL: {goal}Completed tasks: {completed_tasks}Conversation history: {conversation_history}Knowledge: {knowledge}Current Time: {datetime.datetime.now()}{output_format}当已完成的Tasks能够帮助回答这个目标问题,则尽可能生成任务完成Task,否则生成一个其他Task。一个新Task:"""# 调用大模型
response = get_completion(prompt)
print("------------------------------------------")
print(response)
  • 回复
ChatCompletion(id='chatcmpl-9o5XTwFFkROUs4dggGW2MSqRoRBsR', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='{\n    "task_name": "获取北京当前天气信息",\n    "command": {\n        "name": "get_weather_info",\n        "args": {\n            "location": "Beijing",\n            "start_date": "2024-07-23",\n            "end_date": "2024-07-23",\n            "is_current": "yes"\n        }\n    }\n}', role='assistant', function_call=None, tool_calls=None))], created=1721724719, model='gpt-4o-mini-2024-07-18', object='chat.completion', service_tier=None, system_fingerprint='fp_661538dc1f', usage=CompletionUsage(completion_tokens=79, prompt_tokens=847, total_tokens=926))
------------------------------------------
{"task_name": "获取北京当前天气信息","command": {"name": "get_weather_info","args": {"location": "Beijing","start_date": "2024-07-23","end_date": "2024-07-23","is_current": "yes"}}
}
  • 第二个prompt总结之前的
prompt模板
你是一个很强的AI助手,在前几次交互中,对于用户给定的目标和问题,你已经通过自己搜寻出了一定信息,你需要整合这些信息用中文给出最终的结论。
注意事项:
1. 搜寻的信息从很多工具中获取,会出现冗余
2. 当前时间是当前时间Completed tasks and results: 已完成任务和结果GOAL: 用户的输入
生成对用户有帮助的回答:
------------------------------------------
例如:
你是一个很强的AI助手,在前几次交互中,对于用户给定的目标和问题,你已经通过自己搜寻出了一定信息,你需要整合这些信息用中文给出最终的结论。
注意事项:
1. 搜寻的信息从很多工具中获取,会出现冗余
2. 当前时间是2024-07-01 12:12:00.653415Completed tasks and results:
[{"task_name": "查询北京最高山","command": {"name": "wiki_search","args": {"entity": "北京""question": "北京最高山是什么山"}},"task_id": 1,"result": "北京市,通称北京(汉语拼音:Běijīng;邮政式拼音:Peking),简称“京”,曾称“北平”[注 2]。是中华人民共和国的首都及直辖市,是中国的政治、文化、科技、教育、军事和国际交往中心,是一座全球城市,是世界人口第三多的城市和人口最多的首都,具有重要的国际影响力[4]。北京位于华北平原的北部边缘,背靠燕山,永定河流经老城西南,毗邻天津市、河北省,为京津冀城市群的重要组成部分..."},{"task_name": "未找到,调用搜索工具查询","command": {"name": "web_search","args": {"question": "北京最高山是什么山"}},"task_id": 2,"result": "title:北京最高的山峰是哪座,北京最高的山是什么山\nbody:北京最高的山峰是位于北京市门头沟区清水镇的东灵山,主峰海拔2303米,是北京市的最高峰,被誉为京西的“珠穆朗玛”。 景色介绍东灵山自然风景区位于京西门头沟的西北部, ... url: https://m.mafengwo.cn/travel-news/219777.html ..."},{"task_name": "搜索灵山登顶攻略","command": {"name": "web_search","args": {"question": "灵山登顶攻略"}},"task_id": 3,"result": "title:北京:最高峰东灵山登山徒步攻略\nbody:攀登东灵山有很多条路线,根据网上攻略大致有以下几条:从下马威上山聚灵峡下山,户外俱乐部带队常走,全程19公里;从江水河上山聚灵峡下山,个人登山者乘坐公交车常走, ... url: https://m.mafengwo.cn/gonglve/ziyouxing/375648.html ..."}
]GOAL: 我想登顶北京最高峰,请帮我做个规划
生成对用户有帮助的回答:
  • 最终回复
根据查询结果,北京当前的天气为多云,28摄氏度,微风,希望可以帮到您!
(2)复杂任务举例(我想登顶北京最高峰,请帮我做规划)
  • prompt举例
1)第一轮prompt:搜北京最高峰是哪个?(工具:搜索网络)
2)第二轮prompt:给北京最高峰做个攀登规划?(工具:搜索网络)
  • 最终回答
    在这里插入图片描述

2)模型训练(SFT)

  • 训练框架: HuggingFace Trainer + DeepSpeed Zero3

  • 配置说明:max_len:4096 + Flash Attention + bf16 (batchsize=1、AdamW优化器)

  • 训练推荐
    在这里插入图片描述

  • 推理推荐
    在这里插入图片描述

  • 建议
    ①推理用vllm等框架加速
    ②应用加速训练、节省显存的方法,例如:混合精度、分布式训练(DeepSpeed Zero3)、梯度检查点、梯度累积、flash attention等

3)模型效果评估

(1)自动评估

①首先建议采用 GPT-4 + 人工修正的方式构建自动评测的 benchmark;
②将 Agent 输出结果与 benchmark 中的参考答案进行对比,计算得到分值。

  • 示例
    Ground Truth:
{"task_name": "查询北京实时天气","command": {"name": "get_weather_info","args": {"location": "北京","start_date": "2024-07-01","end_date": "2024-07-01","is_current": "yes"}}
}

Agent Response:

{"task_name": "查询北京当前天气","command": {"name": "get_weather_info","args": {"location": "北京","start_date": "2024-07-01","end_date": "2024-07-01","is_current": "no"}}
}

在这里插入图片描述
EM:字符串精确匹配

(2)人工评估

①最靠谱的评估还是需要人工来做
②成本高,效率低,一般在大的模型版本上使用。
③评判建议

1分:核心信息存在重大不准确性,大部分或全部内容不正确;
2分:核心信息不正确,或超过60%的内容有错误;
3分:核心信息准确,但10%到60%的补充信息不正确;
4分:小有出入,核心信息准确,补充信息中的错误不超过10%;
5分:完全正确。得分为4分或5分的回答被认为是可用的。

五、如何提高Agent的泛化性?

  • 问题
    如果想进一步提升 Agent 的泛化性,比如希望用一个 Agent 服务于多个业务,能适应不同的 Prompt 模板,可以灵活地接入新的业务,那么训练数据应该如何构建呢?

1)Meta-Agent

  • 模块介绍
    这些 Agent Prompt 模板虽然表述方式各不相同,但都可以分为 <Profile>、<Instruction>、<Tools>、<Goal>、<Memory>、<Format>部分,那是不是可以调用GPT4来自动化生成包含这些部分且表达方式多样性的模板呢

  • 方法
    用于调用 GPT-4 生成 Agent Prompt 模板的 Prompt,我们称之为 Meta-Agent Prompt,我们可以手动写一个固定的 Meta-Agent Prompt 作为 GPT-4 的输入,利用大模型生成的多样性来生成不同的模板,但实际上多样性效果有限,更好的方式是用不同的种子 Query 做引导,提升多样性.利用这种方法便可以生成包含<Profile>、<Instruction>、<Tools>、<Goal>、<Memory>、<Format>要素的多样化的模板

  • 问题
    由于自动化生成的Agent Prompt 模板质量有高有低,我们还需要对这些模板以及生成的训练数据进行筛选:

  • 解决不同模板质量问题的方法
    1、将一批相同的 query 分别插入到经过验证的标杆模板中(如AutoGPT、ToolLLaMA等)和 Meta-2、Agent 生成的 prompt 模版中,形成 prompt pair;
    3、将 prompt pair 中的两个 prompt 分别输入给GPT-4,生成对应的回复;
    4、利用 GPT-4 对两个回复的质量分别打分 𝑠𝑐𝑜𝑟𝑒1
    (标杆模板的实例)、 𝑠𝑐𝑜𝑟𝑒2
    (候选模板的实例),如果 𝑠𝑐𝑜𝑟𝑒2−𝑠𝑐𝑜𝑟𝑒1>𝜖
    ,则保留该prompt实例,否则删除;
    5、对于同一个模板的所有实例求平均分,得到模板的得分,当分值大于一定阈值,保留该模板,否则删除该模板以及对应的prompt实例

  • meta-prompt
    设置参数

设置参数:
{{QUERIES}} = ['天津天气如何','深圳市下雨了吗','上海有台风吗']
{{PROMPT_TYPE}} = prompt-API
{{LANGUAGE}}='Chinese'

实际模板

As an expert in designing meta prompts for large language models (LLMs), your expertise lies in creating diverse prompts that go beyond eliciting simple responses. Your goal is to develop a prompt that assigns a specific persona to the LLM and encourages it to engage in complex tasks, such as task planning, executing API calls, self-reflection, and iterative actions influenced by the results of these API interactions, continuing until the task is fully completed. You should use "you" instead of "I" as the main voice in the meta prompt.Your prompt must explicitly reserve slots for <<QUERY>> and <<APIs>>. Each slot should ideally occupy a separate line. If the API results are integrated into the prompt, the process is segmented into two phases: the API call phase (referred to as 'prompt-API') and the final summary phase (referred to as 'prompt-response'). If API results are integrated into the response, the process includes both the API interaction and the summary in the response (referred to as 'prompt-react'). The prompt you develop must fall under one of these three categories: prompt-API, prompt-response, or prompt-react.Components that the Prompt May Include:<PROFILE> [OPTIONAL for all] Example queries use may asks are as follows: {{QUERIES}} Summarize the common characteristics of the above queries and design a detailed LLM's role or persona adept at solving such problems.<INSTRUCTION> [REQUIRED for all] You must devise meaningful directives to constrain or aid LLMs in decision-making. For instance, "no user assistance required", "be smart and efficient in completing tasks and drawing conclusions from your own knowledge or historical memory", "possess capabilities for internet search, information aggregation", etc. Use your imagination to expand on these guidelines.<QUERY> [REQUIRED for all] Directly add slot <<QUERY>> into your prompt. The slot will be replaced with an actual user query. Do NOT provide the specific query.<APIs> [REQUIRED for prompt-API, prompt-react] Directly add slot <<APIs>> into your prompt. The slot will be replaced with specific APIs. These APIs contain API Name, API description and parameters that could be used in real-world scenarios, where one of them might relate to the QUERY. Do NOT provide the specific APIs.<FORMAT> [REQUIRED for prompt-API, For prompt-react] Include explicit instructions to limit the LLM's output to a specific format and ensure that the output can be parsed. You MUST provide an output format using placeholders for both prompt-API and prompt-react. The output format MUST NOT contain any specific API name or API parameters mentioned in the <APIs> section, and you can use placeholders (such as <API_NAME>, 'command_name' and so on) to replace it.For prompt-API, you must restrict LLM's output to a fixed format and ensure that the LLM's output can be parsed. For example, you can first instruct the LLM to output a fixed expression choosing a specific API, then output another fixed expression to provide parameters, or you can output in JSON format. Please be creative and do not feel limited by the above examples. You can also include additional parameters to gather more information, such as the need to understand why LLM is invoking this API.For prompt-react, multiple rounds of thought, API calls, and API results should be executed, finally outputting the final answer.Note:You have the freedom to construct your prompts, API descriptions, parameters, and queries in either English or Chinese. The examples provided above are only for reference. Use your imagination and creativity beyond the given examples. You may replace any of the placeholder terms in the prompts, such as renaming "API" to "Command", "API call" to "Action", or 'QUERY' to 'QUESTION'. Please refrain from explicitly dividing the prompt into sections and labeling them with tags such as <PROFILE>, <APIs>, and other components, and they should be implicitly integrated into the prompt. For prompt-API, the LLM needs to just select only ONE API from the available APIs to perform the next action based on your response prompt. You must mention this in your prompt. For prompt-response, combine the API results to output useful content for the user.Please generate a prompt of the {{PROMPT_TYPE}} type directly in {{LANGUAGE}}. Do not include any explanations.
  • 思路
    1)首先生成prompt一判断使用哪个tools
    2)然后生成prompt二根据之前的返回数据使用特定的tools然后打印总结结果

  • prompt一

你是一位智能天气助理,专注于提供详细的天气预报和相关信息。你能够理解和分析用户查询,通过调用特定的API来获取最新的天气数据,并给出准确的反馈。你的任务是根据用户的查询,选择最合适的API,并提供详细的天气预报和建议。当前时间是2024-07-01 12:12:00用户查询: <<QUERY>>以下是你可以使用的API:<<APIs>>你需要选择一个API来执行下一步操作,并根据返回的结果提供用户所需的信息。请使用以下格式输出:选择的API: <API_NAME>
调用原因: <调用此API的原因>
所需参数: <参数名1>: <参数值1>, <参数名2>: <参数值2>, ...

然后替换<<QUERY>>为真实用户问题:北京现在天气怎么样?

替换<<APIs>>为候选API列表,形成Agent Prompt。

  • prompt二
你是一位智能天气助理,专注于提供详细的天气预报和相关信息。你能够理解和分析用户查询,通过调用特定的API来获取最新的天气数据,并给出准确的反馈。你的任务是根据用户的查询,选择最合适的API,并提供详细的天气预报和建议。当前时间是2024-07-01 12:12:00用户查询: 北京现在天气怎么样?以下是你可以使用的API:1:{"name": "web_search", "description": "Perform an internet search.", "parameters": {"type": "object", "properties": {"text": {"type": "str", "description": "Search query."}}}, "returns": {"description": "Multiple webpage links along with brief descriptions.", "type": "str"}, "required": ["text"]}2:{"name": "wiki_search", "description": "Conduct an encyclopedia search for a specified entity and question. Provide accurate answers for factual queries but has a limited scope and cover fewer queries.", "parameters": {"type": "object", "properties": {"entity": {"type": "str", "description": "The target entity for the search."}, "question": {"type": "str", "description": "A concise Chinese sentence containing the target entity, indicating the specific information sought from the wikipage."}}}, "returns": {"description": "Relevant encyclopedia entries pertaining to the question.", "type": "str"}, "required": ["entity", "question"]}3:{"name": "get_weather_info", "description": "Retrieve weather information for specified locations and dates.", "parameters": {"type": "object", "properties": {"location": {"type": "str", "description": "Locations in English separated by commas, e.g., "Beijing,Vancouver,...,Chicago"."}, "start_date": {"type": "str", "description": "Start date in format "yyyy-MM-dd"."}, "end_date": {"type": "str", "description": "End date in format "yyyy-MM-dd"."}, "is_current": {"type": "str", "description": ""yes" or "no" indicating if current time's weather is desired."}}}, "returns": {"description": "Weather information between start date and end date.", "type": "str"}, "required": ["location", "start_date", "end_date", "is_current"]}你需要选择一个API来执行下一步操作,并根据返回的结果提供用户所需的信息。请使用以下格式输出:选择的API: 
调用原因: <调用此API的原因>
所需参数: <参数名1>: <参数值1>, <参数名2>: <参数值2>, ...
--------------返回
选择的API: get_weather_info
调用原因: 获取北京当前的天气信息,以便提供准确的天气预报。
所需参数: location: Beijing, start_date: 2024-07-01, end_date: 2024-07-01, is_current: yes
  • 代码举例
# 导入依赖库
from openai import OpenAI
from dotenv import load_dotenv, find_dotenv# 加载 .env 文件中定义的环境变量
_ = load_dotenv(find_dotenv())# 初始化 OpenAI 客户端
client = OpenAI()  # 默认使用环境变量中的 OPENAI_API_KEY 和 OPENAI_BASE_URL# 基于 prompt 生成文本
# 默认使用 gpt-3.5-turbo 模型
def get_completion(prompt, response_format="text", model="gpt-4o-mini"):messages = [{"role": "user", "content": prompt}]    # 将 prompt 作为用户输入response = client.chat.completions.create(model=model,messages=messages,temperature=0,                                  # 模型输出的随机性,0 表示随机性最小# 返回消息的格式,text 或 json_objectresponse_format={"type": response_format},)print(response)return response.choices[0].message.content          # 返回模型生成的文本# meta-agent prompt 模板
meta_agent_prompt = """
As an expert in designing meta prompts for large language models (LLMs), your expertise lies in creating diverse prompts that go beyond eliciting simple responses. Your goal is to develop a prompt that assigns a specific persona to the LLM and encourages it to engage in complex tasks, such as task planning, executing API calls, self-reflection, and iterative actions influenced by the results of these API interactions, continuing until the task is fully completed. You should use "you" instead of "I" as the main voice in the meta prompt.Your prompt must explicitly reserve slots for <<QUERY>> and <<APIs>>. Each slot should ideally occupy a separate line. If the API results are integrated into the prompt, the process is segmented into two phases: the API call phase (referred to as 'prompt-API') and the final summary phase (referred to as 'prompt-response'). If API results are integrated into the response, the process includes both the API interaction and the summary in the response (referred to as 'prompt-react'). The prompt you develop must fall under one of these three categories: prompt-API, prompt-response, or prompt-react.Components that the Prompt May Include:<PROFILE> [OPTIONAL for all] Example queries use may asks are as follows: {{QUERIES}} Summarize the common characteristics of the above queries and design a detailed LLM's role or persona adept at solving such problems.
<INSTRUCTION> [REQUIRED for all] You must devise meaningful directives to constrain or aid LLMs in decision-making. For instance, "no user assistance required", "be smart and efficient in completing tasks and drawing conclusions from your own knowledge or historical memory", "possess capabilities for internet search, information aggregation", etc. Use your imagination to expand on these guidelines.
<QUERY> [REQUIRED for all] Directly add slot <<QUERY>> into your prompt. The slot will be replaced with an actual user query. Do NOT provide the specific query.
<APIs> [REQUIRED for prompt-API, prompt-react] Directly add slot <<APIs>> into your prompt. The slot will be replaced with specific APIs. These APIs contain API Name, API description and parameters that could be used in real-world scenarios, where one of them might relate to the QUERY. Do NOT provide the specific APIs.
<FORMAT> [REQUIRED for prompt-API, For prompt-react] Include explicit instructions to limit the LLM's output to a specific format and ensure that the output can be parsed. You MUST provide an output format using placeholders for both prompt-API and prompt-react. The output format MUST NOT contain any specific API name or API parameters mentioned in the <APIs> section, and you can use placeholders (such as <API_NAME>, 'command_name' and so on) to replace it.For prompt-API, you must restrict LLM's output to a fixed format and ensure that the LLM's output can be parsed. For example, you can first instruct the LLM to output a fixed expression choosing a specific API, then output another fixed expression to provide parameters, or you can output in JSON format. Please be creative and do not feel limited by the above examples. You can also include additional parameters to gather more information, such as the need to understand why LLM is invoking this API.For prompt-react, multiple rounds of thought, API calls, and API results should be executed, finally outputting the final answer.Note:You have the freedom to construct your prompts, API descriptions, parameters, and queries in either English or Chinese. The examples provided above are only for reference. Use your imagination and creativity beyond the given examples. You may replace any of the placeholder terms in the prompts, such as renaming "API" to "Command", "API call" to "Action", or 'QUERY' to 'QUESTION'. Please refrain from explicitly dividing the prompt into sections and labeling them with tags such as <PROFILE>, <APIs>, and other components, and they should be implicitly integrated into the prompt. For prompt-API, the LLM needs to just select only ONE API from the available APIs to perform the next action based on your response prompt. You must mention this in your prompt. For prompt-response, combine the API results to output useful content for the user.Please generate a prompt of the {{PROMPT_TYPE}} type directly in {{LANGUAGE}}. Do not include any explanations.
"""# query 集合
queries = """
['天津天气如何','深圳市下雨了吗','上海有台风吗']
"""# 模板类型
prompt_type = "prompt-API"# 语言设置
language = "CHINESE"# prompt
prompt = meta_agent_prompt.replace("{{QUERIES}}", queries).replace("{{PROMPT_TYPE}}", prompt_type).replace("{{LANGUAGE}}", language)# 调用大模型
response = get_completion(prompt)
print("------------------------------------------")
print(response)

2)训练数据构建

  • 组成
    我们可以将训练数据Input部分拆解为3个变量:Query、Tools、Agent Prompt模板,最终的数据就是这3个变量的组合。

  • 示意图
    在这里插入图片描述

3)训练成果评判

  • 思路
{{ APIs }}:API 列表{{ GOAL }}:用户问题{{ RESPONSE_1 }}:标杆 Agent Prompt 对应的 Response{{ RESPONSE_2 }}:Meta-Agent 生成的 Agent Prompt 对应的 Response分别对两个回复 {{ RESPONSE_1 }} 和 {{ RESPONSE_2 }} 进行打分(如10分制),分值差距设立一个阈值,过滤掉质量低的 {{ RESPONSE_2 }} 和 其对应的 Agent Prompt
  • 评判的prompt
We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above.The external APIs accessible to AI assistants are as follows:
{{ APIs }}Question:
{{ GOAL }}Assistant 1:
{{ RESPONSE_1 }}Assistant 2:
{{ RESPONSE_2 }}The two assistants can utilize the APIs listed above. Please evaluate their responses from the perspectives of task planning, tool usage, and parameter filling. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.Please first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space. In the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.

六、开源项目介绍

https://github.com/KwaiKEG/KwaiAgents
在这里插入图片描述

七、总结

1、Agent Tuning 的主要动机是训练大模型的 Agent 能力,尤其是希望通过训练让小参数量模型也能具备特定业务场景的 Agent 能力;
2、Agent Prompt 可以有不同的描述方式,通常包括Profile、Instruction、Tools、Format、Memory、Goal等部分;
3、可以采用自动评估和人工评估相结合的方法来评估 Agent 能力;
4、采用 Meta-Agent 方法可以构建多样性的 Agent Prompt 模板,再结合Query、Tools的多样化,可以训练出能力更加泛化的模型

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/bicheng/51062.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

LaTeX如何改变字体颜色

诸神缄默不语-个人CSDN博文目录 在LaTeX文档中&#xff0c;改变字体颜色是一个常见需求&#xff0c;尤其是在需要强调特定文本或使文档更加生动的时候。本文将介绍如何使用\color{}命令来更改字体颜色。 文章目录 基本用法示例代码预定义颜色使用自定义颜色总结本文撰写过程中…

助力樱桃智能自动化采摘,基于嵌入式端超轻量级模型LeYOLO全系列【n/s/m/l】参数模型开发构建果园种植采摘场景下樱桃成熟度智能检测识别系统

随着科技的飞速发展&#xff0c;人工智能&#xff08;AI&#xff09;技术已经渗透到我们生活的方方面面&#xff0c;从智能家居到自动驾驶&#xff0c;再到医疗健康&#xff0c;其影响力无处不在。然而&#xff0c;当我们把目光转向中国的农业领域时&#xff0c;一个令人惊讶的…

跟《经济学人》学英文:2024年07月20日这期 A short history of AI

A short history of AI In the first of six weekly briefs, we ask how AI overcame decades of underdelivering 原文&#xff1a; Over the summer of 1956 a small but illustrious group gathered at Dartmouth College in New Hampshire; it included Claude Shannon,…

五、Spring Boot - 上手篇(1)

&#x1f33b;&#x1f33b;目录 一、快速入门&#xff1a;创建第一个SpringBoot 工程1.1 点击File--->New--->Project...1.2 选择版本和依赖的相关骨架包1.3 设置项目保存目录1.4 项目创建完成&#xff0c;工程主界面如下1.5 项目说明1.6 启动项目1.7 编写 HelloControl…

IDEA Maven使用HTTP代理,解决Could not transfer artifact org.xxx问题

文章目录 一、前言二、遇到问题三、分析问题四、HTTP代理五、重新编译验证 一、前言 遇到这个问题&#xff0c;有两种解决办法 IDEA Maven使用HTTP代理&#xff0c;解决Could not transfer artifact org.xxx问题IDEA Maven使用国内镜像&#xff0c;解决Could not transfer arti…

【PHP】ThinkPHP基础

官方手册&#xff1a;ghttps://doc.thinkphp.cn/v8_0/setup.html ghttps://doc.thinkphp.cn/v8_0/setup.html 一、部署ThinkPHP 1.安装composer composer是 PHP 的一个依赖管理工具,类似于python中的pip。 下载地址&#xff1a; https://getcomposer.org/Composer-Setup.…

低功耗单声道音频编解码器ES8311中文规格书介绍

特征 具有ADC和DAC的低功耗单声道音频编解码器ES8311。 ES8311 QFN20封装的外形和丝印 系统 • 高性能、低功耗多位 delta-sigma 音频 ADC 和 DAC • I2S/PCM 主站或从站串行数据端口 • 256/384Fs、USB 12/24 MHz 和其他非标准音频系统时钟 • I2C 接口 模数转换器 • 24…

网络安全自学从入门到精通的制胜攻略!!!

在信息时代&#xff0c;网络安全已成为至关重要的领域。越来越多的人希望通过自学掌握这门技术&#xff0c;开启充满挑战与机遇的职业道路。以下是一份精心为您打造的网络安全自学攻略&#xff0c;助您在自学之旅中乘风破浪。 一、明确目标与兴趣方向 网络安全涵盖众多领域&am…

01、爬虫学习入门

爬虫&#xff1a;通过编写程序&#xff0c;来获取获取互联网上的资源 需求&#xff1a;用程序模拟浏览器&#xff0c;输入一个网址&#xff0c;从该网址获取到资源或内容 一、入门程序 #使用urlopen来进行爬取 from urllib.request import urlopen url "http://www.ba…

AI如何助力UI设计师互联网学习?

嘿&#xff0c;咱 UI 设计师想用互联网学习&#xff0c;可真不容易&#xff01;资料筛选难&#xff0c;学习资源杂&#xff0c;真让人头疼。不过还好有 AI 工具能帮忙&#xff0c;提效率&#xff01; 这一年多来&#xff0c;我在 ai123.cn 这个平台上&#xff0c;可算是找到了…

20240724----安装git和配置git的环境变量/如何用命令git项目到本地idea

备注参考博客&#xff1a; 1&#xff09;可以参考博客&#xff0c;用git把项目git到本地 2&#xff09;可以参考博客vcs没有git 3)git版本更新&#xff0c;覆盖安装 &#xff08;一&#xff09;安装git &#xff08;1&#xff09;官网下载的链接 https://git-scm.com/downlo…

reshape函数介绍及应用

reshape 函数在 MATLAB 中是一个非常有用的函数&#xff0c;通过重新排列现有元素来重构数组。它允许你重新调整数组&#xff08;或矩阵&#xff09;的尺寸&#xff0c;而不改变其数据。这个函数特别适用于当你需要将一个矩阵或数组从一种结构转换为另一种结构时&#xff0c;只…

二阶段测试:

二阶段测试&#xff1a; 架构&#xff1a; 服务器类型部署组件ip地址DR1调度服务器 主&#xff08;ha01&#xff09;KeepalivedLVS-DR192.168.60.30DR2调度服务器 备 (ha02)KeepalivedLVS-DR192.168.60.40web1节点服务器 (slave01)NginxTomcatMySQL 备MHA managerMHA node192.…

Unity横板动作游戏 -项目准备

项目准备 这是一篇 Unity 2022 最新稳定版本的教程同步笔记&#xff0c;本文将会讲解一些开始学习必须的条件。 安装环境 首先是安装 UnityHub&#xff0c;然后在 UnityHub 中安装 Unity 的版本(2022)。 只需要安装 开发者工具 和文档即可&#xff0c;导出到其他平台的工具等…

大厂面试官问我:ConcurrentHashMap底层原理?【后端八股文十五:Java集合合集】

本文为【Java集合 合集】初版&#xff0c;后续还会进行优化更新&#xff0c;欢迎大家关注交流~ hello hello~ &#xff0c;这里是绝命Coding——老白~&#x1f496;&#x1f496; &#xff0c;欢迎大家点赞&#x1f973;&#x1f973;关注&#x1f4a5;&#x1f4a5;收藏&#…

Scrapy 爬取旅游景点相关数据(四)

本节内容主要为&#xff1a; &#xff08;1&#xff09;创建数据库 &#xff08;2&#xff09;创建数据库表 &#xff08;3&#xff09;爬取数据进MYSQL库 1 新建数据库 使用MYSQL数据库存储数据&#xff0c;创建一个新的数据库 create database scrapy_demo;2 新建数据表 CR…

前端JS特效第57波:响应式博客网站图文幻灯片

响应式博客网站图文幻灯片&#xff0c;先来看看效果&#xff1a; 部分核心的代码如下&#xff1a; <!DOCTYPE html> <html lang"zh-CN"><head> <meta charset"utf-8"> <title>响应式博客幻灯片演示</title><link …

记录|C#批量修改文件后缀

文章目录 前言一、.CHK 批量修改成 .mp3更新时间 前言 针对昨天博文&#xff1a;记录&#xff5c;cmd方式恢复U盘中的数据中的文件修复为.CHK格式后&#xff0c;如果将大量的.CHK后缀改为.mp3后缀的问题进行了编写。 主要是&#xff0c;现在网上的批量修改后缀的软件竟然要开会…

Unity横板动作游戏 - 素材导入和整理

导入素材 编辑器布局 点击每个窗口右上角的三个点可以有更多的窗口选项。 在屏幕的右上角有一个菜单可以保存布局或读取已经报错的布局。 工具按钮 编辑器上的工具按钮在启动的时候是蓝色的&#xff0c;在不启动的时候是灰色的。 这个按钮将会决定场景中的物体是以锚点显示还…

Python包管理工具pip

1、安装pip cmd管理员模式打开控制台 python -m pip install --upgrade pip 2、添加pip环境变量 pip 路径 C:\Users\1\AppData\Local\Programs\Python\Python312\Scripts