LLM function call

简介

在大模型领域，尤其是在自然语言处理（NLP）和人工智能（AI）应用中，function call 是一种非常重要的技术手段。它主要用于让模型在生成或处理文本的过程中调用外部功能或执行特定任务，有以下应用场景：

数据查询和处理

AI模型可以调用数据库查询函数，以获取所需的数据。例如，在客户服务应用中，模型可以调用函数从客户数据库中提取客户信息

执行计算或分析

在需要复杂计算或数据分析的场景中，模型可以调用外部的计算函数。例如，金融分析系统中，模型可以调用函数进行风险评估或收益预测。

第三方API集成

模型可以通过function call集成第三方API服务。例如，调用天气API获取当前天气信息，或者调用地图服务API获取地理位置信息。

自动化工作流程

模型可以调用各种自动化脚本或函数以完成特定任务，例如发送邮件、生成报告或触发警报。

实现流程

使用大模型的function call能力需要进行以下步骤：

函数定义：事先定义好可以调用的函数及其接口。这些函数通常定义在代码库中，并以API形式暴露给AI模型。
函数调用机制：模型生成特定的调用指令，这些指令被解析并触发相应的函数执行。指令通常包含函数名及所需的参数。
响应处理：函数执行完成后，将结果返回给模型，模型根据返回的结果进行后续的生成或决策。例如，获取到的数据库查询结果可以直接用于生成下一步的文本输出。

示例

以下是一个简单的示例，展示如何在AI对话系统中使用function call：

用户请求： “请告诉我今天的天气。”

模型响应：

解析用户意图，识别需要调用天气API。
生成function call指令，如get_weather(location="current location")。
调用天气API获取天气信息。
将API返回的天气信息整合到回复中生成最终输出。

最终输出： “今天的天气是晴天，气温大约25摄氏度。”

OpenAI Tool call

目前OpenAI提供的接口建议使用的是tool call，接口文档：https://platform.openai.com/docs/api-reference/chat/create，使用示例：https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models

用户询问问题

如果提供了多个functions，本轮想指定某个function，可以使用tool_choice参数，如：tool_choice: {"type": "function", "function": {"name": "gaode_weather"}}，默认情况下传auto让模型根据用户的输入去决定是否要调用工具，如果本轮不需要模型调用工具则传none。

curl http://localhost:80/v1/chat/completions \-H "Content-Type: application/json" \-H "Authorization: Bearer $OPENAI_API_KEY" \-d '{"model": "gpt-4","messages": [{"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."},{"role": "user", "content": "今天深圳天气怎么样？"}],"tools": [{"type": "function","function": {"name": "gaode_weather","description": "A tool when you want to ask about the weather or weather-related question.","parameters": {"type": "object","properties": {"city": {"type": "string","description": "If you don't know you can extract the city name from the question or you can reply：Please tell me your city. You have to extract the Chinese city name from the question."}},"required": ["city"]}}}],"tool_choice": "auto",}'

提取模型返回的内容

有的比较新的模型，可以一次性返回多个tool调用。不支持的旧模型则是按多轮调用，比如先调用日期工具，再调用天气工具

返回内容：

{"id": "chatcmpl-abc123","object": "chat.completion","created": 1699896916,"model": "gpt-3.5-turbo-0125","choices": [{"index": 0,"message": {"role": "assistant","content": null,"tool_calls": [{"id": "call_Dn2RJJSxzDm49vlVTehseJ0k","type": "function","function": {"name": "gaode_weather","arguments": "{\n\"city\": \"深圳\"\n}"}}]},"logprobs": null,"finish_reason": "tool_calls"}],"usage": {"prompt_tokens": 82,"completion_tokens": 17,"total_tokens": 99}
}

调用高德天气API，获取返回结果，将工具的调用结果加入到请求的message参数中，并调用模型获取用户问题的结果。

curl http://localhost:80/v1/chat/completions \-H "Content-Type: application/json" \-H "Authorization: Bearer $OPENAI_API_KEY" \-d '{"model": "gpt-4","messages": [{"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."},{"role": "user", "content": "今天深圳天气怎么样？"},{"role": "assistant","tool_calls": [{"id": "call_Dn2RJJSxzDm49vlVTehseJ0k","type": "function","function": {"name": "gaode_weather","arguments": "{\n\"city\": \"深圳\"\n}"}}]},{"role": "tool","tool_call_id": "call_Dn2RJJSxzDm49vlVTehseJ0k","name": "gaode_weather","content": "[{\"date\": \"2024-06-18\", \"week\": \"2\", \"dayweather\": \"阵雨\", \"daytemp_float\": \"32.0\", \"daywind\": \"北\", \"nightweather\": \"多云\", \"nighttemp_float\": \"27.0\"}, {\"date\": \"2024-06-19\", \"week\": \"3\", \"dayweather\": \"阵雨\", \"daytemp_float\": \"32.0\", \"daywind\": \"北\", \"nightweather\": \"多云\", \"nighttemp_float\": \"27.0\"}, {\"date\": \"2024-06-20\", \"week\": \"4\", \"dayweather\": \"阵雨\", \"daytemp_float\": \"32.0\", \"daywind\": \"南\", \"nightweather\": \"多云\", \"nighttemp_float\": \"28.0\"}, {\"date\": \"2024-06-21\", \"week\": \"5\", \"dayweather\": \"多云\", \"daytemp_float\": \"33.0\", \"daywind\": \"南\", \"nightweather\": \"多云\", \"nighttemp_float\": \"28.0\"}]"}]}'

模型根据工具的调用结果回答用户问题

{"id": "chatcmpl-123","object": "chat.completion","created": 1677652288,"model": "gpt-3.5-turbo-0125","system_fingerprint": "fp_44709d6fcb","choices": [{"index": 0,"message": {"role": "assistant","content": "深圳未来几天的天气情况如下：\n\n2024年6月18日，星期六，天气为阵雨，白天温度为32.0摄氏度，北风，晚上多云，温度为27.0摄氏度；\n\n2024年6月19日，星期日，天气为阵雨，白天温度为32.0摄氏度，北风，晚上多云，温度为27.0摄氏度；\n\n2024年6月20日，星期一，天气为阵雨，白天温度为32.0摄氏度，南风，晚上多云，温度为28.0摄氏度；\n\n2024年6月21日，星期二，天气为多云，白天温度为33.0摄氏度，南风，晚上多云，温度为28.0摄氏度。\n\n请注意天气变化，合理安排行程。"},"logprobs": null,"finish_reason": "stop"}],"usage": {"prompt_tokens": 9,"completion_tokens": 12,"total_tokens": 21}
}

OpenAI Function call

目前function call已经被openAI标记成deprecated，建议用新的Tool call方式，但是目前很多项目用的还是function call的方案，未及时升级，细节参数可以参考文档：https://platform.openai.com/docs/api-reference/chat/create

用户询问问题

如果提供了多个functions，本轮想指定某个function，可以指定function_call参数，如：function_call: {"name": "gaode_weather"}，则模型只负责解析该function需要的参数信息。

curl http://localhost:80/v1/chat/completions \-H "Content-Type: application/json" \-H "Authorization: Bearer $OPENAI_API_KEY" \-d '{"model": "gpt-4","messages": [{"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."},{"role": "user", "content": "今天深圳天气怎么样？"}],"functions": [{"name": "gaode_weather","description": "A tool when you want to ask about the weather or weather-related question.","parameters": {"type": "object","properties": {"city": {"type": "string","description": "If you don't know you can extract the city name from the question or you can reply：Please tell me your city. You have to extract the Chinese city name from the question."}},"required": ["city"]}}]}'

提取模型返回的内容，调用工具获取结果

返回内容

{"id": "chatcmpl-123","object": "chat.completion","created": 1677652288,"model": "gpt-3.5-turbo-0125","system_fingerprint": "fp_44709d6fcb","choices": [{"index": 0,"message": {"role": "assistant","function_call": {"name": "gaode_weather","arguments": "{\n  \"city\": \"深圳\"\n}"}},"logprobs": null,"finish_reason": "function_call"}],"usage": {"prompt_tokens": 9,"completion_tokens": 12,"total_tokens": 21}
}

调用高德天气API，获取返回结果，将工具的调用结果加入到请求的message参数中，并调用模型获取用户问题的结果。

curl http://localhost:80/v1/chat/completions \-H "Content-Type: application/json" \-H "Authorization: Bearer $OPENAI_API_KEY" \-d '{"model": "gpt-4","messages": [{"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."},{"role": "user", "content": "今天深圳天气怎么样？"},{"role": "assistant","function_call": {"name": "gaode_weather","arguments": "{\n  \"city\": \"深圳\"\n}"}},{"role": "function","name": "gaode_weather","content": "[{\"date\": \"2024-06-18\", \"week\": \"2\", \"dayweather\": \"阵雨\", \"daytemp_float\": \"32.0\", \"daywind\": \"北\", \"nightweather\": \"多云\", \"nighttemp_float\": \"27.0\"}, {\"date\": \"2024-06-19\", \"week\": \"3\", \"dayweather\": \"阵雨\", \"daytemp_float\": \"32.0\", \"daywind\": \"北\", \"nightweather\": \"多云\", \"nighttemp_float\": \"27.0\"}, {\"date\": \"2024-06-20\", \"week\": \"4\", \"dayweather\": \"阵雨\", \"daytemp_float\": \"32.0\", \"daywind\": \"南\", \"nightweather\": \"多云\", \"nighttemp_float\": \"28.0\"}, {\"date\": \"2024-06-21\", \"week\": \"5\", \"dayweather\": \"多云\", \"daytemp_float\": \"33.0\", \"daywind\": \"南\", \"nightweather\": \"多云\", \"nighttemp_float\": \"28.0\"}]"}]}'

模型根据工具的调用结果回答用户问题

{"id": "chatcmpl-123","object": "chat.completion","created": 1677652288,"model": "gpt-3.5-turbo-0125","system_fingerprint": "fp_44709d6fcb","choices": [{"index": 0,"message": {"role": "assistant","content": "深圳未来几天的天气情况如下：\n\n2024年6月18日，星期六，天气为阵雨，白天温度为32.0摄氏度，北风，晚上多云，温度为27.0摄氏度；\n\n2024年6月19日，星期日，天气为阵雨，白天温度为32.0摄氏度，北风，晚上多云，温度为27.0摄氏度；\n\n2024年6月20日，星期一，天气为阵雨，白天温度为32.0摄氏度，南风，晚上多云，温度为28.0摄氏度；\n\n2024年6月21日，星期二，天气为多云，白天温度为33.0摄氏度，南风，晚上多云，温度为28.0摄氏度。\n\n请注意天气变化，合理安排行程。"},"logprobs": null,"finish_reason": "stop"}],"usage": {"prompt_tokens": 9,"completion_tokens": 12,"total_tokens": 21}
}

ReAct Prompting

更老的模型，并不具备工具调用的能力，为了实现工具调用，通常是通过ReAct（Reasoning and Acting）Prompting来实现的，它旨在通过将推理（Reasoning）和行动（Acting）相结合来增强大语言模型的能力。这种方法特别适用于需要复杂决策和多步骤处理的任务。

ReAct Prompting通过以下步骤对用户问题生成输出：

推理（Reasoning）：模型首先进行推理，理解用户的请求，并确定需要执行的操作。推理过程可以包括问题分解、步骤规划等。
行动（Acting）：根据推理结果，调用外部工具或API，执行具体操作，如检索信息、执行计算、访问数据库等。
整合结果：模型将外部操作的结果整合到最终输出中，生成用户所需的响应。

通用步骤模板如下，每个模型可能适配的提示词不同，需要结合模型做相应的调整：

用户请求：请告诉我今天的天气。
模型推理：用户想知道今天的天气，需要调用天气API。
模型行动：调用get_weather(location="current location")。
整合结果：根据API返回的结果生成最终输出。

以下是基于Qwen模型使用ReAct Prompting的示例，细节可以参考：https://github.com/QwenLM/Qwen/blob/main/examples/react_prompt.md

将工具，用户问题格式化成prompt，注意需要将Observation:、Observation:\n加入stop token，免得模型自问自答。

Answer the following questions as best you can. You have access to the following tools:gaode_weather: Call this tool to interact with the gaode_weather API. What is the gaode_weather API useful for? A tool when you want to ask about the weather or weather-related question. Parameters: [{"name": "city", "description": "If you don't know you can extract the city name from the question or you can reply：Please tell me your city. You have to extract the Chinese city name from the question.", "required": true, "schema": {"type": "string"}}] Format the arguments as a JSON object.Use the following format:Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [quark_search,image_gen]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input questionBegin!Question: 深圳天气

模型返回：

Thought: 我应该用gaode_weather来查询深圳天气
Action: gaode_weather
Action Input: {"city": "深圳"}

调用工具获取结果，并让模型根据工具的结果继续作答

假设工具的返回内容为

[{"date": "2024-06-18", "week": "2", "dayweather": "阵雨", "daytemp_float": "32.0", "daywind": "北", "nightweather": "多云", "nighttemp_float": "27.0"}, {"date": "2024-06-19", "week": "3", "dayweather": "阵雨", "daytemp_float": "32.0", "daywind": "北", "nightweather": "多云", "nighttemp_float": "27.0"}, {"date": "2024-06-20", "week": "4", "dayweather": "阵雨", "daytemp_float": "32.0", "daywind": "南", "nightweather": "多云", "nighttemp_float": "28.0"}, {"date": "2024-06-21", "week": "5", "dayweather": "多云", "daytemp_float": "33.0", "daywind": "南", "nightweather": "多云", "nighttemp_float": "28.0"}]

接下来，将首次请求模型时的prompt和调用工具的结果拼装成如下prompt：

Answer the following questions as best you can. You have access to the following tools:gaode_weather: Call this tool to interact with the gaode_weather API. What is the gaode_weather API useful for? A tool when you want to ask about the weather or weather-related question. Parameters: [{"name": "city", "description": "If you don't know you can extract the city name from the question or you can reply：Please tell me your city. You have to extract the Chinese city name from the question.", "required": true, "schema": {"type": "string"}}] Format the arguments as a JSON object.Use the following format:Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [quark_search,image_gen]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input questionBegin!Question: 深圳天气
Thought: 我应该用gaode_weather来查询深圳天气
Action: gaode_weather
Action Input: {"city": "深圳"}
Observation: [{"date": "2024-06-18", "week": "2", "dayweather": "阵雨", "daytemp_float": "32.0", "daywind": "北", "nightweather": "多云", "nighttemp_float": "27.0"}, {"date": "2024-06-19", "week": "3", "dayweather": "阵雨", "daytemp_float": "32.0", "daywind": "北", "nightweather": "多云", "nighttemp_float": "27.0"}, {"date": "2024-06-20", "week": "4", "dayweather": "阵雨", "daytemp_float": "32.0", "daywind": "南", "nightweather": "多云", "nighttemp_float": "28.0"}, {"date": "2024-06-21", "week": "5", "dayweather": "多云", "daytemp_float": "33.0", "daywind": "南", "nightweather": "多云", "nighttemp_float": "28.0"}]

用新的prompt去调用模型，模型将返回以下结果:

Thought: 我已经成功使用gaode_weather查询到了深圳天气。
Final Answer: 深圳未来几天的天气情况如下：\n\n2024年6月18日，星期六，天气为阵雨，白天温度为32.0摄氏度，北风，晚上多云，温度为27.0摄氏度；\n\n2024年6月19日，星期日，天气为阵雨，白天温度为32.0摄氏度，北风，晚上多云，温度为27.0摄氏度；\n\n2024年6月20日，星期一，天气为阵雨，白天温度为32.0摄氏度，南风，晚上多云，温度为28.0摄氏度；\n\n2024年6月21日，星期二，天气为多云，白天温度为33.0摄氏度，南风，晚上多云，温度为28.0摄氏度。\n\n请注意天气变化，合理安排行程。