本笔记本介绍了如何进行微调以提高函数调用的准确性和可靠性。您可以在此处找到有关函数调用和微调的更多信息
tools 是 Chat Completion API 中的可选参数，可用于提供功能规范。 这样做的目的是使模型能够生成符合所提供规范的函数参数。 请注意，API 实际上不会执行任何函数调用。 开发人员可以使用模型输出执行函数调用。
当函数调用按预期运行时，它是一个非常强大的工具。 然而，我们已经看到，随着函数数量的
增加，手头任务的复杂性增加，函数调用变得不太准确（例如：更多的调用幻觉和不正确的调用）。
在微调函数调用之前，最好从以下开始：
- 函数定义的改进。 让它们更加清晰，彼此更加清晰。
- 尝试提示工程：通常更详细的提示可以帮助模型调用正确的函数。
如果上述步骤未能将函数调用提高到令人满意的水平，那么您可以尝试对函数调用进行微调。
概述
本笔记本包含三个部分
- 评估基线函数调用性能：在给定函数上评估开箱即用的 gpt-3.5-turbo 模型（假设由于延迟和成本原因，我们不会使用 gpt-4 用于自动化）
- 生成合成数据：使用 gpt-4 创建“黄金”提示和函数调用集以用作训练数据
- 微调：运行微调作业，并评估微调后的模型
注意：本文提供了一个示例，说明如何创建合成训练数据，以便仅在给定函数列表的情况下对函数调用进行微调。 虽然现实世界的生产测试评估更可取，但这种方法可以产生强大的结果，并且可以与现实世界的训练数据结合使用。
获取当前函数式调用的基准性能
# !pip install tenacity
# !pip install openai
# !pip install typingimport numpy as np
import json
import os
from openai import OpenAI
import itertools
from tenacity import retry, wait_random_exponential, stop_after_attempt
from typing import Any, Dict, List, Generator
import ast
client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY", "<your OpenAI API key if not set as env var>"))实用工具
让我们定义一些实用函数来调用聊天完成 API，一个用于获取完成，一个用于获取函数调用。
def get_chat_completion(
messages: list[dict[str, str]],
model: str = "gpt-3.5-turbo",
max_tokens=500,
temperature=1.0,
stop=None,
tools=None,
functions=None
) -> str:
params = {
'model': model,
'messages': messages,
'max_tokens': max_tokens,
'temperature': temperature,
'stop': stop,
'tools': tools,
}
if functions:
params['functions'] = functions
completion = client.chat.completions.create(**params)
return completion.choices[0].message
基线测试
让我们打造一个智能自动化代理。 我们希望能够向代理发出命令，并让它调用该命令的函数，或者在该命令不可行时拒绝该请求。 我们可以先为代理定义一个系统提示。
DRONE_SYSTEM_PROMPT = """You are an intelligent AI that controls a drone. Given a command or request from the user,
call one of your functions to complete the request. If the request cannot be completed by your available functions, call the reject_request function.
If the request is ambiguous or unclear, reject the request."""现在让我们为代理可以执行的所有操作定义函数。
function_list = [
{
"name": "takeoff_drone",
"description": "Initiate the drone's takeoff sequence.",
"parameters": {
"type": "object",
"properties": {
"altitude": {
"type": "integer",
"description": "Specifies the altitude in meters to which the drone should ascend."
}
},
"required": ["altitude"]
}
},
{
"name": "land_drone",
"description": "Land the drone at its current location or a specified landing point.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"enum": ["current", "home_base", "custom"],
"description": "Specifies the landing location for the drone."
},
"coordinates": {
"type": "object",
"description": "GPS coordinates for custom landing location. Required if location is 'custom'."
}
},
"required": ["location"]
}
},
{
"name": "control_drone_movement",
"description": "Direct the drone's movement in a specific direction.",
"parameters": {
"type": "object",
"properties": {
"direction": {
"type": "string",
"enum": ["forward", "backward", "left", "right", "up", "down"],
"description": "Direction in which the drone should move."
},
"distance": {
"type": "integer",
"description": "Distance in meters the drone should travel in the specified direction."
}
},
"required": ["direction", "distance"]
}
},
{
"name": "set_drone_speed",
"description": "Adjust the speed of the drone.",
"parameters": {
"type": "object",
"properties": {
"speed": {
"type": "integer",
"description": "Specifies the speed in km/h."
}
},
"required": ["speed"]
}
},
{
"name": "control_camera",
"description": "Control the drone's camera to capture images or videos.",
"parameters": {
"type": "object",
"properties": {
"mode": {
"type": "string",
"enum": ["photo", "video", "panorama"],
"description": "Camera mode to capture content."
},
"duration": {
"type": "integer",
"description": "Duration in seconds for video capture. Required if mode is 'video'."
}
},
"required": ["mode"]
}
},
{
"name": "control_gimbal",
"description": "Adjust the drone's gimbal for camera stabilization and direction.",
"parameters": {
"type": "object",
"properties": {
"tilt": {
"type": "integer",
"description": "Tilt angle for the gimbal in degrees."
},
"pan": {
"type": "integer",
"description": "Pan angle for the gimbal in degrees."
}
},
"required": ["tilt", "pan"]
}
},
{
"name": "set_drone_lighting",
"description": "Control the drone's lighting for visibility and signaling.",
"parameters": {
"type": "object",
"properties": {
"mode": {
"type": "string",
"enum": ["on", "off", "blink", "sos"],
"description": "Lighting mode for the drone."
}
},
"required": ["mode"]
}
},
{
"name": "return_to_home",
"description": "Command the drone to return to its home or launch location.",
"parameters": {
"type": "object",
"properties": {}
}
},
{
"name": "set_battery_saver_mode",
"description": "Toggle battery saver mode.",
"parameters": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": ["on", "off"],
"description": "Toggle battery saver mode."
}
},
"required": ["status"]
}
},
{
"name": "set_obstacle_avoidance",
"description": "Configure obstacle avoidance settings.",
"parameters": {
"type": "object",
"properties": {
"mode": {
"type": "string",
"enum": ["on", "off"],
"description": "Toggle obstacle avoidance."
}
},
"required": ["mode"]
}
},
{
"name": "set_follow_me_mode",
"description": "Enable or disable 'follow me' mode.",
"parameters": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": ["on", "off"],
"description": "Toggle 'follow me' mode."
}
},
"required": ["status"]
}
},
{
"name": "calibrate_sensors",
"description": "Initiate calibration sequence for drone's sensors.",
"parameters": {
"type": "object",
"properties": {}
}
},
{
"name": "set_autopilot",
"description": "Enable or disable autopilot mode.",
"parameters": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": ["on", "off"],
"description": "Toggle autopilot mode."
}
},
"required": ["status"]
}
},
{
"name": "configure_led_display",
"description": "Configure the drone's LED display pattern and colors.",
"parameters": {
"type": "object",
"properties": {
"pattern": {
"type": "string",
"enum": ["solid", "blink", "pulse", "rainbow"],
"description": "Pattern for the LED display."
},
"color": {
"type": "string",
"enum": ["red", "blue", "green", "yellow", "white"],
"description": "Color for the LED display. Not required if pattern is 'rainbow'."
}
},
"required": ["pattern"]
}
},
{
"name": "set_home_location",
"description": "Set or change the home location for the drone.",
"parameters": {
"type": "object",
"properties": {
"coordinates": {
"type": "object",
"description": "GPS coordinates for the home location."
}
},
"required": ["coordinates"]
}
},
{
"name": "reject_request",
"description": "Use this function if the request is not possible.",
"parameters": {
"type": "object",
"properties": {}
}
},
]
首先，让我们看看函数调用如何通过一些直接可行的提示词进行函数执行，然后是对于明显不可能的请求调用“reject_request”函数。
straightforward_prompts = ['Land the drone at the home base',
'Take off the drone to 50 meters',
'change speed to 15 kilometers per hour',
'turn into an elephant!']
for prompt in straightforward_prompts:
messages = []
messages.append({"role": "system", "content": DRONE_SYSTEM_PROMPT})
messages.append({"role": "user", "content": prompt})
completion = get_chat_completion(model="gpt-3.5-turbo",messages=messages,tools=function_list)
print(prompt)
print(completion.function_call,'\n')
输出
Land the drone at the home base
FunctionCall(arguments='{\n "location": "home_base"\n}', name='land_drone')
Take off the drone to 50 meters
FunctionCall(arguments='{\n "altitude": 50\n}', name='takeoff_drone')
change speed to 15 kilometers per hour
FunctionCall(arguments='{ "speed": 15 }', name='set_drone_speed')
turn into an elephant!
FunctionCall(arguments='{}', name='reject_request')
好的！ 该模型对于这些请求表现得很好。 现在让我们尝试一些更困难的请求：几乎可行且与无人机相关的请求，但无人机实际上无法执行，agent应该拒绝。
challenging_prompts = ['Play pre-recorded audio message',
'Initiate live-streaming on social media',
'Scan environment for heat signatures',
'Enable stealth mode',
"Change drone's paint job color"]
for prompt in challenging_prompts:
messages = []
messages.append({"role": "system", "content": DRONE_SYSTEM_PROMPT})
messages.append({"role": "user", "content": prompt})
completion = get_chat_completion(model="gpt-3.5-turbo",messages=messages,tools=function_list)
print(prompt)
try:
print(completion.function_call)
print('\n')
except:
print(completion.content)
print('\n')
输出
Play pre-recorded audio message
FunctionCall(arguments='{}', name='reject_request')
Initiate live-streaming on social media
FunctionCall(arguments='{\n "mode": "video",\n "duration": 0\n}', name='control_camera')
Scan environment for heat signatures
FunctionCall(arguments='{\n "mode": "photo"\n}', name='control_camera')
Enable stealth mode
FunctionCall(arguments='{\n "mode": "off"\n}', name='set_drone_lighting')
Change drone's paint job color
FunctionCall(arguments='{\n "pattern": "solid",\n "color": "blue"\n}', name='configure_led_display')
现在我们遇到了一些问题。 这里的模型应该拒绝所有这些请求，因为给定函数它们是不可能的，但是模型调用与请求有些相关但不正确的函数。 当被要求开始“实时流媒体到社交媒体”时，模型将摄像头设置为视频，当被要求“更改油漆颜色”时，将 LED 更改为蓝色……在这个简单的情况下，更及时的工程可能会解决其中一些问题 问题，但出于本示例的目的，我们将演示如何使用微调来提高性能。 此外，虽然这种情况相对简单，但随着功能数量和复杂性的增加，微调变得越来越有影响力。
生成合成数据
工具函数
我们希望生成每个函数的每次调用，以便我们完全覆盖所有潜在的调用来为其创建合成数据。 然后，我们将使用 gpt-4 提出调用每个调用的提示，并且我们将使用该提示 – 函数调用对作为训练数据。
为具有固定枚举的函数生成每个调用更加简单，但对于像 control_gimbal 这样的函数，我们需要设置倾斜和平移整数值，因此要生成这些合成调用，我们将首先设置一个占位符，然后使用 gpt- 4 得出合理的数值。
placeholder_int = 'fill_in_int'
placeholder_string = 'fill_in_string'
下面的函数接受函数列表中的所有函数，并在给定每个函数的参数的情况下查看这些函数的所有潜在调用。 这些函数还考虑了所需的参数，因此所有调用实际上都是可行的。
def generate_permutations(params: Dict[str, Dict[str, Any]]) -> Generator[Dict[str, Any], None, None]:
"""
Generates all possible permutations for given parameters.
:param params: Parameter dictionary containing required and optional fields.
:return: A generator yielding each permutation.
"""
# Extract the required fields from the parameters
required_fields = params.get('required', [])
# Generate permutations for required fields
required_permutations = generate_required_permutations(params, required_fields)
# Generate optional permutations based on each required permutation
for required_perm in required_permutations:
yield from generate_optional_permutations(params, required_perm)
def generate_required_permutations(params: Dict[str, Dict[str, Any]], required_fields: List[str]) -> List[Dict[str, Any]]:
"""
Generates permutations for the required fields.
:param params: Parameter dictionary.
:param required_fields: List of required fields.
:return: A list of permutations for required fields.
"""
# Get all possible values for each required field
required_values = [get_possible_values(params, field) for field in required_fields]
# Generate permutations from possible values
return [dict(zip(required_fields, values)) for values in itertools.product(*required_values)]
def generate_optional_permutations(params: Dict[str, Dict[str, Any]], base_perm: Dict[str, Any]) -> Generator[Dict[str, Any], None, None]:
"""
Generates permutations for optional fields based on a base permutation.
:param params: Parameter dictionary.
:param base_perm: Base permutation dictionary.
:return: A generator yielding each permutation for optional fields.
"""
# Determine the fields that are optional by subtracting the base permutation's fields from all properties
optional_fields = set(params['properties']) - set(base_perm)
# Iterate through all combinations of optional fields
for field_subset in itertools.chain.from_iterable(itertools.combinations(optional_fields, r) for r in range(len(optional_fields) + 1)):
# Generate product of possible values for the current subset of fields
for values in itertools.product(*(get_possible_values(params, field) for field in field_subset)):
# Create a new permutation by combining base permutation and current field values
new_perm = {**base_perm, **dict(zip(field_subset, values))}
yield new_perm
def get_possible_values(params: Dict[str, Dict[str, Any]], field: str) -> List[Any]:
"""
Retrieves possible values for a given field.
:param params: Parameter dictionary.
:param field: The field for which to get possible values.
:return: A list of possible values.
"""
# Extract field information from the parameters
field_info = params['properties'][field]
# Based on the field's type or presence of 'enum', determine and return the possible values
if 'enum' in field_info:
return field_info['enum']
elif field_info['type'] == 'integer':
return [placeholder_int]
elif field_info['type'] == 'string':
return [placeholder_string]
elif field_info['type'] == 'boolean':
return [True, False]
elif field_info['type'] == 'array' and 'enum' in field_info['items']:
enum_values = field_info['items']['enum']
all_combinations = [list(combo) for i in range(1, len(enum_values) + 1) for combo in itertools.combinations(enum_values, i)]
return all_combinations
return []
让我们先生成每个功能的所有调用
提示词：
INVOCATION_FILLER_PROMPT = """
1) Input reasonable values for 'fill_in_string' and 'fill_in_int' in the invocation here: {invocation}. Reasonable values are determined by the function definition. Use the
the entire function provided here :{function} to get context over what proper fill_in_string and fill_in_int values would be.
Example:
Input: invocation: {{
"name": "control_camera",
"arguments": {{
"mode":"video",
"duration":"fill_in_int"
}}
}},
function:{function}
Output: invocation: {{
"name": "control_camera",
"arguments": {{
"mode":"video",
"duration": 30
}}
}}
MAKE SURE output is just a dictionary with keys 'name' and 'arguments', no other text or response.
Input: {invocation}
Output:
"""
COMMAND_GENERATION_PROMPT= """
You are to output 2 commands, questions or statements that would generate the inputted function and parameters.
Please make the commands or questions natural, as a person would ask, and the command or questions should be varied and not repetitive.
It should not always mirror the exact technical terminology used in the function and parameters, rather reflect a conversational and intuitive request.
For instance, the prompt should not be 'turn on the dome light', as that is too technical, but rather 'turn on the inside lights'.
Another example, is the prompt should not be 'turn on the HVAC', but rather 'turn on the air conditioning'. Use language a normal driver would use, even if
it is technically incorrect but colloquially used.
RULES: ALWAYS put a backwards slash before an apostrophe or single quote '. For example, do not say don't but say don\'t.
Prompts MUST be in double quotes as well.
Example
Input: {{'name': 'calibrate_sensors','arguments': {{}}'' }}
Prompt: ["The sensors are out of whack, can you reset them", "The calibration of the drone is off, fix it please!"]
Input: {{'name': 'set_autopilot','arguments': {{'status': 'off'}}}}
Prompt: ["OK, I want to take back pilot control now","Turn off the automatic pilot I'm ready control it"]
Input: {invocation}
Prompt:
"""
在下面的代码片段中，我们生成除rejection_request函数之外的每个函数的调用。
为了进行有效的微调，我们需要正确标记的数据。 我们可以手动提出示例并标记数据，
或者我们可以在 gpt-4 的帮助下生成合成数据 根据经验，gpt-4 需要更多的帮助才能获得生成reject_request 函数的提示的真实示例，所以我们接下来会这样做……
input_objects = []
all_but_reject = [f for f in function_list if f.get('name') != 'reject_request']
for function in all_but_reject:
func_name = function["name"]
params = function["parameters"]
for arguments in generate_permutations(params):
if any(val in arguments.values() for val in ['fill_in_int', 'fill_in_str']):
input_object = {
"name": func_name,
"arguments": arguments
}
messages = [{"role": "user", "content": INVOCATION_FILLER_PROMPT.format(invocation=input_object,function=function)}]
input_object = get_chat_completion(model='gpt-4', messages=messages, max_tokens = 200, temperature=.1).content
else:
input_object = {
"name": func_name,
"arguments": arguments
}
input_objects.append(input_object)
现在我们已经有了所有调用，让我们使用 gpt-4 生成会导致这些调用的提示
def create_commands(invocation_list):
example_list = []
for i, invocation in enumerate(invocation_list):
if i<10:
print(f'\033[34m{np.round(100*i/len(invocation_list),1)}% complete\033[0m')
print(invocation)
# Format the prompt with the invocation string
request_prompt = COMMAND_GENERATION_PROMPT.format(invocation=invocation)
messages = [{"role": "user", "content": f"{request_prompt}"}]
completion = get_chat_completion(messages,temperature=0.8)
command_dict = {
"Input": invocation,
"Prompt": completion
}
example_list.append(command_dict)
return example_list
#Only printing the first 10 rows
training_examples_unformatted = create_commands(input_objects)
输出：
[34m0.0% complete[0m
{'name': 'takeoff_drone', 'arguments': {'altitude': 100}}
[34m1.8% complete[0m
{'name': 'land_drone', 'arguments': {'location': 'current'}}
[34m3.6% complete[0m
{'name': 'land_drone', 'arguments': {'location': 'home_base'}}
[34m5.4% complete[0m
{'name': 'land_drone', 'arguments': {'location': 'custom'}}
[34m7.1% complete[0m
{'name': 'control_drone_movement', 'arguments': {'direction': 'forward', 'distance': 50}}
[34m8.9% complete[0m
{'name': 'control_drone_movement', 'arguments': {'direction': 'backward', 'distance': 10}}
[34m10.7% complete[0m
{'name': 'control_drone_movement', 'arguments': {'direction': 'left', 'distance': 10}}
[34m12.5% complete[0m
{'name': 'control_drone_movement', 'arguments': {'direction': 'right', 'distance': 10}}
[34m14.3% complete[0m
{'name': 'control_drone_movement', 'arguments': {'direction': 'up', 'distance': 20}}
[34m16.1% complete[0m
{'name': 'control_drone_movement', 'arguments': {'direction': 'down', 'distance': 10}}现在让我们正确格式化训练示例。 有关用于微调函数调用的正确训练数据格式的更多文档，请参阅此处：https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples
training_examples = []
for prompt in training_examples_unformatted:
#adjust formatting for training data specs
#if its not a dict, convert to dict
if type(prompt['Input'])!=dict:
prompt['Input'] = ast.literal_eval(prompt['Input'])
prompt['Input']['arguments']=json.dumps(prompt['Input']['arguments'])
for p in ast.literal_eval(prompt['Prompt'].content):
training_examples.append({"messages": [{"role":"system","content":DRONE_SYSTEM_PROMPT
},{"role":"user","content": p},
{"role":"assistant","function_call": prompt['Input']}],
"functions":function_list})
现在，回到rejection function。 让我们生成一些几乎可能的提示，但应该会导致拒绝rejection function被调用。 为此，我们查询了 gpt-4，询问与给定函数列表相关但不太可能的请求。
reject_list = ['Translate broadcast message to another language',
'Automatically capture photos when face is detected',
'Detect nearby drones',
'Measure wind resistance',
'Capture slow motion video',
"Adjust drone's altitude to ground level changes",
'Display custom message on LED display',
"Sync drone's time with smartphone",
'Alert when drone travels out of designated area',
'Detect moisture levels',
'Automatically follow GPS tagged object',
'Toggle night vision mode',
'Maintain current altitude when battery is low',
'Decide best landing spot using AI',
"Program drone's route based on wind direction"]
reject_training_list = []
for prompt in reject_list:
#Adjust formatting
reject_training_list.append({"messages": [{"role":"system","content":DRONE_SYSTEM_PROMPT
},{"role":"user","content": prompt},
{"role":"assistant","function_call": {"name": "reject_request","arguments": "{}"}}],
"functions":function_list})
现在将所有训练示例组合在一起
training_list_total = training_examples+reject_training_list
training_file = 'data/drone_training.jsonl'
with open(training_file, 'w') as f:
for item in training_list_total:
json_str = json.dumps(item)
f.write(f'{json_str}\n')
微调
下面开始我们的微调任务
if __name__ == "__main__":
file = client.files.create(
file=open(training_file, "rb"),
purpose="fine-tune",
)
file_id = file.id
print(file_id)
ft = client.fine_tuning.jobs.create(
model="gpt-3.5-turbo",
training_file=file_id,
)
file-xrLV8EbNZk31QPT1TB5KIx7E结果评估
非常好！ 我们训练了一个用于函数调用的微调模型。 让我们看看它在我们的评估集上对Agent应自动拒绝的提示的表现如何。
for eval_question in challenging_prompts:
messages = []
messages.append({"role": "system", "content": DRONE_SYSTEM_PROMPT})
messages.append({"role": "user", "content": eval_question})
completion = get_chat_completion(model="ft:gpt-3.5-turbo-0613:openai-internal::8DloQKS2",messages=messages,tools=function_list)
print(eval_question)
print(completion.function_call,'\n')
Play pre-recorded audio message
FunctionCall(arguments='{}', name='reject_request')
Initiate live-streaming on social media
FunctionCall(arguments='{}', name='reject_request')
Scan environment for heat signatures
FunctionCall(arguments='{}', name='reject_request')
Enable stealth mode
FunctionCall(arguments='{}', name='reject_request')
Change drone's paint job color
FunctionCall(arguments='{}', name='reject_request')
伟大的！ 虽然原始模型仅拒绝了 5 个请求中的 1 个，但微调后的模型拒绝了所有 5 个请求。
结论
恭喜！ 您现在已准备好微调函数调用的模型。 我们迫不及待地想看看您构建了什么。
原文：
https://cookbook.openai.com/examples/fine_tuning_for_function_calling