Spaces:
Running
Running
Not all of the tool-calling features shown above are used by all models. Some use tool call IDs, others simply use the function name and | |
match tool calls to results using the ordering, and there are several models that use neither and only issue one tool | |
call at a time to avoid confusion. If you want your code to be compatible across as many models as possible, we | |
recommend structuring your tools calls like we've shown here, and returning tool results in the order that | |
they were issued by the model. The chat templates on each model should handle the rest. | |
Understanding tool schemas | |
Each function you pass to the tools argument of apply_chat_template is converted into a | |
JSON schema. These schemas | |
are then passed to the model chat template. In other words, tool-use models do not see your functions directly, and they | |
never see the actual code inside them. What they care about is the function definitions and the arguments they | |
need to pass to them - they care about what the tools do and how to use them, not how they work! It is up to you | |
to read their outputs, detect if they have requested to use a tool, pass their arguments to the tool function, and | |
return the response in the chat. | |
Generating JSON schemas to pass to the template should be automatic and invisible as long as your functions | |
follow the specification above, but if you encounter problems, or you simply want more control over the conversion, | |
you can handle the conversion manually. Here is an example of a manual schema conversion. | |
thon | |
from transformers.utils import get_json_schema | |
def multiply(a: float, b: float): | |
""" | |
A function that multiplies two numbers | |
Args: | |
a: The first number to multiply | |
b: The second number to multiply | |
""" | |
return a * b | |
schema = get_json_schema(multiply) | |
print(schema) | |
This will yield: | |
json | |
{ | |
"type": "function", | |
"function": { | |
"name": "multiply", | |
"description": "A function that multiplies two numbers", | |
"parameters": { | |
"type": "object", | |
"properties": { | |
"a": { | |
"type": "number", | |
"description": "The first number to multiply" | |
}, | |
"b": { | |
"type": "number", | |
"description": "The second number to multiply" | |
} | |
}, | |
"required": ["a", "b"] | |
} | |
} | |
} | |
If you wish, you can edit these schemas, or even write them from scratch yourself without using get_json_schema at | |
all. JSON schemas can be passed directly to the tools argument of | |
apply_chat_template - this gives you a lot of power to define precise schemas for more complex functions. Be careful, | |
though - the more complex your schemas, the more likely the model is to get confused when dealing with them! We | |
recommend simple function signatures where possible, keeping arguments (and especially complex, nested arguments) | |
to a minimum. | |
Here is an example of defining schemas by hand, and passing them directly to apply_chat_template: | |
thon | |
A simple function that takes no arguments | |
current_time = { | |
"type": "function", | |
"function": { | |
"name": "current_time", | |
"description": "Get the current local time as a string.", | |
"parameters": { | |
'type': 'object', | |
'properties': {} | |
} | |
} | |
} | |
A more complete function that takes two numerical arguments | |
multiply = { | |
'type': 'function', | |
'function': { | |
'name': 'multiply', | |
'description': 'A function that multiplies two numbers', | |
'parameters': { | |
'type': 'object', | |
'properties': { | |
'a': { | |
'type': 'number', | |
'description': 'The first number to multiply' | |
}, | |
'b': { | |
'type': 'number', 'description': 'The second number to multiply' | |
} | |
}, | |
'required': ['a', 'b'] | |
} | |
} | |
} | |
model_input = tokenizer.apply_chat_template( | |
messages, | |
tools = [current_time, multiply] | |
) |