openai_text-embedding-ada-002/fixed_chunks/_chat_templating.txt_chunk_3.txt

From here, just continue training like you would with a standard language modelling task, using the formatted_chat column.
Advanced: Extra inputs to chat templates
The only argument that apply_chat_template requires is messages. However, you can pass any keyword
argument to apply_chat_template and it will be accessible inside the template. This gives you a lot of freedom to use
chat templates for many things. There are no restrictions on the names or the format of these arguments - you can pass
strings, lists, dicts or whatever else you want. 
That said, there are some common use-cases for these extra arguments,
such as passing tools for function calling, or documents for retrieval-augmented generation. In these common cases,
we have some opinionated recommendations about what the names and formats of these arguments should be, which are
described in the sections below. We encourage model authors to make their chat templates compatible with this format,
to make it easy to transfer tool-calling code between models.
Advanced: Tool use / function calling
"Tool use" LLMs can choose to call functions as external tools before generating an answer. When passing tools
to a tool-use model, you can simply pass a list of functions to the tools argument:
thon
import datetime
def current_time():
    """Get the current local time as a string."""
    return str(datetime.now())
def multiply(a: float, b: float):
    """
    A function that multiplies two numbers
Args:
    a: The first number to multiply
    b: The second number to multiply
"""
return a * b

tools = [current_time, multiply]
model_input = tokenizer.apply_chat_template(
    messages,
    tools=tools
)

In order for this to work correctly, you should write your functions in the format above, so that they can be parsed
correctly as tools. Specifically, you should follow these rules:

The function should have a descriptive name
Every argument must have a type hint
The function must have a docstring in the standard Google style (in other words, an initial function description
  followed by an Args: block that describes the arguments, unless the function does not have any arguments. 
Do not include types in the Args: block. In other words, write a: The first number to multiply, not
  a (int): The first number to multiply. Type hints should go in the function header instead.
The function can have a return type and a Returns: block in the docstring. However, these are optional
  because most tool-use models ignore them.

Passing tool results to the model
The sample code above is enough to list the available tools for your model, but what happens if it wants to actually use
one? If that happens, you should:

Parse the model's output to get the tool name(s) and arguments.
Add the model's tool call(s) to the conversation.
Call the corresponding function(s) with those arguments.
Add the result(s) to the conversation

A complete tool use example
Let's walk through a tool use example, step by step. For this example, we will use an 8B Hermes-2-Pro model,
as it is one of the highest-performing tool-use models in its size category at the time of writing. If you have the
memory, you can consider using a larger model instead like Command-R
or Mixtral-8x22B, both of which also support tool use
and offer even stronger performance.
First, let's load our model and tokenizer:
thon
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "NousResearch/Hermes-2-Pro-Llama-3-8B"
tokenizer = AutoTokenizer.from_pretrained(checkpoint, revision="pr/13")
model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, device_map="auto")