Appearance
Tool Use
In this example, we'll put together a simple example of using the Inference Grid SDK to implement tool use. Also, check out setting up tool use using the OpenAI Adapter!
Installation
Let's start by installing the Spark SDK and Inference Grid SDK.
bash
npm install @buildonspark/spark-sdk@0.1.16 inference-grid-sdk
Weather Tool
First, we'll set up a weather tool which uses the OpenWeatherMap API to get the weather given a city and state.
typescript
type WeatherData = {
name: string;
main: {
temp: number;
humidity: number;
pressure: number;
temp_min: number;
temp_max: number;
};
weather: { description: string }[];
};
const API_KEY = "... YOUR API KEY ...";
async function getWeather(
city: string,
state: string
): Promise<WeatherData | null> {
// Combine city and state for specificity, assuming US locations
const location = `${city},${state},US`;
const url = `https://api.openweathermap.org/data/2.5/weather?q=${encodeURIComponent(
location
)}&appid=${API_KEY}&units=imperial`;
try {
const response = await fetch(url);
if (!response.ok) {
throw new Error("City not found or API error");
}
const data: WeatherData = await response.json();
return data;
} catch (error) {
console.error(error);
return null;
}
}
To pass this tool to the model, we'll define the schema. Check out the official OpenAI documentation to learn more about the syntax.
typescript
const WEATHER_TOOL = {
type: "function",
function: {
name: "get_weather",
description: "Get the weather of a city",
parameters: JSON.stringify({
type: "object",
properties: {
city: {
type: "string",
description: "The city, e.g. Los Angeles",
},
state: {
type: "string",
description: "The state, e.g. CA",
},
},
required: ["city", "state"],
}),
},
};
Basic Agent
Let's give the agent access to this tool and then ask it about the weather in Los Angeles.
typescript
const client = new InferenceGrid();
const messages = [
{
role: Role.USER,
content: "What is the weather in Los Angeles, CA?",
},
]
const { invoice, message } = await client.chat(
{
maxTokens: 1000,
temperature: 0.5,
model: {
modelIds: ["openai/gpt-4o"],
flags: [],
},
messages: messages,
tools: [WEATHER_TOOL],
}
);
// Use any Lightning implementation to pay the invoice!
sparkWallet.payInvoice(invoice);
This will return a tool call. We'll call the function and then push the results back!
typescript
const arguments = JSON.parse(message.toolCalls[0]!.function!.arguments);
const result = getWeather(arguments.city, arguments.state)
// Don't forget to add the assistant response!
messages.push({
role: Role.ASSISTANT,
content: result.message,
tool_calls: result.toolCalls,
});
// Add the tool call result
messages.push({
role: Role.TOOL,
tool_call_id: result.toolCalls[0]!.id,
content: JSON.stringify(result),
});
Finally, now that our agent is equipped with the weather report, we'll ask it to give us the final response:
typescript
const { invoice, message } = await client.chat(
{
maxTokens: 1000,
temperature: 0.5,
model: {
modelIds: ["openai/gpt-4o"],
flags: [],
},
messages: messages,
tools: [WEATHER_TOOL],
}
);
// Use any Lightning implementation to pay the invoice!
sparkWallet.payInvoice(invoice);
alert(message);