Tool Use

In this example, we'll put together a simple example of using the Inference Grid SDK to implement tool use. Also, check out setting up tool use using the OpenAI Adapter!

Installation

Let's start by installing the Spark SDK and Inference Grid SDK.

npm install @buildonspark/spark-sdk@0.1.16 inference-grid-sdk

Weather Tool

First, we'll set up a weather tool which uses the OpenWeatherMap API to get the weather given a city and state.

type WeatherData = {
  name: string;
  main: {
    temp: number;
    humidity: number;
    pressure: number;
    temp_min: number;
    temp_max: number;
  };
  weather: { description: string }[];
};

const API_KEY = "... YOUR API KEY ...";

async function getWeather(
  city: string,
  state: string
): Promise<WeatherData | null> {
  // Combine city and state for specificity, assuming US locations
  const location = `${city},${state},US`;
  const url = `https://api.openweathermap.org/data/2.5/weather?q=${encodeURIComponent(
    location
  )}&appid=${API_KEY}&units=imperial`;

  try {
    const response = await fetch(url);
    if (!response.ok) {
      throw new Error("City not found or API error");
    }
    const data: WeatherData = await response.json();
    return data;
  } catch (error) {
    console.error(error);
    return null;
  }
}

To pass this tool to the model, we'll define the schema. Check out the official OpenAI documentation to learn more about the syntax.

const WEATHER_TOOL = {
  type: "function",
  function: {
    name: "get_weather",
    description: "Get the weather of a city",
    parameters: JSON.stringify({
      type: "object",
      properties: {
        city: {
          type: "string",
          description: "The city, e.g. Los Angeles",
        },
        state: {
          type: "string",
          description: "The state, e.g. CA",
        },
      },
      required: ["city", "state"],
    }),
  },
};

Basic Agent

Let's give the agent access to this tool and then ask it about the weather in Los Angeles.

const client = new InferenceGrid();

const messages = [
    {
        role: Role.USER,
        content: "What is the weather in Los Angeles, CA?",
    },
]

const { invoice, message } = await client.chat(
  {
    maxTokens: 1000,
    temperature: 0.5,
    model: {
      modelIds: ["openai/gpt-4o"],
      flags: [],
    },
    messages: messages,
    tools: [WEATHER_TOOL],
  }
);

// Use any Lightning implementation to pay the invoice!
sparkWallet.payInvoice(invoice);

This will return a tool call. We'll call the function and then push the results back!

const arguments = JSON.parse(message.toolCalls[0]!.function!.arguments);
const result = getWeather(arguments.city, arguments.state)

// Don't forget to add the assistant response!
messages.push({
    role: Role.ASSISTANT,
    content: result.message,
    tool_calls: result.toolCalls,
});

// Add the tool call result
messages.push({
    role: Role.TOOL,
    tool_call_id: result.toolCalls[0]!.id,
    content: JSON.stringify(result),
});

Finally, now that our agent is equipped with the weather report, we'll ask it to give us the final response:

const { invoice, message } = await client.chat(
  {
    maxTokens: 1000,
    temperature: 0.5,
    model: {
      modelIds: ["openai/gpt-4o"],
      flags: [],
    },
    messages: messages,
    tools: [WEATHER_TOOL],
  }
);

// Use any Lightning implementation to pay the invoice!
sparkWallet.payInvoice(invoice);
alert(message);