Files

T

wehub-resource-sync c275ba2868 chore: import upstream snapshot with attribution

2026-07-13 13:31:35 +08:00

44 KiB

Raw Permalink Blame History

Creating a client wit LLM

So far, you don see how to create server and client. Di client fit call di server directly to list dia tools, resources, and prompts. Bot, dis no too practical. Your users dey live for di agentic time and dem dey expect sey dem fit use prompts and talk wit LLM. Dem no care if you use MCP store your capabilities; dem just want interact using normal language. So how we go solve dis? Di solution na to add LLM to di client.

Overview

For dis lesson, we go focus on to add LLM to your client and show how e go give better experience to your user.

Learning Objectives

By di end of dis lesson, you go fit:

Create client wit LLM.
Interact easy wit MCP server using LLM.
Provide better end user experience for client side.

Approach

Make we try understand di approach we need take. Adding LLM sound easy, but we really go do am?

Na so di client go take interact wit di server:

Establish connection wit server.
List capabilities, prompts, resources and tools, and save dia schema.
Add LLM and pass di saved capabilities and dia schema for format wey LLM fit understand.
Handle user prompt by passing am to LLM together wit di tools dat di client list.

Great, now we don understand how we fit do am for high level, make we try am for below exercise.

Exercise: Creating a client wit LLM

For dis exercise, we go learn how to add LLM to our client.

Authentication using GitHub Personal Access Token

Creating GitHub token na simple process. Here how you fit do am:

Go GitHub Settings – Click your profile picture for top right corner and select Settings.
Navigate to Developer Settings – Scroll down and click Developer Settings.
Select Personal Access Tokens – Click Fine-grained tokens then Generate new token.
Configure Your Token – Add note for reference, set expiration date, and select scopes (permissions). For dis case, make sure you add Models permission.
Generate and Copy the Token – Click Generate token, then copy am quick quick, because you no go fit see am again.

-1- Connect to server

Make we create our client first:

TypeScript

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
import OpenAI from "openai";
import { z } from "zod"; // Bring zod come do schema validation

class MCPClient {
    private openai: OpenAI;
    private client: Client;
    constructor(){
        this.openai = new OpenAI({
            baseURL: "https://models.inference.ai.azure.com", 
            apiKey: process.env.GITHUB_TOKEN,
        });

        this.client = new Client(
            {
                name: "example-client",
                version: "1.0.0"
            },
            {
                capabilities: {
                prompts: {},
                resources: {},
                tools: {}
                }
            }
            );    
    }
}

For di code we write before:

We import di libraries we need
Create class wit two members, client and openai wey go help us manage client and interact with LLM respectively.
Configure our LLM instance to use GitHub Models by setting baseUrl point to di inference API.

Python

from mcp import ClientSession, StdioServerParameters, types
from mcp.client.stdio import stdio_client

# Make server parameters for stdio connekshun
server_params = StdioServerParameters(
    command="mcp",  # Di program wey fit run
    args=["run", "server.py"],  # Command line arguments wey no mandatory
    env=None,  # Environment variables wey no mandatory
)


async def run():
    async with stdio_client(server_params) as (read, write):
        async with ClientSession(
            read, write
        ) as session:
            # Start di connection
            await session.initialize()


if __name__ == "__main__":
    import asyncio

    asyncio.run(run())

For di previous code we:

Import di libraries we need for MCP
Create client

.NET

using Azure;
using Azure.AI.Inference;
using Azure.Identity;
using System.Text.Json;
using ModelContextProtocol.Client;
using System.Text.Json;

var clientTransport = new StdioClientTransport(new()
{
    Name = "Demo Server",
    Command = "/workspaces/mcp-for-beginners/03-GettingStarted/02-client/solution/server/bin/Debug/net8.0/server",
    Arguments = [],
});

await using var mcpClient = await McpClient.CreateAsync(clientTransport);

Java

First, you need add LangChain4j dependencies for your pom.xml file. Add dis dependencies to enable MCP integration and GitHub Models support:

<properties>
    <langchain4j.version>1.0.0-beta3</langchain4j.version>
</properties>

<dependencies>
    <!-- LangChain4j MCP Integration -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-mcp</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    
    <!-- OpenAI Official API Client -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-open-ai-official</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    
    <!-- GitHub Models Support -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-github-models</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    
    <!-- Spring Boot Starter (optional, for production apps) -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
</dependencies>

Then create your Java client class:

import dev.langchain4j.mcp.McpToolProvider;
import dev.langchain4j.mcp.client.DefaultMcpClient;
import dev.langchain4j.mcp.client.McpClient;
import dev.langchain4j.mcp.client.transport.McpTransport;
import dev.langchain4j.mcp.client.transport.http.HttpMcpTransport;
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.openaiofficial.OpenAiOfficialChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.tool.ToolProvider;

import java.time.Duration;
import java.util.List;

public class LangChain4jClient {
    
    public static void main(String[] args) throws Exception {        // Set up di LLM make e use GitHub Models
        ChatLanguageModel model = OpenAiOfficialChatModel.builder()
                .isGitHubModels(true)
                .apiKey(System.getenv("GITHUB_TOKEN"))
                .timeout(Duration.ofSeconds(60))
                .modelName("gpt-4.1-nano")
                .build();

        // Make MCP transport to connect to di server
        McpTransport transport = new HttpMcpTransport.Builder()
                .sseUrl("http://localhost:8080/sse")
                .timeout(Duration.ofSeconds(60))
                .logRequests(true)
                .logResponses(true)
                .build();

        // Make MCP client
        McpClient mcpClient = new DefaultMcpClient.Builder()
                .transport(transport)
                .build();
    }
}

For di code we previously write:

Add LangChain4j dependencies: Needed for MCP integration, OpenAI official client and GitHub Models support
Import LangChain4j libraries: For MCP integration and OpenAI chat model function
Create ChatLanguageModel: Configured to use GitHub Models wit your GitHub token
Setup HTTP transport: Using Server-Sent Events (SSE) connect to MCP server
Create MCP client: To handle communication wit server
Use LangChain4j built-in MCP support: To simplify integration between LLMs and MCP servers

Rust

Dis example assume you get Rust based MCP server running. If you no get am, refer back to 01-first-server lesson to create di server.

Once you get your Rust MCP server, open terminal and go di same directory as di server. Then run dis command to create new LLM client project:

mkdir calculator-llmclient
cd calculator-llmclient
cargo init

Add dis dependencies to your Cargo.toml file:

[dependencies]
async-openai = { version = "0.29.0", features = ["byot"] }
rmcp = { version = "0.5.0", features = ["client", "transport-child-process"] }
serde_json = "1.0.141"
tokio = { version = "1.46.1", features = ["rt-multi-thread"] }

Note

No get official Rust library for OpenAI, bot async-openai crate na community maintained library wey people dey use.

Open src/main.rs file and replace di content wit dis code:

use async_openai::{Client, config::OpenAIConfig};
use rmcp::{
    RmcpError,
    model::{CallToolRequestParam, ListToolsResult},
    service::{RoleClient, RunningService, ServiceExt},
    transport::{ConfigureCommandExt, TokioChildProcess},
};
use serde_json::{Value, json};
use std::error::Error;
use tokio::process::Command;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    // First message
    let mut messages = vec![json!({"role": "user", "content": "What is the sum of 3 and 2?"})];

    // Arrange OpenAI client
    let api_key = std::env::var("OPENAI_API_KEY")?;
    let openai_client = Client::with_config(
        OpenAIConfig::new()
            .with_api_base("https://models.github.ai/inference/chat")
            .with_api_key(api_key),
    );

    // Arrange MCP client
    let server_dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
        .parent()
        .unwrap()
        .join("calculator-server");

    let mcp_client = ()
        .serve(
            TokioChildProcess::new(Command::new("cargo").configure(|cmd| {
                cmd.arg("run").current_dir(server_dir);
            }))
            .map_err(RmcpError::transport_creation::<TokioChildProcess>)?,
        )
        .await?;

    // TODO: Make we collect MCP tool list

    // TODO: Talk for LLM wit tool call dem

    Ok(())
}

Dis code set up simple Rust app wey go connect MCP server and GitHub Models for LLM interaction.

Important

Make sure sey you set OPENAI_API_KEY environment variable wit your GitHub token before you run di app.

Great, for next step, make we list capabilities wey di server get.

-2- List server capabilities

Now, we go connect to di server and ask for dia capabilities:

Typescript

For di same class, add dis methods:

async connectToServer(transport: Transport) {
     await this.client.connect(transport);
     this.run();
     console.error("MCPClient started on stdin/stdout");
}

async run() {
    console.log("Asking server for available tools");

    // listing tool dem
    const toolsResult = await this.client.listTools();
}

For di previous code we:

Add code to connect to server, connectToServer.
Create run method wey handle app flow. So far e just list tools but we go add more soon.

Python

# List di resources wey dey available
resources = await session.list_resources()
print("LISTING RESOURCES")
for resource in resources:
    print("Resource: ", resource)

# List di tools wey dey available
tools = await session.list_tools()
print("LISTING TOOLS")
for tool in tools.tools:
    print("Tool: ", tool.name)
    print("Tool", tool.inputSchema["properties"])

Here na wetin we add:

List resources and tools and print dem. For tools, we also list inputSchema we go use later.

.NET

async Task<List<ChatCompletionsToolDefinition>> GetMcpTools()
{
    Console.WriteLine("Listing tools");
    var tools = await mcpClient.ListToolsAsync();

    List<ChatCompletionsToolDefinition> toolDefinitions = new List<ChatCompletionsToolDefinition>();

    foreach (var tool in tools)
    {
        Console.WriteLine($"Connected to server with tools: {tool.Name}");
        Console.WriteLine($"Tool description: {tool.Description}");
        Console.WriteLine($"Tool parameters: {tool.JsonSchema}");

        // TODO: convert tool definition from MCP tool to LLm tool     
    }

    return toolDefinitions;
}

For di previous code:

List tools wey dey available on MCP Server
For each tool, list name, description and schema. We go use schema to call tools later.

Java

// Make one tool provider wey dey find MCP tools by itself
ToolProvider toolProvider = McpToolProvider.builder()
        .mcpClients(List.of(mcpClient))
        .build();

// The MCP tool provider dey handle dis kain tin automatic:
// - Show all di tools wey dey for MCP server
// - Change MCP tool schemas to LangChain4j style
// - Manage how tools dey run and how dem respond

For di previous code:

Create McpToolProvider wey automatic discover and register all tools from MCP server
Tool provider handle conversion between MCP tool schemas and LangChain4j's tool format for inside
Dis approach remove manual work of listing and converting tools

Rust

To get tools from MCP server, use list_tools method. For your main function, after you set MCP client, add dis code:

// Comot MCP tool listicle
let tools = mcp_client.list_tools(Default::default()).await?;

-3- Convert server capabilities to LLM tools

Next step after listing server capabilities na to convert dem to format wey LLM go understand. Once we don do am, we fit provide dem as tools to our LLM.

TypeScript

Add dis code to convert MCP Server response to tool format wey LLM fit use:

openAiToolAdapter(tool: {
    name: string;
    description?: string;
    input_schema: any;
    }) {
    // Mak zod schema based on di input_schema
    const schema = z.object(tool.input_schema);

    return {
        type: "function" as const, // Make type na "function" explicitely
        function: {
        name: tool.name,
        description: tool.description,
        parameters: {
        type: "object",
        properties: tool.input_schema.properties,
        required: tool.input_schema.required,
        },
        },
    };
}

Di code above take response from MCP Server and convert am to tool definition format LLM fit understand.

Make we update run method to list server capabilities:

async run() {
    console.log("Asking server for available tools");
    const toolsResult = await this.client.listTools();
    const tools = toolsResult.tools.map((tool) => {
        return this.openAiToolAdapter({
        name: tool.name,
        description: tool.description,
        input_schema: tool.inputSchema,
        });
    });
}

For di code above, we update run method to map through di result and for each item call openAiToolAdapter.

Python

First, make we create dis converter function:

def convert_to_llm_tool(tool):
    tool_schema = {
        "type": "function",
        "function": {
            "name": tool.name,
            "description": tool.description,
            "type": "function",
            "parameters": {
                "type": "object",
                "properties": tool.inputSchema["properties"]
            }
        }
    }

    return tool_schema

For dis function convert_to_llm_tools, we take MCP tool response and convert am to format wey LLM fit understand.

Next, update our client code to use dis function like dis:

functions = []
for tool in tools.tools:
    print("Tool: ", tool.name)
    print("Tool", tool.inputSchema["properties"])
    functions.append(convert_to_llm_tool(tool))

We dey add call to convert_to_llm_tool to convert MCP tool response to wetin we fit feed LLM later.

.NET

Add code to convert MCP tool response to wetin LLM fit understand:

ChatCompletionsToolDefinition ConvertFrom(string name, string description, JsonElement jsonElement)
{ 
    // convert the tool to a function definition
    FunctionDefinition functionDefinition = new FunctionDefinition(name)
    {
        Description = description,
        Parameters = BinaryData.FromObjectAsJson(new
        {
            Type = "object",
            Properties = jsonElement
        },
        new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
    };

    // create a tool definition
    ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsToolDefinition(functionDefinition);
    return toolDefinition;
}

For di previous code:

Create function ConvertFrom wey take name, description and input schema.
Define functionality wey create FunctionDefinition wey go pass to ChatCompletionsDefinition. Dis one na wetin LLM fit understand.

Make we see how to update existing code to use dis function:

async Task<List<ChatCompletionsToolDefinition>> GetMcpTools()
{
    Console.WriteLine("Listing tools");
    var tools = await mcpClient.ListToolsAsync();

    List<ChatCompletionsToolDefinition> toolDefinitions = new List<ChatCompletionsToolDefinition>();

    foreach (var tool in tools)
    {
        Console.WriteLine($"Connected to server with tools: {tool.Name}");
        Console.WriteLine($"Tool description: {tool.Description}");
        Console.WriteLine($"Tool parameters: {tool.JsonSchema}");

        JsonElement propertiesElement;
        tool.JsonSchema.TryGetProperty("properties", out propertiesElement);

        var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
        Console.WriteLine($"Tool definition: {def}");
        toolDefinitions.Add(def);

        Console.WriteLine($"Properties: {propertiesElement}");        
    }

    return toolDefinitions;
}
```    In the preceding code, we've:

- Update the function to convert the MCP tool response to an LLm tool. Let's highlight the code we added:

    ```csharp
    JsonElement propertiesElement;
    tool.JsonSchema.TryGetProperty("properties", out propertiesElement);

    var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
    Console.WriteLine($"Tool definition: {def}");
    toolDefinitions.Add(def);
    ```

    The input schema is part of the tool response but on the "properties" attribute, so we need to extract. Furthermore, we now call `ConvertFrom` with the tool details. Now we've done the heavy lifting, let's see how it call comes together as we handle a user prompt next.

Java

// Make Bot interface for natural language interaction
public interface Bot {
    String chat(String prompt);
}

// Set up the AI service with LLM and MCP tools
Bot bot = AiServices.builder(Bot.class)
        .chatLanguageModel(model)
        .toolProvider(toolProvider)
        .build();

For di previous code:

Define simple Bot interface for natural language interactions
Use LangChain4j AiServices to automatically bind LLM wit MCP tool provider
Framework handle tool schema conversion and function calling automatically
Dis approach remove manual tool conversion - LangChain4j dey handle all di complexity to convert MCP tools make dem LLM compatible

Rust

To convert MCP tool response to format LLM fit understand, we add helper function wey format tools listing. Add dis code to your main.rs file below main function. E go call am when you make requests to LLM:

async fn format_tools(tools: &ListToolsResult) -> Result<Vec<Value>, Box<dyn Error>> {
    let tools_json = serde_json::to_value(tools)?;
    let Some(tools_array) = tools_json.get("tools").and_then(|t| t.as_array()) else {
        return Ok(vec![]);
    };

    let formatted_tools = tools_array
        .iter()
        .filter_map(|tool| {
            let name = tool.get("name")?.as_str()?;
            let description = tool.get("description")?.as_str()?;
            let schema = tool.get("inputSchema")?;

            Some(json!({
                "type": "function",
                "function": {
                    "name": name,
                    "description": description,
                    "parameters": {
                        "type": "object",
                        "properties": schema.get("properties").unwrap_or(&json!({})),
                        "required": schema.get("required").unwrap_or(&json!([]))
                    }
                }
            }))
        })
        .collect();

    Ok(formatted_tools)
}

Great, now we don set to handle user requests, make we do am next.

-4- Handle user prompt request

For dis part, we go handle user requests.

TypeScript

Add method wey go call our LLM:

async callTools(
    tool_calls: OpenAI.Chat.Completions.ChatCompletionMessageToolCall[],
    toolResults: any[]
) {
    for (const tool_call of tool_calls) {
    const toolName = tool_call.function.name;
    const args = tool_call.function.arguments;

    console.log(`Calling tool ${toolName} with args ${JSON.stringify(args)}`);


    // 2. Call di server tool
    const toolResult = await this.client.callTool({
        name: toolName,
        arguments: JSON.parse(args),
    });

    console.log("Tool result: ", toolResult);

    // 3. Do sometin wit di result
    // TODO

    }
}

For di code, we:

Add method callTools.

Di method take LLM response and check which tools dem call, if any:

for (const tool_call of tool_calls) {
const toolName = tool_call.function.name;
const args = tool_call.function.arguments;

console.log(`Calling tool ${toolName} with args ${JSON.stringify(args)}`);

// mek tool call
}

Call tool if LLM say make dem call am:

// 2. Call di server tool
const toolResult = await this.client.callTool({
    name: toolName,
    arguments: JSON.parse(args),
});

console.log("Tool result: ", toolResult);

// 3. Do sometin wit di result
// TODO

Update run method to include calling LLM and callTools:


// 1. Make messages wey go enter di LLM
const prompt = "What is the sum of 2 and 3?"

const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
        {
            role: "user",
            content: prompt,
        },
    ];

console.log("Querying LLM: ", messages[0].content);

// 2. Dey call di LLM
let response = this.openai.chat.completions.create({
    model: "gpt-4.1-mini",
    max_tokens: 1000,
    messages,
    tools: tools,
});    

let results: any[] = [];

// 3. Make sure say you check di LLM response, for each choice, check if e get tool calls
(await response).choices.map(async (choice: { message: any; }) => {
    const message = choice.message;
    if (message.tool_calls) {
        console.log("Making tool call")
        await this.callTools(message.tool_calls, results);
    }
});

Great, here be full code:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
import OpenAI from "openai";
import { z } from "zod"; // Import zod for schema validation

class MyClient {
    private openai: OpenAI;
    private client: Client;
    constructor(){
        this.openai = new OpenAI({
            baseURL: "https://models.inference.ai.azure.com", // fit need change go dis url later: https://models.github.ai/inference
            apiKey: process.env.GITHUB_TOKEN,
        });

        this.client = new Client(
            {
                name: "example-client",
                version: "1.0.0"
            },
            {
                capabilities: {
                prompts: {},
                resources: {},
                tools: {}
                }
            }
            );    
    }

    async connectToServer(transport: Transport) {
        await this.client.connect(transport);
        this.run();
        console.error("MCPClient started on stdin/stdout");
    }

    openAiToolAdapter(tool: {
        name: string;
        description?: string;
        input_schema: any;
          }) {
          // Make zod schema based on di input_schema
          const schema = z.object(tool.input_schema);
      
          return {
            type: "function" as const, // Clear clear set type to "function"
            function: {
              name: tool.name,
              description: tool.description,
              parameters: {
              type: "object",
              properties: tool.input_schema.properties,
              required: tool.input_schema.required,
              },
            },
          };
    }
    
    async callTools(
        tool_calls: OpenAI.Chat.Completions.ChatCompletionMessageToolCall[],
        toolResults: any[]
      ) {
        for (const tool_call of tool_calls) {
          const toolName = tool_call.function.name;
          const args = tool_call.function.arguments;
    
          console.log(`Calling tool ${toolName} with args ${JSON.stringify(args)}`);
    
    
          // 2. Call di server tool
          const toolResult = await this.client.callTool({
            name: toolName,
            arguments: JSON.parse(args),
          });
    
          console.log("Tool result: ", toolResult);
    
          // 3. Do sometin wit di result
          // TODO
    
         }
    }

    async run() {
        console.log("Asking server for available tools");
        const toolsResult = await this.client.listTools();
        const tools = toolsResult.tools.map((tool) => {
            return this.openAiToolAdapter({
              name: tool.name,
              description: tool.description,
              input_schema: tool.inputSchema,
            });
        });

        const prompt = "What is the sum of 2 and 3?";
    
        const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
            {
                role: "user",
                content: prompt,
            },
        ];

        console.log("Querying LLM: ", messages[0].content);
        let response = this.openai.chat.completions.create({
            model: "gpt-4.1-mini",
            max_tokens: 1000,
            messages,
            tools: tools,
        });    

        let results: any[] = [];
    
        // 3. Check di LLM response, for each choice, see if e get tool calls
        (await response).choices.map(async (choice: { message: any; }) => {
          const message = choice.message;
          if (message.tool_calls) {
              console.log("Making tool call")
              await this.callTools(message.tool_calls, results);
          }
        });
    }
    
}

let client = new MyClient();
 const transport = new StdioClientTransport({
            command: "node",
            args: ["./build/index.js"]
        });

client.connectToServer(transport);

Python

Add some imports we need to call LLM:

# llm
import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential
import json

Add function wey go call the LLM:

# llm

def call_llm(prompt, functions):
    token = os.environ["GITHUB_TOKEN"]
    endpoint = "https://models.inference.ai.azure.com"

    model_name = "gpt-4o"

    client = ChatCompletionsClient(
        endpoint=endpoint,
        credential=AzureKeyCredential(token),
    )

    print("CALLING LLM")
    response = client.complete(
        messages=[
            {
            "role": "system",
            "content": "You are a helpful assistant.",
            },
            {
            "role": "user",
            "content": prompt,
            },
        ],
        model=model_name,
        tools = functions,
        # Parameters we fit use if you want
        temperature=1.,
        max_tokens=1000,
        top_p=1.    
    )

    response_message = response.choices[0].message

    functions_to_call = []

    if response_message.tool_calls:
        for tool_call in response_message.tool_calls:
            print("TOOL: ", tool_call)
            name = tool_call.function.name
            args = json.loads(tool_call.function.arguments)
            functions_to_call.append({ "name": name, "args": args })

    return functions_to_call

For di code:

We pass functions we find on MCP server and convert to LLM.
Then call LLM wit those functions.
We inspect result to see functions we suppose call, if any.
Finally, pass list of functions to call.

Final step, update our main code:

prompt = "Add 2 to 20"

# ask LLM wetin tools dem dey for all, if e dey
functions_to_call = call_llm(prompt, functions)

# call di functions wey dem suggest
for f in functions_to_call:
    result = await session.call_tool(f["name"], arguments=f["args"])
    print("TOOLS result: ", result.content)

For di code above:

Call MCP tool via call_tool using function wey LLM talk say make we call based on prompt.
Print di result of tool call to MCP Server.

.NET

Make we show code for doing LLM prompt request:

var tools = await GetMcpTools();

for (int i = 0; i < tools.Count; i++)
{
    var tool = tools[i];
    Console.WriteLine($"MCP Tools def: {i}: {tool}");
}

// 0. Define the chat history and the user message
var userMessage = "add 2 and 4";

chatHistory.Add(new ChatRequestUserMessage(userMessage));

// 1. Define tools
ChatCompletionsToolDefinition def = CreateToolDefinition();


// 2. Define options, including the tools
var options = new ChatCompletionsOptions(chatHistory)
{
    Model = "gpt-4.1-mini",
    Tools = { tools[0] }
};

// 3. Call the model  

ChatCompletions? response = await client.CompleteAsync(options);
var content = response.Content;

For di code above:

Fetch tools from MCP server, var tools = await GetMcpTools().
Define user prompt userMessage.
Construct options object specifying model and tools.
Make request towards LLM.

Last step, check if LLM say make we call a function:

// 4. Check if the response contains a function call
ChatCompletionsToolCall? calls = response.ToolCalls.FirstOrDefault();
for (int i = 0; i < response.ToolCalls.Count; i++)
{
    var call = response.ToolCalls[i];
    Console.WriteLine($"Tool call {i}: {call.Name} with arguments {call.Arguments}");
    //Tool call 0: add with arguments {"a":2,"b":4}

    var dict = JsonSerializer.Deserialize<Dictionary<string, object>>(call.Arguments);
    var result = await mcpClient.CallToolAsync(
        call.Name,
        dict!,
        cancellationToken: CancellationToken.None
    );

    Console.WriteLine(result.Content.First(c => c.Type == "text").Text);

}

For di code above, we:

Loop through list of function calls.
For each tool call, parse name and arguments and call tool on MCP server using MCP client. Print results.

Here be di full code:

using Azure;
using Azure.AI.Inference;
using Azure.Identity;
using System.Text.Json;
using ModelContextProtocol.Client;
using ModelContextProtocol.Protocol;

var endpoint = "https://models.inference.ai.azure.com";
var token = Environment.GetEnvironmentVariable("GITHUB_TOKEN"); // Your GitHub Access Token
var client = new ChatCompletionsClient(new Uri(endpoint), new AzureKeyCredential(token));
var chatHistory = new List<ChatRequestMessage>
{
    new ChatRequestSystemMessage("You are a helpful assistant that knows about AI")
};

var clientTransport = new StdioClientTransport(new()
{
    Name = "Demo Server",
    Command = "/workspaces/mcp-for-beginners/03-GettingStarted/02-client/solution/server/bin/Debug/net8.0/server",
    Arguments = [],
});

Console.WriteLine("Setting up stdio transport");

await using var mcpClient = await McpClient.CreateAsync(clientTransport);

ChatCompletionsToolDefinition ConvertFrom(string name, string description, JsonElement jsonElement)
{ 
    // convert the tool to a function definition
    FunctionDefinition functionDefinition = new FunctionDefinition(name)
    {
        Description = description,
        Parameters = BinaryData.FromObjectAsJson(new
        {
            Type = "object",
            Properties = jsonElement
        },
        new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
    };

    // create a tool definition
    ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsToolDefinition(functionDefinition);
    return toolDefinition;
}



async Task<List<ChatCompletionsToolDefinition>> GetMcpTools()
{
    Console.WriteLine("Listing tools");
    var tools = await mcpClient.ListToolsAsync();

    List<ChatCompletionsToolDefinition> toolDefinitions = new List<ChatCompletionsToolDefinition>();

    foreach (var tool in tools)
    {
        Console.WriteLine($"Connected to server with tools: {tool.Name}");
        Console.WriteLine($"Tool description: {tool.Description}");
        Console.WriteLine($"Tool parameters: {tool.JsonSchema}");

        JsonElement propertiesElement;
        tool.JsonSchema.TryGetProperty("properties", out propertiesElement);

        var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
        Console.WriteLine($"Tool definition: {def}");
        toolDefinitions.Add(def);

        Console.WriteLine($"Properties: {propertiesElement}");        
    }

    return toolDefinitions;
}

// 1. List tools on mcp server

var tools = await GetMcpTools();
for (int i = 0; i < tools.Count; i++)
{
    var tool = tools[i];
    Console.WriteLine($"MCP Tools def: {i}: {tool}");
}

// 2. Define the chat history and the user message
var userMessage = "add 2 and 4";

chatHistory.Add(new ChatRequestUserMessage(userMessage));


// 3. Define options, including the tools
var options = new ChatCompletionsOptions(chatHistory)
{
    Model = "gpt-4.1-mini",
    Tools = { tools[0] }
};

// 4. Call the model  

ChatCompletions? response = await client.CompleteAsync(options);
var content = response.Content;

// 5. Check if the response contains a function call
ChatCompletionsToolCall? calls = response.ToolCalls.FirstOrDefault();
for (int i = 0; i < response.ToolCalls.Count; i++)
{
    var call = response.ToolCalls[i];
    Console.WriteLine($"Tool call {i}: {call.Name} with arguments {call.Arguments}");
    //Tool call 0: add with arguments {"a":2,"b":4}

    var dict = JsonSerializer.Deserialize<Dictionary<string, object>>(call.Arguments);
    var result = await mcpClient.CallToolAsync(
        call.Name,
        dict!,
        cancellationToken: CancellationToken.None
    );

    Console.WriteLine(result.Content.OfType<TextContentBlock>().First().Text);

}

// 6. Print the generic response
Console.WriteLine($"Assistant response: {content}");

Java

try {
    // Make natural language requests wey go automatically use MCP tools
    String response = bot.chat("Calculate the sum of 24.5 and 17.3 using the calculator service");
    System.out.println(response);

    response = bot.chat("What's the square root of 144?");
    System.out.println(response);

    response = bot.chat("Show me the help for the calculator service");
    System.out.println(response);
} finally {
    mcpClient.close();
}

For di code above:

Use simple natural language prompts to interact wit MCP server tools
LangChain4j framework automatically handle:
- Convert user prompts to tool calls if needed
- Call appropriate MCP tools base on LLM decision
- Manage conversation flow between LLM and MCP server
bot.chat() method return natural language responses wey fit include MCP tool results
Dis approach give seamless user experience, users no need know di behind MCP implememtation

Complete example code:

public class LangChain4jClient {
    
    public static void main(String[] args) throws Exception {        ChatLanguageModel model = OpenAiOfficialChatModel.builder()
                .isGitHubModels(true)
                .apiKey(System.getenv("GITHUB_TOKEN"))
                .timeout(Duration.ofSeconds(60))
                .modelName("gpt-4.1-nano")
                .timeout(Duration.ofSeconds(60))
                .build();

        McpTransport transport = new HttpMcpTransport.Builder()
                .sseUrl("http://localhost:8080/sse")
                .timeout(Duration.ofSeconds(60))
                .logRequests(true)
                .logResponses(true)
                .build();

        McpClient mcpClient = new DefaultMcpClient.Builder()
                .transport(transport)
                .build();

        ToolProvider toolProvider = McpToolProvider.builder()
                .mcpClients(List.of(mcpClient))
                .build();

        Bot bot = AiServices.builder(Bot.class)
                .chatLanguageModel(model)
                .toolProvider(toolProvider)
                .build();

        try {
            String response = bot.chat("Calculate the sum of 24.5 and 17.3 using the calculator service");
            System.out.println(response);

            response = bot.chat("What's the square root of 144?");
            System.out.println(response);

            response = bot.chat("Show me the help for the calculator service");
            System.out.println(response);
        } finally {
            mcpClient.close();
        }
    }
}

Rust

Dis na where most work dey happen. We go call LLM wit initial user prompt, then check response to see if we need call any tools. If yes, we go call dem and continue talk wit LLM until no more tools to call and final response ready.

We go call LLM multiple times, so make we define function wey handle LLM call. Add dis function to your main.rs file:

async fn call_llm(
    client: &Client<OpenAIConfig>,
    messages: &[Value],
    tools: &ListToolsResult,
) -> Result<Value, Box<dyn Error>> {
    let response = client
        .completions()
        .create_byot(json!({
            "messages": messages,
            "model": "openai/gpt-4.1",
            "tools": format_tools(tools).await?,
        }))
        .await?;
    Ok(response)
}

Dis function take LLM client, list of messages (including user prompt), tools from MCP server, and send request to LLM, then return response. The response from the LLM go get an array of choices. We go need process the result to see if any tool_calls dey. Dis go make we sabi say the LLM dey request make we call one particular tool with arguments. Add the following code for the bottom of your main.rs file to define one function to handle the LLM response:

async fn process_llm_response(
    llm_response: &Value,
    mcp_client: &RunningService<RoleClient, ()>,
    openai_client: &Client<OpenAIConfig>,
    mcp_tools: &ListToolsResult,
    messages: &mut Vec<Value>,
) -> Result<(), Box<dyn Error>> {
    let Some(message) = llm_response
        .get("choices")
        .and_then(|c| c.as_array())
        .and_then(|choices| choices.first())
        .and_then(|choice| choice.get("message"))
    else {
        return Ok(());
    };

    // Print content if e dey
    if let Some(content) = message.get("content").and_then(|c| c.as_str()) {
        println!("🤖 {}", content);
    }

    // Handle tool calls
    if let Some(tool_calls) = message.get("tool_calls").and_then(|tc| tc.as_array()) {
        messages.push(message.clone()); // Add assistant message

        // Run every tool call
        for tool_call in tool_calls {
            let (tool_id, name, args) = extract_tool_call_info(tool_call)?;
            println!("⚡ Calling tool: {}", name);

            let result = mcp_client
                .call_tool(CallToolRequestParam {
                    name: name.into(),
                    arguments: serde_json::from_str::<Value>(&args)?.as_object().cloned(),
                })
                .await?;

            // Add tool result to messages
            messages.push(json!({
                "role": "tool",
                "tool_call_id": tool_id,
                "content": serde_json::to_string_pretty(&result)?
            }));
        }

        // Continue talk wit tool results
        let response = call_llm(openai_client, messages, mcp_tools).await?;
        Box::pin(process_llm_response(
            &response,
            mcp_client,
            openai_client,
            mcp_tools,
            messages,
        ))
        .await?;
    }
    Ok(())
}

If tool_calls dey, e go extract the tool info, call the MCP server with the tool request, then add the results to the conversation messages. E go continue the conversation with the LLM and the messages go dey updated with the assistant's response and tool call results.

To extract tool call info wey the LLM dey return for MCP calls, we go add another helper function to extract everything we need to make the call. Add the following code for the bottom of your main.rs file:

fn extract_tool_call_info(tool_call: &Value) -> Result<(String, String, String), Box<dyn Error>> {
    let tool_id = tool_call
        .get("id")
        .and_then(|id| id.as_str())
        .unwrap_or("")
        .to_string();
    let function = tool_call.get("function").ok_or("Missing function")?;
    let name = function
        .get("name")
        .and_then(|n| n.as_str())
        .unwrap_or("")
        .to_string();
    let args = function
        .get("arguments")
        .and_then(|a| a.as_str())
        .unwrap_or("{}")
        .to_string();
    Ok((tool_id, name, args))
}

With all the pieces for place, we fit now handle the initial user prompt and call the LLM. Update your main function to include the following code:

// LLM tok tok wit tool dem call
let response = call_llm(&openai_client, &messages, &tools).await?;
process_llm_response(
    &response,
    &mcp_client,
    &openai_client,
    &tools,
    &mut messages,
)
.await?;

Dis one go query the LLM with the initial user prompt wey dey ask for the sum of two numbers, and e go process the response to dynamically handle tool calls.

Great, you don do am!

Assignment

Take the code from the exercise and build the server with some more tools. Then create a client with an LLM, like the one we do for the exercise, and test am out with different prompts to make sure all your server tools dey called dynamically. Dis way wey you build client mean say the end user go get better user experience because dem fit use prompts, instead of exact client commands, and dem no go sabi say any MCP server dey get called.

Solution

Key Takeaways

To add LLM to your client go give better way for users to interact with MCP Servers.
You need make you convert the MCP Server response to something wey the LLM fit understand.

Samples

Additional Resources

What's Next

Next: Consuming a server using Visual Studio Code

Disclaimer: Dis document don translate wit AI translation service Co-op Translator. Even tho we dey try make am correct, abeg make you know say automated translation fit get errors or mistakes. Di original document for dia own language na im be di correct source. For important info, make person wey sabi human translation do am. We no go responsible for any misunderstanding or wrong understanding wey fit happen because of dis translation.

44 KiB Raw Permalink Blame History Unescape Escape

Creating a client wit LLM

Overview

Learning Objectives

Approach

Exercise: Creating a client wit LLM

Authentication using GitHub Personal Access Token

-1- Connect to server

TypeScript

Python

.NET

Java

Rust

-2- List server capabilities

Typescript

Python

.NET

Java

Rust

-3- Convert server capabilities to LLM tools

TypeScript

Python

.NET

Java

Rust

-4- Handle user prompt request

TypeScript

Python

.NET

Java

Rust

Assignment

Solution

Key Takeaways

Samples

Additional Resources

What's Next

44 KiB

Raw Permalink Blame History