Skip to main content

Module 7: Testing Spring AI + MCP Applications

Duration: ~50 minutes | Level: Intermediate | Prerequisites: Module 6 complete. Familiar with JUnit 5, Spring Boot Test, and Mockito.

What You'll Learn​

Testing AI applications has a reputation for being hard. Real LLMs are non-deterministic, slow, and cost money. MCP connections require real server processes.

None of that is actually a testing problem; it's a design problem. AI applications that are hard to test have two structural issues: the AI decision-making is coupled to business logic, and the tool execution is coupled to the AI call.

Fix the design and testing becomes straightforward. This module shows you how.

The Testing Pyramid for AI Applications​

The standard testing pyramid applies, with one additional consideration: AI behaviour is inherently probabilistic, so anything that depends on specific model output is fragile. Design tests that validate structure and integration, not exact text.

                     ┌─────────────┐
                     │  E2E / AI   │  ← Real LLM + real servers, minimal
                     └──────┬──────┘    ("does the agent roughly work?")
                  ┌─────────┴─────────┐
                  │  Integration Tests │  ← Real MCP, mock model
                  └─────────┬─────────┘    (routing, protocol, wiring)
           ┌────────────────┴───────────────┐
           │         Unit Tests              │  ← No AI, no MCP
           └─────────────────────────────────┘    (business logic, tool methods)

Unit tests test your tool methods as plain Java methods, no Spring, no AI, no network. Fast, deterministic, the bulk of your coverage.

Integration tests test the MCP wiring. Do the right tools get registered? Does the server respond correctly to MCP requests? Use a real MCP test client but mock the LLM.

E2E tests test the complete agent. Use them sparingly, a small number of "smoke tests" that verify the agent can answer a broad class of question, run less frequently (not on every PR, perhaps daily).

Unit Testing Tool Methods​

Your @McpTool methods are Spring beans. Strip away the annotation and they're just Java methods. Test them that way.

package com.themcpguy.orders;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.DisplayName;

import java.util.List;

import static org.assertj.core.api.Assertions.*;

class OrderMcpToolsTest {

    private OrderService orderService;
    private OrderMcpTools tools;

    @BeforeEach
    void setUp() {
        orderService = new OrderService();  // uses in-memory seed data
        tools = new OrderMcpTools(orderService);
    }

    @Test
    @DisplayName("getOrder returns correct order for valid ID")
    void getOrder_validId_returnsOrder() {
        Order order = tools.getOrder("ORD-10001");

        assertThat(order.orderId()).isEqualTo("ORD-10001");
        assertThat(order.status()).isEqualTo("SHIPPED");
        assertThat(order.customerId()).isEqualTo("CUST-42");
    }

    @Test
    @DisplayName("getOrder throws descriptive error for unknown ID")
    void getOrder_unknownId_throwsIllegalArgument() {
        assertThatThrownBy(() -> tools.getOrder("ORD-99999"))
            .isInstanceOf(IllegalArgumentException.class)
            .hasMessageContaining("ORD-99999");
    }

    @Test
    @DisplayName("getOrder rejects malformed order IDs")
    void getOrder_malformedId_throwsIllegalArgument() {
        assertThatThrownBy(() -> tools.getOrder("NOT-AN-ORDER-ID"))
            .isInstanceOf(IllegalArgumentException.class)
            .hasMessageContaining("Invalid order ID format");
    }

    @Test
    @DisplayName("getCustomerOrders returns orders sorted newest-first")
    void getCustomerOrders_returnsInDescendingOrder() {
        // Add a second order for the same customer
        orderService.placeOrder("CUST-42",
            List.of(new OrderItem("PROD-010", "Test Widget", 1, 9.99)));

        List<Order> orders = tools.getCustomerOrders("CUST-42");

        assertThat(orders).hasSizeGreaterThanOrEqualTo(1);
        // Verify newest first
        if (orders.size() > 1) {
            assertThat(orders.get(0).createdAt())
                .isGreaterThanOrEqualTo(orders.get(1).createdAt());
        }
    }

    @Test
    @DisplayName("updateOrderStatus changes status and updates timestamp")
    void updateOrderStatus_validTransition_updatesOrder() {
        Order updated = tools.updateOrderStatus("ORD-10002", "PROCESSING");

        assertThat(updated.status()).isEqualTo("PROCESSING");
        assertThat(updated.orderId()).isEqualTo("ORD-10002");
        assertThat(updated.lastUpdated()).isGreaterThan(updated.createdAt());
    }
}

No Spring context, no MCP, no AI. These tests run in milliseconds. Run them on every change.

The key insight: the @McpTool annotation is metadata. The method itself is plain Java. Ignoring the annotation and testing the method directly gives you full coverage of your business logic at zero AI cost.

Integration Testing the MCP Server​

Unit tests verify the tool logic. Integration tests verify that the MCP protocol wiring is correct: are the tools registered with the right names, do the JSON schemas come out as expected, are the underlying beans wired together?

The cheapest reliable pattern is to load the Spring application context with @SpringBootTest(webEnvironment = NONE) and inject the auto-generated MCP tool specifications directly. No transport, no client, no I/O.

package com.themcpguy.orders;

import io.modelcontextprotocol.server.McpServerFeatures.SyncToolSpecification;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import java.util.List;
import java.util.Map;

import static org.assertj.core.api.Assertions.*;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE)
class OrderMcpServerIntegrationTest {

    // With @McpTool there is no ToolCallbackProvider bean on the server side. The
    // annotation scanner turns each @McpTool method into a SyncToolSpecification and
    // publishes them as List<SyncToolSpecification> beans. Collect every such bean
    // (keyed by bean name) and flatten, so the assertions do not depend on which
    // auto-configuration produced a given specification.
    @Autowired
    private Map<String, List<SyncToolSpecification>> toolSpecBeans;

    private List<SyncToolSpecification> specs() {
        return toolSpecBeans.values().stream().flatMap(List::stream).toList();
    }

    @Test
    void allExpectedToolsAreRegistered() {
        assertThat(specs())
            .extracting(spec -> spec.tool().name())
            .containsExactlyInAnyOrder(
                "get_order",
                "get_customer_orders",
                "get_orders_by_status",
                "update_order_status");
    }

    @Test
    void getOrderHasOrderIdInSchema() {
        SyncToolSpecification getOrder = specs().stream()
            .filter(spec -> "get_order".equals(spec.tool().name()))
            .findFirst().orElseThrow();

        assertThat(getOrder.tool().inputSchema().toString()).contains("orderId");
        assertThat(getOrder.tool().description()).containsIgnoringCase("order");
    }
}

This catches: tool registration bugs, schema generation mistakes, broken @McpTool argument binding, and annotation-scanner registration problems. It's deterministic, fast, and runs as part of the standard test suite without any HTTP infrastructure.

For tests that need the real transport layer (Streamable HTTP framing, SSE event encoding, session handling), use @SpringBootTest(webEnvironment = RANDOM_PORT) and connect a real McpSyncClient over HTTP. The Protocol-Level Testing module of the Testing & Observability course covers this pattern in more depth — that course is coming soon.

Mocking the Chat Model for Deterministic Tests​

When you test a ChatClient-based service, you don't want to call the real LLM. It's slow, expensive, and non-deterministic, the model might word its response differently on each call, breaking text-matching assertions.

Mock the ChatModel bean instead:

package com.themcpguy.springai.agent;

import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.boot.test.context.SpringBootTest;
// @MockitoBean (in org.springframework.test.context.bean.override.mockito) replaces
// @MockBean (deprecated in Spring Boot 3.4, removed in Spring Boot 4) but works identically.
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.beans.factory.annotation.Autowired;

import java.util.List;

import static org.assertj.core.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;

@SpringBootTest
class FilesystemAgentServiceTest {

    @MockitoBean
    private ChatModel chatModel;     // Replaces the real Anthropic/OpenAI client

    @Autowired
    private FilesystemAgentService agentService;

    @Test
    void chatReturnsModelResponse() {
        // Arrange, the mock model returns a fixed response
        var mockResponse = new ChatResponse(
            List.of(new Generation(new AssistantMessage("The files are: foo.txt, bar.log")))
        );
        when(chatModel.call(any(Prompt.class))).thenReturn(mockResponse);

        // Act
        String response = agentService.chat("What files are in /tmp?");

        // Assert
        assertThat(response).contains("foo.txt", "bar.log");
        verify(chatModel).call(any(Prompt.class));  // verify it was called once
    }

    @Test
    void chatIncludesUserMessageInRequest() {
        var mockResponse = new ChatResponse(
            List.of(new Generation(new AssistantMessage("I'll help you.")))
        );
        when(chatModel.call(any(Prompt.class))).thenReturn(mockResponse);

        agentService.chat("Delete all files in /tmp");

        // Capture and inspect what was sent to the model
        var captor = ArgumentCaptor.forClass(Prompt.class);
        verify(chatModel).call(captor.capture());

        var prompt = captor.getValue();
        String userContent = prompt.getInstructions().stream()
            .filter(m -> m instanceof org.springframework.ai.chat.messages.UserMessage)
            .map(m -> m.getText())
            .findFirst().orElse("");

        assertThat(userContent).contains("Delete all files in /tmp");
    }
}

With @MockitoBean ChatModel, Spring replaces the real AI client with a Mockito mock. Your ChatClient.Builder is auto-configured but uses the mock model. The MCP tools are still wired up; only the LLM call is mocked.

This lets you test:

  • That the right message is sent to the model
  • That tool results are processed correctly (by controlling what the mock returns)
  • That error handling works when the model returns unexpected responses
  • Conversation memory (verify history accumulates correctly)

Testing Tool Selection (The Hard Part)​

The hardest part of testing AI agents is verifying that the right tool gets called in response to a given prompt. This is fundamentally non-deterministic, you can't guarantee which tool the model will choose.

Two strategies:

Strategy 1: Test at the Tool Level, Not the Agent Level​

Instead of testing "given input X, does the agent call tool Y?", test:

  1. That tool Y behaves correctly when called with appropriate input (unit test)
  2. That the agent can successfully complete a task (integration test at a higher level)

This is more robust and easier to maintain. The model's tool selection is the model's responsibility, your tests verify your code, not the model.

Strategy 2: Scripted Tool Responses​

For scenarios where you need to verify tool selection, use a custom ToolCallback that records calls:

class RecordingToolCallback implements ToolCallback {

    private final List<String> calls = new ArrayList<>();
    private final String name;
    private final String description;

    RecordingToolCallback(String name, String description) {
        this.name = name;
        this.description = description;
    }

    @Override
    public ToolDefinition getToolDefinition() {
        return ToolDefinition.builder()
            .name(name)
            .description(description)
            .inputSchema("{\"type\":\"object\",\"properties\":{}}")
            .build();
    }

    @Override
    public String call(String toolInput) {
        calls.add(toolInput);
        return "{\"result\": \"success\", \"input\": " + toolInput + "}";
    }

    public List<String> getCalls() { return calls; }
    public boolean wasCalled() { return !calls.isEmpty(); }
}

Inject recording callbacks instead of real MCP tools in tests that need to verify tool invocation patterns.

Test Configuration​

Isolate your test configuration to avoid loading production API keys and MCP server processes:

@TestConfiguration
class TestConfig {

    @Bean
    @Primary
    public ChatModel mockChatModel() {
        return mock(ChatModel.class);
    }
}
# src/test/resources/application-test.properties
# Disable the whole MCP client in unit/integration tests
# (Spring AI 2.0.x exposes a single `spring.ai.mcp.client.enabled` flag;
# there is no per-stdio-connection `enabled` property.)
spring.ai.mcp.client.enabled=false
spring.ai.mcp.client.toolcallback.enabled=false

# Use a fake API key (won't be called due to mocked ChatModel)
spring.ai.anthropic.api-key=sk-ant-test-key

One important caveat: with the MCP client disabled, Spring AI never creates the SyncMcpToolCallbackProvider bean, so any bean that injects it (like FilesystemAgentService from Module 4) will fail to load with an UnsatisfiedDependencyException. Use this profile only for tests that never touch MCP tool beans. For FilesystemAgentServiceTest above, either leave the MCP client enabled, which is what keeps the tools "still wired up" as described in that section, or add @MockitoBean SyncMcpToolCallbackProvider mcpTools; alongside the mocked ChatModel. @MockitoBean creates the bean when none exists, and the mock's empty tool list is fine because the test never asks the model to call a tool.

Activate with @ActiveProfiles("test") or @SpringBootTest(properties = "spring.profiles.active=test").

Test the MCP Client Side​

When your application consumes MCP servers, you can test the tool discovery and invocation by starting a real MCP server in-process during tests:

@SpringBootTest
class McpClientIntegrationTest {

    @BeforeAll
    static void startMcpServer() throws Exception {
        // Start the order server on a random port and connect a real HTTP client
        // or start the actual server JAR and connect to it
    }

    @Test
    void agentCanAnswerQuestionsAboutOrders() {
        // Use a mock ChatModel that simulates a model calling getOrder
        // then verify the response includes order data
    }
}

For end-to-end tests, start real server processes using ProcessBuilder and connect to them via stdio from the test. Use JUnit 5's @BeforeAll and @AfterAll to manage the server lifecycle.

CI/CD Considerations​

  • Unit tests: run on every commit. No API keys needed. Fast.
  • Integration tests (mocked model + real MCP server): run on every commit. Require the server JAR but no API keys. Medium speed.
  • E2E tests (real model + real servers): run nightly or on release branches. Require API keys as CI secrets. Slow and have token costs.

Never put API keys in CI configuration files. Use your CI system's secret management (GitHub Actions secrets, AWS Secrets Manager, HashiCorp Vault). Module 8 covers this in detail.

Next: Module 8: Production Configuration. Externalise config, add health indicators, instrument with metrics, and make your AI application production-ready.