MCP Command Panel: Bridging AI to Unreal Editor

What if your AI coding assistant could reach into the Unreal Editor and create actors, modify properties, or trigger builds — without you clicking a single button? That question drove me to build the MCP Command Panel, a full-stack bridge between AI models and the Unreal Engine editor.

What Is the Model Context Protocol

The Model Context Protocol (MCP) is an open standard for connecting AI models to external tools and data sources. Instead of LLMs just generating text, MCP lets them invoke real operations — reading files, querying databases, or in this case, controlling a game editor.

For game development, MCP is transformative. Level designers could describe a scene in natural language and have the editor assemble it. Programmers could ask an AI to scaffold a gameplay system and have the boilerplate code and Blueprints generated in-engine. The MCP Command Panel makes that interaction layer concrete.

Architecture: Two Languages, One Pipeline

The system has two halves that communicate over HTTP and TCP.

The C++ Unreal plugin lives inside the editor as a standard engine plugin. It registers an HTTP client that connects to the Python backend, receives JSON-RPC command payloads, validates them against a schema, and dispatches them to the appropriate editor subsystem. An Editor Utility Widget provides a live dashboard showing incoming commands, execution status, and logs.

The Python FastAPI backend acts as the orchestration layer. It exposes MCP-compatible tool endpoints that AI providers (Claude, Gemini) can call. When a command comes in, the backend translates it into Unreal's expected format, forwards it to the plugin over TCP, and streams the result back to the AI model.

# Simplified FastAPI endpoint for MCP tool registration
@app.post("/mcp/tools/execute")
async def execute_tool(request: ToolRequest):
    """
    Receives an MCP tool call from the AI model,
    translates it into an Unreal command, and
    forwards it to the C++ plugin via TCP.
    """
    command = translate_to_unreal(request.tool_name, request.arguments)
    result = await unreal_bridge.send_command(command)
    return ToolResponse(
        status="success",
        result=result.to_dict()
    )

The C++ Bridge: Editor Integration

The plugin side was the harder engineering challenge. Unreal's editor runs on the game thread, and network operations must not block it. I used Unreal's built-in HTTP module for the initial handshake, then established a persistent TCP connection with keepalive for low-latency command delivery.

Commands arrive as JSON-RPC payloads. Each command type maps to a handler function that interacts with the editor API — spawning actors, modifying component properties, compiling Blueprints, or triggering PIE (Play In Editor) sessions. The handler validates inputs against a schema before execution, preventing malformed AI output from crashing the editor.

// Command dispatch in the C++ plugin
void FMCPCommandHandler::ProcessCommand(
    const FString& Method,
    const TSharedPtr<FJsonObject>& Params)
{
    if (Method == TEXT("editor.spawnActor"))
    {
        HandleSpawnActor(Params);
    }
    else if (Method == TEXT("editor.setProperty"))
    {
        HandleSetProperty(Params);
    }
    else if (Method == TEXT("editor.compileBP"))
    {
        HandleCompileBlueprint(Params);
    }
    // ... additional handlers
}

Automatic reconnection was essential. During long development sessions the TCP connection would occasionally drop — the plugin detects disconnection within two keepalive intervals and re-establishes the link without user intervention.

Python Orchestration Layer

The Python side handles the complexity of multiple AI providers. Each provider has slightly different tool-calling conventions, so the orchestration layer normalizes those into a consistent internal format before forwarding to Unreal.

FastAPI's async support was critical here. Commands from the AI model are non-blocking, and the WebSocket channel to the editor streams results back as they complete. This means the AI model can issue a batch of operations — "create a directional light, set its intensity to 5.0, rotate it 45 degrees" — and receive confirmation for each step individually.

Real-World Usage

In practice, the most useful commands turned out to be surprisingly mundane. Bulk property editing across dozens of actors, scaffolding standard gameplay components from a description, and automated PIE testing saved more time than any flashy level-generation demo. The tool shines at repetitive editor tasks that eat hours during production.

I also use it during my own development workflow. When working on Island Escape, I had the AI set up test scenarios — spawning enemy waves with specific configurations — through MCP commands rather than manually placing actors. That feedback loop of "describe, execute, observe" is genuinely faster than the traditional editor workflow for iteration-heavy tasks.

Looking Ahead

AI-assisted game development is not replacing developers. It is eliminating busywork so developers can focus on creative and architectural decisions that actually matter. The MCP Command Panel is my bet on what that workflow looks like in practice — a clean bridge between natural language intent and editor operations.

The next milestone is Blueprint graph manipulation. Right now the tool handles property editing and actor management, but generating visual scripting nodes programmatically would open up an entirely new category of AI-assisted workflows.