This feature uses the Unity AI Gateway and Unity’s MCP Server. The MCP Server, developed in collaboration with Meta, is available through Unity’s AI Gateway Early Access Beta⁠.

To complete this guide, you must be a member of Unity’s AI Gateway Early Access Beta program to access the required downloads and documentation.

This guide summarizes the steps to set up the Unity MCP Server and Meta XR Unity MCP Extension so you can create and modify Unity scenes using text prompts and AI agents.

Model Context Protocol (MCP) lets you connect AI agents, such as large language models (LLMs), with external services and data sources. The Meta XR Unity MCP Extension is an MCP Server containing tools that control your Unity Editor and project. AI services use an agentic shell with an MCP Client to access MCP Server tools.

The Unity MCP Server acts as a backend server that connects AI agents to Unity.

The Meta XR Unity MCP Extension lets Unity receive and run commands from the MCP Server in Meta XR projects.

After configuring the extension, you can submit text prompts through your AI agent to command the MCP Server to perform tasks such as:

Creating, updating, or deleting a GameObject

Moving or rotating GameObjects using relative transforms

Adding an interaction to make a GameObject grabbable

Creating teleport hotspots

After completing this step-by-step guide, you should be able to:

Add the Meta XR Unity MCP Extension to your project

Set up Unity MCP Server with one of the supported AI agents

Modify your scene by sending text prompts through your AI agent

Prerequisites

Before proceeding with this tutorial, complete the setup steps outlined in Unity Hello World for Meta Quest VR headsets to create a project with the necessary dependencies.

Account requirements

Membership to Unity’s AI Gateway Early Access program: Apply on Unity’s AI Gateway Beta sign-up form⁠.

An account on an AI service that supports MCP Servers. For a list of compatible services, see Unity’s AI Gateway Early Access Beta documentation.

Software requirements

Unity Editor 2022.3.15f1 or later (6.1 or later recommended)

Meta XR SDK packages v78 or later

Instructions

Step 1: Install the Unity MCP package

Note: Without access to Unity’s AI Gateway beta program, you cannot access the documentation and files necessary to complete this step and guide.

For instructions on how to install the Unity MCP package, see Unity’s AI Gateway Beta documentation.

Step 2: Install the Meta XR Unity MCP Extension

The Meta XR Unity MCP Extension is currently only available on GitHub. Functionality might vary by version.

To install the extension from GitHub:

Open the Package Manager by selecting Window > Package Management > Package Manager.

Click the + (plus sign) to show install options and then select Install package from git URL... from the top left.

Select Install package from git URL in the Package Manager

In the text input area, enter https://github.com/meta-quest/Unity-MCP-Extensions.git and then click Install.

In the Package Manager, select In Project on the left and verify that the Meta XR Unity MCP Extension appears in the list.

Confirm Meta XR Unity MCP Extension appears in the list of packages.

Step 3: Install and start the agentic shell

Note: Without access to Unity’s AI Gateway beta program, you cannot access the documentation and files necessary to complete this step and guide.

For instructions on installing an AI agent’s agentic shell with the Unity MCP server, see Unity’s AI Gateway Beta documentation.

After installing the agentic shell and adding the Unity MCP server to it, click Accept in the dialog that appears in Unity Editor, as shown below:

Click Accept to let your LLM make changes to your Unity scene.

Step 4: Send text prompt commands

From the MCP Server provider UI, try submitting the following text prompts to update your scene:

Add a cube with distance grab enabled

Set up ray and poke interactions for all UI elements

Add a teleport hotspot at the center of the scene

Troubleshooting

The MCP Connection Request dialog does not appear after starting the LLM MCP server or the LLM throws an error when attempting to update the project.

Navigate to Project Settings > MCP Server > Other Connections. This displays a list of all the attempted connections and their acceptance or rejection status. For any rejected connection, expand the Connection Details section, verify the details match your connection attempt, and then click Accept.

The extension crashes Unity or behaves unexpectedly.

To view or leave feedback about unexpected behavior of the extension, navigate to the Meta Quest Developer Feedback Center and click View investigations in the VR section.

Did you find this page helpful?