A week ago, OpenAI announced a new SDK to build applications you can chat with, inside ChatGPT.

From a business perspective, this makes sense: by keeping users engaged within the chat rather than sending them elsewhere, they increase the likelihood that ChatGPT will replace traditional search engines as people’s go-to internet gateway.

By the end of the year, OpenAI plans to launch an app marketplace where developers can submit and monetize their applications. Done right, this could spark the same kind of gold rush that the iPhone App Store created in 2008.

Now it’s time to dive deeper into app development.

In my previous post, I showed you how to run the Pizza sample application in ChatGPT. In this post, I’ll walk you through building an application from scratch.

Structure of a ChatGPT application

A ChatGPT app is simply a Model Context Protocol (MCP) server. Rather than inventing yet another specification, OpenAI chose to use this existing standard—a decision that should significantly accelerate adoption.

ChatGPT applications have two parts:

An MCP server (with some additional OpenAI-specific configuration)
A separate web application that handles the user interface. The UI widgets are declared as resources in the MCP server.

Here is an example of what it looks like in ChatGPT using a TicTacToe app, starting with the prompt: show me the tictactoe board :

How the Protocol Works

ChatGPT apps are built on standard MCP servers. The behavior depends on whether your tool includes OpenAI-specific metadata:

If the tool has no OpenAI extensions, it works like any standard MCP tool—the response goes to the language model
If the tool has OpenAI extensions, ChatGPT renders the associated UI widget in the chat

For example:

The interactions with the MCP server look like this:

MCP server development

Some code should help to understand how the MCP protocol works:

server.registerTool(
  "display_board",
  {
    title: "Display TicTacToe Board",
    description: "Display a board",
    inputSchema: {},
    _meta: {
      "openai/outputTemplate": "ui://widget/tictactoe.html",
      "openai/toolInvocation/invoking": "Displaying the board",
      "openai/toolInvocation/invoked": "Displayed the board",
      "openai/widgetAccessible": true,
      "openai/widgetDomain": "https://tunnel.xxx.com",
      "openai/widgetCSP": {
        connect_domains: [],
        resource_domains: ["https://tunnel.xxx.com"],
      },
      "openai/widgetDescription": "TicTacToe board component",
    },
  },
  async () => {
      // code omitted for now
  }
);

This code declares a MCP tool using the official MCP typescript SDK:

the name of the tool is display_board . This function is invoked with a chat like: using the TicTacToe app, display a board
the openai/outputTemplate indicate the widget to be rendered
the invoking/invoked properties define the text rendered during and after the tool invokation
the openai/widgetAccessible property indicate if the widget can make calls to the MCP server
I had to set the widgetDomain, widgetCSP properties to allow the widget to get access to the window.openai object. To be honest, I am not entirely sure if those are required, but the specs will become clearer once OpenAI allow submissions.

The properties above indicate that the widget with the name ui://widget/tictactoe.html should be rendered.

That widget is defined as a MCP resource:

const TIC_TAC_TOE_JS = readFileSync("../web/dist/app.js", "utf8");
const TIC_TAC_TOE_CSS = readFileSync("../web/dist/app.css", "utf8");

// UI resource (no inline data assignment; host will inject data)
server.registerResource(
  "tictactoe-widget",
  "ui://widget/tictactoe.html",
  {},
  async () => ({
    contents: [
      {
        uri: "ui://widget/tictactoe.html",
        mimeType: "text/html+skybridge",
        text: `
  <div id="root"></div>
  <style>${TIC_TAC_TOE_CSS}</style>
  <script type="module">${TIC_TAC_TOE_JS}</script>
          `.trim(),
      },
    ],
  })
);

This looks complex at first glance, but it’s actually quite straightforward:

make sure you keep the same reference consistent for the widget (ui://widget/tictactoe.html)
the mime type must be text/html+skybridge to make sure that the window.openai object is properly passed to the widget
the text section defines the HTML that will be injected in the chat, and so the actual widget code consist of 2 files: CSS and javascript. More on that later

The sample apps coming with the SDK have a different mechanism to define the html part of the application, and defines the javascript and CSS by URLs:

<div id="pizzaz-root"></div>
<link rel="stylesheet" href="https://persistent.oaistatic.com/ecosystem-built-assets/pizzaz-0038.css">
<script type="module" src="https://persistent.oaistatic.com/ecosystem-built-assets/pizzaz-0038.js"></script>

That’s why the Pizza demo app’s widget code is immutable: the code has been uploaded to persistent.oaistatic.com, OpenAI’s static hosting server. This appears to be where all production widgets will eventually be hosted.

In our development, we read the javascript and css code from the file system and serve those inline with the text property of the MCP server:

text: `
  <div id="root"></div>
  <style>${TIC_TAC_TOE_CSS}</style>
  <script type="module">${TIC_TAC_TOE_JS}</script>
`.trim()

SSH tunnel

Before you start: local development requires setting up a secure tunnel so ChatGPT can reach your application over HTTPS. You can use ngrok, or try Cloudflare Tunnel if you prefer using your own domain, with a stable URL.

For the examples below, I’ll use https://tunnel.xxx.com as a placeholder for your tunnel URL.

MCP server communication with UI Widget

The response of a MCP tool indicate wether a UI widget should be displayed (see above).

The server response payload is like this:

return {
  content: [{ type: "text", text: "Displayed the tictactoe board!" }],
  structuredContent: boardState,
};

The model sees both content and structuredContent and can use those fields for reasoning or narration.

The UI widget receives the structuredContent in window.openai.toolOutput : this field should contain all the information needed by the widget to render itself accordingly.

For our TicTacToe app, the content of that field is an object of this type:

interface BoardState {
  positions: string[];
  version: string;
  isGameStarted: boolean;
  error: string;
  lastPlayer: "chatgpt" | "player" | "none";
}

For instance, positions contains the current state of the 9 cells of the board.

UI widget development

That’s definitely the tricky part during the development of the ChatGPT app.

The widget is a single page application (SPA) and the examples from openai use React and they provide a set of hooks to make the work easier, like useOpenAiGlobal, to access the value of the tool output.

For the demo, I went with a lighter web framework (Solid) to better understand the integration mechanism.

Whatever web framework you use, you need to find a way to extract, from all the files that the framework might be generating, the css and typescript files you need to expose from the MCP server.

The UI widget is rendered inside an iframe of chatGPT itself, and once the widget is rendered the value of the tool output is injected in the widget.

The widget can setup an event listener to be notified when the tool output is injected:

window.addEventListener("openai:set_globals", () => {
  refreshDebugInfo();
  checkStateFromToolOutput();
});

  const checkStateFromToolOutput = () => {
    try {
      const openai = (window as any).openai;
      const toolOutput = openai?.toolOutput;

      if (toolOutput && typeof toolOutput === "object") {
        const boardState = toolOutput as BoardState;
        // at this stage, the UI widget has access
        // to the BoardState provided by the MCP server

Note: the code above is not totally safe and you may consider using a validation library like zod to make sure that the payload received is what is expected by the widget.

Once the widget has the state it has all it need to render itself.

This document describes the different actions possible for the widget in the context of a ChatGPT application:

window.openai.setWidgetState: allow the widget to persist state across user sessions (like preferences)
window.openai.callTool: to call tools on the origin MCP server (need to authorize with tool registration _meta properties)
window.openai.sendFollowupMessage to insert a message into the conversation. That’s what we used in our TicTacToe app when the user selects a cell, and we tell the model to play.
there are other actions possible, and most probably more will come as the spec evolves.

TicTacToe application development

application setup in ChatGPT

The configuration is done from the settings menu in your ChatGPT interface:

You need first to make sure that you enabled the developer mode:

And then you can add your application:

The form is pretty straight forward,

In the form:

The MCP server URL is the URL of your public SSH tunnel URL
make sure you select “No authentication”, unless your app do support authentication

The next screen will show informations about your server:

use TicTacToe app in chatGPT

You can invoke the app with a prompt like:

As usual for a MCP server, chatGPT will ask the user to allow the MCP call

and then the board will be displayed:

During the game, the game will also ask the user to allow the MCP call for the move:

The “Allow for this conversation” will prevent ChatGPT to ask for permission at each move.

prompt

The prompt being used to ask ChatGPT to play is:

You are playing tic-tac-toe. You play X, I play O.
The current state of the board is:
O12
3X5
O78
The goal is to win the game. The winner is the one who has 3 in a row, column,
or diagonal.
If the board is full and there is no winner, it's a tie.
Important: Only play in positions that are not already taken by an X or an O.
Choose your move by calling the tool chatgpt_move with a position from 0-8 (where 0 is top-left, 1 is top-center, ..., 8 is bottom-right).

To be honest, ChatGPT doesn’t play that great, and it (he?) somewhat acknowledges it:

caching issues

ChatGPT is caching really aggressively, and don’t expect to have hot reloading while you are developing.

To be sure that you always are interacting with your latest version of your code:

disconnect and delete the app after each change. And then re-add it.
re-open a new chatGPT window to start a new session

debugging your MCP application

As an app is just a MCP server, you can use the MCP inspector to debug it:

npx @modelcontextprotocol/inspector

The web interface will allow you to confirm that all your tools behave as expected:

Useful links

The announcement post: Introducing apps in ChatGPT and the new Apps SDK
The openAI App SDK page: Apps SDK
The MCP Typescript sdk: https://github.com/modelcontextprotocol/typescript-sdk
The examples from OpenAI: https://github.com/openai/openai-apps-sdk-examples

Playing Tic Tac Toe with Chat GPT