On this publish, we’ll introduce the Codex App Server; we’ll proportion our learnings up to now on the most productive tactics to deliver Codex’s features into your product to assist your customers supercharge their workflows. We’ll duvet the App Server’s structure and protocol and the way it integrates with other Codex surfaces, in addition to tips about leveraging Codex, whether or not you wish to have to show Codex right into a code reviewer, an SRE agent, or a coding assistant.
Ahead of diving into structure, it’s useful to grasp the App Server’s backstory. To start with, the App Server was once a sensible approach to reuse the Codex harness throughout merchandise that steadily advanced into our same old protocol.
Codex CLI began as a TUI (terminal consumer interface), that means Codex is accessed during the terminal. After we constructed the VS Code extension (a extra IDE-friendly approach to have interaction with Codex brokers), we would have liked some way to make use of the similar harness so that you could power the similar agent loop from an IDE UI with out re-implementing it. That intended supporting wealthy interplay patterns past request/reaction, corresponding to exploring the workspace, streaming development because the agent causes, and emitting diffs. We first experimented with exposing Codex as an MCP server(opens in a brand new window), however keeping up MCP semantics in some way that made sense for VS Code proved tough. As an alternative, we offered a JSON-RPC protocol that reflected the TUI loop, which changed into the unofficial first model(opens in a brand new window) of the App Server. On the time, we didn’t be expecting different shoppers to rely at the App Server, so it wasn’t designed as a solid API.
As Codex adoption grew over the following couple of months, inside groups and exterior companions sought after the facility to embed the similar harness in their very own merchandise to be able to boost up their customers’ tool building workflows. As an example, JetBrains and Xcode sought after an IDE-grade agent revel in, whilst the Codex desktop app had to orchestrate many Codex brokers in parallel. The ones calls for driven us to design a platform floor that each our merchandise and spouse integrations may safely rely on over the years. It had to be simple to combine and backward suitable, that means lets evolve the protocol with out breaking present shoppers.
Subsequent, we’ll stroll via how we designed the structure and protocol so other shoppers can use the similar harness.
First, let’s zoom in on what’s throughout the Codex harness and the way the Codex App Server exposes it to shoppers. In our final Codex weblog, we broke down the core agent loop that orchestrates the interplay between the consumer, the fashion, and the equipment. That is the core good judgment of the Codex harness, however there’s extra to the entire agent revel in:
1. Thread lifecycle and endurance. A thread is a Codex dialog between a consumer and an agent. Codex creates, resumes, forks, and archives threads, and persists the development historical past so shoppers can reconnect and render a constant timeline.
2. Config and auth. Codex quite a bit configuration, manages defaults, and runs authentication flows like “Check in with ChatGPT,” together with credential state.
3. Device execution and extensions. Codex executes shell/document equipment in a sandbox and wires up integrations like MCP servers and abilities so they may be able to take part within the agent loop underneath a constant coverage fashion.
The entire agent good judgment we discussed right here, together with the core agent loop, lives in part of the Codex CLI codebase referred to as “Codex core(opens in a brand new window).” Codex core is each a library the place the entire agent code lives and a runtime that may be spun as much as run the agent loop and set up the endurance of 1 Codex thread (dialog).
To be helpful, the Codex harness must be obtainable to shoppers. That’s the place the App Server is available in.
The App Server is each the JSON-RPC protocol between the shopper and the server and a long-lived procedure that hosts the Codex core threads. As we will be able to see from the diagram above, an App Server procedure has 4 primary elements: the stdio reader, the Codex message processor, the thread supervisor, and core threads. The thread supervisor spins up one core consultation for each and every thread, and the Codex message processor then communicates with each and every core consultation without delay to publish Jstomer requests and obtain updates.
One Jstomer request may end up in many tournament updates, and those detailed occasions are what let us construct a wealthy UI on best of the App Server. Moreover, the stdio reader and the Codex message processor function the interpretation layer between the shopper and Codex core threads. They translate Jstomer JSON-RPC requests into Codex core operations, concentrate to Codex core’s inside tournament move, after which develop into the ones low-level occasions right into a small set of solid, UI-ready JSON-RPC notifications.
The JSON-RPC protocol between the shopper and the App Server is totally bidirectional. A normal thread has a consumer request and plenty of server notifications. As well as, the server can begin requests when the agent wishes enter, like an approval, after which pause the flip till the shopper responds.
Subsequent, we’ll damage down the dialog primitives, the development blocks of the App Server protocol. Designing an API for an agent loop is hard for the reason that consumer/agent interplay isn’t a easy request/reaction. One consumer request can spread right into a structured collection of movements that the shopper must constitute faithfully: the consumer’s enter, the agent’s incremental development, artifacts produced alongside the way in which (e.g., diffs). To make that interplay move simple to combine and resilient throughout UIs, we landed on 3 core primitives with transparent limitations and lifecycles:
1. Merchandise: An merchandise is the atomic unit of enter/output in Codex. Pieces are typed (e.g., consumer message, agent message, device execution, approval request, diff) and each and every has an specific lifecycle:
merchandise/beganwhen the object starts- not obligatory
merchandise/*/deltaoccasions as content material streams in (for streaming merchandise sorts) merchandise/finishedwhen the object finalizes with its terminal payload
This lifecycle we could shoppers get started rendering instantly on began, move incremental updates on delta, and finalize on finished.
2. Flip: A flip is one unit of agent paintings initiated by way of consumer enter. It starts when the shopper submits an enter (for instance, “run exams and summarize screw ups”) and ends when the agent finishes generating outputs for that enter. A flip accommodates a series of things that constitute the intermediate steps and outputs produced alongside the way in which.
3. Thread: A thread is the sturdy container for an ongoing Codex consultation between a consumer and an agent. It accommodates more than one turns. Threads can also be created, resumed, forked, and archived. Thread historical past is persevered so shoppers can reconnect and render a constant timeline.
Now, we’ll have a look at a simplified dialog between a consumer and an agent, the place the dialog is represented by way of primitives:
At the start of the dialog, the shopper and the server want to identify the initialize handshake. The customer will have to ship a unmarried initialize request earlier than another manner, and the server recognizes with a reaction. This provides the server an opportunity to promote it features and we could either side agree on protocol versioning, function flags, and defaults earlier than the actual paintings starts. Right here’s an instance payload from OpenAI’s VS Code extension:
That is what the server returns:
When a consumer makes a brand new request, it’ll first create a thread after which a flip. The server will ship again notifications for development (thread/began and flip/began). It is going to additionally ship again inputs it registers as pieces, just like the consumer message right here.
Device calls also are despatched again to the shopper as pieces. Moreover, the server would possibly ask for Jstomer approval earlier than it could actually run an motion by way of sending a server request. The approval will pause the flip till the shopper replies with both “permit” or “deny.” That is what the approval waft seems like within the VS Code extension:
In spite of everything, the server sends an agent message after which ends the flip with flip/finished. The agent message delta occasions move items of the message again till the message is finalized with merchandise/finished.
The messages within the diagram are simplified for clarity. If you wish to see the JSON for a complete flip, you’ll be able to run the check Jstomer from the Codex CLI repo:
Now, let’s have a look at how other Jstomer surfaces embed Codex by way of the App Server. We’ll duvet 3 patterns: native apps and IDEs, Codex internet runtime, and the TUI.
Throughout all 3, the shipping is JSON-RPC over stdio (JSONL). JSON-RPC makes it simple to construct Jstomer bindings within the language of your selection. Codex surfaces and spouse integrations have carried out App Server shoppers in languages together with Pass, Python, TypeScript, Swift, and Kotlin. For TypeScript, you’ll be able to generate definitions without delay from the Rust protocol by way of operating:
For different languages, you’ll be able to generate a JSON Schema package and feed it into your most well-liked code generator by way of operating:
Native shoppers generally package or fetch a platform-specific App Server binary, release it as a long-running kid procedure, and stay a bidirectional stdio channel open for JSON-RPC. In our VS Code extension and Desktop App, for instance, the shipped artifact contains the platform-specific Codex binary and is pinned to a examined model so the shopper at all times runs the precise bits we validated.
No longer each and every integration can send Jstomer updates often. Some companions like Xcode decouple liberate cycles by way of preserving the shopper solid and permitting it to indicate to a more recent App Server binary when wanted. That means they may be able to undertake server-side enhancements (for instance, higher auto-compaction in Codex core or newly supported config keys) and roll out malicious program fixes with out looking forward to a consumer liberate. The App Server’s JSON-RPC floor is designed to be backward suitable, so older shoppers can communicate to more recent servers safely.
Codex Internet makes use of the Codex harness, however runs it in a container atmosphere. A employee provisions a container with the checked-out workspace, launches the App Server binary within it, and maintains a long-lived JSON-RPC over stdio2 channel. The internet app (operating within the consumer’s browser tab) talks to the Codex backend over HTTP and SSE, which streams process occasions produced by way of the employee. This helps to keep the browser-side UI light-weight whilst nonetheless giving us a constant runtime throughout desktop and internet.
As a result of internet periods are ephemeral (tabs shut, networks drop), the internet app can’t be the supply of fact for long-running duties. Retaining state and development at the server manner paintings continues even supposing the tab disappears. The streaming protocol and stored thread periods make it simple for a brand new consultation to reconnect, pick out up the place it left off, and catch up with out rebuilding state within the Jstomer.
Traditionally, the TUI was once a “local” Jstomer that ran in the similar procedure because the agent loop and talked without delay to Rust core sorts somewhat than the app-server protocol. That made early iteration speedy, nevertheless it additionally made the TUI a special-case floor.
Now that the App Server exists, we plan to refactor the TUI(opens in a brand new window) to make use of it so it behaves like another Jstomer: release an App Server kid procedure, discuss JSON-RPC over stdio, and render the similar streaming occasions and approvals. This unlocks workflows the place the TUI can connect with a Codex server operating on a faraway device, preserving the agent on the subject of compute and proceeding paintings even supposing the pc sleeps or disconnects, whilst nonetheless turning in reside updates and controls in the neighborhood.
Codex App Server would be the top quality integration manner we deal with shifting ahead, however there also are different strategies with extra restricted capability. By means of default, we’d counsel that shoppers use Codex App Server to combine with Codex, nevertheless it’s value looking at other integration strategies and figuring out their professionals and cons. Beneath are the commonest tactics to power Codex and when each and every could be a excellent are compatible.
Run codex mcp-server(opens in a brand new window) and fasten from any MCP Jstomer that helps stdio servers (e.g., OpenAI Brokers SDK(opens in a brand new window)). This can be a excellent are compatible if you have already got an MCP-based workflow and wish to invoke Codex as a callable device. The drawback is that you simply most effective get what MCP exposes, so Codex-specific interactions that depend on richer consultation semantics (e.g., diff updates) won’t map cleanly via MCP endpoints.
Some ecosystems be offering a conveyable interface that may goal more than one fashion suppliers and runtimes. This is a excellent are compatible if you wish to have one abstraction that coordinates more than one brokers. The tradeoff is that those protocols steadily converge at the commonplace subset of features, which may make richer interactions tougher to constitute, particularly when provider-specific device and consultation semantics topic. This area is evolving temporarily, and we think that extra commonplace requirements will emerge as we work out the most productive primitives to constitute real-world agent workflows (talents(opens in a brand new window) is a superb instance of this).
Select the App Server when you wish to have the entire Codex harness uncovered as a solid, UI-friendly tournament move. You get each the entire capability of the agent loop and different supporting options like Check in with ChatGPT, fashion discovery, and configuration control. The primary value is integration paintings, since you want to construct the client-side JSON-RPC binding on your language. In observe, on the other hand, Codex is in a position to do a large number of the heavy lifting in the event you feed it the JSON schema and documentation. Many groups we labored with had been in a position to make to a running integration temporarily the usage of Codex.
A light-weight, scriptable CLI mode for one-off duties and CI runs. It’s a excellent are compatible for automation and pipelines the place you wish to have a unmarried command to run to crowning glory non-interactively, move structured output for logs, and go out with a transparent luck or failure sign.
A TypeScript library for programmatically controlling native Codex brokers from inside of your personal software. It’s perfect when you wish to have a local library interface for server-side equipment and workflows with out development a separate JSON-RPC Jstomer. Because it shipped previous than the App Server, it lately helps fewer languages and a smaller floor space. If there may be developer hobby, we would possibly upload further SDKs that wrap the App Server protocol so groups can duvet extra of the harness floor with out writing JSON-RPC bindings.
On this publish, we shared how we way designing a brand new same old for interacting with brokers and tips on how to flip the Codex harness right into a solid, client-friendly protocol. We coated how the App Server exposes Codex core, we could shoppers power the entire agent loop, and powers quite a lot of surfaces together with the TUI, native IDE integrations, and the internet runtime.
If this sparked concepts for integrating Codex into your personal workflows, it’s value giving App Server a check out. The entire supply code lives within the Codex CLI open-source repo(opens in a brand new window). Be at liberty to proportion your comments and have requests. We’re excited to listen to from you and to stay making brokers extra obtainable to everybody.
