The Chat API Has No Memory. You Do.
Every call is stateless. What looks like a conversation is a list of messages you resend each time.
The first time you use OpenAI's Chat Completions API expecting it to "remember" the conversation, you're in for a small surprise. It doesn't. Each call is completely stateless — the model has no memory of anything you sent before. Once that one fact clicks, the whole API stops being confusing and starts being obvious.
The shape of a request
A chat completion request is, at its core, a list of messages. Each message has a role and some content, and three roles do almost all the work.
The system message sets the behaviour — who the assistant is, how it should act, what it should and shouldn't do. The user messages are what the human said. The assistant messages are what the model said back. You send this list, the model reads the whole thing, and it generates the next assistant message.
That's the entire interface. Roles and content in, one more assistant message out. Everything fancy is built on top of this.
Where the "conversation" actually lives
Here's the part that trips people up. Because each call is stateless, a multi-turn conversation isn't stored on OpenAI's side between requests. It lives in your code. To continue a conversation you resend the entire history — every previous user and assistant message — plus the new user message, on every single call. The "memory" of the chat is a list you're maintaining and re-uploading each time.
The illusion is good enough that most people never notice. But it explains a lot: why longer conversations cost more (you're resending more history each turn), why the model "forgets" once you trim old messages to fit the context window, and why you — not the API — are responsible for what the model remembers.
Why statelessness is a feature
It sounds like a limitation, and occasionally it's inconvenient. But stateless is the right default for an API. Every request is self-contained and independent: you can retry it, run a thousand in parallel, cache it, or replay it, without worrying about hidden server-side state getting tangled. It also means you have total control over context — you decide exactly what the model sees each turn, which is precisely the control you want when you're building something real rather than a toy chat.
The mental model that helps
Stop thinking of the API as a chatbot with a memory and start thinking of it as a pure function: messages in, next message out, nothing remembered. Every capable thing built on this — assistants, agents, long conversations, retrieval — is just clever management of what goes into that messages list before you call it.
The API doesn't hold the conversation. It holds up a mirror to whatever list you hand it. Getting good with it is mostly getting good at deciding what belongs in that list.
