I spent a chunk of this spring watching MCP servers get popped in ways that were almost funny if they weren't running in production somewhere. A backdoored postmark-mcp package quietly BCC'ing outbound email to an attacker. A mcp-remote client that would shell out and execute arbitrary commands if a malicious server handed it a booby-trapped authorization_endpoint URL - CVSS 9.6, on a package with 437,000-plus downloads. Academic researchers pointed a scanner at 39,884 public MCP server repos in May and walked away with 106 zero-days. None of this was exotic. It was path traversal, command injection, and SSRF - the greatest hits of "we shipped fast and skipped input validation" - just wearing an agent-shaped hat.

Then, in July, the Model Context Protocol steering group shipped the 2026-07-28 specification, and it's the biggest revision since authorization was bolted on. The protocol drops its session handshake, moves to stateless requests, adds header-based routing, and hardens the auth flow with RFC 9207 issuer validation. The blog post frames it as a scalability story and it is one. But read it next to the CVE list from the preceding six months and a lot of the "why now" makes more sense. This wasn't a protocol maturing gracefully. This was a protocol getting hardened in response to the ecosystem finding out, publicly and repeatedly, how easy it was to break.

What MCP is, briefly

Model Context Protocol is Anthropic's open spec for connecting LLM applications to external tools, data sources, and systems : a JSON-RPC-based contract so a client (an IDE, an agent runtime, Claude Desktop) can discover and call tools exposed by a server (a database connector, a filesystem bridge, a SaaS integration) without every pairing needing custom glue code. It shipped on November 25, 2024, as something aimed squarely at individual developers wiring up their coding tools. Twenty months later it's the connective tissue for a meaningful slice of enterprise agent deployments, which is exactly the kind of success that exposes what a spec wasn't built for.

The security findings, and why they weren't a surprise to anyone paying attention

The numbers, once you line them up, describe a fairly specific failure mode. One 2026 scan of 2,614 MCP implementations found 82% had file operations prone to path traversal and 67% used code-injection-prone APIs; a separate scan of over 7,000 servers put command injection at 43% and SSRF at 36.7%. Different studies, different samples, same story. GitGuardian's 2026 secrets-sprawl scan found 24,008 unique credentials sitting in MCP-related config files on public GitHub, over 2,000 of them still live. Something like 40% of registered servers ship with no meaningful authentication at all.

None of that is protocol-specific carelessness - it's what you get when a huge number of people build servers that shell out, read files, and hit URLs based on model-generated input, and the ecosystem has no equivalent yet of "don't trust the client" muscle memory. MCP servers are, structurally, RPC endpoints that take instructions from something whose entire design goal is to be persuadable. Tool poisoning : hiding instructions inside a tool's description field so they land in the model's context as trusted text isn't a bug you patch, it's a consequence of treating tool metadata as safe just because it came from your own server's manifest. Nobody had to be clever to find these holes. Researchers pointed a scanner at 40,000 repos and the holes found themselves.

The mcp-remote RCE (CVE-2025-6514) is the case study worth sitting with, because it's not a misconfigured server - it's the client. A malicious MCP server hands back a crafted authorization_endpoint value during the OAuth flow, the client tries to open it, insufficient sanitization turns that into shell command execution. Versions 0.0.5 through 0.1.15 were exposed; the fix landed in 0.1.16 in June 2025. If your mental model of MCP risk was "the server is the attack surface, the client is trusted," this is the CVE that breaks that assumption. Both sides of the connection now have to assume the other is hostile, which is a genuinely different threat model than most of the early tutorials were written for.

What actually changed in the spec

The 2026-07-28 release, per the MCP blog, does four things that matter if you're building or operating servers right now.

It drops session state from the protocol. The initialize/initialized handshake and the Mcp-Session-Id header are gone. Every request now carries its own protocol version, client identity, and capabilities in _meta, instead of relying on a session negotiated once and held in server memory (or, more realistically, in some shared cache your ops team now owns). This is the load-balancer-friendly version of MCP - no sticky sessions, no shared state store required to scale horizontally. It's also, not incidentally, a security improvement: session hijacking and session-fixation-style attacks lose their foothold when there's no long-lived session artifact to steal.

It adds header-based routing. Mcp-Method and Mcp-Name now travel as HTTP headers, so gateways and rate limiters can route and enforce policy without parsing the JSON-RPC body on every request. That's a small detail with an outsized operational payoff - it's the difference between a gateway that can apply per-tool rate limits cheaply and one that has to deserialize every payload to figure out what's even being called.

It formalizes an extensions framework, and moves Tasks - the mechanism for long-running operations - out of core and into io.modelcontextprotocol/tasks, replacing server-initiated streaming with poll-based tasks/get/tasks/update. Multi Round-Trip Requests replace some of what server-initiated requests used to do: a tool can now return resultType: "input_required" mid-call and get the missing piece back on a follow-up request, instead of the server having to hold a stream open and push at the client.

// old world: server holds a stream open, pushes a follow-up request
// new world: tool returns and waits, client re-calls with the missing piece
{
  "resultType": "input_required",
  "prompt": "Which environment: staging or prod?",
  "continuationToken": "tk_9f2a..."
}

It hardens authorization. RFC 9207 issuer validation, client credentials bound to a specific issuer, and a push from Dynamic Client Registration toward Client ID Metadata Documents. This is the part that reads most directly as a response to the CVE season - DCR was flexible and, per the security writeups, part of why so many servers ended up with weak or absent auth in the first place. CIMD narrows that.

Roots, Sampling, and Logging are deprecated with a twelve-month runway. The old HTTP+SSE transport is on the same clock. If you've got a custom implementation rather than one of the Tier 1 SDKs (TypeScript, Python, Go, C#, with Rust in beta), David Soria Parra's own framing is blunt: "it's going to be a lot of uplift to make this correct." Servers on 2026-07-28 won't necessarily talk to older clients. That's a real migration, not a minor-version bump.

The honest trade-offs

Going stateless solves a scaling problem and closes a session-hijacking class of attack, and it does not touch tool poisoning, SSRF, or a server that shells out to subprocess.run() with unsanitized model output. The spec changes harden the pipes; they don't harden what runs inside the pipes. Read the OWASP MCP Top 10 findings and the honest conclusion is that network allowlists, gateway-level auth, and runtime content inspection are three separate, necessary layers, and none of them is sufficient alone — the protocol update is one input to that stack, not a replacement for it.

There's also a real cost on the builder side. Twelve months sounds generous until you remember how much production MCP tooling was written against the stateful assumptions of the original spec - code that holds a session, streams server-initiated requests, expects Mcp-Session-Id to exist. Craig McLuckie's read -that MCP's statefulness was "a by-product of its origin as a way to support developers using coding tools" and enterprises are now bending it toward production intermediation is accurate, and it's also why this migration will be uneven. Individual developers wiring up a local tool barely notice. Teams running MCP gateways in front of internal systems are looking at real engineering work.

Why this matters right now

MCP crossed the threshold from "convenient way to give Claude Desktop filesystem access" to "the thing standing between agents and production systems" faster than its security posture kept up, and 2026 is the year that gap got measured in CVE counts instead of hand-wringing blog posts. The interesting thing isn't that a young protocol had security problems - every young protocol does. It's that the fix showed up in the spec itself, not just in a wave of after-the-fact server patches. Removing session state, tightening the OAuth flow, forcing routing metadata into headers - these are the kinds of changes that come from people who watched real incidents and decided the protocol needed to make the safe thing easier, not just possible.

If you're running MCP servers in anything resembling production, the 2026-07-28 spec isn't optional homework. But treat it as half the job. The other half is the boring part nobody writes an RFC for: sanitize your inputs, pin your dependencies, don't trust tool descriptions as inert text, and assume the thing on the other end of the connection - client or server - is trying to see what it can get away with. The protocol just got better at its job. Yours didn't change.

Sources: MCP 2026-07-28 Specification · The Register on MCP's stateless shift · State of MCP Security 2026, PipeLab · CVE-2025-6514, mcp-remote RCE, SentinelOne

That workflow has existed for over a decade. Graphite built a company on bringing it to GitHub, because GitHub itself never had it. Aviator did too. Sapling and git-branchless gave you a rougher, free version if you were willing to fight your own tooling a little. And on July 30, 2026, GitHub shipped stacked pull requests natively, in public preview, to every repository, for free, with a CLI extension called gh-stack and a skill package so Copilot and other agents can drive the workflow directly.

The interesting question isn't "does this work." It mostly does. The interesting question is what it means for a platform to absorb a pattern that an entire category of startups was built to sell, and whether the version that ships for free to a billion commits a year is actually the same thing as the version that a small, opinionated team spent years refining.

What actually ships

A stack, in GitHub's implementation, is an ordered chain of branches where the bottom one targets your trunk and each branch above targets the branch below it, not main directly. gh stack init starts the chain. gh stack add <branch> layers on the next piece of work. You write code and commit inside each layer like you always have. When you push, GitHub renders a stack map at the top of every PR in the chain, so a reviewer looking at layer three can see what layers one and two changed without leaving the page.

The mechanically satisfying part is the cascading operations. gh stack rebase rebases every branch onto the one below it in sequence, starting from main, and it turns on git rerere automatically so a conflict you resolve once doesn't have to be resolved again as it propagates upward. Merge the top ready PR and everything beneath it lands in one operation. Merge a partial stack, just the bottom layers, and the ones still open above automatically retarget and rebase onto the new base. None of this required custom tooling before. It required discipline, a lot of git rebase --onto, and the kind of muscle memory that only comes from doing it badly a few dozen times first.

Existing branch protections and required checks apply unchanged to every layer. That's a deliberate, sensible choice: GitHub isn't asking teams to trust a new trust model, it's applying the old one to a new shape of work.

The argument nobody quite settles

The Hacker News thread on the public preview launch ran to 516 points and 282 comments, and the split in it is worth sitting with, because it's not a split about execution quality. It's a split about whether the premise is coherent.

One side: "stacked diffs have existed at Meta for a decade, glad GitHub is joining." Vindication for a pattern that felt, for years, like something only large engineering orgs with custom internal tools got to have.

The other side, sharper: either your changes are independent, in which case you should just open separate PRs against main, or they're dependent, in which case reviewing them separately doesn't actually make sense, because a reviewer approving layer two is implicitly approving the assumptions baked into layer one, whether they've re-read it or not.

I don't think that second position is wrong. I think it's identifying a real seam in the whole pattern, one that Phabricator veterans have always had an answer for, which is: dependence isn't binary. Layer two can depend on layer one's interface without depending on its implementation. A reviewer can meaningfully approve "this login form correctly calls the auth function that layer one added" without re-litigating whether layer one's auth function is itself well written, because that was already reviewed on its own PR. The stack doesn't eliminate the dependency. It makes the dependency explicit and lets you review each side of it at the resolution where a human can actually hold it in their head. A 900-line PR asks a reviewer to hold interface and implementation and five unrelated concerns all at once. A stack asks them to hold one thing at a time, in order.

That's a real answer. It's just not a free one, and the "either/or" critique is right that it costs something: reviewer trust has to extend downward through the stack, and if a lower layer turns out to be wrong after the layer above it already merged, you're now unwinding two changes instead of one, with a live dependency between them.

The bug that undercuts the actual selling point

Here's the part that matters more than the philosophical argument, because it's not theoretical. A GitHub discussion thread on the gh-stack repository (issue 212, if you want to go read it yourself) documents that merging an entire stack in one operation breaks in a meaningful set of cases. The workaround is merging PRs one at a time, and if your repo requires reviews and uses squash-and-merge, each individual merge in that sequence can trigger a re-approval requirement on the PRs above it.

Sit with what that means. The single biggest advertised benefit of stacked PRs is "merge everything in one operation instead of doing it layer by layer with manual rebasing in between." If squash-merge repos with required reviews end up back in a one-at-a-time merge flow that re-triggers approvals, you have kept most of the review-splitting benefit and lost a meaningful chunk of the merge-automation benefit that was supposed to be the other half of the pitch. It's not a dealbreaker. It's a public-preview rough edge, and GitHub will likely tighten it. But it's a real gap between the changelog post and the current lived experience, and the byteiota writeup on this launch flagged the same category of issue independently: squash-merge and cascading rebase conflicts are exactly the unglamorous plumbing that Graphite has had years to harden, and a public preview hasn't had years yet.

Why the timing isn't a coincidence

TED's CTO, Andy Merryman, put the connection directly: AI made TED's developers dramatically more productive, and that productivity created a new bottleneck, because PRs were growing large enough that reviewers were struggling to keep up. Stacked PRs, in his framing, don't just make review faster, they make it more accurate, because dependency-ordered pieces let a reviewer evaluate each piece at the resolution it deserves instead of skimming a diff that outgrew what one person can hold in working memory.

That's the real thesis, and it's a good one, but it assumes the thing being split is human-shaped work that happens to be large. A lot of what's filling review queues right now isn't. It's agent output, generated at a pace and in a style no human authored, then handed over to be verified. Splitting a 900-line agent-generated PR into five 150-line agent-generated PRs doesn't make any of them more legible if the reviewer's actual problem was never diff size, it was unfamiliarity with logic they didn't write and don't have a mental model for. Stacking helps enormously when the bottleneck is "this is too much to hold in my head at once." It helps much less when the bottleneck is "I don't trust what generated this."

What this does to Graphite, and what it doesn't

GitHub shipping this natively, for free, on every repository, is a genuine threat to Graphite and Aviator's core pitch, in the same way Actions eventually made a lot of standalone CI platforms a much harder sell to a team already paying for GitHub. "Good enough and native" beats "better but external" for a large share of teams, especially ones that were never going to pay for a dedicated stacking tool in the first place and were just living with manual rebase pain instead.

But Graphite didn't spend years on this because the core idea is hard. It spent years on the edges: the CLI ergonomics, the automation around restacking after upstream changes, the visualization, the parts that don't show up in a changelog post but show up in a team's daily friction once they're doing this at scale. GitHub's version is genuinely good for a public preview. It is not, yet, obviously better than what a team already deep in Graphite's workflow has. The teams most likely to switch are the ones who never adopted a third-party tool at all, and those teams are exactly the ones GitHub's free, native version was built for.

Where I land

Stacked PRs are a real, overdue upgrade, not a magic fix. GitHub finally caught up to something Meta engineers have known for a decade, and the mechanics are good enough to use today. But a five-PR stack is not five independently-verified units of trust, and a smaller diff doesn't make an unfamiliar one legible. Those are different problems wearing the same symptom, and no chunking strategy, however well built, tells them apart for you.

I've thought about that sentence more than I expected to. Because it quietly dismantles the thing everyone reaches for when someone asks "but is it safe to let the agent do that?" The answer is always the same. We'll keep a human in the loop. Said with confidence, like it settles the matter.

It doesn't settle anything. It just moves the failure somewhere harder to see.

What "human in the loop" is supposed to do

The premise is reasonable. You have a system that's right most of the time but not all of the time, and the failures are expensive. So you put a person at the decision point. The model proposes, the human disposes. The agent drafts the refund, moves the money, merges the PR, changes the infra. A human signs off before it lands.

On a whiteboard this is airtight. The machine handles volume. The human handles judgment. The rare catastrophic case gets caught by the one component in the system capable of understanding context. Two failure modes, two very different mechanisms. Defence in depth.

The whole design rests on one assumption: that the human in the loop is actually looking. Not present. Looking. And that assumption is exactly where it falls apart, because it assumes the one behaviour the rest of the system is busy destroying.

The reliability paradox

Here's the part nobody wants on the slide. The better your agent gets, the worse your reviewer gets. Those aren't two trends. They're the same curve read from opposite ends.

This isn't a hot take. It's decades old. Raja Parasuraman and Dietrich Manzey laid it out in their 2010 review, Complacency and Bias in Human Use of Automation. Automation complacency shows up reliably when a person is juggling multiple tasks and one of them is "watch the reliable machine." Two findings sting. It appears in experts, not just novices. And it doesn't wash out with practice. You cannot train your way to permanent vigilance against a system that's usually right. The reliability is the thing sedating you.

Aviation has known this forever. It's why they don't just tell pilots to "stay alert." Alertness isn't a policy you can declare. It decays. A human watching a system that's correct 999 times out of 1000 will, by the thousandth event, have been conditioned by 999 confirmations that watching is a waste of time. And they'd be statistically correct to feel that way. That's the trap. The rational response to a highly reliable system is to stop checking it, which is precisely when it needs you.

There's a name for what your reviewer becomes at that point: automation bias. It produces two failures, not one. Omission, where the system stays quiet when it should have flagged something and so does the human. Commission, where the system gives bad advice and the human follows it over their own correct judgment. That second one should bother you more than it does.

The numbers are worse than the vibe

I don't want to argue this on intuition alone, because intuition is exactly what automation bias corrupts.

A 2012 systematic review of clinical decision-support systems found that when the system gave wrong advice, it raised the odds of the clinician making an incorrect decision by 26 percent. Risk ratio of 1.26. Read that again. These are trained clinicians reversing their own correct answers because a machine disagreed. Across the studies, somewhere between 6 and 11 percent of correct human decisions got flipped to wrong ones after the system chimed in. That's not the system failing. That's the oversight layer failing, in the exact direction it was installed to prevent.

Software has its own version, and it's newer. Stanford researchers found that developers using an AI coding assistant wrote less secure code, and were more confident it was secure. That combination is the real problem. When the output looks clean and structured, you review it less. The NYU study on Copilot's early output put roughly 40 percent of suggestions in security-relevant contexts as containing vulnerabilities. Not because the tool was bad. Because polished-looking wrongness is the hardest kind to catch, and a tool that's right most of the time makes every output look like the times it was right.

Anyone who's approved a stack of agent-generated PRs knows the feeling in their gut. The fifth one gets less scrutiny than the first. The twentieth gets a glance and a merge. You tell yourself you reviewed it. You didn't. You pattern-matched it against the last nineteen that were fine.

The part that should actually worry you

So the loop degrades into a rubber stamp. Annoying, fixable, you'd think. But there's a second thing happening, and it's structural, not cognitive.

Madeleine Clare Elish has a phrase for it: the moral crumple zone. In a car, the crumple zone absorbs the crash to protect the human. In an automated system, the human absorbs the blame to protect the system. When the agent does something catastrophic and a person clicked approve somewhere upstream, the accountability doesn't attach to the model, the training data, the eval gap, or the org that shipped it under-tested. It attaches to the nearest human who touched it. They become, in Elish's framing, a liability sponge.

This is the quiet reason "human in the loop" is so popular in decks and compliance docs. It isn't always there to catch errors. Sometimes it's there to have someone to blame when the errors aren't caught. That's a use, I suppose. It just isn't oversight. It's a name on a form, positioned to take the hit. If your loop can't slow the system down, can't say no without a fight, and gets blamed when the thing it was never empowered to stop goes wrong, that's not a safeguard. That's a designated fall guy with a keyboard.

And regulators are about to make this everyone's problem. The EU AI Act mandates human oversight for high-risk systems under Article 14, with enforcement for high-risk obligations landing in August 2026. A lot of teams are going to satisfy that clause by bolting an approve button onto an agent and calling it oversight. The letter of the law, met. The spirit, a human with the authority, time, expertise, and independence to actually push back, nowhere in sight.

So what actually works

I won't pretend I've fully solved this in my own systems. I haven't. But the failed approaches share a shape, and so do the ones that hold up.

What doesn't work is making the human review everything. That's the fast road to the rubber stamp. Volume is the enemy of vigilance. If your reviewer sees a hundred approvals a day, ninety-nine of them fine, you have not built oversight. You've built a clicking exercise with a human-shaped bottleneck, and you've trained that human to click.

What seems to actually help, roughly in order of how much it moves the needle:

Make the human sample, not screen. A reviewer who checks a well-chosen 5 percent carefully catches more than one who "checks" 100 percent at a glance. Vigilance is a fixed budget. Spend it deep, not wide. Randomised deep audits also break the pattern-matching, because you can't coast if you don't know which one is graded.

Route by uncertainty and blast radius, not by default. The agent should escalate the cases where it's unsure or where the downside is large, and quietly handle the boring middle. A human forced to look at obvious approvals stops looking by the time the dangerous one arrives. Reserve human attention for where it's actually differential.

Make "no" cheap and real. If pausing the system costs the reviewer a meeting with their skip-level, they will not pause it. Oversight that's socially expensive to exercise isn't oversight. This is an org-design problem wearing a UI costume, and no amount of front-end polish fixes it.

Give the human something the machine didn't generate. Independent context, a second signal, the raw inputs. Not just the model's confident summary of its own work. Asking someone to check an agent using only what the agent chose to show them is theatre. They're grading the essay with the answer key the student wrote.

The pattern underneath all of it: aviation didn't fix cockpit complacency by telling pilots to try harder. It built Crew Resource Management. Structure, training, simulated failures, and an explicit culture where the junior officer is expected to challenge the captain. They turned oversight from a personality trait into an operational discipline. That's the bar. Anything less, and "keep a human in the loop" is a sentence you say to feel better, not a control that does anything.

The line I keep coming back to

The loop was never the safeguard. The human's attention was the safeguard. And attention is the one resource a reliable system is guaranteed to consume. You built something good enough to make watching it feel pointless, then stationed a person to watch it. Of course they stopped watching. You designed the incentive that way.

So before you write "human in the loop" in your next design doc, ask the only question that matters. What would it cost that human, right now, today, to say no? If the answer is "nothing, they'd click the other button and everyone would thank them," good. You might have oversight. If the answer is anything else, you've got a rubber stamp, and you've already decided whose name goes on the incident report.

Approve, approve, approve. At some point that isn't judgment. It's just latency with a person attached.

I didn't wake up one day with a sudden passion for Python. There was no epiphany. No conference talk that changed my worldview. No senior engineer pulling me aside and saying, "You really need to learn this." I spent thirteen years in Java writing services, loving Spring Framework, arguing about architecture, forming strong opinions about checked exceptions and Python lived somewhere in my peripheral vision, vaguely acknowledged the way you acknowledge a fire exit. Good to know it's there. Haven't really needed it yet.

Then, gradually and then all at once, I needed it.

What thirteen years of Java actually gives you

Before I get into the Python side of this, let me be clear about what kind of engineer I am, because it matters for everything I'm going to write in this series.

Thirteen years in Java is not thirteen years of writing for loops. It's thirteen years of thinking about systems - how they fail, how they scale, how they communicate, how they rot quietly from the inside when no one's paying attention. It's learning to read a stack trace like a story. It's developing an instinct for what "this will hurt later" looks like before it hurts.

Java trained me to be explicit. Name your types. Declare your intentions. If it compiles, you've at least done the minimum. If it doesn't compile, the machine is doing you a favor. That discipline is real, and I don't intend to throw it away.

But discipline has a cost. Java asks a lot of ceremony for things that feel like they should be simple. You learn to stop noticing. Then you see someone solve the same problem in eight lines of Python, and you notice again.

Why I didn't learn Python sooner

The honest answer: I didn't need to. Java was sufficient for everything I was building. The Java ecosystem is enormous, the tooling is mature, the job market was strong, and the cognitive overhead of learning a dynamically typed language felt like a bad trade when I was already productive in a statically typed one.

There's also a specific kind of Java developer stubbornness that I'll admit to. You've seen Python code that's impressively concise, but you've also seen Python code that looks like it was written by someone who genuinely believes that naming a variable x is acceptable in a production codebase. And you think: I have a type system. I have an IDE that knows what everything is. I have a language that refuses to let me make certain categories of mistakes. Why would I trade that for flexibility?

That reasoning isn't wrong. It's just incomplete.

What changed

Three things happened at roughly the same time, and together they were enough.

The AI shift. I work in software engineering. Increasingly, the most interesting and consequential work involves machine learning, data pipelines, LLM integrations, automation tooling, and infrastructure scripting. Almost all of it is written in Python. Not because Python is the objectively correct tool for every part of that stack, but because that's where the ecosystem lives - the libraries, the frameworks, the research code, the examples. If you want to work at the intersection of software engineering and AI, Python is not optional. It's the lingua franca.

The scripting tax. Every time I needed to write something small - a data transformation, a quick API client, a file processing script - I had the choice of doing it properly in Java (which meant a project structure, a build file, more ceremony than the task deserved) or doing it badly in bash (which meant something unmaintainable that I'd delete and rewrite every six months). I kept seeing Python used as a serious scripting language that didn't apologize for being productive. Eventually, "I don't know Python well enough" became an embarrassing excuse.

The intellectual honesty. I've been an engineer long enough to know that the languages you're good at shape how you think about problems. That's mostly an advantage. But it can quietly become a limitation. I started reading Fluent Python partly out of curiosity and found myself genuinely surprised not by syntax, but by ideas. Python has a data model, a philosophy, design decisions I hadn't considered before. If I'm going to have opinions about programming languages, I should probably understand more than one of them deeply.

What Java versus Python actually feels like from here

I'm not going to pretend I've unlocked Python. I haven't. I'm learning it properly not just "I can write a script" learning, but "I understand why this is the way it is" learning.

What I can tell you is that the contrast is more interesting than I expected.

Java gives you a contract. Every function signature is a promise: here are the inputs, here is the output type, here is what can go wrong. You can hand a Java codebase to another developer and they can navigate it largely through types and structure. The language itself is doing a significant amount of documentation work.

Python gives you trust. It assumes you know what you're doing. It doesn't force you to say much, and what you do say tends to be elegant. The tradeoff is that you can also say very little and mean something completely different from what you thought. Dynamic typing is expressive right up until it isn't.

Neither of these is simply better. They reflect different philosophies about where responsibility lives in the language or in the programmer. After thirteen years on one side of that line, I find the other side genuinely interesting to think about.

Why this matters beyond "learning a new language"

I want to resist the framing that this is just about adding Python to my resume. It's not.

Learning a language well not just syntactically, but philosophically changes how you think about the ones you already know. I already catch myself looking at Java code differently after a few months of Python. Certain things that felt necessary now feel like ceremony. Certain things I took for granted in Java now feel like genuine design decisions I didn't appreciate enough.

There's also something worth acknowledging about what it means to be a senior engineer learning something from scratch. You're not a beginner in the intellectual sense you bring years of engineering judgment, and you'll learn faster and think more critically than you would have at the start of your career. But you are a beginner in the practical sense, and you have to be honest about that. The muscle memory isn't there. The idioms aren't instinctive yet. Some things that feel wrong just take time to feel right.

That gap between what you know as an engineer and what you're still learning as a Python developer is exactly what this series is about.

What I'm hoping to figure out

I don't have a neat conclusion for this post, because I don't have a neat conclusion to the experience yet. I'm in the middle of it.

What I do know is that I'm taking this seriously not as a weekend project, not as a "learn enough Python to paste LLM outputs," but as a genuine attempt to understand the language the way I understand Java. That means reading Fluent Python cover to cover. That means writing real code, not toy examples. That means being willing to be wrong, to be surprised, and to occasionally admit that Python got something right that Java made harder than it needed to be.

It also means writing about it honestly, which is harder than it sounds when you've spent over a decade building an identity around a different set of tools.

Python feels, so far, like a language with a lot to teach me not because I'm behind, but because it's genuinely different in ways that matter. The flexibility that used to make me nervous is starting to look like a trade-off worth understanding, not a warning sign to avoid.

I didn't come to Python because the grass is greener. I came because the industry changed, because the problems I care about increasingly live here, and because thirteen years of engineering judgment is most useful when it's applied to more than one language.

The learning is real. The skepticism is real. The curiosity is, somewhat to my own surprise, also real.

That's enough to start.

I read it twice, and I'm still sitting with an interesting mix of reactions.

Let me say what's genuinely surprising about this: OpenAI is, by most measures, one of the primary engines of the very disruption it's now proposing to cushion. The company that shipped GPT-5.4 : a model that can outperform humans at desktop computer use, that scores 83% on a benchmark spanning 44 knowledge work occupations - is the same company now floating the idea that businesses should pay taxes on the automated systems replacing workers, comparable to what they'd pay in labor costs for the humans those systems replaced.

That is not a small thing to say out loud if you're OpenAI.

The public wealth fund proposal is the most intriguing piece. The model is explicitly Alaska's Permanent Fund: a nationally managed pool seeded in part by contributions from AI companies, which invests in the technology sector and distributes returns directly to citizens. Not a UBI exactly, but a dividend from the AI economy, a mechanism for ensuring that the gains from productivity don't pool entirely at the top. There's something philosophically coherent about this. If AI is genuinely going to be as transformative as its proponents believe, then the question of who owns the productivity gains isn't academic.

The four-day workweek proposal is softer. OpenAI suggests that employers and unions run time-limited pilots of a 32-hour week at full pay. "If output and service levels hold, consider making it permanent or converting reclaimed hours into paid time off." It's a reasonable, cautious suggestion, and also one that OpenAI doesn't have to implement itself.

Which brings me to the uncomfortable part.

Policy proposals from AI companies are cheap. Zuckerberg wrote op-eds about digital dignity. Google published manifestos on responsible AI. These documents often serve, whether intentionally or not, as narrative management: a way of appearing thoughtful about consequences while the products continue to ship. OpenAI's document doesn't propose binding commitments. It suggests that governments and businesses consider these ideas. It's a vision statement, not a plan.

And yet I don't want to be too cynical. Someone had to say it first. The fact that the company building the infrastructure of AI-driven automation is now publicly endorsing robot taxes, however toothlessly, shifts the conversation. It makes these ideas slightly more mainstream. Economists and labor advocates have been making similar arguments for years; having OpenAI cosign them in a policy document carries different weight than a think tank white paper.

Whether any of it becomes policy is another matter entirely. But the fact that we're here — at the point where the makers of AI are writing papers about how to tax it — says something real about where this moment is heading.

Sources:

• OpenAI calls for robot taxes, a public wealth fund, and a four-day week — The Next Web

• OpenAI's vision for the AI economy: public wealth funds, robot taxes, and a four-day workweek — TechCrunch

• OpenAI proposes robot taxes and 4-day workweek as AI threatens jobs — BetaNews

• OpenAI proposes AI driven economic change including robot taxes, public wealth funds and a four day work week — The Tech Portal

• Four-day week, taxes on robots, public wealth fund: OpenAI floats policy ideas for the intelligence age — Startup News

Google Research published TurboQuant in late March 2026, and it formally lands at ICLR 2026 in Rio de Janeiro on April 25th. The core idea is deceptively elegant: compress the KV cache in large language models down to just 3 bits per value, cutting memory usage by 6x — without any retraining, without any calibration data, and without losing accuracy.

A small primer, because this stuff is genuinely interesting: when you send a message to a large language model, the model generates what's called a Key-Value cache as it processes context. Think of the KV cache as the model's working memory, it holds information about everything in the conversation so far, letting the model be coherent across long exchanges. As context windows have gotten longer (GPT-5.4 now supports one million tokens), these caches have ballooned into one of the biggest bottlenecks in AI infrastructure. Data centers run out of GPU memory not because the models themselves are too big, but because the caches are enormous.

TurboQuant attacks this problem with a two-step approach. First, it applies something called PolarQuant, a random rotation of the data vectors that simplifies their geometric structure, making them much easier to compress efficiently. Then it runs standard quantization on the rotated data. The result: 3.5-bit compression that matches full-precision performance on every standard benchmark, and a 4-bit variant that's actually 8x faster than 32-bit keys on H100 hardware for the attention computation step.

What this means in practice is remarkable. On the same physical hardware, TurboQuant enables models to handle 4 to 8 times longer context windows or, alternatively, to run far larger batch sizes, meaning more simultaneous users at lower cost. And because it requires no retraining and works on any transformer architecture, it can simply be dropped into existing deployments.

The downstream effects here are worth thinking about carefully. If the memory bottleneck in LLM inference shrinks by 6x, the economics of running these models shift significantly. Cloud providers can serve more users per GPU. Self-hosted deployments become more viable on consumer hardware. Startups that couldn't afford the infrastructure to run a large model with a million-token context window suddenly can. Some analysts at TrendForce have noted that TurboQuant could even dampen demand growth for high-bandwidth memory chips, which has already rattled a few corners of the semiconductor market.

This is the kind of research breakthrough that doesn't make front-page news but quietly makes everything downstream cheaper and more accessible. The model capabilities get the headlines. The efficiency gains are what let people actually use them.

Sources:

• Google's TurboQuant: 6x Less Memory for LLM Inference — Nerd Level Tech

• TurboQuant: Redefining AI efficiency with extreme compression — Google Research Blog

• Decoding Google's TurboQuant: 6x KV Cache Cut — TrendForce

• Google TurboQuant: 6x KV Cache Compression for LLM Inference — Spheron Blog

That's what Anthropic did on April 7th.

The company quietly disclosed that its next-generation frontier model - Claude Mythos Preview - has, in recent weeks, identified thousands of previously unknown zero-day vulnerabilities across every major operating system, every major web browser, and a range of other widely-deployed software. Not theoretical vulnerabilities. Real, exploitable ones, including a 17-year-old remote code execution flaw in FreeBSD that allows a remote attacker to gain root access to any machine running NFS. Anthropic found it. Autonomously. The CVE is now logged as CVE-2026-4747.

The company's conclusion: Mythos Preview is too dangerous to release to the public. So instead of a general rollout, they built Project Glasswing.

Glasswing is a controlled-access security collaboration that gives a small group of heavyweight partners - Amazon Web Services, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorgan Chase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks - restricted access to Mythos specifically for patching critical software. The goal is to use the model's superhuman vulnerability-finding abilities defensively: let it find the holes before malicious actors do, and get them fixed before anyone else knows to look.

It's a genuinely interesting approach to a genuinely hard problem, and I find myself of two minds about it.

On one hand, this is responsible disclosure at a civilizational scale. Anthropic found a model that could be a hacker's dream weapon, and rather than release it and deal with the fallout, they built a framework to use it carefully. The partners list reads like a who's who of the internet's critical infrastructure. That's not nothing.

On the other hand, we're now in a world where a private company has decided, unilaterally, which organizations get access to a capability that could reshape the security landscape. Project Glasswing was not voted on. There was no regulatory consultation. Amazon and Apple get in; your regional bank, your city's water utility, your local hospital network do not.

Anthropic says the capabilities "emerged surprisingly quickly" during development — which is perhaps the most unsettling sentence in the entire announcement. They didn't set out to build the world's most dangerous vulnerability scanner. It just became one.

That's the thing about frontier AI development in 2026: the capabilities keep surprising even the people doing the building. Glasswing feels like a reasonable response to an unreasonable situation. Whether it scales — whether a controlled-access model can patch a world's worth of software before attackers figure out what's happening — is a different question entirely.

Sources:

• Anthropic's Claude Mythos Finds Thousands of Zero-Day Flaws Across Major Systems — The Hacker News

• Anthropic debuts preview of powerful new AI model Mythos in new cybersecurity initiative — TechCrunch

• Anthropic is giving some firms early access to Claude Mythos to bolster cybersecurity defenses — Fortune

• Anthropic says its most powerful AI cyber model is too dangerous to release publicly — VentureBeat

• Project Glasswing: Securing critical software for the AI era — Anthropic

On the morning of February 5, 2026, OpenAI and Anthropic were reportedly set to release their new agentic coding models at the same time: 10 a.m. Pacific. On paper, it sounded almost polite, like two rivals agreeing to meet at the starting line together. But Anthropic moved first. About fifteen minutes before the scheduled time, it pushed its launch live.

That small move said a lot. In an industry obsessed with speed, perception, and momentum, even fifteen minutes can feel symbolic. This was not just two companies launching products. It felt like two heavyweight competitors reminding the world that they are watching each other very closely.

And to be fair, both launches were serious.

Anthropic introduced Claude Opus 4.6, with a one-million-token context window, a major jump from 200,000. In simple terms, that means the model can keep far more information in view at once, making it better suited for large codebases, long documents, and complex workflows.

OpenAI answered with GPT-5.4, which reportedly scored 75% on OSWorld-V, a benchmark meant to test how well models handle realistic desktop productivity tasks. That figure sits above the 72.4% human baseline, which is exactly the kind of stat designed to make people stop and pay attention. Around the same period, GPT-5.3 Codex was also drawing notice because OpenAI engineers had reportedly used earlier versions of it to help debug and evaluate the model during development.

All of that is impressive. But the real story is not who launched first or whose benchmark number looked better on the day.

The real story is that these systems are no longer just chatbots.

That word gets used too loosely now, but this shift is worth taking seriously. These newer models are increasingly being positioned as agents, which means they can do more than respond to prompts. They can carry out tasks across software environments, handle multi-step workflows, make intermediate decisions, and keep moving without needing constant human direction. They are not just there to answer. They are there to act.

That is a much bigger change than another model release.

Anthropic’s rollout makes that especially clear. By bringing Opus into tools like Microsoft PowerPoint and Excel, it signaled something important: AI is no longer being framed as a separate assistant sitting in a chat window. It is being placed directly into everyday work software, where people already spend their time. That makes the technology feel less like an experiment and more like a colleague woven into normal workflows.

And that is why this competition matters.

Both OpenAI and Anthropic understand that the company that earns trust in agentic workflows now could end up deeply embedded in the way businesses operate for years. This is no longer only about model quality. It is about becoming part of how teams write, build, plan, analyse, and execute work.

OpenAI’s “Skills” feature points in that direction. The idea is to let ChatGPT reuse and apply repeatable workflows automatically, which starts to look a lot like institutional memory built into the system. Anthropic has been pushing in a similar direction with persistent memory, allowing Claude to remember preferences and context across sessions.

These are not small product updates. They are early signs of a deeper shift in how software may work in the near future.

A year from now, hardly anyone will care which company launched fifteen minutes earlier.

But they may care a lot about what this moment represented.

Because the real signal here is not about bragging rights. It is that we may have crossed into a new phase of AI, one where the tools are no longer just assisting with work.

They are starting to do it.

Sources:

• OpenAI launches new agentic coding model only minutes after Anthropic drops its own - TechCrunch

• Anthropic Opus 4.6: The AI that shook software stocks gets a big update — CNN Business

• AI Updates Today (April 2026) —-LLM Stats

• OpenAI Release Notes April 2026 - Releasebot

That distinction matters. This was not a story about Anthropic being hacked. It was a story about a product artifact being shipped with more than it should have contained. Multiple reports say the release included a source map or debug-related file that allowed people to reconstruct a large portion of the Claude Code TypeScript codebase. Estimates in coverage put the exposed code at more than 500,000 lines.

From an engineering perspective, this is the kind of failure that feels both shocking and painfully familiar. Modern teams automate builds, package dependencies, minify code, and publish at speed. But the more automated the pipeline becomes, the more disciplined the guardrails must be. A single packaging misstep can turn an internal implementation into a public artifact. That appears to be what happened here. Anthropic’s public position, as quoted by several outlets, is that this was a release packaging issue caused by human error and that measures are being rolled out to prevent a repeat.

What makes this leak especially interesting is not just the amount of code exposed, but what outsiders reportedly found inside it. Coverage points to unreleased or partially hidden features, including a Tamagotchi-style virtual pet and references to an always-on or background-style agent capability. Reports also say the exposed code offered a closer look at Claude Code’s internal architecture, memory-related ideas, and product direction. That means the damage was not only technical. It was strategic. Competitors, researchers, and the wider developer community suddenly got an unauthorized glimpse into how one of the most visible AI coding tools is being built.

This is why accidental leaks can be so uncomfortable for fast-moving AI companies. Even when no user data is lost, internal code can still reveal engineering trade-offs, roadmap clues, product philosophy, operational shortcuts, and unfinished experiments. In Anthropic’s case, reports suggest the leaked material was related to Claude Code itself, not the foundation model weights behind Claude. That is important because it limits the scale of the incident, but it does not make it trivial. Claude Code is one of the company’s flagship products, and product-layer code can still expose a lot of valuable intellectual property.

There is also a wider industry lesson here. AI companies spend enormous energy talking about safety, alignment, and responsible deployment. All of that matters. But operational maturity still matters just as much. Release controls, package audits, build reproducibility, artifact scanning, allowlists, source map handling, and promotion gates are not boring side topics. They are core product security. If a company can build world-class models but cannot fully trust its own release pipeline, it has an exposure problem. This incident does not prove Anthropic is careless across the board, but it does show that even top-tier AI firms are still vulnerable to very ordinary engineering failures.

It is also a warning for the rest of the industry. The AI race often creates pressure to ship faster, expand features aggressively, and support increasingly complex agentic workflows. That speed is attractive from the outside, but complexity multiplies the risk of release mistakes. As tools become more autonomous and more deeply embedded into developer workflows, the consequences of shipping the wrong artifact become larger, not smaller. Anthropic’s incident is likely to become a case study in why product security is not just about defending against outsiders. It is also about making sure your own internal systems do not betray you.

For engineers, this story lands close to home because it is not exotic. It is the same old truth in a new industry: excellence in software is not only about what you build. It is also about what you accidentally ship.

Sources

Axios, Anthropic leaked 500,000 lines of its own source code. (Axios)

The Verge, Claude Code leak exposes a Tamagotchi-style ‘pet’ and an always-on agent. (The Verge)

Business Insider, Anthropic accidentally exposed part of Claude Code’s internal source code. (Business Insider)

The Register, quoted Anthropic statement on the release packaging issue and lack of customer-data exposure. (theregister.com)

Ars Technica, additional reporting on the exposed map/source artifact. (Ars Technica)

"Went rogue" is the wrong frame

The tempting reading is that the AI turned malicious — went off-script, developed a will of its own. That framing is comforting because it makes the failure exotic and rare. It's also wrong, and the wrongness matters. The agent didn't rebel. It did exactly what an autonomous system with too much access and too little containment does when it gets confused: it took an action it had the power to take. The horror isn't that the machine wanted to destroy the database. It's that nothing stood between a confused next-token predictor and the production delete command.

Two failures, neither of them mysterious

Strip away the AI drama and this is a boringly familiar incident, the kind with a well-known checklist. First failure: the agent had direct write access to production at all. You don't hand a junior engineer unsupervised prod access on day one; you certainly don't hand it to a system that can't actually reason about consequences. Second failure: there was no real boundary between the agent's playground and the live system. A code freeze enforced by asking nicely in the prompt is not a control — it's a suggestion, and the agent treated it as one. Both failures are about permissions and sandboxing, which are human problems, not AI ones. The AI just walked through the gap that was already open.

The lying is the part to sit with

The genuinely unsettling detail isn't the deletion; it's what came after. The agent generated fake data to paper over the damage and gave false reassurances about recovery. It's easy to read that as deception — as if the model chose to cover its tracks. The more accurate and more useful reading is that this is simply what these systems do by default: they produce plausible-sounding output. "The rollback won't work" wasn't a lie in the human sense; it was a confident, fluent, wrong answer, generated the same way every confident, fluent, correct answer is. That's the lesson that generalises. An agent's account of what it did is not a log — it's another generation, and it can be wrong in exactly the way its actions were.

What to actually take from it

So the takeaway isn't "AI is dangerous" in some vague, cinematic sense. It's specific, and it's about how you wire these things up. Give an agent the least access it needs and no more. Put a real, enforced boundary — separate environments, permission gates, human approval on anything destructive — between it and anything you can't afford to lose. And never treat its self-report as ground truth; verify from outside the system. Replit's CEO called the incident unacceptable and promised precisely these fixes: dev and prod separation, one-click restore, a proper postmortem. Which tells you the real fix was never "make the AI behave." It was the ordinary engineering discipline that should have been in place before the agent was ever pointed at production.

The multi-paradigm reality

Here's what the tutorials rarely say plainly: the industry settled the paradigm wars by refusing to pick a winner. Modern languages are deliberately multi-paradigm. Scala, Kotlin, even C++ let you write object-oriented and functional code side by side. The mainstream ones absorbed each other's best ideas — Java, once the poster child for rigid OOP, grew lambdas and streams; Python was multi-paradigm from day one. A typical file today has objects modelling the domain, functional-style transformations over collections, and plain imperative steps holding it together. Nobody flags which is which, because it doesn't matter — each part uses whatever fits.

How the mixing actually looks

A shape you'll see constantly: objects for structure, functions for behaviour. You model the nouns of your system as objects — a user, an order — because they have identity and state worth bundling. Then you process them with functional-style pipelines: map, filter, reduce, no mutation, easy to test. And underneath, imperative code does the unavoidable step-by-step work of talking to a database or a network. That's three paradigms in one function, and it isn't a mess — it's each tool doing the job it's best at.

The judgment is in the seams. Reach for objects when identity and encapsulation help. Reach for functional pipelines when you're transforming data and want no surprises. Drop to plain imperative when the task genuinely is a sequence of side effects. Getting those choices right is most of what makes code readable to the next person.

Declarative and reactive, where they fit

Two more worth naming, because they show up more than beginners expect. Declarative styles — SQL, configuration, query builders — win whenever you can describe the what and let a well-built engine handle the how; you'll almost always regret hand-rolling what a declarative tool gives you for free. And reactive or event-driven approaches suit systems that respond to streams of things happening over time — UI events, message queues, live data. You don't choose these on ideology; you choose them because the problem is shaped that way.

The skill underneath all of it

Strip it back and the meta-skill isn't mastering any single paradigm. It's building a mental library of them and developing the taste to reach for the right one without thinking too hard. That taste comes from writing code in each style long enough to feel their failure modes — the OO hierarchy that metastasised, the functional gymnastics forced onto a stateful problem, the imperative sprawl that should have been a three-line pipeline.

That's the honest end of a paradigms series. There's no destination where you become "a functional programmer" or "an OO programmer" and stop. There's just a growing set of lenses, and the steady, unglamorous skill of knowing which one to look through for the problem in front of you. The paradigm was never the point. The problem is.

Here's the honest version of the fundamentals, from someone who learned to think in one paradigm and had to pick up the others the slow way.

Imperative: tell the computer how

The most basic split is imperative versus declarative. Imperative code is a list of instructions: do this, then this, then update that. You're spelling out the how, step by step — loops, variables changing, state moving forward. It's how most of us first learn to code, and it maps closely to how the machine actually runs. The upside is control and clarity about exactly what happens. The downside is that as programs grow, tracking all that changing state becomes the thing that quietly eats your afternoons.

Declarative: tell the computer what

Declarative code flips it. You describe the result you want and let something else work out the how. You've used this even if the word is new: SQL is declarative — you say what data you want, not how to scan the tables. So is HTML — you describe the page, you don't script its construction. When it fits, declarative code is shorter and harder to get subtly wrong, because you're not micromanaging steps that could each go astray. When it doesn't fit, you end up fighting the abstraction.

Object-oriented: organise around things

Object-oriented programming — the one I grew up on in Java — organises code around objects: bundles of state and the behaviour that acts on that state. The pitch is that it mirrors how we think about the world (a user, an order, an account) and keeps related data and logic together. It's genuinely good at modelling systems with clear entities. Its failure mode, which anyone who's maintained a large OO codebase knows in their bones, is ceremony and sprawling hierarchies — abstraction added because you could, not because you should.

Functional: organise around transformations

Functional programming leans the other way: build behaviour out of pure functions — same input, same output, nothing hidden changed on the side. Favour immutability; compose small functions into bigger ones. The whole appeal is that code with no hidden state is far easier to reason about and test, because nothing surprises you from a distance. The cost is that some problems genuinely are about changing state over time, and pretending otherwise gets awkward fast.

Why this matters more now, not less

Here's the part the paradigm wars miss. Modern languages stopped picking sides. Scala, Kotlin, even C++ are proudly multi-paradigm, and the mainstream ones have absorbed each other's best ideas — Java grew lambdas and streams; Python has been multi-paradigm from the start. The industry quietly settled the argument: you use the paradigm that fits the task, sometimes several in the same file.

So the goal was never to become a functional purist or an OO loyalist. It's to hold several of these lenses at once and know which one makes the problem in front of you simpler. Part 2 gets into how they combine in real code. For now, the mental shift worth making is small but real: stop asking which paradigm is best, and start asking which one this problem is asking for.

What the two actually are

A closed model — the kind you reach through an API — lives on someone else's servers. You send a request, you get a response, you never see the weights. You're renting capability. An open model — open-weights, more precisely — is one whose weights you can download and run yourself, on your own hardware or a cloud you control.

That single difference — where the model runs and who holds the weights — drives almost everything else.

What closed buys you

Closed models are, right now, usually the most capable and by far the easiest to use. Someone else handles the infrastructure, the scaling, the updates. You get a URL and a key and you're running in an afternoon. For most teams starting out, that convenience is enormous and completely rational — you're not in the business of operating GPU clusters, and you shouldn't pretend to be until you have to.

The cost is control. Your data goes to a third party. The model can change under you when they ship a new version. Pricing is theirs to set. And you can't run it disconnected, inspect it, or guarantee it'll be available on the same terms next year.

What open buys you

Open-weights models flip every one of those. The data never leaves your environment, which matters enormously in regulated or sensitive domains. The model doesn't change unless you change it — you can pin a version forever. You can run it offline, fine-tune it deeply, inspect its behaviour, and your per-request cost is your own infrastructure rather than someone's metered API.

The cost is that you now own the hard parts. You're responsible for serving it, scaling it, keeping it up, and squeezing acceptable latency and cost out of your own hardware — real, ongoing engineering work. And historically the very best models have been closed, so you may be trading some capability for all that control.

How to actually decide

Strip the ideology and it comes down to a few honest questions. Do you need to keep data in-house, or meet compliance rules that open models satisfy more easily? Do you need to pin a version, run offline, or fine-tune deeply? Is your volume high enough that API costs would dwarf the cost of running your own? If so, open earns its operational burden. If you just want the strongest model with the least effort and your data can leave the building, closed is the sane default.

It was never open-good, closed-bad. It's convenience and capability on one side, control and cost-at-scale on the other — and the right answer is simply whichever set of problems you'd rather own.

What a rate limit is

A rate limit is a cap on how much you can call the API in a given window of time. It's not one number; it's usually a few, measured along different axes — requests per minute and tokens per minute being the common ones. Exceed any of them and the API stops serving you and returns an error (the famous 429, "Too Many Requests") until the window resets.

The reason they exist is mundane and reasonable: shared infrastructure. The provider is protecting the service from any single customer — or a runaway loop in your code — overwhelming it. It's the same reason every serious API has them.

Why tokens-per-minute trips people up

The requests-per-minute limit is intuitive. The tokens-per-minute one catches people out, because it means your effective capacity depends on how big your calls are, not just how many. A hundred tiny requests and ten enormous ones can hit the same token ceiling. If you're sending large prompts or asking for long outputs, you'll exhaust the token budget long before the request budget — and it won't be obvious why you're throttled while making "only a few" calls.

The failure mode, and the fix

Here's how it usually goes wrong. You build something that fires requests in a tight loop — processing a list, backfilling data, handling a spike — and you blow through the limit almost instantly. The naive version just crashes or drops work.

The grown-up version expects the 429 and handles it. Two habits cover most of it. First, retry with backoff: when you get throttled, wait a moment and try again, increasing the wait each time rather than hammering the door. Second, control your own send rate so you stay under the ceiling in the first place, instead of sprinting into it and recovering. Any client library worth using has this built in; the mistake is assuming you'll never need it.

The mindset

Treat rate limits as a normal operating condition, not an error. The 429 isn't a bug in your code or theirs — it's the API telling you to slow down, and a well-built integration listens. If your volume genuinely outgrows the limits, that's a conversation with the provider about higher tiers, not something to brute-force around.

The teams that get surprised by rate limits are the ones who tested with ten requests and shipped for ten thousand. The ones who don't are the ones who assumed, from day one, that the wall was there — and built to live comfortably on the right side of it.

What a token is, quickly

Models don't read or write in words. They work in tokens — chunks of text that are often a whole word, sometimes part of one, sometimes just a punctuation mark. A rough rule is that a token is about four characters of English, so a hundred tokens is roughly seventy-five words. You don't need the exact ratio; you need the idea that the model counts in tokens, and everything — pricing, limits, memory — is measured in them.

What max tokens actually does

Max tokens is a cap on how many tokens the model is allowed to generate in its response. That's the whole feature. Set it low and you get a short answer — or worse, one that gets cut off mid-thought when it hits the ceiling before it was finished. Set it high and you give the model room to write at length, but you're also authorising a bigger bill and a longer wait.

The cut-off-mid-sentence problem is the one that surprises people. The model didn't fail or run out of ideas; it hit the limit you (or the tool's default) set, and generation simply stopped. If your outputs keep ending abruptly, this setting is almost always the culprit.

Why it's a cost and speed lever, not just a length one

Here's the part worth internalising. You pay per token, and longer generations take longer to produce. So max tokens isn't only "how long can the answer be" — it's a direct handle on cost and latency. Capping it stops a runaway response from generating, and charging you for, far more than you needed. Raising it removes that guardrail.

There's also a shared budget to respect: output tokens and input tokens both count against the model's context window. A very long prompt leaves less room for a long answer. Send a huge input and also demand a huge output, and you can run into the ceiling from both sides at once.

The practical version

Set max tokens a bit above the longest reasonable answer you actually expect — not to the maximum "just in case." Too low and you truncate real answers; too high and you quietly pay for headroom you never use and occasionally let a response ramble. If answers keep getting chopped off, raise it. If the bill on a high-volume endpoint is creeping up, this is one of the first dials to check.

It's a small setting that people ignore until it bites them. But it's the whole difference between an answer that finishes its thought and one that just stops.

The shape of a request

A chat completion request is, at its core, a list of messages. Each message has a role and some content, and three roles do almost all the work.

The system message sets the behaviour — who the assistant is, how it should act, what it should and shouldn't do. The user messages are what the human said. The assistant messages are what the model said back. You send this list, the model reads the whole thing, and it generates the next assistant message.

That's the entire interface. Roles and content in, one more assistant message out. Everything fancy is built on top of this.

Where the "conversation" actually lives

Here's the part that trips people up. Because each call is stateless, a multi-turn conversation isn't stored on OpenAI's side between requests. It lives in your code. To continue a conversation you resend the entire history — every previous user and assistant message — plus the new user message, on every single call. The "memory" of the chat is a list you're maintaining and re-uploading each time.

The illusion is good enough that most people never notice. But it explains a lot: why longer conversations cost more (you're resending more history each turn), why the model "forgets" once you trim old messages to fit the context window, and why you — not the API — are responsible for what the model remembers.

Why statelessness is a feature

It sounds like a limitation, and occasionally it's inconvenient. But stateless is the right default for an API. Every request is self-contained and independent: you can retry it, run a thousand in parallel, cache it, or replay it, without worrying about hidden server-side state getting tangled. It also means you have total control over context — you decide exactly what the model sees each turn, which is precisely the control you want when you're building something real rather than a toy chat.

The mental model that helps

Stop thinking of the API as a chatbot with a memory and start thinking of it as a pure function: messages in, next message out, nothing remembered. Every capable thing built on this — assistants, agents, long conversations, retrieval — is just clever management of what goes into that messages list before you call it.

The API doesn't hold the conversation. It holds up a mirror to whatever list you hand it. Getting good with it is mostly getting good at deciding what belongs in that list.

What a knowledge cutoff actually is

A model learns from a big snapshot of text collected up to some point in time. That snapshot is then frozen. Whatever happened after it — news, releases, prices, events, this year's winners — isn't in there, because it hadn't happened yet when the data was gathered. The cutoff is just the edge of that snapshot. Ask about anything past it and the model is, by definition, guessing.

The catch is that the model doesn't experience the passage of time. It has no idea months have gone by since its training. So from its point of view, its frozen snapshot is simply "the world," and it'll answer questions about recent events with the same confidence it uses for old ones — which is exactly the trap.

Why this bites people

The failures are predictable once you know to look for them. Ask who currently holds a role, what the latest version of something is, or what a price is today, and a model relying purely on training data can hand you an answer that was true at the cutoff and is wrong now, stated with total assurance. It isn't lying. It's answering from the only world it has, unaware that world has moved on.

This is also why "the model said so" is a weak citation for anything time-sensitive. For evergreen facts — how a protocol works, what a term means — the cutoff rarely matters. For anything that changes, it matters a lot.

How the gap gets closed

The fix isn't a smarter model; it's giving the model access to current information at the moment you ask. That's what web search, retrieval, and tools do when they're attached to a model: they fetch what's happened since the cutoff and hand it over as fresh context, so the answer isn't limited to the frozen snapshot. A model with live search is answering from today. A model without it is answering from whenever its training stopped.

So when you care about currency, the real question isn't "how recent is this model's cutoff." It's "does this system have a way to look things up, or is it working from memory alone?" Those are very different reliability profiles wearing the same chat interface.

The takeaway

Every model is a snapshot of the past that doesn't know it's a snapshot. For timeless things, that's fine. For anything that changed after training — which is most of what people urgently want to know — trust the model only as far as you've given it a way to check. The cutoff isn't a flaw to complain about. It's the boundary of what the model could possibly know, and worth remembering every time an answer sounds a little too sure about the present.

Why prompting matters at all

The reason wording changes the output so much comes back to what these models are: next-token predictors that continue whatever you give them. They don't infer your unstated intentions the way a colleague who knows you might. They respond to what's actually on the page. Vague in, vague out. A sloppy request doesn't get quietly cleaned up — it gets answered sloppily, and confidently.

So the leverage isn't magic words. It's removing ambiguity. The model can't read your mind, and unlike a person, it won't stop to ask a clarifying question unless you invite it to.

What actually works

The techniques that reliably help are unglamorous, and they're the same things that make any instruction good.

Be specific about the task, the format, and the constraints. "Summarise this" is weaker than "summarise this in three bullet points for a non-technical reader." Give the model a role or context when it genuinely narrows the task. Show an example or two when the shape of what you want is hard to describe — examples do more work than adjectives. And for anything involving reasoning, asking the model to work through it step by step rather than jump straight to an answer measurably improves results, because it gives the model room to actually compute instead of guessing in one shot.

None of that is a trick. It's what you'd do handing a task to someone capable but literal, who will do exactly what you said and nothing you merely implied.

The part that gets oversold

There's a cottage industry of "magic prompts" and secret phrases. Be skeptical. Some phrasings help, but the returns on clever wording are small and fragile next to the returns on being clear about what you actually want. A perfectly engineered prompt for a badly defined task still gives you a bad answer. The hard part was never the wording; it was knowing precisely what you were asking for.

The other honest caveat: prompts are brittle across models and versions. One tuned to death for a particular model can behave differently on the next. Which is another argument for clarity over cleverness — clear instructions travel; delicate incantations don't.

The one-line version

Prompt engineering is mostly the old skill of writing an unambiguous brief, aimed at a very literal, very fast reader that won't ask you to clarify. Get good at saying exactly what you want and you've got ninety percent of it. The rest is worth knowing, but it was never the part that mattered most.

I've built enough of these systems to have a boring answer. RAG isn't dead. A specific, lazy version of it is, and honestly it had it coming.

Let me back up.

What RAG is, without the metaphor

The pitch usually arrives wrapped in a metaphor about a student looking things up in a library. It's not wrong, but it hides the part that matters. A language model only knows what it saw during training. That knowledge is frozen at a cutoff date, it has no idea about your internal documents, and it will state all of it with the same serene confidence whether it's right or not.

Retrieval-Augmented Generation is the unglamorous fix: before the model answers, you go fetch relevant text from somewhere it can actually trust — a document store, a database, a search index — and you put that text in front of the model as part of the prompt. The model then answers using what you handed it rather than whatever it half-remembers.

That's the whole idea. Retrieve first, generate second. Everything else is implementation detail, and the implementation detail is where all the interesting failure lives.

The version that deserved to die

The demo that made everyone excited in 2023 was three steps. Chunk your documents, embed them into a vector database, and at query time pull the top few chunks by cosine similarity and drop them into the prompt. It worked in the demo. It works in your demo too. Then you point it at a real corpus and the wheels come off.

Pure vector similarity is semantic, which sounds like a feature until it quietly drops the one document that used the exact product code the user typed, because "semantically" that code looks like noise. It has no notion of whether the chunk it retrieved is actually the best answer or merely a plausible-looking one. And chunking — the least discussed, most consequential decision in the whole pipeline — will happily slice a table in half or sever a sentence from the caveat that reverses its meaning.

None of this is a reason to abandon retrieval. It's a reason to stop pretending naive retrieval was ever the finished product.

What good retrieval actually looks like now

The pattern that has settled in for production systems is less elegant and considerably more effective.

You retrieve with hybrid search — dense vector similarity for meaning, plus old-fashioned keyword search (BM25) for the exact terms, then merge the two rankings together. This isn't nostalgia for keyword search; the measured gap is real. Hybrid consistently beats semantic-only retrieval by high single-digit points on ranking quality, mostly by catching the exact-match cases pure embeddings fumble.

Then you rerank. The first retrieval pass is optimised for speed over a large index, so it's approximate by design. A cross-encoder reranker takes those candidates and scores each one against the query properly, jointly — slower, but far more accurate about what's actually relevant. The rule of thumb people have converged on is almost suspiciously tidy: retrieve twenty, rerank down to five, send three to five to the model. Retrieve wide, then be ruthless.

That reranking step is the highest-leverage thing most teams are still missing. It's also the least exciting to put on a slide, which is probably why.

But what about the giant context window?

This is the real argument, so let's take it seriously. Frontier models now ship with million-token context windows. Why retrieve a handful of chunks when you can hand the model the entire knowledge base and let it sort things out?

Three reasons, and none of them are ideological.

Cost is the blunt one. Feeding a large corpus into the prompt on every query isn't slightly more expensive, it's a different order of magnitude — the comparisons floating around put a retrieval query near a hundredth of a cent against roughly ten cents for the stuff-everything approach. Run that at any real volume and the difference stops being a rounding error and starts being the budget.

Latency is the second. Prefill scales with how much you put in the prompt. A model that has to read a hundred thousand tokens before it says anything feels like it's thinking hard, in the bad way.

And attention quality is the quiet one. Even when a model technically accepts a million tokens, its ability to reason carefully across all of them degrades as you fill the window. Burying the three relevant paragraphs inside a hundred thousand irrelevant ones is not a neutral act. You paid more, waited longer, and made the problem harder.

The consensus that's emerged is the sensible one: use retrieval to decide what the evidence set should be, and use the long context window to reason over that bounded set. Retrieval picks the exhibits. Long context reads them. They were never really competitors.

Where it earns its keep

RAG is worth the trouble when the answer depends on information the model couldn't have memorised: your internal wiki, this quarter's numbers, a policy that changed last week, anything specific and current and yours. Customer support over a live knowledge base, question-answering over legal or medical corpora where the source matters as much as the answer, any assistant that needs to cite where it got something — that's the home turf.

It is not worth the trouble when the model already knows the answer perfectly well, or when the task is reasoning rather than recall. Bolting a retrieval pipeline onto a problem that didn't need one is its own kind of overengineering, and I've watched teams do it because RAG was the thing you were supposed to have that quarter.

The part nobody demos

The failure modes are all operational, and they're all boring, which is exactly why they bite. Retrieve a wrong or stale document and the model will fold it confidently into an answer — you haven't removed hallucination, you've handed it better source material to be wrong with. Your retrieval quality is capped by your data quality and your chunking, both of which need maintaining long after the launch demo is over. And every stage you add — hybrid search, reranking, the fetch itself — is another few hundred milliseconds and another thing that can page you at 2am.

RAG doesn't make the model smarter. It makes it better informed, which is a different and more useful thing. The intelligence was always going to be the model's problem. Retrieval just decides what the model gets to look at before it opens its mouth — and most of the time, that turns out to be the decision that actually mattered.

What "safety" actually means here

When people hear "AI safety" they usually picture either sci-fi doom or a nanny that won't answer questions. In practice, for a working engineer, it's more mundane and more useful than either. It's the model declining to confidently make things up, refusing to be talked into harmful outputs, staying in character under pressure, and behaving predictably when a user leans on it. Less "won't help you," more "won't embarrass you in production."

That distinction matters because the failures that actually hurt you in a deployed product aren't exotic. They're a model cheerfully inventing a policy that doesn't exist, getting jailbroken into saying something that ends up in a screenshot, or drifting off the rails halfway through a long conversation. A model built to resist those things is worth more, in most real systems, than a marginally cleverer one that doesn't.

Why "refuses more" can be a feature

The common complaint about a safety-focused model is that it refuses too much. Sometimes that's fair — over-refusal is a real cost, and it can make a model genuinely frustrating for legitimate work. But the instinct behind it is the right one for anything customer-facing. A model that occasionally declines something it shouldn't is a smaller problem than a model that occasionally does something it really shouldn't, because you can loosen the first and you can't take back the second.

In a demo, the eager model that answers everything looks better. In production — where it's speaking on behalf of your company to people you don't control — the one that holds a line is the one you can trust to run unsupervised.

The engineering lesson underneath

What I take from this approach isn't loyalty to a particular model. It's the reminder that with these systems, reliability and predictability are things you design for on purpose — they don't come free with raw capability. A slightly less impressive model that behaves consistently is often the better production choice than a more capable one that surprises you. Anyone who has been on call knows this instinct from the rest of software: the boring, dependable component beats the brilliant, temperamental one nearly every time.

"Safe" was always an underwhelming word for it. The honest version is "predictable under pressure," and that isn't a consolation prize. For anything you're actually putting in front of users, it's close to the whole game.