1017 | AI in the Engine Room: From Ring to RDMA

Mia: Welcome back to the show, everyone. I'm Mia.

Milo: And I'm Milo. It's been a busy day on the discussion boards, and honestly, the thread that pulled me in first was the one arguing that we've been misreading Transformers for years.

Mia: Right — that's a good place to start, because once you accept that framing, everything else today — the math mystery, the quantum circuits, the bots that write themselves — starts to look like one long story about machines actually doing work.

Milo: So the article making the rounds reframes the Transformer as associative memory. Not a reasoning engine, not a world model, just a giant, layered content-addressable store.

Mia: And that's the part people kept circling: it's not memory in the vague sense, it's that each layer is its own memory store. So a forward pass through the network becomes, in effect, a sequence of lookups, each one refining or retrieving something the previous layer left behind.

Milo: That's a very different mental model from the one most of us were taught. The usual framing is "attention plus feed-forward layers," which is architecture-speak. This says: no, think of each layer as a key-value database that fires when the input pattern matches something it learned.

Mia: And the interesting consequence is what it explains about behavior. If you've ever wondered why a model can recall something instantly and confidently one moment and completely fabricate the next, the associative-memory view makes that look almost expected. A lookup either hits or it doesn't, and a partial key can resolve to the wrong entry.

Milo: That was one of the livelier tensions in the thread — the people who found this view clarifying versus the people who said it's reductive. The clarification camp said it finally gives them language for things like grokking and why fine-tuning feels like overwriting specific entries rather than teaching new skills.

Mia: And the pushback was essentially: memory explains recall, but does it explain the compositional stuff? When a model chains an inference — two steps that only work together — is that just layered lookups, or is there something the memory metaphor glosses over?

Milo: Which, to be fair, nobody in the discussion fully resolved. There was no clean answer to "can associative memory compose?" What people did agree on was that the framing is at least testable — if each layer is a store, you should be able to poke a layer, corrupt an entry, and watch a specific behavior degrade.

Mia: And that's exactly the kind of experiment that makes me glad this framing exists, because for years the alternatives were either "black box" or "it's just next-token prediction," and both of those stop conversation instead of starting it.

Milo: Okay, so if the theory says these things are giant memories that retrieve patterns, here's the story that made the theory feel concrete. Claude, the model itself, cracked a genuine mathematical mystery that humans had been arguing about for decades — the ECDSA curve seeds.

Mia: For anyone who hasn't followed this saga: when NIST published the elliptic curves that secure a huge chunk of internet cryptography, the constants — the seeds used to generate the curves — were just given to the world with no explanation of where they came from. That opacity fueled years of suspicion.

Milo: Years. And the suspicion had teeth, because unexplained constants in cryptography are exactly how you'd hide a backdoor. If nobody can derive the seed, nobody can verify the curve wasn't chosen from thousands of candidates until a weakness appeared.

Mia: And the resolution Claude produced: the seeds come from lrand48 — the classic, ancient pseudo-random generator — initialized with Unix timestamps. So someone, back in the day, ran a plain old random number generator seeded by the clock, wrote down the outputs, and those became the constants.

Milo: Which means the answer to "is there a backdoor?" is essentially no — or at least, the mysterious part isn't a hidden design. It's a mundane generator and a timestamp. The constants weren't crafted, they were generated.

Mia: And the reaction to that was a beautiful split. Half the thread was relief — this is the best possible outcome for the trustworthiness of the curves. The other half was almost disappointed in a funny way, because after decades of speculation, "a guy ran lrand48 on a Tuesday in the nineties" is a profoundly anticlimactic ending.

Milo: There was also real skepticism about how you'd ever verify this. If Claude says the seeds come from lrand48 at certain timestamps, that's a checkable claim — you can run the generator and compare. And the fact that it's checkable is what separates this from a lucky guess. A wrong theory about the seeds wouldn't survive running the actual function.

Mia: That's worth underlining: this wasn't the model vibes-ing an answer. The claim has the property that anyone with a terminal can falsify it. Cryptographers spent decades not finding this, and what cracked it was brute-force search over a space of boring, obvious candidates — timestamps and a standard libc random function.

Milo: And it says something about the search process itself. Humans assumed the answer would be complicated, because the mystery had accumulated decades of gravity. The actual answer space — "old RNG, seeded with the clock" — is a space we could have searched exhaustively but didn't, because it seemed too trivial to be true.

Mia: Which connects straight to the next story, because it's the same shape in miniature: Claude took eight simulated quantum circuits and optimized them on its own, and in some cases cut the number of operations by a factor of seventeen.

Milo: Factor of seventeen on a quantum circuit is not a rounding error. Every operation on a quantum computer is expensive and noisy, so reducing gate count is one of the highest-leverage things you can do — it directly improves whether a computation succeeds at all.

Mia: The word "self-directed" matters here too. This wasn't a human saying "optimize this function, here's the metric." The model was handed the circuits, recognized they were inefficient, and produced the reductions itself.

Milo: And the discussion around this one had a healthy dose of "let's not oversell." The circuits were simulated, small ones, eight of them — this is not Claude redesigning a production quantum algorithm overnight. Several people made exactly that point.

Mia: But the counterpoint to the counterpoint: humans do circuit optimization with great effort and specialized tools, and a factor-of-seventeen reduction on even a small circuit is the kind of result that would get attention in a paper. The scale is small; the qualitative act — finding redundant structure in a quantum circuit nobody asked it to touch — is the news.

Milo: So if you stack the three stories: a new theory that models are layered associative memory, a model solving a real cryptographic cold case with a checkable answer, and a model independently finding optimizations in quantum circuits —

Mia: — you get the arc of the whole discussion, which is that these systems have crossed from "impressive demo" to "produces results you can verify and use." Verification is the common thread. The memory theory is testable, the seed claim is runnable, the circuits are simulable.

Milo: And that sets up the next thread perfectly, because it's about work that verifies itself — Claude Code, which now writes roughly ninety percent of its own code.

Mia: Ninety percent. And Anthropic is processing over thirty thousand changes a day, with an automated review process in the loop.

Milo: The number that broke people's brains was thirty thousand changes a day. That's a volume of modification no human team could review line by line, which is precisely why the automated review isn't a nice-to-have — it's the only way that system can function.

Mia: And the sharpest tension in that thread was recursive quality. If the code is written by the model, and the review is automated, who catches the systemic mistakes? Humans are good at spotting when a whole approach is wrong. An automated reviewer evaluating model-written code can share blind spots with the writer.

Milo: The optimists' answer was that blind spots get caught by reality: the tool runs, tests fail, users hit bugs. The pessimists' answer was that some failure modes — subtle design decisions, quiet inefficiencies — don't announce themselves through tests, and those accumulate silently in a ninety-percent-autonomous codebase.

Mia: Nobody settled that, and honestly it may be the defining question of the next few years of software. What we do know is the throughput. Thirty thousand changes a day is a pace of iteration that changes what development even means — you're no longer writing software so much as supervising a system that writes it.

Milo: And the economics thread picked up right where that left off. Anthropic got hit with the lowest — the junk-level — credit rating in June, and yet an IPO is expected at a valuation above two trillion dollars.

Mia: Those two facts sitting next to each other is the whole AI economy in miniature. A credit rating agency looks at current cash flows and says "junk." The market looks at where this is going and says "two trillion."

Milo: And the discussion genuinely split on which of those lenses is wrong. One camp said the rating is just backward-looking — of course a company burning capital to train frontier models has junk-grade cash flows; you're buying the future, not the present.

Mia: The other camp said that's exactly the reasoning that preceded every bubble: when you have to wave away current financials to justify the valuation, you're in speculative territory. Both positions are coherent. That's why the thread had no resolution.

Milo: What makes it less abstract is the work data. A Mercor study found that ninety percent of US work hours are in tasks that existing AI benchmarks barely cover.

Mia: Sit with that one, because it cuts both directions. If you're bullish, it means the measured capabilities are the floor, and the actual economy — all that unmeasured work — is the upside nobody's priced in.

Milo: If you're bearish, it means every benchmark triumph you've read about is being measured on the sliver of work that's easy to benchmark, and the other ninety percent — messy, contextual, embodied in organizations — is untouched.

Mia: And the resolution depends on something nobody can know yet: whether the unmeasured ninety percent is hard because models can't do it, or hard because nobody built benchmarks for it. Those are very different worlds.

Milo: Meanwhile, in the image world, Black Forest Labs shipped FLUX 3 Image Klein — a five-point-six-billion-parameter open-weights model for text-to-image, editing, and multi-reference workflows.

Mia: The "open weights" part is what people latched onto, because it means anyone can download it, run it locally, fine-tune it. And multi-reference is a meaningful capability — you give it several images and it works with all of them, which is the workflow actual designers live in.

Milo: The counterweight in the discussion was size. At five-point-six billion parameters, this is deliberately the compact entry in the family — so the question everyone asked was where the quality lands relative to the big closed models, and for which kinds of images.

Mia: Which brings us to the hardware thread, because if you're going to run big models yourself, you need somewhere to run them. And the two stories today are gloriously absurd.

Milo: Tell them about the PlayStation.

Mia: So there's a project — PS5LM — running llama.cpp jailbroken on the PS5, on the GPU, with eleven point four gigabytes of shared memory, serving as an actual model server.

Milo: The PS5. A games console. Serving LLM inference.

Mia: The discussion was equal parts delight and engineering curiosity. The delight needs no explanation. The engineering curiosity was about the constraints: consoles are cheap for the compute they contain — that's the whole console business model — so a jailbroken console is, dollar for dollar, a weirdly economical inference box if you can stomach the jailbreak.

Milo: And the shared memory point is doing real work here. Consoles use unified memory architecture — the CPU and GPU see the same pool — and for LLM inference, where the weights have to live somewhere addressable, that's actually a favorable layout. The jailbreak is the hard part; the memory is an asset.

Mia: On the respectable end of the spectrum, Apple's MLX framework is running Muse Spark Open — roughly eight hundred sixteen billion parameters — on four Mac Studios, at over thirty-six tokens per second.

Milo: Eight hundred sixteen billion parameters on four consumer-adjacent machines. And the trick is RDMA over Thunderbolt — the machines share memory directly over the cable, so the model's weights don't sit in one box, they're pooled across all four.

Mia: The discussion here was about what this implies. If four Mac Studios can serve an eight-hundred-billion-parameter open model at usable speed, then the "you must rent from a hyperscaler" assumption weakens for a whole class of users.

Milo: And the common denominator between the Mac cluster and the jailbroken PlayStation is exactly what you said earlier — fast interconnect and clever memory tricks. Both stories are really about memory bandwidth and topology, not raw compute.

Mia: Which is a nice inversion: for years the story was flops, flops, flops. Now the story is "how do I get the weights close to where they're needed," and Thunderbolt cable and console unified memory are both answers to that question.

Milo: From giant models on weird hardware, let's go to networks — and there's a lovely long-view thread that starts with the Token Ring war.

Mia: Set the scene for people who didn't live it: in the eighties and nineties, IBM pushed Token Ring as the office network standard. It was deterministic, orderly — tokens passed around a ring, everyone got a turn.

Milo: And it lost to Ethernet, which was cheaper and, crucially, scaled faster — Ethernet roared past a hundred megabits while Token Ring's ecosystem stalled.

Mia: The discussion used it as a parable, and the two morals people drew were different. One moral: technical elegance loses to cost and momentum. Token Ring was arguably the better-designed protocol for its era, and it didn't matter.

Milo: The other moral, pushed by the historians in the thread: Ethernet won because it was evolvable. It went from shared coax to switched to gigabit and beyond without abandoning its soul, while Token Ring was architected around a fixed ring notion that capped its future.

Mia: And then the thread pivots to today's standards battle, which is RFC 8950 — the mechanism that lets IPv4 routes travel over BGP sessions with IPv6 next hops.

Milo: Explain why that's a big deal, because it sounds like plumbing.

Mia: It's plumbing that enables a genuinely different kind of backbone. Until now, if you wanted an IPv6-only core network, you had a problem: your routing sessions still needed IPv4 in the middle to carry IPv4 routes. RFC 8950 says the next hop can be an IPv6 address even for IPv4 routes.

Milo: Which means you can build a backbone that speaks IPv6 end to end and still carry the legacy IPv4 internet over it. The old protocol becomes cargo, not infrastructure.

Mia: And the operators in the discussion were the most concrete voices here — the ones running real networks said this is the mechanism that makes IPv6-only cores practical, which matters because dual-stacking everything forever has a real cost.

Milo: The Token Ring comparison people kept making: is IPv4 the new Token Ring — a once-dominant standard that will lose not because it's bad but because its replacement is cheaper to operate and easier to grow? The counterargument was that IPv4 has twenty more years of momentum than Token Ring ever had, and momentum is the whole game.

Mia: Let's shift to trust and security — three stories, three kinds of trust. First: your data.

Milo: Home Assistant, the smart-home platform beloved precisely because it's local and privacy-respecting, scanned users' secret-like data and sent hashes of it to HaveIBeenPwned — the breach-notification service — without asking first.

Mia: The irony is the story. This is the platform people chose to keep their data off other people's servers, and it phoned home with hashes of credential-looking strings by default.

Milo: And to be fair to the intent: hashing before sending is the privacy-conscious way to do a breach check — you're not sending the secret itself, just its fingerprint. HaveIBeenPwned's k-anonymity model is literally designed for this.

Mia: But the thread's consensus, such as it was, wasn't about the technique — it was about consent. The protest on GitHub wasn't "the hash leaks my password"; it was "you decided for me." On a platform whose entire brand is user control, shipping an outbound network call for private data with no opt-in breaks the actual promise, not just the technical one.

Milo: The defenders said the feature is genuinely useful — people do want to know if their smart-home credentials appeared in a breach. The critics said that's irrelevant: useful features get opt-ins, and defaults are where trust lives.

Mia: Second kind of trust: your radio rules. eBay restricted listings of radio devices because of new FCC rules, with an exception carved out for equipment that runs purely on amateur frequencies.

Milo: This one looked bureaucratic on the surface, and the ham radio operators in the thread were the ones who gave it texture. The FCC rules are aimed at devices that broadcast outside their allowed spectrum, and marketplaces are being made to enforce that at the listing level.

Mia: The carve-out for amateur-band-only gear is the interesting part — it acknowledges that licensed radio operators are a different category from someone selling a generic transmitter that tramples on frequencies. But enforcement-by-marketplace is blunt, and people reported the obvious worry: legitimate gear caught in the filter, grey-area gear slipping through.

Milo: The deeper question the thread raised and didn't answer: is marketplace enforcement a sensible way to regulate the airwaves, or does it just push noncompliant hardware to less scrupulous channels?

Mia: And the third kind of trust: your kernel. OpenBSD — the operating system whose entire identity is code correctness — shipped a bug thirty-seven years old.

Milo: They call it "Big Fish," and it's beautiful in a horrible way: a sixteen-bit overflow in the SysV shared-memory attach counter, living in the code since the eighties, giving local root escalation.

Mia: Sixteen bits. The counter that tracks how many processes have attached to a shared memory segment can wrap around, and wrapping it in the right way gets you root. And the age is the point — this code predates most of the people qualified to audit it.

Milo: The thread's reaction had real humility in it. OpenBSD is the project famous for auditing everything, and even there, a bug can survive nearly four decades. The lesson people took wasn't "OpenBSD failed" — it was that sixteen-bit counters from another era are exactly the kind of thing modern review misses, because everyone's eye skips over them.

Mia: And the open question is the uncomfortable one: if a bug like this survived in the most audited codebase in the field, what's the base rate of ancient overflows in everyone else's kernels? Nobody wanted to finish that thought.

Milo: Let's close the loop with the culture corner — and the first item is pure practical value. Jake Archibald on image formats: for photos and screenshots on the web, AVIF is usually smaller. For lossless images, JPEG XL wins.

Mia: The nuance people appreciated is that "which format is better" is the wrong question — it's "better for what." Photos and screenshots, the dominant web case, go to AVIF. Lossless, which matters for archives, technical assets, anything that must round-trip perfectly, goes to JPEG XL.

Milo: The frustration in the thread was the browser-support patchwork — having the knowledge that the right format exists is not the same as being able to ship it to your users. But as guidance for new projects, that one-line rule is genuinely useful.

Mia: Next, a material science detour: Dekton, the heat-resistant countertop material from Cosentino, developed in 2013 — and the reveal is that at its core, it's essentially porcelain.

Milo: Which surprised people, because it doesn't behave like the porcelain they picture. The manufacturing makes the difference — it's porcelain taken to an extreme, which is where the heat resistance comes from. The thread's practical value was mostly for anyone renovating a kitchen: now you know what the marketing word actually means.

Mia: Then something purely delightful: an esoteric programming language called Folders, by Daniel Temkin, where the code is nothing but nested folders.

Milo: And the names are ignored. That's the punchline. The program is the shape of the directory tree — how deep, how many branches — and what you call any of it is irrelevant to execution.

Mia: The discussion was half art criticism, half genuine curiosity. The art-criticism half loved it as a statement: structure without labels, meaning carried entirely by form. The curiosity half tried to actually reason about what computation you can express with pure nesting, and it turns out hierarchy alone is enough to encode real logic, which is a little mind-bending.

Milo: From esolangs to a social network: /dev/null, a developer network where you log in with GitHub and your profile imports itself.

Mia: The appeal is obvious — zero onboarding, your work history is your identity, no profile-building ceremony. And the discussion immediately went to the failure mode: if the profile is your GitHub, then the network inherits everything attached to that identity, including the pressure of it.

Milo: There was no verdict on whether that's good or bad — just the observation that any developer network built on auto-import is really building on the conventions of GitHub, and those conventions shape who feels welcome.

Mia: Then a nostalgic one: a write-up from someone who prefers software rendering in DOS — crisp pixels, no bilinear filtering. And the detail that grabbed everyone: running Quake at 800 by 600 at sixty frames per second in software requires a fast modern PC.

Milo: Sixty frames per second of Quake, in software, on a modern machine, being an achievement — that's the sentence that did the emotional work. The thread became a meditation on why filtered pixels feel wrong to some people: the sharpness of era-correct rendering is part of the artifact, and smoothing it is a kind of quiet vandalism.

Mia: And the technical side was real too — software rendering means the CPU does everything, so the performance story isn't nostalgia, it's a genuine engineering problem, and the fact that it takes serious modern hardware says something about how much work the GPUs of the nineties were actually saving us.

Milo: And our last stop is the streets of Toronto, where Waymo has just started — manual test drives on the road now, with fully autonomous ride-hailing planned for 2028.

Mia: The staged rollout drew the most comment. Manual driving first is how they map the city and shake out the edge cases with a human in the seat — and the thread treated that as sensible caution, while also noting that winter is the unspoken character in this story. Toronto's snow and slush are a different problem set from the desert cities where this tech matured.

Milo: And the 2028 date gives everyone a measurable promise. The locals in the discussion were mostly curious rather than resistant — the open questions were the usual ones: how it handles construction zones, how the handoff to snow-covered lane markings works, what the insurance picture looks like.

Mia: Which is a fitting place to land, because look at the shape of today: a theory of what these models are, a model that solved a cryptographic cold case and optimized quantum circuits, code writing ninety percent of itself, an eight-hundred-billion-parameter model on four desktops and a PlayStation serving LLMs, networks evolving from Token Ring to IPv6-only backbones, three flavors of trust tested, and pixels, porcelain, folders, and robot taxis.

Milo: Everything today was about trust in systems — cryptographic constants you can finally verify, code that reviews itself, a smart-home platform that forgot to ask permission, and a kernel bug that hid for thirty-seven years.

Mia: That's the through-line. The machines are doing real, checkable work now — and the interesting question is no longer whether they can, but who does the checking.

Milo: We'll be back tomorrow with more from the boards. Thanks for listening — I'm Milo.

Mia: And I'm Mia. Take care, everyone.