AlphaFold Isn't Really About AI – Michael Nielsen
TL;DR
AlphaFold’s real breakthrough was data, not just AI — Michael Nielsen says the core story is the Protein Data Bank: roughly 180,000 structures built through X-ray diffraction, NMR, cryo-EM, and “several billion dollars” of experimental work over decades, with the model itself only a small fraction of the total investment.
This is a challenge to the classic idea of scientific explanation — Dwarkesh contrasts elegant theories like general relativity, which explain a lot with a few equations and predict surprises like Mercury’s orbital precession, with AlphaFold’s 100M+ parameter system that works powerfully but lacks obvious explanatory reach.
Nielsen lays out three ways to think about models like AlphaFold — the conservative view is that they’re useful but not real explanations, the middle view is that they contain “little explanations” humans can extract, and the boldest view is that they’re a genuinely new kind of scientific object.
Interpretability could turn black-box models into explanation engines — Nielsen compares this to “archaeology of AlphaFold,” where researchers might dig through circuits and internal structure to recover principles, much like people have reportedly inferred strategies from AlphaZero.
He points to chess as an early example of humans learning from opaque models — Nielsen mentions expert speculation that Magnus Carlsen’s style shifted after public forensics on AlphaZero, suggesting models can influence human understanding even before they become fully legible.
The deeper bet is that science may evolve tools for working with complexity directly — Nielsen compares AI models to the way Mathematica let mathematicians and physicists keep working with 100-page equations that would have been hopeless in 1920, opening up a new philosophy of science around what counts as an explanation.
The Breakdown
AlphaFold Is Mostly a Story About Protein Data
Nielsen opens with a strong claim: AlphaFold “really isn’t about AI.” He says a massive share of the success comes from the Protein Data Bank — around 180,000 structures gathered through X-ray diffraction, NMR, and cryo-EM after decades of painstaking lab work and several billion dollars of investment. In his framing, the flashy model sits at the end of a very long pipeline of humans “looking very hard at the world experimentally.”
Dwarkesh Pushes on What Counts as Explanation
Dwarkesh then pivots from engineering to philosophy, almost stopping himself mid-question because Nielsen is “such a careful speaker.” His core challenge: are systems like AlphaFold actually doing science in the same way classic theories do, or are they just fitting gigantic models to reality? He contrasts them with general relativity, where a small set of equations can explain broad phenomena and even predict things like Mercury’s orbital precession.
The Conservative View: Useful Model, Not Scientific Theory
Nielsen’s first answer is the traditional one. If you think science should aim for deep explanatory principles, very few free parameters, and simple models that cover a lot, then AlphaFold does not qualify. On that view, it may be powerful and helpful, but it’s not a scientific explanation in the classic sense.
The Middle Ground: “Archaeology of AlphaFold”
His second answer is more interesting: maybe AlphaFold isn’t itself an explanation, but it contains many small explanations inside it. He says interpretability work could let us do “archaeology of AlphaFold,” digging through the model to extract principles — finding circuits, understanding what they do, and learning from them. The model starts looking less like a final theory and more like a rich mine of partial insights.
AlphaZero and the Possibility of Extracted Meaning
To make that concrete, Nielsen points to chess models, especially AlphaZero. He says there’s been at least some work where people seem to have extracted strategies from the model, and mentions speculation that Magnus Carlsen may have shifted his style after public forensics on AlphaZero’s play. Even without full interpretability, the model may already be teaching humans.
A Third Possibility: A New Kind of Scientific Object
Nielsen saves the most provocative option for last: maybe these systems should be taken seriously as explanations, just not in the old sense. Instead of elegant closed-form theories, they may be a new class of object that scientists can merge, distill, manipulate, and reason over. That, he says, could open a genuinely new area in the philosophy of science.
From Impossible Equations to Computable Objects
His analogy is great: in 1920, if a physicist ended up with a 100-page equation, the problem was basically dead. Today, with tools like Mathematica, those giant expressions become workable objects you can keep operating on until something simpler drops out at the end. Nielsen wonders if models like AlphaFold will become that kind of thing for science — not classic explanations, but serious tools we can operate on to get understanding anyway.