I wish Claude Code had been available when I was a PhD student.
Today Claude Code found and fixed a bug in code I wrote during my PhD in thirty minutes. During the tail end of my PhD I had searched in vain to find that bug for over three months (more on that below).
A large portion of my PhD research was spent sitting in front of a computer for 8-10 hours a day writing code, usually in Fortran.
Today that same code could be written at least 10 times faster by AI and with better style and documentation.
If I had Claude Code when I was a PhD student I could have two or even three times as productive, all else being equal.
Many physicists are not good at software engineering and don’t find coding and computer science intellectually interesting
The initial goal of my PhD research was to achieve greater understanding of liquid water and why it has such a high dielectric constant. We were also interested in what happens at the molecular level when water absorbs microwaves and infrared radiation. Water has been studied for hundreds of years and its properties are well known. However, there have been open questions about how some of those properties arise, owing to the complexity of hydrogen bonding. For instance there has been debate about whether there are low density and high density clusters of water molecules in liquid water.
Writing code to simulate water and analyze the simulations was a means towards achieving greater understanding of the physics of water.
Many computer scientists I have met have a deep intellectual interest in programming language design, software engineering best practices, and the optimization of algorithms. Those topics never interested me much, nor were they within the core intellectual interests of many of the physicists around me.
We didn’t obsess about optimizing our algorithms except in some cases where it was necessary. We coded things in whatever language would get us what we wanted the easiest. In some cases that was Matlab, in other cases Mathematica. In many cases it was Fortran, a language which computer scientists hate but is sometimes faster than C++ and is much easier to learn. (I wrote about why physicists use Fortran in 2015).
We coded things in a procedural manner since it made the code very easy to write and understand. We didn’t spend time thinking much about “best practices” for writing code and making it maintainable. We just wrote code in whatever way made sense to us at the time to get the desired functionality as easily as possible.
Later during my PhD I researched new ways to simulate nuclear quantum effects in a more computationally efficient manner. One of the things that interested me and others was why light causes water molecules to split into hydrogen and oxygen at the surface of catalysts like platinum (photocatalytic water splitting). We were excited about discovering new ways that hydrogen could be produced from sunlight.
Other researchers I worked with were using quantum simulations to understand how lithium ions are stored in materials, or to understand why D2O ice has slightly different properties than H2O ice.
In short, we were primarily intellectually interested in the physics, not the computer science. We were physicists after all!
Of course, this lack of interest in CS best practices among myself and other physicists had a big downside. It was very hard for physicists to build off each other’s code because it wasn’t written with understandability, maintainability, and extensibility in mind.
The fields of quantum simulation and computational materials science were plagued by lack of data standards. The situation is better today with the rise of Python and owing to the work of data standard initiatives, but much of the same situation exists.
Physics PhD students write the minimal code necessary to solve their problem and are not well-incentivized to do much more. Claude Code on the other hand writes code that follows style guidelines perfectly and is well documented.
Claude Code solved an issue in thirty minutes I spent three months agonizing about during my PhD
In 2015 I developed a new method for incorporating nuclear quantum effects into the simulation of water. Unfortunately, in August that year the simulations started to yield an unexpected peak in the calculated absorption spectra of liquid water.
I spent three months trying to figure out the origin of that peak. I had originally planned to graduate in December 2015, but my advisor wanted me to fix the issue with the peak before I graduated. My PhD defense had to be put on hold (which I was mostly OK with since I didn’t have a job lined up anyway).
My advisor and I were both convinced there was a bug in the code.
I remember going home for Christmas that year and being very stressed out and irritable the entire time because I couldn’t solve the issue. I was really in over my head. Instead of enjoying Christmas with my family I was SSHing into a compute cluster from my laptop, running simulations to try to nail down the precise circumstances in which the issue occurred.
In the end, I never figured out where that extra peak came from. My advisor allowed me to finish my PhD but she didn’t want to publish a paper I had written that contained spectra with the spurious peak.
It was not until after I had left graduate school that I convinced my advisor to allow me to publish the paper. Part of what convinced her is that I had noticed a similar artifact in other published work. Additionally, at the location where the peak exists there’s supposed to be a small peak anyway (for the OH-stretch/bend combination mode) and that gave us a bit of cover in case anyone asked about it.
Out of curiosity, I recently gave the above spectra figure and the old codebase from my PhD to Claude Fable 5.1 and asked it to investigate the issue.
A half hour later, Claude had thoroughly analyzed the code, found the bug, fixed the bug, and had run a short simulation to prove it was fixed.1 As a side note, the code runs absurdly fast on my modern desktop compared to my desktop eleven years ago.
I read Claude’s full explanation of the bug. It’s been almost eleven years now since I worked on that codebase, but Claude’s explanation rings true. The issue has to do with how boundary conditions are handled, which I remember I had some trouble with. I think it was also due to me having a poor understanding of the SIESTA codebase. It is hard for humans to understand complex codebases others have written, but trivial for today’s AI.
I had Claude Fable update the code on Github even though nobody is using it and the code was obsoleted by a better codebase a year or two after it was published.
If the paper were highly cited I would ask the journal to publish an erratum, but this is my least cited physics work, having only one citation. Besides, the method we describe is sound, it just had a bug in implementation. I highly doubt anyone cares where the extra peak comes from.
AI can do all the drudge work
During my PhD I spent countless hours trying to get other codes to compile and installing all the libraries they needed. In our lab we used to joke that half our PhD was spent getting codes to compile. This was all before Docker became popular (although Docker has its own headaches).
It was drudge work that consisted of copying compiler errors into Google and randomly trying different solutions people had posted online until the code compiled. We had no idea what we were doing, we were just copypasting stuff from online until the compile finished. Today Claude Code can install and compile codes easily.
The first four months of my time as a Staff Scientist at NIH were spent porting a very complex and highly disorganized C++ code from Windows into Linux. I don’t know how long that work would take with Claude Code, but probably less than a few days, and possibly only a few hours with Claude Code ultracode turned on.
As a PhD student I also spent many hours creating figures in Mathematica and with a program called Grace. I would spend hours carefully tweaking how the figures appeared, since I was very particular about the aesthetics of graphs and I thought having nice graphs would lead to my work being read and cited more. I enjoyed that work, but neither Mathematica or Grace made it particularly easy. It was extremely time consuming.
Nowadays creating figures is absurdly easy with Claude Code. However, the default AI aesthetics is not great, so human aesthetic discernment is still important.
AI can help theoretical physicists understand math
Theoretical physicists have to chart paths through complex mathematical territories. Sometimes these are territories that are completely new (string theory for instance led to many new mathematical vistas). Other theoretical physicists find themselves working at the intersections of mathematical fields where there are few human experts.
The way physicists approach math is very different than the way mathematicians do. As with code, physicists often look at math methods as tools.
If you look at a random theoretical physics paper from arXiv you’ll very rarely see it containing definitions, lemmas, and theorems (although occasionally they might). The way physicists do math is much looser. The way physicists write about math is different — it’s more of a narrative format.
Theoretical physicists often end up using techniques that give the right answer even though they haven’t come up with a rigorous proof that those techniques are always going to work. The work of rigorously proving out methods often falls on mathematicians to complete, sometimes years or decades later.
Recently OpenAI announced that AI had solved half of one of the millennium problems in mathematics. The solution puts Yang-Mills theory, which undergirds the Standard Model of physics, on a more rigorous footing by deriving it from a set of axioms and proving that “the theory actually exists”. One of my friends from graduate school recently wrote about this.
In a recent YouTube video, theoretical physicist Gabriele Carcassi describes how in the past he would have to beg mathematicians to get interested in the math problems he was facing. Sometimes he would wait days, weeks, or months to get a reply from a mathematician. Often the reply would be “sorry, that’s not in my area of expertise”.
Now every theoretical physicist has access to an AI that has an understanding of math that is on par with the best human math professors. Unlike human mathematicians, the AI is always available, doesn’t get tired, and has expertise across all fields of math. The AI is happy to spend hours explaining complex mathematical topics or helping make math more rigorous. And all that expertise is available for only $25-$30 a month.
In a recent arXiv preprint even Ed Witten acknowledged getting help from Claude Opus 5.5.
The preprint describes some of the ways Claude contributed:
(for the record, I have precisely zero understanding of what he’s talking about here!)
I believe in the next few years theoretical physicists working with AI will be able to break new ground on many longstanding unsolved problems in theoretical physics.
The parts of my PhD research I wouldn’t want to replace with AI
What I found most enjoyable during my PhD research was achieving a clear and precise understanding of complex physical phenomena. Writing was how I achieved that. Writing was central to my research process and the intellectual development I experienced during my PhD.
If I had outsourced a lot of my writing to AI I don’t think I would have grown much intellectually. I probably would have gotten papers published, but my brain would not have developed and the papers would not have been as good. I am worried about physicists and other scientists today outsourcing too much of their writing to AI, because they are outsourcing important thinking as well in the process.
I often found papers going back decades that had explored the origin of the dielectric properties of water. These papers used different terminology and modeling approaches, and fitting them altogether was a big puzzle.
At other times I had to reconcile the physics theory I was understanding from the literature with the outputs from my simulations. I found solving such puzzles intrinsically fun and rewarding.
Writing was how I solved all such problems - I used the page as an external memory for organizing information. Writing was how I synthesized large amounts of technical information into something I felt I could understand in my own tiny brain. While tackling such synthesis problems I developed important skills I still use today.
Like all good writers I chose my terms carefully to maximize clarity and for my own sanity so when I returned to my writing I could refresh my memory easily.
I wrote in LaTeX, and a lot of what I wrote made its way into my thesis and papers. My thesis is on the longer side (317 pages) because writing was so central to my research process. I noticed that others did not write as much as I did. Instead, they collected tons of notes and raw data and then wrote everything up later.
I sometimes think that having a lower IQ than other physicists meant that I had to spend more time distilling and organizing things into clear writing. Counter-intuitively, cognitive constraints can force the creation of clearer explanations.
I like to think that one of the reasons why my physics papers are still cited today is due to the clarity of their exposition.
Of course, it is true that AI is also good at synthesizing and distilling large amounts of technical information into efficient, clear, logical prose. However, I still think doing it manually like I did during my PhD will lead to a better result. That may not be true for long, however.
Even as AI advances, I still recommend PhD researchers undertake the work of synthesis and distillation themselves and not rely too much on AI to do everything. The skills I obtained by doing such synthesis and distillation manually have proven useful for me in many ways.
Additionally, doing the synthesis work manually allowed me to be able to answer questions from peers that were couched in different terminology. AI would have given me one way of explaining things. Doing the work manually exposed me to multiple ways, which allowed me to better communicate with scientists that were working with different approximations, models, and terminologies.
Human creativity is still important
Two out of four of my physics papers had their genesis in “a crazy idea” that I have a hard time imagining today’s AI would come up with. My first paper had its genesis in a crazy idea from my advisor that the physics of relaxor ferroelectrics may be relevant to understanding the dielectric properties of water. The idea ended up having a grain of truth and led us in fruitful directions.
If you ask an AI for promising research directions, it will give you a bunch of things that are the logical next steps to build slightly off existing work. It’s not good at coming up with the sort of “crazy ideas” that humans sometimes do (I have tried to prompt this, but haven’t figured out a way — if you have tips please comment below).
In short, I think humans still have an edge when it comes to creativity.
I also enjoyed the physical experience of going to a library and finding hard copies of obscure books and journal articles that were relevant to my research and reading them. Looking at dusty manuscripts from the 1950s I think gave me a bit of an edge in my research because it sometimes led me to make creative happenstance connections that otherwise would not have been made.
I highly doubt an AI asked to do a literature search for ideas and connections would go down the sort of random rabbit holes I explored which sometimes yielded interesting things.
Of course, the field I was working in was especially good for mining forgotten insights from old literature. Water is really important and there has been an absurd amount of experimental and theoretical research published on the properties of water over the last ~150 years.
I’m also not sure how important the specific connections I found from random explorations of old literature actually were. Usually what was surfaced was just underappreciated ties into other bodies of work that were old and outmoded. I like to think that surfacing those connections was useful to other researchers, though.
Parting thoughts as we look towards an AI-dominated future
Terence Tao has recently been talking about how human cognitive constraints mean that we are forced to distill complex mathematical proofs into “insights”. As you may have noticed, AI loves to get into gory details. It has no compunction about spewing out dozens of pages of complex derivations.
Of course, AI can distill things if asked, but according to Tao it doesn’t distill or explain math in a way that is easy for humans to understand. It seems to lack a good theory of mind for how human mathematicians with their puny brains understand things.
Just as AI is making a lot of progress in math on its own, as we look further down the road, it’s clear that at some point a lot of progress in theoretical physics will largely come from AI working in isolation.
As we move to more and more scientific advances being made by AI, what should be the role of humans? Should humans still study math and physics even if AI can solve all problems? These are complex philosophical questions beyond the scope of this present essay.
However, in the near term I think humans will still be better than AI when it comes to distilling what the AIs are finding in ways that are understandable and insightful for other humans.
Here’s Claude’s explanation of the bug in simple terms: “The setup. In the monomer PIMD scheme the forces on a water molecule are split into two parts. The forces inside each molecule, the two O–H stretches and the bend, come from the Partridge–Schwenke monomer surface and are applied to every bead of the ring polymer. The forces between molecules come from SIESTA and are applied once, at the centroid. For this to work, the force SIESTA hands back must contain only the between-molecule part. Your modified SIESTA did this by evaluating a fit to the DFT monomer surface for each molecule and subtracting it from the DFT force, so what came back was DFT minus fit. Adding Partridge–Schwenke on the beads then counts the intramolecular force exactly once. That is the intended scheme, and most of the time it is what ran.
The weak point. The subtraction has a safety check. Before SIESTA subtracts the fit for a molecule it evaluates the fit energy, and if that energy is above 2.5 eV it assumes the fit is being used outside its range and skips the correction for that molecule. When the correction is skipped, SIESTA returns plain DFT for that molecule. Plain DFT already contains an intramolecular force roughly equal to the fit. Adding Partridge–Schwenke on the beads on top of it now counts the intramolecular force twice. A spring that is twice as stiff vibrates √2 times faster, so the 1611 cm⁻¹ bend moves to about 2290 and the 3922 cm⁻¹ stretch to about 5550.
What tripped the safety check. PIMD-F90 wrapped atoms back into the periodic box one atom at a time. Whenever an oxygen had been wrapped and one of its hydrogens had not, the two atoms sat on opposite faces of the box. SIESTA evaluates the fit on the raw coordinates without applying the minimum image, so it saw an O–H distance of nearly a box length, a huge fit energy, and switched the correction off for that molecule. Molecules diffuse, so this is not a fixed set of molecules. At any instant about one molecule in five was straddling the edge, and over time every molecule spent about a fifth of its time in that state…. The single-molecule runs show the same pattern for the same reason. A lone molecule drifts with its thermal velocity, several ångströms per picosecond, and crosses the box edge constantly…”