LK-99 red flags and other miscellany
Finally publishing these misc notes .. better late than never!
For those who need context, read this piece in Nature News: “LK-99 isn’t a superconductor — how science sleuths solved the mystery”
Red flags
There are a number of factors which I believe were legitimate reasons to be skeptical right away even though they are mostly divorced from the physics at hand:
The name of the institute. “Quantum Energy Research Centre”. This sounds like a crackpot institute. Pseudoscience tends to traffic in the wanton use of physics terms to sound scientific. “Quantum energy” reeks of this.
The quality of the graphs. I have found the quality of graphs is a very useful heuristic for quality of work. The graphs were OK but not great. A number of people on Twitter mentioned this (for instance Andrew Cote).
Suspect video on website. Their website is down right now, but previously they had a rather odd video showcased (see below). The video shows the a thin copper plate with LK-99 film deposited on it dangling from a thread. When a magnet is thrust in its direction, the plate swings slightly. One wonders what this video is supposed to prove. A changing magnetic field will induce a current in a conductor. The induced current creates an opposing field, which generates a force. This is Lenz’s law, one of the elementary consequences of Maxwell’s equations. People on social media were quick to show that you can replicate this video with just the copper alone!
Suspect website. The Q-Centre website appeared to be more about trying to attract investors rather than make a rigorous scientific argument.
The circumstances of publication. The finding of room temperature superconductivity was first published in Korean Journal of Crystal Growth and Crystal Technology on April 30, 2023. This is hardly the proper venue for announcing such an important finding. The impact factor of this journal is only 0.25. Journal impact factors are crude metrics, but in general lower impact journals tend to have lower-quality research that is more likely to contain gross mistakes and not replicate.
The resistivity didn’t actually drop to zero. This was probably the biggest red flag I and others noticed right away. The resistivity doesn’t go to zero - not even close. The resistivity of Copper is 1.7 microOhm-cm. The paper’s graphs show the resistivity dropping to a value 100 or 1000 x this. People were quick to point out that this looks more like a metal-insulator transition, not a superconducting transition. Furthermore, the drop is not instantaneous. When a superconductor goes superconducting, it goes in one sharp drop. That’s not what is seen here — instead it drops over the course of several degrees. That can happen if there are different domains or different parts of the material which undergo the phase change at slightly different temperatures (which is very common due to supercooling or inhomogeneous heating). A Twitter user says the authors told him over email that they may have made an error when converting resistance to resistivity, something which does not inspire confidence. It would explain the factor of ~30 difference in the resistivity data reported between the two papers, though:
The patent claims the superconductivity only occurs when current flows through it. (See discussion on Reddit). First, there is no known mechanism that could explain this. Also, the pictures they present of “levitation” are kinda hilarious, given that the “levitating” object is dangling from two thin wires. What they are claiming to show in these pictures is that the levitation only occurs when current flows through the wire.
From the LK-99 patent. Shows “levitation” of a sample when current is flowing, but not when no current is flowing.
Magnetic susceptibility measurement issues. Shortly after publication, a number of people pointed out that the magnetic susceptibility measurements were inconsistent. The susceptibility measurements reported on two different graphs differed by a factor of 4,000. It appears in one of the graphs the authors forgot to divide by H, which is an elementary mistake. The authors claimed the susceptibility was “5,000 times the diamagnetism of graphite”, an incredible claim. The “magnetic susceptibility” they reported (“mass susceptibility”, reported in emu/g) is very non-standard, and apparently one of the measurements translates to a traditional magnetic susceptibility of -15.1 (or -154?) which is impossible (superconductors have χ = -1, which is perfect diamagnetism). More complete discussion here. The author notes that improper sample placement (off just by a few mm) can cause a ferromagnet to appear as a diamagnet, and this is a common issue.
Vague synthesis protocol. For instance, they stated they heated the sample for “5-20 hours”. As far as their process for chemical vapor deposition, they did not explain their process at all, and they also didn’t say what temperature they made certain I-V measurements at with the thin film.
Synthesis was unbalanced and messy. The synthesis was unbalanced - this makes it very likely the end material will be a composite of LK-99, Cu2S, Pb, and Cu.
Their theory discussion didn’t make sense. A number of experts commented on Twitter about this, but they were largely ignored. (For instance, see this note by Andrew Cote… I’m too lazy to dig up the other tweets.)
The measured superconductivity broke down easily. It broke down at at very low currents and very small magnetic fields. Small, delicate effects are more likely to be the result of experimental error, as numerous cases of “pathological” science show.
Residual resistivity in their V-I curves. Normally these would be I-V curves, since voltage is the independent variable it should go on the x-axis. However since R = V/I they probably did it this way so the slope could be the resistivity. Still, the non-standard graphing irks me a bit. Anyway, if you zoom in on their graphs you can see there is residual resistivity.
What went wrong with impurities
(Material taken my Aug 17th tweet)
As of mid August there were at least seven replication attempts that tried to use the same preparation method used by the Korean authors, reacting powders in a crucible at high temperature. All of these papers noted issues with impurities. Note that "impurities" here refers to grains (“domains”) of material embedded within LK-99 that make characterization difficult. Note that this is different than doping, where particular points in the lattice are randomly substituted by a different type of atom. I think there has been some confusion about this point because LK-99 is a lead apatite material doped with copper (more on that later).
As explained by Prof. Michael S. Fuhrer, impurities can cause extreme measurement error when trying to measure resistivity with a four-point probe measurement. If part of the sample is conducting and part is insulating you can get a measurement of very tiny resistance if "the insulating part happens to separate the voltage probes from the current probes".
There is no physical mechanism I can think of where the presence of impurity grains can suddenly turn an insulating material superconducting. In general the presence of impurity grains can't change the bulk properties of the material they are embedded in because the relevant forces are too short-range (ie Van der Waals, Coulomb or exchange repulsion). (A exception to this might be ferroelectric materials, which can generate a fairly large electric field.) So, it is quite a stretch to imagine that the presence of impurity grains is important to the putative superconductivity of LK-99.
Here are the different types of impurity:
Cu2S (Copper(I) sulfide) This impurity was noted in the X-ray diffraction data in the original paper. It undergoes a transition from an insulator to a metal as temperature is cooled below 104 C. This impurity was responsible for the sudden drop in R vs T and discontinuity in heat capacity observed by the Korean group. (see these papers)
Grains of pure copper. This impurity was noted by Hou et al. from Southeast University in their samples as well as others. Hou et al. used energy dispersive spectroscopy (EDS/EDX) to spatially map the composition of their entire sample. They noticed "a substantial amount of Cu particles". Jiang et al. also found unreacted copper which was "distributed homogeneously through the material".
Grains of pure lead. Hou et al. did not notice evidence of unreacted lead in their EDS measurements, but they did measure a drop in resistivity of their sample at 7K, the temperature that lead becomes superconducting. Lead is a conductor and is only very weakly diamagnetic.
Grains of pure iron. Obviously this is a conductor and strongly ferromagnetic material. This was found in samples created by Andew McCalip. Iron contamination can be responsible for the "half-levitation" behavior seen in some grains of material. This was shown by Timokhin in their work. They isolated the grains that exhibited "half levitation" and "flipping" behavior near a strong magnet and analyzed them using EDX. They state: "As confirmed by EDX, these magnet-responsive particles have iron inclusions".
Uneven copper doping. Researchers in both Germany and China have found that the level of copper doping is heterogeneous - some regions have a lot, some have very little. The strongly doped regions are strongly-paramagnetic/soft-ferromagnetic, while the weakly doped regions are diamagnetic. As John Timmer at Ars Technica explains: “When the local copper concentration was high enough, the material behaved like a standard magnet. This provides an additional opportunity for some portions of the material to be repelled by having a magnet placed near it.”
Percolation effects? Hou et al. hypothesize that conducting impurities that are isolated at high temperature can become joined at lower temperature as the LK-99 material they are embedded in shrinks. As the "percolation threshold" is reached, this could lead to a sharp drop in resistivity at low temperature, mimicking a superconductivity transition.
Further notes on this
Overall reflection
Overall I think this episode was good, mainly because of the excitement that was generated online around materials science. As many people said, it was a welcome break from the constant drumbeat of AI hype. It’s hard to think of a technology that has not benefited from advances in materials. While materials breakthroughs are sometimes mentioned in passing, they rarely get the full attention and appreciation they deserve. The LK-99 episode also opened a lot of people’s eyes to how superconductivity could transform the world. In fact, I believe it was Andrew Cote’s July 18th thread on superconductor applications that catapulted LK-99 into public consciousness on Twitter. The LK-99 episode shed a light onto what the process of science is really like (messy, often full of confusion) and the importance of replication. I don’t think anything that happened points towards a viable new way of doing science, for a variety of reasons that would take another post to explain.
Some people said this episode implies there is no need for journals. I somewhat disagree with this. Not every purported breakthrough is worthy of a replication attempt. In fact, most are not. If scientists rushed to replicate every purported breakthrough that got published on a preprint server they would never have time for anything else. Replication attempts generally take a lot of work, and should not be undertaken lightly. In this case the replication process was relatively easy and there was a high expected value if it was true, so I do think a few replication attempts made sense even considering all the red flags listed above. It would take multiple posts to really delve into what the best systems are for filtering out junk science and surfacing the good stuff that really is deserving of a lot of attention and replication attempts. Choosing what to work on and what papers to read and study is one of the central problems in research. The journal system does help with this problem to some degree (if something is published in a high tier journal than it is more likely to be high quality). The journal system has a lot of issues though, of course. In any case, I don’t think “just post every paper on a preprint server” is the best solution here, as many have suggested recently.
Prediction markets didn’t work that well here
A couple problems were illuminated. In brief:
People’s priors should have been very low. Extraordinary claims require extraordinary evidence. Numerous arXiv preprints claiming room-temperature, ambient-pressure superconductivity have been published before, it’s just this one went viral. (See here, here, and here for example). If you look at a the progress of superconductors over time, the superconducting transition temperature (T_c) seems to be plateauing over time. This suggests some sort of fundamental barrier is being reached. There has been no progress in T_c at room temperature and pressure since around 1995 despite intense exploration and effort. Finally, if you understand the (known) physics of superconductivity, you realize that superconductivity probably always requires pairing of electrons, which is fundamentally very difficult. Cooper pairs generally have an interaction energy of 10^-3 eV that is easily disrupted by thermal energy.
The market tracked virality, not scientific plausibility. Eric Hoel wrote in detail about this. Often the market would go up after every “we’re SOO BACK” tweet or viral video of dubious origin. Even after it became clear that LK-99 was not a superconductor, the market remained quite high, around 10%, with upwards buying pressure even many as people were cashing out.
There were too few experts in the market. This is an inference on my part. Ideally, experts should trade larger amounts on the market, countering the vicissitudes of the market. Clearly that didn’t happen, either on Manifold Markets or the real-money Kalshi market.
Theoretical studies don’t mean much
Remember when teams announced they had found evidence for faster-than-light neutrinos? Well most physicists were rightly very skeptical of the finding. Many physicists published theory papers on arXiv attempting to explain the finding. One website I saw (which I’m having trouble finding now) listed well over 100 theory preprints all attempting to explain the finding. Whenever there is some unexplained observation, theorists rush in like bees to honey, seeking the fame and glory that might come with being the first to explain some “new physics”. This case was no different. In fact, in this case it was especially easy for theorists to concoct papers - the crystal structure was apparently already established (not entirely clear on this), so it was easy to take off-the-shelf quantum simulation software and run a bunch of calculations. These simulations involve many approximations and limitations. While Sinead Griffith posted her theory paper with a “mic drop” meme, she was pretty clear about the many limitations of her work in her paper and in a subsequent Twitter thread.
Further reading
Metalessons from the LK-99 saga by Ben Reinhardt
LK-99 as a cautionary tale for prediction markets by Erik Hoel
Great summary, thanks for posting. making room T superconductors is not easy. Sketchy descriptions always worrisome. The hunt for vitality is dangerous ... but, alas, our current reality. The truly great shun it.
Great summary of the early "warning signs" we should have heeded with LK-99. On the bright side though, LK-99 illustrated just how many people would pounce upon the discovery of a room temperature superconductor. If physics allows, I think it is only a matter of time.