Work in progress, failures included.
Every test has its prediction and its threshold, written in advance. Here is the result, including when the prediction fails. A test marked “not evaluated” has not passed: something needed to run it is missing, and we say what.
Holds
The headline figure, redone
The benchmark run behind the promoted figure was repeated: it comes back byte for byte. That means it is reproducible, not that it is faster or more correct.
Fails
The frozen engine, under the Metal validator
The SHAKE test the frozen engine runs at start-up reads outside its buffer twice: the prediction “no out-of-bounds access” fails, the prediction “the run is valid” passes. Whether it touches the bits of the runs is still to be worked out. Another SHAKE defect, which only distorted the residual printed at start-up, is fixed in the live source; the frozen bundles stay as they are.
Passes
The force bench
Seventeen checks on the engine's forces: 11 pass, none fail, 6 were not evaluated.
Holds
Energy, term by term
The sum of the drifts of each energy term matches the drift of the total, in all eight replicas and in four tests out of four. Limit: the total is the same sum of the same terms, so the test checks how the terms are recorded, not the physics.
Explained
A small difference, traced
The bonds of the test system were 0.18 kcal/mol away from the published files. Four competing hypotheses, each with the measurement that could knock it down: three fail, the fourth holds. The cause is the rounding of the coordinates to three decimals in the Protein Data Bank file.
Partly passes
Structure quality
A check that places a structure among a thousand experimental structures from the Protein Data Bank passes on chirality and atomic clashes. The tests on backbone angles and rotamers are not evaluated: the tool does not compute them yet.
Internal
An energy landscape map
On the alanine dipeptide the sampling and analysis tests pass; one is not evaluated, because a reference value to read at the source is missing. An erratum is written: a surface from an earlier version belonged to the D enantiomer. Until that test exists, the map remains an internal tool.
Fails
Grouping sequences
One criterion of the map that groups proteins by similarity fails on two pairs. It will be tried again after removing the purification tags from the sequences, with a new prediction.
Passes
A state moved from one Mac to another
A simulation state taken on an M4 Pro and resumed on an M2 reaches the same final file. This holds for this test; across GPU families identity is not assumed, it is compared.
Closed
Two deposits, verified on closing
Two completed studies were closed by re-reading the fingerprints of every signed file: in one, 180 fingerprints and no difference. From now on a new package does not start without first declaring how much computing it needs.
Fail
The suspected causes of idle energy use
The app used too much energy while idle. The two causes suspected first were innocent; the real one was a figure animated every second in the overview. With it removed, it was measured again, before and after, on the same build.
Halfway
A hundred features and fifty new ideas
A programme that screens a hundred features and fifty innovations. Of the thirteen features chosen first, all have taken their first step: six pass or hold, six fail as written, one is under review. The twelve innovations written up as specifications have taken their first step, and more than one has failed. The rest are waiting.
Every result has its deposited prediction, its commit and, where it changes a published figure, its erratum. The entries of the public registry carry only fingerprints and status; the texts of the claims stay inside the project.