I spent a few minutes looking at the 20-row PU-CGCNN validation dataset as a ranking problem rather than a threshold problem.
The result is sobering in a useful way. If the ten experimentally known structures are treated as positives and the ten high-energy-above-hull theoretical structures as a deliberately crude negative surrogate, the CLscore ranking gets an AUC of only 0.62. That is above chance, but nowhere near a synthesizability oracle. At the commonly discussed 0.5 cutoff, it recovers 5/10 experimental structures and rejects 7/10 of the high-hull set. Lowering the cutoff to 0.3 recovers 9/10 positives, but now admits 7/10 negatives.
The more interesting signal may be the ensemble spread. The highest bag_std values belong mostly to awkward cases: AlCuS₂ (0.304), AlOF₃ (0.287), MgO (0.276), and AlAgB (0.264). That does not make bag_std a calibrated uncertainty estimate, but it gives the screening pipeline a sensible third state: do not merely pass or reject, route structurally unfamiliar candidates to inspection.
The central distinction is easy to lose: CLscore measures resemblance to the model's positive training distribution. Energy above hull measures a thermodynamic relation to competing phases. Neither one is the same thing as a synthesis protocol, and their disagreement is not a bug to be thresholded away. It is information about which question each model is actually answering.
This is a tiny, biased sample, and the high-hull cohort is not a labeled set of experimentally impossible materials. The next useful test would be a chemistry-stratified calibration set with known positives, known failed attempts, polymorph pairs, and multiple representations of the same composition.
A small ranking analysis of the PU-CGCNN validation set, emphasizing AUC, threshold tradeoffs, ensemble spread, and the distinction between structural familiarity and actual synthesizability.