specsrbench.metrics

Reconstruction metrics, and the four guards that keep tuning honest.

MAE alone is not a safe objective, in four separate ways, and no one guard catches the others:

  1. Smoothing – erase every line and no line-shaped residual is incurred.

  2. Shrinkage – scale toward zero and absolute error against a noisy reference falls, whatever the reconstruction quality.

  3. Blurring – a unit-gain Wiener filter with snr <= 1 peaks at zero frequency, so it can only broaden lines, and the amplitude guard is width-invariant.

  4. Merging – all three of the above can pass while two close lines are smeared into one peak, because a Gaussian fitted to a blended doublet has much the same amplitude, S/N and width as one fitted to a separated pair.

mae_scalefree() and std_ratio() are what make (2) visible, and they live here rather than in a build script because the tuner, the cache build, the figures and the tests all have to agree on them. They did not, once: the tuner retuned every method against a corrected kernel while the build went on using the old parameters, and every classical number in the paper was wrong for a day while the whole test suite passed.

Module Attributes

STD_RATIO_LO

Amplitude guard band.

STD_RATIO_HI

Amplitude guard band.

Functions

bias(pred, truth, mask)

bootstrap_std(per_spectrum[, n_boot, seed])

Standard deviation of the mean under resampling of spectra.

fwhm_from_sigma(sigma_um)

Gaussian FWHM in nanometres from a fitted sigma in microns.

global_stats(name, pred, truth, mask, *[, ...])

One row of the global-fidelity table, guards included.

mae(pred, truth, mask)

Mean over spectra of the per-spectrum mean absolute error.

mae_scalefree(pred, truth, mask)

MAE after rescaling each spectrum by its own least-squares optimal gain.

rmse(pred, truth, mask)

std_ratio(pred, truth, mask)

Output scale over target scale.

zscore(arr)

Per-spectrum z-score, the units every cached reconstruction is in.
specsrbench.metrics.zscore(arr)[source]

Per-spectrum z-score, the units every cached reconstruction is in.

Parameters:

arr (ndarray)

Return type:

ndarray

specsrbench.metrics.mae(pred, truth, mask)[source]

Mean over spectra of the per-spectrum mean absolute error.

Return type:

float

specsrbench.metrics.mae_scalefree(pred, truth, mask)[source]

MAE after rescaling each spectrum by its own least-squares optimal gain.

Invariant under any global rescale of pred by construction, which is the entire point: shrinkage cannot improve it. This is the metric on which SR2 ranks eighth of nine while leading the raw-MAE table by 30%.

Return type:

float

specsrbench.metrics.std_ratio(pred, truth, mask)[source]

Output scale over target scale. 1.0 means amplitude was preserved.

Return type:

float

specsrbench.metrics.rmse(pred, truth, mask)[source]
Return type:

float

specsrbench.metrics.bias(pred, truth, mask)[source]
Return type:

float

specsrbench.metrics.bootstrap_std(per_spectrum, n_boot=1000, seed=42)[source]

Standard deviation of the mean under resampling of spectra.

Spectra, not pixels: pixels within one spectrum are correlated, and resampling them would report an error bar several times too small.

Parameters:
Return type:

float

specsrbench.metrics.fwhm_from_sigma(sigma_um)[source]

Gaussian FWHM in nanometres from a fitted sigma in microns.

Return type:

ndarray

specsrbench.metrics.global_stats(name, pred, truth, mask, *, n_boot=1000, seed=42)[source]

One row of the global-fidelity table, guards included.

MAE and MAE_scalefree are reported together, always. Quoting the first without the second is how this paper twice came to state a conclusion that the scale-free number reverses.

Parameters:
Return type:

dict