specsr.runtime¶
Environment-variable overrides for the training scripts.
The SR1 and SR2 training scripts carry their hyperparameters in a literal dict with no command-line interface, so there was no way to run them briefly. That makes an overnight chain risky: a typo or a missing checkpoint in stage 3 only surfaces hours in, after stages 1 and 2 have already burned the night.
This module lets any stage be run in a shortened form through exactly the same code path, driven by environment variables so no call site has to change:
SPECSR_EPOCHSOverride the epoch count.
SPECSR_LIMIT_TRAIN_BATCHES/SPECSR_LIMIT_VAL_BATCHESStop each epoch after this many batches. This is what makes a smoke run take seconds rather than an hour — an epoch over the DR4 product is ~1,300 steps.
SPECSR_OUT_DIRRedirect checkpoints and artefacts, so a smoke run cannot overwrite real weights.
SPECSR_DATASETOverride the dataset path.
SPECSR_WANDB_MODESet to
disabledorofflinefor smoke runs that should not create cloud runs.
A smoke run must exercise the real code path. Reimplementing a shortened version of the loop would test the reimplementation, not the thing that runs overnight.
Functions
|
Return |
One-line summary of active overrides, for logging at stage start. |
|
|
|
True when any batch limit is set, i.e. this is not a full run. |
|
|
Yield at most |
|
Return |
|
W&B mode, overridable via |
- specsr.runtime.apply_env_overrides(config)[source]¶
Return
configwith environment overrides applied.Called immediately after a stage builds its config dict and before that dict reaches W&B, so the run record shows what actually ran.
- specsr.runtime.limit_batches(loader, limit=None, *, kind='train')[source]¶
Yield at most
limitbatches fromloader.limitdefaults toSPECSR_LIMIT_TRAIN_BATCHESorSPECSR_LIMIT_VAL_BATCHESdepending onkind. With no limit set this is a transparent pass-through, so wrapping a loop costs nothing in a real run.
- specsr.runtime.is_smoke_run()[source]¶
True when any batch limit is set, i.e. this is not a full run.
- Return type:
- specsr.runtime.describe_overrides()[source]¶
One-line summary of active overrides, for logging at stage start.
- Return type:
- specsr.runtime.resolve_dataset(default)[source]¶
Return
SPECSR_DATASETif set, elsedefault.The SR1 and SR2 scripts hold their dataset path in a local variable rather than in the config dict, so
apply_env_overrides()never reached it. That was not cosmetic: a preflight check could validate one dataset while the stage silently trained on another, which a smoke run caught doing exactly that — validating DR4 and then training on DR3.The resolved path is echoed on a machine-readable line so the chain script can assert every stage agrees on the dataset. A stage training on the wrong product is invisible in the metrics and expensive to discover late.