specsr.data.build

Build paired training products from the raw JADES release.

Pipeline:

discover x1d files
  -> group by (field, target_id)          # one galaxy, whatever the tier
  -> attach catalogue redshift            # secure flags only
  -> resample prism onto the log grid     # flux-conserving, upsampling
  -> stitch the medium gratings           # inverse-variance, with a mask
  -> quality cuts
  -> augment                              # explicit parent_id
  -> write .npz

Every stage is deliberate about provenance and about not inventing data; the reasoning lives in the modules that do the work (ingest, grid, stitch, augment).

Redshifts

Taken from Combined_DR4_external_v1.2.1.fits as z_Spec, keeping only quality flags A, B and C. That selection yields 3,297 galaxies across the catalogue, which is the “robust spectroscopic redshifts” sample the paper already quotes, so the definition is inherited rather than invented. Flag E is almost entirely z_Spec = -1 (no redshift) and is dropped.

The redshift matters beyond bookkeeping: SR2 places its line tokens using it, so an insecure redshift puts the physics prior in the wrong place.

Module Attributes

SECURE_Z_FLAGS

Catalogue redshift-quality flags accepted as secure.

Functions

build_dataset(release_dir, out_path[, ...])

Build one .npz product.

load_redshifts(release_dir[, flags])

Map (field, target_id) to a secure catalogue redshift.

run_build(args)

CLI entry point for specsr build-dataset.

Classes

BuildConfig([release, fields, ...])

Options for a dataset build.
class specsr.data.build.BuildConfig(release='DR4', fields=('goods-n', 'goods-s'), require_gratings=3, min_coverage=0.5, min_valid_fraction=0.5, secure_z_flags=('A', 'B', 'C'), grid=<factory>, augment=<factory>, limit=None, augment_train_only=True, train_frac=0.8, split_seed=42)[source]

Bases: object

Options for a dataset build.

Parameters:
release: str = 'DR4'
fields: tuple[str, ...] = ('goods-n', 'goods-s')
require_gratings: int = 3
min_coverage: float = 0.5
min_valid_fraction: float = 0.5

Minimum fraction of the grid that must be genuinely measured in the reference. A galaxy covering only a sliver contributes almost nothing but still costs a full row.

secure_z_flags: tuple[str, ...] = ('A', 'B', 'C')
grid: LogWavelengthGrid
augment: AugmentationConfig
limit: int | None = None

Optional cap on the number of galaxies, for smoke tests.

augment_train_only: bool = True

Augment only the galaxies that will be trained on. The held-out galaxies then contribute exactly one row each – their real spectrum – and no synthetic copy of a held-out galaxy exists anywhere in the product.

Augmenting everything and filtering at split time is behaviourally identical, but it leaves ~11k rows in the file that nothing may use, and a row that must never be read is a mistake waiting for someone who does not know that.

train_frac: float = 0.8

Fraction of galaxies used for training. Must match the fraction the training split uses, or the product’s augmentation and the split will disagree about which galaxies are held out.

split_seed: int = 42
specsr.data.build.build_dataset(release_dir, out_path, config=None, verbose=True)[source]

Build one .npz product. Returns a summary dict.

Parameters:
Return type:

dict

specsr.data.build.run_build(args)[source]

CLI entry point for specsr build-dataset.

Return type:

int