Finding files in a data folder

ExamBase and GESeries never hardcode a path to a series’s NIfTI or .mat file — export tools name folders differently (002_3D_Ax_T1_BRAVO vs. Series0002_3D_Ax_T1_BRAVO) and people rename them, so folder names are free-form labels that nothing reads. A NIfTI is found by the SeriesNumber in its dcm2niix JSON sidecar, and every load goes through the same handful of discovery functions in utils.file_helpers. Reaching for them directly is useful any time you want one file without loading a whole series object.

Function	What it does here
`get_exam_folder()`	finds the folder containing raw scanner exam data
`get_mat_data_from_series()`	finds a series’s reconstructed `.mat` file
`get_niftis_from_series()`	finds a series’s NIfTI by its sidecar’s `SeriesNumber`, wherever it sits
`get_all_nifti_series_ids()`	lists every series ID with a NIfTI, and any gaps
`move_files_with_glob()`	archives matching files into an `old/` subfolder, timestamped

from pathlib import Path

from mnutils.utils import file_helpers as fh
from mnutils.testing import build_fake_exam

lg_folder = build_fake_exam("mrsi_missing_series_exam")

The exam folder and a series’s `.mat` file¶

get_exam_folder looks for the one subfolder with “Exam” in its name — that’s where raw scanner exam data (.mat reconstructions) lives, separate from the NIfTIs next to it.

exam_folder = fh.get_exam_folder(lg_folder)
mat_path = fh.get_mat_data_from_series(lg_folder, 7)
print(exam_folder)
print(mat_path)

/tmp/mnutils_fake_exam_cache/a8698bb39c05/mrsi_missing_series_exam/Exam4873anon
/tmp/mnutils_fake_exam_cache/a8698bb39c05/mrsi_missing_series_exam/Exam4873anon/Series7/ScanArchive_Series7.mat

Both raise rather than guess when the folder structure doesn’t match what’s expected:

import pytest

# No "Exam*" folder at this level -- base_path is already inside one.
with pytest.raises(ValueError, match="No folder with"):
    fh.get_exam_folder(lg_folder / "Exam4873anon")

# Series 5 has no .mat file in this dataset.
with pytest.raises(FileNotFoundError, match="No .mat files found"):
    fh.get_mat_data_from_series(lg_folder, 5)

Several ScanArchive*.mat files can sit in one series folder: the raw archive plus files derived from it (..._csi.mat, ..._snr_map.mat). The plain archive wins; when only derived files exist, the one with the shortest suffix does, and a warning names the pick.

NIfTI files, whatever their folder is called¶

A NIfTI counts when it has a JSON sidecar of the same name with a SeriesNumber, so the folder layout is irrelevant: 002_3D_Ax_T1_BRAVO and Series0002_3D_Ax_T1_BRAVO resolve identically. Derived files without a sidecar (_bet, _resampled) are never picked, and if a series has several sidecar pairs (echoes, dynamics) the lowest file name is used, with a warning.

old_style = fh.get_niftis_from_series(build_fake_exam("brain_mrs_mrsi_exam") / "data", 2)
new_style = fh.get_niftis_from_series(build_fake_exam("brain_extraction_exam") / "data", 2)
print(old_style.parent.name)
print(new_style.parent.name)

002_3D_Ax_T1_BRAVO
Series0002_3D_Ax_T1_BRAVO

# STRICT TESTS: get_niftis_from_series across folder naming
assert old_style.parent.name.startswith("002_")
assert new_style.parent.name.startswith("Series0002_")
with pytest.raises(FileNotFoundError, match="dcm2niix"):
    fh.get_niftis_from_series(build_fake_exam("brain_mrs_mrsi_exam") / "data", 99)

Which series IDs actually have a NIfTI¶

get_all_nifti_series_ids returns (found, missing) — every series ID with a NIfTI, and any integers skipped in that range (a genuinely missing/corrupt series, not just “series ID 1 doesn’t exist because numbering starts at 1”). ExamBase uses this internally to build its overview.

Only IDs ≤ 99 are checked for gaps, and only up to the highest one that exists. Those are the protocol steps, numbered densely as the exam runs, so a hole there means something really is absent. Everything from 100 up — reformats, resaves, late additions like 500/501 or 40003 — is sparse by design, and neither gets flagged nor stretches the checked range.

found, missing = fh.get_all_nifti_series_ids(build_fake_exam("brain_mrs_mrsi_exam") / "data")
print(f"{len(found)} series found, gaps at {missing}")

4 series found, gaps at [3, 5, 6, 7, 9, 10, 11, 12, 13]

# STRICT TESTS: get_all_nifti_series_ids
# Series 13 has raw exam data but no NIfTI in this dataset (see docs/data-model/geseries.md) --
# it shows up as a gap here, exactly what ExamBase's classification depends on.
assert 13 in missing
assert 13 not in found
assert found == sorted(found)

# The <= 99 rule, pinned. This regressed once (#34): bounding the checked range by the
# largest series present reported every unused integer below 100 as missing as soon as
# any reformat/resave series existed.
import tempfile


def missing_for(series_ids):
    """Report the gaps for a data folder holding NIfTIs for exactly these series IDs."""
    with tempfile.TemporaryDirectory() as tmp:
        for series_id in series_ids:
            (Path(tmp) / f"s{series_id}.nii.gz").touch()
            (Path(tmp) / f"s{series_id}.json").write_text(f'{{"SeriesNumber": {series_id}}}')
        return fh.get_all_nifti_series_ids(tmp)[1]


# Sparse IDs neither get flagged nor stretch the range up to 99.
assert missing_for([*range(1, 15), 113, 200, 300, 500, 501, 650, 651, 40003]) == []
assert missing_for([1, 2, 4, 113]) == [3]  # a real gap between protocol steps still counts
assert missing_for([113, 500]) == []  # nothing <= 99, so nothing to check
assert missing_for([]) == []  # an empty folder is not an error

Archiving old output files¶

move_files_with_glob is unrelated to loading — it’s what RawMRISeries/MRSISeries use internally before writing new processed-data output, so a re-run doesn’t silently overwrite a previous result. It moves every match into an old/ subfolder, prepending a timestamp so repeat runs don’t collide.

import tempfile

with tempfile.TemporaryDirectory() as tmp:
    folder = Path(tmp)
    (folder / "result_a.h5").touch()
    (folder / "result_b.h5").touch()

    fh.move_files_with_glob(folder, "*.h5")

    moved = sorted(p.name for p in (folder / "old").glob("*.h5"))
    print(moved)

['20261016_094939_result_a.h5', '20261016_094939_result_b.h5']

# STRICT TESTS: move_files_with_glob
import re

with tempfile.TemporaryDirectory() as tmp:
    folder = Path(tmp)
    (folder / "a.h5").touch()
    fh.move_files_with_glob(folder, "*.h5")
    moved_files = list((folder / "old").glob("*.h5"))
    assert len(moved_files) == 1
    assert re.match(r"\d{8}_\d{6}_a\.h5", moved_files[0].name)
    assert not (folder / "a.h5").exists()  # moved, not copied

Finding files in a data folder

The exam folder and a series’s .mat file¶

NIfTI files, whatever their folder is called¶

Which series IDs actually have a NIfTI¶

Archiving old output files¶

The exam folder and a series’s `.mat` file¶