The grid builds a whole Figure just to throw it away

Last edited: 2026-08-18 · #13

inspect_MRSI_spectra() stopped going through matplotlib for its anatomical frames and got ~130 ms per slice back, plus sharper images at the volume’s native resolution — the composite is really cmap(norm(arr)) and an alpha blend, and a Figure/Agg canvas is an expensive way to create these images (see the slice-slider entry). display_images() does the same arithmetic and still pays the same tax: a Figure, a gridspec, per-axis imshow, a PNG encode, once per slice. The reason it wasn’t converted with the MRSI path is that its frames carry things the MRSI frames don’t — panel titles, a figure title, a colorbar. Those are what keep the Figure alive.

What the split looks like¶

matplotlib is not gone — it still supplies the colormap through ScalarMappable, so the colorbar gradient and the pixels come from one source and cannot drift. What it stops doing is laying out and rasterising. _render_frames() does the rasterising for every widget in the module (see below); _render_image_grid_frames() asks it for one frame per panel per slice and returns them slice-major together with the (vmin, vmax) each was normalised with; _widgets/image_grid.{py,js,css} turns that into the page.

# 4D input, four panels, a shared colorbar: displays, returns nothing.
mnutils.plotting.display_images(
    [t1, pdff, r2s, mask], titles=["T1", "PDFF", "R2*", "Mask"], colorbar=True
)

That last line is the breaking part. display_images() used to hand back (Figure, axes) for 2D input and None for a multi-slice stack; with no Figure anywhere it returns None always (and so does display_nifti(), and NiiBase.display()). Callers that post-processed the returned axes lose that hook — deliberately, since keeping it means keeping a second full rendering path for one input shape. A single-slice input is now the same widget with no slider — just the bar’s Border checkbox, which draws each frame’s real extent back in, since NaN pixels are transparent and a mostly-masked panel otherwise has no visible edge.

The measurement that settled it, on the 90×90×100×7 volume docs/plotting/images.md generates — 700 panels against the old path’s 100 whole-grid PNGs:

Native resolution being cheaper here is the opposite of what the MRSI frames found, and for a plain reason: an MRSI frame is one 512² image where the dpi-derived alternative was 400², while a grid frame is one panel out of seven that the old path rendered inside a 1400×600 figure canvas padded with white margins, ticks and text. Dropping the canvas drops most of the pixels.

The second renderer that shouldn’t have been¶

Porting the grid produced a renderer that was, line for line, the one the MRSI inspector already had: normalise, colormap through ScalarMappable, encode WebP on a thread pool. They differed only in their layers — one opaque panel, against a gray T1 under a half-transparent magma overlay — and in which axes an index walked. Two copies of that is how the panels and the inspector drift apart later.

So there is one frame renderer, _render_frames(), and the difference is data. A _Layer names a volume, its colormap, its bounds and its alpha; a job is the trailing index, so volume[:, :, *job] is the image that frame colormaps. Layers composite in order, reproducing what imshow(alpha=…) does — the layer’s alpha scaled by the colormap’s own per-pixel alpha, so “bad” pixels let the layer below through untouched instead of blending toward the bad colour.

_render_image_grid_frames() survives, but it no longer renders: it resolves the bounds each panel is normalised with — one shared pair, or one per panel and slice under colorbar_kws={"mode": "each"} — and nests the flat result the way the widget indexes it.

Handing the layout to CSS means saying what “too big” means¶

A Figure came with a size. A raster doesn’t, so every bound the figure used to imply had to be written down — and the useful version of each turned out to be the one expressed in the other axis from the one it names.

Sizing panels by height was the first attempt and it fails in a narrow pane: height is not a thing CSS can hold constant while width has to give. So the panel fills its column and the column is what yields — and the floor becomes a width too, --mnu-panel-min-h × aspect, so a row of 5×20 frames wraps sooner instead of flattening into strips. --mnu-grid-max-h likewise bounds the scroll box rather than the panels: dividing a fixed height budget between rows made a twelve-panel grid twelve stamps, where scrolling keeps twelve readable panels.

Two decisions on the raster side follow from the same “the browser is doing the drawing now” shift. Frames scaled up get image-rendering: pixelated, set per image by the JS once it can compare naturalWidth against the rendered width, so a 20×20 mask reads as 400 square pixels rather than a blur — and frames under 64×64 are encoded losslessly, because lossy WebP rings around hard edges. The threshold stays low deliberately: at 90×90, lossless costs 4.4× the payload to fix artifacts nobody can see at that scale.

What changed from the plan¶

• The plan expected colorbar_mode="each" to be the hard part, since it nulls vmin/vmax and lets matplotlib autoscale each panel per slice. Recomputing those bounds in Python turned out to be the easy half; the awkward half was that a shared colorbar can also have a degenerate range (a constant image, or zeros_as_nan on an all-zero array), where matplotlib silently rendered the colormap’s low end and a text label would have read “0 to 0”. Both paths now widen a zero-width range to (low, low + 1).

• Pass 1 assumed the docs page’s strict tests could keep asserting on shapes. They can’t — there is no return value to assert on — so docs/plotting/images.md checks the panel count on _render_image_grid_frames() directly, and says in the cell why it reaches for a private helper.

• The plan’s unit of work was “give the grid a renderer”, and that is what shipped first — a second copy of the MRSI inspector’s, which stood on the branch for several commits before the duplication was collapsed into _render_frames(). Reading the two side by side is what showed the difference was only their layers; neither one read that way alone.

• The plan treated the widget’s layout as a detail of the port. It was most of the work: five rounds of it, all driven by the difference between a browser window and a VS Code output pane. The lesson is in the warning above — when the host clips instead of scrolling, “looks right here” is not evidence.