Displaying image arrays of any shape

I have an array — could be one 2D slice, could be a volume, could be several volumes — and I just want to look at it.

display_images() takes 2D, 3D or 4D input and picks a sensible layout automatically.

import numpy as np

from mnutils.plotting.images import display_images

rng = np.random.default_rng(1337)

# 4 D volume creation
size_xy = 90
num_z = 100
num_t = 7  # 4th dimension length (e.g. 30 time steps)

# 4D coordinate grids: shapes (90,1,1,1), (1,90,1,1), (1,1,100,1), (1,1,1,30)
y, x, z, t = np.ogrid[:size_xy, :size_xy, :num_z, :num_t]

# 1. Dynamic center that orbits across dimension 4
x_center = (size_xy // 2) + 15 * np.sin(2 * np.pi * t / num_t)
z_center = 50 + 20 * np.cos(2 * np.pi * t / num_t)

# 2. Dynamic radius/scale that pulses along dimension 4
scale_xy = (size_xy // 2 - 5) + 5 * np.sin(4 * np.pi * t / num_t)

# Normalized radial distance squared in 4D space
r_sq = (
    ((x - x_center) / scale_xy) ** 2
    + ((y - size_xy // 2) / (size_xy // 2 - 5)) ** 2
    + ((z - z_center) / 25) ** 2
)

# Gaussian pattern + 4D noise
pattern = -10 + 28 * np.exp(-r_sq)
noise = rng.normal(0, 0.3, size=(size_xy, size_xy, num_z, num_t))

volume_4d = np.clip(pattern + noise, -10, 20)

The rule: the last axis is “how many images”; everything else is “one volume”¶

2D (H, W) — one image.
3D (H, W, S) — still one image: the middle slice along the last axis, S // 2. Not a grid of S images.
4D (H, W, S, N) — a grid of N images, laid out up to 5 per row. Each of the N still only shows its own middle slice along axis 2 — the same “last axis of each sub-array is slices, not images” rule applies one level down.

single = volume_4d[:, :, num_z // 2, 1]  # single slice of the 4D volume
display_images(single)

A 4D array with one slice per volume is the grid case: seven panels, laid out five per row.

one_slice_grid = volume_4d[:, :, num_z // 2 : num_z // 2 + 1, :]
one_slice_grid.shape

(90, 90, 1, 7)

display_images(one_slice_grid, titles=[f"t = {i}" for i in range(num_t)], colorbar=True)

# STRICT TESTS: display_images axis semantics.
# `display_images` only *displays*, so the panel count is checked on the
# renderer it drives rather than on a return value.
from mnutils.plotting.images import _render_image_grid_frames

frames, bounds = _render_image_grid_frames(
    one_slice_grid, num_images=num_t, cmap="magma", vmin=0, vmax=1, per_panel_bounds=False
)
assert len(frames) == 1  # one slice
assert len(frames[0]) == num_t  # seven panels, one per 4th-dimension entry

frames_2d, _ = _render_image_grid_frames(
    single[:, :, np.newaxis, np.newaxis],
    num_images=1,
    cmap="magma",
    vmin=0,
    vmax=1,
    per_panel_bounds=False,
)
assert len(frames_2d) == 1 and len(frames_2d[0]) == 1

A list of same-shaped arrays works too — it’s stacked along a new last axis before the same rule applies (more on that below, since a list is often exactly how a multi-slice case shows up).

But a volume has more than one slice — can I see the rest?¶

The examples above all used a single slice. Real volumes don’t stop at one: a (H, W, S) array with S > 1 still shows only its middle slice by default — so what about the other S - 1?

display_images() shows them too, as a slider, right here on this page — not just in a live Jupyter session. Every call displays an interactive widget and returns None; with more than one slice that widget gains a slider, and with exactly one it simply doesn’t.

display_images(volume_4d[:, :, :, 1])

The same rule applies one level down for a 4D grid: every panel shares a single slider, since they’re all still slices of the same underlying volume stack.

display_images(volume_4d)

This is also why a list of plain 2D arrays is a multi-slice case, not a static one: stacked along a new last axis, four 2D arrays become one volume with four slices — the same “not a grid” rule as above, just arrived at differently — so it engages the same interactive widget:

list_of_images = [np.full((10, 10), i, dtype=float) for i in range(4)]
display_images(list_of_images)

A list of volumes (each already 3D) becomes 4D instead — a grid, one per volume:

list_of_single_slice_volumes = [np.full((10, 10, 1), i, dtype=float) for i in range(4)]
display_images(list_of_single_slice_volumes, titles=[f"volume {i}" for i in range(4)])

# STRICT TESTS: list input follows np.stack(..., axis=-1), then the same shape rule
stacked = np.stack(list_of_single_slice_volumes, axis=-1).astype(np.float32)
assert stacked.shape == (10, 10, 1, 4)  # stacked to 4D -> grid of 4
frames_list, _ = _render_image_grid_frames(
    stacked, num_images=4, cmap="magma", vmin=0, vmax=3, per_panel_bounds=False
)
assert len(frames_list) == 1 and len(frames_list[0]) == 4

Where the titles and the colorbar come from¶

The panels you see above are plain images, rendered straight from the array at its own pixel size — no matplotlib figure is built at any point, which is why nothing is returned to post-process. Everything drawn around the pixels is HTML instead: fig_title becomes a heading, titles become captions, and colorbar=True becomes a gradient strip labelled with the bounds the panels were scaled to.

display_images(
    volume_4d[:, :, :, :3],
    fig_title="Three time points, one shared slider",
    titles=["t = 0", "t = 1", "t = 2"],
    colorbar=True,
)

Pass colorbar_kws={"mode": "each"} for one bar per panel instead of one for the grid. That mode autoscales every panel to its own slice, so the numbers beside each bar change as you scrub:

display_images(
    volume_4d[:, :, :, :3],
    titles=["t = 0", "t = 1", "t = 2"],
    colorbar=True,
    colorbar_kws={"mode": "each"},
)

Panels are the array’s own pixels, so a non-square array stays non-square — a (30, 120) slice is drawn four times wider than it is tall, not stretched to fill its share of the row:

wide = np.stack([volume_4d[30:60, :, num_z // 2, i] for i in range(7)], axis=-1)[:, :, np.newaxis]
wide.shape  # (30, 90, 1, 3): three wide panels, one slice

(30, 90, 1, 7)

titles = [f"t = {i}" for i in range(wide.shape[-1])]

display_images(wide, titles=titles, colorbar=True)

Once there are enough panels, the columns wrap at --mnu-panel-min-w and the rows that don’t fit go behind a scrollbar — the block stays the same size on the page whether it holds three images or thirty:

many = rng.random((32, 32, 1, 24))
display_images(many, titles=[f"echo {i}" for i in range(24)])

Low-resolution arrays keep their pixels¶

A (20, 20) mask blown up to a few hundred screen pixels is a different problem from a 512² anatomical shrunk to fit: the first wants to look like the twenty-by-twenty grid of values it actually is, the second wants smoothing. Both happen automatically — a panel scaled up is drawn with hard pixel edges, one scaled down keeps the browser’s smooth default — and small frames are encoded losslessly, so no compression ringing creeps in around the blocks.

tiny = rng.integers(0, 4, size=(8, 8)).astype(float)
display_images(tiny, colorbar=True)

The same holds for a non-square low-resolution array, and for a grid of them:

display_images(rng.random((5, 20)))

masks = np.stack([(rng.random((16, 16)) > 0.5).astype(float) for _ in range(4)], axis=-1)
display_images(masks[:, :, np.newaxis, :], titles=[f"mask {i}" for i in range(4)])

# STRICT TESTS: small panels are encoded losslessly, big ones are not.
from mnutils.plotting.images import GRID_LOSSLESS_MAX_PIXELS

assert 8 * 8 <= GRID_LOSSLESS_MAX_PIXELS
tiny_frames, _ = _render_image_grid_frames(
    tiny[:, :, np.newaxis, np.newaxis],
    num_images=1,
    cmap="magma",
    vmin=0,
    vmax=3,
    per_panel_bounds=False,
)
import io as _io

import PIL.Image as _PILImage

with _PILImage.open(_io.BytesIO(tiny_frames[0][0])) as _im:
    assert _im.size == (8, 8)  # native resolution, no upscaling baked in
    # Lossless: every value in an 8x8 of four levels survives the round trip.
    assert len(set(_im.convert("RGBA").get_flattened_data())) == len(np.unique(tiny))

Zeros vs. real signal¶

Background voxels are often exactly 0, not a small real value — treating them as data skews a colorbar built from vmin/vmax/percentile clipping toward the background instead of the signal you actually care about, and paints a slab of colormap-black over whatever you were overlaying onto. zeros_as_nan handles both, and is on by default:

mostly_zero = np.zeros((20, 20))
mostly_zero[8:12, 8:12] = rng.random((4, 4)) * 100

display_images(mostly_zero, zeros_as_nan=True, colorbar=True)

Masked-out voxels are transparent, not a colour: the panels are RGBA images, so the background of the page shows through them in both light and dark themes. Pass zeros_as_nan=False to scale against — and paint — every voxel including the zeros.

The flag is really two decisions that used to be tangled together, and each now lives with the code that acts on it. The numbers come from fast_bounds, which any caller can ask to skip zeros:

from mnutils.plotting.images import fast_bounds

print("zeros counted: ", fast_bounds(mostly_zero))
print("zeros excluded:", fast_bounds(mostly_zero, exclude_zeros=True))

zeros counted:  (0.0, 78.01758659696651)
zeros excluded: (15.321770178044574, 97.0465364229024)

Percentiles are the wrong summary for a segmentation, though. Clipping the tails of a continuous image is what protects it from a handful of hot voxels, but a label map has no tails to clip — on a mask that is mostly background, the 1st and 99th percentile are both 0, and the whole colormap collapses onto the background with the labels invisible above it. So fast_bounds reads an array holding at most max_discrete_levels distinct whole numbers as a label map and takes its exact range instead:

labels = np.zeros((20, 20))
labels[8:12, 8:12] = rng.integers(1, 5, (4, 4))  # four ROIs in a sea of background

print("labels:", fast_bounds(labels))  # (0, 4), not (0, 0)

labels: (0.0, 4.0)

The pixels come from the renderer, one layer at a time — which is what makes it work for the overlays too. overlay_image_data_on_T1, overlay_nifti_data_on_T1, inspect_MRSI_spectra and overlay_voxel_on_T1 mask the data layer’s zeros by default and leave the anatomical beneath alone, on either engine:

from mnutils.plotting.images import overlay_image_data_on_T1

t1 = rng.random((20, 20, 3)) * 100 + 50
sparse_data = np.zeros((20, 20, 3))
sparse_data[8:12, 8:12, :] = rng.random((4, 4, 3))

overlay_image_data_on_T1(
    t1, sparse_data, only_overlay=False, exclude_zeros=True, alpha=1, cmap="okabe_ito"
)

Per layer rather than per call, because the two want opposite answers: an overlay’s zeros are outside its field of view and should drop out, while the T1’s zeros are the air around the head — mask those and the page background appears inside the skull outline. Pass zeros_as_nan=False to switch the data layer back off.