Image Processing Plugins (beta)¶

mesoSPIM-control can apply a configurable chain of image-processing steps to every camera frame, both in the live view and in the data that gets written to disk. This page covers using the built-in processors from the GUI; if you want to write your own processor plugin, see Plugin System instead.

Warning

The image processor plugin system is still in beta testing. Some built-in processors have known performance bottlenecks that can reduce the achievable frame rate. Test a candidate chain with Live at your target frame rate before relying on it during a real acquisition.

Opening the Processor Chain window¶

Open Plugins → Processor Chain in the Main window, or press Ctrl+Shift+P.

Important

Processors affect both the live view and the frames written to disk for the current acquisition. Disable a processor (uncheck it in the Active Chain) if you only want to inspect raw data.

The window has two lists side by side:

Panel	Purpose
Available Processors	Every processor type discovered at startup (built-in and any from `plugins['path_list']` in your config). Select one and click Add → to append it to the chain.
Active Chain	Processors currently applied, in order, top to bottom. Each entry has a checkbox to enable/disable it without removing it. Drag rows to reorder them, or use ↑ Up / ↓ Down. Remove deletes the selected entry entirely.

Order matters: frames pass through the chain top to bottom, so e.g. a background-subtraction step should usually come before a denoising step.

Editing parameters¶

Selecting a processor in the Active Chain shows its parameters below, in Selected Processor Parameters.

By default, parameter edits are only staged — click Apply to push them to the live chain and save them to disk.
Check Auto-apply parameter changes to have edits take effect in the live chain immediately as you type/adjust them. Apply is still required afterwards to persist the change for the next session.
Close exits the dialog; unsaved (non-auto-applied) edits are not written to disk.

Configuration is saved to processor_chain.json next to the active microscope config file, and the active chain is recorded in each acquisition’s metadata.

Built-in processors¶

Processor	Parameters	Description
Identity	—	Pass-through, no change. Useful as a placeholder or for testing.
GaussianBlur	`sigma` (px, default 1.0)	Gaussian smoothing for spatial denoising; larger `sigma` means more smoothing.
DifferenceOfGaussians	`sigma_low` / `sigma_high` (px), `device`	Band-pass filter (difference of two Gaussian blurs) that enhances features between the two length scales while suppressing very fine noise and very broad background. Runs on CPU or CUDA via PyTorch (`device: auto` picks CUDA if available).
BackgroundSubtraction	`method` (`rolling_ball` / `threshold`), `radius`, `threshold`	Removes uneven background. Rolling ball estimates a smooth background with a uniform filter of the given `radius`; threshold subtracts a constant value.
Binning	`bin_factor` (1–16), `method` (`mean` / `sum` / `max`)	Downsamples the frame by `bin_factor` × `bin_factor` blocks, reducing it with the chosen reduction method.
NeuralDenoise	`num_frames` (1/3/5/7/9/11), `device`	Temporal denoising with a PyTorch model that predicts a cleaned-up version of the most recent frame from a short sliding window of the last `num_frames` frames. Requires the matching `denoise_Nframe.pth` weights file (see below) and PyTorch, with optional CUDA acceleration.

Note

NeuralDenoise model weights live in mesoSPIM/src/plugins/support_files/ImageProcessors/NeuralDenoise/. Only some frame counts may ship by default — if you select a num_frames value whose .pth file isn’t present, the processor logs an error and the frame passes through unprocessed. Ask on the image.sc forum if you need a model for a frame count that isn’t bundled.

GPU acceleration¶

DifferenceOfGaussians and NeuralDenoise both accept a device parameter (auto / cpu / cuda) and use PyTorch. auto selects CUDA automatically if a compatible GPU and PyTorch+CUDA install are present, otherwise falls back to CPU. If PyTorch (or a CUDA build of it) is missing, these processors log a warning and pass frames through unchanged rather than crashing the acquisition.

Troubleshooting¶

A processor seems to do nothing — check it’s enabled (checkbox in the Active Chain) and that changes were applied (auto-apply, or Apply).
Live view is fine but saved data looks unprocessed, or vice versa — the same chain applies to both; check you’re comparing frames from the same acquisition and that the chain wasn’t changed (and re-applied) in between.
NeuralDenoise / DifferenceOfGaussians raise an import or file-not-found error — install PyTorch (pip install torch, with a CUDA build if you want GPU acceleration), and for NeuralDenoise confirm the denoise_Nframe.pth file for your chosen num_frames exists.

For plugin authoring (writing a new processor type), see Plugin System.