Your own mask method

How to make TraCell use a mask method that you wrote. You need a little Python; you do not need to know how TraCell works inside.

A mask method is one function: it gets a frame and returns the mask of that frame. Contours, tracking, export and Live Tracking then work as with the built-in methods.

Two ways

	Mask script	Mask service
What it is	One Python file that TraCell loads	A small program of yours that runs next to TraCell
Libraries you can use	NumPy and OpenCV	Any: scikit-image, SciPy, PyTorch, a GPU
Python to install	None	Your own
Settings in the Mask window	Yes, a form is built for them	No, they are in your file
Start with it	Yes	Only when you need another library

What your method gets and returns

The same for both ways:

• You get a frame: 8-bit gray, already cut and adjusted by TraCell's Region and Adjustment settings.
• You return a mask of the same size: 255 where a cell is, 0 elsewhere.
• Frames come one after the other, in order, so your method may remember something between them, for example a background.
• learn is False while the video is paused on a frame. Return the mask then, but do not change what you remember.

Way 1: a mask script

1. Copy the example

In TraCell choose Help > Example Scripts. A folder opens. Copy mask_script.py to a folder of your own, for example Documents\my-mask, and give it a name of your own.

2. Put your method in

Open the file in an editor. It holds one class. This is the example's: it marks what differs from the first frame.

import cv2
import numpy as np
from pydantic import Field

from tracell.model.mask_methods import MaskMethod
from tracell.registry import Params


class DifferenceToFirstFrame(MaskMethod):
    kind = "difference_to_first_frame"  # the name stored in presets
    label = "Difference to first frame"  # the name shown in the Mask window

    class Params(Params):
        threshold: int = Field(25, ge=1, le=255, description="Gray-level difference that counts as a cell")
        blur: int = Field(3, ge=1, le=15, description="Size of the median blur applied first (odd number)")
        invert: bool = Field(False, description="Tick when the cells are the unchanged part")

    def __init__(self, params=None) -> None:
        super().__init__(params)
        self.background: np.ndarray | None = None

    def apply(self, frame: np.ndarray, learn: bool = True) -> np.ndarray:
        blur = self.params.blur | 1  # the blur size must be odd
        frame = cv2.medianBlur(np.asarray(frame, dtype=np.uint8), blur)
        if self.background is None or self.background.shape != frame.shape:
            self.background = frame.copy()
        difference = cv2.absdiff(frame, self.background)
        mask = cv2.threshold(difference, self.params.threshold, 255, cv2.THRESH_BINARY)[1]
        return cv2.bitwise_not(mask) if self.params.invert else mask

What to change:

Part	What it is for
kind and label	The name of your method: kind in presets (no spaces), label in the Mask window.
Params	Your settings, each with a type, a default and limits. TraCell shows them as a form in the Mask window and saves them in the preset. In apply they are self.params.threshold and so on.
__init__	What your method remembers between frames.
apply	Your method: frame in, mask out.

3. Load it

1. Tools > Mask > Settings. Press the folder button next to the Method list and choose your file.
2. Your method is now in the Method list, and its settings are below it.
3. Tools > Mask > Show Mask shows your mask next to the video.

File > Save stores the method, its settings and the path of your file in the preset. Loading the preset loads your file again.

When you change the code, save the file and load it again with the folder button.

Way 2: a mask service

Use this when your method needs a library that TraCell does not have. Your method then runs in a program of its own, with a Python that you install, on the same computer. You start that program; TraCell hands it every frame and takes the mask back.

1. A folder and a Python for it

Install Python 3.9 or newer (python.org). Then make a folder and, in it, a Python environment with the libraries your method needs:

WindowsLinux, macOS

cd Documents\my-mask
py -m venv env
env\Scripts\pip install numpy scikit-image

cd ~/my-mask
python3 -m venv env
env/bin/pip install numpy scikit-image

2. Copy the example

Copy mask_server.py from Help > Example Scripts into the folder and give it a name of your own, for example my_mask_service.py.

3. Put your method in

Near the top of the file is the class Method. Replace it with yours, and add the imports it needs below import numpy as np. Leave everything below the class as it is.

This example uses scikit-image. It is here to show the steps; it is not a better method than MOG2.

from skimage.filters import threshold_triangle
from skimage.morphology import disk, opening


class Method:
    BACKGROUND_FRAMES = 50  # the still background is the average of the first frames
    MIN_DIFFERENCE = 10     # a frame that differs less than this everywhere has no cell
    SPECK = disk(1)         # what is thinner than this shape is noise

    def __init__(self, height: int, width: int) -> None:
        self.background = np.zeros((height, width), np.float32)
        self.seen = 0

    def apply(self, frame: np.ndarray, learn: bool) -> np.ndarray:
        if learn and self.seen < self.BACKGROUND_FRAMES:
            self.seen += 1
            self.background += (frame - self.background) / self.seen
        difference = np.abs(frame - self.background)
        if self.seen == 0 or difference.max() < self.MIN_DIFFERENCE:
            return np.zeros(frame.shape, np.uint8)
        cells = np.where(difference > threshold_triangle(difference), 255, 0).astype(np.uint8)
        return opening(cells, self.SPECK)

Part	What it is for
__init__(height, width)	Called once when a run starts. Prepare here what takes long, such as loading a network.
apply(frame, learn)	Called once per frame. Your method: frame in, mask out.

One thing differs from a script: frame is overwritten by the next frame. To keep a frame, copy it (frame.copy()).

4. Start it

WindowsLinux, macOS

env\Scripts\python my_mask_service.py

env/bin/python my_mask_service.py

It answers mask service on port 52750; Ctrl+C stops it. Leave this window open.

To start it with a double click on Windows, put a file Start mask service.cmd next to it with this line:

"%~dp0env\Scripts\python.exe" "%~dp0my_mask_service.py"

5. Use it in TraCell

1. Tools > Mask > Settings, method Mask service. The port field shows 52750.
2. Tools > Mask > Show Mask shows your mask next to the video.

File > Save stores the method and the port in the preset. From then on, start your mask service first and TraCell second.

When you change the code, stop the service with Ctrl+C and start it again. TraCell connects again with the next frame.

Check your mask

1. Tools > Mask > Show Mask and play the video: cells should be white, everything else black.
2. Set the area limits in Contours so that specks and clumps are left out, and turn on Show Contours.
3. Press Track on a short frame range (Tracker) and look at the traces.

When it does not work

What you see	What to do
… did not load: … (script)	An error in your file; the text names it. Fix it and load the file again.
… defines no MaskMethod class (script)	The class must be derived from MaskMethod, as in the example.
ModuleNotFoundError (script)	Your script imports a library TraCell does not have. Use a mask service.
No mask service answers on port …	Start your mask service, and check that the port in the Mask window is the one it printed.
Mask service: …	The text after the colon is an error of your code. The window of your service shows more.
The mask service on port … did not answer	Your method took longer than 30 seconds for one frame, or your program stopped.
ModuleNotFoundError in the window of your service	The library is not installed in the environment you started it with (step 1).
The mask is black	Your method found no cell. Try a lower threshold, or check it on one frame outside TraCell.

For a mask service in another programming language, the Mask API lists what TraCell sends and expects.