Mask API
Help > Example Scripts > mask_server.py
A mask method of your own normally is a Python file that TraCell loads. That file can only use the libraries TraCell itself has. A method that needs more (scikit-image, PyTorch, a GPU library) runs in a mask service instead: a program of yours on the same computer, with its own Python and its own libraries. This page is everything TraCell expects of it: the mask API.
To build a mask service in Python you do not need this page: Your own mask method shows the steps, starting from the example. This page is the reference, and what a service in another language has to follow.
The mask service is yours
TraCell only hands over frames and reads masks. Writing your mask service, starting it, and what it computes are your responsibility. TraCell ships a complete example to start from.
The mask API can change
This page describes version 1. A later TraCell may use a later version; your mask service then has to follow. TraCell tells you when the versions differ and does not use a service of another version.
Try the example first
Help > Example Scripts opens a folder with mask_server.py: a complete mask service that needs only Python
and NumPy, and nothing of TraCell. Its method marks what differs from the first frame.
python mask_server.py --port 52750
- In TraCell open Tools > Mask > Settings.
- Choose the method Mask service and give the port.
- Tools > Mask > Show Mask shows the mask your service computes.
To serve your own method, copy the file and rewrite its class Method. The rest of the file is the mask API
and can stay as it is. The method and the port are saved in the preset, so tracell track and tracell live
use the same service.
Your mask service must be running before the method is chosen. TraCell never starts it, stops it or replaces it. The settings of your method are your program's: the Mask window shows only the port.
What your mask service does
| Part | What it means |
|---|---|
| Listen | On a TCP port of this computer only (127.0.0.1). |
| Answer commands | One line of JSON in, one line of JSON out. |
| Compute masks | For every frame, in the order they come. |
| Share memory | The frame and its mask are in shared memory. No picture travels through the port. |
A frame is 8-bit gray, after TraCell's region and adjustment settings: one byte per pixel, rows from top to bottom. The mask has the same size: 255 where a cell is, 0 elsewhere. Contours, linking and everything after the mask are TraCell's.
Connections and runs
One connection is one run. TraCell opens a connection, sends the frames of one run in order, and closes it. Your method may keep something between the frames of a connection, such as a background.
TraCell opens several connections at the same time: one for the mask shown in the window, one for each tracking run. Keep them apart: each connection has its own shared memory and its own state of the method.
When a connection closes, free its shared memory.
Messages
Every message is one JSON object on one line, ended by a line feed, as UTF-8.
TraCell sends a command: {"id": 0, "cmd": "apply", "learn": true}. Your service answers with the same
id: {"id": 0, "ok": true}, or {"id": 0, "ok": false, "error": "what went wrong"}. The text of error
is shown to the user, so write it for them. Never leave a command without an answer; TraCell waits 30 seconds
for one.
Commands
cmd |
Fields TraCell sends | Your answer, besides id and ok |
|---|---|---|
hello |
protocol: 1. Optional label: a name for your method. |
|
open |
height, width: the frame size in pixels |
frame and mask: the names of two blocks of shared memory that you create, each height × width bytes. Start the method anew. Sent before the first frame, and again when the frame size changes (free the earlier blocks then). |
apply |
learn: true or false |
Nothing more. TraCell has written a frame into the frame block. Write its mask into the mask block, then answer. |
learn is false while the video is paused on a frame: answer with the mask, and do not update what your
method keeps between frames.
TraCell copies the mask as soon as you answer, so the block may be written again with the next frame.
Shared memory
A block is a named piece of memory that two programs open by its name. Your service creates the blocks and
gives their names in the answer to open; TraCell opens them by name. Use names that no other program uses,
for example with the process number and the time in them.
| System | How to create a block named mymask_0_frame |
|---|---|
| Windows | CreateFileMappingW(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, size, L"mymask_0_frame"), then MapViewOfFile. |
| Linux, macOS | shm_open("/mymask_0_frame", O_CREAT \| O_RDWR, 0600), ftruncate to the size, then mmap. |
| Python | multiprocessing.shared_memory.SharedMemory(create=True, size=size, name="mymask_0_frame") does one or the other. |
Speed
Handing a frame over and taking the mask back costs little next to the method itself. Measured with the example on a small laptop (two cores, 4 GB of memory): about 0.4 ms per frame of 1008 × 136 pixels and about 2 ms per frame of 1280 × 800 pixels. What your method needs for a frame comes on top, and decides how many frames per second a live session can track.
When it does not work
| What you see | What to do |
|---|---|
| No mask service answers on port … | Start your mask service; check the port in Tools > Mask > Settings. |
| … speaks version … of the mask API | Answer hello with the version this page describes, after checking what changed. |
| Mask service: … | It is the error text of your service's answer. |
| The mask service on port … did not answer | Your method took longer than 30 seconds for a frame, or your program stopped. |
| The mask is torn or shifted | The blocks are not height × width bytes, or the rows are written in another order. |