Camera API

Help > Example Scripts > camera_server.py

TraCell does not talk to a camera. It talks to a camera service: a program on the same computer that controls the camera and hands over its frames (Camera setup). You write the camera service for your camera yourself, in any programming language, around the software kit (SDK) of your camera. This page is everything TraCell expects of it: the camera API.

The camera service is yours

TraCell only sends commands and reads frames. Writing your camera service, starting it, and what it does with your camera are your responsibility. TraCell ships a complete example to start from.

The camera API can change

This page describes version 5. A later TraCell may use a later version; your camera service then has to follow. TraCell tells you when the versions differ and does not use a service of another version.

Try the example first

Help > Example Scripts opens a folder with camera_server.py: a complete camera service in about 250 lines that needs only Python and NumPy, and nothing of TraCell. It plays a made-up camera: dark spots drifting over a gray picture.

python camera_server.py --port 52740

In TraCell open Edit > Preferences. In the Camera service row give the port.
Press Check…. The message must say that your camera service answers, and list its camera as ready.
Press OK, open Tools > Live Tracking and press Live View, then AutoTrack.

To serve a real camera, copy the file and rewrite its class Camera with the calls of your SDK. The rest of the file is the camera API and can stay as it is. From the command line the same service is used with tracell live --camera-service 52740.

Your camera service must be running before Live View or AutoTrack. TraCell never starts it, stops it or replaces it.

What your camera service does

Part	What it means
Listen	On a TCP port of this computer only (`127.0.0.1`).
Answer commands	One line of JSON in, one line of JSON out.
Record partitions	A partition is a number of frames recorded in one go. The frames stay with the camera or in your program until TraCell has taken them.
Hand over frames	Through shared memory, a chunk of a partition at a time. No frame travels through the port.

Frames are 8-bit gray: one byte per pixel, rows from top to bottom. A camera with more bits has to be scaled by your program, with the white level of the camera and not with the brightest pixel of each frame. A color camera has to be converted to gray.

Messages

Every message is one JSON object on one line, ended by a line feed, as UTF-8.

TraCell sends a command: {"id": 7, "cmd": "status"}, with more fields for some commands. Your service answers with the same id: {"id": 7, "ok": true, ...}, or {"id": 7, "ok": false, "error": "what went wrong"}. The text of error is shown to the user, so write it for them. Answer a command you do not have with ok: false; never leave a command without an answer.

Your service also sends events by itself, without an id: {"event": "partition_ready", ...}.

Commands

The first nine are needed for AutoTrack. The others are for one feature each; without one of them, that feature shows the error text your service answers.

`cmd`	Fields TraCell sends	Your answer, besides `id` and `ok`
`hello`		`protocol`: `5`. `providers`: a list with a name for your camera, such as `["my-camera"]`. Optional `about`, see below.
`status`		`status`: `{"state": "idle"}` or `"streaming"`; more fields are allowed.
`configure`	`config`: see Configuration	`status` as above. Connect to the camera here and apply the settings. Create the shared memory here.
`subscribe`		Where the frames will be: `slots` (the names of the shared memory blocks), `slot_frames` (frames per block), `height`, `width`, `dtype` (`"uint8"`), `fps` (the rate the camera really uses), `frames_per_partition`, `session` (any text).
`start_stream`		`status`. Start recording partitions, one after the other, until `stop_stream`.
`release`	`slot`: a block's number	Nothing more. TraCell has copied that block; you may fill it again.
`stop_stream`		`status`. Stop recording, soon.
`unsubscribe`		Nothing more.
`release_camera`		`status`. Let go of the camera and free the shared memory.
`live_frame`		For Live View. `captured`: `true` when the picture was written; `name`: the shared memory block that holds it; `height`, `width`, `dtype`. Only asked while not streaming.
`csr`	`lens_covered`: `true` or `false`	For the CSR button (black reference). `done`: `true`, or `false` when the lens has to be covered by hand first; TraCell then asks the user and sends it again with `lens_covered: true`.
`capture_arm`, `capture_trigger`, `capture_stored`, `capture_read`	see Capture	For Capture.
`shutdown`		Nothing more; end your program. TraCell does not send this.

about in the answer to hello is what Check… shows. It is optional:

{"my-camera": {"label": "My camera", "problem": "", "features": ["live view"]}}

problem is empty when the camera can be used, or says in a few words why not ("the SDK is not installed").

Configuration

configure carries what the user typed in Live Tracking. TraCell names no camera: your service knows which one it serves. Use what your camera has and ignore the rest; more keys may come in a later version.

Key	Meaning
`width`, `height`	The picture size asked for, in pixels. Answer `subscribe` with the size you really deliver.
`fps`	Frames per second asked for.
`exposure`	Exposure time in microseconds.
`edr`	Extended dynamic range, in microseconds, for cameras that have it.
`frames_per_partition`	How many frames one partition has.
`chunk_frames`	How many frames to hand over at a time (16 unless changed). A block of shared memory holds this many frames.
`ring_slots`	How many blocks of shared memory to make (at least 2).
`partition_count`	How many partitions the camera's memory is divided into, for cameras that have such a setting.
`capture_post_share`	Only for Capture: the share of the recording made after the trigger, from 0 to 1.

Recording and handing over

After start_stream, repeat until stop_stream:

Record the next partition of frames_per_partition frames.
Take a free block of shared memory. All blocks are free at subscribe; a block is taken when you announce it, and free again when TraCell sends release for it. If none is free, wait: TraCell is still copying.

Copy the next frames of the partition into the block, at most slot_frames of them, and send:

{"event": "partition_ready", "slot": 0, "partition_id": 1, "first_frame": 0,
 "n_frames": 16, "offset": 0, "last": false, "timestamp": 1790000000.0}

Go on with step 2 until the partition is handed over. last is true on its final chunk.

Field	Meaning
`slot`	The block's number, counted from 0 in the order of `slots`.
`partition_id`	1 for the first partition after `configure`, then 2, 3 and so on.
`offset`	The number of the chunk's first frame within its partition, from 0.
`n_frames`	How many frames of the block are filled.
`first_frame`	The number of the chunk's first frame since `configure`, counting all partitions.
`timestamp`	Seconds since 1970, when the chunk was ready.

Other events:

Event	When
`{"event": "end_of_stream", "reason": "..."}`	The source has no more frames (a file played as a camera). The session ends without an error.
`{"event": "error", "message": "..."}`	Recording failed. The session ends and shows the message.
`{"event": "camera_activity", "state": "recording", "partition_id": 2, "frames": 400, "seconds": 2.0, "timestamp": ...}`	Optional, for the Live Activity window: the camera started recording that partition. With `"state": "recorded"` it finished.

Between two partitions a camera may lose frames. TraCell therefore tracks each partition by itself and never follows a cell from one partition into the next.

Shared memory

A block is a named piece of memory that two programs open by its name. Your service creates the blocks and gives their names in subscribe (and in live_frame and capture_read); TraCell opens them by name. Use names that no other program uses, for example with the time in them.

System	How to create a block named `mycam_0`
Windows	`CreateFileMappingW(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, size, L"mycam_0")`, then `MapViewOfFile`.
Linux, macOS	`shm_open("/mycam_0", O_CREAT \\| O_RDWR, 0600)`, `ftruncate` to the size, then `mmap`.
Python	`multiprocessing.shared_memory.SharedMemory(create=True, size=size, name="mycam_0")` does one or the other.

A block for frames holds slot_frames × height × width bytes: frame after frame, each frame row after row. The block for the live picture holds height × width bytes. Keep the blocks until release_camera.

Capture

Capture records in a loop over the camera's whole memory until the user presses Keep, and then shows the stored recording like a video. It is optional.

`cmd`	Fields	Your answer
`capture_arm`		`capacity`: how many frames the memory holds; `fps`. Start recording in a loop.
`capture_trigger`		Nothing more. The moment to keep; go on recording the share given by `capture_post_share`.
`capture_stored`		`stored`: `null` while the recording is not complete, then `{"frames": n, "trigger": i}`: how many frames it holds and which one was the trigger, counted from 0.
`capture_read`	`first`, `count`	Copy frames `first`, `first + 1`, … into a block and answer `n` (how many), `name` (the block), `slot_frames`, `height`, `width`, `dtype`, `fps`. Any order, any number of times.

When it does not work

What you see	What to do
No camera service answers on port …	Start your camera service; check the port in Edit > Preferences.
… speaks another version of the camera API	Answer `hello` with the version this page describes, after checking what changed.
Live View shows an error text	It is the `error` text of your service's answer.
AutoTrack starts but no partition arrives	Is a `partition_ready` event sent for every chunk, and is `slot` free again after `release`?
The picture is torn or shifted	`height` and `width` in `subscribe` do not match what is written into the block.