Skip to content

Usage

This guide covers the DREEM workflow: preparing data, training models, running inference, and evaluating results. For a hands-on introduction, see the Quickstart or the Examples.

Data Preparation

What You Need

For training, you need:

  1. Videos - .mp4, .avi, or .tif stacks (for microscopy)
  2. Labels with ground truth tracks - Detections with temporally consistent identity labels in SLEAP (.slp) or Cell Tracking Challenge format

For inference, you need:

  1. Videos - Same formats as above
  2. Detections only - No ground truth tracks required; just detections in SLEAP (.slp) or Cell Tracking Challenge format

Getting Detections

DREEM decouples detection from tracking, so you can use any detection method. Here are some popular methods:

Directory Structure

SLEAP format (animal tracking): 

dataset/
├── train/
│   ├── video1.mp4
│   ├── video1.slp
│   ├── video2.mp4
│   └── video2.slp
├── val/
│   ├── video3.mp4
│   └── video3.slp
└── test/  # optional
    ├── video4.mp4
    └── video4.slp

Cell Tracking Challenge format (microscopy): 

dataset/
├── train/
│   ├── seq_01/
│   │   ├── frame000.tif
│   │   └── ...
│   └── seq_01_GT/TRA/
│       ├── frame000.tif  # labeled masks
│       └── ...
└── val/
    └── ...

Proofreading Ground Truth

Good tracking results require accurate ground truth. Before training:

  1. Ensure no identity switches in your annotations
  2. Verify detection accuracy - crops are centered on each instance

Use sleap-label for proofreading. For microscopy data, you can convert TrackMate output to SLEAP format using dreem convert (see below) to use the SLEAP GUI.

Training

Train a model with dreem train:

dreem train ./data/train --val-dir ./data/val --crop-size 70

Required Arguments

Argument Description
TRAIN_DIR Path to training data directory
--val-dir Path to validation data directory
--crop-size Size of bounding box around each instance (pixels)

Common Options

dreem train ./data/train \
    --val-dir ./data/val \
    --crop-size 70 \
    --epochs 30 \
    --lr 0.0001 \
    --run-name my_experiment \
    --no-gpu  # use CPU only
Option Default Description
--epochs 20 Maximum training epochs
--lr 0.0001 Learning rate
--d-model 128 Model embedding dimension
--nhead 1 Number of attention heads
--encoder-layers 1 Transformer encoder layers
--decoder-layers 1 Transformer decoder layers
--clip-length 32 Frames per training batch
--run-name dreem_train Name for logging/checkpoints
--gpu/--no-gpu GPU Use GPU or CPU
--video-type mp4 Video file extension (mp4, tif, etc.)

Using Config Files

For advanced options (augmentations, schedulers, callbacks), use a YAML config:

dreem train ./data/train --val-dir ./data/val --crop-size 70 --config ./my_config.yaml

See Training Configuration for all available options.

Output

Training saves checkpoints to ./models/{run_name}/. The final checkpoint is named *final*.ckpt.

Tracking (Inference)

Run tracking on videos without ground truth:

dreem track ./data/inference \
    --checkpoint ./models/my_model.ckpt \
    --output ./results \
    --crop-size 70

Required Arguments

Argument Description
INPUT_DIR Path to input data directory
--checkpoint Path to model checkpoint (.ckpt)
--output Output directory for results
--crop-size Bounding box size (should match training)

Common Options

dreem track ./data/inference \
    --checkpoint ./models/model.ckpt \
    --output ./results \
    --crop-size 70 \
    --max-tracks 5 \
    --confidence-threshold 0.8
Option Description
--max-tracks Maximum number of tracks (set to number of animals/cells)
--confidence-threshold Flag low-confidence predictions for review
--max-dist Maximum center distance between frames
--max-dist-multiplier Penalty multiplier applied when center distance exceeds --max-dist
--max-gap Maximum frame gap for track continuity
--anchor Keypoint name to use as centroid (default: centroid)
--video-type Video file extension
--no-gpu Run on CPU

Using --max-dist to enforce distance or motion based constraints

If you know the maximum distance the instances in your data can travel frame over frame, using --max-dist can help track them better. It penalizes matches that exceed the threshold you set. Using --max-dist-multiplier multiplies this distance penalty relative to the model's association scores, pushing tracking behavior closer to a pure distance-based tracker.

Output

Results are saved as .slp files in the output directory. Open them in SLEAP or the DREEM Visualizer.

Evaluation

Evaluate tracking against ground truth labels:

dreem eval ./data/test \
    --checkpoint ./models/model.ckpt \
    --output ./eval_results \
    --crop-size 70

This computes standard MOT metrics (MOTA, IDF1, ID switches) and saves results to the output directory.

Options

dreem eval accepts the same options as dreem track. The input directory must contain ground truth labels.

Output

  • .slp files with predicted tracks
  • motmetrics.csv with evaluation metrics
  • .h5 file with detailed results

Converting External Formats

Convert tracking data from external tools (e.g., TrackMate) to SLEAP .slp format:

dreem convert trackmate \
    -l ./data/labels1.csv -l ./data/labels2.csv \
    -v ./data/video1.tif -v ./data/video2.tif \
    --output ./converted \
    --to-mp4

Required Arguments

Argument Description
FORMAT Source format to convert from (currently: trackmate)
--labels, -l Paths to label files (repeat for multiple files)
--videos, -v Paths to video files (repeat for multiple files)

Common Options

dreem convert trackmate \
    -l labels.csv \
    -v video.tif \
    --output ./converted \
    --to-mp4  # convert TIF to MP4
Option Default Description
--output, -o . Output directory for converted files
--to-mp4, -m - Convert TIF/ND2 videos to .mp4 format
--to-npy, -n - Convert TIF videos to .npy format

Output

  • .slp files with tracks and detections (one per video)
  • .mp4 or .npy video files (if --to-mp4 or --to-npy is set)

1-indexed frame numbers from TrackMate are automatically converted to 0-indexed.

CLI Reference

Get help for any command:

dreem --help
dreem train --help
dreem track --help
dreem eval --help
dreem convert --help

Config Overrides

Override individual config values with --set:

dreem track ./data --checkpoint model.ckpt --output ./results \
    --crop-size 70 \
    --set tracker.window_size=16 \
    --set tracker.decay_time=0.9
For fine grained control, use a config file as mentioned above.

Next Steps