Documentation

Manual Annotation

Annotation adds structured information to your videos so both humans and AI systems can interpret what’s happening. By labeling objects, and annotating activities, attributes, and relations, you create a foundation for consistent data analysis and automated model training.

Depending on your project setup, you can annotate:

  • Masks and Categories: Define what is visible in a scene (e.g., objects, tools, materials).
  • Relations and Attributes: Describe how these objects interact or connect.
  • Activities: Capture why or when something happens (e.g., a task step or user action).

Starting an Annotation Session

Before you begin, ensure you have opened or created a project, added video(s) to the project and started or created a pipeline.

  1. Open your project and navigate to the Annotations tab.
  2. Locate the video/annotation card you wish to work on and click Start Task.
  3. The interface will open. The timeline below the video displays green diamonds; these are keyframes selected for manual annotation.
  4. Click Edit Frame in the upper right corner to begin.

Creating Annotations (Masks & Categories)

Every annotation requires two things: a Mask (the visual shape) and a Category (the label). You will define both in a single workflow. Masks are generated by AI assistance but guided by your input.

Basic Controls

Regardless of your workflow, use the following controls to define an object:

  • Left-click (Positive): Click on the object to add that area to the mask.
  • Right-click (Negative): Click on unwanted areas (background) to remove them from the mask.
  • Enter or Save Mask: Confirm the mask.

Using a Predefined Vocabulary (Recommended)

Use this workflow if your project already has an assigned vocabulary.

  1. Select Category: In the left sidebar, click the category name you want to annotate.
  2. Create Mask: Hover over the object in the video. A preview appears. Use Left/Right clicks to refine the shape.
  3. Confirm: Press Enter or Save Mask.

  4. Result: The mask is saved and immediately linked to the category you selected.

Without a Vocabulary (Ad-hoc)

Use this workflow if you are defining new categories on the fly.

  1. Enter Mask Editor: Press Enter to enter the Mask Editor.
  2. Create Mask: Hover over the object and use Left/Right clicks to define the shape.
  3. Confirm: Once the mask looks correct, press Enter.
  4. Assign a Category: The Category Editor will open. Type the category name and press Enter to confirm.
  5. Save: Click Save Frame in the upper right corner to finish.

Advanced Masking Tools

If the standard controls do not produce a clean mask – for example, if the object is very small or partially hidden – use these advanced tools.

Adjusting Mask Sensitivity (S/W Keys)

If the AI prediction is too broad or too narrow while hovering:

  • Press S: Focuses on smaller, detailed objects (shrinks the predicted area).
  • Press W: Focuses on larger objects (expands the predicted area). Wider preview is set as default.

Additive Masks (Complex Objects)

If an object is partially occluded, split into parts, or irregularly shaped, a single mask creation with multiple clicks might not cover it cleanly. Use Additive SAM masks:

  1. Set your first positive click to mark the main part of the object.
  2. Press A: This confirms the current segment and allows you to start a new segment for the same object.
  3. Continue this process until all visible parts of the object are covered.
  4. Save once the full object is covered.

Video Consistency: Reusing Objects

In video annotation, it is critical to track specific objects over time rather than creating new instances for every frame.

When an object reappears in a subsequent keyframe:

  1. Do not create a new annotation from scratch.
  2. Look for the object in the object list under Not Annotated.
  3. Select the object from the object list.
  4. Update the mask position/shape if the object has moved.

This ensures the system maintains a consistent Object ID, enabling accurate tracking. Only create a new object if a completely new element enters the scene.

Managing Keyframes

You control which frames are annotated based on the complexity of the scene.

  • Add a Keyframe: If significant movement, occlusion, or a new object appears between existing keyframes, navigate to that frame and click Edit Frame. This adds a yellow diamond to the timeline.
  • Skip a Keyframe: If nothing has changed in the scene, you may leave a keyframe unannotated.

Object Utilities

  • Lock Objects: To prevent accidental edits to a finished mask, press L or click the Lock icon next to the object in the sidebar.
  • Add Notes: Hover over an object and press X to attach a short note. This is helpful for distinguishing between similar items (e.g., "Left Glove" vs "Right Glove") that share the same category.