How to Annotate

A vocabulary defines specific categories, activities, and (soon) relations and attributes used during annotation.

1. Go to Library → Vocabularies → New Vocabulary
2. Enter a name and description
3. Add categories (e.g., mug, kettle) and activities (e.g., pour, fill)
4. Save

Tip: You can always edit or expand your vocabulary later – even while annotating

Projects help organize your annotation work. Each project can have its own vocabulary and automation setup.

1. Go to Annotation → All Projects (or Get Started if no project was created yet) → Create Project
2. Enter a name and description
3. Open your new project and link your vocabulary under: Settings → Annotation Scope → Vocabulary

Tip: Create one project per use case – it keeps automation and vocabularies focused.

1. Go to Library → Videos
2. Upload or select existing videos
3. Click Add to Project → choose your project

Pipelines automate most of the annotation work.

1. Open your project → Videos tab
2. Click New on a video
3. Choose a pipeline template (e.g., Hand-Interaction Video Annotation)
4. Click Start

When it finishes, the task appears in the Annotations tab – click Start Task to open it.

• Click Edit Frame to start annotation
• Use left-click to add to a mask and right-click to remove
• Assign categories automatically (if using a predefined vocabulary)
• Skip frames where nothing changes
• Reuse previous object instances when the same object reappears in the following frames (they are listed under the not annotated section in the object overview)

Optional

Depending on your pipeline, you may receive an additional task to review potential issues after annotation. These are automatically flagged as anomalies—for example, missing masks or category mismatches.

If a refinement task appears:

1. Open it from the Annotations tab.
2. Review the highlighted frames and fix only what’s necessary.
3. Save your changes once done.

Tip: Not all pipelines include this step. If no refinement task appears, your annotation is complete and ready to explore.

After annotation, you can view your results and interact with them directly.

To open your annotations:

Go to the Annotations tab → on the annotation card, click the … menu → View

To ask questions:

Use the built-in chatbot to query your annotations in natural language. It provides grounded answers based on your annotation data (and linked documents, if available).

Examples

• “Have all steps been completed?”
• “Which objects interact in this scene?”
• “Create a step-by-step guide for this video.”

Tip: The chatbot combines annotations with contextual documents to turn your data into easy-to-understand insights.