Skip to content
OATSCHEMA

Docs / Schemas / temporal-segmentation

Temporal Segmentation 0.8.0

Represent labelled time intervals within a media source, such as chapters, scenes, shots, speaker turns or detected events. Segments may overlap, may nest, and need not cover the whole source.

$id
https://oatschema.org/0.8.0/temporal-segmentation
Latest
https://oatschema.org/temporal-segmentation
Versions
Source
v0.8.0 on GitHub

Schema

temporal-segmentation.json Raw 
{    "x-license": {        "id": "Apache-2.0",        "notice": "Copyright 2025-2026 The OATS Authors",        "url": "https://github.com/oatschema/oats/blob/main/LICENSE"    },    "$schema": "https://json-schema.org/draft/2020-12/schema",    "$id": "https://oatschema.org/0.8.0/temporal-segmentation",    "title": "Temporal Segmentation",    "x-version": "0.8.0",    "description": "Represent labelled time intervals within a media source, such as chapters, scenes, shots, speaker turns or detected events. Segments may overlap, may nest, and need not cover the whole source.",    "type": "object",    "properties": {        "producer": {            "type": "string",            "description": "The creator (model, tool or author) of these segments (e.g., 'PySceneDetect', 'pyannote', 'Manual Review').",            "maxLength": 1024        },        "track_id": {            "type": "string",            "description": "(Optional) Identifier for the specific audio or video track in the source media. Numeric identifiers are written as strings (e.g., '0').",            "maxLength": 128        },        "duration": {            "type": "number",            "description": "The total duration of the source media in seconds. When present, no segment's end_time may exceed it.",            "minimum": 0        },        "vocabulary": {            "type": "array",            "description": "A list of the possible segment labels. For more than 100 labels, use 'vocabulary_url'.",            "maxItems": 100,            "uniqueItems": true,            "items": {                "type": "string",                "description": "A single valid label from the vocabulary.",                "maxLength": 128            }        },        "vocabulary_url": {            "type": "string",            "description": "A URL pointing to a more detailed external vocabulary or ontology.",            "format": "uri",            "maxLength": 2048        },        "score_explanation": {            "type": "string",            "description": "Defines the meaning of the 'score' field.",            "maxLength": 256        },        "segments": {            "type": "array",            "description": "The labelled time intervals. Can be empty if no segments were found.",            "maxItems": 100000,            "items": {                "type": "object",                "description": "A single labelled time interval.",                "properties": {                    "id": {                        "type": "string",                        "description": "An identifier for this segment (e.g., 'scene-12'). Must be unique within the record.",                        "maxLength": 128                    },                    "parent_id": {                        "type": "string",                        "description": "The id of the segment that contains this one (e.g., the chapter containing a scene). Must be the id of another segment in the record, and this segment must lie within its parent's interval.",                        "maxLength": 128                    },                    "group_id": {                        "type": "string",                        "description": "An identifier for the entity this segment belongs to. Segments sharing a group_id belong to the same entity (e.g., all turns of speaker 'SPEAKER_00'). Numeric identifiers, such as diarization cluster numbers, are written as strings (e.g., '0').",                        "maxLength": 128                    },                    "label": {                        "type": "string",                        "description": "The category of the segment (e.g., 'chapter', 'scene', 'speaker', 'explosion'). Must be a member of 'vocabulary' when it is present.",                        "maxLength": 128                    },                    "title": {                        "type": "string",                        "description": "A human-readable name for this segment (e.g., a chapter title).",                        "maxLength": 128                    },                    "description": {                        "type": "string",                        "description": "A description of what happens in this segment.",                        "maxLength": 1024                    },                    "start_time": {                        "type": "number",                        "description": "Segment start time in seconds from the start of the source.",                        "minimum": 0                    },                    "end_time": {                        "type": "number",                        "description": "Segment end time in seconds from the start of the source. Must not be earlier than start_time; equal times mark an instantaneous event.",                        "minimum": 0                    },                    "score": {                        "type": "number",                        "description": "The confidence that the label applies to this interval, ranging from 0.0 (uncertain) to 1.0 (certain).",                        "minimum": 0,                        "maximum": 1                    },                    "attributes": {                        "type": "object",                        "description": "Arbitrary metadata relevant to this segment.",                        "maxProperties": 16,                        "propertyNames": {                            "maxLength": 128                        },                        "additionalProperties": {                            "anyOf": [                                {                                    "type": "string",                                    "maxLength": 1024                                },                                {                                    "type": "number"                                },                                {                                    "type": "boolean"                                },                                {                                    "type": "null"                                }                            ]                        }                    }                },                "required": [                    "label",                    "start_time",                    "end_time"                ],                "additionalProperties": false            }        },        "attributes": {            "type": "object",            "description": "Arbitrary metadata relevant to this record (e.g., annotator, source, job ID).",            "maxProperties": 16,            "propertyNames": {                "maxLength": 128            },            "additionalProperties": {                "anyOf": [                    {                        "type": "string",                        "maxLength": 1024                    },                    {                        "type": "number"                    },                    {                        "type": "boolean"                    },                    {                        "type": "null"                    }                ]            }        }    },    "required": [        "segments"    ],    "additionalProperties": false}

Examples

Records that pass validation against this schema.

temporal-segmentation-3
temporal-segmentation-3.json 
{  "producer": "sports-event-detector-v2",  "vocabulary_url": "https://example.com/vocabularies/football-events.json",  "score_explanation": "Detector confidence probability.",  "segments": [    {      "label": "goal",      "start_time": 1312.0,      "end_time": 1312.0,      "score": 0.97    },    {      "label": "celebration",      "start_time": 1312.4,      "end_time": 1341.9,      "score": 0.82,      "attributes": {        "team": "home"      }    },    {      "label": "replay",      "start_time": 1330.0,      "end_time": 1362.5,      "score": 0.76    }  ]}
temporal-segmentation-4
temporal-segmentation-4.json 
{  "producer": "ActionFormer",  "vocabulary_url": "https://github.com/activitynet/ActivityNet/blob/master/Evaluation/data/activity_net.v1-3.min.json",  "score_explanation": "Detection confidence for the action class over the interval.",  "segments": [    {      "label": "Long jump",      "start_time": 12.34,      "end_time": 27.91,      "score": 0.842    },    {      "label": "Triple jump",      "start_time": 11.02,      "end_time": 29.4,      "score": 0.311    },    {      "label": "Long jump",      "start_time": 61.75,      "end_time": 74.08,      "score": 0.127    }  ],  "attributes": {    "source_format": "ActivityNet v1.3 results",    "video_id": "v_8onOVVuN_Is"  }}
temporal-segmentation-5
temporal-segmentation-5.json 
{  "producer": "PySceneDetect",  "vocabulary": [    "scene"  ],  "segments": [    {      "id": "scene-1",      "label": "scene",      "start_time": 0.0,      "end_time": 3.754,      "attributes": {        "start_frame": 1,        "end_frame": 90      }    },    {      "id": "scene-2",      "label": "scene",      "start_time": 3.754,      "end_time": 8.925,      "attributes": {        "start_frame": 91,        "end_frame": 214      }    },    {      "id": "scene-3",      "label": "scene",      "start_time": 8.925,      "end_time": 16.767,      "attributes": {        "start_frame": 215,        "end_frame": 402      }    }  ],  "attributes": {    "source_format": "PySceneDetect scene list CSV"  }}
temporal-segmentation-2
temporal-segmentation-2.json 
{  "producer": "pyannote/speaker-diarization-3.1",  "track_id": "1",  "vocabulary": [    "speaker"  ],  "segments": [    {      "id": "turn-1",      "group_id": "SPEAKER_00",      "label": "speaker",      "start_time": 0.48,      "end_time": 6.12    },    {      "id": "turn-2",      "group_id": "SPEAKER_01",      "label": "speaker",      "start_time": 5.8,      "end_time": 11.04    },    {      "id": "turn-3",      "group_id": "SPEAKER_00",      "label": "speaker",      "start_time": 10.95,      "end_time": 14.3    },    {      "id": "turn-4",      "group_id": "SPEAKER_02",      "label": "speaker",      "start_time": 14.62,      "end_time": 16.725    }  ],  "attributes": {    "source_format": "RTTM",    "uri": "meeting"  }}
temporal-segmentation-1
temporal-segmentation-1.json 
{  "producer": "Manual Review",  "duration": 1834.5,  "vocabulary": [    "chapter",    "scene"  ],  "segments": [    {      "id": "ch-1",      "label": "chapter",      "title": "Intro",      "start_time": 0,      "end_time": 95.2    },    {      "id": "ch-2",      "label": "chapter",      "title": "Building the Frame",      "start_time": 95.2,      "end_time": 1834.5    },    {      "id": "sc-1",      "parent_id": "ch-1",      "label": "scene",      "description": "Host introduces the project in the workshop.",      "start_time": 0,      "end_time": 41.0    },    {      "id": "sc-2",      "parent_id": "ch-1",      "label": "scene",      "description": "Montage of the finished piece.",      "start_time": 41.0,      "end_time": 95.2    },    {      "id": "sc-3",      "parent_id": "ch-2",      "label": "scene",      "start_time": 95.2,      "end_time": 1834.5    }  ]}

Fields

  • producer string

    The creator (model, tool or author) of these segments (e.g., 'PySceneDetect', 'pyannote', 'Manual Review').

    • at most 1024 characters
  • track_id string

    (Optional) Identifier for the specific audio or video track in the source media. Numeric identifiers are written as strings (e.g., '0').

    • at most 128 characters
  • duration number

    The total duration of the source media in seconds. When present, no segment's end_time may exceed it.

    • ≥ 0
  • vocabulary array of string

    A list of the possible segment labels. For more than 100 labels, use 'vocabulary_url'.

    Each item: A single valid label from the vocabulary.

    • at most 100 items
    • items must be unique
    • each at most 128 characters
  • vocabulary_url string

    A URL pointing to a more detailed external vocabulary or ontology.

    • at most 2048 characters
    • format uri
  • score_explanation string

    Defines the meaning of the 'score' field.

    • at most 256 characters
  • segments array of object required

    The labelled time intervals. Can be empty if no segments were found.

    Each item: A single labelled time interval.

    • at most 100000 items
    • id string

      An identifier for this segment (e.g., 'scene-12'). Must be unique within the record.

      • at most 128 characters
    • parent_id string

      The id of the segment that contains this one (e.g., the chapter containing a scene). Must be the id of another segment in the record, and this segment must lie within its parent's interval.

      • at most 128 characters
    • group_id string

      An identifier for the entity this segment belongs to. Segments sharing a group_id belong to the same entity (e.g., all turns of speaker 'SPEAKER_00'). Numeric identifiers, such as diarization cluster numbers, are written as strings (e.g., '0').

      • at most 128 characters
    • label string required

      The category of the segment (e.g., 'chapter', 'scene', 'speaker', 'explosion'). Must be a member of 'vocabulary' when it is present.

      • at most 128 characters
    • title string

      A human-readable name for this segment (e.g., a chapter title).

      • at most 128 characters
    • description string

      A description of what happens in this segment.

      • at most 1024 characters
    • start_time number required

      Segment start time in seconds from the start of the source.

      • ≥ 0
    • end_time number required

      Segment end time in seconds from the start of the source. Must not be earlier than start_time; equal times mark an instantaneous event.

      • ≥ 0
    • score number

      The confidence that the label applies to this interval, ranging from 0.0 (uncertain) to 1.0 (certain).

      • 0 to 1
    • attributes object

      Arbitrary metadata relevant to this segment.

      • at most 16 properties
      • keys at most 128 characters
      • * string | number | boolean | null

        Any other key.

        • string: at most 1024 characters
  • attributes object

    Arbitrary metadata relevant to this record (e.g., annotator, source, job ID).

    • at most 16 properties
    • keys at most 128 characters
    • * string | number | boolean | null

      Any other key.

      • string: at most 1024 characters

Copyright 2025-2026 The OATS Authors. Licensed under Apache-2.0.