Docs / Schemas / temporal-segmentation / 0.7.0

Temporal Segmentation 0.7.0

Represent labelled time intervals within a media source, such as chapters, scenes, shots, speaker turns or detected events. Segments may overlap, may nest, and need not cover the whole source.

This is an older version of temporal-segmentation.

The latest version is 0.8.0.

$id

https://oatschema.org/0.7.0/temporal-segmentation

Latest

https://oatschema.org/temporal-segmentation

Versions

0.8.0
0.7.0

Source

v0.7.0 on GitHub

Schema

temporal-segmentation.json Raw

{    "x-license": {        "id": "Apache-2.0",        "notice": "Copyright 2025-2026 Dorsal Hub LTD",        "url": "https://github.com/oatschema/oats/blob/main/LICENSE"    },    "$schema": "https://json-schema.org/draft/2020-12/schema",    "$id": "https://oatschema.org/0.7.0/temporal-segmentation",    "title": "Temporal Segmentation",    "version": "0.7.0",    "description": "Represent labelled time intervals within a media source, such as chapters, scenes, shots, speaker turns or detected events. Segments may overlap, may nest, and need not cover the whole source.",    "type": "object",    "properties": {        "producer": {            "type": "string",            "description": "The creator (model, tool or author) of these segments (e.g., 'PySceneDetect', 'pyannote', 'Manual Review').",            "maxLength": 1024        },        "track_id": {            "type": "string",            "description": "(Optional) Identifier for the specific audio or video track in the source file. Numeric identifiers are written as strings (e.g., '0').",            "maxLength": 128        },        "duration": {            "type": "number",            "description": "The total duration of the source media in seconds. When present, no segment's end_time may exceed it.",            "minimum": 0        },        "vocabulary": {            "type": "array",            "description": "A list of the possible segment labels. For more than 100 labels, use 'vocabulary_url'.",            "maxItems": 100,            "uniqueItems": true,            "items": {                "type": "string",                "description": "A single valid label from the vocabulary.",                "maxLength": 128            }        },        "vocabulary_url": {            "type": "string",            "description": "A URL pointing to a more detailed external vocabulary or ontology.",            "format": "uri",            "maxLength": 2048        },        "score_explanation": {            "type": "string",            "description": "Defines the meaning of the 'score' field.",            "maxLength": 256        },        "segments": {            "type": "array",            "description": "The labelled time intervals. Can be empty if no segments were found.",            "maxItems": 100000,            "items": {                "type": "object",                "description": "A single labelled time interval.",                "properties": {                    "id": {                        "type": "string",                        "description": "An identifier for this segment (e.g., 'scene-12'). Must be unique within the record.",                        "maxLength": 128                    },                    "parent_id": {                        "type": "string",                        "description": "The id of the segment that contains this one (e.g., the chapter containing a scene). Must be the id of another segment in the record, and this segment must lie within its parent's interval.",                        "maxLength": 128                    },                    "group_id": {                        "type": "string",                        "description": "An identifier for the entity this segment belongs to. Segments sharing a group_id belong to the same entity (e.g., all turns of speaker 'SPEAKER_00'). Numeric identifiers, such as diarization cluster numbers, are written as strings (e.g., '0').",                        "maxLength": 128                    },                    "label": {                        "type": "string",                        "description": "The category of the segment (e.g., 'chapter', 'scene', 'speaker', 'explosion'). Must be a member of 'vocabulary' when it is present.",                        "maxLength": 128                    },                    "title": {                        "type": "string",                        "description": "A human-readable name for this segment (e.g., a chapter title).",                        "maxLength": 128                    },                    "description": {                        "type": "string",                        "description": "A description of what happens in this segment.",                        "maxLength": 1024                    },                    "start_time": {                        "type": "number",                        "description": "Segment start time in seconds from the start of the source.",                        "minimum": 0                    },                    "end_time": {                        "type": "number",                        "description": "Segment end time in seconds from the start of the source. Must not be earlier than start_time; equal times mark an instantaneous event.",                        "minimum": 0                    },                    "score": {                        "type": "number",                        "description": "The confidence that the label applies to this interval, ranging from 0.0 (uncertain) to 1.0 (certain).",                        "minimum": 0,                        "maximum": 1                    },                    "attributes": {                        "type": "object",                        "description": "Arbitrary metadata relevant to this segment.",                        "maxProperties": 16,                        "propertyNames": {                            "maxLength": 128                        },                        "additionalProperties": {                            "anyOf": [                                {                                    "type": "string",                                    "maxLength": 1024                                },                                {                                    "type": "number"                                },                                {                                    "type": "boolean"                                },                                {                                    "type": "null"                                }                            ]                        }                    }                },                "required": [                    "label",                    "start_time",                    "end_time"                ],                "additionalProperties": false            }        },        "attributes": {            "type": "object",            "description": "Arbitrary metadata relevant to this record (e.g., annotator, source, job ID).",            "maxProperties": 16,            "propertyNames": {                "maxLength": 128            },            "additionalProperties": {                "anyOf": [                    {                        "type": "string",                        "maxLength": 1024                    },                    {                        "type": "number"                    },                    {                        "type": "boolean"                    },                    {                        "type": "null"                    }                ]            }        }    },    "required": [        "segments"    ],    "allOf": [        {            "if": {                "properties": {                    "segments": {                        "maxItems": 0                    }                },                "required": [                    "segments"                ]            },            "then": {                "anyOf": [                    {                        "required": [                            "vocabulary"                        ]                    },                    {                        "required": [                            "vocabulary_url"                        ]                    },                    {                        "required": [                            "producer"                        ]                    }                ]            }        }    ],    "additionalProperties": false}

Examples

Records that pass validation against this schema.

temporal-segmentation-3

temporal-segmentation-3.json

{  "producer": "sports-event-detector-v2",  "vocabulary_url": "https://example.com/vocabularies/football-events.json",  "score_explanation": "Detector confidence probability.",  "segments": [    {      "label": "goal",      "start_time": 1312.0,      "end_time": 1312.0,      "score": 0.97    },    {      "label": "celebration",      "start_time": 1312.4,      "end_time": 1341.9,      "score": 0.82,      "attributes": {        "team": "home"      }    },    {      "label": "replay",      "start_time": 1330.0,      "end_time": 1362.5,      "score": 0.76    }  ]}

temporal-segmentation-4

temporal-segmentation-4.json

{  "producer": "ActionFormer",  "vocabulary_url": "https://github.com/activitynet/ActivityNet/blob/master/Evaluation/data/activity_net.v1-3.min.json",  "score_explanation": "Detection confidence for the action class over the interval.",  "segments": [    {      "label": "Long jump",      "start_time": 12.34,      "end_time": 27.91,      "score": 0.842    },    {      "label": "Triple jump",      "start_time": 11.02,      "end_time": 29.4,      "score": 0.311    },    {      "label": "Long jump",      "start_time": 61.75,      "end_time": 74.08,      "score": 0.127    }  ],  "attributes": {    "source_format": "ActivityNet v1.3 results",    "video_id": "v_8onOVVuN_Is"  }}

temporal-segmentation-5

temporal-segmentation-5.json

{  "producer": "PySceneDetect",  "vocabulary": [    "scene"  ],  "segments": [    {      "id": "scene-1",      "label": "scene",      "start_time": 0.0,      "end_time": 3.754,      "attributes": {        "start_frame": 1,        "end_frame": 90      }    },    {      "id": "scene-2",      "label": "scene",      "start_time": 3.754,      "end_time": 8.925,      "attributes": {        "start_frame": 91,        "end_frame": 214      }    },    {      "id": "scene-3",      "label": "scene",      "start_time": 8.925,      "end_time": 16.767,      "attributes": {        "start_frame": 215,        "end_frame": 402      }    }  ],  "attributes": {    "source_format": "PySceneDetect scene list CSV"  }}

temporal-segmentation-2

temporal-segmentation-2.json

{  "producer": "pyannote/speaker-diarization-3.1",  "track_id": "1",  "vocabulary": [    "speaker"  ],  "segments": [    {      "id": "turn-1",      "group_id": "SPEAKER_00",      "label": "speaker",      "start_time": 0.48,      "end_time": 6.12    },    {      "id": "turn-2",      "group_id": "SPEAKER_01",      "label": "speaker",      "start_time": 5.8,      "end_time": 11.04    },    {      "id": "turn-3",      "group_id": "SPEAKER_00",      "label": "speaker",      "start_time": 10.95,      "end_time": 14.3    },    {      "id": "turn-4",      "group_id": "SPEAKER_02",      "label": "speaker",      "start_time": 14.62,      "end_time": 16.725    }  ],  "attributes": {    "source_format": "RTTM",    "uri": "meeting"  }}

temporal-segmentation-1

temporal-segmentation-1.json

{  "producer": "Manual Review",  "duration": 1834.5,  "vocabulary": [    "chapter",    "scene"  ],  "segments": [    {      "id": "ch-1",      "label": "chapter",      "title": "Intro",      "start_time": 0,      "end_time": 95.2    },    {      "id": "ch-2",      "label": "chapter",      "title": "Building the Frame",      "start_time": 95.2,      "end_time": 1834.5    },    {      "id": "sc-1",      "parent_id": "ch-1",      "label": "scene",      "description": "Host introduces the project in the workshop.",      "start_time": 0,      "end_time": 41.0    },    {      "id": "sc-2",      "parent_id": "ch-1",      "label": "scene",      "description": "Montage of the finished piece.",      "start_time": 41.0,      "end_time": 95.2    },    {      "id": "sc-3",      "parent_id": "ch-2",      "label": "scene",      "start_time": 95.2,      "end_time": 1834.5    }  ]}

Fields

producer string

The creator (model, tool or author) of these segments (e.g., 'PySceneDetect', 'pyannote', 'Manual Review').
- at most 1024 characters
track_id string

(Optional) Identifier for the specific audio or video track in the source file. Numeric identifiers are written as strings (e.g., '0').
- at most 128 characters
duration number

The total duration of the source media in seconds. When present, no segment's end_time may exceed it.
- ≥ 0
vocabulary array of string

A list of the possible segment labels. For more than 100 labels, use 'vocabulary_url'.
Each item: A single valid label from the vocabulary.
- at most 100 items
- items must be unique
- each at most 128 characters
vocabulary_url string

A URL pointing to a more detailed external vocabulary or ontology.
- at most 2048 characters
- format uri
score_explanation string

Defines the meaning of the 'score' field.
- at most 256 characters
segments array of object required

The labelled time intervals. Can be empty if no segments were found.
Each item: A single labelled time interval.
- at most 100000 items
- id string
  
  An identifier for this segment (e.g., 'scene-12'). Must be unique within the record.
  - at most 128 characters
- parent_id string
  
  The id of the segment that contains this one (e.g., the chapter containing a scene). Must be the id of another segment in the record, and this segment must lie within its parent's interval.
  - at most 128 characters
- group_id string
  
  An identifier for the entity this segment belongs to. Segments sharing a group_id belong to the same entity (e.g., all turns of speaker 'SPEAKER_00'). Numeric identifiers, such as diarization cluster numbers, are written as strings (e.g., '0').
  - at most 128 characters
- label string required
  
  The category of the segment (e.g., 'chapter', 'scene', 'speaker', 'explosion'). Must be a member of 'vocabulary' when it is present.
  - at most 128 characters
- title string
  
  A human-readable name for this segment (e.g., a chapter title).
  - at most 128 characters
- description string
  
  A description of what happens in this segment.
  - at most 1024 characters
- start_time number required
  
  Segment start time in seconds from the start of the source.
  - ≥ 0
- end_time number required
  
  Segment end time in seconds from the start of the source. Must not be earlier than start_time; equal times mark an instantaneous event.
  - ≥ 0
- score number
  
  The confidence that the label applies to this interval, ranging from 0.0 (uncertain) to 1.0 (certain).
  - 0 to 1
- attributes object
  
  Arbitrary metadata relevant to this segment.
  - at most 16 properties
  - keys at most 128 characters
  - * string | number | boolean | null
    
    Any other key.
    - string: at most 1024 characters
attributes object

Arbitrary metadata relevant to this record (e.g., annotator, source, job ID).
- at most 16 properties
- keys at most 128 characters
- * string | number | boolean | null
  
  Any other key.
  - string: at most 1024 characters

Rules:

When segments is empty: at least one of vocabulary, vocabulary_url, producer must be present.

Show the JSON Schema

{  "if": {    "properties": {      "segments": {        "maxItems": 0      }    },    "required": [      "segments"    ]  },  "then": {    "anyOf": [      {        "required": [          "vocabulary"        ]      },      {        "required": [          "vocabulary_url"        ]      },      {        "required": [          "producer"        ]      }    ]  }}