Documentation
OATS (Open Annotation Task Schemas) is a collection of strict, versioned JSON schemas, defining and validating the structured outputs of machine learning models, workflow steps and annotation.
Quick start
Install the package for your language and validate a record. Every package bundles the schemas, so validation never touches the network.
1Install
pip install 'oatschema[validate]'
The validate extra adds native validation with jsonschema-rs. Without it, pip install oatschema gives you the schemas alone, with no dependencies.
2Validate a record
import oatschemarecord = {"labels": [{"label": "cat", "score": 0.92}]}oatschema.validate(record, "classification") # passesbad = {"labels": [{"label": "cat", "score": 1.5}]}oatschema.is_valid(bad, "classification") # Falseoatschema.validate(bad, "classification")# raises oatschema.ValidationError: classification 0.8.0 at /labels/0/score: … is greater than the maximum of 11Install
npm install oatschema ajv ajv-formats
oatschema bundles the schemas, with TypeScript types. Ajv validates against them.
2Validate a record
import Ajv2020 from "ajv/dist/2020.js";import addFormats from "ajv-formats";import { getSchema } from "oatschema";const ajv = new Ajv2020({ allErrors: true, strictTypes: false, strictTuples: false, strictRequired: false });ajv.addKeyword({ keyword: "x-license", schemaType: "object" });ajv.addKeyword({ keyword: "x-version", schemaType: "string" });addFormats(ajv);const validate = ajv.compile(getSchema("classification"));validate({ labels: [{ label: "cat", score: 0.92 }] }); // truevalidate({ labels: [{ label: "cat", score: 1.5 }] }); // falsevalidate.errors; // [{ instancePath: "/labels/0/score", message: "must be <= 1", ... }]1Install
cargo add oatschema --features serde_jsoncargo add jsonschema serde_jsonoatschema compiles the schemas into your binary. The jsonschema crate validates against them.
2Validate a record
use oatschema::SchemaName;use serde_json::json;fn main() -> Result<(), Box<dyn std::error::Error>> { let schema = oatschema::schema(SchemaName::Classification, None)?; let validator = jsonschema::options().should_validate_formats(true).build(&schema)?; assert!(validator.is_valid(&json!({"labels": [{"label": "cat", "score": 0.92}]}))); for error in validator.iter_errors(&json!({"labels": [{"label": "cat", "score": 1.5}]})) { println!("{}: {error}", error.instance_path()); // /labels/0/score: 1.5 is greater than the maximum of 1 } Ok(())}1Fetch a schema
curl https://oatschema.org/0.8.0/classification
Each schema is served at its $id, which includes the release version. A released version never changes, so you can cache it forever. How the URLs work
2Validate a record
Load the schema into any JSON Schema draft 2020-12 validator. x-license and x-version are the only non-standard keywords, and draft 2020-12 validators treat them as annotations. Turn on format checking, so that values such as uri and date-time are checked too.
Reference
- SchemasAll 12 schemas in 0.8.0, with every field, rule and example.
- Versions and URLsHow releases are versioned, how to pin one, and the URLs each schema is served at.
- Built for model outputsWhy strict, bounded schemas suit the outputs of models and pipelines.
audio-transcriptionclassificationdocument-extractionembeddingentity-extractiongenericgeolocationllm-outputobject-detectionrankingregressiontemporal-segmentation