Contract Format

The full shape ContractParser/ContractValidator accept. See Write a Contract for a guided introduction. The same shape is published as a JSON Schema, for editor validation: invaract-contract.schema.json.

Document shape

id: customer_orders
version: "1.0.0"
status: active

inputs:
  - name: orders
    location: raw/orders
    format: table
    schema:
      fields:
        - name: order_id
          type: string
          required: true
          nullable: false

outputs:
  - name: customer_orders
    location: gold/customer_orders
    format: table
    saveMode: overwrite
    schema:
      fields:
        - name: customer_id
          type: string
          required: true
          nullable: false

rules:
  - type: compatibility
    mode: backward

extensions:
  owner: data-platform-team

Top-level fields

Key	Required	Meaning
id	yes	Stable contract identifier.
version	yes	MAJOR.MINOR[.PATCH], e.g. "1.0.0" or "1.0".
status	no	Defaults to active. Free-form (e.g. active, deprecated, draft).
inputs	no	List of datasets the transformation reads.
outputs	yes	List of datasets the transformation produces. At least one required.
rules	no	List of {type, ...properties}. Most types are recorded, not yet interpreted — three are interpreted and enforced today, see Enforce Row-Level DML Rules.
customRuleTypes	no	Map of type: name → fully-qualified CustomRuleVerifier class name, for a rule type outside the three built-in ones. See Write your own rule type.
extensions	no	Free-form map, merged with any unrecognized top-level keys.

Dataset fields (inputs[] / outputs[])

Key	Required	Meaning
name	yes	Logical name, referenced in lookups and violation messages.
location	yes	Physical location (table name, path, topic) — see location matching below.
format	no	Storage/serialization format, compared case-insensitively. The canonical names are parquet, orc, avro, csv, json, text, table, delta, iceberg, hudi, hive, jdbc, bigquery and kafka. Any other value is accepted but draws a validation warning, since only an adapter that reports exactly that string can match it.
saveMode	no	Expected write behavior toward existing data (append/overwrite/ignore/error, compared case-insensitively). Outputs only; checked against the plan’s actual write mode when both are known. Any other value is accepted but draws a validation warning, since only an adapter that reports exactly that string can match it.
catalog	no	Expected data-catalog registration — see Catalog registration below. Meaningful for both inputs and outputs.
description	no	Free-form human-readable explanation of what this dataset is/contains. Purely documentary — never checked against a real write. An organizational policy can require every dataset to declare one.
type	no	Declared semantic role — DATA_ASSET, SOURCE, or CONTROL, matched case-insensitively. A closed set: any other value is rejected when the contract is parsed. See Declare Input and Output Types.
derivedFrom	no	Outputs only: the names of the contract’s inputs this output is built from. Absent means “all of them” (the default, and the only behavior before this key existed); a list scopes every input check for a write to this output to just those inputs. [] means derived from no declared input. A name that isn’t a declared input is a validation error. See Map Outputs to Their Inputs.
schema.fields	no	List of fields. Leave schema out (or list no fields) for a dataset whose content no contract governs, such as a dead-letter folder of text lines: its columns are not checked, and validation warns so a schema left out by mistake is never silent.

Field fields

Key	Required	Meaning
name	yes	Field name.
type	yes	Logical type — see known types below. An unrecognized type produces a validation warning, not a parse failure.
required	no	Defaults to false. Whether the field must be present.
nullable	no	Defaults to !required. Whether the value may be null when present.
properties	no	Nested field list, for struct/record types. Each nested field is checked against the actual struct with the same rules as a top-level field, and reported by its dotted path (address.zip).
sensitivityTags	no	List of governance labels (e.g. [pii, financial]) — see Sensitivity tags below. Defaults to none.
constraints	no	List of {type, ...} static data-quality constraints (equals/oneOf/range/length) — see Static data-quality constraints below. Defaults to none.

Known types

string, integer, long, short, byte, double, float, decimal, boolean, date, timestamp, timestamp_ntz, binary, time, json, geography, interval, struct, array, map. An unlisted type is a validator warning, not a rejection — a contract using a type Invaract doesn’t yet recognize still validates and parses.

time, json, geography and interval are matched exactly: an engine without that type reports the type it does have (a JSON document held in a string column is a string), and the declaration then fails as a type mismatch instead of being quietly satisfied.

Types can carry parameters, written as Spark writes them — decimal(10,2), array<int>, map<string,long>, struct<a:int,b:string>, nested to any depth. Only the leading keyword is checked against the list above.

These are logical types: Invaract defines what they mean, independent of any one engine, and maps each engine’s own column types onto them before comparing. Inside array<...>, map<...> and struct<...> the usual SQL aliases are also accepted (int, bigint, smallint, tinyint, real, dec/numeric, timestamp_ltz; a struct member may be written name:type or name type, with NOT NULL); a bare top-level type: must use the keyword from the list above — integer, not int. Note that a plain decimal does not match a column of a particular precision: declare decimal(10,2).

How a declared type is compared

• A plain type (integer, string, decimal(10,2)) must equal the actual column’s type.
• array<...>, map<...> and struct<...> are compared structurally against the actual type: element, key, value and member types must match, and a struct’s member names and count must too. Nullability inside a container (containsNull, a nested field’s nullable flag) is not compared; the field’s own nullable is.
• A bare array, map or struct with no properties means “any array / map / struct” and does not look inside. Declare array<int> (or give a struct properties) to have the contents checked.

For example, a contract field tags declared array<int> against a plan that produces array<string> is rejected as OUTPUT_FIELD_TYPE_MISMATCH:

field 'tags' declares type 'array<int>' but the actual OUTPUT schema has type 'array<string>'

and a struct field address with a nested zip: integer against a string column reports the mismatch on address.zip.

How field names are compared

Invaract matches contract field names to the plan’s columns the way Spark does: by the session’s spark.sql.caseSensitive setting. In a default session (false) a contract field ORDER_ID is satisfied by a column order_id; with spark.sql.caseSensitive=true names must match exactly. This applies to nested struct fields and to rejectUndeclaredFields too, and the violation reports the contract’s own spelling.

Validator checks

Parsing succeeding means a document was structurally interpretable; it doesn’t mean it’s well-formed. ContractValidator collects every issue in one pass:

Check	Severity
Empty/missing id	Error
id doesn’t match ^[a-zA-Z][a-zA-Z0-9_.-]*$	Warning
No outputs declared	Error
Empty/missing dataset name or location	Error
Duplicate dataset name within inputs or outputs	Error
Two outputs entries declaring the same location	Warning
Dataset with no schema, or a schema with zero fields (its columns are not checked)	Warning
Duplicate field name within a schema (including nested properties)	Error
Empty field name or type	Error
Field type not in the known-types set (and not a struct)	Warning
Field marked both required and nullable	Warning
Output saveMode, dataset format or catalog.technology outside the canonical names	Warning
Rule with empty type	Error
Interpretable rule type (the three DML rule types or the four transformation shape rule types) with malformed/missing properties	Error

Validation recurses into nested struct fields, so a warning on a deeply nested field reports its full dotted path (e.g. outputs[0].schema.address.zip).

A warning about a field type, a format or a save mode that is one or two letters away from a known one ends with a suggestion (Did you mean 'parquet'?). For a contract read from a file there is one more warning: a key Invaract doesn’t read but that is spelt close to one it does (nulable for nullable, input for inputs), since such a key is otherwise ignored and the setting it was meant to be keeps its default. Other unknown keys, including the Open Data Contract Standard’s own, are kept without a warning.

An error refuses every write the contract governs (on Beam, the whole pipeline, before it runs). Warnings don’t stop anything: they are logged once when a job starts with the contract, each starting Invaract: contract <id>@<version>: (see Something in the contract seems to be ignored).

Sensitivity tags

inputs:
  - name: customers
    location: raw/customers
    schema:
      fields:
        - name: ssn
          type: string
          sensitivityTags: [pii, restricted]

sensitivityTags is an open vocabulary — Invaract doesn’t define or restrict which labels exist, the same way a rule’s type accepts any string. Declared on an input field, a tag propagates forward to every output column that transitively derives from it — directly, or through however many computation steps sit in between — via spark-adapter’s SensitivityLineage, surfaced in demo/output/report.json’s transformationIR.lineage[].sensitivityTags (see Concepts → The Transformation IR’s “Column-level lineage”), including through an aliased read, a self-join, or a .checkpoint() in the middle of the job. This is reporting, for a human or downstream tooling auditing a transformation, not enforcement: a tagged field never causes StructuralVerifier to reject a write on its own — there is no “forbid this tag from reaching that output” rule type yet (see What isn’t part of the contract format below).

Static data-quality constraints

outputs:
  - name: payments
    location: gold/payments
    schema:
      fields:
        - name: currency
          type: string
          constraints:
            - type: equals
              value: GBP
        - name: status
          type: string
          constraints:
            - type: oneOf
              values: [ACTIVE, INACTIVE]
        - name: amount
          type: double
          constraints:
            - type: range
              gte: 0
        - name: identifier
          type: string
          constraints:
            - type: length
              exact: 10
        - name: end_date
          type: long
          constraints:
            - type: fieldRange
              gte: start_date

type	Properties	Meaning
equals	value	The field must always equal this exact value.
oneOf	values (non-empty list)	The field’s value must always be one of these.
range	any of gte/gt/lte/lt (numeric; at least one required; gte+gt or lte+lt together is malformed)	The field’s numeric value must always fall within these bounds.
length	exact, or either/both of min/max (non-negative integers; at least one required; exact combined with min/max is malformed, as is min > max)	The field’s string length must always match exact, or fall within min/max (inclusive).
fieldRange	any of gte/gt/lte/lt (each naming another field in the same schema; at least one required; gte+gt or lte+lt together is malformed)	The field’s numeric value must always satisfy these bounds relative to another field’s own value on the same row (e.g. end_date >= start_date). Only proven for top-level, numeric-typed output fields on both sides — see the guide below.

NOT NULL needs no entry here — nullable: false (above) already states it, and is proven by the same mechanism as any other constraint here. See Verify Static Data Quality for how these are proven, not merely predicted, and why an unproven constraint never fails a write on its own.

Catalog registration

outputs:
  - name: sales
    location: /warehouse/sales
    format: parquet
    catalog:
      required: true
      technology: hive
      catalogName: spark_catalog
      location: thrift://metastore1.example.com:9083
      namespace: [default]
      table: sales

Key	Required	Meaning
required	yes (within a present catalog: block)	Whether this dataset must be registered in a catalog at all. false records an expected shape without gating anything — ContractValidator warns on this combination.
technology	no	The catalog implementation. The canonical names are hive, delta, iceberg, hudi, jdbc, glue, bigquery and in-memory; any other value is accepted but draws a validation warning, since only an adapter that reports exactly that string can match it.
catalogName	no	The Spark-level catalog plugin/session-catalog name, e.g. spark_catalog — a local alias.
location	no	The catalog service’s own address (e.g. a Hive metastore’s thrift://host:port URI) — real and checked for Hive; always unset (and never compared) for Delta/Iceberg/JDBC today, since no reflective accessor exists for those catalogs’ own network endpoints. See Require Catalog Registration for the full explanation.
namespace	no	The database/schema path within the catalog, as an ordered list, e.g. [default].
table	no	The table name within that namespace.

catalog: is opt-in per dataset and orthogonal to format/saveMode — a dataset that omits it behaves exactly as it did before this field existed. Every sub-field beyond required is compared only when declared, and only when the actual side has a real value to compare against (an unset actual sub-field is treated as “unknown,” never a mismatch — see Require Catalog Registration for the full guide, including a real captured mismatch violation).

Location matching

A declared location is not compared with the place a job reads or writes character for character. Both are put in one form first, so the way either is spelt doesn’t decide a match:

• \ becomes /, and repeated, trailing and ./ slashes go: data//orders/ is data/orders.
• A file: prefix goes: file:/data/orders and file:///data/orders are /data/orders.
• A URL scheme is lower-cased, and one store’s spellings are one: s3a:// and s3n:// are s3://, abfss:// is abfs://, wasbs:// is wasb://.

Upper and lower case in the rest still matter (paths and object keys are case-sensitive), and .. is left as written. Then a declared location matches when it is the actual location, or is the end of it, starting at a /.

So a location that starts from a root, such as /data/orders or s3://bucket/orders, matches that one place. A relative one, such as orders or sales/orders, matches any location that ends in it: a contract needn’t know where a deployment keeps its data, but orders can’t tell two teams’ .../orders apart. An organization that wants every location pinned can require it with the require_absolute_location policy.

Contract says	The job reads or writes	Matches?
demo/input/sample.csv	file:/home/u/repo/demo/input/sample.csv	✅
sample.csv	file:/home/u/repo/demo/input/sample.csv	✅ (any file with that name)
put/sample.csv	file:/home/u/repo/demo/input/sample.csv	❌ a match starts at a /
orders	/data/orders/	✅
s3://bucket/orders	s3a://bucket/orders	✅
bucket/orders	s3a://bucket/orders	✅
/data/orders	/archive/data/orders	❌ it starts from the root
/data/orders	hdfs://nn:8020/data/orders	❌ write data/orders, or the whole URL
Orders	/data/orders	❌ case matters
orders	/data/orders/date=2024-01-01	❌ a partition’s folder isn’t the dataset

Tables

A read or write of a catalog table also answers to the table’s name, <catalog>/<namespace>/<table>: spark_catalog/gold/orders, or spark_catalog/default/orders for a table in Spark’s default database. A contract can name the table that way, or by the end of the name (gold/orders, default/orders), instead of where its files are. This matters because Spark doesn’t always report a table the same way: usually by its files (file:/warehouse/gold.db/orders), but a write that creates a Hive, Delta or Iceberg table by the table’s qualified name (spark_catalog.gold.orders). The table’s name is the same either way, so a contract that names the table matches every write to it, and a rollback_to_snapshot or similar CALL on it too.

Write the parts with /. A dotted name such as gold.orders is one segment: it matches only a location spelt exactly that way, never the table. An engine that names other data by parts is converted the same way: BigQuery’s project.dataset.table is project/dataset/table, so dataset/table matches that table in any project and the full form pins one.

Resolving a location at runtime

location can also be ref://<id> — a reference resolved to a real literal location at job startup, instead of a value hardcoded in the contract. This is a spark-adapter-level convention, not something ContractParser/the JSON Schema interpret specially: parsing and validation see it as an ordinary non-empty string either way. See Resolve Dataset Locations at Runtime for the full mechanism.

Multiple outputs

outputs is a list — a contract can declare more than one. Each write a job performs is checked on its own (Invaract verifies one write at a time, as it happens), and is matched against whichever declared output’s location it corresponds to, using the same location-matching rule above. Format, saveMode, catalog, and schema are all then checked against that matched output specifically — a write matching your gold output is never checked against silver’s schema, or vice versa. A write matching none of the declared output locations is rejected (OUTPUT_LOCATION_MISMATCH, naming every declared candidate); two outputs sharing the same location is legal but ambiguous, and ContractValidator warns about it (see below) — the first one declared wins the match.

Interpreted rules

Two independent families, checked by two different verifiers against two different things — see each guide for the full semantics.

Row-level DML rules (RuleVerifier, checked against one extracted MERGE/UPDATE/ DELETE operation):

rules:
  - type: merge_condition
    columns: [customer_id]
  - type: forbid_unconditional_delete
  - type: allowed_update_columns
    columns: [status, updated_at]

See Enforce Row-Level DML Rules.

Transformation shape rules (PlanRuleVerifier, checked against the transformation’s whole plan, independent of whether it’s DML at all):

rules:
  - type: required_group_by
    columns: [customer_id]
  - type: forbid_cross_join
  - type: required_join_columns
    columns: [order_id]
  - type: required_filter_columns
    columns: [is_deleted]

See Enforce Transformation Shape Rules.

A third, independent mechanism — not a rules entry at all, but a constraints list on a field (see Static data-quality constraints above) — is checked by StaticDataQualityVerifier against per-column value-domain facts it can prove from the transformation’s own semantics, opt-in via staticDataQuality. See Verify Static Data Quality.

Every other rules entry type (e.g. compatibility) is recorded on the parsed contract but not yet interpreted or enforced — unless it’s named in customRuleTypes, in which case it’s dispatched to a reflectively-resolved CustomRuleVerifier instead (row-level DML rule types only — see Write your own rule type).

JSON Schema

contract/schema/invaract-contract.schema.json (Draft 2020-12) is a standalone, language-agnostic description of this same shape — useful for IDE validation/autocomplete while authoring a contract by hand, or for generating one from another language. Point your editor at it:

# yaml-language-server: $schema=../../contract/schema/invaract-contract.schema.json
id: my_contract

Note

The schema mirrors ContractParser’s hard failures and ContractValidator’s Error-level checks, but not warning-level or cross-field business rules (like duplicate-name detection, which JSON Schema has no clean way to express) — those stay engine-only.

What isn’t part of the contract format

• Compatibility mode, quality expectations, and anything else rules can carry beyond the three interpreted types — recorded, not yet enforced.
• Governance rules (masking, residency, purpose limitation, or any policy that would reject a write based on where sensitive data ends up) — sensitivityTags (above) gives a field-level labeling vocabulary and propagates it through lineage for reporting, but there is no rule type yet that turns a tag into an enforced constraint.
• Column-level lineage as a first-class contract concept — a contract can label an input field, but it cannot yet declare an expected lineage shape for verification to check an output against (e.g. “this column must derive only from these sources”).

See Data Contracts for how this format relates to ODCS, and What is Invaract? for what’s verified against it today.