Organizational Policy Format

The full shape OrgPolicyParser/OrgPolicyValidator accept. See Enforce an Organizational Policy for a guided introduction.

Document shape

# yaml-language-server: $schema=https://mlltx.github.io/Invaract/schemas/contract/invaract-org-policy.schema.json
version: "1.0"

policies:
  - id: catalog-required
    type: require_catalog
    technology: hive
    scope: outputs
    description: "All outputs must be registered in a discoverable catalog."
  - id: pii-flag-required
    type: require_field
    name: pii_reviewed
    fieldType: boolean
    scope: outputs
  - id: snake-case-fields
    type: field_naming_convention
    pattern: "^[a-z][a-z0-9_]*$"
    mode: warn
  - id: owner-required
    type: require_extension
    key: owner
    description: "Every contract must declare an owning team."
  - id: dataset-described
    type: require_dataset_description
  - id: sunset-date-if-deprecated
    type: require_extension_if
    ifKey: status
    ifValue: deprecated
    thenKey: sunsetDate
    description: "A deprecated contract must declare its sunset date."

inject:
  rules:
    - type: forbid_unconditional_delete
  minVerificationOptions:
    rejectUndeclaredFields: true

exemptions:
  - contractId: legacy_customer_orders
    policyIds: [catalog-required]
    reason: "predates policy; migration tracked in JIRA-1234"
    reviewBy: "2026-12-01"

customPolicyTypes:
  require_approved_location_prefix: com.acme.governance.RequireApprovedLocationPrefix

typeGuarantees:
  enabled: [data_asset_schema_consistency, data_asset_downstream_consumption]
  mode: enforce

controlTables:
  - location: control/watermark_state
    owner: streaming-team
    requiredFields: [watermark_ts, run_id]

Top-level fields

Key	Required	Meaning
version	yes	Free-form revision identifier for this document — your own tracking, not compared against a contract’s ContractVersion.
policies	no	The policy rules themselves — see below.
inject	no	Rules/options this policy contributes to every governed contract — see Rule/option injection.
exemptions	no	Platform-controlled exceptions — see Exemptions.
customPolicyTypes	no	Maps a type: name outside the built-in set to a CustomPolicyEvaluator class — see Custom policy types.
typeGuarantees	no	Configures cross-contract type guarantee checks (data_asset_schema_consistency/data_asset_downstream_consumption, and any registered custom check) — see Type guarantees. Unlike every other field here, only CrossContractLintCli --org-policy consults this one; OrgPolicyEvaluator/ContractEnforcementRule never do, since it governs a check across many contracts, not one contract at a time.
controlTables	no	The org-owned catalogue of known CONTROL-declared datasets, checked by require_control_registration — see Keep a CONTROL table honest.

Policy rule fields (policies[])

Key	Required	Meaning
id	yes	Stable identifier, referenced by exemptions and shown in violation messages. Must be unique within the document.
type	yes	Policy kind — see Policy types below.
description	no	Human-readable explanation, appended to violation messages for self-service remediation.
scope	no	inputs, outputs, or all (default). Which of a contract’s datasets this rule applies to.
when	no	{ sensitivityTag: <tag> } — narrows scope further to only datasets with at least one field (recursively) carrying that sensitivity tag.
mode	no	enforce (default) blocks a write before any plan is analyzed. warn only publishes a notification event and never blocks — the safe-rollout mechanism for a newly-introduced rule.

Every other key on a policy rule is type-specific and captured verbatim, the same way a contract’s own rules: entries work.

Policy types

Type	Properties	Checks
require_catalog	technology (optional)	The dataset declares catalog.required: true, optionally matching a specific technology.
require_field	name (required), fieldType (optional)	The dataset’s schema declares a top-level field named name — not recursive into nested structs — optionally of fieldType (checked case-insensitively).
field_naming_convention	pattern (required)	Every field name in the dataset’s schema — recursing into nested struct properties — fully matches the regular expression pattern. A partial match doesn’t satisfy it.
require_extension	key (required), value (optional)	The contract itself declares key in its top-level extensions map (e.g. extensions: { owner: data-platform-team }), optionally pinning key to an exact value (case-sensitive). A key present but mapped to YAML null counts as absent.
require_format	formats (required)	The dataset declares format as one of formats — a single string (formats: delta) or a list (formats: [delta, iceberg]) — matched case-insensitively. A dataset with no format at all does not satisfy this.
require_dataset_description	(none)	The dataset declares a non-blank description.
require_dataset_type	types (optional)	The dataset declares a type (DATA_ASSET/SOURCE/CONTROL) at all; if types is also set (a single string or a list), the declared type must additionally be one of them.
forbid_control_sensitivity_tags	tags (optional, defaults to [pii, financial])	A dataset already declared type: CONTROL must not carry any schema field (recursing into nested structs) tagged with one of tags. A dataset with no declared type, or declared DATA_ASSET/SOURCE, is never in scope — see Keep a CONTROL table honest below.
require_control_registration	requireRegistration (optional, defaults to true)	A dataset already declared type: CONTROL must match a location entry in the top-level controlTables catalogue, and once matched, its schema must carry every field that entry’s requiredFields names. requireRegistration: false skips the “must be registered at all” half but still enforces conformance against any entry that does match. See Keep a CONTROL table honest below.
require_absolute_location	catalogs (optional)	The dataset’s location names one place on its own: a URL (s3://bucket/orders) or a path from the root (/data/orders), not a relative location (orders, sales/orders) that matches any location ending in it. With catalogs (a single name or a list, e.g. spark_catalog), a table named in full from one of them (spark_catalog/gold/orders) is allowed too. See Allow only absolute locations below.
require_extension_if	ifKey (required), ifValue (optional), thenKey (required), thenValue (optional)	A conditional require_extension: only once the contract itself already satisfies ifKey (optionally =ifValue) must it also satisfy thenKey (optionally =thenValue). A contract not satisfying the if condition at all produces no violation.

A policy type matching neither this set nor a customPolicyTypes entry is recorded but never checked — the same permissive behavior an unrecognized contract rules[].type already has, though OrgPolicyValidator now reports it as a Warning (not an Error) since customPolicyTypes makes this shape more likely to be a typo than intentional. A known type with malformed or missing properties for its shape (e.g. require_field with no name) is an OrgPolicyValidator Error, not silently ignored.

require_field's type pin is fieldType, not type

A policy rule’s own kind (require_field) is itself spelled type: at the same mapping level. SnakeYAML doesn’t reject a document with two type: keys — it silently keeps only the last one, which would overwrite the rule’s own kind rather than raise anything. The type pin therefore uses the distinct key fieldType.

require_extension and require_extension_if check the whole contract, not a dataset

require_catalog, require_field, field_naming_convention, require_format, require_dataset_description, require_dataset_type, forbid_control_sensitivity_tags, require_control_registration and require_absolute_location each check one dataset at a time, honoring scope/when. require_extension and require_extension_if are different: they check the contract document as a whole, exactly once, so scope/when have no effect on either — setting either produces an OrgPolicyValidator Warning (not an Error; the rule still evaluates correctly), since it’s likely not what the author intended.

require_extension_if answers 'only if' requirements

require_extension can require a key to exist, but can’t make that requirement depend on another key’s value. require_extension_if fills that gap — e.g. only a contract whose extensions.status is deprecated needs to also declare extensions.sunsetDate; a contract that isn’t deprecated is entirely unaffected by the rule, not merely exempted from it.

Allow only absolute locations

A relative location such as orders matches any location that ends in /orders (Location matching). That spares a contract from knowing where each deployment keeps its data, but it also means orders can’t tell your team’s orders from another team’s. A platform that wants every contract to say exactly where its data is turns relative locations off:

policies:
  - id: absolute-locations
    type: require_absolute_location
    description: Name each dataset by its full URL or path, so a contract pins one place.

A contract with location: sales/orders is then refused before any write, with a violation like:

organizational policy 'absolute-locations' (Name each dataset by its full URL or path, so a contract pins one place.) requires dataset 'orders' to declare an absolute location (a URL such as 's3://bucket/orders', or a path from the root such as '/data/orders'), but 'sales/orders' is relative: it matches any location that ends in it.

A table name has no leading / or scheme, so it counts as relative too. To allow tables named in full from their catalog, list the catalogs:

  - id: absolute-locations
    type: require_absolute_location
    catalogs: [spark_catalog]

spark_catalog/gold/orders is then allowed and gold/orders is not, because it would match a gold/orders table in any catalog.

The rule is checked on the contract a job enforces, after its ref:// locations are resolved, so a ref:// id that resolves to a relative location is refused too. The standalone lint can’t resolve a ref:// location, so it lets one through. Use scope: outputs to require absolute locations only where a job writes.

Keep a CONTROL table honest

forbid_control_sensitivity_tags exists because type: CONTROL is self-declared by whoever writes the contract — nothing stops a real business dataset from being labeled CONTROL to dodge DATA_ASSET-scoped obligations like require_catalog or role-consistency checking. A genuine control/watermark/processing-calendar/reconciliation signal has no legitimate reason to carry sensitive business data, so a CONTROL-declared dataset whose schema does is a mechanical, single-contract red flag:

policies:
  - id: control-no-sensitive-fields
    type: forbid_control_sensitivity_tags
    mode: enforce

With no tags property, this forbids the default pii/financial set on every CONTROL-declared dataset in the contract. Pin your own vocabulary instead:

policies:
  - id: control-no-restricted-fields
    type: forbid_control_sensitivity_tags
    properties:
      tags: [pii, financial, restricted]

Design your other rules so CONTROL isn't a free pass

If a universal rule like require_catalog or require_dataset_description is scoped or conditioned so it only applies to DATA_ASSET datasets, relabeling something CONTROL becomes the easiest way to dodge it. Keep governance rules that should bind everyone scoped all/outputs with no type-based narrowing, and reserve a type-conditioned rule like forbid_control_sensitivity_tags for checks that are specifically about whether the type claim itself holds up.

Require CONTROL tables to be registered

forbid_control_sensitivity_tags catches one honest signal that a CONTROL claim is wrong — but it proves nothing about an unsuspicious declaration nobody ever actually approved. controlTables is the org-owned catalogue that closes that gap: it lives in the policy document, not the contract, so only whoever owns the policy can add an entry — exactly the same trust boundary exemptions already gives you.

controlTables:
  - location: control/watermark_state
    owner: streaming-team
    purpose: "Tracks the last successfully processed batch."
    requiredFields: [watermark_ts, run_id]
  - location: control/processing_calendar
    owner: data-platform-team
    reviewBy: "2026-12-31"

policies:
  - id: control-table-registry
    type: require_control_registration
    mode: enforce

controlTables is purely documentary until require_control_registration is attached — you can build the catalogue up first, then turn on enforcement once it’s populated. OrgPolicyValidator warns if you have entries but no such rule, since that’s almost always a forgotten step rather than intentional.

The rule has one knob for how far to take it — requireRegistration, default true:

• true (strict) — “CONTROL is the only thing allowed, and only once registered.” A CONTROL-declared dataset with no matching controlTables entry is itself a violation.
• false (loose) — registration is optional; an unregistered CONTROL dataset is never flagged for that alone. But a dataset that is registered still has to conform: if the matching entry names requiredFields, your schema must carry them, or it’s a violation regardless of requireRegistration. This is the shape a platform rolling the catalogue out gradually wants — register real control tables as you find them, without retroactively blocking everything nobody’s gotten to yet.

A registration can lapse, too: give it a reviewBy date (identical semantics to an exemption’s own) and a stale entry stops counting as registered — the dataset falls back to unregistered, forcing periodic re-approval rather than a one-time rubber stamp.

A catalogue, not just a gate

Because controlTables is a plain top-level list, it doubles as a browsable inventory of every control table your organization has approved — owner, purpose, and expected shape all in one place — independent of whether you’ve turned on enforcement yet.

Custom policy types

customPolicyTypes maps a type: name your document uses to the fully-qualified class name of a com.invaract.contract.CustomPolicyEvaluator implementation — the escape hatch for a policy type outside the built-in set above, with no Invaract source change needed. See Write your own policy type for a full worked example.

customPolicyTypes:
  require_approved_location_prefix: com.acme.governance.RequireApprovedLocationPrefix

Key	Meaning
customPolicyTypes.<type>	Fully-qualified class name of a CustomPolicyEvaluator (public no-arg constructor required), resolved reflectively and cached — one instance per class name, reused across every rule/contract that names it.

trait CustomPolicyEvaluator {
  def evaluate(contract: Contract, rule: PolicyRule): List[PolicyViolation]
}

OrgPolicyEvaluator only consults customPolicyTypes for a type: outside the ten built-in ones — a customPolicyTypes key matching a built-in type is dead (the built-in always wins; OrgPolicyValidator warns on this). PolicyExemption coverage and mode: enforce/warn splitting still apply automatically to a custom type’s violations, exactly like a built-in one.

Composes with spark.invaract.orgPolicy for free

A customPolicyTypes entry is just more content in the same YAML document spark.invaract.orgPolicy=<path> already names — the evaluator class itself is supplied via --jars, the same as any other Invaract extension jar. No new spark-submit flag, no job source change.

OrgPolicyValidator outcome	When
Error	customPolicyTypes.<type> has an empty class name, or names a class that doesn’t resolve (missing, no public no-arg constructor, doesn’t implement CustomPolicyEvaluator).
Warning	customPolicyTypes.<type> collides with a built-in type (dead — the built-in always wins).
Warning	A policy rule’s type matches neither a built-in type nor a customPolicyTypes entry (it will never be evaluated).

Type guarantees

typeGuarantees configures com.invaract.contract.TypeGuaranteeValidator’s cross-contract “stronger semantic and guarantee validation” checks — see Declare Input and Output Types for a guided introduction. Only CrossContractLintCli --org-policy reads this block; OrgPolicyEvaluator does not.

typeGuarantees:
  enabled: [data_asset_schema_consistency, my_custom_check]
  mode: warn
  customTypeGuaranteeTypes:
    my_custom_check: com.acme.governance.MyTypeGuaranteeCheck

Key	Meaning
typeGuarantees.enabled	Which checks to run, by name. Two are built in: data_asset_schema_consistency, data_asset_downstream_consumption (see the guide linked above for what each one checks). Empty or omitted means no type guarantee check runs at all.
typeGuarantees.mode	enforce (default) — a real Contradicts result from any enabled check fails the lint run. warn — every enabled check still runs and is reported, but nothing blocks.
typeGuarantees.customTypeGuaranteeTypes.<name>	Fully-qualified class name of a com.invaract.contract.TypeGuaranteeCheck implementation (public no-arg constructor required) — the escape hatch for a check outside the two built in, resolved reflectively by TypeGuaranteeCheckFactory, exactly the same shape customPolicyTypes already uses for CustomPolicyEvaluator.

trait TypeGuaranteeCheck {
  def check(contracts: List[Contract]): List[TypeGuaranteeResult]
}

OrgPolicyValidator outcome	When
Error	typeGuarantees.enabled has an empty entry.
Error	typeGuarantees.customTypeGuaranteeTypes.<name> has an empty class name, or names a class that doesn’t resolve (missing, no public no-arg constructor, doesn’t implement TypeGuaranteeCheck).
Warning	typeGuarantees.customTypeGuaranteeTypes.<name> collides with a built-in check name (dead — the built-in always wins).
Warning	A typeGuarantees.enabled entry matches neither a built-in check nor a customTypeGuaranteeTypes entry (it will never be evaluated).

inject.rules/inject.minVerificationOptions

Key	Meaning
inject.rules	A list of ordinary DML rule entries ({type, ...properties}, the same shape a contract’s own rules: uses) merged into every governed contract’s rules before verification runs. A rule the contract already declares isn’t duplicated.
inject.minVerificationOptions	A map of VerificationOptions flag name (rejectUndeclaredInputs, rejectUndeclaredFields, computeFingerprint) to true — ORed onto a job’s own options. A flag a job left false can be forced true by policy; the reverse never happens. An unrecognized key (e.g. a typo) is rejected — ContractEnforcementRule fails loudly rather than silently ignoring it.

Exemptions

Key	Required	Meaning
contractId	yes	The Contract.id this exemption covers.
policyIds	yes	Which policy rule ids (at least one) this exemption suppresses, for this contractId only.
reason	yes	Why the exemption exists — required, for auditability.
reviewBy	no	An ISO-8601 date (YYYY-MM-DD). Once passed, the exemption stops applying and the violation it was suppressing re-surfaces. Omitted means it never expires.

Exemptions live in the policy file, not the contract

A contract’s own author can’t grant themselves an exemption — that would defeat the purpose of an organization-wide rule. Only whoever owns the policy document can.

Enforce mode vs. warn mode

mode	Blocks a write?	Published event
enforce (default)	Yes — throws before any plan is analyzed	A FAILED-status ContractValidationEvent, if a sink is configured
warn	Never	A PASSED-status ContractValidationEvent still carrying the violation, if a sink is configured

Attaching a policy

Mechanism	How
spark-submit --conf	--conf spark.invaract.orgPolicy=/path/to/org-policy.yaml — attachable to any job that already installs ContractEnforcementRule.forContract, with no change to that job’s source.
Standalone lint	cd contract && sbt "runMain com.invaract.contract.cli.OrgPolicyLintCli org-policy.yaml contracts/" — no Spark involved, for a contracts repository’s own CI/pre-commit gate.

Standalone lint: exit codes and flags

Exit code	Meaning
0	Every discovered contract satisfies every enforce-mode rule, in every layer. A warn violation still prints but never fails the run.
1	The base policy or any overlay is invalid, a contract fails to parse, or no contract files were found.
2	Usage error — too few arguments, a malformed --warn-expiring-within-days value, or --overlays with no value.

Flag	Meaning
--warn-expiring-within-days N	Reports any exemption (in any layer) whose reviewBy falls within the next N days — before it lapses, not after. Defaults to 30; never affects the exit code, since the exemption is still fully in force.
--overlays overlay1.yaml,overlay2.yaml	Lints against the full layered stack a governed job would actually run under, not just the base — see below.

See Enforce an Organizational Policy for both, worked through in full, and Violation Types for ORG_POLICY_VIOLATION.

Policy layering

A stricter business-unit (or team) policy can compose with a looser org-wide baseline, without the business unit’s document duplicating or replacing anything the org-wide one already declares. An overlay is not a different document shape — it’s an ordinary OrgPolicy YAML document, in every respect identical to the base, layered on top of it.

spark-submit \
  --conf spark.invaract.orgPolicy=/policies/org-wide.yaml \
  --conf spark.invaract.orgPolicyOverlays=/policies/bu-finance.yaml,/policies/team-payments.yaml \
  ...

Key	Meaning
spark.invaract.orgPolicy	The org-wide base layer — unchanged from the single-document case above.
spark.invaract.orgPolicyOverlays	Optional, comma-separated, ordered list of additional OrgPolicy documents layered on top of the base. Requires spark.invaract.orgPolicy to also be set — an overlay with no base is rejected rather than silently treated as one.

Each layer is evaluated independently and the violations are unioned, so a stack of layers is always at least as strict as any one layer alone:

• A layer can only add, never loosen. There is no mechanism for an overlay to override, replace, or loosen a rule an earlier layer declares. A business unit wanting a stricter version of a check the org-wide layer already has (e.g. tightening warn to enforce) adds its own additional rule of that type — both rules are then evaluated and unioned, the same as any other pair of rules.
• A layer’s exemptions cover only that layer’s own policies. An overlay’s exemptions can grant an exception to a rule that same overlay declares, but it is structurally incapable of exempting a rule the org-wide layer (or any other layer) declares — OrgPolicyValidator rejects an exemption naming a policy id its own document doesn’t declare, and each layer’s exemptions are only ever checked against that layer’s own policies. Only the document that owns a rule can grant an exemption for it.
• Custom policy types stay per-layer. Each layer resolves its own customPolicyTypes against only its own map, so two layers can independently use the same type: name for unrelated CustomPolicyEvaluator classes without colliding.

A policy id repeated across layers is a Warning, not a problem

Each layer evaluates independently, so a repeated id across layers doesn’t cause shadowing or misattributed exemptions. It’s still worth avoiding, though — a human reading two violations both attributed to catalog-required can’t tell which layer produced which. Prefix a layer’s own rule ids (e.g. bu-finance-catalog-required) rather than relying on id alone to distinguish them.

Linting a layered stack

# run from the repository root; file paths are relative to contract/, or absolute
cd contract && sbt "runMain com.invaract.contract.cli.OrgPolicyLintCli --overlays bu-finance.yaml,team-payments.yaml org-policy.yaml contracts/"

The same base-first, comma-separated convention as the conf key — useful for a business unit’s own CI to lint its contracts against org-wide + its own overlay, with the same argument shape spark-submit would take. Every policy document (base and overlays alike) is excluded from the contract sweep, the same as the single-document case.