Why use it?

The problem

Modern data platforms usually split contract-adjacent concerns across several tools:

• Data contracts describe what a dataset is supposed to look like.
• Lineage systems describe what a job actually did, after the fact.
• Data quality systems check data after (or during) execution.
• Catalogs describe datasets and their relationships.

None of these actually answer the question that matters before data lands:

Does this job’s output match what the contract for this dataset promised?

A pipeline can claim “this job produces customer_orders.” A lineage system can later observe “this job read orders and wrote customer_orders.” Neither one proves that the transformation’s actual output — schema, location, save mode — matches what the contract requires. Usually nobody finds out until a downstream consumer breaks.

Why check before the write, not after

Post-hoc data quality checks (run a validation query after the table is written) catch problems late: bad data has already landed, and something downstream may have already read it. Invaract instead checks the job before the write happens. On Spark it hooks into query analysis and rejects a violating write before Spark executes it; on Beam it checks the whole pipeline before it is handed to a runner. The target file or table is never created. See Verification vs. Enforcement for how this differs from Invaract’s own post-hoc reporting path.

Why the real job, not a schema diff

Comparing “the contract’s declared output schema” against “the DataFrame’s schema right before .write()” sounds sufficient, but it can’t see how the data actually got there — whether a MERGE matched on the columns it should have, or whether a DELETE had a filter at all. Invaract translates the engine’s own description of the job (Spark’s logical plan, Beam’s pipeline graph) into an engine-independent IR (see The Transformation IR), so verification is checking the transformation that is about to run, not a description of it.

Why it’s worth trusting

A verification tool that silently lets through what it doesn’t understand is worse than no tool at all — it creates false confidence. Invaract is deliberately built to fail closed: a write shape it doesn’t recognize is rejected, not passed through unverified. See Fail-Closed by Default for why, and how that policy was arrived at from a real gap found during development (an early version silently let an unrecognized Delta write through with no verification at all).

When Invaract is a good fit

• You already write, or want to write, machine-readable data contracts (ODCS-shaped or close to it) for tables your Spark jobs or Beam pipelines produce.
• You want a write to genuinely fail — not just log a warning — when it violates its contract.
• Your Spark jobs write via Delta Lake, Iceberg, Parquet, CSV, Hive, Avro or ClickHouse, or your Beam pipelines through file, BigQuery, Kafka, JDBC or Pub/Sub IOs — see Connector Support for what’s covered today.

When it isn’t (yet)

• You need to verify business logic correctness (is the aggregation itself right?), not just structural shape — out of scope today, see What is Invaract?.
• You’re on an engine other than Spark or Beam (Flink, a warehouse’s own SQL, dbt). The IR is engine-independent, but no adapter for those exists yet.
• You need a check Invaract can’t make on your engine, such as row-level DML rules on Beam: see What Each Engine Checks. A contract that relies on one is refused, not silently half-checked.