Fail-Closed by Default

A verification tool that silently lets through what it doesn’t understand creates false confidence — worse than no tool at all, since a team relying on it believes writes are being checked when some genuinely aren’t. Invaract is built to fail closed instead: a write it can’t verify is rejected, not passed.

The gap this closes, on Spark

Every Spark write shape Invaract hasn’t been taught to translate — an early gap in Delta Lake support, and any future write shape the adapter doesn’t yet recognize — shares the same failure mode: SparkPlanAdapter produces ir.UnknownPlan instead of ir.Write. Early on, ContractEnforcementRule treated any non-ir.Write plan as “not a write, nothing to gate” — so a write Invaract simply didn’t recognize was indistinguishable from a SELECT or a .count(), and was silently let through, contract or no contract.

The fix: reject unrecognized commands, with an explicit safe list

A plan that’s Command-shaped (Spark’s own marker for “produces a side effect, not rows”) and doesn’t translate to ir.Write is now rejected outright — ContractViolationException with an UNVERIFIABLE_WRITE violation — rather than silently passed.

Why not just 'reject every unrecognized Command'?

That was the first design considered, and it’s unsafe. A real, jar-level survey of every concrete class implementing Spark’s Command trait found that Spark’s own Command hierarchy does not distinguish “writes data” from “pure catalog/session metadata” — a data-writing command and a schema-only CREATE TABLE can implement the exact same base trait, with no structural marker separating them. A blanket “reject every unrecognized Command” policy would also reject ordinary CREATE TABLE, ANALYZE TABLE, CACHE TABLE, SHOW TABLES, and dozens of other legitimate DDL/administrative operations — a severe regression, not a safety improvement.

The actual mechanism is an explicit, documented allowlist of commands judged — by their documented SQL semantics — to never change a table’s committed row content (schema/namespace/function DDL, SHOW/DESCRIBE/ANALYZE/CACHE, session config, storage maintenance like VACUUM/OPTIMIZE). Anything not on that list, if it’s Command-shaped and isn’t a recognized write, is rejected. This is deliberately asymmetric: a safe command missing from the list costs one unnecessary rejection until someone adds it (annoying, cheap to fix, loud); a data-mutating command wrongly added to the list would silently defeat the entire feature (invisible, expensive). Every uncertain case found during the survey was left off the safe list on that basis.

The same principle applies to DML rules

A contract’s merge_condition/forbid_unconditional_delete/allowed_update_columns rules (see Enforce Row-Level DML Rules) need specific facts extracted from a real MERGE/UPDATE/DELETE plan — and one case, Iceberg’s merge-on-read UPDATE, rewrites into a plan shape with no per-column before/after pairing to extract those facts from. Rather than silently reporting “no violation found” with nothing actually checked, this case is classified Unverifiable and rejected with RULE_UNVERIFIABLE_DML — but only when the active contract actually declares a rule that operation kind would need to be checked against; an unrelated rule never blocks an operation it doesn’t apply to.

On Apache Beam

The Beam adapters apply the same rule to a pipeline before it runs:

• A step that ends the pipeline without being a write Invaract recognizes (a function that pushes rows to a service, an IO it doesn’t know, a write whose destination the job computes per row) refuses the pipeline with UNVERIFIABLE_WRITE. A platform can name steps that only report, such as a metrics step, as trusted steps.
• A write whose columns Beam doesn’t state (plain Java or Python objects) is refused if its output declares columns, rather than passed with the schema unchecked.
• A contract that needs a check the engine can’t make (a row-level DML rule, a static data-quality proof on Beam) is refused with UNSUPPORTED_CONTRACT_FEATURE before the pipeline runs.

What this means in practice

If a job you’ve put under contract starts failing with UNVERIFIABLE_WRITE or RULE_UNVERIFIABLE_DML after upgrading Spark, Beam, Delta, or Iceberg, or after changing how a write is issued, that’s Invaract telling you it can no longer confirm the write is safe — not a false positive to work around. See Troubleshooting and Reference → Connector Support for what’s recognized today.