Skip to content

Dev Commands

These scripts, under dev/, are how you build, exercise, and demonstrate Invaract from a cloned repository (see Installation).

CommandPurpose
./dev/testBuild every module, run the demo job on a real Spark session, generate and validate a report. The primary “does everything still work” command.
./dev/buildBuild every module’s jar, in dependency order, without running the demo job.
./dev/dry-runRun the demo job with no contract at all, and print one inferred from its actual inputs/outputs. See Infer a Starting Contract with Dry-Run Mode.
./dev/location-provider-demoRun the demo job against a contract whose input location is a ref:// reference, resolved at startup from a location map. See Resolve Dataset Locations at Runtime.
./dev/catalog-registration-demoA narrated walkthrough of catalog registration: three real Spark sessions against an embedded Hive metastore, one per scenario, printing every event published. See Require Catalog Registration.
./dev/registry-demoStart a real invaract-registry server, register a contract into it, and run the demo job against a registry://<id>@<version> reference resolved from that server. See Use a Contract Registry.
./dev/regressionContract regression pack — proves a satisfied contract executes and a violated one is aborted before any output is written. See Prove Enforcement with the Regression Pack.
./dev/regression-dockerSame regression pack, in a self-contained Docker image — no local JDK/sbt/Spark needed.
./dev/reportStart the results web UI on http://localhost:3000. See View Verification Results.

Each module is its own sbt build; there is no build at the repository root. Run these from inside a module’s directory (contract, ir, fingerprint, verification-core, spark-adapter, beam-adapter, plugin, …):

CommandPurpose
sbt testRun that module’s test suite in isolation.
sbt assemblyBuild the module’s jar. In beam-adapter, this builds the checker jar Beam Python pipelines need, target/scala-2.12/invaract-beam-checker-0.1.0.jar.

These check contracts without running any job, which makes them useful in a contracts repository’s own CI. Run them from the repository root; file paths are relative to contract/, or absolute.

CommandPurposeGuide
cd contract && sbt "runMain com.invaract.contract.cli.MergeDraftsCli --id <id> --version <version> <drafts-dir>"Merge the dry-run drafts of a job with several writes into one contract.Dry-run mode
cd contract && sbt "runMain com.invaract.contract.cli.OrgPolicyLintCli <org-policy.yaml> <contracts-dir>"Check every contract in a directory against an organizational policy.Enforce an organizational policy
cd contract && sbt "runMain com.invaract.contract.cli.CrossContractLintCli <contracts-dir>"Check that contracts which feed each other agree on the dataset types between them.Declare input and output types

For example, from a real run against the demo’s own contracts:

$ cd contract && sbt "runMain com.invaract.contract.cli.CrossContractLintCli ../demo/contracts/"
[ OK ] 10 contract(s) checked, no cross-contract issues found

./dev/test and ./dev/regression wrap this; running it directly is useful for trying a contract against the demo job without the full harness (as in Your First Contract):

Terminal window
spark-submit \
--class com.invaract.runner.DemoJobHarness \
--master local[*] \
--jars plugin/target/scala-2.12/invaract-spark-plugin-0.2.0.jar \
runner/target/scala-2.12/invaract-spark-runner.jar \
<input-path> <output-path> <report-path> <contract-path> <notify-config-path>

All five positional arguments are optional and default to the demo’s own paths (the notification config defaults to none — see Configure a Notification Sink) — you can override only the ones you need, e.g. just the contract path to try a different contract against the same demo data, or just the fifth to turn on notification without changing anything else (pass the demo’s own defaults for the earlier positional arguments to reach it). Add --dry-run anywhere in the argument list (before or after the positional ones) to run with no contract at all — see Infer a Starting Contract with Dry-Run Mode. Add --conf spark.invaract.locationMap=<path> (a Spark configuration flag, positioned before the jar like --jars above — not an argument to DemoJobHarness itself) to resolve any ref:// locations the contract declares from that .properties file before verification runs — see Resolve Dataset Locations at Runtime.

Every script here follows the same convention: exit code 0 means success — for ./dev/test, that the engine actually verified a real Spark write, not just that code compiled. A non-zero exit means something failed; check the script’s console output and demo/output/report.json first (see Troubleshooting).