Skip to content
InvaractInvaractInvaract

Use Prebuilt Jars Without sbt

Invaract hasn’t published a Maven Central release yet (see Installation), so a Spark job built with a different toolchain — Maven, Gradle, or anything else that isn’t sbt — can’t add it as an ordinary dependency coordinate. This guide covers the alternative: a prebuilt jar, published as a GitHub Release asset, that you download instead of building Invaract from source yourself.

Where the jars come from

Section titled “Where the jars come from”

Every push to main that passes Invaract’s full CI suite (including the proof that enforcement actually blocks a bad write) rebuilds the engine’s jars and republishes them to a single rolling GitHub Release tagged spark-jars-latest.

You need one jar. invaract-spark-adapter-0.11.0.jar already contains everything else Invaract runs on (the contract model, the IR, fingerprinting and the shared verification code), so the other jars on the release page are only for building something of your own on a single module.

Terminal window
curl -fLO "https://github.com/mlltx/Invaract/releases/download/spark-jars-latest/invaract-spark-adapter-0.11.0.jar"

Check the release page for the current filename if the version has moved — it follows spark-adapter’s own version in its build.sbt, not the release tag.

Run with spark-submit — no Maven registration needed

Section titled “Run with spark-submit — no Maven registration needed”

Installing Invaract purely via spark-submit --conf is the simplest path: your job’s own source never imports anything from Invaract, so there’s nothing for your build to compile against.

Terminal window
spark-submit \
  --class com.example.MySparkJob \
  --master local[*] \
  --jars invaract-spark-adapter-0.11.0.jar \
  --conf spark.sql.extensions=com.invaract.sparkadapter.InvaractSparkSessionExtension \
  --conf spark.invaract.contract=/path/to/contract.yaml \
  target/my-job.jar \
  ...

The jar goes straight to --jars — no install:install-file, no pom.xml changes. Every other setting is another --conf: see Configuration Settings.

Calling Invaract’s APIs directly in code

Section titled “Calling Invaract’s APIs directly in code”

If you’re instead following Install and Configure Invaract Explicitly in Code — calling ContractEnforcementRule.forContract(...) yourself, because you need something the conf-driven path can’t express — your job’s code compiles against Invaract’s classes, so your build needs the jar at compile time too. With Maven:

  1. Install the jar into your local Maven repository

    Section titled “Install the jar into your local Maven repository”
    Terminal window
    mvn install:install-file \
      -Dfile=invaract-spark-adapter-0.11.0.jar \
      -DgroupId=com.invaract -DartifactId=invaract-spark-adapter \
      -Dversion=0.11.0 -Dpackaging=jar

    This puts it in ~/.m2/repository under the same com.invaract coordinate Invaract’s own build uses.

  2. Add it as a provided-scope dependency

    Section titled “Add it as a provided-scope dependency”
    <dependency>
      <groupId>com.invaract</groupId>
      <artifactId>invaract-spark-adapter</artifactId>
      <version>0.11.0</version>
      <scope>provided</scope>
    </dependency>

    provided is correct here, the same way it’s correct for spark-core/spark-sql in a Spark job’s pom.xml: your code compiles against these classes, but you supply the jar to Spark yourself at submit time via --jars (next step), not by shading it into your job’s own jar.

  3. Write your job

    Section titled “Write your job”

    Follow Install and Configure Invaract Explicitly in Code — nothing about that page’s code changes based on which build tool produced the jar.

  4. Run with spark-submit

    Section titled “Run with spark-submit”
    Terminal window
    spark-submit \
      --class com.example.MySparkJob \
      --master local[*] \
      --jars invaract-spark-adapter-0.11.0.jar \
      target/my-job.jar \
      ...

Picking up a new build

Section titled “Picking up a new build”

Because spark-jars-latest is a rolling release, “updating” is just re-running the download and install:install-file steps above — the same filename gets replaced with whatever main most recently passed CI with. There’s no version bump to track unless spark-adapter’s own version in build.sbt changes.

Apache Beam

Section titled “Apache Beam”

The release holds Spark jars only. For Beam, build the adapter from source (cd beam-adapter && sbt assembly), as the Java and Python guides show.