Skip to content

Load a model with a resolved runtime

By the end of this you will have a program that finds the ONNX Runtime for whatever machine it is on, downloading and verifying it the first time, and reusing the cached copy afterwards.

You will need Go 1.26 or later and about ten minutes, most of it waiting for a 150 MB download the first time you run it.

1. Start a module

$ mkdir ort-demo && cd ort-demo
$ go mod init example/ort-demo
$ go get gitlab.com/phpboyscout/go/onnxruntime

2. Get the release key

The resolver fetches through go/artifacts, which will not construct a client without a verifier — there is no mode that downloads without checking who published the bytes.

The key is published over WKD and nowhere else, deliberately: it is not served from the artefact site, so compromising the site does not hand anyone the ability to publish a key you would then trust.

$ gpg --auto-key-locate clear,wkd \
      --locate-external-keys artifacts-release@phpboyscout.uk
gpg: key 34A711C4B9EAA99A: public key "Artifacts Release Signing
     <artifacts-release@phpboyscout.uk>" imported
$ gpg --armor --export artifacts-release@phpboyscout.uk > release.asc

Put release.asc beside your main.go. You are about to embed it, which is the point: the key travels in your binary rather than being fetched, at the moment it is needed, by the thing it is meant to protect.

Verify the fingerprint against the one published in the key rotation runbook before you trust it. WKD over TLS is good; a fingerprint you checked once is better.

3. Write the program

package main

import (
    "context"
    _ "embed"
    "fmt"
    "log"
    "os"

    "gitlab.com/phpboyscout/go/artifacts"
    "gitlab.com/phpboyscout/go/artifacts/trust"
    "gitlab.com/phpboyscout/go/onnxruntime"
)

//go:embed release.asc
var releaseKey []byte

func main() {
    ctx := context.Background()

    verifier, err := trust.Estate(releaseKey)
    if err != nil {
        log.Fatal(err)
    }

    cache, err := os.UserCacheDir()
    if err != nil {
        log.Fatal(err)
    }

    client, err := artifacts.New(verifier,
        artifacts.WithCache(artifacts.NewDirCache(cache)))
    if err != nil {
        log.Fatal(err)
    }

    resolver, err := onnxruntime.New(client)
    if err != nil {
        log.Fatal(err)
    }

    lib, err := resolver.Resolve(ctx, "")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("runtime at:", lib)
}

A cache is not optional. go/artifacts refuses to resolve without one, because an artefact it cannot store is one it must re-download and re-verify on every call.

The empty string passed to Resolve means decide for me. It is where a configured path goes, and passing it through unconditionally — rather than branching on whether one was set — is how the precedence is meant to be used.

4. Run it

$ go run .
runtime at: /home/you/.cache/phpboyscout/onnxruntime/1.28.0/libonnxruntime.so

The first run downloads the archive, verifies the signature over its manifest, checks the digest, extracts one file, and caches it. Run it again:

$ go run .
runtime at: /home/you/.cache/phpboyscout/onnxruntime/1.28.0/libonnxruntime.so

Instant, and with no network at all. The cache is checked before the channel.

5. Prove the precedence

Point it at something else and watch the configured path win:

$ go run . 2>/dev/null; sed -i 's|Resolve(ctx, "")|Resolve(ctx, "/opt/ort/libonnxruntime.so")|' main.go
$ go run .
runtime at: /opt/ort/libonnxruntime.so

That path does not exist, and the resolver did not care. This is deliberate: if you have said where your runtime is, being quietly given a different one is worse than being given the loader's error about yours.

What next