Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.archal.ai/llms.txt

Use this file to discover all available pages before exploring further.

Archal runs in any CI environment. Set the auth token, pick an output format, set a pass threshold. The build fails if satisfaction drops below it. This guide assumes you’ve already run npx archal init locally so .archal.json, ./.archal/harness.ts, and archal as a devDependency are committed to the repo. The CI job only needs the CLI binary and an ARCHAL_TOKEN.

​
Secrets

The only required secret is your Archal token: 
ARCHAL_TOKEN=arc_...
If your agent needs a model API key, set that too: 
ARCHAL_ENGINE_API_KEY=sk-...

​
GitHub Actions

name: Agent tests
on: [push]

jobs:
  archal:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install Archal
        run: npm install -g archal

      - name: Run scenarios
        env:
          ARCHAL_TOKEN: ${{ secrets.ARCHAL_TOKEN }}
          ARCHAL_ENGINE_API_KEY: ${{ secrets.ENGINE_API_KEY }}
        run: |
          archal run scenarios/close-stale-issues.md \
            --runs 3 \
            --pass-threshold 80 \
            -o json \
            -q

​
GitLab CI

archal:
  image: node:20
  script:
    - npm install -g archal
    - >
      archal run scenarios/close-stale-issues.md
      --runs 3
      --pass-threshold 80
      -o json
      -q
  variables:
    ARCHAL_TOKEN: $ARCHAL_TOKEN
    ARCHAL_ENGINE_API_KEY: $ENGINE_API_KEY

​
Useful flags

FlagWhat it does
--pass-threshold <score>Exit 1 if satisfaction is below this (0-100)
-o jsonMachine-readable JSON output
-qSuppress non-error output
-n, --runs <count>Run the scenario multiple times for a real satisfaction score
--tag <tag>Only run scenarios with a matching tag (exits 0 if no match)
--preflight-onlyValidate config and exit without running

​
Exit codes

CodeMeaning
0Score met the threshold (or scenario skipped by --tag)
1Score below threshold or runtime error
2Validation error (bad flags, missing scenario, invalid config)

​
Running multiple scenarios

If you have multiple scenario files, run them in a loop or use a suite in .archal.json: 
{
  "agent": {
    "command": "npx",
    "args": ["tsx", "src/agent.ts"]
  },
  "twins": ["github"],
  "suites": {
    "ci": {
      "scenarios": [
        "scenarios/close-stale-issues.md",
        "scenarios/triage-new-issues.md"
      ],
      "runs": 3,
      "timeout": 120
    }
  }
}

​
Tips

  • Start with --pass-threshold 60 and tighten as the agent improves.
  • Use --runs 3 or higher — single runs are noisy.
  • -o json for parseable output or CI artifacts.