Skip to content

kyvvu try

What you'll learn: How to see Kyvvu block an unsafe agent step with no account, no API key and no network — and how to run your own manifest and your own behaviour path the same way.

Usage

The output of kyvvu try --help:

 Usage: kyvvu try [OPTIONS]

 Enforce policies offline — no account, no API key, no network.

╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
│ --manifest      -m      <str>  Manifest YAML to enforce (repeatable). Default: the bundled       │
│                                frozen baseline.                                                  │
│ --steps                 <str>  YAML file describing your own behaviour path.                     │
│ --json                         Emit JSON instead of a report.                                    │
│ --record-trial                 Opt in to a local trial marker in ~/.kyvvu. Off by default; sent  │
│                                nowhere.                                                          │
│ --forget-trial                 Delete the local trial marker and exit.                           │
│ --help                         Show this message and exit.                                       │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

The 30-second version

uv tool install kyvvu-cli
kyvvu try

Report excerpt:

kyvvu try — enforcement with no account, no API key and no network

  Policy source   bundled frozen baseline (owasp-agentic-default, eu-ai-act-minimal)
  Policies        15 offered, 15 enforcing (state=enforcing)
                  7 evaluated per step, 8 at agent registration
  Agent           hello-kyvvu   risk=minimal   tools: fetch_user_data, call_llm, send_reply, run_script
  Environment     production   (set KV_ENVIRONMENT=development to change)
  Path            bundled demo path

#  DECISION  RISK  BEHAVIOUR                          VIOLATED POLICY
1  ALLOW     0.00  step.resource GET fetch_user_data  —
2  ALLOW     0.00  step.model call_llm                —
3  WARN      0.75  step.message POST send_reply       AI-generated content must be marked before
                                                      delivery (high, warn); Chatbot disclosure
                                                      gate required before outbound messages
                                                      (high, warn)
4  BLOCK     1.00  step.exec run_script               Code execution requires a preceding gate
                                                      (critical, block)

BLOCKED — run_script was stopped before it ran (risk 1.00)
    Code execution requires a preceding gate (critical)

The same step, after an approval gate
    recorded  step.gate human_approval  output.decision = "pass"
    run_script is now ALLOW (risk 0.00)
    Same step, same policies, different path.
    A gate only counts when it carries a result.

No login, no key, no dashboard, nothing on the network.

What it actually did

Four steps of the agent kyvvu init scaffolds, evaluated against the frozen baseline set bundled in the kyvvu-baseline package, which kyvvu-cli installs:

  1. fetch_user_data — a step.resource GET. In the declared allowlist, so allowed.
  2. call_llm — a step.model. Allowed: the EU AI Act set's Article 50 policies govern delivering model output, not producing it.
  3. send_reply — a step.message POST that delivers the model's answer to a person, with no gate after the model call. It warns (high) twice: the content is not marked as AI-generated (Article 50(2)), and nobody was told they are dealing with an AI (Article 50(1)). That is a compliance finding: a warning is recorded, and the step still runs.
  4. run_script — a step.exec with no approval gate in front of it. The OWASP set's ASI05 policy is critical, so it blocks and the step never runs. That is a security stop. The block is still recorded, as a blocked step, exactly as a real run records it, so a rule that counts attempts sees the same history here as in production.

Then it records a step.gate and re-evaluates step four, which is now allowed. That is the product's thesis in one screen: the decision depends on the path, not on the step. The same run_script call is blocked or allowed depending on what the agent has already done in this task.

The gate has to carry a completed decision. step_requires_gate looks for output.decision == "pass"; a bare step.gate, or one whose decision is "deny", clears nothing.

Enforce your own manifest

Any manifest from Kyvvu/manifests — or one you wrote — runs from disk:

kyvvu try --manifest manifests/security/owasp-agentic-default.yaml
kyvvu try -m owasp.yaml -m my-extras.yaml     # repeatable; both are loaded

An invalid manifest is refused with every problem listed, rather than loading the half that parsed:

Error: Manifest is invalid and was not loaded: my-extras.yaml
  - policies[0] (Model review advisory): invalid severity 'extreme'. Must be one of: critical, high, low, medium.

Run your own behaviour path

Describe your agent's steps in YAML and evaluate them without changing your agent code:

# steps.yaml
agent:
  agent_id: my-agent
  risk_classification: minimal
  allowed_tools: [read_file, run_script]
steps:
  - step_name: read_file
    kind: step.resource
    verb: GET
    input: {path: "notes.md"}
  - step_name: human_approval
    kind: step.gate
    properties:
      kind: human
    output:
      decision: pass
  - step_name: run_script
    kind: step.exec

kyvvu try --steps steps.yaml

allowed_tools matters more than it looks. The OWASP allowlist policy reads the agent's declared tool allowlist and fails closed: with the list empty, every step.resource, step.exec and step.credential blocks on "Tool calls must be in the declared allowlist". When you omit allowed_tools, kyvvu try defaults it to every step name in your path, which is what registration would have declared.

For a blocked simple rule, the verdict line shows the value it checked: the normalised host for domain_allowlist, the field path and value for field_* rules, or the current action kind for current_is. For example, a rejected region prints input.request.region = "forbidden-zone" on the BLOCKED line. Long values are shortened on that line; the policy detail below and --json output retain the untruncated compared value.

A manifest policy whose parameters still contain an RFC 2606 example host (such as example.com) is reported as Warning: Policy '<name>' contains placeholder host '<host>'; replace it before use. in the report, and in the warnings array of --json.

When you pass --manifest, after evaluating the steps kyvvu try warns on stderr for each of your step policies whose field or property filter path resolved on none of the trial steps. The bundled baseline never prints it, with the demo path or with your own --steps: its paths are Kyvvu's, not yours to misspell, and one it reads that your steps do not carry (such as properties.target.trust) is simply not needed by them. The warning names the policy and the path, including paths inside compound rules. For example, input.request.hots warns if the steps only carry input.request.host. A path present on even one step does not warn. The warning is diagnostic: a path may be absent because the trial did not exercise it, and the run's exit status stays the same. It also goes to stderr with --json, leaving stdout as one JSON object.

Machine-readable output

--json emits one object. This command selects its summary fields; here my-manifest.yaml is a copy of the OWASP manifest from the example above:

kyvvu try --manifest my-manifest.yaml --json |
  jq '{policy_source, policy_source_detail, agent_id, policies_offered, policies_enforcing,
       load_state, warnings, environment, counts, blocked}'

{
  "policy_source": "manifest",
  "policy_source_detail": "my-manifest.yaml",
  "agent_id": "hello-kyvvu",
  "policies_offered": 8,
  "policies_enforcing": 8,
  "load_state": "enforcing",
  "warnings": [],
  "environment": "production",
  "counts": {
    "allow": 3,
    "warn": 0,
    "block": 1
  },
  "blocked": true
}

Exit codes

A block is not a failure. kyvvu try exits 0 when it ran and enforced, whether the path was allowed, warned or blocked. A step that was blocked does not change the exit code: a BLOCK row is the thing you asked to see.

Exit code Meaning
0 The policies loaded, at least one is in effect, and the path was evaluated. Any verdict, including BLOCK.
1 The run did not really enforce: the manifest or steps file failed to load, or no policy is in effect (zero policies, for example a valid manifest with policies: []).
2 A usage error (an unknown flag).

Exit 1 means "nothing trustworthy was evaluated", never "a step was blocked". The causes of a load failure are a missing file, a rule_type that is misspelled or unknown, params that do not match their rule, an action kind or verb that is not in the vocabulary, a duplicate policy name, a regex that does not compile, or the retired scope: key. A policy set is loaded whole or refused whole; there is no run that enforces the half that parsed. The command says which on stderr, so --json's stdout stays parseable. A green exit from this command means an agent that is actually governed, not that its path was allowed.

To assert a verdict in CI, parse --json; do not read the exit code. blocked is true when any step was blocked, and decisions has one entry per step:

kyvvu try --manifest my-manifest.yaml --steps steps.yaml --json > verdict.json   # exit 1: did not load, or nothing enforced
jq -e '.blocked == true' verdict.json                      # jq exits 1 when the assertion is false
jq -e '.decisions[-1].action == "block"' verdict.json      # or pin the exact step
jq -r '.decisions[] | "\(.step_name) \(.action)"' verdict.json

With the approved steps.yaml above, both block assertions exit 1. To check that this path is approved, assert blocked == false instead.

action is allow, warn or block. Run the kyvvu try line first and stop the job if it exits non-zero, so a manifest that no longer loads fails the build instead of passing the jq test on a stale verdict.json.

Read load_state, not just policies_enforcing. A run that enforces nothing looks configured while allowing everything, and load_state names which of the four load states you are in: enforcing (at least one policy is in effect), none_configured (the source has no policies), unavailable (nothing is loaded and nothing could be fetched) and stale_cache (a cached set is enforcing because the source stopped answering). A run that enforces nothing announces it once on stderr.

The Environment line, and the environment key in --json, is the environment the run was evaluated as: KV_ENVIRONMENT, else production. Environment-scoped policies behave differently in development and production, so the report says which one it used. When it is not development the line ends with the hint (set KV_ENVIRONMENT=development to change); in a development run there is nothing to change and the line is just the name. kyvvu try reads KV_ENVIRONMENT from the shell and from a .env in the current directory (see where settings are read).

Offline by construction

kyvvu try pins every setting that could redirect the run — the policy source, the credentials and the log sink — in the command itself rather than reading them from the environment. So it stays offline, and enforces the policies it names, even if your shell exports KV_API_KEY, KV_LOG_LOCATION or KV_POLICY_FILE, which is the normal state of anyone who already has an account. The test suite exports all of those and then installs an audit hook that fails on any socket operation, so this is enforced rather than intended.

It is a real KyvvuRunner, though — the same class an instrumented agent uses, with the same policy-load dispatch and the same non-enforcement signalling. The trial is the product, not a demo of it.

The trial marker is opt-in

kyvvu try records nothing by default. --record-trial writes a small JSON file to ~/.kyvvu/try-marker.json:

{
  "schema": 1,
  "trial_id": "<opaque UUID>",
  "first_run_at": "<UTC timestamp>",
  "last_run_at": "<UTC timestamp>",
  "run_count": 1,
  "kyvvu_version": "<installed kyvvu-cli version>",
  "policy_source": "baseline",
  "decisions": {
    "allow": 2,
    "warn": 1,
    "block": 1
  },
  "blocked": true,
  "first_block_at": "<UTC timestamp>"
}

The angle-bracket values stand for the generated UUID, UTC timestamps and installed kyvvu-cli version. Actual timestamps use ISO 8601.

That is the whole feature: the file is written locally and sent nowhere. It carries an opaque random id, timestamps, a run count, the version, which kind of source enforced the run, the decision counts, and when the first block happened. No file paths - not even the manifest's - no step names, no agent content, no hostname, no username.

Nothing reads it to decide what to enforce or what to print, so declining costs you nothing. There is deliberately no environment variable that turns it on by default: an opt-in an installer can pre-enable is not opt-in. kyvvu try --forget-trial deletes it.

When a gate does not help

The approval-gate replay only changes the verdict when the policy that blocked the step is one a gate can clear. A path-taint policy is not: once a sensitive read is on the path, the block stays for the rest of the task. kyvvu try replays the step for real either way, so it says so:

# taint.yaml
name: Taint
policies:
  - name: Block external send after sensitive read
    rule_type: tainted_path_block
    params:
      taint_kind: step.resource
      taint_verb: GET
      taint_property_filter:
        data.classification: pii
      target_kinds: ["step.message"]
      target_verb: POST
    severity: critical
    enforcement_point: step_execution

and a path that reads a PII record and then sends an email:

# taint-steps.yaml
agent:
  agent_id: my-agent
  risk_classification: minimal
  allowed_tools: [read_crm, send_email]
steps:
  - step_name: read_crm
    kind: step.resource
    verb: GET
    properties:
      data: {classification: pii}
  - step_name: send_email
    kind: step.message
    verb: POST

kyvvu try --manifest taint.yaml --steps taint-steps.yaml

send_email is blocked, and the replay ends:

The same step, after an approval gate
    recorded  step.gate human_approval  output.decision = "pass"
    send_email is now BLOCK (risk 1.00)
    The gate did not change the verdict: send_email is still BLOCK.
    Policy Block external send after sensitive read blocks it, and an approval gate is not one of its clearing conditions.

The last line names the policy from the replayed verdict. If the replay only warns, it reads "still WARN" and "warns on it". A run where nothing was blocked has no replay and prints neither.

Next steps