# Cost, privacy, and security

Candidate and reference content is untrusted and may contain personal data,
secrets, retrieved copyrighted material, or prompt-injection instructions.
Default persistence and telemetry are metadata-only.

## Safe defaults

- Store hashes, sizes, stable references, status, usage, cost, and bounded error
  types. Keep raw input/output/context ephemeral.
- Use a separate judge model profile and credentials with the minimum provider
  permissions.
- Give evaluator agents only explicitly allowlisted read-only or
  `evaluation_safe` tools. v0.8.3 does not support evaluator side effects.
- Isolate evaluator memory by namespace and retention. Do not expose target
  operational memory or conversation history.
- Treat candidate text as data. The evaluator system prompt and rubric outrank
  any instruction inside it.
- Cap input tokens, output tokens, retries, tool rounds, time, concurrency, and
  cost per case and online job.
- Keep database DSNs, OTLP headers, and provider keys in named environment
  variables. Diagnostics may report whether they exist, never their values.
- Restrict evaluation databases and evidence stores more tightly than ordinary
  telemetry; evidence can reveal full outputs when content capture is enabled.

`JudgeResult` records usage and cost only when providers report them. A missing
cost is unknown, not zero. Budget breaches become bounded error results.
RAGAS may perform more than one model or embedding operation; test and budget
each selected metric before production use.

Online evaluation should begin with low deterministic sampling. Sampling is
not a privacy control: any selected subject must still satisfy the content,
access, region, encryption, and retention policy.

If explicit redacted-content capture is needed, redact before building the
evaluation context, store the content in an application-owned protected store,
and persist only its `ContentReference`. Never place redacted or raw content in
high-cardinality OpenTelemetry attributes.