SaibleDocs
Evaluation

Candidate models, on your own prompts

Every request is decided by your policy and recorded. Figures are shown side by side, never as a verdict.

Golden setwritten, uploaded,synthetic, baselineRunup to fourcandidatesDecideyour policy,every requestRecordin the ledger,never the textSide by sidefigures,no verdictPropose pack100 itemsor moreASealorganisationadmin + codeBB is never A↺ when the approved models change, the confirmed set is rerun;the difference waits in the review queue
From a golden set to a sealed validation pack · person A proposes, person B seals
  • Your providers' candidatesEach request is decided by the policy in force, under the application's registration, and served from the model endpoint the record names. A refused item is not sent anywhere. Requests are paid for through your own provider keys.
  • Saible-hosted candidatesAmazon Bedrock models on Saible's account, used only when the setting is on. Your policy cannot decide these requests, so an item is sent redacted where the detector found something, and the policy row reads no policy (Bedrock). They are metered at zero credits.
  • Your golden setKept encrypted under your organisation's key, or under your own KMS key when one is set. The set is listed by its item count and hash. The workbench never returns the text of an item.
Destination of each candidate's requests, and storage of the golden set
Steps
  1. 1On Evaluation, choose the Application and make a golden set: Write, Synthesise, From traffic, Upload JSONL, or import a public baseline.
  2. 2Choose the set and up to four candidates, then click Run.
  3. 3Compare the results side by side: the policy outcome, latency, tokens, cost at list price and the detections.
  4. 4With 100 items or more, click Seal as validation pack. A different organisation admin seals it in Review queue.
  5. 5An organisation admin clicks Confirm as the set a change reruns. Regress now reruns the set, and the difference waits in the review queue.
For your agents

This topic is also documented as Agent Skills that your agents can load: evaluation-workbench, model-lab. Request them from Saible.

Next · Control Plane · ExposureConfiguration exposure of each AI platform →