Guardrails (AI)

Guardrails are controls wrapped around an AI system that restrict what it accepts and what it returns. They can include input filters, output validators, topic limits, permission checks on tool calls, rate limits and human approval steps, and they sit outside the model as well as inside its prompt.

Why it matters

A model on its own can produce harmful, off-topic, private or simply malformed output, and prompts alone are not a reliable barrier. Guardrails add defence in depth: they catch personal data before it leaves, block disallowed requests, enforce a schema, and limit what an agent is allowed to do. They reduce risk but cannot eliminate it.

Example

1. Input check: reject messages containing card numbers
2. Model call
3. Output check: response must be valid JSON matching a schema
4. Tool check: refund tool limited to amounts under a set threshold,
   anything larger waits for a human

Common mistakes

• Putting all safety in the system prompt, which can be overridden by prompt injection.
• Checks that only run on input, so unsafe model output passes through.
• Never testing with adversarial inputs, then assuming the filters work.

Related

• System prompt
• AI alignment
• Function calling (tool use)
• PII

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.