Guardrails for Amazon Bedrock

Amazon Web Services

Config-Driven Model Onboarding

Context

Bedrock provides LLM inference APIs across multiple models. Guardrails scans user queries and model responses for unwanted content, masking or blocking it with replacement messages.

Guardrails model selector showing supported models across modalities

Guardrails model selector across modalities

Problem

Before: hardcoded if/else parsing per model, 3 weeks per onboarding

Guardrails needed to parse and write to LLM response text — but each model had a different schema. The existing approach was hardcoded if/else parsing logic per model, embedded in a Bedrock team-owned package. Every code change required their review and deployment cycle. Onboarding a new model with a schema change took an estimated 3 weeks.

Solution

After: two-level routing — provider match then schema fingerprint, collapsing 100+ models into handful of configs

Decoupled model-specific logic into config files loaded at runtime — but a naive 1:1 model-to-config mapping would still require 100+ configs at scale. Designed a two-level routing instead:

Provider matching — route by provider (Anthropic, Meta, Cohere, etc.)
Schema fingerprint — detect response structure at runtime from the LLM response (e.g. messages field → v2, text field → v1) to select the correct parsing config within that provider

This abstraction collapses hundreds of model IDs into a handful of provider × schema combinations, making bulk onboarding trivial.

Result

Onboarding speed: 3 weeks per model → config-only deployment

Onboarding time dropped from 3 weeks to config deployment. The two-level routing collapsed 100+ model IDs into ~8 provider × schema configs. Shipped 100+ Jumpstart catalog models for Guardrails at re:Invent 2024.