Building Type Safe Structured Outputs with Rust and OpenAI

I built a Hacker News AI news summarizer and AI relevancy scorer and, as the type safety and consistency enthusiast I am, I decided to explore structured outputs when interacting with the LLM models.

In short: Structured outputs work by supplying a JSON Schema with your request, ensuring that the reply follows the correct format. This includes specifying what each field should be and how the LLM should fill it. A step forward from telling the LLM to output a JSON object and then praying that it decided to comply.

The news summarizer project may be the subject of a future blog post. For now, let’s dive into what structured outputs are and how they work.

TLDR: Five Easy Steps

This approach assumes we set strict = true as shown in all OpenAI documentation examples.

1. Use Schemars version 1.0.0-alpha.17 to enable with_transform together with RecursiveTransform. See the docs.
2. Tag the type used for the schema with #[serde(deny_unknown_fields)] to comply with strict = true.
3. Utilize the RecursiveTransform to strip format from all fields.
4. Transform the schema to serde_json::Value and send it to OpenAI.
5. Deserialize the response into the schema type from the content of the first message returned by OpenAI.

An example of this implementation can be found in the complementary GitHub repo.

Deep Dive

We need a schema, and we have two options:

1. Using an online schema generator and then manually keeping the types in sync indefinitely.
2. The rusty approach of generating schemas at compile time based on our expected output types.

Naturally, I chose the rusty way.

I find it too tedious to constantly generate and copy external schemas relying only on hope that they’ll work correctly.

I settled on Schemars for my schema generation, though I discovered I needed features from the 1.0 alpha branch to create a compliant schema.

We begin by defining the Chat Completion API request, enabling structured outputs by setting the response_format variable. The messages are our prompts, but I’ll return to that later when we query the OpenAI API.

struct OpenAIChatCompletionQuery {
    // By sending in a response_format we enable structured outputs
    response_format: ResponseFormat,

    model: String,
    messages: Vec<Message>,
    // ...And all other needed parameters for the api call
}

Moving on lets define the ResponseFormat struct. The type is always set to "json_schema" to enable the structured outputs feature.

struct ResponseFormat {
    // Always set to "json_schema" when using structured outputs.
    #[serde(rename = "type")]
    response_type: String,

    // The generated JSON schema.
    json_schema: Schema,
}

Now we supply our schema. First, we need to name it, and then we have a generic serde_json::Value to contain the generated schema.

struct Schema {
    name: String,
    schema: serde_json::Value,
    // In all OpenAI documentation examples strict = true
    strict: bool,
}

Now that we have our correctly defined API request, it’s time to define the schema. We do this by deriving schemars::JsonSchema for our schema.

#[derive(schemars::JsonSchema, serde::Deserialize)]
struct ResponseSchema {
    summary: Vec<String>,
}

Here we encounter our first roadblock. Reading the Schemars documentation, we naively follow the happy path.

let schema = schemars::schema_for!(ResponseSchema);
let schema = serde_json::to_value(schema).unwrap();

Schema {
    name: "my_schema_name".to_string(),
    schema,
    strict: true,
}

And we’re hit with our first error:

Error querying api: HTTP status client error (400 Bad Request)
    for url (https://api.openai.com/v1/chat/completions)
Raw output:
{
  "error": {
    "message":
        "Invalid schema for response_format 'ComplexResponseSchema':
        In context=('properties', 'flair'), 'format' is not permitted.",
    "type": "invalid_request_error",
    "param": "response_format",
    "code": null
  }
}

It turns out that we need to strip the format field from all our types. Fortunately, Schemars has a 1.0-alpha branch allowing us to do that.

let schema = schemars::generate::SchemaSettings::default()
    // The `with_transform` and `RecursiveTransform`
    // comes from the 1.0-alpha branch
    .with_transform(schemars::transform::RecursiveTransform(
        |schema: &mut schemars::Schema| {
            schema.remove("format");
        },
    ))
    .into_generator()
    .into_root_schema_for::<ResponseSchema>();

Let’s try again with the same schema.

Response from OpenAI: Error querying api: HTTP status client error
    (400 Bad Request) for url (https://api.openai.com/v1/chat/completions)
Raw output:
{
  "error": {
    "message":
        "Invalid schema for response_format
        'llm_structured_outputs_tests_SimpleResponseSchema':
        In context=(), 'additionalProperties' is required to be
        supplied and to be false.",
    "type": "invalid_request_error",
    "param": "response_format",
    "code": null
  }
}

Right… since we set strict = true like all documentation examples, we need to deny unknown fields. Lets look into our schema to see what we need to change.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {
    "summary": {
      "description": "Summary of the text in two extremely short paragraphs",
      "items": {
        "type": "string"
      },
      "type": "array"
    }
  },
  "required": ["summary"],
  "title": "SuperSimpleResponseSchema",
  "type": "object"
}

The additionalProperties field is required to be supplied and to be false when strict = true. We can add that our schema by supplying #[serde(deny_unknown_fields)] since Schemars complies with Serde directives when constructing the schema.

#[derive(schemars::JsonSchema, serde::Deserialize)]
#[serde(deny_unknown_fields)]
struct ResponseSchema {
    summary: Vec<String>,
}

Giving us the expected schema.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "additionalProperties": false, // <-- Added by #[serde(deny_unknown_fields)]
  "properties": {
    "summary": {
      "description": "Summary of the text in two extremely short paragraphs",
      "items": {
        "type": "string"
      },
      "type": "array"
    }
  },
  "required": ["summary"],
  "title": "SuperSimpleResponseSchema",
  "type": "object"
}

And it works!

ResponseSchema {
    summary: [
        "Hello! How can I assist you today?",
    ],
}

But we want our answer to be two paragraphs! Let’s add descriptions to our schema.

#[derive(schemars::JsonSchema, serde::Deserialize)]
#[serde(deny_unknown_fields)]
struct ResponseSchema {
    #[schemars(description =
        "Summary of the text in two extremely short paragraphs")
    ]
    summary: Vec<String>,
}

The OpenAI models respond to our guidance, and we get two blathery “paragraphs.”

SimpleResponseSchema {
    summary: [
        "The phrase 'Hello, world!' is commonly used in programming as a
        simple test message to demonstrate a language's syntax.",

        "It symbolizes the first step of learning to code, often used in
        tutorials for beginners.",
    ],
}

There we have it. We now use the schema together with the prompt to generate our structured output.

Let’s explore a few final pieces that make it all come together:

1. Naming the schema
2. Querying the OpenAI API

Naming the Schema

Next, we need to name the schema. Either we let the user provide a name, or we utilize diagnostic functions to automatically generate something adequate. Let’s take the simpler approach.

let name = std::any::type_name::<ResponseSchema>();

Giving us:

let name = "llm_structured_outputs::SimpleResponseSchema"

Seems reasonable! Well… the API informs us that naming things is challenging.

Response from OpenAI: Error querying api: HTTP status client error
    (400 Bad Request) for url (https://api.openai.com/v1/chat/completions)
Raw output:
{
  "error": {
    "message":
        "Invalid 'response_format.json_schema.name': string
        does not match pattern. Expected a string that matches
        the pattern '^[a-zA-Z0-9_-]+$'.",
    "type": "invalid_request_error",
    "param": "response_format.json_schema.name",
    "code": "invalid_value"
  }
}

Of course, there’s a regex pattern to match! This is 2025 after all, and UTF-8 compatibility still presents challenges.

^[a-zA-Z0-9_-]+$

Let’s remove the “special” characters.

let name = std::any::type_name::<ResponseSchema>()
    .replace("::", "_")
    .replace("<", "_")
    .replace(">", "_");

And it works!

I suppose detecting characters not in the regex would also work and might be more robust. But having to deal with a regex once in the error message is enough.

Generic Schema-Safe Querying of OpenAI

Putting it all together, let’s create a generic function for schema generation from any type implementing the schemars::JsonSchema trait.

/// Create an OpenAI compatible schema from a Rust type. Utilizes
/// a diagnostic version of the desired response schema's type name
/// for the schema name sent to OpenAI.
fn get_schema<T: schemars::JsonSchema>() -> Schema {
    let schema = schemars::generate::SchemaSettings::default()
        // The schema generator automatically adds "format"
        // to the items specifying for example int64 or double.
        // OpenAI does not support this.
        .with_transform(schemars::transform::RecursiveTransform(
            |schema: &mut schemars::Schema| {
                schema.remove("format");
            },
        ))
        .into_generator()
        .into_root_schema_for::<T>();
    let schema = serde_json::to_value(schema).unwrap();

    // We need a name for the schema. Get the type name and ensure it
    // is compatible with OpenAI as per the regex "^[a-zA-Z0-9_-]+$"
    // Turning `type_name::<Option<String>>` => `typename__Option_String_`
    let name = std::any::type_name::<T>()
        .replace("::", "_")
        .replace("<", "_")
        .replace(">", "_");

    Schema {
        name,
        schema,
        strict: true,
    }
}

To query an LLM model, we of course need to send our prompts and specify who they are from.

#[derive(serde::Serialize)]
pub struct Message {
    role: Role,
    content: String,
}

#[derive(serde::Serialize)]
#[serde(rename_all = "lowercase")]
enum Role {
    Assistant,
    Developer,
    User,
}

We also need to create some types to deserialize the output.

#[derive(serde::Deserialize)]
struct OpenAIChatCompletionResponse {
    choices: Vec<Choice>,

    // There are a bunch of extra fields in the response
    // that we don't care about. See the OpenAI API docs.
}

#[derive(serde::Deserialize)]
struct Choice {
    message: ResponseMessage,
}

#[derive(serde::Deserialize)]
struct ResponseMessage {
    content: String,
}

Finally, we can utilize the generated schema to query the OpenAI API and directly deserialize the output.

/// Query OpenAI with a message and a schema defined by the generic
/// type T. The schema is used to enforce structured output from the
/// OpenAI API and parse the response into said Rust type.
async fn query_openai<T>(messages: Vec<Message>) -> T
where
    T: for<'a> serde::Deserialize<'a> + schemars::JsonSchema,
{
    let schema = get_schema::<T>();
    let response = query_openai_inner(messages, schema)
        .await
        .expect("Response from OpenAI");

    // The response is inside a string field, so we first need to parse
    // the entire response and then pick out the content field to parse
    // into our structured output type.
    serde_json::from_str(
        &response
            .choices
            .get(0)
            .expect("Response from OpenAI")
            .message
            .content,
    )
    // In a the real world we would have to handle the variety of
    // stop messages and whatever else the LLM can output
    // here instead of crashing when the LLM refuses to answer or
    // does something unexpected.
    .expect("Correctly structured parseable response")
}

/// Query the OpenAI API with a message and a schema.
async fn query_openai_inner(
    messages: Vec<Message>,
    schema: Schema,
) -> anyhow::Result<OpenAIChatCompletionResponse> {
    let query = OpenAIChatCompletionQuery {
        model: CONFIG.model.clone(), // E.g. "o3-mini-2025-01-31"
        messages,
        response_format: ResponseFormat {
            // Always set to json_schema when using structured outputs
            response_type: "json_schema".to_string(),
            json_schema: schema,
        },
    };

    let response = CLIENT
        .post("https://api.openai.com/v1/chat/completions")
        .bearer_auth(CONFIG.api_key.clone())
        .json(&query)
        .send()
        .await?;

    if let Err(e) = response.error_for_status_ref() {
        anyhow::bail!(
            "Error querying api: {e}\nRaw output:\n{}",
            response.text().await.unwrap()
        );
    }

    Ok(response.json().await?)
}

Let’s try it out!

let response: ResponseSchema = query_openai(
    vec![
        Message {
            role: Role::User,
            content: "Hello, world!".to_string(),
        },
    ],
).await;

Giving us the expected output:

ResponseSchema {
    summary: [
        "A simple greeting to the world.",
        "Expresses a friendly and welcoming tone.",
    ]
}

And we’re done! We can now query the OpenAI API with a schema and deserialize the response into our desired type.

This approach allows us to add the complexity we need, as long as we conform to the requirements of structured outputs. Be it lists of objects, enums or even nested schemas.

The full code is available on GitHub.