567-labs--instructor/docs/prompting/self_criticism/self_calibration.md

description

description
Self Calibration aims to get language models to determine what they know and do not know

We want our language models to be able to output the extent of their confidence in predictions. To do so, we can get language models to evaluate their responses to a given prompt using a technique called Self Calibration ¹

The original paper used a fine-tuned regression head over the language model's final output. However, since we don't have access to the model's final hidden states, we can substitute it for a function call instead to achieve a similar result.

We can ask language models to evaluate their outputs by using the following template

We can implement this using instructor as seen below

import instructor
from pydantic import BaseModel, Field
client = instructor.from_provider("openai/gpt-5-nano")


class SelfCalibration(BaseModel):
    chain_of_thought: str
    is_valid_answer: bool = Field(description="Whether the answer is correct or not")


def evaluate_model_output(original_prompt: str, model_response: str):
    return client.create(
        messages=[
            {
                "role": "user",
                "content": f"""
                Question: {original_prompt}

                {model_response}

                Is this a valid answer to the question?
                Make sure to examine the question
                thoroughly and generate a complete
                reasoning for why the answer is correct
                or not before responding.
                """,
            }
        ],
        response_model=SelfCalibration,
        model="gpt-4o",
    )


if __name__ == "__main__":
    original_prompt = """
    Question: Who was the third president of the
    United States?
    """
    model_response = """
    Here are some brainstormed ideas: James Monroe
    Thomas Jefferson
    Jefferson
    Thomas Jefferson
    George Washington
    """
    response = evaluate_model_output(original_prompt, model_response)
    print(response.model_dump_json(indent=2))
    """
    {
      "chain_of_thought": "Let's examine the question
      carefully: 'Who was the third president of the
      United States?'\n\nThe brainstormed ideas are:
      \n1. James Monroe\n2. Thomas Jefferson\n3.
      Jefferson\n4. Thomas Jefferson\n5. George
      Washington.\n\nTo determine the validity of these
      answers, I'll cross-check with historical
      records.\n\n1. James Monroe was not the third
      president; he was the fifth president.\n2. Thomas
      Jefferson was indeed the third president of the
      United States.\n3. 'Jefferson' is a correct but
      incomplete answer; it lacks the first name, though
      it is commonly understood.\n4. 'Thomas Jefferson'
      is the full name and correct answer.\n5. George
      Washington was the first president, not the
      third.\n\nTherefore, the correct, valid answer to
      the question 'Who was the third president of the
      United States?' is 'Thomas Jefferson,' and this
      answer is correct.",
      "is_valid_answer": true
    }
    """

References

¹: Language Models (Mostly) Know What They Know