Files
wehub-resource-sync 97e91a83f3
Ruff / Ruff (push) Has been cancelled
Test / Core Tests (push) Has been cancelled
Test / Offline Coverage Tests (Python 3.10) (push) Has been cancelled
Test / Offline Coverage Tests (Python 3.11) (push) Has been cancelled
Test / Offline Coverage Tests (Python 3.12) (push) Has been cancelled
Test / Offline Coverage Tests (Python 3.13) (push) Has been cancelled
Test / Offline Coverage Tests (Python 3.9) (push) Has been cancelled
Test / Full Coverage (Python 3.11) (push) Has been cancelled
Test / Core Provider Tests (OpenAI) (push) Has been cancelled
Test / Core Provider Tests (Anthropic) (push) Has been cancelled
Test / Core Provider Tests (Google) (push) Has been cancelled
Test / Core Provider Tests (Other) (push) Has been cancelled
Test / Anthropic Tests (push) Has been cancelled
Test / Gemini Tests (push) Has been cancelled
Test / Google GenAI Tests (push) Has been cancelled
Test / Vertex AI Tests (push) Has been cancelled
Test / OpenAI Tests (push) Has been cancelled
Test / Writer Tests (push) Has been cancelled
Test / Auto Client Tests (push) Has been cancelled
ty / type-check (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:36:38 +08:00

78 lines
2.0 KiB
Markdown

---
title: Handling Non-Streaming Requests in OpenAI with Usage Tracking
description: Learn how to manage non-streaming requests in OpenAI, track token usage, and handle exceptions with Python.
---
## See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](./from_provider.md) - Detailed client configuration
- [Response Models](./models.md) - Working with Pydantic models
- [Raw Response](./raw_response.md) - Access original LLM responses
The easiest way to get usage for non streaming requests is to access the raw response.
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider("openai/gpt-4.1-mini")
class UserExtract(BaseModel):
name: str
age: int
user, completion = client.create_with_completion(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
print(completion.usage)
"""
CompletionUsage(
completion_tokens=10,
prompt_tokens=79,
total_tokens=89,
completion_tokens_details=CompletionTokensDetails(
accepted_prediction_tokens=None,
audio_tokens=0,
reasoning_tokens=0,
rejected_prediction_tokens=None,
),
prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0),
)
"""
```
You can catch an IncompleteOutputException whenever the context length is exceeded and react accordingly, such as by trimming your prompt by the number of exceeding tokens.
```python
from instructor.core.exceptions import IncompleteOutputException
import instructor
from pydantic import BaseModel
client = instructor.from_provider("openai/gpt-4.1-mini")
class UserExtract(BaseModel):
name: str
age: int
try:
client.create_with_completion(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
except IncompleteOutputException as e:
token_count = e.last_completion.usage.total_tokens # type: ignore
# your logic here
```