wehub-resource-sync
102 lines
3.5 KiB
Plaintext
102 lines
3.5 KiB
Plaintext
---
|
||
# id: grok
|
||
title: Grok
|
||
sidebar_label: Grok
|
||
---
|
||
|
||
DeepEval allows you to run evals with Grok models via xAI’s SDK, either through the CLI or directly in Python. DeepEval currently validates model names against a supported list—see [Available Grok Models](#available-grok-models).
|
||
|
||
:::info
|
||
To use Grok, you must first install the xAI SDK:
|
||
|
||
```bash
|
||
pip install xai-sdk
|
||
```
|
||
|
||
:::
|
||
|
||
### Command Line
|
||
|
||
To configure Grok through the CLI, run the following command:
|
||
|
||
```bash
|
||
deepeval set-grok --model grok-4.1 \
|
||
--temperature=0
|
||
```
|
||
|
||
The CLI command above sets the specified Grok model as the default llm-judge for all metrics, unless overridden in Python code. To use a different default model provider, you must first unset Grok:
|
||
|
||
```bash
|
||
deepeval unset-grok
|
||
```
|
||
|
||
:::tip[Persisting settings]
|
||
You can persist CLI settings with the optional `--save` flag.
|
||
See [Flags and Configs -> Persisting CLI settings](/docs/evaluation-flags-and-configs#persisting-cli-settings-with---save).
|
||
:::
|
||
|
||
### Python
|
||
|
||
Alternatively, you can specify your model directly in code using `GrokModel` from DeepEval's model collection.
|
||
|
||
<Tabs items={["Python", "ENV"]}>
|
||
<Tab value="Python">
|
||
|
||
```python
|
||
from deepeval.models import GrokModel
|
||
from deepeval.metrics import AnswerRelevancyMetric
|
||
|
||
model = GrokModel(
|
||
model="grok-4.1",
|
||
api_key="your-api-key",
|
||
temperature=0
|
||
)
|
||
|
||
answer_relevancy = AnswerRelevancyMetric(model=model)
|
||
```
|
||
|
||
</Tab>
|
||
<Tab value="ENV">
|
||
|
||
To use any Grok model directly in `deepeval`, set the `USE_GROK_MODEL=1` in your `env` and simply pass the name of your desired model in your metric initialization:
|
||
|
||
```python
|
||
from deepeval.metrics import AnswerRelevancyMetric
|
||
|
||
answer_relevancy = AnswerRelevancyMetric(
|
||
model="grok-4.1",
|
||
)
|
||
```
|
||
|
||
You should also set the other necessary vars like `GROK_API_KEY` to be able to use the Grok models as shown above.
|
||
|
||
</Tab>
|
||
</Tabs>
|
||
|
||
There are **ZERO** mandatory and **SIX** optional parameters when creating a `GrokModel`:
|
||
|
||
- [Optional] `model`: A string specifying the name of the Grok model to use. Defaults to `GROK_MODEL_NAME` if not passed; raises an error at runtime if unset.
|
||
- [Optional] `api_key`: A string specifying your Grok API key for authentication. Defaults to `GROK_API_KEY` if not passed; raises an error at runtime if unset.
|
||
- [Optional] `temperature`: A float specifying the model temperature. Defaults to `TEMPERATURE` if not passed; falls back to `0.0` if unset.
|
||
- [Optional] `cost_per_input_token`: A float specifying the cost for each input token for the provided model. Defaults to `GROK_COST_PER_INPUT_TOKEN` if available in `deepeval`'s model cost registry, else `None`.
|
||
- [Optional] `cost_per_output_token`: A float specifying the cost for each output token for the provided model. Defaults to `GROK_COST_PER_OUTPUT_TOKEN` if available in `deepeval`'s model cost registry, else `None`.
|
||
- [Optional] `generation_kwargs`: A dictionary of additional generation parameters forwarded to the xAI SDK `client.chat.create(...)` call.
|
||
|
||
:::tip
|
||
Any `**kwargs` you would like to use for your model can be passed through the `generation_kwargs` parameter. However, we request you to double check the params supported by the model and your model provider in their [official docs](https://docs.x.ai/docs/guides/function-calling#function-calling-modes).
|
||
:::
|
||
|
||
### Available Grok Models
|
||
|
||
Below is the comprehensive list of available Grok models in DeepEval:
|
||
|
||
- `grok-4.1`
|
||
- `grok-4`
|
||
- `grok-4-heavy`
|
||
- `grok-4-fast`
|
||
- `grok-beta`
|
||
- `grok-3`
|
||
- `grok-2`
|
||
- `grok-2-mini`
|
||
- `grok-code-fast-1`
|