promptfoo--promptfoo/examples/provider-fireworks/promptfooconfig.yaml

# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: 'Fireworks AI Provider Demo'

prompts:
  - 'In one sentence, summarise: {{passage}}'

# These are Fireworks serverless models, so a `FIREWORKS_API_KEY` is all you
# need to run the example. The current serverless catalogue lives at
# https://fireworks.ai/models?deployment=serverless — swap in any other
# serverless id if these rotate out. Fireworks's flagship models are reasoning
# models, so `max_tokens` is set high enough to leave room for the hidden
# reasoning tokens plus the visible answer.
providers:
  - id: fireworks:accounts/fireworks/models/gpt-oss-120b
    label: 'GPT-OSS 120B'
    config:
      temperature: 0.2
      max_tokens: 1024
  - id: fireworks:accounts/fireworks/models/deepseek-v4-pro
    label: 'DeepSeek V4 Pro'
    config:
      temperature: 0.2
      max_tokens: 1024
  - id: fireworks:accounts/fireworks/models/kimi-k2p6
    label: 'Kimi K2.6'
    config:
      temperature: 0.2
      max_tokens: 1024

# Fireworks also serves embedding models on the same key. Here a `similar`
# assertion is graded by the serverless Qwen3 embedding model via the
# `fireworks:embedding:` prefix.
defaultTest:
  options:
    provider:
      embedding:
        id: fireworks:embedding:accounts/fireworks/models/qwen3-embedding-8b

tests:
  - vars:
      passage: |
        On the surface, the Antikythera mechanism was a corroded lump of bronze pulled from a Roman-era
        shipwreck off Greece in 1901. Decades of x-ray and tomographic analysis revealed it to be an
        intricate analogue computer, with dozens of geared dials that tracked the Sun and Moon, predicted
        eclipses, and modelled the irregular motion of the lunar orbit using an epicyclic gear train.
    assert:
      - type: icontains
        value: Antikythera
      - type: icontains-any
        value: [mechanism, computer, gears, astronomical]
      - type: similar
        value: an ancient Greek geared device that predicted astronomical events and eclipses
        threshold: 0.4
  - vars:
      passage: |
        Edsger Dijkstra argued that programs should be derived from their specifications through a
        chain of small, verifiable steps. He took particular issue with the unrestricted use of GOTO,
        which he believed encouraged jumping around in code in ways that made formal reasoning about
        program correctness much harder than it needed to be.
    assert:
      - type: icontains-any
        value: [Dijkstra, GOTO, correctness, specification]