# Output files

The [`trajectories/`](https://github.com/SWE-agent/SWE-agent/tree/main/trajectories) folder is the default location that experiment results (for example invocations of [`sweagent run`](cl_tutorial.md)) will be written to.

## Trajectories

The main output file is `<instance_id>.traj`, which is a `.json` formatted file containing the (thought, action, observation) turns generated by SWE-agent towards solving `<instance_id>`.

Under the `trajectory` key, you can see information for every step of the agent. It looks something like this:

```json
{
  # This is the output of the LM
  "response": "We are indeed seeing the same output as the issue. The issue suggests that we should look at line 1474 of the `fields.py`...",
  # We then parse it into thoughts and actions
  "thought": "We are indeed seeing the same output as the issue. The issue suggests that we should look at line 1474 of the `fields.py`..."
  "action": "ls -F\n",
  # And execute the action, resulting in the output
  "observation": "AUTHORS.rst\nCHANGELOG.rst\nCODE_OF_CONDUCT.md...",
  # In addition, after the action was executed, state can be extracted from the environment
  "state": "{\"open_file\": \"/marshmallow-code__marshmallow/reproduce.py\", \"working_dir\": \"/marshmallow-code__marshmallow\"}\n",
  # For debugging, we also keep all messages that were shown to the LM
  "query": [{"role": "system", "content": "You are a helpful assistant ..."}, ...]
},
```

!!! warning "Query and message field"
    Prior to SWE-agent 1.1.0, we had a `message` field which corresponded (approximately) to the input
    for the LM for the _next_ step. This was replaced with `query`, which shows the exact input
    at the current step.

Here's a full example:

<details>
<summary>Example trajectory</summary>

Note: This trajectory is from SWE-agent v0.7.0. It might look different in newer versions.

```json
--8<-- "trajectories/demonstrations/replay__marshmallow-code__marshmallow-1867__default__t-0.20__p-0.95__c-2.00__install-1___install_from_source/marshmallow-code__marshmallow-1867.traj"
```
</details>

!!! tip "Viewing trajectories"
    We provide a [trajectory viewer](inspector.md) for an easy viewing of trajectories.
    If you want to view the json, we recommend [jsoneditoronline](https://jsoneditoronline.org/),
    because of the way it handles long lines.

Trajectories can also be turned into custom demonstrations for SWE-agent ([more information](../config/demonstrations.md)).

## Other output files

Together with the trajectory, SWE-agent will also generate the following files per instance:

* `config.yaml`: A yaml file containing the configuration used for the experiment. You can directly repeat the run of this
  exact instance by running `sweagent run --config config.yaml`.
* `*.log`: Log files at different levels

At a high level, the experiments folder is organized in the following manner:

```
trajectories:
    user 1:
        experiment 1:
            run_batch.config.yaml
            preds.json
            run_batch.debug.log
            run_batch.info.log
            run_batch.warning.log
            run_batch_exit_statuses.yaml
            instance_1:
                instance_1.traj
                instance_1.config.yaml
                instance_1.debug.log
                instance_1.info.log
                instance_1.warning.log
            instance 2:
                ...
        experiment 2:
            ...
    user 2:
        ...
```

Where every experiment follows the pattern `trajectories/<user name>/<experiment name>`. The `<user name>` is automatically inferred from your system, and the `experiment name` is inferred from the model names and other settings to the run command.


Here are a few more files that are generated with `sweagent run-batch`:

* `run_batch.config.yaml`: A yaml file containing the configuration used for the experiment. You can directly repeat the run of this
  exact experiment by running `sweagent run-batch --config run_batch.config.yaml`.
* `preds.json`: All predictions (i.e., patches produced by the agent) from all instances of the experiment.
  If this file goes out of date, you can regenerate it by running `sweagent merge-preds`.
* `run_batch.*.log`: Log files at different levels.
* `run_batch_exit_statuses.yaml`: A yaml file containing the exit statuses of all instances in the experiment (for this run of `sweagent run-batch`).


!!! tip
    * Evaluation is not completed by `sweagent run-batch`, it is a separate step (see [batch mode](batch_mode.md))
    * `preds.json` can be referenced directly into SWE-bench to run evaluation
    * Trajectories can be turned into custom demonstrations for SWE-agent ([more information](../config/demonstrations.md)).

{% include-markdown "../_footer.md" %}