---
title: Catalog object model
order: 100
---

This page covers the catalog server's object model. For logging and recording basics, see [Recordings](../logging-and-ingestion/recordings.md). For API details, see the [Catalog SDK reference](https://ref.rerun.io/docs/python/stable/common/catalog/).


## Catalog

We refer to the contents stored in a given catalog server as the _catalog_.
The catalog contains top-level objects called _entries_.

There are currently two types of entries: **tables** and **datasets**.
Each is described in more detail below.

Entries share a few common properties:
- **id**: a globally unique identifier
- **name**: a user-provided name, which must be unique within the catalog

### Renaming a catalog entry

The id of a catalog entry is immutable, but the name can be changed provided it remains unique.
In Python, call `set_name()` on an entry to rename it on the catalog server, for example:

```python
client = rr.catalog.CatalogClient(…)
dataset = client.get_dataset("old_name")
dataset.set_name("new_name")
```

### Structuring datasets

When working with larger amounts of data, it can be useful to organize catalog entries in a directory-like structure.
This can be done by using `.` delimiters in the names.
The screenshot below is an example of a dot-delimited dataset name showing up as a directory tree in the viewer's data source browser:

<picture>
  <img src="https://static.rerun.io/catalog_dirs/1a7f0c9a24f418857c94b46fa7fa39381155a91b/full.png" alt="">
  <source media="(max-width: 480px)" srcset="https://static.rerun.io/catalog_dirs/1a7f0c9a24f418857c94b46fa7fa39381155a91b/480w.png">
  <source media="(max-width: 768px)" srcset="https://static.rerun.io/catalog_dirs/1a7f0c9a24f418857c94b46fa7fa39381155a91b/768w.png">
  <source media="(max-width: 1024px)" srcset="https://static.rerun.io/catalog_dirs/1a7f0c9a24f418857c94b46fa7fa39381155a91b/1024w.png">
  <source media="(max-width: 1200px)" srcset="https://static.rerun.io/catalog_dirs/1a7f0c9a24f418857c94b46fa7fa39381155a91b/1200w.png">
</picture>

## Table entries

Table entries model a single table of data.
They use the [Arrow data model](https://arrow.apache.org/docs/format/Columnar.html), so a table is logically equivalent to an [Arrow table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html).
As a result, tables possess an [Arrow schema](https://arrow.apache.org/docs/python/generated/pyarrow.Schema.html).

Tables support the following mutation operations through the Catalog SDK:
- _append_: add new rows to the table
- _overwrite_: replace the entire table with new data
- _upsert_: replace existing rows (based on an index column) with new data

Thanks to [DataFusion](https://datafusion.apache.org/), tables also support most database operations such as querying, filtering, joining, etc.

## Datasets

Dataset entries model a collection of Rerun data organized in episodes such as recorded runs of a given robotic task.
These episodes within datasets are called _segments_, which are identified by a segment ID.

Segments are added to datasets by the process of _registering_ a [recording](../logging-and-ingestion/recordings.md) (typically stored in some object store such as S3) to the dataset using the Catalog SDK.
The recording ID of the `.rrd` file is used as its segment ID.

Recordings registered to a given segment are organized by layers, identified by a layer name.
By default, the `"base"` layer name is used.
Registering two `.rrd` files with the same recording ID (that is, with the same segment ID) to the same dataset, and using the same layer name, will result in the second `.rrd` overwriting the first.
Additive registration can be achieved by using different layer names for different `.rrd`s with the same recording ID/segment ID.

<div class="d2-diagram">
  <img class="d2-dark" src="https://static.rerun.io/6a9690740962bcda73b78a847c04862ba646461a_d2.svg" alt="">
  <img class="d2-light" src="https://static.rerun.io/97520fa2e99046614f595be752eec7c48c405e51_d2-light.svg" alt="">
</div>

Layers are immutable and can only be overwritten by registering a new `.rrd` file. In other words, datasets support the following mutation operations:
- _create segment_: by registering a `.rrd` with a "new" recording ID
- _append to segment_: by registering a `.rrd` with a matching recording ID to a new layer name
- _overwrite segment layer_: by registering a `.rrd` with a matching recording ID to an existing layer name


### Schema

Datasets are based on the Rerun data model, which consists of a collection of [chunks](../logging-and-ingestion/chunks.md) of Arrow data.
These chunks hold data for various [entities and components](../logging-and-ingestion/entity-component.md) corresponding to various indexes (or [timelines](../logging-and-ingestion/timelines.md)).
A given collection of chunks, say, a dataset segment, defines an Arrow schema.
We refer to this as _schema-on-read_, because the schema proceeds from the data, and not the other way around.
This differs from the table model, where the schema is defined upfront (_schema-on-write_).

In this context, the schema of a dataset is the union of schemas of its segments, which themselves are the union of the schemas of their layers.

<div class="d2-diagram">
  <img class="d2-dark" src="https://static.rerun.io/f98523a3b94a18887af13ad8b45a05f685480a2d_d2.svg" alt="">
  <img class="d2-light" src="https://static.rerun.io/43ed31f276a37305d671c76e9ae4083667ccb988_d2-light.svg" alt="">
</div>

Datasets maintain a minimal level of schema self-consistency.
Registering a `.rrd` whose schema is incompatible with the current dataset schema will result in an error.
In this context, _incompatible_ means that the schema of the new `.rrd` contains a column for the same entity, archetype, and component, but with a different Arrow type.
Such an occurrence is rare, and practically impossible when using standard Rerun archetypes.


### Blueprints

A dataset can be assigned a blueprint.
This is done by registering a `.rbl` blueprint file typically stored in object storage to the dataset.
A dedicated API exists for this in the Catalog SDK: [`DatasetEntry.register_blueprint()`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetEntry.register_blueprint).
In that case, the blueprint is applied to all segments of the dataset when visualized in the Rerun Viewer.