wehub-resource-sync
36 lines
1.8 KiB
Markdown
36 lines
1.8 KiB
Markdown
<!--[metadata]
|
|
title = "Interactive 3D annotation app with Rerun and Gradio"
|
|
tags = ["2D", "3D", "Pinhole camera", "Time series", "SAM", "Segmentation"]
|
|
source = "https://github.com/rerun-io/annotation-example"
|
|
thumbnail = "https://static.rerun.io/square-thumbnail/931d51df4f2cfd84293c5c9212e5e84cecbd79b4/480w.png"
|
|
thumbnail_dimensions = [480, 344]
|
|
-->
|
|
|
|
https://vimeo.com/1078165216?autoplay=1&loop=1&autopause=0&background=1&muted=1&ratio=2802:1790
|
|
|
|
## Background
|
|
|
|
This example showcases how to use Rerun with gradio to generate an annotation app. It consists of two different modes both of which leverage Segment Anything 2.
|
|
|
|
The first mode focuses on tracking an object in a monocular video stream. In addition to segmentation masks, it generates real-time depth maps and point clouds to provide full 3D spatial context, enabling users to visualize, inspect, and annotate the tracked object directly in three-dimensional space.
|
|
|
|
The second mode uses a multiview RGB-D video dataset. By obtaining segmentation masks from two synchronized and calibrated RGB-D views, the app triangulates these 2D masks to reconstruct a precise 3D mask of the chosen object. It then propagates this 3D mask across all camera views and through each frame of the videos, resulting in a fully tracked 3D object trajectory over time.
|
|
|
|
## Run the code
|
|
|
|
This is an external example. Check the [repository](https://github.com/rerun-io/annotation-example) for more information on how to run the code.
|
|
|
|
TLDR: make sure you have the [Pixi package manager](https://pixi.sh/latest/#installation) installed and run
|
|
```
|
|
git clone https://github.com/rerun-io/annotation-example
|
|
cd annotation-example
|
|
pixi run app
|
|
```
|
|
|
|
this will run the single view (monocular) app
|
|
|
|
```
|
|
pixi run multiview-app
|
|
```
|
|
will run the multiview rgb-d app
|