wehub/rerun-io--rerun
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:05:14 +08:00
commit 2a547be7fe
7904 changed files with 1000926 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/python/structure_from_motion/.gitignore
+1
View File
@@ -0,0 +1 @@
dataset/**
examples/python/structure_from_motion/README.md
+123
View File
@@ -0,0 +1,123 @@
<!--[metadata]
title = "Structure from motion"
tags = ["2D", "3D", "COLMAP", "Pinhole camera", "Time series"]
thumbnail = "https://static.rerun.io/structure-from-motion/af24e5e8961f46a9c10399dbc31b6611eea563b4/480w.png"
thumbnail_dimensions = [480, 480]
channel = "main"
include_in_manifest = true
build_args = ["--dataset=colmap_fiat", "--resize=800x600"]
-->
Visualize a sparse reconstruction by [COLMAP](https://colmap.github.io/index.html), a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline with a graphical and command-line interface
<picture data-inline-viewer="examples/structure_from_motion">
<source media="(max-width: 480px)" srcset="https://static.rerun.io/structure_from_motion/b17f8824291fa1102a4dc2184d13c91f92d2279c/480w.png">
<source media="(max-width: 768px)" srcset="https://static.rerun.io/structure_from_motion/b17f8824291fa1102a4dc2184d13c91f92d2279c/768w.png">
<source media="(max-width: 1024px)" srcset="https://static.rerun.io/structure_from_motion/b17f8824291fa1102a4dc2184d13c91f92d2279c/1024w.png">
<source media="(max-width: 1200px)" srcset="https://static.rerun.io/structure_from_motion/b17f8824291fa1102a4dc2184d13c91f92d2279c/1200w.png">
<img src="https://static.rerun.io/structure_from_motion/b17f8824291fa1102a4dc2184d13c91f92d2279c/full.png" alt="Structure From Motion example screenshot">
</picture>
## Background
COLMAP is a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline.
In this example, a short video clip has been processed offline using the COLMAP pipeline.
The processed data was then visualized using Rerun, which allowed for the visualization of individual camera frames, estimation of camera poses, and creation of point clouds over time.
By using COLMAP in combination with Rerun, a highly-detailed reconstruction of the scene depicted in the video was generated.
## Used Rerun types
[`Points2D`](https://www.rerun.io/docs/reference/types/archetypes/points2d), [`Points3D`](https://www.rerun.io/docs/reference/types/archetypes/points3d), [`Transform3D`](https://www.rerun.io/docs/reference/types/archetypes/transform3d), [`SeriesLines`](https://www.rerun.io/docs/reference/types/archetypes/series_lines), [`Scalars`](https://www.rerun.io/docs/reference/types/archetypes/scalars), [`Pinhole`](https://www.rerun.io/docs/reference/types/archetypes/pinhole), [`Image`](https://www.rerun.io/docs/reference/types/archetypes/image), [`TextDocument`](https://www.rerun.io/docs/reference/types/archetypes/text_document)
## Logging and visualizing with Rerun
The visualizations in this example were created with the following Rerun code:
### Timelines
All data logged using Rerun in the following sections is connected to a specific frame.
Rerun assigns a frame id to each piece of logged data, and these frame ids are associated with a [`timeline`](https://www.rerun.io/docs/concepts/logging-and-ingestion/timelines).
```python
rr.set_time("frame", sequence=frame_idx)
```
### Images
The images are logged through the [`Image`](https://www.rerun.io/docs/reference/types/archetypes/image) to the `camera/image` entity.
```python
rr.log("camera/image", rr.Image(rgb).compress(jpeg_quality=75))
```
### Cameras
The images stem from pinhole cameras located in the 3D world. To visualize the images in 3D, the pinhole projection has
to be logged and the camera pose (this is often referred to as the intrinsics and extrinsics of the camera,
respectively).
The [`Pinhole`](https://www.rerun.io/docs/reference/types/archetypes/pinhole) is logged to the `camera/image` entity and defines the intrinsics of the camera.
This defines how to go from the 3D camera frame to the 2D image plane. The extrinsics are logged as an
[`Transform3D`](https://www.rerun.io/docs/reference/types/archetypes/transform3d) to the `camera` entity.
```python
rr.log(
"camera",
rr.Transform3D(
translation=image.tvec, rotation=rr.Quaternion(xyzw=quat_xyzw), relation=rr.TransformRelation.ChildFromParent
),
)
```
```python
rr.log(
"camera/image",
rr.Pinhole(
resolution=[camera.width, camera.height],
focal_length=camera.params[:2],
principal_point=camera.params[2:],
),
)
```
### Reprojection error
For each image a [`Scalars`](https://www.rerun.io/docs/reference/types/archetypes/scalars) archetype containing the average reprojection error of the keypoints is logged to the
`plot/avg_reproj_err` entity.
```python
rr.log("plot/avg_reproj_err", rr.Scalars(np.mean(point_errors)))
```
### 2D points
The 2D image points that are used to triangulate the 3D points are visualized by logging as [`Points2D`](https://www.rerun.io/docs/reference/types/archetypes/points2d)
to the `camera/image/keypoints` entity. Note that these keypoints are a child of the
`camera/image` entity, since the points should show in the image plane.
```python
rr.log("camera/image/keypoints", rr.Points2D(visible_xys, colors=[34, 138, 167]))
```
### 3D points
The colored 3D points were added to the visualization by logging the [`Points3D`](https://www.rerun.io/docs/reference/types/archetypes/points3d) archetype to the `points` entity.
```python
rr.log("points", rr.Points3D(points, colors=point_colors), rr.AnyValues(error=point_errors))
```
## Run the code
To run this example, make sure you have the Rerun repository checked out and the latest SDK installed:
```bash
pip install --upgrade rerun-sdk # install the latest Rerun SDK
git clone git@github.com:rerun-io/rerun.git # Clone the repository
cd rerun
git checkout latest # Check out the commit matching the latest SDK release
```
Install the necessary libraries specified in the requirements file:
```bash
pip install -e examples/python/structure_from_motion
```
To experiment with the provided example, simply execute the main Python script:
```bash
python -m structure_from_motion # run the example
```
If you wish to customize it, explore additional features, or save it use the CLI with the `--help` option for guidance:
```bash
python -m structure_from_motion --help
```
examples/python/structure_from_motion/pyproject.toml
+13
View File
@@ -0,0 +1,13 @@
[project]
name = "structure_from_motion"
version = "0.1.0"
readme = "README.md"
dependencies = ["opencv-python>4.6", "numpy", "requests>=2.31,<3", "rerun-sdk", "tqdm"]
[project.scripts]
structure_from_motion = "structure_from_motion.__main__:main"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
examples/python/structure_from_motion/structure_from_motion/__init__.py
View File
examples/python/structure_from_motion/structure_from_motion/__main__.py
Executable
+216
View File
@@ -0,0 +1,216 @@
#!/usr/bin/env python3
"""Example of using Rerun to log and visualize the output of COLMAP's sparse reconstruction."""
from __future__ import annotations
import io
import os
import re
import zipfile
from argparse import ArgumentParser
from pathlib import Path
from typing import Final
import cv2
import numpy as np
import numpy.typing as npt
import requests
from tqdm import tqdm
import rerun as rr # pip install rerun-sdk
import rerun.blueprint as rrb
from .read_write_model import Camera, read_model # type: ignore[attr-defined]
DESCRIPTION = """
# Sparse reconstruction by COLMAP
This example was generated from the output of a sparse reconstruction done with COLMAP.
[COLMAP](https://colmap.github.io/index.html) is a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo
(MVS) pipeline with a graphical and command-line interface.
In this example a short video clip has been processed offline by the COLMAP pipeline, and we use Rerun to visualize the
individual camera frames, estimated camera poses, and resulting point clouds over time.
The full source code for this example is available
[on GitHub](https://github.com/rerun-io/rerun/blob/latest/examples/python/structure_from_motion).
""".strip()
DATASET_DIR: Final = Path(__file__).parent.parent / "dataset"
DATASET_URL_BASE: Final = "https://storage.googleapis.com/rerun-example-datasets/colmap"
# When dataset filtering is turned on, drop views with less than this many valid points.
FILTER_MIN_VISIBLE: Final = 500
def scale_camera(camera: Camera, resize: tuple[int, int]) -> tuple[Camera, npt.NDArray[np.float64]]:
"""Scale the camera intrinsics to match the resized image."""
assert camera.model == "PINHOLE"
new_width = resize[0]
new_height = resize[1]
scale_factor = np.array([new_width / camera.width, new_height / camera.height])
# For PINHOLE camera model, params are: [focal_length_x, focal_length_y, principal_point_x, principal_point_y]
new_params = np.append(camera.params[:2] * scale_factor, camera.params[2:] * scale_factor)
return (Camera(camera.id, camera.model, new_width, new_height, new_params), scale_factor)
def get_downloaded_dataset_path(dataset_name: str) -> Path:
dataset_url = f"{DATASET_URL_BASE}/{dataset_name}.zip"
recording_dir = DATASET_DIR / dataset_name
if recording_dir.exists():
return recording_dir
os.makedirs(DATASET_DIR, exist_ok=True)
zip_file = download_with_progress(dataset_url)
with zipfile.ZipFile(zip_file) as zip_ref:
progress = tqdm(zip_ref.infolist(), "Extracting dataset", total=len(zip_ref.infolist()), unit="files")
for file in progress:
zip_ref.extract(file, DATASET_DIR)
progress.update()
return recording_dir
def download_with_progress(url: str) -> io.BytesIO:
"""Download file with tqdm progress bar."""
chunk_size = 1024 * 1024
resp = requests.get(url, stream=True)
total_size = int(resp.headers.get("content-length", 0))
with tqdm(desc="Downloading dataset", total=total_size, unit="iB", unit_scale=True, unit_divisor=1024) as progress:
zip_file = io.BytesIO()
for data in resp.iter_content(chunk_size):
zip_file.write(data)
progress.update(len(data))
zip_file.seek(0)
return zip_file
def read_and_log_sparse_reconstruction(dataset_path: Path, filter_output: bool, resize: tuple[int, int] | None) -> None:
print("Reading sparse COLMAP reconstruction")
cameras, images, points3D = read_model(dataset_path / "sparse", ext=".bin")
print("Building visualization by logging to Rerun")
if filter_output:
# Filter out noisy points
points3D = {id: point for id, point in points3D.items() if point.rgb.any() and len(point.image_ids) > 4}
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), static=True)
rr.log("/", rr.ViewCoordinates.RIGHT_HAND_Y_DOWN, static=True)
rr.log("plot/avg_reproj_err", rr.SeriesLines(colors=[240, 45, 58]), static=True)
# Iterate through images (video frames) logging data related to each frame.
for image in sorted(images.values(), key=lambda im: im.name):
image_file = dataset_path / "images" / image.name
if not os.path.exists(image_file):
continue
# COLMAP sets image ids that don't match the original video frame
idx_match = re.search(r"\d+", image.name)
assert idx_match is not None
frame_idx = int(idx_match.group(0))
quat_xyzw = image.qvec[[1, 2, 3, 0]] # COLMAP uses wxyz quaternions
camera = cameras[image.camera_id]
if resize:
camera, scale_factor = scale_camera(camera, resize)
else:
scale_factor = np.array([1.0, 1.0])
visible = [id != -1 and points3D.get(id) is not None for id in image.point3D_ids]
visible_ids = image.point3D_ids[visible]
if filter_output and len(visible_ids) < FILTER_MIN_VISIBLE:
continue
visible_xyzs = [points3D[id] for id in visible_ids]
visible_xys = image.xys[visible]
if resize:
visible_xys *= scale_factor
rr.set_time("frame", sequence=frame_idx)
points = [point.xyz for point in visible_xyzs]
point_colors = [point.rgb for point in visible_xyzs]
point_errors = [point.error for point in visible_xyzs]
rr.log("plot/avg_reproj_err", rr.Scalars(np.mean(point_errors)))
rr.log("points", rr.Points3D(points, colors=point_colors), rr.AnyValues(error=point_errors))
# COLMAP's camera transform is "camera from world"
rr.log(
"camera",
rr.Transform3D(
translation=image.tvec,
rotation=rr.Quaternion(xyzw=quat_xyzw),
relation=rr.TransformRelation.ChildFromParent,
),
)
rr.log("camera", rr.ViewCoordinates.RDF, static=True) # X=Right, Y=Down, Z=Forward
# Log camera intrinsics
assert camera.model == "PINHOLE"
rr.log(
"camera/image",
rr.Pinhole(
resolution=[camera.width, camera.height],
focal_length=camera.params[:2],
principal_point=camera.params[2:],
),
)
if resize:
bgr = cv2.imread(str(image_file))
bgr = cv2.resize(bgr, resize)
rr.log("camera/image", rr.Image(bgr, color_model="BGR").compress(jpeg_quality=75))
else:
rr.log("camera/image", rr.EncodedImage(path=dataset_path / "images" / image.name))
rr.log("camera/image/keypoints", rr.Points2D(visible_xys, colors=[34, 138, 167]))
def main() -> None:
parser = ArgumentParser(description="Visualize the output of COLMAP's sparse reconstruction on a video.")
parser.add_argument("--unfiltered", action="store_true", help="If set, we don't filter away any noisy data.")
parser.add_argument(
"--dataset",
action="store",
default="colmap_rusty_car",
choices=["colmap_rusty_car", "colmap_fiat"],
help="Which dataset to download",
)
parser.add_argument("--resize", action="store", help="Target resolution to resize images")
rr.script_add_args(parser)
args = parser.parse_args()
if args.resize:
args.resize = tuple(int(x) for x in args.resize.split("x"))
blueprint = rrb.Vertical(
rrb.Spatial3DView(
name="3D",
origin="/",
line_grid=False, # There's no clearly defined ground plane.
),
rrb.Horizontal(
rrb.TextDocumentView(name="README", origin="/description"),
rrb.Spatial2DView(name="Camera", origin="/camera/image"),
rrb.TimeSeriesView(origin="/plot"),
),
row_shares=[3, 2],
)
rr.script_setup(args, "rerun_example_structure_from_motion", default_blueprint=blueprint)
dataset_path = get_downloaded_dataset_path(args.dataset)
read_and_log_sparse_reconstruction(dataset_path, filter_output=not args.unfiltered, resize=args.resize)
rr.script_teardown(args)
if __name__ == "__main__":
main()
examples/python/structure_from_motion/structure_from_motion/read_write_model.py
+510
View File
@@ -0,0 +1,510 @@
# This file is adapted from
# https://github.com/colmap/colmap/blob/bf3e19140f491c3042bfd85b7192ef7d249808ec/scripts/python/read_write_model.py
# Copyright (c) 2023, ETH Zurich and UNC Chapel Hill.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# * Neither the name of ETH Zurich and UNC Chapel Hill nor the names of
# its contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# Author: Johannes L. Schoenberger (jsch-at-demuc-dot-de)
# type: ignore
from __future__ import annotations
import argparse
import collections
import os
import struct
from pathlib import Path
from typing import Mapping
import numpy as np
CameraModel = collections.namedtuple("CameraModel", ["model_id", "model_name", "num_params"])
Camera = collections.namedtuple("Camera", ["id", "model", "width", "height", "params"])
BaseImage = collections.namedtuple("Image", ["id", "qvec", "tvec", "camera_id", "name", "xys", "point3D_ids"])
Point3D = collections.namedtuple("Point3D", ["id", "xyz", "rgb", "error", "image_ids", "point2D_idxs"])
class Image(BaseImage):
def qvec2rotmat(self):
return qvec2rotmat(self.qvec)
CAMERA_MODELS = {
CameraModel(model_id=0, model_name="SIMPLE_PINHOLE", num_params=3),
CameraModel(model_id=1, model_name="PINHOLE", num_params=4),
CameraModel(model_id=2, model_name="SIMPLE_RADIAL", num_params=4),
CameraModel(model_id=3, model_name="RADIAL", num_params=5),
CameraModel(model_id=4, model_name="OPENCV", num_params=8),
CameraModel(model_id=5, model_name="OPENCV_FISHEYE", num_params=8),
CameraModel(model_id=6, model_name="FULL_OPENCV", num_params=12),
CameraModel(model_id=7, model_name="FOV", num_params=5),
CameraModel(model_id=8, model_name="SIMPLE_RADIAL_FISHEYE", num_params=4),
CameraModel(model_id=9, model_name="RADIAL_FISHEYE", num_params=5),
CameraModel(model_id=10, model_name="THIN_PRISM_FISHEYE", num_params=12),
}
CAMERA_MODEL_IDS = {camera_model.model_id: camera_model for camera_model in CAMERA_MODELS}
CAMERA_MODEL_NAMES = {camera_model.model_name: camera_model for camera_model in CAMERA_MODELS}
def read_next_bytes(fid, num_bytes, format_char_sequence, endian_character="<"):
"""
Read and unpack the next bytes from a binary file.
:param fid:
:param num_bytes: Sum of combination of {2, 4, 8}, e.g. 2, 6, 16, 30, etc.
:param format_char_sequence: List of {c, e, f, d, h, H, i, I, l, L, q, Q}.
:param endian_character: Any of {@, =, <, >, !}
:return: Tuple of read and unpacked values.
"""
data = fid.read(num_bytes)
return struct.unpack(endian_character + format_char_sequence, data)
def write_next_bytes(fid, data, format_char_sequence, endian_character="<"):
"""
Pack and write to a binary file.
:param fid:
:param data: data to send, if multiple elements are sent at the same time,
they should be encapsuled either in a list or a tuple
:param format_char_sequence: List of {c, e, f, d, h, H, i, I, l, L, q, Q}.
should be the same length as the data list or tuple
:param endian_character: Any of {@, =, <, >, !}
"""
if isinstance(data, (list, tuple)):
bytes = struct.pack(endian_character + format_char_sequence, *data)
else:
bytes = struct.pack(endian_character + format_char_sequence, data)
fid.write(bytes)
def read_cameras_text(path: Path):
"""
see: src/base/reconstruction.cc
void Reconstruction::WriteCamerasText(const std::string& path)
void Reconstruction::ReadCamerasText(const std::string& path)
"""
cameras = {}
with open(path) as fid:
while True:
line = fid.readline()
if not line:
break
line = line.strip()
if len(line) > 0 and line[0] != "#":
elems = line.split()
camera_id = int(elems[0])
model = elems[1]
width = int(elems[2])
height = int(elems[3])
params = np.array(tuple(map(float, elems[4:])))
cameras[camera_id] = Camera(id=camera_id, model=model, width=width, height=height, params=params)
return cameras
def read_cameras_binary(path_to_model_file: Path) -> Mapping[int, Camera]:
"""
see: src/base/reconstruction.cc
void Reconstruction::WriteCamerasBinary(const std::string& path)
void Reconstruction::ReadCamerasBinary(const std::string& path)
"""
cameras = {}
with path_to_model_file.open("rb") as fid:
num_cameras = read_next_bytes(fid, 8, "Q")[0]
for _ in range(num_cameras):
camera_properties = read_next_bytes(fid, num_bytes=24, format_char_sequence="iiQQ")
camera_id = camera_properties[0]
model_id = camera_properties[1]
model_name = CAMERA_MODEL_IDS[camera_properties[1]].model_name
width = camera_properties[2]
height = camera_properties[3]
num_params = CAMERA_MODEL_IDS[model_id].num_params
params = read_next_bytes(fid, num_bytes=8 * num_params, format_char_sequence="d" * num_params)
cameras[camera_id] = Camera(
id=camera_id, model=model_name, width=width, height=height, params=np.array(params)
)
assert len(cameras) == num_cameras
return cameras
def write_cameras_text(cameras, path):
"""
see: src/base/reconstruction.cc
void Reconstruction::WriteCamerasText(const std::string& path)
void Reconstruction::ReadCamerasText(const std::string& path)
"""
HEADER = (
"# Camera list with one line of data per camera:\n"
+ "# CAMERA_ID, MODEL, WIDTH, HEIGHT, PARAMS[]\n"
+ f"# Number of cameras: {len(cameras)}\n"
)
with open(path, "w") as fid:
fid.write(HEADER)
for _, cam in cameras.items():
to_write = [cam.id, cam.model, cam.width, cam.height, *cam.params]
line = " ".join([str(elem) for elem in to_write])
fid.write(line + "\n")
def write_cameras_binary(cameras, path_to_model_file):
"""
see: src/base/reconstruction.cc
void Reconstruction::WriteCamerasBinary(const std::string& path)
void Reconstruction::ReadCamerasBinary(const std::string& path)
"""
with open(path_to_model_file, "wb") as fid:
write_next_bytes(fid, len(cameras), "Q")
for _, cam in cameras.items():
model_id = CAMERA_MODEL_NAMES[cam.model].model_id
camera_properties = [cam.id, model_id, cam.width, cam.height]
write_next_bytes(fid, camera_properties, "iiQQ")
for p in cam.params:
write_next_bytes(fid, float(p), "d")
return cameras
def read_images_text(path: Path):
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadImagesText(const std::string& path)
void Reconstruction::WriteImagesText(const std::string& path)
"""
images = {}
with open(path) as fid:
while True:
line = fid.readline()
if not line:
break
line = line.strip()
if len(line) > 0 and line[0] != "#":
elems = line.split()
image_id = int(elems[0])
qvec = np.array(tuple(map(float, elems[1:5])))
tvec = np.array(tuple(map(float, elems[5:8])))
camera_id = int(elems[8])
image_name = elems[9]
elems = fid.readline().split()
xys = np.column_stack([tuple(map(float, elems[0::3])), tuple(map(float, elems[1::3]))])
point3D_ids = np.array(tuple(map(int, elems[2::3])))
images[image_id] = Image(
id=image_id,
qvec=qvec,
tvec=tvec,
camera_id=camera_id,
name=image_name,
xys=xys,
point3D_ids=point3D_ids,
)
return images
def read_images_binary(path_to_model_file: Path) -> Mapping[int, Image]:
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadImagesBinary(const std::string& path)
void Reconstruction::WriteImagesBinary(const std::string& path)
"""
images = {}
with open(path_to_model_file, "rb") as fid:
num_reg_images = read_next_bytes(fid, 8, "Q")[0]
for _ in range(num_reg_images):
binary_image_properties = read_next_bytes(fid, num_bytes=64, format_char_sequence="idddddddi")
image_id = binary_image_properties[0]
qvec = np.array(binary_image_properties[1:5])
tvec = np.array(binary_image_properties[5:8])
camera_id = binary_image_properties[8]
image_name = ""
current_char = read_next_bytes(fid, 1, "c")[0]
while current_char != b"\x00": # look for the ASCII 0 entry
image_name += current_char.decode("utf-8")
current_char = read_next_bytes(fid, 1, "c")[0]
num_points2D = read_next_bytes(fid, num_bytes=8, format_char_sequence="Q")[0]
x_y_id_s = read_next_bytes(fid, num_bytes=24 * num_points2D, format_char_sequence="ddq" * num_points2D)
xys = np.column_stack([tuple(map(float, x_y_id_s[0::3])), tuple(map(float, x_y_id_s[1::3]))])
point3D_ids = np.array(tuple(map(int, x_y_id_s[2::3])))
images[image_id] = Image(
id=image_id,
qvec=qvec,
tvec=tvec,
camera_id=camera_id,
name=image_name,
xys=xys,
point3D_ids=point3D_ids,
)
return images
def write_images_text(images, path):
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadImagesText(const std::string& path)
void Reconstruction::WriteImagesText(const std::string& path)
"""
if len(images) == 0:
mean_observations = 0
else:
mean_observations = sum((len(img.point3D_ids) for _, img in images.items())) / len(images)
HEADER = (
"# Image list with two lines of data per image:\n"
+ "# IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME\n"
+ "# POINTS2D[] as (X, Y, POINT3D_ID)\n"
+ f"# Number of images: {len(images)}, mean observations per image: {mean_observations}\n"
)
with open(path, "w") as fid:
fid.write(HEADER)
for _, img in images.items():
image_header = [img.id, *img.qvec, *img.tvec, img.camera_id, img.name]
first_line = " ".join(map(str, image_header))
fid.write(first_line + "\n")
points_strings = []
for xy, point3D_id in zip(img.xys, img.point3D_ids):
points_strings.append(" ".join(map(str, [*xy, point3D_id])))
fid.write(" ".join(points_strings) + "\n")
def write_images_binary(images, path_to_model_file):
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadImagesBinary(const std::string& path)
void Reconstruction::WriteImagesBinary(const std::string& path)
"""
with open(path_to_model_file, "wb") as fid:
write_next_bytes(fid, len(images), "Q")
for _, img in images.items():
write_next_bytes(fid, img.id, "i")
write_next_bytes(fid, img.qvec.tolist(), "dddd")
write_next_bytes(fid, img.tvec.tolist(), "ddd")
write_next_bytes(fid, img.camera_id, "i")
for char in img.name:
write_next_bytes(fid, char.encode("utf-8"), "c")
write_next_bytes(fid, b"\x00", "c")
write_next_bytes(fid, len(img.point3D_ids), "Q")
for xy, p3d_id in zip(img.xys, img.point3D_ids):
write_next_bytes(fid, [*xy, p3d_id], "ddq")
def read_points3D_text(path):
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadPoints3DText(const std::string& path)
void Reconstruction::WritePoints3DText(const std::string& path)
"""
points3D = {}
with open(path) as fid:
while True:
line = fid.readline()
if not line:
break
line = line.strip()
if len(line) > 0 and line[0] != "#":
elems = line.split()
point3D_id = int(elems[0])
xyz = np.array(tuple(map(float, elems[1:4])))
rgb = np.array(tuple(map(int, elems[4:7])))
error = float(elems[7])
image_ids = np.array(tuple(map(int, elems[8::2])))
point2D_idxs = np.array(tuple(map(int, elems[9::2])))
points3D[point3D_id] = Point3D(
id=point3D_id, xyz=xyz, rgb=rgb, error=error, image_ids=image_ids, point2D_idxs=point2D_idxs
)
return points3D
def read_points3D_binary(path_to_model_file: Path) -> Mapping[int, Point3D]:
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadPoints3DBinary(const std::string& path)
void Reconstruction::WritePoints3DBinary(const std::string& path)
"""
points3D = {}
with open(path_to_model_file, "rb") as fid:
num_points = read_next_bytes(fid, 8, "Q")[0]
for _ in range(num_points):
binary_point_line_properties = read_next_bytes(fid, num_bytes=43, format_char_sequence="QdddBBBd")
point3D_id = binary_point_line_properties[0]
xyz = np.array(binary_point_line_properties[1:4])
rgb = np.array(binary_point_line_properties[4:7])
error = np.array(binary_point_line_properties[7])
track_length = read_next_bytes(fid, num_bytes=8, format_char_sequence="Q")[0]
track_elems = read_next_bytes(fid, num_bytes=8 * track_length, format_char_sequence="ii" * track_length)
image_ids = np.array(tuple(map(int, track_elems[0::2])))
point2D_idxs = np.array(tuple(map(int, track_elems[1::2])))
points3D[point3D_id] = Point3D(
id=point3D_id, xyz=xyz, rgb=rgb, error=error, image_ids=image_ids, point2D_idxs=point2D_idxs
)
return points3D
def write_points3D_text(points3D, path):
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadPoints3DText(const std::string& path)
void Reconstruction::WritePoints3DText(const std::string& path)
"""
if len(points3D) == 0:
mean_track_length = 0
else:
mean_track_length = sum((len(pt.image_ids) for _, pt in points3D.items())) / len(points3D)
HEADER = (
"# 3D point list with one line of data per point:\n"
+ "# POINT3D_ID, X, Y, Z, R, G, B, ERROR, TRACK[] as (IMAGE_ID, POINT2D_IDX)\n"
+ f"# Number of points: {len(points3D)}, mean track length: {mean_track_length}\n"
)
with open(path, "w") as fid:
fid.write(HEADER)
for _, pt in points3D.items():
point_header = [pt.id, *pt.xyz, *pt.rgb, pt.error]
fid.write(" ".join(map(str, point_header)) + " ")
track_strings = []
for image_id, point2D in zip(pt.image_ids, pt.point2D_idxs):
track_strings.append(" ".join(map(str, [image_id, point2D])))
fid.write(" ".join(track_strings) + "\n")
def write_points3D_binary(points3D, path_to_model_file):
"""
see: src/base/reconstruction.cc
void Reconstruction::ReadPoints3DBinary(const std::string& path)
void Reconstruction::WritePoints3DBinary(const std::string& path)
"""
with open(path_to_model_file, "wb") as fid:
write_next_bytes(fid, len(points3D), "Q")
for _, pt in points3D.items():
write_next_bytes(fid, pt.id, "Q")
write_next_bytes(fid, pt.xyz.tolist(), "ddd")
write_next_bytes(fid, pt.rgb.tolist(), "BBB")
write_next_bytes(fid, pt.error, "d")
track_length = pt.image_ids.shape[0]
write_next_bytes(fid, track_length, "Q")
for image_id, point2D_id in zip(pt.image_ids, pt.point2D_idxs):
write_next_bytes(fid, [image_id, point2D_id], "ii")
def detect_model_format(path: Path, ext: str) -> bool:
parts = ["cameras", "images", "points3D"]
if all([(path / p).with_suffix(ext) for p in parts]):
print("Detected model format: '" + ext + "'")
return True
return False
def read_model(path: Path, ext: str = ""):
# try to detect the extension automatically
if ext == "":
if detect_model_format(path, ".bin"):
ext = ".bin"
elif detect_model_format(path, ".txt"):
ext = ".txt"
else:
print("Provide model format: '.bin' or '.txt'")
return
if ext == ".txt":
cameras = read_cameras_text((path / "cameras").with_suffix(ext))
images = read_images_text((path / "images").with_suffix(ext))
points3D = read_points3D_text((path / "points3D").with_suffix(ext))
else:
cameras = read_cameras_binary((path / "cameras").with_suffix(ext))
images = read_images_binary((path / "images").with_suffix(ext))
points3D = read_points3D_binary((path / "points3D").with_suffix(ext))
return cameras, images, points3D
def write_model(cameras, images, points3D, path, ext=".bin"):
if ext == ".txt":
write_cameras_text(cameras, os.path.join(path, "cameras" + ext))
write_images_text(images, os.path.join(path, "images" + ext))
write_points3D_text(points3D, os.path.join(path, "points3D") + ext)
else:
write_cameras_binary(cameras, os.path.join(path, "cameras" + ext))
write_images_binary(images, os.path.join(path, "images" + ext))
write_points3D_binary(points3D, os.path.join(path, "points3D") + ext)
return cameras, images, points3D
def qvec2rotmat(qvec):
return np.array([
[
1 - 2 * qvec[2] ** 2 - 2 * qvec[3] ** 2,
2 * qvec[1] * qvec[2] - 2 * qvec[0] * qvec[3],
2 * qvec[3] * qvec[1] + 2 * qvec[0] * qvec[2],
],
[
2 * qvec[1] * qvec[2] + 2 * qvec[0] * qvec[3],
1 - 2 * qvec[1] ** 2 - 2 * qvec[3] ** 2,
2 * qvec[2] * qvec[3] - 2 * qvec[0] * qvec[1],
],
[
2 * qvec[3] * qvec[1] - 2 * qvec[0] * qvec[2],
2 * qvec[2] * qvec[3] + 2 * qvec[0] * qvec[1],
1 - 2 * qvec[1] ** 2 - 2 * qvec[2] ** 2,
],
])
def rotmat2qvec(R):
Rxx, Ryx, Rzx, Rxy, Ryy, Rzy, Rxz, Ryz, Rzz = R.flat
K = (
np.array([
[Rxx - Ryy - Rzz, 0, 0, 0],
[Ryx + Rxy, Ryy - Rxx - Rzz, 0, 0],
[Rzx + Rxz, Rzy + Ryz, Rzz - Rxx - Ryy, 0],
[Ryz - Rzy, Rzx - Rxz, Rxy - Ryx, Rxx + Ryy + Rzz],
])
/ 3.0
)
eigvals, eigvecs = np.linalg.eigh(K)
qvec = eigvecs[[3, 0, 1, 2], np.argmax(eigvals)]
if qvec[0] < 0:
qvec *= -1
return qvec
def main():
parser = argparse.ArgumentParser(description="Read and write COLMAP binary and text models")
parser.add_argument("--input-model", help="path to input model folder")
parser.add_argument("--input-format", choices=[".bin", ".txt"], help="input model format", default="")
parser.add_argument("--output-model", help="path to output model folder")
parser.add_argument("--output-format", choices=[".bin", ".txt"], help="output model format", default=".txt")
args = parser.parse_args()
cameras, images, points3D = read_model(path=args.input_model, ext=args.input_format)
print("num_cameras:", len(cameras))
print("num_images:", len(images))
print("num_points3D:", len(points3D))
if args.output_model is not None:
write_model(cameras, images, points3D, path=args.output_model, ext=args.output_format)
if __name__ == "__main__":
main()