# ROS bags & robot models

Drop an `.mcap` file into the viewer and it opens with a generated layout —
no config, no conversion. This page is the short map of what decodes, how a
tf tree becomes a moving robot, and where robot models come from.

## What decodes with zero config

Channels are dispatched by the **writer's own declarations** — wire encoding
and schema name — never guessed from shapes:

| in the file | shows as |
|---|---|
| Foxglove schemas (protobuf or json): `CompressedImage` / `RawImage` | camera panes |
| `foxglove.PointCloud` | point-cloud pane |
| `FrameTransform(s)` | 3D scene (tf tree, below) |
| `PoseInFrame` | 3D pose track (an axis triad following the pose) |
| `SceneUpdate` model primitives | meshes in the 3D scene |
| **ROS 2 bags** (`cdr` + `ros2msg` — rosbag2's own MCAP output) | same as their Foxglove twins ↓ |
| **ROS 1 bags** (`ros1` + `ros1msg`) | same as their Foxglove twins ↓ |
| `sensor_msgs/JointState` | joint charts **and**, with a robot description, a moving 3D robot ([below](#joint-angles-are-3d-too-forward-kinematics)) |
| any other numeric/string channel (IMU, diagnostics, …) | line charts / span tracks |

Well-known ROS types decode **like their Foxglove twins** — the message is
normalized, then everything downstream is one code path:
`tf2_msgs/TFMessage` → `FrameTransforms` · `sensor_msgs`
`CompressedImage`/`Image`/`PointCloud2` → `CompressedImage`/`RawImage`/`PointCloud`
· `geometry_msgs/PoseStamped`, `nav_msgs/Odometry` → `PoseInFrame`.
The catalog keeps the honest original name in `meta.schema` and records the
twin in `meta.twin`.

Still declined, each with a console warning naming why: `ros2idl`,
flatbuffers, `Grid`, `CameraCalibration`, compressed video.

## The tf tree is the 3D scene

A tf message states where a child frame sits **in its parent**. Reading a
child `as: transform3d` composes those local transforms down the declared
`parent_frame_id` chain — across `/tf` and `/tf_static` — into fixed-frame
poses, the same semantics Foxglove renders. In the 3D pane:

- every tracked frame draws an **axis triad** following its motion;
- frames with declared parentage are connected by **bones** — a tf-only
  humanoid recording reads as a moving skeleton, no model required;
- charts are untouched: a tf field bound to a `lineChart` still plots the
  schema's own local numbers.

## From skeleton to robot: URDF models

`recon3d` takes external geometry by URL in its `models:` slot, and a
**`.urdf` loads directly** — robots stay in their native distribution form:

```yaml
- view: recon3d
  up: z
  tracks: [{ field: "/tf::*", as: transform3d }]
  models:
    - https://huggingface.co/datasets/live9080/dreamlake-robots/resolve/main/g1/g1_29dof.urdf
```

How a URDF behaves:

- one named node **per link** — each joins its `/tf::<link>` track by name;
- mesh references resolve **relative to the `.urdf`** (STL / OBJ / DAE /
  glTF, plus box/cylinder/sphere primitives). A Collada file's `<up_axis>` is
  ignored, as ROS tooling ignores it: a URDF's mesh is in the link frame, so
  the same part exported as STL has to land identically;
- links no track claims stand at their **zero-angle rest pose** — unless a
  tracked ancestor exists, in which case they ride it;
- `.xacro` is refused with a clear error — expand it first.

Plain `glb` / `gltf` / `obj` / `stl` URLs work in the same slot.

### `package://` — pointing at a description where it lives

`package://<pkg>/meshes/x.dae` is a ROS **package-manager** path. Only a
package registry knows where `<pkg>` is, and a browser has none, so the name
is resolved three ways, most explicit first:

```yaml
models:
  - url: https://…/droid_panda_robotiq/panda_robotiq_85.urdf
    packages:
      franka_description: https://…/franka_description
      robotiq_2f_85_gripper_visualization: https://…/robotiq_2f_85_gripper_visualization
```

1. **the `packages:` map** above — always wins;
2. **the URDF's own URL**, when one of its path segments is the package name.
   `…/franka_description/robots/panda_arm.urdf` referencing
   `package://franka_description/meshes/visual/link0.dae` resolves with no
   config at all — which is the point: upload a description repo unchanged
   (URDF in `robots/`, meshes in `meshes/`) and it works;
3. **the URDF's directory**, as a best effort, with a console warning naming
   the package it could not locate.

Only a description that references a package it does not live inside — a
composition of two upstream packages, say — needs the map.

## Joint angles are 3D too: forward kinematics

Two recordings of the same robot can look completely different on disk:

| the bag carries | what it is |
|---|---|
| `/tf` per link | poses `robot_state_publisher` **already computed** — self-contained geometry |
| `/joint_states` | the joint ANGLES — low-dimensional, and not 3D until a URDF interprets them |

The second shape is extremely common: the angles *are* the robot's state, the
poses are derivable from them, so many rigs never write the poses down. Bind
the topic with **`joints:`** and the viewer does that derivation itself — the
same product `robot_state_publisher` would have published:

```yaml
- view: recon3d
  up: z
  models: [ https://…/g1/g1_29dof.urdf ]
  joints: [ /joint_states ]
  tracks: [{ field: "/tf::pelvis", as: transform3d }]   # the floating base
```

- angles are matched **by joint name**, never by index —
  `sensor_msgs/JointState` carries a `name` array precisely because the order
  is the publisher's business, and each message is aligned on its own names;
- `<mimic>` joints follow their master, which is the only reason a gripper
  opens: one commanded finger, several linkage joints that must track it;
- travel is clamped to each joint's `<limit>`, so a mis-scaled topic bends the
  robot to its stops rather than folding it through itself;
- `/joint_states` says nothing about where the robot *is*. Give the base its
  own `transform3d` track (`world → pelvis`) and the whole solve rides it;
  with no such track the robot stands at the origin.

**Precedence**, most authoritative first: a link's own `transform3d` track
(someone recorded that pose) → forward kinematics → riding a tracked ancestor
→ the rest pose. A published pose always wins over a computed one.

The charts benefit too: a JointState topic's columns are named
`position.<joint>` after the publisher's own joints, so the plot is labelled
with joint names instead of `position.0 … position.28`.

## Preset robots, applied by name match

Known robots live in the registry
[`live9080/dreamlake-robots`](https://huggingface.co/datasets/live9080/dreamlake-robots)
(URDF + meshes, license noted per robot) — currently the Unitree G1, the
Franka Panda, and the Panda + Robotiq 2F-85 pairing DROID-style recordings
publish. The generated default view applies a preset when a file's published
NAMES carry that description's own naming, and there are two fingerprints
because there are two kinds of recording:

- **tf frame ids** ↔ the description's link names — a bag with per-link tf;
- **JointState names** ↔ the description's joint names — a bag that publishes
  no frames at all, where nothing else could say which robot this is.

Either one firing selects the preset. The match is strict — most of that
fingerprint must be present, and never fewer than six exact names — and it
**scales with the description**: a 36-link humanoid needs ~22 matching names,
an 8-link arm needs 6. So a G1 recording opens as the G1, a Panda recording
opens as a Panda, and "looks like a humanoid" never fires: a Valkyrie bag,
which has a pelvis and knees and shoulders under its own naming, keeps the
skeleton.

When several presets fit — and they will, since the Panda's links are a
subset of the Panda-plus-gripper's — the one that explains **more of what the
file published** wins. That is why a DROID episode opens with the gripper
attached rather than as a bare arm.

The applied model appears in the generated `.dreamrc` as a `models:` line —
delete it to opt out, or point it at a different description.

**Adding a robot**: upload its `urdf + meshes` folder to the registry, then
add one entry (id, url, link and joint fingerprints, and a `packages:` map if
it references a package it does not live inside) to
`dataset-viz/formats/robot-presets.ts`. `viz/scripts/fingerprint.py` prints
both fingerprints from a URDF.

## Pointers

- Exact per-format tables (encodings, omission rules, budgets): the
  [reference](/dataset-viz/reference.md)'s `mcap` section.
- All `recon3d` bindings and options: the reference's
  [component table](/dataset-viz/reference.md).
- Writing layouts by hand: [Write the .dreamrc](/dataset-viz/spec.md).