# DreamLake — Full documentation

> @dreamlake/viz visualizes robot-learning datasets in the browser: one `.dreamrc` file at a dataset root renders every episode — LeRobot / zarr / MCAP / plain folders, cameras, depth, point clouds, time series, annotations, and 3D reconstructions, all read in place over HTTP.

Generated from https://viz.dreamlake.ai. 25 pages.

---

Source: https://viz.dreamlake.ai

# Quick start

`@dreamlake/viz` visualizes robot-learning datasets in the browser. One
`.dreamrc` file at a dataset root renders every episode in it — LeRobot /
zarr / MCAP / plain folders; cameras, depth maps, point clouds, time series,
annotations and 3D reconstructions — all read **in place** over HTTP range
requests, never downloaded whole.

The design in one sentence: pre-built view components each declare an input
contract, format adapters normalize whatever is on disk into those contracts,
and the `.dreamrc` states which fields feed which views — the program never
decides what your data means. The full story:
[the architecture](/dataset-viz/overview.md).

## The no-code path: one file on your dataset

If your dataset is public (a HuggingFace repo, any CORS-enabled bucket), you
never need to install anything. Write a `.dreamrc` at its root:

```yaml file=".dreamrc"
version: 1

dataset:
  format: lerobot        # lerobot | folder | umi | mcap
  episodes: auto

views:
  - view: videoStack
    cameras: ["observation.images.*"]
  - view: lineChart
    series:
      - { field: [action, "*"] }
```

Check it from a shell before you ship it — with no config it prints the
dataset's field inventory, which is how you find out what to bind:

```bash file="terminal"
npx tsx scripts/check-dreamrc.mts hf:your-name/your-dataset
```

Then open the dataset in the DreamLake app, or compare against the
[gallery](/dataset-viz/gallery.md) — every entry there is a complete `.dreamrc`
over a real public dataset. Copyable starting points:
[templates](/dataset-viz/templates.md). The grammar:
[the .dreamrc file](/dataset-viz/spec.md).

## The library path: render it yourself

For hosts embedding the viewer. The library is YAML-free (parse upstream) and
credential-free (storage drivers carry identifiers only):

```bash file="terminal"
pnpm add @dreamlake/viz react react-dom
```

```tsx file="app.tsx"

const rc = validateDreamrc(parse(dreamrcText))
const { episodes, warnings } = await resolveDataset(rc, {
  // For a file found at a dataset root, inject that root; a file declaring
  // its own storage: resolves alone and the declaration wins.
  rootStorage: { driver: 'hf', repo: 'live9080/dreamlake-ceramics' },
})

// One episode → one player. Render a list by mapping; wrap it in
// <SyncScrollProvider> and mount lazily for long datasets.
export const App = () => <DatasetViz episode={episodes[0]} views={rc.views} />
```

`validateDreamrc` throws errors written to be fixed mechanically — the
offending key, the allowed values, a did-you-mean — because a `.dreamrc` is
often authored by an agent in a write → validate → fix loop. Hosts extend
every axis at runtime: `registerStorage` for an authorized backend,
`registerFormat` for a dataset layout, `registerComponent` for a view of
their own ([TypeScript API](/dataset-viz/spec.md#typescript-api)).

## What's in this package

| export | what it is |
| --- | --- |
| `@dreamlake/viz/dataset-viz` | the `.dreamrc` engine: validate, resolve, render ([docs](/dataset-viz/overview.md)) |
| `@dreamlake/viz/episode-*`, `…/media-overlay` | the underlying episode components — video stack, line chart, timeline, frame stack, 3D scene ([docs](/components/episode-video-stack.md)) |
| `@dreamlake/viz/file-preview` | single-file preview used by the DreamLake file browser |
| `@dreamlake/viz/schema-viz` | the previous-generation, schema-driven viewer the platform still ships |

## Where to go next

| you want | read |
| --- | --- |
| to understand the design | [the architecture](/dataset-viz/overview.md) |
| to write a `.dreamrc` | [the .dreamrc file](/dataset-viz/spec.md) · [view components](/dataset-viz/views.md) · [reference](/dataset-viz/reference.md) |
| to prepare a dataset | [what your data must look like](/dataset-viz/requirements.md) |
| working examples | [templates](/dataset-viz/templates.md) · [gallery](/dataset-viz/gallery.md) |
| these docs, for your agent | [LLM-readable docs](/llm-readable.md) |

---

Source: https://viz.dreamlake.ai/dataset-viz/overview

# The architecture

**A viewer cannot generate itself.** We do not produce bespoke visualization
code per dataset — the system ships **pre-built view components** (a video
wall, a line chart, a timeline, an animated 3D scene) and one config file
composes them over your data. Everything on these pages follows from what
that takes:

1. A pre-built view is only possible because its **input is known**. Every
   component declares a contract — the structure it consumes, per binding
   slot. Without that contract, a dataset could contain hand keypoints, you
   could know they are hand keypoints, and the viewer still could not draw
   them: nothing would say what structure to hand the view.
2. Datasets do not arrive in that structure, and never will — so **format
   adapters** normalize what is on disk into it.
3. Nothing about the bytes says which fields should feed which views — so a
   config, the **`.dreamrc`**, states it. Meaning is written down once, by a
   person or an agent who read the dataset; the program never guesses it.

## The three layers

```
    bytes on disk                  in memory                     on screen
 ┌───────────────────┐      ┌──────────────────────┐      ┌───────────────────┐
 │  existing formats │      │  payload contracts   │      │  view components  │
 │  LeRobot · zarr   │      │  video · series ·    │      │  videoStack ·     │
 │  MCAP · folders   │  →   │  keypoints · segments│  →   │  lineChart ·      │
 │  WebVTT · COCO ·  │      │  pointcloud · mesh3d │      │  timeline ·       │
 │  glTF · Parquet   │      │  · … (closed set)    │      │  recon3d · …      │
 └───────────────────┘      └──────────────────────┘      └───────────────────┘
    always theirs             ours — in memory only         composed by .dreamrc
          └── adapters normalize ──┘└── views consume contracts only ──┘
```

The middle layer is the narrow waist: N formats × M components costs
**N + M, not N × M**. A new adapter works with every component the day it
lands; a new component works with every format. That holds only while the
contract set stays closed — adding a payload kind is a deliberate act in the
library, never something an adapter does on the side.

## Views declare what they read

Every registered component states, per binding slot, the payload kinds it
consumes. That declaration **is** the input contract:

| view | slot | contract it asks for |
| --- | --- | --- |
| `timeline` | `tracks` | `segments` — `{ start, end, label }` spans |
| `pointCloud` | `cloud` | `pointcloud` — per-frame `xyz` (+ optional `rgb`) |
| `videoStack` | `overlays` | `keypoints` (a skeleton) or `segments` (captions) |
| `recon3d` | `geometry` + `tracks` | `mesh3d` geometry moved by `transform3d` / `vertices3d` / `pose3d` |

One declaration is enforced three ways: it is what the decoder is called
with, what a `*` glob is filtered by, and what an author's `as:` is validated
against. Bind a field to a slot whose contract its bytes cannot satisfy and
the panel says so by name — `observation.state is float32 [6]; keypoints
needs [J,2] or [J,3]` — instead of drawing something wrong. The full tables:
[reference](/dataset-viz/reference.md#kinds--the-catalogs-and-the-views).

## The contract lives in memory, never on disk

The payload kinds are the **only** place a "DreamLake shape" exists, and they
exist only as in-memory structures — a TypeScript union the components are
typed against. Nothing on disk is ever asked to look like them. There is no
DreamLake file format, no required directory layout, no conversion step: a
public dataset renders with exactly one file added (the `.dreamrc`), and
deleting that file leaves the dataset byte-identical to how it was.

## Where the bytes come from — the standards ladder

Each contract is fed from the wire in a fixed order of preference:

1. **The container's own idiom.** Data that lives inside a dataset is
   expressed the way that container already expresses things: a LeRobot
   feature with its `dtype`/`shape`/`names`, a zarr array with its codec, an
   MCAP channel with its schema. A `[J,2]` float feature is already a
   skeleton's worth of numbers; an int column plus a label table is already
   LeRobot's own task pattern, and `segments` reads it as found.
2. **An established standard beside the data**, when the container cannot
   hold it or is not yours to write into: WebVTT/SRT for labelled time spans,
   COCO for 2D keypoints, glTF for geometry (and, animated, for its motion),
   Parquet for per-frame numbers. Every one of them predates us.
3. **A structure of our own, defined deliberately.** Where no existing
   standard fits, we define one — that is the ladder's last rung, not a
   failure of it, and it is how a dataset gets guaranteed a correct
   visualization when the ecosystem offers nothing to lean on. It has an
   admission bar, because the thing to prevent is the casual, unexamined
   format: survey the existing implementations first (LeRobot, Rerun,
   Foxglove, glTF, COCO, … — absorb their experience before writing a line),
   give the result its own spec section and a version, and keep it
   normalizable to and from neighbouring standards where possible. There is
   currently one on disk — the motion-track Parquet profile v1
   ([which format to write](/dataset-viz/requirements.md#which-format-to-write))
   — and one in memory: the payload contracts themselves, versioned with the
   library and [published in full](/dataset-viz/reference.md#payload-contracts-in-full).
4. **A format we do not read yet gets an adapter, not a rule.** Nobody is
   asked to convert: `registerFormat` adds a reader that normalizes the
   foreign layout into the same payloads, and until one exists, a custom
   component can bind the raw file and parse its own wire format —
   [the extension path](/dataset-viz/reference.md#data-the-library-does-not-know).
   A shape that proves general is then promoted into the contract set.

The result is that the requirements on your data are small and mostly not
ours: [what your data must look like](/dataset-viz/requirements.md) is two rules
plus the shape each contract demands — which is not a rule we impose, but
what a skeleton, a point cloud, or a depth map *is*.

## The `.dreamrc` states meaning

The last piece is the wiring. A field's catalog entry records how its bytes
are addressed (a video stream, numbers with a shape, a file with an
extension) and draws no conclusion; the binding in the `.dreamrc` is where a
human or an agent states what the field *is*, by choosing the slot — and,
where a slot reads more than one contract, saying which with `as:`.

```yaml
views:
  - view: videoStack
    cameras: [observation.images.ego]                        # decoded as video
    overlays:
      - { field: observation.keypoints_2d.left.ego, as: keypoints }
  - view: pointCloud
    cloud: [observation.environment_state]                 # the author knows
```

`observation.environment_state` is a `[512,6]` float tensor whose name says
nothing. The program never decides what it means; the config's author read
the inventory once and wrote it down. That one sentence is the design — the
rest is [grammar](/dataset-viz/spec.md).

## What this buys

- **Pre-built views over arbitrary datasets.** The contracts are the reason
  no per-dataset code is ever generated.
- **Configs port.** A `.dreamrc` written for one dataset works on another by
  changing field names, because components only ever see contracts.
- **Wrong renders become named errors.** Every binding is checked against a
  declared contract, so a mistake fails with the fix in the message instead
  of drawing a silently wrong picture.
- **AI-authorable.** Meaning is stated in one small file with a closed
  vocabulary, and every validation error names the offending key and the
  allowed values — a write → validate → fix loop an agent can run alone.

## Where to go

| you are | read |
| --- | --- |
| writing a `.dreamrc` | [the .dreamrc file](/dataset-viz/spec.md), then [view components](/dataset-viz/views.md) |
| preparing or publishing a dataset | [what your data must look like](/dataset-viz/requirements.md) |
| looking up a name or a config key | [reference](/dataset-viz/reference.md) |
| wanting working examples | [templates](/dataset-viz/templates.md) · [gallery](/dataset-viz/gallery.md) |
| changing the library | [library internals](/dataset-viz/internals.md) |

---

Source: https://viz.dreamlake.ai/components/episode-timeline

# EpisodeTimeline

A zoomable episode-detail timeline: time ruler on top, a frame thumbnail strip
beneath it, and N rows of labelled track blocks under that. The cursor is
**hover-driven** — the user's pointer position is the single source of truth
for the highlighted time. An optional `time` prop provides a fallback position
to display when the pointer leaves (e.g., for syncing with a running video
clock).

Pair with [EpisodeVideoStack](/components/episode-video-stack.md) when you have
multi-camera footage that should scrub against the same cursor.

## Basic usage

Pure hover-driven: the cursor follows the pointer; when it leaves, the cursor
disappears.

```tsx file="BasicSpec.tsx"

export const BasicSpec = () => {
  const [t, setT] = useState<number | null>(null)
  return (
    <EpisodeTimeline
      duration={DEMO_DURATION}
      frames={DEMO_FRAMES}
      tracks={DEMO_TRACKS}
      time={t}
      onHover={setT}
      onHoverEnd={() => setT(null)}
    />
  )
}
```

## Frames

`frames` is an **ascending-time-sorted** list of `{ time, image }`. The
component does not re-sort — sorting on every render would be wasteful and
the contract is documented in the type. Frames at the same time deduplicate
naturally because the greedy culler keeps the first one and drops any
followers within `frameWidth + 4px`.

When `frames` is empty, the strip and its footer label collapse entirely —
there's no empty band and no `0 OF 0 FRAMES` readout; the track rows sit
directly beneath the ruler.

Each frame's **left edge** sits at its time on the axis
(`fx = timeToX(f.time)`) — the frame visually represents the chunk starting
at `time`. This keeps the `t = 0` cell fully visible at the timeline's left
edge instead of half-clipped.

```ts
type FrameSample = {
  time: number       // seconds; 0 ≤ time ≤ duration
  image: string      // URL
  alt?: string       // reserved for future a11y surfacing
}

const frames: FrameSample[] = [
  { time: 0,    image: '/keyframes/000.jpg' },
  { time: 0.5,  image: '/keyframes/030.jpg' },
  { time: 1.0,  image: '/keyframes/060.jpg' },
]
```

### Greedy culling

When the visible window is wide (low zoom), many frames would overlap in
screen space. The component sweeps left-to-right and renders a frame only
when its `time` is at least `(frameWidth + 4px) / pxPerSec` later than the
previous rendered frame. Off-screen frames are skipped.

As you zoom in, the visible time window shrinks and `pxPerSec` grows, so the
stride between renderable frames shrinks too — more of the supplied frames
qualify until eventually every one is shown. The footer readout
`{visible} OF {total} FRAMES` updates live as you scrub.

Time positions are exact — each frame's left edge sits at the X
corresponding to its actual `time`, no quantization to evenly-spaced slots.

## Tracks and blocks

Each track is a row; each block is a labelled span `{ start, end, label }`.
Blocks outside the visible window are skipped entirely; blocks that straddle
the edge render with a dashed border on the clipped side to signal
"continues off-screen."

```ts
type TrackBlock = {
  id?: string        // stable key for hover + click callbacks
  start: number
  end: number
  label: string
}

type TimelineTrack = {
  id: string
  name?: string      // accessibility only — not rendered
  blocks: TrackBlock[]
}

const tracks: TimelineTrack[] = [
  {
    id: 'phases',
    blocks: [
      { id: 'p1', start: 0,   end: 2.4, label: 'idle' },
      { id: 'p2', start: 2.4, end: 9.1, label: 'approach' },
    ],
  },
]
```

Provide stable `id` on each block to get reliable hover highlight + callback
identity.

## Interaction model

| Gesture | Effect |
| --- | --- |
| **Hover** | Bright accent cursor pinned to the pointer X; highlights every track block whose `[start, end]` contains the cursor time AND the frame whose visual extent contains it. Fires `onHover(time)` on every move, `onHoverEnd()` on leave. |
| **Click** on empty area | `onSeek(time)` — separate "commit" event distinct from continuous hover. |
| **Click** on a block | `onBlockClick(block, trackId)` — does NOT fire `onSeek`. |
| **Drag** (≥ 4px) | Pans the viewport (when zoomed in). |
| **Shift + drag** vertical | Drag-zoom: drag up to zoom in, down to zoom out. Anchors the time under the cursor. |
| **Wheel + alt/⌘/ctrl** | Zoom at cursor (also fires on trackpad pinch — macOS dispatches `ctrlKey + wheel`). |
| **Wheel** (horizontal-dominant) | Pan when zoomed in. Two-finger horizontal swipes are always swallowed, so a trackpad gesture on the timeline never triggers the browser's back/forward navigation. |
| **ZoomBar** `‹` / `›` | Step zoom by ×1.4 / ÷1.4. |
| **ZoomBar** drag readout | Continuous zoom via `exp(dx × 0.008)`. |

## Controlled vs uncontrolled viewport

`zoom` and `panPct` are optional. Omit them → the component owns viewport
state internally. Provide **either** one → the component flips into
controlled mode for **both** (the omitted value falls back to its default,
but the component no longer writes to its internal state).
`onViewportChange` always fires when the user gestures, so the parent can
persist the new values.

```tsx
// Uncontrolled
<EpisodeTimeline duration={duration} time={t} onSeek={setT} />

// Controlled — persist viewport in URL / localStorage
const [vp, setVp] = useState({ zoom: 1, panPct: 0 })
<EpisodeTimeline
  duration={duration}
  time={t}
  onSeek={setT}
  zoom={vp.zoom}
  panPct={vp.panPct}
  onViewportChange={setVp}
/>
```

```ts
type ViewportState = {
  zoom: number       // ≥ 1
  panPct: number     // 0 ≤ panPct ≤ 1 - 1/zoom
}
```

## Design rules

User-visible behaviour the component enforces:

- **0s strict left edge** — the ruler picks tick steps from a fixed ladder
  (`0.05, 0.1, 0.25, 0.5, 1, 5, 10, …` seconds) and starts ticking at `0`;
  the chart never shows negative time. The `0s` and duration labels stay
  visible at the very edges instead of getting clipped.
- **Anchor zoom** — `⌘` (or `alt` / `ctrl`) + wheel zooms about the cursor:
  the time directly under the pointer stays fixed across the zoom step.
- **Three-tier ruler** — `major / minor / micro`. As zoom changes, ticks
  cross-fade smoothly between tiers instead of popping.

## Props

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `duration` | `number` | — | Episode duration in seconds. Must be `> 0`. |
| `frames` | `FrameSample[]` | — | Frame thumbnails, sorted ascending by time. |
| `tracks` | `TimelineTrack[]` | — | Track rows, drawn top-to-bottom in array order. |
| `time` | `number \| null` | `null` | Fallback cursor position when the user is NOT hovering. Hover always overrides. `null` → pure hover-driven cursor that disappears on leave. |
| `onHover` | `(t: number) => void` | — | Called continuously on every hover move with the time at the cursor. |
| `onHoverEnd` | `() => void` | — | Called when the cursor leaves the timeline. |
| `onSeek` | `(t: number) => void` | — | Called on click in an empty area (single commit). Clicks on a track block fire `onBlockClick` instead and do NOT fire `onSeek`. |
| `zoom` | `number` | `1` | Controlled zoom level. Clamped to `[minZoom, maxZoom]`. Pairing with `panPct` flips the component into controlled mode for both. |
| `panPct` | `number` | `0` | Controlled pan as fraction of duration. Clamped to `[0, 1 - 1/zoom]`. |
| `onViewportChange` | `(v: ViewportState) => void` | — | Fires when the user changes zoom or pan via wheel/drag. |
| `onBlockClick` | `(block, trackId) => void` | — | Click on a track block (suppresses `onSeek`). |
| `onBlockHover` | `(block \| null, trackId \| null) => void` | — | Hover enter/leave on a track block. Receives `null` on leave. |
| `framesLabel` | `string` | `'FRAMES'` | Left-side uppercase caption under the frame strip. The right side is auto-generated as `{visibleCount} OF {totalCount} FRAMES`. |
| `frameHeight` | `number` | auto | Frame thumbnail height in pixels (width auto from 16:9). When omitted, sized to fit 12 cells with 4px gaps. |
| `trackRowHeight` | `number` | `28` | Track row height. |
| `minZoom` | `number` | `1` | Minimum zoom factor. |
| `maxZoom` | `number` | `20` | Maximum zoom factor. |
| `className` | `string` | — | Extra classes on the root wrapper. |

---

Source: https://viz.dreamlake.ai/dataset-viz/spec

# The `.dreamrc` file

**One `.dreamrc` at the root of a dataset renders every episode in it.** The file
answers three questions — where the episodes are, how to parse them, and how to
lay out the visualization — and nothing else. It is plain YAML, never contains
credentials, and works unchanged on any storage backend. Why the system is
shaped this way — views with declared input contracts, meaning stated in the
config — is [the architecture](/dataset-viz/overview.md); this page is the
grammar.

```yaml
# .dreamrc — at the dataset root
version: 1
name: Kitchen Manipulation v2   # optional display name

dataset:                        # ── data entry + parsing ──
  format: lerobot               # lerobot | folder | umi | mcap
  episodes: auto                # ask the format adapter (default)

views:                          # ── visualization: compose views ──
  - view: videoStack
    cameras: ["observation.images.*"]
  - view: lineChart
    series:
      - { field: [action, "*"] }
```

That is a complete, working file. Note what it does **not** say: where the
dataset lives. A `.dreamrc` at its dataset's root inherits the storage it sits
in — the app injects it. A *standalone* file (a demo, a config pointing at
another bucket) adds one more block, `storage:` — see
[below](#storage--where-the-dataset-lives). Everything else is the reference
for the blocks.

<a
  href="/dataset-viz/gallery"
  style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'space-between',
    gap: 16,
    margin: '18px 0 6px',
    padding: '14px 18px',
    borderRadius: 12,
    background: 'color-mix(in srgb, currentColor 6%, transparent)',
    textDecoration: 'none',
    color: 'inherit',
  }}
>
  <span>
    <span style={{ display: 'block', fontWeight: 650, fontSize: 15 }}>
      Gallery — 10 real datasets, one grammar
    </span>
    <span style={{ display: 'block', fontSize: 13, opacity: 0.65, marginTop: 2 }}>
      LeRobot v2/v3 · depth maps · point clouds · UMI zarr · MCAP · annotations + 3D —
      a full-screen two-pane switcher of complete .dreamrc files
    </span>
  </span>
  <span style={{ fontSize: 20, opacity: 0.55, flexShrink: 0 }}>→</span>
</a>

## Live example

One file exercising most of the spec at once — glob enumeration,
auto-discovered annotation tracks (COCO hand keypoints drawn over the video,
WebVTT subtask cues on a timeline), and a real 3D hand–object reconstruction
(glTF geometry + per-frame parquet tracks) driven by the shared cursor. It
uses the `folder` format on
purpose: the **zero-convention layout** — any files you can put in folders,
no conversion, the easiest dataset there is to prepare. An *existing*
LeRobot / zarr / MCAP dataset needs an even shorter file (`format:` +
`episodes: auto` — see the gallery). The source pane is the complete
standalone `.dreamrc`; copy it and it runs anywhere:

## `dataset:` — which episodes, parsed how

`format` names the dataset format; `episodes` says how to enumerate the episodes.
Both live here and nowhere else.

**Which format is mine?** List the dataset root and match the signature:

| you see at the root | `format` | `episodes` |
| --- | --- | --- |
| `meta/info.json` | `lerobot` | `auto` |
| a `*.zarr.zip` or a `.zarr/` directory | `umi` | `auto` |
| `*.mcap` files | `mcap` | `auto` |
| one folder per recording | `folder` | a glob, e.g. `"episodes/*/"` |

If none matches, `folder` is the escape hatch: it asks nothing of the layout
beyond one directory per episode, and reads whatever standard files it finds.
HDF5 (RoboMimic, ManiSkill, AgiBotWorld) and RLDS/TFRecord (Open X-Embodiment)
have no adapter yet — most publishers also ship a LeRobot export, which does.

| key | values | meaning |
| --- | --- | --- |
| `format` | `lerobot` \| `folder` \| `umi` \| `mcap` | the format adapter. One name per format — no aliases. |
| `episodes` | `auto` (default) | ask the format adapter. Container formats know their own episode count (LeRobot `meta/info.json` `total_episodes`, zarr `episode_ends`). |
| | `"episodes/*"` | a glob over storage paths — one episode per match. A trailing `/` matches directories only. Only `*` is supported, one path segment per star. |
| | `{ glob, sort?, limit? }` | glob with options. `sort`: `name` (numeric-aware, default) \| `name-desc` \| `none`. `limit`: cap per pattern (default 1000). |
| `annotations` | `{ <track>: <path> \| { path, kind? } }` | extra annotation tracks merged into every episode's catalog — see below. `kind` is the author stating the payload once in the declaration instead of at every binding. |
| `labels` | `{ "<name or glob>": "Display name" }` | rename a field, or one dimension of one (`"observation.state[3]"`), when the dataset's own names are serial numbers, `null`, or absent. |

Use `auto` for container formats (`lerobot`, `umi`) and a glob for
folder-per-episode layouts (`folder`):

```yaml
dataset:
  format: folder
  episodes: "episodes/*/"       # each matched folder is one episode
```

In glob mode the matched path is handed to the adapter as the episode root —
you never write per-episode config. `{ limit }` on its own (no glob) caps what
a container format enumerated, which is how you preview a 300-episode dataset.

**Writing `views:` for a dataset you have not seen** — start with
`view: fieldsCatalog` and nothing else. It prints the inventory: every
field's address plus the `dtype`, `shape` and `names` the container reported,
and no conclusion drawn from them. Reading that listing is where the judgment
happens — you are the one who knows that `observation.environment_state` is a
point cloud — and the bindings you write next are where it gets recorded.

### `dataset.labels` — names the container got wrong

A camera keyed by its serial number, a 14-dim state whose `names` is `null`:
the data is right and only the label is unreadable. Patterns match field names
(or a `feature[dim]` address), first match wins:

```yaml
dataset:
  format: lerobot
  labels:
    "observation.images.cam_035622060973": Front camera
    "observation.state[3]": wrist_flex
```

`labels` changes what is **displayed** and nothing else; bindings still use the
real names. There is no companion key for meaning — nothing to correct, because
nothing was guessed. What a field is gets stated where it is bound.

### `dataset.annotations` — tracks that live beside the data

Annotations belong **inside** your dataset's container — see
[what your data must look like](/dataset-viz/requirements.md#annotations-belong-inside-the-container).
This block is for the two cases that cannot: static geometry, which no
container models, and a dataset you cannot write into.

```yaml
dataset:
  format: lerobot
  annotations:
    scene: "recon/scene.glb"     # a path, nothing more
```

The merge happens after the format adapter runs, so a declared track lands in
the same catalog as the dataset's own fields, and a declaration **overrides** a
native track of the same name. Paths, formats and the rest:
[files beside the data](/dataset-viz/requirements.md#files-beside-the-data).

## `views:` — compose views

Each entry names a **view** from the registry and binds fields to it.
Everything that is not a binding key is passed through to the view as
props — each view's reference section lists what it accepts. You own the
layout: nest `split` nodes to build any arrangement.

```yaml
views:
  - view: videoStack
    cameras: ["observation.images.*"]  # binding — read as video
    overlays:
      - { field: observation.keypoints_2d.left.ego, as: keypoints }
    columns: 2                         # passthrough prop
  - view: timeline
    tracks: [{ field: subtask_index, as: segments }]   # joins its label table
  - split: row                         # layout node: row | column | grid
    children:
      - view: lineChart
        series:
          - { field: [action, left_waist], label: waist · cmd }
          - { field: [observation.state, left_waist], label: waist · actual, dash: "3 2" }
      - view: lineChart
        series: [{ field: [action, right_waist] }]
```

Rules, all of them:

- **One binding style per view, and the slot's name says what it takes.**
  Camera views (`videoStack`, `frameStack`, `depthStack`) bind `cameras`,
  `lineChart` binds `series`, `timeline` binds `tracks`, `pointCloud` binds
  `cloud`; camera views also take `overlays`, and `recon3d` takes both
  `geometry` (the glTF) and `tracks` (motion). A bare string in
  `series`/`tracks`/`overlays` is shorthand for `{ field }`. A slot a view
  does not read is an error, never ignored.
- **The slot says what the bytes become.** Binding a field to a slot is what
  decides how it is decoded — `series` reads numbers as traces, `tracks` on
  `timeline` reads them as spans, `cloud` on `pointCloud` reads them as a
  cloud. When the field's addressing kind leaves the slot only one possibility,
  nothing is written. When it leaves several, the entry says which with `as`
  (`overlays` draws a skeleton or captions; a `.json` could be either), and
  until it does the panel refuses and prints the choice
  ([the rule](/dataset-viz/reference.md#which-payloads-a-field-can-serve--and-when-you-write-as)).
- **Field references** are `"feature"` or `[feature, dim]` — a feature name
  (never split on dots: `observation.state` is one name), optionally drilled
  into one named dimension. One glob rule: `*` matches within a name.
- **Layout** is `split: row | column | grid` with `children`; nest freely.
  `grid` accepts `columns`. A **`row` is a fixed-height strip** (`height`,
  default 280): the layout never reflows as media loads — extra width
  overflows into a horizontal scrollbar instead, and that scroll **syncs
  across episodes** (the list renderer wraps episodes in
  `SyncScrollProvider`). Per child you choose what to keep: nothing —
  keep the row height (media takes its aspect-derived width, everything
  else stretches to share the leftover, `flex` weighting it and `minWidth`
  flooring it); `width` — a fixed-width box; `height` — that child's own
  strip height.

`views` is **required**. There is no default layout: a dataset with no views
is one nobody has described yet, and inventing an arrangement for it would be
the program deciding what its data means. To find out what there is to bind,
resolve the dataset with no config and read the inventory — `check-dreamrc`
prints every field with its `dtype` and `shape`.

### The views

The initial registry — it grows over time, and a host app can register its own
with `registerComponent(spec)`:

| view | renders | slot → payload it asks for |
| --- | --- | --- |
| `videoStack` | camera videos as a tile grid, with overlay support | `cameras` → `video` \| `image` · `overlays` → `keypoints` \| `segments` |
| `frameStack` | per-frame image sequences (chunked cameras) | `cameras` → `frames` · `overlays` → `keypoints` \| `segments` |
| `lineChart` | time series, styled per-dim traces, synced cursor | `series` → `series` |
| `timeline` | ruler + labelled track blocks | `tracks` → `segments` |
| `metaPanel` | episode name / duration / task strings header card | — (`note` prop) |
| `fieldsCatalog` | the episode's inventory as a table | — |
| `recon3d` | animated 3D scene: glTF geometry moved by per-frame tracks, plus point sets — orbit + cursor-driven playback | `geometry` → `mesh3d` · `tracks` → `transform3d` \| `vertices3d` \| `pose3d` |
| `depthStack` | per-frame depth maps, turbo-colorized | `cameras` → `depth` · `overlays` → `keypoints` \| `segments` |
| `trajectory2d` | planar series as a top-down xy path | `series` → `series` |
| `bandTrack` | discrete series as categorical color bands | `series` → `series` |
| `pointCloud` | per-frame 3D point clouds, orbitable | `cloud` → `pointcloud` |

Every slot, its payload, and the shape that payload needs:
[reference](/dataset-viz/reference.md#view-components).

## `storage:` — where the dataset lives

Every path in the file is **relative to the dataset root**; `storage:` says
where that root is. It is optional, and *who writes it* is the design:

- **At the dataset root, omit it.** The app that found the file injects the
  storage it sits in — a DreamLake source, a project folder, any browsed
  directory. This is the normal, uploaded form: the file never repeats what
  its own location already says, and moving the dataset never breaks it.
- **Standalone files declare it.** A docs example, a demo gallery, a config
  that points at another bucket — anything not sitting at its data's root
  names the root explicitly. Every live example on this page is this form:
  copy the YAML and it resolves the same data anywhere.
- **A declaration wins.** If a file with `storage:` is opened inside the app,
  it renders as written (same rule as `dataset.annotations`: an explicit entry
  is user intent). Omission — not override — is what makes a file portable.

```yaml
storage: { driver: hf, repo: lerobot/pusht }     # public HuggingFace repo
storage: { driver: http, url: https://my-cdn.example.com/kitchen-v2 }
```

Built-in drivers are `http` (`url` — the dataset root URL) and `hf` (`repo`,
plus optional `root`, `revision`, `repoType`) — both credential-free. Host
apps register more: DreamLake registers `dlSource` and `dlProject`, whose
configs carry **identifiers only** — a `.dreamrc` never contains credentials;
drivers that need auth get their tokens from the host at registration time.
Per-driver keys: [reference](/dataset-viz/reference.md#storage-drivers).

The whole storage contract is two methods — `list(path)` and `resolveUrl(path)`
— which is why any backend can be a root.

## Validation

`validateDreamrc(parsed)` checks the file before anything renders and throws
errors written to be fixed mechanically — each names the offending key, the
allowed values, and the episode where expansion failed. Typical messages:

```
.dreamrc: dataset.format 'lerobot3' is not a registered format (available: lerobot, folder, umi, mcap)
.dreamrc: views[2].view 'lineChart2' is not registered (did you mean 'lineChart'?)
.dreamrc: episodes glob "episodes/**" — '**' is not supported, use one '*' per path segment
.dreamrc declares no 'storage:' and no host root storage was supplied — standalone files need storage: { driver, … }
```

## TypeScript API

For hosts and tests — the library is credential-free and YAML-free (parse
upstream, pass the object):

```ts

const rc = validateDreamrc(parseYaml(text))
// A self-contained file (declares storage:) resolves alone; for a file found
// at a dataset root, inject that root — the file's own storage: would win.
const { episodes, warnings } = await resolveDataset(rc, {
  rootStorage: { driver: 'http', url: 'https://…/my-dataset' },
})
// episodes: ResolvedEpisode[] — id, name, meta, and a ready-to-render handle
// <DatasetViz episode={episodes[0]} views={rc.views} />
```

---

Source: https://viz.dreamlake.ai/schema-viz/overview

# Schema viz

`@dreamlake/viz/schema-viz` renders a **dataset** as a **synchronized,
multi-panel visualization** from a small **schema**. You describe _what_ you want
to look at; it fetches lazily and draws — videos, charts, and timelines that
scrub together.

## Quick start

Hand `` a parsed schema. This one points the built-in `http`
storage at a public LeRobot dataset and **omits `panels`** — so viz
**auto-lays-out** the whole episode: a camera stack, a task timeline, and one
chart per numeric field. The entire program:

```tsx file="AutoLayoutSpec.tsx"
// Omit `panels` entirely → viz auto-lays-out the source: a camera stack, a task
// timeline, and one chart per numeric field. The smallest possible schema that
// still produces a full view — the "I don't know this dataset yet" workflow.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      adapter: 'lerobot',
      storage: {
        driver: 'http',
        basePath: 'https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main',
      },
      episode: 0,
    },
  },
  // no `panels` → auto-layout runs against `ep`
}

export const AutoLayoutSpec = () => <DatasetPreview schema={schema} />
```

A schema has two parts:

- **`sources`** — each names an **[adapter](/schema-viz/adapters.md)** (_what format
  the data is_) and its **[storage](/schema-viz/storage.md)** (_where the bytes live_).
- **`panels`** — each is a **[view](/schema-viz/views.md)** (`videoStack`, `lineChart`,
  `timeline`) over some fields. Omit `panels` and viz **auto-lays-out** the dataset.

That is the entire surface area. Everything else is choosing the right adapter,
storage, and views — and, when you need it, writing your own.

## How to read these docs

1. [Schema](/schema-viz/schema.md) — write a schema: sources, panels, field
   binding, and the auto-layout you get when you omit panels.
2. [Storage](/schema-viz/storage.md) — point at the bytes: the public `http` driver,
   and how a host app injects an **authorized** driver for private data.
3. [Adapters](/schema-viz/adapters.md) — which adapter for which dataset
   (LeRobot, Zarr/UMI, egocentric, loose folders), or write your own. Start
   here if you just want to point at a dataset.
4. [Views](/schema-viz/views.md) — the built-in panels, their options, writing your
   own panel, and how auto-layout works.
5. [Concept](/schema-viz/concept.md) — a short read on _why_ it is built in four
   layers. Skip it until you are curious.

---

Source: https://viz.dreamlake.ai/dataset-viz/views

# View components

**Every registered component, each with a minimal `.dreamrc` and its live
render** — this page answers "what do I *get* if I bind this?". Config keys
and the payload each slot asks for are in the
[reference](/dataset-viz/reference.md#view-components);
full multi-component compositions in the [gallery](/dataset-viz/gallery.md).
Each YAML below is complete and standalone — one component, one real public
dataset, first episode only. **Interaction rule**: every component whose
x-axis is time (videos, frames, depth, charts, bands, the timeline) scrubs
the shared cursor on hover — hover any demo to move it. The 3D views leave
the pointer to orbiting, so their demos pair a chart or timeline sibling as
the time source.

## videoStack

Camera videos as a tile grid, each tile at its video's own aspect ratio.
`overlays` is the one slot that draws two different things, so each entry
says which with `as` — here a COCO file `as: keypoints` becomes a hand
skeleton and a WebVTT file `as: segments` becomes captions:

## frameStack

Per-frame image sequences (cameras stored one frame per chunk) — each tile
byte-ranges ONLY the frame under the shared cursor; scrub to step. JPEG-XL
frames need Safari 17+ or Chrome's JXL flag:

## depthStack

Per-frame depth maps colorized on the fly — turbo by default (`colormap: gray`
for grayscale), each frame mapped over its own valid min/max unless pinned
with `min`/`max`; 0/invalid readings stay transparent. The corner chip shows
the mapped range in metres when the format knows the depth scale:

## pointCloud

Per-frame 3D point clouds as an orbitable scene — per-point color when the
data carries rgb, camera auto-fit from the first frame, playback follows the
shared cursor. Default `up: z` (robot-lab convention); `up: y` for y-up
clouds:

## lineChart

Time series with a synced cursor — one `series` entry per trace,
`[feature, dim]` drills into one dimension, `label` / `color` / `dash`
style it:

## trajectory2d

Planar series as a top-down xy path — for 2-dim position series (pusht's
`action` target position) a spatial path reads far better than a line chart.
Hover snaps the shared cursor to the nearest sample; the thick trail is the
last 1.5 s; `invertY: false` flips to math convention:

## timeline

Anything you bind here is read as `segments` — tasks, subtasks, actions,
phases, a `.vtt` file, an index column with its label table — and drawn as
labelled blocks on a ruler. Hover to scrub every panel in the episode:

## bandTrack

Discrete series (gripper open/close, stage indices, success flags) as
categorical color bands — one row per bound column, one colored rect per
contiguous equal-value run, a value→color legend below. Columns busier
than `maxLevels` (12 distinct values) get a one-line "use lineChart"
note instead of a band:

## metaPanel

The episode header: name, duration, frame count, fps, the dataset's task
strings — plus a free-text `note`:

## fieldsCatalog

The episode's inventory as a table — every field's address plus the `dtype`,
`shape` and `names` the container reported, with nothing concluded from them.
The exploration component: ship it first when you don't know what a dataset
holds, decide what the columns are, write the bindings, then replace it:

## recon3d

The animated 3D scene, bound through two slots: `geometry` is the static
scene, and `tracks` is the per-frame motion that moves it — a track binds to the glTF
node whose name matches its ref. All three track kinds are parquet tables of
numbers, so each entry names the one it is. Orbit with the mouse; playback
follows the shared cursor. The motion trail is the **future** — the next
`trail.ahead` seconds (default 1) of each object's path, and nothing more:
it runs out exactly when the clip does. In robot learning the question at
time t is what is about to happen, so that is the segment that glows
(`trail.behind` opts into a dim past tail):

---

Host apps can grow this registry — `registerComponent({ name, component })`
makes a new name available to every `.dreamrc` the app renders
([TypeScript API](/dataset-viz/spec.md#typescript-api)).

---

Source: https://viz.dreamlake.ai/dataset-viz/requirements

# What your data must look like

This page is the **data side** of the
[contract](/dataset-viz/overview.md): what your container must declare, the
shape each payload demands, and where annotation tracks live. The program
never decides what your data means — your `.dreamrc` does
([the architecture](/dataset-viz/overview.md)) — so the requirements are much
smaller than a list of naming rules. There are only two:

1. **The container must say which bytes are encoded media**, so they can be
   addressed a frame at a time instead of being read as numbers. Every format
   already does this — LeRobot's `dtype`, a zarr array's codec, an MCAP
   channel's schema, a file's extension.
2. **A column bound to a view must have the shape that view decodes.** Ask
   for a skeleton and the numbers must be `[J,2]` or `[J,3]`. That is not a
   rule we impose; it is what a skeleton is.

Everything else — which camera a hand belongs to, whether a `[7]` column is a
pose or a gripper command, what a `.json` beside your video contains — is
stated in the `.dreamrc`. You never rename a feature to make it render.

## Which datasets can be read

Four formats, named in `dataset.format`. Each is somebody else's
specification, read as published:

| `format` | the dataset is | episodes come from | how you recognize yours |
| --- | --- | --- | --- |
| `lerobot` | a [LeRobot](https://huggingface.co/docs/lerobot/main/en/lerobot-dataset-v3) dataset, v2.0 / v2.1 / v3.0 | `meta/episodes` (v3) or `meta/episodes.jsonl` (v2) | there is a `meta/info.json` at the root |
| `umi` | a [Zarr](https://zarr-specs.readthedocs.io/) store — a v2 `.zarr.zip` ReplayBuffer or a v3 `.zarr/` directory | `meta/episode_ends`, or the store's attributes | there is a `*.zarr` or `*.zarr.zip` |
| `mcap` | [MCAP](https://mcap.dev/) v1 logs, one file per episode | one `*.mcap` file each, listed or globbed | there are `*.mcap` files |
| `folder` | no container at all — directories of files | a glob you write, e.g. `"episodes/*/"` | none of the above; one directory per recording |

`folder` is the escape hatch and asks nothing of the layout beyond one
directory per episode. HDF5 (RoboMimic, ManiSkill, AgiBotWorld) and
RLDS/TFRecord (Open X-Embodiment) have no adapter yet — most publishers also
ship a LeRobot export, which does. A format we do not read gets an adapter,
not a conversion demand
([the extension path](/dataset-viz/reference.md#data-the-library-does-not-know)).

## What the viewer sees before you configure anything

The catalog is an **inventory**, not a classification. Point the checker at
any dataset and it lists what exists, with the facts the container reported
and no conclusions drawn from them:

```
episode_000000  (7 fields)
  video    observation.images.ego        h264 1080×1920
  tensor   observation.keypoints_2d.left.ego   float32 [21,3]
  tensor   observation.state             float32 [6]  names: shoulder_pan.pos, …
  tensor   subtask_index                 int64 [1]
  text     language_instruction
  file     recon/scene.glb               ext: glb
```

The kinds describe **how the bytes are addressed**, never what they mean:

| kind | means |
| --- | --- |
| `video` | an encoded stream, seekable |
| `frames` / `image` | encoded images, one per frame or one file |
| `tensor` | numbers, with `dtype` and `shape` as the container reported them |
| `text` | strings |
| `file` | an address, with its extension |

**That listing is what you write the config against** — it is the raw truth,
and an agent authoring a `.dreamrc` reads exactly this. Binding a field to a
view slot is what states its meaning, and asking for something the bytes
cannot become fails by name
(`observation.state is float32 [6]; keypoints needs [J,2] or [J,3]`) instead
of rendering a guess.

## What each payload needs

The shape requirements, in full. This is the whole contract on the data side:

| ask for | the column must be | notes |
| --- | --- | --- |
| `series` | any numeric scalar or `[n]` | `names` in the container label the traces |
| `keypoints` | float `[J,2]` or `[J,3]` | third component is a score; **NaN, not 0, for not-measured** |
| `segments` | an int column **plus a label table**, or a string column, or a `.vtt` / `.srt` | equal consecutive values merge into one span |
| `depth` | float `[H,W]` or `[H,W,1]` | metres or millimetres — declare the scale |
| `pointcloud` | float `[N,3]` or `[N,6]` | xyz, or xyz + rgb |
| `transform3d` | float `[7]` or `[N,7]` | translation + quaternion |
| `vertices3d` | float `[V,3]` | topology comes from a bound glTF node |
| `pose3d` | float `[J,3]` or a flat `[J*3]` | a point set per frame |
| `frames` | encoded image bytes per row, or a per-frame path template | |
| `mesh3d` | a `.glb` / `.gltf` / `.obj` | node names bind per-frame tracks |

> **Note:** Zero is a real coordinate — the top-left pixel, the origin. Nothing downstream
> can tell a fabricated zero from a measured one, so a frame written as zeros
> draws a collapsed skeleton in the corner. Write NaN and it renders as nothing,
> because it is nothing.

## Annotations belong inside the container

A payload's preferred home is **inside** the dataset, expressed in the
container's own idiom: a keypoint feature sits in the same parquet as your
state and action columns, gets read by episode row range, and travels with
the dataset.

> **Note:** Measured on our own template: 150 frames of two hands cost **57.7 KB** as
> parquet columns and **142.3 KB** as a JSON file next to them — and the
> columns can be read one episode at a time while the file must be fetched and
> parsed whole. One container, one round trip, no second thing to keep in sync.

What that looks like per container, and the one thing each must get right:

**LeRobot** — `meta/info.json` `features` gives every column its `dtype`,
`shape` and `names`, and those appear verbatim in the listing. The one thing
to get right: **cameras must be `dtype: video` or `dtype: image`**, so they
are addressed as media. Episode boundaries come from `meta/episodes` — v3
packs many episodes into one parquet and one mp4 per camera, and each
episode's row range and video window are read from there. A labelled span is
an index column plus a label table (`meta/<name>s.jsonl` — LeRobot's own
`task_index` + `meta/tasks.jsonl` pattern, one level down); hand keypoints
are a `[J,2]` or `[J,3]` float feature like any other.

**MCAP** — every channel is listed with its schema name and encoding.
Foxglove schemas (`PointCloud`, `FrameTransform`, `CompressedImage`,
`SceneUpdate`, …) have decoders; `cdr` / `ros2msg` / `ros1msg` do not, so a
plain ROS 2 bag currently yields nothing.

**Zarr** — an array's codec says whether it holds encoded images or numbers.
Episodes come from `meta/episode_ends` or the store's attributes.

**A folder of files** — one directory per episode; a file's basename is its
track name; `annotations/` is searched. Three placement conventions, and none
of them says what a file contains.

## Files beside the data

Two cases genuinely cannot go inside, and only two:

1. **The container cannot model it.** Static geometry is the real example —
   there is no LeRobot feature shape for a mesh.
2. **The dataset is not yours.** A public mirror you cannot write into, so
   any extra tracks (and the config itself) have to live elsewhere.

For those, put a file beside the data **in an established format** and name
it in the config. The original dataset is never modified — delete the extra
files and it is untouched.

### Declaring a track

`dataset.annotations` maps a track name to a path. The merge happens *after*
the format adapter runs, so a declared track lands in the same catalog as the
dataset's own fields and binds the same way:

```yaml
dataset:
  format: lerobot
  episodes: auto
  annotations:
    subtasks: "annotations/subtasks/episode_{episode_index:06d}.vtt"
    hands:    "annotations/hands/episode_{episode_index:06d}.json"
    scene:    "recon/scene.glb"
```

- **Paths** are relative to the dataset root; templates use the same
  `{var}` / `{var:06d}` style LeRobot itself uses. Variables:
  `episode_index`, `episode_name`, `episode_path` (glob mode).
- **A declaration is an address, not a claim.** The file lands in the catalog
  as `file` with its extension recorded, and the view that binds it says what
  to make of it. The extension gets its say at read time, where all it picks
  is a PARSER: asked for `segments`, a `.vtt` goes through the WebVTT reader
  and a `.json` through the COCO one.
- **`{ path, kind }` is you saying it once, in the declaration** instead of
  at every binding — legitimate because it is the author speaking, not the
  library deducing. A binding's own `as:` still wins, and either way the
  decoder checks the bytes and fails loudly when they cannot produce what was
  asked.
- **A declaration overrides a native track of the same name.** That is user
  intent: replacing a dataset's coarse task segments with a refined
  re-annotation is exactly what this is for. Undeclared native tracks stay.
- The `folder` format auto-discovers anything in each episode's
  `annotations/` folder, so declaring is never required when files live
  there.

### Which format to write

Use the standard that already exists for the job — none of these is ours:

| for | write | spec |
| --- | --- | --- |
| labelled time spans | **WebVTT** `.vtt` (or SubRip `.srt`) | [W3C](https://www.w3.org/TR/webvtt1/) |
| 2D keypoints | **COCO keypoints** `.json` | [COCO](https://cocodataset.org/#format-data) |
| geometry, and its motion | **glTF 2.0** `.glb` | [Khronos](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html) |
| per-frame numbers | **Apache Parquet** | [Parquet](https://parquet.apache.org/docs/file-format/) |

**Time spans — WebVTT**, the web's own "time range → text" format:

```text file="subtasks.vtt"
WEBVTT

1
00:00:00.000 --> 00:00:02.500
pick up shelf board
```

**2D keypoints — COCO**, stock, plus a top-level `fps` because COCO indexes
images and a player needs seconds (our one documented extension to COCO):

```json file="hands.coco.json"
{
  "fps": 30,
  "images": [{ "id": 0, "file_name": "frame_000000.jpg", "width": 1920, "height": 1080 }],
  "annotations": [
    { "id": 1, "image_id": 0, "category_id": 1,
      "keypoints": [473.4, 714.2, 2, 532.2, 710.5, 2],
      "bbox": [418, 524, 277, 249], "score": 0.79 }
  ],
  "categories": [{ "id": 1, "name": "hand", "keypoints": ["wrist", "thumb_cmc"],
                   "skeleton": [[1, 2]] }]
}
```

Every key is COCO's, including the 1-based `skeleton` pairs. A frame with no
detection simply has **no annotation** — absence, not a zero coordinate.

**Geometry and motion — glTF.** Export one `.glb` with your objects as
**named nodes**: the node name is the join key, so a node called `ruler`
binds the per-frame pose track named `ruler`, and no side file describes the
relationship. **If the objects move, animate the glTF** — per-node
translation/rotation/scale channels with their own keyframe times are already
standardized, so one animated `.glb` carries geometry AND motion in a file
Blender, three.js and every other glTF tool reads. (The first clip's TRS
channels are read; skinning, morph targets and multiple clips are not yet.)

**Per-frame numbers — Parquet**, when the motion is not in a glTF:

| track | columns |
| --- | --- |
| object poses | `frame`, `timestamp`, `object`, `tx,ty,tz` + a quaternion — the `object` column splits it into one track per object |
| mesh vertices | `frame`, `timestamp`, and a **list-of-float** column of flat xyz |

Quaternion order is read from the COLUMN ORDER: `qw` first means `w,x,y,z`,
`qw` last means `x,y,z,w`. These column names are a **decoder's**
requirement, consulted only once a binding has asked for `transform3d` or
`vertices3d` — never evidence about what a file holds.

> **Note:** Those column names are ours — Parquet is Apache's, but "`tx,ty,tz` plus a
> quaternion plus a grouping column" is not written down anywhere else, so we
> define it deliberately: the **motion-track Parquet profile v1**, currently
> the one self-defined on-disk structure in the spec
> ([when we define our own](/dataset-viz/overview.md#where-the-bytes-come-from--the-standards-ladder)).
> An animated glTF carries the same information inside an existing standard
> and opens in Blender and three.js; the profile is the columnar route when
> your pipeline is already writing Parquet. Pick whichever it produces more
> naturally.

## Cameras: the encoding that actually matters

The single most common cause of a sluggish dataset, and it applies to every
container. Scrubbing means seeking, and a seek decodes forward from the
previous keyframe — keyframes 10 seconds apart feel stuck no matter how fast
the machine is:

```bash file="transcode.sh"
ffmpeg -i raw.mp4 \
  -c:v libx264 -pix_fmt yuv420p \
  -g 30 -keyint_min 30 -sc_threshold 0 \   # a keyframe every ~1 s at 30fps
  -movflags +faststart \                   # index up front — playback starts immediately
  out.mp4
```

H.264 + `yuv420p` decodes everywhere; HEVC, AV1 and MPEG-4 Part 2 do not.
**Independent frames scrub better than any video** — one fetch per frame, no
decode chain, exact at any file size.

## Verify

```bash
npx tsx scripts/check-dreamrc.mts hf:your-name/your-dataset   # a Hub dataset
npx tsx scripts/check-dreamrc.mts https://bucket/…/my-dataset # any object storage
npx tsx scripts/check-dreamrc.mts ./draft.dreamrc             # before you upload
```

It resolves the dataset exactly as the app does: reads the `.dreamrc`,
decodes every bound field, and prints what came back. With no config it
prints the inventory instead — which is how you find out what to write.

## Checklist

- [ ] Cameras are declared as media by the container (`dtype: video` /
      `dtype: image` / an image codec / a media extension)
- [ ] Videos are H.264 `yuv420p`, `+faststart`, keyframes ≈ 1 s apart
- [ ] Every column you intend to render has the shape its payload needs
- [ ] Not-measured is NaN, not zero
- [ ] Span index columns have their label table
- [ ] Annotations live inside the container where it can hold them; sidecar
      files are established formats, declared in `dataset.annotations`
- [ ] A `.dreamrc` at the dataset root, no `storage:` block — the app injects
      it ([spec](/dataset-viz/spec.md#storage--where-the-dataset-lives))
- [ ] Resolved once from a shell, with every binding reporting the payload you
      expected

---

Source: https://viz.dreamlake.ai/dataset-viz/reference

# Reference

Every name a `.dreamrc` can use, with its config keys. The
[architecture](/dataset-viz/overview.md) is why the system has this shape, the
[spec](/dataset-viz/spec.md) is the grammar, and the
[contract](/dataset-viz/requirements.md) says what your data must look like; this
page is the lookup table — the names here are exactly the registries the
validator checks, so anything not listed fails validation with a did-you-mean.
For the pipeline and the registries in code, see
[library internals](/dataset-viz/internals.md).

## Kinds — the catalog's, and the view's

Both say "kind" about the same field, and they mean different things. Keeping
the two apart is the whole design.

### Inventory kinds — what the catalog states

`Field.kind` says how the BYTES ARE ADDRESSED and nothing about what they
mean. Six values, closed, and nothing turns a name or a shape into one of them:

| kind | the container said | `meta` carries |
| --- | --- | --- |
| `video` | an encoded stream, seekable | `codec` / dimensions when reported |
| `frames` | encoded images, one per frame | `count`, `fps?`, `ext` |
| `image` | one encoded image | `ext` |
| `tensor` | numbers | `dtype`, `shape`, `names` — verbatim |
| `text` | strings | |
| `file` | an address, and nothing claimed | `ext` |

For a loose file the extension picks the addressing and only that: `.mp4` /
`.webm` → `video`, `.jpg` / `.png` → `image`, `.csv` / `.parquet` → `tensor`.
**`.vtt`, `.glb` and `.json` are all `file`** — "labelled spans", "geometry"
and "keypoints" are meanings, and meanings come from the `.dreamrc`.

`meta` is raw facts, recorded because they are free: `dtype`, `shape`, `names`,
`ext`, `schema`, `codec`. A decoder reads them to do its job — the schema name
selects an MCAP decoder, the extension selects a parser — and nothing reads
them to decide what a field is.

### Payload kinds — what a view asks for

`read(ref, { as })` decodes into one of these. The `as` comes from the slot the
field was bound to, so the interpretation is written in the `.dreamrc`:

| payload | what comes back | the field must be |
| --- | --- | --- |
| `video` | `{ url, window? }` — streamable; `window` is this episode's span of a shared file | a `video` field |
| `image` | `{ url }` | an `image` field |
| `frames` | `{ count, fps?, frameAt(i) }` — lazy per-frame fetch | a `frames` field |
| `series` | `{ timestamps, columns }` — per-dim numeric traces | any numeric scalar or `[n]`; `names` label the traces |
| `segments` | `{ segments: [{ start, end, label }] }` — any time-range → label | an int column **plus its label table**, a string column, or a `.vtt` / `.srt` |
| `keypoints` | `{ keypoints }` — pixel space, fps, sparse frames, skeleton | float `[J,2]` / `[J,3]`, or a COCO `.json` |
| `depth` | `{ count, fps?, at(i) }` — raw values, never pre-colorized | float `[H,W]` or `[H,W,1]`, or `frames` of encoded 16-bit PNGs |
| `pointcloud` | `{ count, fps?, at(i) }` | float `[N,3]` or `[N,6]` (xyz, or xyz + rgb) |
| `transform3d` | `{ timestamps, values, layout }` — position + quaternion | float `[7]` or `[N,7]` |
| `vertices3d` | `{ count, fps?, vertexCount, at(i) }` — lazy | float `[V,3]` |
| `pose3d` | `{ timestamps, joints, shape }` — per-frame point sets | float `[J,3]` or a flat `[J*3]` |
| `mesh3d` | `{ url, format }` — static geometry; its node names bind the tracks above | a `.glb` / `.gltf` / `.obj` |
| `file` | `{ url, ext? }` — an address a component parses itself | anything |

Ask for one the bytes cannot become and the read throws naming both sides
(`observation.state is float32 [6]; keypoints needs [J,2] or [J,3]`) instead of
drawing something wrong.

### Which payloads a field can serve — and when you write `as:`

The bridge between the two tables. It reads the addressing kind and nothing
else — no name, no shape, no extension:

| addressed as | can be read as |
| --- | --- |
| `video` | `video` |
| `image` | `image` |
| `frames` | `frames`, `depth` — depth shipped as 16-bit PNGs is a codec question, not a meaning |
| `tensor` | `series`, `keypoints`, `pose3d`, `transform3d`, `vertices3d`, `depth`, `pointcloud` |
| `text` | `segments` |
| `file` | `file`, `mesh3d`, `segments`, `keypoints` |

Intersect a field's row with the payloads its slot reads
([per component](#view-components)):

- **exactly one survives** — nothing to write; the container already settled it
  (a `video` bound to `videoStack.cameras`, a `tensor` bound to `pointCloud`);
- **more than one** — the binding says which with `as:`, and until it does the
  panel refuses and prints the choice (a `.json` bound to `overlays` is a
  skeleton file or a caption track, and nothing about the bytes says which);
- **none** — that slot cannot show that field, and it says so by name.

A declared `as:` short-circuits the table entirely: it is the author speaking,
and it is how a LeRobot `subtask_index` is bound `as: segments` and joins its
label table.

Where a payload comes from — a feature in the dataset's own container, or a
standard file beside it (WebVTT/SRT, COCO, glTF, parquet) — is in the
[contract](/dataset-viz/requirements.md#what-each-payload-needs).
**No payload requires a format of our own** — where a DreamLake structure
exists (the motion-track Parquet profile v1), an existing standard carries
the same payload.

### Data the library does not know

The payload set versions with the library, but shipping your own end to end
needs no library change:

1. **Declare where it is** —
   `dataset.annotations: { gaze: "annotations/gaze_{episode_index:06d}.json" }`.
   The field lands in every episode's catalog as a `file` with its `ext` in
   `meta`. Nothing is claimed about the contents.
2. **Render it** — the host registers a component
   (`registerComponent({ name: 'gazePanel', component, reads: { fields: ['file'] } })`)
   whose slot reads `file`, fetches the URL and parses its own wire format.
   The `.dreamrc` writes `view: gazePanel` and binds the field — no
   schema change, no library change.
3. **Promote it** — once a payload proves general (the way `segments` covers
   tasks/subtasks/actions/phases), it graduates into the library: a `Payload`
   member, a decoder that accepts the wire shapes found in the wild (see
   `normalizeSegments`), and a built-in component with a slot that reads it.

The same registries extend the other axes: `registerFormat` for a new
dataset layout, `registerStorage` for a new backend — including *composite*
backends (an overlay bucket layered over a read-only base is just another
two-method `Storage` whose `list` merges and whose `resolveUrl` checks the
overlay first).

## Payload contracts in full

The exact structures a `read(ref, { as })` returns — what a component is
handed, and therefore what a host-registered component should expect. These
are **the** in-memory contract: our one self-defined schema layer, versioned
with the library ([why it exists](/dataset-viz/overview.md#the-contract-lives-in-memory-never-on-disk)).
Every type below is exported from `@dreamlake/viz/dataset-viz`, and the
source (`dataset-viz/types.ts`) is the authority — this listing mirrors it.

```ts
type Payload =
  // A camera stream. `window` marks the sub-range of `url` this episode
  // occupies (LeRobot v3 concatenates episodes into one file per camera).
  | { kind: 'video'; url: string; window?: { from: number; to: number } }
  | { kind: 'series'; timestamps: number[]; columns: Record<string, number[]> }
  | { kind: 'image'; url: string }
  | { kind: 'frames'; count: number; fps?: number; frameAt: (i: number) => Promise<string> }
  | { kind: 'file'; url: string; ext?: string }
  | { kind: 'keypoints'; keypoints: KeypointTrack }
  | { kind: 'segments'; segments: Segment[] }
  | { kind: 'pose3d'; timestamps: number[]; joints: number[][]; shape: number[]; layout?: string }
  // Static 3D geometry — node names inside it are the join key for
  // per-frame transform3d / vertices3d tracks.
  | { kind: 'mesh3d'; url: string; format: 'gltf' | 'glb' | 'obj' }
  | { kind: 'transform3d'; timestamps: number[]; values: Float32Array; layout: TransformLayout }
  // Per-frame vertex positions for a deforming mesh, fetched lazily;
  // topology comes from the paired mesh3d node.
  | { kind: 'vertices3d'; count: number; fps?: number; vertexCount: number; at: (i: number) => Promise<Float32Array> }
  // Raw values, never pre-colorized — the component owns the colormap.
  | { kind: 'depth'; count: number; fps?: number; at: (i: number) => Promise<DepthFrame> }
  | { kind: 'pointcloud'; count: number; fps?: number; at: (i: number) => Promise<CloudFrame> }

/** One labelled time span — the canonical shape for every segment-class
 *  track (tasks, subtasks, actions, phases, subtitle files). */
interface Segment {
  start: number
  end: number
  label: string
}

/** Per-frame 2D keypoints, normalized out of whatever the source was (COCO
 *  JSON, a per-frame tensor). Coordinates are pixels in width × height;
 *  frames are sparse — a missing key means "no detection". */
interface KeypointTrack {
  width: number
  height: number
  /** Frame index → detections. Frame k is shown at k / fps seconds. */
  frames: Map<number, KeypointDetection[]>
  fps: number
  /** Index pairs drawn as bones. COCO-17 and Hand-21 presets are built in. */
  skeleton?: [number, number][]
  jointNames?: string[]
}

interface KeypointDetection {
  /** [x, y] per joint, pixels. */
  points: number[][]
  score?: number
  /** Free label — "left" / "right" / a class name. */
  group?: string
  /** [x1, y1, x2, y2] pixels, when the source carries one. */
  box?: [number, number, number, number]
}

/** One depth map. `data` is row-major width×height; `scale` converts a raw
 *  value to metres (0.001 for millimetre uint16; omit when unknown — the
 *  component then normalizes to the frame's own range). 0 = no reading. */
interface DepthFrame {
  width: number
  height: number
  data: Float32Array | Uint16Array | Uint8Array
  scale?: number
}

/** One cloud frame. `xyz` is N×3 metres; `rgb` optional N×3, either 0-255
 *  bytes or 0-1 floats. */
interface CloudFrame {
  xyz: Float32Array
  rgb?: Uint8Array | Float32Array
}

/** Component order of a transform3d row. Both orders occur in the wild
 *  (glTF/three use xyzw; robotics and MANO exports use wxyz), so the
 *  adapter states which it read instead of guessing downstream. */
type TransformLayout = 'txyz_qwxyz' | 'txyz_qxyzw'
```

## Storage drivers

Declared in a standalone file's `storage:` block, or injected by the host for
a file at its dataset root ([who writes it](/dataset-viz/spec.md#storage--where-the-dataset-lives)).
Configs carry identifiers only — never credentials.

| driver | config keys | notes |
| --- | --- | --- |
| `http` | `url` **required** — the dataset root URL (absolute, or site-relative when co-hosted) | `resolveUrl` joins `url` + path. Listing reads a co-located `index.json` manifest per directory — a static host cannot enumerate itself, so glob enumeration needs the manifests; `episodes: auto` formats don't. |
| `hf` | `repo` **required** (e.g. `lerobot/pusht`) · `root` subpath within the repo · `revision` (default `main`) · `repoType` (default `datasets`) | Public HuggingFace repos, credential-free. Listing walks the Hub tree API; resolved URLs support CORS + Range, so parquet/zarr reads work in-browser. |
| `dlSource` | `slug` (namespace) · `sourceId` · `root` subpath | **Registered by the DreamLake app** — a source browsed in the platform (S3, GCS, …). Listing via the source browse API, URLs via presign; the session token is injected at registration, never configured. |
| `dlProject` | `namespace` + `project`, or `root` (a node id) | **Registered by the DreamLake app** — a project folder as the dataset root. |

Hosts add drivers with `registerStorage(name, factory)`; the whole contract is
`list(path)` + `resolveUrl(path)`. (The legacy schema-viz pages use
`basePath`/`id` for their own http/hf drivers — a different subsystem; a
`.dreamrc` always uses the keys above.)

### The http `index.json` manifest

A static host can't enumerate itself, so the `http` driver lists a directory
by fetching `<dir>/index.json`. **Every directory a glob walks needs one** —
for `episodes: "episodes/*/"` that is `episodes/index.json` plus one inside
each episode folder (formats with `episodes: auto` fetch known paths and need
none). The shape mirrors what `list()` returns:

```json
{
  "entries": [
    { "name": "run_a", "path": "episodes/run_a", "type": "dir" },
    { "name": "cam_ego.mp4", "path": "episodes/run_a/cam_ego.mp4", "type": "file" }
  ]
}
```

`name` + `type` (`"file"` | `"dir"`) are required; `path` is the
storage-relative (or absolute) location — entries may point anywhere, which
is how a manifest can reference files hosted elsewhere. Don't list
`index.json` itself; `.dreamrc` need not be listed either (the app fetches
it directly).

## Formats

The `dataset.format` adapters. Common to all: the catalog is an inventory —
each field's address plus the facts the container reported — and `read(ref,
{ as })` is where anything is decoded. `dataset.annotations` declares extra
tracks ([spec](/dataset-viz/spec.md#datasetannotations--tracks-that-live-beside-the-data)),
merged in by the core — never a per-format concern.

### `lerobot`

LeRobot v2.0 / v2.1 / v3.0. The entry file is `meta/info.json`.

| | |
| --- | --- |
| expected layout | `meta/info.json` + `meta/tasks*` + `data/…parquet` + `videos/…mp4` (the paths `info.json` itself declares) |
| episodes | `auto` — `total_episodes` from `info.json`. A glob is never needed. |
| config keys | none |

The inventory is `meta/info.json`'s `features`, verbatim — one entry per
feature, carrying its `dtype`, `shape` and `names` as found. `dtype` is the
only thing read, and only to decide addressing: `video` → `video`, `image` →
`frames`, `language` / `string` → `text`, any numeric dtype → `tensor`.
Anything else is omitted with a warning naming it, including a depth VIDEO
(the feature's own `video.is_depth_map` flag) — no browser decodes 12-bit
log-quantized H.265.

Exactly four bookkeeping columns are dropped — `timestamp`, `frame_index`,
`episode_index`, `index` — and nothing else. `task_index` and `subtask_index`
stay: an integer pointing into a label table is data, and a view binding one
`as: segments` is what fetches `meta/tasks*` and joins it. Timeline:
per-episode length at the dataset's `fps`.

### `folder`

Folder-per-episode, no manifest — the directory layout is the contract.

| | |
| --- | --- |
| expected layout | anything: each matched folder is one episode, its files are the fields |
| episodes | a glob, e.g. `"episodes/*/"` — `auto` errors (a bare folder cannot enumerate itself) |
| config keys | `fps` — the clock for numbered still runs and depth stills (default 30). A directory of JPEGs states no frame rate, so the author does; without it, tracks that carry a real clock (a COCO file's `fps`, parquet timestamps) drift away from the pictures. |

The inventory is the directory listing and nothing more: one field per file,
addressed by its basename, with `path`, `ext` and `size` in `meta`. The
extension picks the ADDRESSING only — `mp4`/`webm` → `video`, `jpg`/`png` →
`image`, `csv`/`parquet` → `tensor`, and **everything else — `.vtt`, `.glb`,
`.json` — is `file`**. Consecutive numbered stills collapse into one `frames`
field. `annotations/` is listed too, and a file there replaces a same-named one
at the root.

No file is opened and no name is read for meaning; a view binding
`subtasks.vtt` as `segments` is what parses it, and the extension's second say
— at read time — picks only which parser. Series tables take their x-axis from
a `timestamp`-like column, row index otherwise. Timeline is `null` — media
components probe durations themselves.

### `umi`

Zarr stores, two modes detected from the store itself.

| | |
| --- | --- |
| expected layout | a `*.zarr.zip` ReplayBuffer (UMI: `data/` arrays + `meta/episode_ends`) **or** a `*.zarr` v3 directory store with a root `zarr.json` manifest (EgoVerse-style, one episode per store) |
| episodes | `auto` — `episode_ends` slices the ReplayBuffer; a directory store is a single episode |
| config keys | `path` — store path when not at the root or ambiguous (default: first `*.zarr.zip` / `*.zarr` found) · `fps` — ReplayBuffer clock (default 60) |

The inventory reads each array's own metadata. An **image codec** is the store
declaring that one chunk is one encoded image, so that array is `frames`
(byte-ranged a frame at a time — the store is never downloaded whole); numeric
arrays are `tensor`, carrying `dtype`, `shape` and `codec`. Nothing is
synthesized: a `names` list whose length does not match the array's width names
nothing, and a `[T,J,3]` keypoint array is a tensor like any other until a view
binds it `as: pose3d`. Arrays this reader cannot window (three dims or more)
are omitted with a warning. Timeline from frame counts at `fps`.

### `mcap`

MCAP v1 indexed containers — one episode per `*.mcap` file, read in place
over HTTP range requests (a 512MB file costs ~130KB before any field is read).

| | |
| --- | --- |
| expected layout | `*.mcap` files at the dataset root — indexed/chunked, lz4- or zstd-compressed chunks |
| episodes | `auto` — one per `*.mcap` at the root, name-sorted (the `http` driver needs its `index.json` manifest to list); a glob (`"runs/*.mcap"`) also works |
| config keys | `path` — a single `.mcap` at a subpath (skips listing) |

The inventory is the channel list, each channel carrying its `schema` name and
encoding in `meta`. Addressing follows what the schema says a message IS: json
channels with numeric leaves → `tensor` (dot paths are the columns, e.g.
`linear_accel.x`; dims come from the jsonschema or the first message),
string-only leaves → `text`; Foxglove image schemas → `frames` (one message is
one image, byte-ranged per frame); `FrameTransform` → one `tensor` per child
frame; a `SceneUpdate` carrying a ModelPrimitive → `file`.

The schema name is then a DECODER selector at read time — a
`foxglove.PointCloud` channel bound `as: pointcloud`, a tf channel bound
`as: transform3d` — never a conclusion at catalog time. Channels outside this
v1 scope — ros1msg/cdr/ros2idl decoding, `Grid`, attachments, bz2 chunks — are
**omitted from the catalog** with a console warning naming topic + encoding,
never mislisted. Unindexed files fail with a clear error (repack with `mcap
recover`). Timeline: message start→end from the summary statistics.

## View components

Live example of each — a minimal `.dreamrc` and its render — on the
[view components catalog](/dataset-viz/views.md). Here: the config surface.

**A slot is where meaning is stated.** Each component declares, per binding
slot, the payload kinds it reads — and that one declaration is what
`read(ref, { as })` is called with, what a `*` glob is filtered by, and what an
author's `as:` is checked against. Bind a field to `series` and it is read as
traces; bind the same field to `tracks` and it is read as spans. A slot a
component does not declare is rejected rather than ignored: `overlays:` on a
`lineChart` is an author expecting to see something, and silence there is the
failure mode this design removes.

Whether a binding needs an `as:` follows from the field, not from the
view — [the rule](#which-payloads-a-field-can-serve--and-when-you-write-as).
The media/geometry slots (`cameras`, `cloud`, `geometry`) take bare refs with
nowhere to write one, so they are always settled by the field's own
addressing kind; `series`, `tracks` and `overlays` take `{ field, as }`
entries. A `*` glob selects only the fields whose addressing kind can serve
the slot — `cameras: ["*"]` on `videoStack` takes the cameras and leaves the
tensors — while an explicit ref always passes and fails loudly if it cannot
be read.

Bindings are the keys the view interprets — its slot names (`cameras` /
`cloud` / `geometry` / `series` / `tracks` / `overlays`); **every other key
passes through as a prop** to the underlying `Episode*` component, so its
documented props are all available.
A `split: row` is a fixed-height strip (`height` on the split, default 280) —
its sizing keys (`width` fixed box · `flex` stretch share · `minWidth` squish
floor · child `height` override) are consumed by the layout, not the
component; media sizes its width from its aspect, charts fill their slot,
and overflow scrolls horizontally, synced across episodes under a
`SyncScrollProvider`. "row" marks components that render in the compact
`layout="row"` strips (dataset lists); the rest appear in grid layout only.

| view | binds | slot → payload | keys of note | row |
| --- | --- | --- | --- | --- |
| `videoStack` | `cameras` (a `*` glob expands to the container's media fields) · `overlays: [ { field, as, on? } ]` — `as` picks skeleton or captions, `on` pins one to a specific camera | `cameras` → `video`, `image` · `overlays` → `keypoints`, `segments` | `columns` (default 3) · `tileAspect` — force one ratio; by default each tile uses its video's intrinsic ratio. Plus [EpisodeVideoStack](/components/episode-video-stack.md) props. Probes video durations when the format has no timeline. | ✓ |
| `frameStack` | `cameras` · `overlays` (same form as `videoStack` — the tiles take the same layer) | `cameras` → `frames` · `overlays` → `keypoints`, `segments` | `columns` (default 3) | ✓ |
| `depthStack` | `cameras` — name the depth columns; a bare `*` would ask every tensor in the episode for a depth map · `overlays` | `cameras` → `depth` · `overlays` → `keypoints`, `segments` | `colormap: turbo \| gray` (default `turbo`) · `min` / `max` (raw units) pin the color range; by default each frame maps its own min/max over valid readings (>0), invalid renders transparent · `columns` (default 3). Corner chip shows the mapped range — metres when the format knows the depth scale, raw units otherwise | ✓ |
| `lineChart` | `series: [ ref \| { field, label?, color?, dash?, … } ]` — field is `feature` (all dims) or `[feature, dim]`; `*` globs work in both halves | `series` → `series` | `height` (default 180) sizes a standalone panel; in a `split: row` slot the chart fills the strip automatically. Plus `title`, `caption` + [EpisodeLineChart](/components/episode-line-chart.md) props | ✓ |
| `trajectory2d` | `series: [ ref \| { field, label?, color? } ]` — each entry is one path; x/y dims are the columns named `x`/`y` (case-insensitive) or the first two | `series` → `series` | `invertY: false` — math convention (y up); default is image convention (top-left origin) · `window: { ahead?, behind? }` — pin the drawn path to the seconds around the cursor (defaults 1 / 0 when given; omitted, the full path draws). `height` (default 260) sizes a standalone panel; in a `split: row` slot the plot fills the strip automatically | ✓ |
| `timeline` | `tracks` — required; nothing in an inventory says a column holds spans | `tracks` → `segments` | [EpisodeTimeline](/components/episode-timeline.md) props | — |
| `bandTrack` | `series: [ ref \| { field, label? } ]` — same field addressing as `lineChart`; each resolved column becomes one band row | `series` → `series` | `maxLevels` (default 12) — a column is discrete when its unique values (rounded to 6 decimals) fit, busier columns get a one-line "use lineChart" note · `bandHeight` (default 18) per-band px. Runs of equal value become colored rects; value→color legend below; natural height. | — |
| `metaPanel` | — (renders `EpisodeInfo`: name, duration, frames, fps, task strings) | — | `note` — a free-text line · `showTasks: false` hides the task strings (single-task datasets repeat one sentence per episode otherwise) | — |
| `fieldsCatalog` | — | — (prints the inventory itself) | — | — |
| `recon3d` | `geometry` — the geometry file(s) that make the scene · `tracks` — the per-frame motion, each entry naming its `as`; a track binds to the glTF node whose name matches its ref | `geometry` → `mesh3d` · `tracks` → `transform3d`, `vertices3d`, `pose3d` | `up` — gravity vector in the data's own frame (uprights the grid) · `trail: { ahead?, behind? }` — motion-trail window in seconds around the playhead (default `{ ahead: 1, behind: 0 }` — pure future, so the trail runs out exactly when the clip does; `behind` opts into a dim past tail; `false` turns it off) · `height` (default 360) sizes a standalone panel; in a `split: row` slot the scene fills the strip automatically | — |
| `pointCloud` | `cloud` — name the column; the first bound field renders (one cloud per panel in v1) | `cloud` → `pointcloud` | `up: y \| z` (default `z`, robot-lab convention → −90° X rotation) · `height` (default 360) sizes a standalone panel; in a `split: row` slot the scene fills the strip automatically. Per-point color when the data carries rgb; camera auto-fits the first frame | — |

Hosts add views with
`registerComponent({ name, component, reads, slotNames?, rows? })` — `reads`
is the slot → payload declaration above (canonical keys: `fields`, `series`,
`tracks`, `overlays`), `slotNames` renames a slot's config-surface key the
way `videoStack` surfaces `fields` as `cameras`, and a component that omits
`reads` opts out of binding checks. A component receives
`{ fields, info?, read, timeline, cursor, config, layout }`.

---

Source: https://viz.dreamlake.ai/components/episode-video-stack

# EpisodeVideoStack

A grid of N camera-view tiles that share a cursor time with each other and
(optionally) with a sibling [EpisodeTimeline](/components/episode-timeline.md).
Hovering any tile updates one shared `time` value; every other tile re-renders
its scrub line at that same time, with a **muted** tone so the actively-hovered
tile stays visually unambiguous.

The caller wires two pieces of state to make this work: a shared **`hover`**
time, and an **`activeId`** marking which tile carries the accent ring. The
component exposes both as controlled props — see
[Combined with EpisodeTimeline](#combined-with-episodetimeline) for the
canonical pattern.

## Tile anatomy

```
┌──────────────────────────────────────────┐
│ REC · 00:10.602              ● LIVE      │
│                                          │
│                  <video>                 │
│                                          │
│ FRONT                          1280×720  │
│ /camera/front/image_compressed           │
└──────────────────────────────────────────┘
```

Four overlays sit on top of the `<video>` element (all are optional via
`show*` props). The scrub line, when present, lives above everything at
`z-index: 4`.

The **bottom-right resolution** is auto-detected from
`<video>.videoWidth × videoHeight` once the metadata loads — there is no
caller-provided field, so the display is always real or absent.

## Standalone stack

Hover any tile to see the per-tile active state (full accent ring + bright
1.5px scrub line); the other two tiles drop to a muted 1px ink scrub line at
the same time.

```tsx file="StandaloneSpec.tsx"

const SAMPLE = '/preview-fixtures/episode-demo.mp4'
const DEMO_DURATION = 21.205 // real duration of episode-demo.mp4

const DEMO_VIDEOS: VideoSource[] = [
  { id: 'front', src: SAMPLE, title: 'front', subtitle: '/camera/front/image_compressed' },
  { id: 'wrist', src: SAMPLE, title: 'wrist', subtitle: '/camera/wrist/image_compressed' },
  { id: 'top',   src: SAMPLE, title: 'top',   subtitle: '/camera/top/image_compressed' },
]

export const StandaloneSpec = () => {
  const [hover, setHover] = useState<number | null>(null)
  const [activeId, setActiveId] = useState<string>(DEMO_VIDEOS[0].id)
  return (
    <EpisodeVideoStack
      videos={DEMO_VIDEOS}
      duration={DEMO_DURATION}
      time={hover}
      onHover={setHover}
      onHoverEnd={() => setHover(null)}
      activeId={activeId}
      onActiveChange={setActiveId}
    />
  )
}
```

> All three tiles render the same `episode-demo.mp4` fixture shipped under
> `docs/public/preview-fixtures/` (21.205s, 1280×720). Each `<video>`
> element seeks independently as you hover. Real apps pass distinct camera
> URLs via `VideoSource.src`. Omit `src` to fall back to the built-in
> placeholder grid (a neutral lattice + crosshair + two demo detection
> boxes in the tile's accent color).

## Playback

The stack has no play state of its own — playback comes from the shared
**`TimelineClock`** (rAF-driven, with `play` / `pause` / `seek` /
`setRate` / `setLoop`), and there is exactly **one playhead**: the clock's
time.

- `time` comes from the clock (`useClockValue`), so every scrub line and
  a paired timeline follow the same playhead.
- **Hover seeks the clock** (`onHover={seek}`) — scrubbing works the same
  paused or mid-playback, and play always resumes from wherever the
  pointer left the playhead. No separate hover state to fall out of sync.
- Provide the clock via `` and the tiles follow its
  play/pause events with the **native** `<video>` controls — smooth
  sequential decode instead of seek-per-update — re-syncing only on real
  jumps (a seek, a loop wrap, &gt;0.3 s drift).
- **A buffering tile holds the clock.** While any playing video is
  stalled for data (slow network, a fresh play, a mid-play seek), the
  playhead *waits* — like a normal video player — instead of running
  ahead and skipping content when data arrives. In a multi-camera stack
  the clock waits for the slowest stream, so every tile stays in sync.
  The play state doesn't change while held (no pause/play flicker).

```tsx
const { clock, state, play, pause, seek } = useTimeline(duration)
const time = useClockValue(30, clock) // the playhead, sampled at 30fps

<ClockProvider clock={clock}>
  <EpisodeVideoStack
    videos={videos}
    duration={duration}
    time={time}
    onHover={seek}
  />
</ClockProvider>
<button onClick={state.playing ? pause : play}>▶ / ⏸</button>
```

`DatasetPreview` wires this same clock behind every schema panel, so
schema-driven video stacks scrub and play the same way. The overlay
example below autoplays with this pattern; the
[combined example](#combined-with-episodetimeline) adds a play button and
a timeline driven by the same playhead.

## Hand keypoint overlays

Each `VideoSource` can carry [annotation overlays](/components/media-overlay.md)
— bounding boxes, keypoint skeletons, and subtitle-style caption tracks;
the tile keeps them glued to the presented frame during playback and
scrubbing. Here, real 21-joint hand detections plus subtask captions, each
converted with one adapter call:

```tsx file="HandJointsSpec.tsx"

// Egocentric recording + two annotation files in STANDARD formats: COCO
// keypoints (21-joint hand skeletons) and WebVTT (subtitle-style captions).
// The overlay layer itself takes normalized shapes — a keypoint track and a
// list of labelled spans — so any parser can feed it.
const EPISODE =
  'https://huggingface.co/datasets/live9080/dreamlake-ceramics/resolve/main/episodes/episode_a'
const VIDEO = `${EPISODE}/cam_ego.mp4`
const KEYPOINTS = `${EPISODE}/annotations/hand_keypoints.json`
const SUBTASKS = `${EPISODE}/annotations/subtasks.vtt`

async function fetchText(url: string): Promise<string> {
  const r = await fetch(url)
  if (!r.ok) throw new Error(`HTTP ${r.status}`)
  return r.text()
}

export const HandJointsSpec = () => {
  const [overlays, setOverlays] = useState<MediaOverlay[]>()
  const [duration, setDuration] = useState(33.7)
  const [error, setError] = useState<string>()

  // One TimelineClock is the single playhead: play advances it, hovering a
  // tile SEEKS it (even mid-playback), pause parks it — playback always
  // resumes from wherever the playhead is. Inside <ClockProvider> the
  // <video> plays natively while the clock runs.
  const { clock, state, play, pause, seek, setLoop } = useTimeline(duration)
  const time = useClockValue(30, clock)
  useEffect(() => {
    setLoop(true)
    play()
  }, [setLoop, play])

  useEffect(() => {
    let cancelled = false
    Promise.all([fetchText(KEYPOINTS), fetchText(SUBTASKS)])
      .then(([cocoText, vtt]) => {
        if (cancelled) return
        const track = parseCocoKeypoints(JSON.parse(cocoText))
        const segments = parseWebVtt(vtt)
        setOverlays([
          keypointsOverlayFromTrack(track), // 21-joint skeletons
          tracksOverlayFromSegments(segments), // subtitle captions
        ])
        const lastFrame = Math.max(...track.frames.keys())
        if (Number.isFinite(lastFrame)) setDuration((lastFrame + 1) / track.fps)
      })
      .catch((e) => {
        if (!cancelled) setError(e instanceof Error ? e.message : String(e))
      })
    return () => {
      cancelled = true
    }
  }, [])

  return (
    <div className="flex flex-col gap-2">
      <ClockProvider clock={clock}>
        <EpisodeVideoStack
          columns={1}
          videos={[
            {
              id: 'ego',
              src: VIDEO,
              title: 'Ceramics',
              subtitle: 'claru_ego · hand skeletons + subtask captions',
              overlays,
            },
          ]}
          duration={duration}
          time={time}
          onHover={seek}
        />
      </ClockProvider>
      <button
        type="button"
        onClick={state.playing ? pause : play}
        className="self-start rounded px-2 py-1 font-mono text-[11px] opacity-60 hover:opacity-100 hover:bg-current/10"
      >
        {state.playing ? '⏸ pause' : '▶ play'}
      </button>
      {error && (
        <p className="font-mono text-[11px] opacity-60">
          hand-joints fetch failed ({error}) — the demo bucket must allow
          cross-origin GET.
        </p>
      )}
    </div>
  )
}
```

See [Media overlays](/components/media-overlay.md) for the data model —
coordinate spaces, by-frame / by-time lookup, and styling.

## Combined with EpisodeTimeline

Two pieces of caller state wire the pair:

- **`hover`** — the shared cursor time. Hovering anywhere updates it; either
  component leaving clears it.
- **`activeId`** — which video tile is currently the focused camera. Set on
  hover-enter; **deliberately not cleared on leave**, so the last-hovered
  tile keeps its accent ring while the pointer moves onto the timeline below.

```tsx file="CombinedSpec.tsx"

const SAMPLE = '/preview-fixtures/episode-demo.mp4'

const DEMO_VIDEOS: VideoSource[] = [
  { id: 'front', src: SAMPLE, title: 'front', subtitle: '/camera/front/image_compressed' },
  { id: 'wrist', src: SAMPLE, title: 'wrist', subtitle: '/camera/wrist/image_compressed' },
  { id: 'top',   src: SAMPLE, title: 'top',   subtitle: '/camera/top/image_compressed' },
]

export const CombinedSpec = () => {
  const [activeId, setActiveId] = useState<string>(DEMO_VIDEOS[0].id)
  // One TimelineClock is the single playhead for the whole pair: hovering
  // any tile or the timeline SEEKS it (even mid-playback); play resumes
  // from wherever the pointer left it. Inside <ClockProvider> the video
  // tiles play natively while the clock runs.
  const { clock, state, play, pause, seek } = useTimeline(DEMO_DURATION)
  const time = useClockValue(30, clock)
  const activeVideo = DEMO_VIDEOS.find(v => v.id === activeId)
  const framesLabel = activeVideo
    ? `KEYFRAMES · ${(activeVideo.title ?? activeVideo.id).toUpperCase()}`
    : 'FRAMES'
  return (
    <div className="flex flex-col gap-2.5">
      <ClockProvider clock={clock}>
        <EpisodeVideoStack
          videos={DEMO_VIDEOS}
          duration={DEMO_DURATION}
          time={time}
          onHover={seek}
          activeId={activeId}
          onActiveChange={setActiveId}
        />
      </ClockProvider>
      <EpisodeTimeline
        duration={DEMO_DURATION}
        frames={DEMO_FRAMES}
        tracks={DEMO_TRACKS}
        time={time}
        onHover={seek}
        onSeek={seek}
        framesLabel={framesLabel}
      />
      <button
        type="button"
        onClick={state.playing ? pause : play}
        className="self-start rounded px-2 py-1 font-mono text-[11px] opacity-60 hover:opacity-100 hover:bg-current/10"
      >
        {state.playing ? '⏸ pause' : '▶ play all cameras'}
      </button>
    </div>
  )
}
```

### Active vs muted coordination

`EpisodeTimeline` and `EpisodeVideoStack` use different mechanisms to decide
which is the "actively scrubbed" surface — they coordinate via shared state,
not via a `tone` prop or `activeSource` flag:

| Component | How it picks active vs muted |
| --- | --- |
| `EpisodeTimeline` | **Auto-derived** from its own internal hover state. When the pointer is on the timeline, its own cursor lights up; when the pointer is elsewhere, an external `time` paints in the muted tone. |
| `EpisodeVideoStack` | **Caller-controlled** via `activeId`. The tile with `activeId === video.id` draws the 2px accent ring + 1.5px bright scrub line; peer tiles draw a 1px ink hairline at the same time. The component never auto-clears `activeId`. |

So when the pointer is over a video tile:
- The actively-hovered tile draws its accent ring + bright scrub line.
- Other tiles in the same stack: 1px 22% ink scrub line at the same time.
- The sibling `EpisodeTimeline` below: 1px 22% ink cursor at the same time
  (because its own internal hover state is empty).

When the pointer moves to the timeline, the roles swap — except the focused
video tile (`activeId`) keeps its **ring** (because `onActiveChange` was set
on hover-enter and the caller never cleared `activeId`), while its scrub line
drops to the muted treatment because the timeline is now the one being
scrubbed.

### Sticky active is a caller pattern, not a component default

The component itself doesn't decide whether to clear `activeId` on leave —
that's the caller's choice:

| Caller code | Behaviour |
| --- | --- |
| `activeId={id}` + `onActiveChange={setId}`, never call `setId(null)` | **Sticky** (recommended). Last-hovered tile keeps its ring forever. |
| Same as above, **plus** `onHoverEnd={() => setId(null)}` | Clear on leave. All tiles drop to muted when the pointer leaves the stack. |
| No `activeId` / `onActiveChange` (uncontrolled) | Sticky by default. First tile is active only if `defaultActiveId` is passed. |

## Notes on the design

- **Native `<video>` only** — supports MP4 / WebM / Ogg via the browser's
  built-in loader. HLS / DASH would require an extra peer-dep and is out of
  scope; pair this with an HLS-aware playlist engine externally if needed.
- **Truthful metadata** — the bottom-right resolution comes straight from
  `<video>.videoWidth × videoHeight` after metadata loads. There is no
  caller-provided fallback, so the display is always real or absent.
- **Seek tolerance** — the component only writes `video.currentTime` when
  the desired time differs by more than 50 ms. Avoids re-seeking on every
  sub-frame hover update.
- **Hover boundary** — `onPointerLeave` is attached to the outer grid, so
  moving between tiles via the 6 px gap doesn't fire `onHoverEnd` (only
  leaving the entire stack does).
- **No `onSeek`** — clicks on a tile are no-ops by design. Tiles are a
  preview surface, not a commit target; listen for clicks on the paired
  `EpisodeTimeline` instead.

## Props

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `videos` | `VideoSource[]` | — | One entry per tile, rendered left-to-right. |
| `duration` | `number` | — | Total duration in seconds. MUST match the sibling timeline. |
| `time` | `number \| null` | `null` | Shared cursor time. Drives `<video>.currentTime` and scrub-line X. `null` hides the cursor. Feed it from a `TimelineClock` for [playback](#playback). |
| `onHover` | `(t: number) => void` | — | Fires on every hover move with the time at the cursor. |
| `onHoverEnd` | `() => void` | — | Fires when the cursor leaves the entire stack (the gap between tiles does not count). |
| `activeId` | `string \| null` | — | Controlled active tile id. When provided, the component does NOT manage internal active state. |
| `defaultActiveId` | `string \| null` | `null` | Uncontrolled initial active tile id. Ignored when `activeId` is provided. |
| `onActiveChange` | `(id: string) => void` | — | Fires on hover-enter a new tile, with that tile's id. Does NOT fire on leave. |
| `columns` | `number` | `3` | Grid column count. |
| `gap` | `number` | `6` | Pixel gap between adjacent tiles. |
| `showRecTimestamp` | `boolean` | `true` | Top-left REC + cursor-time stamp. |
| `showLive` | `boolean` | `false` | Top-right red pulsing dot + "LIVE" label. |
| `showLabel` | `boolean` | `true` | Bottom-left title + subtitle. |
| `showResolution` | `boolean` | `true` | Bottom-right auto-detected resolution. Hidden until metadata loads or when `src` is omitted. |
| `showOverlays` | `boolean` | `true` | Master switch for all `VideoSource.overlays` layers. |
| `overlaySync` | `'media' \| 'cursor'` | `'media'` | What overlay lookup follows: the presented video frame (rVFC) or the `time` prop. See [Media overlays](/components/media-overlay.md#video-sync-modes). |
| `className` | `string` | — | Extra classes on the root grid. |

### VideoSource

| Field | Type | Description |
| --- | --- | --- |
| `id` | `string` | Stable identifier; React key + the value `activeId` matches against. |
| `src` | `string?` | Video URL. Native `<video>` only — MP4 / WebM / Ogg. Omit to render the built-in placeholder grid. |
| `title` | `string?` | Bottom-left primary label; rendered uppercase. Typically the camera name. |
| `subtitle` | `string?` | Bottom-left mono sub-label. Transport-agnostic — ROS topic, HTTP path, file name, anything. |
| `poster` | `string?` | `<video poster>`. Shown before the first frame loads. |
| `overlays` | `MediaOverlay[]?` | Annotation layers (boxes / skeletons) drawn over this video, back-to-front. See [Media overlays](/components/media-overlay.md). |

> **Why no `resolution` field?** It used to be a caller-provided string,
> which made it easy for the displayed value to drift from the actual
> encoded stream. Resolution is now auto-detected from
> `<video>.videoWidth × videoHeight` after metadata loads.

---

Source: https://viz.dreamlake.ai/dataset-viz/templates

# Templates

Two datasets on the Hub, both real recordings, both public. **Open them, look
at the bytes, and make yours look the same.** Each carries its own `.dreamrc`
at its root — the file you see below is fetched from the dataset, not copied
into this page.

| template | when it is yours | Hub |
| --- | --- | --- |
| **LeRobot with annotation features** | you are writing a LeRobot dataset and want annotations INSIDE it | [live9080/dreamlake-lerobot-annotated](https://huggingface.co/datasets/live9080/dreamlake-lerobot-annotated) |
| **Folder of video + sidecars** | you have recordings and files, and no container at all | [live9080/dreamlake-ceramics](https://huggingface.co/datasets/live9080/dreamlake-ceramics) |

Neither template asks you to convert anything into a DreamLake format —
[the contract](/dataset-viz/requirements.md) is entirely container idioms and
existing standards. Copy the shapes; the numbers are ours.

## 1. LeRobot with annotation features

Thirty seconds of a real egocentric recording, repacked as **LeRobot v2.1**,
carrying every annotation as an ordinary feature of the container. Every
number in it was measured on this one recording.

```
dreamlake-lerobot-annotated/
├── .dreamrc                        ← the visualization, no storage: block
├── meta/
│   ├── info.json                   ← features + shapes: the whole contract
│   ├── episodes.jsonl
│   ├── tasks.jsonl
│   └── subtasks.jsonl              ← the label table for subtask_index
├── data/chunk-000/episode_000000.parquet
└── videos/chunk-000/observation.images.ego/episode_000000.mp4
```

What each feature declares, and what the `.dreamrc` binds it to:

| feature in `meta/info.json` | dtype / shape | bound as |
| --- | --- | --- |
| `observation.images.ego` | `video` [1080,1920,3] | `videoStack.cameras` → camera tile |
| `observation.keypoints_2d.left.ego` | `float32` [21,3] | `videoStack.overlays` with `as: keypoints` → hand skeleton |
| `observation.keypoints_2d.right.ego` | `float32` [21,3] | the same, second hand |
| `subtask_index` + `meta/subtasks.jsonl` | `int64` [1] + label table | `tracks: [{ field: subtask_index, as: segments }]` → labelled blocks |

Two details are load-bearing, and both are easy to get wrong:

- **Which camera a hand belongs to is stated in the config, not in the name.**
  `on: observation.images.ego` pins the overlay to that camera, and that is the
  only thing that pairs them — the feature could be called anything. Pixel
  space comes from the camera it is pinned to. (With one camera in the episode
  there is nothing to disambiguate, so `on` is optional here.)
- **A labelled span is an index column plus a label table.** That is LeRobot's
  own `task_index` + `meta/tasks.jsonl` pattern, one level down. The join
  happens when a binding asks for `segments`; without the table there is
  nothing to label the spans with, and the read says so.

A third detail is about honesty rather than mechanics: **a frame where the
tracker found nothing is `NaN`, not zeros.** Zero is a real pixel — the
top-left corner — and a viewer cannot tell a fabricated origin from a measured
one, so zeros draw a collapsed skeleton in the corner on every undetected
frame. 279 of this episode's 900 left-hand frames are NaN, and they render as
nothing at all.

## 2. A folder of video and sidecar files

The zero-conversion route: no container, no manifest, no metadata file. One
directory per episode, files named after the tracks they carry.

```
dreamlake-ceramics/
├── .dreamrc
└── episodes/
    ├── episode_a/
    │   ├── cam_ego.mp4             ← track name = basename
    │   └── annotations/
    │       ├── hand_keypoints.json ← COCO keypoints
    │       └── subtasks.vtt        ← WebVTT
    └── episode_b/                  ← the same recording, second half
        ├── cam_ego.mp4
        └── annotations/…
```

Every file is in a format that already existed before we did. Three
conventions PLACE them — one episode per folder, basename is the track name,
`annotations/` is searched ([how](/dataset-viz/reference.md#folder)) — and not one
of them says what a file contains. `hand_keypoints.json` is catalogued as a
`file` with a `.json` extension; it becomes a skeleton because the `.dreamrc`
binds it `as: keypoints`. Rename every file and the parse result is identical;
that is the line between a convention and a format.

Both episodes carry the same track set here, but they need not: a dataset
where some episodes are annotated and others are not is the normal case, and
it renders as one.

## Upstream datasets, untouched

The two templates above are datasets we built. The more interesting claim is
the other one: **a dataset published by somebody else needs no modification at
all.** Two of the most-downloaded LeRobot datasets, forked into our namespace
with exactly one file added:

| fork | upstream | files changed |
| --- | --- | --- |
| [live9080/pusht](https://huggingface.co/datasets/live9080/pusht) | [`lerobot/pusht`](https://huggingface.co/datasets/lerobot/pusht) — 206 episodes | **+1** (`.dreamrc`) |
| [live9080/svla_so101_pickplace](https://huggingface.co/datasets/live9080/svla_so101_pickplace) | [`lerobot/svla_so101_pickplace`](https://huggingface.co/datasets/lerobot/svla_so101_pickplace) — SO-101, two cameras | **+1** (`.dreamrc`) |

Compare the file trees on the Hub: every parquet, every mp4 and every metadata
file is byte-identical to upstream. The `.dreamrc` describes how to look at the
data; it never asks the data to change. Both render in the
[gallery](/dataset-viz/gallery.md).

## Not ours, and just as valid

Two categories that need no template of ours, because the public dataset
already has the shape — point a `.dreamrc` at them and read
[the contract](/dataset-viz/requirements.md) for the rule each one satisfies:

| you want | a real dataset that does it |
| --- | --- |
| depth maps | [`juyil/libero_spatial_lerobot_depth`](https://huggingface.co/datasets/juyil/libero_spatial_lerobot_depth) — `float32` [256,256,1] named `depth_primary` |
| point clouds | [`rishabhrj11/gym-xarm-pointcloud`](https://huggingface.co/datasets/rishabhrj11/gym-xarm-pointcloud) — `float32` [512,6], xyz + rgb |
| many cameras + bimanual state | [`lerobot/aloha_static_coffee`](https://huggingface.co/datasets/lerobot/aloha_static_coffee) — four cameras, 14-DoF |

The [gallery](/dataset-viz/gallery.md) renders all of them, plus zarr and MCAP.

## Where the `.dreamrc` goes

Both templates above own their data, so the file sits at the dataset root
with no `storage:` block — the app injects the root it found the file in.
The gallery's third-party entries are the other case: read-only data, so
each config lives in
[live9080/dreamlake-configs](https://huggingface.co/datasets/live9080/dreamlake-configs)
and names its dataset in its own `storage:` block, which always wins over an
injected root. The full rule:
[spec](/dataset-viz/spec.md#storage--where-the-dataset-lives).

## Check before you ship

```bash
npx tsx scripts/check-dreamrc.mts hf:your-name/your-dataset
```

It resolves the dataset the way the app does: with no config it prints the
inventory — every field with the `dtype` and `shape` its container reported —
and with one it decodes every binding and prints what came back. That is how
you find out whether the `[21,3]` column you called a skeleton really is one,
before anybody looks at a panel. It also takes an `https://` root, or a local
`.dreamrc` file before you upload anything.

---

Source: https://viz.dreamlake.ai/schema-viz/schema

# Writing a schema

A **schema** names the _sources_ you want to look at and the _panels_ to lay over
them. `` takes the parsed object — parse YAML upstream if you
author in YAML; viz has no YAML dependency.

```ts
interface VizSchema {
  version: 1
  sources: Record<string, SourceConfig>
  timeline?: { source: string } // which source drives the shared clock
  panels?: PanelConfig[] // omit → auto-layout
}

type SourceConfig = {
  adapter: string // 'lerobot' | 'filesystem' | …
  storage?: StorageConfig // where the bytes live; defaults to http
  [param: string]: unknown // per-adapter params, e.g. episode
}
type StorageConfig = { driver: string } & Record<string, unknown>

interface PanelConfig {
  view: string // videoStack | lineChart | timeline | gridLayout | …
  source?: string // which source feeds this panel
  fields?: FieldRef[] // videoStack, or a chart's simple binding
  series?: SeriesCfg[] // lineChart, styled / per-dim
  tracks?: TrackCfg[] // timeline
  children?: PanelConfig[] // gridLayout
  [opt: string]: unknown // columns, title, height, …
}
```

## `sources` — an adapter + storage

Each source names an [adapter](/schema-viz/adapters.md) (_what format_) and its
[storage](/schema-viz/storage.md) (_where the bytes are_); per-adapter params like
`episode` ride alongside.

```yaml
sources:
  ep:
    adapter: lerobot
    storage:
      driver: http
      basePath: https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main
    episode: 0
```

> `http` is the credential-free built-in driver for **public** data. Private
> GitHub / HuggingFace / DreamLake data uses a driver **injected by the host app**
> — see [Storage](/schema-viz/storage.md).

## `panels` — views over fields

A panel names a `view`, points at a `source`, and binds fields. How you bind
depends on the view:

```yaml
# videoStack — list the camera fields (wildcards expand against the catalog)
{ view: videoStack, source: ep, fields: ['observation.images.*'] }

# lineChart — styled series; a field may select one dim of a vector feature
{ view: lineChart, source: ep, series: [
    { field: [action, left_waist],            label: cmd },
    { field: [observation.state, left_waist], label: actual, dash: "3 2.4" },
] }

# timeline — labelled cue tracks
{ view: timeline, source: ep, tracks: [{ field: task_index, label: Task }] }
```

A `field` is a feature name (`action`) or `[feature, dim]` to pick one dim
(`[action, left_waist]`); dim globs work too (`[observation.effort, "left_*"]`).
[Views](/schema-viz/views.md) covers each view's options.

## End-to-end: a folder of clips → one `videoStack`

No manifest, no per-dataset code — list a directory and merge the clips into one
synchronized component:

```yaml
version: 1
sources:
  clips:
    adapter: filesystem
    storage: { driver: http, basePath: /viz-samples/episode_365/ }
panels:
  - { view: videoStack, source: clips, fields: ['*'], columns: 2 }
```

```tsx file="FilesystemFolderSpec.tsx"
// A folder of loose clips → one synchronized stack, no manifest. The `http`
// storage lists the folder via its index.json and resolves each clip's URL; the
// `filesystem` adapter turns each file into a `video` field; `videoStack` merges
// them into one tile grid that scrubs together.
//
// These sample clips are served from the docs site's public/ folder. To read a
// DreamLake project folder instead, swap the storage to the host-injected
// `dlProject` driver:
//   storage: { driver: 'dlProject', root: '<folder node id>' }

const schema: VizSchema = {
  version: 1,
  sources: {
    clips: {
      adapter: 'filesystem',
      storage: { driver: 'http', basePath: '/viz-samples/episode_365/' },
    },
  },
  panels: [{ view: 'videoStack', source: 'clips', fields: ['*'], columns: 2 }],
}

export const FilesystemFolderSpec = () => <DatasetPreview schema={schema} />
```

The folder has no clock, so the stack takes the longest clip as the scrub extent
and every tile scrubs together.

## End-to-end: a LeRobot episode → multi-panel

A manifest indexes everything; one source feeds several synchronized panels. The
adapter's timeline gives them one shared clock, and the line charts overlay
action (cmd) against observation.state (actual, dashed) per joint:

```yaml
version: 1
sources:
  ep:
    adapter: lerobot
    storage:
      driver: http
      basePath: https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main
    episode: 0
panels:
  - { view: videoStack, source: ep, fields: ['observation.images.*'], columns: 2 }
  - { view: timeline, source: ep, tracks: [{ field: task_index, label: Task }] }
  - view: gridLayout
    columns: 2
    children:
      - view: lineChart
        source: ep
        title: Left arm — cmd vs actual
        series:
          - { field: [action, left_waist], label: waist · cmd }
          - { field: [observation.state, left_waist], label: waist · actual, dash: '3 2.4' }
```

```tsx file="LerobotEpisodeSpec.tsx"
// LeRobot episode → multi-panel, against a real PUBLIC HuggingFace dataset.
// Shows the full vocabulary: a camera `videoStack`, a `timeline` task track, and
// per-arm `lineChart`s that overlay action (cmd) against observation.state
// (actual, dashed) — one dim per series. Everything scrubs together.

const BASE = 'https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main'

// cmd (solid) vs actual (dashed) for a few joints of one arm.
const armSeries = (side: 'left' | 'right') =>
  ['waist', 'shoulder', 'elbow', 'gripper'].flatMap((joint) => [
    { field: ['action', `${side}_${joint}`], label: `${joint} · cmd` },
    { field: ['observation.state', `${side}_${joint}`], label: `${joint} · actual`, dash: '3 2.4' },
  ])

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: { adapter: 'lerobot', storage: { driver: 'http', basePath: BASE }, episode: 0 },
  },
  timeline: { source: 'ep' },
  panels: [
    {
      view: 'videoStack',
      source: 'ep',
      columns: 2,
      fields: ['observation.images.*'],
    },
    { view: 'timeline', source: 'ep', tracks: [{ field: 'task_index', label: 'Task' }] },
    {
      view: 'gridLayout',
      columns: 2,
      children: [
        {
          view: 'lineChart',
          source: 'ep',
          title: 'Left arm — cmd vs actual',
          height: 240,
          series: armSeries('left'),
        },
        {
          view: 'lineChart',
          source: 'ep',
          title: 'Right arm — cmd vs actual',
          height: 240,
          series: armSeries('right'),
        },
      ],
    },
  ],
}

export const LerobotEpisodeSpec = () => <DatasetPreview schema={schema} />
```

## End-to-end: hand skeletons via `overlays`

A `videoStack` panel can lay [media overlays](/components/media-overlay.md)
over its tiles. Each `overlays` entry names an annotation-file field in
the same source (globs work) and its `format` — the exact JSON shape each
format expects is specced in
[Media overlays → data formats](/components/media-overlay.md#data-formats).
Here two files annotate one video: 21-joint hand detections
(`format: handJoints` → skeletons) and subtask segments
(`format: subtasks` → subtitle-style captions). The **same** subtasks
field also feeds a `timeline` panel as a track row:

```yaml
version: 1
sources:
  ego:
    adapter: filesystem
    storage: { driver: http, basePath: /viz-samples/hand_joints/ }
panels:
  - view: videoStack
    source: ego
    fields: ['*'] # wildcard binds the video only
    columns: 1
    overlays:
      - { field: '*Ceramics.json', format: handJoints }
      - { field: 'ours_subtasks_713488.json', format: subtasks }
  - view: timeline
    source: ego
    tracks:
      - { field: 'ours_subtasks_713488.json', format: subtasks, label: Subtasks }
```

```tsx file="HandOverlaySchemaSpec.tsx"
// A video + two annotation files, described entirely by a schema:
// hand-joint detections (HandJointsFile → skeleton overlay) and subtask
// segments (SubtasksFile → subtitle-style caption overlay). The SAME
// subtasks field also feeds the timeline panel as a track row — one file,
// two synchronized views of it.
//
// DatasetPreview owns a shared TimelineClock; passing our own lets the
// button drive playback. Hovering a tile SEEKS the clock, so play always
// resumes from wherever you scrubbed.
//
// The video + hand JSON live on a public S3 prefix that can't serve a
// directory listing, so the local index.json manifest lists them with
// absolute URLs; the subtasks JSON is served from the docs site itself.

const schema: VizSchema = {
  version: 1,
  sources: {
    ego: {
      adapter: 'filesystem',
      storage: { driver: 'http', basePath: '/viz-samples/hand_joints/' },
    },
  },
  panels: [
    {
      view: 'videoStack',
      source: 'ego',
      fields: ['*'], // wildcard binds the video only
      columns: 1,
      overlays: [
        { field: '*Ceramics.json', format: 'handJoints' },
        { field: 'ours_subtasks_713488.json', format: 'subtasks' },
      ],
    },
    {
      view: 'timeline',
      source: 'ego',
      tracks: [
        { field: 'ours_subtasks_713488.json', format: 'subtasks', label: 'Subtasks' },
      ],
    },
  ],
}

export const HandOverlaySchemaSpec = () => {
  const { clock, state, play, pause, setLoop } = useTimeline(0)
  return (
    <div className="flex flex-col gap-2">
      <DatasetPreview schema={schema} clock={clock} />
      <button
        type="button"
        onClick={() => {
          setLoop(true)
          if (state.playing) pause()
          else play()
        }}
        className="self-start rounded px-2 py-1 font-mono text-[11px] opacity-60 hover:opacity-100 hover:bg-current/10"
      >
        {state.playing ? '⏸ pause' : '▶ play'}
      </button>
    </div>
  )
}
```

Add `on: <video field>` to target one tile when a panel has several
(omitted, the layer draws on every video tile); `format: raw` reads a file
that already contains `MediaOverlay` JSON.

## Omit `panels` → auto-layout

Don't know the dataset yet? **Leave `panels` out.** viz reads the primary
source's fields and lays them out by kind — a camera stack, a task timeline, and
one chart per numeric field:

```yaml
version: 1
sources:
  ep:
    adapter: lerobot
    storage:
      driver: http
      basePath: https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main
    episode: 0
# no panels → auto-layout
```

```tsx file="AutoLayoutSpec.tsx"
// Omit `panels` entirely → viz auto-lays-out the source: a camera stack, a task
// timeline, and one chart per numeric field. The smallest possible schema that
// still produces a full view — the "I don't know this dataset yet" workflow.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      adapter: 'lerobot',
      storage: {
        driver: 'http',
        basePath: 'https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main',
      },
      episode: 0,
    },
  },
  // no `panels` → auto-layout runs against `ep`
}

export const AutoLayoutSpec = () => <DatasetPreview schema={schema} />
```

Write the smallest schema, get a live view, then hand-author `panels` once you
know what you want. [Views → auto-layout](/schema-viz/views.md#auto-layout) covers
the rules.

## Recap

- [Storage](/schema-viz/storage.md) — _where_ the bytes are (`http`, or a host-injected
  authorized driver).
- [Adapters](/schema-viz/adapters.md) — _what format_ (`filesystem`, `lerobot`).
- [Views](/schema-viz/views.md) — _how to draw_ (`videoStack`, `lineChart`, `timeline`).
- **Schema** — the recipe: `sources` + `panels`.

---

Source: https://viz.dreamlake.ai/dataset-viz/gallery

<div
  style={{
    display: 'flex',
    alignItems: 'baseline',
    gap: 14,
    flexWrap: 'wrap',
    margin: '14px 0 14px',
  }}
>
  <h1 style={{ margin: 0, fontSize: 21, fontWeight: 700, letterSpacing: '-0.01em' }}>Gallery</h1>
  <p style={{ margin: 0, fontSize: 13, opacity: 0.65, flex: '1 1 320px', minWidth: 260 }}>
    Real datasets, one grammar — every file is complete (its{' '}
    <code>storage:</code> inline): copy one and it runs anywhere. At a dataset
    root the file would drop <code>storage:</code> and inherit its location.
    The address bar tracks the selected entry (<code>?example=…</code>) —
    copy it to share one dataset directly.
  </p>
  <a
    href="/dataset-viz/spec"
    style={{
      fontFamily: 'var(--font-doc-template-mono, monospace)',
      fontSize: 12,
      textDecoration: 'none',
      opacity: 0.65,
      whiteSpace: 'nowrap',
    }}
  >
    ← .dreamrc spec
  </a>
</div>

## Notes per entry

- **PushT / ALOHA** — LeRobot v3 repos on the public Hub; `episodes: auto`
  reads `total_episodes`, nothing else to configure. Both pack every episode
  of a camera into ONE mp4 and give each episode a time window in the
  metadata, which is what the viewer plays. The ALOHA entry lays its charts
  out with a `split: row` node — layout belongs to the author, and nested
  `row` / `column` / `grid` compose freely.
- **Cognition pour-water** — a LeRobot **v2.0** dataset served as plain
  HTTPS from a public S3 bucket: same `format: lerobot`, older layout,
  different storage — the `.dreamrc` barely changes. Its chart binds the same
  feature twice with dim globs (`"torso*"`) to overlay command (dashed) on
  state (solid). `meta/info.json` declares 298 episodes and only three are
  mirrored, so the entry resolves five: the last two show the per-panel
  **error state** a missing file produces, which is what a partial mirror
  honestly looks like.
- **LIBERO Spatial (depth)** — float32 depth maps stored raw in parquet:
  read as typed arrays and turbo-colorized on the fly, no image decoding at
  all. Frames in the same parquet row group cost zero extra requests.
- **gym-xarm point clouds** — `observation.environment_state` is a `[512,6]`
  float tensor and its name says nothing; the config binds it to `pointCloud`,
  which reads it as xyz+rgb (Apache-2.0). Orbit the scene while the cursor
  plays. LeRobot declares no semantics for a feature and nothing here invents
  one — a human read the inventory once and wrote it down.
- **nuScenes mini (MCAP)** — a 512MB file on third-party demo infra read IN
  PLACE: the summary section gives 30 fields from 18 of its 41 channels for
  ~2MB of ranged reads, then each field pulls only the chunks it needs.
  Each channel is listed with its schema name, and that name selects a decoder
  when a binding asks: `foxglove.PointCloud` read as `pointcloud` (34,688 LiDAR
  points per frame), one tf child frame read as `transform3d`, `SceneUpdate`
  model primitives, GPS. `cdr`-encoded channels (a plain ROS 2 bag) and
  `Grid` / `CameraCalibration` are still omitted, with console warnings.
- **MV-UMI** — the multi-GB `.zarr.zip` is never downloaded: the frame
  stack byte-ranges one JPEG-XL frame per camera under the cursor
  (JXL needs Safari 17+ or Chrome's flag).
- **EgoVerse** — a `.zarr` v3 directory store; the flattened hand-keypoint
  arrays are numbers like any other until the config binds them `as: pose3d`,
  which animates them as point sets in the 3D scene.
- **Ceramics** — `folder` format: annotations auto-discovered from each
  episode's `annotations/` dir; the `recon` track is a real exported
  reconstruction (ruler + toy, MANO hands) — orbit it while video plays.

---

Source: https://viz.dreamlake.ai/components/episode-frame-stack

# EpisodeFrameStack

The still-image sibling of [EpisodeVideoStack](/components/episode-video-stack.md):
the same tile chrome — REC-style stamp, label, auto-detected resolution,
scrub line, active ring — but each tile scrubs a sequence of `<img>` frames
instead of seeking a `<video>`. Moving the pointer across a tile snaps to the
**nearest frame** for the cursor position; the frame shown stays on screen
until the next one has decoded, so scrubbing doesn't flash.

Unlike the video stack, **there is no `duration` prop**. The total span is
derived from the earliest and latest frame `timestamp` across all sources.

## Derived span

Each frame carries a `timestamp` — a per-frame Unix time in **seconds**
(the upstream `_ts`). The component computes the cursor span from the first
and last frame and maps the pointer's X fraction onto it:

- **Timestamped** — when frames carry real times, `duration = lastTs − firstTs`
  and each tile shows the frame nearest the cursor's time.
- **No valid timestamp** — when a frame's time isn't a real recorded value
  (`0`, negative, `NaN`/`Infinity`, or out of the representable date range),
  frames fall back to uniform index spacing.

Keep a single stack homogeneous: all of its sources should either carry real
timestamps or none should. (A stack mixes one shared cursor span across every
tile, so combining recorded and unrecorded sources in the same stack isn't
meaningful.)

## Timestamped

Three cameras sharing one cursor. Hover any tile — every tile snaps to the
frame nearest that moment, and the top-left stamp shows the hovered frame's
wall-clock time (`YYYY-MM-DD HH:MM:SS`, local).

```tsx file="TimestampedSpec.tsx"

// Three cameras, each a sequence of stills with real per-frame Unix
// timestamps. Total span is derived from the first/last frame, so no
// `duration` prop is needed. The top-left stamp shows the hovered
// frame's wall-clock time.
const SOURCES: FrameImageSource[] = [
  { id: 'front', frames: TIMESTAMPED_FRAMES, title: 'front', subtitle: '/camera/front/image_compressed' },
  { id: 'wrist', frames: TIMESTAMPED_FRAMES, title: 'wrist', subtitle: '/camera/wrist/image_compressed' },
  { id: 'top',   frames: TIMESTAMPED_FRAMES, title: 'top',   subtitle: '/camera/top/image_compressed' },
]

export const TimestampedSpec = () => <EpisodeFrameStack sources={SOURCES} />
```

## Invalid timestamp → frame index

This specimen uses `timestamp: 0`, but the rule is general: any frame whose
time isn't a valid recorded value — `0`, negative, `NaN`/`Infinity`, or out of
the representable date range — is treated the same way. With no real times to
position by, the stack spaces frames uniformly by index and the stamp reads
`FRAME n / N` instead of a date.

```tsx file="FrameIndexSpec.tsx"

// Same frames, but with `timestamp: 0` (the upstream `_ts` was never
// recorded). The stack falls back to uniform index spacing and the
// top-left stamp reads `FRAME n / N` instead of a date.
const SOURCES: FrameImageSource[] = [
  { id: 'front', frames: UNTIMED_FRAMES, title: 'front', subtitle: '/camera/front/image_compressed' },
  { id: 'wrist', frames: UNTIMED_FRAMES, title: 'wrist', subtitle: '/camera/wrist/image_compressed' },
]

export const FrameIndexSpec = () => <EpisodeFrameStack sources={SOURCES} columns={2} />
```

## Bounding-box overlays

Each source can carry [annotation overlays](/components/media-overlay.md); on
a frame stack they key on the displayed image (`frameIndex ?? array index`)
and always describe the frame actually on screen. These synthetic
detections are **normalized** (`0..1` of the frame) — the `cup` box exists
only on frames 30–75, so it appears and disappears with the data:

```tsx file="BBoxOverlaySpec.tsx"

const FRAME_COUNT = UNTIMED_FRAMES.length

// Synthetic detections in NORMALIZED coordinates (0..1 of the frame), keyed
// by frame index. One "gripper" track orbits slowly; a "cup" exists only on
// frames 30–75 — sparse keys simply draw nothing, they never go stale.
function buildDetections(): BBoxOverlay {
  const frames: Record<number, BBoxItem[]> = {}
  for (let i = 0; i < FRAME_COUNT; i++) {
    const a = (i / FRAME_COUNT) * 2 * Math.PI
    const x = 0.38 + 0.22 * Math.cos(a)
    const y = 0.35 + 0.18 * Math.sin(a)
    const items: BBoxItem[] = [
      {
        box: [x, y, x + 0.22, y + 0.3],
        label: 'gripper',
        trackId: 'gripper-0',
        score: 0.93,
      },
    ]
    if (i >= 30 && i <= 75) {
      items.push({
        box: [0.66, 0.12, 0.92, 0.42],
        label: 'cup',
        trackId: 'cup-0',
        score: 0.81,
      })
    }
    frames[i] = items
  }
  return {
    id: 'detections',
    kind: 'bbox',
    space: { units: 'normalized' },
    data: { by: 'frame', frames },
    style: { showScore: true },
  }
}

export const BBoxOverlaySpec = () => {
  const overlays = useMemo(() => [buildDetections()], [])
  return (
    <EpisodeFrameStack
      columns={2}
      sources={[
        {
          id: 'front',
          frames: UNTIMED_FRAMES,
          title: 'front',
          subtitle: 'synthetic detections',
          overlays,
        },
        {
          id: 'top',
          frames: UNTIMED_FRAMES,
          title: 'top',
          subtitle: 'no overlay',
        },
      ]}
    />
  )
}
```

## Frame source shape

```ts
interface FrameImage {
  timestamp: number   // per-frame Unix seconds; invalid (0, NaN, …) = no recorded time
  image: string       // URL — remote, data:, or blob:
  alt?: string
  frameIndex?: number // index in the ORIGINAL media, for subsampled sequences
}

interface FrameImageSource {
  id: string
  frames: FrameImage[]   // sorted ascending by timestamp
  title?: string         // bottom-left label, uppercased
  subtitle?: string      // bottom-left mono sub-label
  poster?: string        // shown before the first frame loads
  overlays?: MediaOverlay[] // annotation layers — see /components/media-overlay
}
```

The tile's top-left stamp is computed per hovered frame: a frame with a valid
recorded time shows the formatted wall-clock; any invalid time (`0`, negative,
`NaN`/`Infinity`, or out of the representable date range) shows its
`FRAME n / N` position instead.

## Props

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `sources` | `FrameImageSource[]` | — | Frame sequences, one tile each, left-to-right. |
| `time` | `number \| null` | `null` | Controlled cursor time (seconds from start). Internal hover overrides it while the pointer is over the stack. |
| `onHover` | `(t: number) => void` | — | Fires on every hover move with the time-from-start at the cursor. |
| `onHoverEnd` | `() => void` | — | Fires when the cursor leaves the entire stack. |
| `activeId` | `string \| null` | — | Controlled active-tile id (the accent-ring tile). |
| `defaultActiveId` | `string \| null` | `null` | Uncontrolled initial active-tile id. |
| `onActiveChange` | `(id: string) => void` | — | Fires on hover-enter of a new tile, with that tile's id. |
| `columns` | `number` | `3` | Grid column count. |
| `gap` | `number` | `6` | Pixel gap between adjacent tiles. |
| `showRecTimestamp` | `boolean` | `true` | Top-left stamp — wall-clock time, or `FRAME n / N` when unrecorded. |
| `showLabel` | `boolean` | `true` | Bottom-left title + subtitle. |
| `showResolution` | `boolean` | `true` | Bottom-right resolution, auto-detected from the rendered `<img>`. |
| `showOverlays` | `boolean` | `true` | Master switch for all `FrameImageSource.overlays` layers. |
| `className` | `string` | — | Extra classes on the root grid. |

---

Source: https://viz.dreamlake.ai/components/media-overlay

# Media overlays

[EpisodeVideoStack](/components/episode-video-stack.md) and
[EpisodeFrameStack](/components/episode-frame-stack.md) accept per-source
**overlays** — annotation layers drawn over the media: bounding boxes for
detections, keypoint skeletons for poses and hands. Attach them to the
source; the tile keeps them aligned with the pixels at any player size.

```tsx

const overlays: MediaOverlay[] = [handSkeletons, objectBoxes]

<EpisodeVideoStack
  videos={[{ id: 'ego', src, overlays }]}
  duration={67.5}
/>
```

A stack-level `showOverlays` prop (default `true`) is the master switch;
each overlay also has its own `visible` flag and `opacity`.

## Hand skeletons + captions on a video

Two layers on one video, each converted with one call: real 21-joint hand
detections (`handOverlayFromDetections` — reads pixel space, fps, and
skeleton from the file, applies the per-finger `HAND21_STYLE`) and
labelled subtask segments rendered as subtitle-style captions
(`tracksOverlayFromSubtasks`). Both stay glued to the frame during
playback and while scrubbing:

```tsx file="HandJointsSpec.tsx"

// Egocentric recording + two annotation files in STANDARD formats: COCO
// keypoints (21-joint hand skeletons) and WebVTT (subtitle-style captions).
// The overlay layer itself takes normalized shapes — a keypoint track and a
// list of labelled spans — so any parser can feed it.
const EPISODE =
  'https://huggingface.co/datasets/live9080/dreamlake-ceramics/resolve/main/episodes/episode_a'
const VIDEO = `${EPISODE}/cam_ego.mp4`
const KEYPOINTS = `${EPISODE}/annotations/hand_keypoints.json`
const SUBTASKS = `${EPISODE}/annotations/subtasks.vtt`

async function fetchText(url: string): Promise<string> {
  const r = await fetch(url)
  if (!r.ok) throw new Error(`HTTP ${r.status}`)
  return r.text()
}

export const HandJointsSpec = () => {
  const [overlays, setOverlays] = useState<MediaOverlay[]>()
  const [duration, setDuration] = useState(33.7)
  const [error, setError] = useState<string>()

  // One TimelineClock is the single playhead: play advances it, hovering a
  // tile SEEKS it (even mid-playback), pause parks it — playback always
  // resumes from wherever the playhead is. Inside <ClockProvider> the
  // <video> plays natively while the clock runs.
  const { clock, state, play, pause, seek, setLoop } = useTimeline(duration)
  const time = useClockValue(30, clock)
  useEffect(() => {
    setLoop(true)
    play()
  }, [setLoop, play])

  useEffect(() => {
    let cancelled = false
    Promise.all([fetchText(KEYPOINTS), fetchText(SUBTASKS)])
      .then(([cocoText, vtt]) => {
        if (cancelled) return
        const track = parseCocoKeypoints(JSON.parse(cocoText))
        const segments = parseWebVtt(vtt)
        setOverlays([
          keypointsOverlayFromTrack(track), // 21-joint skeletons
          tracksOverlayFromSegments(segments), // subtitle captions
        ])
        const lastFrame = Math.max(...track.frames.keys())
        if (Number.isFinite(lastFrame)) setDuration((lastFrame + 1) / track.fps)
      })
      .catch((e) => {
        if (!cancelled) setError(e instanceof Error ? e.message : String(e))
      })
    return () => {
      cancelled = true
    }
  }, [])

  return (
    <div className="flex flex-col gap-2">
      <ClockProvider clock={clock}>
        <EpisodeVideoStack
          columns={1}
          videos={[
            {
              id: 'ego',
              src: VIDEO,
              title: 'Ceramics',
              subtitle: 'claru_ego · hand skeletons + subtask captions',
              overlays,
            },
          ]}
          duration={duration}
          time={time}
          onHover={seek}
        />
      </ClockProvider>
      <button
        type="button"
        onClick={state.playing ? pause : play}
        className="self-start rounded px-2 py-1 font-mono text-[11px] opacity-60 hover:opacity-100 hover:bg-current/10"
      >
        {state.playing ? '⏸ pause' : '▶ play'}
      </button>
      {error && (
        <p className="font-mono text-[11px] opacity-60">
          hand-joints fetch failed ({error}) — the demo bucket must allow
          cross-origin GET.
        </p>
      )}
    </div>
  )
}
```

## Bounding boxes on a frame stack

Hand-authored synthetic detections in **normalized** coordinates, keyed by
frame index. The `cup` only exists on frames 30–75 — sparse keys draw
nothing, so boxes appear and disappear with the data instead of going
stale:

```tsx file="BBoxOverlaySpec.tsx"

const FRAME_COUNT = UNTIMED_FRAMES.length

// Synthetic detections in NORMALIZED coordinates (0..1 of the frame), keyed
// by frame index. One "gripper" track orbits slowly; a "cup" exists only on
// frames 30–75 — sparse keys simply draw nothing, they never go stale.
function buildDetections(): BBoxOverlay {
  const frames: Record<number, BBoxItem[]> = {}
  for (let i = 0; i < FRAME_COUNT; i++) {
    const a = (i / FRAME_COUNT) * 2 * Math.PI
    const x = 0.38 + 0.22 * Math.cos(a)
    const y = 0.35 + 0.18 * Math.sin(a)
    const items: BBoxItem[] = [
      {
        box: [x, y, x + 0.22, y + 0.3],
        label: 'gripper',
        trackId: 'gripper-0',
        score: 0.93,
      },
    ]
    if (i >= 30 && i <= 75) {
      items.push({
        box: [0.66, 0.12, 0.92, 0.42],
        label: 'cup',
        trackId: 'cup-0',
        score: 0.81,
      })
    }
    frames[i] = items
  }
  return {
    id: 'detections',
    kind: 'bbox',
    space: { units: 'normalized' },
    data: { by: 'frame', frames },
    style: { showScore: true },
  }
}

export const BBoxOverlaySpec = () => {
  const overlays = useMemo(() => [buildDetections()], [])
  return (
    <EpisodeFrameStack
      columns={2}
      sources={[
        {
          id: 'front',
          frames: UNTIMED_FRAMES,
          title: 'front',
          subtitle: 'synthetic detections',
          overlays,
        },
        {
          id: 'top',
          frames: UNTIMED_FRAMES,
          title: 'top',
          subtitle: 'no overlay',
        },
      ]}
    />
  )
}
```

## Declaring coordinates

Overlay coordinates live in the **media's** space — never the player's.
Each overlay says which flavor via `space`:

```ts
type OverlaySpace =
  | { units: 'pixel'; width: number; height: number } // annotation-time frame size
  | { units: 'normalized' }                           // x, y already in 0..1
```

- **Pixel coordinates carry the frame size they were produced against**
  (not the runtime `videoWidth`, which can differ after transcode/rotation).
- **Out-of-frame points are legal** — partially visible detections draw and
  get clipped, never clamped.
- Stroke widths and dot radii are CSS px at render size, so a 2px bone
  reads as 2px on any player.

## What draws at time *t*

Each overlay's `data` field says how to look up "the items for right now".
Missing data draws nothing — stale items are never carried over.

```ts
type OverlayFrameData<T> =
  | { by: 'frame'; frames: Record<number, T[]>; fps?: number; frameOffset?: number }
  | { by: 'time'; entries: { time: number; items: T[] }[]; tolerance?: number }
```

| | `by: 'frame'` | `by: 'time'` |
| --- | --- | --- |
| **Video stack** | key = `round(time × fps)`; `fps` required (source fps is often fractional). | nearest entry within `tolerance` (default 0.1 s). |
| **Frame stack** | key = the displayed image's `frameIndex ?? array index`. | matched against `FrameImage.timestamp`. |

On a video, the lookup follows the frame the `<video>` actually presents
(`overlaySync: 'media'`, the default), so drawings never lead or lag the
pixels; `'cursor'` follows the `time` prop instead (deterministic — SSR,
tests).

## Reference

### `bbox`

```ts
interface BBoxItem {
  box: [number, number, number, number]  // [x1, y1, x2, y2]
  label?: string
  score?: number                         // 0..1
  trackId?: string | number              // same id ⇒ same color across frames
  color?: string                         // explicit override
}
```

| `BBoxStyle` | Default | |
| --- | --- | --- |
| `strokeWidth` | `1.5` | |
| `stroke` / `fill` | palette / `transparent` | |
| `showLabel` / `showScore` | `true` / `false` | `showScore` appends: `cup 0.92`. |
| `colorBy` | `'label'` | or `'trackId'` / `'fixed'`. |
| `palette` | built-in | cycled per `colorBy` key. |

### `keypoints`

```ts
interface KeypointsItem {
  points: ([number, number] | [number, number, number])[] // z reserved for 3D
  pointScores?: number[]
  score?: number
  box?: [number, number, number, number]  // drawn when style.showBox
  group?: string                          // coloring key, e.g. 'left' / 'right'
  color?: string
}

interface KeypointsOverlay {
  kind: 'keypoints'
  skeleton?: [number, number][]  // bones as index pairs into points
  jointNames?: string[]
  data: OverlayFrameData<KeypointsItem>
  style?: KeypointsStyle
  // …common fields: id, space, visible, opacity, minScore
}
```

| `KeypointsStyle` | Default | |
| --- | --- | --- |
| `jointRadius` / `boneWidth` | `2.5` / `1.5` | |
| `boneColors` | — | per-bone, same order as `skeleton` — how per-finger coloring works. |
| `jointColors` | derived | each joint takes the color of the bone ending at it. |
| `rootColor` | `#ffffff` | joints no bone ends at (the wrist). |
| `groupColors` | — | whole-instance color per `group` — the simple mode. |
| `showBox` | `false` | also draw the detection `box`. |
| `minPointScore` | `0` | hide low-confidence joints. |

### `tracks` — subtitle-style captions

Labelled time ranges rendered like video subtitles: whichever block spans
the current time draws as an outlined caption at the bottom of the tile.
The data shape is the same `TimelineTrack` / `TrackBlock` that
[EpisodeTimeline](/components/episode-timeline.md) takes, so one dataset feeds
both the caption overlay and a timeline track row:

```ts
interface TracksOverlay {
  kind: 'tracks'
  tracks: { id: string; name?: string; blocks: TrackBlock[] }[]
  style?: { fontSize?: number; color?: string } // default 12px / white
  // …common fields: id, visible, opacity
}

type TrackBlock = { id?: string; start: number; end: number; label: string } // seconds
```

Time-range based — no coordinate `space`, no frame keys.

## Data formats

Each converter — and the matching schema `format:` value — expects
**exactly** the JSON shape below. Data in any other shape needs
`format: raw` (a file that already contains `MediaOverlay` JSON) or a
custom conversion in code.

### `handJoints` → keypoints overlay

`HandJointsFile` — the hand-detection pipeline output (this is the format
of the live example's `claru_ego__Ceramics.json`). Frames are keyed by
**0-based index of the original video**; coordinates are full-resolution
pixels in upright orientation. Field-by-field, the conversion is:

```jsonc
{
  "width": 1920, "height": 1080,       // → space { units: "pixel", width, height }
  "src_fps": 29.987,                   // → data.fps (playback time → frame key)
  "joint_order": ["wrist", "…21"],     // → jointNames
  "bones": [[0, 1], "…20 pairs"],      // → skeleton (4 per finger, thumb → pinky)
  "frames": {                          // sparse → data.frames (missing key = no hands)
    "0": [{
      "is_right": 1,                   // → group "right" | "left"
      "det_conf": 0.98,                // → score
      "bbox": [x1, y1, x2, y2],        // → box
      "keypoints_2d": [[x, y], "…21"]  // → points
    }]
  },
  "total_frames": 2024
}
```

```tsx

const overlay = handOverlayFromDetections(file) // + per-finger HAND21_STYLE
```

### `subtasks` → caption overlay + timeline track

`SubtasksFile` — subtask segment annotations: one labelled range per
segment, in **seconds from video start**:

```jsonc
{
  "video": "713488",
  "task": "mount a wall shelf",        // → track name
  "labeled_subtasks": [
    { "start_sec": 0.0, "end_sec": 2.5, "subtask": "pick up shelf board" },
    { "start_sec": 2.5, "end_sec": 10.5, "subtask": "mount shelf board on wall tracks" }
  ]
}
```

Two converters read it — the **same blocks** drive the subtitle overlay
and an `EpisodeTimeline` track:

```tsx

const overlay = tracksOverlayFromSubtasks(file) // kind: 'tracks' captions
const timelineTracks = [
  { id: 'subtasks', blocks: trackBlocksFromSubtasks(file) }, // <EpisodeTimeline tracks>
]
```

See both live in the
[schema example](/schema-viz/schema.md#end-to-end-hand-skeletons-via-overlays):
a `videoStack` panel names the files by field ref with
`format: handJoints` / `format: subtasks`, and a `timeline` panel shows
the same subtasks as a track row.

---

Source: https://viz.dreamlake.ai/components/episode-recon-3d

# EpisodeRecon3d

**A controlled 3D reconstruction view, shaped like the other Episode\***
**components**: pass a `scene` (geometry + its per-frame tracks) plus
`time`/`duration` and it animates — meshes move along their 6-DoF transform
tracks, deforming meshes get new vertices per frame, and the motion trail
is the future: the next second of each path, running out exactly when the
clip does. Orbit with the mouse.
This is the same scene the platform's annotation viewer renders for
`recon3d` datasets; the example below loads a real exported scene (a ruler +
a toy with MANO hand meshes) from its glTF binary.

```tsx file="BasicSpec.tsx"
// EpisodeRecon3d standalone: a controlled 3D view driven by `time`/`duration`
// props — here a self-running clock; in an app, any shared player clock.
// Everything it renders comes from STANDARD files: geometry from a glTF
// binary (`scene.glb` — nodes `ruler`, `toy`, `left_hand`, `right_hand`),
// and this demo binds nothing per-frame, so the scene shows its rest pose.
// Orbit with the mouse.

const SCENE_URL =
  'https://huggingface.co/datasets/live9080/dreamlake-ceramics/resolve/main/episodes/episode_a/scene.glb'
const DURATION = 5

export function BasicSpec() {
  const [scene, setScene] = useState<Scene3dInput | null>(null)
  const [error, setError] = useState<string | null>(null)
  const [time, setTime] = useState(0)
  const raf = useRef(0)

  useEffect(() => {
    let cancelled = false
    loadGltf(SCENE_URL, 'glb')
      .then((meshes) => {
        if (!cancelled)
          setScene({ meshes, transforms: [], vertexTracks: [], pointTracks: [], upVector: null })
      })
      .catch((e) => {
        if (!cancelled) setError((e as Error).message)
      })
    return () => {
      cancelled = true
    }
  }, [])

  useEffect(() => {
    if (!scene) return
    const start = performance.now()
    const tick = (now: number) => {
      setTime(((now - start) / 1000) % DURATION)
      raf.current = requestAnimationFrame(tick)
    }
    raf.current = requestAnimationFrame(tick)
    return () => cancelAnimationFrame(raf.current)
  }, [scene])

  if (error) return <div className="font-mono text-sm text-red-600">{error}</div>
  if (!scene) return <div className="text-sm opacity-60">Loading scene…</div>
  return (
    <div style={{ height: 360, borderRadius: 8, overflow: 'hidden' }}>
      <EpisodeRecon3d scene={scene} time={time} duration={DURATION} />
    </div>
  )
}
```

## Data in

Geometry comes from a **glTF / GLB** file; everything per-frame comes from the
dataset as separate tracks. **The glTF node name is the join key** — a
`transform3d` track named `ruler` (or `ruler_pose`, `poses[ruler]`) drives the
node named `ruler`.

```ts

const meshes = await loadGltf(url, 'glb')       // nodes: ruler, toy, left_hand, …
const scene: Scene3dInput = {
  meshes,
  transforms: [],    // { name, timestamps, values (rows×7), layout }
  vertexTracks: [],  // { name, count, fps?, vertexCount, at(i) } — deforming meshes
  pointTracks: [],   // animated 3D point sets (21 points draw a hand skeleton)
  upVector: null,    // gravity in the data's frame → upright grid
}
<EpisodeRecon3d scene={scene} time={t} duration={clipSeconds} />
```

`pointTrackFromPose3d(name, payload)` turns a `pose3d` read into a point
track.

## Props

| prop | type | meaning |
| --- | --- | --- |
| `scene` | `Scene3dInput` | meshes + per-frame tracks (see above) |
| `time` | `number \| null` | playback position (seconds); null → first frame |
| `duration` | `number` | clip length — maps time onto each track's own span |
| `trail` | `{ ahead?, behind? } \| false` | motion-trail window, clip seconds around the playhead (default `{ ahead: 1, behind: 0 }` — pure future, gone at the clip's end; `false` for none) |
| `interactive` | `boolean` | orbit controls (default on; off for thumbnails) |

In a `.dreamrc`, the `recon3d` view wraps this — `geometry` and `tracks` bind the
`mesh3d` / `transform3d` / `vertices3d` / `pose3d` tracks and the shared
cursor drives `time`.

---

Source: https://viz.dreamlake.ai/components/episode-line-chart

# EpisodeLineChart

A single time-series plot rendered into a 100×100 SVG viewBox. Each series
is two parallel arrays — `x` (time in seconds, sorted ascending) and `y`
(value at that time) — plus stroke styling. Adjacent samples are connected
with a straight line. The tooltip header reports the **cursor's exact
time** — an interpolated x position between samples, freely landing
anywhere along the chart. Per-channel y values are the **nearest** x
sample to that cursor (no interpolation), so each value the user reads is
always a real recorded number rather than a synthesized one.

Like [EpisodeTimeline](/components/episode-timeline.md) and
[EpisodeVideoStack](/components/episode-video-stack.md), the cursor is
hover-driven and shared across surfaces via three controlled props:
`time`, `onHover`/`onHoverEnd`, and `isScrubOwner`. Hovering any chart in
a group moves the cursor on every peer, and **every chart in the group
shows its own tooltip** at the shared `time` — the readout pattern is
"compare values at one moment across N synced charts." Only the owner
draws the bright accent hairline; peers drop to a muted 22%-ink hairline
so the active chart still stands out.

## Basic

A single chart driving its own cursor. `isScrubOwner` defaults to `true`,
so the bright accent hairline + tooltip appear whenever the pointer is over
the plot.

```tsx file="BasicSpec.tsx"

export const BasicSpec = () => {
  const [time, setTime] = useState<number | null>(null)
  return (
    <div className="h-[180px]">
      <EpisodeLineChart
        title="Left arm"
        caption="signal · joints"
        series={LARM_JOINTS_SERIES}
        duration={DEMO_DURATION}
        unitHint="joint angles · target vs realized"
        time={time}
        onHover={setTime}
        onHoverEnd={() => setTime(null)}
      />
    </div>
  )
}
```

## Multi-chart sync

The canonical Episode-view bottom row: four plots sharing one
`time` + `ownerId` state, with the right-arm chart in disabled / NO SIGNAL
mode. Hovering any chart updates the cursor on every other, and **every
non-disabled chart renders its own tooltip** at the same `t` so you can
compare per-channel readings across plots at a glance.

```tsx file="MultiChartSyncSpec.tsx"

// Mirrors the design's Episode-view bottom row: four time-series plots
// (joints, motor temperatures, right-arm = NO SIGNAL, tactile) sharing
// one hover cursor. Hovering any chart updates the cursor on every
// other chart; only the one under the pointer renders the tooltip.
export const MultiChartSyncSpec = () => {
  const [time, setTime] = useState<number | null>(null)
  const [ownerId, setOwnerId] = useState<string | null>(null)

  const bind = (id: string) => ({
    time,
    isScrubOwner: ownerId === id,
    onHover: (t: number) => { setTime(t); setOwnerId(id) },
    onHoverEnd: () => { setTime(null); setOwnerId(null) },
  })

  return (
    <div className="grid grid-cols-[repeat(auto-fit,minmax(220px,1fr))] gap-2 [grid-auto-rows:160px]">
      <EpisodeLineChart
        {...bind('joints')}
        title="Left arm"
        caption="signal · joints"
        series={LARM_JOINTS_SERIES}
        duration={DEMO_DURATION}
        unitHint="joint angles · target vs realized"
        yRange={[-1, 1]}
      />
      <EpisodeLineChart
        {...bind('temp')}
        title="Left arm motors"
        caption="signal · temperature"
        series={LARM_TEMP_SERIES}
        duration={DEMO_DURATION}
        unitHint="motor temperature · per channel"
        yRange={[-1, 1]}
      />
      <EpisodeLineChart
        {...bind('rarm')}
        title="Right arm"
        caption="signal · joints"
        series={[]}
        duration={DEMO_DURATION}
        disabled
      />
      <EpisodeLineChart
        {...bind('touch')}
        title="Touch"
        caption="signal · tactile"
        series={TOUCH_SERIES}
        duration={DEMO_DURATION}
        unitHint="tactile force magnitude · per finger"
        yRange={[-1, 1]}
      />
    </div>
  )
}
```

## Synced with EpisodeTimeline + EpisodeVideoStack

The full Episode-view layout — video stack + timeline + four time-series
plots, all driven by one shared cursor.

```tsx file="EpisodeViewSpec.tsx"

// Full Episode-view replica: a three-camera video stack + a zoomable
// timeline + four time-series plots, all driven by one shared cursor.
// Mirrors the layout used by `_DLEpisodeDetailItem` in the design
// (videos row · timeline row · time-series row).
const SAMPLE = '/preview-fixtures/episode-demo.mp4'

const VIDEOS: VideoSource[] = [
  { id: 'front', src: SAMPLE, title: 'front', subtitle: '/camera/front/image_compressed' },
  { id: 'wrist', src: SAMPLE, title: 'wrist', subtitle: '/camera/wrist/image_compressed' },
  { id: 'top',   src: SAMPLE, title: 'top',   subtitle: '/camera/top/image_compressed' },
]

export const EpisodeViewSpec = () => {
  const [hover, setHover] = useState<number | null>(null)
  const [ownerId, setOwnerId] = useState<string | null>(null)
  const [activeVideoId, setActiveVideoId] = useState<string>(VIDEOS[0].id)

  // Hover binding shared by every surface in the episode view —
  // any pointer-move flips the cursor and the active surface id, so
  // every other surface re-renders its scrub line at the same time
  // with a muted tone.
  const bind = (id: string) => ({
    time: hover,
    isScrubOwner: ownerId === id,
    onHover: (t: number) => { setHover(t); setOwnerId(id) },
    onHoverEnd: () => { setHover(null); setOwnerId(null) },
  })

  // EpisodeVideoStack doesn't take `isScrubOwner` — it routes hover
  // through activeId. We funnel the stack's hover events into the
  // same shared state, marking the stack as the owner whenever any
  // tile is the active one.
  const stackBind = {
    time: hover,
    onHover: (t: number) => { setHover(t); setOwnerId('stack') },
    onHoverEnd: () => { setHover(null); setOwnerId(null) },
  }

  const framesLabel = activeVideoId
    ? `KEYFRAMES · ${activeVideoId.toUpperCase()}`
    : 'FRAMES'

  return (
    <div className="flex flex-col gap-2">
      <EpisodeVideoStack
        {...stackBind}
        videos={VIDEOS}
        duration={DEMO_DURATION}
        activeId={activeVideoId}
        onActiveChange={setActiveVideoId}
      />
      <EpisodeTimeline
        duration={DEMO_DURATION}
        frames={DEMO_FRAMES}
        tracks={DEMO_TRACKS}
        framesLabel={framesLabel}
        time={hover}
        onHover={(t) => { setHover(t); setOwnerId('timeline') }}
        onHoverEnd={() => { setHover(null); setOwnerId(null) }}
      />
      <div className="grid grid-cols-[repeat(auto-fit,minmax(220px,1fr))] gap-2 [grid-auto-rows:160px]">
        <EpisodeLineChart
          {...bind('joints')}
          title="Left arm"
          caption="signal · joints"
          series={LARM_JOINTS_SERIES}
          duration={DEMO_DURATION}
          unitHint="joint angles · target vs realized"
          yRange={[-1, 1]}
        />
        <EpisodeLineChart
          {...bind('temp')}
          title="Left arm motors"
          caption="signal · temperature"
          series={LARM_TEMP_SERIES}
          duration={DEMO_DURATION}
          unitHint="motor temperature · per channel"
          yRange={[-1, 1]}
        />
        <EpisodeLineChart
          {...bind('rarm')}
          title="Right arm"
          caption="signal · joints"
          series={[]}
          duration={DEMO_DURATION}
          disabled
        />
        <EpisodeLineChart
          {...bind('touch')}
          title="Touch"
          caption="signal · tactile"
          series={TOUCH_SERIES}
          duration={DEMO_DURATION}
          unitHint="tactile force magnitude · per finger"
          yRange={[-1, 1]}
        />
      </div>
    </div>
  )
}
```

### Cursor sync protocol

All three surfaces — `EpisodeLineChart`, `EpisodeTimeline`, `EpisodeVideoStack` —
accept the same `time` + `onHover` + `onHoverEnd` triple, so a single
`useState<number | null>` drives them all. `isScrubOwner` is the
EpisodeLineChart-specific knob that decides which chart in a group renders the
bright hairline + tooltip; peer charts render a muted hairline at the same
`time` with no tooltip:

```tsx
const [hover, setHover] = useState<number | null>(null)
const [ownerId, setOwnerId] = useState<string | null>(null)

const bind = (id: string) => ({
  time: hover,
  isScrubOwner: ownerId === id,
  onHover: (t: number) => { setHover(t); setOwnerId(id) },
  onHoverEnd: () => { setHover(null); setOwnerId(null) },
})

<EpisodeLineChart {...bind('joints')} series={joints} duration={d} ... />
<EpisodeLineChart {...bind('temp')}   series={temp}   duration={d} ... />
```

Every chart with non-null `time` renders its tooltip, regardless of
`isScrubOwner`. Owner-only effects: the bright accent hairline, and the
tooltip's vertical anchor following the pointer Y (peers anchor at
chart-centre).

`EpisodeTimeline` and `EpisodeVideoStack` plug into the same `hover` state
without `isScrubOwner` — the timeline auto-derives its tone from its own
internal hover, and the stack uses `activeId` for the focused tile. The
`bind('timeline')` flow simply marks the timeline as the cursor owner so
every EpisodeLineChart in the group drops to the muted hairline while the user
scrubs on the timeline.

## Series shape

Each `EpisodeLineChartSeries` carries two parallel arrays plus stroke styling:

```ts
interface EpisodeLineChartSeries {
  id: string
  label?: string                            // shown in the tooltip readout row
  data: { x: number[]; y: number[] }        // x = seconds, sorted ascending; y = value
  color: string                             // any CSS color — EPISODE_LINE_CHART_PALETTE.* recommended
  width?: number                            // viewBox units; default 1.4
  dash?: string                             // e.g. '3 2.4'
  opacity?: number                          // 0..1
  linecap?: 'butt' | 'round' | 'square'
  readout?: boolean                         // include in tooltip; default true
  ghost?: boolean                           // target / reference trace — dimmed row, "tgt" label suffix
  formatValue?: (v: number) => string       // tooltip cell text; default v.toFixed(2)
}
```

The sample density is the caller's choice — the chart draws a polyline
through every point, so a 60-sample series is rougher than a 600-sample
one but both work. Non-finite y values (`NaN` / `Infinity`) break the line:
the polyline stops at the last good sample and restarts at the next one,
so a gap in the recording reads as a gap on screen.

The tooltip looks up the nearest x sample to the cursor and displays
that sample's actual y — no interpolation. `x` must be sorted ascending
for the lookup to work.

## Palette

A six-hue oklch palette is exported as `EPISODE_LINE_CHART_PALETTE`. The hues are
hardcoded — the library doesn't ship CSS tokens for them, and the values
don't theme-flip (chart series colours stay legible against both light and
dark backgrounds at these lightness levels).

| Token | Value |
| --- | --- |
| `EPISODE_LINE_CHART_PALETTE.blue` | `oklch(60% 0.13 235)` |
| `EPISODE_LINE_CHART_PALETTE.orange` | `oklch(63% 0.13 50)` |
| `EPISODE_LINE_CHART_PALETTE.green` | `oklch(58% 0.12 150)` |
| `EPISODE_LINE_CHART_PALETTE.purple` | `oklch(58% 0.13 295)` |
| `EPISODE_LINE_CHART_PALETTE.red` | `oklch(58% 0.17 25)` |
| `EPISODE_LINE_CHART_PALETTE.teal` | `oklch(60% 0.10 200)` |

For "ghost" / target traces, pass `color: 'currentColor'` and `opacity: 0.5`
so the dashed line inherits the chart's ink-tinted foreground and matches
in both themes.

## Tooltip behaviour

The tooltip flips against the chart's own midline:

- **Cursor in the left half** → tooltip on the **right** of the hairline.
- **Cursor in the right half** → tooltip on the **left** of the hairline.

Vertically, the owner chart's tooltip follows the pointer Y and flips
above the cursor when it crosses 55% down the chart. Peer charts have no
live pointer Y, so their tooltip anchors at the chart's vertical centre.

The tooltip floats above any clipping ancestor and follows the chart
on scroll, so a user hovering a chart while scrolling the page sees
the tooltip glide with the hairline. The tooltip is **not** clamped
to the viewport — when scrolling pushes part of it off-screen, that's
accepted rather than letting the tooltip jump unpredictably to stay
visible.

## Disabled state

`disabled` renders the chart with a hatched canvas and a centred "NO
SIGNAL" label. No pointer events, no tooltip. The chart still occupies
its grid cell so the surrounding layout doesn't reflow when a sensor
drops out, and the hatch tone follows the chart's text color in both
themes.

## Props

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `series` | `EpisodeLineChartSeries[]` | — | Channel definitions, drawn in array order. |
| `duration` | `number` | — | Total time span in seconds. Must be `> 0`. |
| `title` | `string` | — | Top-left title (UI weight). |
| `caption` | `string` | — | Uppercase mono eyebrow above the title. |
| `yRange` | `[number, number]` | auto-fit | Y-axis domain. Computed from sampled series + 5% padding when omitted. |
| `time` | `number \| null` | `null` | Shared cursor time in seconds. `null` hides the cursor. |
| `onHover` | `(t: number) => void` | — | Fires continuously on hover with the time at the cursor. |
| `onHoverEnd` | `() => void` | — | Fires when the cursor leaves. |
| `isScrubOwner` | `boolean` | `true` | When `true`, the chart draws the bright accent hairline and its tooltip follows the pointer Y. When `false`, the hairline is a muted 22%-ink stroke and the tooltip anchors at chart-centre. Both states still render their tooltip whenever `time` is non-null. |
| `unitHint` | `string` | — | Sub-label under the tooltip's time header — explains units. |
| `showMidline` | `boolean` | `true` | Render the y-axis midline reference. |
| `disabled` | `boolean` | `false` | Render hatched canvas + "NO SIGNAL"; no interaction. |
| `className` | `string` | — | Extra classes on the wrapper. |

### EpisodeLineChartSeries

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `id` | `string` | — | Stable identifier; React key + tooltip readout row key. |
| `label` | `string` | — | Legend label shown in the tooltip readout row. Series with no `label` are drawn on the chart but omitted from the tooltip. |
| `data` | `{ x: number[]; y: number[] }` | — | Sample points. `x` is time in seconds, sorted ascending; `y` is the value at each x. Arrays must be the same length. Adjacent samples are connected with a straight line; `NaN` y values break the line. |
| `color` | `string` | — | Stroke color. Any CSS color string; prefer `EPISODE_LINE_CHART_PALETTE.*` for parity across charts. |
| `width` | `number` | `1.4` | Stroke width in viewBox units (the viewBox is 100×100, so 1.4 ≈ 1.4px when the chart fills 100px wide). |
| `dash` | `string` | — | SVG `stroke-dasharray` pattern, e.g. `'3 2.4'`. Solid when omitted. |
| `opacity` | `number` | `1` | Stroke opacity 0..1. |
| `linecap` | `'butt' \| 'round' \| 'square'` | `'butt'` | SVG `stroke-linecap`. |
| `readout` | `boolean` | `true` | Include this series in the tooltip readout. Set `false` for decorative / per-axis traces that would crowd the legend (e.g. y / z axes when only fx is the headline reading). |
| `ghost` | `boolean` | `false` | Marks the series as a target / reference trace — dims the readout row, suffixes the label with `"tgt"`, and renders the swatch at width 1 instead of 1.6. |
| `formatValue` | `(v: number) => string` | `v => v.toFixed(2)` | Tooltip cell text. Use this to map a normalized signal back to physical units (e.g. `v => (v * 90).toFixed(1) + '°'`). |

---

Source: https://viz.dreamlake.ai/dataset-viz/internals

# Library internals

This page is for people changing the library, not for people using it. The
user-facing statement of the design is [the architecture](/dataset-viz/overview.md);
if you are preparing a dataset, [the contract](/dataset-viz/requirements.md) is
the page you want. This page goes underneath both.

## The pipeline

One file at a dataset root becomes rendered episodes through five steps, and
nothing else crosses between them:

```
.dreamrc text
   │  parse YAML (the host does this — the library never sees YAML)
   ▼
validateDreamrc(obj)              dreamrc.ts   → DreamrcConfig, or an error naming the key
   │
   ▼
resolveDataset(rc, {rootStorage}) resolve.ts   → storage, then a Dataset, then EpisodeInfo[]
   │                                             (rc.storage → opts.storage → opts.rootStorage)
   ▼
dataset.episode(info)             formats/*    → an Episode handle per episode (cheap, no IO)
   │
   ▼
episode.fields() / .read(ref)     formats/*    → Field[] with kinds, then Payloads
   │
   ▼
<DatasetViz episode views>        components/  → components matched to kinds
```

Two rules about that diagram carry most of the design:

**Storage is injected, never declared with credentials.** `pickStorage` takes
the file's own `storage:` first, then a host-supplied instance, then the root
the host found the file in. A dataset you own therefore omits the block and
renders wherever it is; a config that points at someone else's data declares
it. The library holds no tokens — a host registers an authorized driver under
a name, and the `.dreamrc` only ever names it.

**All IO past enumeration is lazy and inside `Episode`.** `resolveDataset`
returns handles, not data. Nothing is fetched until a component asks for a
field, which is what lets the gallery mount a hundred episodes and fetch for
the two on screen.

## The narrow waist

**Format adapters normalize every wire format into a small, closed set of
payload kinds; view components only ever consume kinds.**

```
storage (bytes)  →  format adapter (parse + normalize)  →  fields with kinds  →  view components
   http/hf/…          lerobot/folder/umi/mcap              video, series, …      videoStack, lineChart, …
```

Three consequences worth internalizing:

- **N formats × M components costs N + M, not N × M.** A new adapter works
  with every component the day it lands; a new component works with every
  adapter. That only holds while the kind set stays closed — adding a kind is
  a deliberate act in `types.ts`, not something an adapter does on the side.
- **The kind set is an in-memory contract, not a file format.** It is the one
  place a "DreamLake shape" exists, and it exists only in RAM. Nothing on disk
  is ever asked to look like it.
- **Compatibility is kind-matching.** A component renders a field exactly when
  it consumes that field's kind, which is why a `.dreamrc` written for one
  dataset ports to another by changing names only.

## Adapters decode; they do not classify

Every adapter answers two questions, and only two:

**"What is in here?"** — the catalog is an INVENTORY. It lists each field's
address plus the facts the container reported (`dtype`, `shape`, `names`,
`ext`, `schema`, `codec`) into `meta`, and draws no conclusion from any of
them. `FieldKind` says how the bytes are addressed — `video`, `frames`,
`image`, `tensor`, `text`, `file` — and nothing about what they mean.

**"Give me this, as that."** — `read(ref, { as })` decodes. The `as` comes
from the view that bound the field, so the interpretation is stated in the
`.dreamrc` and never derived at runtime. `meta` is where a decoder finds what
it needs to do the job: the schema name that selects an MCAP decoder, the
extension that selects a parser, the column list that a parquet decoder
validates against.

This is the whole reason there is no `dataset.kinds` key. It used to exist to
correct a wrong guess; there are no guesses to correct.

The discipline that remains:

- **Validate, then decode.** A decoder checks the shape it was handed and
  throws a message naming the mismatch — `observation.state is float32 [6];
  keypoints needs [J,2] or [J,3]`. A wrong render is the one unacceptable
  outcome, because the user cannot see that it is wrong.
- **Omit, never mislist.** Something the adapter cannot address at all stays
  out of the catalog with a `console.warn` naming it.
- **Facts are free, conclusions are not.** Record anything cheap into `meta` at
  catalog time; resolve nothing. A label table is fetched when a binding asks
  for spans, not because a column was called `task_index`.

## The registries

Three `Map`s, one per extension point, in `registry.ts`:

```ts
registerStorage('dlProject', (config) => new DLProjectStorage(config))
registerFormat('myformat', (storage, config) => new MyDataset(storage, config))
registerComponent({ name: 'myView', component: MyView, reads: { series: ['series'] } })
```

A host registers before it resolves. Unknown names fail with a
nearest-match suggestion rather than a stack trace — these names are typed by
hand (and by agents) into YAML, so the error message is part of the API.

This is also the extension path for data we do not know: an unrecognized kind
is preserved verbatim through the catalog and handed to the component as a
plain file URL, so an application can register a component for `myKind` and
parse its own wire format without a change to the library.

## Laziness and caching

The performance rules are structural, not optimizations bolted on:

- **Read only what the summary needs.** MCAP opens through its summary
  section; parquet is classified from its footer; a zarr array from its
  metadata. A 512 MB MCAP becomes a 30-field catalog in ~2 MB of ranged
  requests, and each field then pulls only the chunks that hold it.
- **Dataset-level resources are fetched once**, per-episode resources per
  episode, and every cache stores the in-flight promise rather than the
  result, so parallel panels share one round trip instead of racing.
- **Range requests everywhere.** Nothing is downloaded whole — not a zarr
  store, not an MCAP, not a parquet. That is what makes "read the data in
  place, in the browser" true rather than aspirational.
- **Windowed reads.** `ReadQuery` carries `timeRange` and `maxPoints` so a
  chart can ask for what it will draw.

## Conventions versus formats

The rule the library is built under was once stated as "we define no file
format"; the precise, current version is: **every self-defined structure is
deliberate, surveyed, versioned — and rare.** Worth being precise about what
that does and does not claim.

A **format** dictates how bytes are laid out. We define none — every byte read
is arranged by someone else's specification: LeRobot's parquet layout, Zarr,
MCAP, WebVTT, COCO, glTF, Parquet, H.264.

A **convention** says where to look and what to call things. Since the catalog
stopped drawing conclusions, only three remain, all in the `folder` adapter,
all about PLACEMENT: a directory is an episode, a basename is a track name,
`annotations/` is searched as well as the episode root.

None of them says what a file contains, which is the property that matters —
rename every file in a folder dataset and the parse result is identical,
because no name was ever evidence. They are also not switchable: `folder`'s one config key (`fps`) states a
fact the layout cannot carry — a frame rate — never a placement or a meaning.
If your layout does not match, the answer is an explicit `dataset.annotations`
path or a different `format`, not a knob. Worth revisiting if a real dataset turns up
that the placement rules get wrong.

One on-disk shape is genuinely ours and is named as such: the motion-track
Parquet profile v1 (`tx,ty,tz` plus a quaternion, plus a grouping column).
Parquet is Apache's; that column layout is not written down anywhere else.
An animated glTF — node TRS channels, read into the same tracks — carries the
same information inside an existing standard.

Defining a structure of our own is **allowed, with an admission bar** — the
thing the bar prevents is the casual, unexamined format, because an
un-thought-through structure is how a dataset ends up unvisualizable in a way
no adapter can fix. Before a new DreamLake structure ships: survey how the
existing implementations model the same data (LeRobot, Rerun, Foxglove
schemas, glTF, COCO, nuScenes, …) and write down what was taken from each;
give it its own spec section and a version (the way the motion-track profile
has one); keep it normalizable to and from the nearest standards where
possible. When an existing format genuinely fits, a reader for it still beats
a structure of ours — the ladder in
[the architecture](/dataset-viz/overview.md#where-the-bytes-come-from--the-standards-ladder)
is tried top-down.

---

Source: https://viz.dreamlake.ai/file-preview/composed

# FilePreview

The composed preview layer is the one most apps will mount directly.
Each component pairs a loader with the matching view, owns its
`useLoader` call, and dispatches ``
into the body slot while the data is in flight.

All composed previews follow the same prop contract:
`{ src, metadata?, onSave? }`. Hand them a URL (signed HTTP, `blob:`,
or `data:`) and they handle the rest.

## Usage

Pick the right container for the file shape, or let `FilePreview`
dispatch by extension:

```tsx file="App.tsx"

export function App({ file, src }) {
  return (
    <FilePreview
      src={src}
      metadata={{ name: file.name, ext: file.ext, path: file.path, size: file.size, modified: file.modified }}
    />
  )
}
```

If you want a custom header / chrome, skip `FilePreview` and compose
the matching per-format container yourself:

```tsx file="CustomLayout.tsx"

export function CustomLayout({ file, src }) {
  return (
    <div className="flex flex-col h-full">
      <PreviewHeader name={file.name} ext={file.ext} size={file.size} />
      <div className="flex-1 min-h-0">
        <CsvPreview src={src} metadata={file} />
      </div>
    </div>
  )
}
```

The demos below pass the loaders small `data:` URIs so the page
exercises the full loader → view path end-to-end without a network
fixture. In a real app the URLs would be signed HTTP URLs from your
storage backend.

## Media

### `ImagePreview`

Wraps `ImageView` and surfaces a tiny metadata grid (format,
resolution, size, modified). No loader is involved — the `<img>`
element does its own decode. Pass a `blob:` URL via `src` (e.g.
`URL.createObjectURL(file)`) for just-uploaded files so they render
without a network round-trip.

```tsx file="ImagePreviewSpec.tsx"
// ImagePreview demo — points at placehold.co so the browser reports real
// natural dimensions for the `resolution` row. The placeholder service
// generates the image on the fly; size is omitted from metadata since we
// can't know it ahead of time without a HEAD request.

export const ImagePreviewSpec = () => (
  <ImagePreview
    src="https://placehold.co/640x360?text=ImagePreview+sample"
    metadata={{ name: 'sample.png', ext: 'png' }}
  />
)
```

### `VideoPreview`

Wraps `VideoView` and surfaces duration + resolution once the
demuxer has the header. The browser streams chunks via Range
requests as the user scrubs, so previewing a multi-GB recording
never has to download up-front.

```tsx file="VideoPreviewSpec.tsx"
// VideoPreview demo — points at a small mp4 in /public so the demuxer
// can report real duration + resolution, and the byte size matches the
// actual on-disk fixture.

export const VideoPreviewSpec = () => (
  <VideoPreview
    src="/preview-fixtures/sample.mp4"
    metadata={{ name: 'sample.mp4', ext: 'mp4', size: 19094 }}
  />
)
```

The demo above pulls a small clip from a public CDN. In a real host
you pass the same signed URL your data layer hands out.

#### When the browser can't decode it

A `<video>` that cannot decode its input fails quietly — it paints
black and reports nothing, which reads to the user as "the preview is
broken" rather than "your browser can't play H.265." `VideoPreview`
turns all three shapes of that failure into a stated reason:

- **The element raises a `MediaError`** — network, decode, or
  unsupported source. The numeric code picks the headline; the raw
  browser string (often an internal demuxer code) is appended
  verbatim under it.
- **Metadata parses but the picture is 0×0** — no `MediaError` is
  raised at all. This is the classic black-screen case: the container
  demuxed, the video codec did not. Audio may even play.
- **Nothing happens for `stallMs`** (15s default) — non-fatal, so the
  player stays mounted and only gets an advisory banner. A large file
  on a slow link looks identical to a stuck decode at second five.

```tsx file="VideoErrorSpec.tsx"
// VideoPreview failure demo — the src is a valid data URI that claims to be
// an MP4 but contains no demuxable stream, so every browser rejects it with
// MEDIA_ERR_SRC_NOT_SUPPORTED. Stands in for the real-world case: a file whose
// container or codec (HEVC, AV1, ProRes, .mkv, .avi) the browser can't decode.

const UNDECODABLE = 'data:video/mp4;base64,AAAAAAAAAAAAAAAA'

export const VideoErrorSpec = () => (
  <VideoPreview
    src={UNDECODABLE}
    metadata={{ name: 'capture.mov', ext: 'mov', size: 41288301 }}
  />
)
```

Every message carries a remediation line — re-encode to H.264/AAC MP4
or VP9/Opus WebM — because in practice the fix is nearly always a
transcode on the producing side, not something the viewer can change.
Hosts composing `VideoView` directly get the same classification
through its `onError` callback, typed as `VideoErrorInfo`.

## Data formats

### `CsvPreview`

Calls `loadCsv` and feeds the result into `TableView`. The sub-bar
reports columns, total rows, the detected delimiter, and how many
bytes the range read actually pulled.

```tsx file="CsvPreviewSpec.tsx"
// CsvPreview demo — feeds the loader a small data: URI so the spec runs
// end-to-end in the docs site without needing a network fixture.
//
// The loader does Range GETs internally; browsers return 200 + full body
// on data: URIs (ignoring the Range header), which the loader accepts as
// a small-enough file. For real previews you'd pass a signed HTTP URL.

const SAMPLE = `id,episode,step,reward,done
1,ep-0001,0,0.0,false
2,ep-0001,1,0.12,false
3,ep-0001,2,0.34,false
4,ep-0001,3,0.87,true
5,ep-0002,0,0.0,false
6,ep-0002,1,0.21,false
7,ep-0002,2,0.51,true
8,ep-0002,3,0.93,true
`

const src = 'data:text/csv;charset=utf-8,' + encodeURIComponent(SAMPLE)

export const CsvPreviewSpec = () => (
  <div className="h-[320px] overflow-hidden">
    <CsvPreview src={src} />
  </div>
)
```

### `JsonPreview`

Calls `loadJson` and renders the parsed value through `JsonTreeView`,
with a `tree / raw` toggle in the sub-bar. If parsing fails, the
toggle disables and the raw body shows underneath an inline error.

```tsx file="JsonPreviewSpec.tsx"
// JsonPreview demo — runs loadJson against a data: URI and renders the
// tree view. Click the `tree / raw` toggle in the sub-bar to swap modes.

const VALUE = {
  run: 'rl/2026-05-09/seed-42',
  config: {
    algo: 'ppo',
    horizon: 2048,
    gamma: 0.995,
    optimizer: { kind: 'adam', lr: 3e-4, eps: 1e-8 },
    env: { name: 'PandaReach-v3', n: 32, seed: 42 },
  },
  metrics: {
    reward_mean: 0.873,
    reward_std: 0.214,
    success_rate: 0.91,
    episodes: 1280,
  },
  tags: ['rl', 'panda', 'reach', 'curriculum'],
  finished: true,
}

const src =
  'data:application/json;charset=utf-8,' + encodeURIComponent(JSON.stringify(VALUE, null, 2))

export const JsonPreviewSpec = () => (
  <div className="h-[360px] overflow-hidden">
    <JsonPreview src={src} />
  </div>
)
```

### `JsonlPreview`

Calls `loadJsonl` and feeds the records into `JsonlView`. Long files
report an *estimated* line count derived from the bytes-per-line
ratio of the sliced chunk.

```tsx file="JsonlPreviewSpec.tsx"
// JsonlPreview demo — loadJsonl range-fetches the first ~1 MB and parses
// each line. For the data: URI here the entire body fits in one shot.

const RECORDS = [
  { t: 0.00, kind: 'reset', env: 0, obs_hash: 'a3f1' },
  { t: 0.02, kind: 'step', env: 0, action: [0.12, -0.03, 0.04], r: 0.0,  done: false },
  { t: 0.04, kind: 'step', env: 0, action: [0.18,  0.01, 0.05], r: 0.12, done: false },
  { t: 0.06, kind: 'step', env: 0, action: [0.21, -0.02, 0.07], r: 0.34, done: false },
  { t: 0.08, kind: 'step', env: 0, action: [0.10,  0.04, 0.02], r: 0.87, done: true  },
  { t: 0.10, kind: 'reset', env: 0, obs_hash: 'c0e7' },
  { t: 0.12, kind: 'step', env: 0, action: [0.05,  0.02, 0.01], r: 0.05, done: false },
]
const SAMPLE = RECORDS.map((r) => JSON.stringify(r)).join('\n') + '\n'

const src = 'data:application/x-ndjson;charset=utf-8,' + encodeURIComponent(SAMPLE)

export const JsonlPreviewSpec = () => (
  <div className="h-[280px] overflow-hidden">
    <JsonlPreview src={src} />
  </div>
)
```

### `TextPreview`

Calls `loadText` and feeds the body into `TextView`. Pass `onSave`
to grow an Edit button; the promise drives the saving state and any
thrown error surfaces inline. The demo's fake save fails on payloads
over 4 KB so the error state is reachable too.

```tsx file="TextPreviewSpec.tsx"
// TextPreview demo — load + edit + save. The save handler is faked with a
// 600ms timeout and rejects on >4 KB payloads so the error state is
// reachable too.

const SAMPLE = `# notes.md
- Verify the value-function head still trains on the new env.
- Re-run sweep at gamma in {0.99, 0.995, 0.999}.
- File regressions land in the parquet under datasets/2026-05/.

> The reward shaping change is responsible for the ~7% bump on
> success-rate. Roll forward.
`

export const TextPreviewSpec = () => {
  const [stored, setStored] = useState(SAMPLE)
  const src = 'data:text/markdown;charset=utf-8,' + encodeURIComponent(stored)
  const onSave = async (next: string) => {
    await new Promise((r) => setTimeout(r, 600))
    if (next.length > 4000) throw new Error('payload too large (demo cap = 4 KB)')
    setStored(next)
  }
  return (
    <div className="h-[320px] overflow-hidden">
      <TextPreview src={src} metadata={{ name: 'notes.md', ext: 'md' }} onSave={onSave} />
    </div>
  )
}
```

## Binary formats

These three previews each touch a small fixed slice of the file —
Parquet's footer, the NPy header, MCAP's header + footer + summary —
so the work is roughly constant regardless of file size. They drop
into the same prop contract as the others; no demos here only
because the docs site doesn't ship binary fixtures.

### `ParquetPreview`

```tsx file="ParquetPreview.tsx"

<ParquetPreview src={signedUrl} metadata={file} />
```

Reads the metadata + the first 1000 rows via [hyparquet](https://github.com/hyparam/hyparquet),
renders through `TableView`. Files larger than `MAX_PARQUET_BYTES`
(5 MB) trigger a `too-large` status because Parquet's metadata
layout makes "first N MB" meaningless — the schema lives at the end
of the file. Override via `maxBytes` on the loader if your host can
handle bigger.

### `NpyPreview`

```tsx file="NpyPreview.tsx"

<NpyPreview src={signedUrl} />
```

Range-fetches the first 256 bytes (and one more range if the header
dict claims to be longer), parses dtype / shape / element count, and
renders the result as a `KeyValueView`. Never touches the element
data — previewing a 50 GB array is the same fetch as a 1 KB array.

### `McapPreview`

```tsx file="McapPreview.tsx"

<McapPreview src={signedUrl} metadata={file} />
```

Reads the leading Header record, the trailing Footer, and the
summary section the footer points to. Renders top-level metadata
through `KeyValueView` and the per-channel stats through a nested
`TableView` (with the sub-bar hidden — the parent already owns one).

## `FilePreview` dispatcher

The convenience entry-point — pass `FileMetadata` plus a `src` and
`FilePreview` reads `metadata.ext`, picks the matching container, and
mounts a stable `PreviewHeader` on top.

```tsx file="FilePreviewSpec.tsx"
// FilePreview switcher demo — pick a file type and the dispatcher routes
// to the matching composed preview. PreviewHeader sits on top, so the same
// chrome wraps every body.

interface Sample {
  ext: string
  name: string
  src: string
  size?: number
}

const CSV = `id,episode,step,reward,done
1,ep-0001,0,0.0,false
2,ep-0001,1,0.12,false
3,ep-0001,2,0.34,false
4,ep-0001,3,0.87,true
5,ep-0002,0,0.0,false
`

const JSON_VALUE = {
  run: 'rl/2026-05-09/seed-42',
  config: { algo: 'ppo', horizon: 2048, gamma: 0.995 },
  metrics: { reward_mean: 0.873, success_rate: 0.91 },
  finished: true,
}

const JSONL_RECORDS = [
  { t: 0.00, kind: 'reset', env: 0 },
  { t: 0.02, kind: 'step', env: 0, action: [0.12, -0.03], r: 0.0 },
  { t: 0.04, kind: 'step', env: 0, action: [0.18,  0.01], r: 0.12 },
  { t: 0.06, kind: 'step', env: 0, action: [0.21, -0.02], r: 0.34 },
]

const NOTES = `# notes.md
- Reward shaping change → +7% success rate.
- Roll forward to staging.
`

const IMAGE =
  'data:image/svg+xml;utf8,' +
  encodeURIComponent(
    `<svg xmlns="http://www.w3.org/2000/svg" width="640" height="360" viewBox="0 0 640 360">
       <rect width="640" height="360" fill="#23aaff"/>
       <text x="32" y="56" fill="white" font-family="JetBrains Mono" font-size="22">heatmap.png</text>
     </svg>`,
  )

const SAMPLES: Record<string, Sample> = {
  csv: {
    ext: 'csv', name: 'trajectories.csv',
    src: 'data:text/csv;charset=utf-8,' + encodeURIComponent(CSV),
  },
  json: {
    ext: 'json', name: 'run-summary.json',
    src: 'data:application/json;charset=utf-8,' + encodeURIComponent(JSON.stringify(JSON_VALUE, null, 2)),
  },
  jsonl: {
    ext: 'jsonl', name: 'events.jsonl',
    src: 'data:application/x-ndjson;charset=utf-8,'
      + encodeURIComponent(JSONL_RECORDS.map((r) => JSON.stringify(r)).join('\n') + '\n'),
  },
  md: {
    ext: 'md', name: 'notes.md',
    src: 'data:text/markdown;charset=utf-8,' + encodeURIComponent(NOTES),
  },
  svg: {
    ext: 'svg', name: 'heatmap.svg',
    src: IMAGE,
  },
  bin: {
    ext: 'bin', name: 'weights.bin',
    src: 'data:application/octet-stream;base64,AAAA',
  },
}

const KINDS = Object.keys(SAMPLES) as Array<keyof typeof SAMPLES>

export const FilePreviewSpec = () => {
  const [kind, setKind] = useState<keyof typeof SAMPLES>('csv')
  const sample = SAMPLES[kind]
  return (
    <div className="flex flex-col gap-3">
      <div className="flex flex-wrap gap-1.5">
        {KINDS.map((k) => {
          const active = k === kind
          return (
            <button
              key={k}
              onClick={() => setKind(k)}
              className={
                'inline-flex items-center px-2.5 py-1 rounded-md font-mono text-[10px] '
                + 'font-semibold tracking-wider uppercase border cursor-pointer '
                + (active
                  ? 'bg-[#1a1a1a] dark:bg-[#ececec] text-[#fffefa] dark:text-[#2e2e35] border-[#1a1a1a] dark:border-[#ececec]'
                  : 'bg-transparent text-[#6b6b6b] dark:text-[#8a8a8a] border-black/[0.08] dark:border-white/[0.09] hover:bg-black/[0.04] dark:hover:bg-white/[0.05]')
              }>
              {k}
            </button>
          )
        })}
      </div>
      <div className="h-[400px] overflow-hidden">
        <FilePreview
          src={sample.src}
          metadata={{ name: sample.name, ext: sample.ext, size: sample.size }}
        />
      </div>
    </div>
  )
}
```

The switcher above remounts `FilePreview` against the same prop
contract every time the file kind changes — `csv`, `json`, `jsonl`,
`md` all route to their loader + view pair, `svg` goes through
`ImagePreview` (no loader), and an unknown extension like `bin`
falls back to the `unsupported` status.

Unsupported extensions are anything outside the bundled
`IMAGE_EXTS` / `VIDEO_EXTS` / `TEXT_EXTS` sets and the binary formats
(`csv`, `parquet`, `json`, `jsonl`, `npy`, `mcap`). The host can
either add another `case` upstream, or render its own
`` from the file metadata.

## Props reference

Every per-format container accepts the same `PreviewProps`. The
`FilePreview` dispatcher requires `metadata` (so it can read
`metadata.ext` to pick the body); the per-format containers leave it
optional.

### `PreviewProps`

| Prop | Type | Default | Description |
|---|---|---|---|
| `src` | `string` | — | URL the renderer / loader reads from. Accepts HTTP URLs, `blob:` URLs (e.g. `URL.createObjectURL` of a just-uploaded file), or `data:` URLs. Caller refreshes signed URLs before expiry. Required. |
| `metadata` | `FileMetadata` | — | File metadata used by `PreviewHeader` and the image / video info grids. Loaders read `metadata.size` to skip a `HEAD` round-trip. Optional on per-format containers; required on `FilePreview`. |
| `onSave` | `(text: string) => Promise<void>` | — | Only used by `TextPreview`. When supplied, the sub-bar grows an Edit button. |

### `FileMetadata`

| Field | Type | Description |
|---|---|---|
| `name` | `string` | Required. The filename rendered as the bold second line in the header. |
| `ext` | `string` | Extension (`png`, `mp4`, `parquet`, …). Drives icon selection and `FilePreview`'s dispatch. |
| `path` | `string` | Full file path. The parent directory is shown above the filename; falls back to `/` for bucket-root files. |
| `size` | `number` | Raw bytes. Formatted via `fmtSize` on render. |
| `modified` | `Date \| string` | Last-modified timestamp. Dates are formatted as `YYYY-MM-DD HH:mm`; strings pass through. |

### `LoaderOpts`

Pass-through options the loaders accept. Composed previews thread
`{ size, signal }` through automatically — set the others when you
call the loader directly.

| Field | Type | Description |
|---|---|---|
| `size` | `number` | Skip the `HEAD` probe by passing the byte size you already have. |
| `signal` | `AbortSignal` | Cancellation. `useLoader` plumbs one in for every fetch. |
| `maxBytes` | `number` | Override the default slice cap (per loader). |
| `maxRows` | `number` | Override the default row / line cap (CSV, Parquet, JSONL). |

---

Source: https://viz.dreamlake.ai/schema-viz/storage

# Storage

A source's **storage** answers one question: _where do the bytes live, and how do
I reach them?_ The [adapter](/schema-viz/adapters.md) above it sees one uniform
interface (a `Storage`):

```ts
interface Storage {
  // logical path → a fetchable URL (signed/expiring when the backend needs it)
  resolveUrl(path: string, opts?: { ttlSeconds?: number }): Promise<string>
  // list one directory — bounded, non-recursive
  list(path: string, opts?: { token?: string; limit?: number }): Promise<Listing>
}
interface Listing {
  entries: Entry[]
  nextToken?: string
}
interface Entry {
  name: string
  path: string
  type: 'file' | 'dir'
  size?: number
}
```

A driver is named by `storage.driver` in a [source](/schema-viz/schema.md).

> Storage deals only in `Entry.type` (`'file'` / `'dir'`) — directory structure.
> It knows **nothing** about a field's [`kind`](/schema-viz/concept.md#kind-is-the-joint-the-system-turns-on)
> (`video`/`series`/…); that is the [adapter](/schema-viz/adapters.md)'s job. The two
> are different concepts that both happen to be called "kind" in some backends —
> see the note in the example below.

## Choosing a driver

`schema-viz` is a **stateless public component** — it holds no credentials and
has no login session, so it can never authorize private data on its own. That
single fact drives the choice:

| Your data is…                                                                | Driver                                | Auth                                 |
| ---------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------ |
| **Public** (a public HF/GitHub URL, an S3 public bucket, your own `public/`) | the built-in **`http`**               | none                                 |
| **Private** GitHub / HuggingFace / DreamLake                                 | a driver **injected by the host app** | the host's session, never the schema |

The schema **never contains a token**. Public data needs none; private data is
brokered by the host application, which owns the login session.

## The `http` driver (public)

The only built-in driver. `resolveUrl` joins `basePath` + path (absolute URLs
pass through); `list` reads a co-located `index.json` (a static host can't
enumerate a folder itself).

```yaml
# a public HuggingFace dataset — basePath is its `resolve` URL
storage: { driver: http, basePath: https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main }

# a folder of files you serve yourself — list() reads index.json in that folder
storage: { driver: http, basePath: /viz-samples/episode_365/ }
```

| Key        | Type   | Default | Notes                                                      |
| ---------- | ------ | ------- | ---------------------------------------------------------- |
| `basePath` | string | `""`    | Prefix for every relative path; absolute URLs pass through |

For directory discovery (the [`filesystem`](/schema-viz/adapters.md) adapter), drop
an `index.json` in the folder listing its files.

## Private data — injected by the host

The **host app provides the driver** via the `extensions.storages` prop (or a
global registration). The schema only names it; the driver's `resolveUrl`/`list`
call the app's **own authorized endpoint**, and viz never sees a token. This is
exactly how the DreamLake project storage works — `list` hits `/nodes/children`
and `resolveUrl` hits `/nodes/{id}/download`, both through the app's `apiGet`,
which attaches the session bearer token:

```tsx
// In the DreamLake app (NOT in viz) — apiGet attaches the session token.

class DLProjectStorage implements Storage {
  constructor(private rootId: string) {}
  async list(path = ''): Promise<Listing> {
    const parentId = path || this.rootId
    const res = await apiGet<{ nodes: Node[]; totalPages: number }>(
      BASE_URL,
      `/nodes/children?parentId=${parentId}&page=1`
    )
    const entries: Entry[] = res.nodes.map((n) => ({
      name: n.name,
      path: n.id, // ref = node id
      // n.kind is the DreamLake NODE type (folder/episode/file) — NOT a viz Field
      // `kind`. Storage only uses it to decide dir-vs-file; the adapter assigns
      // the field kind later, from the filename.
      type: n.kind === 'folder' || n.kind === 'episode' ? 'dir' : 'file',
    }))
    return { entries }
  }
  // A node id → a short-lived AUTHORIZED download URL (works for <video>/<img>).
  resolveUrl(path: string): Promise<string> {
    return apiGet<{ url: string }>(BASE_URL, `/nodes/${path}/download`).then((r) => r.url)
  }
}
export const dlProjectStorageFactory: StorageFactory = (cfg) =>
  new DLProjectStorage(cfg.root as string)
```

```tsx
// The schema stays token-free — it only names the driver + the folder node.
const schema = {
  version: 1,
  sources: { clips: { adapter: 'filesystem', storage: { driver: 'dlProject', root: '<node id>' } } },
  panels: [{ view: 'videoStack', source: 'clips', fields: ['*'], columns: 3 }],
}

<DatasetPreview schema={schema} extensions={{ storages: { dlProject: dlProjectStorageFactory } }} />
```

Why this shape:

- **No token in the schema.** It is safe to commit, share, and serialize;
  authorization is the host's concern, resolved at request time.
- **Authorized URLs, not headers.** `<video>`/`<img>` fetch their `src` directly
  and cannot attach an `Authorization` header — so private _media_ must come back
  as a signed/redirected URL. The `/nodes/{id}/download` URL is exactly that.
- **Stateless viz.** Standalone (e.g. this docs site) viz reaches only public data
  via `http`; private access lights up only when a host injects its driver.

### Register once, or per preview

`extensions.storages` adds a driver **for that preview, only if free** — handy for
a one-off. When a page mounts **several** ``s sharing the same
driver, register it **once at app boot** instead — every preview sees it, and a
global registration can also override a built-in:

```tsx

registerStorage('dlProject', dlProjectStorageFactory)
```

`registerStorage` overwrites; an `extensions` entry is skipped if the name is
taken. (`registerAdapter` and `registerView` work the same way.) Because the
factory's `resolveUrl` reads the token **at request time** (via `apiGet`), a
once-registered driver always uses the current session.

## Writing your own driver

Any driver implements the same two methods. Recommended patterns: **dedupe
in-flight requests** (memoize the `Promise` per path), **cache signed URLs with
near-expiry renewal** (a `<video>` may stream for minutes), and **pass absolute
URLs through**.

## What storage does not do

- **It does not know formats.** It hands back URLs/bytes; parsing belongs to the
  [adapter](/schema-viz/adapters.md).
- **It does not fetch media content.** It returns a _URL_; the `<video>`/`<img>`
  streams the bytes.
- **It does not recurse.** `list` is one directory.

Next: [Adapters](/schema-viz/adapters.md) — turning storage bytes into a Model.

---

Source: https://viz.dreamlake.ai/file-preview/views

# Views

The view layer is ten pure-presentation primitives. Seven content views —
**Table**, **KeyValue**, **JsonTree**, **Jsonl**, **Image**, **Video**,
**Text** — plus three chrome pieces (**PreviewHeader**, **PreviewSubBar**,
**StatusView**) that frame whichever body the host plugs in. None of them
fetch — they take parsed data and display props, so the host owns URL
signing, caching, and parsing.

For end-to-end previewing (parse + render), see the
[loaders](/file-preview/loaders.md) page for the parsers, or the
[FilePreview](/file-preview/composed.md) page for ready-made
loader + view containers.

## Usage

Every preview view is a pure visualization primitive. The host owns URL
signing, caching, and parsing; the view takes already-parsed data and
display props and renders pixels. None of them fetch.

```tsx file="FilePreview.tsx"

export function FilePreview({ file, parsed, status }) {
  if (status === 'loading') return <StatusView kind="loading" />
  if (status === 'error')   return <StatusView kind="error" message={parsed.error} />

  return (
    <div className="flex flex-col h-full">
      <PreviewHeader
        name={file.name}
        ext={file.ext}
        path={file.path}
        size={file.size}
        modified={file.modified}
      />
      <TableView cols={parsed.cols} rows={parsed.rows} totalRows={parsed.totalRows} truncated />
    </div>
  )
}
```

The split is the load-bearing rule: views never own data. That keeps
the same component reusable across apps with very different fetch
stories — signed-URL S3, IndexedDB cache, server-side parse — without
forking the rendering.

Pair the right view with each file shape:

| File shape | View |
|---|---|
| CSV, Parquet, MCAP channel | `TableView` |
| Single-record metadata | `KeyValueView` |
| JSON document | `JsonTreeView` |
| JSONL stream | `JsonlView` |
| Image (PNG / JPG / WebP / …) | `ImageView` |
| Video (MP4 / WebM / …) | `VideoView` |
| Plain text / Markdown / code | `TextView` |
| Empty / loading / error / too-large / unsupported | `StatusView` |

## Header & sub-bar

Two thin strips that frame every preview body. `PreviewHeader` is the
top row — extension icon · directory · filename · size · modified.
`PreviewSubBar` is the line below — left slot carries a summary, right
slot carries a status dot or an inline action. Both render at fixed
heights so the body underneath doesn't reflow when files change.

### `PreviewHeader`

```tsx file="PreviewHeaderSpec.tsx"
// Header strip demo — one row with the typical fields a file-preview
// pane shows: extension icon · parent directory · filename · size ·
// modified date. The size is passed as raw bytes; PreviewHeader runs
// it through fmtSize() internally.

export const PreviewHeaderSpec = () => (
  <PreviewHeader
    name="trajectories.parquet"
    ext="parquet"
    path="datasets/2026-05/runs/trajectories.parquet"
    size={4_812_344}
    modified={new Date('2026-05-09T14:22:00')}
  />
)
```

The header always renders two lines so the height is stable across
files. For a bucket-root file (no parent directory), the path line
falls back to `/` rather than collapsing. `size` is raw bytes and is
formatted internally via `fmtSize`.

### `PreviewSubBar`

```tsx file="PreviewSubBarSpec.tsx"
// SubBar demo — thin status line under the header. Left slot carries a
// summary (counts, parse status); right slot carries a status dot or
// inline action. Both slots take arbitrary ReactNode, so callers paint
// whatever fits the surface they're embedding into.

export const PreviewSubBarSpec = () => (
  <PreviewSubBar
    left={
      <>
        <span className="font-semibold text-[#1a1a1a] dark:text-[#ececec]">12</span>
        <span> cols ·</span>
        <span className="font-semibold text-[#1a1a1a] dark:text-[#ececec]"> 5,000</span>
        <span> rows</span>
      </>
    }
    right={
      <>
        <span className="text-[#23aaff]">●</span>
        <span>showing first 100 rows</span>
      </>
    }
  />
)
```

The slot model keeps the bar policy-free — `TableView` paints col/row
counts in the left slot and a truncation dot in the right; `TextView`
paints line count + dirty marker on the left and edit / save buttons
on the right. Anything that fits the mono-10.5px register works.

## Tabular data

### `TableView`

The generic table that every tabular file shape (CSV, Parquet, MCAP
channels) renders through. Pass columns + parsed row tuples; cells are
formatted by type — booleans pick up the `fn` / `tag` code-token hues,
numbers tabular-align, nulls render at 40 % opacity.

```tsx file="TableViewSpec.tsx"
// Table demo — typed columns + a handful of rows. The component is the
// generic visualizer used by every tabular file shape (CSV, Parquet,
// MCAP channels). Pass `totalRows` + `truncated` when the host has
// paged the data — the sub-bar then shows the "showing first N" badge.

const cols: TableColumn[] = [
  { name: 'id', type: 'int64' },
  { name: 'episode', type: 'string' },
  { name: 'step', type: 'int32' },
  { name: 'reward', type: 'float32' },
  { name: 'done', type: 'bool' },
]

const rows: unknown[][] = [
  [1, 'ep-0001', 0, 0.0, false],
  [2, 'ep-0001', 1, 0.12, false],
  [3, 'ep-0001', 2, 0.34, false],
  [4, 'ep-0001', 3, 0.87, true],
  [5, 'ep-0002', 0, 0.0, false],
  [6, 'ep-0002', 1, null, false],
  [7, 'ep-0002', 2, 0.51, true],
]

export const TableViewSpec = () => (
  <div className="h-[320px] overflow-hidden">
    <TableView cols={cols} rows={rows} totalRows={5_000} shownRows={rows.length} truncated />
  </div>
)
```

When the host has paged the data (read the first N rows out of a much
larger file), pass `totalRows` plus `truncated`. The sub-bar then paints
the accent dot and "showing first N rows" caption on the right edge. If
every column carries a `type`, the header grows a second mono line for
type labels; columns without a `type` collapse to a single line.

Set `hideSubBar` when embedding the table inside another preview pane
that already owns the status bar — for example, an MCAP channels table
sitting under a metadata `KeyValueView`.

### `KeyValueView`

Vertical metadata table for flat string-keyed maps. Used for file
headers, MCAP metadata blocks, EXIF, schema dumps — anywhere you want
keys right-aligned next to their values in a stable grid.

```tsx file="KeyValueViewSpec.tsx"
// Key/value demo — vertical metadata table. Keys right-aligned in a
// narrow column, values flow in the remainder. Useful for file headers,
// MCAP metadata blocks, EXIF, or any flat string-keyed map.

const items: Array<[string, string]> = [
  ['format', 'Apache Parquet v2.9'],
  ['compression', 'ZSTD (level 3)'],
  ['rows', '5,000'],
  ['row groups', '4'],
  ['created by', 'pyarrow 15.0.2'],
  ['schema sha', 'a31b…f902'],
]

export const KeyValueViewSpec = () => <KeyValueView items={items} />
```

The key column is `width: 1` plus `whitespace-nowrap` — it auto-sizes
to the longest key, which keeps the value column from wandering as
you switch files.

## JSON & JSONL

### `JsonTreeView`

Foldable tree for one JSON document. Levels deeper than `defaultOpenDepth`
start collapsed; click anywhere on a row to toggle. Scalars are colored
by JSON type from the dreamlake code-token palette — keywords blue,
strings amber, numbers purple — so the rendering matches inline JSON
elsewhere.

```tsx file="JsonTreeViewSpec.tsx"
// JSON tree demo — click the triangle on any row to fold / unfold.
// Levels deeper than `defaultOpenDepth` start collapsed; tune the
// initial fold for the shape of data you're previewing.

const value = {
  run: 'rl/2026-05-09/seed-42',
  config: {
    algo: 'ppo',
    horizon: 2048,
    gamma: 0.995,
    optimizer: { kind: 'adam', lr: 3e-4, eps: 1e-8 },
    env: { name: 'PandaReach-v3', n: 32, seed: 42 },
  },
  metrics: {
    reward_mean: 0.873,
    reward_std: 0.214,
    success_rate: 0.91,
    episodes: 1280,
  },
  tags: ['rl', 'panda', 'reach', 'curriculum'],
  finished: true,
  note: null,
}

export const JsonTreeViewSpec = () => (
  <div className="max-h-[320px] overflow-auto">
    <JsonTreeView value={value} defaultOpenDepth={2} />
  </div>
)
```

Tune `defaultOpenDepth` for the data: pass `1` for flat configs (only
the root opens), `3` or `4` for deeply nested traces where the user
probably wants the leaves visible.

### `JsonlView`

One record per line, gutter on the left, inline JSON on the right.
Visual style intentionally tracks `JsonTreeView` so the two read as a
pair when a host pane offers both modes for the same file.

```tsx file="JsonlViewSpec.tsx"
// JSONL demo — one record per line, gutter on the left, inline JSON on
// the right. Visual style matches JsonTreeView so the two read as a
// pair when a host pane offers both modes for the same file.

const records = [
  { t: 0.00, kind: 'reset', env: 0, obs_hash: 'a3f1' },
  { t: 0.02, kind: 'step', env: 0, action: [0.12, -0.03, 0.04], r: 0.0,  done: false },
  { t: 0.04, kind: 'step', env: 0, action: [0.18,  0.01, 0.05], r: 0.12, done: false },
  { t: 0.06, kind: 'step', env: 0, action: [0.21, -0.02, 0.07], r: 0.34, done: false },
  { t: 0.08, kind: 'step', env: 0, action: [0.10,  0.04, 0.02], r: 0.87, done: true  },
  { t: 0.10, kind: 'reset', env: 0, obs_hash: 'c0e7' },
]

export const JsonlViewSpec = () => (
  <div className="max-h-[260px] overflow-auto">
    <JsonlView records={records} />
  </div>
)
```

The view assumes the host has already split the file on newlines and
JSON-parsed each line — pass `records` as the resulting array. Long
lines are clipped with an ellipsis at the row level; if you need full
records, drop into the tree view for the row instead.

## Media

### `ImageView`

`<img>` wrapped in the shadowed card every preview pane uses. The
`onLoad` callback fires once the browser has decoded the image and
passes the natural `"W×H"` string back — most hosts thread that into
the header or sub-bar so users see the source resolution.

```tsx file="ImageViewSpec.tsx"
// Image demo — wraps an <img> in the same shadowed card every preview
// pane uses. `onLoad` fires once the browser has decoded the image and
// reports the natural resolution back to the host, which usually
// surfaces it in the header / sub-bar.

const src = 'https://placehold.co/640x360?text=ImageView+sample'

export const ImageViewSpec = () => {
  const [res, setRes] = useState<string | null>(null)
  return (
    <div className="flex flex-col gap-2">
      <ImageView src={src} alt="Sample gradient" onLoad={setRes} />
      <div className="font-mono text-[10px] uppercase tracking-wider text-[#6b6b6b]/70 dark:text-[#8a8a8a]/70 text-center">
        resolution · {res ?? '—'}
      </div>
    </div>
  )
}
```

The wrapper caps at `max-w-[560px]` and centers — large originals are
scaled down to fit. The image element keeps its natural aspect ratio,
so tall portraits and wide panoramas both lay out cleanly without
extra props.

### `VideoView`

`<video controls>` on a 16:9 canvas. Browsers stream the file in
chunks via Range requests as the user scrubs, so previewing a
multi-GB recording never has to download up-front. `onLoadedMetadata`
surfaces duration and resolution once the demuxer has the header.

```tsx file="VideoViewSpec.tsx"
// Video demo — <video controls> on a 16:9 canvas. The browser streams
// chunks via Range requests as the user scrubs, so a multi-GB recording
// never has to download up-front. `onLoadedMetadata` surfaces duration
// + resolution once the first frames have been demuxed.

const SAMPLE = '/preview-fixtures/sample.mp4'

export const VideoViewSpec = () => {
  const [info, setInfo] = useState<{ duration: number; resolution: string } | null>(null)
  return (
    <div className="flex flex-col gap-2">
      <VideoView src={SAMPLE} onLoadedMetadata={setInfo} />
      <div className="font-mono text-[10px] uppercase tracking-wider text-[#6b6b6b]/70 dark:text-[#8a8a8a]/70 text-center">
        {info
          ? `${info.duration.toFixed(2)}s · ${info.resolution}`
          : 'metadata · —'}
      </div>
    </div>
  )
}
```

The demo above streams a small sample clip from a public CDN; in a
real host you pass the same signed URL your data layer hands out.
Container formats the `<video>` element can demux are
browser-dependent — MP4 (H.264 + AAC) is the safe default.

Because the unsafe cases fail *silently* — a black frame and no
exception — `VideoView` also watches for them and calls `onError` with
a classified `VideoErrorInfo`: `MediaError`s, and the codec-unsupported
case where metadata parses but the picture is 0×0. If neither lands
within `stallMs` it overlays an advisory banner without unmounting the
player. See [the composed
docs](/file-preview/composed.md#when-the-browser-cant-decode-it) for the
rendered failure states.

## Text

`TextView` is the plain-text viewer / editor. Read-only when `onSave`
is omitted — the sub-bar shows a "read-only" tag and the body renders
as a `<pre>`. Pass an `onSave` and an Edit button appears on the right
of the sub-bar; click it to swap the body for a `<textarea>`.

```tsx file="TextViewSpec.tsx"
// Text demo — read-only when `onSave` is omitted; once provided, an
// edit button toggles a textarea and Cmd/Ctrl-S saves (Esc cancels).
// The save handler returns a Promise so the host can drive the saving
// state and surface errors through the sub-bar.

const SAMPLE = `# notes.md
- Verify the value-function head still trains on the new env.
- Re-run sweep at gamma in {0.99, 0.995, 0.999}.
- File regressions land in the parquet under datasets/2026-05/.

> The reward shaping change is responsible for the ~7% bump on
> success-rate. Roll forward.
`

export const TextViewSpec = () => {
  const [stored, setStored] = useState(SAMPLE)
  // Pretend to talk to a backend — fail every other save so the error
  // state is also reachable in the demo.
  const onSave = async (next: string) => {
    await new Promise((r) => setTimeout(r, 600))
    if (next.length > 4000) throw new Error('payload too large (demo cap = 4 KB)')
    setStored(next)
  }
  return (
    <div className="h-[320px] overflow-hidden">
      <TextView text={stored} ext="md" onSave={onSave} />
    </div>
  )
}
```

Keyboard:

- **Cmd/Ctrl-S** saves while editing. The button is disabled until
  the buffer is dirty, and shows a "saving…" label while the promise
  is in flight.
- **Esc** discards local edits and exits edit mode.

`onSave` returns a `Promise<void>`. Resolve to mark the new text as
the baseline; reject to surface the error message inline in the
sub-bar. The view doesn't know what "save" means — wire it to a PUT,
a Yjs awareness message, an IndexedDB write, whatever fits the host.

When the `text` prop changes underneath you (the parent switched files
or another tab saved over this one), the view snaps local edits back
to the new baseline during render — the React 19 prop-derivation idiom,
not a `useEffect`. That keeps the visible state in sync with the
source of truth without a frame of stale display.

## Status states

`StatusView` covers the five non-content states a preview pane can be
in: nothing selected, loading, error, file-too-large-to-preview, and
unsupported-format. One component because they share the same
"icon-on-tile + headline + detail" layout — only the tone changes.

```tsx file="StatusViewSpec.tsx"
// Status demo — one component covers all five non-content states. The
// switcher below toggles between them so the layout, tone, and copy
// can be compared side-by-side without rebuilding the host.

const KINDS: StatusKind[] = ['empty', 'loading', 'error', 'too-large', 'unsupported']

export const StatusViewSpec = () => {
  const [kind, setKind] = useState<StatusKind>('empty')
  return (
    <div className="flex flex-col gap-3">
      <div className="flex flex-wrap gap-1.5">
        {KINDS.map((k) => {
          const active = k === kind
          return (
            <button
              key={k}
              onClick={() => setKind(k)}
              className={
                'inline-flex items-center px-2.5 py-1 rounded-md font-mono text-[10px] ' +
                'font-semibold tracking-wider uppercase border cursor-pointer ' +
                (active
                  ? 'bg-[#1a1a1a] dark:bg-[#ececec] text-[#fffefa] dark:text-[#2e2e35] border-[#1a1a1a] dark:border-[#ececec]'
                  : 'bg-transparent text-[#6b6b6b] dark:text-[#8a8a8a] border-black/[0.08] dark:border-white/[0.09] hover:bg-black/[0.04] dark:hover:bg-white/[0.05]')
              }>
              {k}
            </button>
          )
        })}
      </div>
      <div className="h-[280px] overflow-hidden">
        <StatusView
          kind={kind}
          label="reading parquet…"
          message="ParquetError: incompatible page version (expected 2, got 1)"
          ext="rosbag"
          sizeBytes={420 * 1024 * 1024}
          capBytes={64 * 1024 * 1024}
        />
      </div>
    </div>
  )
}
```

Switch on `kind`:

- **`empty`** — nothing selected. The neutral state at first paint.
- **`loading`** — show while the parse is in flight. `label` overrides
  the default "loading…" caption when the host wants to be specific
  (e.g. `"reading parquet…"`).
- **`error`** — render a parse / fetch error. Pass the message via
  `message`; long errors wrap inside the centered card.
- **`too-large`** — file exceeds the host's preview cap. Pass
  `sizeBytes` (the actual size) and `capBytes` (the cap) so the
  built-in copy can name both. Use `hint` to swap in a different
  message — e.g. a download link or a per-format escape hatch.
- **`unsupported`** — file extension the host has no view for. Pass
  `ext` so the message names the format.

The component fills its container — drop it into the same body slot
the content views would have occupied, no special wrapper needed.

## Props reference

### `PreviewHeader`

| Prop | Type | Default | Description |
|---|---|---|---|
| `name` | `string` | — | Filename rendered as the bold second line. Required. |
| `ext` | `string` | — | Extension, used to pick the leading icon (`png`, `mp4`, `parquet`, `json`, …). |
| `path` | `string` | — | Full file path. The parent directory is shown above the filename; falls back to `/` when the file sits at the root. |
| `size` | `number` | — | Raw bytes. Formatted internally via `fmtSize`. |
| `modified` | `Date \| string` | — | Last-modified timestamp. Dates are formatted as `YYYY-MM-DD HH:mm`; strings pass through. |

### `PreviewSubBar`

| Prop | Type | Default | Description |
|---|---|---|---|
| `left` | `ReactNode` | — | Left slot. Convention: summary text (col / row counts, line count, parse status). Required. |
| `right` | `ReactNode` | — | Right slot. Convention: status dot or inline action buttons. Required. |

### `TableView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `cols` | `TableColumn[]` | — | Column definitions (`{ name, type? }`). If any column has a `type`, the header grows a second mono line to show it. |
| `rows` | `unknown[][]` | — | Row tuples — one inner array per row, values in column order. Cells are formatted by JS type (boolean, number, null, string). |
| `totalRows` | `number \| null` | — | Total rows in the full file. Painted on the left of the sub-bar; falls back to `rows.length` if omitted. |
| `shownRows` | `number` | `rows.length` | How many rows are actually rendered — used by the "showing first N" caption. |
| `truncated` | `boolean` | `false` | When `true`, the sub-bar shows an accent dot + truncation caption on the right. |
| `subInfoLeft` | `ReactNode` | — | Override the default left slot (col / row counts). |
| `subInfoRight` | `ReactNode` | — | Override the default right slot (truncation caption). |
| `hideSubBar` | `boolean` | `false` | Hide the sub-bar entirely. Use when the table is embedded under a pane that already owns one. |

### `KeyValueView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `items` | `Array<[string, string]>` | — | Ordered list of `[key, value]` pairs. Order is preserved — pre-sort on the host side if you want alphabetical. |

### `JsonTreeView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `value` | `unknown` | — | Already-parsed JSON value (object, array, or scalar). Required. |
| `defaultOpenDepth` | `number` | `2` | Depth below which nodes start collapsed. `0` = everything collapsed; `Infinity` = everything open. |

### `JsonlView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `records` | `unknown[]` | — | Array of already-parsed records, one per JSONL line. Long records are clipped at the row level. |

### `ImageView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `src` | `string` | — | Image URL (signed S3, blob, data URI, …). Required. |
| `alt` | `string` | `''` | Alt text. Empty by default since previews are typically content the user just selected. |
| `onLoad` | `(resolution: string) => void` | — | Fires once the browser has decoded the image. The argument is `"naturalWidth×naturalHeight"`. |

### `VideoView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `src` | `string` | — | Video URL. Browsers fetch chunks via Range requests as the user scrubs — no full download. Required. |
| `ext` | `string` | — | Extension (`mp4`, `mov`, …). Display-only: it words the error copy accurately (".mov files" rather than "this file"). |
| `onLoadedMetadata` | `(info: { duration: number; resolution: string }) => void` | — | Fires once the demuxer has the header. `duration` is in seconds; `resolution` is `"W×H"`. Suppressed when the picture came back 0×0 — that fires `onError` instead. |
| `onError` | `(info: VideoErrorInfo) => void` | — | Fires once per failure, with `{ kind, code, title, message, hint }`. Covers both `MediaError`s and the silent no-video-track case. |
| `stallMs` | `number` | `15000` | How long to wait with no metadata and no error before overlaying an advisory "still loading" banner. `0` disables it. |

### `TextView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `text` | `string` | — | The text to render. Changing this prop snaps any pending local edits back to the new value. Required. |
| `ext` | `string` | — | Extension shown in the sub-bar (e.g. `md`, `yaml`). Display-only, no syntax highlighting yet. |
| `onSave` | `(text: string) => Promise<void>` | — | When supplied, the sub-bar grows an Edit button. The promise drives the saving state; reject to surface the error inline. Omit for a read-only viewer. |

### `StatusView`

| Prop | Type | Default | Description |
|---|---|---|---|
| `kind` | `'empty' \| 'loading' \| 'error' \| 'too-large' \| 'unsupported'` | — | Which state to render. Required. |
| `label` | `string` | `'loading…'` | `kind="loading"`: the caption. `kind="error"`: replaces the `Preview failed` headline, for callers that know what failed. |
| `message` | `string` | `'Preview failed'` | Error detail. Only used for `kind="error"`. |
| `ext` | `string` | — | Extension shown in the unsupported message. Only used for `kind="unsupported"`. |
| `sizeBytes` | `number` | `0` | Actual file size. Only used for `kind="too-large"`. |
| `capBytes` | `number` | `0` | Preview cap. Only used for `kind="too-large"`. |
| `hint` | `ReactNode` | — | `kind="error"`: a remediation line under the message. `kind="too-large"`: replaces the default cap copy entirely — e.g. a download link. |

---

Source: https://viz.dreamlake.ai/schema-viz/adapters

# Adapters

An **adapter** understands _one dataset format_ and turns it into a **Model** —
the live object a [view](/schema-viz/views.md) queries. A format is one adapter.

## Supported datasets

Match the adapter to how the data is stored, point [storage](/schema-viz/storage.md)
at it, done. Full options + a live preview for each are below.

| You have                                                                | `adapter`    | Minimal source                                                                                            |
| ----------------------------------------------------------------------- | ------------ | --------------------------------------------------------------------------------------------------------- |
| A **LeRobot** episode — v2.0/2.1/3.0 (most HF robot datasets)           | `lerobot`    | `{ adapter: lerobot, storage: { driver: hf, id: 'lerobot/aloha_static_coffee' }, episode: 0 }`            |
| A **Zarr ReplayBuffer** `.zarr.zip` — UMI · UMI-3D · MV-UMI             | `umi`        | `{ adapter: umi, storage: { driver: hf, id: 'omarrayyann/mv-umi' }, path: 'bottles_rack_data.zarr.zip' }` |
| An exploded **`.zarr` directory** — Zarr v3, e.g. EgoVerse              | `umi`        | `{ adapter: umi, storage: { driver: http, basePath: '…/' }, path: 'episode.zarr' }`                       |
| **Egocentric** video + annotations — EGO4D · Ego-Exo4D · EgoDex · RH20T | `egocentric` | `{ adapter: egocentric, dataset: 'ego4d', storage: {…} }` (or list `videos`/`annotations`)                |
| A **folder of loose files** — clips, images, csv/parquet                | `filesystem` | `{ adapter: filesystem, storage: { driver: http, basePath: '…/' } }`                                      |

**Storage is separate from format.** `http` reads any public URL/CDN; `hf` reads a
public HuggingFace dataset (`{ driver: hf, id: 'org/name' }` → `…/resolve/main/<path>`).
Both are public-only — private data uses a [host-injected driver](/schema-viz/storage.md).

Each adapter implements one interface, so every dataset above renders through the
same [views](/schema-viz/views.md):

```ts
interface Model {
  fields(): Promise<Field[]> // discover — what's here
  read(ref: string, query?: ReadQuery): Promise<Payload> // fetch one field, lazy
  timeline(): Promise<Timeline | null> // shared clock, or null
}

interface Field {
  ref: string // a feature name ("action") or a file path
  kind: string // video | frames | series | cues | pose3d | pointcloud | image | file
  label?: string
  meta?: Record<string, unknown> // e.g. series dim names in meta.dims
}
```

A field's **`kind`** decides which view draws it and what `read()` returns.

## Choosing an adapter

### `filesystem` — a folder of loose files

No manifest. `fields()` lists the directory (via the [storage](/schema-viz/storage.md)
`list`) and infers each field's kind from its filename; `read()` hands media back
as a URL and parses tabular files into series. A bare folder has no clock, so
`timeline()` is `null`.

```yaml
- adapter: filesystem
  storage: { driver: http, basePath: /viz-samples/episode_365/ }
```

Extension → kind: `mp4/mov/webm…` → `video`, `jpg/png…` → `image`,
`parquet/csv` → `series`, anything else → `file`. Pairs with any driver whose
`list` enumerates the folder (`http` + an `index.json`, or the host's
`dlProject`).

### `lerobot` — a manifest-indexed episode

Reads LeRobot **v2.0 / v2.1 / v3.0** datasets. `fields()` parses
`meta/info.json::features` — cameras become `video`, numeric features become
`series` (vector dims carried in `meta.dims`), and `task_index` becomes a `cues`
track; `timeline()` is the episode's duration/fps.

```yaml
- adapter: lerobot
  storage:
    driver: http
    basePath: https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main
  episode: 0
```

| Param     | Type   | Default | Notes                            |
| --------- | ------ | ------- | -------------------------------- |
| `episode` | number | `0`     | Episode index within the dataset |

Because `read('action')` returns every dim of the `action` feature as columns
keyed `action[left_waist]`, a [lineChart](/schema-viz/views.md) can select one dim
with `field: [action, left_waist]`. The adapter range-reads the episode's parquet
and memoizes each shared fetch (manifest, data buffer) as an in-flight promise.

> **Known limitation (v3.0):** episode metadata is searched only in
> `meta/episodes/chunk-000/file-000.parquet`.

### `umi` / `zarr` — a Zarr ReplayBuffer

Reads a **UMI-style `.zarr.zip`** (a zipped Zarr v2 `ReplayBuffer`): every episode
concatenated along time, with `meta/episode_ends` holding the cumulative frame
boundaries and `data/<name>` holding the per-frame arrays. `fields()` exposes each
low-dim numeric array (`robot0_eef_pos`, …) as a `series` with synthesized
`meta.dims`; `timeline()` is the episode's frame count over `fps`.

The archive is read **in place over HTTP range requests** — the central directory
locates each entry, and only the requested episode's chunks are fetched, so the
archive (the example below is ~6 GB; UMI's `cup_in_the_wild` is ~18 GB) is never
downloaded whole. The only part that scales with the archive is the one-time
central-directory read; the per-episode data is range-read regardless of size.
Low-dim arrays are stored raw, so no Zarr/codec dependency is pulled in.

```yaml
- adapter: umi # alias: zarr
  storage: { driver: hf, id: omarrayyann/mv-umi } # → …/resolve/main/<path>
  path: bottles_rack_data.zarr.zip
  episode: 0
  fps: 59.94
```

| Param     | Type   | Default | Notes                                             |
| --------- | ------ | ------- | ------------------------------------------------- |
| `path`    | string | `''`    | Path (or absolute URL) of the `.zarr.zip`         |
| `episode` | number | `0`     | Episode index; rows come from `meta/episode_ends` |
| `fps`     | number | `60`    | Frame rate for the time axis                      |

```tsx file="UmiEpisodeSpec.tsx"
// A UMI-family ReplayBuffer episode → low-dim state/action time series, read
// straight from a real, public dataset on HuggingFace. The `umi` adapter reads
// arrays out of the `.zarr.zip` (a zipped Zarr v2 store) over HTTP range
// requests, so the multi-GB archive is never downloaded whole — it reads the
// central directory once, then only this episode's chunks.
//
// Dataset: MV-UMI `bottles_rack` (omarrayyann/mv-umi) — a multi-view UMI corpus
// with paired egocentric-wrist + third-person cameras (camera0_rgb / camera1_rgb)
// plus the usual low-dim end-effector trajectory. Public, no gating; HuggingFace
// sends CORS and supports range, which is what makes the in-browser range reads
// work. The cameras are stored one JPEG-XL frame per chunk and shown with the
// `frameStack` view, which lazily byte-ranges just the cursor's frame (so a
// 100k-frame camera costs one fetch per scrub). JPEG-XL decodes natively in
// Safari 17+ and Chrome with the JXL flag.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      // `hf` resolves to https://huggingface.co/datasets/<id>/resolve/main/<path>
      adapter: 'umi',
      storage: { driver: 'hf', id: 'omarrayyann/mv-umi' },
      path: 'bottles_rack_data.zarr.zip',
      episode: 0,
      fps: 59.94,
    },
  },
  timeline: { source: 'ep' },
  panels: [
    {
      view: 'frameStack',
      source: 'ep',
      columns: 2,
      fields: ['camera0_rgb', 'camera1_rgb'], // wrist + third-person, JPEG-XL
    },
    {
      view: 'gridLayout',
      columns: 2,
      children: [
        {
          view: 'lineChart',
          source: 'ep',
          title: 'End-effector position — x / y / z (m)',
          height: 240,
          fields: ['robot0_eef_pos'],
        },
        {
          view: 'lineChart',
          source: 'ep',
          title: 'End-effector rotation — axis-angle',
          height: 240,
          fields: ['robot0_eef_rot_axis_angle'],
        },
      ],
    },
    {
      view: 'lineChart',
      source: 'ep',
      title: 'Gripper width (m)',
      height: 200,
      fields: ['robot0_gripper_width'],
    },
  ],
}

export const UmiEpisodeSpec = () => <DatasetPreview schema={schema} />
```

> **The host must send CORS + support range.** The example reads a real public
> dataset (MV-UMI `bottles_rack`, dual-camera) directly from HuggingFace, which
> sends CORS and supports range. UMI's own `cup_in_the_wild` is hosted on
> `real.stanford.edu`,
> which sends no `Access-Control-Allow-Origin` — mirror it to a CORS-enabled host
> (HuggingFace, S3 + CloudFront, or a host-injected DreamLake driver) to read it
> in the browser. The cameras (`camera0_rgb` / `camera1_rgb`, one JPEG-XL frame
> per chunk) are exposed as `frames` and drawn by the
> [`frameStack`](/schema-viz/views.md) view, which byte-ranges only the cursor's
> frame.

#### Folder mode + Zarr v3

The same adapter also reads an **exploded `.zarr` directory** (no zip), which is
how Zarr is stored natively and on object storage (S3 / R2). It is detected from
the `path`: a `.zip` is the ZipStore above; anything else is a directory, where
**each chunk is its own URL** — there is no central directory, so reads are
size-independent.

Directory stores are typically **Zarr v3** (`zarr.json` metadata, `c/0/0` chunk
keys) and may shard + compress. The adapter handles a v3 group whose root
`zarr.json` carries a feature manifest (`attributes.features`, `fps`,
`total_frames`) — one episode per `.zarr` — and decodes `sharding_indexed` chunks
with inner **zstd**. EgoVerse is the worked example (its EEF / head / gaze pose
arrays below; the full corpus lives on credentialed R2, so this reads a small
same-origin slice):

```yaml
- adapter: umi # alias: zarr — folder mode is detected from the path
  storage: { driver: http, basePath: /viz-samples/egoverse/ }
  path: egoverse_sample.zarr # a directory, not a .zip
```

```tsx file="EgoVerseSpec.tsx"
// EgoVerse — a Zarr v3 *directory* store (not a `.zarr.zip`), demonstrating the
// adapter's folder mode. One `.zarr` per episode; the root `zarr.json` is the
// manifest (features + fps + total_frames); each array is `sharding_indexed` +
// zstd. The adapter auto-detects folder vs zip from the path and reads each chunk
// as its own URL — no central directory, so it is size-independent.
//
// This points at a small same-origin slice (four pose arrays of one episode)
// because the full EgoVerse corpus lives on Cloudflare R2 behind credentials.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      adapter: 'umi', // alias: zarr — folder mode is detected from the path
      storage: { driver: 'http', basePath: '/viz-samples/egoverse/' },
      path: 'egoverse_sample.zarr',
    },
  },
  timeline: { source: 'ep' },
  panels: [
    {
      view: 'gridLayout',
      columns: 2,
      children: [
        {
          view: 'lineChart',
          source: 'ep',
          title: 'Left EEF pose — xyz + quaternion',
          height: 240,
          fields: ['left.obs_ee_pose'],
        },
        {
          view: 'lineChart',
          source: 'ep',
          title: 'Right EEF pose — xyz + quaternion',
          height: 240,
          fields: ['right.obs_ee_pose'],
        },
      ],
    },
    {
      view: 'lineChart',
      source: 'ep',
      title: 'Head pose',
      height: 200,
      fields: ['obs_head_pose'],
    },
    {
      view: 'lineChart',
      source: 'ep',
      title: 'Eye gaze — xyz',
      height: 200,
      fields: ['obs_eye_gaze'],
    },
    // 21 hand joints (63 = 21×3) → parsed into the Model as `pose3d`. No 3D
    // renderer ships yet, so the placeholder view surfaces it.
    { view: 'placeholder3d', source: 'ep', fields: ['left.obs_keypoints'] },
  ],
}

export const EgoVerseSpec = () => <DatasetPreview schema={schema} />
```

### `egocentric` — media + sidecar annotations

The video-centric corpora (**EGO4D, Ego-Exo4D, EgoDex, RH20T**) share one shape —
videos (or per-frame image dirs), annotation JSON, low-dim NPY — but, unlike
LeRobot/Zarr, **no shared self-describing manifest**. So the adapter is told where
things live, two ways.

**1 · By `dataset` profile — open-the-box.** A built-in profile discovers the
videos / annotations / sidecars by `list()`-ing the storage; you write nothing
else. Needs a listable driver (`hf`, or `http` + an `index.json`):

```yaml
- adapter: egocentric
  dataset: ego4d # built-in profiles: ego4d · egoexo4d · rh20t
  storage: { driver: http, basePath: /viz-samples/ego4d/ }
```

```tsx file="EgocentricSpec.tsx"
// Egocentric "media + sidecar" adapter on real EGO4D clips.
//
// `dataset: 'ego4d'` is all that's needed: the EGO4D PROFILE discovers the clips
// (videoStack) and the narration track (timeline) by listing the storage — no
// hand-written `videos`/`annotations`. (A custom or unprofiled layout lists them
// explicitly instead; see the adapter docs.)
//
// Same-origin sample: three real EGO4D clips. EGO4D's actual narration
// annotations are license-gated, so the cue track is a short illustrative file in
// EGO4D's real `{ narrations: [{ timestamp_sec, text }] }` format.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      adapter: 'egocentric',
      dataset: 'ego4d',
      storage: { driver: 'http', basePath: '/viz-samples/ego4d/' },
    },
  },
  panels: [
    { view: 'videoStack', source: 'ep', fields: ['*'], columns: 3 },
    { view: 'timeline', source: 'ep', fields: ['narrations.json'] },
  ],
}

export const EgocentricSpec = () => <DatasetPreview schema={schema} />
```

**2 · By explicit lists — custom or unprofiled layout.** Spell out what to expose;
these keys also override a profile:

```yaml
- adapter: egocentric
  storage: { driver: hf, id: org/my-dataset }
  videos: [{ path: clips/a.mp4, label: A }] # → videoStack
  annotations: [{ path: narrations.json, format: ego4d-narration }] # → timeline cues
  series: [{ path: lowdim/tcp_pose.npy, dims: [x, y, z] }] # → lineChart (NPY)
  pose: [{ path: hands.npy }] # → pose3d
  frames: [{ paths: [rgb/cam0/0001.jpg, …], label: cam0 }] # → frameStack
  fps: 30
```

`videos` → a synchronized `videoStack`; `annotations` → parsed (interval lists,
EGO4D narrations / NLQ, point events) into `timeline` cues; `series` / `pose` →
NPY sidecars into charts / `pose3d`; per-frame image dirs → `frames`.

> EGO4D's real narrations are license-gated, so the example's cue track is a short
> illustrative file in EGO4D's `{ narrations: [{ timestamp_sec, text }] }` format.
> Point storage at a licensed copy (or `hf` at a public mirror) for the full set.

The other built-in profiles are the same one-liner. **Ego-Exo4D** (`dataset:
egoexo4d`) — synchronized ego + exo cameras, discovered as per-camera `frames`:

```tsx file="EgoExo4dSpec.tsx"
// Ego-Exo4D via the `egoexo4d` profile — synchronized egocentric (Aria) + exo
// cameras. The full release ships aligned videos; this same-origin sample is the
// frame-extract form (one dir per camera), which the profile discovers as
// per-camera `frames`. The frameStack scrubs all views on one shared cursor.
//
// Real Ego-Exo4D frames (take cmu_bike01_2): ego + two exo cameras.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      adapter: 'egocentric',
      dataset: 'egoexo4d',
      storage: { driver: 'http', basePath: '/viz-samples/egoexo4d/' },
    },
  },
  panels: [{ view: 'frameStack', source: 'ep', fields: ['*'], columns: 3 }],
}

export const EgoExo4dSpec = () => <DatasetPreview schema={schema} />
```

**RH20T** (`dataset: rh20t`) — low-dim NPY sidecars under `lowdim/` (here, real
joint angles + velocities; the full dataset adds per-camera `rgb/cam_*/` frames):

```tsx file="Rh20tSpec.tsx"
// RH20T via the `rh20t` profile — the egocentric adapter discovers the low-dim
// NPY sidecars under `lowdim/` (and, in the full dataset, per-camera JPEG
// sequences under `rgb/cam_*/`). Same-origin sample: real joint data, recovered
// from the dataset's `transformed/joint.npy` and written in the canonical
// `lowdim/joint_angles.npy` (T×7) form.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      adapter: 'egocentric',
      dataset: 'rh20t',
      storage: { driver: 'http', basePath: '/viz-samples/rh20t/' },
    },
  },
  panels: [
    {
      view: 'lineChart',
      source: 'ep',
      title: 'Joint angles — 7 DoF',
      height: 240,
      fields: ['lowdim/joint_angles.npy'],
    },
    {
      view: 'lineChart',
      source: 'ep',
      title: 'Joint velocities',
      height: 200,
      fields: ['lowdim/joint_velocities.npy'],
    },
  ],
}

export const Rh20tSpec = () => <DatasetPreview schema={schema} />
```

## Writing your own adapter

Implement the three methods and register the format. Here is the `filesystem`
adapter, essentially complete — note where it touches the storage (`list` to
discover, `resolveUrl` for a URL):

```ts

class FilesystemAdapter implements Model {
  private catalog?: Promise<Field[]>
  constructor(
    private storage: Storage,
    private dir: string
  ) {}

  fields() {
    return (this.catalog ??= this.storage
      .list(this.dir)
      .then(({ entries }) =>
        entries
          .filter((e) => e.type === 'file')
          .map((e) => ({ ref: e.path, kind: kindFromExt(e.name), label: e.name }))
      ))
  }
  async read(ref: string): Promise<Payload> {
    const kind = (await this.fields()).find((f) => f.ref === ref)?.kind // kind from catalog
    if (kind === 'video')
      return { kind: 'video', url: await this.storage.resolveUrl(ref), start: 0, end: 0 }
    if (kind === 'series') return readTableColumns(this.storage, ref)
    return { kind: 'file', url: await this.storage.resolveUrl(ref) }
  }
  timeline() {
    return Promise.resolve(null)
  }
}
```

> Resolving `kind` from the **catalog** (not the ref's extension) is what lets
> opaque refs — like DreamLake node ids, which carry no extension — still render
> as video/image/etc.

Register it per-preview, or once at app boot:

```tsx

registerAdapter('myfmt', (storage, params) => new MyAdapter(storage, params.root as string))
// …or: <DatasetPreview schema={schema} extensions={{ adapters: { myfmt: factory } }} />
```

The factory is `(storage, params) => Model`: viz hands it the resolved storage and the
source's params (e.g. `episode`, `dir`), and calls it **once per source**.

Two patterns cover almost everything: **directory discovery** (above) and
**entry-file discovery** (read a manifest, emit fields from it, return a real
`timeline()`). Different discovery in, the **same Model out** — which is why a
folder of clips and a LeRobot episode render through the same
[views](/schema-viz/views.md).

Next: [Views](/schema-viz/views.md) — drawing a Model.

---

Source: https://viz.dreamlake.ai/file-preview/loaders

# Loaders

The loader layer turns bytes into typed data. Every loader is a pure
async function with the same shape — `load(url, opts)` — and only
fetches what the format actually needs to render a preview. Where the
format allows it (CSV, JSONL, NumPy headers, MCAP metadata), loaders
issue `Range` requests so a multi-gigabyte file never gets pulled in
full.

Loaders are also re-exported from `@dreamlake/viz`, so a host that
just wants the data (without the matching view) can pull
`loadParquet` or `loadMcap` directly and feed the result into its own
UI.

## Overview

### Shared shape

Every loader is `async (url: string, opts?: LoaderOpts) => Promise`.
`TData` is per-loader (`CsvData`, `ParquetData`, …), but the input shape
is the same across the board.

```ts file="LoaderOpts.ts"
export interface LoaderOpts {
  size?: number             // skip HEAD if you already know the byte size
  signal?: AbortSignal      // for cancellation on unmount
  maxBytes?: number         // override the default slice cap
  maxRows?: number          // override the default row cap (where applicable)
}
```

`signal` is the load-bearing one — pair it with `useLoader` and the
hook automatically aborts the in-flight fetches when the dependent
URL changes or the component unmounts.

### HTTP helpers

All loaders go through the same four primitives, which you can also

const { data, error, loading } = useLoader(
  (signal) => loadText(url, { signal }),
  [url],
)
```

### `useLoader`

The hook every composed Preview uses. Takes a function that receives
an `AbortSignal` and returns `{ data, error, loading }`. The deps
array drives the React-19 prop-derivation reset — when deps change,
the previous in-flight request is aborted and the state snaps back
to `loading: true` during render, so there's no flash of stale data.

```ts file="useLoader.ts"
export interface LoaderState {
  data: T | null
  error: Error | null
  loading: boolean
}

function useLoader(
  load: (signal: AbortSignal) => Promise,
  deps: unknown[],
): LoaderState
```

## Tabular formats

### `loadCsv`

Range-fetches the first ~5 MB of a CSV, auto-detects the delimiter
(`,` / `;` / `\t` / `|`), and parses header + up to 1000 rows. Handles
the RFC-4180 quote-escape rule and BOM-prefixed Excel exports. Trims
the last partial line if the slice cap was hit.

```ts file="csv.ts"

const data = await loadCsv(url, { signal })
//   data.header     : string[]
//   data.rows       : string[][]
//   data.shown      : number
//   data.totalRows  : number | null   // null when truncated mid-read
//   data.bytesRead  : number
//   data.truncated  : boolean
//   data.delimiter  : ',' | ';' | '\t' | '|'
```

Defaults: `maxBytes = 5 MB`, `maxRows = 1000`. Pass either through
`opts` to widen or narrow the slice — the loader will respect both
caps and trim cleanly.

### `loadParquet`

Reads Parquet via [hyparquet](https://github.com/hyparam/hyparquet).
Parquet metadata lives at the **end** of the file, so unlike CSV
there's no meaningful "first 5 MB" — it's whole-or-nothing. The
loader probes size first; if the file exceeds `MAX_PARQUET_BYTES`
(5 MB by default) it throws `ParquetTooLargeError` so the host can
render a `too-large` status without parsing.

```ts file="parquet.ts"

try {
  const data = await loadParquet(url, { signal })
  //   data.cols      : { name, type }[]
  //   data.rows      : unknown[][]
  //   data.totalRows : number
} catch (e) {
  if (e instanceof ParquetTooLargeError) {
    // e.sizeBytes / e.capBytes
  }
}
```

Defaults: `maxRows = 1000`. Override `maxBytes` to raise or lower the
cap if you know the consumer can handle it. Columns are top-level
only — nested schemas appear as JSON-stringified values inside the
row tuples.

## Structured formats

### `loadJson`

A JSON document is a single value, so range-fetching would yield
unparseable fragments — the loader HEADs first and refuses anything
over `MAX_JSON_BYTES` (10 MB default) by throwing `JsonTooLargeError`.
The success result carries both the raw text (for a "raw" view) and
the parsed value (for a tree view); parse failures are captured into
`parseError` rather than thrown, so the host can still surface the
raw bytes alongside the syntax error.

```ts file="json.ts"

const data = await loadJson(url, { signal })
//   data.raw        : string
//   data.parsed     : unknown
//   data.parseError : string | null
//   data.bytes      : number
```

### `loadJsonl`

Fetches the first ~1 MB of a JSONL file via a single `Range` GET and
parses line-by-line. Lines that fail to parse are silently skipped —
typical for log files where not every line is strict JSON. If the
slice was truncated mid-line, the trailing partial line is dropped
before parsing.

```ts file="jsonl.ts"

const data = await loadJsonl(url, { signal })
//   data.records    : unknown[]
//   data.totalLines : number     // exact when not truncated, estimated otherwise
//   data.truncated  : boolean
```

Defaults: `maxBytes = 1 MB`, `maxRows = 1000`.

## Binary formats

These loaders read only the metadata — never any element / message
data — so previewing is fast and the host doesn't have to gate
on file size.

### `loadNpy`

Parses the header of a NumPy `.npy` file from a 256-byte `Range`
fetch. If the dict header claims to be longer, the loader widens
with one extra range request. We never touch the element data.

```ts file="npy.ts"

const meta = await loadNpy(url, { signal })
//   meta.dtype         : '<f4' | '|i1' | ...
//   meta.shape         : number[]
//   meta.elements      : number
//   meta.itemSize      : number      // bytes per element
//   meta.dataBytes     : number      // total element-data bytes
//   meta.fortranOrder  : boolean
//   meta.version       : '1.0' | '2.0' | '3.0'
//   meta.rangeUsed     : '[0, 256)'  // diagnostic
```

Header dicts are parsed with focused regexes — no `eval`. Supports
NPY format 1/2/3 and the standard scalar dtypes (`f`, `i`, `u`, `b`,
`?`, `e`, `d`, `g`, complex `F` / `D` / `G`).

### `loadMcap`

Parses an MCAP file's metadata: the `Header` record near the start,
the `Footer` at the end, and the summary section pointed to by the
footer. No data records, no decompression — just three range
fetches, regardless of file size.

```ts file="mcap.ts"

const meta = await loadMcap(url, { signal })
//   meta.library          : string
//   meta.profile          : string
//   meta.compression      : string             // dominant chunk codec, or 'none'
//   meta.durationMs       : number
//   meta.messageCount     : number
//   meta.chunkCount       : number
//   meta.attachmentCount  : number
//   meta.channels         : McapChannelStat[]  // per-channel topic / schema / count / rate
//   meta.timestamps       : { start, end }     // ISO-8601 strings
//   meta.rangeUsed        : string
```

The summary section is what makes "metadata-only" practical — it
holds per-channel counts and statistics, so we never need to walk
the data records to compute message rates.

---

Source: https://viz.dreamlake.ai/schema-viz/views

# Views

A **view** draws. Given a source's field catalog and a Model to read from, it
binds the fields named in its panel config and renders, dispatching on each
field's `kind` — which is why any view renders data from any adapter.

```ts
interface View {
  name: string // the `view:` name in a panel
  kinds: string[] // which field kinds it can render
  component: ComponentType<ViewProps>
}

interface ViewProps {
  fields: Field[] // the source's catalog
  read: (ref: string, q?: ReadQuery) => Promise<Payload> // the Model's read, pre-bound
  timeline?: Timeline | null
  cursor?: TimeCursor // shared scrub time
  config: Record<string, unknown> // this panel's options
}
```

## Built-in views

| View            | Renders kinds            | What it does                                                         |
| --------------- | ------------------------ | -------------------------------------------------------------------- |
| `videoStack`    | `video`, `image`, `file` | Many fields merged into **one** synchronized tile grid.              |
| `lineChart`     | `series`                 | Plots styled series (per-dim) over the timeline, windowed by `read`. |
| `timeline`      | `cues`                   | A scrub bar with labelled cue tracks, sharing the cursor.            |
| `fieldsCatalog` | any                      | A discovery table of the source's fields.                            |

Layout: a panel with `children` is a **`gridLayout`** (a CSS grid of nested
panels; `columns` default 2). Omit `panels` (or use `view: autoLayout`) for
[auto-layout](#auto-layout).

### `videoStack`

Binds media fields and merges them into one component whose tiles share a cursor.

```yaml
- view: videoStack
  source: ep
  fields: ['observation.images.*'] # wildcard over the video fields
  columns: 2 # default 3
  gap: 6
  showRecTimestamp: true
  showLive: false
  showLabel: true
  showResolution: true
  overlays: # optional annotation layers
    - { field: '*joints.json', format: handJoints, on: 'observation.images.ego' }
    - { field: '*subtasks*.json', format: subtasks }
```

Each `overlays` entry names an annotation-file field in the same source and
its `format` — `handJoints` (21-joint skeletons), `subtasks`
(subtitle-style captions), or `raw` (the file already contains
`MediaOverlay` JSON); the exact JSON shape per format is specced in
[Media overlays → data formats](/components/media-overlay.md#data-formats).
`on` targets one video field, or every tile when omitted. See the
[schema example](/schema-viz/schema.md#end-to-end-hand-skeletons-via-overlays)
live.

```tsx file="FilesystemFolderSpec.tsx"
// A folder of loose clips → one synchronized stack, no manifest. The `http`
// storage lists the folder via its index.json and resolves each clip's URL; the
// `filesystem` adapter turns each file into a `video` field; `videoStack` merges
// them into one tile grid that scrubs together.
//
// These sample clips are served from the docs site's public/ folder. To read a
// DreamLake project folder instead, swap the storage to the host-injected
// `dlProject` driver:
//   storage: { driver: 'dlProject', root: '<folder node id>' }

const schema: VizSchema = {
  version: 1,
  sources: {
    clips: {
      adapter: 'filesystem',
      storage: { driver: 'http', basePath: '/viz-samples/episode_365/' },
    },
  },
  panels: [{ view: 'videoStack', source: 'clips', fields: ['*'], columns: 2 }],
}

export const FilesystemFolderSpec = () => <DatasetPreview schema={schema} />
```

### `lineChart`

`series` is a list of styled traces. Each `field` is a feature (all dims) or
`[feature, dim]` to select one dim; dim globs (`[feature, "left_*"]`) select
several. Per-series `label`, `dash`, `color`, `width`, `opacity`, `ghost` apply.

```yaml
- view: lineChart
  source: ep
  title: Left arm — cmd vs actual
  caption: joint angle · rad
  height: 240 # px (number) or any CSS length
  yRange: [-2, 2] # auto-fit when omitted
  unitHint: action vs observation.state
  maxPoints: 2000 # downsample cap
  series:
    - { field: [action, left_waist], label: waist · cmd }
    - { field: [observation.state, left_waist], label: waist · actual, dash: '3 2.4' }
```

For a quick, unstyled chart use `fields: [action, observation.state]` instead of
`series` — every dim, auto-colored.

```tsx file="LineChartSpec.tsx"
// `lineChart` in isolation — styled per-dim series against a public LeRobot
// dataset. Each trace overlays action (cmd, solid) against observation.state
// (actual, dashed) for one joint of the left arm. No auth.

const BASE = 'https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main'

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: { adapter: 'lerobot', storage: { driver: 'http', basePath: BASE }, episode: 0 },
  },
  panels: [
    {
      view: 'lineChart',
      source: 'ep',
      title: 'Left arm — cmd vs actual',
      caption: 'joint angle · rad',
      height: 260,
      series: [
        { field: ['action', 'left_waist'], label: 'waist · cmd' },
        { field: ['observation.state', 'left_waist'], label: 'waist · actual', dash: '3 2.4' },
        { field: ['action', 'left_elbow'], label: 'elbow · cmd' },
        { field: ['observation.state', 'left_elbow'], label: 'elbow · actual', dash: '3 2.4' },
      ],
    },
  ],
}

export const LineChartSpec = () => <DatasetPreview schema={schema} />
```

### `timeline`

`tracks` binds cue fields, each with an optional label override. A track
may instead name an annotation FILE field with a `format` — the view
fetches and converts it into blocks (same contract as `videoStack`
overlays):

```yaml
- view: timeline
  source: ep
  tracks:
    - { field: task_index, label: Task }
    - { field: '*subtasks*.json', format: subtasks, label: Subtasks }
  trackRowHeight: 28
  minZoom: 1
  maxZoom: 20
```

```tsx file="TimelineSpec.tsx"
// `timeline` in isolation — a scrub bar with one labelled cue track. The lerobot
// adapter turns the episode's `task_index` column into `cues`; the timeline
// renders them as segments you can scrub. Public dataset, no auth.

const BASE = 'https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main'

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: { adapter: 'lerobot', storage: { driver: 'http', basePath: BASE }, episode: 0 },
  },
  timeline: { source: 'ep' },
  panels: [{ view: 'timeline', source: 'ep', tracks: [{ field: 'task_index', label: 'Task' }] }],
}

export const TimelineSpec = () => <DatasetPreview schema={schema} />
```

### `fieldsCatalog`

A discovery table — point it at an unfamiliar dataset to see field names, kinds,
and dim names.

```yaml
- view: fieldsCatalog
  source: ep
  title: Available fields
```

```tsx file="FieldsCatalogSpec.tsx"
// `fieldsCatalog` in isolation — a discovery table of a source's fields (name,
// kind, dims). Point it at an unfamiliar dataset to see what is inside before
// you write any panels. Public LeRobot dataset, no auth.

const BASE = 'https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main'

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: { adapter: 'lerobot', storage: { driver: 'http', basePath: BASE }, episode: 0 },
  },
  panels: [{ view: 'fieldsCatalog', source: 'ep', title: 'Available fields' }],
}

export const FieldsCatalogSpec = () => <DatasetPreview schema={schema} />
```

## Writing your own panel

A panel is a component that takes `ViewProps`. Inside you bind fields from the
config, `read` them, and draw — using `cursor` to stay in sync with the preview.

```tsx

function HistogramView({ fields, read, config }: ViewProps) {
  const cfg = config as { fields?: string[]; bins?: number }
  const bound = expandRefs(cfg.fields ?? ['*'], fields).filter((f) => f.kind === 'series')
  const state = useLoader(
    () => Promise.all(bound.map((f) => read(f.ref))),
    [bound.map((f) => f.ref).join('|')]
  )
  if (state.loading) return <div>Loading…</div>
  if (state.error) return <div className="text-red-600">⚠ {state.error.message}</div>
  const values = state.data!.flatMap((p) =>
    p.kind === 'series' ? Object.values(p.columns).flat() : []
  )
  return <BarChart data={histogram(values, cfg.bins ?? 30)} />
}

export const histogramView: View = {
  name: 'histogram',
  kinds: ['series'],
  component: HistogramView,
}
```

Register it per-preview with `extensions.views`, or once at app boot with
`registerView` (see [Storage → register once, or per preview](/schema-viz/storage.md#register-once-or-per-preview)):

```tsx

registerView(histogramView)
// …or: <DatasetPreview schema={schema} extensions={{ views: [histogramView] }} />
```

Conventions: a view holds only the Model (it calls `read`, never a storage);
sync via `cursor` (`cursor.time`, `cursor.setHover`, `cursor.seek`); window with
`read(ref, { timeRange, maxPoints })`.

## Auto-layout

Omit `panels` (or place a `{ view: autoLayout, source }`) and viz lays the source
out by kind: media → one `videoStack`, cues → one `timeline`, series → one
`lineChart` per feature (a grid past one) — in that order. It is the _overview_
layout; write `panels` explicitly once you know what to compare.

```tsx file="AutoLayoutSpec.tsx"
// Omit `panels` entirely → viz auto-lays-out the source: a camera stack, a task
// timeline, and one chart per numeric field. The smallest possible schema that
// still produces a full view — the "I don't know this dataset yet" workflow.

const schema: VizSchema = {
  version: 1,
  sources: {
    ep: {
      adapter: 'lerobot',
      storage: {
        driver: 'http',
        basePath: 'https://huggingface.co/datasets/lerobot/aloha_static_coffee/resolve/main',
      },
      episode: 0,
    },
  },
  // no `panels` → auto-layout runs against `ep`
}

export const AutoLayoutSpec = () => <DatasetPreview schema={schema} />
```

Next: [Concept](/schema-viz/concept.md) — why it is built in four layers.

---

Source: https://viz.dreamlake.ai/schema-viz/concept

# Why four layers

You can use schema-viz without reading this page. It explains _why_ the pieces are
shaped the way they are — useful when you write your own adapter or view.

Robot datasets arrive in many formats (LeRobot's `meta/info.json`, a `zarr` root,
a bare folder of files) from many places (S3, GitHub, HuggingFace, DreamLake), but
you want to render them all with the _same_ panels. schema-viz gets there by
splitting the problem into four layers, each meeting the next through one narrow
shape:

```
Storage ──► Adapter ──► Model ──► View
(where)     (format)    (vocab)    (draw)
```

## The Model is the pivot

The hard part of visualizing data is not drawing it — it is _agreeing on what the
data is_. So schema-viz defines one normalized shape, the **Model**: a list of
**fields**, each with a **`kind`** (`video`, `series`, `image`, `cues`, …) and a
lazy way to read it.

```ts
interface Model {
  fields(): Promise<Field[]> // what's here
  read(ref: string, query?: ReadQuery): Promise<Payload> // one field, lazy
  timeline(): Promise<Timeline | null> // shared clock, or null
}
```

A **view** asks the Model only for the fields it can draw, _by kind_. Views speak
**only** Model — they never learn where the data came from. An **adapter** is just
a Model implementation for one format; a **storage** driver feeds the adapter bytes
from one backend.

## `kind` is the joint the system turns on

Every field carries a `kind`, and every `read()` result is discriminated by the
same `kind`. That one string couples three things that otherwise know nothing about
each other:

```
Field.kind ─ selects ─► the View that declares it renders that kind
     └─ determines ─► the Payload shape that read() returns
```

So **a new modality is a new `kind` plus a view that renders it** — no interface
changes. That is why the registry and the panels never grow when you add a format.

### The built-in kinds

`kind` is an **open set** of strings — these five ship built in:

| `kind`   | What it is                         | `read()` returns (`Payload`)            | Default view |
| -------- | ---------------------------------- | --------------------------------------- | ------------ |
| `video`  | a video clip                       | `{ url, start, end }`                   | `videoStack` |
| `image`  | a still image                      | `{ url }`                               | `videoStack` |
| `file`   | an opaque / unrecognized file      | `{ url, ext }`                          | `videoStack` |
| `series` | numeric time-series (multi-column) | `{ timestamps, columns }` (`meta.dims`) | `lineChart`  |
| `cues`   | labeled time intervals (segments)  | `{ cues: Cue[] }`                       | `timeline`   |

Adding a sixth is additive: emit the new `kind` from an adapter and register a view
whose `kinds` includes it. Nothing else changes.

### What each layer does with `kind`

- **[Storage](/schema-viz/storage.md)** — _nothing_. Storage only knows `Entry.type`
  (`file`/`dir`); it never sees a `kind`. (A backend's own `node.kind` is a
  different thing — see the Storage page.)
- **[Adapter](/schema-viz/adapters.md)** — the **producer**. It assigns each field a
  `kind` in `fields()` and tags every `Payload` with the same `kind` in `read()` —
  where raw bytes become a typed modality (`filesystem` infers it from the file
  extension; `lerobot` from the feature type).
- **Model / `Field`** — the **contract**. `Field.kind` answers "what is this";
  `Payload.kind` is the discriminant of the `read()` result union.
- **[View](/schema-viz/views.md)** — the **consumer**. Each view declares
  `kinds: string[]` (the kinds it can draw); the registry indexes views by kind, so
  any view renders any adapter's data as long as the kinds line up.
- **Auto-layout** — groups a source's fields by `kind` to pick default panels:
  media → one `videoStack`, `cues` → one `timeline`, `series` → a `lineChart` each.

In one line: **the adapter produces `kind`, the view consumes it, and neither has
to know about the other.**

## Lazy by construction

Discovery is cheap (`fields()` is a catalog); data is fetched only for what is on
screen, and only for the window asked for. _Where_ the bytes load is decided by
`kind`, so you never wire it by hand:

- **Media** (`video`/`image`/`file`) → `read()` returns a **URL** and stops; the
  `<video>`/`<img>` streams the bytes itself, on play/seek.
- **Data** (`series`/`cues`) → `read()` resolves the URL, range-reads the window,
  **parses** it, and returns arrays. The view gets numbers and never learns the
  file format.

## The payoff

Because each layer meets the next only through these shapes:

- **a new backend is one [Storage](/schema-viz/storage.md)**,
- **a new format is one [Adapter](/schema-viz/adapters.md)**,
- **a new visualization is one [View](/schema-viz/views.md)** —

each added independently, and the [schema](/schema-viz/schema.md) wires them together
with no code.

---

Source: https://viz.dreamlake.ai/llm-readable

# LLM-Readable Docs

These docs are built to be read by agents as easily as by people. Every page
has a markdown twin, and the whole corpus is published in the formats LLM
tooling already looks for — so you can point Claude (or any agent) at DreamLake
viz and have it answer accurately.

## Fetch a single page

Append `.md` to any docs URL to get the raw markdown — no nav, no chrome:

```bash
curl https://viz.dreamlake.ai/index.md
```

Every page also advertises its markdown twin in the HTML head:

```html
<link rel="alternate" type="text/markdown" href="/index.md" />
```

## The whole site, two ways

- **[`/llms.txt`](https://viz.dreamlake.ai/llms.txt)** — a short, linked index
  of every page ([llmstxt.org](https://llmstxt.org) standard). The entry point
  an agent reads first to decide what to fetch.
- **[`/llms-full.txt`](https://viz.dreamlake.ai/llms-full.txt)** — every page
  concatenated into one markdown file. Drop the entire product into a context
  window in a single request.

## Import it as a skill

The docs are also packaged as an [Agent Skill](https://viz.dreamlake.ai/skills/dreamlake-viz.zip)
— a `SKILL.md` plus one markdown reference file per page. Install it so your
agent loads DreamLake viz knowledge on demand:

```bash
# Claude Code: drop it into your project (or ~/.claude) skills directory
curl -L https://viz.dreamlake.ai/skills/dreamlake-viz.zip -o dreamlake-viz.zip
unzip dreamlake-viz.zip -d .claude/skills/
```

> **Note:** **Always current.** Every surface above — the `.md` pages, both `llms` files,
> and the skill — is generated from the same source on each deploy, so none of
> them can drift from what you read on the site.