DreamLake
Browse docs

Dataset API

Use @dreamlake/viz/dataset-viz to embed a configured dataset in an app. The host provides storage access; the dataset browser reads and displays its episodes.

Open and render

This component opens the public PushT dataset and releases it on unmount. fill gives the browser its own scroll region inside the supplied height.

tsx
import { useEffect, useState } from 'react'
import {
  EpisodeBrowser, openDataset, validateDreamrc,
  type DatasetHandle,
} from '@dreamlake/viz/dataset-viz'

const config = validateDreamrc({
  version: 1,
  storage: { driver: 'hf', repo: 'lerobot/pusht' },
  dataset: { format: 'lerobot', episodes: { limit: 3 } },
  views: [
    { view: 'videoStack', cameras: ['observation.image'] },
    { view: 'lineChart', series: ['action'], unit: 'px' },
    { view: 'timeline' },
  ],
})

export function DatasetPage() {
  const [dataset, setDataset] = useState<DatasetHandle | null>(null)
  const [error, setError] = useState<string | null>(null)

  useEffect(() => {
    let closed = false
    let opened: DatasetHandle | undefined
    openDataset(config).then(value => {
      opened = value
      if (closed) value.release()
      else setDataset(value)
    }).catch(error => {
      if (!closed) setError(String(error))
    })
    return () => { closed = true; opened?.release() }
  }, [])

  if (error) return <p>{error}</p>
  if (!dataset) return <p>Opening dataset…</p>
  return <div style={{ height: 600 }}>
    <EpisodeBrowser ds={dataset} views={config.views} fill />
  </div>
}

For an authored YAML file, parse it in the host before calling validateDreamrc. For private data, register an authorized storage driver and pass its identifiers, or supply a storage instance to openDataset. An explicit config storage precedes the host's injected storage or rootStorage.

Choose the rendering level

APIUse
openDataset + EpisodeBrowserBrowse episodes with paging and task-text search
dataset.episodes(infos) + EpisodeListChoose which episode handles to render
DatasetVizRender one episode with a views list
resolveDatasetMaterialize a fixed list; default cap 2,000 episodes

openDataset returns infos, warnings, episodes(infos), and release(). Inspect warnings for truncated or empty enumeration. Use maxEpisodes to cap a session. release() stops dataset prefetch and disposes supported reader resources; call it when changing datasets as well as on unmount.

Read a field

An episode exposes fields(), read(ref, query?), and timeline(). The field catalog supplies addresses and metadata. Narrow the result of a read by its payload kind:

ts
import type { Episode } from '@dreamlake/viz/dataset-viz'

async function readAction(episode: Episode) {
  const payload = await episode.read('action', { as: 'series', maxPoints: 2000 })
  if (payload.kind !== 'series') throw new Error('Expected numeric action data')
  return { time: payload.timestamps, columns: payload.columns }
}

ReadQuery also accepts timeRange, signal, and image dimensions for keypoints. Windowing and sampling depend on the reader; these options do not guarantee a bounded network transfer for every format.

PayloadMain values
video, image, fileA URL; video may include an episode window within a shared file
seriesTimestamps and named numeric columns
frames, depth, pointcloudFrame count, optional fps, and a function to load a frame
segmentsStart, end, and label for each interval
keypointsPixel dimensions, fps, sparse detections, and optional skeleton
transform3d, pose3d, vertices3dRigid poses, point sets, or mesh vertices
mesh3dGeometry URL, format, and optional base transform

Use Extract<Payload, { kind: 'series' }> to obtain a specific public payload type. The full union is defined in dataset-viz/types.ts. Frame URLs belong to the payload; consumers should not revoke them. Progressive series may provide residency notifications, and transform tracks may provide additional windows through source; custom views must handle those when present.

Storage access

A Storage implements resolveUrl(path) and list(path, options?). Listings return entries with name, path, and type, plus an optional pagination token. The built-in hf and http drivers serve public data. A host's driver can resolve authorized or signed URLs using the application's access policy.

The http index.json manifest

A static HTTP root needs a manifest for every directory a reader lists:

json
{
  "entries": [
    { "name": "run_001", "path": "episodes/run_001", "type": "dir" }
  ]
}

This could be episodes/index.json; the episode directory needs its own file listing. MCAP episodes: auto also lists the root. Selecting one MCAP with dataset.path avoids that enumeration.

Formats

lerobot

Reads v2.0, v2.1, and v3.0 through meta/info.json. Feature declarations provide camera addresses, numeric shapes, and dimension labels. Depth-video encodings are not handled by this reader; use supported raw-depth data for depthStack.

folder

Lists each episode root and its annotations/ directory. Numbered images form frame sequences. The default still-frame rate is 30; set dataset.fps when the recording differs.

umi

Reads a Zarr ReplayBuffer zip or the supported v3 feature-manifest layout. Use dataset.path when several stores exist. ReplayBuffer fps defaults to 60; a v3 manifest's own fps takes precedence. Unsupported codecs and numeric shapes are omitted with warnings.

mcap

Requires indexed MCAP v1 with uncompressed, lz4, or zstd chunks. Supported messages include Foxglove protobuf/JSON, ROS 1 ros1 with ros1msg, and ROS 2 cdr with ros2msg. ros2idl, flatbuffers, grids, calibration messages, and compressed-video packets are not decoded by this reader.

defaultMcapViews(fields) generates an initial layout from supported schemas; mcapDreamrcYaml(views) serializes it. Inspect or edit that layout to choose the topics relevant to your recording.

Add a reader or view

registerStorage(name, factory) adds a backend. registerFormat(name, factory) adds a reader implementing Dataset. registerComponent(spec) adds a view with its accepted payloads in reads and optional config-facing slotNames. Register these before validating a config that uses their names.

A custom component receives the fields, bound read function, timeline, cursor, and config. Reuse the cursor to synchronize with existing views. The mujoco and urdf registrations in this package are contracts with placeholders; the DreamLake app supplies their playback implementations.

Collection API

DatasetBrowser({ ds, config, ...episodeBrowserProps }) chooses the configured presentation. CollectionBrowser({ ds, views, labels?, pageSize = 50, groupBy = 'directory', selectedPath?, onSelect?, onClose? }) is also available directly. selectedPath makes selection controlled; onSelect(info) reports the original EpisodeInfo. Pass selectedPath={null} to show the grid in controlled mode, and clear the path in onClose() when the preview close button is pressed. Since 0.31.1, uncontrolled collections start with no selection. Selecting a thumbnail shrinks the grid into a narrower, multi-column grid alongside one preview; closing it unmounts the preview and restores the full grid, filters and grid scroll position. No episode is materialized before selection. The collection owns metadata search, task filtering, scroll-triggered incremental list display (400px bottom margin, one page per intersection; Load more fallback) and one selected DatasetViz. Its search is local and metadata-only; EpisodeBrowser search providers and semantic-search chips are list-mode APIs. Always call ds.release() when the owning dataset is replaced/unmounted.

USD resources

The portable renderer and indexed collection API require version 0.31.0 or later. The exact tested package archive can be installed independently. Its GitHub repository is private: use an authenticated GitHub CLI account with repository access, then install the downloaded archive. The release URL is not an anonymous package endpoint. npm publication of this version has not been verified.

bash
gh release download v0.31.3 --repo dreamlake-ai/viz-workspace \
  --pattern dreamlake-viz-0.31.3.tgz --dir .vendor
pnpm add ./.vendor/dreamlake-viz-0.31.3.tgz react react-dom

The public USD example renders without GitHub authentication; package archive download authorization is separate from runtime USD resource authorization.

UsdViewer is the same lazily loaded renderer used by Libraries and view: usd:

tsx
import { UsdViewer } from '@dreamlake/viz/dataset-viz'
<UsdViewer entry="pick.usda" sceneKey="pick"
  files={[{ path: 'pick.usda', url: 'https://example.com/pick.usda' },
          { path: 'robot.usda', url: 'https://example.com/robot.usda' }]} />

entry is a relative virtual path. files maps allowed relative paths to direct HTTP(S) URLs. sceneKey identifies the mounted scene. Optional resolveResource(path): Promise<string> authorizes additional relative resources on demand, never an arbitrary authored external URL. fallback customizes the error fallback. framing, focusPaths, cameraDirection, maxTextureSize and exposure match the view options. Keep file arrays and callback identities stable to avoid restarting loads.

A dataset host injects authorization without replacing the USD component:

tsx
import { UsdResourceProvider, DatasetBrowser } from '@dreamlake/viz/dataset-viz'
// Define once or memoize in the host. Resolve only against the authorized root.
const resolveAsset = async ({ field, payload, signal }) => {
  signal.throwIfAborted()
  const entry = String(field.meta.path)
  return {
    entry,
    files: [{ path: entry, url: payload.url }],
    resolveResource: async (path) => {
      signal.throwIfAborted()
      return authorizedStorage.resolveUrl(path)
    },
  }
}
<UsdResourceProvider resolveAsset={resolveAsset}>
  <DatasetBrowser ds={dataset} config={config} fill />
</UsdResourceProvider>

The typed UsdAssetResolver receives { field, payload, signal } and returns Promise<UsdAsset>. Its signal is aborted on selection changes and unmount. Project hosts may translate opaque node IDs into virtual paths; Source hosts resolve paths relative to the selected storage root. No credentials enter the package, config, worker messages or authored USD. Without a provider, publicUsdAsset resolves siblings under the public entry URL's directory. Dependencies outside that directory need an explicit manifest or scoped provider.

Importing the package is SSR-safe; WebGL and Worker start only after mounting. The package ships worker/WASM assets with source and built distributions. Serve those emitted assets with your application (WASM MIME application/wasm), allow same-origin workers in CSP and use a bundler that supports static new URL('./asset', import.meta.url) assets. No CDN, remote runtime, patch-package step or DreamLake application code is required. The docs live example uses this same public entry point and public HTTP storage.

For Vite development, exclude the library from dependency prebundling so asset URLs stay relative to their package module (production Vite rewrites them):

ts
export default defineConfig({
  optimizeDeps: {
    exclude: ['@dreamlake/viz', '@dreamlake/viz/dataset-viz'],
    include: ['@react-three/fiber', '@react-three/drei'],
  },
})

Include the package CSS emitted by your build; the source entry imports its CSS. When consuming prebuilt ESM directly, import @dreamlake/viz/dataset-viz.css via your bundler's asset mapping or serve the built CSS alongside it.

DatasetHandle.resolveAssetUrl?(path) resolves a root-relative thumbnail or other catalog asset lazily through the dataset's storage. CollectionBrowser uses it only after a thumbnail approaches the viewport. Manually constructed handles may omit it; their entries then render without thumbnails. Indexed USD catalogs populate EpisodeInfo.meta.thumbnail, tasks, bytes and filename.