Skip to main content

Import and export Runs

Use import and export at the interoperability boundary. Import creates, appends to, or replaces a Dataset; export reads complete Runs from an existing Dataset. Import and export accept ATIF, ACTF, OpenAI Messages, Storyline JSON, and record-level Compact JSONL. Import also accepts decode-only Codex (codex) and Claude Code (claude-code) session JSONL. Export refuses those two formats.

Compact JSONL keeps one JSON object per row without assigning trajectory semantics. Use --input-format compact-jsonl or --output-format compact-jsonl; see the CLI reference for the --column mapping and snapshot-sync restrictions. Records without a usable id receive a stable source_filename#line_number identity; export preserves original input bytes. Successful compact import also writes a leaf chronicle.manifest at the dataset root so later discovery can avoid opening Lance only to classify the tree (RFC-0015).

Import into a new Dataset​

pchronicle import --from input.json \
  --to ./imported --input-format atif

The default --mode create refuses an existing target. Use --mode append for an existing Storyline Dataset; duplicate document_id values receive a #N suffix by default, or can be skipped with --on-duplicate skip. Use --mode replace to stage the complete import and atomically replace an existing local Dataset after confirmation; replacement requires interactive confirmation or --yes. Existing object-store Datasets cannot currently be replaced in place. Regular files can be auto-detected. A directory recursively imports .json, .jsonl, and .ndjson files while preserving their relative paths in the default output. When --input-format is omitted, each file is detected independently; JSON that is not a known run data format is skipped with a warning:

pchronicle import --from ./corpus --to ./imported
pchronicle import --from ./codex-sessions --to ./codex-ds --input-format codex
pchronicle import --from ./claude-sessions --to ./claude-ds --input-format claude-code

The default output preserves input bytes. To normalize and squash all decoded inputs into one Storyline Lance Store at the output root, select Storyline output:

pchronicle import --from ./corpus --to ./normalized \
  --output-format storyline

A validated, non-empty canonical Event Store is detected before JSON scanning and always creates Storyline Lance:

pchronicle import --from ./run/events.lance --to ./run/storyline

This mode accepts local and object-store URIs, never mutates the source, and supports create or confirmed replace (not append). Its JSON result reports format: "events", output_format: "storyline-lance", and fact_rows; it omits input_bytes. Explicit --output-format preserve and JSON exchange --input-format values are invalid for canonical events.

In the squashed Dataset, _file_ is . for all normalized rows:

pchronicle query ./normalized \
  --sql 'SELECT _file_, COUNT(*) AS runs FROM dataset.runs GROUP BY _file_'

document_id is globally unique in Storyline output. Collisions receive a deterministic #N suffix; append can instead skip them with --on-duplicate skip. Successful Storyline output does not retain source paths as queryable information; use preserve output when file boundaries matter.

ATIF .jsonl and .ndjson inputs decode every non-empty record. Symbolic links found while walking a directory are skipped; an explicitly named link to a regular file retains single-file behavior. The directory is published atomically only after every input and the selected storage output succeed. Stdin must be finite and explicit:

cat input.json | pchronicle import --from - \
  --to ./imported --input-format openai-messages

After import, inspect the new boundary:

pchronicle stats ./imported
pchronicle stats overview ./imported

Export complete Runs​

pchronicle export --from ./imported \
  --to restored.json --output-format atif

Narrow the export with file path and external identity when needed:

pchronicle export --from ./imported --to one.json --output-format actf \
  --source source.json --session-id session-42 --strict

--strict fails when the target format cannot preserve the original exchange document. Output files are create-only unless overwrite is requested explicitly.

Import/export is not a storage migration protocol and arbitrary SQL rows are not exportable Runs. For exact flags, see the pchronicle CLI reference. See Run data formats for contracts and data contracts and revisions for the internal layer boundary.