# Deckard — full documentation > Generated from deckard docs content. Site: https://deckard.lol --- # Deckard Documentation **Deckard** is a browser DAW written in [Tish](https://tishlang.com). Humans and LLM agents **co-DJ** by streaming a small line-oriented language called **deck**. The app decodes deck into live-synthesised stems — never WAV/audio files. ## Start here - **Humans:** [Install & run](/docs/getting-started/install/) → [Co-DJ quickstart](/docs/getting-started/co-dj/) - **Agents / LLMs:** read [`/llms.txt`](/llms.txt) first, then [Agent contract](/docs/agents/overview/) and [Agent grammar](/docs/deck/agent-grammar/) - **Language reference:** [deck overview](/docs/deck/overview/) → [full grammar](/docs/deck/grammar/) ## Naming (do not confuse these) | Term | Meaning | |------|---------| | **Deckard** | The DAW product / this app | | **deck** | The streamable patch language (`.deck` files, header `deck 1`) | | **DJ deck A/B/C/D** | Player lanes in the mixer (`deck A` inside a track body) | ## Mental model 1. Every UI edit **round-trips through deck** (emit → apply). The UI is a deck client. 2. Co-DJ peers speak deck over WebSocket (`deck.line`, `deck.block`, `deck.stream_chunk`). 3. Agents compose prompts from [skills](/docs/agents/roles/) + [agent grammar](/docs/deck/agent-grammar/) and are gated by [DJ skills](/docs/agents/skills/). 4. Beats are **quarter notes**. Step `i` (0–15) = beat `i * 0.25` in a 4/4 bar. ## LLM entry points - [`/llms.txt`](/llms.txt) — curated index (llmstxt.org) - [`/llms-full.txt`](/llms-full.txt) — concatenated docs for large context - Repo checkout: root `llms.txt` + `AGENTS.md` + `docs/` --- # Install & run ## Prerequisites - **Node 22.x** (see `engines` in `package.json`) - **pnpm 9.15.4** (pinned via `packageManager`) or npm - A `tish` CLI from `@tishlang/tish` (installed by postinstall) ## Dev ```bash npm install # or pnpm install npm run dev # Vite — opens the DAW; edits to .tish hot-reload the page ``` Click **Play** once to unlock audio. ## Production static site (deckard.lol) ```bash npm run build:docs # generate /docs HTML + llms-full.txt npm run build:static # compile DAW + assemble ./build (includes public/docs) npm run serve:static # http://localhost:3456 ``` `build:static` produces `./build` with the DAW at `/` and docs at `/docs/`. DigitalOcean App Platform deploys that folder (see [DEPLOY.md](https://github.com/spacedevin/deckard/blob/main/DEPLOY.md)). ## Tests ```bash npm test ``` Headless smoke covers deck round-trip, streaming, skills, and co-DJ permissions. --- # Co-DJ quickstart **Order:** gateway → worker → browser. ## 1. Gateway ```bash npm run gateway # listens on ws://127.0.0.1:35987 (or $PORT / CODJ_HUB_PORT) ``` On deckard.lol the gateway is same-origin at `wss://deckard.lol/codj`. ## 2. Agent worker ```bash npm run agent # or: npm run agent:host / npm run agent:client ``` Set `GRADIENT_MODEL_ACCESS_KEY` for real LLM replies. Without a key the worker falls back to a demo patch so the loop still runs offline. ## 3. Browser Open the DAW (`npm run dev` or deckard.lol) → **Co-DJ** → **Connect** (session `default`) → **Play**. On Play the host streams the project as `deck.line`. The worker buffers, calls the LLM, and replies with `deck.stream_chunk` / `deck.block`. The browser applies the block and you hear the new pattern. ## Direct test In Co-DJ, set **Direct→** to the worker's actor, type e.g. `euclid hi-hat`, **Send test direct**. Expect a `deck.block` reply. ## Specs - [WebSocket & actors](/docs/agents/websocket/) - [Stream protocol](/docs/agents/stream-protocol/) - [DJ skills](/docs/agents/skills/) - [Token stream demo](/docs/reference/token-stream-demo/) --- # deck overview **deck** is Deckard's streamable patch language. Files use the `.deck` extension and start with: ``` deck 1 bpm 120 ``` Legacy `tpl 1` is still accepted by the parser; emit always writes `deck 1`. ## Anatomy ``` deck 1 bpm 118 swing 0.12 scale C minor track Kick id c0 gen noise_burst mix gain 0.85 pan 0 step_pitch 36 steps x . . . x . . . x . . . x . . . track Bass id c1 gen bass_acid note 36 0 0.5 v 100 note 39 1 0.5 v 90 ``` - **Header** — `deck 1` then globals (`bpm`, `swing`, `scale`, `xfade`, `main_deck`, `deck_mix`, …) - **Tracks** — `track id gen [* bars]` - **Body** — indented `mix` / `fx` / `voice` / `gen` / `adsr` / `steps` / `note` / `deck A|B|C|D` / `gen_block` … ## Two meanings of `deck` | Context | Syntax | Meaning | |---------|--------|---------| | Version header (top-level) | `deck 1` | Language version | | Track body | ` deck A` / `B` / `C` / `D` | DJ player lane routing | ## Runtime path Language parse / format / highlight: **`@spacedevin/deck`**. Host apply / emit / stream: `src/deckfile/` (`Apply.tish`, `Emit.tish`, `Stream.tish`). Registries boot from `src/generators/DeckIds.tish`. Incremental Co-DJ decode is skill-gated per line. ## Full references - [deck grammar](/docs/deck/grammar/) — canonical language (`@spacedevin/deck`) - [Deckard overlay](/docs/deck/host/) — UI, ownership, clamps - [Agent grammar](/docs/deck/agent-grammar/) — co-DJ lane subset - [Extensions](/docs/deck/extensions/) — `gen_block` patch / matrix engines --- # deck grammar Line-oriented, streamable patch text (`.deck`). This is the **language** reference for `@spacedevin/deck`. **Package responsibilities:** tokenize, `parseProgram` → AST, format helpers, bar selectors, Euclidean step fill, scale root/mode vocab, highlight classify, empty registries (generator id / param key / macro / gen_block dialect). **Host responsibilities:** map AST → project IR (apply/emit), audio engines, ownership/skills, co-DJ, UI. Generators, builtin macro catalogs, and `patch` / `matrix_fm` dialect parsers are **host-registered**. Times are in **quarter-note beats**. One bar = 4 beats = **16** sixteenth steps. --- ## Lexical - Lines are statements. Indentation (2+ spaces or tab) nests a body under the current open block (`track`, `clip`, `auto`, `macro`, `song`, `follow`, `gen_block`). - `#` starts a comment to end of line — but **only at column 0 or after whitespace**, so a `#` inside a token is data. That is what makes sharp note names (`scale F# minor`, a track named `C#maj`) work. - Tokens: whitespace-separated; numbers accepted by `isNumberToken`. - Legacy alias: `tpl` ≡ `deck` for the version header only. --- ## Version header ``` deck 1 ``` Recommended first non-comment line. Emit writes `deck 1`. Distinct from track-body routing `deck A|B|C|D`. --- ## Top-level statements These are recognized by `parseProgram`. | Statement | Form | Notes | |-----------|------|--------| | Version | `deck ` / `tpl ` | | | Tempo | `bpm ` | | | Swing | `swing <0..1>` | Off-beat 16th shuffle; `0` = straight | | Scale lock | `scale ` | `root` = note (`C`, `F#`, `Bb`) or pitch-class `0..11`; modes below. `scale off` / `none` / `chromatic` clears (AST root `-1`) | | Launch quant | `launch_quant ` | Scene/clip launch grid (bars), `n ≥ 1` | | Song seed | `song_seed ` | Seeds deterministic randomness (e.g. step probability) | | Crossfader | `xfade []` | Both `0..1`; if `y` omitted, `y = 0.5` | | Main deck | `main_deck live\|local` | Which booth feeds the main out | | Booth mix | `deck_mix [hi n] [mid n] [lo n] [flt n] [vol n]` | Any subset of keys | | Track | `track id gen [ * ] [ … ]` | Name may be multi-word; anchored on `id` / `gen` | | Remove track | `remove_track ` | Incremental edit; not present in full snapshots | | Macro def | `macro [k=default …]` … `end macro` | Body lines = patch dialect lines | | Automation | `auto …` + indented ` ` | See [Automation](#automation) | | Master mix | `master_mix eq_lo eq_mid eq_hi ` | Keys any order; missing keys unchanged | | Actor mix | `actor_mix …` | `gain`/`trim`, `eq_*`, optional `mute`/`solo` | | Session scenes | `session_scenes ` | `n ≥ 1` | | Session slot | `session_slot ` | `-`/`.` clears | | Clip | `clip channel bars [name …]` + indented body | | | Song | `song` + indented `P [x]` or bare scene index | 1-based `P` | | Follow | `follow` + indented `P [ ]` | 1-based `P` | | Control directive | `@ …` | Collected into `directives[]`; the verb is host-interpreted. See [Control directives](#control-directives-) | ### Scale modes Accepted mode tokens (aliases in parentheses): `major` (`ionian`), `minor` (`aeolian`), `dorian`, `phrygian`, `lydian`, `mixolydian`, `locrian`, `harmonic_minor`, `melodic_minor`, `pentatonic_major` (`penta_major`, `majpenta`), `pentatonic_minor` (`penta_minor`, `minpenta`), `blues`. Package helpers: `parseScaleRoot`, `scaleRootNames`, `scaleModeNames`, `scaleIntervals`. ### Track header ``` track id gen [ * ] [ … ] ``` - `* N` — **pattern length** in bars (default 1). Channel spans `N × 16` steps and repeats. `* inf` / `* infinite` clears an explicit finite length. - Trailing `key value` pairs — **macro parameter overrides** when `gen` is a macro name. - `* N` and the `key value` pairs may appear in **any order** after `gen `. Emit writes `* N` first; a `*` that names no valid length is an error, never a silently dropped token. - `generatorId` spellings are host-registered (`registerGeneratorIdAliases`). Undeclared ids pass through as-is. --- ## Track / clip body `parseProgram` stores indented body lines as token rows (except `gen_block` collection); **`parseBodyLine` / `parseTrackBody`** turn those rows into typed values. The heads below are the standard language. Body parsing is deliberately **parse-only**: an absent optional is `null` so the host applies its own default, and there is no clamping or range checking — that is host policy, and hosts differ (one clamps an out-of-range lock, another rejects it). Range checks needing track context (`note` start vs `* N`) can't live here at all. An unrecognised head comes back as `kind: "unknown"` so a host dialect can claim it via `registerBodyLineDialect` — see [DECK_EXTENSION.md](DECK_EXTENSION.md). ### Mix ``` mix gain pan [mute <0|1>] [solo <0|1>] [eq_lo ] [eq_mid ] [eq_hi ] ``` Boolish: `1`/`true`/`on` vs `0`/`false`/`off`. ### Pattern length vs play cap | Form | Meaning | |------|---------| | `* N` on track header | Pattern **length** (bars); loops forever | | `loops ` | Finite **play cap** since Play / re-apply; then silent | Compose: `* 4` + `loops 8` = 4-bar pattern played twice, then stops. ### Steps ``` steps x . . . x . . . x . . . x . . . steps euclid ``` - On: `x` `X` `1` · Off: `.` `0` - Euclidean: Bjorklund fill (`euclideanPattern` in this package). Common host constraint: `len = 16`. #### Step lock lanes (after `steps`) Emitted only when a step differs from the default: | Lane | Range (default) | Meaning | |------|-----------------|--------| | `step_vel` | `1..127` (100) | Velocity | | `step_prob` | `0..1` (1) | Hit probability (seeded; peers agree) | | `step_ratchet` | `1..8` (1) | Sub-hits over the step | | `step_nudge` | `-0.5..0.5` (0) | Micro-timing as a fraction of a step | A bare `steps` line resets locks; following lanes restore deviations. Optional host extension: `step_lyric` (emitted by some hosts). ### Step pitch ``` step_pitch [ bar ] ``` Base MIDI for step hits when the channel has **no** `note` lines (default **36**). With `bar `, one line per bar/group for multi-bar patterns. ### Notes (piano roll) ``` note v [ p ] [ r ] [ n ] [ bar ] [ l ] ``` - Beats in quarter notes. For pattern length `N`: `0 ≤ startBeat` and `startBeat + durBeats ≤ N×4`. - Optional locks (non-default only on emit): `p`, `r`, `n` — same semantics as step locks. - `bar `: keep `startBeat < 4`; expand onto matching loop bars. - Host optional: `l ` on notes / step lyric lane. **Steps vs notes:** if a track block contains any `note` lines, steps for that channel are cleared. If it contains `steps` and no `note`s, piano notes are cleared. Playback prefers notes when any exist. Also: `notes_clear` — host edit fragment that clears piano notes. ### Transpose ``` transpose ``` Integer shift applied to collected `note` pitches for that block. ### Generator params (fixed / generic) Hosts typically accept: ``` gen … adsr a d s r ``` plus legacy one-line shapes for specific engines (`noise …`, `fm …`, `osc waveform …`). Snake_case keys map via `paramKeyToCamel` / `registerParamKeyAliases`. ### Channel FX / voice / deck routing ``` fx reverb_send drive lfo_rate lfo_depth cutoff res [filter_type ] voice octave arp chord arprate inversion strum deck [slot ] ``` `fx` also accepts `reverb` as alias for `reverb_send`, and `type` as alias for `filter_type`. ### Heavy generators (`gen_block`) ``` gen_block … end gen_block ``` Core language collects lines until `end gen_block`. `parseGenBlock(id, lines)` returns `{ kind, tplHeaderId, version, raw }` until a host dialect is registered. See [DECK_EXTENSION.md](DECK_EXTENSION.md) for the registration API and common `patch` / `matrix_fm` dialects. --- ## Bar selectors Single token (no spaces). Used after `bar` on `note` / `step_pitch`. Bars are **0-indexed** within the track's `* N` length. | Selector | Matches | |----------|---------| | `even` / `odd` | 0,2,4,… / 1,3,5,… | | `` | that bar only | | `n` / `*` / `all` / `every` | every bar | | `n` | `bar % a == 0` | | `n+` | `bar % a == b` | | `-n+` | first `b` bars (`0 .. b-1`) | | `b0,b1,…` | explicit list | Package: `parseBarSelector`, `barSelectorMatches`. --- ## Macros **Define** (top-level): ``` macro [key=default …] … patch-dialect body with $key … end macro ``` **Use:** `track … gen [key val …]` — expands to a `gen_block patch` at load (when the host registers a patch dialect + builtin/user macros). Package provides `lookupMacro`, `expandMacroBody`, `registerBuiltinMacros` (catalog is empty until the host fills it). --- ## Automation ``` auto master_gain auto gen auto mix auto actor mix auto master mix ``` Indented points are `beat value` pairs. Hosts interpolate on the beat timeline (`beat = globalStep × 0.25` for step playback). --- ## Session / scenes / clips ``` session_scenes session_slot clip channel bars [name ] steps … note … loops … ``` Clip grid length = `bars × 16` steps. Clip notes may span the whole clip (`bars × 4` beats). Same steps-vs-notes rule as tracks. ### Song arrangement ``` song P1 P2 x4 3 ``` 1-based scene refs (`P` or bare index). Optional `xN` repeat. ### Follow actions ``` follow P1 next 1 P2 jump 0.7 stay 0.3 ``` `P [ ]`. Host interprets action tokens. --- ## Control directives (`@ …`) Transient **stream** lines. Most are **not** stored in a static project document; hosts apply them for performance / co-DJ. `parseProgram` collects every `@ …` line into `directives[]` as `{ lineNo, verb, tokens }` and does not interpret the verb — that is host policy. A bare `@` with no verb is an error. Hosts typically understand: | Directive | Typical authority | Effect | |-----------|-------------------|--------| | `@ launch scene ` | master | Arm scene clips | | `@ launch clip ` | track owner | Per-track clip / release / stop | | `@ transport play\|song\|sequence [scene] \| stop` | master | Shared transport | | `@ transport preview` | private | Local preview clock | | `@ cue ` | private | Load into local cue | | `@ throw [scene]` | master | Cue → shared main | | `@ fx on\|off …` | master | Live master FX | | `@ deck on\|off` · `spin` | deck owner / master | Vinyl platter moves | | `@ perf_step ` | — | Schedule surrounding block for perf step `n` | Any other verb is collected too, so a host may define its own without a parser change. --- ## Format helpers Package emit helpers (numeric spelling): - `formatTplBeat` — snap to 1/96 beat, trim zeros - `formatTplFloat` — ≤ 4 decimal places, trimmed --- ## Streaming rules 1. Strip comments; ignore empty lines. 2. A line commits when its newline arrives and any open `gen_block` is closed. 3. Partial trailing lines must not mutate state. 4. Incremental merge is by `channelId` / clip id / automation key (host apply). --- ## What is not language (host-only) - Audio engines and Web Audio graphs - Instrument / preset catalogs and generator default param tables - Builtin macro catalogs (register into the package) - Ownership, skills, co-DJ transport plumbing - HTML highlight styling (`tpl-hl-*`) — classify API only lives here - Project JSON / IR schemas beyond what the AST implies --- ## Golden example ``` deck 1 bpm 118 swing 0.08 scale C minor xfade 0.5 0.5 main_deck live track Kick id c0 gen noise_burst mix gain 0.9 pan 0 eq_lo 0 eq_mid 0 eq_hi 0 deck A step_pitch 36 steps x . . . x . . . x . . . x . . . step_vel 120 100 100 100 70 100 100 100 100 100 100 100 90 100 100 100 track Bass id c3 gen fm * 2 mix gain 0.85 pan 0 voice octave -1 fx cutoff 1200 res 0.4 note 48 0.0 0.5 v 90 note 50 1.0 0.5 v 85 bar even track Lead id c4 gen patch gen_block patch osc o1 sawtooth note filter f1 lowpass q 4 freq 1800 gain a1 0 conn o1 f1 1 conn f1 a1 1 conn a1 out 1 env a1.gain set 0 0 lin 0.01 0.9 lin dur 0 end gen_block note 60 0 1 v 80 session_scenes 4 session_slot c0 0 clip_kick_a clip clip_kick_a channel c0 bars 1 steps x . . . x . . . x . . . x . . . auto master_gain 0 0.85 16 0.9 master_mix eq_lo 0 eq_mid 0 eq_hi 0 ``` --- # Deckard host overlay **Canonical language reference** lives in **`@spacedevin/deck`**: - Checkout / npm: [`node_modules/@spacedevin/deck/docs/DECK_GRAMMAR.md`](../node_modules/@spacedevin/deck/docs/DECK_GRAMMAR.md) - Package export: `@spacedevin/deck/grammar` - Upstream: [github.com/spacedevin/deck](https://github.com/spacedevin/deck) Host integration (registries, dialects): package [`docs/HOST.md`](https://github.com/spacedevin/deck/blob/main/docs/HOST.md) · Deckard boot: [`src/generators/DeckIds.tish`](../src/generators/DeckIds.tish), [`src/generators/BuiltinMacros.tish`](../src/generators/BuiltinMacros.tish). This document is the **Deckard overlay** — UI, co-DJ, ownership, clamps, and generator spellings — not a second grammar. --- ## UI: local song vs hub stream There is a **single** deck editor (Apply / Sync, step highlight). **Stream vs local** is tied to **Co-DJ**: | State | deck panel | |-------|-----------| | **Not connected** | Banner: *Local — not on hub.* Edit the song locally. | | **Co-DJ connected** | Banner: **Hub** + **Local → hub** preview (what you send on Play) and **Remote** (agent `deck.stream_chunk` tail). | **Append a line from JS** (e.g. LLM tooling): call `streamAppendLine` on the `DeckardRuntime` held in `App`’s `useRef` (`src/ui/DeckardRuntime.tish`). Example: `runtime.streamAppendLine("track kick id c0 gen noise_burst")`. --- ## Host modules (not in the package) | Concern | Location | |---------|----------| | Parse / format / highlight classify | `@spacedevin/deck` | | AST → project | [`src/deckfile/Apply.tish`](../src/deckfile/Apply.tish) | | project → deck | [`src/deckfile/Emit.tish`](../src/deckfile/Emit.tish) | | Incremental `deck.line` | [`src/deckfile/Stream.tish`](../src/deckfile/Stream.tish) | | `patch` / `matrix_fm` graphs | [`PatchGraph.tish`](../src/deckfile/PatchGraph.tish), [`MatrixFmGraph.tish`](../src/deckfile/MatrixFmGraph.tish) | | Id / dialect registration | [`DeckIds.tish`](../src/generators/DeckIds.tish) | | Builtin macros | [`BuiltinMacros.tish`](../src/generators/BuiltinMacros.tish) | | Ownership / skills | [`src/codj/`](../src/codj/) | | HTML highlight | [`src/ui/SongEditorHighlight.tish`](../src/ui/SongEditorHighlight.tish) | --- ## Apply clamps (Deckard) These are enforced when applying into a project, not by the language parser: | Line | Deckard behaviour | |------|-------------------| | `bpm ` | Clamped **40–300** | | `swing` | `0..1` | | Scale | Snaps melodic triggers live; peers + offline renders agree ([`Scale.tish`](../src/model/Scale.tish)) | | `@ perf_step ` | Stream / Co-DJ scheduling; JSON `effectivePerfStep` on `deck.block` overrides. Song parser treats as no-op | --- ## Generator id spellings (Deckard) Registered in `ensureDeckGeneratorIds()`: | deck / emit | host `generatorId` | |-------------|-------------------| | `noise_burst` | `noiseBurst` | | `fm` / `fm_tone` | `fmTone` | | `basic_osc` / `osc` | `basicOsc` | | `matrix_fm` | `matrixFm` | | `drum` / `drum_synth` | `drumSynth` | | `patch` / `modular` / `synth` | `patch` | Undeclared ids pass through. Other catalog ids use the host registry spelling. Macro names expand via `registerBuiltinMacros` ([`MacroVoice.tish`](../src/model/MacroVoice.tish)). --- ## Ownership & edit fragments - **`remove_track `** — incremental only (never in a full snapshot). **Ownership-gated** by `actorMayEditTrack` (lane removes own track; master removes any). UI × emits it; undo re-creates from captured deck. - Master-scope lines (`bpm`, version header, `scale`, `swing`, `master_mix`, `actor_mix`, `auto`, `session_*`, `clip`, …) require `master_mixer` — see [DJ_SKILLS.md](DJ_SKILLS.md). - Co-DJ lane subset: [DECK_AGENT_GRAMMAR.md](DECK_AGENT_GRAMMAR.md). --- ## Control directives in Co-DJ Language `@ …` table: package grammar. Deckard wire + authority: [STREAM_PROTOCOL.md](STREAM_PROTOCOL.md), [WS_AND_AGENTS.md](WS_AND_AGENTS.md). --- ## Related - [DECK_EXTENSION.md](DECK_EXTENSION.md) — Deckard `patch` / `matrix_fm` engines - Package extension dialect syntax: `@spacedevin/deck/extension` --- # Agent grammar subset The lines a **co-DJ lane** may emit. Human/host stream lines are **read-only context**; you add **new tracks + patterns on your own lane only** (ids prefixed with your `actorId`, e.g. `ai-a_hat`). This is the agent subset of the full language — for the complete grammar (master-scope headers, clips, `gen_block` graphs, automation, control directives) see **[`@spacedevin/deck` grammar](../node_modules/@spacedevin/deck/docs/DECK_GRAMMAR.md)** (canonical). Deckard UI / ownership notes: **[DECK_GRAMMAR.md](DECK_GRAMMAR.md)**. Keep this in lockstep with the agent `SYSTEM_PROMPT` in [`services/agent-worker/main.tish`](../services/agent-worker/main.tish) and the master-scope denylist in [`src/codj/Skills.tish`](../src/codj/Skills.tish). ## Track block ``` track id gen [* ] mix gain <0..1> pan <-1..1> [eq_lo ] [eq_mid ] [eq_hi ] [mute 1] [solo 1] fx [cutoff ] [res <0..1>] [drive <0..1>] [reverb_send <0..1>] [lfo_rate ] [lfo_depth <0..1>] [filter_type ] voice [octave <-2..2>] [chord ] [arp ] [arprate ] [inversion ] [strum <0..150>] …generator param lines (see below)… adsr a d s <0..1> r (generators with an envelope) step_pitch (base pitch for step hits; default 36; `bar ` for per-bar) steps <16 x/. tokens> | steps euclid 16 step_vel <16 ints 1-127> (per-step velocity lock — optional, emit only deviations) step_prob <16 floats 0-1> (per-step probability — seeded, deterministic across peers) step_ratchet <16 ints 1-8> (per-step sub-hits) step_nudge <16 floats -0.5..0.5> (per-step micro-timing, fraction of a step) note v [p ] [r ] [n ] [bar ] (repeat; melodic) loops (finite repetition cap for this channel) ``` Beats are **quarter-notes**: step `i` (0–15) = beat `i*0.25` in the looping bar. `* ` on the header makes a multi-bar pattern; `bar ` (`even`/`odd`/``/`2n+1`/`0,2`) on `note`/`step_pitch` varies it per bar. ## Generators (`gen `) **Fixed generators** (33) — pick by role; don't default everything to `noise_burst`/`fm`: | role | ids | |------|-----| | perc/hat | `drumSynth` (`drum`) · `clap` · `cymbal` · `noise_burst` | | lead/synth | `basic_osc` (`osc`) · `fm` · `aether` · `syncLead` · `obSync` · `laserSync` | | keys/mallet | `tine` · `halo` · `bell` | | pad/texture | `pad` · `noise_burst` | | strings | `guitar` · `arco` | | chip | `chiptune` · `nes2a03` · `gameBoyDmg` · `c64sid` · `ym2612` · `sn76489` · `spc700` · `gbaDirectSound` | | vocal | `formantVocal` · `ttsVocal` · `meSpeakVocal` · `syncChoir` | | modular (advanced) | `matrixFm` · `patch` (use `gen_block … end gen_block`; prefer named generators) | **Macro voices** (expand to a tuned patch; use the macro name directly as `gen `): | role | ids | |------|-----| | kick | `kick_edm` · `kick_deep` · `kick_distorted` | | bass | `bass_reese_punch` · `bass_reese_sc` · `bass_wobble` (plus the bass **generators** `acid303` · `sub808` · `reeseBass`) | Generators vs. macros never share a label (see project memory *macro-generator-boundary-policy*). Only `noise_burst`/`fm`/`basic_osc`/`matrix_fm`/`drum_synth`/`patch` have short aliases — every other id is used verbatim. ### Generator param lines | `gen` | indented param line(s) | |-------|------------------------| | `noise_burst` | `noise attack decay tone <0..1> pitch_follow <0..1>` | | `fm` | `fm ratio mod_index carrier mod ` (+ `adsr`) | | `basic_osc` | `osc waveform ` (+ `adsr`) | | every other named generator | `gen …` — 0–1 designer knobs (e.g. `gen tone 0.5 swell 0.45`) | | `matrix_fm` / `patch` | `gen_block … end gen_block` graph — see [DECK_EXTENSION.md](DECK_EXTENSION.md) / package [extension](../node_modules/@spacedevin/deck/docs/DECK_EXTENSION.md) | ## What you may NOT emit (master-scope) These change the **whole session for every player**, so they require the `master_mixer` skill and are **skipped** for an agent lane (per [`Skills.tish`](../src/codj/Skills.tish)): ``` bpm tpl auto transpose scale swing master_mix actor_mix session_scenes session_slot clip @ ``` Also **never** emit `remove_track ` — deleting tracks is a host/UI action (owner-gated), not something an agent does. You add complementary parts. So **global key (`scale`) and groove (`swing`) are master-only** — you cannot re-key or re-shuffle the mix. Live control directives (`@ launch`/`transport`/`cue`/`throw`/`fx`/`deck`) are a separate master/owner surface — see [package grammar § Control directives](../node_modules/@spacedevin/deck/docs/DECK_GRAMMAR.md#control-directives-). ## Streaming You may stream `deck.line` incrementally — each line decodes progressively (a track sounds the moment its `track …` arrives; the pattern fills as `steps …`/`note …` stream). Lane-unique ids are required. **Goal:** complement the project — add the parts that are missing (hats, perc, bass, chords, melody), match the tempo, and leave space. Don't double what already plays. --- # Generator extensions Core collection and dialect registration live in **`@spacedevin/deck`** — see [package DECK_EXTENSION.md](https://github.com/spacedevin/deck/blob/main/docs/DECK_EXTENSION.md) (`@spacedevin/deck/extension`) and host boot in [`src/generators/DeckIds.tish`](../src/generators/DeckIds.tish). Deckard registers: | Dialect ids | Parser | Audio engine | Spec shape | |-------------|--------|--------------|------------| | `patch`, `modular`, `synth` | [`PatchGraph.tish`](../src/deckfile/PatchGraph.tish) | [`Patch.tish`](../src/generators/Patch.tish) | `{ nodes, conns, envs, dur }` on `generatorSpec.graph` | | `matrixFm`, `matrix_fm` | [`MatrixFmGraph.tish`](../src/deckfile/MatrixFmGraph.tish) | [`MatrixFm.tish`](../src/generators/MatrixFm.tish) | operators, mod matrix, filters, routes (schema [`project-v2.json`](schema/project-v2.json) `$defs/matrixFmGraph`) | Until registered, `parseGenBlock` returns `{ kind, tplHeaderId, version: 1, raw }` only. **Macros** are named patch templates registered via `registerBuiltinMacros` ([`BuiltinMacros.tish`](../src/generators/BuiltinMacros.tish) / [`MacroVoice.tish`](../src/model/MacroVoice.tish)). A `gen ` expands to a `gen_block patch` at load. Simple plugins read `generatorParams`; `matrixFm` and `patch` read `generatorSpec.graph` (each falls back if the graph is empty). Line-level dialect syntax (nodes, `op`/`mod`/`route`, …): package [DECK_EXTENSION.md](https://github.com/spacedevin/deck/blob/main/docs/DECK_EXTENSION.md) (also `@spacedevin/deck/extension` after install). --- # Agent contract This is the editing contract for LLM agents working on Deckard (same intent as repo `AGENTS.md`). ## Prefer these surfaces 1. **Project shape** — `docs/schema/project-v2.json` (`generatorId` + `generatorParams`) 2. **Load / model** — `src/model/Project.tish`, `ProjectLoad.tish` (`loadProjectFromTpl`), default song `projects/default.deckard.deck` 3. **Generators** — `src/generators/` ([docs](/docs/architecture/generators/)) 4. **Safe mutators** — `src/model/Edits.tish` 5. **deck language** — [grammar](/docs/deck/grammar/), [agent subset](/docs/deck/agent-grammar/), `src/deckfile/` ## Co-DJ - Agents emit **only their lane** (track ids prefixed with `actorId`, e.g. `ai-a_hat`) - Master-scope lines (`bpm`, version header, `master_mix`, `auto`, …) need `master_mixer` — see [skills](/docs/agents/skills/) - Prompt composition: [roles & skills](/docs/agents/roles/) + agent grammar ## Invariants - Project `version`: use `2` for generator-based projects - Times: `startBeat` / `durBeats` are **quarter-note beats** - Preserve channel `id` strings - Do **not** put sequencing rules in JSX-only UI files — keep logic in `model/`, schedule, `audio/` - UI is a **deck client**: edits go through emit/apply, not silent project mutation ## Audio / Tish JS Use `new AudioContext()` and `new Uint8Array(n)` on the JS target (see `docs/TISH_JS_BUILTINS.md` in the repo). --- # WebSocket & actors This document specifies how actors (browser, worker) connect to a **session gateway**, exchange deck and direction text, and how workers persist memory in **SQLite + vectors**. ## 1. Session gateway ### 1.1 Transport - **URL**: `ws://:` (e.g. **ws://127.0.0.1:35987**). No path; **first message** from client must be `join` and must include **`sessionId`**. - **Framing**: each message is a **JSON object** (UTF-8 text frame). For high-volume token streams, clients may batch; gateway may reject frames > 256 KiB. ### 1.2 Join handshake First message from client **must** be `join` (must include **`sessionId`** and **`actorId`**): ```json { "type": "join", "sessionId": "default", "actorId": "actor-uuid-123", "channelIds": ["default"], "skillIds": ["add_track", "adjust_instrument", "pattern_steps", "pattern_piano", "channel_mix", "master_mixer"] } ``` Gateway responds: ```json { "type": "joined", "sessionId": "...", "you": { "actorId": "...", "clientId": "...", "channelIds": ["default"] }, "actors": ["actor-1", "actor-xyz"], "replay": [] } ``` - **`actors`**: list of actorIds in the channel. - **`actorId`** (required): for **human** joins, the gateway picks a paired agent lane when exactly one agent is online (v1 prefers **`ai-a`**); otherwise `null`. Browsers show “No agent” until an agent connects. - When anyone joins or leaves, the gateway broadcasts **`presence`**: `{ "type": "presence", "sessionId": "...", "actors": ["agent-1", "human-xyz"] }`. ### Human deck stream (Play) While the browser actor is **playing**, the browser sends: 1. **`control`** `{ "type": "control", "op": "playing_start", "actorId": "...", "authorId": "...", "perfStep": }` — agents mark the session live and may run inference after buffered deck arrives. 2. **`deck.line`** per emitted deck line (throttled). The gateway stamps **`actorId`** from the connection and fans out. 3. **`control`** `{ "op": "playing_stop", ... }` on stop — agents clear live mode. Agents append `deck.line` text from the playing actor to a rolling buffer and respond with **`deck.stream_chunk`** then **`deck.block`** with their own `actorId`. ### 1.3 Message families | `type` | Direction | Fields | |--------|-----------|--------| | `deck.line` | any → gateway → fanout | `actorId`, `line`, `authorId`, `seq` (hub-assigned) | | `deck.block` | any → gateway | `actorId`, `lines[]`, `authorId`, optional `effectivePerfStep`, `submitDeadlinePerfStep`, `asap` — see [STREAM_PROTOCOL.md](./STREAM_PROTOCOL.md) | | `deck.stream_chunk` | worker → gateway → browsers | `actorId`, `chunk`, `authorId` (no seq until line commit) | | `direct` | any → gateway → target actor | `targetActorId` (target), `text`, `authorId`, optional `perfStep` (host 16th index when sent) | | `state.snapshot` | gateway or browser | `hash`, `tplPreview` (truncated), `ts` | | `control` | master | `op`, `payload` — see [STREAM_PROTOCOL.md](./STREAM_PROTOCOL.md) | | `error` | gateway → client | `code`, `message` | Error `code`s **actually emitted** by `services/gateway/main.tish`: **`BAD_JSON`**, **`JOIN_BAD`**, **`JOIN_FIRST`**. Codes such as `SKILL_DENIED`, `LEASE_CONFLICT`, `PARSE_FAIL`, `RATE_LIMIT`, and `AUTH` are **aspirational** — skill / ownership violations are currently handled by the receiver **silently skipping** disallowed lines (see [DJ_SKILLS.md](./DJ_SKILLS.md)). ### 1.4 Ordering - Gateway maintains **`seq` per `actorId`** (monotonic). All `deck.line` / `deck.block` / `direct` get a server timestamp + actor seq. - Cross-lane order is **not** total; merge rules live in the browser ([CO_DJ_SPACE.md](./CO_DJ_SPACE.md)). ### 1.5 Rooms - One **room** per `sessionId`. All joined clients receive fanout for messages they are allowed to see (v1: fanout all). ## 2. Worker contract (actor + skills) ### 2.1 Process model - One worker **per actor** per session (e.g. `actor-1`), or a **pool** with one job per `(sessionId, actorId)`. - Worker connects with `actorId` (e.g. `actor-1`), `channelIds: ["default"]`, `skillIds: [...]`. ### 2.2 Input loop The persistence / RAG steps below (local DB, embeddings, `chunks`) are **planned** — see [§3](#3-sqlite--vector-schema-planned). Today the worker keeps an in-memory rolling context buffer only. 1. On each inbound `deck.line` / `deck.block` (other actors or merged snapshot): append to the context buffer (planned: also persist to local DB and embed chunks). 2. On each `direct` where `targetActorId` matches this actor's `actorId`: enqueue for response. 3. On timer or debounce: build context = system prompt + last N buffered messages (planned: + **RAG** top-k from `chunks`). ### 2.3 Output loop The worker (`services/agent-worker/main.tish`) **calls its LLM**: `handlePlayingStream` and `handleDirect` snapshot the buffered peer stream into one prompt (`llmLinesFrom`), call `callLLM`, and stream the **real model output** — first as `deck.stream_chunk` (the finished text char-by-char), then as a committed `deck.block`. It **falls back to a demo patch only** when `GRADIENT_MODEL_ACCESS_KEY` (or `MODEL_ACCESS_KEY`) is unset or the call fails. The worker also **ingests inbound `deck.block`** from other actors into its context buffer so responses are aware of them. The old 3-second periodic demo `deck.block` flood has been **removed**. > **Note:** the worker streams the *finished* text char-by-char — provider-side SSE token streaming (live provider tokens) is **planned, not implemented**. 1. Snapshot the buffered peer stream into one prompt; call `callLLM` and stream the resulting text. 2. Emit `deck.stream_chunk` (per character of the finished output) for the UI typing indicator, then a `deck.block`. 3. Validate deck line against actor's skills ([DJ_SKILLS.md](./DJ_SKILLS.md)); if invalid, log and skip or send `error` to gateway. 4. For **`deck.block`**: set **`effectivePerfStep`** to **host `perfStep` + lookahead** (default **64** sixteenths ≈ four 4/4 bars). Set **`submitDeadlinePerfStep`** to host step + slack (e.g. **48**) so the host drops the block if it arrives too late. Omit both and use **`asap: true`** for emergency edits. Alternatively put **`@ perf_step N`** as the first line of deck (parsed by host; same as `effectivePerfStep`). ### 2.4 Reconnect - Resend `join` with same `actorId`, `channelIds`, `skillIds`; gateway sends `replay`. - **Planned / not implemented in `services/agent-worker`:** loading state from a SQLite `messages` table on reconnect. The worker is **in-memory only** — its context buffer does not survive a restart. ## 3. SQLite + vector schema (Planned) > **Planned / not implemented in `services/agent-worker`.** The worker is currently **in-memory only**: it holds a rolling context buffer in process and does not open SQLite, embed chunks, or run vector/RAG retrieval. The schema below is a **forward-looking spec** for agent memory, kept here as the target design. Path: e.g. `~/.deckard-sessions/.sqlite` (agent-local) or shared if gateway persists. ### 3.1 Tables (SQL) ```sql CREATE TABLE messages ( id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT NOT NULL, actor_id TEXT NOT NULL, role TEXT NOT NULL, -- tpl | direct | system | tpl_out body TEXT NOT NULL, hub_seq INTEGER, ts INTEGER NOT NULL ); CREATE TABLE tpl_snapshots ( id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT NOT NULL, actor_id TEXT NOT NULL, tpl_text TEXT NOT NULL, ts INTEGER NOT NULL ); CREATE TABLE chunks ( id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT NOT NULL, source TEXT NOT NULL, -- tpl | direct | doc text TEXT NOT NULL, ts INTEGER NOT NULL ); -- Embedding storage: use sqlite-vec / sqlite-vss extension: -- CREATE VIRTUAL TABLE chunk_embeddings USING vec0( -- chunk_id INTEGER, -- embedding FLOAT[1536] -- dimension matches model -- ); CREATE TABLE agent_memory ( session_id TEXT PRIMARY KEY, summary TEXT, updated_ts INTEGER ); ``` ### 3.2 Vector index - **sqlite-vec** or **sqlite-vss**: store `chunk_id` + embedding; on ingest, chunk deck lines (~512 chars) and `direct` messages; query with same embedding model as ingest. - **Fallback**: no extension — keyword search on `chunks.text` only. ### 3.3 Embedding model - Fix **one** model per deployment (e.g. `text-embedding-3-small`, 1536-d). Document in agent `.env`. ## 4. Security notes - **Token** on WS URL for v1 stub auth. - **direct** channel: treat as user input; strip control chars; max length 8 KiB. - Rate-limit `deck.line` per `actorId` (e.g. 30/sec). ## 5. Reference implementation layout - **`services/gateway/`** — Tish: `npm run gateway` (or `tish run --features ws,process services/gateway/main.tish`). Listens on **ws://127.0.0.1:35987** (or `CODJ_HUB_PORT`). - **`services/agent-worker/`** — Tish agent: `npm run agent` (or `tish run --features ws,http,fs,process services/agent-worker/main.tish`). It calls its LLM (`callLLM` over the `http` feature) for **real** responses when `GRADIENT_MODEL_ACCESS_KEY` (or `MODEL_ACCESS_KEY`) is set, and falls back to a demo patch otherwise. Memory persistence and RAG ([§2.4](#24-reconnect), [§3](#3-sqlite--vector-schema-planned)) remain planned — the worker is in-memory only. See repository `package.json` / README for run commands. ## 6. Troubleshooting ### Agent gets "HTTP error: 200 OK" when connecting The WebSocket gateway responds with **101 Switching Protocols**, not 200. A **200 OK** response means another process is handling the port (e.g. an old Node server or another HTTP server). 1. **Start the gateway first**: `npm run gateway` — you should see `WebSocket server listening on ws://0.0.0.0:35987`. 2. **Check what is using the port**: with the gateway running, run: ```bash lsof -i :35987 ``` You should see a single process (the `tish` gateway). If you see another process (e.g. `node`), stop it so only the gateway is listening. 3. **Then start the agent**: `npm run agent`. --- # Stream protocol ## 1. JSON envelope (REST or queued ops) ```json { "v": 1, "sessionId": "uuid", "actorId": "string", "authorId": "string", "layer": "canonical | ui_overlay", "master": false, "op": "DECK_LINE | DECK_BLOCK | DIRECT | CONTROL", "target": { "channelId": "c0", "domain": "mix | steps | gen | notes | auto" }, "payload": "single deck line or object", "clientSeq": 42 } ``` Hub responds with `{ "ok": true, "seq": 17 }` or `{ "ok": false, "code": "SKILL_DENIED", "message": "..." }`. ## 2. Scheduling (perf step / lookahead) The host advances **`perfStep`** in **16th-note steps** (one sequencer column per step; 16 steps per 4/4 bar). Agents should target **future** steps, not “now”. | Field | On `deck.block` | Meaning | |-------|----------------|---------| | `effectivePerfStep` | optional | Apply merge when host `perfStep >= this`. If omitted (and no `@ perf_step` in deck), block is **ASAP**. | | `submitDeadlinePerfStep` | optional | If host `perfStep` **exceeds** this when the message is received, **drop** (late delivery). Omit = no deadline check. | | `asap` | optional | If `true`, ignore schedule and apply immediately when received. | Implemented in [src/codj/Schedule.tish](../src/codj/Schedule.tish) (`coDjHandleIncomingTplBlock` → `coDjFlushScheduledForStep`): late deliveries are dropped, `asap`/due blocks apply immediately, future blocks queue and flush when the playhead reaches `effectivePerfStep`. Queued blocks carry the sender's `skillIds` so skill-gating is re-checked at apply time. **Sequence lookahead**: one sequence = **64** sixteenth steps (four 4/4 bars). Remote lanes may schedule blocks up to **4 sequences** ahead: `effectivePerfStep = hostPerfStep + 256`. Use `submitDeadlinePerfStep` at least `hostPerfStep + 384` (or omit) so delivery is not dropped while the playhead catches up. **`direct`** from browser may include **`perfStep`** (host’s current step when the human sent the message) so the agent can compute `effectivePerfStep` and deadline relative to that instant. ## 3. WebSocket message types See [WS_AND_AGENTS.md](./WS_AND_AGENTS.md). Summary: | type | Purpose | |------|---------| | `join` / `joined` | Handshake | | `deck.line` | One completed deck line | | `deck.block` | Multiple lines atomically (+ optional schedule fields above) | | `deck.stream_chunk` | Live typing (agents) | | `direct` | Natural-language direction to a target actor | | `state.snapshot` | Resync | | `error` | Rejection | Every fanned-out message also carries **`skillIds`** — the gateway stamps the sender's declared `skillIds` (from `join`) onto each forwarded message ([services/gateway/main.tish](../services/gateway/main.tish)) so receivers can enforce skill-gating on apply. See [CO_DJ_SPACE.md](./CO_DJ_SPACE.md) and [src/codj/Skills.tish](../src/codj/Skills.tish). ## 4. Control ops (master) Payload for `type: control`, `op`: | op | payload | status | |----|---------|--------| | `clear_overlay` | `{ channelId? }` empty = all | **Implemented** (browser-side, [src/ui/CoDjPanel.tish](../src/ui/CoDjPanel.tish)) | | `take_track` | `{ channelId }` | *Planned* | | `release_track` | `{ channelId }` | *Planned* | | `set_master` | `{ authorId }` — host-only | *Planned* | | `master_overwrite` | `{ channelId, tplFragment }` | *Planned* | Only `clear_overlay` is handled today (it drops all overlays in scope on the receiving browser); the other ops are not yet wired. ## 5. Error codes Codes the gateway actually emits ([services/gateway/main.tish](../services/gateway/main.tish)): | code | Meaning | |------|---------| | `BAD_JSON` | Message was not valid JSON | | `JOIN_BAD` | `join` missing a valid `actorId` | | `JOIN_FIRST` | A non-`join` message arrived before joining | These are *Planned* (not yet emitted): | code | Meaning | |------|---------| | `SKILL_DENIED` | Lane lacks skill for op | | `LEASE_CONFLICT` | Track owned by another lane / master lock | | `PARSE_FAIL` | deck invalid | | `RATE_LIMIT` | Too many lines/sec | | `AUTH` | Bad token | Skill-gating is enforced today, but **not** via an error code: the receiver silently skips master-scope lines an actor's `skillIds` do not permit ([src/codj/Skills.tish](../src/codj/Skills.tish), [src/codj/Merge.tish](../src/codj/Merge.tish)). ## 6. Examples **Human line** ```json { "type": "deck.line", "actorId": "human-xyz", "line": " mix gain 0.9 pan 0", "authorId": "u1" } ``` **Direct to AI-A** (with perf step for scheduling) ```json { "type": "direct", "targetActorId": "agent-1", "text": "add euclidean 5/16 hi-hat pattern", "authorId": "u1", "perfStep": 120 } ``` **deck.block scheduled for step 200, must arrive by 180** ```json { "type": "deck.block", "actorId": "agent-1", "authorId": "agent", "lines": ["deck 1", "track H id h1 gen noise_burst", " steps euclid 5 16"], "effectivePerfStep": 200, "submitDeadlinePerfStep": 180 } ``` --- # DJ skills & gating Skills limit what a **lane** (especially AI) may emit. Hub or browser validates. | skill id | Allowed deck / ops | |----------|-------------------| | `add_track` | New `track ... id gen ` (id allocation may be server-assisted) | | `remove_track` | `remove_track ` — delete a channel. NOT master-scope (not in the denylist); per-track **ownership-gated** by `actorMayEditTrack` (a lane removes its own; a master removes any). Round-trips as absence (a deleted channel is simply not emitted). | | `adjust_instrument` | `gen `, `gen_block patch` (`osc`/`noise`/`filter`/`shaper`/`gain`/`conn`/`env`/`dur`), `osc`, `fm`, `noise`, `adsr`, `fx cutoff|reverb_send` on **owned** tracks | | `pattern_steps` | `steps`, `steps euclid`, `step_pitch` (incl. `bar `) on owned tracks | | `pattern_piano` | `note` lines on owned tracks | | `channel_mix` | `mix gain|pan|mute|solo|eq_lo|eq_mid|eq_hi`, `fx`, `voice`, `step_vel|prob|ratchet|nudge` locks on owned tracks | | `master_mixer` | the master-scope denylist — `bpm`, `tpl`, `auto`, `transpose`, `scale`, `swing`, `master_mix`, `actor_mix`, `session_scenes`, `session_slot`, `clip` — usually **human + master** only | | `transpose_track` | `transpose` in body | | `promote_song` | Append to full song doc | ## Agent presets & synthesis vocabulary The agent's **primary contribution is a deck preset** on its own lane, not a single hand-written track. Implemented in [`services/agent-worker/main.tish`](../services/agent-worker/main.tish): - On a styled/preset direct (`"play nebula pulse"`, `"give me something dark"`) or on **auto-jam** (when a peer presses Play), the agent picks one of the 15 deck sets in [`src/model/DeckSets.tish`](../src/model/DeckSets.tish) and streams it as one `deck.block`. - **Selection order:** `matchDeckSetId` (keyword/name match on the directive) → `llmPickDeckSetId` (LLM chooses an id from the catalog, only when `GRADIENT_MODEL_ACCESS_KEY` is set) → `rotateDeckSetId` (round-robin, for variety). - `prefixTrackIds` rewrites every `track … id ` → `_`, so the whole preset is **owned by this lane** and lands on **Deck B** (the human stays on Deck A; the crossfader blends them). - A **fine single-element** direct (`"add a hihat"`) still produces one LLM/demo track instead of a full preset (`looksLikeSingleElement`). **Synthesis vocabulary a preset (or the LLM) may emit** — all of these are non-master, so they apply on the receiver. The **full, current palette** the agent should use lives in **[DECK_AGENT_GRAMMAR.md](DECK_AGENT_GRAMMAR.md)** (kept in lockstep with the agent `SYSTEM_PROMPT`); in brief: - **33 fixed generators** by role — perc (`drumSynth`/`clap`/`cymbal`/`noise_burst`), bass (`acid303`/`sub808`/`reeseBass`), lead (`basic_osc`/`fm`/`aether`/`syncLead`/`obSync`/`laserSync`), keys (`tine`/`halo`/`bell`), pad (`pad`), strings (`guitar`/`arco`), chip (`chiptune`/`nes2a03`/`gameBoyDmg`/`c64sid`/`ym2612`/`sn76489`/`spc700`/`gbaDirectSound`), vocal (`formantVocal`/`ttsVocal`/`meSpeakVocal`/`syncChoir`), modular (`matrixFm`/`patch`). - **8 macro voices** — kick (`kick_edm`/`kick_deep`/`kick_distorted`), bass (`bass_reese_punch`/`bass_reese_sc`/`bass_wobble`; `bass_acid`/`bass_reese` are legacy, superseded in the picker by the `acid303`/`reeseBass` generators — see project memory *macro-generator-boundary-policy*). Catalog: [`src/model/MacroVoice.tish`](../src/model/MacroVoice.tish) / [`BuiltinMacros.tish`](../src/generators/BuiltinMacros.tish). - **Modular voices**: `gen_block patch` (`osc`/`noise`/`string`/`filter`/`shaper`/`gain`/`conn`/`env`/`dur`) and `gen_block matrix_fm`. - **Per-track**: `steps`/`steps euclid`/`step_pitch` (`bar `), `note` (with `p`/`r`/`n` locks), `step_vel|prob|ratchet|nudge` lock lanes, `mix … eq_*`, `fx cutoff|res|drive|reverb_send|lfo_rate|lfo_depth|filter_type`, `voice octave|chord|arp|arprate|inversion|strum`, `adsr`, `* `, `loops`. The agent's declared skills are `add_track`, `adjust_instrument`, `pattern_steps`, `pattern_piano`, `channel_mix` (no `master_mixer`) — exactly the set a preset needs, since presets never emit master-scope lines. ## Lane matrix (default) Same skill ids for every row; **human** and **ai-a** columns show who may use each skill today. Toggling AI access later = flip flags in code, not new skill types. | skill id | human | ai-a / ai-b | |----------|-------|-------------|-------------------| | `add_track` | yes | yes | | `adjust_instrument` | yes | yes | | `pattern_steps` | yes | yes | | `pattern_piano` | yes | yes | | `channel_mix` | yes | yes | | `master_mixer` | yes | no (`bpm`, `auto`) | | `transpose_track` | no* | no | | `promote_song` | no* | no | | `remove_track` | yes (UI ×) | no | \*Same registry entry shape for future use. Actors with **`master_mixer`** may stream full project shape including `bpm` / `tpl` / `auto`. Agents (without `master_mixer`) must not emit any master-scope head — `bpm`, `tpl`, top-level `auto`, `transpose`, **`scale`**, **`swing`**, `master_mix`, `actor_mix`, `session_scenes`, `session_slot`, `clip` (enforced in [`coDjLineAllowedForSkills`](../src/codj/Skills.tish), with [`skillAllowsLine`](../src/codj/Skills.tish) as a back-compat facade). `scale`/`swing` are master-scope because they re-key / re-shuffle the **whole** session for every player. **Implementation:** [`src/codj/Skills.tish`](../src/codj/Skills.tish) — `coDjLineAllowedForSkills`, `skillIdsAllowMaster`, `hasSkill`, `actorHasSkill`, `skillAllowsLine`. ## Default bundles (informal) - **AI lane (typical)**: `adjust_instrument`, `pattern_steps`, `channel_mix`, `pattern_piano` on leased tracks only. - **Human**: all implemented rows + master. - **Viewer**: none (read-only). ## Enforcement Skill-gating **is now enforced**, on the **receiver** side: - The gateway stamps the sender's declared `skillIds` onto every fanned-out message (`out.skillIds = conn.skillIds` in [`services/gateway/main.tish`](../services/gateway/main.tish)). - The browser ([`src/ui/CoDjPanel.tish`](../src/ui/CoDjPanel.tish)) threads `msg.skillIds` into [`applyCoDjTplSource`](../src/codj/Merge.tish) and `coDjHandleIncomingTplBlock`. - [`applyCoDjTplSource`](../src/codj/Merge.tish) gates master-scope lines (`bpm`, top-level `auto`, session clips, scenes) via [`skillIdsAllowMaster`](../src/codj/Skills.tish), falling back to the legacy `actorId.indexOf("human")` check **only** when `skillIds` are absent. - The incremental line decoder ([`src/deckfile/Stream.tish`](../src/deckfile/Stream.tish)) gates **every** streamed line via [`coDjLineAllowedForSkills`](../src/codj/Skills.tish). Enforcement = the **receiver silently skips disallowed lines** (master-scope lines without `master_mixer`). There is **no `SKILL_DENIED` error code** yet — see [WS_AND_AGENTS.md §1.3](./WS_AND_AGENTS.md) for the error codes actually emitted. ## Denied examples - AI emits `bpm 200` without `master_mixer` skill → receiver **silently skips** the line (planned: `SKILL_DENIED`). - AI edits `c0` when `ownerActorId` is another actor and not leased → skipped by `actorMayEditTrack` (planned error: `LEASE_CONFLICT`). ## Implementation - `src/codj/Skills.tish` — `coDjLineAllowedForSkills`, `skillIdsAllowMaster`, `hasSkill`, `actorHasSkill`, `skillAllowsLine`. - `src/codj/Merge.tish` — `applyCoDjTplSource` gates master-scope lines; `actorMayEditTrack` enforces per-track ownership / master-lock. - `src/deckfile/Stream.tish` — incremental line decoder gates every streamed line. - Hub duplicate check optional. --- # Roles & skill files This directory is the **knowledge library the co-DJ agents read at boot** (`services/agent-worker/main.tish`, via `readFile` from `tish:fs`). Each `.md` is one *musical skill* — the agent composes its system prompt from a **role file** + a **persona-selected subset of capability skills** + the shared deck grammar (`docs/DECK_AGENT_GRAMMAR.md`). Editing a skill changes the agent on its next boot — no rebuild. ## Roles - **`_role-host.md`** — the host agent (`--role host`, joins with `master_mixer`). Owns the **mix + master + cohesion**: balances levels/EQ/pan across every lane, sets tempo/key/swing, arranges, and *responds* to what clients add (clients steer style/genre by CONTENT; the host adapts the master/mix to fit). - **`_role-client.md`** — a client agent (`--role client`, follower, no master). Owns **production**: adds beats/bass/melody/texture, fills gaps, pushes the arrangement, and *steers* genre/energy by what it ADDS (a halftime break, a key-implying line, a genre pivot) — never by touching master. ## Capability skills (composable persona library) `beats` · `bass` · `melody` · `harmony-chords` · `texture-atmos` · `vocal` · `groove-humanize` · `mixing` · `arrangement` · `style-genre`. At boot each agent assembles a **unique persona** by selecting + weighting a subset of these (the LLM picks the blend from the library + the live session; falls back to a seeded blend keyed on the actorId when no inference key is set). ## File format Each skill file is YAML frontmatter + a markdown body: ``` --- id: beats role: client # host | client | any gates: pattern_steps, channel_mix # the Skills.tish gating skills this maps to (enforcement layer) weight: 1.0 # default persona-selection weight (higher = more likely chosen) --- # Beats — drums & percussion ## Intent ## How to think - ## deck it emits - ## Examples \`\`\`tpl track Hat id _hat gen drumSynth steps . . x . . . x . . . x . . . x . \`\`\` ``` Rules: every track id is prefixed `_` (the worker substitutes the actorId). Examples must be **valid co-DJ-lane deck** (non-master-scope unless the file's `role: host`). The body is injected verbatim into the prompt, so keep it tight and imperative. ``` --- # Architecture > A token-streamed, live-coding DAW. Agents and humans **co-DJ** by streaming a small > line-oriented language (**deck**) that the app **decodes into live-synthesised stems** — > never WAV/audio files. Think *MIDI × live-coding, designed for LLM token streams, with a > traditional DAW built on top.* This document is the map of the whole project: where it came from, the core idea, the end-to-end signal path, and how the subsystems fit together. For the language reference see **`@spacedevin/deck`** ([grammar in `node_modules`](../node_modules/@spacedevin/deck/docs/DECK_GRAMMAR.md)); Deckard overlay (UI / ownership) is [DECK_GRAMMAR.md](DECK_GRAMMAR.md). Wire protocol: [WS_AND_AGENTS.md](WS_AND_AGENTS.md) and [STREAM_PROTOCOL.md](STREAM_PROTOCOL.md). --- ## 1. Product thesis — where this comes from A normal collaborative DAW moves **audio** between participants (stems, WAVs, OT on a timeline). That is heavy, hard for an LLM to author, and impossible to "improvise" token by token. Deckard inverts it. The shared artifact is **text** — a compact patch language (**deck**, the *deck*) that describes instruments, patterns, mixing and automation. Every participant — human or AI agent — **streams deck lines** as the music evolves. The app holds the synthesiser, so a streamed line like `steps euclid 5 16` *becomes sound* in the browser the moment it arrives. Nothing is pre-rendered; **all audio is generated in-app from the token stream.** That makes three things possible at once: 1. **LLMs are first-class performers.** A model emits deck the same way a human edits a pattern. "Decode a song into live stems and DJ from there" is literally: stream deck → synth graph → sound. 2. **Live coding meets a DAW.** The familiar surfaces (channel rack, piano roll, mixer, session/scene launcher) sit *on top of* the language; the language is the source of truth, the UI is a view. 3. **Many performers, one groove.** Multiple agents and humans share a session, each owning lanes and tracks, scheduling their contributions to **future beats** so they lock to the bar. The project has been through several iterations (a Node.js service tier was rewritten in Tish; the project schema went v1 → v2 with a per-channel *generator plugin* model; the streaming/co-DJ layer was built out incrementally). This document reflects the **current** state after an architecture-cleanup pass. --- ## 2. The stack Everything is written in **[Tish](https://github.com/tishlang/tish)**, a language that compiles to JavaScript. The UI uses **Lattish** ([LATTISH.md](LATTISH.md)), a small React-like layer (`useState/useMemo/useRef/useEffect`, `createRoot`, and JSX that lowers to `h()`calls). The browser app is built with `tish build --target js src/main.tish -o dist/bundle.js`; the services run under the Tish interpreter with the `ws` / `http` / `process` features. | Concern | Where | |---------|-------| | `.deck` language (parse / format / registries / highlight) | `@spacedevin/deck` | | Apply / emit / stream + graph dialects | `src/deckfile/` | | Generator id + macro registration | `src/generators/DeckIds.tish`, `BuiltinMacros.tish` | | Data model (project = single source of truth) | `src/model/` | | Synthesis & scheduling (the "stems") | `src/audio/`, `src/schedule/`, `src/generators/` | | Co-DJ collaboration (lanes, merge, skills, scheduling) | `src/codj/` | | DAW UI (Lattish/JSX) | `src/ui/` | | WebSocket gateway, agent worker, demo bot | `services/` | --- ## 3. The core data model — project is the source of truth A **project** (`src/model/Project.tish`) is a plain object that everything reads and writes: ``` project = { version: 2, bpm, transportMainDeck: "live"|"local", channels: [ channel… ], // the instruments / tracks instrumentPresets: [ … ], // named patches (incl. 13 factory matrix-FM presets) automation: { masterGain[], pitchBend[] }, paramAutomations: [ … ], // per-channel generator-param curves mixerAutomations: [ … ], // track / actor-bus / master mixer curves session: { sceneCount, slots[][] }, // Session-view scene grid masterMixer, actorMixer, // mixer state for master + per-actor buses coDjMeta: { tracks: { : { ownerActorId, authorId, masterLock, lastTouchedPerfStep } } }, coDjOverlays: [ … ] // temporary UI overlays (e.g. MIDI gain) } ``` Each **channel** is an FL-style *generator slot* (see [FL_STUDIO_GENERATORS.md](FL_STUDIO_GENERATORS.md)): routing (`gain/pan/mute/solo` + 3-band `eqLo/eqMid/eqHi`), a `generatorId` selecting one instrument plugin, and a `generatorParams` object whose **shape depends on the generator** (this is where ADSR lives — *not* on the channel root). Pattern data is either a 16-step row (`steps`) or piano-roll `pianoNotes`; `stepPitch` is the base MIDI note for step hits. The model is the contract for agents: edit `Project.tish` / the schema ([schema/project-v2.json](schema/project-v2.json)) and the safe mutators in `src/model/Edits.tish` rather than reaching into the UI. The UI channel-strip controls now route through those `Edits.tish` setters, so UI edits and programmatic/agent edits share one mutation path. --- ## 4. deck — the streaming token language (the centerpiece) deck is line-oriented and streamable. Canonical grammar: `@spacedevin/deck`. Shape: ``` deck 1 bpm 118 track Kick id c0 gen noise_burst # one channel = one generator mix gain 0.9 pan 0 eq_lo 0 eq_mid 0 eq_hi 0 step_pitch 36 noise attack 0.002 decay 0.12 tone 0.15 pitch_follow 0.35 steps x . . . x . . . x . . . x . . . track Bass id c3 gen fm fm ratio 1 mod_index 6 carrier sine mod sine adsr a 0.008 d 0.12 s 0.35 r 0.15 note 48 0 0.5 v 90 master_mix eq_lo 0 eq_mid 0 eq_hi 0 # static mixer lines actor_mix local gain 1 eq_lo 0 eq_mid 0 eq_hi 0 auto master_gain # automation curves 0 1.0 16 0.8 ``` It also round-trips the **Session view** (`session_scenes`, `session_slot`, and `clip … bars …` blocks) and heavy generators (`gen_block matrix_fm … end gen_block`, see [DECK_EXTENSION.md](DECK_EXTENSION.md)). Deck routing (LIVE/CUE) is JSON/UI state and is intentionally **not** part of deck. ### Two decode paths — block and stream | Path | Unit | Module | Use | |------|------|--------|-----| | **Atomic** | whole program / block | `@spacedevin/deck` `parseProgram` → `Apply.tish` (`applyTplSource`) | Editor *Apply*, JSON import, `deck.block` over the wire | | **Incremental** | one line at a time | `src/deckfile/Stream.tish` (`tplLineStreamPush`) | `deck.line` over the wire — progressive decode | The atomic path (`parseProgram` → `applyParsed`) merges a complete program into the project by channel id. The **incremental path** is what makes "stream a song into live stems" literal: a non-indented statement (`track…`, `auto…`, `clip…`) opens a block, indented lines extend it, and the growing block is re-applied (idempotently) on every line — so a remote actor's track **sounds the instant its `track …` header arrives**, then the pattern fills in as `steps …` streams. Both paths share the same ownership/skill enforcement via `applyCoDjTplSource`. `src/deckfile/Emit.tish` does the reverse — `project → deck` — for the editor mirror, JSON↔deck, and the "what you send on Play" preview. Package parse + host apply/emit are a verified round-trip (see `test/smoke.tish`). --- ## 5. From tokens to sound — the synthesis path The audio engine (`src/audio/Engine.tish`) builds a Web Audio graph with a three-tier mixer: ``` generator voice → [channel bus: lowpass → 3-band EQ → trim → pan] → [actor bus: EQ → trim] (one bus per lane / actor) → [master: EQ → masterGain] → analyser → destination ``` The transport (`src/ui/App.tish` playback loop → `src/audio/Playback.tish:transportTick`) advances a **16th-note `perfStep`**. Each tick: commit any queued Session scene on the bar line, prune stale agent tracks, flush co-DJ blocks scheduled for this step, interpolate all automation at the current beat, and fire the due step/notes. The loop is **self-scheduling** and reads `project.bpm` every tick, so tempo changes take effect live. Generators are modular plugins ([GENERATORS.md](GENERATORS.md)) dispatched by `generatorId`: | `generatorId` | deck `gen` | sound | |---------------|-----------|-------| | `noiseBurst` | `noise_burst` | filtered-noise percussion (kick/snare/hat) | | `fmTone` | `fm` | 2-operator FM + ADSR | | `basicOsc` | `basic_osc` | single oscillator + ADSR | | `matrixFm` | `matrix_fm` | Sytrus-style multi-operator graph via `gen_block` | To add an instrument: drop a module in `src/generators/`, register it, branch in `Dispatch.tish`. This is the only place sound is defined — there is no separate hand-written JS engine. --- ## 6. Co-DJ — agents and humans performing together The collaboration layer (`src/codj/`) is the differentiator. A **session** is a room on the gateway; **actors** (browsers `human-*`, agents `agent-*`/`actor-*`) join with an `actorId` and a declared `skillIds` set. - **Ownership / merge** (`Merge.tish`, `CoDjMeta.tish`): each channel id has an owner lane; an actor may only edit tracks it owns (or new tracks), and never a **master-locked** track. A per-track **LOCK/OPEN** toggle in the channel rack sets `masterLock`. - **Skills** (`Skills.tish`): an actor's `skillIds` gate which lines it may emit. Master-scope lines (`bpm`, `auto`, `transpose`, `master_mix`, `actor_mix`, `session_*`, `clip`) require the `master_mixer` skill. The gateway stamps each sender's `skillIds` onto fan-out; the receiver enforces them on apply (disallowed lines are silently skipped). See [DJ_SKILLS.md](DJ_SKILLS.md). - **Scheduling** (`Schedule.tish`): blocks target a **future** `perfStep` (`effectivePerfStep`, with `submitDeadlinePerfStep` / `asap`) so remote edits land on the bar instead of "now". The transport flushes them at the right step. - **Overlays** (`Overlay.tish`): temporary, non-committing changes (e.g. a Web-MIDI note → channel gain) applied on the read path until cleared or promoted. - **Pruning** (`Prune.tish`): agent-owned tracks untouched for a couple of sequences are removed, so an improvising agent doesn't accumulate clutter. ### The end-to-end happy path ``` agent worker gateway browser (host) ───────────── ─────── ────────────── join (skillIds) ───────────────▶ room/presence ◀───────────── join (Connect) Play → stream project as deck.line buffer peer deck.line ◀────────── fanout (+skillIds) ◀──────────── deck.line per line (throttled) debounce → snapshot buffer → callLLM → deck lines deck.stream_chunk (live tokens) ──▶ fanout ──────────────────────▶ "Hub → you" preview deck.block @ effectivePerfStep ───▶ fanout ──────────────────────▶ schedule → apply on that step merge (ownership/skills) → synth → sound ``` Humans stream **`deck.line`** (decoded incrementally); agents commit **`deck.block`** scheduled to a future bar. The worker only collapses the rolling stream into a single prompt **at the LLM boundary** — everything on the wire stays a stream. With no `GRADIENT_MODEL_ACCESS_KEY`, the worker falls back to a built-in demo patch so the loop is exercisable offline. --- ## 7. The DAW UI `src/ui/App.tish` is the orchestrator. It holds the project in `useState` and a mutable **`DeckardRuntime`** bag (`DeckardRuntime.tish`) in a `useRef` for transport/Co-DJ/editor/WS state (no `window.__*` globals). Workspace tabs: - **Sequencer** — channel rack + step grid (`ChannelRack.tish`), piano roll (`PianoRoll.tish`, canvas), and the **Co-DJ** panel (`CoDjPanel.tish`: connect, stream previews, activity log). - **Session** — Ableton-style scene launcher (`SessionView.tish`, model in `Session.tish`). - **Patch** / **Instrument** — per-track generator editor (`InstrumentPanel.tish`, `GeneratorParams.tish`, `MatrixFmPanel.tish` for the matrix-FM graph). - Always-docked **deck editor** (`CodeDebugView.tish`) — Apply/Sync, step highlight, the emit-mirror of the project. The mixer (`Mixer.tish`) renders the track → actor → master tiers; a master **scope** (`Scope.tish`) draws the analyser. The rule (`.cursor/rules/tish-midi.mdc`): **business logic lives in model/schedule/audio; UI files are layout + wiring.** --- ## 8. Services | Service | File | Role | |---------|------|------| | **Gateway** | `services/gateway/main.tish` | One room per `sessionId`; JSON fan-out; per-actor `seq`; presence; stamps each sender's `skillIds` onto fan-out. `ws://127.0.0.1:35987` (or `CODJ_HUB_PORT`). | | **Agent worker** | `services/agent-worker/main.tish` | Joins as an actor; buffers the peer stream; on debounce snapshots it into one prompt, calls the LLM, and streams real deck out (`deck.stream_chunk` → `deck.block`); demo fallback without a key. | | **Token-stream demo** | `services/token-stream-demo/main.tish` | A bot that streams rotating patches (kick / hats / bass) to prove the wire path end-to-end. | Run order: **gateway → worker → browser** (`npm run gateway`, `npm run agent`, `npm run serve`). See the [README](../README.md) quick-start. --- ## 9. Maturity map | Area | State | |------|-------| | Project model, schema, v1→v2 migration | **Solid** | | deck parse / apply / emit round-trip (incl. step_pitch, mixer lines, sessions, gen_block) | **Solid** | | Web Audio synthesis, 3-tier mixer, automation, deck routing | **Solid** | | Session / scene launcher (arm / queue / commit) | **Solid** | | Co-DJ gateway, ownership/merge, perf-step scheduling, overlays, pruning | **Solid** | | Incremental `deck.line` decode; skill-gating enforcement | **Wired** | | Agent worker LLM call + real outbound streaming | **Wired** (needs `GRADIENT_MODEL_ACCESS_KEY`) | | matrix_fm generator + graph editor | **Working** | | **Planned / not yet built** | SQLite + vector/RAG agent memory; provider-side SSE token streaming (currently the finished reply is streamed char-by-char); control ops `take_track`/`release_track`/`set_master`/`master_overwrite`; named MIDI controller profiles (only note%8 → gain overlay exists); inline `@lane` author tags; Session-view scene authoring (add/remove/duplicate/clear). | --- ## 10. File index (start here) - **Model:** [Project.tish](../src/model/Project.tish), [Session.tish](../src/model/Session.tish), [Edits.tish](../src/model/Edits.tish), [Migrate.tish](../src/model/Migrate.tish), [MixerRouting.tish](../src/model/MixerRouting.tish), [DeckRouting.tish](../src/model/DeckRouting.tish) - **deck language:** [`@spacedevin/deck`](../node_modules/@spacedevin/deck/docs/DECK_GRAMMAR.md) - **deck host:** [Apply.tish](../src/deckfile/Apply.tish), [Emit.tish](../src/deckfile/Emit.tish), [Stream.tish](../src/deckfile/Stream.tish), [PatchGraph.tish](../src/deckfile/PatchGraph.tish), [MatrixFmGraph.tish](../src/deckfile/MatrixFmGraph.tish), [DeckIds.tish](../src/generators/DeckIds.tish) - **Audio:** [Engine.tish](../src/audio/Engine.tish), [Playback.tish](../src/audio/Playback.tish), [schedule/Engine.tish](../src/schedule/Engine.tish), [generators/](../src/generators/) - **Co-DJ:** [Merge.tish](../src/codj/Merge.tish), [Skills.tish](../src/codj/Skills.tish), [Schedule.tish](../src/codj/Schedule.tish), [Overlay.tish](../src/codj/Overlay.tish), [CoDjMeta.tish](../src/codj/CoDjMeta.tish), [Prune.tish](../src/codj/Prune.tish) - **UI:** [App.tish](../src/ui/App.tish) (orchestrator), [CoDjPanel.tish](../src/ui/CoDjPanel.tish) - **Services:** [gateway](../services/gateway/main.tish), [agent-worker](../services/agent-worker/main.tish) - **Tests:** [test/smoke.tish](../test/smoke.tish) — `npm test` (round-trip, incremental decode, skill-gating, permissions) --- # Generators > **Catalog (current):** there are **33 fixed generators** + **8 macro voices**. The authoritative id list is > `generatorCatalog()` in [`src/generators/Registry.tish`](../src/generators/Registry.tish) and `macroCatalog()` > in [`src/model/MacroVoice.tish`](../src/model/MacroVoice.tish); the picker grouping is `VOICE_GROUPS` in > [`src/ui/InstrumentStack.tish`](../src/ui/InstrumentStack.tish). For the agent-facing list by role + the > `gen ` deck aliases, see [`DECK_AGENT_GRAMMAR.md`](DECK_AGENT_GRAMMAR.md). This page is the **how-to-add** guide. ## 1. Pick an `id` Stable string, e.g. `mySynth`. Used in project JSON and dispatch. ## 2. Register in [`src/generators/Registry.tish`](../src/generators/Registry.tish) - Add `{ id, label, description }` to `generatorCatalog()`. - Add defaults in `defaultParamsForGeneratorId()`. ## 3. Implement audio in `src/generators/YourGenerator.tish` Export `playYourGenerator(ctx, bus, t, midi, vel, durSec, ch, bendSemis)`: - Build a short-lived Web Audio subgraph. - Connect the **last node** to `bus.input` (channel filter → gain → pan → master). - Read patch and envelope from `ch.generatorParams` if applicable. The ADSR lives in `generatorParams`, not on the channel root — do `let p = ch.generatorParams` then read `p.attack` / `p.decay` / `p.sustain` / `p.release` (see [`src/generators/BasicOsc.tish`](../src/generators/BasicOsc.tish)). Use [`midiToHz`](../src/schedule/Engine.tish) for pitched notes. ## 4. Dispatch in [`src/generators/Dispatch.tish`](../src/generators/Dispatch.tish) Call your `play...` when `ch.generatorId === "yourId"`. Unknown ids fall back to `basicOsc`. ## 5. UI - **Generator picker + per-voice designers** live in [`src/ui/InstrumentStack.tish`](../src/ui/InstrumentStack.tish) — the `VOICE_GROUPS` dropdown + the inline designer/raw cards for the **selected track**. - Shared knob/select widgets are in [`src/ui/InstrumentKit.tish`](../src/ui/InstrumentKit.tish) and [`src/ui/GeneratorParams.tish`](../src/ui/GeneratorParams.tish). The channel rack only shows the **instrument label badge**, not parameters. ## 6. Schema Extend [`docs/schema/project-v2.json`](schema/project-v2.json) with a `generatorParams` shape for your id (optional JSON Schema oneOf). ## Matrix FM (`matrixFm`) Sytrus-style multi-operator graph: define the patch in deck with `track … gen matrix_fm` and an indented `gen_block matrix_fm` … `end gen_block`. Parsed graph lives in `channel.generatorSpec.graph`; see [`docs/DECK_GRAMMAR.md`](DECK_GRAMMAR.md) and [`docs/DECK_EXTENSION.md`](DECK_EXTENSION.md). ### Factory matrix FM presets [`src/model/MatrixFmPresets.tish`](../src/model/MatrixFmPresets.tish) ships **25** factory `matrixFm` patches that are auto-loaded into `project.instrumentPresets` for every new project ([`src/model/Project.tish`](../src/model/Project.tish) `emptyProjectShell`). They cover bass (`mx_dub_growl`, `mx_reese_wide`, `mx_reese_grind_rm`, `mx_dx_knock_bass`, `mx_deep_house_bass`, `mx_sub_layer`, `mx_trap_808`, `mx_future_bass_wobble`), leads/plucks (`mx_pluck_neon`, `mx_chrome_pluck_lead`, `mx_bright_lead`, `mx_supersaw_stack`, `mx_psy_squelch`), keys/organ (`mx_dx_mark_v_suitcase`, `mx_wurli_barky_reed`, `mx_drawbar_cathedral_organ`), bells/mallets (`mx_cathedral_tubular_bells`, `mx_bloom_gong_gamelan`, `mx_vibe_marimba_mallet`, `mx_metallic_clank`), pads/choirs (`mx_airy_pad`, `mx_choir_of_the_void`, `mx_aurora_drift`, `mx_fanfare_swell`), and `mx_stab_chord`. The file is the source of truth — don't hand-maintain this list elsewhere. --- # Author tagging ## Sidecar model (v1) Stored in project as `coDjMeta` (parallel to channels): ```json { "coDjMeta": { "tracks": { "c0": { "ownerActorId": "agent-1", "authorId": "agent-a", "masterLock": false, "lastTouchedPerfStep": 0 }, "c3": { "ownerActorId": "human-xyz", "authorId": "u1", "masterLock": true, "lastTouchedPerfStep": 240 } } } } ``` `ensureCoDjMeta` in `src/codj/CoDjMeta.tish` only ever builds the `tracks` object, keyed by `channelId`. A `lanes` object is _not_ created (Planned). - **`ownerActorId`**: which lane’s deck is authoritative for this channel’s body (after merge). - **`masterLock`**: if true, only master/human actor may change this track until released. Now has a UI toggle: a per-track **LOCK/OPEN** button in the channel rack (`src/ui/ChannelRack.tish`) calls `setMasterLock`. - **`authorId`**: last writer or creator. - **`lastTouchedPerfStep`**: performance step at which the track was last written (`setTrackTouched`). ## Merge precedence 1. If `masterLock` and actor is not master → ignore non-master deck for that `channelId` from other actors (or require `control` overwrite). 2. Else use **ownerActorId**’s latest deck for that track id. 3. New track from actor X → `ownerActorId = X`, `authorId = emitter`. ## Planned: inline deck Optional parser extension (still unimplemented): ``` track Kick id c0 gen noise_burst @lane ai-a ``` The inline `@lane ai-a` tag form is not yet parsed; the sidecar (`coDjMeta.tracks`) remains the source of truth. ## UI - Channel rack badge: color by `ownerActorId`. - Mixer strip: tint if overlay active. --- # Token stream demo End-to-end flow: 1. **`services/token-stream-demo`** (Tish) connects to the gateway as actor **`stream-demo`**. 2. For each cycle it sends **`deck.stream_chunk`** (single characters) so the Co-DJ panel shows a live “token” preview. 3. Then it sends **`deck.block`** with **`asap: true`** so the patch applies immediately (no perf-step queue). 4. You **Play** in the app to hear **WsKick** / **WsHat** / **WsBass** on channel ids **`wsdemo_k1`** / **`wsdemo_h1`** / **`wsdemo_b1`** (the demo cycles rotating kick, hat, and bass variations). ## Run ```bash # Terminal A npm run gateway # from repo root (Tish gateway) # Terminal B npm run token-demo # from repo root (runs Tish stream demo) ``` Open the app (e.g. `npm run serve` → http://localhost:3456), go to **Co-DJ**, **Connect** (session **default**), click **Play**. ## Human in the loop - Open **Song** and edit steps / mix on **`wsdemo_k1`** or **`wsdemo_h1`**, then **Apply** (human lane). That sets **human ownership** on those tracks. - After that, the stream service **no longer overwrites** those tracks (lane rules in `Merge.tish`), but it still updates any tracks it still owns. - Add your own tracks in Song as usual; they stay yours. ## Env | Variable | Default | Meaning | |----------|---------|---------| | `CODJ_HUB` | `ws://127.0.0.1:35987` | Gateway base URL | | `CODJ_SESSION` | `default` | Session id (must match browser) | | `STREAM_INTERVAL_MS` | `5000` | Interval between deck updates (ms) | | `TOKEN_CHUNK_MS` | `15` | Delay between stream chunks (ms) |