VT100-to-PWA Protocol

How TUICommander transforms raw PTY output into structured, styled terminal content for the mobile Progressive Web App.

Architecture Overview

PTY (raw bytes)
  │
  ▼
VtLogBuffer (TerminalGrid — the patched alacritty_terminal fork)
  ├── scrollback → LogLine[] (styled spans)
  ├── screen_rows → String[] (plain text)
  └── prompt_input_text → String (user-typed, no ghost text)
        │
        ▼
  HTTP / WebSocket handlers
  ├── trim_screen_chrome() → remove agent UI footer
  ├── is_separator_line() → detect decorated separators
  └── SessionState → accumulated parsed events
        │
        ▼
  Frontend (SolidJS)
  ├── OutputView → log + screen rendering, auto-scroll
  ├── CommandInput → bidirectional PTY ↔ textarea sync
  └── SlashMenuOverlay → arrow navigation, prefill

1. VtLogBuffer — VT Parsing Engine

File: src-tauri/crates/tuic-terminal/src/vt_log.rs (buffer) · src-tauri/crates/tuic-terminal/src/terminal_grid.rs (grid)

There is no vt100 crate. VtLogBuffer wraps a TerminalGrid, TUIC’s patched alacritty_terminal fork — see alacritty-integration.md. Every parsing, changed-row and scrollback query below is delegated to that grid.

Configuration

#![allow(unused)]
fn main() {
VtLogBuffer::new(rows, cols, capacity)
// Grid created with: TerminalGrid::new(rows, cols, GRID_SCROLLBACK)
}

process(data: &[u8]) -> Vec<ChangedRow> — Core Pipeline

Called for every PTY read chunk. Steps:

1. Detect a screen switch (primary ↔ alternate). On a switch, grid.clear_prev_rows() forces the next diff to report every non-empty row.
2. Feed bytes to the grid — grid.process(data) parses and returns the changed rows itself; VtLogBuffer keeps no prev_rows of its own.
3. Extract the scrollback delta, primary screen only:

   total_sb  = self.grid.scrollback_count();
   delta     = total_sb.saturating_sub(self.scrollback_read);
   new_lines = self.grid.read_scrollback_log_lines(delta);
   

4. Mark agent chrome on the new lines (mark_agent_chrome).
5. Stamp cols with the current PTY width and push to the log ring buffer.
6. Return the changed rows (for the output parser).

Changed rows are reported for both screens, so parsers still see status lines and intent tokens from alternate-screen agents (Claude Code / Ink). Log capture is skipped on the alternate screen, while suppress_capture is set (a side panel halved the width), and for inline TUIs (mouse reporting on the primary screen, e.g. grok --no-alt-screen) — but scrollback_read is still advanced so the cursor stays in sync.

Scrollback Extraction

The grid accumulates history when lines scroll off the top of the primary screen (a real line feed at the bottom), but NOT when cursor-based TUI redraws repaint in place.

Key invariant: scrollback_read is monotonically increasing. Each process() call reads only the delta since the last call. resize() re-syncs it to grid.primary_scrollback_count() — the primary coordinate space — so a resize taken while an alt-screen app is active cannot strand the durable-log cursor.

TerminalGrid::extract_log_line(&self, line: Line) -> LogLine — Styled Cell Extraction

Converts one grid line into a LogLine with colored spans:

#![allow(unused)]
fn main() {
struct LogLine {
    spans: Vec<LogSpan>,
    cols: u16,     // PTY width at capture time — lets the frontend spot narrow-captured lines
    chrome: bool,  // Agent UI chrome; never serialized, readers skip it
}
struct LogSpan {
    text: String,
    fg: Option<LogColor>,   // Idx(u8) or Rgb(r,g,b)
    bg: Option<LogColor>,
    bold: bool,
    italic: bool,
    underline: bool,
}
}

Algorithm:

• Iterate columns left-to-right
• Skip wide-char continuation cells
• Group consecutive cells with identical attributes into one span
• Flush span when attributes change
• Trim trailing empty/whitespace-only spans with default styling

mark_agent_chrome flags lines from find_scrollback_chrome_cutoff down as chrome: true; it does not delete them. A misclassification therefore hides text rather than destroying it.

screen_rows() — Current Screen Content

Returns the visible terminal content as plain text strings, from the grid’s cached snapshot (grid.screen_text_rows()). screen_rows_ref() borrows the same snapshot so a caller already holding the lock avoids the clone.

prompt_input_text() — User Input Extraction

Extracts what the user has typed at the prompt, excluding ghost/suggestion text.

Algorithm:

1. Scan rows bottom-to-top for prompt character (❯, >, > )
2. Walk cells left-to-right after the prompt char
3. Collect cell contents while !cell.dim()
4. Stop at first dim cell — that’s ghost/autocomplete text
5. Return trimmed result

This is critical: Claude Code shows inline suggestions in dim text (e.g., ❯ /wiz:status where /wiz:status is grey). Without the dim check, the suggestion text would appear in the PWA textarea.

mark_agent_chrome() — Log Line Filtering

Flags agent UI chrome (prompt box, status bar, separator) on captured log lines.

• Uses find_scrollback_chrome_cutoff() — anchors only on a bare prompt row (is_agent_prompt_row), never on a standalone separator
• Extends cutoff upward past separator, empty and task-list lines
• Sets LogLine::chrome; the line stays in the buffer and lines_since_owned() skips it. Nothing is deleted from history, so a misclassification hides text rather than destroying it — the failure mode that made the mobile log show paragraphs starting mid-sentence.

2. HTTP/WS Session Handlers

File: src-tauri/src/mcp_http/session.rs

GET /sessions/:id/output?format=log

Initial data fetch before WebSocket connects.

Response:

{
  "lines": [LogLine, ...],
  "total_lines": 1234,
  "offset": 1134,
  "screen": ["row1", "row2", ...],
  "input_line": "user typed text"
}

• total_lines serves as the offset cursor for WS catch-up
• offset is the absolute start of the returned window. Clients must use it instead of total_lines - lines.length: chrome lines occupy offset slots without being returned, so the subtraction lands inside the window already held and replays those lines on scroll-up
• screen has chrome trimmed via trim_screen_chrome()
• input_line from prompt_input_text() (dim text excluded). Only agent prompts (❯, ›, >) are recognized — a plain shell prompt ($/#/%/➜) yields null, so the textarea gets no PTY-driven reconciliation for shells.

WS /sessions/:id/stream?format=log&offset=N

Real-time bidirectional stream. Two concurrent tasks:

Server → Client (polling task)

Runs in a tokio::select! loop with two branches:

Branch 1: 200ms timer — polls VtLogBuffer for changes:

#![allow(unused)]
fn main() {
let (lines, new_offset) = buf.lines_since_owned(offset);
let trim = trim_screen_chrome(buf.screen_rows());
let input_line = buf.prompt_input_text();
}

Change detection via hash:

#![allow(unused)]
fn main() {
let mut hasher = DefaultHasher::new();
screen.hash(&mut hasher);
input_line.hash(&mut hasher);
let screen_hash = hasher.finish();
let screen_changed = screen_hash != prev_screen_hash;
}

Frame sent only when !lines.is_empty() || screen_changed:

{
  "type": "log",
  "offset": 100,
  "total_lines": 142,
  "lines": [...],
  "screen": [...],
  "input_line": "text"
}

• offset is the start position of lines (where the delta begins).
• total_lines is the post-read monotonic cursor (== offset when no new lines). The client stores it and passes it back as ?offset= on reconnect, so catch-up resumes from the last consumed line instead of replaying the whole scrollback from the mount offset. The catch-up frame on connect carries total_lines too. (Without this, every WS reconnect — frequent on mobile: background/foreground, lock, network change — re-injected the entire session scrollback, duplicating it.)

Branch 2: event bus — forwards SessionState on parsed events:

{
  "type": "state",
  "state": { "awaiting_input": true, "agent_type": "claude-code", ... }
}

Client → Server (input passthrough)

WebSocket Text/Binary messages are written directly to the PTY.

trim_screen_chrome() — Screen Footer Removal

Removes Claude Code’s TUI footer (separator, status bar, permissions line, prompt).

Scan window: Last 15 rows from content end (handles Claude Code’s ~12-row footer).

Two anchor strategies:

1. Separator line — is_separator_line():

   #![allow(unused)]
   fn main() {
   fn is_separator_line(s: &str) -> bool {
       // 4+ consecutive: ─ ━ ═ — ╌ ╍
       // Tolerates decorated separators:
       // "──────── ■■■ Medium /model ─"
   }
   }

2. Prompt line — starts with ❯, >, >

Takes the higher anchor (closer to content), extends upward past empty/separator lines, truncates.

write_to_session() — Slash Mode Tracking

When users type / from the mobile PWA, the backend tracks this to enable slash menu detection in the output parser:

#![allow(unused)]
fn main() {
if data == "/" || data.starts_with('/') {
    slash_mode.store(true);
} else if data.contains('\r') || data.contains('\n') {
    slash_mode.store(false);
}
}

The output parser checks slash_mode before scanning screen rows for slash command menus.

3. Frontend Transport

File: src/transport.ts

subscribePty(sessionId, onData, onExit, options)

Auto-detects Tauri (native) vs browser (HTTP/WS) mode.

Browser mode connects a WebSocket:

ws(s)://host/sessions/{sessionId}/stream?format=log&offset={logOffset}

Options:

interface SubscribePtyOptions {
  format?: "log";
  logOffset?: number;
  onLogLines?: (lines: unknown[]) => void;
  onScreenRows?: (rows: string[]) => void;
  onInputLine?: (text: string | null) => void;
  onStateChange?: (state: Record<string, unknown>) => void;
}

Frame dispatch:

On reconnect the WebSocket reopens with ?offset=<last total_lines> (log mode) so the server’s catch-up only sends lines committed since the last one the client received. The raw (format omitted) path uses total_written byte offsets for the same purpose.

Returns a PtySubscription: callable to dispose, plus pause() / resume(). A backgrounded PWA renders nothing, so OutputView pauses on visibilitychange and the socket is dropped. The cursor above is what makes that safe — resume() reopens from it, so a hide/show cycle neither duplicates nor drops scrollback, and onExit is not raised for a close the client asked for.

rpc("write_pty", { sessionId, data })

Maps to POST /sessions/{sessionId}/write with body { data: "..." }.

Used for all PTY input: typed characters, escape sequences (arrow keys), control characters (Ctrl-U, Ctrl-C).

4. Mobile Components

OutputView

File: src/mobile/components/OutputView.tsx

Renders combined log history + current screen with auto-scroll management.

Text wrapping strategy: reflowDisplayLines() joins prose rows that the agent hard-wrapped before they reached the phone. It estimates the agent’s wrap width from the widest row in a contiguous prose run; LogLine.cols provides a lower-bound gate because Claude may wrap near 80 columns inside a much wider PTY. Short lines, paragraph breaks, lists, indented code, and box drawing stay separate. The browser then wraps the joined paragraph once at the phone width. Normal text uses pre-wrap; consecutive box-drawing lines stay in scrollable tableBlock containers with white-space: pre and overflow-x: auto. The rendering helpers live in src/mobile/utils/logLine.ts.

Initialization:

1. HTTP fetch: GET /sessions/{id}/output?format=log
2. WebSocket connect with logOffset from step 1 (avoids duplicate lines)

Data model:

displayedLines = [...logLines, ...screenRows]  // combined
                 .filter(searchQuery)            // optional filter

• logLines: Accumulated scrollback (max 500, ring buffer)
• screenRows: Current terminal screen (replaced each update)

Auto-scroll suppression:

• Track userScrolledUp via scroll event listener
• “At bottom” = within 80px of scroll end
• scrollToBottom(force?): skips if user scrolled up, unless force=true
• Initial load and session exit force-scroll

CommandInput — Delta Sync (textarea is source of truth)

File: src/mobile/components/CommandInput.tsx Helpers: src/mobile/components/syncGuards.ts

The textarea is the source of truth; the PTY is a write-only sink fed character deltas as the user types. Echoes from the PTY are accepted back only under strict conditions, so a laggy link can’t clobber what the user sees.

Direction 2: Textarea → PTY (syncDelta)

On every input event, syncDelta(newText) streams a minimal end-anchored delta computed by computeInputDelta(syncedText, newText):

// keep longest common prefix, backspace the divergent tail from the end,
// then type the new tail
function computeInputDelta(oldText, newText) {
  let prefix = 0;
  const max = Math.min(oldText.length, newText.length);
  while (prefix < max && oldText[prefix] === newText[prefix]) prefix++;
  return "\x7f".repeat(oldText.length - prefix) + newText.slice(prefix);
}

Append (no backspaces) and truncate (no retype) fall out as special cases. A mid-line edit backspaces only the divergent tail instead of nuking and retyping the whole line. This is correct as long as the remote cursor is at end-of-line (true while the user only appends/backspaces and hasn’t moved the readline cursor via arrow keys).

Why minimal, not full-nuke: the old “complex edit” branch sent \x7f×oldLen + newText on any mid-line change. That keystroke storm flickered readline and corrupted the line when a write dropped/reordered over a laggy mobile link — the “typing fa casino” symptom. Minimal deltas send the fewest keystrokes and keep the textarea consistent with the screen.

Direction 1: PTY → Textarea (guarded echo)

Source: ptyInputLine prop (from WebSocket input_line). An echo is accepted into the textarea only when both gates pass (syncGuards.ts):

1. Post-send guard — within POST_SEND_GUARD_MS (500ms) of Enter, every echo is ignored (suppresses the ghost flash of the just-sent command).
2. Strict-extension rule — isSupersetEcho(echo, syncedText): accept only if the echo strictly extends what we’ve sent (tab completion / autocomplete). Prompt redraws, lagging echoes, and history-nav replacements are ignored so the textarea can’t be clobbered.

For a plain shell, ptyInputLine is null (see prompt_input_text — it only recognizes agent prompts), so there is no PTY-driven reconciliation; the authoritative display of the shell line is the screen rendered by OutputView.

Send (Enter)

The typed text is already in the PTY via live delta sync, so Enter just writes \r (and clears the textarea + arms the post-send guard).

Mid-line editing

Moving the readline cursor is done with the ← / → keys in TerminalKeybar (\x1b[D / \x1b[C), not by moving the textarea caret. When arrows move the remote cursor, the textarea’s end-anchored model can diverge from the screen — the screen stays authoritative.

SlashMenuOverlay

File: src/mobile/components/SlashMenuOverlay.tsx

Displays slash command menu items detected from the terminal.

Item source: sessionState.slash_menu_items — populated by the output parser when it detects a slash command menu on screen (gated by slash_mode).

Arrow navigation:

┌─────────────────────────┐
│ /help     Get help...   │
│ /compact  Compact...    │
│ /review   Review code   │
│ ⏫  ▲  ▼  ⏬           │  ← arrow bar
└─────────────────────────┘

• Single arrow (▲/▼): sends one \x1b[A or \x1b[B
• Page arrow (⏫/⏬): sends items.length arrows (scrolls one page)
• Anti-zoom: touch-action: manipulation on all buttons

Selection flow:

1. User taps item → onSelect(command) callback
2. CommandInput replaces the slash fragment, updates the textarea, and sends its text delta to the PTY
3. User reviews, optionally edits, and submits; CommandInput waits for the last write and the agent’s Enter gap before sending Enter

SessionDetailScreen — Orchestration

File: src/mobile/screens/SessionDetailScreen.tsx

Wires everything together:

OutputView ──onStateChange──→ wsState signal
           ──onInputLine───→ ptyInputLine signal
                                    │
                                    ▼
                              CommandInput ←── SlashMenuOverlay selection

State merging: WebSocket state is authoritative when present; 3s poll state fills gaps.

5. SessionState Accumulation

File: src-tauri/src/state.rs — apply_event_to_session_state()

The output parser emits PtyParsed events. These accumulate into SessionState:

Lifecycle:

• Created on SessionCreated
• Removed on SessionClosed
• is_busy cleared on PtyExit

6. Key Escape Sequences

7. Debugging

Check backend logs

GET /logs?source=terminal&limit=20

Inspect raw WS frames

In browser DevTools → Network → WS → filter by /stream:

• "type":"log" frames show line count, screen rows, input_line
• "type":"state" frames show accumulated session state

Verify input_line extraction

GET /sessions/{id}/output?format=log

Response includes "input_line" — should show only user-typed text (no ghost/dim suggestions).

Verify separator detection

If agent chrome leaks through, check trim_screen_chrome() scan window (15 rows) and is_separator_line() threshold (4+ consecutive box-drawing chars).