Architecture chapter 08

lashlang/rlm

Lashlang is the agent-facing execution language for RLM mode. It lets the model run local programs against host tools, preserve state across turns, use async handles, and pass typed values through print and submit.

Language Reference

Lashlang is a compact CodeAct language for model-authored REPL blocks. Syntax is brace-delimited (statements inside { ... }, no significant whitespace), with familiar token-level cues for agents — if / else / for x in xs, and / or / not, raw r"""...""" strings. The shape of the language, though, is its own: tool calls go through (call name { ... })?, fail-fast unwrap uses Rust-style ?, async work uses the contextual start call form plus await, the ternary is cond ? yes : no, and submit / print / parallel are statement-level keywords. Effects are routed through explicit host calls, typed values, and persistent runtime state.

Values
null, booleans, numbers, strings, lists, records, projected host values, result wrappers, async handles, and immutable Image handles. Lists and records are value-shaped and copy-on-write at mutation roots.
Statements
Simple and path assignment, bare expression statements, call, print, if/else, for, break, continue, parallel, cancel, and submit.
Expressions
Literals, variables, builtin calls, tool calls, start call, await, field/index access, unary operators, binary operators, ternaries, result unwrap with ?, and Type { ... } literals.
Effects
All external work crosses ToolHost. Synchronous calls return wrapper records, background starts return handles, await resolves handles, and print creates model-visible observations.

Program Form

RLM mode extracts the first closed ```lashlang fence from model output. Only the lashlang language tag is recognised — rlm and other labels are treated as plain prose by first_lashlang_fence_span. Opening fences may be three or more backticks (CommonMark variable-length), the closer must match the opener length, and anything after the first closed block is ignored by the driver — so the block must contain the next action or final submit.

REPL state
Variables persist across fenced blocks in the active RLM execution state. Read-only host bindings projected through ProjectedBindings are already in scope (e.g., history) and should be used directly instead of reconstructed from prompt prose.
Completion
submit ends the program and turn. call continue_as { task: ..., seed: { ... } } is a host tool (not a keyword) that ends the current trajectory and queues a fresh-window successor session. In schema-constrained runs, a bad submitted shape produces a runtime nudge asking the model to fix the value and submit again from another fenced block.
```lashlang
text = (call read_file { path: "Cargo.toml" })?
print { chars: len(text), head: slice(text, 0, 1200) }
```

A typical handoff packs the concrete state the successor needs and ends the block. Computed expressions default to global state on the successor; values rooted at a projected binding (like input.prompt) stay projected.

```lashlang
findings = []
for path in (call glob { pattern: "src/**/*.rs" })? {
  text = (call read_file { path: path })?
  if contains(text, "TODO") {
    findings = push(findings, { path: path, chars: len(text) })
  }
}
call continue_as {
  task: "summarize the TODO audit and propose a follow-up plan",
  seed: { problem: input.prompt, findings: findings }
}
```

Syntax And Values

The language accepts newline-separated statements. Commas are accepted in lists, records, type fields, and parallel branch lists where they make generated code easier to read.

Literals
Use null, true, false, numbers, strings, lists like [a, b], and records like { name: "lash", "with space": 1 }. Record insertion order is preserved for display and serialization.
Strings
"..." supports \n, \r, \t, \", and \\. """...""" is multiline with escapes. r"""...""" and r'''...''' preserve content exactly for patches, JSON, scripts, Markdown, and heredocs.
Fields and indexes
value.field reads record and image fields. value[index] reads lists and records; record indexes are string-coerced and missing keys read as null. List indexes support negative offsets for reads and slices.
Images
Image values expose .id, .label, .size, .width, and .height. Image fields are immutable. len(image) is invalid; use image.size.

Expressions

Expressions are deliberately small but cover the operations agents need for filtering, shaping, validation, and tool orchestration.

FormMeaning
a + b, a - b, a * b, a / b, a % bNumeric arithmetic. + also concatenates strings and lists where supported by runtime value semantics.
==, !=, <, <=, >, >=Comparison operators used in conditionals and filters.
a and b, a or b, !a, not aBoolean composition and negation.
cond ? yes : noExpression ternary. There is no expression-form if.
name(args...)Builtin function call. User-defined functions are not part of the language surface.
(call tool { ... })?, (await handle)?Fail-fast unwrap of a result wrapper: return .value on success or abort the block with the wrapper error.

Assignment And Mutation

Assignments are rooted at named variables. Path assignment mutates the variable's current value through copy-on-write root updates, not through shared-alias mutation — there is no aliasing of inner records or lists.

Supported targets
name = expr, record.field = value, record[key] = value, list[i] = value, and nested forms like state.groups[g].count = count + 1.
Rules
Record field/index assignment inserts or replaces fields. List assignment replaces an existing integer index only. Assigning through a missing nested record key or image field is an error.
counts = {}
for group in groups {
  current = counts[group]
  counts[group] = (current == null ? 0 : current) + 1
}
submit counts

Control Flow

Control flow is statement-oriented. Loop cleanup is part of the runtime path, so loop escapes are explicit statements rather than arbitrary jumps.

if / else
Conditionals use statement blocks. else if parses as an else block containing another if.
for
for item in iterable { ... } iterates lists. Non-list iteration is a runtime error. range is the usual integer-loop helper.
break / continue
break exits the nearest loop through the iterator cleanup path. continue skips to the next iteration. They are rejected outside loops and cannot cross parallel branch boundaries.
submit
submit value ends the whole program or turn. submit inside a loop is not a loop control statement, and submit inside parallel is rejected.

A short walkthrough that exercises iteration, else if chains, break/continue, and format. Loop variables are scoped to the loop body and the prior binding of label is restored on break.

nums = [1, 2, 3, 4, 5, 6]
seen = []
total = 0
for n in nums {
  if n == 2 {
    continue
  }
  if n > 4 {
    break
  }
  seen = push(seen, n)
  total = total + n
}
if total > 10 {
  label = "large"
} else if total > 5 {
  label = "medium"
} else {
  label = "small"
}
submit format("seen={} total={} label={}", join(seen, ","), total, label)

Projected Host Bindings

The host can inject named, read-only values into Lashlang scope without copying them into VM State. They behave like ordinary variables for read access but rejecting reassignment keeps the host-managed values authoritative.

ProjectedBindings
Map of names to ProjectedValues the host hands the executor at the start of each fenced block. RLM rebuilds it from RlmGlobalsPatch + the live history projection.
ProjectedValue
Either a scalar FlowValue or a custom Arc<dyn ProjectedHostValue> for non-trivial shapes (lazy field access, length, indexing).
ProjectedHostValue
Trait that lets the host serve get_field, get_index, len, render, and materialize queries on demand (returning a ProjectedRead for field/index reads) so large host-side structures need not be cloned into Lashlang values.
Reserved names
history and any host-projected globals are reserved. Attempting name = ... over a projected name raises `name` is a read-only projected binding. The host-side seed channel (continue_as / spawn_agent seed:) can land projected entries that lashlang code then sees as read-only.
Tool arguments
RLM serializes projected values with an internal {"__projected__": ...} marker, then its before-tool hook materializes those markers before ordinary tools validate or execute. Tool implementations should parse ToolCall.args as plain JSON. The only preserved markers are root entries under seed: for projection-aware RLM control tools such as continue_as and RLM spawn_agent, where the marker means "re-project this value in the successor".

Tool Calls And Async Handles

Host work is explicit and typed at the boundary. Lashlang itself does not know provider APIs, file systems, shells, or subagents; it asks the registered ToolHost.

call
call tool { arg: expr } returns { ok: true, value: ... } or { ok: false, error: "..." }. Use ? for ordinary happy-path calls and keep the wrapper for retries or expected failures.
start call
start call tool { ... } returns a handle directly. Monitors and subagents are long-lived handle-producing tools; generic async tool starts use the same shape.
await
await handle resolves one handle. await [h1, h2] resolves a list and returns wrapper records in order; await { a: h1, b: h2 } resolves a record of handles and returns a record of wrappers. (await handle)? unwraps the resolved result.
cancel
cancel handle asks the host to stop background work. Cancellation is best-effort and uses the same handle identity exposed by list_async_handles.

Use ? for the happy path and inspect the wrapper when failures are expected. Mixing both styles in one block keeps fatal errors fail-fast while letting probes branch on outcome.

// Fail-fast: the block aborts if the file can't be read.
manifest = (call read_file { path: "Cargo.toml" })?
lines = split(manifest, "\n")

// Inspect-the-wrapper: we expect this read might miss.
optional = call read_file { path: "CHANGELOG.md" }
notes = optional.ok ? slice(optional.value, 0, 200) : format("no changelog: {}", optional.error)

// Iterate over a glob and skip files that fail to read.
items = []
for path in (call glob { pattern: "src/**/*.rs" })? {
  hit = call read_file { path: path }
  if !hit.ok {
    continue
  }
  items = push(items, { path: path, chars: len(hit.value) })
}
submit { line_count: len(lines), notes: notes, items: items }

Background work uses start call; results come back as wrappers when you await. await over a list returns wrappers in order; over a record it returns wrappers keyed by name. The unwrap operator ? applies to (await h) just like to (call ...).

// Kick off two long-running subagents in the background.
left = start call spawn_agent {
  agent_name: "audit_a",
  task: "audit module a",
  capability: "explore"
}
right = start call spawn_agent {
  agent_name: "audit_b",
  task: "audit module b",
  capability: "explore"
}

// Resolve both concurrently as a record of wrappers.
results = await { a: left, b: right }
if !results.a.ok {
  submit { error: results.a.error }
}
submit {
  a: results.a.value,
  b: results.b.ok ? results.b.value : null
}

Parallel Blocks

parallel is for independent work. Branches must not depend on each other's output, and branch-local assignments are merged only when unambiguous.

Named branches
parallel { first: call ... second: call ... } returns a record when used as an expression. Named branches are preferred because results can be read as results.first.
Positional branches
parallel { call ..., call ... } (or with newlines as separators) returns a list when used as an expression. Bare expressions inside a parallel block contribute values to that list.
a = start call spawn_agent { agent_name: "one", task: "Read file A", capability: "explore" }
b = start call spawn_agent { agent_name: "two", task: "Read file B", capability: "explore" }
results = parallel { one: (await a)?, two: (await b)? }
submit { a: results.one, b: results.two }

Builtins

Builtins are pure value helpers except for validation errors. They are called as ordinary functions and are optimized by the compiler where static type literals allow it.

BuiltinBehavior
len(x), empty(x)Length/emptiness for strings, lists, records, and projected lists. len(null) is 0.
slice(x, start, end)Substring or sublist. null bounds mean start/end; negative bounds count from the end.
range(end), range(start, end), range(start, end, step)Integer lists with an exclusive end bound. step may be positive or negative, but not 0.
ceil_div(a, b), floor_div(a, b)Finite-integer division helpers for chunk/count math; b must not be 0.
push(list, item)Returns a new list with item appended.
split, join, trimCommon string shaping helpers.
find(s, needle, start?)Returns the zero-based character index of the first literal match, or null. start defaults to 0 and is a non-negative character index. An empty needle returns start when it is in bounds.
grep_text(s, needle)Literal in-memory line search; needle must be non-empty. Returns one record per matching line: { line: int, text: str, match: str, start: int, end: int }; line is 1-based, text is the line without its line ending, and start/end are zero-based character offsets within that line's text, with end exclusive.
starts_with, ends_with, containsPrefix/suffix checks plus membership for strings, lists, projected lists, and record keys.
keys(record), values(record)Record key and value extraction in record order.
to_string, to_int, to_floatConversion helpers. to_string(image) emits metadata, not binary image data.
json_parse(s)Parses JSON into Lashlang values.
format(template, ...args)Positional interpolation with {}, explicit slots like {0}, and escaped braces via {{/}}.
validate(value, Type { ... })Returns value unchanged when it matches the type literal; aborts with a validation error otherwise.

Type Literals

Type { ... } values describe structured records for validation and dynamic tool output contracts. They are runtime values, not a static type checker for all code.

Shapes
Scalars: str, int, float, bool, dict, any, null. Collections: list[shape], enum["a", "b"], nested Type { ... }, named refs, and unions with |.
Optional vs nullable
email: str? means the field may be absent, but if present it must be a string. email: str | null means the field is required and may be null.
Nested objects
Nested record shapes must use the Type keyword: profile: Type { name: str }. A bare { name: str } is a record value form and is rejected in type positions.
Contract boundary
Tools such as llm_query and spawn_agent can expose output schemas derived from these shapes, but Lashlang v1 keeps this as contract truthfulness rather than model-authored generics.
Payload = Type {
  id: str,
  score: float | null,
  tags: list[str],
  status: enum["new", "done"],
  note: str?
}
validated = validate(candidate, Payload)

Type literals can be passed directly to tools that accept output schemas, and they double as runtime guards on JSON parsed from untrusted text. Validation errors carry a JSON-pointer-style path to the bad field.

Package = Type {
  name: str,
  version: str,
  labels: list[str],
  meta: Type {
    pages: int,
    published: int
  }
}

raw = (call read_file { path: "package.json" })?
parsed = json_parse(raw)
package = validate(parsed, Package)

// Constrain a subagent's output shape via the same Type literal.
result = (call spawn_agent {
  agent_name: "summarizer",
  task: format("summarize {} v{}", package.name, package.version),
  capability: "summarize",
  output: Type { headline: str, bullets: list[str] }
})?

submit { package: package, summary: result }

RLM Execution Path

The model emits code, the runtime executes it locally with the host's ProjectedBindings in scope, and the resulting trajectory is persisted as mode events. A block ending with continue_as finishes the turn as TurnOutcome::Handoff instead of looping back to the model.

RLM and Lashlang execution
sequenceDiagram participant Model as LLM participant Driver as RlmDriver participant Runtime as RuntimeTurnDriver participant Executor as RlmExecutionState participant Host as HostBridge / ModeExecutionContext participant Commit as TurnCommitPipeline Model-->>Driver: prose + ```lashlang fence Driver-->>Runtime: DriverAction::StartExec(code) Runtime->>Executor: execute_code(code, ProjectedBindings) Executor->>Host: call / start / await / cancel tool Host-->>Executor: ToolResult / handle / error Executor-->>Runtime: ExecResponse + observations + images Runtime->>Commit: records tool calls + RLM trajectory in turn events Runtime-->>Driver: Response::ExecResult or TurnOutcome::Handoff (continue_as)

An end-to-end fanout-and-merge program. It enumerates candidate files, kicks off per-file analyses in parallel, awaits them as a record, builds a histogram with nested path assignment, and either finishes the trajectory or hands the partial work to a successor when the budget would be exceeded.

```lashlang
paths = (call glob { pattern: "lash/src/**/*.rs" })?
if empty(paths) {
  submit { reviewed: 0, by_owner: {} }
}

// Fan out: one subagent per file, capped at the first eight to keep
// the parallel batch small.
batch = slice(paths, null, 8)
handles = {}
for path in batch {
  handles[path] = start call spawn_agent {
    agent_name: format("review:{}", path),
    task: format("review {} for TODOs and panics", path),
    capability: "explore",
    output: Type { owner: str, todos: int, panics: int }
  }
}

// Resolve the whole record concurrently, then collapse into a histogram.
resolved = await handles
by_owner = {}
errors = []
for path in keys(resolved) {
  wrapper = resolved[path]
  if !wrapper.ok {
    errors = push(errors, { path: path, error: wrapper.error })
    continue
  }
  report = wrapper.value
  current = by_owner[report.owner]
  by_owner[report.owner] = {
    todos: (current == null ? 0 : current.todos) + report.todos,
    panics: (current == null ? 0 : current.panics) + report.panics
  }
}

// Hand off the rest if we only got through a slice; otherwise submit.
if len(paths) > len(batch) {
  call continue_as {
    task: "continue the review over the remaining files",
    seed: { remaining: slice(paths, 8, null), by_owner: by_owner, errors: errors }
  }
}
submit { reviewed: len(batch), by_owner: by_owner, errors: errors }
```

Images And Attachments

Image support is a first-class runtime feature, not a plain record convention.

print(image)
Collects image values recursively, registers their attachment refs, and sends provider-visible LlmContentBlock::Image content alongside descriptor text.
submit(image)
Serializes metadata only, using a JSON-compatible descriptor with id, label, size, width, and height. It does not embed base64 data.

Runtime Internals

The implementation compiles AST to a compact VM instruction stream, then executes async host effects without blocking the RLM runtime.

  1. Parser

    lexer.rs tokenizes strings (including r"""...""" / r'''...''' raws), // comments, operators, and keywords. parser.rs builds Program, Stmt, Expr, assignment targets, parallel branches, and type expressions.

  2. Compiler

    runtime/compiler.rs lowers the AST into a Chunk of Instructions (defined in runtime/instruction.rs), folds static type literals, compiles assignment paths, loop cleanup jumps, builtin calls (specialized to Instruction::Len, Range, Push, FormatCompiled, ValidateCompiled, etc.), and optimized parallel call sets. cache.rs reuses compiled programs across fenced blocks via an LRU.

  3. VM

    runtime/vm.rs walks the compiled Chunk, stores globals in state.rs's State, records assignment conflicts inside parallel branches, calls host effects through host.rs's ToolHost, and consults the ProjectedBindings defined in value.rs for any name not present in State. ops.rs holds value coercions and builtin bodies; access.rs handles field/index reads and path assignment; format.rs and json.rs own format(...) and json_parse/serialization helpers. entry_points.rs wires the public compile_source / execute_compiled* / profile_compiled* surface re-exported from the crate root.

  4. Profiling

    examples/perf.rs, examples/profile.rs, and Scenario::ALL (in examples/bench_support/mod.rs) cover whole-language performance sweeps, including compiler and builtin hot paths. ProfileReport aggregates per-instruction and per-builtin counts/times.

Important Files

Keep prompt reference, parser/runtime behavior, tests, and docs synchronized when changing the language.

PathRole
lashlang/src/lexer.rsTokens, including raw triple-single-quote strings used for embedded code and heredoc-like content.
lashlang/src/ast.rsAST node types (Program, Stmt, Expr, AssignTarget, ParallelBranches, TypeExpr, …) shared by the parser and compiler.
lashlang/src/parser.rsAST construction for expressions, statements, type literals, calls, async, and control flow.
lashlang/src/runtime/mod.rsCross-cutting runtime types (RuntimeError, RuntimeFailure, ExecutionScratch, ExecutionOutcome, CompiledProgram, ProfileReport, CompileStats) and the pub use wiring that re-exports each focused submodule.
lashlang/src/runtime/compiler.rsAST → bytecode Chunk lowering, type-literal const-folding, parallel-call optimization, builtin specialization.
lashlang/src/runtime/instruction.rsInstruction opcode enum and supporting types (ParallelCallBranch, profile tags, loop ops).
lashlang/src/runtime/vm.rsBytecode executor — walks instructions, materializes intermediate values, dispatches to ToolHost, emits trace/profile data.
lashlang/src/runtime/value.rsValue, ListValue, ImageValue, LASH_TYPE_KEY, and the ProjectedBindings / ProjectedValue / ProjectedHostValue / ProjectedRead* seam.
lashlang/src/runtime/state.rsState globals, mutation rules, and Snapshot/restore (including projected snapshot tagging).
lashlang/src/runtime/host.rsToolHost, ToolHostCall, and ToolHostError contracts plus the default call_batch/start_call/await_handle/cancel_handle/print/yield_now impls.
lashlang/src/runtime/cache.rsCompiledProgramCache with LRU stats for fenced-block reuse.
lashlang/src/runtime/record.rsRecord value form, ordered keys, interned symbols, and shared structural helpers.
lashlang/src/runtime/schema.rsRuntime schema validation and Lash type descriptor support (ValidationPlan, execute_validate_builtin).
lashlang/src/runtime/access.rsField/index read and path-assignment helpers.
lashlang/src/runtime/ops.rsValue coercions, comparison/arithmetic, and most builtin function bodies.
lashlang/src/runtime/format.rsformat(...) template engine and compiled-format fast paths.
lashlang/src/runtime/json.rsjson_parse, JSON serialization, and round-trip helpers shared by snapshots and tool args.
lashlang/src/runtime/entry_points.rsTop-level compile_source / execute_compiled* / profile_compiled* / prewarm entry points re-exported from the crate root.
lash-mode-rlm/src/protocol.rsShared RLM execution prompt, RlmPromptFeatures, language reference shown to the model, and the first_lashlang_fence_span / extract_first_lashlang_fence extractors.
lash-mode-rlm/src/executor.rsBridge between Lashlang tool calls and runtime tool execution, host-projected bindings (history and globals), snapshot/restore, and image conversion.
lashlang/tests/rlm_prompt_claims.rsTests that prompt claims about language behavior remain true.
lashlang/examples/perf.rs, profile.rsWhole-language profiling and benchmark entrypoints.