xlab-start(7)

NAME

xlab-start — tell your agent the question, review the plan, and read the evidence.

SYNOPSIS

Tell my agent to… set up a lab → freeze the inputs → write the experiment and follow-up rules → show the spend → run the approved plan → explain the verdict.

You can use a coding agent such as Codex, Claude Code, or Cursor with access to the lab machine's shell and files. Paste the prompts below into that agent.

DESCRIPTION

You supply the question, the constraints, and the decisions that matter. Your agent reads the README and adapter docstrings, writes TOML specs and trees, checks them with the CLI, runs the approved work, and explains the evidence. You do not need to memorize every command before starting.

Driving xlab with an agent and measuring an agent as an experiment arm are different roles. This walkthrough teaches the first: your coding agent operates the lab using the simulated, compression, and command adapters. Agent-specific examples are being added separately; inspect xlab adapters in your checkout before choosing a measurement adapter. These prompts carry the operating method without requiring a dedicated agent-experiment skill.

TRY THE SIMULATED LAB FIRST

Start here if you want to see the whole method without a real workload, codec installation, or API key. A model you use to drive the shell may have its own usage cost; the simulated experiment itself makes no model calls.

Tell my agent to try the lab:

Read the xlab README and the built-in adapter docstrings. Help me try the lab with synthetic data before measuring anything real. If xlab is not installed, clone https://github.com/s4njee/xlab and install it in a virtual environment with a Python that supports the project. Create ~/labs/demo with the simulated examples, run xlab doctor and xlab selftest from that lab, and tell me what passed or needs fixing. Explain the question, control, smallest effect that matters, and randomized blocks in S1-demo. Validate it and show me its plan and declared spend. Stop there before enqueueing; after I have reviewed the plan, I will tell you to run it and explain the verdicts.

After the plan has been shown, continue with:

Run the simulated spec we just reviewed. When it finishes, explain the ratio,
interval, and verdict for each candidate in each cell. Show me the run id,
any voids, and a local report. These are known synthetic effects, so do not
describe them as a benchmark of this machine or a model.

The corresponding shell path, after installing xlab in the repository, is:

xlab init ~/labs/demo --example simulated
cd ~/labs/demo
xlab doctor
xlab selftest
xlab check experiments/S1-demo.toml
xlab plan experiments/S1-demo.toml --seed 1
# Read the plan and spend before continuing.
xlab enqueue experiments/S1-demo.toml --seed 1
xlab work
xlab analyze S1-demo
xlab report S1-demo

For a simulated tree instead of a single run, start trees/T0-demo.toml, then use xlab work --trees or xlab up. The latter also serves the local dashboard.

WHAT TO TELL YOUR AGENT

If you know only the question, start with it. Ask the agent to fill the gaps with you before it writes a spec or starts a run:

Bring to the conversation	Why it matters
The question and hypothesis	What you are trying to decide and what would support it
The current configuration	The control every candidate is paired against
The primary metric and meaningful effect	What “better” means and how large a change matters
Candidate settings and conditions	Which arms and cells are in the comparison
Frozen inputs and an independent check	How each sample is reproducible and verified
The measurement host and quiet-time constraints	Where the runner works and when it should wait
Resource allowances	The declared spend and the rolling-window budget
Follow-up decisions	What should be confirmed, held for you, or stopped

An opening prompt can be as simple as:

I want to know whether changing this command will make my nightly job faster.
Read the xlab README. Ask me for the current command, candidates, frozen input,
verification method, meaningful effect, measurement host, and budget. Help me
turn those answers into a valid experiment. Show the plan before running it.

A WORKED SCENARIO: THE TELEMETRY LAB

The README follows a server that receives hourly sensor JSONL files, archives each completed day with gzip -6, and aggregates it into SQLite. You want to speed up the nightly work while the collector keeps running. The next prompts carry that same example from setup to a decision tree and a report.

The paths, dates, codec levels, block counts, and thresholds below are example configuration choices, not measured findings. Substitute your host's values. Send each prompt when you reach that stage; the planning prompts deliberately stop before execution, and the start prompt authorizes the plan you reviewed.

1. TELL MY AGENT TO SET UP THE LAB

Clone https://github.com/s4njee/xlab onto the machine that will take the measurements. Read its README and adapter docstrings, then install it with pip install -e . in a virtual environment using Python 3.11 or newer. Check that this interpreter can import xlab.cli. Make a lab with xlab init ~/labs/telemetry --example compression. From that lab, run xlab doctor and xlab selftest and tell me what passed and what needs fixing. The host collects sensor readings all day, so set max_load_per_core = 0.5 in lab.toml. Check which codecs the example specs require and which are installed. Show me the lab path and configuration. Do not enqueue experiments yet.

What you should get: A usable Python environment, the selftest result, a lab directory, and an explicit list of missing tools. A busy host should wait at its load gate.

The repository holds the implementation; the lab directory holds your specs, queue, records, and reports. Keep those roles distinct. If a required codec is missing, install it or deliberately revise the candidate list before the spec is frozen. Do not bypass preflight to make a missing tool disappear.

From the lab directory, the setup checks include:

xlab doctor
xlab selftest
xlab adapters

The example block counts are starting configurations, not a promise that a particular effect will be resolved. Before trusting a real comparison, run the calibration protocol on the host and conditions you will use.

2. TELL MY AGENT WHAT IS BEING COLLECTED

Sensor readings arrive in /srv/telemetry/YYYY-MM-DD/HH.jsonl, about 2 GB per day. The archive job currently compresses a completed day with gzip -6 into /srv/archive/. Use /srv/telemetry/2026-10-17 as the example completed day; ask me for the real completed date and paths before using this on my host.

Never read the live collection directory during a timed sample. Freeze a copy outside it, pack that copy once as a deterministic corpus, and record its SHA-256. Call the corpus telemetry-day and pin the actual digest in the experiment specs so every arm sees the same bytes. Show me the frozen source, corpus definition, size, and digest before planning the comparison.

What you should get: A frozen input and a telemetry-day corpus definition pinned to the digest of the packed corpus, with preparation outside the timed path.

The date and paths above come from the README's worked scenario; they are not instructions to read a directory that happens to exist on your machine. Replace them with your own completed, immutable input.

The agent should create a definition shaped like this in the spec:

[factors]
corpus = ["telemetry-day"]

[corpora.telemetry-day]
path = "/srv/frozen/telemetry/2026-10-17"
# sha256 = the real digest of the packed corpus, supplied after preparation

The compression adapter packs directories with sorted members and normalized ownership. Pin the resulting corpus digest, not a guessed value or a digest of one member file. Preparation and round-trip verification are outside the timed compression interval. Keep the frozen source and the corpus manifest with the lab.

See corpora and digests for generated inputs and real directories.

3. TELL MY AGENT TO PLAN THE EXPERIMENT

Set up a decision tree in ~/labs/telemetry for these questions, in this order. Write the specs and branch rules before measuring anything.

Screen zstd levels 1 through 9, lz4, brotli 4, and xz 1 against gzip -6 as the control on the pinned telemetry-day corpus. Use six screening blocks, compress_s as the primary metric, and a 5% smallest effect that matters. A candidate qualifies only if it is faster and loses no more than 3% of the control's compression ratio.

Confirm exactly the qualifying settings with twelve fresh blocks in a confirmatory experiment. If a setting is faster only by giving up too much ratio, hold for my decision. If nothing qualifies, stop. Only after a zstd setting is confirmed should the thread follow-up compare 2, 4, and 8 threads.

Validate the tree and every concrete spec with xlab check and preflight. Show me the randomized plans, declared spend, budget settings, and the tree's branch order. Explain what remains unknown about the cost of later nodes. Do not start the tree or enqueue a run until I have reviewed the plan.

What you should get: Validated specs and a decision tree: screen, confirm with fresh blocks, hold on a trade-off, and run the thread follow-up only after its declared condition.

Use the bundled T1-log-archives tree and the C2-replace-gzip and C3-zstd-threads specs as starting points. The agent must adapt them to your corpus, tools, and question; copying a filename does not make the plan correct.

Check the branch order. The shipped tree queues both confirm and threads from its qualifying screening branch. This prompt asks for a stricter sequence: the thread node must instead follow the appropriate confirmation outcome. Have the agent show that change explicitly before starting the tree.

For a concrete spec file, inspect both validity and planned work:

xlab check experiments/C2-replace-gzip.toml
xlab preflight experiments/C2-replace-gzip.toml
xlab plan experiments/C2-replace-gzip.toml --seed 1
xlab check trees/T1-log-archives.toml

Your adapted files may have different names. The plan should identify arms, cells, blocks, retry allowance, and declared resource estimates. Review the confirmation and sentinel share too. Later promotion costs depend on which arms qualify. An unknown spend is not zero; current budgets are rolling-window admission checks, not automatic token pricing or a hard mid-run dollar stop. See planning and budgets.

4. TELL MY AGENT TO START THE TREE

I have reviewed the specs, tree, plans, and declared spend. Start the approved tree in ~/labs/telemetry. Add the fixed sentinel spec as a standing run every six hours so we can see the host's noise floor and drift.

Use xlab service to print the launchd or systemd user unit for this machine. Show me the unit and its interpreter and lab paths, then install it so xlab up survives a reboot. Verify that the service is running, and tell me the dashboard address and where the records and reports are stored.

When I ask for status, tell me which node is active, which branch the tree took and why, and whether any run is waiting for load, budget, or a decision. If you can monitor this host, tell me when the first tree decision occurs; otherwise explain how I can check it or ask you to read it later.

What you should get: One running lab service, a standing sentinel, a reachable local dashboard, and clear commands for checking the queue and tree.

The start prompt is a separate instruction from the planning prompt. It tells the agent that you have reviewed the work. A chat that cannot keep running on the host cannot promise a later notification; use the service for execution, and your agent's own monitoring feature if you want notifications.

The relevant commands, using the bundled tree name, are:

xlab tree start trees/T1-log-archives.toml
xlab standing add experiments/S0-sentinel.toml --every-hours 6
xlab service
xlab queue
xlab tree status T1-log-archives

Review and install the printed service using its host-specific instructions. The service runs xlab up; avoid starting a second runner alongside it. For a foreground session instead of a service, use xlab up directly. For a finite foreground batch, use xlab work --trees.

The dashboard defaults to http://127.0.0.1:8710/ on the lab host. If that host is remote, use an SSH tunnel rather than assuming your laptop's loopback address is the remote machine. The sentinel uses the same queue and budgets; it does not interrupt a sample in progress. Records are under data/runs/; reports are under reports/. See running unattended.

5. TELL MY AGENT TO COMPARE ANOTHER COMMAND

The aggregation job is /srv/jobs/aggregate.py --day DAY --out DB. Set up a command-adapter experiment in ~/labs/telemetry: does --parallel 4 make the completed day's aggregation faster than the default? Use the default as the control, eight blocks, wall_s as primary, and a 5% smallest effect that matters. Both arms must use the same frozen input.

Give every sample a fresh output database. In the untimed after command, check its row count against an independently established expected result; a failed command or failed verification must be void, never a timing. Keep setup, input preparation, and verification out of the timed command.

Validate it, run preflight, and show me the plan, declared spend, and how it fits the lab's current budgets. Wait for me to review those. When I tell you to enqueue it, use --front so it runs before other waiting work.

What you should get: A separate command-adapter spec, independent output verification, and a reviewed plan that can be queued ahead of waiting runs.

Choose a new experiment id and filename so its records stay separate from the compression tree. --front changes queue priority; it never interrupts the current run. Declare the meaningful effect before sampling, and use output checks independent of either candidate's claim that it succeeded.

For example, if the agent names the spec K2-aggregate.toml:

xlab check experiments/K2-aggregate.toml
xlab preflight experiments/K2-aggregate.toml
xlab plan experiments/K2-aggregate.toml
# After reviewing the plan:
xlab enqueue experiments/K2-aggregate.toml --front
xlab queue

That filename is illustrative; the commands require the spec your agent writes. No additional runner is needed. The existing service picks it up from the queue. See the README's custom command experiment and the adapter guide for before, timed command, and untimed after hooks.

6. TELL MY AGENT TO EXPLAIN THE VERDICT

Analyse the compression screening experiment in ~/labs/telemetry. For each candidate and cell, tell me its ratio to gzip -6, the interval, the number of complete paired blocks, and the verdict. Explain what the metric's direction and the declared smallest effect mean before naming a winner.

Tell me which candidates kept the required compression ratio, whether the decision tree moved, and the exact branch condition that caused the move. Show the run ids and distinguish screening leads from confirmation results. Summarize void reasons and any sentinel drift that affects interpretation.

For inconclusive results, report the analysis's estimated blocks needed. Use observed sample durations to estimate the extra hours, stating the assumptions and any unknown spend. Do not change the rubric after seeing the results. Write a local report and show me where to find its source records.

What you should get: An evidence-backed explanation with intervals, complete-block counts, run ids, tree decisions, and a report; an inconclusive result remains inconclusive.

Ask for the experiment id your agent actually created. With the bundled screening id, the commands are:

xlab analyze C2-replace-gzip
xlab analyze C2-replace-gzip --json
xlab tree status T1-log-archives
xlab standing list
xlab report C2-replace-gzip

A ratio below one is less compression time; it is not by itself a better verdict. The intervals must clear the meaningful-effect boundary, and the compression-ratio requirement must also hold. A confirmation uses fresh blocks and the declared multiple-comparison correction.

The blocks-needed figure is an estimate from the observed spread, not a guarantee of the next verdict. Additional elapsed time depends on complete blocks, retries, gates, and queue contention. If the records do not support an hours or dollar estimate, the agent should say so rather than invent one. See analysis and verdicts and the record format.

WHEN SOMETHING DOES NOT GO TO PLAN

Situation	What to ask your agent
xlab cannot be found or Python cannot import it	Show the interpreter and virtual environment; reinstall from the checkout using that interpreter
Preflight reports a missing tool or changed input	Name the failure and fix the dependency or corpus pin before queueing
The queue waits without starting a run	Read xlab queue and explain whether the cause is a load gate, budget, pause, or halt
A tree is held	Show the branch and its evidence; wait for my trade-off decision
The shell or machine stops	Restart the same lab service and verify it resumes from unfinished work
The dashboard is on a remote host	Explain the host's loopback address and help me open an SSH tunnel
A result is inconclusive	Show uncertainty, estimated blocks needed, and the added resource cost; keep the declared rubric

THE RULES YOUR AGENT SHOULD KEEP

Name the control and smallest meaningful effect before the first sample. Show the plan and declared spend before enqueueing. Keep live data and verification out of the timed path. Record failed measurements as voids with reasons; do not turn them into zero or drop slow samples. Report a verdict with its interval and run ids, and treat screening as a lead to confirm with fresh blocks.

Those are operating rules, not a claim that the current CLI enforces every conversational step. You and the agent still need to follow them.

SEE ALSO

Lab directory · Specs · Corpora · Adapters · Budgets · Trees · Calibration · Running unattended · CLI reference · Original README