An agent writes the code fast. It cannot make the archive fast.
An AI agent can write a scanner in a few hours. To make it correct took us weeks of real scans, because the worst errors do not look like errors. Pay for the result, or let your agent pay for it.
Each test is one hour of real archive.
An agent learns from errors. Here, a useful test is a real scan: about one hour for each epoch, on a machine that bills each hour. A unit test with a small fixture does not find the problems below. Some of them appear only in epochs from the last few months.
The two worst failures we found wrote zero rows and gave no error. The scan only looked slow. An agent that sees a slow scan adds threads or a bigger machine. It does not look at the reward type of slot 439,344,281.
What an agent does not have
- A record of which epochs break which engine version
- A fixture set of real transactions for each preset IDL
- A measured scan rate for the machine it rents
- A way to see zero rows before the run ends
- Your time, for each review and each restart
Twelve problems. Most give no error message.
Each one is fixed in the scan binary or in the control plane that runs it.
Version 1 transactions
About a third of mainnet transactions are version 1 by Oct 2026. Upstream Jetstreamer 0.7.0 fails each data node with Attempting to read 8256 bytes, retries without end, and writes nothing.
Fixed: The scan runs on a fork that reads legacy, v0 and v1 transactions.
A new reward type
In July 2026 Solana added the DeactivatedStake reward type. The engine rejected each block that held one, and retried. One real job read 709 GB of a 968 GB epoch, wrote nothing and hit its timeout.
Fixed: The fork reads the new type. The release of 19 Oct 2026 scanned slices of epochs 1020 and 1058 before it shipped.
Inner instructions
Jupiter calls Raydium, Orca and Meteora as inner instructions (CPI). A filter that reads only top-level instructions misses most swaps on those programs.
Fixed: The filter matches top-level and inner instructions. Four columns show where each one sat.
Events written two ways
Pump.fun, PumpSwap and Raydium LaunchLab write each event as a self-CPI and also as a log line. Count both and the volume doubles. Read only the logs and you lose the events after a 10 KB log cut.
Fixed: The scan keeps the self-CPI copy and drops the log copy with the same bytes. One event is one row.
Shared event names
Jupiter and Raydium both name an event SwapEvent, with the same discriminator. A log line from Raydium under a Jupiter route then decodes as a Jupiter event.
Fixed: The scan reads a log line only in the frame of the program that wrote it.
Integers in JSON
A JSON number is a double. A u64 lamport amount above 2^53 silently loses digits.
Fixed: A u64 column uses the exact Arrow type. u128, and any large integer inside JSON, is a string.
Arguments left out
Pump.fun accepts an instruction that leaves out the arguments at the end. A strict Borsh decoder fails on it.
Fixed: A missing argument at the end reads as null. A cut in the middle is still an error.
The size limit of one upload
R2 refuses one PUT over 4.995 GiB. One epoch of SPL Token gives about 6.9 GB of Parquet. A first test with a small result never sees this.
Fixed: Files rotate at 2 GB and upload while the scan runs. A large file uses a multipart upload.
The engine build
The optimized build of the engine ran out of memory on 3.3 GB of RAM. On an 8 GB machine it takes 13 minutes, and the tests take 15 more.
Fixed: Built and tested once. Each job boots the same prebuilt binary.
A database you did not ask for
By default the engine starts a local ClickHouse. On a small machine, that runs out of memory.
Fixed: Off, on every machine.
One wrong setting
JETSTREAMER_NETWORK_CAPACITY_MB=125, from an old setup note, gives one firehose thread: 38.6 MB/s, not 162 MB/s.
Fixed: Measured on the real plan. The quote uses the slower of two measured rates.
Machines that stay alive
A stopped cloud machine still bills. A hung machine bills until somebody sees it. A block volume outlives the machine it was attached to.
Fixed: Each machine is deleted, never stopped. A watchdog runs every 10 minutes. There is no block volume.
Orchestrate it, or order it.
- Rent a machine with 8 GB of RAM or more, and build the engine.13 to 15 min, again after each engine change
- Write the plugin, the filter and the IDL decoder.hours of agent time, and your review
- Test on one real epoch.about 1 hour for each test
- Find the silent failures.only if somebody counts the rows after a full run
- Fix, rebuild and test again.for each trap on this page
- Run the real job, and delete the machine after it.you watch it
- Send one request.a firm price in seconds
- Pay with x402.it settles in seconds
- The scan runs.about 1 to 2 hours for each TB
- Download, or query in place.wget, DuckDB, Polars, Spark
One recent epoch costs $4. A failed job gives a credit, so a failure costs you time, not money.
Give it this API, not the job.
x402 is an HTTP payment standard for machines. Your agent sends the request and gets a 402 with the firm price. It pays in USDC and sends the request again. It gets a job id. There is no account, no API key and no form.
The agent spends its effort on your question, not on the pipeline.
Point the agent at /llms.txt. It describes the API, the filter and the
output in plain text.
POST https://trawl.run/scan {"preset": "jupiter-v6", "start_epoch": 1058, "end_epoch": 1060, "filter": {"accounts": ["<wallet>"]}} ← 402 accepts[0]: price, asset, payTo, network POST https://trawl.run/scan X-PAYMENT: <signed x402 payment> ← 202 {"job_id": "job_…", "status_url": "…"} GET https://trawl.run/job/job_… ← 200 {"state": "done", "manifest_url": "…", "duckdb": "SELECT * FROM read_parquet([…]);"}
When an agent is the correct choice
You need a live stream, not history. You need stateful Rust for each transaction that a filter cannot express. You will scan every day, and you want to own the pipeline. For a question about the past, the scan here is faster and costs less than the debug runs.
Order the table. Skip the debug runs.
Get a firm price for your range. Pay by hand, or let your agent pay.