The request, the filter and the output

Send a request

Send POST /scan with a JSON body. Name a preset, or send a filter and an optional IDL. Add the first and the last epoch; both are inclusive.

curl -X POST https://trawl.run/scan \
  -H 'Content-Type: application/json' -d '{
  "preset": "pumpfun",
  "start_epoch": 1050,
  "end_epoch": 1052
}'

The answer is 402 Payment Required, with the firm price in quote and the x402 payment details in accepts. Send the same request again with an X-PAYMENT header from any x402 client. The answer is then 202 with a job_id.

A failed job issues a credit. Send the same job again with X-CREDIT: <credit id> in place of a payment.

The filter

A transaction must pass every field that is set. programs or accounts must hold at least one key.

{
  "programs": ["TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"],
  "accounts": [],
  "discriminators": [],
  "include_votes": false,
  "include_failed": true,
  "include_success": true,
  "only_idl_instructions": false,
  "start_slot": null, "end_slot": null,
  "min_fee": null, "max_fee": null
}

The account set compiles to a bloom filter, and each hit is checked in a hash set, so the match is exact.

Presets

A preset is one program, a filter on it, and its IDL, checked against real transactions of that program. A filter sent with a preset replaces the preset fields one by one. This request keeps the program and adds a wallet:

{"preset": "pumpfun", "filter": {"accounts": ["<wallet>"]}, "start_epoch": 1058, "end_epoch": 1058}

A preset with an idl beside it is refused. GET /presets lists them, and GET /presets/<slug> returns one with its filter and IDL.

raw: not an Anchor program. The result holds the raw instruction bytes and all context columns.

Your own IDL

Send an Anchor IDL in the idl field, in the 0.29 layout or the 0.30 layout, up to 512 KB. The scan derives each discriminator from the IDL. An explicit discriminator array wins; without one it is the first eight bytes of sha256("global:<snake_case name>").

Send one program for each IDL. The IDL applies to every program in the filter, so a second program can decode with the wrong layout.

Output files

part-tNN-SSSS.parquet         matched instructions, with a raw data column
ix-<name>-tNN-SSSS.parquet    one IDL instruction, decoded
ev-<name>-tNN-SSSS.parquet    one IDL event, decoded

NN is the scan thread. SSSS is the file sequence: a file closes at 2 GB and the next one opens. Each file is its own object in R2. There is no archive to unpack. Anything the IDL does not name, and anything that fails to decode, keeps its raw bytes in the part-* files.

Columns

Every row carries the transaction context:

part-* files add data, the raw bytes. A decoded table adds one column for each account the IDL names, then one column for each argument. Event tables add log_index and one column for each field. A name that matches a context column gets a prefix: acct_, arg_ or field_. Pump.fun's TradeEvent.fee is field_fee.

Decoded types

Every decoded column is nullable. A missing argument at the end of the data reads as null. A named account column is null when the transaction passed fewer accounts than the IDL names.

Events

An Anchor program writes an event in one of two ways, and the scan reads both: emit_cpi!, an inner instruction from the program to itself, and emit!, a Program data: log line. When a program writes one event both ways, the scan keeps the self-CPI copy, so one event is one row.

The scan reads a log line only in the frame of a program that passed the filter. The runtime cuts a log at 10 KB, and an emit! event after the cut is lost. (slot, tx_index, ix_index, inner_index, log_index) is unique.

Inner instructions

programs matches a top-level instruction and an inner one. Jupiter is called top-level, but the Raydium, Orca and Meteora swaps it routes are inner. (slot, tx_index, ix_index, inner_index) is unique for one instruction, and to sort by it replays the transaction in call order.

Download and query

GET /job/<id> returns the state. When it is done, it holds a URL for each file, a manifest URL and a DuckDB query. The URLs stay valid for 7 days from the end of the job.

# every file, in one command
wget -i https://trawl.run/job/<id>/urls

-- or query in place, with no download
SELECT mint, count(*) AS trades, sum(sol_amount) / 1e9 AS sol
FROM read_parquet(['https://…/ev-trade_event-t00-0000.parquet', …])
WHERE is_buy
GROUP BY mint ORDER BY sol DESC LIMIT 10;

API routes

Limits