# Trawl

> Scan any epoch range of Solana history with a filter and an optional Anchor IDL. Get decoded Parquet files in R2.
> Pay $2 for each TB of archive scanned (launch price; list price $5), minimum $4 for each job, in USDC on Solana through x402. No account.

## Flow

1. POST https://trawl.run/scan with a JSON body. The answer is 402 with the firm price in "quote" and the x402 terms in "accepts".
2. Send the same request with an X-PAYMENT header (a signed x402 payment). The answer is 202 with "job_id" and "status_url".
3. GET https://trawl.run/job/<job_id> until "state" is "done". It then holds "download_urls", "manifest_url" and a "duckdb" query.
4. Download with: wget -i https://trawl.run/job/<job_id>/urls . URLs stay valid for 7 days.

A 503 with Retry-After means: send the same request again after that many seconds.
A failed job ("failed" or "timeout") issues a credit. Send the same job with X-CREDIT: <credit id> to re-run it for free.

## Request body

{"preset": "<slug>", "start_epoch": <int>, "end_epoch": <int>}
or
{"filter": {...}, "idl": <Anchor IDL JSON, optional>, "start_epoch": <int>, "end_epoch": <int>}

Epochs are inclusive. A "filter" sent with a preset replaces the preset fields one by one.
A preset with an "idl" beside it is refused.

## Filter fields

- programs: base58 list. Matches top-level and inner instructions.
- accounts: base58 list. Keeps transactions that touch one of these accounts.
- discriminators: hex list, 8 bytes each.
- include_votes (default false), include_failed (default true), include_success (default true).
- only_idl_instructions (default false; needs an IDL).
- start_slot, end_slot (end exclusive), min_fee, max_fee (lamports, inclusive).
programs or accounts must hold at least one key.

## Output

- part-tNN-SSSS.parquet: matched instructions, raw "data" column.
- ix-<name>-tNN-SSSS.parquet: one IDL instruction, one column for each named account and argument.
- ev-<name>-tNN-SSSS.parquet: one IDL event, "log_index" and one column for each field.
Context columns on every row: slot, tx_index, signature, success, fee, program_id, ix_index, is_inner, inner_index, stack_height, accounts.
u128 and i128 are decimal strings. Compound types are JSON strings.

## Limits

- 13 TB of archive and 100 epochs for one job; 60 GB of output.
- The archive is 2 to 4 days behind the chain. GET https://trawl.run/health returns the newest epoch.

## Presets

- pumpfun: Pump.fun (Launchpad), program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P
- raydium-launchpad: Raydium LaunchLab (Launchpad), program LanMV9sAd7wArD4vJFi2qDdfnVhFxYSUg6eADduJ3uj
- meteora-dbc: Meteora Dynamic Bonding Curve (Launchpad), program dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN
- pump-swap: PumpSwap (DEX), program pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA
- raydium-clmm: Raydium CLMM (DEX), program CAMMCzo5YL8w4VFF8KVHrK22GGUsp5VTaW7grrKgrWqK
- raydium-cpmm: Raydium CPMM (DEX), program CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C
- meteora-dlmm: Meteora DLMM (DEX), program LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo
- meteora-damm-v2: Meteora DAMM v2 (DEX), program cpamdpZCGKUy5JxQXB4dcpGPiikHawvSWAd6mEn1sGG
- orca-whirlpool: Orca Whirlpool (DEX), program whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc
- raydium-amm-v4: Raydium AMM v4 (DEX), program 675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8, raw bytes only
- jupiter-v6: Jupiter Swap v6 (Aggregator), program JUP6LkbZbjS1jKKwapdHNy74zcZ3tLUZoi5QNyVTaV4
- jupiter-perpetuals: Jupiter Perpetuals (Perpetuals), program PERPHjGBqRHArX4DySjwM6UJHiR3sWAatqfdBS2qQJu
- kamino-lending: Kamino Lending (Lending), program KLend2g3cP87fffoy8q1mQqGKjrxjC8boSyAYavgmjD
- marginfi-v2: marginfi v2 (Lending), program MFv2hWf31Z9kbCa1snEPYctwafyhdvnV7FZnsebVacA
- openbook-v2: OpenBook v2 (Order book), program opnb2LAfJYbRMAHHvqjCwQxanZn7ReEHp1k81EohpZb
- phoenix: Phoenix (Order book), program PhoeNiXZ8ByJGLkxNfZRnkUfjvmuYqLR89jjFHGqdXY, raw bytes only
- jupiter-dca: Jupiter DCA (Limit order and DCA), program DCA265Vj8a9CEuX1eb1LWRnDT7uK6q1xMipnNyatn23M
- jupiter-limit-order-v2: Jupiter Limit Order v2 (Limit order and DCA), program j1o2qRpjcyUwEvwtcfhEQefh773ZgjxcVRry7LDqg5X
- spl-token: SPL Token (Core), program TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA, raw bytes only
- token-2022: Token-2022 (Core), program TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb, raw bytes only
- metaplex-token-metadata: Metaplex Token Metadata (NFT), program metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s, raw bytes only

## Pages

- [How it works](https://trawl.run/how-it-works)
- [Pricing](https://trawl.run/pricing)
- [Docs](https://trawl.run/docs)
- [FAQ](https://trawl.run/faq)

## Contact

contact@trawl.run