We got tired of rebuilding the same scraper
Every team we talked to had quietly built — and was quietly maintaining — its own fragile version of the same thing: browsers, proxies, and parsers held together with cron jobs.
Shoal started as an internal tool at a much smaller company, built because three different engineers had each written their own scraper for the same handful of competitor sites. None of the three versions worked the same way, and all three broke the same week a target site shipped a redesign.
We figured the actual problem wasn't scraping — it was that reading the web reliably had quietly become infrastructure, and infrastructure shouldn't be reinvented by every team that needs it. So we built the version we wished had existed, made it fast enough to run at the volumes AI products actually need, and opened it up.
Today Shoal exists for exactly one job: make the open web as easy to query as a database, whether the caller is a person, a script, or an agent that's going to ask it the same question ten thousand times before lunch.
A small, early-stage team
Shoal is early — a small group focused on the infrastructure, not yet a large org chart. Full founder bios are coming as the team grows.
Founding engineering
Builds and operates the fetch/traverse/search infrastructure and the API surface on top of it.
Product & Autopilot
Designs how a plain-language instruction turns into a reliable, editable pipeline.
Support & success
Answers the inbox, helps new integrations succeed, and feeds real usage back into the roadmap.
Built on AWS, designed to scale in bursts
Shoal's architecture is built around AWS-native, serverless primitives — chosen so cost and capacity scale with actual usage instead of a fleet of servers sitting idle.
Serverless fetch & traverse workers
Fetch, Traverse, and Bulk run as horizontally-scalable serverless compute, so a ten-thousand-URL burst and a quiet Tuesday cost the same per request — no fleet of always-on browsers to keep warm.
Vector-backed retrieval
Seek and Brief lean on a managed vector index for semantic retrieval, layered on top of traditional keyword search, so results stay relevant even when a query doesn't match a page's exact wording.
Hosted foundation models for Autopilot
Autopilot's natural-language planning calls hosted foundation models (including Amazon Bedrock) for the reasoning step, rather than a model we train and operate ourselves.
Global edge delivery
Static assets and API responses are served from edge locations close to the caller, keeping median response times low regardless of where a request originates.
What we optimize for
Reliable over clever
A boring API that returns the same shape every time beats a clever one that occasionally surprises you.
Priced like infrastructure
No charge for failures, no bandwidth surcharges — the price per request is the price, full stop.
Support that answers
A scaling question on a Saturday gets a real answer, not a ticket number.
99.9%
API uptime, measured monthly
6.2B+
Pages processed to date
<900ms
Median time to first byte
60%
Typical cost drop vs. self-hosted scraping
Give your agents a real connection to the web
Simple usage-based pricing, priced to scale, and built by people who'd rather ship your feature than argue about proxies.