Web data for AI agents and pipelines
Scrape, crawl, map, search, extract and monitor websites through one API. Results come back as clean markdown, structured JSON and screenshots. Botnom is hosted in the EU and charges one credit per page.
1,000 pages free every month · No credit card required
# markdown, links and a screenshot of one page
curl -X POST https://botnom.com/api/v1/scrape \
-H "Authorization: Bearer $BOTNOM_API_KEY" \
-H "content-type: application/json" \
-d '{"url": "https://example.com/pricing",
"formats": ["markdown", "links", "screenshot"]}'/api/v1/jobs/:id · 1 credit for all three formatsSix operations, one API
Each operation runs as an async job. You get an id back straight away, then poll it or wait for a webhook. The same operations are available as MCP tools.
| Operation | Endpoint | What you get |
|---|---|---|
| Scrape | POST /api/v1/scrape | Markdown, HTML, links, a summary or a screenshot of one URL. JavaScript is rendered and navigation and boilerplate are stripped. |
| Crawl | POST /api/v1/crawl | Every page of a site, following links up to a depth and page limit, with include and exclude path filters. |
| Map | POST /api/v1/map | The list of URLs on a site, collected from its sitemaps and links in a few seconds. |
| Search | POST /api/v1/search | Web search results, with the option to scrape each result page in the same job. |
| Extract | POST /api/v1/extract | Structured JSON from one or more pages, shaped by a plain-language prompt or a JSON schema. |
| Batch | POST /api/v1/batch/scrape | Up to 1,000 URLs scraped in parallel as a single job. |
Built to be predictable
One page costs one credit. Markdown, HTML, links, a summary and a screenshot of the same page all fit in that credit. The cost of every operation is published, and nothing is priced dynamically.
Made for teams. Each organization has its own members, roles and API keys, so access can be granted and revoked per person and per project.
Stored in the EU. Botnom runs in Helsinki, Finland. Crawl results, screenshots and logs are kept there.
You choose which model reads your data. Extraction runs on the LLM your admin picks, and every prompt is logged with its token count and cost.
Know when a page changes
Put any scrape, crawl or search on a schedule and turn on change tracking. Botnom keeps the latest snapshot, compares each run with the previous one, and sends a signed monitor.changed webhook when something moved.
- Added and removed line counts, with a preview of what changed
- A run history for every monitor in the dashboard
- Standard cron schedules, from every few minutes to monthly
- Monitor runs cost the same credits as any other job
## Changelog+ ### 4.2.0 — 6 October 2026+ - Bulk export to CSV and JSONL+ - Webhook retries with backoff ### 4.1.3 — 28 September 2026 - Fixed time zones in scheduled reports
https://hooks.example.com/botnomSigned with X-Botnom-Signature| Time | Model | Tokens in / out | Cost | OK |
|---|---|---|---|---|
| 12:04:11 | claude-sonnet-5.5 | 3,812 / 420 | $0.0178 | yes |
| 12:03:58 | claude-sonnet-5.5 | 2,104 / 388 | $0.0121 | yes |
| 11:59:20 | gpt-5-mini | 5,330 / 611 | $0.0041 | yes |
Choose the model that reads your data
Extraction runs on the model your admin selects from a live catalog of Anthropic, OpenAI, Google and x.ai models, with current prices shown next to each one. Switching models is a setting, not a deploy.
- Every prompt and response is stored with tokens, cost and latency
- Spend totals are visible in the admin console
- Models from other providers can be entered by name
Start on the free plan. Paid plans add more pages, higher request limits and larger crawls.
For trying Botnom and small projects.
- 1,000 credits a month
- 20 requests a minute
- Crawls of up to 100 pages
- REST API, SDKs and MCP
- Schedules, monitors and webhooks
For production apps and agents.
- 50,000 credits a month
- 100 requests a minute
- Crawls of up to 5,000 pages
- Everything in Free
- Email support
For teams and data pipelines.
- 500,000 credits a month
- 500 requests a minute
- Crawls of up to 50,000 pages
- Everything in Starter
- Priority support
Need more than 500,000 pages or single sign-on? Email us.
Questions
What does a credit buy?
Scraping or mapping one page costs 1 credit, whatever formats you ask for. Crawls cost 1 credit per page crawled, batch jobs 1 credit per URL, search 2 credits per 10 results, and an extract request 5 credits. The free plan includes 1,000 credits a month.
Which sites can Botnom fetch?
Any publicly reachable page, including pages that need JavaScript to render. Private networks and cloud metadata addresses are blocked, and requests to each domain are rate-limited across all customers.
Where is my data stored?
On servers in Helsinki, Finland. When you use extraction, the page text is sent to the LLM provider your admin has selected.
Do I have to poll for results?
No. Add a webhook endpoint in the dashboard and Botnom sends a signed POST with the job id when a job completes or fails. You then fetch the result from the API.
Does it work with Claude and Cursor?
Yes. Botnom has an MCP server with eight tools that works with Claude, Cursor and any other MCP client, alongside the REST API. Official TypeScript and Python SDKs are on the way.
What happens when I run out of credits?
New jobs are rejected with a 429 quota_exceeded response until the next month starts or you move to a bigger plan. Nothing is billed automatically.
Try it on your own site
Create an account, make an API key and send one request. The first 1,000 pages each month are free.