Crawl

Crawl a URL (and optional linked pages) and return full + filtered text.

POST/v2/diligence/crawl

Crawl

Crawl a URL with optional link expansion and LLM-guided selection.

cURL

curl -X POST https://api.omtx.ai/v2/diligence/crawl \
  -H "x-api-key: YOUR_API_KEY" \
  -H "idempotency-key: crawl-$(date +%s)" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.org/egfr-review",
    "preset": "quick"
  }'

Response

{
  "job_id": "job_123abc",
  "status": "running"
}

This endpoint runs asynchronously and returns 202 with job_id.
Poll /v2/jobs/{job_id} until status is succeeded.
The final crawl payload is in response_payload on /v2/jobs/{job_id}.
Public callers may pass preset as quick, deep, deeper, or deepest.