Skip to main content

​
What this does

Searchable’s LLM Analytics dashboard shows you, in near real-time, which AI assistants — ChatGPT, Claude, Perplexity, Google’s AI Overviews, and the rest — are crawling your site, what pages they’re hitting, and which of those crawls turn into referred users. To do that, Searchable needs to see your inbound HTTP requests. The setup below points your hosting platform’s request logs at our ingest endpoint. We classify the AI bots, drop everything else, and never store query strings or other PII.
Setup takes about 5 minutes. There are no agents to install and no code to change in your application.

​
Pick your platform

Vercel

One-click Log Drain. Works on Pro and Enterprise.

Cloudflare Worker

Edge Worker — works on any Cloudflare plan, including Free.

Cloudflare Logpush

Native Logpush job — Enterprise plan only.

Amazon CloudFront

Standard Logging V2 via Firehose — any CloudFront distribution.

Netlify Edge Function

Edge Function — works on every Netlify plan, including Free.

Netlify Log Drain

HTTP Log Drain — Enterprise plan only.

Fastly

Real-Time Log Streaming via HTTPS endpoint.

Akamai

DataStream 2 HTTPS push — any property with DataStream 2 entitled.

Google Cloud Platform

HTTPS Load Balancer / Cloud CDN logs via Pub/Sub push.

WordPress

Must-use plugin — captures AI crawlers at the origin on any self-hosted site.

Other platforms

Custom REST API or middleware for everything else.

​
Which one should I use?

Use Vercel Log Drain. It’s the most direct path — Vercel streams request logs to Searchable with one entry in your project settings. No code changes.
Use the Cloudflare Worker. It works on every Cloudflare plan including Free, and the Free tier easily covers ~100k requests/day. The Worker forwards AI-bot traffic to Searchable from the edge — no latency added to real users.
You can use either path. Cloudflare Logpush is the most “native” option — Cloudflare batches and ships logs directly with no Worker involved. If you’d rather avoid the Enterprise-only Logpush feature, the Worker path works on Enterprise too.
Use Amazon CloudFront → Firehose. CloudFront’s free Standard Logging V2 feature ships records to Amazon Data Firehose, which forwards them to Searchable. Works on any CloudFront distribution — you only pay for the Firehose ingestion.
Use the Netlify Edge Function on any plan (including Free), or Netlify Log Drain if you’re on the Enterprise plan. The Edge Function path runs at Netlify’s edge with no added latency for real users.
Use Fastly Real-Time Log Streaming. Configure an HTTPS Logging endpoint inside the Fastly UI — no code changes — and Fastly streams request logs to Searchable.
Use Akamai DataStream 2. Create a DataStream 2 stream in Akamai Control Center, pick the property fronting your domain, and point it at Searchable’s HTTPS endpoint. Requires the DataStream 2 product on your Akamai contract — your account team can confirm if it’s enabled.
Use Google Cloud → Pub/Sub push. Cloud Logging captures every HTTPS Load Balancer or Cloud CDN request; a Pub/Sub topic with a push subscription forwards those logs to Searchable.
Use the WordPress plugin — a must-use plugin that forwards AI-crawler requests from the origin, with no theme changes and no JavaScript. One caveat: if your site is behind Cloudflare, or runs a full-page cache that serves HTML without booting PHP, prefer the Cloudflare Worker — it sits in front of the cache and sees every request.
Use the custom integration. Searchable accepts a small REST POST from your application middleware or any platform that can ship structured request logs.

​
What gets sent to Searchable

For each request that matches an AI-bot user agent, we receive:
  • HTTP method, path, and host (no query strings)
  • User agent
  • Referer
  • Country code (geo-IP, no precise location)
  • Response status and bytes
  • Timestamp
That’s it. Query strings, request bodies, response bodies, headers other than User-Agent / Referer, and full IP addresses are never sent.

​
Verifying the connection

After completing setup, return to LLM Analytics → Setup in your Searchable dashboard. The status strip at the bottom updates as events arrive:
  1. Waiting for first event — your platform hasn’t sent anything yet. AI bots typically hit any indexed site within a few hours.
  2. Connected — events are flowing. The strip shows the count from the last 24 hours.
If the status doesn’t update within a few hours, see the troubleshooting section in the platform-specific guide you followed.

​
Next steps

Add a domain

LLM Analytics is scoped to your project’s primary domain.

Connect Search Console

Layer in keyword data so you can correlate AI crawls with search demand.