Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.searchable.com/llms.txt

Use this file to discover all available pages before exploring further.

​
What this does

Vercel’s Log Drain feature streams every inbound request to a destination of your choosing. We point that drain at Searchable’s ingest endpoint, classify the AI bots, and drop everything else.
No code changes. All configuration is done inside the Vercel dashboard.

​
Prerequisites

A Vercel project on the Pro or Enterprise plan (Log Drains aren’t available on Hobby)
Owner or Member-with-admin access to the Vercel project
A Searchable project with your domain confirmed

​
Setup

1

Generate an integration token in Searchable

  1. Open your Searchable dashboard
  2. Go to Agent Analytics → Setup
  3. Pick Vercel as your crawler source
  4. Click Generate token
Copy the token now — it starts with sa_… and won’t be shown again. You can always generate a new one if you lose it.
2

Open Vercel's Log Drain settings

Navigate to your Vercel project:Project Settings → Log Drains → AddOr open it directly: vercel.com/dashboard/log-drains
3

Configure the drain

Fill in these fields:
FieldValue
Delivery formatNDJSON
Endpointhttps://searchable-tracker.searchable.workers.dev/v1/vercel-logs
Custom headersAuthorization: Bearer <your-sa_-token>
SourcesFunctions, Edge Functions, Static Files, Firewall, Redirects
Sampling rate100%
Selecting all five sources is what gives Searchable a complete picture. AI crawlers hit static files (sitemaps, robots.txt) almost as often as they hit pages.
4

Save and verify

Click Save. Vercel sends a verification ping to the endpoint immediately — Searchable accepts it.Then return to Agent Analytics → Setup in Searchable. The status strip at the bottom should show Connected within a few minutes once an AI bot hits your site.

​
What Searchable receives

For each request, the Log Drain sends a small NDJSON record with:
  • HTTP method, path, host (query strings stripped before storage)
  • User agent
  • Referer
  • Status code, response bytes
  • Timestamp
  • Geo country (from Vercel’s edge metadata)
Bodies, headers, cookies, and full IPs are never sent or stored.

​
Verifying the connection

In Searchable:
  1. Go to Agent Analytics → Setup
  2. Look at the status strip at the bottom of the page
  3. Click Check if it still shows “Waiting for first event”
StatusWhat it means
Waiting for first eventThe drain is configured but no AI bot has hit your site yet. Typical wait is a few hours for sites that are already indexed.
ConnectedEvents are arriving. The strip shows the count from the last 24 hours.
You can also confirm in Vercel: Log Drains → your drain → Deliveries. Successful deliveries return 204 No Content.

​
Troubleshooting

The Authorization header is missing or wrong.
  • Make sure you added a custom header (not a query parameter) named exactly Authorization
  • The value must be Bearer followed by the full sa_… token, with one space and no quotes
  • If you’ve recently revoked the token in Searchable, generate a new one and update the drain
The delivery format isn’t NDJSON.
  • In the drain settings, set Delivery format to NDJSON (not JSON or Logfmt)
  • Save the drain — Vercel will retry pending deliveries
A few possible causes:
  • The drain isn’t enabled (check Vercel — drains can be paused)
  • The selected sources don’t include the routes AI bots are hitting (re-check Sources includes Static Files and Functions)
  • Your domain in Searchable doesn’t match the project on Vercel (check Agent Analytics → Setup → Confirm your domain)
If everything looks right, Vercel’s Deliveries tab tells you whether deliveries are succeeding. Successful deliveries that don’t appear in Searchable point to a domain mismatch.
Log Drains are a Pro / Enterprise feature on Vercel. On Hobby, your options are:

​
Removing the integration

To stop sending traffic to Searchable:
  1. Vercel → Project Settings → Log Drains → delete the drain
  2. Searchable → Agent Analytics → Setup → Tokens → revoke the token
Both sides are independent — revoking the token alone is enough to stop ingestion immediately, even if the drain stays configured in Vercel (its deliveries will start returning 401).

​
Next steps

See the data

Open Agent Analytics to see which assistants are crawling your site.

Add Search Console

Correlate AI crawls with search demand.