> ## Documentation Index
> Fetch the complete documentation index at: https://docs.searchable.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Connect WordPress to Searchable
> Install the Searchable must-use plugin on your WordPress site to forward AI-crawler requests to LLM Analytics — no theme changes, no JavaScript.
## What this is
A small **must-use plugin** that runs inside WordPress, inspects the user agent of every request that reaches PHP, and forwards the ones that match Searchable's canonical AI-crawler list to LLM Analytics.
It feeds the same server-side pipeline as the Vercel, Cloudflare, and Netlify connectors — same classifier, same dashboards. The difference is where it runs: in your application, not at a CDN edge.
Must-use plugins load automatically and can't be deactivated from the admin UI, so the forwarder
survives plugin and theme updates.
## First, check this is the right path
**If your site sits behind Cloudflare, use the [Cloudflare Worker](/setup/cloudflare-worker)
instead.** It runs at the edge, so it sees every AI-crawler request — including the ones a
full-page cache serves without ever booting PHP. It works on every Cloudflare plan including Free,
and it's less to maintain.
Use the WordPress plugin when your site **isn't** behind a CDN we support natively, or when you specifically want the forwarder in the application layer.
The same caveat applies to any full-page cache. See [Troubleshooting](#troubleshooting) below.
## Before you start
Self-hosted WordPress (this does not work on WordPress.com's hosted plans, which don't allow
`mu-plugins` or `wp-config.php` edits)
SFTP, SSH, or a file manager that can write to `wp-content/` and `wp-config.php`
Your project's domain confirmed in **LLM Analytics → Setup → Confirm your domain**
LLM Analytics is available on every plan — no upgrade required.
## Step 1: Open the WordPress connector
In Searchable, open **LLM Analytics** (left nav, under Analytics) → **Setup**, then pick **WordPress** under "Connect your crawler source".
Click **Generate key**. Searchable mints a key with the `log_events` permission, scoped to this project, and shows it **once** — copy it now. Keys start with `sk_live_…`.
The key is a live write credential. Treat it like a password. If you lose it, revoke and regenerate from the same panel or **Settings → API Keys**.
The panel shows a `define()` block with your project's site token already filled in. Copy it verbatim — you'll paste it in Step 3.
## Step 2: Install the plugin
Click **Download plugin** in the WordPress panel. It downloads `searchable-ai-traffic.php`, served from your project's tracker host.
Download from the button rather than reusing a copy from a colleague. The artifact is refreshed in place on every release, so the button always gives you the current plugin — and the download is pinned to the tracker host your project actually posts to.
Create `wp-content/mu-plugins/` if it doesn't already exist, then copy `searchable-ai-traffic.php` **directly** into it.
WordPress does not load must-use plugins from subdirectories. The file has to sit at `wp-content/mu-plugins/searchable-ai-traffic.php`, not `wp-content/mu-plugins/searchable/searchable-ai-traffic.php`.
There is nothing to activate. Must-use plugins load on every request automatically, and appear under the **Must-Use** filter on the admin Plugins screen (not in the main list, and with no activate/deactivate controls).
## Step 3: Add your credentials
Open `wp-config.php` and paste the snippet from Step 1 **above** the line that reads `/* That's all, stop editing! Happy publishing. */`.
```php wp-config.php theme={null}
define('SEARCHABLE_API_KEY', 'sk_live_YOUR_API_KEY');
define('SEARCHABLE_SITE_TOKEN', 'st_YOUR_SITE_TOKEN');
```
Replace `sk_live_YOUR_API_KEY` with the key you generated. The site token is already filled in for you in the in-app snippet.
The plugin deliberately stays inert — it sends nothing at all — while either constant still
contains `YOUR_`. If nothing arrives in your dashboard, this is the first thing to check.
**Projects on a dedicated tracker hostname** see two extra `define()` lines in the in-app snippet
(`SEARCHABLE_API_URL` and `SEARCHABLE_BOTS_URL`). Don't drop them — without them the plugin falls
back to the shared host, and you'd end up downloading the plugin from one endpoint while posting
events to another. Most projects use the shared host and won't see these lines.
### Why `wp-config.php` and not the plugin file
`wp-config.php` sits outside the plugin directory, is excluded from most deployment pipelines by default, and won't leak the key if `mu-plugins/` is ever served as plain text.
### Optional constants
| Constant | Default | What it does |
| --------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- |
| `SEARCHABLE_SEND_IP` | `true` | Set to `false` to omit the (already anonymized) IP entirely. Geo enrichment then falls back to edge metadata. |
| `SEARCHABLE_API_URL` | `https://tracker.searchableanalytics.com/v1/events` | Ingest endpoint. Set automatically by the in-app snippet for projects on a dedicated tracker hostname. |
| `SEARCHABLE_BOTS_URL` | `https://tracker.searchableanalytics.com/v1/bots.json` | Canonical bot list. Same rule as above. |
## Step 4: Verify
Send a request to your site with a known crawler user agent:
```bash theme={null}
curl -A "GPTBot/1.0" https://yourdomain.com/
```
**Expected result:** the request appears in LLM Analytics within roughly 30 seconds.
1. Check the **Logs** tab first — it shows individual bot requests as they land.
2. On the **Setup** tab, the WordPress card flips from **Waiting for the first event from your WordPress site…** to **Connected — the plugin is sending events.**
Real crawler traffic follows on its own schedule. AI bots typically hit any indexed site within a
few hours, so don't read an empty dashboard an hour after install as a failure — the `curl` above
is the definitive test that the wiring works.
## How it works
* **Bot list.** The plugin fetches Searchable's canonical bot list from `/v1/bots.json`, validates each pattern, and caches the compiled list in a transient for one hour. If the fetch fails it falls back to the last good list persisted in `wp_options`, so a network blip doesn't silently disarm the filter. Failures are cached for five minutes, so an outage never adds a fetch attempt to every request.
* **Matching.** On the `shutdown` hook the plugin matches the request's user agent against that list. Non-matching requests do nothing — no outbound call, no work beyond the regex pass. WP-CLI, cron, and CLI requests are skipped entirely.
* **Latency.** On PHP-FPM the response is flushed to the crawler via `fastcgi_finish_request()` *before* the outbound POST, so forwarding adds no latency. On other SAPIs the POST is blocking with a 2-second timeout, and only ever fires for matched crawler requests.
* **IP handling.** The client IP is resolved proxy-aware (`CF-Connecting-IP` → left-most `X-Forwarded-For` → `X-Real-IP` → `REMOTE_ADDR`, each validated) and **anonymized on your server before sending** — the last IPv4 octet or IPv6 group is zeroed, so the full IP never leaves the site.
## What gets sent
Only requests from user agents matching Searchable's canonical AI-crawler list are forwarded. Human traffic is never sent.
Each matched request sends:
* HTTP method, path, and full URL (query strings are stripped server-side before storage)
* Response status code and server response time
* User agent and referrer
* An **anonymized** client IP — last octet zeroed, or omitted entirely with `SEARCHABLE_SEND_IP`
* A `source` tag of `wordpress`, so these events are distinguishable from Middleware SDK or edge-connector events
Cookies, request and response bodies, other headers, and full IP addresses are never sent.
## Optional: enhance the data
The plugin is a **crawler-log source**. It powers the Overview, Pages, AI Crawlers and Logs tabs.
To populate the **Visitors** tab — humans arriving on your site from AI answers — connect [Google Analytics 4](/integrations/google-analytics-4), [Search Console](/integrations/google-search-console), or Bing in step 3 of the Setup wizard. These are additive; they don't replace the crawler-log source.
## Troubleshooting
Work through these in order:
1. **Placeholders still in place.** The plugin no-ops while `SEARCHABLE_API_KEY` or `SEARCHABLE_SITE_TOKEN` contains `YOUR_`. Confirm both were replaced with real values.
2. **Wrong directory.** The file must be directly in `wp-content/mu-plugins/` — not `plugins/`, and not a subdirectory. Confirm it appears under the **Must-Use** filter on the admin Plugins screen. If that filter isn't shown at all, WordPress isn't seeing any mu-plugin — check the path over SFTP.
3. **Constant names.** They're case-sensitive and must be spelled exactly as in the snippet.
4. **Outbound HTTP blocked.** Some managed hosts restrict `wp_remote_post` to an allowlist. Ask your host to allow `tracker.searchableanalytics.com` (or your dedicated tracker hostname).
Full-page caches — WP Rocket, WP Super Cache, W3 Total Cache, Varnish, Cloudflare APO, or host-level caches on WP Engine and Kinsta — serve cached HTML without booting PHP. When that happens the `shutdown` hook never fires and the crawler visit is never recorded.
Two options:
* **Exclude known crawler user agents from the cache**, so their requests always reach PHP.
* **Move to an edge-level integration** — the [Cloudflare Worker](/setup/cloudflare-worker) is the usual answer, since it runs in front of the cache.
If you do neither, you'll still capture the origin-level hits that pass through on cache misses and TTL expiry — an undercount, not a zero.
You've connected Google Analytics or Search Console but no crawler-log source. The Overview,
Pages, AI Crawlers and Logs tabs need this plugin (or a Cloudflare, Vercel, or other edge source)
working.
Searchable's canonical bot list is regenerated **once daily** at 03:00 UTC, and the plugin caches its own copy for up to another hour. A newly added crawler can therefore take up to \~25 hours to start being captured.
Deleting the `searchable_bot_patterns` transient forces a refetch, but that only helps once the daily upstream publish has picked the new bot up. To see what the plugin is working from, fetch the list yourself:
```bash theme={null}
curl https://tracker.searchableanalytics.com/v1/bots.json
```
On PHP-FPM they shouldn't be — the response is flushed before the POST. If your host runs mod\_php or another non-FPM SAPI, matched crawler requests carry a blocking POST bounded at 2 seconds. Human traffic is unaffected either way, since non-matching requests never make an outbound call.
## Next steps
Open LLM Analytics to see which assistants are crawling your site.
Layer in keyword data so you can correlate AI crawls with search demand.