> ## Documentation Index
> Fetch the complete documentation index at: https://docs.searchable.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Send Fastly traffic to Searchable (Real-Time Log Streaming)

> Configure a Fastly HTTPS Logging endpoint to stream request logs to Searchable. No code changes — all configuration is done inside the Fastly UI.

## What this does

Fastly's Real-Time Log Streaming ships request log records to a destination of your choosing in near-real-time. We point that stream at Searchable's tracker endpoint, classify the AI bots, and drop everything else. No code changes — all configuration happens inside your Fastly service.

<Info>
  **No code changes.** All configuration is done inside the Fastly UI.
</Info>

## Prerequisites

<Check>A Fastly account on any paid plan (HTTPS Logging is available on all paid plans)</Check>
<Check>Admin or Engineer-level access to the Fastly service you want to instrument</Check>
<Check>A Searchable project with your domain confirmed</Check>

## Setup

<Steps>
  <Step title="Generate an integration token in Searchable">
    1. Open your Searchable dashboard
    2. Go to **LLM Analytics → Setup**
    3. Pick **Fastly** as your crawler source
    4. Click **Generate token**

    Copy the token now — it starts with `sa_…` and won't be shown again. You can always generate a new one if you lose it.
  </Step>

  <Step title="Create an HTTPS logging endpoint in Fastly">
    In the Fastly UI, navigate to your service:

    **Service → Logging → HTTPS → Create endpoint**

    This creates a new draft version of your service config; you'll activate it at the end.
  </Step>

  <Step title="Configure the endpoint">
    Fill in these fields:

    | Field                   | Value                                                    |
    | ----------------------- | -------------------------------------------------------- |
    | **Name**                | `Searchable LLM Analytics` (or any label you prefer)     |
    | **URL**                 | `https://tracker.searchableanalytics.com/v1/fastly-logs` |
    | **Method**              | `POST`                                                   |
    | **Content type**        | `application/json`                                       |
    | **Log format version**  | `2`                                                      |
    | **Custom header name**  | `Authorization`                                          |
    | **Custom header value** | `Bearer <your-sa_-token>`                                |
    | **Log format**          | (paste the template below)                               |

    <Tip>
      The custom header value must be `Bearer ` followed by the full `sa_…` token, with one space and no quotes. The header name must be exactly `Authorization`.
    </Tip>
  </Step>

  <Step title="Paste the log format template verbatim">
    Paste this into the **Log format** field exactly as-is. Do **not** edit any of the `%{...}` substitutions — they are Fastly VCL/log-format directives that we depend on for correct request attribution and deduplication:

    ```json theme={null}
    {
      "timestamp": "%{begin:%Y-%m-%dT%H:%M:%SZ}t",
      "client_ip": "%{req.http.Fastly-Client-IP}V",
      "method": "%{req.method}V",
      "url": "%{req.url}V",
      "host": "%{req.http.host}V",
      "status": %{resp.status}V,
      "response_time": %{time.elapsed.msec}V,
      "user_agent": "%{json.escape(req.http.User-Agent)}V",
      "referer": "%{json.escape(req.http.Referer)}V",
      "fastly_request_id": "%{req.xid}V",
      "country": "%{client.geo.country_code}V",
      "region": "%{client.geo.region}V",
      "city": "%{json.escape(client.geo.city)}V"
    }
    ```

    <Warning>
      `fastly_request_id` must use `%{req.xid}V` — Fastly's canonical per-request unique identifier. Substituting any other token (e.g. `Fastly-SOA-ID` or a custom `X-Request-ID`) will silently break deduplication on our side and events will appear to be lost.
    </Warning>
  </Step>

  <Step title="Save and activate the new service version">
    Click **Create** to save the endpoint, then **Activate** the new service version. Fastly batches log records and ships them within a few seconds; expect events to appear in Searchable within a couple of minutes once an AI bot hits your site.
  </Step>
</Steps>

## Verifying the connection

In Searchable:

1. Go to **LLM Analytics → Setup**
2. Look at the Fastly card status
3. Click **Check** if it still shows "Waiting for first event"

| Status                      | What it means                                                                                                                   |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| **Waiting for first event** | The endpoint is configured but no AI bot has hit your site yet. Typical wait is a few hours for sites that are already indexed. |
| **Connected**               | Events are arriving. The card shows the count from the last 24 hours.                                                           |

## Geo enrichment

`country`, `region`, and `city` arrive from Fastly's edge geo lookups (`client.geo.*`). If geo lookups aren't enabled on your service (the default is on for paid plans), those fields will arrive empty — everything else still works, and the integration counts as healthy. You can enable geo at **Service → Settings → Geolocation**.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Searchable shows 401 errors or the card stays 'Not connected'">
    The Authorization header is missing or wrong.

    * Make sure you added a **custom header** named exactly `Authorization`
    * The value must be `Bearer ` followed by the full `sa_…` token, with one space and no quotes
    * If you've recently revoked the token in Searchable, generate a new one and update the endpoint
  </Accordion>

  <Accordion title="Fastly reports delivery errors or 400 responses">
    The log format is likely wrong.

    * Confirm **Log format version** is set to **2**, not 1 — version 1 wraps each record in a different shape that our endpoint doesn't accept
    * Confirm the log format template was pasted exactly, including all `%{...}` substitutions. Do not remove any directive, even if it looks redundant
    * Confirm the **Content type** is `application/json`
  </Accordion>

  <Accordion title="Events look deduplicated incorrectly (e.g. many requests collapse to one row)">
    `fastly_request_id` is being populated with a non-unique value.

    * Confirm the template uses `%{req.xid}V` (not `%{req.http.Fastly-SOA-ID}V` or any other header)
    * `req.xid` is auto-populated by Fastly on every request and requires no special VCL setup — if it's empty in your logs, contact Fastly support
  </Accordion>

  <Accordion title="Status stays on 'Waiting for first event' for more than 24 hours">
    A few possible causes:

    * The new service version was never activated — go to **Service → Versions** and confirm the version containing the HTTPS endpoint is currently **Active**
    * Your domain in Searchable doesn't match the site served by Fastly (check **LLM Analytics → Setup → Confirm your domain**)
    * The endpoint is configured but no AI bot has visited yet — try visiting your site with a known AI user agent (e.g. `Mozilla/5.0 (compatible; GPTBot/1.0)`) to trigger a test event

    If deliveries are succeeding in Fastly's logs view but nothing appears in Searchable, that points to a domain mismatch or an Authorization-header issue.
  </Accordion>
</AccordionGroup>

## Removing the integration

To stop sending traffic to Searchable:

1. Fastly → **Service → Logging → HTTPS** → delete the endpoint and activate a new service version
2. Searchable → **LLM Analytics → Setup → Tokens** → revoke the token

Both sides are independent — revoking the token alone is enough to stop ingestion immediately, even if the endpoint stays configured in Fastly (its deliveries will start returning `401`).

## Next steps

<CardGroup cols={2}>
  <Card title="See the data" icon="chart-line" href="/using-searchable/visibility-tracking">
    Open LLM Analytics to see which assistants are crawling your site.
  </Card>

  <Card title="Add Search Console" icon="google" href="/integrations/google-search-console">
    Correlate AI crawls with search demand.
  </Card>
</CardGroup>