# HTTP receiver

Takes what senders POST to a path on its port, and answers each request once every destination has its records.

A webhook, a log shipper or a script sends to it. Every worker of the group listens on the receiver's port, and the sender, or a balancer in front of the workers, picks one worker for each request. More workers take more.

## Example

Takes what a CI system POSTs to `/hooks/ci` on port 8088, one record per line, over TLS with the [identity](/docs/secrets#identities) `receiver-tls`. Senders present the token held in the worker-host variable `HOOK_TOKEN`.

```
{
  "id": "ci-hooks",
  "type": "httpReceiver",
  "port": 8088,
  "path": "/hooks/ci",
  "tokens": {
    "ci": "HOOK_TOKEN"
  },
  "identity": "receiver-tls",
  "records": "lines"
}
```

![The HTTP receiver form on the Sources page: id ci-hooks, port 8088, path /hooks/ci, records one per line, no tokens yet.](/http-receiver-source-form-light.png)

The same source in the Sources page's add dialog, before you add its token and identity.

A sender posts a file of lines like this:

```
curl --data-binary @build.log \
  -H "Authorization: Bearer $HOOK_TOKEN" \
  https://workers.example.com:8088/hooks/ci
```

## Before you start

-   Pick a port no other receiver of the group listens on.
-   Decide how senders prove who they are: a token, a client certificate, or both. See [Tokens and certificates](#senders).
-   Put each token in a [credential](/docs/secrets).
-   Open the port on every worker of the group, and give senders one address that reaches them. See [Open its port](#port).

## Settings

It also takes the settings [every source](/docs/sources#settings) takes.

`port`  integer required

The port it listens on. Every worker of the group binds it, so no two receivers share one.

Examples:

- `8088`
- `9880`

`path`  string default `/`

The path senders POST to. It starts with `/` and holds no `?`, `#`, `:`, `*` or space. Not `/health`, where it answers health checks.

Examples:

- `/hooks/github`
- `/ingest`

`tokens`  map default none

Each token's name to the [credential](/docs/secrets) holding it. A sender presents one as `Authorization: Bearer <token>`, and any other request is refused `401`. A receiver with no token needs an `identity` with authorities.

Examples:

- `ci = HOOK_TOKEN`
- `github = GITHUB_HOOK_TOKEN`
- `jenkins = JENKINS_HOOK_TOKEN`

`records`  string default `body`

Where a request body's records are.

`body`

Gives the whole body as one record.

`lines`

Gives one record per line.

`array`

Gives one record per element of the JSON array at `recordsAt`.

`recordsAt`  string default the body is the array

The [JSON pointer](https://www.rfc-editor.org/rfc/rfc6901) to the array `array` reads. A body with no array there is refused `400`. Only with `records` `array`.

Examples:

- `/events`
- `/records`

### Advanced

`identity`  identity default plain HTTP

The [connection identity](/docs/secrets#identities) it serves TLS under: its certificate and key. With authorities, a sender must present a certificate that chains to one of them.

Example:

- `receiver-tls`

`maxRequestBytes`  integer default `10485760`

The largest request body it reads, between 1024 and 67108864. A larger one is refused `413`. A body sent as gzip is held to it as it inflates.

Examples:

- `1048576`
- `67108864`

## Fields it gives

Each record holds its text exactly as the sender wrote it. `__source.remote` holds the IP address the request came from: the balancer's, when one is in front of the workers.

| Field | Type | Holds |
| --- | --- | --- |
| `__source.name` | `str` | The name of the source that read the record. |
| `__source.type` | `str` | The kind of source that read it, its `type`, such as `kafka`, `s3List`, `httpApi` or `hecReceiver`. |
| `__source.remote` | `str` | The address of the sender that delivered the record to a receiver. |
| `__source.bytes` | `bytes` | The record's payload as the source read it, before any decoding. The source decodes `_raw` from it. A `bytes` destination writes it unless the destination names another field. Text functions refuse it; `decode(__source.bytes, encoding)` reads it as text. |

## Tokens and certificates

Whoever a receiver admits puts records into the group's pipelines, so a receiver needs `tokens`, an `identity` with authorities, or both. A receiver with neither is refused at load.

-   **Tokens:** a sender presents one as `Authorization: Bearer <token>`. Any other request is refused 401 before any of its body is read. Without an identity, the token crosses the network in the clear, which suits a receiver behind a balancer that ends TLS for it.
-   **Client certificates:** under an [identity](/docs/secrets#identities), the receiver serves TLS with the identity's certificate and key. When the identity has authorities, every connection must present a certificate that chains to one of them, health checks included.

With both, a sender presents a certificate and a token.

## What it answers

The receiver answers 200 only once every destination has the request's records, so a sender that retries an unanswered request loses nothing.

**200**

Every destination has the request's records.

**503** with `Retry-After`

The receiver is full, [blocked](/docs/delivery#hold), or stopping. It answers a new request at once, and a stopping receiver answers every request it holds. A sender may send it again.

**401**

The request presents none of the receiver's tokens.

**400**

The body does not read as `records` says, such as one with no array at `recordsAt`. It is refused whole: none of its records is taken.

**413**

The body, or what it inflates to, is larger than `maxRequestBytes`.

**415**

The body uses a `Content-Encoding` other than `gzip` or `identity`. The receiver inflates a gzip body itself.

**408**

The sender did not send the whole body within 30 seconds.

## Health check

`GET /health` on the receiver's port answers 200 while the receiver takes requests and 503 while it is full, blocked or stopping. It needs no token, and says only whether the receiver takes requests. Point a balancer's health check or a readiness probe at it, so a busy worker gets no requests until it can take them. Under an identity with authorities it needs a client certificate like any other request.

## Open its port

Every worker of the group listens on the receiver's `port`. Open that port to senders on each worker, and give senders one address that reaches all of them, such as a balancer's. On a Unix host, a request sent while a worker deploys a new configuration waits for it and is answered by the new one.

-   On Kubernetes, add the port to the worker pods and a Service in front of them: [Expose a receiver](/docs/enroll-worker/kubernetes#receiver).
-   On a host or in a container, open the host's firewall or publish the container's port: [Enroll a worker on a host](/docs/enroll-worker/host).

## Verify

POST a record you can recognize, as in the [example](#example). The receiver answers 200 once every destination has it. Find it at the destination.

## Troubleshooting

**The sender cannot connect**

Check that the port is open on every worker and that the sender's address reaches them. An identity with authorities refuses a connection with no certificate that chains to them.

**Every request is answered 401**

Check that the sender presents `Authorization: Bearer <token>` with a token one of the receiver's credentials holds.

**Every request is answered 503**

Check the source's status. A blocked receiver takes nothing until its destinations take records again. A full one takes more with more workers.

**The same records arrive twice**

A sender that hears no answer may send the request again. Give the sender a timeout longer than its destinations take to deliver.

## See also

-   [Sources](/docs/sources)
-   [HTTP API source](/docs/sources/http-api)
-   [Credentials](/docs/secrets)
-   [Delivery](/docs/delivery)