Secrets & credentials

Use a secret to give a connector access to a data system. Choose whether its value lives on the leader or on the worker host, then reference it in your configuration.

Create and rotate a secret

  1. Open Admin → Secrets and choose + Add secret. Enter Name and Description. Only an administrator creates or changes a secret; an editor picks a stored secret by name in a source or destination form.
  2. Choose Value lives on: The leader stores an encrypted value. The worker host reads the value from an environment variable on each worker. You cannot change where the value lives after creating the secret.
  3. For a value stored on the leader, supply Value. Then select Add. You cannot read a value back afterward.
  4. Reference the secret in the relevant connector configuration and review/deploy the branch. On Status, check for missing-credential warnings.
  5. To rotate a leader-stored secret, select New value, enter it and choose Store. Review the current branch and any reference/version changes, deploy, and confirm the Running column on Admin → Secrets. Stored does not mean deployed.

Check tests a candidate value without revealing the stored value. Keep old external credentials valid until deployments and rollback requirements no longer need them. Keep secrets that deployed configurations still use. Change a secret through New value; replacing the sealing key would stop the leader from decrypting existing values.

A stored value belongs to no branch. The leader keeps the newest values of each secret, as many as leader.secrets.versions allows, whichever branch you were working on. Deleting a branch does not remove a value stored while working on it: an older value it pushed out is gone, and a rollback to a deployment that used that value fails, unless a running deployment still uses it.

Connection identities

A connection identity contains the certificates a worker trusts when connecting to a data system and, when needed, the client certificate it uses to identify itself. Create it under Build → Resources → Connection identities, then select it in the source or destination's identity field.

  • For a private certificate authority (CA), supply its public certificates in PEM format (text files containing certificate blocks). A source or destination trusts them in addition to the public roots: Mozilla's and the worker host's own.
  • To trust only your own authorities, turn off trust public roots under Advanced. A receiver always admits by its identity's authorities alone.
  • For client authentication, supply the client certificate chain and a secret reference for its private key. Provide the certificate and key together.
  • An identity can contain trust authorities, a client certificate, or both. With no identity selected, the connector trusts the public roots and presents no client certificate.

Review and deploy the identity with the connector that uses it. Check certificate names and secret availability from each worker. You configure leader/browser certificates separately in Certificates & TLS.

Kafka security

The same security controls apply to Kafka sources and Kafka destinations.

  1. Select the broker's security protocol. SSL uses TLS; SASL_SSL adds SASL authentication over TLS. PLAINTEXT is the default when you set no protocol.
  2. For SASL, select the mechanism your brokers use. PLAIN and SCRAM need a username and password. OAUTHBEARER uses the OIDC token endpoint, client id, client secret, and optional scope.
  3. Select named secrets for password or client-secret fields. Make worker-host credentials available in every worker process that uses the connection.
  4. Select a connection identity for private trust authorities or a client certificate. An identity requires a TLS protocol, and you must not duplicate its TLS settings in the extra properties.

Kerberos (GSSAPI) requires a build that supports it. Configure the service name and principal; a Linux keytab path must exist on each worker host. A Windows worker uses its own ticket cache.

For example, you can add these fields to a Kafka source or destination. Set the worker-host environment variable KAFKA_PASSWORD to the password and replace the username. The broker must support SCRAM-SHA-512 over TLS:

{
  "properties": {
    "security.protocol": "SASL_SSL",
    "sasl.mechanism": "SCRAM-SHA-512",
    "sasl.username": "smeltr"
  },
  "credentials": {
    "sasl.password": "KAFKA_PASSWORD"
  }
}

Strings in credentials name environment variables; they are not literal passwords. Select a leader-stored secret in the form to use a versioned secret reference instead. Keep ordinary settings in properties.