Using Azure Key Vault as a Secrets Backend
Bruin supports using Azure Key Vault as a secrets backend for managing connection credentials. This is controlled via the --secrets-backend flag.
Enabling Azure Key Vault
To use Azure Key Vault as your secrets backend, pass the flag:
bruin run --secrets-backend azureYou can also set the backend via environment variable:
export BRUIN_SECRETS_BACKEND=azure
bruin runConfiguring the Azure Connection
Bruin needs the URL of your vault and a way to authenticate to Microsoft Entra ID:
export BRUIN_AZURE_KEYVAULT_URL=https://my-vault.vault.azure.net
bruin run --secrets-backend azureEnvironment Variables
| Variable | Required | Description |
|---|---|---|
BRUIN_AZURE_KEYVAULT_URL | Yes | The vault URL, e.g. https://my-vault.vault.azure.net. Must use https. See Sovereign clouds for other hosts. |
BRUIN_AZURE_AUTH_METHOD | No | How to authenticate: default (the default), workload_identity, managed_identity, client_credentials, or cli. See Authentication. |
BRUIN_AZURE_TENANT_ID | For client_credentials | Microsoft Entra tenant ID. With workload_identity, overrides AZURE_TENANT_ID. |
BRUIN_AZURE_CLIENT_ID | For client_credentials | Application (client) ID. With managed_identity, selects a user-assigned identity. With workload_identity, overrides AZURE_CLIENT_ID. |
BRUIN_AZURE_CLIENT_SECRET | For client_credentials | Client secret of the app registration. |
BRUIN_AZURE_SECRET_PREFIX | No | Prepended to connection names to form secret names, e.g. bruin-prod-. See Namespacing secrets with a prefix. |
Authentication
BRUIN_AZURE_AUTH_METHOD selects one credential type:
| Value | Credential | Use it for |
|---|---|---|
default | DefaultAzureCredential | Most cases. Tries, in order: the AZURE_CLIENT_ID / AZURE_TENANT_ID / AZURE_CLIENT_SECRET (or certificate) environment variables, AKS workload identity, managed identity, the Azure CLI, and the Azure Developer CLI. |
workload_identity | Workload identity | AKS pods with Microsoft Entra Workload ID. Reads AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE, which the AKS webhook injects. |
managed_identity | Managed identity | Azure VMs, Container Apps, App Service, or AKS node identities. Set BRUIN_AZURE_CLIENT_ID to pick a user-assigned identity. |
client_credentials | Client secret | A service principal. Needs BRUIN_AZURE_TENANT_ID, BRUIN_AZURE_CLIENT_ID and BRUIN_AZURE_CLIENT_SECRET. |
cli | Azure CLI | Local development after az login. |
default already picks up AKS workload identity and managed identity, so you don't have to change anything to run on Azure. Use workload_identity or managed_identity when you want Bruin to use exactly that identity and fail clearly when it isn't available, instead of falling through to other credentials.
Credential Precedence
- An explicit
BRUIN_AZURE_AUTH_METHODalways wins: Bruin uses only that credential. client_credentialsuses only theBRUIN_AZURE_*variables, never theAZURE_*ones.- With
workload_identity,BRUIN_AZURE_CLIENT_IDandBRUIN_AZURE_TENANT_IDtake precedence over theAZURE_CLIENT_IDandAZURE_TENANT_IDset by the webhook. - With
default, the order is theDefaultAzureCredentialorder in the table above.BRUIN_AZURE_*credentials are not used, so setBRUIN_AZURE_AUTH_METHOD=client_credentialsif you want Bruin-specific static credentials to win.
Example: Service Principal
export BRUIN_AZURE_KEYVAULT_URL=https://my-vault.vault.azure.net
export BRUIN_AZURE_AUTH_METHOD=client_credentials
export BRUIN_AZURE_TENANT_ID=00000000-0000-0000-0000-000000000000
export BRUIN_AZURE_CLIENT_ID=11111111-1111-1111-1111-111111111111
export BRUIN_AZURE_CLIENT_SECRET=my-client-secret
bruin run --secrets-backend azureExample: Local Development
az login
export BRUIN_AZURE_KEYVAULT_URL=https://my-vault.vault.azure.net
export BRUIN_AZURE_AUTH_METHOD=cli
bruin run --secrets-backend azureSovereign Clouds
Bruin accepts vault URLs in the public cloud and the sovereign clouds:
| Cloud | Vault URL |
|---|---|
| Azure public cloud | https://<name>.vault.azure.net |
| Azure China (21Vianet) | https://<name>.vault.azure.cn |
| Azure US Government | https://<name>.vault.usgovcloudapi.net |
| Azure Germany | https://<name>.vault.microsoftazure.de |
For China and US Government vaults, Bruin signs in against that cloud's Microsoft Entra authority automatically. Set AZURE_AUTHORITY_HOST to override it, for example for Azure Germany (https://login.microsoftonline.de/). On AKS, the workload identity webhook sets AZURE_AUTHORITY_HOST for you. The cli method uses the cloud selected with az cloud set.
Storing Secrets in Azure Key Vault
Each connection is one secret, named after the connection. The secret value is a JSON string with two required fields:
type: The connection type (must match a valid Bruin connection type)details: An object containing the connection-specific configuration
Example: PostgreSQL Connection
Create a secret named my-postgres:
az keyvault secret set --vault-name my-vault --name my-postgres --value '{
"type": "postgres",
"details": {
"host": "localhost",
"port": 5432,
"username": "myuser",
"password": "mypassword",
"database": "mydatabase",
"schema": "public"
}
}'Example: Snowflake Connection
Create a secret named my-snowflake with this value:
{
"type": "snowflake",
"details": {
"account": "my-account",
"username": "myuser",
"password": "mypassword",
"warehouse": "my-warehouse",
"database": "my-database",
"schema": "my-schema"
}
}Connection Names
Key Vault secret names may only contain letters, digits and hyphens (0-9, a-z, A-Z, -) and be at most 127 characters long, and they are case-insensitive. Connection names stored in Key Vault must follow the same rule, including any prefix. A connection named my_postgres can't be stored in Key Vault; Bruin reports:
connection 'my_postgres' can't be read from Azure Key Vault: secret 'my_postgres' is not a valid Key Vault secret name, which may only contain letters, digits and hyphens (0-9, a-z, A-Z, -) and be at most 127 characters; rename the connection to matchRename the connection, e.g. to my-postgres, in both the secret and the assets that use it. Bruin doesn't rewrite names for you.
Namespacing Secrets with a Prefix
Set BRUIN_AZURE_SECRET_PREFIX to keep Bruin's secrets apart from others in the same vault, or to keep one set of secrets per environment in a shared vault:
export BRUIN_AZURE_SECRET_PREFIX=bruin-prod-With this prefix, the connection my-postgres is read from the secret bruin-prod-my-postgres. The prefix must only use letters, digits and hyphens, since Key Vault names can't contain /. Without a prefix, the secret name is the connection name. Using a separate vault per environment is often simpler than a prefix.
Listing Connections
bruin connections list --secrets-backend azure lists the connections stored in the vault without a local .bruin.yml:
bruin connections list --secrets-backend azure -o jsonBruin lists the enabled secrets in the vault (only those starting with BRUIN_AZURE_SECRET_PREFIX, if set) and reads each one. Disabled secrets, secrets backing Key Vault certificates, and secrets that aren't in the Bruin format are skipped.
Permissions
Bruin only needs to read secrets.
- With Azure RBAC (recommended), assign the Key Vault Secrets User role to Bruin's identity on the vault. It allows reading secret values and listing secrets.
- With vault access policies, grant the Get and List secret permissions. List is only needed for
bruin connections list.
az role assignment create \
--role "Key Vault Secrets User" \
--assignee <client-id-of-bruin-identity> \
--scope "$(az keyvault show --name my-vault --query id -o tsv)"Running on Kubernetes (AKS)
On AKS, use Microsoft Entra Workload ID to give the Bruin pod an Azure identity through its Kubernetes ServiceAccount. No client secret is needed.
Enable the OIDC issuer and workload identity on the cluster:
bashaz aks update --resource-group my-rg --name my-cluster --enable-oidc-issuer --enable-workload-identityCreate a user-assigned managed identity and give it the Key Vault Secrets User role on the vault, as shown in Permissions:
bashaz identity create --resource-group my-rg --name bruin-secrets-readerFederate the identity with the ServiceAccount the Bruin pod runs as:
bashaz identity federated-credential create \ --name bruin \ --identity-name bruin-secrets-reader \ --resource-group my-rg \ --issuer "$(az aks show --resource-group my-rg --name my-cluster --query oidcIssuerProfile.issuerUrl -o tsv)" \ --subject system:serviceaccount:bruin:bruin \ --audience api://AzureADTokenExchangeAnnotate the ServiceAccount with the identity's client ID:
yamlapiVersion: v1 kind: ServiceAccount metadata: name: bruin namespace: bruin annotations: azure.workload.identity/client-id: 11111111-1111-1111-1111-111111111111Run the Bruin pod with that ServiceAccount and the
azure.workload.identity/uselabel. The label is on the pod, not the ServiceAccount, and is what makes the webhook injectAZURE_CLIENT_ID,AZURE_TENANT_ID,AZURE_FEDERATED_TOKEN_FILEandAZURE_AUTHORITY_HOST:yamlapiVersion: v1 kind: Pod metadata: name: bruin namespace: bruin labels: azure.workload.identity/use: "true" spec: serviceAccountName: bruin containers: - name: bruin image: ghcr.io/bruin-data/bruin:latest # /project holds your Bruin project, e.g. baked into the image or mounted from a volume. command: ["bruin", "run", "--secrets-backend", "azure", "/project"] env: - name: BRUIN_AZURE_KEYVAULT_URL value: https://my-vault.vault.azure.net - name: BRUIN_AZURE_AUTH_METHOD value: workload_identity
BRUIN_AZURE_AUTH_METHOD=default also works here. workload_identity makes Bruin fail with a clear error if the webhook didn't inject its variables, for example because the pod label is missing, instead of falling back to the node's managed identity.
To use the node pool's or a VM's managed identity instead of workload identity, set BRUIN_AZURE_AUTH_METHOD=managed_identity (and BRUIN_AZURE_CLIENT_ID for a user-assigned identity).
How It Works
When you run Bruin with --secrets-backend azure:
- Bruin authenticates to Microsoft Entra ID with the configured credential
- For each connection referenced in your pipeline, Bruin fetches the latest version of the secret with that name (with
BRUIN_AZURE_SECRET_PREFIXprepended, if set) - The secret is parsed and validated according to the connection type
- Each secret is fetched once and cached in memory for the duration of the run, even when several assets use it at the same time
Troubleshooting
Invalid Vault URL
invalid Azure Key Vault URL: must be https://<name>.vault.azure.net, ...Use the vault's Vault URI from the Azure portal (or az keyvault show --name my-vault --query properties.vaultUri). It must start with https://.
Workload Identity Not Available
failed to create Azure workload identity credential; it needs AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE, ...The workload identity webhook didn't mutate the pod. Check that the pod has the azure.workload.identity/use: "true" label, that its ServiceAccount has the azure.workload.identity/client-id annotation, and that workload identity is enabled on the cluster. Pods have to be recreated after these change.
Secret Not Found
secret 'my-connection' not found in Azure Key VaultVerify that the secret exists in this vault with the connection's name (plus BRUIN_AZURE_SECRET_PREFIX, if set) and is enabled.
Access Denied
A 403 Forbidden response means the identity authenticated but lacks permission. Assign Key Vault Secrets User on the vault, or add Get and List secret permissions to the vault's access policy. Role assignments can take a few minutes to apply.