If Airflow raises cryptography.fernet.InvalidToken while reading a stored connection password or Variable, first check whether it is using the key that encrypted that value. Generating a new key does not recover existing encrypted values, and resetting the metadata database should not be the first troubleshooting step. Restore the original key where possible, or use a controlled rotation if you are changing keys on purpose. This guide covers wrong-key errors, HMAC verification failures and a recovery process that keeps existing metadata.
What does cryptography.fernet.InvalidToken mean?
InvalidToken means Fernet could not validate or decrypt the supplied token.
Possible causes include:
- The wrong decryption key.
- A malformed, truncated or modified token.
- Signature verification failure.
- Token expiration when the application explicitly supplies a time-to-live value.
The exception alone does not identify the cause. Start with the operation immediately before it in the traceback.
See the official cryptography Fernet documentation (opens in a new tab) for the exception’s definition.
What does “Fernet invalid token: HMAC” mean?
An error mentioning HMAC generally indicates that the token’s authenticity check failed. The exact wording varies between libraries.
A wrong key can cause this failure, as can a changed or damaged token. An HMAC error does not, by itself, prove that data was maliciously modified.
Fernet authenticates a token before returning its decrypted contents. Do not bypass that check to make an error disappear.
If the message comes from a Node.js application or another Fernet implementation, investigate that application’s keys and token handling. Changing Airflow’s configuration will not fix an unrelated application’s encryption.
Why does Airflow fail to decrypt a connection password?
Airflow uses Fernet to protect connection passwords and Variable values stored in its metadata database.
The key is configured through:
[core]fernet_key = ...
It can also be supplied through the environment variable:
AIRFLOW__CORE__FERNET_KEY
If stored values were encrypted with one key and the process now uses another, decryption fails.
Airflow documents the configuration and rotation procedure in its Fernet guide (opens in a new tab).
For an existing environment, investigate what changed:
| When the error started | What to investigate |
|---|---|
| After rebuilding containers | Was the original key preserved? |
| After restoring a database | Was its corresponding key restored? |
| After updating a secret | Did the key change unintentionally? |
| During a rollout | Are processes using different configuration versions? |
| After copying encrypted records | Was the required key available in the destination? |
These are diagnostic questions, not confirmed causes.
Step 1: Confirm which operation is failing
Distinguish two situations:
Airflow cannot read a stored credential. The traceback points to loading a connection or Variable before the task can use it.
Task code cannot decrypt application data. The task calls Fernet on a file, message or other payload using an application-specific key.
Establish this distinction before changing configuration. Otherwise, you may alter a working Airflow key while leaving the actual error unresolved.
Record the affected task, connection or Variable name, failing component and first observed failure. Keep keys and decrypted values out of diagnostic logs.
Step 2: Identify where the credential comes from
Airflow can retrieve connections and Variables from its metadata database, environment variables or an external secrets backend.
The UI does not show every externally defined value. A duplicate definition can also cause reads to return a different source from the entry you edited.
Check the secrets-backend search order (opens in a new tab) for your version and deployment.
If a credential comes from an external system, investigate that retrieval path. Do not assume every credential error involves an encrypted database record.
Step 3: Check the effective Fernet configuration
Inspect the configuration used by the failing process, rather than relying only on a local airflow.cfg.
Airflow supports environment overrides and command-based or secrets-backend configuration for fernet_key. A correct value in the file may therefore be overridden.
Review:
- The configured source and secret version.
- Whether the deployment changed that source.
- Whether running processes loaded the update.
- Whether the process points to the intended metadata database.
Consult Airflow configuration precedence (opens in a new tab).
Apply corrections through your deployment’s configuration mechanism. An environment change in an interactive terminal does not update an already-running service.
Step 4: Recover the original key when possible
Before modifying production configuration:
- Back up the metadata database.
- Preserve the currently available keys securely.
- Identify whether new records were encrypted after the change.
- Plan validation and rollback.
Look for the original key in approved secret storage, retained secret versions or secured deployment backups.
If only the original key was used, restoring it may restore access. If records were encrypted under several keys, preserve all relevant keys before deciding how to consolidate them.
Do not overwrite the remaining working key while investigating.
Step 5: Rotate the Fernet key safely
Rotation replaces an encryption key while retaining access to existing values. It requires the keys needed to decrypt those values.
Airflow’s documented sequence is:
- Configure the new key first, followed by the old key.
- Run the rotation command.
- Remove the old key after successful rotation.
The intermediate configuration looks like:
[core]fernet_key = NEW_KEY,OLD_KEY
These are placeholders, not valid keys. Update the authoritative configuration source, including any override.
For production, use a coordinated maintenance procedure. Avoid concurrent configuration changes or writes from processes still using outdated settings.
From an authorized administration environment connected to the correct metadata database, run:
airflow rotate-fernet-key
This command re-encrypts stored connection credentials and Variables. It changes data, so confirm its behavior against your installed version in the Airflow CLI reference (opens in a new tab).
Validate representative credentials and workflows before completing the change to the new key alone. If rotation fails, investigate before removing an old key.
Docker and Kubernetes deployment checks
For containers, verify that the key survives replacement and is supplied consistently through the intended configuration source.
For the official Helm chart, review the Fernet secret behavior for your chart version. The chart’s production guide (opens in a new tab) documents using a managed secret through fernetKeySecretName and reloading components after secret changes.
Test recovery using the combination of a database backup and its required key material. Preserving one without the other can leave recovery incomplete.
For managed Airflow services, use the provider’s supported recovery process rather than modifying platform-managed secrets directly.
What if the Fernet key is lost?
A newly generated key cannot decrypt values encrypted with the missing key.
If no usable key remains, recovery must use another source of the underlying credentials or configuration. That may mean retrieving secrets from an approved store or reissuing credentials.
Work in this order:
- Inventory the affected entries.
- Recover or reissue their underlying values.
- Recreate affected entries through supported administration interfaces.
- Validate dependent workflows.
A database backup containing ciphertext is not a substitute for the missing key. Fernet’s documentation states that losing the key prevents decryption.
Should you reset the Airflow database?
Do not use a database reset as a routine Fernet fix.
Airflow’s db reset operation rebuilds the metadata database. It does not recover an encryption key. Review the command’s destructive behavior in the CLI documentation (opens in a new tab).
An intentional reset may suit a disposable development environment. For an existing environment, investigate configuration correction, key recovery and targeted restoration first.
Frequently asked questions
Does InvalidToken mean the destination database password is wrong?
Not necessarily. If Airflow fails while decrypting the stored password, it may not have reached destination authentication. Identify the failure stage before changing credentials.
Will restarting Airflow fix the error?
A restart can load corrected configuration. It cannot recover a missing key or repair damaged encrypted data by itself.
Should I generate a new key immediately?
First establish which key encrypted the existing values. Generate a new key for an intentional installation or rotation, rather than treating it as a decryption repair.
Is every InvalidToken error caused by a wrong key?
No. Inspect the complete traceback and token source. Wrong keys, invalid token contents and application-specific validation can lead to the same exception.
What if only one connection fails?
Compare that entry’s history with working entries. Check whether it was migrated, edited or encrypted under a different key before changing the whole environment.
Prevent the next Fernet incident
Maintain a controlled key source, restricted access, secured key backups and a documented rotation process. Test that your recovery procedure restores both the metadata and the ability to decrypt it.
Running Airflow in production? See how we build data platforms and the cloud infrastructure they run on.