Drift: secrets and redacted values
cdkd avoids storing secret values in its state file, so drift cannot compare a
secret property by reading state alone. For a property that refers to a
secret, drift fetches the secret to compare it. For a property that state
holds only as the mask ***, drift cannot compare it at all. This page covers
both, and what --accept and --revert do in each case.
It applies to a stack with properties like this one:
new lambda.Function(this, 'Handler', {
// ...
environment: {
API_KEY: SecretValue.secretsManager('prod/api-key').unsafeUnwrap(),
},
});
A property that refers to a secret
CloudFormation templates refer to a secret with a dynamic reference, a string
of the form {{resolve:...}}. cdkd state stores that string in place of the
value behind it. This covers {{resolve:secretsmanager:...}},
{{resolve:ssm-secure:...}}, and {{resolve:ssm:...}} when it names a
SecureString parameter.
To compare such a property, cdkd drift fetches the secret, holds the value
in memory, and compares it with what AWS reports. A run without --accept or
--revert writes nothing back.
When AWS holds a different value, the report shows the property as drifted and hides the AWS side:
~ Handler (AWS::Lambda::Function)
- Environment.Variables.API_KEY: {{resolve:secretsmanager:prod/api-key:SecretString:::}}
+ Environment.Variables.API_KEY: ***
The most common cause is a Secrets Manager rotation: the secret has a new version and the deployed resource still carries the previous one. An edit someone made by hand looks the same, and cdkd cannot tell the two apart, so it prints neither value.
What each result means
| AWS holds at the property | Reported as |
|---|---|
| The value the reference resolves to | Clean |
| Any other value | Drifted, AWS side *** |
| Nothing, for this one property | Neither clean nor drifted |
| Nothing, for the whole block | Drifted |
The third row is a write-only credential such as MasterUserPassword. AWS
never returns it, so its absence means "cannot be checked" and cdkd does not
call that drift.
The fourth row is a whole block disappearing, for example when someone removes
all of a function's environment variables. That is reported. --accept
declines it, because accepting would delete the reference from state. Use
--revert.
Permissions drift needs
Fetching the secret needs read access to it: secretsmanager:GetSecretValue,
or ssm:GetParameter with kms:Decrypt on the parameter's key.
Without that access, or when the secret has been deleted, cdkd warns and carries on. The secret properties of that one resource are listed as not compared, and every other resource is checked as usual.
--accept and --revert on a secret property
--accept declines a drifted secret property and says so in the plan. It will
not write *** into state, and it will not write a value it cannot identify.
The drift keeps being reported.
--revert fetches the secret first, so the resource receives the secret
value. It does not receive the {{resolve:...}} text.
Finding out whether state holds a secret in plaintext
cdkd drift does not answer that question. Use
cdkd scrub:
cdkd scrub --dry-run --fail
Redacted (NoEcho) baselines
Some properties in state hold the literal mask *** with no reference behind
it. cdkd writes the mask when a value is secret but there is no
{{resolve:...}} string it could store in its place. Drift has nothing to
compare against at such a property.
How drift reports it depends on where the mask came from:
| Source of the mask | Effect on drift |
|---|---|
A NoEcho custom-resource response value used in a property |
Drifted on every run |
The Fn::Base64 encoding of a secret, such as UserData |
Drifted on every run |
A NoEcho template parameter |
Listed as noEchoParameter |
A mask that is reported as drift on every run
This applies to the first two rows. State holds the mask and AWS holds the
real value, so the two never match. The report prints *** on both sides:
~ Handler (AWS::Lambda::Function)
- Environment.Variables.TOKEN: ***
+ Environment.Variables.TOKEN: ***
A run without --accept or --revert exits 1 every time, and no deploy
clears it. cdkd reports the property because the live value is readable and
cdkd cannot say whether it is the right one.
The two flags treat the masked property like this:
--acceptdeclines it.--revertkeeps the value AWS has there.--revertrefuses the whole resource, and exits2, in two cases: AWS reports nothing at the property, or the mask sits in a list whose elements cdkd cannot match to the recorded ones.
If a stack like this has to pass a drift check in CI, you have two choices.
Stop marking the custom resource's response NoEcho, or check the --json
output and filter the known property out.
When --revert refused because AWS reports nothing at the property, the
custom resource has to supply the value again. Change a property of the custom
resource and redeploy, so that its handler runs. A nonce property is the usual
way.
A property whose real value is the string *** is treated the same way; see
Secrets in state. How
cdkd matches list elements, and the remaining refusal cases, are in
cdkd drift internals.
A mask from a NoEcho template parameter
The third row behaves differently. The resource is listed as not compared,
with the cause noEchoParameter and the path of the property. Every other
property of the resource is compared, and the exit code is not affected.
A masked position cdkd could not certify
A mask can also appear where the template did use a {{resolve:...}}
reference. A stack adopted with cdkd import can show this on its first
cdkd drift.
It comes from how cdkd records the snapshot of a resource. AWS returns the
decrypted secret, and cdkd replaces it with the reference from the template,
matching the two by position. When cdkd cannot line the AWS response up with
the reference, it writes *** so that no secret can end up in state. That
happens in these situations:
- AWS restructures or reorders the property.
- AWS changes the case of a name, such as
dbreturned asDB. - A
cdkd importrecorded a rawFn::Joinobject.
What drift reports
When the mask is the only difference at that position, the resource is listed
as not compared with the cause uncertifiedBaseline, and a run without
--accept or --revert exits 2. --accept writes nothing there, and
--revert sends AWS its own value back unchanged.
When anything else differs there, such as an edited neighbouring value or a
list that was resized or reordered, the property is reported as drifted with
the AWS side masked. --accept declines it, and --revert often refuses the
whole resource.
The rest of the resource is compared as usual. A secret that changed exactly at a masked position cannot be seen.
Clearing the mask
Run cdkd deploy, even one that changes nothing:
cdkd deploy MyStack
At the start of every deploy, cdkd reads each such resource again. Where the value AWS holds is exactly what the reference resolves to, cdkd replaces the mask with the reference.
Three limits apply:
- A secret that was rotated since the resource was last deployed keeps its mask. It clears on a deploy that creates or updates that resource.
cdkd state refresh-observeddoes not clear a mask.- A resource whose only snapshot is a raw
Fn::JoinorFn::Subobject is refused by--revert, which exits2. The samecdkd deployfixes it.
The property shapes that produce the mask, and the rules of the re-read, are in cdkd drift internals.
Tokens that are not references
A {{resolve:...}} string that cdkd does not resolve is either text that only
looks like a reference, or a reference to a service AWS added after this cdkd
release. Drift cannot compare the property. It lists the resource as not
compared with the cause unresolvedToken, and a warning names the string once
per resource. The exit code is not affected.
--revert meets such a string when another property of the same resource
drifted. It then does one of two things at the string's position: it leaves
the live value unchanged, or it writes the string literally, as cdkd deploy
would. The warning printed before the confirmation prompt says which. The
rules are in
cdkd drift internals.
Literal ssm-secure text left in AWS by an older cdkd
Older cdkd releases did not resolve ssm-secure references. A resource one of
them deployed can hold the literal {{resolve:ssm-secure:...}} text in AWS.
cdkd drift reports it for a property it can read back, and --revert writes
the resolved value. A write-only property such as MasterUserPassword cannot
be read back, so it needs one deploy that updates it.
Related
cdkd drift: the report, the exit codes and the options- Drift: what was not compared: the
noEchoParameter,uncertifiedBaselineandunresolvedTokencauses cdkd scrub: finding and removing secret plaintext in state- State Management: how state stores secret values
- cdkd drift internals: the rules behind the edge cases