cdkd scrub internals
This page holds the detail behind cdkd scrub and its
sub-pages (cdkd scrub: what it covers,
cdkd scrub: across stacks and
cdkd scrub: findings and refusals): the matching
rules, the reasoning for each refusal, and the cases those pages summarize in
one line. It is written for someone debugging an unexpected scrub result or
changing cdkd itself. Start from the public pages; every section here is
linked from the place it expands.
Rollback-orphan records
A rollback that leaves a DeletionPolicy: Retain resource behind records what
it left, as orphans in the state file. Each record carries a whole resource
state — properties and attributes — so scrub examines it too.
Those records need their own treatment, because the sentence that opens this
section does not hold for them: an orphan's logical id may be GONE from the
template, which is what happens when you remove the failing resource from your
CDK app. There is then nothing to re-resolve for it. So scrub derives that
record's secrets from the record ITSELF — any {{resolve:...}} expression it
still holds — in addition to every secret the run learned from the live
resources and outputs.
Both sources are needed, and they cover different failures. The record's own expressions find a secret in a record the template can no longer describe. The run-wide set finds a PLAINTEXT the write side should have replaced and did not — that value carries no expression to derive from, but it is usually the same secret a live resource still references.
One gap remains, and a clean verdict does not rule it out: a record whose
logical id is gone from the template AND which holds plaintext matches neither
source, because nothing in the run knows that plaintext. cdkd diff will still
show you the record, and cdkd state show prints it.
Nested stacks
How a child's secrets are learned
A child's template consumes a secret through a parameter — {Ref: DbPassword}
— so the child alone does not say which values are secrets. scrub learns them
from the PARENT: it resolves the Parameters the parent passes the child, and
scrubs the child with those values and the secrets they came from, exactly as a
deploy hands them down. A secret is looked for in the records of the child
resources that consumed the parameter, not in an unrelated resource whose
literal merely contains the same text. This is what repairs a child record a cdkd
older than the nested-parameter redaction wrote with the decrypted secret — a
record no redeploy rewrites while the child's resources stay unchanged.
The parent's Outputs.<Name> attributes
The parent's own row mirrors each child output as an Outputs.<Name>
attribute, and on a parent record older than the redaction of nested-stack
outputs that attribute can hold the plaintext of a child output sourced from
the CHILD's own {{resolve:...}}. The parent is scrubbed before its children
and has no needle for it, so after each child's scrub scrub re-opens the
parent's record, under the parent's lock, and rewrites such an attribute to
the child's output — reported as Scrubbed N nested-stack output attribute(s) in <Parent>. It rewrites one only when the attribute is EXACTLY what the
child's output resolves to in this run, so an unrelated value is never
touched. An attribute that still holds a plaintext this run recorded but does
not match any output exactly is reported instead, and like the rewrite it keeps
--dry-run --fail red; deploying the child rewrites it. The parent's own
per-stack line says it covers the parent's own records only.
How the version purge decides
scrub rewrites state.json with a plain S3 PutObject, and cdkd bootstrap
turns versioning on for the state bucket (it skips that step for a bucket
that already existed, unless you pass --force, so confirm with
aws s3api get-bucket-versioning --bucket <state-bucket> if you supplied your
own). Where versioning is on, the rewrite makes the pre-scrub body a
NONCURRENT VERSION of the same key, readable — plaintext and all — to anyone
who can GetObject the key with a VersionId.
So after each state.json it rewrites, a real cdkd scrub purges that
key's noncurrent versions, keeping only the current object. It does
the same for the shared exports index at
{state-prefix}/_index/{region}/exports.json, once per region whose entries it
rewrote. That object is shared by every cdkd-managed stack in the region, so
its purged history includes other stacks' earlier entries; nothing is lost by
that, because the index is a derived view that cdkd rebuilds from the state
records. The index is purged after any write scrub attempted on it, failed or
not, because its history has no recovery value. A state.json is purged when
its write succeeded or may have landed: a server error other than a throttle, a
timeout, a dropped connection, or any failure the SDK reached after retrying
(an earlier attempt may have committed). A definite refusal on the FIRST
attempt, such as a precondition failure or a denied PutObject, wrote nothing,
so that record keeps its history. On an unversioned bucket there is nothing to
purge. A record still in the pre-region
layout (<state-prefix>/<stack>/state.json) is migrated by the rewrite, as
any other state write migrates it: written to the region-scoped key, the old
key deleted, and the old key's earlier versions purged too. If that delete
fails, the old key still holds the pre-scrub record as its current object, so
the stack fails with SCRUB_LEGACY_STATE_KEY_SURVIVES rather than being
reported scrubbed.
A state record scrub cannot read
A state record whose resources map cannot be read exits 2. A
hand-edited or truncated state.json can carry a resources field that is
absent, null, or not an object at all. A real run REFUSES such a record
outright, because scrub saves whenever anything changed — an Outputs change
alone is enough — and saving would replace the unreadable map with a
well-formed empty one, destroying the only evidence that the record is broken.
Under --dry-run, where scrub provably cannot write, it audits the record's
outputs instead, warns that the resource half was never examined, and still
exits 2 — ranked above --fail's exit 1 on purpose. Reporting 1 there
would name the opposite remedy ("scrub looked and found a leak — rotate the
secret") for a record scrub could not look at, and exit 0 would be the
false-clean this whole check exists to prevent, in the mode a CI gate uses.
Either way the message names the record. cdkd deploy and cdkd destroy
refuse the same map themselves, so neither can be run against it by accident.
One row of a readable resources map that is not a resource record exits 2
the same way. A row that is null, a string, a list, or an object with no
resourceType is one the rewrite scrub saves cannot handle: a null row either
stops it on a bare TypeError under the lock (when the template still
positions the row and the stack recorded any secret) or is copied into the
rebuilt map as it stands — reported clean when no secret was recorded at all,
saved back the moment another record changed; a string or number the template
positions is spread into an object and saved as one; and a row with no type is
rewritten and saved still without one. A real run refuses, naming the rows;
--dry-run drops them,
warns, audits the rest of the record, and still exits 2 through the same
audited-record refusal.
A record whose outputs map cannot be read exits 2 as well, and it is
decided separately: a record can be damaged in either container alone, so the
message names the one that is actually broken. The reasoning is the same and
the consequence is different. Scrub REBUILDS the outputs bag before saving it,
and Object.entries walks a string as readily as a map — a six-character
value comes back as a well-formed six-key map, a null one as {} — so a
real run refuses rather than laundering the record. What is at stake is not
the stack being re-created (the resource map is intact) but the shared exports
index: state.outputs is what a redeploy republishes into
cdkd/_index/<region>/exports.json, which every other stack's
Fn::ImportValue resolves against.
Under --dry-run scrub audits the resource half instead, warns that the
outputs were never examined, and still exits 2 — otherwise every
outputs-side counter is legitimately zero and the run would print
No plaintext secrets found over a bag it replaced with an empty one.
An absent outputs field is not a defect and is never refused: a record
with no outputs is one cdkd writes on purpose, and scrub round-trips it
without materializing {}.
Findings a real run reports as exit 1
A state key that renders a secret
A key counts when it renders a secret, not only when it contains one
literally. An Export.Name is a resolved, template-controlled string, so it
can carry invisible characters — control bytes, bidi marks, zero-width
joiners, and zero-width combining marks (nonspacing diacritics) — and one
placed inside a secret splits the plaintext so a literal scan misses it
while a reader of the log sees the secret unbroken. Such a key is
reported and the run exits 1.
The same holds for a secret spelled in compatibility characters —
full-width letters and digits, mathematical alphanumerics, superscripts,
ligatures, circled digits, an ideographic space — which Unicode NFKC folds to
their plain forms. A key caught only through a combining mark, a precomposed
letter standing for its decomposed spelling (the two render identically), or
a compatibility character is reported with its name withheld, since the
printed text keeps a name's own characters. Look-alike letters from another
script (Cyrillic small a, U+0430, standing for a Latin a) have no
compatibility mapping and are not detected, nor is a secret with a Hangul
jamo at its start or end, or ending in an open Hangul syllable, whose other
letters are spelled in compatibility characters, since that edge can join
a neighbouring jamo in the name into a different syllable.
If this starts firing on a state that used to pass, the key was already
leaking — the change is what cdkd can see, not what the state holds.
Rotate the secret and change the Export.Name.
A cross-stack read that could not be verified
Two shapes land here and they call for different things, so the per-read
warning above the summary is what names the remedy. One is a read cdkd
declines by design — a cross-account reference whose producer stores a
secret expression cdkd will not resolve under the consumer's credentials —
and no re-run clears it; export a non-secret value such as the secret's ARN,
or reference it from within its own account. The other is a producer whose
own outputs map cannot be read, which is repairable: fix that record
and scrub it first.
The two also exit differently, because the remedies differ. The by-design
read is a --fail finding: exit 1, and only with the flag. A producer
whose outputs map cannot be read exits 2 on its own, with or without
--fail — including under --dry-run. It means scrub could not tell
whether that producer still holds the plaintext this stack imports, which is
"cdkd did not finish", not "cdkd looked and found a leak".
Neither shape refuses the stack. Refusing would strand this stack's own plaintext over a record that belongs to another stack — possibly one the operator does not own — so scrub reports the finding, scrubs everything else, and declines to call the stack clean.
A scan abandoned part-way
The resolver stops at the first {{resolve:...}} token it cannot resolve —
a deleted SSM parameter, a secret with no SecretString, a missing
JSON_KEY, a secret that is not JSON, or an AWS rejection such as
ParameterNotFound / AccessDeniedException — and every later token in the
SAME value is then never resolved, so it contributes no needle. A plaintext
sitting behind such a token survives a scan that finds nothing. cdkd cannot
rewrite it (there is no needle to match) and does not refuse the stack, so it
is reported and counted instead. Restore the parameter or secret and re-run.
The same finding covers a reference an Fn::Sub placeholder left
unresolvable: {{resolve:secretsmanager:${Typo}-db:...}}, where Typo names
nothing the template declares (no resource, no parameter), keeps its
${Typo} (the keeping placeholder warning), so the reference is never
looked up — also when the {{resolve: around it comes from an enclosing
Fn::Join or Fn::Sub. Declare the variable, fix its name, or escape it as
${!Typo}, and re-run.
A placeholder naming a DECLARED parameter with no Default (scrub takes no
--parameters, so it cannot bind one), or a declared resource, is different:
it gets only the keeping placeholder warning. No ABANDONED line is
printed, --fail does not count it, and the stack can still print
No plaintext secrets found. A parameter that
has a non-empty Default is bound even when another parameter of the stack has none.
A scan --fail warns about but does not count
A failure is scoped to the PROPERTY that caused it — an unresolvable Ref in
one property does not stop the scan of a {{resolve:...}} in another
property of the same resource — so the warning names the property, not just the
record. It is scoped to the top-level property only: inside one property's
value, a failing key still stops the keys after it.
Each record left unscanned is named in a warning at default verbosity, and
a property that fails while carrying no reference of its own is silent, because
nothing was lost when it stopped. So a green --dry-run --fail does
not by itself mean every record was examined: read the warnings. A record cdkd
could not certify may still hold a plaintext from an older binary, and giving
the parameter a Default — or resolving the reference — is what lets a re-run
certify it. Where neither is possible, cdkd cannot certify that record at all.
Refusals
A cross-stack read that cannot be resolved
scrub learns which plaintexts to hunt for by re-resolving the template, so a
leaf that arrives through Fn::ImportValue / Fn::GetStackOutput yields a
needle only if the producer's state can actually be read. With no needle,
nothing matches, and the command would report no plaintext found over state
that may still hold it.
Such a read is therefore attempted before the main pass, and a failure refuses
the stack, naming the resource or output it could not resolve. A stack whose
producer state is unreadable — a deleted producer, a cross-account export, a
missing state file — exits 2.
A conditional import does NOT refuse when its branch is not taken: the
pre-pass walks Fn::If the way the resolver does, selected branch only.
Neither does an Fn::ImportValue inside an output that this run's conditions
SUPPRESS — such an output wrote no state key, so there is nothing behind it to
protect.
Which Fn::If branch scrub selects
Branch selection is evaluated against the template's default parameter
values. scrub takes no --parameters, so it has nothing else to evaluate
a Conditions entry with, and a condition it cannot evaluate reads as false.
When a parameter's Default or SSM value changed after the last deploy, that
means scrub can pick a branch the deploy never took. In a RESOURCE position a
cross-stack read on that branch still refuses, so the stack can be refused over
a producer that legitimately does not exist for the parameters it was actually
deployed with.
An output position is spared this: state.outputs records what the deploy
really wrote, and a key absent from it disarms the refusal. There is no
equivalent record for a resource-position branch, so the remedy is to make the
read resolvable — deploy or scrub the producer that branch names, which
cdkd scrub --all does in one run — rather than to re-run unchanged.
The same branch selection decides whether the PRODUCER's export counts as secret-bearing at all, so one selection feeds both halves of the question.
A producer that still stores the plaintext
scrub can only replace a stored plaintext with the {{resolve:...}}
expression the PRODUCER holds. A producer whose own state has not been
scrubbed yet still holds the plaintext itself, so the read succeeds, there is
no expression to write, and the consumer would be reported clean over a record
that still holds the secret.
The verdict is taken by READING the producer's own stored value, not by inspecting what the read returned: the read RESOLVES a stored expression to plaintext before handing it over, so a healthy producer and an unscrubbed one are indistinguishable from the consumer's side. The refusal names the producer and the fix.
Which exports count as secret-bearing
Whether the export is secret-bearing at all is the half of the question the stored value cannot answer — a bucket name and a leaked password are both bare strings, so refusing on the stored value alone would refuse every multi-stack app that imports anything. That half is taken from the app's TEMPLATES, and the refusal fires only when they say the value carries a secret:
- the producer declares that export from a
{{resolve:...}}expression; or - the producer RE-EXPORTS a value that a stack further up the chain declares from one.
Both arms read the export through the Fn::If branch selection above, so an
expression sitting only in a branch this run does not select answers neither
of them: a secret reachable only through that branch is not detected. An
ordinary import of a bucket name or an ARN is unaffected under either arm.
The chain arm is not a refinement. A middle stack's output IS the
Fn::ImportValue, so asking that one template alone answers "not
secret-bearing", and cdkd scrub <the stack at the end of the chain> would
then report clean over its own surviving plaintext. The walk follows
Fn::ImportValue / Fn::GetStackOutput through the synthesized templates and
terminates on a cycle by never revisiting a (stack, export) pair. For a
chain the remedy names EVERY stack in it, head first, because a middle stack
cannot store the expression until its own producer has been scrubbed.
Four shapes stay unclassifiable from the consumer's side and are NOT refused:
- a producer outside the synthesized app;
- one whose export cdkd resolved through CloudFormation rather than through cdkd state;
- a re-export whose upstream reference cannot be read statically — an
assembled export name, or an
Fn::GetStackOutputwhose stack or output name is itself an intrinsic; - one whose upstream export is declared under a name this run cannot
reproduce, which usually means an intrinsic
Export.Nameone hop up, so a stack DOES declare it and the walk simply cannot match it.
When the DIRECT producer's Export.Name is an intrinsic this run cannot
reproduce, the check widens to every output of that producer — an
over-approximation in the safe direction, and the message says so rather than
claiming the producer declares that particular key from an expression.
The asymmetry between that widening and the dropped case one hop up is
deliberate. At the root the key came from an actual read, so some output of
that producer really did answer it, and refusing over the set is the safe
reading. One hop up the key is a literal name read out of a template, so a
miss means that template does not declare it, and widening there would refuse
the consumer over an unrelated secret two stacks away in a refusal no
cdkd scrub could clear.
Assembled references and cross-region secrets
A reference the intrinsics build out of parts — an Fn::Sub placeholder
inside it, an Fn::Join that splits it — does not exist as a complete
expression until it is resolved, so scrub's region pre-pass cannot classify it
and hands it to the resolver instead of refusing. The resolver decides the
region AFTER assembly and either routes the read to the region the ARN names
or refuses it as ambiguous.
If such a reference then goes UNRESOLVED — the region refuses the read, the
secret was deleted, or an Fn::Sub placeholder it needs names nothing the
template declares — the record is reported as an
abandoned scan (see Exit codes): the stack is not summarised clean and --fail exits 1. It is a finding rather than the
SCRUB_CROSS_REGION_SECRET_UNRESOLVED refusal a complete reference gets,
because the failure belongs to one assembled value, and refusing would strand
every other secret in the stack.
Known residual: a placeholder scrub cannot bind still leaves the stack
summarised CLEAN. When the Fn::Sub placeholder inside such a reference
names a declared parameter with no Default, or a declared resource, the
reference is never looked up and the only sign is the keeping placeholder
warning. Run
cdkd scrub --verbose when a stack you expect findings from reports clean.
A reference built from a parameter has the limit described in A reference built from a parameter.
A read cdkd declines by design is a finding
The cross-account Fn::GetStackOutput of a redacted value is never resolved:
cdkd will not look up a producer account's secret with the consumer's
credentials. That read cannot be made to succeed by re-running, so refusing
the whole stack would strand every other secret in it.
Instead the stack is scrubbed for everything else, the read is reported
(N cross-stack read(s) in <stack> could NOT be verified), the summary says
so, and --fail exits non-zero — the same treatment a secret-bearing output
KEY gets. That treatment is scoped to THAT ONE read.
Other refusals the resolver raises deliberately — a stale placeholder ARN, an
unresolvable account id, an unenriched Fn::GetAtt, --strict-getatt, a
malformed Fn::Split — are all things you can FIX in the template, so they
refuse the stack (exit 2) with the resolver's own message, and a re-run
after the fix scrubs it. They are reachable here whenever the reference's
export name is built by an Fn::Sub over one of them.
SSM parameters on the diff path
On the diff and no-op comparison path cdkd still has to learn the type, so it
issues GetParameter with WithDecryption: false. A SecureString comes
back as its encrypted blob, which is never substituted, cached or persisted,
and the comparison stays expression-versus-expression. Once a reference is
known to be SecureString, later comparisons short-circuit with no AWS call
at all.
What gets redacted inside a record
Scrub rewrites the leaves that hold secret material, not whole records. Two cases decide what a leaf becomes.
Two spellings of one secret value
Two dynamic references that resolve to the SAME value — the same JSON key written once with and once without an explicit version stage, for example — must each keep their own expression, or both sites persist whichever expression was recorded last and the stack reports a spurious UPDATE on every deploy. No plaintext is exposed by that; the wrong EXPRESSION would be stored.
cdkd therefore redacts by POSITION as well as by value: each leaf is matched
against the UNRESOLVED template at the same path, so it keeps its own
expression. Where the template leaf is an intrinsic — Fn::Join / Fn::Sub,
what CDK emits whenever the secret's ARN is a Ref, so the common case — the
reference is identified by the shape of that intrinsic instead.
A narrow residual remains, and it degrades to the old behaviour rather than to
anything worse. When the intrinsic's literal parts cannot tell the two
references apart — the part that differs is itself behind a Ref — cdkd
declines to guess and both leaves persist the same expression. The same
applies to a pair of {{resolve:ssm:...}} references whose parameter Type
AWS did not report. If you hit a spurious UPDATE on a resource holding two
spellings of one secret, make the differing part a literal in the template.
Secrets inside a list
A reference nested in an array — an ECS task definition's
ContainerDefinitions[].Environment[] is the usual shape — is redacted like
any other leaf, including on a resource that did not change on that deploy.
cdkd matches list elements by their identity field (Name / Key) rather
than by position, which does not depend on the order AWS returns them in.
Elements that carry no such identity field are left alone rather than guessed
at, so nothing is ever written onto the wrong element; a redaction cdkd cannot
place falls back to matching by value.
A secret whose value is itself a reference
Such a value is byte-identical to an already-redacted expression, so cdkd asks a narrower question: does the reference it is being compared against describe the SAME generation of the resource? Only a resource's own stored properties can answer yes; a template never can, whichever command is running.
When the answer is no, the leaf is matched by VALUE instead — so a plaintext
cdkd resolved this run is still replaced by its own reference, while a
reference already in state is left exactly as it is. A legacy leaf of this
shape is cleaned the next time either cdkd deploy or cdkd scrub resolves
that secret.
Fragments inside a complete reference
A stored value can hold a reference inside surrounding text —
jdbc://appdb:{{resolve:secretsmanager:appdb/creds:SecretString:password}}@host
is what a joined connection string looks like after redaction — and a later
deploy can record a secret whose plaintext (appdb) also occurs inside that
reference's own text. Rewriting it there would produce
{{resolve:secretsmanager:{{resolve:ssm:/app/dbname}}/creds:...}}, which no
service can resolve: cdkd rollback reads it as a request for the secret id
{{resolve:ssm:/app/dbname and either refuses or applies the wrong value.
cdkd therefore replaces every match of a recorded secret EXCEPT one that lies
wholly inside a complete reference and is shorter than it. "Reference" means a
secretsmanager, ssm or ssm-secure one: a {{resolve:...}} of any other
service is not something cdkd resolves, so a secret inside it
({{resolve:<secret>}}, which an Fn::Sub placing a secret where the service
name goes produces) is replaced like any other text. A stored secret
whose own value IS a reference is still replaced, and so is one that CONTAINS
a whole reference plus surrounding text — dropping those would leave the
plaintext in state, which is worse than the mangling this rule prevents. An
embedded secret in ordinary text is repaired.
Three limits are worth knowing, all of them narrow. A STRAY {{resolve: — an
opener that is not part of a real reference — is read by the same grammar the
resolver uses, and which way it falls depends on what follows it in that same
value:
- With no later
}}it is not a reference at all, so it protects nothing and a secret after it is still replaced. Leaving the plaintext there instead would hide it behind two characters any string can contain. - With a
}}anywhere later, the opener and that}}bracket one region, and when the opener spells one of the three services above ({{resolve:ssm:...) a secret inside it is left alone. This is the one shape where cdkd redacts less than a naive value match would. Reading the braces differently is not the fix: that would disagree with the resolver about the same string, and would re-mangle values an older cdkd already mangled. Such a value cannot come from a template — it would fail to resolve at deploy time — so the way it arrives is a drift read-back (observedProperties), which is arbitrary text from AWS.
And a value already mangled by an older cdkd is not repaired: it parses as a
valid reference now, so neither a redeploy nor cdkd scrub rewrites it.
Fixing such a record means editing it out of state, or redeploying the
resource so the leaf is written afresh.
A reference you have edited but not deployed
Such a reference is never rewritten — by cdkd scrub or by cdkd deploy. If
state holds ...:AWSPREVIOUS and the template now says ...:AWSCURRENT,
scrub leaves the record alone and reports nothing to scrub, and a deploy that
fails and rolls back leaves the reverted reference in place.
Rewriting either would make the next cdkd deploy compare the new expression
against itself, see no change, and never push the edit to AWS — a credential
rotation that silently never happens, invisible to cdkd drift because the
baseline would have been rewritten too. The same rule covers the drift
baseline: observedProperties keeps the reference AWS was last seen holding,
so cdkd drift --revert cannot push an undeployed one.
A value AWS reports at a position your template does not name
The drift baseline in observedProperties is whatever AWS returned, so it
routinely carries fields the template never set and list elements the template
does not have — and a secret can land in one of them: a copied environment
variable, an entry AWS added. Those positions have no template leaf to match
against.
cdkd takes the plaintext it learned at a position it COULD match — the same secret's own leaf, elsewhere in the same record — and replaces the remaining occurrences of that value in that record with the same reference. Positions the template already accounted for keep the answer the template gave them, so this only ever ADDS a replacement.
Nothing is fetched and no extra permission is needed: the value comes out of the read-back cdkd already has. A value that is NOT one of those secrets is left exactly as AWS reported it, so the baseline still describes the live resource.
How a NoEcho parameter value is masked
A NoEcho parameter's value is masked the way a cdkd deploy stores it (see
version: 11 stores NoEcho values as ***):
- By position. Every property today's template fills from a
NoEchoparameter holds***, whatever the value's type or length, and the record names the position innoEchoLeaves. So does the observed baseline there. A record that already names its positions keeps them, as a deploy does. A nested child's parameter its parent's row fills from aNoEchosource counts as one too. One whose everyNoEchoread sits inside anFn::Ifcounts whichever branch reads the source, because scrub cannot tell which branch the deploy took (and so does a grandchild's parameter filled from it): the child stores***there, but a plaintext it held there is not masked elsewhere in the record (it may be the other branch's literal). A position marked this way when the deploy took the plain branch reads back from AWS until the next deploy rewrites it. - By the stored value. Where the record still holds a plaintext at such a
position (a stack deployed before state
version: 11, or under an olderDefault), that value is masked wherever else the same record holds it, a leaf embedding it included (4 characters or more). Another record holding the same literal is left alone, and so is a value only anFn::Ifrow parameter positions. - Attributes. An attribute of the same name as such a property that
equals its value (an SSM parameter's
Value) is masked at any length and declaredNoEcho, so a resource reading it throughFn::GetAttis positioned too, whatever order the records are in. - Outputs. A declared output the parameter serves holds
***, and so do its own export alias and any key no other output publishes that holds the same stored value. The exports index entry is converged onto***. - Not found: a value at a position a record that names no positions no
longer reads. A
cdkd deployof the stack masks it.
Stack outputs
Stack outputs are scrubbed too, including an output you have since DELETED.
A stored output key today's template can still name — a declared output, or an
Export.Name alias this run can fully resolve — is redacted by POSITION
against that template, like any resource property.
A key the template can no longer name is repaired whenever its stored value MATCHES a secret plaintext recorded anywhere in this run, including one only a RESOURCE still references, which is the usual shape after deleting the output that used to expose it.
A deleted output is the motivating case but not the whole population: any
stored key this run cannot COMPUTE is repaired the same way. The standing
example is a parameterized Export.Name — scrub takes no --parameters, so
a name that resolves to a literal prefix-${Foo} here leaves the real alias
key your deploy wrote unaccounted for on every run, and that key is
value-matched rather than positioned for as long as the parameter stays
unresolvable. Declare the export name literally, or give the parameter a
Default the template resolves from, if you want that key positioned instead.
A key the template can no longer name
A stored key the template can no longer name, whose value no pass rewrote, is
removed from the outputs a scrub writes. Its value is one scrub cannot
identify — most often the output was deleted, and the value may be a plaintext
an older cdkd stored — and left beside the {{resolve:...}} references scrub
writes, it would be read by cdkd diff as part of a redacted record and
printed on the output's removal row. A deploy of today's template does not
write the key either. Each dropped key is named, never its value:
Dropped 1 output key(s) from MyStack that its template no longer declares: OldDbUrl. ...
--dry-run says Would drop, and counts the stack as one it would scrub, so
--dry-run --fail exits 1 until a real run (or a deploy) removes the key. A
name that holds a secret, or the key's own stored value, is masked or withheld;
a name that may be an export alias and carries a character an output's logical
id cannot is withheld outright, as cdkd diff withholds it.
An Export.Name a deploy now refuses because it holds or reads a NoEcho
parameter's value is never published, so its missing key does not make every
other key a possible alias. An alias key an older cdkd published under such a
value is reported as a key that renders a secret (scrub cannot rewrite a key;
a deploy drops it).
A dropped export alias leaves the record's export set too; its entry in the
exports index is reported as a name state.outputs no
longer holds, and a redeploy rewrites the index.
The drop runs only when this run recorded at least one secret for the stack — the pass that rewrites the outputs at all. A stack whose template resolves no secret is not rewritten, and its undeclared keys stay as they are.
These keys are kept:
one a pass rewrote, whole or in part — it now carries the reference, and the exports index entry of that name is converged to it. Text a part-rewritten value keeps beside the reference (
postgres://u:{{resolve:...}}@host) is withheld bycdkd diffitself;one no string of which can be a plaintext — only whole
{{resolve:...}}references, or no string at all;one whose name holds a secret this run recorded — the state KEY leak scrub reports and cannot rewrite, because the exports index still publishes that name;
one that may be a live export alias whose name this run could not reproduce. A key the record lists as an export (or any key, for a record written before cdkd recorded which keys are exports) is dropped only when every
Export.Namein today's template resolved to a key the record holds and lists as an export (a literal name that collides with another output is exempt, since a deploy never publishes it; on a record with no export list, or one listing anything but names, an intrinsic name matching a declared output name proves nothing). Otherwise — a parameterized name whoseDefaultor SSM value changed after the last deploy, one that does not resolve here, or an export the last deploy did not write — scrub cannot tell that alias from a deleted one, keeps the key, and warns:... were LEFT as they are. Such a key's value can still be printed bycdkd diff, so the stack is not reported clean and--failexits1; a deploy rewrites the outputs;one another stack still reads. Before dropping, scrub reads every state record in the bucket once per run. A key another stack records reading, with
Fn::ImportValueorFn::GetStackOutput, is kept and named with that stack. Every key the stack would drop is kept when such a read's name or producer is stored redacted or damaged and cannot be compared, and when another record predates the field that records such reads (importsbefore schema v4,outputReadsbefore v8). Dropping it would break the read. The rest of the record is still scrubbed, the stack is not reported clean, and--failexits1. Stop the consumer reading it (or declare the output again) and deploy, then re-run.This protects the reads cdkd recorded, and no others. Two residuals of the version test: it misses a record older than schema v8 that a command other than
cdkd deployhas rewritten since (cdkd scrub,cdkd drift --accept,cdkd import, thecdkd statecommands), because every write stamps the current version while carrying nooutputReads— such a record reads as having no readers; and it over-refuses, because one old record ANYWHERE in the bucket, related or not, keeps every stack's drop candidates, and--failred, until that record is redeployed.
When the listing or any record cannot be read — a listed record that reads as
absent counts — no key is dropped from that stack, the rest of it is still
scrubbed, and the run ends with
SCRUB_DROPPED_OUTPUT_READERS_UNVERIFIED (exit 2, with or without --fail).
Cross-stack read names
When a stale name is reported
A name whose secret has since been rotated is left as it is. Scrub learns which plaintexts to look for by re-resolving your template, so it holds the secret's CURRENT value, while the stored name holds the one it had when that record was written — a value nothing in the run can see. Rotating again does not help; a deploy that updates the stack rewrites both lists from the reads it performs. A deploy that finds nothing to change keeps the stale entry.
Scrub still reports such a name, so the stack is not called clean and
--fail exits 1. The warning names the list and index
(state.imports[1]), never the stored value. A stored entry counts when all
of these hold:
| Condition | Why |
|---|---|
| No read in today's template produces the same entry | An ordinary import this run re-read is healthy, even from a producer that also publishes a secret. |
| A read in today's template of the same producer and region has a name that carries a secret | The stored entry has to be tied to a secret-bearing reference. |
| The stored name matches that read's name, each secret reference standing for any text, and one such position still holds text | A rotated name differs only where the secret sits; with two secrets and one rotated, the old one stays text. |
An entry left behind by a reference you REMOVED from the template can also match, when its name happens to have the same shape as a secret-bearing read of the same producer. It is reported the same way; a deploy that updates the stack drops it.
A name that is never re-resolved is the case this repair exists for: a stable stack resolves its cross-stack reference once and the resource never changes again, so no later deploy would rewrite it.
As everywhere else in this command, repairing a record does not un-expose a value that was already stored in plaintext. A real run purges the rewritten record's earlier versions, within the limits listed in What a real run removes, and what it cannot. Rotate the secret.
What scrub deliberately does not do
state.outputs is re-applied VERBATIM to other stacks — cdkd's exports index
and every Fn::ImportValue / Fn::GetStackOutput read it — so a value
rewritten that was never a secret would ship a literal {{resolve:...}} token
into a CONSUMER stack's AWS call. Hence three rules:
- It never guesses. When nothing this run recorded that plaintext — the secret was deleted, rotated away, or its reference is gone from the template as well — the value is never rewritten onto a reference and no key is invented. A key the template still names is left exactly as it is; one it cannot name is dropped, as above, which ships no token to a consumer — a key another stack reads is kept. ROTATE the secret and redeploy; that rewrites the record. A degenerately short plaintext, under 4 characters, is excluded from the WIDENED match specifically: it is never used as a cross-resource needle, since it would match unrelated values. A key the template still names is unaffected — it is redacted by template POSITION, so a short secret stored there is still repaired.
- It matches inside a value, and that has a stated cost. A recorded
plaintext of 4 characters or more is repaired even when it is EMBEDDED in a
longer stored value, which is what repairs a connection string built around
a password — the shape this exists for. The consequence is that a short,
word-like secret (
adminas asecretValueFromJson('username')) occurring inside an unrelated key the template can no longer name is rewritten too, and a consumer importing that key then receives a literal{{resolve:...}}token. The trade is deliberate: that failure is loud and fixable on the next deploy, whereas an unrepaired plaintext under a key no template declares is silent and no redeploy ever clears it. If it bites, edit the key out of the state record — and rotate the secret, which was exposed either way. - It does not widen the match for a key the template still names. Those
keep their template position, so one resource's secret value can never
rewrite a declared output's coinciding literal into that resource's
reference. The residual: a declared output whose template value no longer
resolves a secret, but whose STORED value is still the stale plaintext of
one, is not repaired by
scrubeither — a redeploy rewrites it.
The exports index pass
cdkd scrub runs one pass over that object per region it touched, as a step
after the state.json write for each stack. For every entry the index already
holds whose producerStack / producerRegion name a stack this run scrubbed:
- when
state.outputsholds a value under the same name, that value contains{{resolve:or carries the redaction mask***(any masked output, such as one aNoEchoparameter serves), and the entry's value differs from it, the entry is rewritten to the state value. Until it is written, the entry is a finding:--dry-run --failexits 1 over it; - when it does not differ, nothing is written.
The rule is a comparison against the state record, not against the plaintext
scrub resolved. scrub builds its plaintext map by resolving the template's
references against Secrets Manager / SSM, so the map holds the secret's value
AS IT IS NOW; an entry written before a rotation holds the value as it was
THEN, and the two are different strings. Comparing against state.outputs
instead reads the value a redeploy writes, which is the same value whether or
not the secret has rotated since.
--dry-run performs the read and issues no write. The audit is a real read
because scrub's map comes from live resolution rather than from the state
record, so cdkd scrub --all --dry-run --fail exits 1 on a divergence here
even when every state.json already holds the expression.
What the index pass reports rather than writing
Three cases produce a message and no write. The first exits 2; the other two
do not affect the exit code, with one exception noted under the second.
- An entry it could not write. S3 refused the
PutObject, or a concurrent writer exhausted the If-Match retry budget. The run fails withSCRUB_EXPORT_INDEX_INCOMPLETE, naming each entry and its region, because such an entry keeps the value it holds. An entry whose name holds a secret this run recorded, or carries any character outside printable ASCII, is named as withheld rather than printed. Re-run the same command once the cause is cleared: an entry already matchingstate.outputsis left alone, so the re-run writes the remainder. - An owned entry whose name is not a key of
state.outputs. There is no value to converge it to, so it keeps what it holds. Redeploy that producer, which rewrites the index from its own outputs. When that entry's value still holds a secret this run recorded — typically after scrub dropped the key — it is a FINDING instead: the stack is not reported clean and--failexits1. An alias-shaped name this run dropped or kept is withheld on these lines, as it was on the drop line; and an absent entry's name that carries a character an output's logical id cannot is withheld on every run, even once the template no longer references a secret (the entry is still the one an earlier drop left). The stack and region still print. - An entry published by a producer this run did not scrub.
--alltargets every stack in the SYNTHESIZED APP, not every stack with a state record, and one bucket and region are legitimately shared by several CDK apps. Those entries are reported as coverage and never fail the gate — a gate that reddened on another app's entries could not be cleared from here. Coverage composes: each app clears its own entries by running its owncdkd scrub.
What the index pass does not do
- It changes no membership. Only the value of an entry the index already
holds is rewritten; no name is added and none is removed.
scrubtakes no--parametersand reads a state record whose template may not be the one that produced it, so any export SET it derived would be a guess. - Its write needs no additional IAM permission. The index key sits under
the same prefix as the state records, and
cdkd deployalready writes that exact object, so any principal that can deploy the stack can write it. A policy hand-narrowed to{state-prefix}/{stackName}/*breaks here — and already breaks cross-stack deploys for the same reason. Purging the index's earlier versions needs the same two version grants as thestate.jsonpurge, and warns rather than fails without them. - It does not widen
--all. A stack outside the synthesized app has no template in reach, so scrub could not learn which of its values are secrets even if the entry were targeted.
A region whose index was rewritten here has its earlier versions purged — see
What a real run removes, and what it cannot,
which covers exports.json as well as state.json.