2026-08-21 · RESEARCH · The Offensive Technology team

Deterministic CBOR is not deterministic across libraries

Deterministic CBOR is supposed to guarantee that one value has exactly one byte encoding. We ran the same 3,061 values through five widely used CBOR libraries in their deterministic or canonical mode, and 26.3% came out with more than one distinct canonical encoding. On 40% of the values the libraries could not even agree on whether the input was already valid canonical form. That matters because COSE and CWT signatures are computed over the encoded bytes, so when a value is signed as canonical on one library and re-encoded as canonical on another, the two can disagree on the bytes and the signature fails to verify. The sharpest case: one library, under a function literally named encodeCanonical, silently rewrites the floating-point value 2.0 into the integer 2, changing the bytes a verifier will hash.

What deterministic CBOR is supposed to do

CBOR, the Concise Binary Object Representation defined in RFC 8949, is the compact binary cousin of JSON that underpins COSE, CWT, WebAuthn/FIDO2, and a growing pile of IoT and supply-chain formats. Plain CBOR lets the same value be written many ways: an integer can be padded into more bytes than it needs, a map's keys can appear in any order, a float can be stored in half, single, or double width. That flexibility is fine until you need to hash or sign a value, because a signature is over bytes, and if the bytes can vary, the signature is meaningless.

Deterministic encoding, described in RFC 8949 Section 4.2, is the fix: a set of rules that collapse every value to one and only one encoding. Older material calls this canonical CBOR; the current word is deterministic, and the two mean the same thing. The core profile requires four things: shortest-form integers and lengths, definite lengths only, shortest-form floats, and map keys sorted in bytewise order of their encoded form. Get all four right and, in principle, any two encoders emit identical bytes for identical values. The whole value of the scheme is that word identical. So we tested it.

What we measured, and why a skeptic should trust it

We built a differential oracle: one shared set of test values, fed through several independent CBOR libraries in their deterministic or canonical mode, comparing what each one produced. This is the same method we used to tabulate cross-library TLS behavior under FIPS. It needs no third party and touches no one's infrastructure; it is our own code, in our own throwaway lab, encoding numbers we made up. The libraries appear only in a neutral compatibility table, the way a browser compatibility chart names browsers.

The five implementations, chosen to span languages and to include a strict dCBOR baseline:

The corpus was 3,061 values: 61 hand-crafted edge vectors covering ten known decision points, plus 3,000 randomly generated well-typed values from a fixed seed so the run reproduces exactly. For each value and each library we recorded two things: the canonical re-encoding as hex, and a verdict, whether the library accepted the input as already-valid canonical, decoded it as non-canonical, or rejected it outright. Then we diffed. The method is deliberately dumb, which is its strength: it does not judge who is right, it only counts where independent implementations, all claiming to be canonical, disagree.

The result: 26% of values, more than one canonical encoding

Across the 3,061 values, 806 (26.3%) produced more than one distinct canonical encoding among the five libraries, and 1,223 (40.0%) produced more than one accept-or-reject verdict. The random corpus alone, ordinary values a real application might serialize, split on encoding 26.2% of the time. This is not an edge-case-only phenomenon; a quarter of everyday values encode differently depending on which canonicalizer you hand them.

The hand-crafted vectors show where the disagreement concentrates. Each row below is a known deterministic-encoding decision point, and the percentages are how often the five libraries split on the encoding and on the validity verdict.

Read the last three rows carefully, because they are the subtle ones. On integer minimality, indefinite lengths, and trailing garbage, the libraries that produce an encoding all produce the same encoding, so the encoding-split column is 0%. But they sharply disagree on whether the input was acceptable in the first place: 60% to 80% verdict splits. One library shrugs and re-encodes a non-minimal integer; another flags it; a third rejects it. For a signing pipeline, an accept-versus-reject disagreement is just as dangerous as a bytes disagreement, because it decides whether a message is processed at all.

The payoff: a signature that verifies on one stack and fails on another

Here is why this leaves the realm of trivia. COSE (RFC 9052) and CWT (RFC 8392) do not sign an abstract value. COSE constructs a Sig_structure, a CBOR array holding the context, the protected headers, external data, and the payload, encodes that array as CBOR, and the signature is computed over those bytes. The verifier reconstructs the same structure, encodes it, and checks the signature against its bytes. The entire scheme assumes both sides produce the same bytes for the same value. Deterministic encoding is that assumption.

Now watch it break. The single clearest case in our data is the floating-point value 2.0, CBOR f94000:

Three distinct outcomes for one trivial number, under functions all advertised as canonical or deterministic. A service that signs a token containing the number 2.0 using the Node library, whose canonical mode applies dCBOR-style numeric reduction and emits 02, and a verifier that re-encodes the value with the Python or Go library and gets f94000, will hash different byte strings. The signature will not verify, and the failure will look like a mysterious intermittent interop bug rather than what it is: two libraries that both think they are canonical and are not the same canonical.

The duplicate-key case is worse in a different way. Given the map {1:1, 1:2} (input a201010102), Go and the dCBOR baseline reject it, Python and Node silently collapse it to a one-entry map {1:2}, and the default Rust encoder passes the duplicate through untouched. Five libraries, three security-relevant behaviors: reject, silently drop data, or preserve an ambiguous structure. Any of those mismatched across a signer and verifier is a place where an attacker gets to choose which side sees which value.

Two canons, and a rejection rate that tells the story

The reason this is a structural problem and not a set of isolated bugs is that there is more than one canon. Plain RFC 8949 deterministic encoding is one target. The stricter dCBOR profile, draft-mcnally-deterministic-cbor, is another, and it deliberately does more: it mandates numeric reduction so 2.0 becomes 2, canonicalizes every NaN to a single form, and rejects duplicate keys outright. There are still more proposals in flight, including the IETF draft-ietf-cbor-cde Common Deterministic Encoding work, which exists precisely because the ecosystem has not converged. Google's Adam Langley catalogued at least three conflicting map-key orderings back in 2022, and the disagreement is still shipping: real interoperability issues are open against .NET's CBOR library over RFC 7049 versus RFC 8949 ordering.

Our own numbers show the two-canon split as a single stark figure. The strict dCBOR library rejected 1,221 of the 3,061 values (40%) as not valid dCBOR, even though the other libraries happily accepted and canonically encoded them under plain RFC 8949 rules. On the 1,840 values dCBOR did accept, it agreed with the others more than 99.9% of the time. In other words, dCBOR and core deterministic CBOR are not a stricter-but-compatible pair; they are two different languages that overlap on a subset. Pick one on the signer and the other on the verifier and 40% of your values fall into the gap.

The pairwise agreement numbers make the same point from the other direction:

• cbor2 (Python) and fxamacker/cbor (Go) agreed on 99.93% of values. Two independent implementations, in different languages, that genuinely converge on RFC 8949 core deterministic encoding. It is achievable.
• cbor (Node.js) and ciborium (Rust) agreed on only 73.7%. The least-similar pair, more than a quarter of values apart, because one applies numeric reduction and the other does not canonicalize at all.
• The default Rust encoder never sorted maps or shortened floats. That is not a bug; ciborium simply has no dedicated canonical mode, which is exactly the trap. A developer who reaches for a CBOR library and does not find a canonical switch will happily sign non-canonical bytes and never know.

How we measured it, and the honest limits

The dataset is ours, built from values we invented, and reported in aggregate. We probed no external system and named no library as vulnerable; a compatibility difference is not a vulnerability, it is a compatibility difference. The point of naming versions is reproducibility, not blame.

• The corpus. 61 hand-crafted vectors targeting ten decision points (integer minimality, map ordering, duplicate keys, shortest float, numeric reduction, NaN, negative zero, indefinite length, trailing bytes, big integers), plus 3,000 seeded random values so the whole run is deterministic and repeatable.
• The verdict. For each value and library we recorded the canonical re-encoding in hex and whether the library treated the input as valid-canonical, non-canonical, or rejected. Divergence is counted over the libraries that returned an encoding, so a library that rejects a value is not miscounted as producing a different one.
• Modes, not defaults where a canonical mode exists. Each library was run in its documented deterministic or canonical mode, except ciborium, which has none; we included its default encoder precisely to show what a developer gets when no canonical mode is offered.
• The limits. Five libraries are not the whole ecosystem, and each library exposes several strictness knobs; a different knob could move a number. These are also library behaviors, not proof that any deployed product signs across a mismatch. The finding is that the disagreement is real, common, and lands on the exact decision points that break signatures. It is a warning about an assumption, not an accusation about a product.
• Not the same as a parser-safety bug. CBOR decoders have a hardening track record of their own, for example the memory and denial-of-service fixes in cbor2's advisories. Those are decoder-robustness issues, a separate class. Canonicalization confusion is not a crash; it is two honest implementations quietly disagreeing about the truth.

What to do if you sign CBOR

The defensive takeaways are boring, which is how you know they are right.

• Sign and verify with the same library and version on both ends wherever you can. Cross-stack signing is exactly where the 26% bites.
• Pin the profile explicitly. Decide whether you require plain RFC 8949 core deterministic or strict dCBOR, write it down, and make every participant use that one. They are not interchangeable.
• Never rely on defaults. If a library has no canonical mode, its default output is almost certainly not canonical, and you will not get a warning.
• Test against fixed vectors, including the nasty ones: duplicate keys, 2.0, negative zero, NaN, and maps whose keys have different encoded lengths. If your signer and verifier produce different bytes for any of those, you have found your interop bug before an attacker did.

Deterministic CBOR is a good idea that mostly works, and two of our five libraries proved you can converge to the byte. But mostly is not the guarantee the word deterministic implies, and signatures do not tolerate mostly. The gap is a quarter of ordinary values wide, and it sits under formats a lot of security depends on.

Frequently asked questions

What is deterministic CBOR, and how is it different from canonical CBOR?

Deterministic CBOR is a set of extra rules on top of ordinary CBOR (RFC 8949) that force one value to have exactly one byte encoding. It is what older documents call canonical CBOR; the current standard uses the word deterministic, and they mean the same thing. RFC 8949 Section 4.2 gives the core rules: shortest-form integers and lengths, definite lengths only, shortest-form floats, and sorted map keys. The goal is that two independent encoders emit the same bytes for the same value, which is what signing and hashing depend on.

What does RFC 8949 Section 4.2 require for deterministic encoding?

Four things. Preferred serialization, so every integer, length, and tag argument uses the fewest bytes possible. Definite lengths only, so no indefinite-length strings, arrays, or maps. Map key ordering by bytewise lexicographic comparison of the encoded keys, the one-step ordering. And shortest float, so a value uses the smallest of half, single, or double that represents it exactly. Section 4.2.3 also documents an older length-first key ordering kept for RFC 7049 compatibility, which is one direct source of cross-library disagreement.

How are CBOR map keys ordered in deterministic encoding?

Under RFC 8949 keys are sorted by comparing their fully encoded byte strings lexicographically, byte by byte. RFC 7049, the previous version, sorted a shorter encoded key before a longer one first. The two rules disagree whenever keys have different encoded lengths, for example a one-byte integer key next to a two-byte one, and libraries built to different versions therefore sort the same map differently while both call the result canonical.

What is dCBOR and how does it differ from RFC 8949 deterministic CBOR?

dCBOR is a stricter profile defined in the draft draft-mcnally-deterministic-cbor. On top of RFC 8949 it adds numeric reduction, so a float with no fractional part must be encoded as an integer where one fits and 2.0 must become 2. It also canonicalizes every NaN to one half-width form and rejects maps with duplicate keys as a decode error. Because it reduces 2.0 to 2, a map that is valid under plain deterministic CBOR can become an invalid dCBOR map when two keys collapse to the same reduced value.

Why does deterministic encoding matter for COSE and CWT signatures?

COSE (RFC 9052) and CWT (RFC 8392) compute a signature over encoded CBOR bytes, not over an abstract value. COSE builds a Sig_structure, encodes it as CBOR, and signs those bytes. If the signer canonicalizes a value one way and the verifier re-encodes it another way, the verifier hashes different bytes and the signature fails, or a boundary-crafted value passes one side and means something else on the other. Deterministic encoding is the assumption that makes signing a structured value safe.

Do different CBOR libraries produce the same canonical bytes?

Not always. In our test of five widely used libraries in canonical mode, 26.3% of 3,061 values produced more than one distinct canonical encoding, and 40% produced more than one validity verdict. The disagreements cluster in duplicate keys, float reduction, map ordering, and shortest-float. Two libraries agreed on more than 99.9% of values while the least similar pair matched on under 74%, so agreement depends heavily on which implementations you pair.

Is 2.0 encoded the same as 2 in CBOR?

It depends on the library and profile. In plain RFC 8949 deterministic CBOR the float 2.0 stays a float and encodes differently from the integer 2. Under dCBOR, numeric reduction requires 2.0 to be encoded as the integer 2. In our test one library rewrote 2.0 to 2 under a mode named encodeCanonical, three kept it as a float, and the dCBOR baseline rejected the float form, so the same value canonicalized on two stacks can produce different signed bytes.

How do you encode CBOR deterministically?

Turn on the library's explicit deterministic or canonical mode rather than trusting defaults, which do not sort maps or shorten floats. Pin the exact profile, plain RFC 8949 or strict dCBOR, and make signer and verifier use the same one. Test with edge cases, especially duplicate keys, floats with no fractional part, negative zero, NaN, and mixed-length map keys. If you sign CBOR, the safest design is identical library and version on both ends, or comparing bytes against fixed test vectors.

Related reading

• JCS signature verification splits across RFC 8785 libraries. The same differential one encoding over, on JSON canonicalization, where six of seven libraries agree and the one that does not breaks signatures.
• Does FIPS disable post-quantum TLS? The same differential method, one set of inputs across many libraries, applied to the TLS handshake instead of a serialization format.
• Certificate transparency post-quantum: 43% of static-CT logs already sign with ML-DSA-44. Another place where signatures are computed over a precise byte encoding, and where reading the bytes tells the real story.
• What the certs prove. Reading a cryptographic guarantee off the wire by construction instead of trusting the label, the same instinct behind counting bytes here.