feat!: default-on size limits on CAR decode and encode

[tabcat]

Aligns js-car's size limits with go-car: two
caps that are on by default, bounding the largest single allocation a decoder
makes and ensuring the library never emits a CAR it would refuse to read back.

Changes

• maxAllowedHeaderSize (default 32 MiB) and maxAllowedSectionSize
  (default 8 MiB) on CarCodecOptions, using go-car's names and defaults. The
  section cap bounds the CID and block body combined. Both count the
  varint-declared length; a length strictly greater than a cap throws
  RangeError (decode: from the length prefix before buffering; encode: before
  writing). undefined uses the default, 0 rejects every section,
  Number.MAX_SAFE_INTEGER disables a cap, and any other non-negative-safe-integer
  value throws TypeError.
• readCid is bounded by its declared section, so a CID cannot read past the
  section it lives in.
• A hardcoded 32 MiB digest cap (go-cid's maxDigestAlloc) bounds the
  multihash digest independently, so raising maxAllowedSectionSize stays safe.
• Enforced on every decode and encode entry point (streaming, in-memory,
  CarBufferReader, CarBufferWriter, CarIndexedReader.fromFile), so one
  CarCodecOptions profile holds on both read and write.
  updateRootsInBytes / updateRootsInFile keep their signatures under the
  default header cap.

Breaking change

With no options passed, decode now throws for a section over 8 MiB or a header
over 32 MiB on every entry point (including fromBytes and CarBufferReader),
and encode throws for a section over 8 MiB. Opt out per-cap with
Number.MAX_SAFE_INTEGER.

Closes #185

[rvagg]

we know length at this point, we should make sure that readCid can't read beyond it, short length, long cid = error; currently this allows a badly declared CID in here to blow past any limitation

[rvagg]

well, you have maxCidLength in there, but still if the section is declared to be 1Kb and the CID is longer than that it shoudl error or if there is no maxCidLength it still should be capped at length

[rvagg]

probably rework this whole function I reckon, so we pass in the unmodified section length to readCid something like:

 const sectionLength = decodeVarint(await reader.upTo(8), reader)
 const lengthPrefixLength = Number(reader.pos - start)

 const { cid, cidLength } = await readCid(reader, options, sectionLength)
 const blockLength = sectionLength - cidLength

 if (blockLength < 0) {
   throw new Error('Invalid CAR section')
 }

 return {
   cid,
   length: lengthPrefixLength + sectionLength,
   blockLength
 }

[rvagg]

As per slack, I really think we should just default-on limits and let the user override them; this is already done in Go and we should do it here. The fact that the go parser isn't going to let large values through already puts an effective protocol limit on this format so we're not really going backward here. Anyone streaming bytes is going to have the same problems with potentially malicious (or just erroneous) input so we should solve it properly.

I think we should adopt the Go limits for defaults. But we also have a parity problem because here you're doing CID and block limits separately rather than section limits. It would be easier if we had a whole section limit and not a block limit, kept the CID separate.

In Go we arrived at the following limits:

• Header 32 << 20
• Section (all of it) 8 << 20
• CID (per go-cid) 32 << 20

There is also const DefaultMaxIndexCidSize = 2 << 10 in Go which you're conflating with CID limits ("and 2 KiB per CID"), that's only for index limiting and doesn't really apply to js-car IIRC, certainly not general section parsing, so we should ignore that. The actual, correct version of the Go limits would be:

go-car defaults to 32 MiB for the CARv1 header and 8 MiB per section. Its CARv2 index path also defaults indexed CIDs to 2 KiB, while go-cid refuses digest allocations above 32 MiB.

And note that that CID limit is bigger than the section limit anyway, so it's just a separate safety hatch. If we adopt a section limit and cap readCid to it then we don't even need an explicit CID limit, it'll just have to be under 8 << 20 although you could put in an optional limiter in the options for people that want to cap it. But, identity CIDs are a thing to account for, they do have uses and they can be big even though we discourage use of large identity CIDs.

[rvagg]

it'd be good to do a cheap validation of options here to not let invalid and non-uint values get by

[tabcat]

I'll rework to use go's section limits with same defaults like you suggested originally 👍

[tabcat]

reworked to the default-on go-parity design per your review. the pr description has been updated.