Skip to content

ENCODINGS_LOOKUP mixes string and bytes keys in the same dict #44

Description

@sumanjeet0012

ENCODINGS_LOOKUP is a single dict that uses both str keys (encoding names like "base16") and bytes keys (prefix codes like b"f"). This is confusing, type-unsafe, and could cause subtle bugs if a name happens to match a prefix byte.

Problem

In multibase/multibase.py:

ENCODINGS_LOOKUP = {}
for codec in ENCODINGS:
    ENCODINGS_LOOKUP[codec.encoding] = codec  # str key: "base16"
    ENCODINGS_LOOKUP[codec.code] = codec      # bytes key: b"f"

The resulting dict has mixed key types:

ENCODINGS_LOOKUP = {
    "identity": Encoding(...),     # str
    b"\x00": Encoding(...),        # bytes
    "base2": Encoding(...),        # str
    b"0": Encoding(...),           # bytes
    "base16": Encoding(...),       # str
    b"f": Encoding(...),           # bytes
    # ...
}

Problems:

  1. Type confusion — Functions like is_encoding_supported() accept str but the dict also has bytes keys, so is_encoding_supported(b"f") returns True even though the docstring says it takes a str
  2. Potential collision — If an encoding name were a single character that matches another encoding's prefix byte, there would be a collision (currently safe but fragile)
  3. get_codec() relies on this — It checks data[:4] for emoji, then data[:1] for single-byte prefixes, both against the same mixed dict

Proposed Solution

Split into two separate dicts:

_ENCODINGS_BY_NAME: dict[str, Encoding] = {}
_ENCODINGS_BY_CODE: dict[bytes, Encoding] = {}

for codec in ENCODINGS:
    _ENCODINGS_BY_NAME[codec.encoding] = codec
    _ENCODINGS_BY_CODE[codec.code] = codec

Update all functions to use the appropriate dict:

def is_encoding_supported(encoding: str) -> bool:
    return encoding in _ENCODINGS_BY_NAME

def get_codec(data):
    data = ensure_bytes(data, "utf8")
    # Check emoji first (4-byte prefix)
    if len(data) >= 4 and data[:4] in _ENCODINGS_BY_CODE:
        return _ENCODINGS_BY_CODE[data[:4]]
    # Check single-byte prefix
    if data[:1] in _ENCODINGS_BY_CODE:
        return _ENCODINGS_BY_CODE[data[:1]]
    raise InvalidMultibaseStringError(...)

Keep ENCODINGS_LOOKUP as a backward-compatible alias if needed:

# Backward compatibility
ENCODINGS_LOOKUP = {**_ENCODINGS_BY_NAME, **_ENCODINGS_BY_CODE}

Related

  • File: multibase/multibase.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions