Skip to content

Enhancement: Use string_view to reduce string allocations #144

Description

@clank-hayen

Problem

The lexer and parser create many temporary strings during parsing. For example:

// In lexer.cpp
while (isalnum(current()) || current() == '_') {
    value += current();  // String allocation per character
    advance();
}
// In parser - string concatenation
string qualified_name = ident + "." + member_name;  // Allocation
decl->type = "List<" + element_type + ">";          // Allocation

Suggested Solution

Use `std::string_view` where possible:

  1. Lexer: Return views into the source string

    struct Token {
        TokenType type;
        string_view value;  // View into source
        int line;
    };
  2. Parser lookups: Use string_view for comparisons

    bool is_primitive_type(string_view type);
    const StructDef* get_struct(string_view name);
  3. Type extraction: Return views for type components

    string_view extract_element_type(string_view generic_type, string_view prefix);

Trade-offs

Pros:

  • Reduced allocations during parsing
  • Faster keyword/type comparisons
  • Lower memory pressure

Cons:

  • Must ensure source string lifetime exceeds tokens
  • Some APIs may still need owned strings

Priority

Low - This is a performance optimization. Profile first to confirm benefit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions