Problem
In `lexer/lexer.cpp`, there are four nearly identical string reading functions:
- read_string() (lines 106-138): Double-quoted with escapes
- read_single_quoted() (lines 145-177): Single-quoted with escapes
- read_raw_string() (lines 183-199): Raw double-quoted
- read_raw_single_quoted() (lines 205-221): Raw single-quoted
Functions 1 and 2 differ only in the quote character ('"' vs '\'').
Functions 3 and 4 differ only in the quote character.
Suggested Solution
Consolidate into two functions with a quote parameter:
Token read_string_literal(char quote_char, bool handle_escapes) {
int start_line = line;
advance(); // skip opening quote
string value;
while (current() != quote_char && current() != '\0') {
if (handle_escapes && current() == '\\\\') {
advance();
switch (current()) {
case 'n': value += '\\n'; break;
case 't': value += '\\t'; break;
case 'r': value += '\\r'; break;
case '\\\\': value += '\\\\'; break;
case '\"': value += '\"'; break;
case '\\'': value += '\\''; break;
default: value += current(); break;
}
advance();
} else {
value += current();
advance();
}
}
if (current() == '\0') {
throw runtime_error("Unterminated string literal at line " + to_string(start_line));
}
advance(); // skip closing quote
return {TokenType::STRING, value, start_line};
}
Usage:
// In tokenize():
if (current() == '\"') {
tokens.push_back(read_string_literal('\"', true));
}
if (current() == '\\'') {
tokens.push_back(read_string_literal('\\'', true));
}
// For raw strings: read_string_literal(quote, false)
Impact
- Reduces ~80 lines to ~30 lines
- Single place to add new escape sequences
- Consistent behavior across string types
Problem
In `lexer/lexer.cpp`, there are four nearly identical string reading functions:
Functions 1 and 2 differ only in the quote character ('"' vs '\'').
Functions 3 and 4 differ only in the quote character.
Suggested Solution
Consolidate into two functions with a quote parameter:
Usage:
Impact