Conversation
* Add design spec for pure-Python GLM parser
Replaces the Nim-based `glm` pip package with a pure-Python `glmparser`
module, removing the Nim toolchain from the from-source build.
Benchmarking showed the compiled parser has no speed advantage to give up:
a pure-Python prototype matched it on every sample model, because the Nim
lexer allocates a ref-object Token per whitespace character and the binding
serializes its AST to a JSON string for Python to re-parse.
Spec also records three confirmed data-losing bugs in the current parser
that the rewrite fixes (dropped #include on export, collapsed dotted
attribute keys, discarded class blocks).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Add implementation plan for pure-Python GLM parser
Ten TDD tasks covering the glmparser package, its pytest suite, the
Nim-differential golden bootstrap, the glmhelper cutover, and doc updates.
The module code in the plan was assembled and run against models/ before
writing, which surfaced three bugs now fixed and pinned with tests: ${...}
substitutions lexing their braces as block delimiters (broke 6 of 17
models), directive values swallowing trailing // comments, and a slow
per-character regex form of the first fix.
Verified: 0 unexpected diffs vs the Nim parser across all 17 models,
round-trip stable, peak memory 61.6 -> 17.0 MB.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(plan): oversized-file test was 2.7 MB, under the 5 MB cap it asserts
* feat(glmparser): add package scaffold, error type, pytest wiring
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(plan): clamp GlmParseError offset at EOF; it crashed Tasks 3 and 5
The lexer's EOF token carries start == len(source), and both unterminated-block
and unterminated-schedule errors are raised on it. splitlines() yields no
phantom trailing entry for newline-terminated source, so line was one past the
last real line and _render raised IndexError instead of the GlmParseError the
tests expect. Found by the Task 1 reviewer.
* fix(glmparser): clamp EOF offset to prevent IndexError in _render
The lexer's EOF token carries start == len(source), and unterminated
blocks/schedules raise GlmParseError on it. Without clamping, the line
number can exceed len(splitlines()), causing IndexError in _render().
Add two regression tests: offset at EOF with trailing newline and without.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(glmparser): add streaming regex lexer
* fix(plan): lexer word pattern needs three branches, not two
Two independent bugs, each caught by a test the other regex passes:
- $\{...\}|[^\s{};]+ leaks lbrace/rbrace on prefix${A}suffix
- $\{...\}|[^\s{}$;]+ silently drops a bare $: a$b covers only ab
Three branches (value | substitution | bare $) fix both at identical speed.
Verified byte-identical to the prior regex across all 17 sample models.
* fix(glmparser): use three-branch word regex to preserve bare dollar signs
* fix(plan): bound ${...} substitution to one line
An unterminated ${ with [^}]* runs to the next } anywhere in the file,
swallowing a brace that closes an unrelated block. Measured on a two-object
malformed input: the loose form silently drops the second object entirely.
Excluding whitespace and ; from the fragment contains it. Verified byte-
identical across all 17 sample models. Also documents the (?<!:) guard as
currently-unreachable defense-in-depth rather than the active protection the
old comment claimed.
* fix(glmparser): bound substitution pattern and clarify comment defense
* feat(glmparser): parse clock, module, class, and directives
* test(plan): add regression net for parser helper edge cases
Reviewer confirmed all four behave correctly today, but _attributes,
_value_to_semicolon and _value_to_eol are the helpers Tasks 4 and 5 build
directly on, and this plan has already shipped three bugs hiding in untested
error paths. Pins: value missing ; before }, directive with no =, directive
as last line with no trailing newline, empty block.
* fix(plan): correct swapped test counts for Tasks 4 and 5
* test(glmparser): pin parser helper edge-case behavior
* feat(glmparser): parse objects, nesting, and anonymous hoisting
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(glmparser): parse schedules and sub-schedule blocks
* feat(glmparser): add GLM writer with correct directive punctuation
* fix(plan): give the lexer real quoted-string support
The writer's defensive quoting protected nothing: the lexer had no quoted-string
token, so a ; inside quotes still lexed as semi and terminated the value.
Exporting {"weird": "a;b"} wrote weird "a;b"; and read back as
{'weird': 'a', 'b"': ''} -- value truncated and a junk attribute fabricated,
silently, in a file users load into simulations.
Adds "[^"\n]*" and '[^'\n]*' branches ahead of the ordinary-value branch.
Verified: fixes all corruption cases, 0 differences across all 17 sample models,
round-trip stable.
Also replaces the writer tests' substring assertions with real round-trip
assertions -- the substring form passed 59/59 while the corruption was live.
* fix(glmparser): add quoted-string lexing to enable writer defensive quoting
The writer's _quote_if_needed function wrapped special values in quotes,
but the lexer treated quotes as ordinary word characters and semicolons
inside quotes still terminated values. Exporting {'weird': 'a;b'} would
write 'weird "a;b";' but read back as {'weird': 'a', 'b"': ''} --
silent corruption with a fabricated junk attribute.
Update lexer to consume quoted strings (both single and double quotes) as
atomic tokens, preceding the ordinary-value branch so the entire quoted
run is consumed before the fast path tries to match. This enables the
writer's defensive quoting to genuinely protect special characters.
Add round-trip tests that parse writer output back through the parser,
catching malformations that substring assertions alone would miss.
* fix(plan): writer refuses values it cannot round-trip
Quoted-string lexing fixed the measured case but two combinations still
corrupted silently: a value with both ; and a newline (the quoted branch
excludes newline, by design, so the bound on unterminated quotes holds), and
any value containing a literal " (the writer's backslash escaping was
decorative -- GLM has no escape mechanism for the lexer to honor).
Rather than extend the lexer a fifth time, the writer now raises a ValueError
naming the offending attribute. server.py surfaces it as an HTTP error, so a
user sees a real failure instead of a quietly corrupted model.
Zero values in the 17 sample models contain ; " or newline, so this path is
defensive. Verified: representable values still round-trip, all 17 models still
export and round-trip cleanly.
* fix(glmparser): refuse to write unrepresentable attribute values
Values containing a double quote, or both a newline and semicolon, cannot
survive a GLM round-trip due to lexer limitations (no escape mechanism,
quoted-string branch stops at newline). Writing them anyway silently
corrupts the model on reload, with truncated values and fabricated junk
attributes appearing in GridLAB-D simulations.
Instead of attempting (and failing) to protect these with quoting, refuse
loudly in the writer with a ValueError. The export endpoint wraps this in
exception handling that returns the error message to the user as an HTTP
error, making the problem visible rather than silent.
Clarify lexer comment that the quoted-string support is partial -- it
protects semicolons and newlines alone, but not their combination with
a double quote.
No values in any of the 17 sample models contain these shapes, so this
is defensive rather than routine.
* feat(glmparser): add public load/loads/dump/dumps API
* fix(plan): round-trip test now asserts full equality
The structural-only comparison was justified by a claim I got wrong: that uuid4
hoisted names regenerate on each parse. They regenerate only when the same
SOURCE TEXT is parsed twice. This cycle parses source once then re-parses the
writer's output, and the writer emits hoisted children as ordinary top-level
objects carrying their generated name as a literal, so nothing re-hoists.
Measured: full equality holds for all 17 models (zero of which trigger hoisting)
and for a synthetic model that does. The weak comparison skipped every object's
attributes -- where essentially all real GLM content lives -- so a regression
mangling attribute values would have gone uncaught.
Also tests dump() with a str path, matching load()'s coverage.
* test(glmparser): assert full round-trip equality for sample models
* test(glmparser): freeze goldens after Nim differential parity
* refactor(glmhelper): use pure-Python glmparser instead of Nim glm
* docs: drop Nim toolchain requirement
The .glm parser is now local-server/glmparser/, pure Python with no
build step, so the Nim prerequisite, the separate GLM-parser install
step, and the Apple Silicon build-from-source instructions are all
stale. (CLAUDE.md was updated to match but is gitignored in this repo
and has no git history, so it is not part of this commit.)
* fix(docker): drop dead Nim toolchain build, exclude test goldens
Dockerfile.backend still built the Nim glm compiler from source (Nim
toolchain, choosenim, nimble) even though the backend now uses the
pure-Python glmparser and nothing in the image imports the Nim
extension — pure dead weight in every build.
.dockerignore excluded **/glm but not local-server/tests, so the 15 MB
of golden JSON fixtures shipped into the runtime image via COPY
local-server/ ./. Excluded local-server/tests too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(glmparser): correct directive EOL bound and value round-trip checks
Three correctness fixes found in final review:
- _value_to_eol anchored the line bound to the NEXT token's start
instead of the directive keyword's own line. An empty `#set`/
`#define`/`#include` (nothing following it on that line) therefore
consumed the following line's statement into its own value, silently
dropping it. Anchor to the hash token's own end instead.
- _unrepresentable rejected any value containing a double quote, even
though `3"x5` round-trips fine -- it only breaks when the value
starts/ends with a quote, needs quoting (has `;`/newline) and also
contains `"`, or has both `;` and a newline. The old blanket rule
made any model with an inch mark permanently un-exportable.
- _schedule interpolated values directly instead of going through
_quote_if_needed like every other value writer, so a schedule value
hitting the above cases would silently corrupt on export instead of
raising.
test_glmparser.py: added regression tests for the empty-directive case
and for interior-quote representability; adjusted the one existing
test whose fixture value (an interior quote with no `;`/newline) is
now correctly accepted rather than rejected. Also asserts ALL_MODELS
resolves to exactly 17 files so a models/ path drift fails loudly
instead of silently skipping the 34 golden/round-trip parametrized
tests that are the only ones touching real GLM sample data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: document the pytest suite in README and package.json
The only place documenting local-server's 111-test pytest suite was
CLAUDE.md, which is gitignored and invisible to a fresh clone. Add a
"Running Tests" section to README.md (how to run it, where pytest
comes from, how it relates to the separate socket-testing/ scripts)
and a test:server npm script alongside the other local-server-rooted
scripts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(glmparser): keep every same-typed property in a class block
class bodies were parsed into a dict keyed by property TYPE
(_named_block), so a class declaring more than one property of the
same type silently dropped all but the last. The only class in the
17 sample models is `class player { double value; }` -- one
property, so the collision never fired and this survived every
review.
Give class bodies their own parse path (_class_block) producing an
ORDERED LIST of {type, name} entries under a `properties` key, and a
matching writer (_class). Module and object bodies are untouched --
this is classes-only.
Regenerated the one golden with a non-empty classes array
(model_base__model_startup.json) and updated the AST schema in the
design doc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
simulation log now stacks showing latest log at the top decomposed the graph helper class
…stalled GLIMPSE on MacOS
A CRLF checkout of docker/40-glimpse-env.sh gives it a "#!/bin/sh\r" shebang, so nginx's entrypoint fails with a misleading "40-glimpse-env.sh: not found" and the container exits 127. Add a .gitattributes rule pinning *.sh to LF, plus a defensive sed in the image build for working trees that already contain CRLF. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
improved the responsiveness to distribution area highlighting
Greatly improved GridAPPS-D simulation log, output, and command response time
…ng dark and light modes
new host ready build of GLIMPSE
fixed a small issue with the clear logs tooltip cliping through the window making the view flicker
* Add GitHub Actions workflow for building and pushing Docker images * chore: update actions/checkout to version 4 * feat(docker): add README files for backend and frontend images * chore: update actions/checkout to version 7 * chore: update dockerhub-description action to version 5 --------- Co-authored-by: Craig <3979063+craig8@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.