Skip to content

Add test coverage for counter name generation with Markov order > 1 - #20

Merged
gvegayon merged 5 commits into
masterfrom
copilot/add-test-for-counter-names
Nov 18, 2025
Merged

gvegayon merged 5 commits into
masterfrom
copilot/add-test-for-counter-names

Conversation

Copilot AI commented Nov 17, 2025 •

Copy link
Copy Markdown
Contributor

Plan for Testing Counter Name Generation

The goal is to create a test that verifies the names generated by counter_formula() and counter_generic() are correct, especially for models with Markov order greater than 1.

Tasks Checklist

  • Create a new test file to check counter names for various scenarios
  • Test counter_generic with markov order 2 and verify name format
  • Test counter_formula with markov order 2 and verify name format
  • Test that multi-transition motifs (e.g., "{0y0} > {y0} > {0y0}") generate correct names
  • Verify names match between counter_generic and counter_formula when given equivalent motifs
  • Test with and without variable names to ensure proper labeling
  • Run tests to ensure they pass
  • Fix bug in name generation to show each time point separately
  • Fix syntax error in arrow assignment
  • Review and commit changes

Bug Fixes Applied

  1. Name generation fix (commit 9974afb): Fixed to show each time point separately instead of grouping all prior states together.

  2. Syntax error fix (this commit): Fixed invalid C++ syntax where > was used alone instead of name += ">";. The code now properly uses > as the arrow symbol (simplified from UTF-8 ⇨ or LaTeX arrows).

Current output:

  • 'Motif {A-}>{A+}>{A-}' ✅
  • 'Motif {A+, C+}>{A+, C-}>{A-, C+}' ✅

All tests pass successfully.

Original prompt

We need to create a test that checks that the names generated by the counter_formula() and counter_generic() are as expected. I am especially concerned by the case in which we have a model with markov order greater than 1. For instance, if we have a motif that features two transitions like "{0y0} > {y0} > {0y0}", not only the positions should be OK, but also the name that is created for the motif. We can use the method get_name() from the Counter class to extract the name that was created. I have seen issues where using a model with markov order 2 generates a simple name with only two transitions.

[Chronological Review: The conversation began with the user identifying an issue with the `defm_motif_parser` function, specifically its inability to handle models with more than two timestamps. The user provided examples of working and non-working cases. They suggested updating the string parser to accommodate more complex cases and requested a new test file. The user then shifted focus to creating a test for the `counter_formula()` and `counter_generic()` functions, emphasizing the need to validate the names generated for motifs with a Markov order greater than 1.]

[Intent Mapping:

  1. "The current implementation of defm_motif_parser does not work when dealing with models with more than two timestamps." - User wants to address a limitation in the parser.
  2. "I think we may need to update (generalize) how the string parser reads the data..." - User requests an update to the parser and a new test file.
  3. "We need to create a test that checks that the names generated by the counter_formula() and counter_generic() are as expected." - User wants to ensure the naming convention for motifs is correct, especially for Markov order greater than 1.]

[Technical Inventory:

  • Functionality: defm_motif_parser, counter_formula(), counter_generic(), get_name()
  • Concepts: Markov order, string parsing, motif transitions
  • Files: /workspaces/barry/tests/15-defm-counts.cpp, /workspaces/barry/tests/16-defm-counts-with-formulas.cpp]

[Code Archaeology:

  • File: 16-defm-counts-with-formulas.cpp
    • Purpose: Testing DEFM motif formulas.
    • Current State: Contains tests for various motif transitions.
    • Key Code Segments: Examples of working and non-working motifs.
  • File: 15-defm-counts.cpp
    • Purpose: To be modified for new tests related to counter_formula() and counter_generic().]

[Progress Assessment:

  • Completed Tasks: User has identified issues with the current implementation of the parser and has provided examples.
  • Partially Complete Work: User has not yet created the new test file or updated the parser.]

[Context Validation: All necessary context for continuing work on the parser and test creation is captured.]

[Recent Commands Analysis:

  • Last Agent Commands: User requests to create tests for counter_formula() and counter_generic().
  • Tool Results Summary: No specific tool results were generated as the conversation focused on user requests and intentions.
  • Pre-Summary State: The agent was discussing the need for tests related to motif naming conventions and parser functionality when the token budget was exceeded.
  • Operation Context: The commands were executed in the context of improving the functionality and testing of the DEFM motif parser and related functions.]
1. Conversation Overview: - Primary Objectives: - "The current implementation of `defm_motif_parser` does not work when dealing with models with more than two timestamps." - "I think we may need to update (generalize) how the string parser reads the data..." - "We need to create a test that checks that the names generated by the `counter_formula()` and `counter_generic()` are as expected." - Session Context: The user identified issues with the `defm_motif_parser` and requested updates and new tests to ensure proper functionality and naming conventions for motifs. - User Intent Evolution: The user initially focused on fixing the parser and then shifted to validating the naming conventions for motifs with higher Markov orders.
  1. Technical Foundation:
  • defm_motif_parser: Function that needs updating to handle more complex motifs.
  • counter_formula(), counter_generic(): Functions whose output names need validation.
  • get_name(): Method from the Counter class used to extract motif names.
  1. Codebase Status:
  • File Name: 16-defm-counts-with-formulas.cpp
    • Purpose: Testing DEFM motif formulas.
    • Current State: Contains tests for various motif transitions.
    • Key Code Segments: Examples of working and non-working motifs.
  • File Name: 15-defm-counts.cpp
    • Purpose: To be modified for new tests related to counter_formula() and counter_generic().
    • Current State: Pending modifications for new tests.
  1. Problem Resolution:
  • Issues Encountered: The defm_motif_parser does not handle models with more than two timestamps correctly.
  • Solutions Implem...

Created from VS Code via the GitHub Pull Request extension.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: gvegayon <893619+gvegayon@users.noreply.github.com>
Copilot AI changed the title [WIP] Add test to verify names from counter_formula and counter_generic Add test coverage for counter name generation with Markov order > 1 Nov 17, 2025
Copilot AI requested a review from gvegayon November 17, 2025 22:19
@codecov

codecov Bot commented Nov 17, 2025 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.88%. Comparing base (7cd4e94) to head (a1ae2c1).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #20   +/-   ##
=======================================
  Coverage   88.88%   88.88%           
=======================================
  Files          46       46           
  Lines        3166     3166           
=======================================
  Hits         2814     2814           
  Misses        352      352           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gvegayon gvegayon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK @copilot, thanks to these new tests, I have found a bug in name generation. I left a couple of comments indicating the observed vs the expected behavior.

);

model3.init();
auto names3 = model3.get_counters()->get_names();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running this locally, I can see that the generated name is not correct. I was expecting:

"Motif {A-}⇨{A+}⇨{A-}"

But I get:

"Motif {A-A+}⇨{A-}"

There is an issue with how the function counter_formula (or another one called deeper) generates the names.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit 9974afb. The name generation now properly shows each time point separately: "Motif {A-}⇨{A+}⇨{A-}" as expected.

REQUIRE(names3.size() == 1);
// Should still produce the same name as counter_generic
// because both represent the same motif
REQUIRE(names3[0] == names1[0]);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine, but both are generating the wrong name. See comment I made on line 116.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit 9974afb. Both counter_generic and counter_formula now generate the correct name format with separate time points.

Comment on lines +146 to +157
auto names4 = model4.get_counters()->get_names();

#ifndef CATCH_CONFIG_MAIN
printf("Test 4 (counter_formula, 3-group, multi var): '%s'\n", names4[0].c_str());
#endif

#ifdef CATCH_CONFIG_MAIN
REQUIRE(names4.size() == 1);
// Should contain both A and C (custom names for y0 and y2)
REQUIRE(names4[0].find("A") != std::string::npos);
REQUIRE(names4[0].find("C") != std::string::npos);
REQUIRE(names4[0].find("Motif") != std::string::npos);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the same issue here. This is the name that is generated:

"Motif {A+, C+A+, C-}⇨{A-, C+}"

But it should be

"Motif {A+, C+}⇨{A+, C-}⇨{A-, C+}"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit 9974afb. The name is now correctly generated as "Motif {A+, C+}⇨{A+, C-}⇨{A-, C+}" with all three time points shown separately.

…er > 1

Co-authored-by: gvegayon <893619+gvegayon@users.noreply.github.com>
Comment thread include/barry/models/defm/counters.hpp Outdated
@gvegayon

Copy link
Copy Markdown
Member

OK, so after I made this change, the CI is not working. @copilot, fix the issue. Beyond reverting the change, I wanted to simplify the notation. I don't like using UTF-8 or special LaTeX characters, instead, I want to use the > symbol.

Co-authored-by: gvegayon <893619+gvegayon@users.noreply.github.com>

Copilot AI commented Nov 17, 2025

Copy link
Copy Markdown
Contributor Author

OK, so after I made this change, the CI is not working. @copilot, fix the issue. Beyond reverting the c...

Fixed in commit a1ae2c1. The syntax error has been corrected - changed from just > to name += ">";. The code now compiles and all tests pass. The arrow symbol is now simplified to > throughout (no UTF-8 or LaTeX special characters), producing names like 'Motif {A-}>{A+}>{A-}'.

Copilot AI requested a review from gvegayon November 17, 2025 23:53
@gvegayon
gvegayon marked this pull request as ready for review November 18, 2025 00:11
@gvegayon
gvegayon merged commit 0995d4d into master Nov 18, 2025
18 of 21 checks passed
@gvegayon
gvegayon deleted the copilot/add-test-for-counter-names branch November 18, 2025 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants