Skip to content

DEMO.md

Latest commit

 

History

History
152 lines (113 loc) · 3.48 KB

DEMO.md

File metadata and controls

152 lines (113 loc) · 3.48 KB
Raw

Demo: Coverage-Guided Test Improvement

This demo uses FasterXML/jackson-core, a real open-source Java library.

The goal is to show the full workflow:

  1. target one low-coverage class
  2. generate a JUnit test
  3. run Maven validation
  4. repair failures when needed
  5. leave reviewable coverage and run artifacts

Setup

git clone https://github.com/zemeng2015/JTestGen.git
cd JTestGen

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -e .

git clone https://github.com/FasterXML/jackson-core.git C:\tmp\jackson-core

Set your API key:

$env:OPENAI_API_KEY="..."

Optional readiness check:

java-testgen doctor C:\tmp\jackson-core

Optional target scan without calling an LLM:

java-testgen scan C:\tmp\jackson-core --top 5 --verify-arg=-DskipITs

Optional model override:

$env:OPENAI_MODEL="gpt-4o-mini"

Run

If Maven is already on your PATH, run:

java-testgen run C:\tmp\jackson-core `
  --target-class tools.jackson.core.io.DataOutputAsStream `
  --test-suffix GeneratedTest `
  --target-coverage 0.80 `
  --verify-arg=-DskipITs

If Maven is not on your PATH, pass the executable explicitly:

java-testgen run C:\tmp\jackson-core `
  --target-class tools.jackson.core.io.DataOutputAsStream `
  --test-suffix GeneratedTest `
  --target-coverage 0.80 `
  --maven-command C:\tmp\apache-maven-3.9.11\bin\mvn.cmd `
  --verify-arg=-DskipITs

For a pilot audit, you can also process up to three automatically selected targets and write a patch:

java-testgen run C:\tmp\jackson-core `
  --max-targets 3 `
  --test-suffix GeneratedTest `
  --target-coverage 0.80 `
  --patch-output C:\tmp\jtestgen-jackson-core.patch `
  --verify-arg=-DskipITs

Observed Result

Selected target: tools.jackson.core.io.DataOutputAsStream with 55.56% line coverage
Repairing DataOutputAsStreamGeneratedTest, attempt 1/3
Generated test passed

Generation report
- Target class: tools.jackson.core.io.DataOutputAsStream
- Class line coverage: 55.56% -> 100.00%
- Class covered lines: 5/9 -> 9/9
- Project line coverage: 81.72% -> 81.75%

What to Show in a Demo

When presenting JTestGen, do not describe it as "AI generated a test." The stronger message is:

JTestGen turns a Java coverage gap into a Maven-passing, reviewable test improvement artifact.

Show these four things:

  1. the target class and baseline coverage
  2. the generated test file
  3. Maven pass/fail logs and repair attempts
  4. summary.md or report.html with before/after coverage
  5. optional generated.patch or --patch-output file for code review

Artifacts

Each run writes observability artifacts under the target project:

.jtestgen/runs/<run-id>/
  report.json
  summary.md
  report.html
  prompt.initial.system.txt
  prompt.initial.user.txt
  prompt.repair.1.system.txt
  prompt.repair.1.user.txt
  maven.baseline.log
  maven.test.0.log
  maven.test.1.log
  maven.final.log
  generated.initial.java
  generated.repair.1.java
  generated.final.java
  generated.patch

These files make the LLM workflow inspectable and reproducible.

Clean Up

The demo writes a generated test into the target repo. If you want to reset jackson-core afterward:

cd C:\tmp\jackson-core
git status
git clean -fd .jtestgen
git restore src/test/java

See also: