Skip to content

feat(skills): standardize pytest-mock as Python mocking framework#1170

Merged
WilliamBerryiii merged 6 commits intomainfrom
feat/893-pytest-mock-standardization
Mar 24, 2026
Merged

feat(skills): standardize pytest-mock as Python mocking framework#1170
WilliamBerryiii merged 6 commits intomainfrom
feat/893-pytest-mock-standardization

Conversation

@WilliamBerryiii
Copy link
Copy Markdown
Member

Description

Standardized pytest-mock as the mocking framework for all Python skills, added an instructions document codifying the convention, and migrated the entire powerpoint skill test suite from unittest.mock.patch patterns to the mocker fixture. Also fixed per-skill virtual environment resolution in the PowerShell test and lint runners so they invoke pytest and ruff from each skill's .venv rather than relying on a global installation.

pytest-mock Standardization

The mocker fixture from pytest-mock provides automatic cleanup, reduces decorator nesting, and keeps mock setup visually close to the code under test — replacing @patch decorators and with patch() context managers that scattered setup away from assertions.

  • Added python-tests.instructions.md (238 lines) to the coding-standards instructions, establishing mocker.patch() as the standard mocking mechanism with guidance on migration, mocker.AsyncMock(), mocker.patch.dict(), and prohibited patterns
  • Migrated all 6 test files in the powerpoint skill from unittest.mock.patch to the mocker fixture
    • Converted @patch decorator chains, with patch(...) context managers, and @patch.dict calls to sequential mocker.patch() calls
    • Replaced AsyncMock imports with mocker.AsyncMock() in test_validate_slides.py
    • Decomposed parenthesized multi-context-manager blocks into individual mocker.patch() and mocker.patch.object() calls in test_render_pdf_images.py
    • MagicMock remains imported from unittest.mock where used as a standalone mock object, consistent with the instructions
  • Added pytest-mock>=3.14 to pyproject.toml dev dependencies (resolved as 3.15.1 in uv.lock)

Per-Skill venv Resolution Fix

  • Updated Invoke-PythonTests.ps1 and Invoke-PythonLint.ps1 to resolve pytest and ruff from each skill's .venv directory before falling back to global installations
    • Cascading resolution: .venv/bin/{tool} (Linux) → .venv/Scripts/{tool}.exe (Windows) → global → per-skill error
  • Updated both Pester test files to match the new per-skill error reporting behavior

Collection and Plugin Updates

  • Updated coding-standards and hve-core-all collection manifests to include the new python-tests.instructions.md entry
  • Regenerated plugin READMEs and added symlinks for the new instructions file

Related Issue(s)

Closes #893

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

"How should I mock dependencies in my Python tests?"

Execution Flow:

The python-tests.instructions.md file is attached automatically via its applyTo glob pattern when editing Python test files in skills. It provides inline guidance on using the mocker fixture from pytest-mock, including migration patterns from unittest.mock.

Output Artifacts:

No files created — the instructions file provides contextual guidance during test development.

Success Indicators:

New or modified Python tests use mocker.patch() instead of @patch decorators or with patch() context managers. MagicMock remains acceptable as a standalone mock object.

Testing

  • All 626 pytest tests passing across the powerpoint skill test suite
  • All 52 Pester tests passing for PowerShell scripts
  • Coverage unchanged at 90.76%

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

  • MagicMock remains imported from unittest.mock in test files where it creates standalone mock objects. The instructions document explicitly permits this pattern — only patch, patch.object, patch.dict, and AsyncMock are migrated to mocker equivalents.
  • The venv resolution fix is a separate commit (5b5b314d) from the pytest-mock migration (82cd78d4) since it addresses a pre-existing bug discovered during testing.

Bill Berry added 3 commits March 20, 2026 12:47
- add python-tests.instructions.md with mocker vs monkeypatch guidance
- add pytest-mock>=3.14 to powerpoint skill dev dependencies
- migrate 6 test files from unittest.mock to mocker fixture
- update coding-standards and hve-core-all collection manifests

🧪 - Generated by Copilot
- use skill .venv/bin/pytest and .venv/bin/ruff instead of bare commands
- add cross-platform fallback to .venv/Scripts/ for Windows
- fall back to global tool when no venv exists
- update Pester tests for per-skill resolution behavior

🐛 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii requested a review from a team as a code owner March 21, 2026 03:36
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 21, 2026

Codecov Report

❌ Patch coverage is 77.14286% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.05%. Comparing base (63d4c4d) to head (5e1d78b).
⚠️ Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
scripts/linting/Invoke-PythonLint.ps1 76.47% 4 Missing ⚠️
scripts/linting/Invoke-PythonTests.ps1 77.77% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1170      +/-   ##
==========================================
- Coverage   87.68%   87.05%   -0.63%     
==========================================
  Files          61       59       -2     
  Lines        9239     8799     -440     
==========================================
- Hits         8101     7660     -441     
- Misses       1138     1139       +1     
Flag Coverage Δ
pester 85.25% <77.14%> (-0.09%) ⬇️
pytest 91.53% <ø> (-1.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/linting/Invoke-PythonLint.ps1 84.50% <76.47%> (-3.63%) ⬇️
scripts/linting/Invoke-PythonTests.ps1 80.95% <77.77%> (-2.15%) ⬇️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 21, 2026

Dependency Review Summary

The full dependency review summary was too large to display here (6206KB, limit is 1024KB).

Please download the artifact named "dependency-review-summary" to view the complete report.

View full job summary

- collapse wrapped mocker.patch() calls onto single lines

🎨 - Generated by Copilot
Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition!
There are two PRs that touch on Python based skills and will be affected by this PR. Question is should we merge them in first and then this, then add a backlog item to update the code from PRs #1128 and #978 - or merge the current one first and ask contributors to update based on the new requirements?

@WilliamBerryiii
Copy link
Copy Markdown
Member Author

Great addition! There are two PRs that touch on Python based skills and will be affected by this PR. Question is should we merge them in first and then this, then add a backlog item to update the code from PRs #1128 and #978 - or merge the current one first and ask contributors to update based on the new requirements?

Great addition! There are two PRs that touch on Python based skills and will be affected by this PR. Question is should we merge them in first and then this, then add a backlog item to update the code from PRs #1128 and #978 - or merge the current one first and ask contributors to update based on the new requirements?

I'd like to merge the other PRs first so they are not blocked and then circle back to update these to match. There's also some type system work for python that I'd like to enforce, so I may do all that in one pass.

Copy link
Copy Markdown
Contributor

@nguyena2 nguyena2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this PR were a credit score, I’d rate it around 790/850.

Strong payment history: core changes are coherent and tests pass in the updated areas.
Low risk debt: only one known concern remains, and you explicitly marked it non-blocking.
Small utilization penalty: broad applyTo scope is a minor policy-quality issue, not a functional blocker.

@WilliamBerryiii WilliamBerryiii merged commit 3b92227 into main Mar 24, 2026
41 checks passed
WilliamBerryiii pushed a commit that referenced this pull request Mar 24, 2026
## Pre-Release 3.3.10

### ✨ Features

- standardize pytest-mock as Python mocking framework (#1170)
- add Jira backlog workflows and Jira/GitLab skills (#978)
- add centralized version bump script and supply-chain attestation
(#1183)

### 🐛 Bug Fixes

- add post-release triggers to scorecard workflow (#1186)
- add missing .md extensions to relative links in agent documentation
(#1180)

### 🔧 Maintenance

- bump markdownlint-cli2 from 0.21.0 to 0.22.0 in the npm-dependencies
group (#1175)
- bump the github-actions group with 3 updates (#1176)
- pre-release 3.3.1 (#1165)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
WilliamBerryiii pushed a commit that referenced this pull request Mar 30, 2026
## Pre-Release 3.3.27

### ✨ Features

- standardize pytest-mock as Python mocking framework (#1170)
- add Jira backlog workflows and Jira/GitLab skills (#978)
- add centralized version bump script and supply-chain attestation
(#1183)

### 🐛 Bug Fixes

- sync package-lock.json after pre-release version bump (#1236)
- separate mermaid node declarations and add dynamic diagram generation
with tests (#1215)
- replace anchor links in meeting-analyst with bold text references
(#1201)
- remove recursive symlinks in jira and gitlab skill directories (#1233)
- validate-installation scripts now check .github/skills directory
(#1010) (#1206)
- resolve npm audit vulnerabilities via dependency overrides (#1200)
- add post-release triggers to scorecard workflow (#1186)
- add missing .md extensions to relative links in agent documentation
(#1180)

### ♻️ Refactoring

- align agent and prompt folder names to collection identifier (#1210)

### 🔧 Maintenance

- bump pygments from 2.19.2 to 2.20.0 in /.github/skills/gitlab/gitlab
(#1234)
- bump path-to-regexp from 0.1.12 to 0.1.13 in /docs/docusaurus (#1226)
- bump the github-actions group with 4 updates (#1231)
- add missing folders and alphabetize location lists (#1193)
- bump brace-expansion (#1224)
- bump handlebars from 4.7.8 to 4.7.9 in /docs/docusaurus (#1217)
- bump brace-expansion from 5.0.3 to 5.0.5 in /docs/docusaurus (#1213)
- pre-release 3.3.10 (#1187)
- bump markdownlint-cli2 from 0.21.0 to 0.22.0 in the npm-dependencies
group (#1175)
- bump the github-actions group with 3 updates (#1176)
- pre-release 3.3.1 (#1165)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
WilliamBerryiii pushed a commit that referenced this pull request Apr 2, 2026
## Pre-Release 3.3.41

### ✨ Features

- add agentic workflows for automated issue triage, implementation, PR
review, dependency review, and doc-staleness detection (#1219)
- consolidate package-lock.json version sync into
Update-VersionFiles.ps1 (#1240)
- add standards code review agent and full review orchestrator (#1174)
- standardize pytest-mock as Python mocking framework (#1170)
- add Jira backlog workflows and Jira/GitLab skills (#978)
- add centralized version bump script and supply-chain attestation
(#1183)

### 🐛 Bug Fixes

- replace invalid bare tool names with official tool identifiers (#1198)
- fix broken links and remove orphaned reference in code review docs
(#1257)
- exclude Python env dirs from skill validation warnings (#1255)
- pin happy-dom and serialize-javascript to resolve Dependabot
vulnerabilities (#1253)
- remove Mermaid diagram and add missing collection cards (#1247)
- disable MCP servers by default to prevent token limit errors (#1144)
- sync package-lock.json after pre-release version bump (#1236)
- separate mermaid node declarations and add dynamic diagram generation
with tests (#1215)
- replace anchor links in meeting-analyst with bold text references
(#1201)
- remove recursive symlinks in jira and gitlab skill directories (#1233)
- validate-installation scripts now check .github/skills directory
(#1010) (#1206)
- resolve npm audit vulnerabilities via dependency overrides (#1200)
- add post-release triggers to scorecard workflow (#1186)
- add missing .md extensions to relative links in agent documentation
(#1180)

### 📚 Documentation

- CLI getting-started: clarify plugin install commands as alternatives
(-all vs base) (#1251)

### ♻️ Refactoring

- align agent and prompt folder names to collection identifier (#1210)

### 🔧 Maintenance

- streamline RAI Planner phase structure and documentation (#1273)
- bump happy-dom from 20.8.8 to 20.8.9 in /docs/docusaurus (#1237)
- pre-release 3.3.27 (#1191)
- bump pygments from 2.19.2 to 2.20.0 in /.github/skills/gitlab/gitlab
(#1234)
- bump path-to-regexp from 0.1.12 to 0.1.13 in /docs/docusaurus (#1226)
- bump the github-actions group with 4 updates (#1231)
- add missing folders and alphabetize location lists (#1193)
- bump brace-expansion (#1224)
- bump handlebars from 4.7.8 to 4.7.9 in /docs/docusaurus (#1217)
- bump brace-expansion from 5.0.3 to 5.0.5 in /docs/docusaurus (#1213)
- pre-release 3.3.10 (#1187)
- bump markdownlint-cli2 from 0.21.0 to 0.22.0 in the npm-dependencies
group (#1175)
- bump the github-actions group with 3 updates (#1176)
- pre-release 3.3.1 (#1165)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
WilliamBerryiii pushed a commit that referenced this pull request Apr 24, 2026
## Pre-Release 3.3.101

### ✨ Features

- add removed maturity tier and retire owasp-docker (#1444)
- add evaluation dataset creator (#1279)
- align RAI planner with guide, remove scoring, improve UX (#1287)
- add PSGallery staleness check and BOM cleanup (#1379)
- ISA-95 network planner agent (#1177)
- auto-generate collection.md with maturity filtering (#1316)
- add folder-consistency check and standardize WARN outp… (#1350)
- add synth-data-generate prompt to data-science collection (#1419)
- add canonical deck workflow and customer-card rendering for design
thinking (#1413)
- add Figma MCP integration for DT artifact export (#1222)
- introduce `owasp-docker` (#1245)
- replace hve-core-specific references with portable discovery-based
language (#1335)
- introduce `owasp-cicd` (#1246)
- add secure-by-design knowledge skill (#1223)
- introduce `owasp-infrastructure` (#1244)
- introduce `owasp-mcp` (#1207)
- add OutputPath parameter to Invoke-LinkLanguageCheck.ps1 (#1229)
- add -OutputPath parameter to Validate-SkillStructure.ps1 (#1225)
- add maintainer-only skip-review label guard (#1293)
- add extension collections overview and integrate into getting started
flow (#950)
- add agentic workflows for automated issue triage, implementation, PR
review, dependency review, and doc-staleness detection (#1219)
- consolidate package-lock.json version sync into
Update-VersionFiles.ps1 (#1240)
- add standards code review agent and full review orchestrator (#1174)
- standardize pytest-mock as Python mocking framework (#1170)
- add Jira backlog workflows and Jira/GitLab skills (#978)
- add centralized version bump script and supply-chain attestation
(#1183)

### 🐛 Bug Fixes

- pin PowerShell-Yaml to 0.4.7 across all install sites (#1378)
- close fork-PR/workflow-file-PR secret-strip gap and normalize
upload-artifact version (#1421)
- replace stream-based lookahead with array indexing in
list-changed-files.sh (#1376)
- centralize ISO 8601 timestamp regex in CIHelpers (#1343)
- update stale documentation date in release-process.md (#1363)
- pin basic-ftp to 5.3.0 to resolve GHSA-rp42-5vxx-qpwr (#1374)
- add bot filter to dependency PR review workflow (#1362)
- resolve pip-audit findings in powerpoint, gitlab, and jira skill lock
files (#1360)
- standardize Timestamp JSON key casing across all lint result files
(#1314)
- add synchronize trigger to PR Review workflow (#1323)
- standardize timestamp in Validate-SkillStructure.ps1 to use
Get-StandardTimestamp (#1280)
- add parallel subagent dispatch and structured JSON contracts to
code-review-full (#1304)
- standardize timestamp in SecurityHelpers.psm1 to use
Get-StandardTimestamp (#1284)
- standardize timestamps in Test-DependencyPinning.ps1 and
SecurityClasses.psm1 (#1282)
- derive collection artifact counts from YAML at build time (#1275)
- standardize timestamp in FrontmatterValidation.psm1 to use
Get-StandardTimestamp (#1285)
- standardize timestamp in Markdown-Link-Check.ps1 to use
Get-StandardTimestamp (#1283)
- escape hyphens in Mermaid diagram on Collections page (#1262)
- add summary timestamp to PSScriptAnalyzer output (#1211)
- fix plugin compatibility and robustness for coding-standards code
review agents (#1289)
- standardize timestamp in Test-CopyrightHeaders.ps1 to use
Get-StandardTimestamp (#1278)
- standardize timestamp in Invoke-YamlLint.ps1 to use
Get-StandardTimestamp (#1270)
- standardize timestamp in Invoke-LinkLanguageCheck.ps1 to use
Get-StandardTimestamp (#1264)
- fix dependency-review path filters and sparse-checkout cone mode
(#1259)
- replace invalid bare tool names with official tool identifiers (#1198)
- fix broken links and remove orphaned reference in code review docs
(#1257)
- exclude Python env dirs from skill validation warnings (#1255)
- pin happy-dom and serialize-javascript to resolve Dependabot
vulnerabilities (#1253)
- remove Mermaid diagram and add missing collection cards (#1247)
- disable MCP servers by default to prevent token limit errors (#1144)
- sync package-lock.json after pre-release version bump (#1236)
- separate mermaid node declarations and add dynamic diagram generation
with tests (#1215)
- replace anchor links in meeting-analyst with bold text references
(#1201)
- remove recursive symlinks in jira and gitlab skill directories (#1233)
- validate-installation scripts now check .github/skills directory
(#1010) (#1206)
- resolve npm audit vulnerabilities via dependency overrides (#1200)
- add post-release triggers to scorecard workflow (#1186)
- add missing .md extensions to relative links in agent documentation
(#1180)

### 📚 Documentation

- broaden Security Review description beyond OWASP (#1385)
- document maintainer advisory mode and skip-review label guard (#1386)
- document ExcludePaths/OutputPath for Invoke-LinkLanguageCheck (#1383)
- CLI getting-started: clarify plugin install commands as alternatives
(-all vs base) (#1251)

### ♻️ Refactoring

- align agent and prompt folder names to collection identifier (#1210)

### 🔧 Maintenance

- pin PSScriptAnalyzer to 1.25.0 and sync stale workflow version
comments (#1389)
- bump lxml from 6.0.2 to 6.1.0 in
/.github/skills/experimental/powerpoint (#1424)
- bump @vscode/vsce from 3.7.1 to 3.9.1 in the npm-dependencies group
(#1390)
- bump the github-actions group across 1 directory with 7 updates
(#1391)
- bump follow-redirects from 1.15.11 to 1.16.0 in /docs/docusaurus
(#1356)
- upgrade Node.js from 20 to 24 and bump cspell to v10 (#1353)
- bump basic-ftp from 5.2.0 to 5.2.1 (#1324)
- update github/gh-aw-actions requirement to
536ea1bad8c6715d098a9dc1afea8d403733acfe in the github-actions group
across 1 directory (#1298)
- update security instruction attributions and compliance (#1294)
- bump the npm-dependencies group with 2 updates (#1297)
- pre-release 3.3.41 (#1252)
- streamline RAI Planner phase structure and documentation (#1273)
- bump happy-dom from 20.8.8 to 20.8.9 in /docs/docusaurus (#1237)
- pre-release 3.3.27 (#1191)
- bump pygments from 2.19.2 to 2.20.0 in /.github/skills/gitlab/gitlab
(#1234)
- bump path-to-regexp from 0.1.12 to 0.1.13 in /docs/docusaurus (#1226)
- bump the github-actions group with 4 updates (#1231)
- add missing folders and alphabetize location lists (#1193)
- bump brace-expansion (#1224)
- bump handlebars from 4.7.8 to 4.7.9 in /docs/docusaurus (#1217)
- bump brace-expansion from 5.0.3 to 5.0.5 in /docs/docusaurus (#1213)
- pre-release 3.3.10 (#1187)
- bump markdownlint-cli2 from 0.21.0 to 0.22.0 in the npm-dependencies
group (#1175)
- bump the github-actions group with 3 updates (#1176)
- pre-release 3.3.1 (#1165)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(skills): Standardize pytest-mock as Python mocking framework

4 participants