diff --git a/CHANGELOG.md b/CHANGELOG.md index 45085ad..50c4e99 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,13 +2,13 @@ All notable portfolio milestones are documented in this file. -## Unreleased — Day 6 Automation +## Unreleased — v1.1.0 Day 6 Automation ### Added - Newman as a project dependency - `package.json` and `package-lock.json` -- Public-safe CI smoke collection +- Public-safe 19-request CI smoke/regression collection - GitHub Actions API workflow - Repository-secret credential handling - Smoke JSON and JUnit reporting @@ -19,6 +19,7 @@ All notable portfolio milestones are documented in this file. - API, database and automation executive summary - CI troubleshooting case study - Day 6 screenshot evidence index +- 45 passing CI assertions across 19 requests ### Changed @@ -28,6 +29,8 @@ All notable portfolio milestones are documented in this file. - Expanded the project journal through Day 6 - Expanded the screenshot index with automation evidence - Finalized workflow triggers for `main`, pull requests and manual runs +- Merged the Day 6 pull request into `main` +- Verified the final `main` run with 19 requests and 45 passing assertions ### Preserved @@ -45,8 +48,8 @@ All notable portfolio milestones are documented in this file. - Generated Newman output remains ignored locally - Public assets remain sanitized -After the Day 6 pull request is merged and the final `main` workflow is -verified, rename this section to: +After the documentation alignment is committed and the GitHub release +is published, rename this section to: ```text ## 1.1.0 — Day 6 Newman and GitHub Actions Automation diff --git a/README.md b/README.md index 23a8e3d..27ddd00 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,35 @@ # API, Database and CI Test Automation Portfolio -[![Newman API Tests](https://github.com/daryal89/api-database-testing-portfolio/actions/workflows/newman-api-tests.yml/badge.svg)](https://github.com/daryal89/api-database-testing-portfolio/actions/workflows/newman-api-tests.yml) +[![Newman API Tests](https://github.com/daryal89/api-database-testing-portfolio/actions/workflows/newman-api-tests.yml/badge.svg?branch=main)](https://github.com/daryal89/api-database-testing-portfolio/actions/workflows/newman-api-tests.yml) ## Project Overview -This portfolio demonstrates an end-to-end quality-assurance workflow for -a sample online booking system. - -The project progresses from requirements and manual test design through -database validation, REST API testing, defect reporting and continuous -integration: +This portfolio demonstrates a complete quality-assurance workflow for a +sample online booking system: - Requirements analysis and software test planning -- Manual and REST API test-case design +- Manual and REST API test design - Requirements traceability - PostgreSQL database validation -- Postman collection development and execution -- Positive, negative, boundary-value and authorization testing -- Automated API assertions +- Postman API execution - Defect investigation and reporting -- Newman command-line execution -- GitHub Actions continuous-integration testing -- JSON and JUnit test-report generation -- Secure credential handling with repository secrets -- Public-safe evidence and technical documentation +- Newman command-line automation +- GitHub Actions continuous integration +- JSON and JUnit reporting +- Secure credential handling +- Public-safe technical evidence The REST API phase uses the public **Restful Booker** demonstration API. The database phase uses a separate, locally created PostgreSQL portfolio database. The public API does not provide authorized access to its underlying -database. API results and local database results are therefore -documented as separate testing activities rather than as direct -API-to-database reconciliation. +database. API and database results are therefore documented as separate +testing activities rather than as direct API-to-database reconciliation. ## Key Results -### Day 4 — PostgreSQL Validation +### PostgreSQL Database Validation | Metric | Result | |---|---:| @@ -45,45 +38,43 @@ API-to-database reconciliation. | Database-constraint negative tests | 6 of 6 passed | | Production-style integrity failures | 0 | -### Day 5 — Postman REST API Testing +### Complete Postman and Newman Validation | Metric | Result | |---|---:| | Manual/API test cases executed | 25 | | Test cases passed | 18 | | Test cases failed | 7 | -| Postman requests executed | 26 | -| Postman assertions executed | 61 | +| Requests executed | 26 | +| Assertions executed | 61 | | Assertions passed | 51 | -| Assertions failed | 10 | -| Postman runtime errors | 0 | -| Average response time | 36 ms | +| Known assertion failures | 10 | +| Runtime errors | 0 | | Confirmed API defects | 3 | -### Day 6 — Newman and GitHub Actions +### GitHub Actions CI Quality Gate | Metric | Result | |---|---:| -| CI smoke requests | 8 | -| CI smoke assertions | 22 | -| CI smoke assertions passed | 22 | -| CI smoke assertions failed | 0 | -| CI smoke runtime errors | 0 | -| Full Newman validation requests | 26 | -| Full Newman validation assertions | 61 | -| Known full-suite assertion failures | 10 | -| Full-suite runtime errors | 0 | -| GitHub Actions smoke quality gate | Pass | +| CI requests executed | 19 | +| CI test scripts executed | 19 | +| CI pre-request scripts executed | 2 | +| CI assertions executed | 45 | +| CI assertions passed | 45 | +| CI assertions failed | 0 | +| CI runtime errors | 0 | +| GitHub Actions result | Pass | | Automated report formats | JSON and JUnit | | Workflow artifact groups | 2 | -The full validation suite intentionally preserves the Day 5 requirement -mismatches. The separate smoke suite is the strict passing CI quality -gate. +The final `main` workflow completed successfully. + +Newman uses `test-scripts` to describe post-response scripts. The +project's documented manual/API baseline remains 25 test cases. ## Quick Links -### Planning and Requirements +### Planning and Traceability - [Software Test Plan](test-plan/booking-system-test-plan.md) - [Requirements Baseline](requirements-traceability/booking-system-requirements.md) @@ -100,215 +91,185 @@ gate. - [DEF-API-002 — Nonpositive Total Price](defect-reports/DEF-API-002-nonpositive-total-price.md) - [DEF-API-003 — Missing Required Fields Return HTTP 500](defect-reports/DEF-API-003-missing-required-fields-return-500.md) -### Postman API Testing +### Postman and Automation - [Postman Documentation](postman/README.md) - [Complete Public-Safe Collection](postman/restful-booker-api-portfolio.public-sanitized.postman_collection.json) -- [Public-Safe CI Smoke Collection](postman/restful-booker-ci-smoke.public-sanitized.postman_collection.json) +- [Stable CI Collection](postman/restful-booker-ci-smoke.public-sanitized.postman_collection.json) +- [CI Coverage Mapping](docs/day6-ci-coverage-mapping.md) - [Sanitized Environment Template](postman/restful-booker-template.public-sanitized.postman_environment.json) -- [Sanitized Day 5 Runner Result](postman/day5-postman-collection-run-results.public-sanitized.json) -- [Day 5 API Execution Report](reports/day5-api-validation-execution.md) - -### Automation and CI/CD - - [Automation Documentation](automation/README.md) - [GitHub Actions Workflow](.github/workflows/newman-api-tests.yml) -- [Day 6 Newman and GitHub Actions Report](reports/day6-newman-and-github-actions-execution.md) -- [API, Database and Automation Executive Summary](reports/api-database-automation-executive-summary.md) -- [Day 6 CI Troubleshooting Case Study](docs/day6-ci-troubleshooting-case-study.md) -- [Day 6 Automation Evidence](screenshots/README.md#day-6-automation-evidence) -### PostgreSQL Database Testing +### Reports and Evidence -- [SQL Database Testing Documentation](sql/README.md) -- [SQL Validation Queries](sql/03_validation_queries.sql) -- [Automated Database Validation Summary](reports/day4-database-validation-summary.csv) - [Day 4 Database Execution Report](reports/day4-database-validation-execution.md) - -### Evidence and Project History - +- [Day 5 API Execution Report](reports/day5-api-validation-execution.md) +- [Day 6 Newman and GitHub Actions Report](reports/day6-newman-and-github-actions-execution.md) +- [API, Database and Automation Executive Summary](reports/api-database-automation-executive-summary.md) - [Screenshot Evidence Index](screenshots/README.md) +- [Day 6 CI Troubleshooting Case Study](docs/day6-ci-troubleshooting-case-study.md) - [Project Development Journal](docs/project-development-journal.md) - [Project Changelog](CHANGELOG.md) -## Release Status +## Portfolio Highlights -The current stable release is: +- Designed and executed 25 manual/API test cases. +- Built a 26-request complete Postman collection with 61 assertions. +- Documented seven failed test cases and three confirmed API defects. +- Created 19 SQL validation queries, 13 automated database checks and + six database-constraint negative tests. +- Installed Newman as a project dependency with a committed lockfile. +- Created a stable 19-request CI smoke/regression collection. +- Executed 45 CI assertions with zero failures. +- Protected credentials with GitHub repository secrets. +- Generated JSON and JUnit reports as workflow artifacts. +- Investigated and corrected an initial CI design failure without + weakening valid defect assertions. +- Preserved clear boundaries between the public API and local database. + +## Test and Automation Strategy -[Day 5 — Manual, API and Database Testing Baseline](https://github.com/daryal89/api-database-testing-portfolio/releases/tag/v1.0.0) +The repository contains two public-safe Postman collections: -After the Day 6 pull request is merged and the final `main` workflow is -verified, the automation milestone can be published as `v1.1.0`. +| Collection | Purpose | Actual Baseline | CI Role | +|---|---|---|---| +| Complete validation collection | Full positive, negative, boundary, authorization and defect coverage | 26 requests, 61 assertions, 10 known failures | Reporting suite | +| Stable CI smoke/regression collection | Health check plus 18 passing Day 5 scenarios selected for CI | 19 requests, 45 assertions, zero failures | Strict quality gate | -## Portfolio Highlights +The CI collection excludes the seven requests associated with confirmed +Day 5 requirement mismatches: -- Designed and executed a complete booking CRUD workflow using Postman. -- Connected failed test cases to requirements, defect reports and - screenshot evidence. -- Preserved the original defect-draft history to show which hypotheses - were confirmed or rejected. -- Built a local PostgreSQL validation suite with data-quality and - constraint-negative testing. -- Automated API execution with Node.js, npm and Newman. -- Separated complete requirement validation from a stable CI smoke gate. -- Protected demonstration credentials with GitHub repository secrets. -- Generated JSON and JUnit results as downloadable workflow artifacts. -- Investigated and corrected an initial CI failure without weakening - valid defect assertions. -- Sanitized public assets to prevent credential and token exposure. -- Documented the separation between the public API and local database. - -## Business Scenario - -The sample online booking system allows users to: - -- Authenticate -- Create a booking -- Retrieve booking information -- Search for bookings -- Fully update a booking -- Partially update a booking -- Delete a booking - -Testing validates whether the system: - -- Processes valid information correctly -- Rejects invalid or missing information appropriately -- Enforces authentication and authorization requirements -- Returns accurate API responses -- Maintains database integrity -- Handles expected and unexpected error conditions clearly -- Continues to support a stable critical workflow in CI - -## Testing Scope +```text +TC-BOOK-002 +TC-BOOK-003 +TC-BOOK-004 +TC-BOOK-005 +TC-BOOK-006 +TC-VAL-002 +TC-UPD-005 +``` -### In Scope +Those scenarios remain unchanged in the complete collection. -- Requirement analysis and test planning -- Authentication and authorization testing -- Booking creation, retrieval, search, update and deletion -- Required-field, date and price validation -- Positive, negative and boundary-value testing -- API status-code, response-body and response-time validation -- PostgreSQL data-integrity and data-quality validation -- Requirements traceability -- Defect reporting -- Newman command-line execution -- API smoke-test design -- GitHub Actions workflow execution -- JSON and JUnit reporting -- Workflow artifact retention -- Public repository security review +The smoke/regression suite determines the final workflow result. The +complete suite runs with `continue-on-error` so its known failures remain +visible and its JSON and JUnit reports are still uploaded. -### Out of Scope +## Confirmed API Defects -- Production security penetration testing -- High-volume load and stress testing -- Production performance testing -- Payment-card processing -- Mobile application testing -- Production customer data -- Production deployment -- Disaster-recovery testing -- Full accessibility certification -- Direct API-to-database validation without authorized database access +| Defect | Finding | Related Test Cases | Severity | +|---|---|---|---| +| [DEF-API-001](defect-reports/DEF-API-001-invalid-booking-date-sequence.md) | Checkout before check-in accepted during creation and update | TC-BOOK-004; TC-UPD-005 | High | +| [DEF-API-002](defect-reports/DEF-API-002-nonpositive-total-price.md) | Zero and negative total prices accepted | TC-BOOK-005; TC-VAL-002 | High | +| [DEF-API-003](defect-reports/DEF-API-003-missing-required-fields-return-500.md) | Missing required fields return HTTP 500 | TC-BOOK-002; TC-BOOK-003; TC-BOOK-006 | Medium | + +The severity and priority classifications are based on the assumed +portfolio requirement baseline. Restful Booker is a demonstration API. ## Skills Demonstrated -### Software Testing +### Testing -- Requirement analysis -- Test planning -- Test-case development -- Functional, positive and negative testing -- Boundary-value analysis +- Requirements analysis +- Test planning and traceability +- Functional, positive, negative and boundary testing - Authentication and authorization testing -- Business-rule and error-handling validation +- Status-code, response-body and response-time validation - Defect reporting -- Requirements traceability - Test-execution reporting -### REST API Testing - -- HTTP methods and CRUD workflows -- Status-code and response-body validation -- Response-time validation -- Authentication-token handling -- Postman environment variables -- Automated booking-ID workflow -- Collection Runner execution -- Automated Postman assertions -- Public-safe Postman exports - -### Test Automation and CI/CD +### API Automation and CI/CD +- Postman collection design +- Environment and dynamic-variable workflows - Node.js and npm - Newman command-line execution -- API smoke-test design -- npm dependency and lockfile management - GitHub Actions - YAML workflow configuration -- Strict CI quality gates -- Known-failure regression reporting -- GitHub repository secrets -- JSON and JUnit test reports +- CI quality-gate design +- Repository-secret management +- JSON and JUnit reporting - Workflow artifact management -- CI failure investigation and resolution +- CI troubleshooting and root-cause analysis ### Database Testing - PostgreSQL and SQL -- Schema and table validation -- Primary-key and foreign-key validation -- Null, duplicate and orphan detection -- Check-constraint validation -- Staging-data and data-quality testing -- Automated validation-summary checks +- Schema and relationship validation +- Required-field, duplicate and orphan detection +- Date and price validation +- Staging-data quality checks - Database-constraint negative testing -### Documentation and Tools +## Reproducing the Tests -- Git and GitHub -- Markdown -- Microsoft Excel -- Postman Desktop -- PostgreSQL and pgAdmin 4 -- Structured test, defect, RTM, execution and automation documentation +The complete collection preserves the full requirement and defect +baseline. -## Technologies and Tools +The CI collection must complete: -- Git and GitHub -- GitHub Actions -- Node.js -- npm -- Newman -- Postman Desktop -- PostgreSQL -- pgAdmin 4 -- SQL -- YAML -- JSON -- JUnit XML -- Microsoft Excel -- Markdown -- Windows and Git Bash - -## Automation Strategy +```text +19 requests +19 test scripts +2 pre-request scripts +45 assertions +0 failures +0 runtime errors +``` -The repository contains two public-safe Postman collections: +Detailed instructions: -| Collection | Purpose | Expected Result | CI Role | -|---|---|---|---| -| Complete validation collection | Full positive, negative, boundary, authorization and defect coverage | Known failed assertions may remain | Reporting suite | -| CI smoke collection | Stable critical booking workflow | Zero failed assertions | Strict quality gate | +- [Postman Reproduction Guide](postman/README.md) +- [Newman and GitHub Actions Guide](automation/README.md) +- [Day 6 Execution Report](reports/day6-newman-and-github-actions-execution.md) -The smoke suite determines the GitHub Actions workflow result. +## Scope and Environment Boundaries -The complete suite runs with its original expectations preserved. -`continue-on-error` allows the workflow to retain and publish known -findings without converting them into artificial passes. +### In Scope -## Project Structure +- Booking authentication and CRUD workflows +- Positive, negative and boundary API testing +- Required-field, date and price validation +- PostgreSQL data-integrity testing +- Traceability and defect reporting +- Local and CI Newman execution +- Secure automated reporting + +### Out of Scope + +- Production penetration testing +- High-volume load and stress testing +- Production deployment +- Payment-card processing +- Production customer information +- Direct API-to-database reconciliation without authorized access + +## Security Controls + +- Public Postman credentials remain blank. +- Local credentials remain in an ignored environment file. +- CI credentials use `BOOKER_USERNAME` and `BOOKER_PASSWORD` repository + secrets. +- Active tokens are not committed. +- Generated local Newman results remain ignored. +- Public screenshots and exports are sanitized. +- Workflow permissions are limited to repository-content read access. + +## Limitations + +- Restful Booker is a public demonstration service. +- Booking data may reset. +- Dynamic IDs and response times may vary. +- The requirement baseline is assumed for portfolio testing. +- The local PostgreSQL database is separate from the public API. +- The CI collection excludes seven known-failure scenarios by design. +- The complete collection remains authoritative for defect coverage. +- Workflow artifacts expire according to the configured retention + period. + +
+Repository Structure ```text api-database-testing-portfolio/ @@ -325,315 +286,97 @@ api-database-testing-portfolio/ │ └── README.md ├── docs/ │ ├── README.md +│ ├── day6-ci-coverage-mapping.md │ ├── day6-ci-troubleshooting-case-study.md │ └── project-development-journal.md ├── test-plan/ -│ ├── README.md -│ └── booking-system-test-plan.md ├── test-cases/ -│ ├── README.md -│ ├── booking-system-test-cases.xlsx -│ └── booking-system-test-cases.csv ├── defect-reports/ -│ ├── README.md -│ ├── booking-system-defect-reports.xlsx -│ ├── booking-system-defect-reports.csv -│ ├── DEF-API-001-invalid-booking-date-sequence.md -│ ├── DEF-API-002-nonpositive-total-price.md -│ └── DEF-API-003-missing-required-fields-return-500.md ├── requirements-traceability/ -│ ├── README.md -│ ├── booking-system-requirements.md -│ ├── booking-system-requirements-traceability-matrix.xlsx -│ └── booking-system-requirements-traceability-matrix.csv ├── postman/ -│ ├── README.md -│ ├── restful-booker-api-portfolio.public-sanitized.postman_collection.json -│ ├── restful-booker-ci-smoke.public-sanitized.postman_collection.json -│ ├── restful-booker-template.public-sanitized.postman_environment.json -│ └── day5-postman-collection-run-results.public-sanitized.json ├── sql/ -│ ├── README.md -│ ├── 01_create_schema.sql -│ ├── 02_seed_test_data.sql -│ ├── 03_validation_queries.sql -│ ├── 04_validation_summary.sql -│ └── 05_constraint_negative_tests.sql ├── reports/ -│ ├── README.md -│ ├── day4-database-validation-summary.csv -│ ├── day4-database-validation-execution.md -│ ├── day5-api-validation-execution.md -│ ├── day6-newman-and-github-actions-execution.md -│ └── api-database-automation-executive-summary.md └── screenshots/ - ├── README.md ├── day4-database/ ├── day5-api-execution/ ├── day5-defects/ └── day6-automation/ ``` -Generated Newman JSON and JUnit files are stored locally or as GitHub -Actions artifacts. They are not committed automatically. - -## Current Project Status +
-### Completed +
+Completed Milestones -#### Foundation and Test Design +### Foundation and Test Design -- [x] Professional repository structure and documentation - [x] Requirements baseline and software test plan - [x] 25 detailed manual/API test cases -- [x] Excel and GitHub-previewable CSV test assets - [x] Requirements Traceability Matrix -- [x] Self-navigating folder README files -- [x] Recruiter-focused root README optimization +- [x] Professional folder documentation -#### PostgreSQL Database Testing +### PostgreSQL Validation -- [x] Local PostgreSQL portfolio database -- [x] Relational and staging tables -- [x] Key, required-field and business-rule constraints -- [x] Synthetic production-style records and controlled anomalies +- [x] Local schema and synthetic data - [x] 19 SQL validation and analysis queries - [x] 13 automated validation-summary checks - [x] Six database-constraint negative tests -- [x] Database execution report and screenshots - -#### Postman REST API Testing - -- [x] 26-request complete validation collection -- [x] Authentication-token and booking-ID workflows -- [x] Booking CRUD execution -- [x] Positive, negative, boundary, authorization and performance assertions -- [x] Full Collection Runner execution -- [x] Public-safe Postman exports -- [x] Reproducibility instructions - -#### API Results and Defects +- [x] Database execution report and evidence -- [x] All 25 manual/API test cases executed -- [x] 18 test cases passed -- [x] Seven test cases failed -- [x] Three confirmed API defects documented -- [x] Original defect-draft history preserved -- [x] Two hypothetical defect drafts rejected -- [x] Clickable defect evidence -- [x] Day 5 API execution report - -#### Repository Quality and Security +### Postman API Testing -- [x] Public link verification -- [x] Public repository security review -- [x] Cross-document consistency review -- [x] Organized screenshot evidence indexes +- [x] Complete 26-request validation collection +- [x] 61 automated assertions +- [x] 18 passed and seven failed test cases +- [x] Three confirmed API defects +- [x] Public-safe exports and evidence -#### Day 6 Test Automation +### Newman and GitHub Actions -- [x] Node.js, npm, Git and Newman verified - [x] Newman installed as a project dependency -- [x] Complete validation collection executed through Newman -- [x] Stable eight-request CI smoke collection created -- [x] Smoke collection passed in Postman -- [x] Smoke collection passed locally through Newman +- [x] Complete collection executed through Newman +- [x] Stable 19-request CI collection created +- [x] 45 CI assertions passed - [x] GitHub repository secrets configured -- [x] Initial GitHub Actions failure investigated -- [x] Root cause and corrective action documented -- [x] GitHub Actions smoke quality gate completed successfully -- [x] Complete validation findings preserved in CI +- [x] GitHub Actions workflow completed successfully - [x] JSON and JUnit reports generated -- [x] Workflow artifacts preserved -- [x] Day 6 evidence and execution reporting completed - -### Release Finalization - -- [ ] Merge the Day 6 pull request into `main` -- [ ] Verify the final `main` workflow -- [ ] Verify public documentation links after merge -- [ ] Publish the `v1.1.0` Day 6 release - -## Day 4 Database Validation Summary - -| Metric | Result | -|---|---:| -| SQL validation and analysis queries | 19 | -| Automated summary checks executed | 13 | -| Automated summary checks passed | 13 | -| Automated summary checks failed | 0 | -| Constraint negative tests executed | 6 | -| Constraint negative tests passed | 6 | -| Constraint negative tests failed | 0 | -| Production integrity failures identified | 0 | - -The local database is a portfolio testing environment and is not the -database used by the public demonstration booking API. - -## Day 5 API Execution Summary - -| Metric | Result | -|---|---:| -| Requests executed | 26 | -| Assertions executed | 61 | -| Assertions passed | 51 | -| Assertions failed | 10 | -| Runtime errors | 0 | -| Average response time | 36 ms | -| Manual/API test cases executed | 25 | -| Test cases passed | 18 | -| Test cases failed | 7 | -| Confirmed API defects | 3 | - -The failed assertions are intentionally retained as evidence of observed -requirement mismatches. They are not hidden or changed to create an -artificially passing collection. - -## Day 6 Automation Summary - -### CI Smoke Quality Gate - -| Metric | Result | -|---|---:| -| Requests executed | 8 | -| Assertions executed | 22 | -| Assertions passed | 22 | -| Assertions failed | 0 | -| Runtime errors | 0 | -| Local Newman exit code | 0 | -| GitHub Actions result | Pass | - -### Complete Validation Reporting - -| Metric | Result | -|---|---:| -| Requests executed | 26 | -| Assertions executed | 61 | -| Assertions passed | 51 | -| Known assertion failures | 10 | -| Runtime errors | 0 | -| CI behavior | Findings retained; failure tolerated | - -### Automated Evidence - -| Item | Result | -|---|---| -| Report formats | JSON and JUnit | -| Smoke artifact | Generated | -| Full-validation artifact | Generated | -| Artifact retention | 14 days | -| Credentials | GitHub repository secrets | - -## Current Test Coverage - -| Test Area | Test Cases | Execution Result | -|---|---:|---| -| Authentication | 5 | 5 Pass | -| Booking Creation and Validation | 8 | 3 Pass / 5 Fail | -| Booking Retrieval | 3 | 3 Pass | -| Booking Update | 5 | 4 Pass / 1 Fail | -| Booking Deletion | 4 | 4 Pass | -| **Total** | **25** | **18 Pass / 7 Fail** | - -The smoke suite reuses selected stable scenarios. It does not replace, -renumber or change the complete 25-test-case baseline. +- [x] Two artifact groups preserved +- [x] Pull request merged into `main` +- [x] Final `main` workflow verified +- [x] Documentation aligned to the actual CI result -## Confirmed API Defects - -| Defect | Summary | Related Test Cases | Severity | -|---|---|---|---| -| [DEF-API-001](defect-reports/DEF-API-001-invalid-booking-date-sequence.md) | Checkout before check-in accepted during creation and update | TC-BOOK-004; TC-UPD-005 | High | -| [DEF-API-002](defect-reports/DEF-API-002-nonpositive-total-price.md) | Zero and negative total prices accepted | TC-BOOK-005; TC-VAL-002 | High | -| [DEF-API-003](defect-reports/DEF-API-003-missing-required-fields-return-500.md) | Missing required fields return HTTP 500 | TC-BOOK-002; TC-BOOK-003; TC-BOOK-006 | Medium | - -## Requirements Traceability Summary - -| Metric | Result | -|---|---:| -| Requirements documented | 35 | -| Covered | 30 | -| Partially Covered | 3 | -| Planned | 2 | -| Execution Pass | 28 | -| Execution Fail | 5 | -| Not Run | 2 | +
-`DATA-002` and `DATA-008` remain planned because the public API is not -connected to the local PostgreSQL portfolio database. - -## Reproducing the API Tests - -The project provides two public-safe Postman collections: - -1. The complete collection preserves the full requirement, negative and - confirmed-defect baseline. -2. The CI smoke collection validates the stable critical booking - workflow. - -The complete collection may report the ten known failed assertions. - -The smoke collection must complete with zero failed assertions and zero -runtime errors. - -See: - -- [Postman Reproduction Instructions](postman/README.md) -- [Automation Documentation](automation/README.md) -- [GitHub Actions Workflow](.github/workflows/newman-api-tests.yml) -- [Day 6 Execution Report](reports/day6-newman-and-github-actions-execution.md) +## Release Status -## Data Security +The Day 6 work is merged and verified on `main`. -The public repository does not contain: +Current published release: -- Real customer information -- Real passwords -- Active authentication tokens -- Private API keys -- Confidential employer information -- Payment-card information -- Production database records -- Personal health information -- Completed private environment files -- Raw unsanitized execution output +[Day 5 — Manual, API and Database Testing Baseline](https://github.com/daryal89/api-database-testing-portfolio/releases/tag/v1.0.0) -Sensitive values are blank, represented through variables, injected from -repository secrets or replaced with `` markers. +Next release: -## Project Limitations +```text +Tag: v1.1.0 +Title: Day 6 — Newman and GitHub Actions Automation +Target: main +``` -- Restful Booker is a public demonstration API. -- Its records may reset periodically. -- Dynamic booking IDs and response times may differ between runs. -- The requirement baseline is assumed for portfolio testing. -- The local PostgreSQL database is separate from the public API. -- Direct API-to-database reconciliation was not performed. -- The CI smoke suite proves the selected critical workflow, not every - negative requirement in the complete suite. -- Workflow artifacts are execution evidence, not permanent production - monitoring. +The remaining Day 6 step is to publish `v1.1.0` after this +documentation-only alignment is merged. ## Project Development History -The portfolio was completed through structured milestones: - - **Day 1** — Repository foundation - **Day 2** — Requirements and test planning - **Day 3** — Test cases, RTM and defect drafts -- **Day 4** — PostgreSQL database validation +- **Day 4** — PostgreSQL validation - **Day 5** — Postman execution and confirmed defects -- **Day 6** — Newman, smoke-test design and GitHub Actions automation +- **Day 6** — Newman, CI collection design and GitHub Actions automation [View the complete project development journal](docs/project-development-journal.md) -## Next Planned Activities - -1. Merge the Day 6 pull request after all required checks pass. -2. Verify the final workflow and links on `main`. -3. Publish the `v1.1.0` automation release. -4. Add the project to the résumé and prepare the CI troubleshooting - interview story. - ## Author **Dhruba Aryal** @@ -642,5 +385,4 @@ QA, REST API, Database and CI Test Automation Portfolio ## License -This project is licensed under the MIT License. See the -[LICENSE](LICENSE) file for details. +This project is licensed under the MIT License. See [LICENSE](LICENSE). diff --git a/automation/README.md b/automation/README.md index 7205b3a..952109a 100644 --- a/automation/README.md +++ b/automation/README.md @@ -10,7 +10,7 @@ The automation uses: - Node.js and npm - Newman - A complete validation collection -- A separate CI smoke collection +- A separate stable CI smoke/regression collection - GitHub Actions - GitHub repository secrets - JSON and JUnit reports @@ -26,7 +26,7 @@ The repository deliberately separates two testing purposes: | Suite | Purpose | Expected Result | Workflow Behavior | |---|---|---|---| | Complete validation suite | Execute the full positive, negative, boundary, authorization and defect baseline | Known failed assertions may remain | Runs for reporting; failure is tolerated | -| CI smoke suite | Validate the stable critical booking workflow | Zero failed assertions | Strict quality gate | +| CI smoke/regression suite | Validate the health check and 18 passing Day 5 scenarios selected for CI | 19 requests, 45 assertions, zero failures | Strict quality gate | The smoke suite determines the final GitHub Actions result. @@ -37,6 +37,7 @@ not weakened merely to produce a green workflow. - [Complete Validation Collection](../postman/restful-booker-api-portfolio.public-sanitized.postman_collection.json) - [CI Smoke Collection](../postman/restful-booker-ci-smoke.public-sanitized.postman_collection.json) +- [CI Coverage Mapping](../docs/day6-ci-coverage-mapping.md) - [Sanitized Environment Template](../postman/restful-booker-template.public-sanitized.postman_environment.json) - [GitHub Actions Workflow](../.github/workflows/newman-api-tests.yml) - [Day 6 Execution Report](../reports/day6-newman-and-github-actions-execution.md) @@ -172,28 +173,68 @@ Expected documented baseline: | Metric | Result | |---|---:| -| Requests | 8 | -| Assertions | 22 | -| Passed assertions | 22 | +| Requests | 19 | +| Test scripts | 19 | +| Pre-request scripts | 2 | +| Assertions | 45 | +| Passed assertions | 45 | | Failed assertions | 0 | | Runtime errors | 0 | | Exit code | 0 | ## CI Smoke Workflow -The smoke collection covers the stable core workflow: +The workflow step is named **CI smoke collection**, but the collection +also provides stable regression coverage. -1. API health check -2. Valid authentication -3. Valid booking creation -4. Retrieve the created booking -5. Authorized full booking update -6. Authorized partial booking update -7. Authorized booking deletion -8. Verify the deleted booking returns HTTP 404 +It contains 19 requests: -Known-defect and broad negative scenarios remain in the complete -validation collection. +- One API health check +- The 18 Day 5 test scenarios that passed +- Authentication behavior +- Valid booking creation +- Booking retrieval and search +- Authorized and unauthorized updates +- Nonexistent-resource update handling +- Authorized and unauthorized deletion +- Verification after deletion +- Nonexistent-resource deletion handling + +The CI collection excludes the seven Day 5 failed scenarios: + +```text +TC-BOOK-002 +TC-BOOK-003 +TC-BOOK-004 +TC-BOOK-005 +TC-BOOK-006 +TC-VAL-002 +TC-UPD-005 +``` + +Those seven requests remain in the complete validation collection and +continue to provide confirmed-defect evidence. + +### Recorded Main-Branch CI Result + +| Metric | Result | +|---|---:| +| Requests executed | 19 | +| Test scripts executed | 19 | +| Pre-request scripts executed | 2 | +| Assertions executed | 45 | +| Assertions passed | 45 | +| Assertions failed | 0 | +| Runtime errors | 0 | +| Total duration | 2.4 seconds | +| Average response time | 12 ms | + +Duration and response-time values describe the recorded run and may vary +in later executions. + +In Newman's summary, `test-scripts` refers to post-response scripts. It +does not create a separate 19-test-case baseline; the project's manual/API +baseline remains 25 test cases. ## GitHub Actions Workflow @@ -323,9 +364,10 @@ The documented requirement mismatches are: These findings remain visible in the complete validation suite. -The smoke suite does not redefine the affected requirements. It simply -uses stable scenarios to determine whether the critical API workflow is -available for CI. +The CI suite does not redefine the affected requirements. It uses the +health check and 18 passing Day 5 scenarios selected for CI to provide a broad, +stable quality gate while the complete suite remains authoritative for +known-failure coverage. ## Security Controls @@ -380,7 +422,8 @@ consistent. - Restful Booker is a public demonstration service. - API data may reset. - Response times and booking identifiers may vary. -- The smoke suite covers only the critical workflow. +- The CI suite covers 19 stable requests but excludes the seven known- + failure requests by design. - The complete suite remains the authoritative requirement-validation baseline. - The local PostgreSQL database is separate from the public API. @@ -392,4 +435,5 @@ consistent. - [Day 6 Execution Report](../reports/day6-newman-and-github-actions-execution.md) - [Executive Summary](../reports/api-database-automation-executive-summary.md) - [CI Troubleshooting Case Study](../docs/day6-ci-troubleshooting-case-study.md) +- [CI Coverage Mapping](../docs/day6-ci-coverage-mapping.md) - [Screenshot Evidence](../screenshots/README.md#day-6-automation-evidence) diff --git a/docs/README.md b/docs/README.md index bc26e15..1870b98 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,6 +8,7 @@ README. - [Project Development Journal](project-development-journal.md) - [Day 6 CI Troubleshooting Case Study](day6-ci-troubleshooting-case-study.md) +- [Day 6 CI Coverage Mapping](day6-ci-coverage-mapping.md) - [Main Project README](../README.md) - [API, Database and Automation Executive Summary](../reports/api-database-automation-executive-summary.md) - [Day 6 Execution Report](../reports/day6-newman-and-github-actions-execution.md) @@ -21,7 +22,8 @@ The development journal preserves the chronological Day 1 through Day 6 history. The troubleshooting case study documents the initial CI failure, -root-cause analysis, corrective action and final automation strategy. +root-cause analysis, corrective action and final 19-request, +45-assertion automation strategy. Historical `Not Run`, draft and planned statuses describe the project at that earlier milestone. They are not the current final status. diff --git a/docs/day6-ci-coverage-mapping.md b/docs/day6-ci-coverage-mapping.md new file mode 100644 index 0000000..4ed6a78 --- /dev/null +++ b/docs/day6-ci-coverage-mapping.md @@ -0,0 +1,117 @@ +# Day 6 CI Coverage Mapping + +## Purpose + +This document explains how the 19-request CI collection relates to the +complete 26-request Postman validation collection and the 25 documented +manual/API test cases. + +The CI collection does not create new manual test cases and does not +change any Day 5 expected result. + +## Final CI Baseline + +| Metric | Result | +|---|---:| +| Requests executed | 19 | +| Test scripts executed | 19 | +| Pre-request scripts executed | 2 | +| Assertions executed | 45 | +| Assertions passed | 45 | +| Assertions failed | 0 | +| Runtime errors | 0 | +| GitHub Actions result | Pass | + +Newman uses `test-scripts` for post-response scripts. The 19 test scripts +shown in the runner summary do not mean that 19 new manual test cases +were created. + +## Included CI Requests + +The CI collection contains the API health check and the 18 Day 5 +scenarios that passed in the complete validation baseline. + +### Health Check + +- API health check + +### Authentication + +- `TC-AUTH-001` — Valid authentication +- `TC-AUTH-002` — Invalid password +- `TC-AUTH-003` — Missing username +- `TC-AUTH-004` — Missing password +- `TC-AUTH-005` — Blank credentials + +### Booking Creation and Validation + +- `TC-BOOK-001` — Valid booking creation +- `TC-VAL-001` — Minimum valid positive price + +### Booking Retrieval and Search + +- `TC-RET-001` — Retrieve an existing booking +- `TC-RET-002` — Retrieve a nonexistent booking +- `TC-RET-003` — Search by supported customer-name criteria + +### Booking Update + +- `TC-UPD-001` — Authorized full update +- `TC-UPD-002` — Update without authentication +- `TC-UPD-003` — Update a nonexistent booking +- `TC-UPD-004` — Authorized partial update + +### Booking Deletion + +- `TC-DEL-001` — Authorized deletion +- `TC-DEL-002` — Retrieve a deleted booking +- `TC-DEL-003` — Delete without authentication +- `TC-DEL-004` — Delete a nonexistent booking + +## Excluded Known-Failure Requests + +The following seven scenarios remain unchanged in the complete +validation collection but are excluded from the strict passing CI gate: + +| Test Case | Documented Finding | +|---|---| +| `TC-BOOK-002` | Missing `firstname` returns HTTP 500 | +| `TC-BOOK-003` | Missing `bookingdates` returns HTTP 500 | +| `TC-BOOK-004` | Checkout before check-in is accepted | +| `TC-BOOK-005` | Zero total price is accepted | +| `TC-BOOK-006` | Missing `lastname` returns HTTP 500 | +| `TC-VAL-002` | Negative total price is accepted | +| `TC-UPD-005` | Invalid date sequence is accepted during update | + +These scenarios account for the seven failed Day 5 test cases and the +three confirmed defects. + +## Relationship to the Complete Suite + +| Suite | Requests | Assertions | Result | Purpose | +|---|---:|---:|---|---| +| CI smoke/regression collection | 19 | 45 | Pass | Strict CI quality gate | +| Complete validation collection | 26 | 61 | 51 pass / 10 known failures | Full requirement and defect reporting | + +The CI suite validates stable functionality. The complete suite remains +the authoritative source for negative testing and defect evidence. + +## Maintenance Rule + +When API behavior changes: + +1. Reproduce the affected scenario manually. +2. Compare the result with the assumed requirement baseline. +3. Update defect status when appropriate. +4. Preserve historical evidence in the existing release. +5. Change CI membership only after documenting the reason. +6. Never weaken a valid assertion merely to keep the workflow green. + +## Related Documentation + +- [Main Project README](../README.md) +- [Postman Documentation](../postman/README.md) +- [Automation Documentation](../automation/README.md) +- [Day 6 Execution Report](../reports/day6-newman-and-github-actions-execution.md) +- [CI Troubleshooting Case Study](day6-ci-troubleshooting-case-study.md) +- [Screenshot Evidence](../screenshots/README.md#day-6-automation-evidence) diff --git a/docs/day6-ci-troubleshooting-case-study.md b/docs/day6-ci-troubleshooting-case-study.md index 86684f0..936fe47 100644 --- a/docs/day6-ci-troubleshooting-case-study.md +++ b/docs/day6-ci-troubleshooting-case-study.md @@ -87,7 +87,7 @@ The project adopted a two-suite strategy. ### CI Smoke Suite - Contains eight stable critical requests -- Executes 22 assertions +- Executes 45 assertions across 19 requests - Must pass all assertions - Uses Newman's normal exit-code behavior - Determines the final workflow result @@ -100,12 +100,14 @@ The corrected design was validated at three levels: 2. Local Newman execution 3. GitHub Actions -The smoke suite completed with: +The stable CI smoke/regression suite completed with: ```text -8 requests -22 assertions -22 passed +19 requests +19 test scripts +2 pre-request scripts +45 assertions +45 passed 0 failed 0 runtime errors ``` @@ -173,7 +175,8 @@ meaningful passing CI design without weakening tests. - Confirmed Newman executed successfully - Mapped failed assertions to existing defect reports - Preserved the complete collection -- Created a focused eight-request smoke collection +- Created a stable 19-request CI smoke/regression collection from the + health check and 18 passing Day 5 scenarios selected for CI - Configured repository secrets - Used the smoke suite as the strict gate - Used the complete suite for known-failure reporting @@ -181,7 +184,7 @@ meaningful passing CI design without weakening tests. ### Result -The smoke suite passed locally and in GitHub Actions with 22 of 22 +The CI suite passed locally and in GitHub Actions with 45 of 45 assertions, while the complete suite continued to expose ten documented requirement mismatches. diff --git a/docs/project-development-journal.md b/docs/project-development-journal.md index fbf547f..546a6f1 100644 --- a/docs/project-development-journal.md +++ b/docs/project-development-journal.md @@ -232,8 +232,9 @@ Day 6 accomplishments included: - Installed Newman as a project dependency - Executed the complete Postman collection through Newman - Preserved the 61-assertion baseline and ten known failures -- Created an eight-request stable CI smoke collection -- Executed 22 passing smoke assertions +- Created a stable 19-request CI smoke/regression collection from the + health check and 18 passing Day 5 scenarios selected for CI +- Executed 45 passing CI assertions - Configured GitHub repository secrets - Created a GitHub Actions workflow - Investigated an initial exit-code-1 workflow result @@ -248,10 +249,10 @@ Day 6 accomplishments included: | Metric | Result | |---|---:| -| Smoke requests | 8 | -| Smoke assertions | 22 | -| Smoke assertions passed | 22 | -| Smoke assertions failed | 0 | +| CI requests | 19 | +| CI assertions | 45 | +| CI assertions passed | 45 | +| CI assertions failed | 0 | | Smoke runtime errors | 0 | | Full Newman requests | 26 | | Full Newman assertions | 61 | @@ -262,8 +263,9 @@ Day 6 accomplishments included: | Report formats | JSON and JUnit | | Artifact groups | 2 | -The smoke suite does not replace the complete test baseline. It -validates the stable critical workflow for CI. +The CI suite does not replace the complete test baseline. It validates +the health check and 18 passing Day 5 scenarios selected for CI while the complete +suite retains the seven known-failure scenarios. ## Current Final Status @@ -277,9 +279,9 @@ After Day 6: | Complete Postman/Newman requests | 26 | | Complete assertions | 61 | | Known complete-suite failures | 10 | -| CI smoke requests | 8 | -| CI smoke assertions passed | 22 | -| CI smoke failures | 0 | +| CI smoke/regression requests | 19 | +| CI smoke/regression assertions passed | 45 | +| CI smoke/regression failures | 0 | | Confirmed API defects | 3 | | GitHub Actions smoke gate | Pass | | Automated report formats | JSON and JUnit | @@ -287,5 +289,5 @@ After Day 6: The Day 1 through Day 3 planning and draft history remains preserved for auditability and learning purposes. -The Day 6 pull request must still be merged, verified on `main` and -published as the `v1.1.0` milestone. +The Day 6 pull request has been merged and verified on `main`. The +remaining milestone is publication of the `v1.1.0` release. diff --git a/postman/README.md b/postman/README.md index be842db..461f594 100644 --- a/postman/README.md +++ b/postman/README.md @@ -12,6 +12,7 @@ demonstration service. - [Sanitized Day 5 Collection Runner Result](day5-postman-collection-run-results.public-sanitized.json) - [Day 5 API Execution Report](../reports/day5-api-validation-execution.md) - [Day 6 Automation Report](../reports/day6-newman-and-github-actions-execution.md) +- [CI Coverage Mapping](../docs/day6-ci-coverage-mapping.md) - [Automation Documentation](../automation/README.md) - [GitHub Actions Workflow](../.github/workflows/newman-api-tests.yml) - [Screenshot Evidence Index](../screenshots/README.md) @@ -44,41 +45,53 @@ It demonstrates: ## CI Smoke Collection Coverage -The CI smoke collection contains eight requests: +The workflow names this the CI smoke collection. In practice, it is a +stable smoke/regression suite containing 19 requests and 45 assertions. -| Order | Request Purpose | -|---:|---| -| 1 | Verify API health | -| 2 | Authenticate successfully | -| 3 | Create a valid booking | -| 4 | Retrieve the created booking | -| 5 | Perform an authorized full update | -| 6 | Perform an authorized partial update | -| 7 | Delete the booking | -| 8 | Confirm the deleted booking returns HTTP 404 | +Its composition is: -The documented smoke baseline is: +| Component | Count | +|---|---:| +| API health check | 1 request | +| Day 5 passing test scenarios | 18 requests | +| **Total** | **19 requests** | + +Recorded main-branch result: | Metric | Result | |---|---:| -| Requests | 8 | -| Assertions | 22 | -| Passed assertions | 22 | +| Requests | 19 | +| Test scripts | 19 | +| Pre-request scripts | 2 | +| Assertions | 45 | +| Passed assertions | 45 | | Failed assertions | 0 | | Runtime errors | 0 | +The seven known-failure requests remain only in the complete collection: + +```text +TC-BOOK-002 +TC-BOOK-003 +TC-BOOK-004 +TC-BOOK-005 +TC-BOOK-006 +TC-VAL-002 +TC-UPD-005 +``` + ## Automation Strategy | Collection | Purpose | Expected Result | CI Role | |---|---|---|---| | Complete collection | Full positive, negative, boundary, authorization and defect validation | Known failures may remain | Reporting suite | -| CI smoke collection | Stable critical workflow validation | Zero failures | Strict quality gate | +| CI smoke/regression collection | Health check plus 18 passing Day 5 scenarios selected for CI | 19 requests and 45 assertions with zero failures | Strict quality gate | The complete collection retains failed assertions associated with confirmed requirement mismatches. -The CI smoke collection does not replace or renumber the complete test -baseline. +The CI collection does not replace or renumber the complete test +baseline. It reuses the health check and 18 passing Day 5 scenarios selected for CI. ## How to Run the Complete Collection in Postman @@ -101,8 +114,8 @@ baseline. 3. Confirm valid demonstration credentials are present. 4. Clear dynamic token and booking variables. 5. Run one iteration. -6. Confirm all eight requests execute. -7. Confirm all 22 assertions pass. +6. Confirm all 19 requests execute. +7. Confirm all 45 assertions pass. 8. Confirm zero runtime errors. ## Run the Complete Collection with Newman diff --git a/reports/README.md b/reports/README.md index 485dbbf..e2f1aae 100644 --- a/reports/README.md +++ b/reports/README.md @@ -59,13 +59,15 @@ reports for the portfolio. ## Day 6 Automation Results -### Smoke Quality Gate +### CI Smoke/Regression Quality Gate | Metric | Result | |---|---:| -| Requests executed | 8 | -| Assertions executed | 22 | -| Assertions passed | 22 | +| Requests executed | 19 | +| Test scripts executed | 19 | +| Pre-request scripts executed | 2 | +| Assertions executed | 45 | +| Assertions passed | 45 | | Assertions failed | 0 | | Runtime errors | 0 | | GitHub Actions result | Pass | @@ -99,9 +101,9 @@ confirmed defects. They remain intentionally visible. Day 6 does not redefine those failures as passes. -Instead, the stable eight-request smoke suite acts as the strict CI gate, -while the complete suite remains available for requirement and defect -reporting. +Instead, the stable 19-request CI smoke/regression suite acts as the +strict gate, while the complete suite remains available for requirement +and defect reporting. The Day 4 and Day 5/Day 6 reports describe separate environments: diff --git a/reports/api-database-automation-executive-summary.md b/reports/api-database-automation-executive-summary.md index 3c7e0ad..0059a50 100644 --- a/reports/api-database-automation-executive-summary.md +++ b/reports/api-database-automation-executive-summary.md @@ -66,10 +66,10 @@ Two automation suites were used: | Suite | Purpose | Result | |---|---|---| -| CI smoke suite | Stable critical booking workflow | 8 requests, 22 of 22 assertions passed | +| CI smoke/regression suite | Health check plus 18 passing Day 5 scenarios selected for CI | 19 requests, 45 of 45 assertions passed | | Complete validation suite | Full requirement and defect evidence | 26 requests, 61 assertions, 10 known failures retained | -The smoke suite is the strict quality gate. +The 19-request CI smoke/regression suite is the strict quality gate. The complete suite continues to report known findings but does not override a successful smoke result. @@ -81,7 +81,7 @@ The GitHub Actions workflow: - Installs dependencies with `npm ci` - Uses Node.js and Newman - Validates required repository secrets -- Runs the smoke suite as a strict gate +- Runs the 19-request, 45-assertion CI suite as a strict gate - Runs the complete suite for broader reporting - Produces JSON and JUnit files - Uploads separate artifact groups @@ -105,7 +105,7 @@ which intentionally contains failed defect assertions, was used as the required passing step. The correction preserved the complete suite and introduced a separate -passing smoke suite. +19-request passing CI smoke/regression suite. This approach: @@ -120,7 +120,8 @@ This approach: - Restful Booker is a public demonstration API. - Its data may reset without notice. - Booking IDs and response times may vary. -- A passing smoke suite proves only the selected critical workflow. +- A passing CI suite proves the health check and 18 stable scenarios; + it does not convert the seven known-failure scenarios into passes. - The local PostgreSQL database is not connected to the public API. - Direct persistence validation was not possible. diff --git a/reports/day6-newman-and-github-actions-execution.md b/reports/day6-newman-and-github-actions-execution.md index 845429e..b91291f 100644 --- a/reports/day6-newman-and-github-actions-execution.md +++ b/reports/day6-newman-and-github-actions-execution.md @@ -9,7 +9,7 @@ | Milestone | Day 6 | | Prepared By | Dhruba Aryal | | Execution Period | August 4–5, 2026 | -| Execution Status | Completed; pull request pending merge | +| Execution Status | Completed; merged into `main`; release pending | | Automation Tools | Node.js, npm, Newman and GitHub Actions | ## Execution Summary @@ -80,6 +80,7 @@ reports/README.md screenshots/day6-automation/ screenshots/README.md docs/day6-ci-troubleshooting-case-study.md +docs/day6-ci-coverage-mapping.md docs/project-development-journal.md README.md CHANGELOG.md @@ -89,7 +90,7 @@ CHANGELOG.md | Suite | Purpose | Expected Result | CI Behavior | |---|---|---|---| -| CI smoke suite | Validate the stable critical booking workflow | Zero failures | Strict quality gate | +| CI smoke/regression suite | Validate the health check and 18 passing Day 5 scenarios selected for CI | 19 requests, 45 assertions, zero failures | Strict quality gate | | Complete validation suite | Preserve full requirement, negative and defect coverage | Known failures may remain | Failure tolerated; results published | The smoke collection does not replace the complete collection. @@ -99,28 +100,51 @@ the workflow pass. ## CI Smoke Collection -The smoke suite contains eight critical requests: +The workflow step is named the CI smoke collection. The implemented +collection is broader than a minimal smoke test and functions as a stable +smoke/regression suite. -1. Health check -2. Valid authentication -3. Valid booking creation -4. Retrieve created booking -5. Authorized full update -6. Authorized partial update -7. Authorized deletion -8. Retrieve deleted booking +It contains: -### Smoke Results +- One API health check +- All 18 Day 5 scenarios that passed +- 19 total requests +- 45 total assertions + +It excludes the seven requests associated with confirmed Day 5 +requirement mismatches: + +```text +TC-BOOK-002 +TC-BOOK-003 +TC-BOOK-004 +TC-BOOK-005 +TC-BOOK-006 +TC-VAL-002 +TC-UPD-005 +``` + +### Smoke/Regression Results | Metric | Result | |---|---:| -| Requests executed | 8 | -| Assertions executed | 22 | -| Assertions passed | 22 | +| Requests executed | 19 | +| Test scripts executed | 19 | +| Pre-request scripts executed | 2 | +| Assertions executed | 45 | +| Assertions passed | 45 | | Assertions failed | 0 | | Runtime errors | 0 | | Local Newman exit code | 0 | | GitHub Actions result | Pass | +| Recorded main-branch duration | 2.4 seconds | +| Recorded average response time | 12 ms | + +The duration and response time are session-specific and may change in +later runs. + +Newman uses `test-scripts` for post-response scripts. The 19 executed +test scripts do not replace the separate 25-test-case project baseline. ## Complete Validation Execution @@ -177,7 +201,8 @@ The solution was not to remove or weaken the failed assertions. Instead: 1. The complete validation suite was preserved. -2. A separate eight-request smoke suite was created. +2. A separate 19-request stable CI smoke/regression suite was created + from the health check and 18 passing Day 5 scenarios selected for CI. 3. The smoke suite was verified in Postman. 4. The smoke suite was verified locally through Newman. 5. The smoke suite became the strict GitHub Actions quality gate. @@ -307,7 +332,7 @@ The corrected full run retained: | JUnit reports | Generated | | Artifact uploads | Completed | | Day 6 documentation | Completed | -| Pull-request merge | Pending | +| Pull-request merge | Completed | ## Limitations @@ -355,8 +380,8 @@ The corrected full run retained: ## Next Steps -1. Merge the pull request after all required checks pass. -2. Verify the workflow on `main`. -3. Verify all public documentation links. -4. Publish the `v1.1.0` release. -5. Add the project to the résumé and interview portfolio. +1. Commit the documentation-only alignment to the actual 19-request, + 45-assertion main-branch result. +2. Verify all public documentation links and screenshot filenames. +3. Publish the `v1.1.0` release. +4. Add the project to the résumé and interview portfolio. diff --git a/screenshots/README.md b/screenshots/README.md index 51c0301..bbc3c5e 100644 --- a/screenshots/README.md +++ b/screenshots/README.md @@ -126,24 +126,25 @@ by the public Restful Booker API. | [Node, npm and Git versions](day6-automation/day6-01-node-npm-git-versions.png) | Confirms the local command-line environment and feature branch | | [Newman installation](day6-automation/day6-02-newman-installation.png) | Confirms Newman was installed as a project dependency | | [Complete Newman run](day6-automation/day6-03-full-newman-run-known-findings.png) | Shows 26 requests, 61 assertions and known findings retained | -| [Smoke collection structure](day6-automation/day6-04-postman-smoke-collection-structure.png) | Shows the eight critical requests selected for CI | +| [Smoke collection structure](day6-automation/day6-04-postman-smoke-collection-structure.png) | Shows the stable CI collection built from the health check and 18 passing Day 5 scenarios | | [Passing Postman smoke run](day6-automation/day6-05-postman-smoke-run-passed.png) | Shows the smoke suite passing in Postman | | [Initial GitHub Actions failure](day6-automation/day6-06-initial-github-actions-failure.png) | Documents the first CI result and generated artifact | | [Initial failure log](day6-automation/day6-07-github-actions-failure-log.png) | Shows that retained defect assertions caused exit code 1 | | [Passing local Newman smoke run](day6-automation/day6-08-local-newman-smoke-pass.png) | Shows zero smoke failures and local exit code 0 | | [Repository secret names](day6-automation/day6-09-github-actions-secret-names.png) | Confirms secure secret configuration without exposing values | | [Green GitHub Actions summary](day6-automation/day6-11-github-actions-green-summary.png) | Shows the final successful workflow | -| [Passing CI smoke log](day6-automation/day6-12-github-actions-newman-pass.png) | Shows 22 passing smoke assertions | +| [Passing CI smoke log](day6-automation/day6-12-github-actions-newman-pass.png) | Shows 19 requests and 45 assertions completing with zero failures | | [Full validation findings](day6-automation/day6-13-full-validation-known-findings.png) | Shows the complete suite findings retained in CI | | [Workflow strategy summary](day6-automation/day6-14-workflow-test-strategy-summary.png) | Shows smoke as the strict gate and full validation as reporting | | [Workflow artifacts](day6-automation/day6-15-github-actions-artifacts.png) | Shows both JSON/JUnit artifact groups | | [Pull-request checks](day6-automation/day6-16-pull-request-checks-passed.png) | Confirms validation before merge | +| [Final main-branch workflow](day6-automation/day6-17-main-branch-workflow-success.png) | Confirms the merged workflow passed on `main`, preserved complete-suite findings and generated both artifact groups | The initial failed workflow is retained as troubleshooting evidence. It does not represent the final automation result. -The final CI quality gate is the smoke suite, which completed with zero -failed assertions and zero runtime errors. +The final CI quality gate is the 19-request smoke/regression suite, +which completed 45 assertions with zero failures and zero runtime errors. ## Screenshot Security Rules @@ -179,7 +180,9 @@ Therefore: - Screenshots represent specific recorded sessions. - The public API is not connected to the local PostgreSQL portfolio database. -- A passing smoke workflow proves only the selected critical path. +- A passing CI workflow proves the health check and 18 stable passing + scenarios; the complete collection remains the source for known- + failure and defect coverage. ## Related Documentation diff --git a/screenshots/day6-automation/day6-16-pull-request-checks-passed.png b/screenshots/day6-automation/day6-16-pull-request-checks-passed.png new file mode 100644 index 0000000..03bc9f6 Binary files /dev/null and b/screenshots/day6-automation/day6-16-pull-request-checks-passed.png differ diff --git a/screenshots/day6-automation/day6-17-main-branch-workflow-success.png b/screenshots/day6-automation/day6-17-main-branch-workflow-success.png new file mode 100644 index 0000000..be0be12 Binary files /dev/null and b/screenshots/day6-automation/day6-17-main-branch-workflow-success.png differ