Introducing Sonargit, an automated bot scraper engineered to streamline data extraction from GitHub pull requests and capture dynamic SonarQube screenshots. Built specificaly for Catalyst engineers. Increase productivity with a bot that effortlessly extracts, analyzes, and visualizes KPI data.
Node 18.0 and later.
npm install -g sonargitExecute with the desired start date:
sonargit -s 10/10/2023Execute with date range:
sonargit -s 10/10/2023 -e 10/12/2023Execute with custom output:
sonargit -s 10/10/2023 -o my_output.csvLog files are generated in the ~/Users/<username>/sonargit directory.
Sonargit will fetches closed pull requests from a GitHub repository within a specified date range and meeting certain criteria (e.g., merged into a specific branch, closed by a specific author). The GitHub API is utilized to retrieve information about pull requests, including the total count and an array of items.
The fetchIssueComments function fetches comments on a GitHub pull request, extracts coverage percentage and SonarQube URL from the latest comment, and returns the result as an object.
The extractSonarQubeUrl function uses a regular expression to extract the SonarQube URL from a given text.
Puppeteer is used to launch a headless browser and navigate to a SonarQube login page. It then logs in with user provided credentials. After logging in, it iterates through an array of pull requests and takes a screenshot of the corresponding SonarQube page.
The sequentialProcess function used to handle an array of pull requests in a sequential execution to avoid exceeding GitHub rate limits and prevent high memory usage when using Puppeteer.
The moment library is used for date and time formatting.
Sonargit requires a GitHub personal access token (token) for making authenticated requests to the GitHub API.
Sonargit Scraper is licensed under the MIT License.
For information on contributing to Sonargit Scraper, please refer to the Contributing Guidelines.
For support and bug reporting, please open an issue on the issue tracker.