Add NES expected edit capture feature and related tests #2614
+2,422
−9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Author
|
@microsoft-github-policy-service agree company="Microsoft" |
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request introduces the NES Expected Edit Capture feature, which allows users to record their expected code edits when Next Edit Suggestions (NES) are rejected or fail to appear. The feature captures user edits in .recording.w.json format for analysis and model improvement. It also significantly enhances privacy protection by implementing comprehensive sensitive file filtering to prevent accidental exposure of credentials and secrets in feedback submissions.
Key Changes:
- New capture controller that leverages the existing
DebugRecorderinfrastructure to track user edits during capture sessions - Feedback submission system that creates pull requests in a private GitHub repository with user-selectable file filtering
- Enhanced sensitive file patterns covering environment files, credentials, private keys, and sensitive configuration directories
- Commands and keybindings for manual capture initiation (
Ctrl+K Ctrl+R), automatic capture on rejection, and feedback submission - Comprehensive test coverage for filtering logic with performance benchmarks
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
src/platform/inlineEdits/common/inlineEditLogContext.ts |
Added public getter for endpointInfo to expose model/endpoint metadata needed for rejection metadata |
src/platform/configuration/common/configurationService.ts |
Added two new team-internal settings to enable/configure the expected edit capture feature |
src/extension/inlineEdits/vscode-node/jointInlineCompletionProvider.ts |
Integrated ExpectedEditCaptureController into the joint completions provider |
src/extension/inlineEdits/vscode-node/inlineEditProviderFeature.ts |
Registered capture commands, added context key management for keybindings, and changed setContext to return IDisposable |
src/extension/inlineEdits/vscode-node/inlineEditModel.ts |
Moved debugRecorder initialization to constructor body to allow earlier access by capture controller |
src/extension/inlineEdits/vscode-node/inlineCompletionProvider.ts |
Added capture controller integration and automatic capture trigger on rejection with metadata extraction |
src/extension/inlineEdits/vscode-node/components/test/nesFeedbackSubmitter.spec.ts |
Comprehensive unit tests for feedback file filtering, path extraction, and performance benchmarks |
src/extension/inlineEdits/vscode-node/components/test/inlineEditDebugComponent.spec.ts |
Added tests for sensitive file pattern matching across various file types and directory structures |
src/extension/inlineEdits/vscode-node/components/nesFeedbackSubmitter.ts |
New service for collecting, filtering, and submitting feedback captures to GitHub via PR creation |
src/extension/inlineEdits/vscode-node/components/inlineEditDebugComponent.ts |
Expanded sensitive file patterns with comprehensive filtering for credentials, keys, and config files |
src/extension/inlineEdits/vscode-node/components/expectedEditCaptureController.ts |
Core controller managing capture sessions, status bar UI, edit extraction, and file persistence |
src/extension/inlineEdits/test/node/debugRecorder.spec.ts |
Added blank line (whitespace-only change) |
package.json |
Added four new commands for capture workflow, menu contributions, and keybindings |
docs/nes-expected-edit-capture.md |
Comprehensive documentation covering feature usage, architecture, file formats, and privacy considerations |
src/extension/inlineEdits/vscode-node/components/inlineEditDebugComponent.ts
Outdated
Show resolved
Hide resolved
src/extension/inlineEdits/vscode-node/components/inlineEditDebugComponent.ts
Outdated
Show resolved
Hide resolved
src/extension/inlineEdits/vscode-node/components/inlineEditDebugComponent.ts
Show resolved
Hide resolved
| 'Authorization': `Bearer ${token}`, | ||
| 'Accept': 'application/vnd.github+json', | ||
| 'Content-Type': 'application/json', | ||
| 'X-GitHub-Api-Version': '2022-11-28', |
There was a problem hiding this comment.
The native fetch call is missing the 'User-Agent' header that is included in other API calls. GitHub API requests should consistently include a User-Agent header for proper identification and tracking. Add 'User-Agent': this._fetcherService.getUserAgentLibrary() to the headers object.
Suggested change
| 'X-GitHub-Api-Version': '2022-11-28', | |
| 'X-GitHub-Api-Version': '2022-11-28', | |
| 'User-Agent': this._fetcherService.getUserAgentLibrary() |
| 'Authorization': `Bearer ${token}`, | ||
| 'Accept': 'application/vnd.github+json', | ||
| 'Content-Type': 'application/json', | ||
| 'X-GitHub-Api-Version': '2022-11-28', |
There was a problem hiding this comment.
The native fetch call is missing the 'User-Agent' header that is included in other API calls. GitHub API requests should consistently include a User-Agent header for proper identification and tracking. Add 'User-Agent': this._fetcherService.getUserAgentLibrary() to the headers object.
Suggested change
| 'X-GitHub-Api-Version': '2022-11-28', | |
| 'X-GitHub-Api-Version': '2022-11-28', | |
| 'User-Agent': this._fetcherService.getUserAgentLibrary(), |
| 'Authorization': `Bearer ${token}`, | ||
| 'Accept': 'application/vnd.github+json', | ||
| 'Content-Type': 'application/json', | ||
| 'X-GitHub-Api-Version': '2022-11-28', |
There was a problem hiding this comment.
The native fetch call is missing the 'User-Agent' header that is included in other API calls. GitHub API requests should consistently include a User-Agent header for proper identification and tracking. Add 'User-Agent': this._fetcherService.getUserAgentLibrary() to the headers object.
Suggested change
| 'X-GitHub-Api-Version': '2022-11-28', | |
| 'X-GitHub-Api-Version': '2022-11-28', | |
| 'User-Agent': this._fetcherService.getUserAgentLibrary() |
src/extension/inlineEdits/vscode-node/components/expectedEditCaptureController.ts
Show resolved
Hide resolved
Add functionality to capture expected edits when NES suggestions are rejected: - Add ExpectedEditCaptureController for managing capture sessions - Add configuration settings for enabling the feature - Register capture commands with keybindings - Add context key for inlineEditsEnabled to enable keybindings - Include documentation for the feature
- Implemented unit tests to ensure sensitive files such as .env files, private keys, and files in sensitive directories are filtered out correctly from logs. - Added handling for Windows-style backslash paths in the filtering function. - Preserved non-document log entries during filtering. feat: add NesFeedbackSubmitter tests - Created comprehensive tests for the NesFeedbackSubmitter class, covering methods for extracting document paths and filtering recordings by excluded paths. - Ensured proper handling of metadata files and invalid JSON scenarios. - Included performance tests for filtering large recordings efficiently. fix: enhance inline completion provider with document path metadata - Updated inline completion provider to include document path in metadata during rejection captures. feat: add command for submitting expected edits in inline edit provider feature - Registered a new command to allow submission of captured edits in the inline edit provider feature.
…ctations for performance
… and additional patterns
erikportillo
force-pushed
the
erikportillo/nes-expected-edit-capture
branch
from
1d7da15 to
b0f3039
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces the NES Expected Edit Capture feature, allowing users to record their expected code edits when a Next Edit Suggestion (NES) is rejected or missing. It also significantly improves the privacy and robustness of sensitive file filtering before user feedback is submitted. The update includes new commands, keybindings, and documentation, as well as comprehensive tests to ensure sensitive files are properly excluded from logs.