Replace_file command breaks if returned format is malformed

[pistachiomatt]

Super great start! Worked flawlessly in a dummy test, but I found a first real world issue:

Steps

1. Have a repo with a styled-components file that contains multi-line backticks
2. Get Codemaster to edit that file
3. When Codemaster produces an replace_file command an error occurs

Action log:

2023-03-26T04:50:51.0949494Z Creating a new issue comment:
2023-03-26T04:50:51.0952092Z ## AI Response
2023-03-26T04:50:51.0952553Z replace_file ```
2023-03-26T04:50:51.0953945Z import styled from 'styled-components'
2023-03-26T04:50:51.0954249Z 
2023-03-26T04:50:51.0954644Z export const Cont = styled.nav`
2023-03-26T04:50:51.0956236Z   display: flex;
2023-03-26T04:50:51.0956798Z   align-items: center;
2023-03-26T04:50:51.0957391Z   justify-content: space-between;
2023-03-26T04:50:51.0958308Z   width: 100%;
2023-03-26T04:50:51.0958802Z   height: var(--top-nav-height);
2023-03-26T04:50:51.0976303Z   border-bottom: 2px solid var(--border-color);
2023-03-26T04:50:51.0977670Z   background-color: var(--neutral-color-D);
2023-03-26T04:50:51.0978003Z 
2023-03-26T04:50:51.0978104Z   &>.left, &>.right {
2023-03-26T04:50:51.0978372Z     display: flex;
2023-03-26T04:50:51.0978656Z     align-items: center;
2023-03-26T04:50:51.0979007Z     height: calc(var(--top-nav-height) - 2px);
2023-03-26T04:50:51.0979399Z   }
2023-03-26T04:50:51.0979512Z 
2023-03-26T04:50:51.0979597Z   .navItem {
2023-03-26T04:50:51.0979805Z     display: flex;
2023-03-26T04:50:51.0980066Z     align-items: center;
2023-03-26T04:50:51.0980374Z     border-right: 2px solid var(--border-color);
2023-03-26T04:50:51.0980722Z     padding: 0 calc(var(--body-size) * 2);
2023-03-26T04:50:51.0980969Z     height: 100%;
2023-03-26T04:50:51.0981427Z     cursor: pointer;
2023-03-26T04:50:51.0981618Z 
2023-03-26T04:50:51.0981702Z     &.isActive {
2023-03-26T04:50:51.0982044Z       background-color: var(--neutral-color-C);
2023-03-26T04:50:51.0982291Z     }
2023-03-26T04:50:51.0982404Z 
2023-03-26T04:50:51.0982538Z     &:first-of-type {
2023-03-26T04:50:51.0982843Z       padding-left: calc(var(--body-size) * 2);
2023-03-26T04:50:51.0983082Z     }
2023-03-26T04:50:51.0983261Z   }
2023-03-26T04:50:51.0983371Z 
2023-03-26T04:50:51.0983451Z   &>.right {
2023-03-26T04:50:51.0983708Z     &>.navItem:first-of-type {
2023-03-26T04:50:51.0984043Z       border-left: 2px solid var(--border-color);
2023-03-26T04:50:51.0984284Z     }
2023-03-26T04:50:51.0984395Z 
2023-03-26T04:50:51.0984546Z     &>.navItem:last-of-type {
2023-03-26T04:50:51.0984806Z       border-right: none;
2023-03-26T04:50:51.0985017Z     }
2023-03-26T04:50:51.0985283Z   }
2023-03-26T04:50:51.0985393Z 
2023-03-26T04:50:51.0985491Z   .fingerprintIcon {
2023-03-26T04:50:51.0985739Z     font-size: 24px;
2023-03-26T04:50:51.0986037Z     color: var(--neutral-color-B);
2023-03-26T04:50:51.0986286Z     cursor: pointer;
2023-03-26T04:50:51.0986513Z     transition: color 0.3s;
2023-03-26T04:50:51.0986655Z 
2023-03-26T04:50:51.0986717Z     &:hover {
2023-03-26T04:50:51.0986998Z       color: var(--primary-color);
2023-03-26T04:50:51.0987220Z     }
2023-03-26T04:50:51.0987335Z 
2023-03-26T04:50:51.0987417Z     &:before {
2023-03-26T04:50:51.0987604Z       content: "";
2023-03-26T04:50:51.0987872Z       display: inline-block;
2023-03-26T04:50:51.0988100Z       width: 24px;
2023-03-26T04:50:51.0988310Z       height: 24px;
2023-03-26T04:50:51.0988647Z       background-image: url('/images/fingerprint24.svg');
2023-03-26T04:50:51.0988974Z       background-size: cover;
2023-03-26T04:50:51.0989312Z       mask-image: url('/images/fingerprint24.svg');
2023-03-26T04:50:51.0989610Z       mask-size: cover;
2023-03-26T04:50:51.0989799Z     }
2023-03-26T04:50:51.0989978Z   }
2023-03-26T04:50:51.0990156Z `
2023-03-26T04:50:51.0991174Z ``` Added fingerprintIcon class with the required styles and removed the border-right from the last navItem in the right section.
2023-03-26T04:50:51.0991596Z Got Chat API response in 32.1 seconds
2023-03-26T04:50:51.9361724Z Solving Issue failed. See exception below:
2023-03-26T04:50:51.9363185Z AiProgrammer.Solving.Utils.UnexpectedAiOutputException: Invalid format for replace file command.. First 50 characters of output: 'replace_file ```
2023-03-26T04:50:51.9364087Z import styled from 'styled-compon'
2023-03-26T04:50:51.9366991Z    at AiProgrammer.Solving.Steps.ConcreteSteps.StepModifyExistingFilesOneByOne.ExecuteReplaceFileCommand(FilePath currentFile, List`1 alreadyReviewedFilesTexts, String stepResult) in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/ConcreteSteps/StepModifyExistingFilesOneByOne.cs:line 128
2023-03-26T04:50:51.9369133Z    at AiProgrammer.Solving.Steps.ConcreteSteps.StepModifyExistingFilesOneByOne.ExecuteForSingleFile(FilePath currentFile, List`1 alreadyReviewedFilesTexts, IReadOnlyCollection`1 filesLeftToReview) in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/ConcreteSteps/StepModifyExistingFilesOneByOne.cs:line 101
2023-03-26T04:50:51.9371595Z    at AiProgrammer.Solving.Steps.ConcreteSteps.StepModifyExistingFilesOneByOne.GetChangedFiles(IReadOnlyList`1 filesToPreview) in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/ConcreteSteps/StepModifyExistingFilesOneByOne.cs:line 33
2023-03-26T04:50:51.9373219Z    at AiProgrammer.Solving.Steps.EntrySteps.StepSolveGithubIssueFileByFile.SolveGithubIssue() in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/EntrySteps/StepSolveGithubIssueFileByFile.cs:line 94
2023-03-26T04:50:51.9374650Z    at AiProgrammer.Solving.GithubIssue.GithubIssueSolverFileByFile.SolveGithubIssue() in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/GithubIssue/GithubIssueSolverFileByFile.cs:line 17
2023-03-26T04:50:51.9376068Z    at GithubIssueListener.IssueSolvingStarter.Solve(GithubIssueContext requestContext) in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/IssueSolvingStarter.cs:line 57
2023-03-26T04:50:51.9377174Z    at GithubIssueListener.IssueSolvingStarter.Solve(GithubIssueContext requestContext) in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/IssueSolvingStarter.cs:line 64
2023-03-26T04:50:51.9378447Z    at GithubIssueListener.IssueSolvingStarter.SendIssueToSolveInBrain() in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/IssueSolvingStarter.cs:line 46
2023-03-26T04:50:51.9379465Z    at GithubIssueListener.ListenerEntryPoint.Main() in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/ListenerEntryPoint.cs:line 22
2023-03-26T04:50:51.9847389Z Unhandled exception. AiProgrammer.Solving.Utils.UnexpectedAiOutputException: Invalid format for replace file command.. First 50 characters of output: 'replace_file ```
2023-03-26T04:50:51.9848455Z import styled from 'styled-compon'
2023-03-26T04:50:51.9849848Z    at AiProgrammer.Solving.Steps.ConcreteSteps.StepModifyExistingFilesOneByOne.ExecuteReplaceFileCommand(FilePath currentFile, List`1 alreadyReviewedFilesTexts, String stepResult) in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/ConcreteSteps/StepModifyExistingFilesOneByOne.cs:line 128
2023-03-26T04:50:51.9852067Z    at AiProgrammer.Solving.Steps.ConcreteSteps.StepModifyExistingFilesOneByOne.ExecuteForSingleFile(FilePath currentFile, List`1 alreadyReviewedFilesTexts, IReadOnlyCollection`1 filesLeftToReview) in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/ConcreteSteps/StepModifyExistingFilesOneByOne.cs:line 101
2023-03-26T04:50:51.9854456Z    at AiProgrammer.Solving.Steps.ConcreteSteps.StepModifyExistingFilesOneByOne.GetChangedFiles(IReadOnlyList`1 filesToPreview) in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/ConcreteSteps/StepModifyExistingFilesOneByOne.cs:line 33
2023-03-26T04:50:51.9856733Z    at AiProgrammer.Solving.Steps.EntrySteps.StepSolveGithubIssueFileByFile.SolveGithubIssue() in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/Steps/EntrySteps/StepSolveGithubIssueFileByFile.cs:line 94
2023-03-26T04:50:51.9858485Z    at AiProgrammer.Solving.GithubIssue.GithubIssueSolverFileByFile.SolveGithubIssue() in /home/runner/work/good-thinking/good-thinking/action-repo/AiProgrammer/Solving/GithubIssue/GithubIssueSolverFileByFile.cs:line 17
2023-03-26T04:50:51.9859726Z    at GithubIssueListener.IssueSolvingStarter.Solve(GithubIssueContext requestContext) in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/IssueSolvingStarter.cs:line 57
2023-03-26T04:50:51.9860870Z    at GithubIssueListener.IssueSolvingStarter.Solve(GithubIssueContext requestContext) in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/IssueSolvingStarter.cs:line 64
2023-03-26T04:50:51.9862910Z    at GithubIssueListener.IssueSolvingStarter.SendIssueToSolveInBrain() in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/IssueSolvingStarter.cs:line 46
2023-03-26T04:50:51.9864706Z    at GithubIssueListener.ListenerEntryPoint.Main() in /home/runner/work/good-thinking/good-thinking/action-repo/GithubIssueListener/ListenerEntryPoint.cs:line 22
2023-03-26T04:50:51.9865330Z    at GithubIssueListener.ListenerEntryPoint.<Main>()
2023-03-26T04:50:52.2000171Z ##[error]Process completed with exit code 134.

[pistachiomatt]

Also breaks when response is:

replace_file
```tsx
'use client'
import { FC } fro...

I've found GPT produces more reliable results when you ask it to return JSON.

[dex3r]

Yeah, I was thinking about moving to JSON as well. However, JSON has a really poor multi line string support. I will try with YAML to see if GPT4 can handle that.

[pistachiomatt]

GPT4 will return multi-line data properly escaped in one giant double-quoted string