Capture output of eval plugin and some girls scout changes

[dschrempf]

This PR contains some girl-scout changes as well as changes to how the eval plugin treats stdout and stderr.

Eval plugin

We now capture output to stdout and stderr induced as a possible side effect by the statement being evaluated. This is fragile because the output may be scrambled in a concurrent setting when HLS is writing to one of these file handles from a different thread.

Fixes #1977, most likely also #4390 (although this is debatable, we could do something more sophisticated).

-- >>> putStrLn "Hello"
-- Hello

-- >>> hPutStrLn stderr "Hello"
-- Hello

-- >>> putStrLn "Hello," >> pure "World!"
-- Hello,
-- "World!"

Girl scout changes

• Docs: Fix some links in eval plugin in-source documentation
• Clarify documentation of expected test failure
• Update Nix Flake (Darwin issues seem to be solved)
• Update some tests and avoid mentioning GHC 9.2
• Remove comment referring to GHC 9.2 and fix code

[fendor]

Needs a proper test in hls-eval-plugin, but otherwise looks good to me, thank you for the clean up! Feel free to merge once you've added a test :)

[fendor]

Slightly weird, probably needs to be fixed one way or another?

[dschrempf]

I didn't realize the eval plugin changed some other results. Thanks for picking that up. I will investigate.

[dschrempf]

I fixed those WAS NOW errors, one question though:

Does the plugin also evaluate all actions in the same comment block in your case?

If I evaluate one action in this comment block, all of them get evaluated and a lot of WAS NOW messages get added. Some of them are wrong. I am not sure if this PR added this discrepancy, or if the problem was already there before. I will check.

[fendor]

After locally running, I can confirm I also get these errors, independent of your changes.
This is how this file looks like after running eval:

diff --git a/plugins/hls-eval-plugin/src/Ide/Plugin/Eval/Handlers.hs b/plugins/hls-eval-plugin/src/Ide/Plugin/Eval/Handlers.hs
index 1f19b5b47..0ae407bcb 100644
--- a/plugins/hls-eval-plugin/src/Ide/Plugin/Eval/Handlers.hs
+++ b/plugins/hls-eval-plugin/src/Ide/Plugin/Eval/Handlers.hs
@@ -426,8 +426,14 @@ A, possibly multi line, error is returned for a wrong declaration, directive or
 Some flags have not been recognized: -XNonExistent
 
 >>> cls C
-Variable not in scope: cls :: t0 -> t
-Data constructor not in scope: C
+WAS Variable not in scope: cls :: t0 -> t
+WAS Data constructor not in scope: C
+NOW Illegal term-level use of the class `C'
+NOW   defined at <interactive>:1:2
+NOW In the first argument of `cls', namely `C'
+NOW In the expression: cls C
+NOW In an equation for `it_a2KTO': it_a2KTO = cls C
+NOW Variable not in scope: cls :: t0_a2KVe[tau:1] -> t1_a2KVg[tau:1]
 
 >>> "A
 lexical error in string/character literal at end of input
@@ -437,16 +443,20 @@ in GHCi or doctest. This allows it to be used as a hack to simulate print until
 get proper IO support. See #1977
 
 >>> 3 `div` 0
-divide by zero
+WAS divide by zero
+NOW *** Exception: divide by zero
 
 >>> error "Something went wrong\nbad times" :: E.SomeException
-Something went wrong
+WAS Something went wrong
+NOW *** Exception: Something went wrong
 bad times
 
 Or for a value that does not have a Show instance and can therefore not be displayed:
 >>> data V = V
 >>> V
-No instance for (Show V) arising from a use of ‘evalPrint’
+WAS No instance for (Show V) arising from a use of ‘evalPrint’
+NOW No instance for `Show V' arising from a use of `evalPrint'
+NOW In a stmt of an interactive GHCi command: evalPrint it_a2LvA
 -}
 evals :: Recorder (WithPriority Log) -> Bool -> TEnv -> DynFlags -> [Statement] -> Ghc [Text]
 evals recorder mark_exception fp df stmts = do

[fendor]

I think, we probably need to ignore these.

[dschrempf]

Alright! Thanks for confirming. I will make some room this week ;-).

Did we conclude on whether we should limit the maximum output? I think we decided not to have a limit, did we?

[theGhostJW]

who are you asking?

[dschrempf]

@fendor

But also everybody else if they have an opinion or a fact to add, please go ahead!

[theGhostJW]

This could be an interesting surprise for some users. I often eval expressions that produce > 1000 lines of IO. In my case detailed pretty printed logs. If this was to suddenly be rendered in the code editor I'd need to change my workflow. If i wasn't expecting it, it would be quite confusing.

[dschrempf]

Yes, if this is your workflow. In theory, we could limit the amount of output. The underlying library redirects output to a temporary file and then reads this temporary file. We could amend the function so that it only takes the first N lines of this file (or bytes, or whatever).

[dschrempf]

Okay, finally some updates on this one.

• I had to revamp the method how we capture stdout and stderr. We now inject a setup and teardown function into the GHCi call. These calls setup the stdout and stderr capture, and also make sure the handles are closed afterwards. This is further ensured using bracket functions (new dependency on exceptions).
• I have changed the existing IO test to exercise the capture of stdout (instead of testing that it is not captured).
• I have added another test that exercises capture of stderr.

I think we can merge this now, but waiting for your OK @fendor @xsebek.

[dschrempf]

(I still have to refresh the in-source eval examples).

[xsebek]

I tested it on the example from #4410 and it now works with just putStrLn, great! 🚀

[xsebek]

Could you please check if it is also possible to redirect standard input? The docs mention it is currently broken:

Standard input is shared with HLS, so e.g. getLine breaks the connection to server.

But I guess fewer people run into that issue, while (pretty)print is very common. 🙂

[dschrempf]

I retract my statement about LGTM; there are still minor issues to fix (?).

Also, some lenses get out of date (not sure if this is a problem of my editor setup, or a more general problem with HLS).

[dschrempf]

OK, I gave it another go.

I fixed one more bug, improved large parts of the code (it is piece of art, but still a mess), and updated the Readme.

I think it is good to merge now. @xsebek, I didn't implement your feature request (I ran out of time); I think it can easily go in a separate PR.

Let's discuss tomorrow.