enhancements for livecoding

[claudeha]

Is your feature request related to a problem? Please describe.

As TOPLAP says:

Live coding is where people write code while it runs to make live music/visuals/dance/art/etc.

Currently doing this in FragM isn't so easy due to a small number of small issues.

Describe the solution you'd like

1. Double-buffering when rebuilding shaders, so that the last compiled shader can be used to continue rendering until there is a new successfully-built shader to use.
2. Animation not to stop playback when compile fails.
3. Animation time not to reset to 0 when compile fails.

Then there are some relatively simple additions that would further improve things:

4. Allow main display widget to be popped out (so it can be made fullscreen on another desktop/projector, or easily input to OBS or other streaming/recording software that works based on system windows, while still being able to edit code, unlike full screen ESC with all subwindows disabled)
5. A uniform int CompileFailures; and uniform float TimeSinceCompiled; that counts the number of times shader compilation was attempted, and wallclock time, since this shader was compiled (eg for adding visual feedback for viewers if the log window is not shown, eg red mist descending...) Would need to be opt-in via preferences to avoid clashes with any existing frags out there that might already use these names (unlikely, but who knows).

Then some further advanced/niche things while I'm brainstorming just so I don't forget:

6. Display code editor with (semi-)transparent background on top of the display window (might be possible with some Qt magic?) with a keyboard toggle to hide/show it (maybe by adjusting transparency)
7. Support OSC (Open Sound Control) to allow setting uniform values (and maybe control other FragM things) by external software in real time (for example this could allow Supercollider or Pure-data or whatever else to control FragM visuals in sync with their music)
8. Do the shader recompile in another thread with a shared OpenGL context, so animation rendering continues with the previous shader until the new one is ready (with my current shader library, the pause of a few 100ms is very unpleasant); not sure what to do if there are more edits before the background thread is ready

Describe alternatives you've considered

• Writing my own live coding tool(s) from scratch, but that's way too much work to get something as solid as FragM.
• Finding some exisiting tool better suited to live coding of GLSL and port all my shaders to it (still lots of work, awkward porting any fixes between two code bases as I'll still want to use FragM, a total nightmare with my shader library if multiple includes are not supported by the other tools)

Additional context

I was thinking about trying to use FragM live for a gig this Sunday, but the animation stop-and-reset-time-on-error is a show stopper (literally)... so I made other plans for now.

I'll try to fix 1,2,3 myself, 4 might be difficult due to not knowing much Qt, the rest are much lower priority,,,,

[3Dickulus]

Remember... FragM is a GLSL development environment not at all designed for live coding or performance presentation at a live gig.

I've seen some live coding tools but they usually have the code editor superimposed on the shader output, re-compiling with every keystroke.

Personally I don't want to go down this road, FragM shaders are very large and not intended for quick display, rather, it's designed to accumulate pixels in 32bit buffer for photo realistic scenes. And, to render frames for post processing into a mix with live footage.

I strongly feel that a purpose built app would serve this task much better than trying to hack FragM.

[3Dickulus]

Click in the GL area
Press "Esc" (gets full screen)
drag each control piece to the other screen.

result: one monitor has fullscreen GL display with all of the tools on the secondary monitor.

thanks to Qt ;)

[3Dickulus]

...but to do anything with this mode you need a LOT of number crunching power like a Titan V or A100 or...

maybe shadertoy.com would be better, it has audio and other capabilities built in already or...

https://github.com/kazzy2501/glsl-livecoder might be a better starting point to develop what you are after.

[claudeha]

I won't make a pull request for the branch containing the commit with the fix (see above) because it also contains the commit that deletes the splash screen (because it was causing segfaults for me). You may apply the commit 0f3f442 onto develop with git cherry-pick (after adding and fetching a remote to my repo, probably)

[claudeha]

result: one monitor has fullscreen GL display with all of the tools on the secondary monitor.

Ah, nice! Misses the code editor though, and I don't have 2 monitors to try it at the moment.

[3Dickulus]

the textedit widget and the GL widget are tied together

you can enable "Auto load on change by external editor" in Edit->Preferences

I pushed a splash fix before seeing this... will look at 0f3f442 as "double-buffer shader compiles for more-graceful failure" sounds like it might be a good thing

[3Dickulus]

...been thinking about this...

when working in a development environment I do want "graceful fail" I do not want "fallback to last working frag" because I want to see the results of the bad code failing so that I can fix it, if it keeps running I wouldn't know that the code fails unless I'm watching the log window.

it might be advantageous to create a live presentation tool that can use FragM style scripts in a stripped down fashion ie:no fancy gui just the editor widget and the GL widget... or maybe use a midi library to make connections between FragM widgets and a midi control track so you could "play" sliders from a keyboard ???