Runtime generation

[jjnicholls]

I am thinking in a game in which the objects in the scene can change by specific data give from the gamer, will be useful generate images to show like backgrounds or pictures on the walls. Is possible generate images in runtime? Of course, I understand that involve have access to Stable Diffusion server and get the image using UnityWebRequest

[pedroka-dev]

Hello @jjnicholls ,
I am working on research project that does exactly that.
I do not believe that Stable-Diffusion-Unity-Integration currently supports this.

But I think it is possible to implement such feature. It certainly would take some time and might result in a big refactor or even completely a new tool entirely.

Even if it works, two challenges of such implementation would to handle what happens if the generation fails (timeout? exception?) and how to handle the generation time in-game, since it is not worth it for a player to wait minutes of loading time just to generate a few backgrounds, texture or characters.

I'm not sure if this implementation is on @dobrado76 's priorities tho.
However, I will personally be investigating what is possible about this in the following months.

[jjnicholls]

Hi @Pedro-ca , thank you for your comments, I just finished doing a project using a simple method I found on Youtube, https://www.youtube.com/watch?v=LqVNiKaOWrI, it wasn't exactly a game but rather an activity for an event. I installed a Stable Diffusion locally on a Boxx laptop with a 3070 TI, the waiting time in the generation of the image at 720 x 720 px was around 15 seconds, however, while the image was generated I showed a small video to simulate the image was being processed. My development was not the most careful due to the time I had, but in the end, the activity was a success.