[Feature request] Move model management out of python into `.cpp` + `hf` cli

[06kellyjac]

It's a bit annoying to also have to bring along python + the dependencies for the fetcher script in order to use the UI buttons.

Adopting a similar approach to llama.cpp leveraging the hf cli would be great.
https://github.com/ggml-org/llama.cpp/blob/84e908c625fb60992b4cdef8180fb12fa9b4c4bf/common/hf-cache.cpp#L198-L227

Many thanks

[0xShug0]

Thanks! Removing the Python dependencies makes sense.

@mirek190 Would you like to take a stab at this? Also would it make sense to remove the legacy Python UI in this release to avoid confusion, in case existing legacy UI users keep using the old UI out of habit?

[mirek190]

Thanks! Removing the Python dependencies makes sense.

@mirek190 Would you like to take a stab at this? Also would it make sense to remove the legacy Python UI in this release to avoid confusion, in case existing legacy UI users keep using the old UI out of habit?

sure ;)

[nomonkeynodeal]

Wait, the buttons are known to be not working? I've been trying to get this downloading for hours.

[0xShug0]

@nomonkeynodeal Are you using the Python UI?

[nomonkeynodeal]

@nomonkeynodeal Are you using the Python UI?

No idea. I'm using the pre-made docker. It appears to be missing half the project. I was about to try building it myself, but I don't know if that will have the same result.

[0xShug0]

okay I will look into the docker CI.

[nomonkeynodeal]

okay I will look into the docker CI.

I got it working. I had to clone the repo and edit the CUDA dockerfile to include the /tools directory and install python/pip. Figuring this out took me 4+ hours of trial and error. If I were to make a suggestion, it would be that there are zero users who just want the UI without the ability to download.

[06kellyjac]

UI is still on the beta branch so things are bound to be missing/broken if devs mostly use it locally & haven't tested the UI fetching models via docker image yet 🙂
Just the nature of dev. Even actually testing the UI using the docker image it's quite understandable to mount your already downloaded models rather than waiting to download them again 😅

---

Thanks for considering this change @0xShug0 @mirek190 🙂