FastAPI - Mixtral 8x7B - Text Generation Inference

Building an simple Generative AI application to run text inference on Mixtral 8x7B running on Modal's Serverless Platform.

References

• https://huggingface.co/mistralai/Mixtral-8x7B-v0.1
• https://mistral.ai/news/mixtral-of-experts/
• https://huggingface.co/docs/text-generation-inference/index
• https://github.com/huggingface/text-generation-inference
• https://modal.com/
• https://fastapi.tiangolo.com/