Olive Recipes For AI Model Optimization Toolkit

This repository compliments Olive, the AI model optimization toolkit, and includes recipes demonstrating its extensive features and use cases. Users of Olive can use these recipes as a reference to either optimize publicly available AI models or to optimize their own proprietary models.

Supported models, architectures, devices and execution providers

Below are list of available recipes grouped by different criteria. Click the link to expand.

Models grouped by model architecture

bert	clip	deepseek	llama	mistral	phi	phi3	qwen2	resnet	vit
google-bert-bert-base-multilingual-cased	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	meta-llama-Llama-3.2-1B-Instruct	mistralai-Mistral-7B-Instruct-v0.3	microsoft-Phi-3.5-mini-instruct	microsoft-Phi-3.5-mini-instruct	Qwen-Qwen2.5-1.5B-Instruct	microsoft-resnet-50	google-vit-base-patch16-224
intel-bert-base-uncased-mrpc	openai-clip-vit-base-patch16		meta-llama-Llama-3.2-1B-Instruct		microsoft-Phi-4-mini-reasoning		Qwen-Qwen2.5-1.5B-Instruct
	openai-clip-vit-base-patch32						deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B

Models grouped by device

cpu	gpu	npu
Qwen-Qwen2.5-1.5B-Instruct	DeepSeek_R1_1.5B_FP16_Model_Builder	Qwen-Qwen2.5-1.5B-Instruct
deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	Llama3.2_1B_Instruct_NVMO_INT4_AWQ	Qwen-Qwen2.5-1.5B-Instruct
google-bert-bert-base-multilingual-cased	Phi3.5_Mini_Instruct_NVMO_INT4_AWQ	Qwen-Qwen2.5-1.5B-Instruct
google-vit-base-patch16-224	Qwen-Qwen2.5-1.5B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
intel-bert-base-uncased-mrpc (ov)	Qwen-Qwen2.5-1.5B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
laion-CLIP-ViT-B-32-laion2B-s34B-b79K	Qwen2.5_1.5B_Instruct_NVMO_INT4_AWQ	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
meta-llama-Llama-3.2-1B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	google-bert-bert-base-multilingual-cased
microsoft-Phi-3.5-mini-instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	google-bert-bert-base-multilingual-cased
microsoft-resnet-50	google-bert-bert-base-multilingual-cased	google-bert-bert-base-multilingual-cased
openai-clip-vit-base-patch16	google-bert-bert-base-multilingual-cased	google-vit-base-patch16-224
openai-clip-vit-base-patch32	google-bert-bert-base-multilingual-cased	google-vit-base-patch16-224
	google-vit-base-patch16-224	google-vit-base-patch16-224
	google-vit-base-patch16-224	intel-bert-base-uncased-mrpc (AMD)
	google-vit-base-patch16-224	intel-bert-base-uncased-mrpc (ov)
	intel-bert-base-uncased-mrpc (ov)	intel-bert-base-uncased-mrpc
	intel-bert-base-uncased-mrpc	laion-CLIP-ViT-B-32-laion2B-s34B-b79K (Text)
	intel-bert-base-uncased-mrpc	laion-CLIP-ViT-B-32-laion2B-s34B-b79K (Vision)
	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	laion-CLIP-ViT-B-32-laion2B-s34B-b79K
	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	laion-CLIP-ViT-B-32-laion2B-s34B-b79K
	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	meta-llama-Llama-3.2-1B-Instruct
	meta-llama-Llama-3.2-1B-Instruct	meta-llama-Llama-3.2-1B-Instruct
	meta-llama-Llama-3.2-1B-Instruct	meta-llama-Llama-3.2-1B-Instruct
	microsoft-Phi-3.5-mini-instruct	microsoft-Phi-3.5-mini-instruct
	microsoft-Phi-3.5-mini-instruct	microsoft-Phi-3.5-mini-instruct
	microsoft-resnet-50	microsoft-Phi-3.5-mini-instruct
	microsoft-resnet-50	microsoft-Phi-4-mini-reasoning
	microsoft-resnet-50	microsoft-resnet-50
	mistralai-Mistral-7B-Instruct-v0.3	microsoft-resnet-50
	openai-clip-vit-base-patch16	microsoft-resnet-50
	openai-clip-vit-base-patch16	openai-clip-vit-base-patch16 (Text)
	openai-clip-vit-base-patch16	openai-clip-vit-base-patch16 (Vision)
	openai-clip-vit-base-patch32	openai-clip-vit-base-patch16
	openai-clip-vit-base-patch32	openai-clip-vit-base-patch16
	openai-clip-vit-base-patch32	openai-clip-vit-base-patch32 (Text)
		openai-clip-vit-base-patch32 (Vision)
		openai-clip-vit-base-patch32
		openai-clip-vit-base-patch32

Models grouped by EP

Dml	NvTensorRTRTX	OpenVINO	QNN	VitisAI
Qwen-Qwen2.5-1.5B-Instruct	DeepSeek_R1_1.5B_FP16_Model_Builder	Qwen-Qwen2.5-1.5B-Instruct	Qwen-Qwen2.5-1.5B-Instruct	Qwen-Qwen2.5-1.5B-Instruct
deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	Llama3.2_1B_Instruct_NVMO_INT4_AWQ	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
google-bert-bert-base-multilingual-cased	Phi3.5_Mini_Instruct_NVMO_INT4_AWQ	google-bert-bert-base-multilingual-cased	google-bert-bert-base-multilingual-cased	google-bert-bert-base-multilingual-cased
google-vit-base-patch16-224	Qwen2.5_1.5B_Instruct_NVMO_INT4_AWQ	google-vit-base-patch16-224	google-vit-base-patch16-224	google-vit-base-patch16-224
intel-bert-base-uncased-mrpc	google-bert-bert-base-multilingual-cased	intel-bert-base-uncased-mrpc (ov)	intel-bert-base-uncased-mrpc	intel-bert-base-uncased-mrpc (AMD)
laion-CLIP-ViT-B-32-laion2B-s34B-b79K	google-vit-base-patch16-224	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	laion-CLIP-ViT-B-32-laion2B-s34B-b79K (Text)	laion-CLIP-ViT-B-32-laion2B-s34B-b79K
meta-llama-Llama-3.2-1B-Instruct	intel-bert-base-uncased-mrpc	meta-llama-Llama-3.2-1B-Instruct	laion-CLIP-ViT-B-32-laion2B-s34B-b79K (Vision)	meta-llama-Llama-3.2-1B-Instruct
microsoft-Phi-3.5-mini-instruct	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	microsoft-Phi-3.5-mini-instruct	meta-llama-Llama-3.2-1B-Instruct	microsoft-Phi-3.5-mini-instruct
microsoft-resnet-50	microsoft-resnet-50	microsoft-Phi-4-mini-reasoning	microsoft-Phi-3.5-mini-instruct	microsoft-resnet-50
openai-clip-vit-base-patch16	openai-clip-vit-base-patch16	microsoft-resnet-50	microsoft-resnet-50	openai-clip-vit-base-patch16
openai-clip-vit-base-patch32	openai-clip-vit-base-patch32	mistralai-Mistral-7B-Instruct-v0.3	openai-clip-vit-base-patch16 (Text)	openai-clip-vit-base-patch32
		openai-clip-vit-base-patch16	openai-clip-vit-base-patch16 (Vision)
		openai-clip-vit-base-patch32	openai-clip-vit-base-patch32 (Text)
			openai-clip-vit-base-patch32 (Vision)

Learn more

• Olive Github Repository
• Getting Started
• Olive Documentation

🤝 Contributions and Feedback

• We welcome contributions! Please read the contribution guidelines for more details on how to contribute to the Olive project.
• For feature requests or bug reports, file a GitHub Issue.
• For general discussion or questions, use GitHub Discussions.

⚖️ License

Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License.