Skip to content

Latest commit

 

History

61 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MindHub简介

MindHub是一个MindSpore生态的模型应用工具,致力于为用户提供方便快捷的模型加载和推理功能。MindHub 的主要特点如下:

  • 快速加载:用户使用MindHub可以快速加载MindSpore的预训练模型,而无需自己下载和处理模型参数。只需几行代码,即可轻松加载模型并进行推理。
  • 简单易用MindHub 提供了简单易用的 Python API,使用户可以在自己的 Python 代码中轻松使用预训练模型进行推理。API 的使用方法简单明了, 具有较好的可读性和易用性。

安装

依赖

  • tqdm
  • Pillow
  • numpy>=1.17.0
  • mindspore>=2.0.0
pip install -r requirements.txt

用户可遵从官方指导 并根据自身使用的硬件平台选择最适合您的MindSpore版本来进行安装。如果需要在分布式条件下使用,还需安装openmpi

之后的说明将默认用户已正确安装好相关依赖。

源码安装

Github上最新的MindCV可以通过以下指令安装。

pip install pip install https://github.com/xiuyu0000/mindhub/releases/download/v0.0.1/mindhub-0.0.1-py3-none-any.whl

快速入门

在开始上手MindHub前,可以阅读MindHub的自定义模型教程,该教程可以帮助用户快速了解MindHub的各个重要组件以及 模型各项功能的使用。

以下是一些供您快速体验的代码样例。

import mindhub as hub

net = hub.Model("tinydarknet_imagenet", pretrained=True)
print(net.infer(data_path="./data/infer_example.JPEG", json_path="./label_map.json"))
Matching local models: []
Matching remote models: ['tinydarknet_imagenet']
tinydarknet_imagenet is not installed!
4096B [00:00, 2369637.13B/s]           
20480B [00:00, 6247225.16B/s]           
4096B [00:00, 1779006.85B/s]           
5120B [00:00, 2905538.69B/s]           
4096B [00:00, 2582274.04B/s]           
Downloading data from https://download.mindspore.cn/models/r1.9/tinydarknet_ascend_v190_imagenet2012_official_cv_top1acc59.0_top5acc81.84.ckpt (10.4 MB)

file_sizes: 100%|██████████████████████████| 10.9M/10.9M [00:00<00:00, 23.1MB/s]
Successfully downloaded file to ./tinydarknet_ascend_v190_imagenet2012_official_cv_top1acc59.0_top5acc81.84.ckpt
Data Path: ./data/infer_example.JPEG
[{'275': 'African hunting dog'}]

infer_example

功能介绍

MindHub目前版本包括模型搜索,模型安装/移除和模型加载等功能,具体使用方法如下所示。

模型搜索

MindHub提供了list_models接口,通过输出的可能的模型名称模糊匹配本地和远程仓库中的模型名称,返回可能的模型名称的列表,供用户做进一步判断。

import mindhub as hub
# 列出满足条件的预训练模型名称
hub.list_models("tinydarknet")
Matching local models: []
Matching remote models: ['tinydarknet_imagenet']
([], ['tinydarknet_imagenet'])

示例中搜索的tinydarknet在本地没有匹配,在远程仓库匹配到了tinydarknet_imagenet

模型安装

MindHub提供了install_model接口,来安装所需模型。

hub.Model.install_model("tinydarknet_imagenet")
Matching local models: []
Matching remote models: ['tinydarknet_imagenet']
tinydarknet_imagenet is not installed!
4096B [00:00, 2120447.94B/s]           
20480B [00:00, 7109107.50B/s]           
4096B [00:00, 1644006.62B/s]           
5120B [00:00, 1940438.83B/s]           
4096B [00:00, 2142128.33B/s]           
'~/.mindhub/tinydarknet_imagenet'

同时MindHub还提供了local_models_info接口,可以进一步查看本地模型的详细信息。

print(hub.local_models_info("tinydarknet_imagenet"))
{'model_name': 'tinydarknet_imagenet', 'pretrained': True, 'paper': '', 'model_type': 'image/classification'}

示例中由于tinydarknet在模型搜索中发现,所需模型未存在于本地模型注册表中,但存在于远程仓库。所以可以通过install_model来进行安装。安装后, tinydarknet_imagenet的详细信息注册在了本地模型信息表中,可以通过local_models_info接口来进行查看。

模型移除

对于已经安装的不需要的模型,MindHub提供了remove_model接口,在本地模型注册表中删除该模型,并删除对应的模型文件。

hub.Model.remove_model("tinydarknet_imagenet")
~/.mindhub/tinydarknet_imagenet has been successfully deleted.

示例中我们删除了刚刚安装的tinydarknet_imagenet模型。

模型加载

MindHub提供了Model类,可以通过加载本地模型文件或者模型名称来加载所需的模型。

  1. 本地模型文件加载

通过本地模型文件加载,需要同时输入所需的模型名称和模型文件所在文件夹路径(关于模型文件的详细信息, 请参阅自定义模型教程)以及是否需要下载预训练模型。

net = hub.Model(model_name="tinydarknet_imagenet", 
                diretory="{YOUR_PATH}/tinydarknet_imagenet/", pretrained=True)
Downloading data from https://download.mindspore.cn/models/r1.9/tinydarknet_ascend_v190_imagenet2012_official_cv_top1acc59.0_top5acc81.84.ckpt (10.4 MB)

file_sizes: 100%|██████████████████████████| 10.9M/10.9M [00:00<00:00, 22.3MB/s]
Successfully downloaded file to ./tinydarknet_ascend_v190_imagenet2012_official_cv_top1acc59.0_top5acc81.84.ckpt

示例中通过加载./tinydarknet_imagenet/文件夹中的模型文件加载了tinydarknet_imagenet模型,并加载了预训练模型。

  1. 模型名称加载

通过输入模型名称加载,只需要输入模型名称以及是否需要下载预训练模型。

net = hub.models.Model(model_name="tinydarknet_imagenet", pretrained=True)
Matching local models: []
Matching remote models: ['tinydarknet_imagenet']
tinydarknet_imagenet is not installed!
4096B [00:00, 2369637.13B/s]           
20480B [00:00, 6247225.16B/s]           
4096B [00:00, 1779006.85B/s]           
5120B [00:00, 2905538.69B/s]           
4096B [00:00, 2582274.04B/s]           
Downloading data from https://download.mindspore.cn/models/r1.9/tinydarknet_ascend_v190_imagenet2012_official_cv_top1acc59.0_top5acc81.84.ckpt (10.4 MB)

file_sizes: 100%|██████████████████████████| 10.9M/10.9M [00:00<00:00, 23.1MB/s]
Successfully downloaded file to ./tinydarknet_ascend_v190_imagenet2012_official_cv_top1acc59.0_top5acc81.84.ckpt

示例中通过模型搜索发现本地模型注册表中不存在该模型,但是远程仓库中存在,因此进行模型安装,之后进行模型加载以及预训练权重的下载和加载。

注:如果模型名称已经注册在本地模型注册表中,不需要其他操作可以直接加载;如果模型名称未在 本地模型注册表中,则需要进一步确认是否存在于远程仓库, 如果存在则需要先进行,如果不存在则直接报错。

模型推理

推理是MindHub中的模型所要求的基本功能,根据贡献者定义的方法来进行推理。

  • 推理单张图片。
print(net.infer(data_path="./data/infer_example.JPEG", json_path="./label_map.json"))
Data Path: ./data/infer_example.JPEG
[{'275': 'African hunting dog'}]
  • 推理文件夹下所有图片。
import os

dataset_url = "https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/intermediate/Canidae_data.zip"
root_dir = "./"

if not os.path.exists(os.path.join(root_dir, 'data/Canidae')):
    hub.DownLoad().download_and_extract_archive(dataset_url, root_dir)

print(net.infer(data_path="./data/Canidae/val/dogs/", json_path="./label_map.json"))
Data Path: ./data/Canidae/val/dogs/
[{'158': 'toy terrier'}, {'151': 'Chihuahua'}, {'151': 'Chihuahua'}, {'161': 'basset'}, {'151': 'Chihuahua'}, 
{'263': 'Pembroke'}, {'263': 'Pembroke'}, {'151': 'Chihuahua'}, {'151': 'Chihuahua'}, {'151': 'Chihuahua'}, 
{'171': 'Italian greyhound'}, {'151': 'Chihuahua'}, {'151': 'Chihuahua'}, {'151': 'Chihuahua'}, {'6': 'stingray'}, 
{'253': 'basenji'}, {'151': 'Chihuahua'}, {'227': 'kelpie'}, {'151': 'Chihuahua'}, {'151': 'Chihuahua'}, 
{'151': 'Chihuahua'}, {'151': 'Chihuahua'}, {'158': 'toy terrier'}, {'151': 'Chihuahua'}, {'669': 'mosquito net'}, 
{'151': 'Chihuahua'}, {'173': 'Ibizan hound'}, {'156': 'Blenheim spaniel'}, {'237': 'miniature pinscher'}, 
{'416': 'balance beam'}]

示例中的模型要进行推理只需要输入图片所在的路径和标签对应表所在的路径,就可以正常输出推理结果。

教程

我们提供了相关教程,帮助用户学习如何使用和贡献MindHub

模型列表

目前,MindHub支持以下模型。

模型 论文 支持
Vision - Segmentation
U-Net U-Net: Convolutional Networks for Biomedical Image Segmentation
DeepLabV3+ Encoder-decoder with atrous separable convolution for semantic image segmentation.
FCN8s Fully convolutional networks for semantic segmentation.
Mask R-CNN Mask R-CNN
PointNet PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Vision - Classification
tinydarknet Tiny Darknet
resnet Deep Residual Learning for Image Recognition
mobilenetv2 MobileNetV2: Inverted Residuals and Linear Bottlenecks
efficientnet EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
cspnet CSPNet: A New Backbone that can Enhance Learning Capability of CNN
vit An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Vision - Face Verification
facenet FaceNet: A Unified Embedding for Face Recognition and Clustering
arcface ArcFace: Additive Angular Margin Loss for Deep Face Recognition
retinaface RetinaFace: Single-stage Dense Face Localisation in the Wild
Vision - Detection
yolov3 YOLOv3: An Incremental Improvement
yolov4 YOLOv4: Optimal Speed and Accuracy of Object Detection
yolov7 YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors
detr End-to-End Object Detection with Transformers
ssd SSD: Single Shot MultiBox Detector
fast r-cnn Fast R-CNN
Vision - OCR
DBNet DBNET: DOA-driven beamforming network for end-to-end farfield sound source separation
CRNN CRNN: A Joint Neural Network for Redundancy Detection
Vision - Video
fairmot FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking
swin3d Video Swin Transformer
Vision - 3D
second Video Swin Transformer
PointNet++ PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
Vision - 3D Restruction
DeepLM DeepLM: Large-Scale Nonlinear Least Squares on Deep Learning Frameworks Using Stochastic Domain Decomposition
DecoMR 3D Human Mesh Regression with Dense Correspondence
NLP - Machine Translation
seq2seq Sequence to Sequence Learning with Neural Networks
speech_transformer The Speechtransformer for Large-scale Mandarin Chinese Speech Recognition
NLP - Talking
dgu DGU: Dialogue General Understanding
dam Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network
NLP - Language Modeling
gpt-3 Language Models are Few-Shot Learners
pangu PanGu-α : Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
bert BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Recommand - CTR
Wide & Deep Wide & Deep Learning for Recommender Systems
DeepFM DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
EDCN Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models
Audio - Speech Recognition
conformer Conformer: Convolution-augmented Transformer for Speech Recognition
deepspeechv2 Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Generation - Image Generation
dcgan Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
stable diffusion v2 High-Resolution Image Synthesis with Latent Diffusion Models
wukong-huahua Wukong-Huahua

贡献

欢迎开发者用户提issue或提交代码PR,或贡献更多的算法和模型,一起让MindHub变得更好。

有关贡献指南,请参阅CONTRIBUTING。请遵循自定义模型模型教程所规定的规则来贡献模型接口:)

许可证

Apache License 2.0

引用

如果你觉得MindHub对你的项目有帮助,请考虑引用:

@misc{MindSpore-Lab Hub 2023,
    title={{MindSpore-Lab Hub}:MindSpore_Lab Models Hub},
    author={MindSpore-Lab Hub Contributors},
    howpublished = {\url{https://github.com/mindspore-lab/mindhub/}},
    year={2023}
}

About

MindHub是一个MindSpore生态的模型应用工具,致力于为用户提供方便快捷的模型加载和推理功能。

Resources

Contributing

Stars

2 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages