Hello,
By following the repo instructions, right after running the example code from the docs:
docker run -it rabbitllm python3 scripts/inference_example.py --model Qwen/Qwen2.5-7B-Instruct
I get this error:
INFO: Loading shard 1/4 | 0/31 [00:00<?, ?it/s]
0%| | 0/31 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/app/scripts/inference_example.py", line 94, in
model = AutoModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/models/registry.py", line 109, in from_pretrained
return class_(pretrained_model_name_or_path, *inputs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/models/qwen2.py", line 27, in init
super(RabbitLLMQWen2, self).init(*args, **kwargs)
File "/app/src/rabbitllm/engine/base.py", line 246, in init
self.model_local_path, self.checkpoint_path = find_or_create_local_splitted_path(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/utils/hub.py", line 85, in find_or_create_local_splitted_path
return Path(hf_cache_path), split_and_save_layers(
^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/utils/splitting.py", line 255, in split_and_save_layers
huggingface_hub.snapshot_download(
^^^^^^^^^^^^^^^
NameError: name 'huggingface_hub' is not defined
It appears to be that rabbitllm library relies on the huggingface_hub library to download models, but soemone forgot to add the line import huggingface_hub at the top of one of their internal files, but trying to use it in the code, may you please check?
Hello,
By following the repo instructions, right after running the example code from the docs:
docker run -it rabbitllm python3 scripts/inference_example.py --model Qwen/Qwen2.5-7B-Instruct
I get this error:
INFO: Loading shard 1/4 | 0/31 [00:00<?, ?it/s]
0%| | 0/31 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/app/scripts/inference_example.py", line 94, in
model = AutoModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/models/registry.py", line 109, in from_pretrained
return class_(pretrained_model_name_or_path, *inputs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/models/qwen2.py", line 27, in init
super(RabbitLLMQWen2, self).init(*args, **kwargs)
File "/app/src/rabbitllm/engine/base.py", line 246, in init
self.model_local_path, self.checkpoint_path = find_or_create_local_splitted_path(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/utils/hub.py", line 85, in find_or_create_local_splitted_path
return Path(hf_cache_path), split_and_save_layers(
^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/rabbitllm/utils/splitting.py", line 255, in split_and_save_layers
huggingface_hub.snapshot_download(
^^^^^^^^^^^^^^^
NameError: name 'huggingface_hub' is not defined
It appears to be that rabbitllm library relies on the huggingface_hub library to download models, but soemone forgot to add the line import huggingface_hub at the top of one of their internal files, but trying to use it in the code, may you please check?