Skip to content

fsspec==2023.12.2 does not allow '**' in path #32

Description

@jacobvsdanniel
  File "/lib/python3.8/site-packages/retriv/dense_retriever/dense_retriever.py", line 175, in index
    self.index_aux(
  File "/lib/python3.8/site-packages/retriv/dense_retriever/dense_retriever.py", line 137, in index_aux
    self.ann_searcher.build()
  File "/lib/python3.8/site-packages/retriv/dense_retriever/ann_searcher.py", line 27, in build
    index, index_infos = build_index(
  File "/lib/python3.8/site-packages/autofaiss/external/quantize.py", line 205, in build_index
    embedding_reader = EmbeddingReader(
  File "/lib/python3.8/site-packages/embedding_reader/embedding_reader.py", line 20, in __init__
    self.reader = NumpyReader(embeddings_folder)
  File "/lib/python3.8/site-packages/embedding_reader/numpy_reader.py", line 67, in __init__
    self.fs, embeddings_file_paths = get_file_list(embeddings_folder, "npy")
  File "/lib/python3.8/site-packages/embedding_reader/get_file_list.py", line 15, in get_file_list
    return _get_file_list(path, file_format)
  File "/lib/python3.8/site-packages/embedding_reader/get_file_list.py", line 46, in _get_file_list
    file_paths = fs.glob(glob_pattern)
  File "/lib/python3.8/site-packages/fsspec/spec.py", line 606, in glob
    pattern = glob_translate(path + ("/" if ends_with_sep else ""))
  File "/lib/python3.8/site-packages/fsspec/utils.py", line 734, in glob_translate
    raise ValueError(
ValueError: Invalid pattern: '**' can only be an entire path component

this error does not occur with fsspec==2023.5.0

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions