Skip to content

RocksIOError during bulk_search #62

Description

@tolot27

I have configured my rocksdb.yaml with nproc: 4' as shown in the example config at step 2 of https//bigsi.readme.io/docs/your-first-bigsi. If I do a bulk_searchI run intoRocksIOError`. The stack trace is:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/usr/local/lib/python3.6/dist-packages/bigsi/__main__.py", line 76, in search_bigsi_parallel
    bigsi = BIGSI(l[0][0])
  File "/usr/local/lib/python3.6/dist-packages/bigsi/graph/bigsi.py", line 134, in __init__
    self.storage = get_storage(config)
  File "/usr/local/lib/python3.6/dist-packages/bigsi/storage/__init__.py", line 19, in get_storage
    return STORAGE_DICT[config["storage-engine"]](config["storage-config"])
  File "/usr/local/lib/python3.6/dist-packages/bigsi/storage/rocksdb.py", line 49, in __init__
    read_only=self.storage_config.get("read_only", False),
  File "rocksdb/_rocksdb.pyx", line 1636, in rocksdb._rocksdb.DB.__cinit__
  File "rocksdb/_rocksdb.pyx", line 85, in rocksdb._rocksdb.check_status
rocksdb.errors.RocksIOError: b'IO error: While lock file: bigsi.rocks/LOCK: Resource temporarily unavailable'
"""

If I lower nproc to one, the error is gone, obviously. It looks like search_bigsi_parallel is the entry point to fix this error.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions