Subdirectory indexing error: WARN [f.p.e.c.f.FsParserAbstract] Error while crawling ... Range [0, -1) out of bounds for length 21

[miroslavpechacek]

Describe the bug

If there is subdirectory in indexed directory, I will get error:

00:12:11,913 WARN  [f.p.e.c.f.FsParserAbstract] Error while crawling c:/Elastic/fscdocs: Range [0, -1) out of bounds for length 21

and no further files are indexed.

continue_on_error: true will not help.

index_folders: false will eliminate error.

I am using fscrawler on windows

It appears both on elasticsearch 8 and 9

• fscrawler-distribution-2.10-20250305.083042-479.zip works ok
• fscrawler-distribution-2.10-20250818.072527-86.zip generates error above
• fscrawler-distribution-2.10-20250821.064644-87.zip generates error too

Job Settings

name: "Storage"

elasticsearch:
  nodes:
  - url: "http://127.0.0.1:9200"
  semantic_search: "false"
  ssl_verification: false
  index: "Storage"
  create_index: true

fs:
  url: "c:/Elastic/fscdocs"
  update_rate: "10s"
  includes:
    - "*"
  excludes:
    - "*/~*"
  json_support: false
  # filename_as_id: true
  add_filesize: true
  remove_deleted: true
  index_content: true
  # index_folders: false
  continue_on_error: true
  ocr:
    enabled: true
    language: "ces+eng"

[dadoonet]

Thanks for reporting. Really appreciate that you tried with different builds. This helps!

Would it be possible to run the same test again, always with the latest build but with the trace level?

FS_JAVA_OPTS="-DLOG_LEVEL=trace" bin/fscrawler

Not sure if there's an equivalent for this command line on Windows though.

It should generate a full stack trace of the error which could help more to locate the exact issue and reproduce.

Thanks!

[miroslavpechacek]

Hi David, thank you for your work on fscrawler.

I have tested it now on fscrawler-distribution-2.10-20250821.064644-87.zip
With
set FS_JAVA_OPTS=-DLOG_LEVEL=trace -Xmx2g -Xms2g -DLOG_DIR=c:/ExtrasData/fscrawler/logs

On following directory

c:\Elastic\fscdocs>tree /F
Folder PATH listing for volume SYSTEM
Volume serial number is xxx
C:.
│   aa.docx
│
└───cc
        bb.docx

And log is in the attachment

fscrawler.log

[dadoonet]

Great. Thanks!

The full stacktrace is:

13:06:21,821 �[36mDEBUG�[m [f.p.e.c.f.FsParserAbstract] Full stacktrace
java.lang.StringIndexOutOfBoundsException: Range [0, -1) out of bounds for length 21
	at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:55)
	at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:52)
	at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:213)
	at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:210)
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromToIndex(Preconditions.java:112)
	at java.base/jdk.internal.util.Preconditions.checkFromToIndex(Preconditions.java:349)
	at java.base/java.lang.String.checkBoundsBeginEnd(String.java:4950)
	at java.base/java.lang.String.substring(String.java:2912)
	at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.indexDirectory(FsParserAbstract.java:606)
	at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.addFilesRecursively(FsParserAbstract.java:357)
	at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.run(FsParserAbstract.java:166)
	at java.base/java.lang.Thread.run(Thread.java:1447)

I'm going to reproduce this behavior and find a fix for it. Stay tuned.

[dadoonet]

Could you try to replace fs.url with "c:\Elastic\fscdocs" or "c:\\Elastic\\fscdocs" instead of "c:/Elastic/fscdocs"?

[miroslavpechacek]

Wow, it works with
url: "c:\\Elastic\\fscdocs"
Thanks and sorry, I did not found this way by myself.
To be sure, I am attaching log when it works.

fscrawler.log

[dadoonet]

Thanks! As I documented it with:

For Windows users, use a form like c:/tmp or c:\\tmp.

I think this should work well. I'm adding some IT on windows as well to make sure all that is tested in the future.

[miroslavpechacek]

Unfortunatelly there is "deadlock" with #1904

1904 can be fixed if / slashes are used
this issue can be fixed \ slashes are used

So there is no possibility to fix both in configuration.

If I can help with testing or log sending to be fixed one or other, I will be happy to help.
M