Question: Should TreeSitter.create() / newParser() used with multiple threads?

[sshaaf]

1. Is TreeSitter.create() / newParser() intended to be thread-safe when called concurrently on one instance?
2. If not, should TreeSitter be per-thread (or per-task), or is there a recommended pattern for parallel batch parsing?

  TreeSitter ts = TreeSitter.create();
  try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
      for (Path file : javaFiles) {
          executor.submit(() -> {
              String source = Files.readString(file);
              TreeSitterParser parser = ts.newParser();
              parser.setLanguage(Language.JAVA);
              TreeSitterTree tree = parser.parseString(source);
              tree.close();
              parser.close();
          });
      }
  }

[cmoulliard]

As, when we parse an application, we would like to discover the files belonging to different languages (for later query/analyze), then the process to walk through the files locally could run using virtual threads in parallel as they will not conflict (I mean a virtual thread) between the languages.

As VirtualThread is only available since JDK21+, by consequence, a fallback is also needed !

[cmoulliard]

I opened a ticket as I got memory issue reported by chicory: dylibso/chicory#1301