Skip to content

Create a new class able to handle a pool of TreeSitter wasm module#13

Open
cmoulliard wants to merge 2 commits into
mainfrom
add-thread-pool
Open

Create a new class able to handle a pool of TreeSitter wasm module#13
cmoulliard wants to merge 2 commits into
mainfrom
add-thread-pool

Conversation

@cmoulliard

Copy link
Copy Markdown
Collaborator
  • Create a new class able to handle a pool of TreeSitter wasm module: TreeSitterPool

…reeSitterPool

Signed-off-by: Charles Moulliard <cmoulliard@ibm.com>
Signed-off-by: Charles Moulliard <cmoulliard@ibm.com>
@andreaTP

Copy link
Copy Markdown
Collaborator

For pooling we successfully followed this pattern in the past:
https://github.com/StyraOSS/opa-java-wasm/blob/main/core/src/main/java/com/styra/opa/wasm/OpaPolicyPool.java

Did you checked that it's effective and the increase of complexity(and infrastructure) is really saving time at the end?

@cmoulliard

Copy link
Copy Markdown
Collaborator Author

Did you checked that it's effective and the increase of complexity(and infrastructure) is really saving time at the end?

The most important thing that I would like to achieve here is to be able to have x instances where an instance can handle a tree-sitter grammar/language to parse files (= and to populate ASTree) and to allow to query AST trees using S-expression queries

As a the beginning of the process, the tool using this lib will load from a path (= java application) collections of file where a collection corresponds to a language: java, properties, etc then the idea is to allow to parallel the work to build the AST trees for the different languages in parallel

@andreaTP

@andreaTP

Copy link
Copy Markdown
Collaborator

what you are asking is achievable also without pooling, let see what the bottleneck really is and fix it

@cmoulliard

Copy link
Copy Markdown
Collaborator Author

bottleneck really is and fix it

So this is here that we should have a look to parallelize the process reading the files and building the ASTrees for each language: https://github.com/ch007m/java-tree-sitter/blob/5dfe908a8ad4eeed75f1d885ca441f207bf20fa1/tree-sitter4j-client/src/main/java/dev/snowdrop/treesitter4j/util/ASTParserUtil.java#L134-L172

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants