feat: add Spark integration

[doorgan]

This is an experiment I started over the weekend, and I think it's very viable

It implements support for Spark with the skeleton for an integrations system. Rather than a plugin system, it's just some file organization and integration points to make it easier for contributors to figure out how to add other integrations. The main assumption I am making is that user integrations will always be managed by the Engine. I do not know what a proper extensible plugin system will look like, but I do foresee it running in the engine and be contained within the engine.

RIght now there's two integration points: beam indexing, and completions, which is what the Spark integration needs.

During indexing, the beam data is passed to the enabled integrations, which produce indexer entries specific to them via the new :integration subtype and :metadata field for arbitrary information.

When requesting completions, integrations also run against the completion request and can either replace, augment or leave untouched the list of completions provided by core expert. Integration completions are returned at the top of the list as they are more likely(in my opinion) to be what the user expects, and we treat the core expert completions are generic completions for that context.

As for the Spark integration itself, it takes advantage of Spark compiling all its DSL as data within the compiled module, so the indexer basically extracts all the DSL metadata from the beam and converts it to integration entires. The completion side of it figures out the cursor context and provides the appropriate completions and docs from the DSL.

For example, for the Ash framework, if the module you are editing has use Ash.Resource, the Spark integration will:

• provide completions for actions, attributes, etc:

• provide completions for things inside a DSL block:

• Ash.create completions

[zachdaniel]

🤤