Language Model Extraction to Vindex Architecture

[metavacua]

Greetings @chrishayuk ,
I've been working on infrastructure for doing language model synthesis and distillation, and I came across a wide matrix of issues with the extraction of various language models which I am documenting on my fork, metavacua#166.

Feel free to use the issues and the reported information--I'd appreciate independent verification of some of these issues as I don't generally trust the coverage or consistency of the LLM coding agents at this time; I consider this to be strictly under the upstream (Apache 2.0 without AGPL and CC-SA-BY 4.0) licensing terms as bug fixing, optimization, and maintenance of prior art.

I am thinking of setting up a series of PRs for contribution to chrishayuk/larql, but I wanted to check with you, and the community first about how I should do that properly.

I'd appreciate PR and issue templates, workflows, instructions for the coding agents, and anything else that might be relevant to producing a well-formed contribution.

Thanks,
Ian D.L.N. McLean

[metavacua]

At current, the extract command and the knowledge pipeline are not clearly and cleanly separable concerns due to how DESCRIBE works.

The language model to vindex and vindex to language model pipelines and frameworks need significant architectural attention.

There's a seam here that cleaves LQL into two distinct languages where one version of vindexes does not rely upon the knowledge pipeline but certain LQL commands as currently implemented and designed would not work for those vindexes and the other version of vindexes is "enriched" with global knowledge graph relations and nodes such that we have at least partial isomorphisms between the language model's internal knowledge and the global knowledge graph.

[dfparker2002]

Can you share insights about identifying attention & FFN/weights layers?
My aim is to choose any LLM and dis-aggregate ATN form FFN in order to use larger models on distributed systems.
I appreciate constructive guidance.