feat: Perpendicular add

[ljleb]

Perpendicular add:

$$ M = A + \alpha \times ((B - C) + (A - C) \frac{(A - C) \cdot (B - C)}{(A - C) \cdot (A - C)}) $$

model A: https://civitai.com/models/81359?modelVersionId=86337
model B: https://civitai.com/models/108882/ink-prt
model C: SD1.5 base model from runwayml
alpha: 1.0

[image]

[ljleb]

I'll open a new PR when I find a good candidate for negative weights clipping.

[ljleb]

@Clybius (apologies if I got the wrong handle!) came up with a more appropriate approach for the projection part of the formulae, which I just implemented. It seems to work much better on models that have a low cosine similarity from one another. On models that are similar, it resembles a weighted sum. I updated the formulae above in the OP to reflect this.

I tested rebasin with add perpendicular difference and it works really well. Since plain add difference doesn't work with rebasin, I think this method might be a valuable replacement in some cases. It also doesn't need weights clipping to crank alpha up to 1. More tests needed with dj cats I guess.