Difference with speculative

[WilliamsToTo]

What's the difference between CleanGen and speculative decoding baseline? I think CleanGen is very similar to speculative decoding. The main difference is that two models in CleanGen have the same size but speculative decoding requires a smaller model and a larger model.

[1GaryLi]

Thank you for your question. The main difference is that CleanGen has different algorithms to replace a "suspicious" token during decoding. Speculative's main goal is that the decoding output should always follow the distribution of the target model (thus if the target model is backdoored, the output is still compromised). CleanGen's main goal is that when the input contains trigger, the output should follow the distribution of reference model, when the input has no trigger, the output should follow the target model. Hope this will help your question!