How you evaluate the in-domain generalization of the honesty probe?

[wenjunli-0]

I am curious how you evaluate the in-domain generalization of the honesty probe. I found this in the paper

With this setup, the resulting LAT reading vector reaches a classification accuracy of over 90% in distinguishing between held-out examples where the model is instructed to be honest or dishonest.

The trained honesty probe can predict the honesty score at each token position in the response and the honesty scores associated with a complete response should consist of many scores. How do you determine whether the model is instructed to be honest or dishonest (i.e., a binary prediction) based on the long sequence of honest scores?

[yitong241]

Same question here, simply taking the average of all honest scores of generated tokens does not seem to be a good indicator of honest of dishonest.