Understanding score of rule based predictors

[rickgoud]

Hi!

Maybe I am overlooking it, but can nowhere seem to find how the scoring of a rule based predictor works. Maybe it helps for everyone to do this based on a sample ad-hoc recognizer.
Let's assume I want to detect a UK bank account number with the regex ^(\d){7,8}$
Let's assume I want to specify the context words 'bank' and 'account'.
Questions:

• What is the default score if the regex pattern does match, but no context words were found?
• Is there a way to influence the score of a positive regex match only?
• What will the score be if it also finds 1 context word?
• What will the score be if it also finds 2 context words?
• At what distance (in words or characters) is a context word considered a match?
• Does the distance influence the score?
• Can the score based on the distance be influenced?
• Is there a threshold before a positive match is returned?
• Is there a way to influence the threshold before a positive match is returned?

Thanks for providing these answers. Would be very helpful to understand the power/flexibility of presidio!

[omri374]

Hello @rickgoud, please find a previous discussion here: #761 (comment)

And let us know if you have additional questions.

[rickgoud]

Thanks @omri374 ,

That is indeed helpful to at least understand the how to configure the distance!
is, looking at the code, my hypothesis correct that

1. a positive regex match give a default score of 0.4? (MIN_SCORE_WITH_CONTEXT_SIMILARITY = 0.4)
2. that a positive context word by default adds 0.35? (CONTEXT_SIMILARITY_FACTOR = 0.35)
3. that the threshold for returning a positive match is 0.65? (CONTEXT_SIMILARITY_THRESHOLD = 0.65)

if that is true. What happens to the score if you evaluate a regex without context words? Will the score be 0.4+0.35, so reaching the threshold by default? If not would be interested how to use the same set-up of presidio using both regex with or without context.

Thanks again, you Rock!

[omri374]

Hi @rickgoud,

1. Each regex has its specified default score. For example, in this one the default score is 0.5. Confidence can increase or decrease based on further analysis, such as validation (checksum or other), invalidation, or context words. The MIN_SCORE_WITH_CONTEXT_SIMILARITY means that if regex default score + CONTEXT_SIMILARITY_FACTOR < MIN_SCORE_WITH_CONTEXT_SIMILARITY is true, then the score would be MIN_SCORE_WITH_CONTEXT_SIMILARITY and not regex score + CONTEXT_SIMILARITY_FACTOR. There are regex patterns with a 0.01 score, and together with context support they get to 0.36. In such a case Presidio increases the score to 0.4.
2. Yes.
3. CONTEXT_SIMILARITY_THRESHOLD is currently not used. The threshold is defined in the .analyze call. For example:

analyzer = AnalyzerEngine()
analyzer.analyze(text="My SSN is 1234", language="en", score_threshold=0.7)