How do you calculate the individual fairness based on exposure from implicit feedback?

[Ronosu]

Hello ashudeep,

I read your paper
https://papers.nips.cc/paper/8782-policy-learning-for-fairness-in-ranking
and I am impressed with your idea and would like to do research based on this paper.
However, there are two things I don't understand after reading your paper.

1. When calculating the fairness of individual i, I think the following formula is used.
   exposure (i) / merit (i)
   But how do you calculate if the denominator merit (i) is 0 in implicit feedback?

2. Should fairness be calculated against the overall ranking?
   In general, people will use the top k(<<n) rankings for n items.
   Do you need to calculate group fairness for all n items, not just top k?
   In other words, is it a fair ranking if groups A and B have the following for the items in top k only?
   group exposure(A) / group merit(A)
   = group exposure(B) / group merit(B)

When I read your paper, it seems to me that you are doing the math for every item...

Thank you

[ashudeep]

Thank you for reading the paper and the questions @Ronosu.

Here are the answers to your questions:

1. For the groups where merit(i) is 0, you do not need to satisfy a constraint since any amount of exposure will be proportional to 0. Yes, it a problem of describing the constraints as exposure(i)/merit(i) but we basically just skip the items that have 0 average relevance.
   One notion of fairness that you can think of is exposure(i)/avg-merit(i) is equal for all items, where avg-merit is the average of relevance/merit across all queries (rather than the relevance for the current query). But you need to use such a notion carefully as you are assuming that average relevance over all the queries has a meaning.

2. Exposure is a function that can be defined according to the application. So, in case there are a lot of items and you are only concerned about a top-k ranking, you can set the exposure@k' for k'>k to be 0. This will save computation on the tail of the ranking.

[Ronosu]

Thank you for replying @ashudeep

I've understood that when I consider the fairness of exposure, I should skip the item which relevance is 0 .
I have more questions for you,

1.. When I consider the group fairness of exposure about group A, that
avg-exposure(A) / avg-merit(A)
how do I calculate the average (avg) ?
If skipping negative items (relevance is 0) as you say, I shouldn't include negative items in the average, right?
That is, in implicit feedback, is the denominator avg-merit(A) always 1?
Should I average only for positive items (relevance is 1) when calculating the numerator avg-exposure(A)?

2. When I think of about exposure@k what happens to the denominator ? Is the denominator a constant? Or does it change according to the top k?

Does the paper explain the handling of exposure in implicit feedback?
(If so, please tell me the section)

Thanks