fairlearn.org - everything related to the Fairlearn package. Please feel free to ask any kinds of questions!
1 reply
15 replies
The biggest issue I see in this generic approach is, well, that it’s generic. 😅 If we go down this path, it emphasizes that fairness can be seen as an intrinsic problem of the model, rather than being a sociotechnical one.
On the other hand, it’s a lot better than using an inappropriate data set. So it might be a good alternative until we find a better one.
What do you think, does this make any sense?
1 reply
I’d be okay with gender queer as a term, as it makes clear the intention is to represent people outside a binary gender scheme. I can’t speak for all non-binary though. 😬
@Roman that’s actually a good idea. Without having read the original papers, I wouldn’t have understood it either. Having a approachable theoretical section is a unique selling point for the fairlearn documentation I guess.
2 replies
Re. collecting gender demographics, I have recently chatted with Queer in AI folks, and they suggest the following:
Gender: (please select the options which are applicable to you)
[] Woman
[] Man
[] Non-binary / Genderqueer / Third gender
[] Genderfluid / Gender non-conforming
[] Questioning
[] Prefer not to say
[] Specify your own (open text box)
The key aspect is that this is a multi-select. When this kind of data is collected, there should be an explanation that the data is collected to audit for fairness and/or mitigate any fairness issues, and that if you pick "prefer not to say" you will not be included in any fairness audits.
I also like these two sources:
Gender (apa.org)
Respectful Collection of Demographic Data | by Sarai Rosenberg | Managing on the Margins | Medium
10 replies
3 replies
3 replies
I'll be there both days. I believe they're planning to set up channels for each project (incl. one for Fairlearn).
I'm also available right now for the next 3.5h (until 1pm Pacific Time) if you want to discuss anything or pair program etc.
24 replies
In scikit repo conf.py which you pointed in description of the issue:
https://github.com/scikit-learn/scikit-learn/blob/309f135c3284d7db6e23ca81a87948c7066a3949/doc/conf.py#L338.
I am confused what should we put for in the case of branch and dependencies - requirements.txt. If you look at scikit-repo requirements - https://github.com/scikit-learn/scikit-learn/blob/309f135c3284d7db6e23ca81a87948c7066a3949/doc/binder/requirements.txt, it's confusing.
What should we put in the requirements.txt in case of fairlearn repository?
2 replies
predict_proba. Note that our meta estimators like ExponentiatedGradient do NOT produce probabilities the same way, so we've very consciously chosen not to name it predict_proba.
This will allow us to do things like
- hanging out!
- pair programming and ride-alongs
quite easily. It's worth noting that this doesn't mean this Gitter channel isn't active anymore. We're just trying out Discord for now. Feedback is more than welcome!
If it's used by trolls I can still deactivate it, though. But for now this one should work and the above one may not work for you
Assessment: our user guide covers both classification and regression metrics https://fairlearn.org/main/user_guide/assessment.html#metrics
Mitigation: The table shows which techniques should work for regression: https://fairlearn.org/main/user_guide/mitigation.html (everything but ThresholdOptimizer)
Admittedly, the user guide isn't super comprehensive yet and has some gaps, but there is a regression section for fairness constraints that can be plugged into our fairlearn.reductions techniques: https://fairlearn.org/main/user_guide/mitigation.html#fairness-constraints-for-regression
We don't have examples with regression mitigation yet, but this test case code might be useful to you: https://github.com/fairlearn/fairlearn/blob/62fc80c77bcd3bef6a3d7bc44e54827ec9fb8d09/test/unit/reductions/grid_search/test_grid_search_regression.py#L64
If you're willing to explain your use case we could try to advise on how to use Fairlearn. Let us know!
1 reply
3 replies
6 replies