Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Feb 25 06:32
    kalikhademi commented #339
  • Feb 24 00:24
    romanlutz commented #703
  • Feb 23 20:52
    riedgar-ms commented #703
  • Feb 22 10:47
    hildeweerts commented #703
  • Feb 20 07:08
    adrinjalali closed #705
  • Feb 20 07:08
    adrinjalali commented #705
  • Feb 20 07:07
    adrinjalali commented #703
  • Feb 19 23:53
    manasatramalinga opened #705
  • Feb 19 23:30

    romanlutz on main

    add CNAME file command (#702) … (compare)

  • Feb 19 23:30
    romanlutz closed #702
  • Feb 19 23:30
    romanlutz commented #702
  • Feb 19 22:51
    romanlutz commented #702
  • Feb 19 07:48
    adrinjalali opened #704
  • Feb 19 07:43
    adrinjalali edited #703
  • Feb 19 07:41
    adrinjalali opened #703
  • Feb 19 07:06
    adrinjalali commented #698
  • Feb 19 07:04
    adrinjalali commented #702
  • Feb 19 05:12
    github-actions[bot] assigned #702
  • Feb 19 05:11
    romanlutz review_requested #702
  • Feb 19 05:11
    romanlutz review_requested #702
Richard Edgar
@riedgar-ms
Happy New Year to all.... a PR to get our builds working again:
fairlearn/fairlearn#669
1 reply
Richard Edgar
@riedgar-ms
One more PR to fix a problem with the TensorFlow tests:
fairlearn/fairlearn#671
Richard Edgar
@riedgar-ms
And another build break:
fairlearn/fairlearn#675
This was caused by a flake8 rule updating. Now I look at the code, I think that one of these is actually a legitimate complaint - why replace the actual exception by a ValueError?
4 replies
trivenigandhi
@trivenigandhi
Hi everyone! I am so excited to come across the fairlearn community and I am eager to help out however I can. I was thinking about an example notebook using publicly available health data, but I would also be happy to work through any simple code base issues...please let me know where I can best direct my energy!
8 replies
vincent d warmerdam
@koaning
So ... not to start a rant. But! Anybody seen this announcement

Quote

BCG GAMMA FACET Helps Human Operators Understand Advanced Machine Learning Models So They Can Make Better and More Ethical Decisions

Their guide meanwhile uses the load_boston dataset to explain the tool without acknowledging the B column.
Feel free to pitch in BCG-Gamma/facet#202
Also, can anybody confirm if the scikit-learn team is actually contributing to this project? It feels strange that this didn't get flagged before.
vincent d warmerdam
@koaning
The original announcement suggests a collaboration, but it feels strange since I thought it was common knowledge what the issues with load_boston were.
Hilde Weerts
@hildeweerts
Good to see they're immediately taking action :D
trivenigandhi
@trivenigandhi
That's disappointing to see, but good they responded quickly. I think it only highlights the need for people to have the social context before diving into solutioning.
Roman Lutz
@romanlutz
Indeed... I think we need to write that user guide section about the Boston dataset so that we can point them to something more structured :-)
vincent d warmerdam
@koaning

That's disappointing to see, but good they responded quickly. I think it only highlights the need for people to have the social context before diving into solutioning.

This is especially true when strategy consultancies are marketing these tools. Especially if it is at global scale.

vincent d warmerdam
@koaning
Also, just sayin' ... if folks are interested in speaking about fairlearn this year. Scipy talks are open for submission https://www.scipy2021.scipy.org/talk-poster-presentations
trivenigandhi
@trivenigandhi
Hi all! I am working in a community edition of Pycharm. Any suggestions on how to render the FairLearnDashboard without a Jupyter notebook?
Roman Lutz
@romanlutz
@trivenigandhi that's not supported. The dashboard is moving to another package (called raiwidgets, to be released at the end of this month) and there's no further work planned before we remove it from Fairlearn. We are adding matplotlib-based plots, though, but only in the next version.
Manojit Nandi
@LeJit
Here's the PyCon talk proposal I presented in today's community call for those of you who want to take a closer look at it: https://github.com/fairlearn/fairlearn/discussions/674#discussioncomment-300005
MiroDudik
@MiroDudik
@hildeweerts , @trivenigandhi , @romanlutz : potato!
3 replies
trivenigandhi
@trivenigandhi
Following up on the same threat as @LeJit here is the link to the SciPy proposal! Looking forward to your feedback! https://github.com/fairlearn/fairlearn/discussions/674#discussioncomment-306760
André Cruz
@AndreFCruz
hey all! I'm having an error when fitting an Exponentiated Gradient with FPR constraint ValueError: Phase 1 of the simplex method failed to find a feasible solution. (...) Consider increasing the tolerance to be greater than 3.9e-01. (...) ;
Any idea which parameter I should change if I want to use the classifier with the solution that was found as is ? Is it EG's nu ? Or the constraint's ratio_bound_slack ?
5 replies
vincent d warmerdam
@koaning

To anybody who is bored. I've found another global consultancy company that didn't look at the contents of the load_boston dataset.

Upvotes appreciated. quantumblacklabs/causalnex#91

17 replies
vincent d warmerdam
@koaning
@romanlutz crazy idea. But would it perhaps be an idea to build a github bot that automatically warns repos of the load_boston dataset?
Adrin Jalali
@adrinjalali
I mean, they should already be getting a warning that it's deprecated I think
vincent d warmerdam
@koaning
True, but not everyone tests their docs.
I just checked, there's like 30K+ repos on github that uses load_boston in the docs/tests/script.
vincent d warmerdam
@koaning
Do we know any Belgian journalists in this group? I may have found something that deserves investigation.
https://traicie.com/
9 replies
Richard Edgar
@riedgar-ms
Can I ping for a second review for:
fairlearn/fairlearn#689
trivenigandhi
@trivenigandhi
@romanlutz @MiroDudik @hildeweerts @LisaIbanez let me know if you would like access to the scipy proposal! I posted the link on the github discussion and manojit already has access
Roman Lutz
@romanlutz
FYI this week Prof. Steven Wu from CMU is speaking about “Involving Stakeholders in Building Fair ML Systems" at the TrustML community:
https://groups.google.com/g/trustworthyml/c/GP2YHDOEG2g/m/erQq7FlZEAAJ
Richard Edgar
@riedgar-ms
I'm looking to get the v0.6.0 release rolling (starting tomorrow, I think) - reply to this if there's some critical bugfix you think we're missing. The release process will likely result in some PRs that get put out and merged without the usual 24 hour opportunity for comment. Tagging @adrinjalali @hildeweerts @romanlutz @MiroDudik in particular.
15 replies
Frédéric Branchaud-Charron
@Dref360

Hello, I had a question on the different types of biases in datasets. I couldn't find much of it online.
So far, my team and I explored two types of biases:

  1. There is a sensitive feature that correlates with y but it would be unfair to use it for making the prediction. Even in the infinite amount of data and a perfect predictor, it would still use that feature for making the prediction and it would be unfair.

  2. The minority class does not correlate with y, but there is much less samples for training the model in that subspace (since the distribution is uneven). In the limit of infinite data, this is no longer a problem, you converge to the optimal classifier and this variable was not a sensitive variable.

Are you aware of other types of biases in datasets?
Thanks for your help!

4 replies
Richard Edgar
@riedgar-ms
Another small update.... Python 3.9 is starting to look promising:
fairlearn/fairlearn#646
Unfortunately, this doesn't cover all potential 3.9 builds, but a couple of weeks ago, even this limited set wasn't working
LisaIbanez
@LisaIbanez
@trivenigandhi yes, I would love access to review! Unfortunately, I can't make tomorrow's meeting, but will def check back here for updates.
Richard Edgar
@riedgar-ms
@adrinjalali to your query on the Python 3.9 PR just merged.... does SciKit-Learn have Python 3.9 wheels out for v0.22.1?
7 replies
Hilde Weerts
@hildeweerts
@trivenigandhi @LisaIbanez @brkifle @WesDeng @kevinrobinson during yesterday's community call we discussed some ideas for fairlearn's roadmap. As frequent attendees we'd love to get your opinion as well, so if you have any ideas on where you would like to see the project going, please feel free to reach out! :D
7 replies
(i apologize in advance if i forgot somebody)
Roman Lutz
@romanlutz
Since v0.6.0 is out I moved us from master to main as the default branch. To make things work with main please approve my PR fairlearn/fairlearn#694 @hildeweerts @adrinjalali @riedgar-ms @MiroDudik @mmadaio
2 replies
vincent d warmerdam
@koaning

Hi all.

With my PR on the correlation removed out there, I feel like I might start working on some educational content for fairlearn. My plan is to host it on https://calmcode.io but I imagine that content could be re-used on the docs as well.

There's one thing though, I don't have a clear intuition on how the reductions actually work. The docs host an example that's a bit too theoretical for me to immediately get my head around ... so is there a call that I can join with a whiteboard soon-ish? I'm not sure if I'm the only one who feels this way, or if the usual meetings are the best place for this, but I'd certainly appreciate a better "paper-napkin"-level of intuition.

4 replies
Hilde Weerts
@hildeweerts
Hi everyone! In this Thursday's community call, we will discuss @hannawallach 's recent work on fairness through the lens of measurement modeling and think about ways to translate these ideas into a toolkit/educational materials. I think this will be a super interesting discussion, here is a link to the paper: https://arxiv.org/abs/1912.05511?
Richard Edgar
@riedgar-ms
PR to cover a testing gap I noticed today:
fairlearn/fairlearn#701
We didn't have any purely public tests of the MetricFrame with sample parameters
Adrin Jalali
@adrinjalali
@romanlutz could you please add the link to the notes (https://hackmd.io/QcJ8WBAbQTq6iOgoi0JT4g?both) to the calendar invite?
2 replies
Roman Lutz
@romanlutz
As you may know we have fairlearn.org as the domain name. So far https was not set up for that, so I'm currently trying to do that. If you have issues accessing the page in the meanwhile please respond by creating a thread here. https still doesn't work since GitHub first needs to issue a certificate, but http://fairlearn.org and fairlearn.github.io should both work and redirect to fairlearn.org (watch what your browser resolves to). Works at least on my computer and my phone. Https should work later tonight/early tomorrow depending on which timezone you're in.
6 replies
Hilde Weerts
@hildeweerts
Hi everybody! Because some people couldn't make it last week, the discussion of @hannawallach's measurement and fairness paper is postponed to this Thursday (25/2/2021). @MiroDudik's talk on reductions will be next week (4/3/2021).
8 replies
Adrin Jalali
@adrinjalali
I'm trying to figure out what the vibe around the model-card-toolkit is. Anybody here knows anything? Seems like there isn't really a community around it. I'm wondering if we have any contacts in the team that we could talk to, to see what their plans are. Also wondering if it makes sense or if there's enough support to actually fork it and create a community around it.
9 replies
It also has a hard dependency on a bunch of tensorflow related packages, which I don't think is necessary for a model card toolkit kinda library.
Adrin Jalali
@adrinjalali
Is this something we'd wanna talk about in our weekly?
Adrin Jalali
@adrinjalali
I was trying to find out where we say how often and how we meet in our docs (to check when it was NY time to make sure I don't miss it due to day light saving changes), but I can't seem to find it
1 reply
Hilde Weerts
@hildeweerts
Thanks everyone for the interesting discussion! I felt like I needed a brain dump, so I opened a discussion (https://github.com/fairlearn/fairlearn/discussions/707)
Please feel free to add your thoughts, perhaps its helpful to collect some of the things we're thinking about to structure next week's followup :)