FAT* 2020 Hands-on Tutorial: Probing ML Models for Fairness with the What-If Tool and SHAP
Presented by James Wexler (Software Developer, Google) and Andrew Zaldivar (Responsible AI Developer Advocate, Google).
Monday, January 27th 2020. Barceló Sants Hotel, Barcelona, Spain. Rooms MR5+MR6.
Additional work by Tolga Bolukbasi (Software Developer, Google), Mahima Pushkarna (UX Designer, Google), and Sara Robinson (Cloud Developer Advocate, Google).
About the Tutorial
As more and more industries use machine learning, it's important to understand how these models make predictions, and where bias can be introduced in the process. This tutorial will walk you through two open source frameworks for analyzing machine learning (ML) models from a fairness perspective.
The tutorial will first introduce the What-If Tool, a visualization tool that you can run inside a Python notebook to analyze an ML model. With the What-If Tool, you will learn to identify dataset imbalances, see how individual features impact your model's prediction through partial dependence plots, and analyze human-centered ML models from a fairness perspective using various optimization strategies.
Then we will look at SHAP, a tool for interpreting the output of any ML model and seeing how a model arrived at predictions for individual datapoints. We will then use SHAP and the What-If Tool together.
After the tutorial, you'll have the skills to get started with both of these tools on your own datasets, and be better equipped to analyze your models from a fairness perspective.
Participants should have a basic knowledge of Python (although all tutorial steps can be run with no python knowledge), and should bring a laptop in order to run the Colaboratory notebook containing the code for the tutorial. The only requirement for running Colaboratory notebooks is either a GMail or GSuite account. No software installations are necessary; you can run Colaboratory notebooks directly from the browser.
After the tutorial, we would love to get your feedback on the tool. Click here to provide feedback.
What-If Tool Community
Join the What-If Tool discussion group to stay up to date on all things WIT.