WIT - Getting Started

pip install witwidget

Use the What If Tool directly in your Notebooks, in TensorBoard, and with Cloud AI models.

Model understanding in Notebooks

The What-If Tool is available as an extension in Jupyter, Colaboratory, and Cloud AI Platform notebooks. Use the What-If Tool to analyze classification or regression models on datapoints as inputs directly from within the notebook.

A custom prediction function can be used to load any model, and provide additional customizations to the What-If Tool, including feature attribution methods like SHAP, Integrated Gradients, or SmoothGrad.

See setup guide

Explore

Notebooks

Compare income classification on UCI census data

binary classification model comparison

DATA SOURCE

UCI Census Income Dataset

Compare two binary classification models that predict whether a person earns more than $50k a year, based on their census information. Examine how different features affect each models' prediction, in relation to each other.

Text toxicity classifiers

binary classification model comparison keras model custom distance

DATA SOURCE

Wikpedia Comments Dataset

Use the What-If Tool to compare two pre-trained models from ConversationAI that determine sentence toxicity, one of which was trained on a more balanced dataset. Examine their performance side-by-side on the Wikipedia Comments dataset. These are keras models which do not use TensorFlow examples as an input format.

Easily explore Cloud AI model results

The What-If Tool can be easily configured to analyze AI Platform Prediction-hosted classification or regression models.

Use the What-If Tool to display and investigate attribution values for individual input features in relation to model predictions.

Visit Cloud AI

Explore

Notebooks

Mortgage classification with AI Platform

binary classification cloud ai platform

DATA SOURCE

Home Mortgage Disclosure Act Dataset

Explore a mortgage classification model that has been deployed on Cloud AI Platform. This model was created with the XGBoost platform and not TensorFlow.

Training and comparing wine quality models with AI Platform

regression model comparison cloud ai platform keras model scikit-learn model

DATA SOURCE

UCI Wine Quality Dataset

Train both a scikit-learn and keras model to predict wine quality and deploy them to Cloud AI Platform. Then use the What-If Tool to compare the two models. This demo requires a Google Cloud Platform account.

Use the What-If Tool in TensorBoard

Quickly explore your models in the What-If Tool from directly within TensorBoard, by providing the What-If Tool with a model server host and port, and a dataset for the model to perform predictions on.

The What-If Tool accepts a variety of data types. Upload data as tf.Examples, tf.SequenceExamples, or even a CSV file.

See setup guide

Explore

Web demos

Compare income classification on UCI census data

binary classification model comparison

DATA SOURCE

UCI Census Income Dataset

Explore celebrity face image smile classification

binary classification image recognition

DATA SOURCE

CelebA Dataset

Predict whether an image contains a smiling face using this binary classification model on the CelebA dataset. Can you identify which group was missing from the training data, resulting in a biased model?