What-If Tool
pip install wit-widget
Use the What If Tool directly in your Notebooks, in TensorBoard, and with Cloud AI models.

Model understanding in Notebooks

The What-If Tool is available as an extension in Jupyter, Colaboratory, and Cloud AI Platform notebooks. Use the What-If Tool to analyze classification or regression models on datapoints as inputs directly from within the notebook.

A custom prediction function can be used to load any model, and provide additional customizations to the What-If Tool, including feature attribution methods like SHAP, Integrated Gradients, or SmoothGrad.

Explore

Notebooks

binary classification model comparison
DATA SOURCE
Compare two binary classification models that predict whether a person earns more than $50k a year, based on their census information. Examine how different features affect each models' prediction, in relation to each other.
binary classification model comparison keras model custom distance
DATA SOURCE
Use the What-If Tool to compare two pre-trained models from ConversationAI that determine sentence toxicity, one of which was trained on a more balanced dataset. Examine their performance side-by-side on the Wikipedia Comments dataset. These are keras models which do not use TensorFlow examples as an input format.

Easily explore Cloud AI model results

The What-If Tool can be easily configured to analyze AI Platform Prediction-hosted classification or regression models.

Use the What-If Tool to display and investigate attribution values for individual input features in relation to model predictions.

Explore

Notebooks

binary classification cloud ai platform
DATA SOURCE
Explore a mortgage classification model that has been deployed on Cloud AI Platform. This model was created with the XGBoost platform and not TensorFlow.
regression model comparison cloud ai platform keras model scikit-learn model
DATA SOURCE
Train both a scikit-learn and keras model to predict wine quality and deploy them to Cloud AI Platform. Then use the What-If Tool to compare the two models. This demo requires a Google Cloud Platform account.

Use the What-If Tool in TensorBoard

Quickly explore your models in the What-If Tool from directly within TensorBoard, by providing the What-If Tool with a model server host and port, and a dataset for the model to perform predictions on.

The What-If Tool accepts a variety of data types. Upload data as tf.Examples, tf.SequenceExamples, or even a CSV file.

Explore

Web demos

binary classification model comparison
DATA SOURCE
Compare two binary classification models that predict whether a person earns more than $50k a year, based on their census information. Examine how different features affect each models' prediction, in relation to each other.
binary classification image recognition
DATA SOURCE
Predict whether an image contains a smiling face using this binary classification model on the CelebA dataset. Can you identify which group was missing from the training data, resulting in a biased model?