SHAP & Shapley ValuesSHAP & Shapley Values

In 1953, the mathematician Lloyd Shapley published 'A Value for n-person Games', tackling a question from cooperative game theory: if a group of players cooperates to win some total payoff, how should they split it fairly according to what each contributed?

His answer, now called the Shapley value, considers every possible order in which players could join the coalition and averages each player's marginal contribution across all of them. It is the unique way to divide the winnings that satisfies a handful of common-sense fairness axioms.

For over half a century this lived in economics and game theory, far from computing. The bridge came in 2017, when Scott Lundberg and Su-In Lee reframed a machine learning model's prediction as a cooperative game in which the players are the input features.

Their method, SHAP, short for SHapley Additive exPlanations, asks how much each feature contributed to pushing a particular prediction above or below the average. Was this loan denied mostly because of income, or credit history, or age? SHAP assigns each feature a fair share of the blame or the credit.

Because it rests on Shapley's axioms, SHAP gives explanations with guarantees that ad hoc methods lack, which made it enormously popular for auditing models in finance, healthcare, and any setting where a decision must be justified.

It is a striking piece of intellectual reuse: a Cold War-era theorem about fairness among cooperating players, resurrected to hold modern black-box models accountable one prediction at a time.