Model with conditional interaction
In this example, we show global effects of a model with conditional interactions using PDP, ALE, and RHALE. In particular, we:
- show how to use
effector
to estimate the global effects using PDP, ALE, and RHALE - provide the analytical formulas for the global effects
- test that (1) and (2) match
We will use the following model:
where the features
The model has an interaction between
In contrast,
import numpy as np
import matplotlib.pyplot as plt
import effector
np.random.seed(21)
model = effector.models.ConditionalInteraction()
dataset = effector.datasets.IndependentUniform(dim=3, low=-1, high=1)
x = dataset.generate_data(1_000)
PDP
Effector
Let's see below the PDP effects for each feature, using effector
.
pdp = effector.PDP(x, model.predict, dataset.axis_limits)
pdp.fit(features="all", centering=True)
for feature in [0, 1, 2]:
pdp.plot(feature=feature, centering=True, y_limits=[-2, 2])
PDP states that:
has a zero average effect on the model output has a constant effect when or , but when moving from to it adds (on average) units to has an effect of
Derivations
How PDP leads to these explanations? Are they meaningfull? Let's have some analytical derivations. If you don't care about the derivations, skip the following three cells and go directly to the coclusions.
For
For
For
Tests
def compute_centering_constant(func, start, stop, nof_points):
x = np.linspace(start, stop, nof_points)
y = func(x)
return np.mean(y)
def pdp_ground_truth(feature, xs):
if feature == 0:
ff = lambda x: np.zeros_like(x)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
elif feature == 1:
ff = lambda x: -1 / 3 * (x < 0) + 1 / 3 * (x >= 0)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
elif feature == 2:
ff = lambda x: np.exp(x)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
xx = np.linspace(-1, 1, 100)
y_pdp = []
for feature in [0, 1, 2]:
y_pdp.append(pdp_ground_truth(feature, xx))
plt.figure()
plt.title("PDP effects (ground truth)")
color_pallette = ["blue", "red", "green"]
for feature in [0, 1, 2]:
plt.plot(
xx,
y_pdp[feature],
color=color_pallette[feature],
linestyle="--",
label=f"feature {feature + 1}"
)
plt.legend()
plt.xlim([-1.1, 1.1])
plt.ylim([-2, 2])
plt.show()
# make a test
xx = np.linspace(-1, 1, 100)
for feature in [0, 1, 2]:
y_pdp = pdp.eval(feature=feature, xs=xx, centering=True)
y_gt = pdp_ground_truth(feature, xx)
np.testing.assert_allclose(y_pdp, y_gt, atol=1e-1)
Conclusions
Are the PDP effects intuitive?
- For
the effect is zero. The terms related to are and . Both terms involve an interaction with . Since , almost half of the instances have and the the other half , so the the two terms cancel out. - For
, the effect is constant when or but has a positive jump of when moving from to . It makes sense; when the active term is $-(x_1^i)^2 \mathbb{1}_{x_2 < 0} $ which adds a negative quantity to the output and when the active term is that adds something postive. Therefore in the transmission we observe a non-linearity. - For
, the effect is , as expected, since only the this term corresponds to and has no interaction with other variables.
ALE
Effector
Let's see below the PDP effects for each feature, using effector
.
ale = effector.ALE(x, model.predict, axis_limits=dataset.axis_limits)
ale.fit(features="all", centering=True, binning_method=effector.axis_partitioning.Fixed(nof_bins=31))
for feature in [0, 1, 2]:
ale.plot(feature=feature, centering=True, y_limits=[-2, 2])
ALE states that:
-
Derivations
For all bins, except the central, it holds that bin limits are either both negative or both positive, so the effects cancel out.
For central bin, i.e., the one from
Tests
def ale_ground_truth(feature, xs):
if feature == 0:
ff = lambda x: np.zeros_like(x)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
elif feature == 1:
K = 51
ff = lambda x: -1/3 * (x < 0) + 1/3 * (x >= 0)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
elif feature == 2:
ff = lambda x: np.exp(x)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
xx = np.linspace(-1, 1, 100)
y_ale = []
for feature in [0, 1, 2]:
y_ale.append(ale_ground_truth(feature, xx))
plt.figure()
plt.title("ALE effects (ground truth)")
color_pallette = ["blue", "red", "green"]
for feature in [0, 1, 2]:
plt.plot(
xx,
y_ale[feature],
color=color_pallette[feature],
linestyle="--",
label=f"feature {feature + 1}"
)
plt.legend()
plt.xlim([-1.1, 1.1])
plt.ylim([-2, 2])
plt.show()
xx = np.linspace(-1, 1, 100)
for feature in [1]:# [0, 1, 2]:
y_ale = ale.eval(feature=feature, xs=xx, centering=True)
y_gt = ale_ground_truth(feature, xx)
# hack to remove the effect at undefined region
if feature == 1:
K = 31
ind = np.logical_and(xx > -1/K, xx < 1/K)
y_ale[ind] = 0
y_gt[ind] = 0
np.testing.assert_allclose(y_ale, y_gt, atol=1e-1)
Conclusions
Are the ALE effects intuitive?
ALE effects are identical to PDP effects which, as discussed above, can be considered intutive.
RHALE
Effector
Let's see below the RHALE effects for each feature, using effector
.
rhale = effector.RHALE(x, model.predict, model.jacobian, axis_limits=dataset.axis_limits)
rhale.fit(features="all", centering=True)
for feature in [0, 1, 2]:
rhale.plot(feature=feature, centering=True, y_limits=[-2, 2])
RHALE states that:
-
Derivations
def rhale_ground_truth(feature, xs):
if feature == 0:
ff = lambda x: np.zeros_like(x)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
elif feature == 1:
K = 31
ff = lambda x: np.zeros_like(x)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
elif feature == 2:
ff = lambda x: np.exp(x)
z = compute_centering_constant(ff, -1, 1, 1000)
return ff(xs) - z
xx = np.linspace(-1, 1, 100)
y_rhale = []
for feature in [0, 1, 2]:
y_rhale.append(rhale_ground_truth(feature, xx))
plt.figure()
plt.title("RHALE effects (ground truth)")
color_pallette = ["blue", "red", "green"]
for feature in [0, 1, 2]:
plt.plot(
xx,
y_rhale[feature],
color=color_pallette[feature],
linestyle="-" if feature == 0 else "--",
label=f"feature {feature + 1}"
)
plt.legend()
plt.xlim([-1.1, 1.1])
plt.ylim([-2, 2])
plt.show()
for feature in [0, 1, 2]:
y_ale = rhale.eval(feature=feature, xs=xx, centering=True)
y_gt = rhale_ground_truth(feature, xx)
np.testing.assert_allclose(y_ale, y_gt, atol=1e-1)
Conclusions
Are the RHALE effects intuitive?
RHALE does not add something new, compared to ALE and PDP, for features