Integrated Gradient for Multi-Task Models

[senjed]

How can the integrated gradient explainer be used for a model with multiple outputs (e.g. a multi-task classification model where each output can have few possible values)? Can you please provide a working example? How can one choose to get attributions with respect to each of the outputs?

[gipster]

At the moment there's no option to select an output within the IntegratedGradient class, but If you have a keras or tf.keras model with multiple outputs, you can do something like that

from tensorflow.keras.models import Model 

explanations = []
for out in model.output:
    model_2 = Model(inputs=model.input, outputs=out)
    ig  = IntegratedGradients(model_2)
    explanation = ig.explain(X)
    explanations.append(explanation)

It's good to keep this issue open to understand whether it is convenient to add an option that allow to pass an output as a parameter.

[senjed]

I am getting a Graph disconnected error when I try to do this.

[jklaise]

@senjed could you provide a summary view of what your model looks like (i.e. using the summary() method) for the original and the new model?

[jklaise]

We should have an example of this.

[nouna99]

Hello;
I have the same issue: I need to apply the Integrated gradients to a multi-task model with two inputs and two outputs. but I'm getting a Graph disconnected error when I try to create a loop .:
explanations = []
out= new_model.output[0]:
inp=new_model.inputs[0]:
model_task = Model(inp, out)
ig = IntegratedGradients(model_task)
explanation = ig.explain(Z)
explanations.append(explanation)

[jklaise]

@nouna99 could you provide a short, self-contained example of the multi-task model with the error? This will help us debug the issue.

[nouna99]

new_model=tf.keras.models.load_model(my_wd, custom_objects={'f1_score': f1_score, 'precision' : precision, 'recall' : recall, 'AdamWeightDecay' : optimizer})
new_model.summary()

from tensorflow.keras.models import Model
predictions = pred_class
explanations = []
for out in new_model.output:
for inp in new_model.inputs:
model_task = Model(inputs=inp, outputs=out)
ig = IntegratedGradients(model_task)
explanation = ig.explain(X_test_sample)
explanations.append(explanation)

---

ValueError Traceback (most recent call last)
in ()
5 for out in new_model.output:
6 for inp in new_model.inputs:
----> 7 model_task = Model(inputs=inp, outputs=out)
8 ig = IntegratedGradients(model_task)
9 explanation = ig.explain(X_test_sample)

4 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py in _map_graph_network(inputs, outputs)
982 'The following previous layers '
983 'were accessed without issue: ' +
--> 984 str(layers_with_complete_input))
985 for x in tf.nest.flatten(node.outputs):
986 computable_tensors.add(id(x))

ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 114), dtype=tf.int32, name='input_word_ids2'), name='input_word_ids2', description="created by layer 'input_word_ids2'") at layer "keras_layer". The following previous layers were accessed without issue: []

[jklaise]

@nouna99 is it possible to share the model architecture, how it was constructed?

[nouna99]

Hello;
can i share it in private with you please?
Regards

[gipster]

Hi @nouna99 . Probably there's no need to share it in private. Would you mind share a minimal piece of code that reproduce the issue and that can be run as it is? It would be very helpful.