Enterprise AI Analysis

Summary of Key Findings

Our experiments demonstrate that AgentSHAP provides reliable and meaningful tool importance scores. The consistency experiment shows that Monte Carlo sampling produces stable results with 0.945 mean cosine similarity between runs.

The faithfulness experiment confirms that SHAP values reflect actual importance with 13x quality difference when removing important versus unimportant tools. The irrelevant tool injection experiment shows AgentSHAP assigns the expected tool 7x higher scores than irrelevant ones. The cross-domain experiment demonstrates 86% accuracy across different query types. Together, these results validate AgentSHAP as a practical tool for understanding and optimizing LLM agent behavior.

Limitations & Future Work

AgentSHAP has several limitations that suggest directions for future research. First, it measures individual tool contributions without capturing synergies between tools. When Calculator and Wiki together produce better results than either alone, this interaction effect is not directly measured. Extending to pairwise or higher-order Shapley interactions could address this.

Second, AgentSHAP analyzes single prompt-response pairs rather than multi-turn conversations where tool importance may shift over time. Tracking importance across conversation turns would provide richer insights for dialogue agents. Third, when agents call multiple tools in sequence, AgentSHAP attributes to the tool set rather than the specific calling order.

Future work could also use SHAP patterns to automatically recommend adding or removing tools, integrate with agent training pipelines for tool-aware fine-tuning, and extend to multi-agent systems where multiple agents collaborate using shared tools.