Transparency in multi-agent systems

: capturing, interpreting and visualizing execution traces

Abstract

Effective collaboration and use of tools require users to build sound mental models which are validated, strengthened and revised through increased interaction and learning.

When working with tools, mental models enable us to make predictions and estimate the reliability of said tools. With AI proliferating in our day-to-day lives, there is a need to ensure we properly understand the limitations of these systems to appropriately build our trust in them. Transparency serves as a means of improving our understanding. It is communicated via a number of media such as text, sound and visualisation.

We observe that traces captured from most multi-agent platforms (where available) have architecture and/or platform-specific representations. This is an issue as it limits the reuse of tools built for debugging agent platforms and/or providing transparency related information about agents. It is also not always possible to adequately map one architecture to another to facilitate tool reuse.

To address this, the research presented here creates a taxonomy of actively maintained agent development platforms, which are then used to identify similarities amongst the different development platforms and languages and corresponding execution traces to develop an architecture and platform-independent means of representing these traces.

Additionally, we observe that while tools exist for debugging multi-agent systems, most are tailored towards the development community and not the end-users, though a lot of research has been done on the different requirements of end-users and developers where transparency is concerned.

To address this, we develop an interactive visualisation for explaining agent decision making, taking into consideration the principle of cooperative communication, that is, Grice’s maxims of communication, namely: quality, quantity, relation and manner, which are the principles observed to be practised amongst parties engaged in cooperative communication.

We then evaluate the transparency dashboard developed as part of this work across a variety of end users and experts, and further investigate how these user groups interact with the visualisation, particularly given research to date suggests end users perform worse in comparison to experts when presented with visualisation-based explanations. We carried out a series of studies, interleaved with tool development, to achieve a degree of co-creation, leading to the form of the tool described here through two iterations. Our studies allow us to draw three conclusions: (i) a preference for compact visualisation that can provide an overview of a decision without requiring interaction; (ii) interaction is an effective way of allowing users to explore the bounds of their agents; (iii) a need for visualisations whose complexity is not perceived to be greater than that of the underlying agent.

Thus, the contribution of this work has three aspects: (i) a platform-agnostic logging framework for multi-agent systems, (ii) a visualisation dashboard to offer transparency related information, co-created with a range of users, that provides for operator adjustment of the focus and quantity of information displayed and (iii) a series of qualitative studies across a skill-diverse user base that supports the usability objectives of the work.

Date of Award	10 Sept 2025
Original language	English
Awarding Institution	University of Bath
Supervisor	Marina De Vos (Supervisor) & Julian Padget (Supervisor)