In my doctoral work at Leiden University, I am interested in finding a new way to understand interactions in interactive art. As interactive art is evolving rapidly, more and more novel forms of interaction are being explored and developed. The conventional way of understanding audience-artwork interaction as a dialogue between the two partners seems insufficient to capture the different types of dynamics at play. Taking this observation as a starting point, we (together with my supervision team) developed a new interaction model and practical tool for describing, visualising and generative interactive dialogues and used it to analyse diverse range of interactive artworks focusing on co-located audience interaction and more-than-human interaction.
This research is supervised by Maarten H. Lamers, Edwin van der Heide, Fons J. Verbeek.
A Relational Model of Interaction
We started by reviewing existing interaction models and tools for describing and analysing interaction in interactive art, discussing their strengths and limitations. Drawing on this analysis, we outlined the key considerations for developing an interaction model that can capture a wide range of interactions. We then proposed a relational model of interaction with an emphasis on how the interacting elements relate to and influence each other based on their actions and the resulted forms of communication between them.
Publication:
From the Relational Model to the Relational Modelling Tool
Following the introduction of the relational model, we developed a web-based application, referred to as the Relational Modelling Tool (RMT). RMT offers a structured and formal input area that guides users in describing an interaction using the concepts of the relational model. Given an artwork's formal description, it automatically generates a visual graph that provides a direct overview of the interaction. The formal structure of the tool enforces rigorous reasoning about the relationships between the actions of the elements, thereby making the description more detailed and precise. It also generates data with consistent types and structures that allow us not only to easily compare different interactions, but also to generate new interacting elements and draw new connections between them using computational techniques. As such, we believe that RMT not only benefits the analysis of interactive art, but also facilitates the discovery of new forms of interaction.
Publication:
Describing and Comparing Co-located Interaction
 by Rafael Lozano-Hemmer.jpg)
Co-located interaction refers to scenarios in which two or more audience members engage simultaneously with an interactive artwork within a shared physical space. In such contexts, the presence of and interactions among audience members are not merely incidental, but are integral to the artwork itself. Here we applied the tool to analyse a collection of carefully selected interactive artworks focusing on co-located interaction. Based on the outcomes, we observe that the interacting elements not only can act and react to one another in various ways, but their actions also influence each other in ways that are essential to the interaction and for establishing complex relationships among the elements. One aspect of co-located interaction that was highlighted in our analysis of the selected artworks pertains to its inherently social nature. To facilitate this, an art system can employ various strategies to foster connections between audience members, drawing upon their ability to communicate directly in different ways, and devising novel forms of mediated communication. Since the relational model does not presuppose differences in the behaviours of the art system and the audience, it also opens up new possibilities to conceive new interactive dialogues, such as by subverting or exchanging the roles of the elements.
Publication:
Describing and Comparing More-than-Human Interaction
 by Jiabao Li.jpg)
Interactive art is also evolving towards a more-than-human turn, embracing the significance and opportunities of involving non-human participants. We select five interspecies interactive artworks and model and visualise the more-than-human interaction within them using the tool. From these descriptions, we identify a general pattern of active participation of human participants and passive participation of non-human participants. Furthermore, we also observe diverse forms of mediated communication, including communications from non-human organisms to humans via art systems and vice versa, as well as the communication from art systems to non-human participants via humans. Lastly, we discuss the different ways art systems reconfigure the interspecies relationships between humans and non-human organisms, either by creating a collaborative context for both participants or creating and mediating physical connections between them. By uncovering patterns in the roles of human and non-human elements within these artworks, we not only demonstrate that the relational model is effective in analysing such interactions but also propose novel roles for future more-than-human interactive art.
Publication:
Workshop on Modelling and Creating Interactions
To evaluate the tool with a broader user group, we organised a workshop at the 13th EAI ArtsIT conference, inviting researchers and practitioners from interactive art and interaction design to model existing interactions and explore new interactive dialogues using it. The workshop enabled collecting feedback on usability and functionalities of the tool while exploring additional benefits, opportunities, and potential improvements.
To facilitate the workshop, we developed a generative component that allows users to explore new forms of interaction by randomly generating interacting elements and drawing connections between them:
During the workshop, the participants were instructed to use the tool to model an existing interactive work of their choice and generate new forms of interaction:
Key insights from the workshop centred on the interface design, potential improvements to the tool, and its various application scenarios and potentials. Based on the feedback received, we iteratively refined the tool and reflected on lessons related to designing intuitive interfaces, involving users in the development process, understanding the tool's benefits and limitations, as well as clarifying its scope and positioning. We believe that these lessons can also inform the design and development of other interaction modelling tools in general.