A multidimensional activity theory framework for human computer interaction with Digital Twins

Abstract

This paper introduces a human-centred framework to address the interaction and usability challenges of complex Digital Twin (DT) systems by proposing a new generation of Activity Theory, named Pyramidal Activity Theory (PAT). Digital twins are virtual representations of physical processes, systems, or components that are continuously synchronised with real-world data. Despite their growing adoption across industries, their graphical representation remains challenging due to the diversity of application domains, where multiple models—spanning different users, phases, and scales—are typically distributed across heterogeneous software packages. These complexities often result in inconsistencies, fragmented workflows, and communication barriers across domains. The proposed model builds upon the previous generation of activity theory, extending it into a 3D pyramidal structure while excluding motivational factors to focus exclusively on interaction mechanics. PAT provides a coherent model for designing unified, user-centred interfaces by capturing dynamic interactions among the key elements of DT systems—Users, Tools (Models), Live Data, Interfaces, and Outcomes—and their interrelationships. Two case studies are presented to demonstrate its applicability. Case Study 1 maps human–system interactions in advanced manufacturing, clarifying roles and activity flows to make the framework understandable and directly applicable for practitioners. Case Study 2 validates PAT through an implemented prototype for industrial energy optimisation.

The proposed model introduces a novel 3D interaction paradigm, providing a scalable and adaptable framework for digital twin interface design. It improves usability, standardisation, and decision-making by clarifying stakeholder roles, reducing cognitive load through “black-box” model integration, and ensuring consistent logic from regional planning to unit-level control. The paper concludes with future research directions, including usability testing, interface refinement, and alignment with interoperability and accessibility standards.