Data-driven Simulation Framework for Expressive Piano Playing by Anthropomorphic Hand with Variable Passive Properties

Abstract

The expressive piano playing generally requires a delicate control over the keystroke by considering passive dynamics of the hand and piano keyboard. It becomes even more challenging when the passive properties change during the performance. In this paper, we develop a framework for expressive piano playing that simulates the interaction between a soft anthropomorphic hand with variable passive properties and a piano keyboard. The passive dynamics of each system component in our framework are identified using the data collected during real piano playing, which in turn, helps to reduce the sim-to-real gap. To evaluate the proposed framework, we have designed and fabricated a three-fingered soft-rigid hybrid hand system with variable stiffness in each finger, enabling diverse compliant behaviors for piano key pressing. The results show that our proposed simulator achieves less than 5% error rate in coordination with real-world piano playing.