Hand gesture-based interactive puppetry system to assist storytelling for children

Hui Liang, Jian Chang, Ismail Khalid Kazmi, Jian Jun Zhang, Peifeng Jiao

    Research output: Contribution to journalArticlepeer-review

    184 Downloads (Pure)


    Digital techniques have been used to assist narrative and storytelling, especially in many pedagogical practices. With the rapid development of HCI techniques, saturated with digital media in their daily lives, young children, demands more interactive learning methods and meaningful immersive learning experiences. In this paper, we propose a novel hand gesture-based puppetry storytelling system which provides a more intuitive and natural human computer interaction method for young children to develop narrative ability in virtual story world. Depth motion sensing and hand gestures control technology is utilized in the implementation of user-friendly interaction. Young players could intuitively use hand gestures to manipulate virtual puppet to perform story and interact with different items in virtual environment to assist narration. Based on the result of the evaluation, this novel digital storytelling system shows positive pedagogical functions on children’s narrating ability as well as the competencies of cognitive and motor coordination. The usability of the system is preliminary examined in our test, and the results which showed that young children can benefit from playing with Puppet Narrator.
    Original languageEnglish
    Pages (from-to)517-531
    JournalThe Visual Computer
    Issue number4
    Early online date9 Jun 2016
    Publication statusPublished - Apr 2017


    Dive into the research topics of 'Hand gesture-based interactive puppetry system to assist storytelling for children'. Together they form a unique fingerprint.

    Cite this