HOME Project 1 Project 2

Niklas Krome

Social Cognitive Systems at Bielefeld University

Home Page

I am a PhD student in the Social Cognitive Systems Group at Bielefeld University headed by Prof. Dr.-Ing. Stefan Kopp.

As part of the HiAvA-Project, I am developing AI-assisted methods to drive gesticulation on virtual agents in a multimodal XR-Cyberspace. My research interests lie in the analysis of human behaviour and its simulation using modern machine learning techniques. My goal is to build an adaptive system, that is able to generate appropriate co-speech gestures in realtime to be deployed in an avatar-based face-to-face interaction scenario, using the modalities available on different platforms.

News

January 2024: Our paper titled "Minimal Latency Speech-Driven Gesture Generation for Continuous Interaction in Social XR won the Best Poster Award at AIxVR 2024
January 2024: I presented our paper titled "Minimal Latency Speech-Driven Gesture Generation for Continuous Interaction in Social XR" virtually at AIxVR 2024 in Los Angeles, USA
December 2023: Our paper "Minimal Latency Speech-Driven Gesture Generation for Continuous Interaction in Social XR" was accepted at AIxVR 2024
September 2023: I presented our paper titled "Towards Real-time Co-speech Gesture Generation in Online Interaction in Social XR" at IVA 2023 in Würzburg, Germany
June 2023: Our paper "Towards Real-time Co-speech Gesture Generation in Online Interaction in Social XR" was accepted at IVA 2023
April 2023: I presented my work during a poster session on MMSYM 2023

Publications

Niklas Krome, Stefan Kopp. (2024). Minimal Latency Speech-Driven Gesture Generation for Continuous Interaction in Social XR [Paper] [Video] [Project Page]
Niklas Krome, Stefan Kopp. (2023). Towards Real-time Co-speech Gesture Generation in Online Interaction in Social XR [Paper] [Video] [Project Page]