Haopeng Geng is a Ph.D. candidate in Electrical Engineering and Information Systems at the University of Tokyo. His research in Human-Centered Speech AI develops perceptually meaningful acoustic and phonetic models and speech representations for understanding L2, accented, and pathological speech.

Before beginning his Ph.D., he worked full-time as an AI Software Engineer at Laronix, an Australian MedTech start-up, developing practical speech technologies to support communication for the voice-loss community.

📣 News

• [Feb 2026] 🥈 Our UTokyo team placed 2nd in the Iqra’Eval2 Challenge at INTERSPEECH 2026.
  See our prompt-free MDD paper, code, and checkpoints.

• [Aug 2025] We released the full implementation of IF-MDD, our prompt-free framework for mispronunciation detection and diagnosis.

• [Feb 2025] Our paper on perception-based L2 intelligibility was accepted at INTERSPEECH 2025.
  Read the paper and explore the demo of our shadowing-based framework.

📝 Publications

Submitted to SLT 2026

• Subphonetic Acoustic Modeling via Optimal Transport for Pronunciation Assessment
  Double Blind Review.

  • Subphonetic Acoustic Modeling - Expands each phone into ordered internal states, producing dense frame-level acoustic evidence beyond sparse CTC peaks.
  • Optimal Transport Training - Learns monotonic frame-to-state alignments with topology-aware optimal temporal transport, without requiring manual frame labels.
  • Pronunciation Assessment - Provides more precise phone-internal timing and acoustic cues for segmentation, mispronunciation detection, and automatic pronunciation assessment.

INTERSPEECH 2026 (Long Paper)

• Beyond Acoustic Sparsity and Linguistic Bias: A Prompt-Free Paradigm for Mispronunciation Detection and Diagnosis
  Haopeng Geng, Longfei Yang, Xi Chen et al.

  🥈 Ranked 2nd in the Iqra'Eval2 Challenge 2026
  • CROTTC Front-end - Introduces a dense acoustic front-end for capturing fine-grained phonetic deviations beyond the sparse peaks produced by conventional CTC models.
  • Indirect Fusion & Prompt-free Inference - Transfers pronunciation-specific cues into a language model during training, enabling inference without a text prompt.
  • LLM Limitation Analysis - Examines how linguistic priors can override acoustic evidence in fine-grained phonetic recognition.

Preprint, 2026

• IF-MDD: Indirect Fusion for Prompt-free Mispronunciation Detection and Diagnosis
  Haopeng Geng, Daisuke Saito, Nobuaki Minematsu.
  🎧Demo, 💻GitHub Repo
  
  • Prompt-free Mispronunciation Detection - Developed IF-MDD, an indirect fusion framework that leverages canonical phonemes only during training, enabling inference without text prompts.
  • Strong Diagnostic Performance - Achieved 60.67% F1 and 19.98% error diagnosis rate on L2-ARCTIC, showing competitive results even with limited training data.
  • Robust Generalization - Demonstrated reliable performance across unseen speakers from diverse L1 backgrounds, highlighting scalability for real-world CALL applications.

INTERSPEECH 2025

• A Perception-Based L2 Speech Intelligibility Indicator: Leveraging a Rater’s Shadowing and Sequence-to-sequence Voice Conversion
  Haopeng Geng, Daisuke Saito, Nobuaki Minematsu.
  🎧Demo
  • Customized Intelligibility Indicator – Proposed a novel metric leveraging native raters’ shadowing data, focusing on perceptual cues rather than purely native-like pronunciation.
  • Seq2Seq Voice Conversion Framework – Applied alignment and acoustic reconstruction modules to simulate how native listeners detect unintelligible segments.
  • Multi-Task Learning for Feedback – Jointly optimized speech reconstruction and disfluency detection, achieving closer alignment with native raters’ judgments than mainstream ASR and enabling more personalized CALL feedback.

• SlaTE 2025

  
  Synthesizing True Golden Voices to Enhance Pronunciation Training for Individual Language Learners
  Ryoga Yamanaka, Kento Osa, Akari Fujiwara, Haopeng Geng, Daisuke Saito, Nobuaki Minematsu, Yusuke Inoue.

• Preprint, 2025

  
  Simulating Native Speaker Shadowing for Nonnative Speech Assessment with Latent Speech Representations
  Haopeng Geng, Daisuke Saito, Nobuaki Minematsu
  🎧Demo

• APSIPA ASC 2024

  
  A Pilot Study of Applying Sequence-to-Sequence Voice Conversion to Evaluate the Intelligibility of L2 Speech Using a Native Speaker’s Shadowings
  Haopeng Geng, Daisuke Saito, Nobuaki Minematsu.
  🎧Demo, 💻GitHub Repo

• ASJ 2022

  
  Disfluency Removal with Speech Inpainting on Spontaneous Lecture Speech
  Haopeng Geng, YASUDA Yusuke, Tomoki Toda.

📖 Education

• The University of Tokyo, Japan — Ph.D. Candidate in Engineering, Apr 2024–Present
  Supervisor: Prof. Nobuaki Minematsu
  Doctoral research: fine-grained acoustic and phonetic modeling, speech representation learning, and spoken-language assessment.
• Nagoya University, Japan — M.S. in Informatics, Apr 2020–Mar 2022
  Supervisor: Prof. Tomoki Toda
  Thesis: Speech conversion and inpainting for editing disfluencies in spontaneous speech.
• Dalian University of Technology, China — B.S. in Computer Science and Technology; B.A. in Japanese, Sep 2014–Jun 2019

💬 Full-Time Work Experience

💻 Internships & Research Experience

🎖 Honors and Awards

• 2nd Place, Iqra’Eval2 Challenge, INTERSPEECH 2026 · Feb 2026
• Miyabi Supercomputer Resource Grant, The University of Tokyo · Mar 2025–Present
• SPRING GX Fellowship, The University of Tokyo · Apr 2024–Present
• Graduate Program for Real-World Data Circulation Leaders, Nagoya University · Apr 2020–Mar 2022
• Scholarship for Outstanding Undergraduate Students, China Scholarship Council · Sep 2017–Aug 2018