Ke-Han Lu

Ke-Han Lu

I’m Ke-Han Lu, a Ph.D. student at National Taiwan University, advised by Prof. Hung-Yi Lee. My research focuses on multimodal language models, particularly on cross-modal alignment and utilizing large language models to enhance multimodal understanding.

News

  • Sep 2026 AKB and SpeechKV is accepted by SLT 2026šŸ‡®šŸ‡¹.
  • May 2026 Our Spoken LLM tutorial is accepted by Interspeech 2026šŸ‡¦šŸ‡ŗ.
  • Apr 2026 I joined Microsoft as a Research Intern in Redmond, working with Jinyu Li.

Selected Publications

For the full publication list, please refer to my Google Scholar page.

  • SLT 2026
    Compress the Cache, Not the Speech Embedding: KV Compression for Efficient Speech LLMs
    Ke-Han Lu, Keqi Deng, Ruchao Fan, Rui Zhao, Jinyu Li
    Paper
  • SLT 2026
    How Auditory Knowledge in LLM Backbones Shapes Audio Language Models: A Holistic Evaluation
    Ke-Han Lu, Szu-Wei Fu, Chao-Han Huck Yang, et al.
    Paper
  • TASLP 2026
    DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
    Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu, et al.
  • Interspeech 2025
    Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
    Ke-Han Lu, Chun-Yi Kuan, Hung-yi Lee
    Paper GitHub
  • ICASSP 2025
    Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
    Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu, et al.
    Paper GitHub
  • Interspeech 2024
    DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
    Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu, et al.
    Paper
  • ICASSP 2024
    Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
    Chien-yu Huang, Ke-Han Lu, Shih-Heng Wang, et al.
    Paper GitHub
  • SLT 2022
    A Context-aware Knowledge Transferring Strategy for CTC-based ASR
    Ke-Han Lu, Kuan-Yu Chen
    Paper GitHub
  • TASLP 2022
    Non-autoregressive ASR Modeling using Pre-trained Language Models for Chinese Speech Recognition
    Fu-Hao Yu, Kuan-Yu Chen, Ke-Han Lu
    Paper
  • CVPR-W 2021
    A Transformer-based Cross-modal Fusion Model with Adversarial Training for VQA Challenge 2021
    Ke-Han Lu, Bo-Han Fang, Kuan-Yu Chen
    Poster spotlight, VQA Workshop @ CVPR 2021

Experience

Microsoft
Research Intern, Microsoft
Redmond, USA
Apr 2026 - Jul 2026
  • Work with Jinyu Li on inference efficiency and zero-shot ability of Speech LLMs.

Education

National Taiwan University
Ph.D. student in Communication Engineering, National Taiwan University
Feb 2024 - Present
  • Supervisor: Prof. Hung-Yi Lee
TaiwanTech
M.S. in Computer Science and Information Engineering, TaiwanTech
Sep 2020 - Feb 2023
  • Supervisor: Prof. Kuan-Yu Chen
TaiwanTech
B.S. in Computer Science and Information Engineering, TaiwanTech
Sep 2016 - Jun 2020

Honors & Awards

  • Industry-Academia Collaborative Doctoral Scholarship, NTU GICE & NVIDIA
  • NVIDIA Academic Grant
  • NSTC Graduate Research Fellowship (NSTC-GRF)
  • 16th TaiwanTech Outstanding Youth Award

Professional Services

  • Program Chair: ROCLing 2025
  • Tutorial Organizer: Interspeech 2026
  • Reviewer: ACML 2025, ASRU 2025, ICASSP 2026, Interspeech 2026, SLT 2026

Skills

  • Programming: Python, PyTorch, Javascript, Latex
  • Software and tools: Linux, Docker, Git, NeMo, Megatron-LM, ESPNET, Huggingface Transformers, fairseq
  • Language: Mandarin(native), English(fluent)

© 2026 Ke-Han Lu.