Boosting Large Language Model for Speech Synthesis: An Empirical Study
Hongkun Hao, Long Zhou, Shujie Liu, Jinyu Li, Shujie Hu, Rui Wang, Furu Wei
ICASSP | April 2025
Jinyu Li
Partner Applied Science Manager
Publications
Publications by Year
-
Target word activity detector: An approach to obtain ASR word boundaries without lexicon
Sunit Sivasankaran, Eric Sun, Jinyu Li, Yan Huang, Jing Pan
ICASSP | April 2025
V2SFlow: Video-to-Speech Generation with Speech Decomposition and Rectified Flow
Jeongsoo Choi, Ji-Hoon Kim, Jinyu Li, Joon Son Chung, Shujie Liu
ICASSP | April 2025
-
CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation
Rui Zhao, Jinyu Li, Ruchao Fan, Matt Post
SLT Workshop | December 2024
Investigating neural audio codecs for speech language model-based speech generation
Jiaqi Li , Dongmei Wang, Xiaofei Wang, Yao Qian, Long Zhou, Shujie Liu, Midia Yousefi, Canrun Li , Chung-Hsien Tsai, Zhen Xiao, Yanqing Liu, Junkun Chen, Sheng Zhao, Jinyu Li, Zhizheng Wu , Michael Zeng
SLT Workshop | December 2024
Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech
Haibin Wu, Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Daniel Tompkins, Chung-Hsien Tsai , Canrun Li, Zhen Xiao, Sheng Zhao , Jinyu Li, Naoyuki Kanda
SLT Workshop | December 2024
ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation
Zongyi Li, Shujie Hu, Shujie Liu, Long Zhou, Jeongsoo Choi, Lingwei Meng, Xun Guo, Jinyu Li, Hefei Ling, Furu Wei
October 2024
October 2024
An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS
Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Hemin Yang, Zirun Zhu, Min Tang, Yufei Xia, Jinzhu Li, sheng zhao , Jinyu Li, Naoyuki Kanda
Interspeech 2024 | September 2024
COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning
Jing Pan, Jian Wu, Yashesh Gaur , Sunit Sivasankaran, Zhuo Chen , Shujie Liu, Jinyu Li
Interspeech 2024 | September 2024
Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translation
Peidong Wang, Jian Xue, Jinyu Li, Junkun Chen, Aswin Shanmugam Subramanian
Interspeech 2024 | September 2024
Total-Duration-Aware Duration Modeling for Text-to-Speech Systems
Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Chung-Hsien Tsai, Canrun Li, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Jinyu Li, Sheng Zhao, Naoyuki Kanda
Interspeech 2024 | September 2024
Autoregressive Speech Synthesis without Vector Quantization
Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen Meng, Furu Wei
July 2024
VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment
Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, Furu Wei
June 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei
June 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei
June 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiang-Yang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, Sheng Zhao
ICML | June 2024
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
Chenyang Le, Yao Qian, Dongmei Wang, Long Zhou, Shujie Liu, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Sheng Zhao, Michael Zeng
NeurIPS 2024 | May 2024
End-to-End Automatic Speech Recognition
MSR-TR-2024-30 | May 2024
Published by Microsoft
IEEE SPS Webinar
CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations
Leying Zhang, Yao Qian, Long Zhou, Shujie Liu (shujliu), Dongmei Wang, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Lei He, Sheng Zhao, Michael Zeng
NeurIPS 2024 | April 2024
DiariST: Streaming Speech Translation with Speaker Diarization
Mu Yang, Naoyuki Kanda, Xiaofei Wang, Junkun Chen, Peidong Wang, Jian Xue, Jinyu Li, Takuya Yoshioka
ICASSP | April 2024
Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation
Sara Pap, Peidong Wang, Junkun Chen, Jian Xue, Naoyuki Kanda, Jinyu Li, Yashesh Gaur
ICASSP | April 2024
ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
Yiming Wang, Jinyu Li
ICASSP | April 2024
t-SOT FNT: streaming multi-talker ASR with text-only domain adaptation capability
Jian Wu, Naoyuki Kanda, Takuya Yoshioka, Rui Zhao, Zhuo Chen, Jinyu Li
ICASSP | April 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model
Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, Furu Wei
March 2024
SpeechLM: Enhanced Speech Pre-Training With Unpaired Textual Data
Ziqiang Zhang, Sanyuan Chen, Long Zhou, Yu Wu, Shuo Ren, Shujie Liu, Zhuoyuan Yao, Xun Gong, Lirong Dai, Jinyu Li, Furu Wei
IEEE/ACM Transactions on Audio, Speech, and Language Processing | March 2024
Advanced Long-Content Speech Recognition with Factorized Neural Transducer
Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian
IEEE/ACM Transactions on Audio, Speech, and Language Processing | January 2024
-
A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capability
Jian Xue, Peidong Wang, Jinyu Li, Eric Sun
Workshop of Automatic Speech Recognition and Understanding | December 2023
Building High-accuracy Multilingual ASR with Gated Language Experts and Curriculum Training
Eric Sun, Jinyu Li, Yuxuan Hu, Yimeng Zhu, Long Zhou, Jian Xue, Peidong Wang, Linquan Liu, Shujie Liu, Edward Lin, Yifan Gong
December 2023
Decoder-only Modeling for Speech
December 2023
Invited Talk at NTU and SJTU
Improving Stability in Simultaneous Speech Translation: A Revision-Controllable Decoding Approach
Junkun Chen, Jian Xue, Peidong Wang, Jing Pan, Jinyu Li
Workshop of Automatic Speech Recognition and Understanding | December 2023
Large-Scale Streaming End-to-End Speech Translation
December 2023
Invited Talk at NTU and SJTU
On decoder-only architecture for speech-to-text and large language model integration
Jian Wu, Yashesh Gaur, Zhuo Chen, Long Zhou, Yimeng Zhu, Tianrui Wang, Jinyu Li, Shujie Liu, Bo Ren, Linquan Liu, Yu Wu
Workshop of Automatic Speech Recognition and Understanding | December 2023
Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition
Yuang Li, Yu Wu, Jinyu Li, Shujie Liu
Workshop of Automatic Speech Recognition and Understanding | December 2023
Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments
Sara Papi, Peidong Wang, Junkun Chen, Jian Xue, Jinyu Li, Yashesh Gaur
Workshop of Automatic Speech Recognition and Understanding | December 2023
Recent Advances in End-to-End Automatic Speech Recognition (Invited Talk at ASC 2023)
| November 2023
Published by Microsoft
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
Xiaofei Wang, Manthan Thakker, Zhuo Chen, Naoyuki Kanda, Sefik Emre Eskimez, Sanyuan Chen, Min Tang, Shujie Liu, Jinyu Li, Takuya Yoshioka
August 2023
Accelerating Transducers through Adjacent Token Merging
Yuang Li, Yu Wu, Jinyu Li, Shujie Liu
Interspeech | August 2023
LAMASSU: A Streaming Language-Agnostic Multilingual Speech Recognition and Translation Model Using Neural Transducers
Peidong Wang, Eric Sun, Jian Xue, Yu Wu, Long Zhou, Yashesh Gaur, Shujie Liu, Jinyu Li
Interspeech | August 2023
CTCBERT: Advancing Hidden-unit BERT with CTC Objectives
Ruchao Fan, Yiming Wang, Yashesh Gaur, Jinyu Li
ICASSP | June 2023
Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition models
Rui Zhao, Jian Xue, Partha Parthasarathy, Veljko Miljanic, Jinyu Li
ICASSP | June 2023
Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentation
Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Sheng zhao
ICASSP | June 2023
Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation
Kun Wei, Long Zhou, Ziqiang Zhang, Liping Chen, Shujie Liu, Lei He, Jinyu Li, Furu Wei
ICASSP | June 2023
LongFNT: Long-form Speech Recognition with Factorized Neural Transducer
Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian
ICASSP | June 2023
Self-supervised learning with bi-label masked speech prediction for streaming multi-talker speech recognition
Zili Huang, Zhuo Chen, Naoyuki Kanda, Jian Wu, Yiming Wang, Jinyu Li, Takuya Yoshioka, Xiaofei Wang, Peidong Wang
ICASSP | June 2023
Simulating realistic speech overlaps improves multi-talker ASR
Muqiao Yang, Naoyuki Kanda, Xiaofei Wang, Jian Wu, Sunit Sivasankaran, Zhuo Chen, Jinyu Li, Takuya Yoshioka
ICASSP | June 2023
Speaker Change Detection for Transformer Transducer ASR
Jian Wu, Zhuo Chen, Min Hu, Xiong Xiao, Jinyu Li
June 2023
Speech separation with large-scale self-supervised learning
Zhuo Chen, Naoyuki Kanda, Jian Wu, Yu Wu, Xiaofei Wang, Takuya Yoshioka, Jinyu Li, Sunit Sivasankaran, Sefik Emre Eskimez
ICASSP | June 2023
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
ICASSP | June 2023
VatLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning
Qiushi Zhu, Long Zhou, Ziqiang Zhang, Shujie Liu, Binxing Jiao, Jie Zhang, Lirong Dai, Daxin Jiang, Jinyu Li, Furu Wei
IEEE Transactions on Multimedia | May 2023
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei
March 2023
STREAMING, FAST AND ACCURATE ON-DEVICE INVERSE TEXT NORMALIZATION FOR AUTOMATIC SPEECH RECOGNITION
Yashesh Gaur, Nick Kibre, Jian Xue, Kangyuan Shu, Yuhui Wang, Issac Alphanso, Jinyu Li, Yifan Gong
Spoken Language Technology Workshop | January 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei
January 2023
-
Advancing end-to-end automatic speech recognition and beyond
International Symposium on Chinese Spoken Language Processing | December 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
Ziqiang Zhang, Long Zhou, Junyi Ao, Shujie Liu, Lirong Dai, Jinyu Li, Furu Wei
EMNLP 2022 | October 2022
Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Naoyuki Kanda, Jinyu Li, Xie Chen, Yu Wu, Yifan Gong
Interspeech 2022 | September 2022
Large-Scale Streaming End-to-End Speech Translation with Neural Transducers
Jian Xue, Peidong Wang, Jinyu Li, Matt Post, Yashesh Gaur
Interspeech | September 2022
Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Data
Junyi Ao, Ziqiang Zhang, Long Zhou, Shujie Liu, Haizhou Li, Tom Ko, Lirong Dai, Jinyu Li, Yao Qian, Furu Wei
Interspeech | September 2022
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Wangyou Zhang, Zhuo Chen, Naoyuki Kanda, Shujie Liu, Jinyu Li, Sefik Emre Eskimez, Takuya Yoshioka, Xiong Xiao, Zhong Meng, Yanmin Qian, Furu Wei
Interspeech 2022 | September 2022
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training
Chengyi Wang, Yiming Wang, Yu Wu, Sanyuan Chen, Jinyu Li, Shujie Liu, Furu Wei
Interspeech | September 2022
Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Zhuo Chen, Gang Liu, Jinyu Li, Jian Wu, Xiangzhan Yu, Furu Wei
Interspeech | September 2022
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
arXiv:2209.04974 | September 2022
Towards Contextual Spelling Correction for Customization of End-to-End Speech Recognition Systems
Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Veljko Miljanic, Sheng Zhao, Hosam Khalil
IEEE/ACM Transactions on Audio, Speech, and Language Processing | September 2022, Vol 30: pp. 3089-3097
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, Furu Wei
IEEE Journal of Selected Topics in Signal Processing | July 2022, Vol 16(6): pp. 1505-1518
A Configurable Multilingual Model is All You Need to Recognize All Languages
Long Zhou, Jinyu Li, Eric Sun, Shujie Liu
ICASSP 2022 | May 2022
Continuous Streaming Multi-Talker ASR with Dual-path Transducers
Desh Raj, Liang Lu, Zhuo Chen, Yashesh Gaur, Jinyu Li
ICASSP 2022 | May 2022
Endpoint Detection for Streaming End-to-End Multi-talker ASR
Liang Lu, Jinyu Li, Yifan Gong
ICASSP 2022 | May 2022
Factorized Neural Transducer for Efficient Language Model Adaptation
Xie Chen, Zhong Meng, Sarangarajan Parthasarathy, Jinyu Li
ICASSP 2022 | May 2022
Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition
Guoli Ye, Vadim Mazalov, Jinyu Li, Yifan Gong
ICASSP 2022 | May 2022
Self-Supervised Learning for speech recognition with Intermediate layer supervision
Chengyi Wang, Yu Wu, Sanyuan Chen, Shujie Liu, Jinyu Li, Yao Qian, Zhenglu Yang
ICASSP 2022 | May 2022
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu
ICASSP | May 2022
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu
ICASSP 2022 | May 2022
Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition
Yiming Wang, Jinyu Li, Heming Wang, Yao Qian, Chengyi Wang, Yu Wu
ICASSP 2022 | May 2022
Continuous Speech Separation with Recurrent Selective Attention Network
Yixuan Zhang, Zhuo Chen, Jian Wu, Takuya Yoshioka, Peidong Wang, Zhong Meng, Jinyu Li
ICASSP 2021 | May 2022
Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction
Heming Wang, Yao Qian, Xiaofei Wang, Yiming Wang, Chengyi Wang, Shujie Liu, Takuya Yoshioka, Jinyu Li, DeLiang Wang
ICASSP 2022 | May 2022
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
ACL 2022 | May 2022
Recent Advances in End-to-End Automatic Speech Recognition
APSIPA Transactions on Signal and Information Processing | April 2022
-
On Addressing Practical Challenges For RNN-Transducer
Rui Zhao, Jian Xue, Jinyu Li, Wenning Wei, Lei He, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2021
Advancing end-to-end automatic speech recognition
MSR-TR-2021-32 | October 2021
Published by Microsoft
Keynote talk at the conference on Computational Linguistics and Speech Processing, 2021.
A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems
Xiaoqiang Wang, Yanqing Liu, Sheng Zhao, Jinyu Li
Interspeech 2021 | August 2021
Improving Multilingual Transformer Transducer Models by Reducing Language Confusions
Eric Sun, Jinyu Li, Zhong Meng, Yu Wu, Jian Xue, Shujie Liu, Yifan Gong
Interspeech 2021 | August 2021
Improving RNN-T for Domain Scaling Using Semi-Supervised Training with Neural TTS
Yan Deng, Rui Zhao, Zhong Meng, Xie Chen, Bing Liu, Jinyu Li, Yifan Gong, Lei He
Interspeech 2021 | August 2021
Investigation of Practical Aspects of Single Channel Speech Separation for ASR
Jian Wu, Zhuo Chen, Sanyuan Chen, Yu Wu, Takuya Yoshioka, Naoyuki Kanda, Shujie Liu, Jinyu Li
Interspeech 2021 | August 2021
Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
Zhong Meng, Yu Wu, Naoyuki Kanda, Liang Lu, Xie Chen, Guoli Ye, Eric Sun, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Multiple Softmax Architecture for Streaming Multilingual End-to-End ASR Systems
Vikas Joshi, Amit Das, Eric Sun, Rupesh Mehta, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer
Liang Lu, Zhong Meng, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Rapid Speaker Adaptation for Conformer Transducer: Attention and Bias are All You Need
Yan Huang, Guoli Ye, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Streaming Multi-talker Speech Recognition with Joint Speaker Identification
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Ultra Fast Speech Separation Model with Teacher Student Learning
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Takuya Yoshioka, Shujie Liu, Jinyu Li, Xiangzhan Yu
Interspeech 2021 | August 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Naoyuki Kanda, Yashesh Gaur, Sarangarajan Parthasarathy, Eric Sun, Liang Lu, Xie Chen, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020
Xiong Xiao, Naoyuki Kanda, Zhuo Chen, Tianyan Zhou, Takuya Yoshioka, Sanyuan Chen, Yong Zhao, Gang Liu, Yu Wu, Jian Wu, Shujie Liu, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Continuous speech separation with Conformer
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Jinyu Li, Takuya Yoshioka, Chengyi Wang, Shujie Liu, Ming Zhou
ICASSP | June 2021
Developing real-time streaming transformer transducer for speech recognition on large-scale dataset
Xie Chen, Yu Wu, Zhenghao Wang, Shujie Liu, Jinyu Li
ICASSP | June 2021
Don’t shoot butterfly with rifles: multi-channel continuous speech separation with early exit transformer
Sanyuan Chen, Yu Wu, Zhuo Chen, Takuya Yoshioka, Shujie Liu, Jinyu Li, Xiangzhan Yu
ICASSP | June 2021
Streaming end-to-end multi-talker speech recognition
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
IEEE Signal Processing Letter | April 2021, Vol 28
Adaptation Algorithms for Speech Recognition: An Overview
Peter Bell, Joachim Fainberg, Ondrej Klejch, Jinyu Li, Steve Renals, Pawel Swietojanski
IEEE Open Journal of Signal Processing | February 2021
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen, Hakan Erdogan, Zili Huang, Maokui He, Shinji Watanabe, Jun Du, Takuya Yoshioka, Yi Luo, Naoyuki Kanda, Jinyu Li, Scott Wisdom, John R. Hershey
SLT 2021 | January 2021
Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, Yifan Gong
SLT 2021 | January 2021
Dual-Path RNN For Long Recording Speech Separation
Chenda Li, Yi Luo, Cong Han, Jinyu Li, Takuya Yoshioka, Tianyan Zhou, Marc Delcroix, Keisuke Kinoshita, Christoph Boeddeker, Yanmin Qian, Shinji Watanabe, Zhuo Chen
Spoken Language Technology Workshop | January 2021
Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations
Shahram Ghorbani, Yashesh Gaur, Yu Shi, Jinyu Li
Spoken Language Technology Workshop | January 2021
-
Speaker Separation Using Speaker Inventories and Estimated Speech
Peidong Wang, Zhuo Chen, DeLiang Wang, Jinyu Li, Yifan Gong
IEEE Transactions on Audio, Speech, and Language Processing | December 2020, Vol 29: pp. 537-546
Developing streaming end-to-end models for automatic speech recognition in industry
MSR-TR-2020-45 | December 2020
Published by Microsoft
Sequence-Level Self-Learning with Multiple Hypotheses
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
Interspeech | October 2020
Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability
Jinyu Li, Rui Zhao, Zhong Meng, Yanqing Liu, Wenning Wei, Sarangarajan Parthasarathy, Vadim Mazalov, Zhenghao Wang, Lei He, Sheng Zhao, Yifan Gong
Interspeech | October 2020
On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Jinyu Li, Yu Wu, Yashesh Gaur, Chengyi Wang, Rui Zhao , Shujie Liu
Interspeech | October 2020
Sequence-level self-learning with multi-task learning framework
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
October 2020
An End-to-end Architecture of Online Multi-channel Speech Separation
Jian Wu, Zhuo Chen, Jinyu Li, Takuya Yoshioka, Zhili Tan, Ed Lin, Yi Luo , Lei Xie
Interspeech | October 2020
Combination of end-to-end and hybrid models for speech recognition
Jeremy Wong, Yashesh Gaur, Rui Zhao, Liang Lu, Eric Sun, Jinyu Li, Yifan Gong
Interspeech | October 2020
Exploring Transformers for Large-Scale Speech Recognition
Liang Lu, Changliang Liu, Jinyu Li, Yifan Gong
Interspeech | October 2020
Low Latency End-to-End Streaming Speech Recognition with a Scout Network
Chengyi Wang, Yu Wu, Liang Lu, Shujie Liu, Jinyu Li, Guoli Ye, Ming Zhou
Interspeech | October 2020
Rapid RNN-T Adaptation Using Personalized Speech Synthesis and Neural Language Generator
Yan Huang, Jinyu Li, Lei He, Wenning Wei, William Gale, Yifan Gong
Interspeech | October 2020
Semantic Mask for Transformer based End-to-End Speech Recognition
Chengyi Wang, Yu Wu, yujiao du, Jinyu Li, Shujie Liu, Liang Lu, Shuo Ren, Guoli Ye, Sheng Zhao, Ming Zhou
Interspeech | October 2020
Transfer Learning Approaches for Streaming End-to-End Speech Recognition System
Vikas Joshi, Rui Zhao, Rupesh R. Mehta, Kshitiz Kumar, Jinyu Li
Interspeech | October 2020
High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model
Jinyu Li, Rui Zhao, Eric Sun, Jeremy Wong, Amit Das, Zhong Meng, Yifan Gong
ICASSP | April 2020
Continuous speech separation: dataset and analysis
Zhuo Chen, Takuya Yoshioka, Liang Lu, Tianyan Zhou, Zhong Meng, Yi Luo, Jian Wu, Xiong Xiao, Jinyu Li
ICASSP | April 2020
Exploring pre-training with alignments for RNN transducer based end-to-end speech recognition
Hu Hu, Rui Zhao, Jinyu Li, Liang Lu, Yifan Gong
ICASSP | April 2020
L-Vector: Neural Label Embedding for Domain Adaptation
Zhong Meng, Hu Hu, Jinyu Li, Changliang Liu, Yan Huang, Yifan Gong, Chin-Hui Lee
ICASSP | April 2020
Minimum latency training strategies for streaming sequence-to-sequence ASR
Hirofumi Inaguma, Yashesh Gaur, Liang Lu, Jinyu Li, Yifan Gong
ICASSP | April 2020
Recent Progresses On Deep Learning For Speech Recognition
Jinyu Li, Dong Yu
Handbook of Pattern Recognition and Computer Vision: 6th Edition | 2020
Using personalized speech synthesis and neural language generator for rapid speaker adaptation
Yan Huang, Lei He, Wenning Wei, William Gale, Jinyu Li, Yifan Gong
ICASSP | April 2020
-
Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
Amit Das, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong
IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING | December 2019, pp. 1880-1892
Character-aware attention-based end-to-end speech recognition
Zhong Meng, Yashesh Gaur, Jinyu Li, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
CNN with phonetic attention for text-independent speaker verification
Tianyan Zhou, yong zhao, Jinyu Li, Yifan Gong, Jian Wu
Automatic Speech Recognition and Understanding Workshop | December 2019
Domain adaptation via teacher-student learning for end-to-end speech recognition
Zhong Meng, Jinyu Li, Yashesh Gaur, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Improving RNN Transducer Modeling for End-to-End Speech Recognition
Jinyu Li, Rui Zhao, Hu Hu, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Speech Separation Using Speaker Inventory
Peidong Wang, Zhuo Chen, Xiong Xiao, Zhong Meng, Takuya Yoshioka, Tianyan Zhou, Liang Lu, Jinyu Li
Automatic Speech Recognition and Understanding Workshop | November 2019
Acoustic-to-Phrase Models for Speech Recognition
Yashesh Gaur, Jinyu Li, Zhong Meng, Yifan Gong
Interspeech | September 2019
Layer Trajectory BLSTM
Eric Sun, Jinyu Li, Yifan Gong
Interspeech | September 2019
Speaker Adaptation for Attention-Based End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Jinyu Li, Yifan Gong
Interspeech | September 2019
Adversarial Speaker Adaptation
Zhong Meng, Jinyu Li, Yifan Gong
ICASSP | May 2019
Adversarial Speaker Verification
Zhong Meng, Yong Zhao, Jinyu Li, Yifan Gong
ICASSP | May 2019
Attentive Adversarial Learning for Domain-Invariant Training
Zhong Meng, Jinyu Li, Yifan Gong
ICASSP | May 2019
Conditional Teacher-Student Learning
Zhong Meng, Jinyu Li, Yong Zhao, Yifan Gong
ICASSP | May 2019
Improving Layer Trajectory LSTM With Future Context Frames
Jinyu Li, Liang Lu, Changliang Liu, Yifan Gong
ICASSP | May 2019
Towards Code-Switching ASR for End-to-End CTC Models
Ke Li, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong
ICASSP | May 2019
Universal Acoustic Modeling Using Neural Mixture Models
Amit Das, Jinyu Li, Changliang Liu, Yifan Gong
ICASSP | May 2019
-
Multi-Channel Overlapped Speech Recognition with Location Guided Speech Extraction Network
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
SLT 2018 | December 2018
Exploring layer trajectory LSTM with depth processing units and attention
Jinyu Li, Liang Lu, Changliang Liu, Yifan Gong
Spoken Language Technology | December 2018
MULTI-CHANNEL OVERLAPPED SPEECH RECOGNITION WITH LOCATION GUIDED SPEECH EXTRACTION NETWORK
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
Spoken Language Technology | December 2018
Speaker Adaptation for End-to-End CTC Models
Ke Li, Jinyu Li, yong zhao, Yifan Gong (ygong), Kshitiz Kumar
Spoken Language Technology | December 2018
Adversarial Feature-Mapping for Speech Enhancement
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang Juang
Interspeech 2018 | September 2018
Cycle-Consistent Speech Enhancement
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang Juang
Interspeech 2018 | September 2018
Improved training for online end-to-end speech recognition systems
Suyoun Kim, Michael L. Seltzer, Jinyu Li, Rui Zhao
Interspeech 2018 | September 2018
Layer Trajectory LSTM
Jinyu Li, Changliang Liu, Yifan Gong
Interspeech 2018 | September 2018
Advancing Acoustic-to-Word CTC Model
Jinyu Li, Guoli Ye, Amit Das, Rui Zhao, Yifan Gong
ICASSP | April 2018
Advancing Connectionist Temporal Classification With Attention Modeling
Amit Das, Jinyu Li, Rui Zhao, Yifan Gong
ICASSP | April 2018
Developing Far-Field Speaker System Via Teacher-Student Learning
Jinyu Li, Rui Zhao, Zhuo Chen, Changliang Liu, Xiong Xiao, Guoli Ye, Yifan Gong
ICASSP | April 2018
Efficient Integration of Fixed Beamformers and Speech Separation Networks for Multi-Channel Far-Field Speech Separation
Zhuo Chen, Takuya Yoshioka, Xiong Xiao, Jinyu Li, Michael L. Seltzer, Yifan Gong
ICASSP 2018 | April 2018
Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang (Fred) Juang
ICASSP | April 2018
DOMAIN AND SPEAKER ADAPTATION FOR CORTANA SPEECH RECOGNITION
Yong Zhao, Jinyu Li, Shixiong Zhang, Liping Chen, Yifan Gong
ICASSP | April 2018
Efficient integration of fixed beamformers and speech separation networks for multi-channel far-field speech separation
Zhuo Chen, Takuya Yoshioka, Xiong Xiao, Jinyu Li, Michael L. Seltzer, Yifan Gong
ICASSP | April 2018
Speaker-Invariant Training via Adversarial Learning
Zhong Meng, Jinyu Li, Zhuo Chen, Yong Zhao, Vadim Mazalov, Yifan Gong, Biing-Hwang (Fred) Juang
ICASSP | April 2018
Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition
Zhehuai Chen, Jasha Droppo, Jinyu Li, Wayne Xiong
IEEE/ACM Transactions on Audio, Speech, and Language Processing | January 2018
-
Acoustic-To-Word Model Without OOV
Jinyu Li, Guoli Ye, Rui Zhao, Jasha Droppo, Yifan Gong
ASRU | December 2017
Cracking the cocktail party problem by multi-beam deep attractor network
Zhuo Chen, Jinyu Li, Xiong Xiao, Takuya Yoshioka, Huaming Wang, Zhenghao Wang, Yifan Gong
ASRU 2018 | December 2017
Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
Zhong Meng, Zhuo Chen, Vadim Mazalov, Jinyu Li, Yifan Gong
ASRU | December 2017
Deep Learning Acoustic Model in Microsoft Cortana Voice Assistant
MSR-TR-2017-60 | November 2017
Published by Microsoft
Improving mask learning based speech enhancement system with restoration layers and residual connection
Zhuo Chen, Yan Huang, Jinyu Li, Yifan Gong
Interspeech | August 2017
Large-Scale Domain Adaptation via Teacher-Student Learning
Jinyu Li, Michael Seltzer, Xi Wang, Rui Zhao, Yifan Gong
Interspeech | August 2017
Challenges and Solutions to Deep Learning Network Acoustic Modeling in Speech Recognition Products at Microsoft
New Era for Robust Speech Recognitino: Exploiting Deep Learning | Published by Springer | 2017
Recent Progresses in Deep Learning Based Acoustic Models
Dong Yu, Jinyu Li
IEEE/CAA JOURNAL OF AUTOMATICA SINICA | July 2017
Extended low-rank plus diagonal adaptation for deep and recurrent neural networks
yong zhao, Jinyu Li, kshitiz kumar, Yifan Gong
ICASSP | April 2017
Improved Cepstra Minimum-Mean-Square-Error Noise Reduction Algorithm For Robust Speech Recognition
Jinyu Li, Yan Huang, Yifan Gong
ICASSP | April 2017
-
End-to-End Attention Based Text-Dependent Speaker Verification
Shixiong Zhang, Zhuo Chen, Yong Zhao, Jinyu Li, Yifan Gong
SLT workshop | December 2016
Deep Convolutional Neural Networks with Layer-wise Context Expansion and Attention
Dong Yu, Wayne Xiong, Jasha Droppo, Andreas Stolcke, Guoli Ye, Jinyu Li, Geoffrey Zweig
Proc. Interspeech | September 2016
EXPLORING MULTIDIMENSIONAL LSTMS FOR LARGE VOCABULARY ASR
Jinyu Li, Abdelrahman Mohamed, Geoffrey Zweig, Yifan Gong
ICASSP | March 2016
Low-rank Plus Diagonal Adaption for Deep Neural Networks
Yong Zhao, Jinyu Li, Yifan Gong
ICASSP | March 2016
ICASSP
Recurrent Support Vector Machines for Speech Recognition
Shi-Xiong Zhang, Rui Zhao, Chaojun Liu, Jinyu Li, Yifan Gong
ICASSP | March 2016
ICASSP
Simplifying Long Short-term Memory Acoustic Models for Fast Training and Decoding
Yajie Miao, Jinyu Li, Yongqiang Wang, Shi-Xiong Zhang, Yifan Gong
ICASSP | March 2016
ICASSP
-
LSTM Time and Frequency Recurrence for Automatic Speech Recognition
Jinyu Li, Abdelrahman Mohamed, Geoffrey Zweig, Yifan Gong
IEEE Automatic Speech Recognition and Understanding Workshop | December 2015
Compensation with Prior Knowledge
Jinyu Li, Li Deng, Yifan Gong
Robust Automatic Speech Recognition: A Bridge to Practical Applications | Published by Elsevier | 2015 | Robust Automatic Speech Recognition: A Bridge to Practical Applications edition
Robust Automatic Speech Recognition – A Bridge to Practical Applications (1st Edition)
Jinyu Li, Li Deng, R. Haeb-Umbach, Y. Gong
Published by Elsevier | October 2015
Maximum a Posteriori Adaptation of Network Parameters in Deep Models
Zheng Huang, Sabato Marco Siniscalchi, I-Fan Chen, Jinyu Li, Jiadong Wu, Chin-Hui Lee
September 2015
Rapid Adaptation for Deep Neural Networks through Multi-Task Learning
Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Ji Wu, Chin-Hui Lee
Interspeech | September 2015
SVD-based Universal DNN Modeling for Multiple Scenarios
Changliang Liu, Jinyu Li, Yifan Gong
September 2015
An Analysis of Convolutional Neural Networks for Speech Recognition
Jui-Ting Huang, Jinyu Li, Yifan Gong
ICASSP | April 2015
Investigating Online Low-Footprint Speaker Adaptation Using Generalized Linear Regression and Click-Through Data
Yong Zhao, Jinyu Li, Jian Xue, Yifan Gong
ICASSP | April 2015
ICASSP
Small-footprint high-performance deep neural network-based speech recognition using split-VQ
Yongqiang Wang, Jinyu Li, Yifan Gong
ICASSP | April 2015
-
variable-activation and variable-input deep neural network for robust speech recognition
Rui Zhao, Jinyu Li, Yifan Gong
IEEE SLT workshop | December 2014
Beyond Cross-entropy: Towards Better Frame-level Objective Functions For Deep Neural Network Training In Automatic Speech Recognition
Zhen Huang, Jinyu Li, Chao Weng, Chin-Hui Lee
Interspeech | September 2014
Interspeech
Deep Neural Network for Automatic Speech Recognition: from the Industry’s View
September 2014
Feature Space Maximum A Posteriori Linear Regression for Adaptation of Deep Neural Networks
Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Chao Weng, Chin-Hui Lee
Interspeech | September 2014
Interspeech
Learning Small-Size DNN with Output-Distribution-Based Criteria
Jinyu Li, Rui Zhao, Jui-Ting Huang, Yifan Gong
Interspeech | September 2014
Interspeech
Variable-Component Deep Neural Network for Robust Speech Recognition
Interspeech | September 2014
Interspeech
An Overview of Noise-Robust Automatic Speech Recognition
Jinyu Li, Li Deng, Yifan Gong, Reinhold Haeb-Umbach
IEEE/ACM Transactions on Audio, Speech, and Language Processing | April 2014, Vol 22: pp. 745
Factorized adaptation for deep neural network
Jinyu Li, Jui-Ting Huang, Yifan Gong
ICASSP | January 2014
Feature Compensation Using Linear Combination of Speaker and Environment Dependent Correction Vectors
Xiong Xiao, Jinyu Li, Eng Siong Chng, Haizhou Li
ICASSP | January 2014
Investigation of Maxout Networks For Speech Recognition
Pawel Swietojanski, Jinyu Li, Jui-Ting Huang
ICASSP | January 2014
Singular Value Decomposition Based Low-footprint Speaker Adaptation and Personalization for Deep Neural Network
Jian Xue, Jinyu Li, Dong Yu, Mike Seltzer, Yifan Gong
ICASSP | January 2014
-
Cross-language Knowledge Transfer using Multilingual Deep Neural Network with Shared Hidden Layers
Jui-Ting Huang, Jinyu Li, Dong Yu, Li Deng, Yifan Gong
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) | May 2013
Feature Learning in Deep Neural Networks – Studies on Speech Recognition
Dong Yu, Mike Seltzer, Jinyu Li, Jui-Ting Huang, Frank Seide
International Conference on Learning Representations | May 2013
Recent Advances in Deep Learning for Speech Research at Microsoft
Li Deng, Jinyu Li, Jui-Ting Huang, Kaisheng Yao, Dong Yu, Frank Seide, Mike Seltzer, Geoff Zweig, Xiaodong He, Jason Williams, Yifan Gong, Alex Acero
May 2013
Hermitian Polynomial for Speaker Adaptation of Connectionist Speech Recognition Systems
Sabato Marco Simiscalchi, Jinyu Li, Chin-Hui Lee
IEEE Transactions on Audio, Speech, and Language Processing | January 2013
Investigations on Hessian-Free Optimization for Cross-Entropy Training of Deep Neural Networks
Simon Wiesler, Jinyu Li, Jian Xue
Interspeech | January 2013
Interspeech
Model-Based Margin Estimation for Hidden Markov Model Learning and Generalization
Sabato Marco Siniscalchi, Jinyu Li, Chin-Hui Lee
IET Signal Processing | January 2013
Restructuring of Deep Neural Network Acoustic Models with Singular Value Decomposition
Jian Xue, Jinyu Li, Yifan Gong
Interspeech | January 2013
-
Improvements to VTS Feature Enhancement
Jinyu Li, Mike Seltzer, Yifan Gong
March 2012
Efficient VTS Adaptation Using Jacobian Approximation
Jinyu Li, Mike Seltzer, Yifan Gong
Interspeech | January 2012
Improvements to VTS feature enhancement
Jinyu Li, Mike Seltzer, Yifan Gong
Proc. ICASSP | January 2012
Improving wideband speech recognition using mixed-bandwidth training data in CD-DNN-HMM
Jinyu Li, Dong Yu, Jui-Ting Huang, Tifan Gong
IEEE Workshop on Spoken Language Technology | January 2012
LASSO environment model combination for robust speech recognition
Proc. ICASSP | January 2012
-
Calibration of Confidence Measures In Speech Recognition
Dong Yu, Jinyu Li, Li Deng
IEEE Transactions on Audio, Speech, and Language Processing | November 2011
Towards High-Accuracy Low-Cost Noisy Robust Speech Recognition Exploiting Structured Model
Jinyu Li, Dong Yu, Li Deng, Yifan Gong
ICML Workshop 2011 | June 2011
Calibration of Confidence Measures In Speech Recognition
Dong Yu, Jinyu Li, Li Deng
IEEE Transactions on Audio, Speech and Language Processing | January 2011
Feature normalization using structured full transforms for robust speech recognition
Proc. Interspeech | January 2011
Proc. Interspeech
LASSO model adaptation for automatic speech recognition
Jinyu Li, Ming Yuan, Chin-Hui Lee
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing | January 2011
Maximum likelihood adaptation of histogram equalization with constraint for robust speech recognition
Proc. ICASSP | January 2011
Proc. ICASSP
Towards High-Accuracy Low-Cost Noisy Robust Speech Recognition Exploiting Structured Model
Jinyu Li, Li Deng, Dong Yu
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing | January 2011
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing
-
Unscented Transform with Online Distortion Estimation for HMM Adaptation
Jinyu Li, Dong Yu, Li Deng, Yifan Gong
Interspeech 2010 | September 2010
Word Confidence Calibration Using A Maximum Entropy Model With Constraints On Confidence and Word Distributions
Dong Yu, Shizhen Wang, Jinyu Li, Li Deng
March 2010
A study on the generalization capability of acoustic models for robust speech recognition
IEEE Transactions on Audio, Speech and Language Processing | January 2010
Shrinkage Model Adaptation in Automatic Speech Recognition
Jinyu Li, Yu Tsao, Chin-Hui Lee
Interspeech | January 2010
Unscented transform with online distortion estimation for HMM adaptation
Proc. Interspeech | January 2010
Word Confidence Calibration Using A Maximum Entropy Model With Constraints On Confidence and Word Distributions
Jinyu Li, Li Deng, Dong Yu
Proc. ICASSP | January 2010
-
A study on hidden Markov model’s generalization capability for speech recognition
Proc. ASRU | January 2009
A study on soft margin estimation of linear regression parameters for speaker adaptation
Proc. Interspeech | January 2009
A unified framework of HMM adaptation with joint compensation of additive and convolutive distortions
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Computer Speech and Language | January 2009, Vol 23: pp. 389-405
Ensemble speaker and speaking environment modeling approach with advanced online estimation process
Yu Tsao, Jinyu Li, Chin-Hui Lee
Proc. ICASSP | January 2009
Proc. ICASSP
Soft margin estimation on improving environment structures for ensemble speaker and speaking environment modeling
ACM Proc. the 3rd International Universal Communication Symposium | January 2009
ACM Proc. the 3rd International Universal Communication Symposium
-
SOFT MARGIN ESTIMATION FOR AUTOMATIC SPEECH RECOGNITION
December 2008
December 2008
Adaptation of Compressed HMM Parameters for Resource-Constrained Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Jian Wu, Yifan Gong, Alex Acero
April 2008
HMM Adaptation Using a Phase-Sensitive Acoustic Distortion Model for Environment-Robust Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
April 2008
HMM adaptation using a phase-sensitive acoustic distortion model for environment-robust speech recognition
Jinyu Li, Li Deng, Dong Yu, Alex Acero
Proc. ICASSP | January 2008
Proc. ICASSP
On a Generalization of Margin-Based Discriminative Training to Robust Speech Recognition
Proc. Interspeech | January 2008
Soft Margin Estimation with Various Separation Levels for LVCSR
Proc. Interspeech | January 2008
-
High-Performance HMM Adaptation With Joint Compensation of Additive and Convolutive Distortions Via Vector Taylor Series
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Proceedings IEEE Workshop on ASRU | April 2007
Proceedings IEEE Workshop on ASRU
A study on soft margin estimation for LVCSR
Proc. IEEE Automatic Speech Recognition and Understanding | January 2007
Approximate Test Risk Bound Minimization through Soft Margin Estimation
IEEE Trans. on Audio, Speech, and Language Proc | January 2007
Approximate test risk minimization through soft margin estimation
Proc. ICASSP | January 2007
Detection-Based ASR in the Automatic Speech Attribute Transcription Project
Proc. Interspeech | January 2007
Proc. Interspeech
High-Performance HMM Adaptation with Joint Compensation of Additive and Convolutive Distortions via Vector Taylor Series
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Proc. IEEE Automatic Speech Recognition and Understanding | January 2007
Proc. IEEE Automatic Speech Recognition and Understanding
Soft Margin Feature Extraction for Automatic Speech Recognition
Proc. Interspeech | January 2007
-
A study on lattice rescoring with knowledge scores for automatic speech recognition
Proc. Interspeech | January 2006
Proc. Interspeech
Application of EαNets to feature recognitionof articulation manner in knowledge-based automatic speech recognition
Lecture Notes in Computer Science | January 2006
Language recognition based on score distribution feature vectors and discriminative classifier fusion
Proc. IEEE Odyssey Workshop on Speaker and Language Recognition | January 2006
Soft Margin Estimation of Hidden Markov Model Parameters (best student paper)
Proc. Interspeech | January 2006
Best Student Paper
-
A study on knowledge source integration for candidate rescoring in automatic speech recognition
Proc. ICASSP | January 2005
A study on separation between acoustic models and its applications
Proc. Interspeech | January 2005
On Designing and Evaluating Speech Event Detectors
Proc. Interspeech | January 2005
-
A complexity reduction of ETSI advanced front-end for DSR
Proc. ICASSP | January 2004
Dimensionality reduction using MCE-optimized LDA transformation
Proc. ICASSP | January 2004
-
A novelsearch algorithm for LSF VQ
Proc. ICSLP | January 2000
Proc. ICSLP
Publications by Research Area
-
Boosting Large Language Model for Speech Synthesis: An Empirical Study
Hongkun Hao, Long Zhou, Shujie Liu, Jinyu Li, Shujie Hu, Rui Wang, Furu Wei
ICASSP | April 2025
V2SFlow: Video-to-Speech Generation with Speech Decomposition and Rectified Flow
Jeongsoo Choi, Ji-Hoon Kim, Jinyu Li, Joon Son Chung, Shujie Liu
ICASSP | April 2025
Target word activity detector: An approach to obtain ASR word boundaries without lexicon
Sunit Sivasankaran, Eric Sun, Jinyu Li, Yan Huang, Jing Pan
ICASSP | April 2025
Investigating neural audio codecs for speech language model-based speech generation
Jiaqi Li , Dongmei Wang, Xiaofei Wang, Yao Qian, Long Zhou, Shujie Liu, Midia Yousefi, Canrun Li , Chung-Hsien Tsai, Zhen Xiao, Yanqing Liu, Junkun Chen, Sheng Zhao, Jinyu Li, Zhizheng Wu , Michael Zeng
SLT Workshop | December 2024
CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation
Rui Zhao, Jinyu Li, Ruchao Fan, Matt Post
SLT Workshop | December 2024
Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech
Haibin Wu, Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Daniel Tompkins, Chung-Hsien Tsai , Canrun Li, Zhen Xiao, Sheng Zhao , Jinyu Li, Naoyuki Kanda
SLT Workshop | December 2024
Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translation
Peidong Wang, Jian Xue, Jinyu Li, Junkun Chen, Aswin Shanmugam Subramanian
Interspeech 2024 | September 2024
An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS
Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Hemin Yang, Zirun Zhu, Min Tang, Yufei Xia, Jinzhu Li, sheng zhao , Jinyu Li, Naoyuki Kanda
Interspeech 2024 | September 2024
COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning
Jing Pan, Jian Wu, Yashesh Gaur , Sunit Sivasankaran, Zhuo Chen , Shujie Liu, Jinyu Li
Interspeech 2024 | September 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei
June 2024
End-to-End Automatic Speech Recognition
MSR-TR-2024-30 | May 2024
Published by Microsoft
IEEE SPS Webinar
ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
Yiming Wang, Jinyu Li
ICASSP | April 2024
t-SOT FNT: streaming multi-talker ASR with text-only domain adaptation capability
Jian Wu, Naoyuki Kanda, Takuya Yoshioka, Rui Zhao, Zhuo Chen, Jinyu Li
ICASSP | April 2024
Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation
Sara Pap, Peidong Wang, Junkun Chen, Jian Xue, Naoyuki Kanda, Jinyu Li, Yashesh Gaur
ICASSP | April 2024
DiariST: Streaming Speech Translation with Speaker Diarization
Mu Yang, Naoyuki Kanda, Xiaofei Wang, Junkun Chen, Peidong Wang, Jian Xue, Jinyu Li, Takuya Yoshioka
ICASSP | April 2024
SpeechLM: Enhanced Speech Pre-Training With Unpaired Textual Data
Ziqiang Zhang, Sanyuan Chen, Long Zhou, Yu Wu, Shuo Ren, Shujie Liu, Zhuoyuan Yao, Xun Gong, Lirong Dai, Jinyu Li, Furu Wei
IEEE/ACM Transactions on Audio, Speech, and Language Processing | March 2024
Advanced Long-Content Speech Recognition with Factorized Neural Transducer
Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian
IEEE/ACM Transactions on Audio, Speech, and Language Processing | January 2024
Decoder-only Modeling for Speech
December 2023
Invited Talk at NTU and SJTU
Large-Scale Streaming End-to-End Speech Translation
December 2023
Invited Talk at NTU and SJTU
Building High-accuracy Multilingual ASR with Gated Language Experts and Curriculum Training
Eric Sun, Jinyu Li, Yuxuan Hu, Yimeng Zhu, Long Zhou, Jian Xue, Peidong Wang, Linquan Liu, Shujie Liu, Edward Lin, Yifan Gong
December 2023
Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition
Yuang Li, Yu Wu, Jinyu Li, Shujie Liu
Workshop of Automatic Speech Recognition and Understanding | December 2023
On decoder-only architecture for speech-to-text and large language model integration
Jian Wu, Yashesh Gaur, Zhuo Chen, Long Zhou, Yimeng Zhu, Tianrui Wang, Jinyu Li, Shujie Liu, Bo Ren, Linquan Liu, Yu Wu
Workshop of Automatic Speech Recognition and Understanding | December 2023
Improving Stability in Simultaneous Speech Translation: A Revision-Controllable Decoding Approach
Junkun Chen, Jian Xue, Peidong Wang, Jing Pan, Jinyu Li
Workshop of Automatic Speech Recognition and Understanding | December 2023
Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments
Sara Papi, Peidong Wang, Junkun Chen, Jian Xue, Jinyu Li, Yashesh Gaur
Workshop of Automatic Speech Recognition and Understanding | December 2023
A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capability
Jian Xue, Peidong Wang, Jinyu Li, Eric Sun
Workshop of Automatic Speech Recognition and Understanding | December 2023
Recent Advances in End-to-End Automatic Speech Recognition (Invited Talk at ASC 2023)
| November 2023
Published by Microsoft
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
Xiaofei Wang, Manthan Thakker, Zhuo Chen, Naoyuki Kanda, Sefik Emre Eskimez, Sanyuan Chen, Min Tang, Shujie Liu, Jinyu Li, Takuya Yoshioka
August 2023
Accelerating Transducers through Adjacent Token Merging
Yuang Li, Yu Wu, Jinyu Li, Shujie Liu
Interspeech | August 2023
LAMASSU: A Streaming Language-Agnostic Multilingual Speech Recognition and Translation Model Using Neural Transducers
Peidong Wang, Eric Sun, Jian Xue, Yu Wu, Long Zhou, Yashesh Gaur, Shujie Liu, Jinyu Li
Interspeech | August 2023
Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentation
Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Sheng zhao
ICASSP | June 2023
Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition models
Rui Zhao, Jian Xue, Partha Parthasarathy, Veljko Miljanic, Jinyu Li
ICASSP | June 2023
Speaker Change Detection for Transformer Transducer ASR
Jian Wu, Zhuo Chen, Min Hu, Xiong Xiao, Jinyu Li
June 2023
Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation
Kun Wei, Long Zhou, Ziqiang Zhang, Liping Chen, Shujie Liu, Lei He, Jinyu Li, Furu Wei
ICASSP | June 2023
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
ICASSP | June 2023
Simulating realistic speech overlaps improves multi-talker ASR
Muqiao Yang, Naoyuki Kanda, Xiaofei Wang, Jian Wu, Sunit Sivasankaran, Zhuo Chen, Jinyu Li, Takuya Yoshioka
ICASSP | June 2023
Speech separation with large-scale self-supervised learning
Zhuo Chen, Naoyuki Kanda, Jian Wu, Yu Wu, Xiaofei Wang, Takuya Yoshioka, Jinyu Li, Sunit Sivasankaran, Sefik Emre Eskimez
ICASSP | June 2023
LongFNT: Long-form Speech Recognition with Factorized Neural Transducer
Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian
ICASSP | June 2023
Self-supervised learning with bi-label masked speech prediction for streaming multi-talker speech recognition
Zili Huang, Zhuo Chen, Naoyuki Kanda, Jian Wu, Yiming Wang, Jinyu Li, Takuya Yoshioka, Xiaofei Wang, Peidong Wang
ICASSP | June 2023
CTCBERT: Advancing Hidden-unit BERT with CTC Objectives
Ruchao Fan, Yiming Wang, Yashesh Gaur, Jinyu Li
ICASSP | June 2023
VatLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning
Qiushi Zhu, Long Zhou, Ziqiang Zhang, Shujie Liu, Binxing Jiao, Jie Zhang, Lirong Dai, Daxin Jiang, Jinyu Li, Furu Wei
IEEE Transactions on Multimedia | May 2023
STREAMING, FAST AND ACCURATE ON-DEVICE INVERSE TEXT NORMALIZATION FOR AUTOMATIC SPEECH RECOGNITION
Yashesh Gaur, Nick Kibre, Jian Xue, Kangyuan Shu, Yuhui Wang, Issac Alphanso, Jinyu Li, Yifan Gong
Spoken Language Technology Workshop | January 2023
Advancing end-to-end automatic speech recognition and beyond
International Symposium on Chinese Spoken Language Processing | December 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
Ziqiang Zhang, Long Zhou, Junyi Ao, Shujie Liu, Lirong Dai, Jinyu Li, Furu Wei
EMNLP 2022 | October 2022
Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Zhuo Chen, Gang Liu, Jinyu Li, Jian Wu, Xiangzhan Yu, Furu Wei
Interspeech | September 2022
Large-Scale Streaming End-to-End Speech Translation with Neural Transducers
Jian Xue, Peidong Wang, Jinyu Li, Matt Post, Yashesh Gaur
Interspeech | September 2022
Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Data
Junyi Ao, Ziqiang Zhang, Long Zhou, Shujie Liu, Haizhou Li, Tom Ko, Lirong Dai, Jinyu Li, Yao Qian, Furu Wei
Interspeech | September 2022
Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training
Chengyi Wang, Yiming Wang, Yu Wu, Sanyuan Chen, Jinyu Li, Shujie Liu, Furu Wei
Interspeech | September 2022
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Wangyou Zhang, Zhuo Chen, Naoyuki Kanda, Shujie Liu, Jinyu Li, Sefik Emre Eskimez, Takuya Yoshioka, Xiong Xiao, Zhong Meng, Yanmin Qian, Furu Wei
Interspeech 2022 | September 2022
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Naoyuki Kanda, Jinyu Li, Xie Chen, Yu Wu, Yifan Gong
Interspeech 2022 | September 2022
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
arXiv:2209.04974 | September 2022
Towards Contextual Spelling Correction for Customization of End-to-End Speech Recognition Systems
Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Veljko Miljanic, Sheng Zhao, Hosam Khalil
IEEE/ACM Transactions on Audio, Speech, and Language Processing | September 2022, Vol 30: pp. 3089-3097
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, Furu Wei
IEEE Journal of Selected Topics in Signal Processing | July 2022, Vol 16(6): pp. 1505-1518
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu
ICASSP | May 2022
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
ACL 2022 | May 2022
Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction
Heming Wang, Yao Qian, Xiaofei Wang, Yiming Wang, Chengyi Wang, Shujie Liu, Takuya Yoshioka, Jinyu Li, DeLiang Wang
ICASSP 2022 | May 2022
Continuous Speech Separation with Recurrent Selective Attention Network
Yixuan Zhang, Zhuo Chen, Jian Wu, Takuya Yoshioka, Peidong Wang, Zhong Meng, Jinyu Li
ICASSP 2021 | May 2022
Recent Advances in End-to-End Automatic Speech Recognition
APSIPA Transactions on Signal and Information Processing | April 2022
On Addressing Practical Challenges For RNN-Transducer
Rui Zhao, Jian Xue, Jinyu Li, Wenning Wei, Lei He, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2021
Advancing end-to-end automatic speech recognition
MSR-TR-2021-32 | October 2021
Published by Microsoft
Keynote talk at the conference on Computational Linguistics and Speech Processing, 2021.
Rapid Speaker Adaptation for Conformer Transducer: Attention and Bias are All You Need
Yan Huang, Guoli Ye, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Multiple Softmax Architecture for Streaming Multilingual End-to-End ASR Systems
Vikas Joshi, Amit Das, Eric Sun, Rupesh Mehta, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Improving RNN-T for Domain Scaling Using Semi-Supervised Training with Neural TTS
Yan Deng, Rui Zhao, Zhong Meng, Xie Chen, Bing Liu, Jinyu Li, Yifan Gong, Lei He
Interspeech 2021 | August 2021
Improving Multilingual Transformer Transducer Models by Reducing Language Confusions
Eric Sun, Jinyu Li, Zhong Meng, Yu Wu, Jian Xue, Shujie Liu, Yifan Gong
Interspeech 2021 | August 2021
Investigation of Practical Aspects of Single Channel Speech Separation for ASR
Jian Wu, Zhuo Chen, Sanyuan Chen, Yu Wu, Takuya Yoshioka, Naoyuki Kanda, Shujie Liu, Jinyu Li
Interspeech 2021 | August 2021
Ultra Fast Speech Separation Model with Teacher Student Learning
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Takuya Yoshioka, Shujie Liu, Jinyu Li, Xiangzhan Yu
Interspeech 2021 | August 2021
A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems
Xiaoqiang Wang, Yanqing Liu, Sheng Zhao, Jinyu Li
Interspeech 2021 | August 2021
Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
Zhong Meng, Yu Wu, Naoyuki Kanda, Liang Lu, Xie Chen, Guoli Ye, Eric Sun, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Streaming Multi-talker Speech Recognition with Joint Speaker Identification
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer
Liang Lu, Zhong Meng, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Naoyuki Kanda, Yashesh Gaur, Sarangarajan Parthasarathy, Eric Sun, Liang Lu, Xie Chen, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020
Xiong Xiao, Naoyuki Kanda, Zhuo Chen, Tianyan Zhou, Takuya Yoshioka, Sanyuan Chen, Yong Zhao, Gang Liu, Yu Wu, Jian Wu, Shujie Liu, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Developing real-time streaming transformer transducer for speech recognition on large-scale dataset
Xie Chen, Yu Wu, Zhenghao Wang, Shujie Liu, Jinyu Li
ICASSP | June 2021
Continuous speech separation with Conformer
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Jinyu Li, Takuya Yoshioka, Chengyi Wang, Shujie Liu, Ming Zhou
ICASSP | June 2021
Don’t shoot butterfly with rifles: multi-channel continuous speech separation with early exit transformer
Sanyuan Chen, Yu Wu, Zhuo Chen, Takuya Yoshioka, Shujie Liu, Jinyu Li, Xiangzhan Yu
ICASSP | June 2021
Streaming end-to-end multi-talker speech recognition
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
IEEE Signal Processing Letter | April 2021, Vol 28
Adaptation Algorithms for Speech Recognition: An Overview
Peter Bell, Joachim Fainberg, Ondrej Klejch, Jinyu Li, Steve Renals, Pawel Swietojanski
IEEE Open Journal of Signal Processing | February 2021
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen, Hakan Erdogan, Zili Huang, Maokui He, Shinji Watanabe, Jun Du, Takuya Yoshioka, Yi Luo, Naoyuki Kanda, Jinyu Li, Scott Wisdom, John R. Hershey
SLT 2021 | January 2021
Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, Yifan Gong
SLT 2021 | January 2021
Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations
Shahram Ghorbani, Yashesh Gaur, Yu Shi, Jinyu Li
Spoken Language Technology Workshop | January 2021
Dual-Path RNN For Long Recording Speech Separation
Chenda Li, Yi Luo, Cong Han, Jinyu Li, Takuya Yoshioka, Tianyan Zhou, Marc Delcroix, Keisuke Kinoshita, Christoph Boeddeker, Yanmin Qian, Shinji Watanabe, Zhuo Chen
Spoken Language Technology Workshop | January 2021
Speaker Separation Using Speaker Inventories and Estimated Speech
Peidong Wang, Zhuo Chen, DeLiang Wang, Jinyu Li, Yifan Gong
IEEE Transactions on Audio, Speech, and Language Processing | December 2020, Vol 29: pp. 537-546
Developing streaming end-to-end models for automatic speech recognition in industry
MSR-TR-2020-45 | December 2020
Published by Microsoft
On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Jinyu Li, Yu Wu, Yashesh Gaur, Chengyi Wang, Rui Zhao , Shujie Liu
Interspeech | October 2020
Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability
Jinyu Li, Rui Zhao, Zhong Meng, Yanqing Liu, Wenning Wei, Sarangarajan Parthasarathy, Vadim Mazalov, Zhenghao Wang, Lei He, Sheng Zhao, Yifan Gong
Interspeech | October 2020
Exploring Transformers for Large-Scale Speech Recognition
Liang Lu, Changliang Liu, Jinyu Li, Yifan Gong
Interspeech | October 2020
An End-to-end Architecture of Online Multi-channel Speech Separation
Jian Wu, Zhuo Chen, Jinyu Li, Takuya Yoshioka, Zhili Tan, Ed Lin, Yi Luo , Lei Xie
Interspeech | October 2020
Combination of end-to-end and hybrid models for speech recognition
Jeremy Wong, Yashesh Gaur, Rui Zhao, Liang Lu, Eric Sun, Jinyu Li, Yifan Gong
Interspeech | October 2020
Low Latency End-to-End Streaming Speech Recognition with a Scout Network
Chengyi Wang, Yu Wu, Liang Lu, Shujie Liu, Jinyu Li, Guoli Ye, Ming Zhou
Interspeech | October 2020
Transfer Learning Approaches for Streaming End-to-End Speech Recognition System
Vikas Joshi, Rui Zhao, Rupesh R. Mehta, Kshitiz Kumar, Jinyu Li
Interspeech | October 2020
Rapid RNN-T Adaptation Using Personalized Speech Synthesis and Neural Language Generator
Yan Huang, Jinyu Li, Lei He, Wenning Wei, William Gale, Yifan Gong
Interspeech | October 2020
Recent Progresses On Deep Learning For Speech Recognition
Jinyu Li, Dong Yu
Handbook of Pattern Recognition and Computer Vision: 6th Edition | 2020
Minimum latency training strategies for streaming sequence-to-sequence ASR
Hirofumi Inaguma, Yashesh Gaur, Liang Lu, Jinyu Li, Yifan Gong
ICASSP | April 2020
Exploring pre-training with alignments for RNN transducer based end-to-end speech recognition
Hu Hu, Rui Zhao, Jinyu Li, Liang Lu, Yifan Gong
ICASSP | April 2020
Continuous speech separation: dataset and analysis
Zhuo Chen, Takuya Yoshioka, Liang Lu, Tianyan Zhou, Zhong Meng, Yi Luo, Jian Wu, Xiong Xiao, Jinyu Li
ICASSP | April 2020
Using personalized speech synthesis and neural language generator for rapid speaker adaptation
Yan Huang, Lei He, Wenning Wei, William Gale, Jinyu Li, Yifan Gong
ICASSP | April 2020
L-Vector: Neural Label Embedding for Domain Adaptation
Zhong Meng, Hu Hu, Jinyu Li, Changliang Liu, Yan Huang, Yifan Gong, Chin-Hui Lee
ICASSP | April 2020
CNN with phonetic attention for text-independent speaker verification
Tianyan Zhou, yong zhao, Jinyu Li, Yifan Gong, Jian Wu
Automatic Speech Recognition and Understanding Workshop | December 2019
Character-aware attention-based end-to-end speech recognition
Zhong Meng, Yashesh Gaur, Jinyu Li, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Domain adaptation via teacher-student learning for end-to-end speech recognition
Zhong Meng, Jinyu Li, Yashesh Gaur, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Improving RNN Transducer Modeling for End-to-End Speech Recognition
Jinyu Li, Rui Zhao, Hu Hu, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
Amit Das, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong
IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING | December 2019, pp. 1880-1892
Speech Separation Using Speaker Inventory
Peidong Wang, Zhuo Chen, Xiong Xiao, Zhong Meng, Takuya Yoshioka, Tianyan Zhou, Liang Lu, Jinyu Li
Automatic Speech Recognition and Understanding Workshop | November 2019
Speaker Adaptation for Attention-Based End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Jinyu Li, Yifan Gong
Interspeech | September 2019
Acoustic-to-Phrase Models for Speech Recognition
Yashesh Gaur, Jinyu Li, Zhong Meng, Yifan Gong
Interspeech | September 2019
Layer Trajectory BLSTM
Eric Sun, Jinyu Li, Yifan Gong
Interspeech | September 2019
Improving Layer Trajectory LSTM With Future Context Frames
Jinyu Li, Liang Lu, Changliang Liu, Yifan Gong
ICASSP | May 2019
Adversarial Speaker Verification
Zhong Meng, Yong Zhao, Jinyu Li, Yifan Gong
ICASSP | May 2019
Adversarial Speaker Adaptation
Zhong Meng, Jinyu Li, Yifan Gong
ICASSP | May 2019
Towards Code-Switching ASR for End-to-End CTC Models
Ke Li, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong
ICASSP | May 2019
Attentive Adversarial Learning for Domain-Invariant Training
Zhong Meng, Jinyu Li, Yifan Gong
ICASSP | May 2019
Conditional Teacher-Student Learning
Zhong Meng, Jinyu Li, Yong Zhao, Yifan Gong
ICASSP | May 2019
Universal Acoustic Modeling Using Neural Mixture Models
Amit Das, Jinyu Li, Changliang Liu, Yifan Gong
ICASSP | May 2019
Multi-Channel Overlapped Speech Recognition with Location Guided Speech Extraction Network
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
SLT 2018 | December 2018
Exploring layer trajectory LSTM with depth processing units and attention
Jinyu Li, Liang Lu, Changliang Liu, Yifan Gong
Spoken Language Technology | December 2018
MULTI-CHANNEL OVERLAPPED SPEECH RECOGNITION WITH LOCATION GUIDED SPEECH EXTRACTION NETWORK
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
Spoken Language Technology | December 2018
Speaker Adaptation for End-to-End CTC Models
Ke Li, Jinyu Li, yong zhao, Yifan Gong (ygong), Kshitiz Kumar
Spoken Language Technology | December 2018
Improved training for online end-to-end speech recognition systems
Suyoun Kim, Michael L. Seltzer, Jinyu Li, Rui Zhao
Interspeech 2018 | September 2018
Adversarial Feature-Mapping for Speech Enhancement
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang Juang
Interspeech 2018 | September 2018
Cycle-Consistent Speech Enhancement
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang Juang
Interspeech 2018 | September 2018
Layer Trajectory LSTM
Jinyu Li, Changliang Liu, Yifan Gong
Interspeech 2018 | September 2018
Advancing Connectionist Temporal Classification With Attention Modeling
Amit Das, Jinyu Li, Rui Zhao, Yifan Gong
ICASSP | April 2018
Advancing Acoustic-to-Word CTC Model
Jinyu Li, Guoli Ye, Amit Das, Rui Zhao, Yifan Gong
ICASSP | April 2018
Developing Far-Field Speaker System Via Teacher-Student Learning
Jinyu Li, Rui Zhao, Zhuo Chen, Changliang Liu, Xiong Xiao, Guoli Ye, Yifan Gong
ICASSP | April 2018
Efficient Integration of Fixed Beamformers and Speech Separation Networks for Multi-Channel Far-Field Speech Separation
Zhuo Chen, Takuya Yoshioka, Xiong Xiao, Jinyu Li, Michael L. Seltzer, Yifan Gong
ICASSP 2018 | April 2018
DOMAIN AND SPEAKER ADAPTATION FOR CORTANA SPEECH RECOGNITION
Yong Zhao, Jinyu Li, Shixiong Zhang, Liping Chen, Yifan Gong
ICASSP | April 2018
Efficient integration of fixed beamformers and speech separation networks for multi-channel far-field speech separation
Zhuo Chen, Takuya Yoshioka, Xiong Xiao, Jinyu Li, Michael L. Seltzer, Yifan Gong
ICASSP | April 2018
Speaker-Invariant Training via Adversarial Learning
Zhong Meng, Jinyu Li, Zhuo Chen, Yong Zhao, Vadim Mazalov, Yifan Gong, Biing-Hwang (Fred) Juang
ICASSP | April 2018
Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang (Fred) Juang
ICASSP | April 2018
Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition
Zhehuai Chen, Jasha Droppo, Jinyu Li, Wayne Xiong
IEEE/ACM Transactions on Audio, Speech, and Language Processing | January 2018
Acoustic-To-Word Model Without OOV
Jinyu Li, Guoli Ye, Rui Zhao, Jasha Droppo, Yifan Gong
ASRU | December 2017
Cracking the cocktail party problem by multi-beam deep attractor network
Zhuo Chen, Jinyu Li, Xiong Xiao, Takuya Yoshioka, Huaming Wang, Zhenghao Wang, Yifan Gong
ASRU 2018 | December 2017
Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
Zhong Meng, Zhuo Chen, Vadim Mazalov, Jinyu Li, Yifan Gong
ASRU | December 2017
Deep Learning Acoustic Model in Microsoft Cortana Voice Assistant
MSR-TR-2017-60 | November 2017
Published by Microsoft
Improving mask learning based speech enhancement system with restoration layers and residual connection
Zhuo Chen, Yan Huang, Jinyu Li, Yifan Gong
Interspeech | August 2017
Large-Scale Domain Adaptation via Teacher-Student Learning
Jinyu Li, Michael Seltzer, Xi Wang, Rui Zhao, Yifan Gong
Interspeech | August 2017
Challenges and Solutions to Deep Learning Network Acoustic Modeling in Speech Recognition Products at Microsoft
New Era for Robust Speech Recognitino: Exploiting Deep Learning | Published by Springer | 2017
Recent Progresses in Deep Learning Based Acoustic Models
Dong Yu, Jinyu Li
IEEE/CAA JOURNAL OF AUTOMATICA SINICA | July 2017
Extended low-rank plus diagonal adaptation for deep and recurrent neural networks
yong zhao, Jinyu Li, kshitiz kumar, Yifan Gong
ICASSP | April 2017
Improved Cepstra Minimum-Mean-Square-Error Noise Reduction Algorithm For Robust Speech Recognition
Jinyu Li, Yan Huang, Yifan Gong
ICASSP | April 2017
End-to-End Attention Based Text-Dependent Speaker Verification
Shixiong Zhang, Zhuo Chen, Yong Zhao, Jinyu Li, Yifan Gong
SLT workshop | December 2016
Deep Convolutional Neural Networks with Layer-wise Context Expansion and Attention
Dong Yu, Wayne Xiong, Jasha Droppo, Andreas Stolcke, Guoli Ye, Jinyu Li, Geoffrey Zweig
Proc. Interspeech | September 2016
EXPLORING MULTIDIMENSIONAL LSTMS FOR LARGE VOCABULARY ASR
Jinyu Li, Abdelrahman Mohamed, Geoffrey Zweig, Yifan Gong
ICASSP | March 2016
Recurrent Support Vector Machines for Speech Recognition
Shi-Xiong Zhang, Rui Zhao, Chaojun Liu, Jinyu Li, Yifan Gong
ICASSP | March 2016
ICASSP
Compensation with Prior Knowledge
Jinyu Li, Li Deng, Yifan Gong
Robust Automatic Speech Recognition: A Bridge to Practical Applications | Published by Elsevier | 2015 | Robust Automatic Speech Recognition: A Bridge to Practical Applications edition
Rapid Adaptation for Deep Neural Networks through Multi-Task Learning
Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Ji Wu, Chin-Hui Lee
Interspeech | September 2015
Maximum a Posteriori Adaptation of Network Parameters in Deep Models
Zheng Huang, Sabato Marco Siniscalchi, I-Fan Chen, Jinyu Li, Jiadong Wu, Chin-Hui Lee
September 2015
Small-footprint high-performance deep neural network-based speech recognition using split-VQ
Yongqiang Wang, Jinyu Li, Yifan Gong
ICASSP | April 2015
An Analysis of Convolutional Neural Networks for Speech Recognition
Jui-Ting Huang, Jinyu Li, Yifan Gong
ICASSP | April 2015
variable-activation and variable-input deep neural network for robust speech recognition
Rui Zhao, Jinyu Li, Yifan Gong
IEEE SLT workshop | December 2014
Beyond Cross-entropy: Towards Better Frame-level Objective Functions For Deep Neural Network Training In Automatic Speech Recognition
Zhen Huang, Jinyu Li, Chao Weng, Chin-Hui Lee
Interspeech | September 2014
Interspeech
Deep Neural Network for Automatic Speech Recognition: from the Industry’s View
September 2014
An Overview of Noise-Robust Automatic Speech Recognition
Jinyu Li, Li Deng, Yifan Gong, Reinhold Haeb-Umbach
IEEE/ACM Transactions on Audio, Speech, and Language Processing | April 2014, Vol 22: pp. 745
Investigation of Maxout Networks For Speech Recognition
Pawel Swietojanski, Jinyu Li, Jui-Ting Huang
ICASSP | January 2014
Feature Compensation Using Linear Combination of Speaker and Environment Dependent Correction Vectors
Xiong Xiao, Jinyu Li, Eng Siong Chng, Haizhou Li
ICASSP | January 2014
Hermitian Polynomial for Speaker Adaptation of Connectionist Speech Recognition Systems
Sabato Marco Simiscalchi, Jinyu Li, Chin-Hui Lee
IEEE Transactions on Audio, Speech, and Language Processing | January 2013
Restructuring of Deep Neural Network Acoustic Models with Singular Value Decomposition
Jian Xue, Jinyu Li, Yifan Gong
Interspeech | January 2013
Improvements to VTS Feature Enhancement
Jinyu Li, Mike Seltzer, Yifan Gong
March 2012
Improvements to VTS feature enhancement
Jinyu Li, Mike Seltzer, Yifan Gong
Proc. ICASSP | January 2012
Efficient VTS Adaptation Using Jacobian Approximation
Jinyu Li, Mike Seltzer, Yifan Gong
Interspeech | January 2012
Improving wideband speech recognition using mixed-bandwidth training data in CD-DNN-HMM
Jinyu Li, Dong Yu, Jui-Ting Huang, Tifan Gong
IEEE Workshop on Spoken Language Technology | January 2012
Calibration of Confidence Measures In Speech Recognition
Dong Yu, Jinyu Li, Li Deng
IEEE Transactions on Audio, Speech and Language Processing | January 2011
A study on the generalization capability of acoustic models for robust speech recognition
IEEE Transactions on Audio, Speech and Language Processing | January 2010
A unified framework of HMM adaptation with joint compensation of additive and convolutive distortions
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Computer Speech and Language | January 2009, Vol 23: pp. 389-405
A study on hidden Markov model’s generalization capability for speech recognition
Proc. ASRU | January 2009
HMM adaptation using a phase-sensitive acoustic distortion model for environment-robust speech recognition
Jinyu Li, Li Deng, Dong Yu, Alex Acero
Proc. ICASSP | January 2008
Proc. ICASSP
High-Performance HMM Adaptation With Joint Compensation of Additive and Convolutive Distortions Via Vector Taylor Series
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Proceedings IEEE Workshop on ASRU | April 2007
Proceedings IEEE Workshop on ASRU
High-Performance HMM Adaptation with Joint Compensation of Additive and Convolutive Distortions via Vector Taylor Series
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Proc. IEEE Automatic Speech Recognition and Understanding | January 2007
Proc. IEEE Automatic Speech Recognition and Understanding
-
ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation
Zongyi Li, Shujie Hu, Shujie Liu, Long Zhou, Jeongsoo Choi, Lingwei Meng, Xun Guo, Jinyu Li, Hefei Ling, Furu Wei
October 2024
October 2024
Total-Duration-Aware Duration Modeling for Text-to-Speech Systems
Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Chung-Hsien Tsai, Canrun Li, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Jinyu Li, Sheng Zhao, Naoyuki Kanda
Interspeech 2024 | September 2024
Autoregressive Speech Synthesis without Vector Quantization
Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen Meng, Furu Wei
July 2024
VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment
Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, Furu Wei
June 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei
June 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiang-Yang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, Sheng Zhao
ICML | June 2024
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
Chenyang Le, Yao Qian, Dongmei Wang, Long Zhou, Shujie Liu, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Sheng Zhao, Michael Zeng
NeurIPS 2024 | May 2024
CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations
Leying Zhang, Yao Qian, Long Zhou, Shujie Liu (shujliu), Dongmei Wang, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Lei He, Sheng Zhao, Michael Zeng
NeurIPS 2024 | April 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model
Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, Furu Wei
March 2024
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
Xiaofei Wang, Manthan Thakker, Zhuo Chen, Naoyuki Kanda, Sefik Emre Eskimez, Sanyuan Chen, Min Tang, Shujie Liu, Jinyu Li, Takuya Yoshioka
August 2023
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei
March 2023
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
Ziqiang Zhang, Long Zhou, Junyi Ao, Shujie Liu, Lirong Dai, Jinyu Li, Furu Wei
EMNLP 2022 | October 2022
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Wangyou Zhang, Zhuo Chen, Naoyuki Kanda, Shujie Liu, Jinyu Li, Sefik Emre Eskimez, Takuya Yoshioka, Xiong Xiao, Zhong Meng, Yanmin Qian, Furu Wei
Interspeech 2022 | September 2022
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Naoyuki Kanda, Jinyu Li, Xie Chen, Yu Wu, Yifan Gong
Interspeech 2022 | September 2022
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
arXiv:2209.04974 | September 2022
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, Furu Wei
IEEE Journal of Selected Topics in Signal Processing | July 2022, Vol 16(6): pp. 1505-1518
Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
Zhong Meng, Yu Wu, Naoyuki Kanda, Liang Lu, Xie Chen, Guoli Ye, Eric Sun, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Streaming Multi-talker Speech Recognition with Joint Speaker Identification
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer
Liang Lu, Zhong Meng, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Naoyuki Kanda, Yashesh Gaur, Sarangarajan Parthasarathy, Eric Sun, Liang Lu, Xie Chen, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020
Xiong Xiao, Naoyuki Kanda, Zhuo Chen, Tianyan Zhou, Takuya Yoshioka, Sanyuan Chen, Yong Zhao, Gang Liu, Yu Wu, Jian Wu, Shujie Liu, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Streaming end-to-end multi-talker speech recognition
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
IEEE Signal Processing Letter | April 2021, Vol 28
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen, Hakan Erdogan, Zili Huang, Maokui He, Shinji Watanabe, Jun Du, Takuya Yoshioka, Yi Luo, Naoyuki Kanda, Jinyu Li, Scott Wisdom, John R. Hershey
SLT 2021 | January 2021
Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, Yifan Gong
SLT 2021 | January 2021
Speaker Separation Using Speaker Inventories and Estimated Speech
Peidong Wang, Zhuo Chen, DeLiang Wang, Jinyu Li, Yifan Gong
IEEE Transactions on Audio, Speech, and Language Processing | December 2020, Vol 29: pp. 537-546
Sequence-Level Self-Learning with Multiple Hypotheses
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
Interspeech | October 2020
Sequence-level self-learning with multi-task learning framework
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
October 2020
Speech Separation Using Speaker Inventory
Peidong Wang, Zhuo Chen, Xiong Xiao, Zhong Meng, Takuya Yoshioka, Tianyan Zhou, Liang Lu, Jinyu Li
Automatic Speech Recognition and Understanding Workshop | November 2019
Multi-Channel Overlapped Speech Recognition with Location Guided Speech Extraction Network
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
SLT 2018 | December 2018
Robust Automatic Speech Recognition – A Bridge to Practical Applications (1st Edition)
Jinyu Li, Li Deng, R. Haeb-Umbach, Y. Gong
Published by Elsevier | October 2015
Small-footprint high-performance deep neural network-based speech recognition using split-VQ
Yongqiang Wang, Jinyu Li, Yifan Gong
ICASSP | April 2015
Recent Advances in Deep Learning for Speech Research at Microsoft
Li Deng, Jinyu Li, Jui-Ting Huang, Kaisheng Yao, Dong Yu, Frank Seide, Mike Seltzer, Geoff Zweig, Xiaodong He, Jason Williams, Yifan Gong, Alex Acero
May 2013
Feature Learning in Deep Neural Networks – Studies on Speech Recognition
Dong Yu, Mike Seltzer, Jinyu Li, Jui-Ting Huang, Frank Seide
International Conference on Learning Representations | May 2013
Towards High-Accuracy Low-Cost Noisy Robust Speech Recognition Exploiting Structured Model
Jinyu Li, Li Deng, Dong Yu
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing | January 2011
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing
Word Confidence Calibration Using A Maximum Entropy Model With Constraints On Confidence and Word Distributions
Jinyu Li, Li Deng, Dong Yu
Proc. ICASSP | January 2010
Adaptation of Compressed HMM Parameters for Resource-Constrained Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Jian Wu, Yifan Gong, Alex Acero
April 2008
HMM Adaptation Using a Phase-Sensitive Acoustic Distortion Model for Environment-Robust Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
April 2008
-
Total-Duration-Aware Duration Modeling for Text-to-Speech Systems
Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Chung-Hsien Tsai, Canrun Li, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Jinyu Li, Sheng Zhao, Naoyuki Kanda
Interspeech 2024 | September 2024
Autoregressive Speech Synthesis without Vector Quantization
Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen Meng, Furu Wei
July 2024
VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment
Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, Furu Wei
June 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei
June 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiang-Yang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, Sheng Zhao
ICML | June 2024
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
Chenyang Le, Yao Qian, Dongmei Wang, Long Zhou, Shujie Liu, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Sheng Zhao, Michael Zeng
NeurIPS 2024 | May 2024
CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations
Leying Zhang, Yao Qian, Long Zhou, Shujie Liu (shujliu), Dongmei Wang, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Lei He, Sheng Zhao, Michael Zeng
NeurIPS 2024 | April 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model
Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, Furu Wei
March 2024
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
Xiaofei Wang, Manthan Thakker, Zhuo Chen, Naoyuki Kanda, Sefik Emre Eskimez, Sanyuan Chen, Min Tang, Shujie Liu, Jinyu Li, Takuya Yoshioka
August 2023
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei
March 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei
January 2023
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
Ziqiang Zhang, Long Zhou, Junyi Ao, Shujie Liu, Lirong Dai, Jinyu Li, Furu Wei
EMNLP 2022 | October 2022
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Wangyou Zhang, Zhuo Chen, Naoyuki Kanda, Shujie Liu, Jinyu Li, Sefik Emre Eskimez, Takuya Yoshioka, Xiong Xiao, Zhong Meng, Yanmin Qian, Furu Wei
Interspeech 2022 | September 2022
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Naoyuki Kanda, Jinyu Li, Xie Chen, Yu Wu, Yifan Gong
Interspeech 2022 | September 2022
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
arXiv:2209.04974 | September 2022
Factorized Neural Transducer for Efficient Language Model Adaptation
Xie Chen, Zhong Meng, Sarangarajan Parthasarathy, Jinyu Li
ICASSP 2022 | May 2022
Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition
Guoli Ye, Vadim Mazalov, Jinyu Li, Yifan Gong
ICASSP 2022 | May 2022
Continuous Streaming Multi-Talker ASR with Dual-path Transducers
Desh Raj, Liang Lu, Zhuo Chen, Yashesh Gaur, Jinyu Li
ICASSP 2022 | May 2022
Endpoint Detection for Streaming End-to-End Multi-talker ASR
Liang Lu, Jinyu Li, Yifan Gong
ICASSP 2022 | May 2022
Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition
Yiming Wang, Jinyu Li, Heming Wang, Yao Qian, Chengyi Wang, Yu Wu
ICASSP 2022 | May 2022
Self-Supervised Learning for speech recognition with Intermediate layer supervision
Chengyi Wang, Yu Wu, Sanyuan Chen, Shujie Liu, Jinyu Li, Yao Qian, Zhenglu Yang
ICASSP 2022 | May 2022
A Configurable Multilingual Model is All You Need to Recognize All Languages
Long Zhou, Jinyu Li, Eric Sun, Shujie Liu
ICASSP 2022 | May 2022
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu
ICASSP 2022 | May 2022
Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction
Heming Wang, Yao Qian, Xiaofei Wang, Yiming Wang, Chengyi Wang, Shujie Liu, Takuya Yoshioka, Jinyu Li, DeLiang Wang
ICASSP 2022 | May 2022
Continuous Speech Separation with Recurrent Selective Attention Network
Yixuan Zhang, Zhuo Chen, Jian Wu, Takuya Yoshioka, Peidong Wang, Zhong Meng, Jinyu Li
ICASSP 2021 | May 2022
Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
Zhong Meng, Yu Wu, Naoyuki Kanda, Liang Lu, Xie Chen, Guoli Ye, Eric Sun, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Streaming Multi-talker Speech Recognition with Joint Speaker Identification
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer
Liang Lu, Zhong Meng, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Naoyuki Kanda, Yashesh Gaur, Sarangarajan Parthasarathy, Eric Sun, Liang Lu, Xie Chen, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020
Xiong Xiao, Naoyuki Kanda, Zhuo Chen, Tianyan Zhou, Takuya Yoshioka, Sanyuan Chen, Yong Zhao, Gang Liu, Yu Wu, Jian Wu, Shujie Liu, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Streaming end-to-end multi-talker speech recognition
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
IEEE Signal Processing Letter | April 2021, Vol 28
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen, Hakan Erdogan, Zili Huang, Maokui He, Shinji Watanabe, Jun Du, Takuya Yoshioka, Yi Luo, Naoyuki Kanda, Jinyu Li, Scott Wisdom, John R. Hershey
SLT 2021 | January 2021
Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, Yifan Gong
SLT 2021 | January 2021
Speaker Separation Using Speaker Inventories and Estimated Speech
Peidong Wang, Zhuo Chen, DeLiang Wang, Jinyu Li, Yifan Gong
IEEE Transactions on Audio, Speech, and Language Processing | December 2020, Vol 29: pp. 537-546
An End-to-end Architecture of Online Multi-channel Speech Separation
Jian Wu, Zhuo Chen, Jinyu Li, Takuya Yoshioka, Zhili Tan, Ed Lin, Yi Luo , Lei Xie
Interspeech | October 2020
Continuous speech separation: dataset and analysis
Zhuo Chen, Takuya Yoshioka, Liang Lu, Tianyan Zhou, Zhong Meng, Yi Luo, Jian Wu, Xiong Xiao, Jinyu Li
ICASSP | April 2020
Speech Separation Using Speaker Inventory
Peidong Wang, Zhuo Chen, Xiong Xiao, Zhong Meng, Takuya Yoshioka, Tianyan Zhou, Liang Lu, Jinyu Li
Automatic Speech Recognition and Understanding Workshop | November 2019
Multi-Channel Overlapped Speech Recognition with Location Guided Speech Extraction Network
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
SLT 2018 | December 2018
MULTI-CHANNEL OVERLAPPED SPEECH RECOGNITION WITH LOCATION GUIDED SPEECH EXTRACTION NETWORK
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
Spoken Language Technology | December 2018
Efficient Integration of Fixed Beamformers and Speech Separation Networks for Multi-Channel Far-Field Speech Separation
Zhuo Chen, Takuya Yoshioka, Xiong Xiao, Jinyu Li, Michael L. Seltzer, Yifan Gong
ICASSP 2018 | April 2018
Cracking the cocktail party problem by multi-beam deep attractor network
Zhuo Chen, Jinyu Li, Xiong Xiao, Takuya Yoshioka, Huaming Wang, Zhenghao Wang, Yifan Gong
ASRU 2018 | December 2017
-
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Sequence-level self-learning with multi-task learning framework
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
October 2020
Semantic Mask for Transformer based End-to-End Speech Recognition
Chengyi Wang, Yu Wu, yujiao du, Jinyu Li, Shujie Liu, Liang Lu, Shuo Ren, Guoli Ye, Sheng Zhao, Ming Zhou
Interspeech | October 2020
High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model
Jinyu Li, Rui Zhao, Eric Sun, Jeremy Wong, Amit Das, Zhong Meng, Yifan Gong
ICASSP | April 2020
Feature Learning in Deep Neural Networks – Studies on Speech Recognition
Dong Yu, Mike Seltzer, Jinyu Li, Jui-Ting Huang, Frank Seide
International Conference on Learning Representations | May 2013
Calibration of Confidence Measures In Speech Recognition
Dong Yu, Jinyu Li, Li Deng
IEEE Transactions on Audio, Speech, and Language Processing | November 2011
Towards High-Accuracy Low-Cost Noisy Robust Speech Recognition Exploiting Structured Model
Jinyu Li, Dong Yu, Li Deng, Yifan Gong
ICML Workshop 2011 | June 2011
Unscented Transform with Online Distortion Estimation for HMM Adaptation
Jinyu Li, Dong Yu, Li Deng, Yifan Gong
Interspeech 2010 | September 2010
Word Confidence Calibration Using A Maximum Entropy Model With Constraints On Confidence and Word Distributions
Dong Yu, Shizhen Wang, Jinyu Li, Li Deng
March 2010
A unified framework of HMM adaptation with joint compensation of additive and convolutive distortions
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Computer Speech and Language | January 2009, Vol 23: pp. 389-405
Adaptation of Compressed HMM Parameters for Resource-Constrained Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Jian Wu, Yifan Gong, Alex Acero
April 2008
HMM Adaptation Using a Phase-Sensitive Acoustic Distortion Model for Environment-Robust Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
April 2008
-
Sequence-level self-learning with multi-task learning framework
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
October 2020
Factorized adaptation for deep neural network
Jinyu Li, Jui-Ting Huang, Yifan Gong
ICASSP | January 2014
-
Simplifying Long Short-term Memory Acoustic Models for Fast Training and Decoding
Yajie Miao, Jinyu Li, Yongqiang Wang, Shi-Xiong Zhang, Yifan Gong
ICASSP | March 2016
ICASSP
Low-rank Plus Diagonal Adaption for Deep Neural Networks
Yong Zhao, Jinyu Li, Yifan Gong
ICASSP | March 2016
ICASSP
Learning Small-Size DNN with Output-Distribution-Based Criteria
Jinyu Li, Rui Zhao, Jui-Ting Huang, Yifan Gong
Interspeech | September 2014
Interspeech
Cross-language Knowledge Transfer using Multilingual Deep Neural Network with Shared Hidden Layers
Jui-Ting Huang, Jinyu Li, Dong Yu, Li Deng, Yifan Gong
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) | May 2013
Model-Based Margin Estimation for Hidden Markov Model Learning and Generalization
Sabato Marco Siniscalchi, Jinyu Li, Chin-Hui Lee
IET Signal Processing | January 2013
Shrinkage Model Adaptation in Automatic Speech Recognition
Jinyu Li, Yu Tsao, Chin-Hui Lee
Interspeech | January 2010
Ensemble speaker and speaking environment modeling approach with advanced online estimation process
Yu Tsao, Jinyu Li, Chin-Hui Lee
Proc. ICASSP | January 2009
Proc. ICASSP
HMM adaptation using a phase-sensitive acoustic distortion model for environment-robust speech recognition
Jinyu Li, Li Deng, Dong Yu, Alex Acero
Proc. ICASSP | January 2008
Proc. ICASSP
Approximate Test Risk Bound Minimization through Soft Margin Estimation
IEEE Trans. on Audio, Speech, and Language Proc | January 2007
A study on soft margin estimation for LVCSR
Proc. IEEE Automatic Speech Recognition and Understanding | January 2007
-
LSTM Time and Frequency Recurrence for Automatic Speech Recognition
Jinyu Li, Abdelrahman Mohamed, Geoffrey Zweig, Yifan Gong
IEEE Automatic Speech Recognition and Understanding Workshop | December 2015
Singular Value Decomposition Based Low-footprint Speaker Adaptation and Personalization for Deep Neural Network
Jian Xue, Jinyu Li, Dong Yu, Mike Seltzer, Yifan Gong
ICASSP | January 2014
-
Compensation with Prior Knowledge
Jinyu Li, Li Deng, Yifan Gong
Robust Automatic Speech Recognition: A Bridge to Practical Applications | Published by Elsevier | 2015 | Robust Automatic Speech Recognition: A Bridge to Practical Applications edition
SVD-based Universal DNN Modeling for Multiple Scenarios
Changliang Liu, Jinyu Li, Yifan Gong
September 2015
-
Investigating Online Low-Footprint Speaker Adaptation Using Generalized Linear Regression and Click-Through Data
Yong Zhao, Jinyu Li, Jian Xue, Yifan Gong
ICASSP | April 2015
ICASSP
Feature Space Maximum A Posteriori Linear Regression for Adaptation of Deep Neural Networks
Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Chao Weng, Chin-Hui Lee
Interspeech | September 2014
Interspeech
-
A novelsearch algorithm for LSF VQ
Proc. ICSLP | January 2000
Proc. ICSLP
Variable-Component Deep Neural Network for Robust Speech Recognition
Interspeech | September 2014
Interspeech
LASSO environment model combination for robust speech recognition
Proc. ICASSP | January 2012
Maximum likelihood adaptation of histogram equalization with constraint for robust speech recognition
Proc. ICASSP | January 2011
Proc. ICASSP
Feature normalization using structured full transforms for robust speech recognition
Proc. Interspeech | January 2011
Proc. Interspeech
Unscented transform with online distortion estimation for HMM adaptation
Proc. Interspeech | January 2010
A study on soft margin estimation of linear regression parameters for speaker adaptation
Proc. Interspeech | January 2009
Soft margin estimation on improving environment structures for ensemble speaker and speaking environment modeling
ACM Proc. the 3rd International Universal Communication Symposium | January 2009
ACM Proc. the 3rd International Universal Communication Symposium
SOFT MARGIN ESTIMATION FOR AUTOMATIC SPEECH RECOGNITION
December 2008
December 2008
On a Generalization of Margin-Based Discriminative Training to Robust Speech Recognition
Proc. Interspeech | January 2008
Soft Margin Estimation with Various Separation Levels for LVCSR
Proc. Interspeech | January 2008
Approximate test risk minimization through soft margin estimation
Proc. ICASSP | January 2007
Detection-Based ASR in the Automatic Speech Attribute Transcription Project
Proc. Interspeech | January 2007
Proc. Interspeech
Soft Margin Feature Extraction for Automatic Speech Recognition
Proc. Interspeech | January 2007
Soft Margin Estimation of Hidden Markov Model Parameters (best student paper)
Proc. Interspeech | January 2006
Best Student Paper
Application of EαNets to feature recognitionof articulation manner in knowledge-based automatic speech recognition
Lecture Notes in Computer Science | January 2006
Language recognition based on score distribution feature vectors and discriminative classifier fusion
Proc. IEEE Odyssey Workshop on Speaker and Language Recognition | January 2006
A study on lattice rescoring with knowledge scores for automatic speech recognition
Proc. Interspeech | January 2006
Proc. Interspeech
A study on knowledge source integration for candidate rescoring in automatic speech recognition
Proc. ICASSP | January 2005
A study on separation between acoustic models and its applications
Proc. Interspeech | January 2005
On Designing and Evaluating Speech Event Detectors
Proc. Interspeech | January 2005
Dimensionality reduction using MCE-optimized LDA transformation
Proc. ICASSP | January 2004
A complexity reduction of ETSI advanced front-end for DSR
Proc. ICASSP | January 2004
-
Investigations on Hessian-Free Optimization for Cross-Entropy Training of Deep Neural Networks
Simon Wiesler, Jinyu Li, Jian Xue
Interspeech | January 2013
Interspeech
LASSO model adaptation for automatic speech recognition
Jinyu Li, Ming Yuan, Chin-Hui Lee
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing | January 2011
Publications by Type
-
A novelsearch algorithm for LSF VQ
Proc. ICSLP | January 2000
Proc. ICSLP
Boosting Large Language Model for Speech Synthesis: An Empirical Study
Hongkun Hao, Long Zhou, Shujie Liu, Jinyu Li, Shujie Hu, Rui Wang, Furu Wei
ICASSP | April 2025
V2SFlow: Video-to-Speech Generation with Speech Decomposition and Rectified Flow
Jeongsoo Choi, Ji-Hoon Kim, Jinyu Li, Joon Son Chung, Shujie Liu
ICASSP | April 2025
Target word activity detector: An approach to obtain ASR word boundaries without lexicon
Sunit Sivasankaran, Eric Sun, Jinyu Li, Yan Huang, Jing Pan
ICASSP | April 2025
Investigating neural audio codecs for speech language model-based speech generation
Jiaqi Li , Dongmei Wang, Xiaofei Wang, Yao Qian, Long Zhou, Shujie Liu, Midia Yousefi, Canrun Li , Chung-Hsien Tsai, Zhen Xiao, Yanqing Liu, Junkun Chen, Sheng Zhao, Jinyu Li, Zhizheng Wu , Michael Zeng
SLT Workshop | December 2024
CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation
Rui Zhao, Jinyu Li, Ruchao Fan, Matt Post
SLT Workshop | December 2024
Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech
Haibin Wu, Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Daniel Tompkins, Chung-Hsien Tsai , Canrun Li, Zhen Xiao, Sheng Zhao , Jinyu Li, Naoyuki Kanda
SLT Workshop | December 2024
Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translation
Peidong Wang, Jian Xue, Jinyu Li, Junkun Chen, Aswin Shanmugam Subramanian
Interspeech 2024 | September 2024
An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS
Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Hemin Yang, Zirun Zhu, Min Tang, Yufei Xia, Jinzhu Li, sheng zhao , Jinyu Li, Naoyuki Kanda
Interspeech 2024 | September 2024
COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning
Jing Pan, Jian Wu, Yashesh Gaur , Sunit Sivasankaran, Zhuo Chen , Shujie Liu, Jinyu Li
Interspeech 2024 | September 2024
Total-Duration-Aware Duration Modeling for Text-to-Speech Systems
Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Chung-Hsien Tsai, Canrun Li, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Jinyu Li, Sheng Zhao, Naoyuki Kanda
Interspeech 2024 | September 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiang-Yang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, Sheng Zhao
ICML | June 2024
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
Chenyang Le, Yao Qian, Dongmei Wang, Long Zhou, Shujie Liu, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Sheng Zhao, Michael Zeng
NeurIPS 2024 | May 2024
CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations
Leying Zhang, Yao Qian, Long Zhou, Shujie Liu (shujliu), Dongmei Wang, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Lei He, Sheng Zhao, Michael Zeng
NeurIPS 2024 | April 2024
ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
Yiming Wang, Jinyu Li
ICASSP | April 2024
t-SOT FNT: streaming multi-talker ASR with text-only domain adaptation capability
Jian Wu, Naoyuki Kanda, Takuya Yoshioka, Rui Zhao, Zhuo Chen, Jinyu Li
ICASSP | April 2024
Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation
Sara Pap, Peidong Wang, Junkun Chen, Jian Xue, Naoyuki Kanda, Jinyu Li, Yashesh Gaur
ICASSP | April 2024
DiariST: Streaming Speech Translation with Speaker Diarization
Mu Yang, Naoyuki Kanda, Xiaofei Wang, Junkun Chen, Peidong Wang, Jian Xue, Jinyu Li, Takuya Yoshioka
ICASSP | April 2024
Building High-accuracy Multilingual ASR with Gated Language Experts and Curriculum Training
Eric Sun, Jinyu Li, Yuxuan Hu, Yimeng Zhu, Long Zhou, Jian Xue, Peidong Wang, Linquan Liu, Shujie Liu, Edward Lin, Yifan Gong
December 2023
Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition
Yuang Li, Yu Wu, Jinyu Li, Shujie Liu
Workshop of Automatic Speech Recognition and Understanding | December 2023
On decoder-only architecture for speech-to-text and large language model integration
Jian Wu, Yashesh Gaur, Zhuo Chen, Long Zhou, Yimeng Zhu, Tianrui Wang, Jinyu Li, Shujie Liu, Bo Ren, Linquan Liu, Yu Wu
Workshop of Automatic Speech Recognition and Understanding | December 2023
Improving Stability in Simultaneous Speech Translation: A Revision-Controllable Decoding Approach
Junkun Chen, Jian Xue, Peidong Wang, Jing Pan, Jinyu Li
Workshop of Automatic Speech Recognition and Understanding | December 2023
Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments
Sara Papi, Peidong Wang, Junkun Chen, Jian Xue, Jinyu Li, Yashesh Gaur
Workshop of Automatic Speech Recognition and Understanding | December 2023
A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capability
Jian Xue, Peidong Wang, Jinyu Li, Eric Sun
Workshop of Automatic Speech Recognition and Understanding | December 2023
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
Accelerating Transducers through Adjacent Token Merging
Yuang Li, Yu Wu, Jinyu Li, Shujie Liu
Interspeech | August 2023
LAMASSU: A Streaming Language-Agnostic Multilingual Speech Recognition and Translation Model Using Neural Transducers
Peidong Wang, Eric Sun, Jian Xue, Yu Wu, Long Zhou, Yashesh Gaur, Shujie Liu, Jinyu Li
Interspeech | August 2023
Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentation
Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Sheng zhao
ICASSP | June 2023
Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition models
Rui Zhao, Jian Xue, Partha Parthasarathy, Veljko Miljanic, Jinyu Li
ICASSP | June 2023
Speaker Change Detection for Transformer Transducer ASR
Jian Wu, Zhuo Chen, Min Hu, Xiong Xiao, Jinyu Li
June 2023
Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation
Kun Wei, Long Zhou, Ziqiang Zhang, Liping Chen, Shujie Liu, Lei He, Jinyu Li, Furu Wei
ICASSP | June 2023
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
ICASSP | June 2023
Simulating realistic speech overlaps improves multi-talker ASR
Muqiao Yang, Naoyuki Kanda, Xiaofei Wang, Jian Wu, Sunit Sivasankaran, Zhuo Chen, Jinyu Li, Takuya Yoshioka
ICASSP | June 2023
Speech separation with large-scale self-supervised learning
Zhuo Chen, Naoyuki Kanda, Jian Wu, Yu Wu, Xiaofei Wang, Takuya Yoshioka, Jinyu Li, Sunit Sivasankaran, Sefik Emre Eskimez
ICASSP | June 2023
LongFNT: Long-form Speech Recognition with Factorized Neural Transducer
Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian
ICASSP | June 2023
Self-supervised learning with bi-label masked speech prediction for streaming multi-talker speech recognition
Zili Huang, Zhuo Chen, Naoyuki Kanda, Jian Wu, Yiming Wang, Jinyu Li, Takuya Yoshioka, Xiaofei Wang, Peidong Wang
ICASSP | June 2023
CTCBERT: Advancing Hidden-unit BERT with CTC Objectives
Ruchao Fan, Yiming Wang, Yashesh Gaur, Jinyu Li
ICASSP | June 2023
STREAMING, FAST AND ACCURATE ON-DEVICE INVERSE TEXT NORMALIZATION FOR AUTOMATIC SPEECH RECOGNITION
Yashesh Gaur, Nick Kibre, Jian Xue, Kangyuan Shu, Yuhui Wang, Issac Alphanso, Jinyu Li, Yifan Gong
Spoken Language Technology Workshop | January 2023
Advancing end-to-end automatic speech recognition and beyond
International Symposium on Chinese Spoken Language Processing | December 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
Ziqiang Zhang, Long Zhou, Junyi Ao, Shujie Liu, Lirong Dai, Jinyu Li, Furu Wei
EMNLP 2022 | October 2022
Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Zhuo Chen, Gang Liu, Jinyu Li, Jian Wu, Xiangzhan Yu, Furu Wei
Interspeech | September 2022
Large-Scale Streaming End-to-End Speech Translation with Neural Transducers
Jian Xue, Peidong Wang, Jinyu Li, Matt Post, Yashesh Gaur
Interspeech | September 2022
Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Data
Junyi Ao, Ziqiang Zhang, Long Zhou, Shujie Liu, Haizhou Li, Tom Ko, Lirong Dai, Jinyu Li, Yao Qian, Furu Wei
Interspeech | September 2022
Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training
Chengyi Wang, Yiming Wang, Yu Wu, Sanyuan Chen, Jinyu Li, Shujie Liu, Furu Wei
Interspeech | September 2022
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Wangyou Zhang, Zhuo Chen, Naoyuki Kanda, Shujie Liu, Jinyu Li, Sefik Emre Eskimez, Takuya Yoshioka, Xiong Xiao, Zhong Meng, Yanmin Qian, Furu Wei
Interspeech 2022 | September 2022
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Interspeech 2022 | September 2022
Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Naoyuki Kanda, Jinyu Li, Xie Chen, Yu Wu, Yifan Gong
Interspeech 2022 | September 2022
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu
ICASSP | May 2022
Factorized Neural Transducer for Efficient Language Model Adaptation
Xie Chen, Zhong Meng, Sarangarajan Parthasarathy, Jinyu Li
ICASSP 2022 | May 2022
Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition
Guoli Ye, Vadim Mazalov, Jinyu Li, Yifan Gong
ICASSP 2022 | May 2022
Continuous Streaming Multi-Talker ASR with Dual-path Transducers
Desh Raj, Liang Lu, Zhuo Chen, Yashesh Gaur, Jinyu Li
ICASSP 2022 | May 2022
Endpoint Detection for Streaming End-to-End Multi-talker ASR
Liang Lu, Jinyu Li, Yifan Gong
ICASSP 2022 | May 2022
Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition
Yiming Wang, Jinyu Li, Heming Wang, Yao Qian, Chengyi Wang, Yu Wu
ICASSP 2022 | May 2022
Self-Supervised Learning for speech recognition with Intermediate layer supervision
Chengyi Wang, Yu Wu, Sanyuan Chen, Shujie Liu, Jinyu Li, Yao Qian, Zhenglu Yang
ICASSP 2022 | May 2022
A Configurable Multilingual Model is All You Need to Recognize All Languages
Long Zhou, Jinyu Li, Eric Sun, Shujie Liu
ICASSP 2022 | May 2022
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu
ICASSP 2022 | May 2022
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
ACL 2022 | May 2022
Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction
Heming Wang, Yao Qian, Xiaofei Wang, Yiming Wang, Chengyi Wang, Shujie Liu, Takuya Yoshioka, Jinyu Li, DeLiang Wang
ICASSP 2022 | May 2022
Continuous Speech Separation with Recurrent Selective Attention Network
Yixuan Zhang, Zhuo Chen, Jian Wu, Takuya Yoshioka, Peidong Wang, Zhong Meng, Jinyu Li
ICASSP 2021 | May 2022
On Addressing Practical Challenges For RNN-Transducer
Rui Zhao, Jian Xue, Jinyu Li, Wenning Wei, Lei He, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2021
Rapid Speaker Adaptation for Conformer Transducer: Attention and Bias are All You Need
Yan Huang, Guoli Ye, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Multiple Softmax Architecture for Streaming Multilingual End-to-End ASR Systems
Vikas Joshi, Amit Das, Eric Sun, Rupesh Mehta, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Improving RNN-T for Domain Scaling Using Semi-Supervised Training with Neural TTS
Yan Deng, Rui Zhao, Zhong Meng, Xie Chen, Bing Liu, Jinyu Li, Yifan Gong, Lei He
Interspeech 2021 | August 2021
Improving Multilingual Transformer Transducer Models by Reducing Language Confusions
Eric Sun, Jinyu Li, Zhong Meng, Yu Wu, Jian Xue, Shujie Liu, Yifan Gong
Interspeech 2021 | August 2021
Investigation of Practical Aspects of Single Channel Speech Separation for ASR
Jian Wu, Zhuo Chen, Sanyuan Chen, Yu Wu, Takuya Yoshioka, Naoyuki Kanda, Shujie Liu, Jinyu Li
Interspeech 2021 | August 2021
Ultra Fast Speech Separation Model with Teacher Student Learning
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Takuya Yoshioka, Shujie Liu, Jinyu Li, Xiangzhan Yu
Interspeech 2021 | August 2021
A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems
Xiaoqiang Wang, Yanqing Liu, Sheng Zhao, Jinyu Li
Interspeech 2021 | August 2021
Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
Zhong Meng, Yu Wu, Naoyuki Kanda, Liang Lu, Xie Chen, Guoli Ye, Eric Sun, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Streaming Multi-talker Speech Recognition with Joint Speaker Identification
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer
Liang Lu, Zhong Meng, Naoyuki Kanda, Jinyu Li, Yifan Gong
Interspeech 2021 | August 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Naoyuki Kanda, Yashesh Gaur, Sarangarajan Parthasarathy, Eric Sun, Liang Lu, Xie Chen, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020
Xiong Xiao, Naoyuki Kanda, Zhuo Chen, Tianyan Zhou, Takuya Yoshioka, Sanyuan Chen, Yong Zhao, Gang Liu, Yu Wu, Jian Wu, Shujie Liu, Jinyu Li, Yifan Gong
ICASSP 2021 | June 2021
Developing real-time streaming transformer transducer for speech recognition on large-scale dataset
Xie Chen, Yu Wu, Zhenghao Wang, Shujie Liu, Jinyu Li
ICASSP | June 2021
Continuous speech separation with Conformer
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Jinyu Li, Takuya Yoshioka, Chengyi Wang, Shujie Liu, Ming Zhou
ICASSP | June 2021
Don’t shoot butterfly with rifles: multi-channel continuous speech separation with early exit transformer
Sanyuan Chen, Yu Wu, Zhuo Chen, Takuya Yoshioka, Shujie Liu, Jinyu Li, Xiangzhan Yu
ICASSP | June 2021
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen, Hakan Erdogan, Zili Huang, Maokui He, Shinji Watanabe, Jun Du, Takuya Yoshioka, Yi Luo, Naoyuki Kanda, Jinyu Li, Scott Wisdom, John R. Hershey
SLT 2021 | January 2021
Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, Yifan Gong
SLT 2021 | January 2021
Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations
Shahram Ghorbani, Yashesh Gaur, Yu Shi, Jinyu Li
Spoken Language Technology Workshop | January 2021
Dual-Path RNN For Long Recording Speech Separation
Chenda Li, Yi Luo, Cong Han, Jinyu Li, Takuya Yoshioka, Tianyan Zhou, Marc Delcroix, Keisuke Kinoshita, Christoph Boeddeker, Yanmin Qian, Shinji Watanabe, Zhuo Chen
Spoken Language Technology Workshop | January 2021
Sequence-Level Self-Learning with Multiple Hypotheses
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
Interspeech | October 2020
On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Jinyu Li, Yu Wu, Yashesh Gaur, Chengyi Wang, Rui Zhao , Shujie Liu
Interspeech | October 2020
Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability
Jinyu Li, Rui Zhao, Zhong Meng, Yanqing Liu, Wenning Wei, Sarangarajan Parthasarathy, Vadim Mazalov, Zhenghao Wang, Lei He, Sheng Zhao, Yifan Gong
Interspeech | October 2020
Exploring Transformers for Large-Scale Speech Recognition
Liang Lu, Changliang Liu, Jinyu Li, Yifan Gong
Interspeech | October 2020
An End-to-end Architecture of Online Multi-channel Speech Separation
Jian Wu, Zhuo Chen, Jinyu Li, Takuya Yoshioka, Zhili Tan, Ed Lin, Yi Luo , Lei Xie
Interspeech | October 2020
Combination of end-to-end and hybrid models for speech recognition
Jeremy Wong, Yashesh Gaur, Rui Zhao, Liang Lu, Eric Sun, Jinyu Li, Yifan Gong
Interspeech | October 2020
Low Latency End-to-End Streaming Speech Recognition with a Scout Network
Chengyi Wang, Yu Wu, Liang Lu, Shujie Liu, Jinyu Li, Guoli Ye, Ming Zhou
Interspeech | October 2020
Semantic Mask for Transformer based End-to-End Speech Recognition
Chengyi Wang, Yu Wu, yujiao du, Jinyu Li, Shujie Liu, Liang Lu, Shuo Ren, Guoli Ye, Sheng Zhao, Ming Zhou
Interspeech | October 2020
Transfer Learning Approaches for Streaming End-to-End Speech Recognition System
Vikas Joshi, Rui Zhao, Rupesh R. Mehta, Kshitiz Kumar, Jinyu Li
Interspeech | October 2020
Rapid RNN-T Adaptation Using Personalized Speech Synthesis and Neural Language Generator
Yan Huang, Jinyu Li, Lei He, Wenning Wei, William Gale, Yifan Gong
Interspeech | October 2020
High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model
Jinyu Li, Rui Zhao, Eric Sun, Jeremy Wong, Amit Das, Zhong Meng, Yifan Gong
ICASSP | April 2020
Minimum latency training strategies for streaming sequence-to-sequence ASR
Hirofumi Inaguma, Yashesh Gaur, Liang Lu, Jinyu Li, Yifan Gong
ICASSP | April 2020
Exploring pre-training with alignments for RNN transducer based end-to-end speech recognition
Hu Hu, Rui Zhao, Jinyu Li, Liang Lu, Yifan Gong
ICASSP | April 2020
Continuous speech separation: dataset and analysis
Zhuo Chen, Takuya Yoshioka, Liang Lu, Tianyan Zhou, Zhong Meng, Yi Luo, Jian Wu, Xiong Xiao, Jinyu Li
ICASSP | April 2020
Using personalized speech synthesis and neural language generator for rapid speaker adaptation
Yan Huang, Lei He, Wenning Wei, William Gale, Jinyu Li, Yifan Gong
ICASSP | April 2020
L-Vector: Neural Label Embedding for Domain Adaptation
Zhong Meng, Hu Hu, Jinyu Li, Changliang Liu, Yan Huang, Yifan Gong, Chin-Hui Lee
ICASSP | April 2020
CNN with phonetic attention for text-independent speaker verification
Tianyan Zhou, yong zhao, Jinyu Li, Yifan Gong, Jian Wu
Automatic Speech Recognition and Understanding Workshop | December 2019
Character-aware attention-based end-to-end speech recognition
Zhong Meng, Yashesh Gaur, Jinyu Li, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Domain adaptation via teacher-student learning for end-to-end speech recognition
Zhong Meng, Jinyu Li, Yashesh Gaur, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Improving RNN Transducer Modeling for End-to-End Speech Recognition
Jinyu Li, Rui Zhao, Hu Hu, Yifan Gong
Automatic Speech Recognition and Understanding Workshop | December 2019
Speech Separation Using Speaker Inventory
Peidong Wang, Zhuo Chen, Xiong Xiao, Zhong Meng, Takuya Yoshioka, Tianyan Zhou, Liang Lu, Jinyu Li
Automatic Speech Recognition and Understanding Workshop | November 2019
Speaker Adaptation for Attention-Based End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Jinyu Li, Yifan Gong
Interspeech | September 2019
Acoustic-to-Phrase Models for Speech Recognition
Yashesh Gaur, Jinyu Li, Zhong Meng, Yifan Gong
Interspeech | September 2019
Layer Trajectory BLSTM
Eric Sun, Jinyu Li, Yifan Gong
Interspeech | September 2019
Improving Layer Trajectory LSTM With Future Context Frames
Jinyu Li, Liang Lu, Changliang Liu, Yifan Gong
ICASSP | May 2019
Adversarial Speaker Verification
Zhong Meng, Yong Zhao, Jinyu Li, Yifan Gong
ICASSP | May 2019
Adversarial Speaker Adaptation
Zhong Meng, Jinyu Li, Yifan Gong
ICASSP | May 2019
Towards Code-Switching ASR for End-to-End CTC Models
Ke Li, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong
ICASSP | May 2019
Attentive Adversarial Learning for Domain-Invariant Training
Zhong Meng, Jinyu Li, Yifan Gong
ICASSP | May 2019
Conditional Teacher-Student Learning
Zhong Meng, Jinyu Li, Yong Zhao, Yifan Gong
ICASSP | May 2019
Universal Acoustic Modeling Using Neural Mixture Models
Amit Das, Jinyu Li, Changliang Liu, Yifan Gong
ICASSP | May 2019
Multi-Channel Overlapped Speech Recognition with Location Guided Speech Extraction Network
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
SLT 2018 | December 2018
Exploring layer trajectory LSTM with depth processing units and attention
Jinyu Li, Liang Lu, Changliang Liu, Yifan Gong
Spoken Language Technology | December 2018
MULTI-CHANNEL OVERLAPPED SPEECH RECOGNITION WITH LOCATION GUIDED SPEECH EXTRACTION NETWORK
Zhuo Chen, Xiong Xiao, Takuya Yoshioka, Hakan Erdogan, Jinyu Li, Yifan Gong
Spoken Language Technology | December 2018
Speaker Adaptation for End-to-End CTC Models
Ke Li, Jinyu Li, yong zhao, Yifan Gong (ygong), Kshitiz Kumar
Spoken Language Technology | December 2018
Improved training for online end-to-end speech recognition systems
Suyoun Kim, Michael L. Seltzer, Jinyu Li, Rui Zhao
Interspeech 2018 | September 2018
Adversarial Feature-Mapping for Speech Enhancement
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang Juang
Interspeech 2018 | September 2018
Cycle-Consistent Speech Enhancement
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang Juang
Interspeech 2018 | September 2018
Layer Trajectory LSTM
Jinyu Li, Changliang Liu, Yifan Gong
Interspeech 2018 | September 2018
Advancing Connectionist Temporal Classification With Attention Modeling
Amit Das, Jinyu Li, Rui Zhao, Yifan Gong
ICASSP | April 2018
Advancing Acoustic-to-Word CTC Model
Jinyu Li, Guoli Ye, Amit Das, Rui Zhao, Yifan Gong
ICASSP | April 2018
Developing Far-Field Speaker System Via Teacher-Student Learning
Jinyu Li, Rui Zhao, Zhuo Chen, Changliang Liu, Xiong Xiao, Guoli Ye, Yifan Gong
ICASSP | April 2018
Efficient Integration of Fixed Beamformers and Speech Separation Networks for Multi-Channel Far-Field Speech Separation
Zhuo Chen, Takuya Yoshioka, Xiong Xiao, Jinyu Li, Michael L. Seltzer, Yifan Gong
ICASSP 2018 | April 2018
DOMAIN AND SPEAKER ADAPTATION FOR CORTANA SPEECH RECOGNITION
Yong Zhao, Jinyu Li, Shixiong Zhang, Liping Chen, Yifan Gong
ICASSP | April 2018
Efficient integration of fixed beamformers and speech separation networks for multi-channel far-field speech separation
Zhuo Chen, Takuya Yoshioka, Xiong Xiao, Jinyu Li, Michael L. Seltzer, Yifan Gong
ICASSP | April 2018
Speaker-Invariant Training via Adversarial Learning
Zhong Meng, Jinyu Li, Zhuo Chen, Yong Zhao, Vadim Mazalov, Yifan Gong, Biing-Hwang (Fred) Juang
ICASSP | April 2018
Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang (Fred) Juang
ICASSP | April 2018
Acoustic-To-Word Model Without OOV
Jinyu Li, Guoli Ye, Rui Zhao, Jasha Droppo, Yifan Gong
ASRU | December 2017
Cracking the cocktail party problem by multi-beam deep attractor network
Zhuo Chen, Jinyu Li, Xiong Xiao, Takuya Yoshioka, Huaming Wang, Zhenghao Wang, Yifan Gong
ASRU 2018 | December 2017
Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
Zhong Meng, Zhuo Chen, Vadim Mazalov, Jinyu Li, Yifan Gong
ASRU | December 2017
Improving mask learning based speech enhancement system with restoration layers and residual connection
Zhuo Chen, Yan Huang, Jinyu Li, Yifan Gong
Interspeech | August 2017
Large-Scale Domain Adaptation via Teacher-Student Learning
Jinyu Li, Michael Seltzer, Xi Wang, Rui Zhao, Yifan Gong
Interspeech | August 2017
Extended low-rank plus diagonal adaptation for deep and recurrent neural networks
yong zhao, Jinyu Li, kshitiz kumar, Yifan Gong
ICASSP | April 2017
Improved Cepstra Minimum-Mean-Square-Error Noise Reduction Algorithm For Robust Speech Recognition
Jinyu Li, Yan Huang, Yifan Gong
ICASSP | April 2017
End-to-End Attention Based Text-Dependent Speaker Verification
Shixiong Zhang, Zhuo Chen, Yong Zhao, Jinyu Li, Yifan Gong
SLT workshop | December 2016
Deep Convolutional Neural Networks with Layer-wise Context Expansion and Attention
Dong Yu, Wayne Xiong, Jasha Droppo, Andreas Stolcke, Guoli Ye, Jinyu Li, Geoffrey Zweig
Proc. Interspeech | September 2016
EXPLORING MULTIDIMENSIONAL LSTMS FOR LARGE VOCABULARY ASR
Jinyu Li, Abdelrahman Mohamed, Geoffrey Zweig, Yifan Gong
ICASSP | March 2016
Simplifying Long Short-term Memory Acoustic Models for Fast Training and Decoding
Yajie Miao, Jinyu Li, Yongqiang Wang, Shi-Xiong Zhang, Yifan Gong
ICASSP | March 2016
ICASSP
Recurrent Support Vector Machines for Speech Recognition
Shi-Xiong Zhang, Rui Zhao, Chaojun Liu, Jinyu Li, Yifan Gong
ICASSP | March 2016
ICASSP
Low-rank Plus Diagonal Adaption for Deep Neural Networks
Yong Zhao, Jinyu Li, Yifan Gong
ICASSP | March 2016
ICASSP
LSTM Time and Frequency Recurrence for Automatic Speech Recognition
Jinyu Li, Abdelrahman Mohamed, Geoffrey Zweig, Yifan Gong
IEEE Automatic Speech Recognition and Understanding Workshop | December 2015
Rapid Adaptation for Deep Neural Networks through Multi-Task Learning
Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Ji Wu, Chin-Hui Lee
Interspeech | September 2015
Maximum a Posteriori Adaptation of Network Parameters in Deep Models
Zheng Huang, Sabato Marco Siniscalchi, I-Fan Chen, Jinyu Li, Jiadong Wu, Chin-Hui Lee
September 2015
SVD-based Universal DNN Modeling for Multiple Scenarios
Changliang Liu, Jinyu Li, Yifan Gong
September 2015
Small-footprint high-performance deep neural network-based speech recognition using split-VQ
Yongqiang Wang, Jinyu Li, Yifan Gong
ICASSP | April 2015
Investigating Online Low-Footprint Speaker Adaptation Using Generalized Linear Regression and Click-Through Data
Yong Zhao, Jinyu Li, Jian Xue, Yifan Gong
ICASSP | April 2015
ICASSP
An Analysis of Convolutional Neural Networks for Speech Recognition
Jui-Ting Huang, Jinyu Li, Yifan Gong
ICASSP | April 2015
variable-activation and variable-input deep neural network for robust speech recognition
Rui Zhao, Jinyu Li, Yifan Gong
IEEE SLT workshop | December 2014
Variable-Component Deep Neural Network for Robust Speech Recognition
Interspeech | September 2014
Interspeech
Learning Small-Size DNN with Output-Distribution-Based Criteria
Jinyu Li, Rui Zhao, Jui-Ting Huang, Yifan Gong
Interspeech | September 2014
Interspeech
Beyond Cross-entropy: Towards Better Frame-level Objective Functions For Deep Neural Network Training In Automatic Speech Recognition
Zhen Huang, Jinyu Li, Chao Weng, Chin-Hui Lee
Interspeech | September 2014
Interspeech
Feature Space Maximum A Posteriori Linear Regression for Adaptation of Deep Neural Networks
Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Chao Weng, Chin-Hui Lee
Interspeech | September 2014
Interspeech
Investigation of Maxout Networks For Speech Recognition
Pawel Swietojanski, Jinyu Li, Jui-Ting Huang
ICASSP | January 2014
Factorized adaptation for deep neural network
Jinyu Li, Jui-Ting Huang, Yifan Gong
ICASSP | January 2014
Singular Value Decomposition Based Low-footprint Speaker Adaptation and Personalization for Deep Neural Network
Jian Xue, Jinyu Li, Dong Yu, Mike Seltzer, Yifan Gong
ICASSP | January 2014
Feature Compensation Using Linear Combination of Speaker and Environment Dependent Correction Vectors
Xiong Xiao, Jinyu Li, Eng Siong Chng, Haizhou Li
ICASSP | January 2014
Recent Advances in Deep Learning for Speech Research at Microsoft
Li Deng, Jinyu Li, Jui-Ting Huang, Kaisheng Yao, Dong Yu, Frank Seide, Mike Seltzer, Geoff Zweig, Xiaodong He, Jason Williams, Yifan Gong, Alex Acero
May 2013
Cross-language Knowledge Transfer using Multilingual Deep Neural Network with Shared Hidden Layers
Jui-Ting Huang, Jinyu Li, Dong Yu, Li Deng, Yifan Gong
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) | May 2013
Feature Learning in Deep Neural Networks – Studies on Speech Recognition
Dong Yu, Mike Seltzer, Jinyu Li, Jui-Ting Huang, Frank Seide
International Conference on Learning Representations | May 2013
Investigations on Hessian-Free Optimization for Cross-Entropy Training of Deep Neural Networks
Simon Wiesler, Jinyu Li, Jian Xue
Interspeech | January 2013
Interspeech
Restructuring of Deep Neural Network Acoustic Models with Singular Value Decomposition
Jian Xue, Jinyu Li, Yifan Gong
Interspeech | January 2013
Improvements to VTS Feature Enhancement
Jinyu Li, Mike Seltzer, Yifan Gong
March 2012
Improvements to VTS feature enhancement
Jinyu Li, Mike Seltzer, Yifan Gong
Proc. ICASSP | January 2012
LASSO environment model combination for robust speech recognition
Proc. ICASSP | January 2012
Efficient VTS Adaptation Using Jacobian Approximation
Jinyu Li, Mike Seltzer, Yifan Gong
Interspeech | January 2012
Improving wideband speech recognition using mixed-bandwidth training data in CD-DNN-HMM
Jinyu Li, Dong Yu, Jui-Ting Huang, Tifan Gong
IEEE Workshop on Spoken Language Technology | January 2012
Towards High-Accuracy Low-Cost Noisy Robust Speech Recognition Exploiting Structured Model
Jinyu Li, Dong Yu, Li Deng, Yifan Gong
ICML Workshop 2011 | June 2011
Calibration of Confidence Measures In Speech Recognition
Dong Yu, Jinyu Li, Li Deng
IEEE Transactions on Audio, Speech and Language Processing | January 2011
Towards High-Accuracy Low-Cost Noisy Robust Speech Recognition Exploiting Structured Model
Jinyu Li, Li Deng, Dong Yu
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing | January 2011
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing
Maximum likelihood adaptation of histogram equalization with constraint for robust speech recognition
Proc. ICASSP | January 2011
Proc. ICASSP
Feature normalization using structured full transforms for robust speech recognition
Proc. Interspeech | January 2011
Proc. Interspeech
LASSO model adaptation for automatic speech recognition
Jinyu Li, Ming Yuan, Chin-Hui Lee
ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing | January 2011
Unscented Transform with Online Distortion Estimation for HMM Adaptation
Jinyu Li, Dong Yu, Li Deng, Yifan Gong
Interspeech 2010 | September 2010
Word Confidence Calibration Using A Maximum Entropy Model With Constraints On Confidence and Word Distributions
Dong Yu, Shizhen Wang, Jinyu Li, Li Deng
March 2010
Unscented transform with online distortion estimation for HMM adaptation
Proc. Interspeech | January 2010
Shrinkage Model Adaptation in Automatic Speech Recognition
Jinyu Li, Yu Tsao, Chin-Hui Lee
Interspeech | January 2010
Word Confidence Calibration Using A Maximum Entropy Model With Constraints On Confidence and Word Distributions
Jinyu Li, Li Deng, Dong Yu
Proc. ICASSP | January 2010
Ensemble speaker and speaking environment modeling approach with advanced online estimation process
Yu Tsao, Jinyu Li, Chin-Hui Lee
Proc. ICASSP | January 2009
Proc. ICASSP
A study on hidden Markov model’s generalization capability for speech recognition
Proc. ASRU | January 2009
A study on soft margin estimation of linear regression parameters for speaker adaptation
Proc. Interspeech | January 2009
Soft margin estimation on improving environment structures for ensemble speaker and speaking environment modeling
ACM Proc. the 3rd International Universal Communication Symposium | January 2009
ACM Proc. the 3rd International Universal Communication Symposium
Adaptation of Compressed HMM Parameters for Resource-Constrained Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Jian Wu, Yifan Gong, Alex Acero
April 2008
HMM Adaptation Using a Phase-Sensitive Acoustic Distortion Model for Environment-Robust Speech Recognition
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
April 2008
On a Generalization of Margin-Based Discriminative Training to Robust Speech Recognition
Proc. Interspeech | January 2008
Soft Margin Estimation with Various Separation Levels for LVCSR
Proc. Interspeech | January 2008
HMM adaptation using a phase-sensitive acoustic distortion model for environment-robust speech recognition
Jinyu Li, Li Deng, Dong Yu, Alex Acero
Proc. ICASSP | January 2008
Proc. ICASSP
High-Performance HMM Adaptation With Joint Compensation of Additive and Convolutive Distortions Via Vector Taylor Series
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Proceedings IEEE Workshop on ASRU | April 2007
Proceedings IEEE Workshop on ASRU
A study on soft margin estimation for LVCSR
Proc. IEEE Automatic Speech Recognition and Understanding | January 2007
Approximate test risk minimization through soft margin estimation
Proc. ICASSP | January 2007
Detection-Based ASR in the Automatic Speech Attribute Transcription Project
Proc. Interspeech | January 2007
Proc. Interspeech
High-Performance HMM Adaptation with Joint Compensation of Additive and Convolutive Distortions via Vector Taylor Series
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Proc. IEEE Automatic Speech Recognition and Understanding | January 2007
Proc. IEEE Automatic Speech Recognition and Understanding
Soft Margin Feature Extraction for Automatic Speech Recognition
Proc. Interspeech | January 2007
Soft Margin Estimation of Hidden Markov Model Parameters (best student paper)
Proc. Interspeech | January 2006
Best Student Paper
Application of EαNets to feature recognitionof articulation manner in knowledge-based automatic speech recognition
Lecture Notes in Computer Science | January 2006
Language recognition based on score distribution feature vectors and discriminative classifier fusion
Proc. IEEE Odyssey Workshop on Speaker and Language Recognition | January 2006
A study on lattice rescoring with knowledge scores for automatic speech recognition
Proc. Interspeech | January 2006
Proc. Interspeech
A study on knowledge source integration for candidate rescoring in automatic speech recognition
Proc. ICASSP | January 2005
A study on separation between acoustic models and its applications
Proc. Interspeech | January 2005
On Designing and Evaluating Speech Event Detectors
Proc. Interspeech | January 2005
Dimensionality reduction using MCE-optimized LDA transformation
Proc. ICASSP | January 2004
A complexity reduction of ETSI advanced front-end for DSR
Proc. ICASSP | January 2004
-
ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation
Zongyi Li, Shujie Hu, Shujie Liu, Long Zhou, Jeongsoo Choi, Lingwei Meng, Xun Guo, Jinyu Li, Hefei Ling, Furu Wei
October 2024
October 2024
-
Autoregressive Speech Synthesis without Vector Quantization
Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen Meng, Furu Wei
July 2024
VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment
Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, Furu Wei
June 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei
June 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei
June 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model
Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, Furu Wei
March 2024
Decoder-only Modeling for Speech
December 2023
Invited Talk at NTU and SJTU
Large-Scale Streaming End-to-End Speech Translation
December 2023
Invited Talk at NTU and SJTU
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei
March 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei
January 2023
Deep Neural Network for Automatic Speech Recognition: from the Industry’s View
September 2014
-
End-to-End Automatic Speech Recognition
MSR-TR-2024-30 | May 2024
Published by Microsoft
IEEE SPS Webinar
Recent Advances in End-to-End Automatic Speech Recognition (Invited Talk at ASC 2023)
| November 2023
Published by Microsoft
Advancing end-to-end automatic speech recognition
MSR-TR-2021-32 | October 2021
Published by Microsoft
Keynote talk at the conference on Computational Linguistics and Speech Processing, 2021.
Developing streaming end-to-end models for automatic speech recognition in industry
MSR-TR-2020-45 | December 2020
Published by Microsoft
Deep Learning Acoustic Model in Microsoft Cortana Voice Assistant
MSR-TR-2017-60 | November 2017
Published by Microsoft
-
SpeechLM: Enhanced Speech Pre-Training With Unpaired Textual Data
Ziqiang Zhang, Sanyuan Chen, Long Zhou, Yu Wu, Shuo Ren, Shujie Liu, Zhuoyuan Yao, Xun Gong, Lirong Dai, Jinyu Li, Furu Wei
IEEE/ACM Transactions on Audio, Speech, and Language Processing | March 2024
Advanced Long-Content Speech Recognition with Factorized Neural Transducer
Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian
IEEE/ACM Transactions on Audio, Speech, and Language Processing | January 2024
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
Xiaofei Wang, Manthan Thakker, Zhuo Chen, Naoyuki Kanda, Sefik Emre Eskimez, Sanyuan Chen, Min Tang, Shujie Liu, Jinyu Li, Takuya Yoshioka
August 2023
VatLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning
Qiushi Zhu, Long Zhou, Ziqiang Zhang, Shujie Liu, Binxing Jiao, Jie Zhang, Lirong Dai, Daxin Jiang, Jinyu Li, Furu Wei
IEEE Transactions on Multimedia | May 2023
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Naoyuki Kanda, Jian Wu, Xiaofei Wang, Zhuo Chen, Jinyu Li, Takuya Yoshioka
arXiv:2209.04974 | September 2022
Towards Contextual Spelling Correction for Customization of End-to-End Speech Recognition Systems
Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Veljko Miljanic, Sheng Zhao, Hosam Khalil
IEEE/ACM Transactions on Audio, Speech, and Language Processing | September 2022, Vol 30: pp. 3089-3097
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, Furu Wei
IEEE Journal of Selected Topics in Signal Processing | July 2022, Vol 16(6): pp. 1505-1518
Recent Advances in End-to-End Automatic Speech Recognition
APSIPA Transactions on Signal and Information Processing | April 2022
Streaming end-to-end multi-talker speech recognition
Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
IEEE Signal Processing Letter | April 2021, Vol 28
Adaptation Algorithms for Speech Recognition: An Overview
Peter Bell, Joachim Fainberg, Ondrej Klejch, Jinyu Li, Steve Renals, Pawel Swietojanski
IEEE Open Journal of Signal Processing | February 2021
Speaker Separation Using Speaker Inventories and Estimated Speech
Peidong Wang, Zhuo Chen, DeLiang Wang, Jinyu Li, Yifan Gong
IEEE Transactions on Audio, Speech, and Language Processing | December 2020, Vol 29: pp. 537-546
Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
Amit Das, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong
IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING | December 2019, pp. 1880-1892
Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition
Zhehuai Chen, Jasha Droppo, Jinyu Li, Wayne Xiong
IEEE/ACM Transactions on Audio, Speech, and Language Processing | January 2018
Recent Progresses in Deep Learning Based Acoustic Models
Dong Yu, Jinyu Li
IEEE/CAA JOURNAL OF AUTOMATICA SINICA | July 2017
An Overview of Noise-Robust Automatic Speech Recognition
Jinyu Li, Li Deng, Yifan Gong, Reinhold Haeb-Umbach
IEEE/ACM Transactions on Audio, Speech, and Language Processing | April 2014, Vol 22: pp. 745
Hermitian Polynomial for Speaker Adaptation of Connectionist Speech Recognition Systems
Sabato Marco Simiscalchi, Jinyu Li, Chin-Hui Lee
IEEE Transactions on Audio, Speech, and Language Processing | January 2013
Model-Based Margin Estimation for Hidden Markov Model Learning and Generalization
Sabato Marco Siniscalchi, Jinyu Li, Chin-Hui Lee
IET Signal Processing | January 2013
Calibration of Confidence Measures In Speech Recognition
Dong Yu, Jinyu Li, Li Deng
IEEE Transactions on Audio, Speech, and Language Processing | November 2011
A study on the generalization capability of acoustic models for robust speech recognition
IEEE Transactions on Audio, Speech and Language Processing | January 2010
A unified framework of HMM adaptation with joint compensation of additive and convolutive distortions
Jinyu Li, Li Deng, Dong Yu, Yifan Gong, Alex Acero
Computer Speech and Language | January 2009, Vol 23: pp. 389-405
Approximate Test Risk Bound Minimization through Soft Margin Estimation
IEEE Trans. on Audio, Speech, and Language Proc | January 2007
-
Recent Progresses On Deep Learning For Speech Recognition
Jinyu Li, Dong Yu
Handbook of Pattern Recognition and Computer Vision: 6th Edition | 2020
Challenges and Solutions to Deep Learning Network Acoustic Modeling in Speech Recognition Products at Microsoft
New Era for Robust Speech Recognitino: Exploiting Deep Learning | Published by Springer | 2017
Compensation with Prior Knowledge
Jinyu Li, Li Deng, Yifan Gong
Robust Automatic Speech Recognition: A Bridge to Practical Applications | Published by Elsevier | 2015 | Robust Automatic Speech Recognition: A Bridge to Practical Applications edition
-
Sequence-level self-learning with multi-task learning framework
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
October 2020
-
Robust Automatic Speech Recognition – A Bridge to Practical Applications (1st Edition)
Jinyu Li, Li Deng, R. Haeb-Umbach, Y. Gong
Published by Elsevier | October 2015
-
SOFT MARGIN ESTIMATION FOR AUTOMATIC SPEECH RECOGNITION
December 2008
December 2008