Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Qingwei Lin 林庆维, Jianguang Lou, Shifeng Chen, Yansong Tang, Weizhu Chen
July 2024
Preprint
Weizhu Chen
Technical Fellow and CVP, Microsoft GenAI
Publications
Publications by Year
-
-
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
NeurIPS 2023 | December 2023
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, Tuo Zhao
ICLR 2024 | October 2023
In-Context Learning Unlocked for Diffusion Models
Zhendong Wang, Yifan Jiang, Yadong Lu, Yelong Shen, Pengcheng He, Weizhu Chen, Zhangyang Wang, Mingyuan Zhou
NeurIPS 2023 | October 2023
Meet in the Middle: A New Pre-training Paradigm
Anh Nguyen, Nikos Karampatziakis, Weizhu Chen
NeurIPS 2023 | October 2023
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen
ICLR 2024 | September 2023
Making Language Models Better Reasoners with Step-Aware Verifier
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, Weizhu Chen
ACL 2023 | June 2023
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, Tuo Zhao
11th International Conference on Learning Representations (ICLR 2023) | May 2023
Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou
NeurIPS 2023 | April 2023
-
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen
ICLR 2022 | April 2022
TAPEX: Table Pre-training via Learning a Neural SQL Executor
Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou
ICLR 2022 | April 2022
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Greg Yang, Edward J. Hu, Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki, Xiaodong Liu, Weizhu Chen, Jianfeng Gao
NeurIPS 2021 | March 2022
-
On the Variance of the Adaptive Learning Rate and Beyond
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
Eighth International Conference on Learning Representations (ICLR) | April 2020
-
Multi-Task Deep Neural Networks for Natural Language Understanding
Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
ACL 2019 | June 2019
Publications by Research Area
-
Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Qingwei Lin 林庆维, Jianguang Lou, Shifeng Chen, Yansong Tang, Weizhu Chen
July 2024
Preprint
On the Variance of the Adaptive Learning Rate and Beyond
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
Eighth International Conference on Learning Representations (ICLR) | April 2020
-
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
NeurIPS 2023 | December 2023
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, Tuo Zhao
ICLR 2024 | October 2023
Meet in the Middle: A New Pre-training Paradigm
Anh Nguyen, Nikos Karampatziakis, Weizhu Chen
NeurIPS 2023 | October 2023
In-Context Learning Unlocked for Diffusion Models
Zhendong Wang, Yifan Jiang, Yadong Lu, Yelong Shen, Pengcheng He, Weizhu Chen, Zhangyang Wang, Mingyuan Zhou
NeurIPS 2023 | October 2023
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen
ICLR 2024 | September 2023
Making Language Models Better Reasoners with Step-Aware Verifier
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, Weizhu Chen
ACL 2023 | June 2023
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, Tuo Zhao
11th International Conference on Learning Representations (ICLR 2023) | May 2023
Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou
NeurIPS 2023 | April 2023
TAPEX: Table Pre-training via Learning a Neural SQL Executor
Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou
ICLR 2022 | April 2022
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen
ICLR 2022 | April 2022
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Greg Yang, Edward J. Hu, Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki, Xiaodong Liu, Weizhu Chen, Jianfeng Gao
NeurIPS 2021 | March 2022
On the Variance of the Adaptive Learning Rate and Beyond
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
Eighth International Conference on Learning Representations (ICLR) | April 2020
Multi-Task Deep Neural Networks for Natural Language Understanding
Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
ACL 2019 | June 2019
-
Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou
NeurIPS 2023 | April 2023
On the Variance of the Adaptive Learning Rate and Beyond
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
Eighth International Conference on Learning Representations (ICLR) | April 2020
-
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, Tuo Zhao
11th International Conference on Learning Representations (ICLR 2023) | May 2023
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen
ICLR 2022 | April 2022
On the Variance of the Adaptive Learning Rate and Beyond
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
Eighth International Conference on Learning Representations (ICLR) | April 2020
Publications by Type
-
Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Qingwei Lin 林庆维, Jianguang Lou, Shifeng Chen, Yansong Tang, Weizhu Chen
July 2024
Preprint
-
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
NeurIPS 2023 | December 2023
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, Tuo Zhao
ICLR 2024 | October 2023
Meet in the Middle: A New Pre-training Paradigm
Anh Nguyen, Nikos Karampatziakis, Weizhu Chen
NeurIPS 2023 | October 2023
In-Context Learning Unlocked for Diffusion Models
Zhendong Wang, Yifan Jiang, Yadong Lu, Yelong Shen, Pengcheng He, Weizhu Chen, Zhangyang Wang, Mingyuan Zhou
NeurIPS 2023 | October 2023
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen
ICLR 2024 | September 2023
Making Language Models Better Reasoners with Step-Aware Verifier
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, Weizhu Chen
ACL 2023 | June 2023
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, Tuo Zhao
11th International Conference on Learning Representations (ICLR 2023) | May 2023
Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou
NeurIPS 2023 | April 2023
TAPEX: Table Pre-training via Learning a Neural SQL Executor
Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou
ICLR 2022 | April 2022
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen
ICLR 2022 | April 2022
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Greg Yang, Edward J. Hu, Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki, Xiaodong Liu, Weizhu Chen, Jianfeng Gao
NeurIPS 2021 | March 2022
On the Variance of the Adaptive Learning Rate and Beyond
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
Eighth International Conference on Learning Representations (ICLR) | April 2020
Multi-Task Deep Neural Networks for Natural Language Understanding
Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
ACL 2019 | June 2019