Cyril Zhang

Phi-4 Technical Report

Marah I Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio CT Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, Yi Zhang

MSR-TR-2024-57 | December 2024

Published by Microsoft

Publication

ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles

Kayo Yin, Chinmay Singh, Fyodor O. Minakov, Vanessa Milan, Hal Daumé, Cyril Zhang, Alex Lu, Danielle Bragg

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing | November 2024

Publication Publication Project

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Marah I Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Hassan Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Parul Chopra, Allie Del Giorno, Gustavo de Rosa, Matthew Dixon, Ronen Eldan, Dan Iter, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Jamie Huynh, Mojan Javaheripi, Xin Jin, Piero Kauffmann, Nikos Karampatziakis, Dongwoo Kim, Mahmoud Khademi, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Chen Liang, Weishung Liu, Xihui (Eric) Lin, Zeqi Lin, Piyush Madan, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Corby Rosset, Sambudha Roy, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Xia Song, Olatunji Ruwase, Xin Wang, Rachel Ward, Guanhua Wang, Philipp Witte, Michael Wyatt, Can Xu, Jiahang Xu, Weijian Xu, Sonali Yadav, Fan Yang, Ziyi Yang, Donghan Yu, Chengruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yunan Zhang, Xiren Zhou

MSR-TR-2024-12 | August 2024

Published by Microsoft

Publication Video Download Project

Can large language models explore in-context?

Akshay Krishnamurthy, Keegan Harris, Dylan Foster, Cyril Zhang, Aleksandrs Slivkins

NeurIPS 2024 | March 2024

DOI Publication Publication

Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression

Adam Block, Dylan Foster, Akshay Krishnamurthy, Max Simchowitz, Cyril Zhang

ICLR 2024 | October 2023

DOI Publication

Exposing Attention Glitches with Flip-Flop Language Modeling

Bingbin Liu, Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

NeurIPS 2023 | October 2023

Publication

Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck

Benjamin L. Edelman, Surbhi Goel, Sham Kakade, Eran Malach, Cyril Zhang

September 2023

Publication

Transformers Learn Shortcuts to Automata

Bingbin Liu, Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

ICLR | May 2023

oral presentation

Publication

Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit

Boaz Barak, Benjamin L. Edelman, Surbhi Goel, Sham M. Kakade, Eran Malach, Cyril Zhang

NeurIPS 2022 | November 2022

Publication

Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms

Surbhi Goel, Sham Kakade, Adam Tauman Kalai, Cyril Zhang

2022 Neural Information Processing Systems | September 2022

Publication

Sparsity in Partially Controllable Linear Systems

Yonathan Efroni, Sham Kakade, Akshay Krishnamurthy, Cyril Zhang

ICML 2022 | July 2022

Publication

Understanding Contrastive Learning Requires Incorporating Inductive Biases

Nikunj Umesh Saunshi, Jordan Ash, Surbhi Goel, Dipendra Kumar Misra, Cyril Zhang, Sanjeev Arora, Sham Kakade, Akshay Krishnamurthy

ICML 2022 | July 2022

Publication

Inductive Biases and Variable Creation in Self-Attention Mechanisms

Benjamin Edelman, Surbhi Goel, Sham Kakade, Cyril Zhang

ICML 2022 | June 2022

Publication

Anti-Concentrated Confidence Bonuses for Scalable Exploration

Jordan T. Ash, Cyril Zhang, Surbhi Goel, Akshay Krishnamurthy, Sham M. Kakade

ICLR 2022 | April 2022

Publication Publication

Acceleration via Fractal Learning Rate Schedules

Naman Agarwal, Surbhi Goel, Cyril Zhang

2021 International Conference on Machine Learning | July 2021

Publication Publication

Phi-4 Technical Report

Marah I Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio CT Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, Yi Zhang

MSR-TR-2024-57 | December 2024

Published by Microsoft

Publication

ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles

Kayo Yin, Chinmay Singh, Fyodor O. Minakov, Vanessa Milan, Hal Daumé, Cyril Zhang, Alex Lu, Danielle Bragg

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing | November 2024

Publication Publication Project

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Marah I Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Hassan Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Parul Chopra, Allie Del Giorno, Gustavo de Rosa, Matthew Dixon, Ronen Eldan, Dan Iter, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Jamie Huynh, Mojan Javaheripi, Xin Jin, Piero Kauffmann, Nikos Karampatziakis, Dongwoo Kim, Mahmoud Khademi, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Chen Liang, Weishung Liu, Xihui (Eric) Lin, Zeqi Lin, Piyush Madan, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Corby Rosset, Sambudha Roy, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Xia Song, Olatunji Ruwase, Xin Wang, Rachel Ward, Guanhua Wang, Philipp Witte, Michael Wyatt, Can Xu, Jiahang Xu, Weijian Xu, Sonali Yadav, Fan Yang, Ziyi Yang, Donghan Yu, Chengruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yunan Zhang, Xiren Zhou

MSR-TR-2024-12 | August 2024

Published by Microsoft

Publication Video Download Project

Can large language models explore in-context?

Akshay Krishnamurthy, Keegan Harris, Dylan Foster, Cyril Zhang, Aleksandrs Slivkins

NeurIPS 2024 | March 2024

DOI Publication Publication

Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression

Adam Block, Dylan Foster, Akshay Krishnamurthy, Max Simchowitz, Cyril Zhang

ICLR 2024 | October 2023

DOI Publication

Exposing Attention Glitches with Flip-Flop Language Modeling

Bingbin Liu, Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

NeurIPS 2023 | October 2023

Publication

Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck

Benjamin L. Edelman, Surbhi Goel, Sham Kakade, Eran Malach, Cyril Zhang

September 2023

Publication

Transformers Learn Shortcuts to Automata

Bingbin Liu, Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

ICLR | May 2023

oral presentation

Publication

Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit

Boaz Barak, Benjamin L. Edelman, Surbhi Goel, Sham M. Kakade, Eran Malach, Cyril Zhang

NeurIPS 2022 | November 2022

Publication

Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms

Surbhi Goel, Sham Kakade, Adam Tauman Kalai, Cyril Zhang

2022 Neural Information Processing Systems | September 2022

Publication

Understanding Contrastive Learning Requires Incorporating Inductive Biases

Nikunj Umesh Saunshi, Jordan Ash, Surbhi Goel, Dipendra Kumar Misra, Cyril Zhang, Sanjeev Arora, Sham Kakade, Akshay Krishnamurthy

ICML 2022 | July 2022

Publication

Sparsity in Partially Controllable Linear Systems

Yonathan Efroni, Sham Kakade, Akshay Krishnamurthy, Cyril Zhang

ICML 2022 | July 2022

Publication

Inductive Biases and Variable Creation in Self-Attention Mechanisms

Benjamin Edelman, Surbhi Goel, Sham Kakade, Cyril Zhang

ICML 2022 | June 2022

Publication

Anti-Concentrated Confidence Bonuses for Scalable Exploration

Jordan T. Ash, Cyril Zhang, Surbhi Goel, Akshay Krishnamurthy, Sham M. Kakade

ICLR 2022 | April 2022

Publication Publication

Acceleration via Fractal Learning Rate Schedules

Naman Agarwal, Surbhi Goel, Cyril Zhang

2021 International Conference on Machine Learning | July 2021

Publication Publication

ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles

Kayo Yin, Chinmay Singh, Fyodor O. Minakov, Vanessa Milan, Hal Daumé, Cyril Zhang, Alex Lu, Danielle Bragg

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing | November 2024

Publication Publication Project

Acceleration via Fractal Learning Rate Schedules

Naman Agarwal, Surbhi Goel, Cyril Zhang

2021 International Conference on Machine Learning | July 2021

Publication Publication

Phi-4 Technical Report

Marah I Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio CT Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, Yi Zhang

MSR-TR-2024-57 | December 2024

Published by Microsoft

Publication

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Marah I Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Hassan Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Parul Chopra, Allie Del Giorno, Gustavo de Rosa, Matthew Dixon, Ronen Eldan, Dan Iter, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Jamie Huynh, Mojan Javaheripi, Xin Jin, Piero Kauffmann, Nikos Karampatziakis, Dongwoo Kim, Mahmoud Khademi, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Chen Liang, Weishung Liu, Xihui (Eric) Lin, Zeqi Lin, Piyush Madan, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Corby Rosset, Sambudha Roy, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Xia Song, Olatunji Ruwase, Xin Wang, Rachel Ward, Guanhua Wang, Philipp Witte, Michael Wyatt, Can Xu, Jiahang Xu, Weijian Xu, Sonali Yadav, Fan Yang, Ziyi Yang, Donghan Yu, Chengruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yunan Zhang, Xiren Zhou

MSR-TR-2024-12 | August 2024

Published by Microsoft

Publication Video Download Project

ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles

Kayo Yin, Chinmay Singh, Fyodor O. Minakov, Vanessa Milan, Hal Daumé, Cyril Zhang, Alex Lu, Danielle Bragg

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing | November 2024

Publication Publication Project

Can large language models explore in-context?

Akshay Krishnamurthy, Keegan Harris, Dylan Foster, Cyril Zhang, Aleksandrs Slivkins

NeurIPS 2024 | March 2024

DOI Publication Publication

Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression

Adam Block, Dylan Foster, Akshay Krishnamurthy, Max Simchowitz, Cyril Zhang

ICLR 2024 | October 2023

DOI Publication

Exposing Attention Glitches with Flip-Flop Language Modeling

Bingbin Liu, Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

NeurIPS 2023 | October 2023

Publication

Transformers Learn Shortcuts to Automata

Bingbin Liu, Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

ICLR | May 2023

oral presentation

Publication

Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit

Boaz Barak, Benjamin L. Edelman, Surbhi Goel, Sham M. Kakade, Eran Malach, Cyril Zhang

NeurIPS 2022 | November 2022

Publication

Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms

Surbhi Goel, Sham Kakade, Adam Tauman Kalai, Cyril Zhang

2022 Neural Information Processing Systems | September 2022

Publication

Understanding Contrastive Learning Requires Incorporating Inductive Biases

Nikunj Umesh Saunshi, Jordan Ash, Surbhi Goel, Dipendra Kumar Misra, Cyril Zhang, Sanjeev Arora, Sham Kakade, Akshay Krishnamurthy

ICML 2022 | July 2022

Publication

Sparsity in Partially Controllable Linear Systems

Yonathan Efroni, Sham Kakade, Akshay Krishnamurthy, Cyril Zhang

ICML 2022 | July 2022

Publication

Inductive Biases and Variable Creation in Self-Attention Mechanisms

Benjamin Edelman, Surbhi Goel, Sham Kakade, Cyril Zhang

ICML 2022 | June 2022

Publication

Anti-Concentrated Confidence Bonuses for Scalable Exploration

Jordan T. Ash, Cyril Zhang, Surbhi Goel, Akshay Krishnamurthy, Sham M. Kakade

ICLR 2022 | April 2022

Publication Publication

Acceleration via Fractal Learning Rate Schedules

Naman Agarwal, Surbhi Goel, Cyril Zhang

2021 International Conference on Machine Learning | July 2021

Publication Publication

Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck

Benjamin L. Edelman, Surbhi Goel, Sham Kakade, Eran Malach, Cyril Zhang

September 2023

Publication