Kunal Jain

Performance Aware LLM Load Balancer for Mixed Workloads

Kunal Jain, A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan

EuroMLSys 2025 | April 2025

Publication Project

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication

Performance Aware LLM Load Balancer for Mixed Workloads

Kunal Jain, A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan

EuroMLSys 2025 | April 2025

Publication Project

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication

Performance Aware LLM Load Balancer for Mixed Workloads

Kunal Jain, A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan

EuroMLSys 2025 | April 2025

Publication Project

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication