arXiv 2024 / LLM Agents

P/D-Serve: Serving Disaggregated Large Language Model at Scale

A systems paper on disaggregated large language model serving, separating serving stages to support large-scale inference.

Research area

AI search, skill retrieval, NL2SQL benchmarks, LLM serving systems, and capability governance are treated as system surfaces that should be inspected before they are trusted.

Publication record

Authors
Yibo Jin, Tao Wang, Huimin Lin, Mingyang Song, Peiyang Li, Yipeng Ma, Yicheng Shan, Zhengfan Yuan, Cailong Li, Yajing Sun, Tiandeng Wu, Xing Chu, Ruizhi Huan, Li Ma, Xiao You, Wenting Zhou, Yunpeng Ye, Wen Liu, Xiangkun Xu, Yongsheng Zhang, Tiantian Dong, Jiawei Zhu, Zhe Wang, Xijian Ju, Jianxun Song, Haoliang Cheng, Xiaojing Li, Jiandong Ding, Hefei Guo, Zhengyong Zhang
Venue
arXiv preprint arXiv:2408.08147
Year
2024
Area
LLM Agents