Author of the publication

copy delete add this publication to your clipboard
community post
history of this post
URL
DOI
BibTeX
EndNote
APA
Chicago
DIN 1505
Harvard
MSOffice XML

DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters.

J. Rasley, S. Rajbhandari, O. Ruwase, and Y. He. KDD, page 3505-3506. ACM, (2020)

Please choose a person to relate this publication to

To differ between persons with the same name, the academic degree and the title of an important publication will be displayed. You can also use the button next to the name to display some publications already assigned to the person.

Shrija Rajbhandari

Sujan Rajbhandari

Meena Rajbhandari

Rajita Rajbhandari

Kishor Kumar Rajbhandari

Other publications of authors with the same name

Scalable and Efficient MoE Training for Multitask Multilingual Models.Y. Kim, A. Awan, A. Muzio, A. Cruz-Salinas, L. Lu, A. Hendy, S. Rajbhandari, Y. He, and H. Awadalla. CoRR, (2021)Fast LSTM Inference by Dynamic Decomposition on Cloud Systems.Y. You, Y. He, S. Rajbhandari, W. Wang, C. Hsieh, K. Keutzer, and J. Demmel. ICDM, page 748-757. IEEE, (2019)ZeRO-Offload: Democratizing Billion-Scale Model Training.J. Ren, S. Rajbhandari, R. Aminabadi, O. Ruwase, S. Yang, M. Zhang, D. Li, and Y. He. USENIX Annual Technical Conference, page 551-564. USENIX Association, (2021)SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement.H. Qin, S. Rajbhandari, O. Ruwase, F. Yan, L. Yang, and Y. He. NeurIPS, page 20531-20544. (2021)Optimizing the Four-Index Integral Transform Using Data Movement Lower Bounds Analysis.S. Rajbhandari, F. Rastello, K. Kowalski, S. Krishnamoorthy, and P. Sadayappan. PPoPP, page 327-340. ACM, (2017)DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models.S. Jacobs, M. Tanaka, C. Zhang, M. Zhang, S. Song, S. Rajbhandari, and Y. He. CoRR, (2023)DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales.Z. Yao, R. Aminabadi, O. Ruwase, S. Rajbhandari, X. Wu, A. Awan, J. Rasley, M. Zhang, C. Li, C. Holmes and 9 other author(s). CoRR, (2023)DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale.S. Rajbhandari, C. Li, Z. Yao, M. Zhang, R. Aminabadi, A. Awan, J. Rasley, and Y. He. ICML, volume 162 of Proceedings of Machine Learning Research, page 18332-18346. PMLR, (2022)A Communication-Optimal Framework for Contracting Distributed Tensors.S. Rajbhandari, A. Nikam, P. Lai, K. Stock, S. Krishnamoorthy, and P. Sadayappan. SC, page 375-386. IEEE Computer Society, (2014)International Conference on Computational Science, ICCS 2012.P. Lai, H. Zhang, S. Rajbhandari, E. Valeev, K. Kowalski, and P. Sadayappan. ICCS, volume 9 of Procedia Computer Science, page 412-421. Elsevier, (2012)

BibSonomy

Disambiguation of "Rajbhandari, Samyam"

copy delete add this publication to your clipboard
community post
history of this post
URL
DOI
BibTeX
EndNote
APA
Chicago
DIN 1505
Harvard
MSOffice XML

DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters.

Please choose a person to relate this publication to

Shrija Rajbhandari

Sujan Rajbhandari

Meena Rajbhandari

Rajita Rajbhandari

Kishor Kumar Rajbhandari

Other publications of authors with the same name

Disambiguation

BibSonomy

Disambiguation of "Rajbhandari, Samyam"

copydeleteadd this publication to your clipboardcommunity posthistory of this postURLDOIBibTeXEndNoteAPAChicagoDIN 1505HarvardMSOffice XML DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters.

Please choose a person to relate this publication to

Shrija Rajbhandari

Sujan Rajbhandari

Meena Rajbhandari

Rajita Rajbhandari

Kishor Kumar Rajbhandari

Other publications of authors with the same name

Disambiguation

copy delete add this publication to your clipboard
community post
history of this post
URL
DOI
BibTeX
EndNote
APA
Chicago
DIN 1505
Harvard
MSOffice XML

DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters.