cs.DC · 2026-10-18 · No. 147
Distributed, Parallel, and Cluster Computing, 2026-10-18.
5 new papers in cs.DC. Titles, authors,
abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →
01 — The papers
5 entries-
01
Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs
Haoyu Zhang, Arshad Rahman, Adnan Aziz, Ajith Prabhudev, Amin Farmahini, Ben Rhoads, Benson Ma, Brian Shi, Cen Zhao,...
cs.DC · cs.AR · cs.LG
Meta deploys a heterogeneous fleet comprising NVIDIA GPUs, AMD GPUs, and proprietary MTIA accelerators to reduce costs and mitigate vendor lock-in. However, the extensive history of NVIDIA-specific optimizations hinders performance portability to other hardware. This paper presents our methods for optimizing alternative accelerators, using AMD Instinct MI350X versus NVIDIA B200 for recommendation model training as a case study. We propose...
-
02
Efficient KV Cache Migration for Geo-Distributed LLM Inference in Collaborative Edge Computing
Mingjin Zhang, Jiannong Cao, Tao Wu, Xiangchun Chen, Ne Wang
cs.DC
Large Language Models (LLMs) have shown exceptional capabilities but rely heavily on centralized cloud GPU clusters, leading to high bandwidth costs, latency, and privacy issues. Edge computing offers a solution by deploying LLMs closer to users on edge servers, yet single edge servers often cannot meet LLMs' heavy computational demands. Collaborative Edge Computing (CEC) addresses this challenge by integrating resources of multiple...
-
03
ETH: Data-Balanced Heterogeneous Pipeline Parallelism for Training Large Models
Mianjie Yu, Zhongmin Zhao, Zizhao Mo, Jianxiong Liao, Wenxuan Li, Yulin Qiao, Huanle Xu, Wenquan Yang, Yongqiang...
cs.DC · cs.LG
Current heterogeneity-aware training systems for large models balance workloads by assigning unequal numbers of layers across pipeline stages, a strategy aimed at equalizing execution time through a unique pipeline configuration. However, this approach fails to address critical memory constraints and suffers from inherently coarse balancing granularity. In this paper, we propose Heterogeneous Pipeline Parallelism (HPP), a novel approach...
-
04
Resource-aware Sub-network Allocation for Heterogeneous Decentralized Learning
Zhuojun Tian, Ming Xiao, Mikael Skoglund
cs.DC
System heterogeneity in computation, communication, and memory can substantially limit synchronous decentralized learning, where each training round is constrained by resource-limited nodes. Existing approaches provide limited support for jointly determining node-specific model capacities from deployment resources. We propose RASA-DL, a resource-aware sub-network allocation framework that treats the maximum sub-network width assigned to each...
-
05
Exact Makespan Bounds and the GPipe Homogeneity Bias for Heterogeneous LLM Inference Pipelines
Zhu Wenbo
cs.DC
We give an exact critical-path characterization of the makespan of micro-batch pipelines serving LLMs on heterogeneous GPU clusters, quantifying how the classic homogeneous GPipe formula errs. In the pure-forward, communication-free, infinite-buffer setting, T(M) = (M-1)tau_max + Sum_j tau_j, where tau_j is stage j's per-micro-batch service time and M the micro-batch count (Theorem 1). Under asynchronous communication -- modern NCCL-based...
This edition is part of The Daily Abstract — cs.DC archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.
#D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.