Posts tagged Group Collective

Support Native Group Collective

• 24 January 2026
Yunpeng Huang , Zewei Tao
China
English
MagiAttention
Group Collective DeepEP Collective Communication Distributed Attention Context Parallelism

With the release of MagiAttention-v1.1.0, we are excited to announce the support for native group collective CUDA kernels for both intranode and internode communication, based upon the amazing work of DeepEP [Zhao et al., 2025].

Read more ...

---

MagiAttention

• 21 April 2025
Zewei Tao , Yunpeng Huang , Qiangang Wang , Hanwen Sun , Jin Li , Tao Bu , Bowen Zeng
China
English
MagiAttention
Attention Slice Representation Computation Load-Balance Zero-Redundant Communication Multi-Stage Overlap Flex-Flash-Attention Group Collective Flash-Attention Distributed Attention Context Parallelism

A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Mask Training

Read more ...

---