Posts by Zewei Tao

Attention Engine for Inference

• 08 February 2026
Zewei Tao
China
English
MagiAttention
AF Disaggregation Distributed Attention Context Parallelism

The upcoming blog post will be released in the near future. Stay tuned!

Read more ...

---

Support Blackwell with FFA_FA4 Backend

• 07 February 2026
Jerry Chen , Yujia Liu , Yufeng Yang , Yunpeng Huang , Zewei Tao , Qiangang Wang , Kunlun Li
China
English
MagiAttention
Blackwell Flex-Flash-Attention Flash-Attention HSTU Function Representation

The upcoming blog post will be released in the near future. Stay tuned!

Read more ...

---

Optimize Sparse Attention in FFA

• 25 January 2026
Zewei Tao , Hanwen Sun , Bowen Zeng , Jin Li , Yunpeng Huang
China
English
MagiAttention
Sparse Attention NSA DSA Flex-Flash-Attention Flash-Attention

The upcoming blog post will be released in the near future. Stay tuned!

Read more ...

---

Support Native Group Collective Based on DeepEP

• 24 January 2026
Yunpeng Huang , Zewei Tao
China
English
MagiAttention
Group Collective DeepEP Distributed Attention Context Parallelism

The upcoming blog post will be released in the near future. Stay tuned!

Read more ...

---

Dynamic Attention Solver

• 21 January 2026
Jin Li , Zewei Tao , Qiangang Wang , Yunpeng Huang
China
English
MagiAttention
Dynamic Load Balance Sparse Attention Hybrid Attention Distributed Attention Context Parallelism

The upcoming blog post will be released in the near future. Stay tuned!

Read more ...

---

Long-Context Attention Benchmark

• 19 October 2025
Tao Bu , Qiangang Wang , Bowen Zeng , Hanwen Sun , Yunpeng Huang , Zewei Tao
China
English
MagiAttention
Benchmark Blackwell Flex-Flash-Attention Distributed Attention Context Parallelism

From Kernel Efficiency to Distributed Scalability

Read more ...

---

MagiAttention

• 21 April 2025
Zewei Tao , Yunpeng Huang , Qiangang Wang , Hanwen Sun , Jin Li , Tao Bu , Bowen Zeng
China
English
MagiAttention
Attention Slice Representation Computation Load-Balance Zero-Redundant Communication Multi-Stage Overlap Flex-Flash-Attention Group Collective Flash-Attention Distributed Attention Context Parallelism

A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Mask Training

Read more ...

---