News

Based on the MoGE architecture, we built a Pangu Pro MoE model with a total parameter size of 72B and an activation parameter size of 16B: MoGE configuration: 4 shared experts, 64 routing experts ...