Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
Paper
•
2504.20966
•
Published
•
25
SEACrowd is a community movement project aimed at centralizing and standardizing AI resources for Southeast Asian languages, cultures, and/or regions. This movement is co-initiated by SEA researchers and practitioners from various institutions.