Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Posts
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up
Chat-UniVi 's Collections
Chat-UniVi
MoH
MoE++

MoH

updated Apr 5

MoH: Multi-Head Attention as Mixture-of-Head Attention

Upvote
1

  • Chat-UniVi/MoH-LLaMA3-8B

    Text Generation • Updated Dec 7, 2024 • 15 • 3

  • Chat-UniVi/MoH-DiT-XL-90

    Updated Oct 17, 2024 • 3

  • Chat-UniVi/MoH-ViT-B-75

    Updated Oct 17, 2024

  • Chat-UniVi/MoH-ViT-B-50

    Updated Oct 17, 2024

  • Chat-UniVi/MoH-ViT-S-80

    Updated Oct 17, 2024

  • Chat-UniVi/MoH-ViT-S-75

    Updated Oct 17, 2024

  • MoH: Multi-Head Attention as Mixture-of-Head Attention

    Paper • 2410.11842 • Published Oct 15, 2024 • 22
Upvote
1
  • Collection guide
  • Browse collections
Company
TOS Privacy About Jobs
Website
Models Datasets Spaces Pricing Docs