Patel

Kimish's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

updated a model about 18 hours ago

Kimish/Qwen3-4B-untied-8da4w

Text Generation • Updated about 18 hours ago • 10

published a model about 18 hours ago

Kimish/Qwen3-4B-untied-8da4w

Text Generation • Updated about 18 hours ago • 10

updated a model about 18 hours ago

Kimish/Qwen3-4B-untied-weights

Updated about 18 hours ago

published a model about 18 hours ago

Kimish/Qwen3-4B-untied-weights

Updated about 18 hours ago

reacted to wassemgtk's post with 😎 22 days ago

view post

Post

2854

I’ve been diving into the iRoPE architecture from Llama 4—a game-changer for long-context models! It interleaves local attention (with RoPE) for short contexts and global attention (with inference-time temp scaling) for long-range reasoning, aiming for infinite context. I’m going to try writing iRoPE—who wants to help?

Code: https://github.com/wassemgtk/iRoPE-try/blob/main/iRoPE.ipynb

See translation

• 
• 1 reply

·

👀

11

11

😎

3

3

+

Reply