README.md

metadata

license: apache-2.0
datasets:
  - EleutherAI/fineweb-edu-dedup-10b

We trained these skip-transcoders using signum, over 1B tokens. Trained with input and output normalized.