EleutherAI-pythia-410m-finetuned-nvidia-faq

This model is a fine-tuned version of EleutherAI/pythia-410m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2026

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use adagrad and the args are: No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
0.2307	0.2813	50	0.2261
0.2178	0.5626	100	0.2179
0.213	0.8439	150	0.2135
0.1883	1.1294	200	0.2109
0.2153	1.4107	250	0.2091
0.2183	1.6920	300	0.2075
0.1855	1.9733	350	0.2063
0.1723	2.2588	400	0.2056
0.1971	2.5401	450	0.2050
0.1724	2.8214	500	0.2043
0.1954	3.1069	550	0.2038
0.169	3.3882	600	0.2035
0.1937	3.6695	650	0.2032
0.1786	3.9508	700	0.2029
0.2031	4.2363	750	0.2028
0.186	4.5176	800	0.2027
0.1797	4.7989	850	0.2026

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cu121
• Datasets 3.3.1
• Tokenizers 0.21.0