zinc10M_gpt2_SMILES_bpe_combined_step1_finetune_covid

This model is a fine-tuned version of jarod0411/zinc10M_gpt2_SMILES_bpe_combined_step1 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5168
• Accuracy: 0.8253

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 64
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	388	0.5532	0.8113
0.5991	2.0	776	0.5424	0.8149
0.5579	3.0	1164	0.5373	0.8165
0.5472	4.0	1552	0.5332	0.8180
0.5472	5.0	1940	0.5306	0.8188
0.5401	6.0	2328	0.5285	0.8196
0.5343	7.0	2716	0.5270	0.8202
0.5298	8.0	3104	0.5254	0.8208
0.5298	9.0	3492	0.5245	0.8211
0.5256	10.0	3880	0.5235	0.8216
0.5222	11.0	4268	0.5226	0.8219
0.519	12.0	4656	0.5215	0.8223
0.516	13.0	5044	0.5209	0.8226
0.516	14.0	5432	0.5206	0.8228
0.5135	15.0	5820	0.5197	0.8231
0.511	16.0	6208	0.5191	0.8234
0.5088	17.0	6596	0.5189	0.8235
0.5088	18.0	6984	0.5187	0.8237
0.5067	19.0	7372	0.5183	0.8239
0.505	20.0	7760	0.5182	0.8241
0.503	21.0	8148	0.5179	0.8242
0.5013	22.0	8536	0.5177	0.8243
0.5013	23.0	8924	0.5177	0.8244
0.4999	24.0	9312	0.5175	0.8244
0.4985	25.0	9700	0.5174	0.8245
0.4972	26.0	10088	0.5174	0.8246
0.4972	27.0	10476	0.5172	0.8247
0.4958	28.0	10864	0.5173	0.8247
0.4949	29.0	11252	0.5173	0.8248
0.4937	30.0	11640	0.5172	0.8249
0.4928	31.0	12028	0.5170	0.8250
0.4928	32.0	12416	0.5169	0.8250
0.492	33.0	12804	0.5170	0.8251
0.4912	34.0	13192	0.5169	0.8252
0.4904	35.0	13580	0.5171	0.8251
0.4904	36.0	13968	0.5168	0.8252
0.4899	37.0	14356	0.5169	0.8252
0.4895	38.0	14744	0.5169	0.8252
0.4889	39.0	15132	0.5168	0.8253
0.4885	40.0	15520	0.5168	0.8253

Framework versions

• Transformers 4.36.0.dev0
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0