Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: openlm-research/open_llama_3b
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "openlm-research/open_llama_3b",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:607c3f2ed0bcf7175d2653b204b1d9456d5338a559ae6f2b0882238f2a4d2ae0
+size 10663320

checkpoint-500/README.md

@@ -0,0 +1,202 @@
+---
+base_model: openlm-research/open_llama_3b
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-500/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "openlm-research/open_llama_3b",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:869ce8cb8525e2e9d25141da07e7e2fd7f96b6e826dd4bd204dbf60b5653c067
+size 10663320

checkpoint-500/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82db044f769362afc41ac564211afeeb387d8a43bdbd7e6bcde3f11f6f08d480
+size 21386746

checkpoint-500/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f37c40ce327861a7ca13b719d3aa37510a143368b6e74358bdb14becb3899e1e
+size 14244

checkpoint-500/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c40b1ef83e0ab438de8d7316f6b9e11585b931afa2859bc18f6b2472417e80ad
+size 1064

checkpoint-500/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-500/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab1b681ec7fc02fed5edd3026687d7a692a918c4dd8e150ca2e3994a6229843b
+size 534194

checkpoint-500/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "legacy": true,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-500/trainer_state.json

@@ -0,0 +1,784 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 4.0,
+  "eval_steps": 500,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.08048289738430583,
+      "grad_norm": 1.8390964269638062,
+      "learning_rate": 1.976e-05,
+      "logits/chosen": -7.928730010986328,
+      "logits/rejected": -7.768202304840088,
+      "logps/chosen": -126.51554870605469,
+      "logps/rejected": -141.75454711914062,
+      "loss": 0.6957,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": -0.002580838045105338,
+      "rewards/margins": -0.004946361295878887,
+      "rewards/rejected": 0.0023655227851122618,
+      "step": 10
+    },
+    {
+      "epoch": 0.16096579476861167,
+      "grad_norm": 2.1244845390319824,
+      "learning_rate": 1.9493333333333335e-05,
+      "logits/chosen": -8.07243537902832,
+      "logits/rejected": -8.191374778747559,
+      "logps/chosen": -140.3431854248047,
+      "logps/rejected": -127.8196029663086,
+      "loss": 0.6941,
+      "rewards/accuracies": 0.4749999940395355,
+      "rewards/chosen": 0.0015918923309072852,
+      "rewards/margins": -0.0016587398713454604,
+      "rewards/rejected": 0.0032506324350833893,
+      "step": 20
+    },
+    {
+      "epoch": 0.2414486921529175,
+      "grad_norm": 2.229215383529663,
+      "learning_rate": 1.922666666666667e-05,
+      "logits/chosen": -7.767237663269043,
+      "logits/rejected": -7.859239101409912,
+      "logps/chosen": -140.90402221679688,
+      "logps/rejected": -136.17660522460938,
+      "loss": 0.6931,
+      "rewards/accuracies": 0.4749999940395355,
+      "rewards/chosen": 0.0016952038276940584,
+      "rewards/margins": 0.0003137874882668257,
+      "rewards/rejected": 0.001381416223011911,
+      "step": 30
+    },
+    {
+      "epoch": 0.32193158953722334,
+      "grad_norm": 1.4213330745697021,
+      "learning_rate": 1.896e-05,
+      "logits/chosen": -7.892869472503662,
+      "logits/rejected": -7.617051124572754,
+      "logps/chosen": -121.54151916503906,
+      "logps/rejected": -132.56336975097656,
+      "loss": 0.6901,
+      "rewards/accuracies": 0.574999988079071,
+      "rewards/chosen": 0.009255774319171906,
+      "rewards/margins": 0.006459495518356562,
+      "rewards/rejected": 0.0027962785679847,
+      "step": 40
+    },
+    {
+      "epoch": 0.4024144869215292,
+      "grad_norm": 2.9331471920013428,
+      "learning_rate": 1.8693333333333333e-05,
+      "logits/chosen": -7.751120090484619,
+      "logits/rejected": -7.877864837646484,
+      "logps/chosen": -125.05207824707031,
+      "logps/rejected": -139.2479248046875,
+      "loss": 0.6888,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.013416772708296776,
+      "rewards/margins": 0.008954327553510666,
+      "rewards/rejected": 0.004462444689124823,
+      "step": 50
+    },
+    {
+      "epoch": 0.482897384305835,
+      "grad_norm": 1.9477262496948242,
+      "learning_rate": 1.8426666666666668e-05,
+      "logits/chosen": -7.746337890625,
+      "logits/rejected": -7.790966987609863,
+      "logps/chosen": -108.3724365234375,
+      "logps/rejected": -141.29953002929688,
+      "loss": 0.6947,
+      "rewards/accuracies": 0.44999998807907104,
+      "rewards/chosen": 0.0028126670513302088,
+      "rewards/margins": -0.0029423837549984455,
+      "rewards/rejected": 0.005755049642175436,
+      "step": 60
+    },
+    {
+      "epoch": 0.5633802816901409,
+      "grad_norm": 2.476123809814453,
+      "learning_rate": 1.8160000000000002e-05,
+      "logits/chosen": -7.934849739074707,
+      "logits/rejected": -7.784144401550293,
+      "logps/chosen": -148.48281860351562,
+      "logps/rejected": -140.32388305664062,
+      "loss": 0.6895,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.016439538449048996,
+      "rewards/margins": 0.007786408998072147,
+      "rewards/rejected": 0.008653131313621998,
+      "step": 70
+    },
+    {
+      "epoch": 0.6438631790744467,
+      "grad_norm": 1.9035556316375732,
+      "learning_rate": 1.7893333333333337e-05,
+      "logits/chosen": -7.956778526306152,
+      "logits/rejected": -7.880636692047119,
+      "logps/chosen": -134.6743621826172,
+      "logps/rejected": -142.11549377441406,
+      "loss": 0.6855,
+      "rewards/accuracies": 0.6499999761581421,
+      "rewards/chosen": 0.005495529621839523,
+      "rewards/margins": 0.015812702476978302,
+      "rewards/rejected": -0.010317172855138779,
+      "step": 80
+    },
+    {
+      "epoch": 0.7243460764587525,
+      "grad_norm": 2.0266778469085693,
+      "learning_rate": 1.762666666666667e-05,
+      "logits/chosen": -7.979184627532959,
+      "logits/rejected": -8.19025993347168,
+      "logps/chosen": -129.11795043945312,
+      "logps/rejected": -133.7857666015625,
+      "loss": 0.6911,
+      "rewards/accuracies": 0.5249999761581421,
+      "rewards/chosen": 0.00029927687137387693,
+      "rewards/margins": 0.0047690351493656635,
+      "rewards/rejected": -0.004469756968319416,
+      "step": 90
+    },
+    {
+      "epoch": 0.8048289738430584,
+      "grad_norm": 2.324070453643799,
+      "learning_rate": 1.736e-05,
+      "logits/chosen": -7.854997158050537,
+      "logits/rejected": -7.669028282165527,
+      "logps/chosen": -129.60366821289062,
+      "logps/rejected": -129.22799682617188,
+      "loss": 0.6872,
+      "rewards/accuracies": 0.5249999761581421,
+      "rewards/chosen": 0.005187320522964001,
+      "rewards/margins": 0.012428809888660908,
+      "rewards/rejected": -0.007241488434374332,
+      "step": 100
+    },
+    {
+      "epoch": 0.8853118712273642,
+      "grad_norm": 2.3035223484039307,
+      "learning_rate": 1.7093333333333335e-05,
+      "logits/chosen": -7.684693336486816,
+      "logits/rejected": -8.104289054870605,
+      "logps/chosen": -135.45704650878906,
+      "logps/rejected": -142.0008544921875,
+      "loss": 0.6842,
+      "rewards/accuracies": 0.625,
+      "rewards/chosen": -0.0016596360364928842,
+      "rewards/margins": 0.0194082073867321,
+      "rewards/rejected": -0.021067844703793526,
+      "step": 110
+    },
+    {
+      "epoch": 0.96579476861167,
+      "grad_norm": 2.0869996547698975,
+      "learning_rate": 1.682666666666667e-05,
+      "logits/chosen": -8.05040168762207,
+      "logits/rejected": -8.025721549987793,
+      "logps/chosen": -127.3593978881836,
+      "logps/rejected": -122.40535736083984,
+      "loss": 0.6837,
+      "rewards/accuracies": 0.550000011920929,
+      "rewards/chosen": -0.012504220008850098,
+      "rewards/margins": 0.0199703611433506,
+      "rewards/rejected": -0.0324745811522007,
+      "step": 120
+    },
+    {
+      "epoch": 1.040241448692153,
+      "grad_norm": 1.9564281702041626,
+      "learning_rate": 1.656e-05,
+      "logits/chosen": -8.038365364074707,
+      "logits/rejected": -8.249411582946777,
+      "logps/chosen": -122.29254913330078,
+      "logps/rejected": -139.63780212402344,
+      "loss": 0.628,
+      "rewards/accuracies": 0.6486486196517944,
+      "rewards/chosen": -0.008433423936367035,
+      "rewards/margins": 0.0293881893157959,
+      "rewards/rejected": -0.03782161325216293,
+      "step": 130
+    },
+    {
+      "epoch": 1.1207243460764587,
+      "grad_norm": 2.4709627628326416,
+      "learning_rate": 1.6293333333333335e-05,
+      "logits/chosen": -7.3343305587768555,
+      "logits/rejected": -7.533148288726807,
+      "logps/chosen": -125.78892517089844,
+      "logps/rejected": -131.87570190429688,
+      "loss": 0.6687,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.01648072525858879,
+      "rewards/margins": 0.05134889483451843,
+      "rewards/rejected": -0.03486816957592964,
+      "step": 140
+    },
+    {
+      "epoch": 1.2012072434607646,
+      "grad_norm": 2.2346677780151367,
+      "learning_rate": 1.6026666666666667e-05,
+      "logits/chosen": -7.965804100036621,
+      "logits/rejected": -8.229939460754395,
+      "logps/chosen": -134.5069580078125,
+      "logps/rejected": -150.25515747070312,
+      "loss": 0.6559,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.00035553425550460815,
+      "rewards/margins": 0.0779074877500534,
+      "rewards/rejected": -0.0775519534945488,
+      "step": 150
+    },
+    {
+      "epoch": 1.2816901408450705,
+      "grad_norm": 2.3957595825195312,
+      "learning_rate": 1.576e-05,
+      "logits/chosen": -7.930548667907715,
+      "logits/rejected": -8.198356628417969,
+      "logps/chosen": -127.7699203491211,
+      "logps/rejected": -140.8756866455078,
+      "loss": 0.6581,
+      "rewards/accuracies": 0.7250000238418579,
+      "rewards/chosen": 0.0083924550563097,
+      "rewards/margins": 0.07435399293899536,
+      "rewards/rejected": -0.06596153974533081,
+      "step": 160
+    },
+    {
+      "epoch": 1.3621730382293762,
+      "grad_norm": 2.6251115798950195,
+      "learning_rate": 1.5493333333333333e-05,
+      "logits/chosen": -8.110880851745605,
+      "logits/rejected": -8.14248275756836,
+      "logps/chosen": -124.23824310302734,
+      "logps/rejected": -133.23915100097656,
+      "loss": 0.6484,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.02494233287870884,
+      "rewards/margins": 0.09594295918941498,
+      "rewards/rejected": -0.07100063562393188,
+      "step": 170
+    },
+    {
+      "epoch": 1.442655935613682,
+      "grad_norm": 2.497619867324829,
+      "learning_rate": 1.5226666666666668e-05,
+      "logits/chosen": -7.945010185241699,
+      "logits/rejected": -7.939410209655762,
+      "logps/chosen": -116.3001937866211,
+      "logps/rejected": -115.17778015136719,
+      "loss": 0.6531,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.0008405310800299048,
+      "rewards/margins": 0.08627375215291977,
+      "rewards/rejected": -0.085433229804039,
+      "step": 180
+    },
+    {
+      "epoch": 1.5231388329979878,
+      "grad_norm": 2.756197690963745,
+      "learning_rate": 1.496e-05,
+      "logits/chosen": -7.959776401519775,
+      "logits/rejected": -7.889138698577881,
+      "logps/chosen": -137.98397827148438,
+      "logps/rejected": -134.6807403564453,
+      "loss": 0.6609,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.0001740553416311741,
+      "rewards/margins": 0.06992082297801971,
+      "rewards/rejected": -0.06974677741527557,
+      "step": 190
+    },
+    {
+      "epoch": 1.6036217303822937,
+      "grad_norm": 3.126654624938965,
+      "learning_rate": 1.4693333333333336e-05,
+      "logits/chosen": -7.728564262390137,
+      "logits/rejected": -7.694819450378418,
+      "logps/chosen": -140.72019958496094,
+      "logps/rejected": -146.0216064453125,
+      "loss": 0.6504,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.004489220678806305,
+      "rewards/margins": 0.09480009973049164,
+      "rewards/rejected": -0.09031088650226593,
+      "step": 200
+    },
+    {
+      "epoch": 1.6841046277665996,
+      "grad_norm": 2.8069660663604736,
+      "learning_rate": 1.4426666666666669e-05,
+      "logits/chosen": -8.206178665161133,
+      "logits/rejected": -7.909379482269287,
+      "logps/chosen": -133.2583465576172,
+      "logps/rejected": -140.89820861816406,
+      "loss": 0.6309,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.0008678575977683067,
+      "rewards/margins": 0.14069929718971252,
+      "rewards/rejected": -0.13983140885829926,
+      "step": 210
+    },
+    {
+      "epoch": 1.7645875251509056,
+      "grad_norm": 3.165580987930298,
+      "learning_rate": 1.416e-05,
+      "logits/chosen": -7.514456748962402,
+      "logits/rejected": -7.704525947570801,
+      "logps/chosen": -133.6655731201172,
+      "logps/rejected": -133.7900390625,
+      "loss": 0.6461,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": -0.018594294786453247,
+      "rewards/margins": 0.1046491265296936,
+      "rewards/rejected": -0.12324341386556625,
+      "step": 220
+    },
+    {
+      "epoch": 1.8450704225352113,
+      "grad_norm": 2.8950154781341553,
+      "learning_rate": 1.3893333333333335e-05,
+      "logits/chosen": -7.968588352203369,
+      "logits/rejected": -7.763016700744629,
+      "logps/chosen": -121.14384460449219,
+      "logps/rejected": -138.0037384033203,
+      "loss": 0.6169,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.04061353951692581,
+      "rewards/margins": 0.16627803444862366,
+      "rewards/rejected": -0.12566450238227844,
+      "step": 230
+    },
+    {
+      "epoch": 1.925553319919517,
+      "grad_norm": 2.657349109649658,
+      "learning_rate": 1.3626666666666668e-05,
+      "logits/chosen": -7.7075958251953125,
+      "logits/rejected": -7.8631134033203125,
+      "logps/chosen": -135.63150024414062,
+      "logps/rejected": -142.60678100585938,
+      "loss": 0.6269,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.0013963343808427453,
+      "rewards/margins": 0.1436794400215149,
+      "rewards/rejected": -0.1422831118106842,
+      "step": 240
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 1.3372068405151367,
+      "learning_rate": 1.3360000000000003e-05,
+      "logits/chosen": -8.093542098999023,
+      "logits/rejected": -7.9440999031066895,
+      "logps/chosen": -126.51072692871094,
+      "logps/rejected": -128.38294982910156,
+      "loss": 0.6052,
+      "rewards/accuracies": 0.7027027010917664,
+      "rewards/chosen": 0.015508824028074741,
+      "rewards/margins": 0.10147809982299805,
+      "rewards/rejected": -0.08596926927566528,
+      "step": 250
+    },
+    {
+      "epoch": 2.080482897384306,
+      "grad_norm": 2.8451366424560547,
+      "learning_rate": 1.3093333333333334e-05,
+      "logits/chosen": -7.710860252380371,
+      "logits/rejected": -7.813695430755615,
+      "logps/chosen": -131.70631408691406,
+      "logps/rejected": -132.4761962890625,
+      "loss": 0.6136,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": 0.05115525797009468,
+      "rewards/margins": 0.1773657351732254,
+      "rewards/rejected": -0.12621048092842102,
+      "step": 260
+    },
+    {
+      "epoch": 2.160965794768612,
+      "grad_norm": 3.0381789207458496,
+      "learning_rate": 1.2826666666666667e-05,
+      "logits/chosen": -7.992362976074219,
+      "logits/rejected": -8.11289119720459,
+      "logps/chosen": -148.1260986328125,
+      "logps/rejected": -150.82127380371094,
+      "loss": 0.6069,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": -0.01732814498245716,
+      "rewards/margins": 0.19258996844291687,
+      "rewards/rejected": -0.20991814136505127,
+      "step": 270
+    },
+    {
+      "epoch": 2.2414486921529173,
+      "grad_norm": 3.104029417037964,
+      "learning_rate": 1.2560000000000002e-05,
+      "logits/chosen": -7.620333671569824,
+      "logits/rejected": -7.614747524261475,
+      "logps/chosen": -120.11073303222656,
+      "logps/rejected": -146.62948608398438,
+      "loss": 0.6026,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.06914319843053818,
+      "rewards/margins": 0.22265009582042694,
+      "rewards/rejected": -0.15350690484046936,
+      "step": 280
+    },
+    {
+      "epoch": 2.3219315895372232,
+      "grad_norm": 2.918400526046753,
+      "learning_rate": 1.2293333333333335e-05,
+      "logits/chosen": -8.447749137878418,
+      "logits/rejected": -8.199603080749512,
+      "logps/chosen": -123.7292709350586,
+      "logps/rejected": -141.75552368164062,
+      "loss": 0.6137,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": -0.0003969132958445698,
+      "rewards/margins": 0.17746230959892273,
+      "rewards/rejected": -0.17785921692848206,
+      "step": 290
+    },
+    {
+      "epoch": 2.402414486921529,
+      "grad_norm": 3.638888120651245,
+      "learning_rate": 1.202666666666667e-05,
+      "logits/chosen": -7.351523399353027,
+      "logits/rejected": -7.5525007247924805,
+      "logps/chosen": -114.2864990234375,
+      "logps/rejected": -128.71466064453125,
+      "loss": 0.5855,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.08879393339157104,
+      "rewards/margins": 0.24962946772575378,
+      "rewards/rejected": -0.16083553433418274,
+      "step": 300
+    },
+    {
+      "epoch": 2.482897384305835,
+      "grad_norm": 2.6371285915374756,
+      "learning_rate": 1.1760000000000001e-05,
+      "logits/chosen": -7.6833176612854,
+      "logits/rejected": -7.768864631652832,
+      "logps/chosen": -119.72042083740234,
+      "logps/rejected": -116.38240051269531,
+      "loss": 0.5752,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.0737290009856224,
+      "rewards/margins": 0.267278254032135,
+      "rewards/rejected": -0.1935492306947708,
+      "step": 310
+    },
+    {
+      "epoch": 2.563380281690141,
+      "grad_norm": 3.9384307861328125,
+      "learning_rate": 1.1493333333333334e-05,
+      "logits/chosen": -7.878905296325684,
+      "logits/rejected": -7.912691593170166,
+      "logps/chosen": -130.33595275878906,
+      "logps/rejected": -141.26541137695312,
+      "loss": 0.5933,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.0017801120411604643,
+      "rewards/margins": 0.22636179625988007,
+      "rewards/rejected": -0.22458168864250183,
+      "step": 320
+    },
+    {
+      "epoch": 2.6438631790744465,
+      "grad_norm": 3.0923478603363037,
+      "learning_rate": 1.1226666666666669e-05,
+      "logits/chosen": -8.036542892456055,
+      "logits/rejected": -8.248276710510254,
+      "logps/chosen": -125.16792297363281,
+      "logps/rejected": -132.63623046875,
+      "loss": 0.5676,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.07399231195449829,
+      "rewards/margins": 0.29761144518852234,
+      "rewards/rejected": -0.22361913323402405,
+      "step": 330
+    },
+    {
+      "epoch": 2.7243460764587524,
+      "grad_norm": 4.881553649902344,
+      "learning_rate": 1.0960000000000002e-05,
+      "logits/chosen": -7.877285957336426,
+      "logits/rejected": -8.097951889038086,
+      "logps/chosen": -126.46671295166016,
+      "logps/rejected": -139.54888916015625,
+      "loss": 0.5812,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": -0.05153612047433853,
+      "rewards/margins": 0.2616890072822571,
+      "rewards/rejected": -0.3132251501083374,
+      "step": 340
+    },
+    {
+      "epoch": 2.8048289738430583,
+      "grad_norm": 3.55159068107605,
+      "learning_rate": 1.0693333333333333e-05,
+      "logits/chosen": -7.992476463317871,
+      "logits/rejected": -7.782661437988281,
+      "logps/chosen": -144.28924560546875,
+      "logps/rejected": -154.6568145751953,
+      "loss": 0.578,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": -0.04911986365914345,
+      "rewards/margins": 0.29945996403694153,
+      "rewards/rejected": -0.3485798239707947,
+      "step": 350
+    },
+    {
+      "epoch": 2.885311871227364,
+      "grad_norm": 3.865807056427002,
+      "learning_rate": 1.0426666666666668e-05,
+      "logits/chosen": -8.330907821655273,
+      "logits/rejected": -8.05485725402832,
+      "logps/chosen": -145.71182250976562,
+      "logps/rejected": -136.82615661621094,
+      "loss": 0.5965,
+      "rewards/accuracies": 0.7250000238418579,
+      "rewards/chosen": -0.02360793948173523,
+      "rewards/margins": 0.23897425830364227,
+      "rewards/rejected": -0.2625822126865387,
+      "step": 360
+    },
+    {
+      "epoch": 2.96579476861167,
+      "grad_norm": 5.761505126953125,
+      "learning_rate": 1.0160000000000001e-05,
+      "logits/chosen": -7.840609073638916,
+      "logits/rejected": -7.903719425201416,
+      "logps/chosen": -130.0011444091797,
+      "logps/rejected": -134.54214477539062,
+      "loss": 0.5836,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.01680201105773449,
+      "rewards/margins": 0.30428779125213623,
+      "rewards/rejected": -0.2874857783317566,
+      "step": 370
+    },
+    {
+      "epoch": 3.0402414486921527,
+      "grad_norm": 5.530464172363281,
+      "learning_rate": 9.893333333333334e-06,
+      "logits/chosen": -7.832857608795166,
+      "logits/rejected": -7.836492538452148,
+      "logps/chosen": -126.37843322753906,
+      "logps/rejected": -122.53112030029297,
+      "loss": 0.5751,
+      "rewards/accuracies": 0.7027027010917664,
+      "rewards/chosen": -0.007279620040208101,
+      "rewards/margins": 0.2105865776538849,
+      "rewards/rejected": -0.2178661823272705,
+      "step": 380
+    },
+    {
+      "epoch": 3.1207243460764587,
+      "grad_norm": 2.370408296585083,
+      "learning_rate": 9.626666666666667e-06,
+      "logits/chosen": -7.422645568847656,
+      "logits/rejected": -7.677459716796875,
+      "logps/chosen": -125.61036682128906,
+      "logps/rejected": -142.198486328125,
+      "loss": 0.5097,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.1445888727903366,
+      "rewards/margins": 0.4941856265068054,
+      "rewards/rejected": -0.3495967984199524,
+      "step": 390
+    },
+    {
+      "epoch": 3.2012072434607646,
+      "grad_norm": 2.9659812450408936,
+      "learning_rate": 9.360000000000002e-06,
+      "logits/chosen": -7.914555549621582,
+      "logits/rejected": -7.748204708099365,
+      "logps/chosen": -126.63890075683594,
+      "logps/rejected": -142.98391723632812,
+      "loss": 0.5004,
+      "rewards/accuracies": 0.925000011920929,
+      "rewards/chosen": 0.15164130926132202,
+      "rewards/margins": 0.46555933356285095,
+      "rewards/rejected": -0.31391802430152893,
+      "step": 400
+    },
+    {
+      "epoch": 3.2816901408450705,
+      "grad_norm": 4.255645751953125,
+      "learning_rate": 9.093333333333333e-06,
+      "logits/chosen": -8.225273132324219,
+      "logits/rejected": -8.452564239501953,
+      "logps/chosen": -139.03164672851562,
+      "logps/rejected": -141.5215606689453,
+      "loss": 0.5326,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.07439279556274414,
+      "rewards/margins": 0.40740150213241577,
+      "rewards/rejected": -0.33300870656967163,
+      "step": 410
+    },
+    {
+      "epoch": 3.3621730382293764,
+      "grad_norm": 3.6998543739318848,
+      "learning_rate": 8.826666666666668e-06,
+      "logits/chosen": -7.763664245605469,
+      "logits/rejected": -7.92657470703125,
+      "logps/chosen": -138.3274383544922,
+      "logps/rejected": -139.1911163330078,
+      "loss": 0.5505,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.045943666249513626,
+      "rewards/margins": 0.3697761595249176,
+      "rewards/rejected": -0.3238324820995331,
+      "step": 420
+    },
+    {
+      "epoch": 3.442655935613682,
+      "grad_norm": 2.9942128658294678,
+      "learning_rate": 8.560000000000001e-06,
+      "logits/chosen": -7.518572807312012,
+      "logits/rejected": -7.9670090675354,
+      "logps/chosen": -120.34236145019531,
+      "logps/rejected": -144.70004272460938,
+      "loss": 0.4991,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.11089307069778442,
+      "rewards/margins": 0.49521318078041077,
+      "rewards/rejected": -0.38432011008262634,
+      "step": 430
+    },
+    {
+      "epoch": 3.523138832997988,
+      "grad_norm": 4.232895851135254,
+      "learning_rate": 8.293333333333334e-06,
+      "logits/chosen": -8.115842819213867,
+      "logits/rejected": -7.9553937911987305,
+      "logps/chosen": -131.760986328125,
+      "logps/rejected": -138.90676879882812,
+      "loss": 0.5524,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.02769925631582737,
+      "rewards/margins": 0.36288028955459595,
+      "rewards/rejected": -0.3351810574531555,
+      "step": 440
+    },
+    {
+      "epoch": 3.6036217303822937,
+      "grad_norm": 3.185037851333618,
+      "learning_rate": 8.026666666666667e-06,
+      "logits/chosen": -8.362442970275879,
+      "logits/rejected": -8.021516799926758,
+      "logps/chosen": -136.4081573486328,
+      "logps/rejected": -128.19985961914062,
+      "loss": 0.5657,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.07184217125177383,
+      "rewards/margins": 0.3203974664211273,
+      "rewards/rejected": -0.2485552728176117,
+      "step": 450
+    },
+    {
+      "epoch": 3.6841046277665996,
+      "grad_norm": 4.260532855987549,
+      "learning_rate": 7.76e-06,
+      "logits/chosen": -7.6452765464782715,
+      "logits/rejected": -7.829669952392578,
+      "logps/chosen": -118.22314453125,
+      "logps/rejected": -128.7630615234375,
+      "loss": 0.5738,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": 0.026561100035905838,
+      "rewards/margins": 0.29474154114723206,
+      "rewards/rejected": -0.2681804597377777,
+      "step": 460
+    },
+    {
+      "epoch": 3.7645875251509056,
+      "grad_norm": 3.9909141063690186,
+      "learning_rate": 7.493333333333333e-06,
+      "logits/chosen": -8.017112731933594,
+      "logits/rejected": -7.949077606201172,
+      "logps/chosen": -111.22465515136719,
+      "logps/rejected": -145.2229461669922,
+      "loss": 0.5364,
+      "rewards/accuracies": 0.7250000238418579,
+      "rewards/chosen": 0.08945528417825699,
+      "rewards/margins": 0.4134772717952728,
+      "rewards/rejected": -0.32402199506759644,
+      "step": 470
+    },
+    {
+      "epoch": 3.845070422535211,
+      "grad_norm": 4.2791523933410645,
+      "learning_rate": 7.226666666666667e-06,
+      "logits/chosen": -8.009933471679688,
+      "logits/rejected": -7.980963706970215,
+      "logps/chosen": -143.8748779296875,
+      "logps/rejected": -151.89736938476562,
+      "loss": 0.5372,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.024521049112081528,
+      "rewards/margins": 0.40748023986816406,
+      "rewards/rejected": -0.38295918703079224,
+      "step": 480
+    },
+    {
+      "epoch": 3.925553319919517,
+      "grad_norm": 3.2610886096954346,
+      "learning_rate": 6.96e-06,
+      "logits/chosen": -7.851205348968506,
+      "logits/rejected": -7.731414794921875,
+      "logps/chosen": -123.91703033447266,
+      "logps/rejected": -123.8227767944336,
+      "loss": 0.5066,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.10982272773981094,
+      "rewards/margins": 0.46656376123428345,
+      "rewards/rejected": -0.3567410409450531,
+      "step": 490
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 2.3405067920684814,
+      "learning_rate": 6.693333333333334e-06,
+      "logits/chosen": -8.007993698120117,
+      "logits/rejected": -7.974534511566162,
+      "logps/chosen": -136.6571502685547,
+      "logps/rejected": -155.05227661132812,
+      "loss": 0.5037,
+      "rewards/accuracies": 0.7837837934494019,
+      "rewards/chosen": 0.05127580463886261,
+      "rewards/margins": 0.39702004194259644,
+      "rewards/rejected": -0.34574422240257263,
+      "step": 500
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 750,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 7,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-500/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:221328b488cbd7c8001fec74cbc28a889d631151b99e80fa9d7de1e2595f7246
+size 6200

checkpoint-750/README.md

@@ -0,0 +1,202 @@
+---
+base_model: openlm-research/open_llama_3b
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-750/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "openlm-research/open_llama_3b",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-750/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:607c3f2ed0bcf7175d2653b204b1d9456d5338a559ae6f2b0882238f2a4d2ae0
+size 10663320

checkpoint-750/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:000610572ec96ce116395efd49923a098ee3865ece676de762d5cc76f629b24b
+size 21386746

checkpoint-750/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95b6047bd8cc6f4cdf7c46dea47edb8e542435510070c6cd1e0a7d9ccf5fd7da
+size 14244

checkpoint-750/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a504ee88fd535edcd1ff9dcbaad07a4a854c27733e5765fb9b8035b9ab72d593
+size 1064

checkpoint-750/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-750/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab1b681ec7fc02fed5edd3026687d7a692a918c4dd8e150ca2e3994a6229843b
+size 534194

checkpoint-750/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "legacy": true,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-750/trainer_state.json

@@ -0,0 +1,1159 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.0,
+  "eval_steps": 500,
+  "global_step": 750,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.08048289738430583,
+      "grad_norm": 1.8390964269638062,
+      "learning_rate": 1.976e-05,
+      "logits/chosen": -7.928730010986328,
+      "logits/rejected": -7.768202304840088,
+      "logps/chosen": -126.51554870605469,
+      "logps/rejected": -141.75454711914062,
+      "loss": 0.6957,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": -0.002580838045105338,
+      "rewards/margins": -0.004946361295878887,
+      "rewards/rejected": 0.0023655227851122618,
+      "step": 10
+    },
+    {
+      "epoch": 0.16096579476861167,
+      "grad_norm": 2.1244845390319824,
+      "learning_rate": 1.9493333333333335e-05,
+      "logits/chosen": -8.07243537902832,
+      "logits/rejected": -8.191374778747559,
+      "logps/chosen": -140.3431854248047,
+      "logps/rejected": -127.8196029663086,
+      "loss": 0.6941,
+      "rewards/accuracies": 0.4749999940395355,
+      "rewards/chosen": 0.0015918923309072852,
+      "rewards/margins": -0.0016587398713454604,
+      "rewards/rejected": 0.0032506324350833893,
+      "step": 20
+    },
+    {
+      "epoch": 0.2414486921529175,
+      "grad_norm": 2.229215383529663,
+      "learning_rate": 1.922666666666667e-05,
+      "logits/chosen": -7.767237663269043,
+      "logits/rejected": -7.859239101409912,
+      "logps/chosen": -140.90402221679688,
+      "logps/rejected": -136.17660522460938,
+      "loss": 0.6931,
+      "rewards/accuracies": 0.4749999940395355,
+      "rewards/chosen": 0.0016952038276940584,
+      "rewards/margins": 0.0003137874882668257,
+      "rewards/rejected": 0.001381416223011911,
+      "step": 30
+    },
+    {
+      "epoch": 0.32193158953722334,
+      "grad_norm": 1.4213330745697021,
+      "learning_rate": 1.896e-05,
+      "logits/chosen": -7.892869472503662,
+      "logits/rejected": -7.617051124572754,
+      "logps/chosen": -121.54151916503906,
+      "logps/rejected": -132.56336975097656,
+      "loss": 0.6901,
+      "rewards/accuracies": 0.574999988079071,
+      "rewards/chosen": 0.009255774319171906,
+      "rewards/margins": 0.006459495518356562,
+      "rewards/rejected": 0.0027962785679847,
+      "step": 40
+    },
+    {
+      "epoch": 0.4024144869215292,
+      "grad_norm": 2.9331471920013428,
+      "learning_rate": 1.8693333333333333e-05,
+      "logits/chosen": -7.751120090484619,
+      "logits/rejected": -7.877864837646484,
+      "logps/chosen": -125.05207824707031,
+      "logps/rejected": -139.2479248046875,
+      "loss": 0.6888,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.013416772708296776,
+      "rewards/margins": 0.008954327553510666,
+      "rewards/rejected": 0.004462444689124823,
+      "step": 50
+    },
+    {
+      "epoch": 0.482897384305835,
+      "grad_norm": 1.9477262496948242,
+      "learning_rate": 1.8426666666666668e-05,
+      "logits/chosen": -7.746337890625,
+      "logits/rejected": -7.790966987609863,
+      "logps/chosen": -108.3724365234375,
+      "logps/rejected": -141.29953002929688,
+      "loss": 0.6947,
+      "rewards/accuracies": 0.44999998807907104,
+      "rewards/chosen": 0.0028126670513302088,
+      "rewards/margins": -0.0029423837549984455,
+      "rewards/rejected": 0.005755049642175436,
+      "step": 60
+    },
+    {
+      "epoch": 0.5633802816901409,
+      "grad_norm": 2.476123809814453,
+      "learning_rate": 1.8160000000000002e-05,
+      "logits/chosen": -7.934849739074707,
+      "logits/rejected": -7.784144401550293,
+      "logps/chosen": -148.48281860351562,
+      "logps/rejected": -140.32388305664062,
+      "loss": 0.6895,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.016439538449048996,
+      "rewards/margins": 0.007786408998072147,
+      "rewards/rejected": 0.008653131313621998,
+      "step": 70
+    },
+    {
+      "epoch": 0.6438631790744467,
+      "grad_norm": 1.9035556316375732,
+      "learning_rate": 1.7893333333333337e-05,
+      "logits/chosen": -7.956778526306152,
+      "logits/rejected": -7.880636692047119,
+      "logps/chosen": -134.6743621826172,
+      "logps/rejected": -142.11549377441406,
+      "loss": 0.6855,
+      "rewards/accuracies": 0.6499999761581421,
+      "rewards/chosen": 0.005495529621839523,
+      "rewards/margins": 0.015812702476978302,
+      "rewards/rejected": -0.010317172855138779,
+      "step": 80
+    },
+    {
+      "epoch": 0.7243460764587525,
+      "grad_norm": 2.0266778469085693,
+      "learning_rate": 1.762666666666667e-05,
+      "logits/chosen": -7.979184627532959,
+      "logits/rejected": -8.19025993347168,
+      "logps/chosen": -129.11795043945312,
+      "logps/rejected": -133.7857666015625,
+      "loss": 0.6911,
+      "rewards/accuracies": 0.5249999761581421,
+      "rewards/chosen": 0.00029927687137387693,
+      "rewards/margins": 0.0047690351493656635,
+      "rewards/rejected": -0.004469756968319416,
+      "step": 90
+    },
+    {
+      "epoch": 0.8048289738430584,
+      "grad_norm": 2.324070453643799,
+      "learning_rate": 1.736e-05,
+      "logits/chosen": -7.854997158050537,
+      "logits/rejected": -7.669028282165527,
+      "logps/chosen": -129.60366821289062,
+      "logps/rejected": -129.22799682617188,
+      "loss": 0.6872,
+      "rewards/accuracies": 0.5249999761581421,
+      "rewards/chosen": 0.005187320522964001,
+      "rewards/margins": 0.012428809888660908,
+      "rewards/rejected": -0.007241488434374332,
+      "step": 100
+    },
+    {
+      "epoch": 0.8853118712273642,
+      "grad_norm": 2.3035223484039307,
+      "learning_rate": 1.7093333333333335e-05,
+      "logits/chosen": -7.684693336486816,
+      "logits/rejected": -8.104289054870605,
+      "logps/chosen": -135.45704650878906,
+      "logps/rejected": -142.0008544921875,
+      "loss": 0.6842,
+      "rewards/accuracies": 0.625,
+      "rewards/chosen": -0.0016596360364928842,
+      "rewards/margins": 0.0194082073867321,
+      "rewards/rejected": -0.021067844703793526,
+      "step": 110
+    },
+    {
+      "epoch": 0.96579476861167,
+      "grad_norm": 2.0869996547698975,
+      "learning_rate": 1.682666666666667e-05,
+      "logits/chosen": -8.05040168762207,
+      "logits/rejected": -8.025721549987793,
+      "logps/chosen": -127.3593978881836,
+      "logps/rejected": -122.40535736083984,
+      "loss": 0.6837,
+      "rewards/accuracies": 0.550000011920929,
+      "rewards/chosen": -0.012504220008850098,
+      "rewards/margins": 0.0199703611433506,
+      "rewards/rejected": -0.0324745811522007,
+      "step": 120
+    },
+    {
+      "epoch": 1.040241448692153,
+      "grad_norm": 1.9564281702041626,
+      "learning_rate": 1.656e-05,
+      "logits/chosen": -8.038365364074707,
+      "logits/rejected": -8.249411582946777,
+      "logps/chosen": -122.29254913330078,
+      "logps/rejected": -139.63780212402344,
+      "loss": 0.628,
+      "rewards/accuracies": 0.6486486196517944,
+      "rewards/chosen": -0.008433423936367035,
+      "rewards/margins": 0.0293881893157959,
+      "rewards/rejected": -0.03782161325216293,
+      "step": 130
+    },
+    {
+      "epoch": 1.1207243460764587,
+      "grad_norm": 2.4709627628326416,
+      "learning_rate": 1.6293333333333335e-05,
+      "logits/chosen": -7.3343305587768555,
+      "logits/rejected": -7.533148288726807,
+      "logps/chosen": -125.78892517089844,
+      "logps/rejected": -131.87570190429688,
+      "loss": 0.6687,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.01648072525858879,
+      "rewards/margins": 0.05134889483451843,
+      "rewards/rejected": -0.03486816957592964,
+      "step": 140
+    },
+    {
+      "epoch": 1.2012072434607646,
+      "grad_norm": 2.2346677780151367,
+      "learning_rate": 1.6026666666666667e-05,
+      "logits/chosen": -7.965804100036621,
+      "logits/rejected": -8.229939460754395,
+      "logps/chosen": -134.5069580078125,
+      "logps/rejected": -150.25515747070312,
+      "loss": 0.6559,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.00035553425550460815,
+      "rewards/margins": 0.0779074877500534,
+      "rewards/rejected": -0.0775519534945488,
+      "step": 150
+    },
+    {
+      "epoch": 1.2816901408450705,
+      "grad_norm": 2.3957595825195312,
+      "learning_rate": 1.576e-05,
+      "logits/chosen": -7.930548667907715,
+      "logits/rejected": -8.198356628417969,
+      "logps/chosen": -127.7699203491211,
+      "logps/rejected": -140.8756866455078,
+      "loss": 0.6581,
+      "rewards/accuracies": 0.7250000238418579,
+      "rewards/chosen": 0.0083924550563097,
+      "rewards/margins": 0.07435399293899536,
+      "rewards/rejected": -0.06596153974533081,
+      "step": 160
+    },
+    {
+      "epoch": 1.3621730382293762,
+      "grad_norm": 2.6251115798950195,
+      "learning_rate": 1.5493333333333333e-05,
+      "logits/chosen": -8.110880851745605,
+      "logits/rejected": -8.14248275756836,
+      "logps/chosen": -124.23824310302734,
+      "logps/rejected": -133.23915100097656,
+      "loss": 0.6484,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.02494233287870884,
+      "rewards/margins": 0.09594295918941498,
+      "rewards/rejected": -0.07100063562393188,
+      "step": 170
+    },
+    {
+      "epoch": 1.442655935613682,
+      "grad_norm": 2.497619867324829,
+      "learning_rate": 1.5226666666666668e-05,
+      "logits/chosen": -7.945010185241699,
+      "logits/rejected": -7.939410209655762,
+      "logps/chosen": -116.3001937866211,
+      "logps/rejected": -115.17778015136719,
+      "loss": 0.6531,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.0008405310800299048,
+      "rewards/margins": 0.08627375215291977,
+      "rewards/rejected": -0.085433229804039,
+      "step": 180
+    },
+    {
+      "epoch": 1.5231388329979878,
+      "grad_norm": 2.756197690963745,
+      "learning_rate": 1.496e-05,
+      "logits/chosen": -7.959776401519775,
+      "logits/rejected": -7.889138698577881,
+      "logps/chosen": -137.98397827148438,
+      "logps/rejected": -134.6807403564453,
+      "loss": 0.6609,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.0001740553416311741,
+      "rewards/margins": 0.06992082297801971,
+      "rewards/rejected": -0.06974677741527557,
+      "step": 190
+    },
+    {
+      "epoch": 1.6036217303822937,
+      "grad_norm": 3.126654624938965,
+      "learning_rate": 1.4693333333333336e-05,
+      "logits/chosen": -7.728564262390137,
+      "logits/rejected": -7.694819450378418,
+      "logps/chosen": -140.72019958496094,
+      "logps/rejected": -146.0216064453125,
+      "loss": 0.6504,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.004489220678806305,
+      "rewards/margins": 0.09480009973049164,
+      "rewards/rejected": -0.09031088650226593,
+      "step": 200
+    },
+    {
+      "epoch": 1.6841046277665996,
+      "grad_norm": 2.8069660663604736,
+      "learning_rate": 1.4426666666666669e-05,
+      "logits/chosen": -8.206178665161133,
+      "logits/rejected": -7.909379482269287,
+      "logps/chosen": -133.2583465576172,
+      "logps/rejected": -140.89820861816406,
+      "loss": 0.6309,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.0008678575977683067,
+      "rewards/margins": 0.14069929718971252,
+      "rewards/rejected": -0.13983140885829926,
+      "step": 210
+    },
+    {
+      "epoch": 1.7645875251509056,
+      "grad_norm": 3.165580987930298,
+      "learning_rate": 1.416e-05,
+      "logits/chosen": -7.514456748962402,
+      "logits/rejected": -7.704525947570801,
+      "logps/chosen": -133.6655731201172,
+      "logps/rejected": -133.7900390625,
+      "loss": 0.6461,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": -0.018594294786453247,
+      "rewards/margins": 0.1046491265296936,
+      "rewards/rejected": -0.12324341386556625,
+      "step": 220
+    },
+    {
+      "epoch": 1.8450704225352113,
+      "grad_norm": 2.8950154781341553,
+      "learning_rate": 1.3893333333333335e-05,
+      "logits/chosen": -7.968588352203369,
+      "logits/rejected": -7.763016700744629,
+      "logps/chosen": -121.14384460449219,
+      "logps/rejected": -138.0037384033203,
+      "loss": 0.6169,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.04061353951692581,
+      "rewards/margins": 0.16627803444862366,
+      "rewards/rejected": -0.12566450238227844,
+      "step": 230
+    },
+    {
+      "epoch": 1.925553319919517,
+      "grad_norm": 2.657349109649658,
+      "learning_rate": 1.3626666666666668e-05,
+      "logits/chosen": -7.7075958251953125,
+      "logits/rejected": -7.8631134033203125,
+      "logps/chosen": -135.63150024414062,
+      "logps/rejected": -142.60678100585938,
+      "loss": 0.6269,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.0013963343808427453,
+      "rewards/margins": 0.1436794400215149,
+      "rewards/rejected": -0.1422831118106842,
+      "step": 240
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 1.3372068405151367,
+      "learning_rate": 1.3360000000000003e-05,
+      "logits/chosen": -8.093542098999023,
+      "logits/rejected": -7.9440999031066895,
+      "logps/chosen": -126.51072692871094,
+      "logps/rejected": -128.38294982910156,
+      "loss": 0.6052,
+      "rewards/accuracies": 0.7027027010917664,
+      "rewards/chosen": 0.015508824028074741,
+      "rewards/margins": 0.10147809982299805,
+      "rewards/rejected": -0.08596926927566528,
+      "step": 250
+    },
+    {
+      "epoch": 2.080482897384306,
+      "grad_norm": 2.8451366424560547,
+      "learning_rate": 1.3093333333333334e-05,
+      "logits/chosen": -7.710860252380371,
+      "logits/rejected": -7.813695430755615,
+      "logps/chosen": -131.70631408691406,
+      "logps/rejected": -132.4761962890625,
+      "loss": 0.6136,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": 0.05115525797009468,
+      "rewards/margins": 0.1773657351732254,
+      "rewards/rejected": -0.12621048092842102,
+      "step": 260
+    },
+    {
+      "epoch": 2.160965794768612,
+      "grad_norm": 3.0381789207458496,
+      "learning_rate": 1.2826666666666667e-05,
+      "logits/chosen": -7.992362976074219,
+      "logits/rejected": -8.11289119720459,
+      "logps/chosen": -148.1260986328125,
+      "logps/rejected": -150.82127380371094,
+      "loss": 0.6069,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": -0.01732814498245716,
+      "rewards/margins": 0.19258996844291687,
+      "rewards/rejected": -0.20991814136505127,
+      "step": 270
+    },
+    {
+      "epoch": 2.2414486921529173,
+      "grad_norm": 3.104029417037964,
+      "learning_rate": 1.2560000000000002e-05,
+      "logits/chosen": -7.620333671569824,
+      "logits/rejected": -7.614747524261475,
+      "logps/chosen": -120.11073303222656,
+      "logps/rejected": -146.62948608398438,
+      "loss": 0.6026,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.06914319843053818,
+      "rewards/margins": 0.22265009582042694,
+      "rewards/rejected": -0.15350690484046936,
+      "step": 280
+    },
+    {
+      "epoch": 2.3219315895372232,
+      "grad_norm": 2.918400526046753,
+      "learning_rate": 1.2293333333333335e-05,
+      "logits/chosen": -8.447749137878418,
+      "logits/rejected": -8.199603080749512,
+      "logps/chosen": -123.7292709350586,
+      "logps/rejected": -141.75552368164062,
+      "loss": 0.6137,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": -0.0003969132958445698,
+      "rewards/margins": 0.17746230959892273,
+      "rewards/rejected": -0.17785921692848206,
+      "step": 290
+    },
+    {
+      "epoch": 2.402414486921529,
+      "grad_norm": 3.638888120651245,
+      "learning_rate": 1.202666666666667e-05,
+      "logits/chosen": -7.351523399353027,
+      "logits/rejected": -7.5525007247924805,
+      "logps/chosen": -114.2864990234375,
+      "logps/rejected": -128.71466064453125,
+      "loss": 0.5855,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.08879393339157104,
+      "rewards/margins": 0.24962946772575378,
+      "rewards/rejected": -0.16083553433418274,
+      "step": 300
+    },
+    {
+      "epoch": 2.482897384305835,
+      "grad_norm": 2.6371285915374756,
+      "learning_rate": 1.1760000000000001e-05,
+      "logits/chosen": -7.6833176612854,
+      "logits/rejected": -7.768864631652832,
+      "logps/chosen": -119.72042083740234,
+      "logps/rejected": -116.38240051269531,
+      "loss": 0.5752,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.0737290009856224,
+      "rewards/margins": 0.267278254032135,
+      "rewards/rejected": -0.1935492306947708,
+      "step": 310
+    },
+    {
+      "epoch": 2.563380281690141,
+      "grad_norm": 3.9384307861328125,
+      "learning_rate": 1.1493333333333334e-05,
+      "logits/chosen": -7.878905296325684,
+      "logits/rejected": -7.912691593170166,
+      "logps/chosen": -130.33595275878906,
+      "logps/rejected": -141.26541137695312,
+      "loss": 0.5933,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.0017801120411604643,
+      "rewards/margins": 0.22636179625988007,
+      "rewards/rejected": -0.22458168864250183,
+      "step": 320
+    },
+    {
+      "epoch": 2.6438631790744465,
+      "grad_norm": 3.0923478603363037,
+      "learning_rate": 1.1226666666666669e-05,
+      "logits/chosen": -8.036542892456055,
+      "logits/rejected": -8.248276710510254,
+      "logps/chosen": -125.16792297363281,
+      "logps/rejected": -132.63623046875,
+      "loss": 0.5676,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.07399231195449829,
+      "rewards/margins": 0.29761144518852234,
+      "rewards/rejected": -0.22361913323402405,
+      "step": 330
+    },
+    {
+      "epoch": 2.7243460764587524,
+      "grad_norm": 4.881553649902344,
+      "learning_rate": 1.0960000000000002e-05,
+      "logits/chosen": -7.877285957336426,
+      "logits/rejected": -8.097951889038086,
+      "logps/chosen": -126.46671295166016,
+      "logps/rejected": -139.54888916015625,
+      "loss": 0.5812,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": -0.05153612047433853,
+      "rewards/margins": 0.2616890072822571,
+      "rewards/rejected": -0.3132251501083374,
+      "step": 340
+    },
+    {
+      "epoch": 2.8048289738430583,
+      "grad_norm": 3.55159068107605,
+      "learning_rate": 1.0693333333333333e-05,
+      "logits/chosen": -7.992476463317871,
+      "logits/rejected": -7.782661437988281,
+      "logps/chosen": -144.28924560546875,
+      "logps/rejected": -154.6568145751953,
+      "loss": 0.578,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": -0.04911986365914345,
+      "rewards/margins": 0.29945996403694153,
+      "rewards/rejected": -0.3485798239707947,
+      "step": 350
+    },
+    {
+      "epoch": 2.885311871227364,
+      "grad_norm": 3.865807056427002,
+      "learning_rate": 1.0426666666666668e-05,
+      "logits/chosen": -8.330907821655273,
+      "logits/rejected": -8.05485725402832,
+      "logps/chosen": -145.71182250976562,
+      "logps/rejected": -136.82615661621094,
+      "loss": 0.5965,
+      "rewards/accuracies": 0.7250000238418579,
+      "rewards/chosen": -0.02360793948173523,
+      "rewards/margins": 0.23897425830364227,
+      "rewards/rejected": -0.2625822126865387,
+      "step": 360
+    },
+    {
+      "epoch": 2.96579476861167,
+      "grad_norm": 5.761505126953125,
+      "learning_rate": 1.0160000000000001e-05,
+      "logits/chosen": -7.840609073638916,
+      "logits/rejected": -7.903719425201416,
+      "logps/chosen": -130.0011444091797,
+      "logps/rejected": -134.54214477539062,
+      "loss": 0.5836,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.01680201105773449,
+      "rewards/margins": 0.30428779125213623,
+      "rewards/rejected": -0.2874857783317566,
+      "step": 370
+    },
+    {
+      "epoch": 3.0402414486921527,
+      "grad_norm": 5.530464172363281,
+      "learning_rate": 9.893333333333334e-06,
+      "logits/chosen": -7.832857608795166,
+      "logits/rejected": -7.836492538452148,
+      "logps/chosen": -126.37843322753906,
+      "logps/rejected": -122.53112030029297,
+      "loss": 0.5751,
+      "rewards/accuracies": 0.7027027010917664,
+      "rewards/chosen": -0.007279620040208101,
+      "rewards/margins": 0.2105865776538849,
+      "rewards/rejected": -0.2178661823272705,
+      "step": 380
+    },
+    {
+      "epoch": 3.1207243460764587,
+      "grad_norm": 2.370408296585083,
+      "learning_rate": 9.626666666666667e-06,
+      "logits/chosen": -7.422645568847656,
+      "logits/rejected": -7.677459716796875,
+      "logps/chosen": -125.61036682128906,
+      "logps/rejected": -142.198486328125,
+      "loss": 0.5097,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.1445888727903366,
+      "rewards/margins": 0.4941856265068054,
+      "rewards/rejected": -0.3495967984199524,
+      "step": 390
+    },
+    {
+      "epoch": 3.2012072434607646,
+      "grad_norm": 2.9659812450408936,
+      "learning_rate": 9.360000000000002e-06,
+      "logits/chosen": -7.914555549621582,
+      "logits/rejected": -7.748204708099365,
+      "logps/chosen": -126.63890075683594,
+      "logps/rejected": -142.98391723632812,
+      "loss": 0.5004,
+      "rewards/accuracies": 0.925000011920929,
+      "rewards/chosen": 0.15164130926132202,
+      "rewards/margins": 0.46555933356285095,
+      "rewards/rejected": -0.31391802430152893,
+      "step": 400
+    },
+    {
+      "epoch": 3.2816901408450705,
+      "grad_norm": 4.255645751953125,
+      "learning_rate": 9.093333333333333e-06,
+      "logits/chosen": -8.225273132324219,
+      "logits/rejected": -8.452564239501953,
+      "logps/chosen": -139.03164672851562,
+      "logps/rejected": -141.5215606689453,
+      "loss": 0.5326,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.07439279556274414,
+      "rewards/margins": 0.40740150213241577,
+      "rewards/rejected": -0.33300870656967163,
+      "step": 410
+    },
+    {
+      "epoch": 3.3621730382293764,
+      "grad_norm": 3.6998543739318848,
+      "learning_rate": 8.826666666666668e-06,
+      "logits/chosen": -7.763664245605469,
+      "logits/rejected": -7.92657470703125,
+      "logps/chosen": -138.3274383544922,
+      "logps/rejected": -139.1911163330078,
+      "loss": 0.5505,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.045943666249513626,
+      "rewards/margins": 0.3697761595249176,
+      "rewards/rejected": -0.3238324820995331,
+      "step": 420
+    },
+    {
+      "epoch": 3.442655935613682,
+      "grad_norm": 2.9942128658294678,
+      "learning_rate": 8.560000000000001e-06,
+      "logits/chosen": -7.518572807312012,
+      "logits/rejected": -7.9670090675354,
+      "logps/chosen": -120.34236145019531,
+      "logps/rejected": -144.70004272460938,
+      "loss": 0.4991,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.11089307069778442,
+      "rewards/margins": 0.49521318078041077,
+      "rewards/rejected": -0.38432011008262634,
+      "step": 430
+    },
+    {
+      "epoch": 3.523138832997988,
+      "grad_norm": 4.232895851135254,
+      "learning_rate": 8.293333333333334e-06,
+      "logits/chosen": -8.115842819213867,
+      "logits/rejected": -7.9553937911987305,
+      "logps/chosen": -131.760986328125,
+      "logps/rejected": -138.90676879882812,
+      "loss": 0.5524,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.02769925631582737,
+      "rewards/margins": 0.36288028955459595,
+      "rewards/rejected": -0.3351810574531555,
+      "step": 440
+    },
+    {
+      "epoch": 3.6036217303822937,
+      "grad_norm": 3.185037851333618,
+      "learning_rate": 8.026666666666667e-06,
+      "logits/chosen": -8.362442970275879,
+      "logits/rejected": -8.021516799926758,
+      "logps/chosen": -136.4081573486328,
+      "logps/rejected": -128.19985961914062,
+      "loss": 0.5657,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.07184217125177383,
+      "rewards/margins": 0.3203974664211273,
+      "rewards/rejected": -0.2485552728176117,
+      "step": 450
+    },
+    {
+      "epoch": 3.6841046277665996,
+      "grad_norm": 4.260532855987549,
+      "learning_rate": 7.76e-06,
+      "logits/chosen": -7.6452765464782715,
+      "logits/rejected": -7.829669952392578,
+      "logps/chosen": -118.22314453125,
+      "logps/rejected": -128.7630615234375,
+      "loss": 0.5738,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": 0.026561100035905838,
+      "rewards/margins": 0.29474154114723206,
+      "rewards/rejected": -0.2681804597377777,
+      "step": 460
+    },
+    {
+      "epoch": 3.7645875251509056,
+      "grad_norm": 3.9909141063690186,
+      "learning_rate": 7.493333333333333e-06,
+      "logits/chosen": -8.017112731933594,
+      "logits/rejected": -7.949077606201172,
+      "logps/chosen": -111.22465515136719,
+      "logps/rejected": -145.2229461669922,
+      "loss": 0.5364,
+      "rewards/accuracies": 0.7250000238418579,
+      "rewards/chosen": 0.08945528417825699,
+      "rewards/margins": 0.4134772717952728,
+      "rewards/rejected": -0.32402199506759644,
+      "step": 470
+    },
+    {
+      "epoch": 3.845070422535211,
+      "grad_norm": 4.2791523933410645,
+      "learning_rate": 7.226666666666667e-06,
+      "logits/chosen": -8.009933471679688,
+      "logits/rejected": -7.980963706970215,
+      "logps/chosen": -143.8748779296875,
+      "logps/rejected": -151.89736938476562,
+      "loss": 0.5372,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.024521049112081528,
+      "rewards/margins": 0.40748023986816406,
+      "rewards/rejected": -0.38295918703079224,
+      "step": 480
+    },
+    {
+      "epoch": 3.925553319919517,
+      "grad_norm": 3.2610886096954346,
+      "learning_rate": 6.96e-06,
+      "logits/chosen": -7.851205348968506,
+      "logits/rejected": -7.731414794921875,
+      "logps/chosen": -123.91703033447266,
+      "logps/rejected": -123.8227767944336,
+      "loss": 0.5066,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.10982272773981094,
+      "rewards/margins": 0.46656376123428345,
+      "rewards/rejected": -0.3567410409450531,
+      "step": 490
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 2.3405067920684814,
+      "learning_rate": 6.693333333333334e-06,
+      "logits/chosen": -8.007993698120117,
+      "logits/rejected": -7.974534511566162,
+      "logps/chosen": -136.6571502685547,
+      "logps/rejected": -155.05227661132812,
+      "loss": 0.5037,
+      "rewards/accuracies": 0.7837837934494019,
+      "rewards/chosen": 0.05127580463886261,
+      "rewards/margins": 0.39702004194259644,
+      "rewards/rejected": -0.34574422240257263,
+      "step": 500
+    },
+    {
+      "epoch": 4.0804828973843055,
+      "grad_norm": 3.0611572265625,
+      "learning_rate": 6.426666666666668e-06,
+      "logits/chosen": -7.8229827880859375,
+      "logits/rejected": -8.056346893310547,
+      "logps/chosen": -123.7149887084961,
+      "logps/rejected": -154.68173217773438,
+      "loss": 0.5072,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.04092025011777878,
+      "rewards/margins": 0.4962732791900635,
+      "rewards/rejected": -0.4553530812263489,
+      "step": 510
+    },
+    {
+      "epoch": 4.160965794768612,
+      "grad_norm": 3.2638542652130127,
+      "learning_rate": 6.16e-06,
+      "logits/chosen": -7.906125068664551,
+      "logits/rejected": -7.711597442626953,
+      "logps/chosen": -120.27784729003906,
+      "logps/rejected": -128.5182647705078,
+      "loss": 0.5079,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.06980352103710175,
+      "rewards/margins": 0.5142830610275269,
+      "rewards/rejected": -0.4444795250892639,
+      "step": 520
+    },
+    {
+      "epoch": 4.241448692152917,
+      "grad_norm": 5.444788932800293,
+      "learning_rate": 5.893333333333334e-06,
+      "logits/chosen": -8.111169815063477,
+      "logits/rejected": -7.884283542633057,
+      "logps/chosen": -140.63882446289062,
+      "logps/rejected": -140.9823760986328,
+      "loss": 0.566,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": -0.051231205463409424,
+      "rewards/margins": 0.33118587732315063,
+      "rewards/rejected": -0.38241708278656006,
+      "step": 530
+    },
+    {
+      "epoch": 4.321931589537224,
+      "grad_norm": 3.564617156982422,
+      "learning_rate": 5.626666666666667e-06,
+      "logits/chosen": -8.093330383300781,
+      "logits/rejected": -8.017361640930176,
+      "logps/chosen": -139.41275024414062,
+      "logps/rejected": -138.95274353027344,
+      "loss": 0.5066,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.10449226200580597,
+      "rewards/margins": 0.4961455762386322,
+      "rewards/rejected": -0.39165326952934265,
+      "step": 540
+    },
+    {
+      "epoch": 4.402414486921529,
+      "grad_norm": 4.48131799697876,
+      "learning_rate": 5.36e-06,
+      "logits/chosen": -8.066251754760742,
+      "logits/rejected": -7.98406982421875,
+      "logps/chosen": -134.0875244140625,
+      "logps/rejected": -134.54095458984375,
+      "loss": 0.5085,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.18554718792438507,
+      "rewards/margins": 0.5177052617073059,
+      "rewards/rejected": -0.33215808868408203,
+      "step": 550
+    },
+    {
+      "epoch": 4.482897384305835,
+      "grad_norm": 2.9168806076049805,
+      "learning_rate": 5.093333333333333e-06,
+      "logits/chosen": -8.315153121948242,
+      "logits/rejected": -8.331533432006836,
+      "logps/chosen": -130.7205352783203,
+      "logps/rejected": -128.97933959960938,
+      "loss": 0.495,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 0.07991756498813629,
+      "rewards/margins": 0.5530857443809509,
+      "rewards/rejected": -0.47316819429397583,
+      "step": 560
+    },
+    {
+      "epoch": 4.563380281690141,
+      "grad_norm": 4.712265968322754,
+      "learning_rate": 4.826666666666667e-06,
+      "logits/chosen": -7.964537143707275,
+      "logits/rejected": -8.250347137451172,
+      "logps/chosen": -107.0856704711914,
+      "logps/rejected": -131.0066680908203,
+      "loss": 0.5051,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": 0.03752124309539795,
+      "rewards/margins": 0.5069595575332642,
+      "rewards/rejected": -0.46943825483322144,
+      "step": 570
+    },
+    {
+      "epoch": 4.6438631790744465,
+      "grad_norm": 5.547153472900391,
+      "learning_rate": 4.56e-06,
+      "logits/chosen": -7.999167442321777,
+      "logits/rejected": -7.8645429611206055,
+      "logps/chosen": -123.46644592285156,
+      "logps/rejected": -145.62319946289062,
+      "loss": 0.5287,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.018197838217020035,
+      "rewards/margins": 0.43459415435791016,
+      "rewards/rejected": -0.4163963198661804,
+      "step": 580
+    },
+    {
+      "epoch": 4.724346076458753,
+      "grad_norm": 4.132951259613037,
+      "learning_rate": 4.2933333333333334e-06,
+      "logits/chosen": -7.352658271789551,
+      "logits/rejected": -7.5392165184021,
+      "logps/chosen": -122.26692962646484,
+      "logps/rejected": -144.80618286132812,
+      "loss": 0.4491,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.1904078871011734,
+      "rewards/margins": 0.6379453539848328,
+      "rewards/rejected": -0.4475374221801758,
+      "step": 590
+    },
+    {
+      "epoch": 4.804828973843058,
+      "grad_norm": 3.974529981613159,
+      "learning_rate": 4.026666666666667e-06,
+      "logits/chosen": -7.725615501403809,
+      "logits/rejected": -7.880255222320557,
+      "logps/chosen": -126.33785247802734,
+      "logps/rejected": -141.9647979736328,
+      "loss": 0.4566,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": 0.041528504341840744,
+      "rewards/margins": 0.6287237405776978,
+      "rewards/rejected": -0.5871952176094055,
+      "step": 600
+    },
+    {
+      "epoch": 4.885311871227364,
+      "grad_norm": 5.846194744110107,
+      "learning_rate": 3.7600000000000004e-06,
+      "logits/chosen": -7.711016654968262,
+      "logits/rejected": -7.9464921951293945,
+      "logps/chosen": -140.57571411132812,
+      "logps/rejected": -134.4864959716797,
+      "loss": 0.5305,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": 0.10763657093048096,
+      "rewards/margins": 0.5108700394630432,
+      "rewards/rejected": -0.40323343873023987,
+      "step": 610
+    },
+    {
+      "epoch": 4.96579476861167,
+      "grad_norm": 4.602907180786133,
+      "learning_rate": 3.4933333333333335e-06,
+      "logits/chosen": -7.861893653869629,
+      "logits/rejected": -7.691662788391113,
+      "logps/chosen": -134.75808715820312,
+      "logps/rejected": -151.4270782470703,
+      "loss": 0.5428,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.012985095381736755,
+      "rewards/margins": 0.4465901255607605,
+      "rewards/rejected": -0.4336049556732178,
+      "step": 620
+    },
+    {
+      "epoch": 5.040241448692153,
+      "grad_norm": 4.058650493621826,
+      "learning_rate": 3.226666666666667e-06,
+      "logits/chosen": -7.957271099090576,
+      "logits/rejected": -8.070477485656738,
+      "logps/chosen": -139.47348022460938,
+      "logps/rejected": -148.40493774414062,
+      "loss": 0.4508,
+      "rewards/accuracies": 0.8108108043670654,
+      "rewards/chosen": 0.11836089193820953,
+      "rewards/margins": 0.5762597918510437,
+      "rewards/rejected": -0.457898885011673,
+      "step": 630
+    },
+    {
+      "epoch": 5.120724346076459,
+      "grad_norm": 3.104494571685791,
+      "learning_rate": 2.96e-06,
+      "logits/chosen": -8.098637580871582,
+      "logits/rejected": -8.007506370544434,
+      "logps/chosen": -117.58427429199219,
+      "logps/rejected": -141.4518585205078,
+      "loss": 0.5134,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.013196405954658985,
+      "rewards/margins": 0.4996967315673828,
+      "rewards/rejected": -0.4865003228187561,
+      "step": 640
+    },
+    {
+      "epoch": 5.201207243460765,
+      "grad_norm": 4.8590216636657715,
+      "learning_rate": 2.6933333333333335e-06,
+      "logits/chosen": -7.682847499847412,
+      "logits/rejected": -7.851078987121582,
+      "logps/chosen": -128.17315673828125,
+      "logps/rejected": -143.27740478515625,
+      "loss": 0.4639,
+      "rewards/accuracies": 0.925000011920929,
+      "rewards/chosen": 0.1449861079454422,
+      "rewards/margins": 0.6247243881225586,
+      "rewards/rejected": -0.4797382950782776,
+      "step": 650
+    },
+    {
+      "epoch": 5.28169014084507,
+      "grad_norm": 4.140720367431641,
+      "learning_rate": 2.426666666666667e-06,
+      "logits/chosen": -8.153270721435547,
+      "logits/rejected": -7.843749046325684,
+      "logps/chosen": -145.75784301757812,
+      "logps/rejected": -151.5276641845703,
+      "loss": 0.4924,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": -0.024539103731513023,
+      "rewards/margins": 0.5701876878738403,
+      "rewards/rejected": -0.5947268009185791,
+      "step": 660
+    },
+    {
+      "epoch": 5.362173038229376,
+      "grad_norm": 3.363086223602295,
+      "learning_rate": 2.16e-06,
+      "logits/chosen": -7.592160701751709,
+      "logits/rejected": -8.117759704589844,
+      "logps/chosen": -136.26693725585938,
+      "logps/rejected": -131.3207550048828,
+      "loss": 0.5143,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.13678143918514252,
+      "rewards/margins": 0.4933203160762787,
+      "rewards/rejected": -0.356538861989975,
+      "step": 670
+    },
+    {
+      "epoch": 5.442655935613682,
+      "grad_norm": 4.244340896606445,
+      "learning_rate": 1.8933333333333333e-06,
+      "logits/chosen": -7.977016448974609,
+      "logits/rejected": -7.939810276031494,
+      "logps/chosen": -135.91845703125,
+      "logps/rejected": -139.4970703125,
+      "loss": 0.4766,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": 0.14392784237861633,
+      "rewards/margins": 0.6197702884674072,
+      "rewards/rejected": -0.4758424162864685,
+      "step": 680
+    },
+    {
+      "epoch": 5.523138832997988,
+      "grad_norm": 4.6983747482299805,
+      "learning_rate": 1.6266666666666666e-06,
+      "logits/chosen": -8.18427848815918,
+      "logits/rejected": -8.351648330688477,
+      "logps/chosen": -122.5636978149414,
+      "logps/rejected": -129.7288360595703,
+      "loss": 0.5299,
+      "rewards/accuracies": 0.7749999761581421,
+      "rewards/chosen": 0.012566042132675648,
+      "rewards/margins": 0.44475775957107544,
+      "rewards/rejected": -0.43219175934791565,
+      "step": 690
+    },
+    {
+      "epoch": 5.603621730382294,
+      "grad_norm": 3.8425047397613525,
+      "learning_rate": 1.3600000000000001e-06,
+      "logits/chosen": -8.02586555480957,
+      "logits/rejected": -7.971312046051025,
+      "logps/chosen": -128.48104858398438,
+      "logps/rejected": -124.56768798828125,
+      "loss": 0.5179,
+      "rewards/accuracies": 0.7250000238418579,
+      "rewards/chosen": 0.08885373175144196,
+      "rewards/margins": 0.564620316028595,
+      "rewards/rejected": -0.4757665991783142,
+      "step": 700
+    },
+    {
+      "epoch": 5.684104627766599,
+      "grad_norm": 4.374882221221924,
+      "learning_rate": 1.0933333333333334e-06,
+      "logits/chosen": -7.965134620666504,
+      "logits/rejected": -7.833965301513672,
+      "logps/chosen": -131.6035919189453,
+      "logps/rejected": -149.52066040039062,
+      "loss": 0.5213,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": -0.04283960908651352,
+      "rewards/margins": 0.503414511680603,
+      "rewards/rejected": -0.5462541580200195,
+      "step": 710
+    },
+    {
+      "epoch": 5.7645875251509056,
+      "grad_norm": 5.414626598358154,
+      "learning_rate": 8.266666666666668e-07,
+      "logits/chosen": -7.947386264801025,
+      "logits/rejected": -8.022677421569824,
+      "logps/chosen": -120.8558120727539,
+      "logps/rejected": -135.9299774169922,
+      "loss": 0.4254,
+      "rewards/accuracies": 0.925000011920929,
+      "rewards/chosen": 0.11365139484405518,
+      "rewards/margins": 0.7651789784431458,
+      "rewards/rejected": -0.6515275239944458,
+      "step": 720
+    },
+    {
+      "epoch": 5.845070422535211,
+      "grad_norm": 2.958667278289795,
+      "learning_rate": 5.6e-07,
+      "logits/chosen": -7.740857124328613,
+      "logits/rejected": -7.85732364654541,
+      "logps/chosen": -140.94491577148438,
+      "logps/rejected": -145.21192932128906,
+      "loss": 0.4625,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.13008669018745422,
+      "rewards/margins": 0.6117427349090576,
+      "rewards/rejected": -0.4816560745239258,
+      "step": 730
+    },
+    {
+      "epoch": 5.925553319919517,
+      "grad_norm": 3.8819665908813477,
+      "learning_rate": 2.9333333333333337e-07,
+      "logits/chosen": -7.447596549987793,
+      "logits/rejected": -7.661311149597168,
+      "logps/chosen": -119.31522369384766,
+      "logps/rejected": -155.86141967773438,
+      "loss": 0.4112,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 0.2565927505493164,
+      "rewards/margins": 0.7656124234199524,
+      "rewards/rejected": -0.509019672870636,
+      "step": 740
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 2.323993444442749,
+      "learning_rate": 2.6666666666666667e-08,
+      "logits/chosen": -7.972095489501953,
+      "logits/rejected": -7.644292831420898,
+      "logps/chosen": -119.33430480957031,
+      "logps/rejected": -137.5045928955078,
+      "loss": 0.4655,
+      "rewards/accuracies": 0.7567567825317383,
+      "rewards/chosen": 0.04585576057434082,
+      "rewards/margins": 0.5475000143051147,
+      "rewards/rejected": -0.5016443133354187,
+      "step": 750
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 750,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 7,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-750/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:221328b488cbd7c8001fec74cbc28a889d631151b99e80fa9d7de1e2595f7246
+size 6200

runs/Apr18_06-26-04_81a10bb95825/events.out.tfevents.1744957564.81a10bb95825.14299.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2dc45b7117e3c4454389fc7f22e33e8fb9be5d9d4259d13f9ad7c9bdf7735d6
+size 57765

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab1b681ec7fc02fed5edd3026687d7a692a918c4dd8e150ca2e3994a6229843b
+size 534194

tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "legacy": true,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}