ypdated model training deyails
Browse files
README.md
CHANGED
|
@@ -81,8 +81,8 @@ The model has been trained on source code from 80+ programming languages. The pr
|
|
| 81 |
## Model
|
| 82 |
|
| 83 |
- **Architecture:** GPT-2 model with multi-query attention and Fill-in-the-Middle objective
|
| 84 |
-
-
|
| 85 |
-
- **Pretraining tokens:**
|
| 86 |
- **Precision:** bfloat16
|
| 87 |
|
| 88 |
## Hardware
|
|
|
|
| 81 |
## Model
|
| 82 |
|
| 83 |
- **Architecture:** GPT-2 model with multi-query attention and Fill-in-the-Middle objective
|
| 84 |
+
- **•Fine-training steps:** 50k
|
| 85 |
+
- **Pretraining tokens:** 22 Billion
|
| 86 |
- **Precision:** bfloat16
|
| 87 |
|
| 88 |
## Hardware
|