Update README.md
Browse files
README.md
CHANGED
@@ -27,7 +27,9 @@ This project fine-tunes a language model using supervised fine-tuning (SFT) and
|
|
27 |
For optimal performance, **chunk your text** into smaller, coherent pieces before providing it to the model. Long documents can lead the model to focus on specific details rather than the overall context.
|
28 |
|
29 |
- **Training Setup:**
|
30 |
-
The model is fine-tuned using the Unsloth framework with LoRA adapters, taking advantage of an A100 GPU for efficient training.
|
|
|
|
|
31 |
|
32 |
## Quick Usage
|
33 |
|
@@ -70,6 +72,7 @@ _ = model.generate(
|
|
70 |
min_p=0.1,
|
71 |
eos_token_id=tokenizer.eos_token_id, # Ensures proper termination.
|
72 |
)
|
|
|
73 |
|
74 |
# Uploaded model
|
75 |
|
|
|
27 |
For optimal performance, **chunk your text** into smaller, coherent pieces before providing it to the model. Long documents can lead the model to focus on specific details rather than the overall context.
|
28 |
|
29 |
- **Training Setup:**
|
30 |
+
The model is fine-tuned using the Unsloth framework with LoRA adapters, taking advantage of an A100 GPU for efficient training. See W&B loss curve here: https://wandb.ai/prdev/lora_model_training/panel/jp2r24xk7?nw=nwuserprdev
|
31 |
+
|
32 |
+
|
33 |
|
34 |
## Quick Usage
|
35 |
|
|
|
72 |
min_p=0.1,
|
73 |
eos_token_id=tokenizer.eos_token_id, # Ensures proper termination.
|
74 |
)
|
75 |
+
```
|
76 |
|
77 |
# Uploaded model
|
78 |
|