prdev commited on
Commit
c0f7621
·
verified ·
1 Parent(s): 579e358

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -27,7 +27,9 @@ This project fine-tunes a language model using supervised fine-tuning (SFT) and
27
  For optimal performance, **chunk your text** into smaller, coherent pieces before providing it to the model. Long documents can lead the model to focus on specific details rather than the overall context.
28
 
29
  - **Training Setup:**
30
- The model is fine-tuned using the Unsloth framework with LoRA adapters, taking advantage of an A100 GPU for efficient training.
 
 
31
 
32
  ## Quick Usage
33
 
@@ -70,6 +72,7 @@ _ = model.generate(
70
  min_p=0.1,
71
  eos_token_id=tokenizer.eos_token_id, # Ensures proper termination.
72
  )
 
73
 
74
  # Uploaded model
75
 
 
27
  For optimal performance, **chunk your text** into smaller, coherent pieces before providing it to the model. Long documents can lead the model to focus on specific details rather than the overall context.
28
 
29
  - **Training Setup:**
30
+ The model is fine-tuned using the Unsloth framework with LoRA adapters, taking advantage of an A100 GPU for efficient training. See W&B loss curve here: https://wandb.ai/prdev/lora_model_training/panel/jp2r24xk7?nw=nwuserprdev
31
+
32
+
33
 
34
  ## Quick Usage
35
 
 
72
  min_p=0.1,
73
  eos_token_id=tokenizer.eos_token_id, # Ensures proper termination.
74
  )
75
+ ```
76
 
77
  # Uploaded model
78