Likhith003/dpo-llmjudge-lora-adapter

DPO Fine-Tuned Adapter - LLM Judge Dataset

🧠 Model

• Base: meta-llama/Llama-3.2-1B-Instruct
• Fine-tuned using TRL's DPOTrainer with the LLM Judge preference dataset (50 pairs)

⚙️ Training Parameters

Parameter	Value
Learning Rate	5e-5
Batch Size	4
Epochs	3
Beta (DPO regularizer)	0.1
Max Input Length	1024 tokens
Max Prompt Length	512 tokens
Padding Token	eos_token

📦 Dataset

• Source: llm_judge_preferences.csv
• Size: 50 human-labeled pairs with prompt, chosen, and rejected columns

📂 Output

• Adapter saved and uploaded as Likhith003/dpo-llmjudge-lora-adapter