Summary
This model is a fine-tuned version of the Mistral-7B-Instruct-v0.3 optimised for answering questions in the domain of forensic investigations. The model has been trained using a specialised dataset titled Advanced_Forensic_Investigations_Knowledge_Library_v1, which consists of approximately 100 domain-specific question-answer pairs. The objective is to support advanced forensic investigative reasoning, rapid knowledge retrieval, and high-precision forensic domain assistance.
Model Details
Model Description
Model type: Instruction-following Language Model (LoRA-based fine-tuning)
Language(s): English
Fine-tuned from model: Mistral-7B-Instruct-v0.3
Training Details
Training Data
Dataset:
Advanced_Forensic_Investigations_Knowledge_Library_v1Data size: ~100 high-quality, domain-specific QA pairs
Training Procedure
Preprocessing
Template:
mistralToken truncation/cutoff: 2048
No vocab resizing or prompt packing
Hyperparameters
Finetuning type: LoRA
Precision:
bf16LoRA rank: 16
LoRA alpha: 32
Batch size: 4
Gradient accumulation: 8
Learning rate:
3e-4Epochs: 35
LR scheduler: cosine
Quantisation: 4-bit (bitsandbytes)
Cutoff length: 2048
Compute
Training time: close to 30 minutes
Framework: LLaMA-Factory
Evaluation
Testing Data, Factors & Metrics
Metrics Used: BLEU-4, ROUGE-1, ROUGE-2
Results:
BLEU-4: 100%
ROUGE-1: 100%
ROUGE-2: 100%
These scores reflect perfect overlap with reference answers within the scope of the evaluation dataset.
Technical Specifications
Model Architecture and Objective
Base: Transformer (Mistral-7B architecture)
Fine-tuning method: LoRA
Objective: Instruction-following with forensic legal knowledge adaptation
Compute Infrastructure
Hardware: 2xL40s
Software: LLaMA-Factory, PyTorch, Transformers, bitsandbytes
- Downloads last month
- 35