Update README.md

README.md

@@ -1,3 +1,135 @@
----
-license: cc-by-nc-4.0
----
+---
+license: cc-by-nc-4.0
+language:
+- ru
+- en
+pipeline_tag: document-question-answering
+tags:
+- DocumentQA
+- QuestionAnswering
+- NLP
+- DeepLearning
+- Transformers
+- Multimodal
+- HuggingFace
+- ruBert
+- MachineLearning
+- DeepQA
+- AIForDocs
+- Docs
+- NeuralNetworks
+- torch
+- pytorch
+- large
+- text-generation-inference
+library_name: transformers
+metrics:
+- accuracy
+- f1
+- recall
+- exact_match
+- precision
+base_model:
+- ai-forever/ruBert-large
+---
+
+![Official Kaleidoscope Logo](https://huggingface.co/LaciaStudio/Kaleidoscope_large_v1/resolve/main/Kaleidoscope.png)
+
+# Document Question Answering Model - Kaleidoscope_large_v1
+This model is a fine-tuned version of sberbank-ai/ruBert-large designed for the task of document question answering. It has been adapted specifically for extracting answers from a provided document context and fine-tuned on a custom JSON dataset containing context, question, and answer triples.
+
+# Key Features
+* Objective: Extract answers from documents based on user questions.
+* Base Model: sberbank-ai/ruBert-large.
+* Dataset: A custom JSON file with fields: context, question, and answer.
+* Preprocessing: The input is formed by concatenating the question and the document context, guiding the model to focus on the relevant segments.
+# Training Settings:
+* Number of epochs: 20.
+* Batch size: 4 per device.
+* Warmup steps: 0.1 of total steps.
+* FP16 training enabled (if CUDA is available).
+* Hardware: Training was performed on an 1xRTX 3070.
+
+# Description
+The model was fine-tuned using the Transformers library with a custom training pipeline. Key aspects of the training process include:
+
+Custom Dataset: A loader reads a JSON file containing context, question, and answer triples.
+
+* *Feature Preparation: The script tokenizes the document and question with a sliding window approach to handle long texts.*
+* *Training Process: Leveraging mixed precision training and the AdamW optimizer to improve optimization.*
+* *Evaluation and Checkpointing: The training script evaluates model performance on a validation set, saves checkpoints, and employs early stopping based on validation loss.*
+* *This model is ideal for interactive document question answering tasks, making it a powerful tool for applications such as customer support, document search, and automated Q&A systems.*
+
+While primarily focused on Russian texts, the model also supports English language inputs.
+**The model also supports English language, but its support was not tested**
+
+# Example Usage
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForQuestionAnswering
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+tokenizer = AutoTokenizer.from_pretrained("LaciaStudio/Kaleidoscope_large_v1")
+model = AutoModelForQuestionAnswering.from_pretrained("LaciaStudio/Kaleidoscope_large_v1")
+model.to(device)
+
+file_path = input("Enter document path: ")
+with open(file_path, "r", encoding="utf-8") as f:
+    context = f.read()
+
+while True:
+    question = input("Enter question (or 'exit' to quit): ")
+    if question.lower() == "exit":
+        break
+    inputs = tokenizer(question, context, return_tensors="pt", truncation=True, max_length=384)
+    inputs = {k: v.to(device) for k, v in inputs.items()}
+    outputs = model(**inputs)
+    start_logits = outputs.start_logits
+    end_logits = outputs.end_logits
+    start_index = torch.argmax(start_logits)
+    end_index = torch.argmax(end_logits)
+    answer_tokens = inputs["input_ids"][0][start_index:end_index + 1]
+    answer = tokenizer.decode(answer_tokens, skip_special_tokens=True)
+    print("Answer:", answer)
+```
+
+# Example of answering
+**RU**
+*Context:*
+
+```
+Альберт Эйнштейн разработал теорию относительности.
+```
+
+*Question:*
+
+```
+Кто разработал теорию относительности?
+```
+
+*Answer:*
+
+```
+альберт эинштеин
+```
+**EN**
+*Context:*
+
+```
+I had a red car.
+```
+
+*Question:*
+
+```
+What kind of car did I have?
+```
+
+*Answer:*
+
+```
+a red car
+```
+
+**Finetuned by LaciaStudio | LaciaAI**