Amarthya7 commited on
Commit
5e763a6
·
verified ·
1 Parent(s): b34fbc9

Upload 5 files

Browse files
Files changed (5) hide show
  1. DOCUMENTATION.md +335 -0
  2. README.md +64 -0
  3. mediSync.log +80 -0
  4. requirements.txt +11 -0
  5. run.py +71 -0
DOCUMENTATION.md ADDED
@@ -0,0 +1,335 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # MediSync: Multi-Modal Medical Analysis System
2
+
3
+ ## Comprehensive Technical Documentation
4
+
5
+ ### Table of Contents
6
+ 1. [Introduction](#introduction)
7
+ 2. [System Architecture](#system-architecture)
8
+ 3. [Installation](#installation)
9
+ 4. [Usage](#usage)
10
+ 5. [Core Components](#core-components)
11
+ 6. [Model Details](#model-details)
12
+ 7. [API Reference](#api-reference)
13
+ 8. [Extending the System](#extending-the-system)
14
+ 9. [Troubleshooting](#troubleshooting)
15
+ 10. [References](#references)
16
+
17
+ ---
18
+
19
+ ## Introduction
20
+
21
+ MediSync is a multi-modal AI system that combines X-ray image analysis with medical report text processing to provide comprehensive medical insights. By leveraging state-of-the-art deep learning models for both vision and language understanding, MediSync can:
22
+
23
+ - Analyze chest X-ray images to detect abnormalities
24
+ - Extract key clinical information from medical reports
25
+ - Fuse insights from both modalities for enhanced diagnosis support
26
+ - Provide comprehensive visualization of analysis results
27
+
28
+ This AI system demonstrates the power of multi-modal fusion in the healthcare domain, where integrating information from multiple sources can lead to more robust and accurate analyses.
29
+
30
+ ## System Architecture
31
+
32
+ MediSync follows a modular architecture with three main components:
33
+
34
+ 1. **Image Analysis Module**: Processes X-ray images using pre-trained vision models
35
+ 2. **Text Analysis Module**: Analyzes medical reports using NLP models
36
+ 3. **Multimodal Fusion Module**: Combines insights from both modalities
37
+
38
+ The system uses the following high-level workflow:
39
+
40
+ ```
41
+ ┌─────────────────┐
42
+ │ X-ray Image │
43
+ └────────┬────────┘
44
+
45
+
46
+ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
47
+ │ Preprocessing │───▶│ Image Analysis │───▶│ │
48
+ └─────────────────┘ └─────────────────┘ │ │
49
+ │ Multimodal │
50
+ ┌─────────────────┐ ┌─────────────────┐ │ Fusion │───▶ Results
51
+ │ Medical Report │───▶│ Text Analysis │───▶│ │
52
+ └─────────────────┘ └─────────────────┘ │ │
53
+ └─────────────────┘
54
+ ```
55
+
56
+ ## Installation
57
+
58
+ ### Prerequisites
59
+ - Python 3.8 or higher
60
+ - Pip package manager
61
+
62
+ ### Setup Instructions
63
+
64
+ 1. Clone the repository:
65
+ ```bash
66
+ git clone [repository-url]
67
+ cd mediSync
68
+ ```
69
+
70
+ 2. Install dependencies:
71
+ ```bash
72
+ pip install -r requirements.txt
73
+ ```
74
+
75
+ 3. Download sample data:
76
+ ```bash
77
+ python -m mediSync.utils.download_samples
78
+ ```
79
+
80
+ ## Usage
81
+
82
+ ### Running the Application
83
+
84
+ To launch the MediSync application with the Gradio interface:
85
+
86
+ ```bash
87
+ python run.py
88
+ ```
89
+
90
+ This will:
91
+ 1. Download sample data if not already present
92
+ 2. Initialize the application
93
+ 3. Launch the Gradio web interface
94
+
95
+ ### Web Interface
96
+
97
+ MediSync provides a user-friendly web interface with three main tabs:
98
+
99
+ 1. **Multimodal Analysis**: Upload an X-ray image and enter a medical report for combined analysis
100
+ 2. **Image Analysis**: Upload an X-ray image for image-only analysis
101
+ 3. **Text Analysis**: Enter a medical report for text-only analysis
102
+
103
+ ### Command Line Usage
104
+
105
+ You can also use the core components directly from Python:
106
+
107
+ ```python
108
+ from mediSync.models import XRayImageAnalyzer, MedicalReportAnalyzer, MultimodalFusion
109
+
110
+ # Initialize models
111
+ fusion_model = MultimodalFusion()
112
+
113
+ # Analyze image and text
114
+ results = fusion_model.analyze("path/to/image.jpg", "Medical report text...")
115
+
116
+ # Get explanation
117
+ explanation = fusion_model.get_explanation(results)
118
+ print(explanation)
119
+ ```
120
+
121
+ ## Core Components
122
+
123
+ ### Image Analysis Module
124
+
125
+ The `XRayImageAnalyzer` class is responsible for analyzing X-ray images:
126
+
127
+ - Uses the DeiT (Data-efficient image Transformers) model fine-tuned on chest X-rays
128
+ - Detects abnormalities and classifies findings
129
+ - Provides confidence scores and primary findings
130
+
131
+ Key methods:
132
+ - `analyze(image_path)`: Analyzes an X-ray image
133
+ - `get_explanation(results)`: Generates a human-readable explanation
134
+
135
+ ### Text Analysis Module
136
+
137
+ The `MedicalReportAnalyzer` class processes medical report text:
138
+
139
+ - Extracts medical entities (conditions, treatments, tests)
140
+ - Assesses severity level
141
+ - Extracts key findings
142
+ - Suggests follow-up actions
143
+
144
+ Key methods:
145
+ - `extract_entities(text)`: Extracts medical entities
146
+ - `assess_severity(text)`: Determines severity level
147
+ - `extract_findings(text)`: Extracts key clinical findings
148
+ - `suggest_followup(text, entities, severity)`: Suggests follow-up actions
149
+ - `analyze(text)`: Performs comprehensive analysis
150
+
151
+ ### Multimodal Fusion Module
152
+
153
+ The `MultimodalFusion` class combines insights from both modalities:
154
+
155
+ - Calculates agreement between image and text analyses
156
+ - Determines confidence-weighted findings
157
+ - Provides comprehensive severity assessment
158
+ - Merges follow-up recommendations
159
+
160
+ Key methods:
161
+ - `analyze_image(image_path)`: Analyzes image only
162
+ - `analyze_text(text)`: Analyzes text only
163
+ - `analyze(image_path, report_text)`: Performs multimodal analysis
164
+ - `get_explanation(fused_results)`: Generates comprehensive explanation
165
+
166
+ ## Model Details
167
+
168
+ ### X-ray Analysis Model
169
+
170
+ - **Model**: facebook/deit-base-patch16-224-medical-cxr
171
+ - **Architecture**: Data-efficient image Transformer (DeiT)
172
+ - **Training Data**: Chest X-ray datasets
173
+ - **Input Size**: 224x224 pixels
174
+ - **Output**: Classification probabilities for various conditions
175
+
176
+ ### Medical Text Analysis Models
177
+
178
+ - **Entity Recognition Model**: samrawal/bert-base-uncased_medical-ner
179
+ - **Classification Model**: medicalai/ClinicalBERT
180
+ - **Architecture**: BERT-based transformer models
181
+ - **Training Data**: Medical text and reports
182
+
183
+ ## API Reference
184
+
185
+ ### XRayImageAnalyzer
186
+
187
+ ```python
188
+ from mediSync.models import XRayImageAnalyzer
189
+
190
+ # Initialize
191
+ analyzer = XRayImageAnalyzer(model_name="facebook/deit-base-patch16-224-medical-cxr")
192
+
193
+ # Analyze image
194
+ results = analyzer.analyze("path/to/image.jpg")
195
+
196
+ # Get explanation
197
+ explanation = analyzer.get_explanation(results)
198
+ ```
199
+
200
+ ### MedicalReportAnalyzer
201
+
202
+ ```python
203
+ from mediSync.models import MedicalReportAnalyzer
204
+
205
+ # Initialize
206
+ analyzer = MedicalReportAnalyzer()
207
+
208
+ # Analyze report
209
+ results = analyzer.analyze("Medical report text...")
210
+
211
+ # Access specific components
212
+ entities = results["entities"]
213
+ severity = results["severity"]
214
+ findings = results["findings"]
215
+ recommendations = results["followup_recommendations"]
216
+ ```
217
+
218
+ ### MultimodalFusion
219
+
220
+ ```python
221
+ from mediSync.models import MultimodalFusion
222
+
223
+ # Initialize
224
+ fusion = MultimodalFusion()
225
+
226
+ # Multimodal analysis
227
+ results = fusion.analyze("path/to/image.jpg", "Medical report text...")
228
+
229
+ # Get explanation
230
+ explanation = fusion.get_explanation(results)
231
+ ```
232
+
233
+ ## Extending the System
234
+
235
+ ### Adding New Models
236
+
237
+ To add a new image analysis model:
238
+
239
+ 1. Create a new class that follows the same interface as `XRayImageAnalyzer`
240
+ 2. Update the `MultimodalFusion` class to use your new model
241
+
242
+ ```python
243
+ class NewXRayModel:
244
+ def __init__(self, model_name, device=None):
245
+ # Initialize your model
246
+ pass
247
+
248
+ def analyze(self, image_path):
249
+ # Implement analysis logic
250
+ return results
251
+
252
+ def get_explanation(self, results):
253
+ # Generate explanation
254
+ return explanation
255
+ ```
256
+
257
+ ### Custom Preprocessing
258
+
259
+ You can extend the preprocessing utilities in `utils/preprocessing.py` for custom data preparation:
260
+
261
+ ```python
262
+ def my_custom_preprocessor(image_path, **kwargs):
263
+ # Implement custom preprocessing
264
+ return processed_image
265
+ ```
266
+
267
+ ### Visualization Extensions
268
+
269
+ To add new visualization options, extend the utilities in `utils/visualization.py`:
270
+
271
+ ```python
272
+ def my_custom_visualization(results, **kwargs):
273
+ # Create custom visualization
274
+ return figure
275
+ ```
276
+
277
+ ## Troubleshooting
278
+
279
+ ### Common Issues
280
+
281
+ 1. **Model Loading Errors**
282
+ - Ensure you have a stable internet connection for downloading models
283
+ - Check that you have sufficient disk space
284
+ - Try specifying a different model checkpoint
285
+
286
+ 2. **Image Processing Errors**
287
+ - Ensure images are in a supported format (JPEG, PNG)
288
+ - Check that the image is a valid X-ray image
289
+ - Try preprocessing the image manually using the utility functions
290
+
291
+ 3. **Performance Issues**
292
+ - For faster inference, use a GPU if available
293
+ - Reduce image resolution if processing is too slow
294
+ - Use the text-only analysis for quicker results
295
+
296
+ ### Logging
297
+
298
+ MediSync uses Python's logging module for debug information:
299
+
300
+ ```python
301
+ import logging
302
+ logging.basicConfig(level=logging.DEBUG)
303
+ ```
304
+
305
+ Log files are saved to `mediSync.log` in the application directory.
306
+
307
+ ## References
308
+
309
+ ### Datasets
310
+
311
+ - [MIMIC-CXR](https://physionet.org/content/mimic-cxr/2.0.0/): Large dataset of chest radiographs with reports
312
+ - [ChestX-ray14](https://www.nih.gov/news-events/news-releases/nih-clinical-center-provides-one-largest-publicly-available-chest-x-ray-datasets-scientific-community): NIH dataset of chest X-rays
313
+
314
+ ### Papers
315
+
316
+ - He, K., et al. (2020). "Vision Transformers for Medical Image Analysis"
317
+ - Irvin, J., et al. (2019). "CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison"
318
+ - Johnson, A.E.W., et al. (2019). "MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs"
319
+
320
+ ### Tools and Libraries
321
+
322
+ - [Hugging Face Transformers](https://huggingface.co/docs/transformers/index)
323
+ - [PyTorch](https://pytorch.org/)
324
+ - [Gradio](https://gradio.app/)
325
+
326
+ ---
327
+
328
+ ## License
329
+
330
+ This project is licensed under the MIT License - see the LICENSE file for details.
331
+
332
+ ## Acknowledgments
333
+
334
+ - The development of MediSync was inspired by recent advances in multi-modal learning in healthcare.
335
+ - Special thanks to the open-source community for providing pre-trained models and tools.
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # MediSync: Multi-Modal Medical Analysis System
2
+
3
+ MediSync is an AI-powered healthcare solution that combines X-ray image analysis with patient report text processing to provide comprehensive medical insights.
4
+
5
+ ## Features
6
+
7
+ - **X-ray Image Analysis**: Detects abnormalities in chest X-rays using pre-trained vision models from Hugging Face.
8
+ - **Medical Report Processing**: Extracts key information from patient reports using NLP models.
9
+ - **Multi-modal Integration**: Combines insights from both image and text data for more accurate diagnosis suggestions.
10
+ - **User-friendly Interface**: Simple web interface for uploading images and reports.
11
+
12
+ ## Project Structure
13
+
14
+ ```
15
+ mediSync/
16
+ ├── app.py # Main application with Gradio interface
17
+ ├── models/
18
+ │ ├── image_analyzer.py # X-ray image analysis module
19
+ │ ├── text_analyzer.py # Medical report text analysis module
20
+ │ └── multimodal_fusion.py # Fusion of image and text insights
21
+ ├── utils/
22
+ │ ├── preprocessing.py # Data preprocessing utilities
23
+ │ └── visualization.py # Result visualization utilities
24
+ ├── data/
25
+ │ └── sample/ # Sample data for testing
26
+ └── tests/ # Unit tests
27
+ ```
28
+
29
+ ## Setup Instructions
30
+
31
+ 1. Clone this repository:
32
+ ```bash
33
+ git clone [repository-url]
34
+ cd MediSync
35
+ ```
36
+
37
+ 2. Install dependencies:
38
+ ```bash
39
+ pip install -r requirements.txt
40
+ ```
41
+
42
+ 3. Run the application:
43
+ ```bash
44
+ python app.py
45
+ ```
46
+
47
+ 4. Access the web interface at `http://localhost:7860`
48
+
49
+ ## Models Used
50
+
51
+ - **X-ray Analysis**: facebook/deit-base-patch16-224-medical-cxr
52
+ - **Medical Text Analysis**: medicalai/ClinicalBERT
53
+ - **Additional Support Models**: Medical question answering and entity recognition models
54
+
55
+ ## Use Cases
56
+
57
+ - Preliminary screening of chest X-rays
58
+ - Cross-validation of radiologist reports
59
+ - Educational tool for medical students
60
+ - Research tool for studying correlation between visual findings and written reports
61
+
62
+ ## Note
63
+
64
+ This system is designed as a support tool and should not replace professional medical diagnosis. Always consult with healthcare professionals for medical decisions.
mediSync.log ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2025-03-10 22:44:41,854 - __main__ - INFO - Starting MediSync setup
2
+ 2025-03-10 22:44:41,854 - __main__ - INFO - Checking for sample data
3
+ 2025-03-10 22:44:42,626 - __main__ - INFO - No sample images found. Downloading...
4
+ 2025-03-10 22:44:42,626 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/53448173/322830a37f0fa0852773ca2db3e8d8_big_gallery.jpeg to D:\Multi Model AI Problem Solving\mediSync\data\sample\normal_chest_xray.jpg
5
+ 2025-03-10 22:44:43,532 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/53448173/322830a37f0fa0852773ca2db3e8d8_big_gallery.jpeg: HTTP Error 404: Not Found
6
+ 2025-03-10 22:44:43,532 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/52465460/e4d8791bd7502ab72af8d9e5c322db_big_gallery.jpg to D:\Multi Model AI Problem Solving\mediSync\data\sample\pneumonia_xray.jpg
7
+ 2025-03-10 22:44:43,988 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/52465460/e4d8791bd7502ab72af8d9e5c322db_big_gallery.jpg: HTTP Error 404: Not Found
8
+ 2025-03-10 22:44:43,988 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/556520/cf17c05750adb04b2a6e23afb47c7d_big_gallery.jpg to D:\Multi Model AI Problem Solving\mediSync\data\sample\cardiomegaly_xray.jpg
9
+ 2025-03-10 22:44:44,605 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/556520/cf17c05750adb04b2a6e23afb47c7d_big_gallery.jpg: HTTP Error 404: Not Found
10
+ 2025-03-10 22:44:44,605 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/19972291/41eed1a2cdad06d26c3f415a6ed65a_big_gallery.jpeg to D:\Multi Model AI Problem Solving\mediSync\data\sample\nodule_xray.jpg
11
+ 2025-03-10 22:44:45,107 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/19972291/41eed1a2cdad06d26c3f415a6ed65a_big_gallery.jpeg: HTTP Error 404: Not Found
12
+ 2025-03-10 22:44:45,107 - mediSync.utils.download_samples - INFO - Downloaded 0 out of 4 images
13
+ 2025-03-10 22:44:45,107 - mediSync.utils.download_samples - INFO - Created sample info file: D:\Multi Model AI Problem Solving\mediSync\data\sample\sample_info.txt
14
+ 2025-03-10 22:44:45,107 - __main__ - INFO - Launching MediSync application
15
+ 2025-03-10 22:44:48,918 - httpx - INFO - HTTP Request: GET https://api.gradio.app/gradio-messaging/en "HTTP/1.1 200 OK"
16
+ 2025-03-10 22:45:02,402 - mediSync.app - INFO - Initializing MediSync application
17
+ 2025-03-10 22:45:03,005 - httpx - INFO - HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
18
+ 2025-03-10 22:45:03,294 - httpx - INFO - HTTP Request: GET http://127.0.0.1:7860/gradio_api/startup-events "HTTP/1.1 200 OK"
19
+ 2025-03-10 22:45:03,722 - httpx - INFO - HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
20
+ 2025-03-10 22:46:47,162 - mediSync.app - INFO - Loading models...
21
+ 2025-03-10 22:46:47,162 - models.multimodal_fusion - INFO - Using device: cpu
22
+ 2025-03-10 22:46:47,178 - models.image_analyzer - INFO - Using device: cpu
23
+ 2025-03-10 22:46:47,391 - models.image_analyzer - ERROR - Failed to load model: facebook/deit-base-patch16-224-medical-cxr is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
24
+ If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
25
+ 2025-03-10 22:46:47,391 - models.multimodal_fusion - ERROR - Failed to initialize image analyzer: facebook/deit-base-patch16-224-medical-cxr is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
26
+ If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
27
+ 2025-03-10 22:46:47,391 - models.text_analyzer - INFO - Using device: cpu
28
+ 2025-03-10 22:46:47,490 - models.text_analyzer - ERROR - Failed to load NER model: samrawal/bert-base-uncased_medical-ner is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
29
+ If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
30
+ 2025-03-10 22:46:58,283 - models.text_analyzer - INFO - Successfully loaded classifier model: medicalai/ClinicalBERT
31
+ 2025-03-10 22:46:58,283 - models.multimodal_fusion - INFO - Successfully initialized text analyzer
32
+ 2025-03-10 22:46:58,283 - mediSync.app - INFO - Models loaded successfully
33
+ 2025-03-10 22:46:58,379 - mediSync.app - INFO - Performing multimodal analysis
34
+ 2025-03-10 22:46:58,379 - models.multimodal_fusion - WARNING - Image analyzer not available
35
+ 2025-03-10 22:46:58,379 - models.text_analyzer - WARNING - NER model not available
36
+ 2025-03-10 22:46:58,379 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
37
+ 2025-03-10 22:47:44,547 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp19zqepvi\upload.png
38
+ 2025-03-10 22:47:55,128 - mediSync.app - INFO - Analyzing medical report text
39
+ 2025-03-10 22:47:55,128 - models.text_analyzer - WARNING - NER model not available
40
+ 2025-03-10 22:48:20,220 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpowd6qwty\upload.png
41
+ 2025-03-10 22:55:22,019 - mediSync.app - INFO - Performing multimodal analysis
42
+ 2025-03-10 22:55:22,019 - models.multimodal_fusion - WARNING - Image analyzer not available
43
+ 2025-03-10 22:55:22,025 - models.text_analyzer - WARNING - NER model not available
44
+ 2025-03-10 22:55:22,025 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
45
+ 2025-03-10 22:55:35,606 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpipzmsxch\upload.png
46
+ 2025-03-10 22:55:45,116 - mediSync.app - INFO - Analyzing medical report text
47
+ 2025-03-10 22:55:45,116 - models.text_analyzer - WARNING - NER model not available
48
+ 2025-03-10 23:01:32,009 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpo88f1uaw\upload.png
49
+ 2025-03-10 23:05:24,892 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpnckk4w27\upload.png
50
+ 2025-03-10 23:05:45,332 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpxab_m30m\upload.png
51
+ 2025-03-10 23:06:01,387 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpcit78i2e\upload.png
52
+ 2025-03-10 23:06:04,157 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpivli_abx\upload.png
53
+ 2025-03-10 23:11:47,429 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp4cpwvo5h\upload.png
54
+ 2025-03-10 23:22:27,118 - mediSync.app - INFO - Performing multimodal analysis
55
+ 2025-03-10 23:22:27,118 - models.multimodal_fusion - WARNING - Image analyzer not available
56
+ 2025-03-10 23:22:27,118 - models.text_analyzer - WARNING - NER model not available
57
+ 2025-03-10 23:22:27,118 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
58
+ 2025-03-10 23:24:22,378 - mediSync.app - INFO - Performing multimodal analysis
59
+ 2025-03-10 23:24:22,378 - models.multimodal_fusion - WARNING - Image analyzer not available
60
+ 2025-03-10 23:24:22,378 - models.text_analyzer - WARNING - NER model not available
61
+ 2025-03-10 23:24:22,378 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
62
+ 2025-03-10 23:24:35,252 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp947ie5e1\upload.png
63
+ 2025-03-10 23:24:37,554 - mediSync.app - INFO - Performing multimodal analysis
64
+ 2025-03-10 23:24:37,554 - models.multimodal_fusion - WARNING - Image analyzer not available
65
+ 2025-03-10 23:24:37,554 - models.text_analyzer - WARNING - NER model not available
66
+ 2025-03-10 23:24:37,554 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
67
+ 2025-03-10 23:24:58,770 - mediSync.app - INFO - Analyzing medical report text
68
+ 2025-03-10 23:24:58,770 - models.text_analyzer - WARNING - NER model not available
69
+ 2025-03-10 23:25:25,652 - mediSync.app - INFO - Analyzing medical report text
70
+ 2025-03-10 23:25:25,652 - models.text_analyzer - WARNING - NER model not available
71
+ 2025-03-10 23:27:06,100 - mediSync.app - INFO - Performing multimodal analysis
72
+ 2025-03-10 23:27:06,100 - models.multimodal_fusion - WARNING - Image analyzer not available
73
+ 2025-03-10 23:27:06,100 - models.text_analyzer - WARNING - NER model not available
74
+ 2025-03-10 23:27:06,100 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
75
+ 2025-03-10 23:27:46,183 - mediSync.app - INFO - Performing multimodal analysis
76
+ 2025-03-10 23:27:46,183 - models.multimodal_fusion - WARNING - Image analyzer not available
77
+ 2025-03-10 23:27:46,183 - models.text_analyzer - WARNING - NER model not available
78
+ 2025-03-10 23:27:46,183 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
79
+ 2025-03-10 23:28:15,394 - mediSync.app - INFO - Analyzing medical report text
80
+ 2025-03-10 23:28:15,394 - models.text_analyzer - WARNING - NER model not available
requirements.txt ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ torch>=2.0.0
2
+ transformers>=4.30.0
3
+ datasets>=2.12.0
4
+ pillow>=9.0.0
5
+ matplotlib>=3.7.0
6
+ pandas>=2.0.0
7
+ numpy>=1.24.0
8
+ scikit-learn>=1.2.0
9
+ gradio>=4.0.0
10
+ huggingface_hub>=0.16.0
11
+ opencv-python>=4.7.0
run.py ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ MediSync: Multi-Modal Medical Analysis System - Runner Script
3
+ ============================================================
4
+ This script downloads sample data and launches the MediSync application.
5
+ """
6
+
7
+ import logging
8
+ import sys
9
+ from pathlib import Path
10
+
11
+ # Set up logging
12
+ logging.basicConfig(
13
+ level=logging.INFO,
14
+ format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
15
+ handlers=[logging.StreamHandler(), logging.FileHandler("mediSync.log")],
16
+ )
17
+ logger = logging.getLogger(__name__)
18
+
19
+
20
+ def main():
21
+ """
22
+ Main function to set up and run the MediSync application.
23
+ """
24
+ logger.info("Starting MediSync setup")
25
+
26
+ # Add the current directory to Python path
27
+ current_dir = Path(__file__).resolve().parent
28
+ sys.path.append(str(current_dir))
29
+
30
+ # Download sample data if needed
31
+ try:
32
+ logger.info("Checking for sample data")
33
+ from mediSync.utils.download_samples import (
34
+ create_sample_info_file,
35
+ download_sample_images,
36
+ )
37
+
38
+ # Check if sample directory exists and contains images
39
+ sample_dir = current_dir / "mediSync" / "data" / "sample"
40
+ sample_dir.mkdir(parents=True, exist_ok=True)
41
+
42
+ image_files = list(sample_dir.glob("*.jpg")) + list(sample_dir.glob("*.png"))
43
+
44
+ if not image_files:
45
+ logger.info("No sample images found. Downloading...")
46
+ download_sample_images()
47
+ create_sample_info_file()
48
+ else:
49
+ logger.info(f"Found {len(image_files)} existing sample images")
50
+
51
+ except Exception as e:
52
+ logger.error(f"Error setting up sample data: {e}")
53
+ print(f"Warning: Could not set up sample data: {e}")
54
+
55
+ # Launch the application
56
+ try:
57
+ logger.info("Launching MediSync application")
58
+ from mediSync.app import create_interface
59
+
60
+ create_interface()
61
+
62
+ except Exception as e:
63
+ logger.error(f"Error launching application: {e}")
64
+ print(f"Error: Failed to launch MediSync application: {e}")
65
+ return 1
66
+
67
+ return 0
68
+
69
+
70
+ if __name__ == "__main__":
71
+ sys.exit(main())