Initial commit

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,158 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+# *.log  (Covered by generic *.log below)
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# PEP 582; used by PDM, Flit and potentially other packaging tools.
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# Hugging Face cache (often large)
+.cache/huggingface/
+
+# Gradio
+.gradio/
+
+# General cache directory (covers some cases like pytest)
+.cache/
+
+# IDE / Editor specific files
+.vscode/
+.idea/
+*.swp
+*~
+*.sublime-project
+*.sublime-workspace
+
+# OS generated files
+.DS_Store
+Thumbs.db
+
+# Project-specific
+temp_downloads/
+test.wav
+*.log # General log files
+
+# Large media files potentially downloaded by yt-dlp (uncomment if needed)
+*.mp4
+*.mkv
+*.webm
+*.mp3
+*.m4a
+*.flv
+*.aac
+*.ogg
+*.opus
+*.wav # Ignoring all .wav files, including test.wav

README.md

@@ -0,0 +1,113 @@
+---
+title: TranscriptTool - Gradio MCP Server for Transcription
+emoji: 💬
+colorFrom: green
+colorTo: green
+sdk: gradio
+sdk_version: 5.29.0
+app_file: app.py
+pinned: false
+license: apache-2.0
+short_description: Gradio-based MCP server to transcribe audio & video from URLs
+---
+
+# TranscriptTool: A Gradio MCP Server for Audio/Video Transcription from URLs
+
+## Overview
+
+`TranscriptTool` is a Gradio application configured to function as an MCP (Model Control Protocol) server. It is designed to transcribe audio and video from URLs into text. Implementing OpenAI's Whisper and `ffmpeg` (via `yt-dlp`), this server enables MCP clients (like Cline) to process multimedia inputs efficiently by downloading and converting content from a given URL. It supports robust handling, including format conversion to WAV and dynamic device selection (CPU or GPU).
+
+The repository contains the following main components:
+- **`app.py`**: The main Gradio application file that runs the MCP server.
+- **`transcription_tool.py`**: The core logic for handling file conversion and calling the transcription function.
+- **`transcription.py`**: Contains the implementation for Whisper transcription using the `transformers` library.
+- **`tool_config.json`**: Configuration details for the `TranscriptTool`.
+- **`requirements.txt`**: Lists the necessary Python dependencies.
+- **`ffmpeg_setup.py`**: Script to ensure ffmpeg is available.
+- **`logging_config.py`**: Configuration for logging.
+
+---
+
+## Installation
+
+1. Clone this repository:
+   ```bash
+   git clone https://huggingface.co/spaces/bismay/TranscriptTool
+   cd TranscriptTool
+   ```
+2. Install dependencies:
+   ```bash
+   pip install -r requirements.txt
+   ```
+   This will install the necessary libraries, including `gradio[mcp]`, `yt-dlp`, `transformers`, and `torch`.
+
+---
+## Usage
+
+### Running the Gradio App / MCP Server
+
+To run the Gradio application which also starts the MCP server, execute:
+   ```bash
+   python app.py
+   ```
+
+This will launch a local Gradio web interface and start the MCP server. The server will expose the `transcribe_url` function as an MCP tool.
+
+### Using as an MCP Server
+
+When you run `python app.py`, the application starts an MCP server accessible to MCP clients.
+
+**Exposed Tool:**
+
+The server exposes one tool: `transcribe_url`.
+
+*   **Description:** Transcribes audio or video from a given URL. Downloads the media from the URL, converts it to WAV format, and then uses the TranscriptTool to perform the transcription in English.
+*   **Input:**
+    *   `url` (string): The URL of the audio or video file.
+*   **Output:** (string): The transcription of the audio/video in English, or an error message if download or transcription fails.
+
+**Connecting an MCP Client:**
+
+The MCP server will typically be accessible at `http://127.0.0.1:7860/gradio_api/mcp/sse` when run locally. You can find the exact URL printed in your console when the Gradio app launches.
+
+To connect an MCP client (like Cline) to this server, you need to add a configuration entry in your client's settings. The exact format depends on your client, but it generally involves specifying a name for the server and its URL.
+
+Example configuration for a client (like Cline) that supports SSE:
+
+```json
+{
+  "mcpServers": {
+    "localTranscript": {
+      "url": "http://127.0.0.1:7860/gradio_api/mcp/sse"
+    }
+  }
+}
+```
+
+*Note: If your MCP client does not directly support SSE-based servers (like some versions of Claude Desktop), you may need to use a tool like `mcp-remote` as an intermediary. Refer to your client's documentation for details.*
+
+### Connecting to the Hosted Server on Hugging Face Spaces
+
+This application is also hosted on Hugging Face Spaces, providing a publicly accessible MCP server. You can connect to this hosted server using the following URL:
+
+`https://bismay-transcripttool.hf.space/gradio_api/mcp/sse`
+
+To connect your MCP client (like Cline) to this hosted server, add a configuration entry similar to this:
+
+```json
+{
+  "mcpServers": {
+    "remoteTranscript": {
+      "url": "https://maguid28-transcripttool.hf.space/gradio_api/mcp/sse"
+    }
+  }
+}
+```
+
+---
+## License
+This project is licensed under the Apache-2.0 License. See the LICENSE file for more details.
+
+---
+## Contributing
+Contributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.

app.py

@@ -0,0 +1,103 @@
+import gradio as gr
+import subprocess
+import tempfile
+import os
+import shutil
+from logging_config import logger
+from transcription_tool import TranscriptTool  # Assuming TranscriptionTool is in `transcription_tool.py`
+
+# smolagent transcription tool
+transcript_tool = TranscriptTool()
+
+
+def transcribe_url(url):
+    """
+    Transcribes audio or video from a given URL.
+
+    Downloads the media from the URL, converts it to WAV format,
+    and then uses the TranscriptTool to perform the transcription in English
+
+    Args:
+        url (str): The URL of the audio or video file.
+
+    Returns:
+        str: The transcription of the audio/video in english, or an error message if
+             download or transcription fails.
+    """
+    local_file_path = None
+
+    try:
+        if not url:
+            return "Error: Please provide a URL." # Removed the second empty string as the function only returns one value
+
+        logger.info(f"Attempting to download audio from URL: {url}")
+        temp_download_dir = "./temp_downloads"
+        os.makedirs(temp_download_dir, exist_ok=True)
+
+        # Use yt-dlp to download the best audio format and convert to wav
+        # -f bestaudio: selects the best audio format
+        # -x: extracts audio
+        # --audio-format wav: converts to wav format
+        # -o: specifies output template
+        output_template = os.path.join(temp_download_dir, "downloaded_audio.%(ext)s")
+
+        # --- Start Added Code ---
+        # Ensure any previous download is removed before starting a new one
+        logger.info(f"Checking for existing files matching 'downloaded_audio.*' in {temp_download_dir}")
+        for filename in os.listdir(temp_download_dir):
+            if filename.startswith("downloaded_audio."):
+                file_to_delete = os.path.join(temp_download_dir, filename)
+                try:
+                    os.remove(file_to_delete)
+                    logger.info(f"Removed existing file: {file_to_delete}")
+                except OSError as e:
+                    logger.error(f"Error removing file {file_to_delete}: {e.strerror}")
+        # --- End Added Code ---
+
+        cmd = ["yt-dlp", "-f", "bestaudio", "-x", "--audio-format", "wav", "-o", output_template, url]
+        process = subprocess.run(cmd, check=True, capture_output=True, text=True)
+        logger.info("yt-dlp stdout:\n" + process.stdout)
+        logger.info("yt-dlp stderr:\n" + process.stderr)
+
+        # Find the downloaded file (yt-dlp might add .wav extension)
+        downloaded_files = [f for f in os.listdir(temp_download_dir) if f.startswith("downloaded_audio.")]
+        if not downloaded_files:
+            raise RuntimeError("yt-dlp failed to download or convert audio.")
+        local_file_path = os.path.join(temp_download_dir, downloaded_files[0])
+        logger.info(f"Downloaded audio to temporary file: {local_file_path}")
+        print(f"Downloaded audio to temporary file: {local_file_path}")
+
+        # Perform transcription
+        transcription_result = transcript_tool.forward(local_file_path)
+
+        return transcription_result
+
+    except subprocess.CalledProcessError as e:
+        error_message = f"yt-dlp error: {e.stderr}"
+        logger.error(error_message)
+        return f"An error occurred during download: {error_message}"
+    except Exception as e:
+        error_message = f"An unexpected error occurred: {str(e)}"
+        logger.error(error_message)
+        return error_message
+
+
+with gr.Blocks() as app:
+    gr.Markdown("# TranscriptTool: Transcribe Audio/Video")
+    gr.Markdown("TranscriptTool is a smolagent tool used to transcribe audio and video files into text. This tool allows agents to process multimedia inputs efficiently. Can be used within a smolagent via the Hugging Face API.")
+
+    url_input = gr.Textbox(label="Enter Audio/Video URL", placeholder="e.g., https://www.youtube.com/watch?v=dQw4w9WgXcQ")
+    transcribe_button = gr.Button("Transcribe")
+
+    gr.Markdown("Provide a URL to transcribe audio or video.")
+
+    transcription_output = gr.Textbox(label="Transcription", lines=10)
+
+    transcribe_button.click(
+        fn=transcribe_url,
+        inputs=[url_input],
+        outputs=[transcription_output]
+    )
+
+if __name__ == "__main__":
+    app.launch(mcp_server=True)

ffmpeg_setup.py

@@ -0,0 +1,59 @@
+import os
+import stat
+import shutil
+import subprocess
+import imageio_ffmpeg
+from logging_config import logger
+
+
+def is_ffmpeg_in_path() -> bool:
+    try:
+        subprocess.run(
+            ["ffmpeg", "-version"],
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            check=True
+        )
+        return True
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return False
+
+
+def ensure_ffmpeg_in_path():
+
+    if is_ffmpeg_in_path():
+        logger.info("FFmpeg is already available in PATH.")
+        return
+
+    try:
+        ffmpeg_path_original = imageio_ffmpeg.get_ffmpeg_exe()
+        ffmpeg_dir = os.path.dirname(ffmpeg_path_original)
+        binary_name = os.path.basename(ffmpeg_path_original)
+
+        logger.info(f"imageio-ffmpeg reported path: {ffmpeg_path_original}")
+        logger.info(f"Directory contents: {os.listdir(ffmpeg_dir)}")
+        logger.info(f"Binary name: {binary_name}")
+
+        expected_binary_name = "ffmpeg"
+        copied_path = os.path.join(ffmpeg_dir, expected_binary_name)
+
+        if not os.path.exists(copied_path):
+            logger.info(f"Copying {binary_name} to {expected_binary_name} in {ffmpeg_dir}.")
+            shutil.copy2(ffmpeg_path_original, copied_path)
+            st = os.stat(copied_path)
+            os.chmod(copied_path, st.st_mode | stat.S_IEXEC)
+        else:
+            logger.info(f"'{copied_path}' already exists; skipping copy.")
+
+        # Add directory to PATH
+        os.environ["PATH"] = ffmpeg_dir + os.pathsep + os.environ["PATH"]
+        logger.info(f"PATH updated to include: {ffmpeg_dir}")
+
+        if is_ffmpeg_in_path():
+            logger.info("FFmpeg is now accessible in PATH.")
+        else:
+            logger.warning("FFmpeg is still not found in PATH after attempting to add it.")
+            raise RuntimeError("Failed to make FFmpeg accessible in PATH.")
+    except Exception as e:
+        logger.error(f"Failed to ensure FFmpeg is in PATH: {str(e)}")
+        raise RuntimeError("Failed to ensure FFmpeg is in PATH.") from e

logging_config.py

@@ -0,0 +1,17 @@
+import logging
+import sys # Import sys for console output
+
+logger = logging.getLogger("transcription_logger")
+logger.setLevel(logging.DEBUG)
+
+# Formatter for the logs
+formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
+
+# Log handler that writes to the console (stdout/stderr)
+console_handler = logging.StreamHandler(sys.stdout) # Use sys.stdout for console output
+console_handler.setLevel(logging.DEBUG)
+console_handler.setFormatter(formatter)
+
+logger.addHandler(console_handler)
+
+logger.propagate = False

requirements.txt

@@ -0,0 +1,11 @@
+huggingface_hub==0.30.1
+gradio==5.29.0
+gradio[mcp]
+pydantic==2.10.6
+youtube-transcript-api==0.6.3
+yt-dlp==2025.1.15
+transformers==4.48.1
+torch==2.2.2
+imageio-ffmpeg==0.6.0
+numpy==1.24.3
+smolagents==1.4.1

tool_config.json

@@ -0,0 +1,12 @@
+{
+  "description": "\n    A smolagent tool for transcribing audio and video from URLs into text. This tool utilizes Whisper for transcription \n    and ffmpeg (via yt-dlp) for media conversion, enabling agents to process multimedia inputs into text by downloading \n    and converting content from a given URL. It supports robust handling, including format conversion to WAV and dynamic \n    device selection for optimal performance. It empowers agents to integrate audio and video transcription capabilities \n    into workflows for enhanced data accessibility.",
+  "inputs": {
+    "file_path": {
+      "description": "Path to the audio or video file for transcription.",
+      "type": "string"
+    }
+  },
+  "name": "transcription_tool",
+  "output_type": "string",
+  "tool_class": "TranscriptionTool"
+}

transcription.py

@@ -0,0 +1,29 @@
+import torch
+from transformers import pipeline
+from logging_config import logger
+
+def run_whisper_transcription(wav_file_path: str, device: str):
+    try:
+        model_name = "distil-whisper/distil-small.en"
+        logger.info(f"Initialising Whisper ASR pipeline with model: {model_name}")
+        logger.info(f"Running pipeline on device: {device}")
+
+        asr_pipeline = pipeline(
+            "automatic-speech-recognition",
+            model=model_name,
+            device=0 if device == "cuda" else -1,
+            return_timestamps=True
+        )
+        logger.info("Whisper ASR pipeline initialised.")
+        logger.info(f"Starting transcription for file: {wav_file_path}")
+
+        # Perform transcription
+        result = asr_pipeline(wav_file_path)
+        transcription = result.get("text", "")
+        logger.info("Transcription completed successfully.")
+
+        yield transcription # Yield only the transcription string
+    except Exception as e:
+        err_msg = f"Error during transcription: {str(e)}"
+        logger.error(err_msg)
+        yield err_msg # Yield only the error message string

transcription_tool.py

@@ -0,0 +1,100 @@
+from smolagents import Tool
+import os
+import tempfile
+import shutil
+import torch
+import subprocess
+from transcription import run_whisper_transcription
+from logging_config import logger
+from ffmpeg_setup import ensure_ffmpeg_in_path
+
+
+class TranscriptTool(Tool):
+    name = "TranscriptTool"
+    description = """
+    A smolagent tool for transcribing audio and video files into text. This tool utilises Whisper for transcription 
+    and ffmpeg for media conversion, enabling agents to process multimedia inputs into text. The tool supports robust 
+    file handling, including format conversion to WAV and dynamic device selection for optimal performance.
+    """
+    inputs = {
+        "file_path": {
+            "type": "string",
+            "description": "Path to the audio or video file for transcription."
+        }
+    }
+    output_type = "string"
+
+    def __init__(self, audio_directory=None):
+        super().__init__()
+        ensure_ffmpeg_in_path()
+        self.audio_directory = audio_directory or os.getcwd()
+
+    def locate_audio_file(self, file_name):
+        for root, _, files in os.walk(self.audio_directory):
+            if file_name in files:
+                return os.path.join(root, file_name)
+        return None
+
+    def convert_audio_to_wav(self, input_file: str, output_file: str, ffmpeg_path: str) -> str:
+        logger.info(f"Converting {input_file} to WAV format: {output_file}")
+        cmd = [
+            ffmpeg_path,
+            "-y",  # Overwrite output files without asking
+            "-i", input_file,
+            "-ar", "16000",  # Set audio sampling rate to 16kHz
+            "-ac", "1",      # Set number of audio channels to mono
+            output_file
+        ]
+        try:
+            subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            logger.info("Audio conversion to WAV completed successfully.")
+            return output_file
+        except subprocess.CalledProcessError as e:
+            ffmpeg_error = e.stderr.decode()
+            logger.error(f"ffmpeg error: {ffmpeg_error}")
+            raise RuntimeError("Failed to convert audio to WAV.") from e
+
+    def forward(self, file_path: str) -> str:
+        try:
+            # Locate the file if it does not exist
+            logger.info(f"Attempting to transcribe file: {file_path}")
+            if not os.path.exists(file_path):
+                file_name = os.path.basename(file_path)
+                file_path = self.locate_audio_file(file_name)
+                if not file_path:
+                    logger.error(f"File '{file_name}' not found in '{self.audio_directory}'.")
+                    return f"Error: File '{file_name}' not found in '{self.audio_directory}'."
+
+            with tempfile.TemporaryDirectory() as tmpdir:
+                # Copy file to temp dir
+                filename = os.path.basename(file_path)
+                input_file_path = os.path.join(tmpdir, filename)
+                shutil.copy(file_path, input_file_path)
+                logger.info(f"Copied input file to temporary directory: {input_file_path}")
+
+                # Check if already WAV, otherwise convert
+                if input_file_path.lower().endswith(".wav"):
+                    logger.info(f"File {filename} is already in WAV format. Skipping conversion.")
+                    transcription_input_path = input_file_path
+                else:
+                    # Convert to wav
+                    wav_file_path = os.path.join(tmpdir, "converted_audio.wav")
+                    ffmpeg_path = shutil.which("ffmpeg")
+                    if not ffmpeg_path:
+                        logger.error("ffmpeg is not accessible in PATH.")
+                        raise RuntimeError("ffmpeg is not accessible in PATH.")
+                    self.convert_audio_to_wav(input_file_path, wav_file_path, ffmpeg_path)
+                    transcription_input_path = wav_file_path
+
+                device = "cuda" if torch.cuda.is_available() else "cpu"
+                logger.info(f"Using device for transcription: {device}")
+
+                # Transcribe audio
+                transcription_generator = run_whisper_transcription(transcription_input_path, device)
+                # The generator now yields only the result string (transcription or error)
+                for result_string in transcription_generator:
+                    return result_string # Return the first (and only) result
+
+        except Exception as e:
+            logger.error(f"An unexpected error occurred in TranscriptTool: {str(e)}")
+            return f"An error occurred: {str(e)}"