Upload folder using huggingface_hub

README.md

@@ -4,3 +4,80 @@ app_file: main.py
 sdk: gradio
 sdk_version: 5.23.1
 ---
+
+# Model Recommender
+
+An intelligent agent that searches for and recommends the best AI models for specific tasks by analyzing leaderboards, benchmarks, and arenas across the Hugging Face ecosystem. This tool represents the first step towards self-assembling and self-improving AI systems.
+
+## Overview
+
+The Model Recommender automates the process of finding optimal AI models for specific tasks by searching through Hugging Face's extensive collection of leaderboards, benchmarks, and model arenas. Instead of manually comparing models across different evaluation metrics, this agent can quickly identify the top-performing models for your specific use case, saving valuable research time and ensuring you're using state-of-the-art solutions.
+
+## How It Works
+
+This tool leverages the Hugging Face API to search for and analyze model benchmarking spaces. When given a task description, the agent:
+
+1. Searches for relevant leaderboards, benchmarks, and arenas using targeted queries
+2. Analyzes the search results to identify the most relevant evaluation spaces
+3. Examines the content of these spaces to extract model performance data
+4. Compiles a ranked list of the best models for the specified task
+5. Provides detailed information about each model, including performance metrics and links
+
+The agent can also download and examine specific files from Hugging Face Spaces to extract more detailed information about model performance and implementation details.
+
+## Features
+
+- **Intelligent Leaderboard Search**: Finds relevant model evaluation spaces based on task descriptions
+- **Comprehensive Model Analysis**: Examines multiple evaluation metrics to identify truly optimal models
+- **Detailed Reporting**: Provides performance metrics, model links, and implementation details
+- **Space Content Examination**: Can inspect the contents of Hugging Face Spaces to extract additional information
+- **File Download Capability**: Can download specific files or entire repositories for deeper analysis
+
+
+
+## Installation
+
+Install the required dependencies:
+
+```bash
+pip install -r requirements.txt
+```
+
+## Usage
+
+Run the application:
+
+```bash
+python main.py
+```
+
+This will launch a Gradio interface where you can:
+
+1. Enter a task description (e.g., "text classification", "image generation", "question answering")
+2. The agent will search for relevant leaderboards and benchmarks
+3. Review the recommended models, their performance metrics, and links to learn more
+
+## Example Queries
+
+- "Find the best models for text classification"
+- "What are the top performing models for image generation?"
+- "I need a model for question answering in a production environment"
+- "Which models perform best on the GLUE benchmark?"
+
+## Towards Self-Assembling AI Systems
+
+This Model Recommender represents the first step towards truly self-assembling and self-improving AI systems. By automating the discovery and evaluation of model components, we're building the foundation for systems that can:
+
+1. **Self-Optimize**: Automatically identify and integrate better-performing components
+2. **Self-Assemble**: Combine specialized models to solve complex tasks
+3. **Self-Evaluate**: Understand their own performance characteristics and limitations
+
+Future iterations will expand beyond model discovery to include automated fine-tuning, integration, and deployment - moving us closer to AI systems that can improve themselves with minimal human intervention.
+
+## Customization
+
+To modify the search behavior or add additional tools, edit the `main.py` file:
+
+- Adjust the `search_words` list in the `leaderboard_search` function to target different types of evaluation spaces
+- Add new tools to extract more specific information from Hugging Face repositories
+- Modify the agent description to focus on specific types of models or evaluation metrics

main.py

@@ -1,11 +1,19 @@
 from smolagents import CodeAgent, InferenceClientModel, GradioUI, tool
-from huggingface_hub import HfApi
-from huggingface_hub import snapshot_download
+import os
+from huggingface_hub import HfApi, login, snapshot_download
 import requests
 from typing import List, Dict
 from typing import Optional, Union
 from pathlib import Path
 
+# read token from environment
+HF_TOKEN = os.getenv("HUGGINGFACE_API_KEY")
+if not HF_TOKEN:
+    raise RuntimeError("HUGGINGFACE_API_KEY environment variable is not set")
+
+# tell huggingface_hub to use it
+login(token=HF_TOKEN)
+
 @tool
 def leaderboard_search(query: str) -> str:
     """
@@ -29,7 +37,7 @@ def leaderboard_search(query: str) -> str:
                 "full": True  # Get full information
             }
             
-            response = requests.get(api_url, params=params)
+            response = requests.get(api_url, params=params, headers={"Authorization": f"Bearer {HF_TOKEN}"})
             print(response)
             
             spaces = response.json()
@@ -137,7 +145,7 @@ def get_file_from_space(space_id: str, file_path: str) -> str:
     """
     try:
         url = f"https://huggingface.co/spaces/{space_id}/raw/main/{file_path}"
-        response = requests.get(url)
+        response = requests.get(url, headers={"Authorization": f"Bearer {HF_TOKEN}"})
         
         if response.status_code == 200:
             return f"Content of {file_path} from {space_id}:\n\n{response.text}"