week4/community-contributions/day4_with_inference_provider.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4a6ab9a2-28a2-445d-8512-a0dc8d1b54e9",
   "metadata": {},
   "source": [
    "# Code Generator\n",
    "\n",
    "The requirement: use an Open Source model to generate high performance C++ code from Python code\n",
    "\n",
    "To replicate this, you'll need to set up a HuggingFace endpoint as I do in the video. It's simple to do, and it's quite satisfying to see the results!\n",
    "\n",
    "It's also an important part of your learning; this is the first example of deploying an open source model to be behind an API. We'll return to this in Week 8, but this should plant a seed in your mind for what's involved in moving open source models into production.\n",
    "\n",
    "Added the use of inference providers that was introduced recently by Hugging Face to convert the code.\n",
    "Improved the user prompt to include algorithic efficeiny and performance optimization.\n",
    "\n",
    "Added Java as a conversion option.\n",
    "\n",
    "Note: C++ commands work on windows environment.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22e1567b-33fd-49e7-866e-4b635d15715a",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left;\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h1 style=\"color:#900;\">Important - Pause Endpoints when not in use</h1>\n",
    "            <span style=\"color:#900;\">\n",
    "            If you do decide to use HuggingFace endpoints for this project, you should stop or pause the endpoints when you are done to avoid accruing unnecessary running cost. The costs are very low as long as you only run the endpoint when you're using it. Navigate to the HuggingFace endpoint UI <a href=\"https://ui.endpoints.huggingface.co/\">here,</a> open your endpoint, and click Pause to put it on pause so you no longer pay for it.  \n",
    "Many thanks to student John L. for raising this.\n",
    "<br/><br/>\n",
    "In week 8 we will use Modal instead of HuggingFace endpoints; with Modal you only pay for the time that you use it and you should get free credits.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "id": "e610bf56-a46e-4aff-8de1-ab49d62b1ad3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "\n",
    "import os\n",
    "import io\n",
    "import sys\n",
    "import json\n",
    "import requests\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "import google.generativeai\n",
    "import anthropic\n",
    "from IPython.display import Markdown, display, update_display\n",
    "import gradio as gr\n",
    "import subprocess, re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "4f672e1c-87e9-4865-b760-370fa605e614",
   "metadata": {},
   "outputs": [],
   "source": [
    "# environment\n",
    "\n",
    "load_dotenv()\n",
    "os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY', 'your-key-if-not-using-env')\n",
    "os.environ['ANTHROPIC_API_KEY'] = os.getenv('ANTHROPIC_API_KEY', 'your-key-if-not-using-env')\n",
    "os.environ['HF_TOKEN'] = os.getenv('HF_TOKEN', 'your-key-if-not-using-env')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "id": "8aa149ed-9298-4d69-8fe2-8f5de0f667da",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize\n",
    "\n",
    "openai = OpenAI()\n",
    "claude = anthropic.Anthropic()\n",
    "OPENAI_MODEL = \"gpt-4o\"\n",
    "CLAUDE_MODEL = \"claude-3-5-sonnet-20240620\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "id": "2db60a72-d098-42ca-8ce2-1e037c86b718",
   "metadata": {},
   "outputs": [],
   "source": [
    "def system_prompt_for(language: str) -> str:\n",
    "    system_prompt = (\n",
    "        f\"You are an assistant that reimplements Python code in high performance {language.upper()} for an Windows intel i7.\"\n",
    "        f\"Respond only with {language.upper()} code; use comments sparingly and do not provide any explanation other than occasional comments.\"\n",
    "        f\"The {language.upper()} response needs to produce an identical output in the fastest possible time. Keep implementations of random number generators identical so that results match exactly.\"\n",
    "    )\n",
    "    return system_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 243,
   "id": "70583432-e851-40d1-a219-2fb32b830dc8",
   "metadata": {},
   "outputs": [],
   "source": [
    "#updated the original prompt to include algorithic efficeiny and performance optimization\n",
    "def user_prompt_for(python: str, language: str) -> str:\n",
    "    if language.lower() not in {\"cpp\", \"java\"}:\n",
    "        raise ValueError(\"Unsupported language. Please choose 'C++' or 'Java'.\")\n",
    "    \n",
    "    optimization_notes = {\n",
    "        \"cpp\": (\n",
    "            \"- Use `int64_t` instead of `int` where necessary to prevent overflows.\\n\"\n",
    "            \"- Ensure random number generation in C++ matches Python's output as closely as possible.\\n\"\n",
    "            \"- Avoid undefined behavior, such as bit shifts that exceed type width (`1UL << 32` is incorrect for `uint32_t`).\\n\"\n",
    "            \"- Utilize `std::vector` for dynamic arrays and prefer preallocation for efficiency.\\n\"\n",
    "            \"- Consider `std::array` or `std::span` when fixed-size arrays are sufficient.\\n\"\n",
    "            \"- Optimize with **SIMD**, cache-friendly structures, and memory alignment where necessary.\\n\"\n",
    "        ),\n",
    "        \"java\": (\n",
    "            \"- Use `long` instead of `int` where necessary to prevent overflows.\\n\"\n",
    "            \"- Ensure random number generation in Java matches Python's output as closely as possible.\\n\"\n",
    "            \"- Use `ArrayList` instead of primitive arrays if dynamic resizing is needed.\\n\"\n",
    "            \"- Utilize `BigInteger` if handling large numbers that could exceed `long`.\\n\"\n",
    "            \"- Optimize with **parallel streams** (`IntStream.parallel()`) and **efficient data structures** (`HashMap`, `LinkedList`, etc.).\\n\"\n",
    "        )\n",
    "    }\n",
    "\n",
    "    user_prompt = (\n",
    "        f\"First, analyze the given Python code to understand its core purpose and algorithmic approach. \"\n",
    "        f\"Then, implement a {language} solution that achieves the same output while prioritizing:\\n\"\n",
    "        \"1. **Algorithmic Efficiency** - Optimize time and space complexity, even if it means using a different approach.\\n\"\n",
    "        \"2. **Numerical Correctness** - Prevent integer overflows, use appropriate data types (`long`, `BigInteger`, etc.), \"\n",
    "        \"and ensure correct handling of edge cases.\\n\"\n",
    "        \"3. **Performance Optimization** - Utilize language-specific features for efficiency.\\n\\n\"\n",
    "        \n",
    "        \"### **Important Notes:**\\n\"\n",
    "        + optimization_notes[language.lower()] +\n",
    "        \"\\n### **Expected Response:**\\n\"\n",
    "        f\"Respond **only with {language} code**, including all necessary imports and ensuring the output matches the Python version exactly.\\n\\n\"\n",
    "        \n",
    "        \"Here's the Python code to analyze and optimize:\\n\\n\"\n",
    "        + python\n",
    "    )\n",
    "    \n",
    "    return user_prompt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "id": "c6190659-f54c-4951-bef4-4960f8e51cc4",
   "metadata": {},
   "outputs": [],
   "source": [
    "def messages_for(python, language=\"cpp\"):\n",
    "    return [\n",
    "        {\"role\": \"system\", \"content\": system_prompt_for(language)},\n",
    "        {\"role\": \"user\", \"content\": user_prompt_for(python, language)}\n",
    "    ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "id": "71e1ba8c-5b05-4726-a9f3-8d8c6257350b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# write to a file called optimized.cpp\n",
    "\n",
    "def write_output(code, file_name):\n",
    "    with open(file_name, \"w\") as f:\n",
    "        f.write(code)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "id": "e7d2fea8-74c6-4421-8f1e-0e76d5b201b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "def optimize_gpt(python, language=\"cpp\"):    \n",
    "    stream = openai.chat.completions.create(model=OPENAI_MODEL, messages=messages_for(python, language), stream=True)\n",
    "    reply = \"\"\n",
    "    for chunk in stream:\n",
    "        fragment = chunk.choices[0].delta.content or \"\"\n",
    "        reply += fragment\n",
    "        print(fragment, end='', flush=True)\n",
    "    file_name= f\"optimized.{language}\"\n",
    "    if language == \"java\":\n",
    "        # Extract class name from Java code\n",
    "        match = re.search(r\"\\b(public\\s+)?class\\s+(\\w+)\", reply)\n",
    "        class_name = match.group(2) if match else \"OptimizedJava\"\n",
    "        file_name = f\"{class_name}.java\"\n",
    "    else:\n",
    "        file_name = f\"optimized.{language}\"\n",
    "    write_output(reply, file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "id": "7cd84ad8-d55c-4fe0-9eeb-1895c95c4a9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def optimize_claude(python, language=\"cpp\"):\n",
    "    result = claude.messages.stream(\n",
    "        model=CLAUDE_MODEL,\n",
    "        max_tokens=2000,\n",
    "        system=system_message,\n",
    "        messages=[{\"role\": \"user\", \"content\": user_prompt_for(python, language)}],\n",
    "    )\n",
    "    reply = \"\"\n",
    "    with result as stream:\n",
    "        for text in stream.text_stream:\n",
    "            reply += text\n",
    "            print(text, end=\"\", flush=True)\n",
    "    if language == \"java\":\n",
    "        # Extract class name from Java code\n",
    "        match = re.search(r\"\\b(public\\s+)?class\\s+(\\w+)\", reply)\n",
    "        class_name = match.group(2) if match else \"OptimizedJava\"\n",
    "        file_name = f\"{class_name}.java\"\n",
    "    else:\n",
    "        file_name = f\"optimized.{language}\"\n",
    "    write_output(reply, file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "id": "a1cbb778-fa57-43de-b04b-ed523f396c38",
   "metadata": {},
   "outputs": [],
   "source": [
    "pi = \"\"\"\n",
    "import time\n",
    "\n",
    "def calculate(iterations, param1, param2):\n",
    "    result = 1.0\n",
    "    for i in range(1, iterations+1):\n",
    "        j = i * param1 - param2\n",
    "        result -= (1/j)\n",
    "        j = i * param1 + param2\n",
    "        result += (1/j)\n",
    "    return result\n",
    "\n",
    "start_time = time.time()\n",
    "result = calculate(100_000_000, 4, 1) * 4\n",
    "end_time = time.time()\n",
    "\n",
    "print(f\"Result: {result:.12f}\")\n",
    "print(f\"Execution Time: {(end_time - start_time):.6f} seconds\")\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "7fe1cd4b-d2c5-4303-afed-2115a3fef200",
   "metadata": {},
   "outputs": [],
   "source": [
    "exec(pi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "105db6f9-343c-491d-8e44-3a5328b81719",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimize_gpt(pi, \"java\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf26ee95-0c77-491d-9a91-579a1e96a8a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "exec(pi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4194e40c-04ab-4940-9d64-b4ad37c5bb40",
   "metadata": {},
   "outputs": [],
   "source": [
    "!g++ -O3 -std=c++17 -march=native -o optimized optimized.cpp\n",
    "!optimized.exe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "983a11fe-e24d-4c65-8269-9802c5ef3ae6",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimize_claude(pi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5a766f9-3d23-4bb4-a1d4-88ec44b61ddf",
   "metadata": {},
   "outputs": [],
   "source": [
    "!g++ -O3 -std=c++17 -march=native -o optimized optimized.cpp\n",
    "!optimized.exe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "id": "c3b497b3-f569-420e-b92e-fb0f49957ce0",
   "metadata": {},
   "outputs": [],
   "source": [
    "python_hard = \"\"\"# Be careful to support large number sizes\n",
    "\n",
    "def lcg(seed, a=1664525, c=1013904223, m=2**32):\n",
    "    value = seed\n",
    "    while True:\n",
    "        value = (a * value + c) % m\n",
    "        yield value\n",
    "        \n",
    "def max_subarray_sum(n, seed, min_val, max_val):\n",
    "    lcg_gen = lcg(seed)\n",
    "    random_numbers = [next(lcg_gen) % (max_val - min_val + 1) + min_val for _ in range(n)]\n",
    "    max_sum = float('-inf')\n",
    "    for i in range(n):\n",
    "        current_sum = 0\n",
    "        for j in range(i, n):\n",
    "            current_sum += random_numbers[j]\n",
    "            if current_sum > max_sum:\n",
    "                max_sum = current_sum\n",
    "    return max_sum\n",
    "\n",
    "def total_max_subarray_sum(n, initial_seed, min_val, max_val):\n",
    "    total_sum = 0\n",
    "    lcg_gen = lcg(initial_seed)\n",
    "    for _ in range(20):\n",
    "        seed = next(lcg_gen)\n",
    "        total_sum += max_subarray_sum(n, seed, min_val, max_val)\n",
    "    return total_sum\n",
    "\n",
    "# Parameters\n",
    "n = 10000         # Number of random numbers\n",
    "initial_seed = 42 # Initial seed for the LCG\n",
    "min_val = -10     # Minimum value of random numbers\n",
    "max_val = 10      # Maximum value of random numbers\n",
    "\n",
    "# Timing the function\n",
    "import time\n",
    "start_time = time.time()\n",
    "result = total_max_subarray_sum(n, initial_seed, min_val, max_val)\n",
    "end_time = time.time()\n",
    "\n",
    "print(\"Total Maximum Subarray Sum (20 runs):\", result)\n",
    "print(\"Execution Time: {:.6f} seconds\".format(end_time - start_time))\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "dab5e4bc-276c-4555-bd4c-12c699d5e899",
   "metadata": {},
   "outputs": [],
   "source": [
    "exec(python_hard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8d24ed5-2c15-4f55-80e7-13a3952b3cb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimize_gpt(python_hard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0b3d073-88a2-40b2-831c-6f0c345c256f",
   "metadata": {},
   "outputs": [],
   "source": [
    "!g++ -O3 -std=c++17 -march=native -o optimized optimized.cpp\n",
    "!optimized.exe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9305446-1d0c-4b51-866a-b8c1e299bf5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimize_claude(python_hard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c181036-8193-4fdd-aef3-fc513b218d43",
   "metadata": {},
   "outputs": [],
   "source": [
    "!g++ -O3 -std=c++17 -march=native -o optimized optimized.cpp\n",
    "!optimized.exe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "id": "0be9f47d-5213-4700-b0e2-d444c7c738c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def stream_gpt(python, language=\"cpp\"):    \n",
    "    stream = openai.chat.completions.create(model=OPENAI_MODEL, messages=messages_for(python, language), stream=True)\n",
    "    reply = \"\"\n",
    "    code_block = f\"```{language}\\n\"\n",
    "    for chunk in stream:\n",
    "        fragment = chunk.choices[0].delta.content or \"\"\n",
    "        reply += fragment\n",
    "        cleaned_reply = reply.replace(code_block,'').replace('```','')\n",
    "        yield cleaned_reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "id": "8669f56b-8314-4582-a167-78842caea131",
   "metadata": {},
   "outputs": [],
   "source": [
    "def stream_claude(python, language=\"cpp\"):\n",
    "    result = claude.messages.stream(\n",
    "        model=CLAUDE_MODEL,\n",
    "        max_tokens=2000,\n",
    "        system=system_message,\n",
    "        messages=[{\"role\": \"user\", \"content\": user_prompt_for(python, language)}],\n",
    "    )\n",
    "    reply = \"\"\n",
    "    code_block = f\"```{language}\\n\"\n",
    "    with result as stream:\n",
    "        for text in stream.text_stream:\n",
    "            reply += text\n",
    "            cleaned_reply = reply.replace(code_block,'').replace('```','')\n",
    "            yield cleaned_reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "id": "2f1ae8f5-16c8-40a0-aa18-63b617df078d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def optimize(python, model):\n",
    "    if model==\"GPT\":\n",
    "        result = stream_gpt(python)\n",
    "    elif model==\"Claude\":\n",
    "        result = stream_claude(python)\n",
    "    else:\n",
    "        raise ValueError(\"Unknown model\")\n",
    "    for stream_so_far in result:\n",
    "        yield stream_so_far        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "id": "f1ddb38e-6b0a-4c37-baa4-ace0b7de887a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7888/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with gr.Blocks() as ui:\n",
    "    with gr.Row():\n",
    "        python = gr.Textbox(label=\"Python code:\", lines=10, value=python_hard)\n",
    "        cpp = gr.Textbox(label=\"C++ code:\", lines=10)\n",
    "    with gr.Row():\n",
    "        model = gr.Dropdown([\"GPT\", \"Claude\"], label=\"Select model\", value=\"GPT\")\n",
    "        convert = gr.Button(\"Convert code\")\n",
    "\n",
    "    convert.click(optimize, inputs=[python, model], outputs=[cpp])\n",
    "\n",
    "ui.launch(inbrowser=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "id": "19bf2bff-a822-4009-a539-f003b1651383",
   "metadata": {},
   "outputs": [],
   "source": [
    "def execute_python(code):\n",
    "    try:\n",
    "        output = io.StringIO()\n",
    "        sys.stdout = output\n",
    "        exec(code)\n",
    "    finally:\n",
    "        sys.stdout = sys.__stdout__\n",
    "    return output.getvalue()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "id": "9a2274f1-d03b-42c0-8dcc-4ce159b18442",
   "metadata": {},
   "outputs": [],
   "source": [
    "css = \"\"\"\n",
    ".python {background-color: #306998;}\n",
    ".cpp {background-color: #050;}\n",
    ".java {background-color: #306775;}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "f1303932-160c-424b-97a8-d28c816721b2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7868/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with gr.Blocks(css=css) as ui:\n",
    "    gr.Markdown(\"## Convert code from Python to C++\")\n",
    "    with gr.Row():\n",
    "        python = gr.Textbox(label=\"Python code:\", value=python_hard, lines=10)\n",
    "        cpp = gr.Textbox(label=\"C++ code:\", lines=10)\n",
    "    with gr.Row():\n",
    "        model = gr.Dropdown([\"GPT\", \"Claude\"], label=\"Select model\", value=\"GPT\")\n",
    "    with gr.Row():\n",
    "        convert = gr.Button(\"Convert code\")\n",
    "    with gr.Row():\n",
    "        python_run = gr.Button(\"Run Python\")\n",
    "        cpp_run = gr.Button(\"Run C++\")\n",
    "    with gr.Row():\n",
    "        python_out = gr.TextArea(label=\"Python result:\", elem_classes=[\"python\"])\n",
    "        cpp_out = gr.TextArea(label=\"C++ result:\", elem_classes=[\"cpp\"])\n",
    "\n",
    "    convert.click(optimize, inputs=[python, model], outputs=[cpp])\n",
    "    python_run.click(execute_python, inputs=[python], outputs=[python_out])\n",
    "    cpp_run.click(execute_cpp, inputs=[cpp], outputs=[cpp_out])\n",
    "\n",
    "ui.launch(inbrowser=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "id": "bb8c5b4e-ec51-4f21-b3f8-6aa94fede86d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from huggingface_hub import login, InferenceClient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "13347633-4606-4e38-9927-80c39e65c1f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Note: Environment variable`HF_TOKEN` is set and is the current active token independently from the token you've just configured.\n"
     ]
    }
   ],
   "source": [
    "hf_token = os.environ['HF_TOKEN']\n",
    "login(hf_token)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "id": "ef60a4df-6267-4ebd-8eed-dcb917af0a5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "code_qwen = \"Qwen/CodeQwen1.5-7B-Chat\"\n",
    "code_gemma = \"google/codegemma-7b-it\"\n",
    "messages=messages_for(pi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "3825d77a-03c6-42b2-89bc-ccbcb1585740",
   "metadata": {},
   "outputs": [
    {
     "ename": "HfHubHTTPError",
     "evalue": "402 Client Error: Payment Required for url: https://huggingface.co/api/inference-proxy/sambanova/v1/chat/completions (Request ID: Root=1-67afb729-1eb9aff1704314144ef14e59;2df843ad-b7d2-4145-bb7b-1cfd94ae19ef)\n\nYou have exceeded your monthly included credits for Inference Endpoints. Subscribe to PRO to get 20x more monthly allowance.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mHTTPError\u001b[0m                                 Traceback (most recent call last)",
      "File \u001b[1;32m~\\anaconda3\\envs\\llms\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:406\u001b[0m, in \u001b[0;36mhf_raise_for_status\u001b[1;34m(response, endpoint_name)\u001b[0m\n\u001b[0;32m    405\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m--> 406\u001b[0m     \u001b[43mresponse\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mraise_for_status\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    407\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m HTTPError \u001b[38;5;28;01mas\u001b[39;00m e:\n",
      "File \u001b[1;32m~\\anaconda3\\envs\\llms\\Lib\\site-packages\\requests\\models.py:1024\u001b[0m, in \u001b[0;36mResponse.raise_for_status\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m   1023\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m http_error_msg:\n\u001b[1;32m-> 1024\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m HTTPError(http_error_msg, response\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m)\n",
      "\u001b[1;31mHTTPError\u001b[0m: 402 Client Error: Payment Required for url: https://huggingface.co/api/inference-proxy/sambanova/v1/chat/completions",
      "\nThe above exception was the direct cause of the following exception:\n",
      "\u001b[1;31mHfHubHTTPError\u001b[0m                            Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[119], line 5\u001b[0m\n\u001b[0;32m      1\u001b[0m client \u001b[38;5;241m=\u001b[39m InferenceClient(\n\u001b[0;32m      2\u001b[0m \tprovider\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124msambanova\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m      3\u001b[0m \tapi_key\u001b[38;5;241m=\u001b[39mhf_token\n\u001b[0;32m      4\u001b[0m )\n\u001b[1;32m----> 5\u001b[0m stream \u001b[38;5;241m=\u001b[39m \u001b[43mclient\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mchat\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompletions\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m      6\u001b[0m \u001b[43m\t\u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mQwen/Qwen2.5-Coder-32B-Instruct\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\n\u001b[0;32m      7\u001b[0m \u001b[43m\t\u001b[49m\u001b[43mmessages\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\n\u001b[0;32m      8\u001b[0m \u001b[43m\t\u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m500\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m      9\u001b[0m \u001b[43m\t\u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\n\u001b[0;32m     10\u001b[0m \u001b[43m)\u001b[49m\n\u001b[0;32m     12\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m chunk \u001b[38;5;129;01min\u001b[39;00m stream:\n\u001b[0;32m     13\u001b[0m     \u001b[38;5;28mprint\u001b[39m(chunk\u001b[38;5;241m.\u001b[39mchoices[\u001b[38;5;241m0\u001b[39m]\u001b[38;5;241m.\u001b[39mdelta\u001b[38;5;241m.\u001b[39mcontent, end\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "File \u001b[1;32m~\\anaconda3\\envs\\llms\\Lib\\site-packages\\huggingface_hub\\inference\\_client.py:970\u001b[0m, in \u001b[0;36mInferenceClient.chat_completion\u001b[1;34m(self, messages, model, stream, frequency_penalty, logit_bias, logprobs, max_tokens, n, presence_penalty, response_format, seed, stop, stream_options, temperature, tool_choice, tool_prompt, tools, top_logprobs, top_p)\u001b[0m\n\u001b[0;32m    943\u001b[0m parameters \u001b[38;5;241m=\u001b[39m {\n\u001b[0;32m    944\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel\u001b[39m\u001b[38;5;124m\"\u001b[39m: payload_model,\n\u001b[0;32m    945\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfrequency_penalty\u001b[39m\u001b[38;5;124m\"\u001b[39m: frequency_penalty,\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m    961\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstream_options\u001b[39m\u001b[38;5;124m\"\u001b[39m: stream_options,\n\u001b[0;32m    962\u001b[0m }\n\u001b[0;32m    963\u001b[0m request_parameters \u001b[38;5;241m=\u001b[39m provider_helper\u001b[38;5;241m.\u001b[39mprepare_request(\n\u001b[0;32m    964\u001b[0m     inputs\u001b[38;5;241m=\u001b[39mmessages,\n\u001b[0;32m    965\u001b[0m     parameters\u001b[38;5;241m=\u001b[39mparameters,\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m    968\u001b[0m     api_key\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtoken,\n\u001b[0;32m    969\u001b[0m )\n\u001b[1;32m--> 970\u001b[0m data \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_inner_post\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest_parameters\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    972\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m stream:\n\u001b[0;32m    973\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m _stream_chat_completion_response(data)  \u001b[38;5;66;03m# type: ignore[arg-type]\u001b[39;00m\n",
      "File \u001b[1;32m~\\anaconda3\\envs\\llms\\Lib\\site-packages\\huggingface_hub\\inference\\_client.py:327\u001b[0m, in \u001b[0;36mInferenceClient._inner_post\u001b[1;34m(self, request_parameters, stream)\u001b[0m\n\u001b[0;32m    324\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m InferenceTimeoutError(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInference call timed out: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mrequest_parameters\u001b[38;5;241m.\u001b[39murl\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01merror\u001b[39;00m  \u001b[38;5;66;03m# type: ignore\u001b[39;00m\n\u001b[0;32m    326\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m--> 327\u001b[0m     \u001b[43mhf_raise_for_status\u001b[49m\u001b[43m(\u001b[49m\u001b[43mresponse\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    328\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m response\u001b[38;5;241m.\u001b[39miter_lines() \u001b[38;5;28;01mif\u001b[39;00m stream \u001b[38;5;28;01melse\u001b[39;00m response\u001b[38;5;241m.\u001b[39mcontent\n\u001b[0;32m    329\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m HTTPError \u001b[38;5;28;01mas\u001b[39;00m error:\n",
      "File \u001b[1;32m~\\anaconda3\\envs\\llms\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:477\u001b[0m, in \u001b[0;36mhf_raise_for_status\u001b[1;34m(response, endpoint_name)\u001b[0m\n\u001b[0;32m    473\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m _format(HfHubHTTPError, message, response) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01me\u001b[39;00m\n\u001b[0;32m    475\u001b[0m \u001b[38;5;66;03m# Convert `HTTPError` into a `HfHubHTTPError` to display request information\u001b[39;00m\n\u001b[0;32m    476\u001b[0m \u001b[38;5;66;03m# as well (request id and/or server error message)\u001b[39;00m\n\u001b[1;32m--> 477\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m _format(HfHubHTTPError, \u001b[38;5;28mstr\u001b[39m(e), response) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01me\u001b[39;00m\n",
      "\u001b[1;31mHfHubHTTPError\u001b[0m: 402 Client Error: Payment Required for url: https://huggingface.co/api/inference-proxy/sambanova/v1/chat/completions (Request ID: Root=1-67afb729-1eb9aff1704314144ef14e59;2df843ad-b7d2-4145-bb7b-1cfd94ae19ef)\n\nYou have exceeded your monthly included credits for Inference Endpoints. Subscribe to PRO to get 20x more monthly allowance."
     ]
    }
   ],
   "source": [
    "client = InferenceClient(\n",
    "\tprovider=\"sambanova\",\n",
    "\tapi_key=hf_token\n",
    ")\n",
    "stream = client.chat.completions.create(\n",
    "\tmodel=\"Qwen/Qwen2.5-Coder-32B-Instruct\", \n",
    "\tmessages=messages, \n",
    "\tmax_tokens=500,\n",
    "\tstream=True\n",
    ")\n",
    "\n",
    "for chunk in stream:\n",
    "    print(chunk.choices[0].delta.content, end=\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "cc0c3e9c-2572-41d1-a476-6eae96b20695",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using inference providers\n",
    "def stream_code_qwen(python):\n",
    "    messages = messages_for(python)\n",
    "    client = InferenceClient(\n",
    "    \tprovider=\"sambanova\",\n",
    "    \tapi_key=hf_token\n",
    "    )\n",
    "    stream = client.chat.completions.create(\n",
    "    \tmodel=\"Qwen/Qwen2.5-Coder-32B-Instruct\", \n",
    "    \tmessages=messages, \n",
    "    \tmax_tokens=500,\n",
    "    \tstream=True\n",
    "    )\n",
    "    result = \"\"\n",
    "    for chunk in stream:\n",
    "        if chunk.choices and chunk.choices[0].delta.content:\n",
    "            result += chunk.choices[0].delta.content\n",
    "            yield result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "id": "a82387d1-7651-4923-995b-fe18356fcaa6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def optimize(python, model, language):\n",
    "    if model==\"GPT\":\n",
    "        result = stream_gpt(python, language)\n",
    "    elif model==\"Claude\":\n",
    "        result = stream_claude(python, language)\n",
    "    elif model==\"CodeQwen\":\n",
    "        result = stream_code_qwen(python, language)\n",
    "    else:\n",
    "        raise ValueError(\"Unknown model\")\n",
    "    for stream_so_far in result:\n",
    "        yield stream_so_far    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "id": "4ba311ec-c16a-4fe0-946b-4b940704cf65",
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_sample_program(sample_program):\n",
    "    if sample_program==\"pi\":\n",
    "        return pi\n",
    "    elif sample_program==\"python_hard\":\n",
    "        return python_hard\n",
    "    else:\n",
    "        return \"Type your Python program here\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "id": "06148e88-501b-4686-a41d-c3be528d8e6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def execute_cpp(code):\n",
    "        write_output(code, \"optimized.exe\")\n",
    "        try:\n",
    "            compile_cmd = [\"g++\", \"-Ofast\", \"-std=c++17\", \"-march=native\", \"-mtune=intel\", \"-o\", \"optimized\", \"optimized.cpp\"]\n",
    "            compile_result = subprocess.run(compile_cmd, check=True, text=True, capture_output=True)\n",
    "            run_cmd = [\"optimized.exe\"]\n",
    "            run_result = subprocess.run(run_cmd, check=True, text=True, capture_output=True)\n",
    "            return run_result.stdout\n",
    "        except subprocess.CalledProcessError as e:\n",
    "            return f\"An error occurred:\\n{e.stderr}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "id": "a42e3871-f3a5-4f14-836c-1e8ecacb56b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "def execute_java(code):\n",
    "    # Extract the class name from the Java code\n",
    "    match = re.search(r\"\\b(public\\s+)?class\\s+(\\w+)\", code)\n",
    "    class_name = match.group(2) if match else \"OptimizedJava\"\n",
    "\n",
    "    file_name = f\"{class_name}.java\"\n",
    "    write_output(code, file_name)\n",
    "    try:\n",
    "        compile_cmd =[\"javac\", file_name]\n",
    "        subprocess.run(compile_cmd, check=True, text=True, capture_output=True)\n",
    "        run_cmd = [\"java\", class_name]\n",
    "        run_result = subprocess.run(run_cmd, check=True, text=True, capture_output=True)\n",
    "        return run_result.stdout\n",
    "    except subprocess.CalledProcessError as e:\n",
    "        return f\"Error during compilation or execution:\\n{e.stderr}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "id": "f9ca2e6f-60c1-4e5f-b570-63c75b2d189b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7901/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 238,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with gr.Blocks(css=css) as ui:\n",
    "    gr.Markdown(\"## Convert code from Python to C++ or Java\")\n",
    "    #input and output\n",
    "    with gr.Row():\n",
    "        python = gr.Textbox(label=\"Python code:\", value=python_hard, lines=10)\n",
    "        converted_code = gr.Textbox(label=\"Converted code:\", lines=10)\n",
    "        # java = gr.Textbox(label=\"Java code:\", lines=10)\n",
    "    #sample programs\n",
    "    with gr.Row():\n",
    "        with gr.Column():\n",
    "            sample_program = gr.Radio([\"pi\", \"python_hard\"], label=\"Sample program\", value=\"python_hard\")\n",
    "    #select model and language\n",
    "    with gr.Row():\n",
    "        with gr.Column():\n",
    "            model = gr.Dropdown([\"GPT\", \"Claude\", \"CodeQwen\"], label=\"Select model\", value=\"GPT\")\n",
    "            language = gr.Dropdown([\"C++\",\"Java\"], label=\"Select language\", value=\"C++\")\n",
    "    with gr.Row():\n",
    "        convert = gr.Button(\"Convert\")\n",
    "    #Code execution\n",
    "    with gr.Row():\n",
    "        python_run = gr.Button(\"Run Python\")\n",
    "        converted_run = gr.Button(\"Run converted code\")\n",
    "    with gr.Row():\n",
    "        python_out = gr.TextArea(label=\"Python result:\", elem_classes=[\"python\"])\n",
    "        output = gr.TextArea(label=\"Converted code result:\", elem_classes=[\"cpp\"])\n",
    "        \n",
    "    # Function to convert Python code based on language\n",
    "    def convert_code(python_code, model, selected_language):\n",
    "        if selected_language == \"C++\":\n",
    "            for chunk in optimize(python_code, model, \"cpp\"):\n",
    "                yield chunk  # Stream each chunk\n",
    "        elif selected_language == \"Java\":\n",
    "            for chunk in optimize(python_code, model, \"java\"):\n",
    "                yield chunk\n",
    "        return \"\"\n",
    "\n",
    "    # Function to execute converted code\n",
    "    def run_code(converted_code, selected_language):\n",
    "        if selected_language == \"C++\":\n",
    "            return execute_cpp(converted_code)\n",
    "        elif selected_language == \"Java\":\n",
    "            return execute_java(converted_code)\n",
    "        return \"Invalid language selection\"\n",
    "\n",
    "    sample_program.change(select_sample_program, inputs=[sample_program], outputs=[python])\n",
    "    convert.click(convert_code, inputs=[python, model, language], outputs=[converted_code])\n",
    "    converted_run.click(run_code, inputs=[converted_code, language], outputs=[output])   \n",
    "    python_run.click(execute_python, inputs=[python], outputs=[python_out])\n",
    "\n",
    "ui.launch(inbrowser=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d0ad093-425b-488e-8c3f-67f729dd9c06",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}