week5/day1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dfe37963-1af6-44fc-a841-8e462443f5e6",
   "metadata": {},
   "source": [
    "## Expert Knowledge Worker\n",
    "\n",
    "### A question answering agent that is an expert knowledge worker\n",
    "### To be used by employees of Insurellm, an Insurance Tech company\n",
    "### The agent needs to be accurate and the solution should be low cost.\n",
    "\n",
    "This project will use RAG (Retrieval Augmented Generation) to ensure our question/answering assistant has high accuracy.\n",
    "\n",
    "This first implementation will use a simple, brute-force type of RAG..\n",
    "\n",
    "### Sidenote: Business applications of this week's projects\n",
    "\n",
    "RAG is perhaps the most immediately applicable technique of anything that we cover in the course! In fact, there are commercial products that do precisely what we build this week: nuanced querying across large databases of information, such as company contracts or product specs. RAG gives you a quick-to-market, low cost mechanism for adapting an LLM to your business area."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba2779af-84ef-4227-9e9e-6eaf0df87e77",
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "\n",
    "import os\n",
    "import glob\n",
    "from dotenv import load_dotenv\n",
    "import gradio as gr\n",
    "from openai import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58c85082-e417-4708-9efe-81a5d55d1424",
   "metadata": {},
   "outputs": [],
   "source": [
    "# price is a factor for our company, so we're going to use a low cost model\n",
    "\n",
    "MODEL = \"gpt-4o-mini\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee78efcb-60fe-449e-a944-40bab26261af",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load environment variables in a file called .env\n",
    "\n",
    "load_dotenv(override=True)\n",
    "os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY', 'your-key-if-not-using-env')\n",
    "openai = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e0652c2-3d76-40c7-8313-9dc1895155a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# With massive thanks to student Dr John S. for fixing a bug in the below for Windows users!\n",
    "\n",
    "context = {}\n",
    "\n",
    "employees = glob.glob(\"knowledge-base/employees/*\")\n",
    "\n",
    "for employee in employees:\n",
    "    name = employee.split(' ')[-1][:-3]\n",
    "    doc = \"\"\n",
    "    with open(employee, \"r\", encoding=\"utf-8\") as f:\n",
    "        doc = f.read()\n",
    "    context[name]=doc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c85a11b-b04d-4066-b243-f96139ca106f",
   "metadata": {},
   "outputs": [],
   "source": [
    "context[\"Lancaster\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1d231f9-091e-4c72-b0f8-6af578a74e22",
   "metadata": {},
   "outputs": [],
   "source": [
    "products = glob.glob(\"knowledge-base/products/*\")\n",
    "\n",
    "for product in products:\n",
    "    name = product.split(os.sep)[-1][:-3]\n",
    "    doc = \"\"\n",
    "    with open(product, \"r\", encoding=\"utf-8\") as f:\n",
    "        doc = f.read()\n",
    "    context[name]=doc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aba46a57-d973-4195-8fe3-70fc60687192",
   "metadata": {},
   "outputs": [],
   "source": [
    "context.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "129c7d1e-0094-4479-9459-f9360b95f244",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_message = \"You are an expert in answering accurate questions about Insurellm, the Insurance Tech company. Give brief, accurate answers. If you don't know the answer, say so. Do not make anything up if you haven't been provided with relevant context.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d40e390b-c110-42d5-8d80-daf3295b9862",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_relevant_context(message):\n",
    "    relevant_context = []\n",
    "    for context_title, context_details in context.items():\n",
    "        if context_title.lower() in message.lower():\n",
    "            relevant_context.append(context_details)\n",
    "    return relevant_context          "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d126cfcb-e85c-4dd9-837e-9d2b8436d4b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "get_relevant_context(\"Who is lancaster?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d94c768d-c47a-4c34-85e9-7b786da96507",
   "metadata": {},
   "outputs": [],
   "source": [
    "get_relevant_context(\"Who is Avery and what is carllm?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a7cef7f-f214-4bac-8217-3f9ab9ba1bf0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_context(message):\n",
    "    relevant_context = get_relevant_context(message)\n",
    "    if relevant_context:\n",
    "        message += \"\\n\\nThe following additional context might be relevant in answering this question:\\n\\n\"\n",
    "        for relevant in relevant_context:\n",
    "            message += relevant + \"\\n\\n\"\n",
    "    return message"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b36399c-440b-4049-9d39-68d208283c71",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(add_context(\"Who is Alex Lancaster?\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "968e7bf2-e862-4679-a11f-6c1efb6ec8ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_message}] + history\n",
    "    message = add_context(message)\n",
    "    messages.append({\"role\": \"user\", \"content\": message})\n",
    "\n",
    "    stream = openai.chat.completions.create(model=MODEL, messages=messages, stream=True)\n",
    "\n",
    "    response = \"\"\n",
    "    for chunk in stream:\n",
    "        response += chunk.choices[0].delta.content or ''\n",
    "        yield response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbbcb659-13ce-47ab-8a5e-01b930494964",
   "metadata": {},
   "source": [
    "## Now we will bring this up in Gradio using the Chat interface -\n",
    "\n",
    "A quick and easy way to prototype a chat with an LLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3536590-85c7-4155-bd87-ae78a1467670",
   "metadata": {},
   "outputs": [],
   "source": [
    "view = gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48873d11-2fbd-4329-af27-46c781788561",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}