week5/day3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dfe37963-1af6-44fc-a841-8e462443f5e6",
   "metadata": {},
   "source": [
    "## Expert Knowledge Worker\n",
    "\n",
    "### A question answering agent that is an expert knowledge worker\n",
    "### To be used by employees of Insurellm, an Insurance Tech company\n",
    "### The agent needs to be accurate and the solution should be low cost.\n",
    "\n",
    "This project will use RAG (Retrieval Augmented Generation) to ensure our question/answering assistant has high accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba2779af-84ef-4227-9e9e-6eaf0df87e77",
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "\n",
    "import os\n",
    "import glob\n",
    "from dotenv import load_dotenv\n",
    "import gradio as gr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "802137aa-8a74-45e0-a487-d1974927d7ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports for langchain and Chroma and plotly\n",
    "\n",
    "from langchain.document_loaders import DirectoryLoader, TextLoader\n",
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "from langchain.schema import Document\n",
    "from langchain_openai import OpenAIEmbeddings, ChatOpenAI\n",
    "from langchain_chroma import Chroma\n",
    "import numpy as np\n",
    "from sklearn.manifold import TSNE\n",
    "import plotly.graph_objects as go"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58c85082-e417-4708-9efe-81a5d55d1424",
   "metadata": {},
   "outputs": [],
   "source": [
    "# price is a factor for our company, so we're going to use a low cost model\n",
    "\n",
    "MODEL = \"gpt-4o-mini\"\n",
    "db_name = \"vector_db\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee78efcb-60fe-449e-a944-40bab26261af",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load environment variables in a file called .env\n",
    "\n",
    "load_dotenv(override=True)\n",
    "os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY', 'your-key-if-not-using-env')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "730711a9-6ffe-4eee-8f48-d6cfb7314905",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read in documents using LangChain's loaders\n",
    "# Take everything in all the sub-folders of our knowledgebase\n",
    "\n",
    "folders = glob.glob(\"knowledge-base/*\")\n",
    "\n",
    "# With thanks to CG and Jon R, students on the course, for this fix needed for some users \n",
    "text_loader_kwargs = {'encoding': 'utf-8'}\n",
    "# If that doesn't work, some Windows users might need to uncomment the next line instead\n",
    "# text_loader_kwargs={'autodetect_encoding': True}\n",
    "\n",
    "documents = []\n",
    "for folder in folders:\n",
    "    doc_type = os.path.basename(folder)\n",
    "    loader = DirectoryLoader(folder, glob=\"**/*.md\", loader_cls=TextLoader, loader_kwargs=text_loader_kwargs)\n",
    "    folder_docs = loader.load()\n",
    "    for doc in folder_docs:\n",
    "        doc.metadata[\"doc_type\"] = doc_type\n",
    "        documents.append(doc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f065d4b1-80b7-4e15-abd4-60a83e752ea8",
   "metadata": {},
   "source": [
    "# Please note:\n",
    "\n",
    "In the next cell, we split the text into chunks.\n",
    "\n",
    "2 students let me know that the next cell crashed their computer.  \n",
    "They were able to fix it by changing the chunk_size from 1,000 to 2,000 and the chunk_overlap from 200 to 400.  \n",
    "This shouldn't be required; but if it happens to you, please make that change!  \n",
    "(Note that LangChain may give a warning about a chunk being larger than 1,000 - this can be safely ignored).\n",
    "\n",
    "_With much thanks to Steven W and Nir P for this valuable contribution._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7310c9c8-03c1-4efc-a104-5e89aec6db1a",
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\n",
    "chunks = text_splitter.split_documents(documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd06e02f-6d9b-44cc-a43d-e1faa8acc7bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(chunks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c54b4b6-06da-463d-bee7-4dd456c2b887",
   "metadata": {},
   "outputs": [],
   "source": [
    "doc_types = set(chunk.metadata['doc_type'] for chunk in chunks)\n",
    "print(f\"Document types found: {', '.join(doc_types)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77f7d2a6-ccfa-425b-a1c3-5e55b23bd013",
   "metadata": {},
   "source": [
    "## A sidenote on Embeddings, and \"Auto-Encoding LLMs\"\n",
    "\n",
    "We will be mapping each chunk of text into a Vector that represents the meaning of the text, known as an embedding.\n",
    "\n",
    "OpenAI offers a model to do this, which we will use by calling their API with some LangChain code.\n",
    "\n",
    "This model is an example of an \"Auto-Encoding LLM\" which generates an output given a complete input.\n",
    "It's different to all the other LLMs we've discussed today, which are known as \"Auto-Regressive LLMs\", and generate future tokens based only on past context.\n",
    "\n",
    "Another example of an Auto-Encoding LLMs is BERT from Google. In addition to embedding, Auto-encoding LLMs are often used for classification.\n",
    "\n",
    "### Sidenote\n",
    "\n",
    "In week 8 we will return to RAG and vector embeddings, and we will use an open-source vector encoder so that the data never leaves our computer - that's an important consideration when building enterprise systems and the data needs to remain internal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "78998399-ac17-4e28-b15f-0b5f51e6ee23",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Put the chunks of data into a Vector Store that associates a Vector Embedding with each chunk\n",
    "\n",
    "embeddings = OpenAIEmbeddings()\n",
    "\n",
    "# If you would rather use the free Vector Embeddings from HuggingFace sentence-transformers\n",
    "# Then replace embeddings = OpenAIEmbeddings()\n",
    "# with:\n",
    "# from langchain.embeddings import HuggingFaceEmbeddings\n",
    "# embeddings = HuggingFaceEmbeddings(model_name=\"sentence-transformers/all-MiniLM-L6-v2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "763e51ff-5787-4a56-8176-36b7c5796fe3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check if a Chroma Datastore already exists - if so, delete the collection to start from scratch\n",
    "\n",
    "if os.path.exists(db_name):\n",
    "    Chroma(persist_directory=db_name, embedding_function=embeddings).delete_collection()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99fe3a37-480f-4d55-be48-120588d5846b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create our Chroma vectorstore!\n",
    "\n",
    "vectorstore = Chroma.from_documents(documents=chunks, embedding=embeddings, persist_directory=db_name)\n",
    "print(f\"Vectorstore created with {vectorstore._collection.count()} documents\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "057868f6-51a6-4087-94d1-380145821550",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get one vector and find how many dimensions it has\n",
    "\n",
    "collection = vectorstore._collection\n",
    "sample_embedding = collection.get(limit=1, include=[\"embeddings\"])[\"embeddings\"][0]\n",
    "dimensions = len(sample_embedding)\n",
    "print(f\"The vectors have {dimensions:,} dimensions\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61e393a0-dd4c-419f-842f-60c1cb3b716b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b0d45462-a818-441c-b010-b85b32bcf618",
   "metadata": {},
   "source": [
    "## Visualizing the Vector Store\n",
    "\n",
    "Let's take a minute to look at the documents and their embedding vectors to see what's going on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b98adf5e-d464-4bd2-9bdf-bc5b6770263b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prework\n",
    "\n",
    "result = collection.get(include=['embeddings', 'documents', 'metadatas'])\n",
    "vectors = np.array(result['embeddings'])\n",
    "documents = result['documents']\n",
    "doc_types = [metadata['doc_type'] for metadata in result['metadatas']]\n",
    "colors = [['blue', 'green', 'red', 'orange'][['products', 'employees', 'contracts', 'company'].index(t)] for t in doc_types]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "427149d5-e5d8-4abd-bb6f-7ef0333cca21",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We humans find it easier to visalize things in 2D!\n",
    "# Reduce the dimensionality of the vectors to 2D using t-SNE\n",
    "# (t-distributed stochastic neighbor embedding)\n",
    "\n",
    "tsne = TSNE(n_components=2, random_state=42)\n",
    "reduced_vectors = tsne.fit_transform(vectors)\n",
    "\n",
    "# Create the 2D scatter plot\n",
    "fig = go.Figure(data=[go.Scatter(\n",
    "    x=reduced_vectors[:, 0],\n",
    "    y=reduced_vectors[:, 1],\n",
    "    mode='markers',\n",
    "    marker=dict(size=5, color=colors, opacity=0.8),\n",
    "    text=[f\"Type: {t}<br>Text: {d[:100]}...\" for t, d in zip(doc_types, documents)],\n",
    "    hoverinfo='text'\n",
    ")])\n",
    "\n",
    "fig.update_layout(\n",
    "    title='2D Chroma Vector Store Visualization',\n",
    "    scene=dict(xaxis_title='x',yaxis_title='y'),\n",
    "    width=800,\n",
    "    height=600,\n",
    "    margin=dict(r=20, b=10, l=10, t=40)\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1418e88-acd5-460a-bf2b-4e6efc88e3dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's try 3D!\n",
    "\n",
    "tsne = TSNE(n_components=3, random_state=42)\n",
    "reduced_vectors = tsne.fit_transform(vectors)\n",
    "\n",
    "# Create the 3D scatter plot\n",
    "fig = go.Figure(data=[go.Scatter3d(\n",
    "    x=reduced_vectors[:, 0],\n",
    "    y=reduced_vectors[:, 1],\n",
    "    z=reduced_vectors[:, 2],\n",
    "    mode='markers',\n",
    "    marker=dict(size=5, color=colors, opacity=0.8),\n",
    "    text=[f\"Type: {t}<br>Text: {d[:100]}...\" for t, d in zip(doc_types, documents)],\n",
    "    hoverinfo='text'\n",
    ")])\n",
    "\n",
    "fig.update_layout(\n",
    "    title='3D Chroma Vector Store Visualization',\n",
    "    scene=dict(xaxis_title='x', yaxis_title='y', zaxis_title='z'),\n",
    "    width=900,\n",
    "    height=700,\n",
    "    margin=dict(r=20, b=10, l=10, t=40)\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b3ada26-b4b7-42fc-b943-933c14adf89b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}