{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Similarity Score\n",
    "This Notebook imports the questions and answers from the QuestionGeneration.ipynb output and scores the similarity\n",
    "\n",
    "Heavy inspiration taken from:\n",
    "\n",
    "https://github.com/karndeepsingh/sentence_similarity/blob/main/Finding_Similar_Sentence.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\evant\\anaconda3\\envs\\docu_compare\\Lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "import numpy as np\n",
    "from sentence_transformers import SentenceTransformer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the model\n",
    "Uncomment to store the model locally for easy retrieval, but delete the model before uploading to GitHub as too large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = SentenceTransformer('nli-distilroberta-base-v2')\n",
    "# model.save(\"./Model/model\")\n",
    "# model = SentenceTransformer(\"./Model/model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load in the questions and responses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "Data = pd.read_csv(\"./Results/Compare.csv\", index_col=0)\n",
    "questions = Data.index.values\n",
    "company = Data['Company'].values\n",
    "gold = Data['Gold'].values\n",
    "sentences = np.concatenate((company, gold))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the sentence embeddings and obtain scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "sentence_embeddings = model.encode(sentences)\n",
    "similarity_score = []\n",
    "for i in range(len(company)):\n",
    "    similarity_score.append(cosine_similarity(\n",
    "        [sentence_embeddings[i]],\n",
    "        [sentence_embeddings[len(company) + i]]\n",
    "    ).flatten()[0])\n",
    "Similarity = pd.DataFrame({'Score': similarity_score}, index=questions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Print a few of the scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                       Score\n",
      "Legal and regulatory requirements involving AI ...  0.820508\n",
      "The characteristics of trustworthy AI are integ...  0.847748\n",
      "Processes, procedures, and practices are in pla...  0.793549\n",
      "The risk management process and its outcomes ar...  0.837404\n",
      "Ongoing monitoring and periodic review of the r...  0.810052\n"
     ]
    }
   ],
   "source": [
    "print(Similarity.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the scores to a .csv file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "Similarity.to_csv('./Results/Similarity_Scores.csv')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "docu_compare",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}