merge

Lab7Cook.ipynb

@@ -2672,6 +2672,16 @@
     "plot_history(history1, title='Model with Glove embeddings')"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "da7249c3",
+   "metadata": {},
+   "source": [
+    "As can be seen below, the Glove Model does an overall excellent job classifying the tweets correctly, with 2204 correct positives and 4475 correct negatives.  However, this benchmark still falls below the Transformer model with 2 multi-heads, as the Glove model has nearly twice as many false positives and false negatives (323 and 464 compared to 247 and 247 respectively).  As can be seen in the graphs, the model converges around 13 epochs, as the validation accuracy and validation loss appear to plateau.  However, the training accuracy and training loss continue to increase marginally to peak at an accuracy of about 0.98 based on the graph.\n",
+    "\n",
+    "As can be seen in the classification report below, the precision for positive tweets is slightly less than that of negative tweets, with a score of 0.87 compared to 0.91.  This is consistent with the rate of false positives shown in the confusion matrix, along with the fact that there are more negative tweets than positive tweets.  The relatively high scores of accuracy, macro average, and weighted average further contribute to the fact that this is a good classification model.  However, it still falls short of the marks achieved by the Transformer model.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 43,
@@ -2905,6 +2915,16 @@
     "plot_history(history2, title='Model with ConceptNet Numberbatch embeddings')"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "93a9e95c",
+   "metadata": {},
+   "source": [
+    "Similarly to the Glove Model, the ConceptNet Model performs very well, albeit still below the mark of the Transformer model previously discussed.  2258 tweets were correctly identified as positive and 4400 as negative, with the positive rate slightly higher than Glove, but the negative rate slightly lower.  Again, the model falls short of the 247 false positives and negatives from the Transformer model, and has 398 false positives and 410 false negatives.  The false positive rate is slightly higher than that of the Glove model, a sign that the Glove model may be the better model.  The graphs demonstrate a convergence around 8 epochs for the validation accuracy and validation loss, but the training accuracy and training loss continue to increase as the epochs increase.  These values never plateau to the degree that the Glove model does.\n",
+    "\n",
+    "As can be seen in the classification report below, the ConceptNet Model shows very similar results to the Glove model, with only marginal differences in the precision, recall, and F1 scores of all of the values.  This makes sense, given that the confusion matrix was extremely similar to that of the Glove model.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 47,
@@ -2931,6 +2951,14 @@
     "print(classification_report(test_df['labels'], y_pred_binary, target_names=['Positive', 'Negative']))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "883a5cd3",
+   "metadata": {},
+   "source": [
+    "There isn't much difference between the pre-trained ConceptNet Numberbatch embedding and the pre-trained GloVe models for our specific application, as they performed extremely similarly.  I would argue that the pre-trained GloVe model is likely better for this purpose, though, as it tends to capture statistical information about word co-occurrences better, a useful tool for sentiment analysis.  With that being said, the inability of GloVe to take into account contextual semantic understandings of words makes it difficult for some classifications.  There doesn't appear to be much of a difference between the models for our purpose, though, and both models fall below the previously used models, particularly the Transformer model with 2 multi-heads."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "b6725e16",