Journal Finder

Recommend the top-5 most relevant computer-science journals for a given abstract, and produce interpretable topic clusters over the article corpus.

---

Quickstart

Requirements: Python 3.9+, the file CompSciencePub.sqlite placed one folder above this one (i.e. Data Mining Assignment/CompSciencePub.sqlite).

Step 1 — Install dependencies

pip install scikit-learn pandas numpy scipy matplotlib jupyter

Step 2 — Open the notebook

cd journal-finder
jupyter notebook journal_finder.ipynb

Step 3 — Run all cells (in order)

In Jupyter: Kernel → Restart & Run All

The notebook will:

• Load and clean 22 000+ abstracts from the database (~10 s)
• Fit TF-IDF and train the LDA topic model (~3–5 min, one-time)
• Print the 25 discovered topic clusters with their top words
• Evaluate top-1 / top-5 / top-10 accuracy on a held-out sample
• Show a dendrogram of journals grouped by topic profile

Step 4 — Try your own abstract

Scroll to Section 8 ("Use the model interactively"), paste any abstract into the abstract = ''' ... ''' string, and run that cell. A ranked table of the top-5 journals will appear directly below the cell.

Where is the output? Every cell displays its result immediately below it after it runs. The journal recommendation table appears as a styled pandas DataFrame in Section 5 (demo) and Section 8 (interactive). If you see no output, make sure you ran the cells in order from the top — the model must be fitted before the ranking cells can run.

---

Data

CompSciencePub.sqlite (provided with the assignment). 23 801 articles across 466 publications; 23 061 of them carry abstract text.

Method

• Text representation: TF-IDF over title + abstract + author keywords + Web-of-Science KeywordPlus. Title and keywords are repeated so they weigh more heavily in the journal signal.
• Topic clustering (the non-k-means method): Latent Dirichlet Allocation, 25 topics. LDA is a generative probabilistic model that assigns soft topic memberships, which is what the assignment's "cluster of topics for subject areas" actually wants.
• Journal grouping: Ward agglomerative clustering on the per-journal topic fingerprint. Used for the dendrogram visualisation only.
• Ranking: convex combination of TF-IDF cosine similarity and LDA-topic cosine similarity against per-journal mean vectors.

Files

File	Purpose
journal_finder.py	Library: corpus loader, vectoriser, LDA, finder.
journal_finder.ipynb	Walk-through notebook with evaluation + plots.
README.md	This file.