LinkedIn Job Scraper & AI Matcher

Automated job scraper that fetches job listings from LinkedIn, then uses AI (Ollama) to match them with your profile. Perfect for those who are tired of manually browsing LinkedIn every day to find suitable jobs.

Features

• Scrape jobs from LinkedIn (no login required)
• AI-powered job matching using Ollama
• Save results to Appwrite database
• Filter jobs that match your profile
• Built with Bun (blazingly fast)

Tech Stack

• Runtime: Bun
• Language: TypeScript
• Database: Appwrite
• AI: Ollama (Local LLM)
• Scraper: Puppeteer

Prerequisites

Before you start, make sure you have installed:

1. Bun - JavaScript runtime
2. Ollama - Local LLM
3. Appwrite - Backend database (Cloud or self-hosted)

Setup

1. Clone & Install Dependencies

git clone github.com/dionarya23/job-scraper.git
cd job-scraper
bun install

2. Setup Ollama

# Install model (choose one)
ollama pull llama3       # Recommended
ollama pull llama3.2     # Lighter
ollama pull phi3         # Lightest

# Run Ollama server
ollama serve

3. Setup Appwrite

Option 1: Using Appwrite Cloud

• Sign up at cloud.appwrite.io
• Create a new project
• Create a new database
• Create a collection with the following attributes:

Attribute	Type	Size	Required
title	String	255	Yes
company	String	255	Yes
location	String	255	No
link	String	500	Yes
description	String	10000	No
seniority	String	100	No
employmentType	String	100	No
industry	String	255	No
postedDate	String	100	No
scrapedAt	String	100	No
isSuitable	Boolean	-	No

Important: Don't forget to create indexes for the link and isSuitable fields for faster queries!

4. Environment Variables

Create a .env file in the root folder:

# Appwrite Config
APPWRITE_ENDPOINT=https://cloud.appwrite.io/v1
APPWRITE_PROJECT_ID=your_project_id
APPWRITE_API_KEY=your_api_key
DATABASE_ID=your_database_id
COLLECTION_ID=your_collection_id

# Ollama Config
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL_NAME=llama3

5. Setup Profile

Edit the src/data/profile.ts file with your CV data. The more complete it is, the more accurate the matching will be.

Usage

1. Scraping Jobs

bun scrape.ts

This script will:

• Open LinkedIn Jobs
• Scrape job listings based on keyword & location (you can modify it in the code)
• Fetch details for each job
• Save to Appwrite
• Skip duplicates

Customize keyword & location:

Edit the last line in scrape.ts:

const jobs = await scrapeLinkedInJobs('software engineer', 'Jakarta');

2. Analyze with AI

bun analyze.ts

This script will:

• Fetch all scraped jobs
• Send them to Ollama for analysis
• Compare with your profile
• Update the isSuitable field (true/false)
• Process in batches to avoid overload

Note: This process might take a while depending on:

• Number of jobs
• Your computer specs
• The Ollama model you're using

3. View Results

bun index.ts

This script will display:

• List of all suitable jobs
• Details for each job (title, company, link, etc.)
• Summary stats (total jobs, match rate, etc.)

Project Structure

.
├── src/
│   ├── db/
│   │   └── appwrite.ts      # Appwrite client config
│   └── data/
│       └── profile.ts       # Your profile (CV)
├── scrape.ts                # Job scraper
├── analyze.ts               # AI matcher
├── index.ts                 # View results
├── .env                     # Environment variables
└── package.json

Tips & Troubleshooting

Scraping Issues

Problem: LinkedIn blocks or rate limits you

Solution:

• Add longer delays in scrape.ts
• Use a VPN or proxy
• Don't scrape too many jobs in one run

Ollama Slow/Timeout

Solution:

• Use a smaller model (llama3.2 or phi3)
• Reduce chunkSize in analyze.ts (from 10 to 5 or 1)
• Upgrade your computer's RAM

Appwrite Connection Error

Solution:

• Make sure the API key is valid and has documents.read & documents.write permissions
• Check the endpoint URL (should end with /v1)
• Verify that the database & collection IDs are correct

Customization

Change Scraping Target

Edit in scrape.ts:

// Change keyword or location
const jobs = await scrapeLinkedInJobs('frontend developer', 'Remote');

Modify Matching Criteria

Edit the prompt in the sendToLLM() function in analyze.ts to customize how the AI judges job matching.

Add Filters

Add Query in index.ts:

// Example: Filter jobs in Jakarta only
[
  Query.equal('isSuitable', true),
  Query.equal('location', 'Jakarta')
]