Fix: Timeline shows some irrelevant news articles

[DanielSongShen]

The timeline shows many irrelevant news articles. Let's brainstorm ideas to fix this!

Proposals

• Smart topic filter using an LLM to decide if an article is relevant or not
• Limit news sources to a few dozen reputable sources
• Look at meta information like popularity of an article

Please feel free to add-on. This is a hard problem so we should work together on it.

[JulienDarve]

I did some research into GDELT, we could use their database for getting article metadata. Their search features are significantly more robust from what I can see.

They also provide some NLP analysis of the articles we can query. They also have no API limit.

[DanielSongShen]

We may have to use Google BigQuery since the database is very large. Luckily they have a free tier that looks good enough.

• https://cloud.google.com/bigquery/pricing#free-usage-tier
• https://cloudplatform.googleblog.com/2014/05/worlds-largest-event-dataset-now-publicly-available-in-google-bigquery.html