Skip to content

feat: add /llms.txt for AI crawlers - #59

Open
HarshRajSinghania wants to merge 2 commits into
Ymax27:mainfrom
HarshRajSinghania:feat/add-llms-txt
Open

HarshRajSinghania wants to merge 2 commits into
Ymax27:mainfrom
HarshRajSinghania:feat/add-llms-txt

Conversation

@HarshRajSinghania

Copy link
Copy Markdown

Summary
Adds a /llms.txt endpoint that serves a short, factual machine-readable description of OpenToAll for AI systems.

Motivation
Addresses #50: publish /llms.txt explaining what OpenToAll is, key pages, and preferred crawl paths.

Implementation

  • New view llms_txt in core/views.py returning plain text.
  • URL pattern added in core/urls.py.

Testing

  • Manual review of the generated content for accuracy and brevity.
  • Change is isolated; no existing functionality altered.

Closes #50.

Addresses Ymax27#50 by serving a short, factual machine-readable description of OpenToAll at /llms.txt.

Includes key pages and preferred crawl paths.

@Ymax27 Ymax27 left a comment •

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up, and welcome to OpenToAll! 🎉 The content is accurate and nicely concise.

Two small things before we merge:

  1. Charset: the text contains non-ASCII characters (the "—"), so please use content_type="text/plain; charset=utf-8" to avoid garbled output in some browsers.
  2. Test: could you add a small test in core/tests.py checking that /llms.txt returns 200 with a text/plain content type?

I've approved the CI run (it's green ✅). For next time, a quick comment on the issue before starting helps us avoid duplicate work. Thanks!

— Max

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an llms.txt file for AI crawlers

2 participants