Local link verification in source code

[cbachhuber]

I find markdown link checks highly useful, even if they only check relative links in the repo. Examples:

• https://docs.rs/crate/rumdl/latest/source/docs/md052.md
• https://github.com/lycheeverse/lychee
• https://github.com/tcort/markdown-link-check

I still often encounter dead relative links from source code, though. For example, people often reference other files from source code comments.

Do you think it would make sense to offer a pre-commit hook that verifies relative links in source code? It could expect some pattern (possibly r"<[^\n]>") to define a relative, checked link, and then verify existence in the repo.

[hofbi]

Rumdl works pretty robust. Could we use rumdl for the actual link verification so that we only have to focus on passing the content to rumdl via stdin for instance?

But given that there is probably way more source code than markdown we have to be sure that this runs fast enough.

[cbachhuber]

Good points. Having this highly performant speaks for keeping it within one language (possibly the current craze rust). Or would you call that premature optimization?

[hofbi]

I think we only know python native vs rumdl calls if we do a benchmark. The reason for proposing rumdl was mainly to avoid reimplementing the check. But checking for existence is of course not a big deal. So feel free to make a proposal.

[cbachhuber]

When reusing rumdl, we'd require the markdown [text](link) pattern. However, that pattern is used for different purposes in many other languages. So I don't think it'll work.

I considered the following designs:

1. Have an explicit, language-agnostic rarely used markup for links to check, as done in feat: add check-repo-links hook to verify @repo relative links #163.
   1. Good: no false positives, works in any plain-text file
   2. Bad: opt-in, so usage will often times simply be forgotten
2. Language-aware parsing: identify comments using https://tree-sitter.github.io/tree-sitter/ or with a per-language regex and look for local link patterns in them. Something like this or even with an explicit extension whitelist to reduce false positives.
   1. Good: no accidental forgetting of the special markup
   2. Bad: false positives, need to maintain language-support
3. Inverted approach: Detect (re-)moved files and grep the repo for references to (parts of) the deleted path
   1. Good: no false positives, grepping should be quick
   2. Bad: doesn't catch references that were always wrong because of typos, for example

I'm not calling out my favorite to not bias you. What are your thoughts about these?

[hofbi]

To me option 3 sounds like a good approach. No dependencies and no opt-in logic. The downside is acceptable IMO as its better that it was before.