Skip to content

Infrastructure: Update link-checker User-Agent string #3264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 20, 2025

Conversation

howard-e
Copy link
Contributor

@howard-e howard-e commented May 5, 2025

Fix #3263


WAI Preview Link (Last built on Tue, 20 May 2025 13:59:09 GMT).

@howard-e howard-e changed the title Update link-checker User-Agent string Update link-checker User-Agent string May 5, 2025
@howard-e howard-e added the Infrastructure Related to maintaining task force and repo operations, processes, systems, documentation label May 5, 2025
@howard-e howard-e changed the title Update link-checker User-Agent string Infrastructure: Update link-checker User-Agent string May 5, 2025
Copy link

@gnarf gnarf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems good enough, and will fix the bug in the short term, but will probably break again in a few months.

I did a little research and ran into https://www.npmjs.com/package/user-agents - a package that someone has automated to collect the most popular user agents used on their own websites and it can generate "always up to date" user agents. We may want to look into using this package instead.

After a bit more research and a comment from @howard-e - https://github.com/DavideViolante/useragents-me-api/ looks like a much better choice

@mcking65
Copy link
Contributor

@howard-e

Given this is infra, I was waiting on you to merge.

@howard-e
Copy link
Contributor Author

howard-e commented May 20, 2025

This seems good enough, and will fix the bug in the short term, but will probably break again in a few months.

Certainly true.

I did a little research and ran into https://www.npmjs.com/package/user-agents - a package that someone has automated to collect the most popular user agents used on their own websites and it can generate "always up to date" user agents. We may want to look into using this package instead.

After a bit more research and a comment from @howard-e - https://github.com/DavideViolante/useragents-me-api/ looks like a much better choice

Not so much a fan for adding an extra lib for this small use case. Agreed on making this more future-proof, with the hardcoded string as a fallback. Fetching from the source site of that lib (specifically https://www.useragents.me/#most-common-desktop-useragents) is a good enough option to do in the future, to move this PR along.

Edit: tracking this in #3270

@howard-e howard-e merged commit 7b7d576 into main May 20, 2025
7 checks passed
@howard-e howard-e deleted the link-checker-update-user-agent-string branch May 20, 2025 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Infrastructure Related to maintaining task force and repo operations, processes, systems, documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

link-checker unexpectedly failing on paciellogroup.com
3 participants