[feat] add dead link checker to CI/CD

[Anonymitaet]

Issue

Sometimes 404 errors occur in docs.

Currently, we use the following ways to check them:

Method 1: use tools to check them manually at regular intervals

• Dr. Link Checker

• W3C Link Checker

• Dead Link Checker

Disadvantages

• It's easy to forget to do this task.

• It entirely depends on the contributor's willingness.

Method 2: check the links through the local preview process

It reminds you of 404 errors if any.

Disadvantages

• Some contributors need to preview docs on the local machines and the link check step is skipped.

• Even though contributors preview docs and get reminders of 404 issues, they can ignore this error and still submit doc PRs.

Solution

Can we implement a dead link check to the CI/CD process in pulsar-site?

Example

Skywalking

Benefits

• Docs submitted from all contributors can be checked in a prominent and easy way

• If we want to ensure no 404 links in docs, we can set this check as required. In this way, PRs can not be merged with 404 issues.

• (Other doc-related checks, e.g., grammar checks, typo checks, etc can also be implemented)

cc @momo-jun @DaveDuggins @D-2-Ed

[tisonkun]

FYI: I'm going to enable Docusaurus builtin broken links checker in the site repo in this month.

• https://docusaurus.io/docs/api/docusaurus-config#onBrokenLinks
• https://docusaurus.io/docs/api/docusaurus-config#onBrokenMarkdownLinks

I'll enable the warning level first to get a report and see how to fix.

Other integrations proposed here can be good to have a try. But I think we should enable them on the site repo, right?

[Anonymitaet]

I think we should enable them on the site repo, right?

Yes

[momo-jun]

The Docu built-in checker sounds better. Does the detection happen during the local preview?
If yes, it would be more proactive than a checker in the CI/CD process. But on the other hand, not all contributors run the local preview. It's not a mandatory step for minor fixes. The checker integrated in the CI/CD process can be a gate for this.

[tisonkun]

Does the detection happen during the local preview

Yes. It will.