Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure entire docs #3447

Merged
merged 91 commits into from
Mar 24, 2025
Merged

Restructure entire docs #3447

merged 91 commits into from
Mar 24, 2025

Conversation

htahir1
Copy link
Contributor

@htahir1 htahir1 commented Mar 20, 2025

Describe changes

This PR introduces a new structure for our docs. We have now split the docs into multiple GitBook sites that are all linked together in one big collection. This new "monorepo" structure with 5 different syncs of the same branch for different sites for gitbook, is a bit more complicated. The biggest drawback is that linking between sections now involves using absolute links (starting with docs.zenml.io), and more likely to break. Therefore we have added a GitHub action that checks broken links periodically.

I am making this PR ready for review for now to test. You do not need to review it until I make a comment

Pre-requisites

Please ensure you have done the following:

  • I have read the CONTRIBUTING.md document.
  • I have added tests to cover my changes.
  • I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
  • IMPORTANT: I made sure that my changes are reflected properly in the following resources:
    • ZenML Docs
    • Dashboard: Needs to be communicated to the frontend team.
    • Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
    • Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Other (add details above)

Sorry, something went wrong.

htahir1 and others added 30 commits March 18, 2025 10:39
* format script

* Enhance link replacement functionality by adding optional substring filter in replace_links_in_file. This allows users to specify a substring to limit which links are replaced, improving control over the link transformation process.

* Fixed links to how-to pages from stacks guides

* Fix links to reference pages from stacks guides

* Update GCP CLI configuration link in container registries documentation

* Enhance link checking functionality in check_links_with_substring to differentiate between internal documentation paths and other links. Update docstring for clarity and improve link replacement logic in replace_links_in_file by removing the substring parameter, ensuring only relevant links are processed. This improves the accuracy of link validation and replacement in markdown files.

* Fix buggy link checker

* Fix links to getting-started pages from stacks guides

* Document the link_checker script

* Fix links to starter-guide pages from stack guides

* add url mapping + documentation

* Fix links to production-guide pages from stack guides

* Fix links to tutorials from getting-started section

* Fix links to tutorials from how-to section

* Fix links to tutorials from referenc section

* Update default source directories in format.sh to include link_checker.py

* Refine link filtering logic in link_checker.py to ensure only relevant internal documentation paths are included. Update comments for clarity.

* Fix links to stacks guides from getting-started section

* Fix links to stacks guides from how-to section

* Fix links to stacks guides from reference section

* Fix links to stacks guides from 'framework' meta-section

* Enhance link replacement logic in link_checker.py to store line-specific replacements and apply them in a single pass. This change improves the accuracy of link transformations while maintaining the original line structure during modifications.

* Fix multi-links

* Fix links to framework section from tutorials

* Fix links to stacks guides from tutorials

* Fix broken links

* Fix duplicated path in URL
Copy link
Contributor

Classification template updates in examples/mlops_starter have been pushed.

Copy link
Contributor

Images automagically compressed by Calibre's image-actions

Compression reduced images by 18.7%, saving 175.02 KB.

Filename Before After Improvement Visual comparison
docs/book/.gitbook/assets/org_hierarchy_pro.png 182.69 KB 167.72 KB -8.2% View diff
docs/book/api-docs/.gitbook/assets/oss-api.png 364.19 KB 323.88 KB -11.1% View diff
docs/book/api-docs/.gitbook/assets/pro-api.png 387.07 KB 267.34 KB -30.9% View diff

475 images did not require optimisation.

Update required: Update image-actions configuration to the latest version before 1/1/21. See README for instructions.

Copy link
Contributor

Images automagically compressed by Calibre's image-actions

Compression reduced images by 33.7%, saving 131.54 KB.

Filename Before After Improvement Visual comparison
docs/book/getting-started/zenml-pro/.gitbook/assets/pro-projects.png 389.90 KB 258.36 KB -33.7% View diff

478 images did not require optimisation.

Update required: Update image-actions configuration to the latest version before 1/1/21. See README for instructions.

Copy link
Contributor

Images automagically compressed by Calibre's image-actions

Compression reduced images by 31.5%, saving 70.94 KB.

Filename Before After Improvement Visual comparison
docs/book/getting-started/zenml-pro/org_hierarchy_pro.png 224.89 KB 153.95 KB -31.5% View diff

478 images did not require optimisation.

Update required: Update image-actions configuration to the latest version before 1/1/21. See README for instructions.

Copy link
Contributor

Images automagically compressed by Calibre's image-actions

Compression reduced images by 65.3%, saving 159.05 KB.

Filename Before After Improvement Visual comparison
docs/book/user-guide/.gitbook/assets/eurorate.jpg 14.12 KB 10.41 KB -26.3% View diff
docs/book/user-guide/.gitbook/assets/gamesense.jpg 32.05 KB 9.06 KB -71.7% View diff
docs/book/user-guide/.gitbook/assets/magicphoto.jpg 31.47 KB 9.80 KB -68.9% View diff
docs/book/user-guide/.gitbook/assets/nightwatch.jpg 40.18 KB 12.18 KB -69.7% View diff
docs/book/user-guide/.gitbook/assets/researchradar.jpg 38.62 KB 11.12 KB -71.2% View diff
docs/book/user-guide/.gitbook/assets/support.jpg 35.88 KB 10.58 KB -70.5% View diff
docs/book/user-guide/.gitbook/assets/yolo.jpg 37.13 KB 10.55 KB -71.6% View diff
docs/book/user-guide/.gitbook/assets/zencoder.jpg 14.16 KB 10.86 KB -23.3% View diff

479 images did not require optimisation.

Update required: Update image-actions configuration to the latest version before 1/1/21. See README for instructions.

@htahir1 htahir1 merged commit 2eece44 into develop Mar 24, 2025
9 checks passed
@htahir1 htahir1 deleted the doc/restructureentiredocs branch March 24, 2025 17:09
htahir1 added a commit that referenced this pull request Mar 24, 2025
* Add legacy docs section to table of contents

* Add redirects and new table of contents for component guide

* Add user guide for ZenML with starter and production guides. (50 chars)

* Update link to artifact store in cache documentation

* GITBOOK-1: No subject

* Revert "GITBOOK-1: No subject"

This reverts commit 81ba95a.

* Update caching behavior to include artifact store link.<commit message>

* Update caching behavior and artifact store link

* Update artifact store link in cache-previous-executions.md

* Update caching behavior in ZenML user guide

* Update artifact store link in cache documentation

* Update cache-previous-executions.md with corrected link

* GITBOOK-2: No subject

* GITBOOK-1: No subject

* Remove SDK & CLI reference link from TOC

* GITBOOK-1: No subject

* GITBOOK-2: No subject

* Add link checker script for markdown files

* Update links in markdown files to absolute URLs

* Fixes for docs links (#3425)

* format script

* Enhance link replacement functionality by adding optional substring filter in replace_links_in_file. This allows users to specify a substring to limit which links are replaced, improving control over the link transformation process.

* Fixed links to how-to pages from stacks guides

* Fix links to reference pages from stacks guides

* Update GCP CLI configuration link in container registries documentation

* Enhance link checking functionality in check_links_with_substring to differentiate between internal documentation paths and other links. Update docstring for clarity and improve link replacement logic in replace_links_in_file by removing the substring parameter, ensuring only relevant links are processed. This improves the accuracy of link validation and replacement in markdown files.

* Fix buggy link checker

* Fix links to getting-started pages from stacks guides

* Document the link_checker script

* Fix links to starter-guide pages from stack guides

* add url mapping + documentation

* Fix links to production-guide pages from stack guides

* Fix links to tutorials from getting-started section

* Fix links to tutorials from how-to section

* Fix links to tutorials from referenc section

* Update default source directories in format.sh to include link_checker.py

* Refine link filtering logic in link_checker.py to ensure only relevant internal documentation paths are included. Update comments for clarity.

* Fix links to stacks guides from getting-started section

* Fix links to stacks guides from how-to section

* Fix links to stacks guides from reference section

* Fix links to stacks guides from 'framework' meta-section

* Enhance link replacement logic in link_checker.py to store line-specific replacements and apply them in a single pass. This change improves the accuracy of link transformations while maintaining the original line structure during modifications.

* Fix multi-links

* Fix links to framework section from tutorials

* Fix links to stacks guides from tutorials

* Fix broken links

* Fix duplicated path in URL

* Added API docs

* Added API docs

* Add Pro API section to Table of Contents

* Update API endpoints in OSS API documentation

* GITBOOK-5: No subject

* Add redirects for project setup and management paths

* Update SDK Docs URLs to include ".html" extension.- Update SDK Docs URLs to include ".html" extension

* Add ZenML Pro API documentation and authentication instructions

* Update Pro API documentation link in API reference

* Update ZenML Client documentation links

* Fix more links (#3436)

* Enhance link validity checks in link_checker.py

- Added an additional check for Gitbook URLs to identify pages that return a 200 status code but contain a "noindex" meta tag, indicating a non-existent page.
- Updated the transform_relative_link function to skip links to assets and Gitbook resources.
- Modified the replace_links_in_file function to also skip links to assets and Gitbook resources, ensuring cleaner link processing.

* Fix internal links inside component-guide

* Fix internal links inside getting-started section

* Refactor link replacement logic in link_checker.py

- Introduced a mechanism to track valid replacements, ensuring only valid links are applied during the replacement process.
- Enhanced the handling of inline and reference-style links by collecting potential replacements before applying them.
- Updated the logic to apply replacements conditionally based on validation results, improving the accuracy of link processing.
- Removed unnecessary dry run checks and streamlined the writing of modified content back to the file.

* Fix internal links in how-to section

* Fixing edge case links inside how-to section

---------

Co-authored-by: Hamza Tahir <[email protected]>

* Add redirects to ZenML Pro and project setup pages

* Add link checking workflow and script (and fix some broken links) (#3438)

* Add link checking workflow and script

- Introduced a GitHub Actions workflow (`check-links.yml`) to automate the checking of absolute and relative links in markdown files upon push and pull request events.
- Added a new script (`check_relative_links.py`) to verify that relative links in markdown files point to existing files in the repository, enhancing documentation integrity.
- The script scans markdown files, extracts relative links, and checks their validity without making HTTP requests, providing detailed output for any broken links found.

This addition aims to improve the reliability of documentation links and streamline the review process for changes in markdown files.

* Fixing static pro links

* Fix more pro links

* Update internal links in Kubernetes orchestrator documentation

- Changed links for the remote artifact store and remote container registry sections to point to the correct README files, ensuring accurate navigation within the documentation.

This update aims to enhance the clarity and usability of the Kubernetes orchestrator guide.

* Update documentation for CloudpickleMaterializer warning

- Clarified the warning regarding the use of the built-in CloudpickleMaterializer, emphasising the potential security risks associated with uploading arbitrary objects.
- Improved the link to the custom Materializer documentation for better guidance on creating robust and efficient formats.

This update aims to enhance user awareness of security implications and provide clearer instructions for safer practices in ZenML step output handling.

* Update internal links in API documentation

- Modified links in the OSS and Pro API sections to remove unnecessary README.md suffixes, ensuring cleaner and more direct navigation to the documentation.

This update aims to enhance the clarity and usability of the API documentation.

* ignore mailto links

* Don't check local urls

* Fix SDK docs links

* Fix SDK docs links

* Update CI checker script

* Enhance GitHub Actions workflow for link checking

- Updated the check-links.yml workflow to ignore changes in the 'src' and 'tests' directories, as well as Python files, during push and pull request events on the main branch. This aims to streamline the link checking process by focusing on relevant changes only.

This update improves the efficiency of the CI workflow by reducing unnecessary checks.

* Handle backslashes in docs better

* Fix links broken in projects renaming

* Fix broken links

* Link checker in CI should actually fail now

* Update documentation links for clarity and accuracy

- Changed the reference from the "starter guide" to the "production guide" in the migration guide.
- Updated the service connector link to point to the correct section on infrastructure deployment in the remote storage documentation.
- Minor formatting adjustment in the workspaces documentation for consistency.

These changes ensure that users have access to the most relevant and accurate resources.

* Refine paths-ignore syntax in CI workflow

- Updated the paths-ignore entries in the check-links.yml workflow to use double quotes for consistency.
- Ensured that the Python version is specified with double quotes as well.

These changes enhance the readability and maintainability of the workflow configuration.

* Fix the backslash problem (#3439)

* Refactor documentation for clarity

- Removed unnecessary line breaks in the PyTorch integration documentation, enhancing readability and ensuring a smoother flow of information.
- This change aims to improve the overall user experience when navigating the component guide.

* Fix some backslashes

* Fix remaining backslashes

* Fix line break

* GITBOOK-1: Updating Pro pages with new structure, images, and steppers

* GITBOOK-5: Adding the introductory text back to the main page

* GITBOOK-6: No subject

* GITBOOK-2: No subject

* GITBOOK-2: No subject

* Add Deployments section to table of contents

* Optimised images with calibre/image-actions

* Update ZenML Pro feature sign-up links in templates

* Update ZenML Pro hierarchy documentation

* GITBOOK-3: No subject

* Optimised images with calibre/image-actions

* GITBOOK-2: No subject

* Update ZenML Pro links in deployment and access management guides

* Update AWS and Databricks links in documentation

* Update SDK and Deepchecks links in documentation

* Update links for Linux and OpenAI API key in docs

* Reflect Deepchecks integration with ZenML and SDK updates

* GITBOOK-6: No subject

* GITBOOK-7: No subject

* Update API documentation to improve clarity and consistency

* Update API reference links in READMEs

* Fix GH Action for link checking (#3448)

* Enhance GitHub Actions workflow by initializing FAILED variable for better error handling in link checks

* Refine GitHub Actions workflow to ensure link checks report errors correctly by adding failure handling for absolute and relative link checks

* try again

* and another

* Add comments to PR

* bump versions

* Refine GitHub Actions workflow to improve link check error handling by preventing immediate exit on failure and adding a reporting step after posting comments.

* Update GitHub Actions workflow to standardise quotes and enhance summary comment generation for link checks

* Fix date display on PR output

* GITBOOK-3: No subject

* GITBOOK-4: No subject

* Optimised images with calibre/image-actions

* Update API token and API documentation links

* Optimised images with calibre/image-actions

* GITBOOK-3: No subject

* Auto-update of Starter template

* Update org_hierarchy_pro.png asset image

* Optimised images with calibre/image-actions

* Add projects to ZenML Pro workspaces

* Add instructions for using projects and workspaces with CLI

* Update project login instructions in CLI documentation

* Fix typo in hierarchy file name- Rename 'heirarchy.md' to 'hierarchy.md' in file paths

* Optimised images with calibre/image-actions

* Remove deprecated methods from ZenML Client documentation

* Update title to capitalize in hierarchy.md

* Update ZenML Pro features and deployment options

* Update heading to title case in TOC.md file

* Update terminology from "tenant" to "workspace" in ZenML Pro

* Refactor roles and workspaces documentation formatting

* Add note about creating custom organization roles via API

* Add organization hierarchy diagram for ZenML Pro

* Optimised images with calibre/image-actions

* Update asset path and note in hierarchy.md

* GITBOOK-5: No subject

* Optimised images with calibre/image-actions

---------

Co-authored-by: Alex Strick van Linschoten <[email protected]>
Co-authored-by: Alex Strick van Linschoten <[email protected]>
Co-authored-by: Zuri Negrín <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <[email protected]>
(cherry picked from commit 2eece44)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation internal To filter out internal PRs and issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants