Refactor/tools func stability #10351
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- fetch_and_deploy_gh_release: add validation for empty app names - Derives app name from repo if not provided - Prevents '/root/.: Is a directory' error (fixes #10342) - setup_hwaccel: fix Intel driver app names for fetch_and_deploy_gh_release - Add proper app names: intel-igc-core, intel-igc-opencl, libigdgmm12, intel-opencl-icd - setup_mariadb: use distro packages by default - Default: apt packages (default-mysql-server, mariadb-server) - Optional: USE_MARIADB_REPO=true for official MariaDB repo - Fixes GPG key/mirror availability issues - setup_mysql: use distro packages by default - Default: apt packages (default-mysql-server, mysql-server) - Optional: USE_MYSQL_REPO=true for official MySQL repo - Keeps Debian Trixie 8.4 LTS handling when using official repo - setup_postgresql: use distro packages by default - Default: apt packages (postgresql, postgresql-client) - Optional: USE_PGDG_REPO=true for official PGDG repo - setup_docker: use distro packages by default - Default: docker.io package - Optional: USE_DOCKER_REPO=true for official Docker repo - Maintains Portainer support in both modes This refactoring prioritizes stability by using well-tested distro packages while maintaining the option to use official repos for specific version requirements.
New helper functions: - curl_with_retry: Robust curl wrapper with retry logic (3 attempts) - curl_api_with_retry: API calls with HTTP status handling - download_gpg_key: GPG key download with retry and dearmor support - debug_log: Conditional debug output when TOOLS_DEBUG=true Replaced critical curl calls: - MongoDB GPG key download - NodeSource GPG key download - PostgreSQL GPG key download - PHP (Sury) keyring download - MySQL GPG key download - setup_deb822_repo GPG import Benefits: - Automatic retry on transient network failures - Configurable timeouts (CURL_TIMEOUT, CURL_CONNECT_TO) - Debug mode for troubleshooting (TOOLS_DEBUG=true) - Consistent error handling across all GPG key imports
Added curl_with_retry to all critical download operations: - Adminer download - Composer installer - FFmpeg (binary and source) - Go tarball - Ghostscript source - ImageMagick source - rbenv and ruby-build - uv (astral-sh) - yq binary - Go version check Extended timeouts for large downloads: - CURL_TIMEOUT=300 for FFmpeg, Go (large tarballs) - CURL_TIMEOUT=180 for Ghostscript, ImageMagick Remaining without retry (intentional): - download_with_progress (specialized function) - Rustup installer (piped to shell) - Portainer version check (non-critical) Total curl_with_retry/download_gpg_key usage: 27 locations
- Add libmfx-gen1.2 back for Intel Quick Sync Video encoding (Debian 12+13) - Restore tmpfiles.d configuration for MariaDB /run/mysqld persistence - Fix MariaDB fallback version from 11.4 to 12.2 (latest GA version) These changes were incorrectly removed in the refactor commits.
… PG backup, Intel VPL
… pre-check - curl_with_retry: DNS pre-check + exponential backoff - download_gpg_key: Auto-detect key format, validation - ensure_dependencies: Batch dpkg-query check, individual fallback - install_packages_with_retry: Progressive recovery (dpkg fix, broken deps, individual packages) - verify_repo_available: Caching with TTL to avoid repeated HTTP requests - get_fallback_suite: Dynamic HTTP availability check cascade - ensure_apt_working: APT lock handling, progressive recovery - safe_service_restart: Wait-for-ready with configurable timeout, retry logic - get_latest_github_release: Fallback to tags API, prerelease support, rate limit handling
- get_parallel_jobs: Add memory-based limiting (1.5GB/job), load awareness, and container detection for conservative limits - get_default_php_version: Add future versions (Debian 14, Ubuntu 26.04), update defaults to 8.3 - get_default_python_version: Add future versions, update defaults to 3.12
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✍️ Description
tbd
🔗 Related Issue
Fixes #
✅ Prerequisites (X in brackets)
🛠️ Type of Change (X in brackets)
README,AppName.md,CONTRIBUTING.md, or other docs.