Skip to content

GH-48904: [C++][FlightRPC][CI][Packaging] Upload ODBC installer into GitHub release as RC #48934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kou merged 10 commits into from
Feb 10, 2026
Merged

GH-48904: [C++][FlightRPC][CI][Packaging] Upload ODBC installer into GitHub release as RC #48934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
65b9bd4
Upload ODBC installer into GitHub release as RC
alinaliBQ Jan 21, 2026
defaba1
Add check for install ODBC MSI and check ODBC dll
alinaliBQ Jan 28, 2026
7f2d797
Minor: add comment to 04-binary-download.sh
alinaliBQ Jan 29, 2026
6bb67c5
Add periods to ODBC msi artifact names
alinaliBQ Jan 29, 2026
74549fe
Change to PowerShell format
alinaliBQ Feb 3, 2026
51acfe6
Replace space with `-` in generated ODBC MSI file name
alinaliBQ Feb 4, 2026
b1cf045
Work on comments from Kou
alinaliBQ Feb 5, 2026
2fb7436
Use`startsWith` and `contain` for checking tags
alinaliBQ Feb 5, 2026
7bd8429
Nit - comment removal
alinaliBQ Feb 9, 2026
195ed13
Fix odbc-msvc
alinaliBQ Feb 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 69 additions & 1 deletion .github/workflows/cpp_extra.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -530,8 +530,43 @@ jobs:
uses: actions/upload-artifact@v6
with:
name: flight-sql-odbc-msi-installer
path: build/cpp/Apache Arrow Flight SQL ODBC-*-win64.msi
path: build/cpp/Apache-Arrow-Flight-SQL-ODBC-*-win64.msi
if-no-files-found: error
- name: Install ODBC MSI
run: |
cd build/cpp
$odbc_msi = Get-ChildItem -Filter "Apache-Arrow-Flight-SQL-ODBC-*-win64.msi"
if (-not $odbc_msi) {
Write-Error "ODBC MSI not found"
exit 1
}

foreach ($msi in $odbc_msi) {
Write-Host "Installing $($msi.Name) with logs"
$log = "odbc-install.log"
Start-Process msiexec.exe -Wait -ArgumentList "/i `"$msi`"", "/qn", "/L*V `"$log`""
Get-Content $log
}
- name: Check ODBC DLL installation
run: |
$dirs = Get-ChildItem "C:\Program Files" -Directory -Filter "Apache-Arrow-Flight-SQL-ODBC*"

foreach ($dir in $dirs) {
$bin = Join-Path $dir.FullName "bin"

if (Test-Path $bin) {
tree $bin /f

$dll = Join-Path $bin "arrow_flight_sql_odbc.dll"
if (Test-Path $dll) {
Write-Host "Found ODBC DLL: $dll"
exit 0
}
}
}

Write-Error "ODBC DLL not found"
exit 1

odbc-nightly:
needs: odbc-msvc
Expand Down Expand Up @@ -579,6 +614,39 @@ jobs:
remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}
remote_host_key: ${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}

odbc-release:
needs: odbc-msvc
name: ODBC release
runs-on: ubuntu-latest
if: ${{ startsWith(github.ref_name, 'apache-arrow-') && contains(github.ref_name, '-rc') }}
permissions:
# Upload to GitHub Release
contents: write
steps:
- name: Checkout Arrow
uses: actions/checkout@v6
with:
fetch-depth: 0
submodules: recursive
- name: Download the artifacts
uses: actions/download-artifact@v7
with:
name: flight-sql-odbc-msi-installer
- name: Wait for creating GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
dev/release/utils-watch-gh-workflow.sh \
${GITHUB_REF_NAME} \
release_candidate.yml
- name: Upload the artifacts to GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release upload ${GITHUB_REF_NAME} \
--clobber \
Apache-Arrow-Flight-SQL-ODBC-*-win64.msi

report-extra-cpp:
if: github.event_name == 'schedule' && always()
needs:
Expand Down
2 changes: 2 additions & 0 deletions cpp/src/arrow/flight/sql/odbc/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,8 @@ if(ARROW_FLIGHT_SQL_ODBC_INSTALLER)
set(CPACK_PACKAGE_VERSION_PATCH ${ODBC_PACKAGE_VERSION_PATCH})

set(CPACK_PACKAGE_NAME ${ODBC_PACKAGE_NAME})
# Make sure the MSI name contains only hyphens, not spaces
string(REPLACE " " "-" CPACK_PACKAGE_NAME "${CPACK_PACKAGE_NAME}")
set(CPACK_PACKAGE_VENDOR ${ODBC_PACKAGE_VENDOR})
set(CPACK_PACKAGE_DESCRIPTION_SUMMARY "Apache Arrow Flight SQL ODBC Driver")
set(CPACK_PACKAGE_CONTACT "dev@arrow.apache.org")
Expand Down
2 changes: 1 addition & 1 deletion cpp/src/arrow/flight/sql/odbc/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ should show as an available ODBC driver in the x64 ODBC Driver Manager.
3. `cd` to `build` folder.
4. Run `cpack`.

If the generation is successful, you will find `Apache Arrow Flight SQL ODBC-<version>-win64.msi` generated under the `build` folder.
If the generation is successful, you will find `Apache-Arrow-Flight-SQL-ODBC-<version>-win64.msi` generated under the `build` folder.


## Steps to Enable Logging
Expand Down
3 changes: 2 additions & 1 deletion dev/release/04-binary-download.sh
View file
Open in desktop
Copy link
Member

@kou kou Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you revert needless permission change?

Copy link
Collaborator Author

@alinaliBQ alinaliBQ Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, I reverted it. it was changed accidentally

Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,14 @@ tag="apache-arrow-${version_with_rc}"

archery crossbow download-artifacts --no-fetch ${CROSSBOW_JOB_ID} "$@"

# Download Linux packages.
# Download Linux packages and ODBC MSI.
gh release download "${tag}" \
--dir "packages/${CROSSBOW_JOB_ID}" \
--pattern "almalinux-*.tar.gz" \
--pattern "amazon-linux-*.tar.gz" \
--pattern "centos-*.tar.gz" \
--pattern "debian-*.tar.gz" \
--pattern "ubuntu-*.tar.gz" \
--pattern "Apache-Arrow-Flight-SQL-ODBC-*-win64.msi" \
--repo "${REPOSITORY:-apache/arrow}" \
--skip-existing
5 changes: 5 additions & 0 deletions dev/release/05-binary-upload.sh
View file
Open in desktop
Copy link
Member

@kou kou Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add signing command line something like the following here:

diff --git a/dev/release/05-binary-upload.sh b/dev/release/05-binary-upload.sh
index f628cce0e0..e66b5af646 100755
--- a/dev/release/05-binary-upload.sh
+++ b/dev/release/05-binary-upload.sh
@@ -98,6 +98,9 @@ upload_to_github_release() {
       shasum -a 512 "${base_name}" >"${base_name}.sha512"
       popd
     fi
+    if [[ "${base_name}" = *.msi ]]; then
+      jsign ... "${dist_dir}/${base_name}"
+    fi
   done
   gh release upload \
     --repo apache/arrow \

Copy link
Collaborator Author

@alinaliBQ alinaliBQ Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion @kou. My understanding is that this change will be done by a PMC member outside of this PR. Please correct me if that’s not the case

Copy link
Member

@kou kou Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

05-binary-upload.sh is executed by a PMC member but all Apache Arrow contributors can update 05-binary-upload.sh. Could you implement MSI signing in this PR because it's related to the official MSI.

Copy link
Member

@lidavidm lidavidm Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears there's documentation here: https://github.com/apache/infrastructure-website/blob/145ccc3f7dadf0b95418ebb31c86c349b482fb01/content/pages/code-signing-use.md?plain=1

That provides the jsign command, we should document what the release manager needs to sign up for/install.

Do we need to sign the DLL inside the MSI as well or just sign the MSI?

Copy link
Member

@amoeba amoeba Feb 6, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @kou, I looked at this the other week and found this page from ASF: https://infra.apache.org/code-signing-use.html. There's an automated signing option that I haven't tested but it looks like we might be able to obtain a secret, the "local file" they refer to here:

For automated code signing you create a local file that contains you ssl.com username and password along with the 2FA secret code (not the PIN) and code signing will not require any additional input when you sign a file.

and then we could do this in CI. Otherwise it would be done manually which isn't ideal. Have you done this before?

Edit: I re-read the page and the jsign method in Step 4 looks very doable in CI so long as a PMC member applies for an account to set up the secret.

Copy link
Collaborator Author

@alinaliBQ alinaliBQ Feb 6, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it is considered best practice to sign the DLL inside MSI: https://stackoverflow.com/a/4716079
That said, signing the DLL is optional as Windows doesn't check for signature in DLL after DLL is installed via the MSI. During install, Windows only checks for MSI signing.
Edit: Just saw Bryce's comment after posting, thanks for asking on Slack about this

Copy link
Collaborator Author

@alinaliBQ alinaliBQ Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have access to the Slack thread, so please let me know if there's any action item I should do regards to ODBC MSI signing, or if PMC members will be handling it. 🙂 cc @amoeba @kou @lidavidm

Copy link
Member

@amoeba amoeba Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @alinaliBQ, a PMC member can take the next steps.

@kou @lidavidm: Signing in CI is acceptable (per Slack) and it sounds like having a reproducible build for the installer is key. I assume the work done on setting up reproducible builds at the project level can be extended to the ODBC installer. Do we just need a volunteer at this point to figure the rest of this out?

Copy link
Member

@kou kou Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Let's work on reproducible build as a separated task.

Copy link
Member

@kou kou Feb 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we can't release the ODBC installer without signing. If we can't prepare reproducible build until the next release, we may need to prepare manual signing script like the diff in the #48934 (comment) comment.

Copy link
Member

@kou kou Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you revert needless permission change?

Copy link
Collaborator Author

@alinaliBQ alinaliBQ Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, I reverted it. it was changed accidentally

Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ cd "${SOURCE_DIR}"
: "${UPLOAD_CENTOS:=${UPLOAD_DEFAULT}}"
: "${UPLOAD_DEBIAN:=${UPLOAD_DEFAULT}}"
: "${UPLOAD_DOCS:=${UPLOAD_DEFAULT}}"
: "${UPLOAD_ODBC:=${UPLOAD_DEFAULT}}"
: "${UPLOAD_PYTHON:=${UPLOAD_DEFAULT}}"
: "${UPLOAD_R:=${UPLOAD_DEFAULT}}"
: "${UPLOAD_UBUNTU:=${UPLOAD_DEFAULT}}"
Expand Down Expand Up @@ -108,6 +109,10 @@ upload_to_github_release() {
if [ "${UPLOAD_DOCS}" -gt 0 ]; then
upload_to_github_release docs "${ARROW_ARTIFACTS_DIR}"/*-docs/*
fi
if [ "${UPLOAD_ODBC}" -gt 0 ]; then
upload_to_github_release odbc \
"${ARROW_ARTIFACTS_DIR}"/Apache-Arrow-Flight-SQL-ODBC-*-win64.msi
fi
if [ "${UPLOAD_PYTHON}" -gt 0 ]; then
upload_to_github_release python \
"${ARROW_ARTIFACTS_DIR}"/{python-sdist,wheel-*}/*
Expand Down
Loading