Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to Azure Linux queues #112024

Closed
wants to merge 1 commit into from
Closed

Conversation

richlander
Copy link
Member

@richlander richlander commented Jan 31, 2025

Switch to Azure Linux 3 queues for an initial set of pipelines as a test.

@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jan 31, 2025
@richlander richlander added area-Infrastructure and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Jan 31, 2025
Copy link
Contributor

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

@richlander
Copy link
Member Author

PTAL @vcsjones. Looks like there is a mismatch with Azure Linux being the VM host for some reason.

@vcsjones
Copy link
Member

@richlander will take a look today. It's going to take me a bit to get my AZL test / dev environment working again, so answers might not come until Monday.

@richlander
Copy link
Member Author

We should probably wait until this issue is resolved: dotnet/dnceng#4892

@vcsjones
Copy link
Member

@richlander Looks like we found another edge case in SCOSSL. I filed issue #112036, and opened PR #112037 to disable the tests for now.

there is a mismatch with Azure Linux being the VM host for some reason.

I suspect the VM has a slightly newer version of Azure Linux than the containers. I would expect the containers to start failing if/when they are re-created that have the latest symcrypt-openssl and openssl-devel packages.

@richlander
Copy link
Member Author

richlander commented Jan 31, 2025

It appears that the newest OpenSSL is installed in our images. I don't think the VM has another source of packages. Are we missing packages in our helix images, perhaps?

$ docker run --rm -it mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-helix-amd64 bash -c "tdnf list --installed | grep ssl"
openssl.x86_64                               3.3.2-1.azl3              @System
openssl-libs.x86_64                          3.3.2-1.azl3              @System

These seem to be the total list available.

$ docker run --rm -it mcr.microsoft.com/azurelinux/base/core:3.0 bash -c "tdnf list | grep ssl"
openssl.x86_64                               3.3.2-1.azl3              @System
openssl-libs.x86_64                          3.3.2-1.azl3              @System
apr-util-openssl.x86_64                      1.6.3-2.azl3         azurelinux-official-base
mod_ssl.x86_64                               2.4.61-1.azl3        azurelinux-official-base
mod_ssl.x86_64                               2.4.62-1.azl3        azurelinux-official-base
openssl.x86_64                               3.3.0-2.azl3         azurelinux-official-base
openssl.x86_64                               3.3.2-1.azl3         azurelinux-official-base
openssl-devel.x86_64                         3.3.0-2.azl3         azurelinux-official-base
openssl-devel.x86_64                         3.3.2-1.azl3         azurelinux-official-base
openssl-libs.x86_64                          3.3.0-2.azl3         azurelinux-official-base
openssl-libs.x86_64                          3.3.2-1.azl3         azurelinux-official-base
openssl-perl.x86_64                          3.3.0-2.azl3         azurelinux-official-base
openssl-perl.x86_64                          3.3.2-1.azl3         azurelinux-official-base
openssl-pkcs11.x86_64                        0.4.12-1.azl3        azurelinux-official-base
openssl-static.x86_64                        3.3.0-2.azl3         azurelinux-official-base
openssl-static.x86_64 

@@ -98,9 +98,9 @@ jobs:
# Linux x64
- ${{ if eq(parameters.platform, 'linux_x64') }}:
- ${{ if eq(variables['System.TeamProject'], 'public') }}:
- Ubuntu.2204.Amd64.Open
- azurelinux.3.amd64.open
Copy link
Member

@akoeplinger akoeplinger Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add an Ubuntu container run so we don't lose coverage for that platform?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great question, in two different ways.

We have Ubuntu coverage with Arm64. I think that's sufficient:

- (Ubuntu.2004.Arm64.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-20.04-helix-arm64v8
- ${{ if eq(variables['System.TeamProject'], 'internal') }}:
- (Ubuntu.2004.Arm64)[email protected]/dotnet-buildtools/prereqs:ubuntu-20.04-helix-arm64v8
.

However, I'm not sure why we use this raw VM pattern at all. We only do that because we can (matching arch). I think I should convert x64 to look the same as the other architectures (use containers). That would be cleaner and easier for people looking at this scheme for the first time. I know it confused me. In theory, we could then skip putting all extra dependencies on the Azure Linux VM.

Thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main benefit of the raw VM pattern is that it removes one layer of complexity (containers), e.g. there are small differences between running in the container or not like cgroup constraints etc. I'm also not sure whether it might have infrastructure benefits i.e. workitems start faster when they don't need to fetch the container image before running.

Question is whether that's enough reason to keep the existing setup.

@vcsjones
Copy link
Member

It appears that the newest OpenSSL is installed in our images.

Do the tests pass in our container images then? I think AZL3 recently took an update to OpenSSL 3.3 and that's when the functionality started not working.

@richlander
Copy link
Member Author

CI is a bit broken right now. We should wait to see what results look like after it is back.

@dougbu
Copy link
Member

dougbu commented Jan 31, 2025

We should probably wait until this issue is resolved: dotnet/dnceng#4892

resolution is at least temporarily going to be a rollout revert unfortunately. that'll break the azurelinux.3.amd64* queues again 😦

@richlander
Copy link
Member Author

Closing until the queue we need is active again.

@richlander richlander closed this Feb 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants