Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Azure Fusion env misses credentials when no key or SAS provided #5287

Closed
wants to merge 15 commits into from
Closed

fix: Azure Fusion env misses credentials when no key or SAS provided #5287

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
92c974c
Fix: Azure Fusion missing authentication if accountKey is provided
adamrtalbot Sep 5, 2024
89710df
Catch when service principal exists but no keys or SAS
adamrtalbot Sep 5, 2024
a6006f0
Mock AzStorageOpts
adamrtalbot Sep 5, 2024
0171fa3
Improve comment for Fusion env with SAS
adamrtalbot Sep 5, 2024
8b92f8f
Update plugins/nf-azure/src/main/nextflow/cloud/azure/fusion/AzFusion…
adamrtalbot Sep 5, 2024
f11d62d
Ensure proper SAS tokens are generated for each Azure authentication …
alberto-miranda Sep 6, 2024
dd20318
test [platform stage]
pditommaso Sep 16, 2024
a832bde
Merge branch 'master' into fix_azure_fusion_missing_auth
pditommaso Sep 16, 2024
61374f8
test [platform stage]
pditommaso Sep 16, 2024
5b67abe
Merge branch 'master' into fix_azure_fusion_missing_auth
adamrtalbot Sep 23, 2024
3139dae
[platform stage]
adamrtalbot Sep 23, 2024
cab3ffe
debug build workflow
adamrtalbot Sep 23, 2024
d05cae5
Revert "debug build workflow"
adamrtalbot Sep 23, 2024
577bfb4
Merge branch 'master' into fix_azure_fusion_missing_auth
pditommaso Sep 24, 2024
f93e25b
test [e2e stage]
pditommaso Sep 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions plugins/nf-azure/src/main/nextflow/cloud/azure/fusion/AzFusionEnv.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,13 @@ class AzFusionEnv implements FusionEnv {
// the account name is always required
result.AZURE_STORAGE_ACCOUNT = cfg.storage().accountName

// If a Managed Identity or Service Principal is configured, Fusion only needs to know the account name
if (cfg.managedIdentity().isConfigured() || cfg.activeDirectory().isConfigured()) {
// If a Managed Identity is configured, Fusion only needs to know the account name
if (cfg.managedIdentity().isConfigured()) {
return result
}

// If a SAS token is configured, instead, Fusion also requires the token value
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamrtalbot Could we please update this comment to something like:

        // Otherwise, Fusion also requires a SAS Token
        // (yes, even if `cfg.storage().sasToken` has not been explicitly defined)

so that this doesn't happen again?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this:

// If Fusion does not use a managed identity, get or create a SAS token for Fusion to use

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not much different from what the code itself already tells. I would prefer to have a explicit comment telling us why it's needed despite the configuration not requesting it explicitly.

if (cfg.storage().sasToken) {
if (cfg.storage().sasToken || cfg.storage().accountKey || cfg.activeDirectory().isConfigured()) {
result.AZURE_STORAGE_SAS_TOKEN = cfg.storage().getOrCreateSasToken()
}

Expand Down
22 changes: 22 additions & 0 deletions plugins/nf-azure/src/test/nextflow/cloud/azure/fusion/AzFusionEnvTest.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,27 @@ class AzFusionEnvTest extends Specification {

}

def 'should return env environment with SAS token config when accountKey is provided'() {
given:
Global.session = Mock(Session) {
getConfig() >> [azure: [storage: [accountName: 'myaccount', accountKey: 'myaccountkey']]]
}
def mockStorageObject = Mock(Object) {
getOrCreateSasToken() >> 'generatedSasToken'
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pditommaso o great Spock wizard, why does it not mock the getOrCreateSasToken method properly here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't it be Mock(AzStorageOpts)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried that and I get the same result, env.AZURE_STORAGE_SAS_TOKEN is sv=2024-05-04&ss=bf&srt=sco&se=2024-09-07T11%3A24%3A38Z&sp=rwdlacut&sig=SIGNATURE%3D

def config = Mock(FusionConfig) {
storage() >> mockStorageObject
}

when:
def env = new AzFusionEnv().getEnvironment('az', config)

then:
env.AZURE_STORAGE_ACCOUNT == 'myaccount'
env.AZURE_STORAGE_SAS_TOKEN
env.size() == 2
}

def 'should return env environment with SAS token config'() {
given:
Global.session = Mock(Session) {
Expand Down Expand Up @@ -99,4 +120,5 @@ class AzFusionEnvTest extends Specification {
then:
thrown(IllegalArgumentException)
}

}