Azure storage authentication errors in 24.08.0-edge with Fusion enabled

[alberto-miranda]

Bug report

Recent PR #5081 slightly changed the authentication behavior of nextflow with Azure by not creating a SAS token for all cases where it's required. If Fusion is enabled, failing to create the SAS token provokes an error in Fusion which crashes nextflow run with errors similar to the following:

9:29AM ERR downloading stream error="GET https://seqeralabs.blob.core.windows.net/scidev-useast/scratch/21laGA30YXQGBQ/43/f98f42ed5ad9d27d971b40c2bb04e6/.command.run\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nERROR CODE: AuthenticationFailed\n--------------------------------------------------------------------------------\n\ufeff<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:8ac53056-201e-00ec-2376-ff738d000000\nTime:2024-09-05T09:29:10.0998308Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request 'ngsTkY1vIU3LzUYQLpKgVPSWnp5P4/1xsBz67dcX7oQ=' is not the same as any computed signature. Server used following string to sign: 'GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:Thu, 05 Sep 2024 09:29:10 GMT\nx-ms-version:2023-11-03\n/seqeralabs/scidev-useast/scratch%2F21laGA30YXQGBQ%2F43%2Ff98f42ed5ad9d27d971b40c2bb04e6%2F.command.run'.</AuthenticationErrorDetail></Error>\n--------------------------------------------------------------------------------\n"
9:29AM WRN expected .command.run file not found :: GET https://seqeralabs.blob.core.windows.net/scidev-useast/scratch/21laGA30YXQGBQ/43/f98f42ed5ad9d27d971b40c2bb04e6/.command.run
--------------------------------------------------------------------------------
RESPONSE 403: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
ERROR CODE: AuthenticationFailed
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:8ac53056-201e-00ec-2376-ff738d000000
Time:2024-09-05T09:29:10.0998308Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request 'ngsTkY1vIU3LzUYQLpKgVPSWnp5P4/1xsBz67dcX7oQ=' is not the same as any computed signature. Server used following string to sign: 'GET











x-ms-date:Thu, 05 Sep 2024 09:29:10 GMT
x-ms-version:2023-11-03
/seqeralabs/scidev-useast/scratch%2F21laGA30YXQGBQ%2F43%2Ff98f42ed5ad9d27d971b40c2bb04e6%2F.command.run'.</AuthenticationErrorDetail></Error>
--------------------------------------------------------------------------------

9:29AM ERR querying pager error="GET https://seqeralabs.blob.core.windows.net/scidev-useast\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nERROR CODE: AuthenticationFailed\n--------------------------------------------------------------------------------\n\ufeff<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:8ac5305b-201e-00ec-2776-ff738d000000\nTime:2024-09-05T09:29:10.1081463Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request '4DZkymMysDuE0NnH0PhTKjdbNQ8o6OLTWHgrDESF2Rc=' is not the same as any computed signature. Server used following string to sign: 'GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:Thu, 05 Sep 2024 09:29:10 GMT\nx-ms-version:2023-11-03\n/seqeralabs/scidev-useast\ncomp:list\ndelimiter:/\nprefix:scratch/21laGA30YXQGBQ/43/f98f42ed5ad9d27d971b40c2bb04e6/\nrestype:container'.</AuthenticationErrorDetail></Error>\n--------------------------------------------------------------------------------\n"
9:29AM FTL validating work path error="validating remote store, prefix '/az/scidev-useast/scratch/21laGA30YXQGBQ/43/f98f42ed5ad9d27d971b40c2bb04e6' must exists - GET https://seqeralabs.blob.core.windows.net/scidev-useast\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nERROR CODE: AuthenticationFailed\n--------------------------------------------------------------------------------\n\ufeff<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:8ac5305b-201e-00ec-2776-ff738d000000\nTime:2024-09-05T09:29:10.1081463Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request '4DZkymMysDuE0NnH0PhTKjdbNQ8o6OLTWHgrDESF2Rc=' is not the same as any computed signature. Server used following string to sign: 'GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:Thu, 05 Sep 2024 09:29:10 GMT\nx-ms-version:2023-11-03\n/seqeralabs/scidev-useast\ncomp:list\ndelimiter:/\nprefix:scratch/21laGA30YXQGBQ/43/f98f42ed5ad9d27d971b40c2bb04e6/\nrestype:container'.</AuthenticationErrorDetail></Error>\n--------------------------------------------------------------------------------\n"

Affected versions

• nextflow 24.08.0-edge

[pditommaso]

I believe this has been solved by #5328