You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/modules/airflow/pages/troubleshooting/index.adoc
+48Lines changed: 48 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,53 @@
1
1
= Troubleshooting
2
2
3
+
== Azure Blob Storage Logging:
4
+
5
+
Azure's `ADLS` can be used to store task logs Airflow.
6
+
7
+
Assume a regular storage container in Azures ADLS backend, this can be accessed with either the `adls[s]` or `wasb` connector using the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/adls_v2.html[Azure Data Lake Storage Gen2 Connection] or the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/wasb.html[Microsoft Azure Blob Storage Connection] respectively.
8
+
9
+
Using `ADLS` as a task log backend, requires to access it via `wasb` and thus the configuration in the environment should look like:
<1> This env var is only used for wasb connections.
26
+
<2> Note that the <container-name> is *not* referenced.
27
+
<3> This connection can be defined in the AirflowUI or constructed as env variable.
28
+
29
+
Due to this open https://github.com/apache/airflow/issues/58946[issue] with Airflow, it's recommended to use `wasb-<folder-name>` rather then `wasb://<folder-name>` as it would cause your backend looking like:
30
+
[source,text]
31
+
----
32
+
<container-name>
33
+
└── wasb:/
34
+
└── tasklogs/
35
+
└── dag_id=...
36
+
----
37
+
However the workaround will result in
38
+
[source,text]
39
+
----
40
+
<container-name>
41
+
└── wasb-tasklogs/
42
+
└── dag_id=...
43
+
----
44
+
45
+
The `Azure Blob Storage Connection` will offer the optional field `Host` which should have the a value looking like
0 commit comments