[Telemetry] use existed env variable instead of introducing new one

[louie-tsai]

Description

Remove new environment variable and use existed one to disable/enable telemetry feature
PR#1168

Issues

NA

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

no

Tests

manually testing

[eero-t]

Approved, matches env var used in telemetry code:

GenAIComps/comps$ git grep TELEMETRY | grep py:
__init__.py:if os.getenv("ENABLE_OPEA_TELEMETRY", "false").lower() == "true":
cores/telemetry/opea_telemetry.py:telemetry_endpoint = os.environ.get("TELEMETRY_ENDPOINT", "http://localhost:4318/v1/traces")

[lianhao]

Instead of enabling/disabling the tracing in comps/init.py, could we do this in the module comps.cores.telemetry.opea_telemetry.opea_telemetry itself.

Doing a simple grep, I found other places just directly import the opea_telemetry.

$ grep -r 'import opea_telemetry' `find -name "*.py"`
./comps/__init__.py:    from comps.cores.telemetry.opea_telemetry import opea_telemetry
./comps/rerankings/src/opea_reranking_microservice.py:from comps.cores.telemetry.opea_telemetry import opea_telemetry
./comps/embeddings/src/opea_embedding_microservice.py:from comps.cores.telemetry.opea_telemetry import opea_telemetry
./comps/cores/mega/orchestrator.py:from ..telemetry.opea_telemetry import opea_telemetry, tracer
./comps/llms/src/text-generation/opea_llm_microservice.py:from comps.cores.telemetry.opea_telemetry import opea_telemetry

[louie-tsai]

Instead of enabling/disabling the tracing in comps/init.py, could we do this in the module comps.cores.telemetry.opea_telemetry.opea_telemetry itself.

Doing a simple grep, I found other places just directly import the opea_telemetry.

$ grep -r 'import opea_telemetry' `find -name "*.py"`
./comps/__init__.py:    from comps.cores.telemetry.opea_telemetry import opea_telemetry
./comps/rerankings/src/opea_reranking_microservice.py:from comps.cores.telemetry.opea_telemetry import opea_telemetry
./comps/embeddings/src/opea_embedding_microservice.py:from comps.cores.telemetry.opea_telemetry import opea_telemetry
./comps/cores/mega/orchestrator.py:from ..telemetry.opea_telemetry import opea_telemetry, tracer
./comps/llms/src/text-generation/opea_llm_microservice.py:from comps.cores.telemetry.opea_telemetry import opea_telemetry

Actually, we probably should remove all Telemetry Endpoint env variables from default compose.yaml and enable it only using an additional telemetry.yaml file.
We used this mechanism in below PR.
opea-project/GenAIExamples#1488
Does it work?

[lianhao]

Actually, we probably should remove all Telemetry Endpoint env variables from default compose.yaml and enable it only using an additional telemetry.yaml file.
We used this mechanism in below PR.
opea-project/GenAIExamples#1488
Does it work?

I think this PR is not related whether we configure the tracing endpoint through environment variable or command line parameter(I'm fine to any of the option). My point is that we should disable the tracing by default, currently it assumes a tracing endpoint is available at localhost:4318 by default which will result connection refused errors even when users are not set any tracing related configurations:

Below is the excerpt of container logs of the retriever service in latest chatqna example without any tracing configuration enabled.

retriever-redis-server  | The above exception was the direct cause of the following exception:
retriever-redis-server  |
retriever-redis-server  | Traceback (most recent call last):
retriever-redis-server  |   File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
retriever-redis-server  |     resp = conn.urlopen(
retriever-redis-server  |            ^^^^^^^^^^^^^
retriever-redis-server  |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retriever-redis-server  |     retries = retries.increment(
retriever-redis-server  |               ^^^^^^^^^^^^^^^^^^
retriever-redis-server  |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
retriever-redis-server  |     raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
retriever-redis-server  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                   retriever-redis-server  | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4318): Max retries exceeded with url: /v1/traces (Caused by NewConnectionError
('<urllib3.connection.HTTPConnection object at 0x7fabf9c97e90>: Failed to establish a new connection: [Errno 111] Connection refused'))  

[xiguiw]

Actually, we probably should remove all Telemetry Endpoint env variables from default compose.yaml and enable it only using an additional telemetry.yaml file.
We used this mechanism in below PR.
opea-project/GenAIExamples#1488
Does it work?

I think this PR is not related whether we configure the tracing endpoint through environment variable or command line parameter(I'm fine to any of the option). My point is that we should disable the tracing by default, currently it assumes a tracing endpoint is available at localhost:4318 by default which will result connection refused errors even when users are not set any tracing related configurations:

Below is the excerpt of container logs of the retriever service in latest chatqna example without any tracing configuration enabled.

retriever-redis-server  | The above exception was the direct cause of the following exception:
retriever-redis-server  |
retriever-redis-server  | Traceback (most recent call last):
retriever-redis-server  |   File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
retriever-redis-server  |     resp = conn.urlopen(
retriever-redis-server  |            ^^^^^^^^^^^^^
retriever-redis-server  |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retriever-redis-server  |     retries = retries.increment(
retriever-redis-server  |               ^^^^^^^^^^^^^^^^^^
retriever-redis-server  |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
retriever-redis-server  |     raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
retriever-redis-server  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                   retriever-redis-server  | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4318): Max retries exceeded with url: /v1/traces (Caused by NewConnectionError
('<urllib3.connection.HTTPConnection object at 0x7fabf9c97e90>: Failed to establish a new connection: [Errno 111] Connection refused'))  

There is code confliction. Please help to resolve it.

@louie-tsai
Could you confirm the telemetry is off by default? although it not related to this PR.