Can I convert a javaagent.shaded.io.opentelemetry.context to a io.opentelemetry.context.Context

[amccague]

I am hoping to add attributes to an active ktor client span specifically trying to get this context https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/ktor/ktor-2-common/library/src/main/kotlin/io/opentelemetry/instrumentation/ktor/v2_0/common/internal/KtorClientTelemetryUtil.kt#L27 from the pipieline.

However I easily hit:

Exception in thread "main" java.lang.ClassCastException: class io.opentelemetry.javaagent.shaded.io.opentelemetry.context.ArrayBasedContext cannot be cast to class io.opentelemetry.context.Context (io.opentelemetry.javaagent.shaded.io.opentelemetry.context.ArrayBasedContext is in unnamed module of loader 'bootstrap'; io.opentelemetry.context.Context is in unnamed module of loader 'app')

One thing I'm not sure about is why this particular context is shaded, but if I use https://github.com/open-telemetry/opentelemetry-java/blob/main/extensions/kotlin/src/main/kotlin/io/opentelemetry/extension/kotlin/ContextExtensions.kt#L27 then the context is intact.

Are there any helper libraries I can use to 'unshade' the ktor context so that I can call setAttribute on it from the application?

[amccague]

Installing the instrumentation manually does of course work around it, and the span can be customised.

1. Disable the autoinstrumentation:

-Dotel.instrumentation.ktor-client.enabled=false

2. Add the dependency:

implementation("io.opentelemetry.instrumentation:opentelemetry-ktor-2.0")

2. Configure the library plugin

HttpClient(Java) {
    install(KtorClientTracing) {
        setOpenTelemetry(GlobalOpenTelemetry.get()) // Uses OTEL from the agent.
    }
}

[steverao]

I am hoping to add attributes to an active ktor client span

You can check out related document and estimate whether it can meet your requirement by adding an extension.

One thing I'm not sure about is why this particular context is shaded

You can have a look at https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/contributing/javaagent-structure.md#bootstrap-class-loader

[amccague]

@steverao thanks for this that clarified some aspects. I'm still not 100% sure how the Kotlin extensions get an unshaded context from the auto instrumentation.

There's reference to a context bridge here but the module is not directly referenced in the readme you linked?

opentelemetry-java-instrumentation/instrumentation/kotlinx-coroutines/kotlinx-coroutines-1.0/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/kotlinxcoroutines/KotlinCoroutinesInstrumentationModule.java

Lines 31 to 32 in 234d619

	// This module uses the api context bridge helpers, therefore must be in the same classloader
	return "opentelemetry-api-bridge";

It doesn't seem like a stretch then to have a generic unshading mapper extension?

[steverao]

I'm still not 100% sure how the Kotlin extensions get an unshaded context from the auto instrumentation.

You can check out whether they can help you solve your final purpose that you want to add attributes to specific spans by using extensions.

[laurit]

Agent uses shaded opentelemetry api to not conflict when the application also included opentelemetry api. If application provides opentelemetry api agent will instrument the api in the application and bride it with the api in the agent.

opentelemetry-java-instrumentation/instrumentation/opentelemetry-api/opentelemetry-api-1.0/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/opentelemetryapi/context/AgentContextStorage.java

Line 127 in 083a797

public static Context toApplicationContext(io.opentelemetry.context.Context agentContext) {

this method is used to convert context from the agent to context in the application api. Instead of trying to read the context from the attribute perhaps you could add your own interceptor. When otel interceptor calls proceed in

opentelemetry-java-instrumentation/instrumentation/ktor/ktor-2-common/library/src/main/kotlin/io/opentelemetry/instrumentation/ktor/v2_0/common/internal/KtorClientTelemetryUtil.kt

Line 55 in 083a797

withContext(openTelemetryContext.asContextElement()) { proceed() }

the next interceptor should be able to observe the context in coroutine context using coroutineContext.getOpenTelemetryContext() you'll need https://github.com/open-telemetry/opentelemetry-java/tree/main/extensions/kotlin. Didn't try it myself so could be wrong.

[amccague]

@laurit thanks for this it clarifies a lot!

I suppose my intended usage would look like this:

val data = client
    .preparePost(...)
    .execute { response ->
        response.body<Dto>().also { data ->
            with(Span.fromContext(coroutineContext.getOpenTelemetryContext())) {
                setAttribute(SomeKey, data.interestingValue)
            }
        }
    }

With manual instrumentation as I described in #13274 (comment) I can do this:

    .execute { response ->
        response.body<Dto>().also { data ->
            with(Span.fromContext(response.call.coroutineContext.getOpenTelemetryContext())) {
                setAttribute(SomeKey, data.interestingValue)
            }
        }
    }

This doesn't bridge with the agent.

Following your guidance I can install a plugin:

install(createClientPlugin("OTELIntercept") {
    this.client.sendPipeline.intercept(HttpSendPipeline.Monitoring) {
        context.attributes.put(AttributeKey<Span>("OTELSpan"), Span.fromContext(coroutineContext.getOpenTelemetryContext()))
        proceed()
    }
})

And then this works with the agent (could have stored the context but this felt nicer if the plugin is innersourced):

val data = post.execute { response ->
    response.body<Dto>().also { data ->
        with(response.call.attributes[AttributeKey<Span>("OTELSpan")]) {
            setAttribute(SomeKey, data.interestingValue)
        }
    }
}

It would be great if the client context could be found from that execute context out of the box.
For now I'm not sure which is preferred between the additional interceptor or dropping to manual installation.

[laurit]

With manual instrumentation as I described in #13274 (comment) I can do this: ...
This doesn't bridge with the agent.

If it works with the library instrumentation then it is reasonable to assume that it could also work with the agent. Please provide a minimal application that reproduces the issue along with any necessary instructions.