Skip to content

langchain: streamed calls are not marked as streaming, thus no streaming timing signals emitted #481

Description

@sfc-gh-zeningchen

Describe your environment

OS: Ubuntu 22.04
Python version: Python 3.12
Package version: opentelemetry-instrumentation-genai-langchain 1.1b1, opentelemetry-util-genai 1.1b0 (also reproduces on main)
GenAI library (e.g. anthropic, openai) and version: langchain 1.3.16, langchain-openai 1.4.3

What happened?

opentelemetry-util-genai 1.1b0 added streaming telemetry in #269 — the
gen_ai.request.stream and gen_ai.response.time_to_first_chunk span
attributes, and the gen_ai.client.operation.time_to_first_chunk and
gen_ai.client.operation.time_per_output_chunk histograms. None of it reaches
the LangChain path.
Those values are set by the shared stream wrappers, which need an SDK stream to
wrap. The LangChain instrumentation is callback-based: it opens an
InferenceInvocation in on_chat_model_start, finishes it in on_llm_end, and
never sees a stream. on_llm_new_token is not implemented, so there is no
per-chunk hook either.
A streamed call is therefore indistinguishable from a synchronous one, and
time-to-first-token cannot be separated from total duration.

Steps to Reproduce

Stream a chat call with the instrumentation enabled:

LangChainInstrumentor().instrument()
list(ChatOpenAI(model="gpt-4o-mini", stream_usage=True).stream("Say hello."))

Then inspect the finished span's attributes and the emitted metric names.

Expected Result

The chat span carries gen_ai.request.stream = true and a plausible
gen_ai.response.time_to_first_chunk, and both streaming histograms record
points. Non-streamed calls are unchanged.

Actual Result

Both span attributes are absent and neither histogram is emitted.

Additional context

No response

Would you like to implement a fix?

Yes

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status
    New issues

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions