All articles

Tracing LiteLLM proxy calls from .NET with Langfuse

, 4 min read

To trace LLM calls from a .NET service through a LiteLLM proxy into Langfuse, make one service the only place that talks to models, wrap every call in an OpenTelemetry activity that carries the model, messages and token counts, and refuse to start if tracing is enabled but any credential is missing. The last part matters most: a half-configured exporter fails silently, and you only discover it when the traces you need are not there.

Why put every LLM call behind one service?

I worked on a setup where several applications needed AI-generated responses. Rather than let each one hold model credentials, prompts and retry logic, one internal .NET service became the gateway. Consumers send it a request and get a response back. That gave me a single place to add tracing, so every call is observed the same way regardless of who asked.

The gateway talks to models through a LiteLLM proxy. The proxy gives one OpenAI-compatible endpoint in front of several providers, so switching a model is a configuration change rather than a code change. Langfuse sits on the other side as the observability tool: it shows each call as a generation with its model, input, output, token usage and cost.

How do you record a call as a Langfuse generation from .NET?

Langfuse accepts OpenTelemetry traces, so in .NET the natural unit is an Activity. The OpenTelemetry GenAI semantic conventions define attribute names for the model and token counts, and Langfuse can read those. Here is a minimal wrapper, written from scratch to show the shape:

using System.Diagnostics;

public sealed class LlmTracer
{
    private static readonly ActivitySource Source = new("Gateway.Llm");

    public async Task<string> TraceAsync(
        string model,
        IReadOnlyList<(string Role, string Content)> messages,
        Func<Task<(string Reply, int InputTokens, int OutputTokens)>> call)
    {
        using var activity = Source.StartActivity("chat " + model);
        activity?.SetTag("gen_ai.request.model", model);
        activity?.SetTag("gen_ai.prompt",
            string.Join("\n", messages.Select(m => m.Role + ": " + m.Content)));

        var (reply, input, output) = await call();

        activity?.SetTag("gen_ai.completion", reply);
        activity?.SetTag("gen_ai.usage.input_tokens", input);
        activity?.SetTag("gen_ai.usage.output_tokens", output);
        return reply;
    }
}

You register the source with the OpenTelemetry SDK using AddSource("Gateway.Llm") and point an OTLP exporter at your Langfuse instance. In my case I used an open-source .NET client for Langfuse rather than hand-rolling the exporter. It was pre-1.0, so I hid it behind a single wrapper type. If its API shifts, one file changes and the rest of the service never notices.

What changes when the reply sits inside a tool call?

The gateway has two client paths: one talks to a provider directly, the other goes through the proxy. Both are instrumented, but they do not return the answer in the same place. On the proxy path I forced a tool call so the model returns structured output, which means the useful reply is in the tool call's arguments and the message content is empty. A trace that reads only the message content shows blank outputs for every proxied call.

The fix is to have the tracing code read the reply from wherever the path puts it. If the response contains a tool call, record its arguments as the completion; otherwise record the message text. Check this early. An empty output column in the observability tool looks like a tracing bug, but it is a reading bug.

Why keep the real message roles?

It is tempting to flatten the input into one string. Don't. Keep each message's actual role, whether system, developer, user or assistant, so a multi-turn conversation reads as a conversation in the trace. When you are debugging why the model answered oddly on the third turn, seeing which earlier turns were assistant output and which were instructions is the whole point of tracing.

How do you stop tracing from failing silently?

OpenTelemetry exporters are built not to crash your application. If the endpoint credentials are wrong or blank, the exporter logs a warning and carries on, and every export fails. Nothing breaks, so nobody notices until someone opens Langfuse and finds it empty.

There is a second trap. If your secrets store substitutes values into configuration at deploy time, a failed substitution can leave the literal placeholder text in the setting. That is not blank, so a simple null check passes it.

My answer was to validate at startup. When tracing is switched on, the service checks every required setting, treats blank values and unsubstituted placeholders as missing, and throws one exception that names every missing key at once. Reporting them all together saves a round of deploy, fail, fix, redeploy for each key.

public sealed class TracingOptions
{
    public bool Enabled { get; set; }
    public string? Endpoint { get; set; }
    public string? PublicKey { get; set; }
    public string? SecretKey { get; set; }

    public void Validate()
    {
        if (!Enabled) return;

        var missing = new List<string>();
        Check(nameof(Endpoint), Endpoint);
        Check(nameof(PublicKey), PublicKey);
        Check(nameof(SecretKey), SecretKey);

        if (missing.Count > 0)
            throw new InvalidOperationException(
                "Tracing is enabled but these settings are missing: "
                + string.Join(", ", missing));

        void Check(string name, string? value)
        {
            if (string.IsNullOrWhiteSpace(value) || value.StartsWith("${"))
                missing.Add(name);
        }
    }
}

Tracing ships switched off by default and is enabled through configuration, so local development and tests never need credentials. The strict check applies only when someone has asked for tracing.

Key takeaways

  • Funnel every LLM call through one service so tracing is added once and applies to every consumer.
  • Record each call as an OpenTelemetry activity with the model, the real message roles, the reply and the token counts, and Langfuse will show it as a generation.
  • Read the reply from the tool call arguments when you force a tool call, or the proxied path will show empty outputs.
  • Wrap a pre-1.0 tracing library in a single type so a breaking release touches one file.
  • Validate tracing configuration at startup, treat unsubstituted placeholders as missing, and name every missing key in one error.