CubeAPM
CubeAPM CubeAPM

How to Instrument Go Applications with OpenTelemetry 

How to Instrument Go Applications with OpenTelemetry 

Table of Contents

The way you instrument Go applications with OpenTelemetry differs from Java, Python, and .NET in one fundamental way: Go has no bytecode manipulation or runtime agent mechanism. There is no -javaagent equivalent. 

Every span, metric, and log record in a Go application must come from either manual SDK instrumentation in your code or from instrumentation libraries that wrap the frameworks you already use (such as net/http, gRPC, or database drivers). This guide covers both approaches end-to-end.

Key Takeaways

  • Latest stable release: v1.43.0 of go.opentelemetry.io/otel (April 2, 2026). Requires Go 1.25 or above. Go 1.41.0 was the last release to support Go 1.24. Go 1.43.0 adds support for testing against Go 1.26
  • Go has no automatic zero-code agent. All instrumentation requires either manual SDK calls or wrapping your HTTP/gRPC handlers with instrumentation middleware from go.opentelemetry.io/contrib/instrumentation
  • An experimental eBPF-based zero-code instrumentation project exists (opentelemetry-go-instrumentation) that uses eBPF probes on Linux. It does not require code changes but is Linux-only and supports a limited set of libraries
  • Context propagation in Go is explicit. You must pass context.Context through every function call that needs trace context. Spans created without a context carrying a parent trace start a new root trace
  • The OTel Go SDK has a multi-module versioning scheme. The core API (go.opentelemetry.io/otel) and SDK (go.opentelemetry.io/otel/sdk) share the same version tag. Contrib instrumentation libraries have independent version numbers
  • Default OTLP export protocol is HTTP/protobuf (otlptracehttp, otlpmetrichttp, otlploghttp). gRPC exporters are available as separate packages

The Key Difference: Go Requires Manual Instrumentation

In Java, Python, or .NET, you can attach an agent at startup and get automatic spans for HTTP requests, database queries, and outbound calls without touching application code. In Go, the language runtime does not expose the hooks needed for that approach.

What this means in practice:

  • Every HTTP handler you want to trace needs to be wrapped with otelhttp.NewHandler()
  • Every database call you want to trace needs a traced driver or explicit span creation
  • Every outbound HTTP call needs to use a traced transport via otelhttp.NewTransport()
  • Context must be explicitly passed through your call chain

The upside: Go instrumentation is explicit and predictable. There are no surprises from automatic instrumentation injecting unexpected spans or attributes.

Step 1: Install the Required Packages

# Core API and SDK

go get go.opentelemetry.io/[email protected]

go get go.opentelemetry.io/otel/[email protected]

go get go.opentelemetry.io/otel/sdk/[email protected]

go get go.opentelemetry.io/otel/sdk/[email protected]

go get go.opentelemetry.io/otel/sdk/[email protected]

# OTLP exporters (HTTP/protobuf — default and recommended)

go get go.opentelemetry.io/otel/exporters/otlp/otlptrace/[email protected]

go get go.opentelemetry.io/otel/exporters/otlp/otlpmetric/[email protected]

go get go.opentelemetry.io/otel/exporters/otlp/otlplog/[email protected]

# HTTP server and client instrumentation (contrib, independent versioning)

go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@latest

Or add directly to go.mod and run go mod tidy:

require (

    go.opentelemetry.io/otel v1.43.0

    go.opentelemetry.io/otel/sdk v1.43.0

    go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0

    go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.43.0

    go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp v1.43.0

    go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0

)

Note: contrib instrumentation packages have independent versioning from the core SDK. The otelhttp package at v0.67.0 is compatible with SDK v1.43.0. Always run go mod tidy to resolve the full dependency graph.

Step 2: Initialize the SDK

Create a telemetry.go file to initialize all three signal providers before your application starts handling requests:

package telemetry

import (

    "context"

    "fmt"

    "go.opentelemetry.io/otel"

    "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"

    "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp"

    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"

    "go.opentelemetry.io/otel/propagation"

    sdklog "go.opentelemetry.io/otel/sdk/log"

    sdkmetric "go.opentelemetry.io/otel/sdk/metric"

    "go.opentelemetry.io/otel/sdk/resource"

    sdktrace "go.opentelemetry.io/otel/sdk/trace"

    semconv "go.opentelemetry.io/otel/semconv/v1.40.0"

)

// Setup initializes the OTel SDK and returns a shutdown function.

// The caller must call shutdown() before the application exits to

// flush all buffered telemetry.

func Setup(ctx context.Context) (shutdown func(context.Context) error, err error) {

    res, err := resource.New(ctx,

        resource.WithAttributes(

            semconv.ServiceName("order-service"),

            semconv.ServiceVersion("1.0.0"),

        ),

    )

    if err != nil {

        return nil, fmt.Errorf("failed to create resource: %w", err)

    }

    // Trace exporter

    traceExporter, err := otlptracehttp.New(ctx,

        otlptracehttp.WithEndpoint("otel-collector:4318"),

        otlptracehttp.WithInsecure(),

    )

    if err != nil {

        return nil, fmt.Errorf("failed to create trace exporter: %w", err)

    }

    tp := sdktrace.NewTracerProvider(

        sdktrace.WithBatcher(traceExporter),

        sdktrace.WithResource(res),

        sdktrace.WithSampler(sdktrace.AlwaysSample()),

    )

    otel.SetTracerProvider(tp)

    // Metric exporter

    metricExporter, err := otlpmetrichttp.New(ctx,

        otlpmetrichttp.WithEndpoint("otel-collector:4318"),

        otlpmetrichttp.WithInsecure(),

    )

    if err != nil {

        return nil, fmt.Errorf("failed to create metric exporter: %w", err)

    }

    mp := sdkmetric.NewMeterProvider(

        sdkmetric.WithReader(sdkmetric.NewPeriodicReader(metricExporter)),

        sdkmetric.WithResource(res),

    )

    otel.SetMeterProvider(mp)

    // Log exporter

    logExporter, err := otlploghttp.New(ctx,

        otlploghttp.WithEndpoint("otel-collector:4318"),

        otlploghttp.WithInsecure(),

    )

    if err != nil {

        return nil, fmt.Errorf("failed to create log exporter: %w", err)

    }

    lp := sdklog.NewLoggerProvider(

        sdklog.WithProcessor(sdklog.NewBatchProcessor(logExporter)),

        sdklog.WithResource(res),

    )

    // Set global propagator to W3C TraceContext + Baggage

    otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(

        propagation.TraceContext{},

        propagation.Baggage{},

    ))

    shutdown = func(ctx context.Context) error {

        var errs []error

        if err := tp.Shutdown(ctx); err != nil {

            errs = append(errs, err)

        }

        if err := mp.Shutdown(ctx); err != nil {

            errs = append(errs, err)

        }

        if err := lp.Shutdown(ctx); err != nil {

            errs = append(errs, err)

        }

        if len(errs) > 0 {

            return fmt.Errorf("shutdown errors: %v", errs)

        }

        return nil

    }

    return shutdown, nil

}

Call Setup at the start of main() and defer the shutdown function:

func main() {

    ctx := context.Background()

    shutdown, err := telemetry.Setup(ctx)

    if err != nil {

        log.Fatalf("failed to set up telemetry: %v", err)

    }

    defer func() {

        if err := shutdown(ctx); err != nil {

            log.Printf("telemetry shutdown error: %v", err)

        }

    }()

    startServer()

}

Calling Shutdown on the provider is critical. It flushes all buffered spans and metrics before the process exits. Forgetting to call it causes data loss on graceful shutdown.

Step 3: Instrument HTTP Handlers

The otelhttp package wraps net/http handlers to create a span per request automatically:

import (

    "net/http"

    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"

)

func main() {

    // ... telemetry setup ...

    mux := http.NewServeMux()

    mux.HandleFunc("/orders", handleOrders)

    mux.HandleFunc("/health", handleHealth)

    // Wrap the entire mux with otelhttp to instrument all routes

    handler := otelhttp.NewHandler(mux, "order-service",

        otelhttp.WithFilter(func(r *http.Request) bool {

            // Return false to exclude health checks from tracing

            return r.URL.Path != "/health"

        }),

    )

    http.ListenAndServe(":8080", handler)

}

Each request creates a span named after the service name argument. To get per-route span names, call otelhttp.WithRouteTag(r, “/orders/{id}”) inside the handler, or use a router like gorilla/mux or chi, which have dedicated otelhttp integration.

Step 4: Instrument Outbound HTTP Calls

Wrap the http.Client transport with otelhttp.NewTransport to create spans for every outbound HTTP call and propagate trace context headers automatically:

import (

    "net/http"

    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"

)

var httpClient = &http.Client{

    Transport: otelhttp.NewTransport(http.DefaultTransport),

}

func callPaymentService(ctx context.Context, orderID string) error {

    req, err := http.NewRequestWithContext(ctx, "POST",

        "http://payment-service/charge", nil)

    if err != nil {

        return err

    }

    // otelhttp.NewTransport automatically injects traceparent header

    resp, err := httpClient.Do(req)

    if err != nil {

        return err

    }

    defer resp.Body.Close()

    return nil

}

The traced transport reads the active span from the context, creates a child span for the outbound call, and injects traceparent and tracestate headers so the downstream service can continue the same trace.

Step 5: Create Manual Spans for Business Logic

For business logic inside your handlers, create spans using the global tracer:

import (

    "context"

    "go.opentelemetry.io/otel"

    "go.opentelemetry.io/otel/attribute"

    "go.opentelemetry.io/otel/codes"

    "go.opentelemetry.io/otel/trace"

)

var tracer = otel.Tracer("order-service")

func processOrder(ctx context.Context, order Order) error {

    ctx, span := tracer.Start(ctx, "order.process",

        trace.WithAttributes(

            attribute.String("order.id", order.ID),

            attribute.String("order.customer_id", order.CustomerID),

            attribute.Int("order.item_count", len(order.Items)),

        ),

    )

    defer span.End()

    if err := validateOrder(ctx, order); err != nil {

        span.RecordError(err)

        span.SetStatus(codes.Error, err.Error())

        return err

    }

    span.SetAttributes(attribute.String("order.status", "validated"))

    return chargeOrder(ctx, order)

}

Important rules for Go span management:

  • Always call defer span.End() immediately after tracer.Start(). A span that is not ended is never exported
  • Pass ctx (the updated context containing the new span) to child function calls so they can create child spans automatically
  • Use span.RecordError(err) to attach error details and span.SetStatus(codes.Error, …) to mark the span as failed. Both are needed: RecordError alone does not mark the span as an error in your trace backend

Step 6: Create Custom Metrics

import (

    "context"

    "go.opentelemetry.io/otel"

    "go.opentelemetry.io/otel/attribute"

    "go.opentelemetry.io/otel/codes"

    "go.opentelemetry.io/otel/trace"

)

var tracer = otel.Tracer("order-service")

func processOrder(ctx context.Context, order Order) error {

    ctx, span := tracer.Start(ctx, "order.process",

        trace.WithAttributes(

            attribute.String("order.id", order.ID),

            attribute.String("order.customer_id", order.CustomerID),

            attribute.Int("order.item_count", len(order.Items)),

        ),

    )

    defer span.End()

    if err := validateOrder(ctx, order); err != nil {

        span.RecordError(err)

        span.SetStatus(codes.Error, err.Error())

        return err

    }

    span.SetAttributes(attribute.String("order.status", "validated"))

    return chargeOrder(ctx, order)

}

eBPF Zero-Code Instrumentation (Linux Only, Experimental)

The opentelemetry-go-instrumentation project uses eBPF probes to instrument Go applications without code changes on Linux. It supports net/http servers and clients, gRPC, and the Go standard database/sql package. The latest release includes offset caches for go.opentelemetry.io/otel v1.38.0, Go 1.24.7, and Go 1.25.1.

This is not a production replacement for manual instrumentation. eBPF probes attach to specific function offsets in compiled Go binaries and break when the binary is recompiled with a different Go or library version until new offset caches are added. It is useful for instrumenting third-party binaries you cannot modify.

Configure the OTel Collector

receivers:

  otlp:

    protocols:

      grpc:

        endpoint: 0.0.0.0:4317

      http:

        endpoint: 0.0.0.0:4318

processors:

  batch:

    timeout: 5s

    send_batch_size: 512

exporters:

  otlp:

    endpoint: your-backend:4317

service:

  pipelines:

    traces:

      receivers: [otlp]

      processors: [batch]

      exporters: [otlp]

    metrics:

      receivers: [otlp]

      processors: [batch]

      exporters: [otlp]

    logs:

      receivers: [otlp]

      processors: [batch]

      exporters: [otlp]

Common Setup Problems

ProblemLikely causeFix
No spans appearing in the backendTracerProvider not set globally, or span.End() never calledConfirm otel.SetTracerProvider(tp) is called before any spans are created. Add defer span.End() immediately after tracer.Start()
Spans have no parent even inside HTTP handlersContext not passed through the call chainEnsure the handler reads request context via r.Context() and passes it to all child calls. Use otelhttp.NewHandler() to extract incoming trace context
Outbound HTTP calls not linked to the parent traceUsing http.DefaultClient instead of a traced clientWrap the transport with otelhttp.NewTransport(http.DefaultTransport)
Metrics not appearingMeterProvider not set globally or PeriodicReader interval not elapsedConfirm otel.SetMeterProvider(mp) is called. Default export interval is 30 seconds
Data lost on process exitShutdown() not called before exitUse defer shutdown(ctx) in main(). The Shutdown call flushes all buffered telemetry
Module version conflictsMismatched versions between go.opentelemetry.io/otel and contrib packagesRun go mod tidy and verify contrib packages declare compatible minimum SDK versions in their go.mod
Build failing with Go version errorUsing Go 1.24 with SDK v1.43.0SDK v1.43.0 requires Go 1.25 or above. Either upgrade Go or pin to SDK v1.41.0 which was the last release supporting Go 1.24

Your Traces Are Flowing. Now Make Them Useful.

Go’s explicit context propagation means your traces are structurally correct from the start: every span has the right parent, every outbound call carries the right headers, every database query is a child of the right request span. What that still does not answer is the harder production question: why is this specific request slow when most are not?

When a slow trace surfaces, the span breakdown shows you where time was spent. What it does not show is whether the slowness correlates with a memory pressure spike on that host, a connection pool exhaustion event in your database client, or a deployment that happened 20 minutes earlier. Those answers live in infrastructure metrics and logs, not in the trace itself.

CubeAPM as the best Go application monitoring tool
How to Instrument Go Applications with OpenTelemetry  2

CubeAPM accepts OTLP directly from your existing Go OTel setup and correlates traces, infrastructure metrics, and logs using the trace ID that Go’s context propagation carries through your entire call chain. When a slow trace appears, you move from the span breakdown to the infrastructure metric at that exact moment to the log records from that exact request, without switching tools or matching timestamps manually. It runs self-hosted inside your own infrastructure at $0.15/GB ingestion with no per-user fees, so your telemetry data never leaves your environment.

Summary

Instrumenting Go with OpenTelemetry requires manual SDK initialization, explicit context propagation, and wrapping HTTP handlers and clients with instrumentation middleware. There is no zero-code agent for production use. Initialize TracerProvider, MeterProvider, and LoggerProvider before handling requests, always call Shutdown() on exit, and pass context through every function call that needs trace continuity. SDK v1.43.0 requires Go 1.25 or above.

StepWhat to doKey detail
Install packagesgo.opentelemetry.io/[email protected], SDK, OTLP exporters, contrib instrumentationGo 1.25 minimum. Use semconv/v1.40.0 in code
Initialize SDKCreate providers, set globally, configure OTLP exportersUse otlptracehttp, otlpmetrichttp, otlploghttp for HTTP/protobuf
Register shutdowndefer shutdown(ctx) in main()Flushes all buffered telemetry before process exit
Instrument HTTP serverotelhttp.NewHandler(mux, “service-name”)Wrap the entire mux, use WithFilter to exclude health checks
Instrument HTTP clientotelhttp.NewTransport(http.DefaultTransport)Injects traceparent headers automatically
Create manual spanstracer.Start(ctx, “operation.name”) then defer span.End()Pass updated ctx to child calls. Use RecordError and SetStatus on failure
Custom metricsmeter.Int64Counter() or meter.Float64Histogram()Pass ctx and metric.WithAttributes() when recording
Context propagationPass context.Context explicitly through all function callsFor non-HTTP entry points, extract context with otel.GetTextMapPropagator().Extract()

Disclaimer: Package versions are verified from the opentelemetry-go GitHub releases page and CHANGELOG: go.opentelemetry.io/otel v1.43.0 released April 2, 2026, requires Go 1.25 minimum (v1.41.0 was the last release supporting Go 1.24). Contrib package otelhttp v0.67.0 verified from pkg.go.dev (published April 7, 2026). eBPF instrumentation details verified from github.com/open-telemetry/opentelemetry-go-instrumentation releases as of May 2026.

Also read:

What Is the Difference Between OpenTelemetry and Zipkin? 

What Is the Difference Between OpenTelemetry and Jaeger? 

How to Set Up OpenTelemetry in .NET Applications 

×
×