Add OpenTelemetry OTLP exporter with full SDK support by etiennep · Pull Request #218 · segmentio/stats

etiennep · 2026-02-06T10:14:39Z

Summary

Adds production-ready OpenTelemetry Protocol (OTLP) exporter using the official OpenTelemetry SDK with comprehensive support for both gRPC and HTTP/Protobuf transports.

Features

✅ Dual Transport Support: gRPC and HTTP/Protobuf protocols
✅ Environment Variables: Full OTEL_* environment variable support
✅ Resource Detection: AWS (EC2, ECS, EKS, Lambda), GCP, Azure, K8s, host, process
✅ All Metric Types: Counter, Gauge, Histogram with proper semantics
✅ Tag Conversion: Automatic stats tags → OpenTelemetry attributes
✅ Production Ready: Thread-safe, tested, documented

Usage

// Simple - uses environment variables
handler, err := otlp.NewSDKHandlerFromEnv(ctx)
stats.Register(handler)

// Or explicit configuration
handler, err := otlp.NewSDKHandler(ctx, otlp.SDKConfig{
    Protocol: otlp.ProtocolGRPC,
    Endpoint: "localhost:4317",
})

Implementation Highlights

Gauge Semantics: Uses UpDownCounter with delta calculation to maintain absolute value semantics (workaround until stable OTel SDK adds Gauge)
Context Handling: Background context for metric recording prevents context cancellation issues
Performance: Lock-free reads for instrument lookup, efficient caching
Resource Detection: Automatic cloud provider metadata detection

Documentation

📖 Complete OTLP README with examples
📝 Implementation Notes explaining design decisions
🔧 Example code for all major use cases
☁️ Cloud resource detector guides (AWS, GCP, Azure)

Testing

✅ Unit tests for all metric types
✅ Gauge behavior verification
✅ HTTP and gRPC protocol tests
✅ Value type conversion tests
✅ Performance benchmarks

Changes

Added otlp/sdk_handler.go - Main OpenTelemetry SDK integration
Added otlp/sdk_handler_test.go - Comprehensive tests
Added otlp/example_test.go - Usage examples
Added otlp/README.md - Complete documentation
Added otlp/IMPLEMENTATION_NOTES.md - Design decisions
Updated README.md - Added OpenTelemetry backend overview
Updated HISTORY.md - Added v5.9.0 release notes
Updated version/version.go - Bumped to 5.9.0
Updated otlp/go.mod - Added OpenTelemetry SDK dependencies

Backward Compatibility

✅ Fully backward compatible - This is a new feature addition that doesn't change existing APIs. The legacy otlp.Handler remains available for existing users.

🤖 Generated with Claude Code

kevinburkesegment · 2026-02-19T00:10:59Z

+
+### 3. Instrument Caching
+
+**Implementation**: Thread-safe two-level locking pattern


I can't tell if this is internal to the stats library or external - if users need to know about this.

kevinburkesegment · 2026-02-19T00:12:10Z

+
+### Default: Cumulative Temporality
+
+**Decision**: Use cumulative temporality for all metric instruments (Prometheus-compatible)


I think we should rewrite this to be aimed more at people who might be curious - it's weird to talk about a "decision" without a discussion of the tradeoffs that led to that

Or maybe it just needs to be presented in reverse order

kevinburkesegment · 2026-02-19T00:15:43Z

+   - ✅ Resource detection
+   - ✅ Production-ready
+
+2. **Handler** (Legacy): Custom OTLP implementation


Do we have internal use of this?

If so I'd like to expand on why people shouldn't use this.

Asked claude to search for usage and it didn't find any. Adding clear deprecation notice.

kevinburkesegment · 2026-02-19T02:13:31Z

+	// For gRPC: "localhost:4317"
+	// For HTTP: "http://localhost:4318"
+	// If empty, uses OTEL_EXPORTER_OTLP_ENDPOINT environment variable
+	Endpoint string


Note people frequently get tripped up between "Endpoint" and "EndpointURL" we should probably note the difference here and say this is explicitly "Endpoint"

oooh good point.

kevinburkesegment · 2026-03-12T18:11:53Z

-go 1.23.0
+go 1.24.0

 require (


Should this really have its own separate go.mod? I guess so the other callers don't pull in all of the otel dependencies?

kevinburkesegment · 2026-03-12T18:12:12Z

+//	    EndpointURL: "http://localhost:4318",
+//	})
+//
 // Status: Alpha. This Handler is still in heavy development phase. Do not use


Let's remove this comment since this is now deprecated

kevinburkesegment · 2026-03-12T18:14:59Z

+    defer stats.Flush()
+
+    // Or use environment variables (simplest)
+    // export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318


This example switches from GRPC to HTTP in addition to switching from env vars to in memory - that's fine but let's be explicit this is making two changes (env var AND protocol) instead of just one

kevinburkesegment · 2026-03-12T18:16:44Z

+
+### Prometheus
+
+The [github.com/segmentio/stats/v5/prometheus](https://godoc.org/github.com/segmentio/stats/v5/prometheus) package exposes an HTTP handler that serves metrics in Prometheus format.


This is dumb but for people who have never used Prometheus before, the pull model vs. push can be counterintuitive, could we just add a sentence explaining that's how this works? "Note that with Prometheus, the metric server will poll your client for changes - metrics are not pushed from a client to the server"

kevinburkesegment · 2026-03-12T18:18:46Z

+	// If zero or not set, uses the SDK default (60 seconds)
+	ExportInterval time.Duration
+
+	// ExportTimeout specifies the timeout for exports


I'd be more specific - "the maximum amount of time to wait for a request to the server to complete"

As written it could be confused with what ExportInterval does

kevinburkesegment · 2026-03-12T18:19:05Z

+
+	// HTTPOptions are additional options for HTTP protocol
+	// Only used when Protocol is ProtocolHTTPProtobuf
+	HTTPOptions []otlpmetrichttp.Option


My first question with this and GRPCOptions is "what options exist" can we link to the docs.

kevinburkesegment · 2026-03-12T18:19:39Z

+	// Set defaults for histogram configuration
+	if config.ExponentialHistogram {
+		if config.ExponentialHistogramMaxSize == 0 {
+			config.ExponentialHistogramMaxSize = 160


let's pull into DefaultHistogramMaxSize const please

kevinburkesegment · 2026-03-12T18:19:43Z

+			config.ExponentialHistogramMaxSize = 160
+		}
+		if config.ExponentialHistogramMaxScale == 0 {
+			config.ExponentialHistogramMaxScale = 20


kevinburkesegment · 2026-03-12T18:20:02Z

+	res := config.Resource
+	if res == nil {
+		var err error
+		res, err = resource.New(ctx,


let's put a timeout on this - we shouldn't hang forever because we couldn't get a resource

kevinburkesegment · 2026-03-12T18:20:52Z

+		}
+
+	default:
+		return nil, fmt.Errorf("unsupported protocol: %s", protocol)


Suggested change

return nil, fmt.Errorf("unsupported protocol: %s", protocol)

return nil, fmt.Errorf("unsupported protocol: %q", protocol)

"%q" will make clear 'you passed the empty string' vs. 'we forgot to include the variable in the error message'

kevinburkesegment · 2026-03-12T18:21:25Z

+	case stats.Counter:
+		counter, err := meter.Int64Counter(name)
+		if err != nil {
+			log.Printf("stats/otlp: failed to create counter %s: %v", name, err)


slog.Info() ?

same for all in this file

Implement a production-ready OpenTelemetry Protocol (OTLP) exporter using the official OpenTelemetry SDK, supporting both gRPC and HTTP transports, and deprecate the legacy alpha handler in preparation for v6. Features: - gRPC and HTTP/Protobuf protocol support - Counter, Gauge, and Histogram metric types - Optional exponential histogram aggregation - Configurable temporality (cumulative default, Prometheus-compatible) - Tag to attribute conversion - Thread-safe instrument caching Implementation: - Gauges use the native Float64Gauge instrument for instantaneous value recording - Background context for recording to avoid cancellation issues - Lock-free reads for instrument lookup in hot path - A single Meter is created once and reused across recordings Environment variables: - The transport protocol is resolved from OTEL_EXPORTER_OTLP_PROTOCOL, with OTEL_EXPORTER_OTLP_METRICS_PROTOCOL taking precedence; an unrecognized value is rejected. An explicit SDKConfig.Protocol always wins. We resolve only this variable ourselves because the otlpmetricgrpc/otlpmetrichttp exporters do not read the protocol selector. - The exporters read the remaining OTEL_EXPORTER_OTLP_* variables (endpoint, headers, timeout, compression, ...) themselves; programmatic overrides are available via WithEndpointURL, GRPCOptions, and HTTPOptions. - Resource attributes come from OTEL_RESOURCE_ATTRIBUTES/OTEL_SERVICE_NAME plus host and process detection. Cloud and Kubernetes detection is opt-in via the contrib/detectors/* packages. Config API: - SDKConfig.EndpointURL takes a full URL with scheme (http:// or https://); WithEndpointURL is used to avoid a known gRPC bug with the http:// scheme - ExportInterval and ExportTimeout fall back to SDK defaults (60s and 30s) when unset Deprecations: - Deprecate otlp.Handler (Alpha since 2022, minimal usage) - Deprecate otlp.HTTPClient - Deprecate otlp.NewHTTPClient() All will be removed in v6.0.0. Migration path provided in deprecation notices with code examples. Testing: - Unit tests and benchmarks for instrument handling and value conversion - Integration tests that export to an in-process gRPC OTLP collector and assert on the metrics received over the wire, covering protocol resolution precedence and the invalid-protocol error path Documentation: - Complete README with configuration examples - Cloud resource detector usage guides - Implementation notes explaining design decisions and temporality - Example code for common use cases - HISTORY.md release notes for v5.9.0 Performance: - Preallocate tag/attribute slices to their exact length and assign by index instead of appending, in the OTLP handlers and in the core stats.M and tagFuncMap.namedTagFuncs helpers Bumps version to 5.9.0. All tests pass. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Co-Authored-By: Kevin Burke <kburke@twilio.com>

etiennep force-pushed the add-opentelemetry-otlp-support branch from bad79e6 to e6d05d8 Compare February 6, 2026 10:26

sccoache previously approved these changes Feb 6, 2026

View reviewed changes

etiennep dismissed sccoache’s stale review via 4fbb39e February 6, 2026 16:16

etiennep force-pushed the add-opentelemetry-otlp-support branch 3 times, most recently from a82c5e2 to 82249fe Compare February 6, 2026 17:08

kevinburkesegment reviewed Feb 19, 2026

View reviewed changes

kevinburkesegment reviewed Mar 12, 2026

View reviewed changes

kevinburkesegment force-pushed the add-opentelemetry-otlp-support branch 7 times, most recently from 136c900 to 3441b0b Compare June 15, 2026 16:17

petedannemann approved these changes Jun 15, 2026

View reviewed changes

frajuqeb approved these changes Jun 15, 2026

View reviewed changes

kevinburkesegment force-pushed the add-opentelemetry-otlp-support branch from 3441b0b to 7e7070d Compare June 15, 2026 22:56

kevinburkesegment merged commit 7e7070d into main Jun 15, 2026
7 of 9 checks passed

kevinburkesegment deleted the add-opentelemetry-otlp-support branch June 15, 2026 22:56


		### 3. Instrument Caching

		Implementation: Thread-safe two-level locking pattern


		### Default: Cumulative Temporality

		Decision: Use cumulative temporality for all metric instruments (Prometheus-compatible)


		### Prometheus

		The [github.com/segmentio/stats/v5/prometheus](https://godoc.org/github.com/segmentio/stats/v5/prometheus) package exposes an HTTP handler that serves metrics in Prometheus format.

	return nil, fmt.Errorf("unsupported protocol: %s", protocol)
	return nil, fmt.Errorf("unsupported protocol: %q", protocol)

Uh oh!

Conversation

etiennep commented Feb 6, 2026

Summary

Features

Usage

Implementation Highlights

Documentation

Testing

Changes

Backward Compatibility

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants