Summary
authservice's Redis session store constructs its client via go-redis's standalone NewClient path, which is not Sentinel-aware. In deployments backed by a Sentinel-managed HA Redis (or Valkey) cluster, this means authservice continues talking to the former primary after a Sentinel-driven failover, surfacing a sustained stream of READONLY You can't write against a read only replica errors to users until the authservice pods are manually restarted.
Raising this to confirm the problem framing and see whether maintainers would welcome a contribution in this direction.
Current behavior
NewRedisClient at internal/oidc/redis.go:71,116 parses the server_uri and hands it to redis.NewClient:
opts, err := redis.ParseURL(config.GetServerUri())
// ...
return redis.NewClient(opts), nil
redis.ParseURL only accepts redis://, rediss://, and unix:// (go-redis v9.7.3 options.go:298-305). The standalone client has no mechanism to discover a new primary on its own.
Consequence on Sentinel-driven failover:
- Former primary is demoted and starts rejecting writes with
READONLY You can't write against a read only replica.
- The standalone client surfaces the error to authservice with no Sentinel-aware retry or re-resolution.
- Pool-level teardown eventually forces a reconnect, but DNS re-resolution on the same Service name lands back on the same endpoints.
- Writes remain broken until authservice pods are restarted.
Related issue #240 (still open) describes the same class of breakage attributed to replica sync lag; PR #241 attempted a fix and was closed. I suspect the root cause of both is this standalone-client construction.
Proposed direction
Use go-redis's built-in Sentinel client. NewFailoverClient returns the same *redis.Client type as NewClient, so the rest of the code (via redis.Cmdable at internal/oidc/redis.go:62) is unchanged. The failover client subscribes to Sentinel's +switch-master channel and updates its internal master address on each event — no additional client-side logic needed.
The API is already available in the go-redis/v9 v9.7.3 authservice is pinned on, so this wouldn't require a dependency bump. oauth2-proxy uses this exact pattern (its buildSentinelClient) and has shipped it in production for years.
Config-wise, an optional Sentinel sub-message on RedisConfig would be additive and preserve the existing server_uri path.
Summary
authservice's Redis session store constructs its client via go-redis's standalone
NewClientpath, which is not Sentinel-aware. In deployments backed by a Sentinel-managed HA Redis (or Valkey) cluster, this means authservice continues talking to the former primary after a Sentinel-driven failover, surfacing a sustained stream ofREADONLY You can't write against a read only replicaerrors to users until the authservice pods are manually restarted.Raising this to confirm the problem framing and see whether maintainers would welcome a contribution in this direction.
Current behavior
NewRedisClientatinternal/oidc/redis.go:71,116parses theserver_uriand hands it toredis.NewClient:redis.ParseURLonly acceptsredis://,rediss://, andunix://(go-redis v9.7.3options.go:298-305). The standalone client has no mechanism to discover a new primary on its own.Consequence on Sentinel-driven failover:
READONLY You can't write against a read only replica.Related issue #240 (still open) describes the same class of breakage attributed to replica sync lag; PR #241 attempted a fix and was closed. I suspect the root cause of both is this standalone-client construction.
Proposed direction
Use go-redis's built-in Sentinel client.
NewFailoverClientreturns the same*redis.Clienttype asNewClient, so the rest of the code (viaredis.Cmdableatinternal/oidc/redis.go:62) is unchanged. The failover client subscribes to Sentinel's+switch-masterchannel and updates its internal master address on each event — no additional client-side logic needed.The API is already available in the
go-redis/v9 v9.7.3authservice is pinned on, so this wouldn't require a dependency bump. oauth2-proxy uses this exact pattern (itsbuildSentinelClient) and has shipped it in production for years.Config-wise, an optional Sentinel sub-message on
RedisConfigwould be additive and preserve the existingserver_uripath.