mirror of
https://github.com/we-promise/sure.git
synced 2026-04-07 06:21:23 +00:00
Add Redis Sentinel support for Sidekiq high availability (#457)
* Initial plan * Add Redis Sentinel support for Sidekiq configuration Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com> * Add documentation for Redis Sentinel configuration Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com> * Improve Sentinel host parsing with better validation and error handling Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com> * Add port validation to prevent invalid Sentinel configurations Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com> * Linter * Add Sentinel authentication credentials to fix NOAUTH error - Add sentinel_username (defaults to "default") - Add sentinel_password (uses REDIS_PASSWORD) - Update documentation for sentinel auth configuration Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com> * Add CHANGELOG.md for Helm chart documenting Redis Sentinel support Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com> * Set REDIS_URL alongside in Sentinel mode also --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
This commit is contained in:
@@ -70,8 +70,16 @@ POSTGRES_PASSWORD=postgres
|
||||
POSTGRES_USER=postgres
|
||||
|
||||
# Redis configuration
|
||||
# Standard Redis URL (for direct connection)
|
||||
REDIS_URL=redis://localhost:6379/1
|
||||
|
||||
# Redis Sentinel configuration (for high availability)
|
||||
# When REDIS_SENTINEL_HOSTS is set, it takes precedence over REDIS_URL
|
||||
# REDIS_SENTINEL_HOSTS=sentinel1:26379,sentinel2:26379,sentinel3:26379
|
||||
# REDIS_SENTINEL_MASTER=mymaster
|
||||
# REDIS_SENTINEL_USERNAME=default
|
||||
# REDIS_PASSWORD=your-redis-password
|
||||
|
||||
# App Domain
|
||||
# This is the domain that your Sure instance will be hosted at. It is used to generate links in emails and other places.
|
||||
APP_DOMAIN=
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
### 0.0.0
|
||||
# Changelog
|
||||
|
||||
All notable changes to the Sure Helm chart will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
### [0.0.0]
|
||||
|
||||
- First (nightly/test) releases via <https://we-promise.github.io/sure/index.yaml>
|
||||
|
||||
### 0.6.5
|
||||
### [0.6.5]
|
||||
|
||||
- First version/release that aligns versions with monorepo
|
||||
- CNPG: render `Cluster.spec.backup` from `cnpg.cluster.backup`.
|
||||
@@ -10,3 +17,19 @@
|
||||
- For snapshot backups, `backup.volumeSnapshot.className` is required (template fails early if missing).
|
||||
- Example-only keys like `backup.ttl` and `backup.volumeSnapshot.enabled` are stripped to avoid CRD warnings.
|
||||
- CNPG: render `Cluster.spec.plugins` from `cnpg.cluster.plugins` (enables barman-cloud plugin / WAL archiver configuration).
|
||||
|
||||
## [0.6.7-alpha] - 2026-01-10
|
||||
|
||||
### Added
|
||||
- **Redis Sentinel support for Sidekiq high availability**: Application now automatically detects and configures Sidekiq to use Redis Sentinel when `redisOperator.mode=sentinel` and `redisOperator.sentinel.enabled=true`
|
||||
- New Helm template helpers (`sure.redisSentinelEnabled`, `sure.redisSentinelHosts`, `sure.redisSentinelMaster`) for Sentinel configuration detection
|
||||
- Automatic injection of `REDIS_SENTINEL_HOSTS` and `REDIS_SENTINEL_MASTER` environment variables when Sentinel mode is enabled
|
||||
- Sidekiq configuration supports Sentinel authentication with `sentinel_username` (defaults to "default") and `sentinel_password`
|
||||
- Robust validation of Sentinel endpoints with port range checking (1-65535) and graceful fallback to direct Redis URL on invalid configuration
|
||||
- Production-ready HA timeouts: 200ms connect, 1s read/write, 3 reconnection attempts
|
||||
- Backward compatible with existing `REDIS_URL` deployments
|
||||
|
||||
## Notes
|
||||
- Chart version and application version are kept in sync
|
||||
- Requires Kubernetes >= 1.25.0
|
||||
- When upgrading from pre-Sentinel configurations, existing deployments using `REDIS_URL` continue to work unchanged
|
||||
@@ -246,7 +246,11 @@ redisOperator:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
managed:
|
||||
enabled: true # render a RedisSentinel CR
|
||||
enabled: true # render Redis CRs for in-cluster Redis
|
||||
mode: sentinel # enables RedisSentinel CR in addition to RedisReplication
|
||||
sentinel:
|
||||
enabled: true # must be true when mode=sentinel
|
||||
masterGroupName: mymaster
|
||||
name: "" # defaults to <fullname>-redis
|
||||
replicas: 3
|
||||
auth:
|
||||
@@ -258,9 +262,14 @@ redisOperator:
|
||||
```
|
||||
|
||||
Notes:
|
||||
- When `redisOperator.mode=sentinel` and `redisOperator.sentinel.enabled=true`, the chart automatically configures Sidekiq to use Redis Sentinel for high availability.
|
||||
- The application receives `REDIS_SENTINEL_HOSTS` (comma-separated list of Sentinel endpoints) and `REDIS_SENTINEL_MASTER` (master group name) environment variables instead of `REDIS_URL`.
|
||||
- Sidekiq will connect to Sentinel nodes for automatic master discovery and failover support.
|
||||
- Both the Redis master and Sentinel nodes use the same password from `REDIS_PASSWORD` (via `redisOperator.auth.existingSecret`).
|
||||
- Sentinel authentication uses username "default" by default (configurable via `REDIS_SENTINEL_USERNAME`).
|
||||
- The operator master service is `<name>-redis-master.<ns>.svc.cluster.local:6379`.
|
||||
- The CR references your existing password secret via `kubernetesConfig.redisSecret { name, key }`.
|
||||
- Provider precedence for auto-wiring is: explicit `rails.extraEnv.REDIS_URL` → `redisOperator.managed` → `redisSimple`.
|
||||
- Provider precedence for auto-wiring is: explicit `rails.extraEnv.REDIS_URL` → `redisOperator.managed` (with Sentinel if configured) → `redisSimple`.
|
||||
- Only one in-cluster Redis provider should be enabled at a time to avoid ambiguity.
|
||||
|
||||
### HA scheduling and topology spreading
|
||||
|
||||
@@ -68,6 +68,13 @@ The helper always injects:
|
||||
key: {{ include "sure.redisPasswordKey" $ctx }}
|
||||
- name: REDIS_URL
|
||||
value: {{ $redis | quote }}
|
||||
{{- $sentinelHosts := include "sure.redisSentinelHosts" $ctx -}}
|
||||
{{- if $sentinelHosts }}
|
||||
- name: REDIS_SENTINEL_HOSTS
|
||||
value: {{ $sentinelHosts | quote }}
|
||||
- name: REDIS_SENTINEL_MASTER
|
||||
value: {{ include "sure.redisSentinelMaster" $ctx | quote }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- range $k, $v := $ctx.Values.rails.settings }}
|
||||
|
||||
@@ -76,6 +76,38 @@ app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/* Check if Redis Sentinel is enabled and configured */}}
|
||||
{{- define "sure.redisSentinelEnabled" -}}
|
||||
{{- if and .Values.redisOperator.managed.enabled .Values.redisOperator.sentinel.enabled (eq (.Values.redisOperator.mode | default "replication") "sentinel") -}}
|
||||
true
|
||||
{{- else -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/* Compute Redis Sentinel hosts (comma-separated list of host:port) */}}
|
||||
{{- define "sure.redisSentinelHosts" -}}
|
||||
{{- if eq (include "sure.redisSentinelEnabled" .) "true" -}}
|
||||
{{- $name := .Values.redisOperator.name | default (printf "%s-redis" (include "sure.fullname" .)) -}}
|
||||
{{- $replicas := .Values.redisOperator.replicas | default 3 -}}
|
||||
{{- $port := .Values.redisOperator.probes.sentinel.port | default 26379 -}}
|
||||
{{- $hosts := list -}}
|
||||
{{- range $i := until (int $replicas) -}}
|
||||
{{- $host := printf "%s-sentinel-%d.%s-sentinel-headless.%s.svc.cluster.local:%d" $name $i $name $.Release.Namespace (int $port) -}}
|
||||
{{- $hosts = append $hosts $host -}}
|
||||
{{- end -}}
|
||||
{{- join "," $hosts -}}
|
||||
{{- else -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/* Get Redis Sentinel master group name */}}
|
||||
{{- define "sure.redisSentinelMaster" -}}
|
||||
{{- if eq (include "sure.redisSentinelEnabled" .) "true" -}}
|
||||
{{- .Values.redisOperator.sentinel.masterGroupName | default "mymaster" -}}
|
||||
{{- else -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
|
||||
{{/* Common secret name helpers to avoid complex inline conditionals in env blocks */}}
|
||||
{{- define "sure.appSecretName" -}}
|
||||
|
||||
@@ -10,6 +10,60 @@ if Rails.env.production?
|
||||
end
|
||||
end
|
||||
|
||||
# Configure Redis connection for Sidekiq
|
||||
# Supports both Redis Sentinel (for HA) and direct Redis URL
|
||||
redis_config = if ENV["REDIS_SENTINEL_HOSTS"].present?
|
||||
# Redis Sentinel configuration for high availability
|
||||
# REDIS_SENTINEL_HOSTS should be comma-separated list: "host1:port1,host2:port2,host3:port3"
|
||||
sentinels = ENV["REDIS_SENTINEL_HOSTS"].split(",").filter_map do |host_port|
|
||||
parts = host_port.strip.split(":", 2)
|
||||
host = parts[0]&.strip
|
||||
port_str = parts[1]&.strip
|
||||
|
||||
next if host.blank?
|
||||
|
||||
# Parse port with validation, default to 26379 if invalid or missing
|
||||
port = if port_str.present?
|
||||
port_int = port_str.to_i
|
||||
(port_int > 0 && port_int <= 65535) ? port_int : 26379
|
||||
else
|
||||
26379
|
||||
end
|
||||
|
||||
{ host: host, port: port }
|
||||
end
|
||||
|
||||
if sentinels.empty?
|
||||
Rails.logger.warn("REDIS_SENTINEL_HOSTS is set but no valid sentinel hosts found, falling back to REDIS_URL")
|
||||
{ url: ENV.fetch("REDIS_URL", "redis://localhost:6379/0") }
|
||||
else
|
||||
{
|
||||
url: "redis://#{ENV.fetch('REDIS_SENTINEL_MASTER', 'mymaster')}/0",
|
||||
sentinels: sentinels,
|
||||
password: ENV["REDIS_PASSWORD"],
|
||||
sentinel_username: ENV.fetch("REDIS_SENTINEL_USERNAME", "default"),
|
||||
sentinel_password: ENV["REDIS_PASSWORD"],
|
||||
role: :master,
|
||||
# Recommended timeouts for Sentinel
|
||||
connect_timeout: 0.2,
|
||||
read_timeout: 1,
|
||||
write_timeout: 1,
|
||||
reconnect_attempts: 3
|
||||
}
|
||||
end
|
||||
else
|
||||
# Standard Redis URL configuration (no Sentinel)
|
||||
{ url: ENV.fetch("REDIS_URL", "redis://localhost:6379/0") }
|
||||
end
|
||||
|
||||
Sidekiq.configure_server do |config|
|
||||
config.redis = redis_config
|
||||
end
|
||||
|
||||
Sidekiq.configure_client do |config|
|
||||
config.redis = redis_config
|
||||
end
|
||||
|
||||
Sidekiq::Cron.configure do |config|
|
||||
# 10 min "catch-up" window in case worker process is re-deploying when cron tick occurs
|
||||
config.reschedule_grace_period = 600
|
||||
|
||||
Reference in New Issue
Block a user