Kubernetes Deployment

Enterprise

Zephyr's Kubernetes deployment option allows you to run the Zephyr edge worker on your own Kubernetes infrastructure. This deployment model provides maximum control over your infrastructure while leveraging Zephyr's deployment and versioning capabilities.

Enterprise Only

Kubernetes deployment is available exclusively for Enterprise customers. If you're interested in deploying Zephyr on your own K8S infrastructure, please contact our sales team to discuss your requirements.

Overview

The Kubernetes edge worker (ze-k8s-worker) is a containerized service that handles asset uploads and serves your deployed applications. It integrates with:

  • S3-compatible storage (AWS S3, MinIO, Ceph, etc.) for asset storage
  • Redis-compatible KV store for environment configuration and snapshots

Configuration

The recommended way to configure the worker is using a JSON config file mounted as a Kubernetes Secret. This approach keeps all your configuration in one place and makes it easier to manage secrets securely.

Alternatively, you can use environment variables directly, though this is less recommended for production deployments.

Create a JSON file containing all your configuration. All keys use the ZE_WORKER_ prefix:

{
  "ZE_WORKER_JWT_SECRET": "your-secret-key",
  "ZE_WORKER_S3_ENDPOINT": "https://s3.amazonaws.com",
  "ZE_WORKER_S3_REGION": "us-east-1",
  "ZE_WORKER_S3_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE",
  "ZE_WORKER_S3_SECRET_ACCESS_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "ZE_WORKER_S3_BUCKET": "zephyr-assets",
  "ZE_WORKER_S3_FORCE_PATH_STYLE": "false",
  "ZE_WORKER_REDIS_HOST": "redis.default.svc.cluster.local",
  "ZE_WORKER_REDIS_PORT": "6379",
  "ZE_WORKER_REDIS_PASSWORD": "your-redis-password",
  "ZE_WORKER_REDIS_TLS": "true",
  "ZE_WORKER_PORT": "8080",
  "ZE_WORKER_LOG_LEVEL": "info",
  "ZE_WORKER_DELIMITER": "-"
}

Deploying with Kubernetes Secret

Mount the config file as a Secret for secure configuration:

apiVersion: v1
kind: Secret
metadata:
  name: ze-worker-config
type: Opaque
stringData:
  config.json: |
    {
      "ZE_WORKER_JWT_SECRET": "your-production-secret",
      "ZE_WORKER_S3_ENDPOINT": "https://s3.us-west-2.amazonaws.com",
      "ZE_WORKER_S3_REGION": "us-west-2",
      "ZE_WORKER_S3_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE",
      "ZE_WORKER_S3_SECRET_ACCESS_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
      "ZE_WORKER_S3_BUCKET": "zephyr-production",
      "ZE_WORKER_S3_FORCE_PATH_STYLE": "false",
      "ZE_WORKER_REDIS_HOST": "redis-cluster.example.com",
      "ZE_WORKER_REDIS_PORT": "6380",
      "ZE_WORKER_REDIS_PASSWORD": "your-redis-password",
      "ZE_WORKER_REDIS_TLS": "true",
      "ZE_WORKER_PORT": "8080",
      "ZE_WORKER_LOG_LEVEL": "info",
      "ZE_WORKER_DELIMITER": "-"
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ze-k8s-worker
spec:
  replicas: 2
  selector:
    matchLabels:
      app: ze-k8s-worker
  template:
    metadata:
      labels:
        app: ze-k8s-worker
    spec:
      containers:
        - name: worker
          image: ze-k8s-worker:latest
          ports:
            - containerPort: 8080
          env:
            - name: ZE_WORKER_CONFIG_JSON
              value: /etc/ze-worker/config.json
          volumeMounts:
            - name: config
              mountPath: /etc/ze-worker
              readOnly: true
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /readyz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
      volumes:
        - name: config
          secret:
            secretName: ze-worker-config
---
apiVersion: v1
kind: Service
metadata:
  name: ze-k8s-worker
spec:
  selector:
    app: ze-k8s-worker
  ports:
    - port: 80
      targetPort: 8080

Development Configuration (MinIO + Redis)

For local development or testing with MinIO:

{
  "ZE_WORKER_JWT_SECRET": "dev-secret-key",
  "ZE_WORKER_S3_ENDPOINT": "http://minio:9000",
  "ZE_WORKER_S3_REGION": "us-east-1",
  "ZE_WORKER_S3_ACCESS_KEY_ID": "minioadmin",
  "ZE_WORKER_S3_SECRET_ACCESS_KEY": "minioadmin",
  "ZE_WORKER_S3_BUCKET": "zephyr-dev",
  "ZE_WORKER_S3_FORCE_PATH_STYLE": "true",
  "ZE_WORKER_REDIS_HOST": "redis",
  "ZE_WORKER_REDIS_PORT": "6379",
  "ZE_WORKER_REDIS_TLS": "false",
  "ZE_WORKER_PORT": "8080",
  "ZE_WORKER_LOG_LEVEL": "debug",
  "ZE_WORKER_DELIMITER": "-"
}
TIP

When using MinIO or other S3-compatible storage, set ZE_WORKER_S3_FORCE_PATH_STYLE to "true".

Health Check Endpoints

The worker exposes the following health check endpoints for Kubernetes probes:

EndpointTypeDescription
GET /healthzLiveness probeAlways returns 200 if the process is running
GET /readyzReadiness probeChecks S3 and Redis connectivity; returns 200 if healthy, 503 if not

Configuration Reference

Below is the complete reference for all available configuration options. These can be set in your JSON config file or as environment variables.

Required Options

KeyDescriptionExample
ZE_WORKER_JWT_SECRETSecret key for JWT token validationyour-secret-key-here
ZE_WORKER_S3_ENDPOINTS3-compatible storage endpoint URLhttps://s3.amazonaws.com or http://minio:9000
ZE_WORKER_S3_ACCESS_KEY_IDS3 access key IDAKIAIOSFODNN7EXAMPLE
ZE_WORKER_S3_SECRET_ACCESS_KEYS3 secret access keywJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
ZE_WORKER_S3_BUCKETS3 bucket name for storing assetszephyr-assets
ZE_WORKER_REDIS_HOSTRedis-compatible KV store hostnameredis.default.svc.cluster.local

Server Options

KeyDescriptionDefaultExample
ZE_WORKER_PORTHTTP server port80803000
ZE_WORKER_LOG_LEVELLogging levelinfodebug, trace, warn, error, fatal
ZE_WORKER_DELIMITERDelimiter for hostname parsing-. or _

S3 Options

KeyDescriptionDefaultExample
ZE_WORKER_S3_REGIONS3 regionus-east-1eu-west-1
ZE_WORKER_S3_FORCE_PATH_STYLEUse path-style URLs (required for MinIO/Ceph)truefalse

Redis Options

KeyDescriptionDefaultExample
ZE_WORKER_REDIS_PORTRedis port63796380
ZE_WORKER_REDIS_PASSWORDRedis password (if required)(empty)your-redis-password
ZE_WORKER_REDIS_TLSEnable TLS for Redis connectionfalsetrue
ZE_WORKER_REDIS_DBRedis database number01, 2, etc.
ZE_WORKER_REDIS_PREFIX_ENVSKey prefix for environment variables{ze-k8s-worker}:envs:{myapp}:envs:
ZE_WORKER_REDIS_PREFIX_SNAPSHOTSKey prefix for snapshots{ze-k8s-worker}:snapshots:{myapp}:snapshots:
Redis Key Prefixes

The curly braces {} in the default prefixes enable Redis Cluster hash tags. This ensures all keys with the same hash tag are stored on the same cluster node, enabling multi-key operations, transactions, and better performance. For standalone Redis instances, the braces have no special meaning but don't cause issues.

Using Environment Variables

While the JSON config file is recommended, you can also configure the worker using environment variables directly. This can be useful for simple setups or when integrating with external secret management systems.

Set ZE_WORKER_CONFIG_JSON to the path of your config file. If both are provided, values in the JSON file take precedence over environment variables.

env:
  - name: ZE_WORKER_JWT_SECRET
    valueFrom:
      secretKeyRef:
        name: ze-worker-secrets
        key: jwt-secret
  - name: ZE_WORKER_S3_ENDPOINT
    value: 'https://s3.us-west-2.amazonaws.com'
  - name: ZE_WORKER_S3_ACCESS_KEY_ID
    valueFrom:
      secretKeyRef:
        name: ze-worker-secrets
        key: s3-access-key
  # ... additional environment variables

Troubleshooting

Common Issues

Worker fails to start

  1. Verify all required configuration options are set
  2. Check that S3 endpoint is reachable from within the cluster
  3. Ensure Redis host is resolvable and accessible
  4. Check the worker logs: kubectl logs -l app=ze-k8s-worker

Readiness probe failing

  1. Check S3 bucket exists and credentials have read/write access
  2. Verify Redis connection (host, port, password, TLS settings)
  3. Review worker logs for specific error messages

Assets not serving correctly

  1. Confirm ZE_WORKER_DELIMITER matches your DNS configuration
  2. Verify S3 bucket policy allows the worker to read objects
  3. Check that the Ingress or load balancer is routing traffic correctly

Getting Help

If you encounter issues with your Kubernetes deployment, reach out to your Zephyr account representative or contact us on Discord.