API Gateway in Go: From Zero to Production

Table of Contents

1. Why We Built Our Own Gateway
2. Core Architecture
3. Rate Limiting Implementation
4. JWT Authentication
5. Load Balancing
6. Circuit Breaker Pattern
7. Request/Response Transformation
8. Performance Optimizations
9. Monitoring & Metrics
10. Security Considerations
11. Testing Strategy
12. Production Deployment
13. Lessons Learned

Why We Built Our Own Gateway

Kong wanted $50,000/year. AWS API Gateway was $3.50 per million requests (that's $1,275/month for us). Nginx+ was complex to configure and lacked observability we needed.

Requirements:

• Handle 1M+ requests per day
• Sub-10ms latency overhead
• Advanced rate limiting
• JWT authentication
• Load balancing with health checks
• Circuit breaking
• Request/response transformation
• Real-time metrics

Built it in 6 weeks. Running production for 18 months. Total cost: $200/month.

Core Architecture

The gateway follows a middleware chain pattern where each request passes through multiple layers: authentication, rate limiting, load balancing, and finally proxying to upstream services.

The main components work together to provide a robust, scalable solution:

• Router: Fast path matching using radix trees for O(log n) lookup performance
• Rate Limiter: Distributed sliding window algorithm using Redis for coordination
• Auth Manager: Pluggable authentication supporting JWT, API keys, and OAuth2
• Load Balancer: Multiple algorithms with active health checking
• Circuit Breaker: Fail-fast mechanism to prevent cascading failures
• Metrics: Real-time Prometheus metrics for observability

Each route configuration is independent, allowing different services to have different authentication, rate limiting, and load balancing strategies. This flexibility was crucial for our multi-tenant environment.

type Gateway struct {
    config        *Config
    router        *chi.Mux
    services      *ServiceRegistry
    rateLimiter   *RateLimiter
    auth          *AuthManager
    loadBalancer  *LoadBalancer
    circuitBreaker *CircuitBreaker
    metrics       *Metrics
}

HTTP Router and Middleware Chain

The router architecture emphasizes performance and flexibility. We chose Chi router for its zero-allocation routing and middleware composability. Dynamic route updates happen without restart, crucial for our microservices environment where services frequently change.

Connection pooling configuration was critical for performance. Initially, we had connection leaks causing memory issues. The final configuration balances resource usage with connection reuse, achieving 95% connection reuse rate.

Middleware order matters significantly. Request ID generation comes first for tracing, followed by real IP detection for accurate rate limiting, then metrics collection before any processing that might fail.

func (gw *Gateway) setupMiddleware() {
    // Core middleware chain - order matters!
    gw.router.Use(middleware.RequestID)
    gw.router.Use(middleware.RealIP)
    gw.router.Use(gw.loggingMiddleware)
    gw.router.Use(gw.metricsMiddleware)
    gw.router.Use(middleware.Recoverer)
    gw.router.Use(middleware.Timeout(60 * time.Second))
}

The admin API provides runtime configuration management. Initially, we restarted the gateway for config changes, causing brief downtime. The admin API enables zero-downtime deployments and A/B testing of routing rules.

Request Proxying Engine

The core of the gateway: routing requests to backend services.

func (gw *Gateway) proxyHandler(w http.ResponseWriter, r *http.Request) {
    start := time.Now()
    defer func() {
        gw.metrics.RequestDuration.Observe(time.Since(start).Seconds())
    }()
    // Find matching route
    route := gw.matchRoute(r)
    if route == nil {
        gw.metrics.RequestsTotal.WithLabelValues("404", "not_found").Inc()
        http.NotFound(w, r)
        return
    }
    // Apply middleware chain
    ctx := context.WithValue(r.Context(), "route", route)
    r = r.WithContext(ctx)
    // Rate limiting
    if route.RateLimit != nil {
        if err := gw.rateLimiter.Check(r, route.RateLimit); err != nil {
            gw.writeError(w, http.StatusTooManyRequests, "Rate limit exceeded")
            return
        }
    }
    // Authentication
    if route.Auth != nil {
        if err := gw.auth.Authenticate(r, route.Auth); err != nil {
            gw.writeError(w, http.StatusUnauthorized, "Authentication failed")
            return
        }
    }
    // Transform request if needed
    if route.Transform != nil {
        if err := gw.transformRequest(r, route.Transform); err != nil {
            gw.writeError(w, http.StatusBadRequest, "Request transformation failed")
            return
        }
    }
    // Select upstream
    upstream, err := gw.loadBalancer.SelectUpstream(route)
    if err != nil {
        gw.writeError(w, http.StatusServiceUnavailable, "No healthy upstream")
        return
    }
    // Circuit breaker check
    if !gw.circuitBreaker.Allow(upstream.URL) {
        gw.writeError(w, http.StatusServiceUnavailable, "Circuit breaker open")
        return
    }
    // Proxy the request
    gw.proxyRequest(w, r, route, upstream)
}
func (gw *Gateway) proxyRequest(w http.ResponseWriter, r *http.Request, route *Route, upstream *Upstream) {
    // Build upstream URL
    targetURL, err := gw.buildUpstreamURL(r, route, upstream)
    if err != nil {
        gw.writeError(w, http.StatusInternalServerError, "Invalid upstream URL")
        return
    }
    // Create proxy request
    proxyReq, err := http.NewRequestWithContext(r.Context(), r.Method, targetURL, r.Body)
    if err != nil {
        gw.writeError(w, http.StatusInternalServerError, "Failed to create proxy request")
        return
    }
    // Copy headers
    gw.copyHeaders(proxyReq.Header, r.Header)
    // Add/modify headers
    proxyReq.Header.Set("X-Forwarded-For", r.RemoteAddr)
    proxyReq.Header.Set("X-Forwarded-Proto", "https")
    proxyReq.Header.Set("X-Request-ID", middleware.GetReqID(r.Context()))
    // Execute request
    start := time.Now()
    resp, err := gw.httpClient.Do(proxyReq)
    duration := time.Since(start)
    // Record metrics
    gw.recordUpstreamMetrics(upstream.URL, duration, err)
    if err != nil {
        gw.circuitBreaker.RecordFailure(upstream.URL)
        gw.writeError(w, http.StatusBadGateway, "Upstream request failed")
        return
    }
    defer resp.Body.Close()
    gw.circuitBreaker.RecordSuccess(upstream.URL)
    // Transform response if needed
    if route.Transform != nil {
        if err := gw.transformResponse(resp, route.Transform); err != nil {
            gw.writeError(w, http.StatusInternalServerError, "Response transformation failed")
            return
        }
    }
    // Copy response
    gw.copyResponse(w, resp)
}

Rate Limiting Implementation

Rate limiting prevents API abuse and ensures fair resource usage. We chose a sliding window algorithm over fixed windows to avoid the "thundering herd" problem where clients hit the API exactly when windows reset.

The implementation uses Redis for distributed coordination across gateway instances. Each request timestamp is stored in a sorted set, allowing efficient cleanup of old entries and accurate counting within the time window.

Key generation strategy matters significantly. IP-based limiting works for public APIs, while user-based limiting is essential for authenticated endpoints. API key limiting prevents single clients from overwhelming the system.

type RateLimiter struct {
    redis  redis.UniversalClient
    script *redis.Script
}
type RateLimitConfig struct {
    Requests   int           `json:"requests"`
    Window     time.Duration `json:"window"`
    KeyBy      string        `json:"key_by"` // ip, user_id, api_key
    BurstSize  int           `json:"burst_size"`
}
func NewRateLimiter(redisClient redis.UniversalClient) *RateLimiter {
    // Lua script for atomic sliding window rate limiting
    script := redis.NewScript(`
        local key = KEYS[1]
        local window = tonumber(ARGV[1])
        local limit = tonumber(ARGV[2])
        local now = tonumber(ARGV[3])
        -- Remove old entries
        redis.call('ZREMRANGEBYSCORE', key, 0, now - window)
        -- Count current requests
        local current = redis.call('ZCARD', key)
        if current < limit then
            -- Add new request
            redis.call('ZADD', key, now, now)
            redis.call('EXPIRE', key, math.ceil(window / 1000))
            return {1, limit - current - 1}
        else
            return {0, 0}
        end
    `)
    return &RateLimiter{
        redis:  redisClient,
        script: script,
    }
}
func (rl *RateLimiter) Check(r *http.Request, config *RateLimitConfig) error {
    key := rl.generateKey(r, config)
    result, err := rl.script.Run(
        context.Background(),
        rl.redis,
        []string{key},
        config.Window.Milliseconds(),
        config.Requests,
        time.Now().UnixNano()/1e6,
    ).Result()
    if err != nil {
        // Fail open on Redis errors
        return nil
    }
    values := result.([]interface{})
    allowed := values[0].(int64)
    remaining := values[1].(int64)
    // Set rate limit headers
    w := r.Context().Value("response_writer").(http.ResponseWriter)
    w.Header().Set("X-RateLimit-Limit", strconv.Itoa(config.Requests))
    w.Header().Set("X-RateLimit-Remaining", strconv.FormatInt(remaining, 10))
    w.Header().Set("X-RateLimit-Reset", strconv.FormatInt(time.Now().Add(config.Window).Unix(), 10))
    if allowed == 0 {
        return fmt.Errorf("rate limit exceeded")
    }
    return nil
}
func (rl *RateLimiter) generateKey(r *http.Request, config *RateLimitConfig) string {
    var identifier string
    switch config.KeyBy {
    case "ip":
        identifier = r.RemoteAddr
    case "user_id":
        identifier = r.Header.Get("X-User-ID")
    case "api_key":
        identifier = r.Header.Get("X-API-Key")
    default:
        identifier = r.RemoteAddr
    }
    return fmt.Sprintf("rate_limit:%s:%s", config.KeyBy, identifier)
}

JWT Authentication

Authentication architecture supports multiple methods with pluggable providers. JWT validation uses RSA-256 signatures with key rotation support. We cache valid tokens for 5 minutes to reduce CPU overhead from signature verification.

Token validation includes expiry checks, issuer verification, and audience claims. The claims are injected into request context for downstream services to access user information without re-parsing tokens.

Error handling distinguishes between malformed tokens (400) and expired tokens (401) to help clients handle authentication failures appropriately. This improved our mobile app's token refresh logic significantly.

type AuthManager struct {
    jwtVerifier *JWTVerifier
    apiKeys     *APIKeyStore
    oauth       *OAuthVerifier
}
type AuthConfig struct {
    Type     string            `json:"type"` // jwt, api_key, oauth2
    Config   map[string]string `json:"config"`
    Required bool              `json:"required"`
}
func (am *AuthManager) Authenticate(r *http.Request, config *AuthConfig) error {
    switch config.Type {
    case "jwt":
        return am.verifyJWT(r, config)
    case "api_key":
        return am.verifyAPIKey(r, config)
    case "oauth2":
        return am.verifyOAuth(r, config)
    default:
        return fmt.Errorf("unknown auth type: %s", config.Type)
    }
}
func (am *AuthManager) verifyJWT(r *http.Request, config *AuthConfig) error {
    authHeader := r.Header.Get("Authorization")
    if authHeader == "" {
        return fmt.Errorf("missing authorization header")
    }
    if !strings.HasPrefix(authHeader, "Bearer ") {
        return fmt.Errorf("invalid authorization header format")
    }
    token := strings.TrimPrefix(authHeader, "Bearer ")
    claims, err := am.jwtVerifier.Verify(token)
    if err != nil {
        return fmt.Errorf("invalid token: %w", err)
    }
    // Add claims to request context
    ctx := context.WithValue(r.Context(), "user_claims", claims)
    *r = *r.WithContext(ctx)
    return nil
}
type JWTVerifier struct {
    publicKey interface{}
    issuer    string
    audience  string
}
func (jv *JWTVerifier) Verify(tokenString string) (jwt.MapClaims, error) {
    token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
        if _, ok := token.Method.(*jwt.SigningMethodRSA); !ok {
            return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
        }
        return jv.publicKey, nil
    })
    if err != nil {
        return nil, err
    }
    if !token.Valid {
        return nil, fmt.Errorf("invalid token")
    }
    claims, ok := token.Claims.(jwt.MapClaims)
    if !ok {
        return nil, fmt.Errorf("invalid claims")
    }
    // Verify issuer and audience
    if claims["iss"] != jv.issuer {
        return nil, fmt.Errorf("invalid issuer")
    }
    if claims["aud"] != jv.audience {
        return nil, fmt.Errorf("invalid audience")
    }
    return claims, nil
}

Load Balancing

Load balancing ensures even traffic distribution across healthy upstream services. We support three algorithms: round-robin for equal servers, weighted for heterogeneous capacity, and IP hash for session affinity.

Health checking runs every 10 seconds with configurable failure thresholds. Failed upstreams are automatically removed from rotation and re-added when healthy. This prevented cascading failures during service deployments.

The weighted round-robin implementation uses a deficit counter algorithm to ensure precise weight distribution over time, avoiding the "clumping" seen in naive weighted selection.

type LoadBalancer struct {
    mu         sync.RWMutex
    algorithms map[string]Algorithm
    health     *HealthChecker
}
type Algorithm interface {
    Select(upstreams []Upstream, r *http.Request) (*Upstream, error)
}
// Round-robin implementation
func (rr *RoundRobinAlgorithm) Select(upstreams []Upstream, r *http.Request) (*Upstream, error) {
    healthy := filterHealthy(upstreams)
    if len(healthy) == 0 {
        return nil, fmt.Errorf("no healthy upstreams")
    }
    rr.mu.Lock()
    current := rr.current[key] % len(healthy)
    rr.current[key] = current + 1
    rr.mu.Unlock()
    return &healthy[current], nil
}
// IP hash for session affinity
func (ih *IPHashAlgorithm) Select(upstreams []Upstream, r *http.Request) (*Upstream, error) {
    healthy := filterHealthy(upstreams)
    hash := fnv.New32a()
    hash.Write([]byte(r.RemoteAddr))
    index := hash.Sum32() % uint32(len(healthy))
    return &healthy[index], nil
}

Health Checking

Active health checks run every 10 seconds against configured endpoints. We learned that overly frequent checks (every second) created unnecessary load, while infrequent checks (every minute) led to slow failure detection.

Health check configuration supports custom paths, expected status codes, and timeout settings. Most services expose a simple /health endpoint, but some legacy services required checking specific business logic endpoints.

The health checker runs in a separate goroutine and updates upstream status atomically. This prevents race conditions between health updates and load balancing decisions that we experienced in early versions.

type HealthChecker struct {
    mu        sync.RWMutex
    checks    map[string]*HealthCheck
    client    *http.Client
    ticker    *time.Ticker
}
type HealthCheck struct {
    URL          string
    Method       string
    Path         string
    ExpectedCode int
    Timeout      time.Duration
    Interval     time.Duration
    // State
    Healthy      bool
    LastCheck    time.Time
    FailureCount int
    MaxFailures  int
}
func NewHealthChecker() *HealthChecker {
    hc := &HealthChecker{
        checks: make(map[string]*HealthCheck),
        client: &http.Client{
            Timeout: 5 * time.Second,
        },
        ticker: time.NewTicker(10 * time.Second),
    }
    go hc.runChecks()
    return hc
}
func (hc *HealthChecker) runChecks() {
    for range hc.ticker.C {
        hc.mu.RLock()
        checks := make([]*HealthCheck, 0, len(hc.checks))
        for _, check := range hc.checks {
            checks = append(checks, check)
        }
        hc.mu.RUnlock()
        // Run checks concurrently
        var wg sync.WaitGroup
        for _, check := range checks {
            wg.Add(1)
            go func(check *HealthCheck) {
                defer wg.Done()
                hc.performCheck(check)
            }(check)
        }
        wg.Wait()
    }
}
func (hc *HealthChecker) performCheck(check *HealthCheck) {
    url := check.URL + check.Path
    ctx, cancel := context.WithTimeout(context.Background(), check.Timeout)
    defer cancel()
    req, err := http.NewRequestWithContext(ctx, check.Method, url, nil)
    if err != nil {
        hc.recordFailure(check)
        return
    }
    resp, err := hc.client.Do(req)
    if err != nil {
        hc.recordFailure(check)
        return
    }
    defer resp.Body.Close()
    if resp.StatusCode == check.ExpectedCode {
        hc.recordSuccess(check)
    } else {
        hc.recordFailure(check)
    }
}
func (hc *HealthChecker) performCheck(check *HealthCheck) {
    resp, err := hc.client.Get(check.URL + check.Path)
    if err != nil || resp.StatusCode != check.ExpectedCode {
        hc.recordFailure(check)
        return
    }
    hc.recordSuccess(check)
}

Circuit Breaker Pattern

Circuit breakers prevent cascading failures when upstream services become unresponsive. The pattern has three states: Closed (normal operation), Open (failing fast), and Half-Open (testing recovery).

Our implementation tracks consecutive failures per upstream service. After 5 consecutive failures, the breaker opens for 30 seconds. During this time, requests fail immediately without hitting the troubled service.

The half-open state allows limited requests to test service recovery. If these succeed, the breaker closes; if they fail, it reopens. This gradual recovery prevents overwhelming recovering services.

Tuning circuit breaker sensitivity was critical. Too sensitive caused false positives during normal load spikes. Too lenient allowed cascading failures. We settled on 5 failures over 60 seconds after extensive testing.

const (
    StateClosed State = iota
    StateHalfOpen
    StateOpen
)
func (cb *CircuitBreaker) Allow(upstream string) bool {
    breaker := cb.getBreaker(upstream)
    switch breaker.state {
    case StateOpen:
        if time.Since(breaker.lastFailure) > breaker.timeout {
            breaker.state = StateHalfOpen
        } else {
            return false // Fail fast
        }
    case StateHalfOpen:
        if breaker.requests >= breaker.maxRequests {
            return false
        }
    }
    breaker.requests++
    return true
}
func (cb *CircuitBreaker) RecordResult(upstream string, success bool) {
    breaker := cb.getBreaker(upstream)
    if success {
        breaker.consecutiveFailures = 0
        if breaker.state == StateHalfOpen {
            breaker.state = StateClosed // Recovery successful
        }
    } else {
        breaker.consecutiveFailures++
        breaker.lastFailure = time.Now()
        if breaker.consecutiveFailures >= 5 {
            breaker.state = StateOpen // Trip the breaker
        }
    }
}

Monitoring and Metrics

Comprehensive monitoring with Prometheus metrics.

type Metrics struct {
    RequestsTotal      *prometheus.CounterVec
    RequestDuration    prometheus.Histogram
    UpstreamDuration   *prometheus.HistogramVec
    UpstreamErrors     *prometheus.CounterVec
    CircuitBreakerState *prometheus.GaugeVec
    RateLimitHits      *prometheus.CounterVec
}
func NewMetrics() *Metrics {
    return &Metrics{
        RequestsTotal: prometheus.NewCounterVec(
            prometheus.CounterOpts{
                Name: "gateway_requests_total",
                Help: "Total number of requests processed",
            },
            []string{"status", "route"},
        ),
        RequestDuration: prometheus.NewHistogram(
            prometheus.HistogramOpts{
                Name:    "gateway_request_duration_seconds",
                Help:    "Request duration in seconds",
                Buckets: prometheus.DefBuckets,
            },
        ),
        UpstreamDuration: prometheus.NewHistogramVec(
            prometheus.HistogramOpts{
                Name:    "gateway_upstream_duration_seconds",
                Help:    "Upstream request duration in seconds",
                Buckets: prometheus.DefBuckets,
            },
            []string{"upstream"},
        ),
        UpstreamErrors: prometheus.NewCounterVec(
            prometheus.CounterOpts{
                Name: "gateway_upstream_errors_total",
                Help: "Total number of upstream errors",
            },
            []string{"upstream", "error_type"},
        ),
        CircuitBreakerState: prometheus.NewGaugeVec(
            prometheus.GaugeOpts{
                Name: "gateway_circuit_breaker_state",
                Help: "Circuit breaker state (0=closed, 1=half-open, 2=open)",
            },
            []string{"upstream"},
        ),
        RateLimitHits: prometheus.NewCounterVec(
            prometheus.CounterOpts{
                Name: "gateway_rate_limit_hits_total",
                Help: "Total number of rate limit hits",
            },
            []string{"key_type"},
        ),
    }
}
func (gw *Gateway) metricsMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        ww := middleware.NewWrapResponseWriter(w, r.ProtoMajor)
        next.ServeHTTP(ww, r)
        duration := time.Since(start)
        status := strconv.Itoa(ww.Status())
        route := "unknown"
        if routeCtx := r.Context().Value("route"); routeCtx != nil {
            route = routeCtx.(*Route).ID
        }
        gw.metrics.RequestsTotal.WithLabelValues(status, route).Inc()
        gw.metrics.RequestDuration.Observe(duration.Seconds())
    })
}

Configuration Management

Dynamic configuration updates without restart.

type ConfigManager struct {
    mu       sync.RWMutex
    config   *Config
    routes   map[string]*Route
    watchers []ConfigWatcher
    storage  ConfigStorage
}
type ConfigWatcher interface {
    OnConfigChange(*Config)
    OnRouteChange(string, *Route)
    OnRouteDelete(string)
}
func (cm *ConfigManager) WatchConfig(ctx context.Context) {
    ticker := time.NewTicker(30 * time.Second)
    defer ticker.Stop()
    for {
        select {
        case <-ctx.Done():
            return
        case <-ticker.C:
            if err := cm.refreshConfig(); err != nil {
                log.Printf("Failed to refresh config: %v", err)
            }
        }
    }
}
func (cm *ConfigManager) UpdateRoute(route *Route) error {
    cm.mu.Lock()
    defer cm.mu.Unlock()
    // Validate route
    if err := cm.validateRoute(route); err != nil {
        return err
    }
    // Store in persistent storage
    if err := cm.storage.SaveRoute(route); err != nil {
        return err
    }
    // Update in-memory
    oldRoute := cm.routes[route.ID]
    cm.routes[route.ID] = route
    // Notify watchers
    for _, watcher := range cm.watchers {
        watcher.OnRouteChange(route.ID, route)
    }
    log.Printf("Route updated: %s -> %s", route.Path, route.Service)
    return nil
}
// Hot reload implementation
func (gw *Gateway) OnRouteChange(id string, route *Route) {
    gw.router.HandleFunc(route.Path, func(w http.ResponseWriter, r *http.Request) {
        gw.proxyWithRoute(w, r, route)
    })
    log.Printf("Route reloaded: %s", route.Path)
}

Production Metrics

After 18 months in production, here's what we learned:

• 1.2M requests/day at peak
• 8ms median latency overhead
• 99.9% uptime (only 43 minutes downtime/month)
• 300+ routes across 50 services
• 95% cache hit rate for auth validation
• $200/month total infrastructure cost

Performance Optimizations

1. Connection pooling: Reduced latency by 40%
2. Response streaming: Handle large responses without memory issues
3. JWT caching: Cache valid tokens for 5 minutes
4. Route caching: In-memory route lookup with LRU eviction
5. Health check optimization: Shared health status across instances

Lessons Learned

• Fail-safe defaults: When Redis is down, bypass rate limiting
• Circuit breaker tuning: Too sensitive causes false positives
• Health check frequency: 10 seconds is optimal balance
• Connection pooling is critical: Don't create new connections per request
• Monitor everything: Latency, error rates, circuit breaker states

Docker and Deployment

# Multi-stage build for small image
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o gateway ./cmd/gateway
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /app/gateway .
COPY --from=builder /app/config.yaml .
EXPOSE 8080
CMD ["./gateway"]

Security Considerations

1. Authentication & Authorization

// Secure JWT validation with key rotation
type JWTValidator struct {
    keys        map[string]*rsa.PublicKey
    keyRotation time.Duration
    mu          sync.RWMutex
}
func (v *JWTValidator) ValidateToken(tokenString string) (*Claims, error) {
    token, err := jwt.ParseWithClaims(tokenString, &Claims{}, func(token *jwt.Token) (interface{}, error) {
        if _, ok := token.Method.(*jwt.SigningMethodRSA); !ok {
            return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
        }
        kid, ok := token.Header["kid"].(string)
        if !ok {
            return nil, fmt.Errorf("missing key ID")
        }
        v.mu.RLock()
        key, exists := v.keys[kid]
        v.mu.RUnlock()
        if !exists {
            return nil, fmt.Errorf("unknown key ID: %s", kid)
        }
        return key, nil
    })
    if err != nil {
        return nil, fmt.Errorf("token validation failed: %w", err)
    }
    claims, ok := token.Claims.(*Claims)
    if !ok || !token.Valid {
        return nil, fmt.Errorf("invalid token claims")
    }
    // Additional security checks
    if time.Now().Unix() > claims.ExpiresAt {
        return nil, fmt.Errorf("token expired")
    }
    if claims.Issuer != expectedIssuer {
        return nil, fmt.Errorf("invalid issuer")
    }
    return claims, nil
}

2. Input Validation & Sanitization

// Prevent injection attacks
func (gw *Gateway) validateRequest(r *http.Request) error {
    // Check request size
    if r.ContentLength > maxRequestSize {
        return fmt.Errorf("request too large: %d bytes", r.ContentLength)
    }
    // Validate headers
    for name, values := range r.Header {
        if len(name) > maxHeaderNameLength {
            return fmt.Errorf("header name too long: %s", name)
        }
        for _, value := range values {
            if len(value) > maxHeaderValueLength {
                return fmt.Errorf("header value too long for %s", name)
            }
            // Check for injection attempts
            if containsSQLInjection(value) || containsXSS(value) {
                return fmt.Errorf("potential injection in header %s", name)
            }
        }
    }
    // Validate path
    if !isValidPath(r.URL.Path) {
        return fmt.Errorf("invalid path: %s", r.URL.Path)
    }
    return nil
}

3. DDoS Protection

• Rate limiting per IP, user, and API key
• Connection limits and timeouts
• Request size limits
• Graceful degradation under load
• IP blacklisting for repeat offenders

4. Secure Communication

// TLS configuration with modern ciphers
tlsConfig := &tls.Config{
    MinVersion: tls.VersionTLS12,
    CipherSuites: []uint16{
        tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
        tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
        tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
        tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
    },
    PreferServerCipherSuites: true,
    CurvePreferences: []tls.CurveID{
        tls.X25519,
        tls.CurveP256,
    },
}

Testing Strategy

Testing an API Gateway requires comprehensive coverage of routing, middleware, and edge cases.

1. Unit Testing Core Components

func TestRateLimiter(t *testing.T) {
    limiter := NewRateLimiter(10, time.Second) // 10 req/sec
    // Should allow 10 requests
    for i := 0; i < 10; i++ {
        if !limiter.Allow("test-key") {
            t.Errorf("Request %d should be allowed", i+1)
        }
    }
    // 11th request should be rejected
    if limiter.Allow("test-key") {
        t.Error("11th request should be rejected")
    }
    // After 1 second, should allow again
    time.Sleep(time.Second)
    if !limiter.Allow("test-key") {
        t.Error("Request should be allowed after window reset")
    }
}

2. Integration Testing

func TestGatewayIntegration(t *testing.T) {
    // Start mock upstream services
    upstream1 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"service":"upstream1"}`))
    }))
    defer upstream1.Close()
    // Configure gateway
    config := &Config{
        Routes: []Route{
            {
                ID:       "test-route",
                Path:     "/api/test",
                Upstream: upstream1.URL,
                Methods:  []string{"GET"},
            },
        },
    }
    gateway := NewGateway(config)
    server := httptest.NewServer(gateway)
    defer server.Close()
    // Test routing
    resp, err := http.Get(server.URL + "/api/test")
    require.NoError(t, err)
    assert.Equal(t, http.StatusOK, resp.StatusCode)
    body, _ := ioutil.ReadAll(resp.Body)
    assert.Contains(t, string(body), "upstream1")
}

3. Load Testing

func TestGatewayUnderLoad(t *testing.T) {
    gateway := setupTestGateway(t)
    server := httptest.NewServer(gateway)
    defer server.Close()
    // Concurrent requests
    concurrency := 100
    requests := 10000
    var wg sync.WaitGroup
    errors := make(chan error, requests)
    latencies := make(chan time.Duration, requests)
    for i := 0; i < concurrency; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            client := &http.Client{Timeout: 5 * time.Second}
            for j := 0; j < requests/concurrency; j++ {
                start := time.Now()
                resp, err := client.Get(server.URL + "/api/test")
                latency := time.Since(start)
                if err != nil {
                    errors <- err
                    continue
                }
                if resp.StatusCode != http.StatusOK {
                    errors <- fmt.Errorf("unexpected status: %d", resp.StatusCode)
                }
                latencies <- latency
                resp.Body.Close()
            }
        }()
    }
    wg.Wait()
    close(errors)
    close(latencies)
    // Analyze results
    var errorCount int
    for err := range errors {
        t.Logf("Error: %v", err)
        errorCount++
    }
    var totalLatency time.Duration
    var maxLatency time.Duration
    count := 0
    for latency := range latencies {
        totalLatency += latency
        if latency > maxLatency {
            maxLatency = latency
        }
        count++
    }
    avgLatency := totalLatency / time.Duration(count)
    // Assertions
    assert.Less(t, float64(errorCount)/float64(requests), 0.01) // <1% error rate
    assert.Less(t, avgLatency, 50*time.Millisecond) // <50ms avg latency
    assert.Less(t, maxLatency, 500*time.Millisecond) // <500ms max latency
}

4. Chaos Testing

func TestCircuitBreakerWithFailures(t *testing.T) {
    failureRate := 0.5 // 50% failure rate
    // Mock unreliable upstream
    upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if rand.Float64() < failureRate {
            w.WriteHeader(http.StatusInternalServerError)
            return
        }
        w.WriteHeader(http.StatusOK)
    }))
    defer upstream.Close()
    gateway := NewGateway(&Config{
        CircuitBreaker: CircuitBreakerConfig{
            Threshold:   5,
            Timeout:     time.Second,
            MaxRequests: 10,
        },
    })
    // Should trip circuit breaker after failures
    var tripCount int
    for i := 0; i < 100; i++ {
        resp := gateway.ProxyRequest(upstream.URL)
        if resp.StatusCode == http.StatusServiceUnavailable {
            tripCount++
        }
    }
    assert.Greater(t, tripCount, 0, "Circuit breaker should trip")
}

Conclusion

Building a production API Gateway in Go isn't trivial, but it's entirely feasible. Our implementation saved us $60,000/year compared to commercial solutions and gives us complete control over the system.

Key takeaways:

• Start with basic proxying, add features incrementally
• Connection pooling and caching are critical for performance
• Circuit breakers prevent cascading failures
• Comprehensive monitoring is essential
• Hot reloading enables zero-downtime deployments

The complete gateway is ~5,000 lines of Go code and handles all our traffic routing needs. Sometimes building your own infrastructure makes sense when you understand the requirements and have the team to maintain it.