Building a Plugin System in Go: Make Your Monolith Extensible Without Microservices

Table of Contents

1. Why Not Go's plugin Package?
2. A Practical Architecture
3. RPC-Based Plugins (Default Pattern)
4. The Plugin Side
5. Sandboxing
6. WebAssembly Plugins
7. Embedded Scripting
8. Plugin Discovery and Hot Reload
9. Plugin API Gateway
10. Plugin Marketplace
11. Production Monitoring
12. Security Considerations
13. Testing Strategy
14. Operational Expectations

Why Not Just Use Go's plugin Package?

Go's built-in plugin package looks cool on paper, but in real apps it just falls apart:

• Only works on Linux/macOS
• Can't unload plugins
• Version conflicts are a nightmare
• One bad plugin crashes everything

Most teams just need something boring that boots in prod today. The rest of this guide walks through a practical setup you can copy-paste and tweak.

A Practical Architecture

You usually mix one of three approaches, depending on how wild the custom logic gets:

1. RPC-based plugins (most flexible)
2. WebAssembly plugins (for untrusted code)
3. Embedded scripting (for simple logic)

RPC-Based Plugins: The Default Pattern

Run every plugin as its own process and talk to it over RPC. That way when someone ships a crashing plugin, the main app just shrugs. Each plugin keeps its own memory and can be written in whatever language the customer is stubborn about.

Process isolation also means resource limits. Throw cgroups, containers, or Windows job objects at the process, set CPU/RAM ceilings, and a runaway loop only nukes its own sandbox.

The manager is just a babysitter: start processes, wire up RPC, ping for health, and restart the moment a heartbeat dies. No magic, just scripts that refuse to babysit dead code for long.

Local RPC hop adds roughly 50–100 microseconds. That overhead is basically noise compared to having to reboot the whole app when a plugin touches nil.

Versioning still matters. Every plugin should declare the host API version it expects. On load, the manager checks against a compatibility table (maybe host 2.0 accepts plugins from 1.8+). That breathing room lets customers upgrade on their own weird schedule.

stdin/stdout pipes plus `net/rpc` are enough. Because the protocol is literally JSON over pipes, teams regularly write plugins in Python, Node.js, Rust, or whatever other stack is lying around.

type Manager struct {
    plugins   map[string]*ManagedPlugin
    maxMemory int64    // Resource limits
    maxCPU    float64  // Prevent runaway plugins
}
func (m *Manager) Load(path string) error {
    // Start plugin process with resource limits
    cmd := exec.Command(path)
    cmd.Env = append(cmd.Env, "GOMEMLIMIT=100MiB")
    // Connect via RPC over stdin/stdout
    stdin, _ := cmd.StdinPipe()
    stdout, _ := cmd.StdoutPipe()
    cmd.Start()
    client := rpc.NewClient(stdout, stdin)
    // Validate plugin and start monitoring
    var info PluginInfo
    client.Call("Plugin.Info", &info)
    go m.monitorHealth(plugin)
    return nil
}

The Plugin Side: What Developers Write

package main
import (
    "context"
    "net/rpc"
    "os"
)
type MyPlugin struct{}
func (p *MyPlugin) Info(_ struct{}, reply *PluginInfo) error {
    *reply = PluginInfo{
        Name:    "customer-validator",
        Version: "1.0.0",
        Author:  "ACME Corp",
    }
    return nil
}
func (p *MyPlugin) Execute(args ExecuteArgs, reply *ExecuteReply) error {
    // Plugin logic here
    ctx := context.Background()
    // Example: Custom validation
    data := args.Input.(map[string]interface{})
    if email, ok := data["email"].(string); ok {
        if !strings.HasSuffix(email, "@acme.com") {
            reply.Error = "Only ACME emails allowed"
            return nil
        }
    }
    reply.Output = data
    reply.Success = true
    return nil
}
func main() {
    plugin := &MyPlugin{}
    server := rpc.NewServer()
    server.Register(plugin)
    // Serve RPC over stdin/stdout
    server.ServeConn(struct {
        io.Reader
        io.Writer
        io.Closer
    }{
        Reader: os.Stdin,
        Writer: os.Stdout,
        Closer: os.Stdin,
    })
}

Sandboxing: Because Customers Write Terrible Code

func (m *Manager) Execute(name string, input interface{}) (interface{}, error) {
    plugin, exists := m.getPlugin(name)
    if !exists {
        return nil, fmt.Errorf("plugin not found: %s", name)
    }
    // Check health
    if !plugin.healthy {
        return nil, fmt.Errorf("plugin unhealthy: %s", name)
    }
    // Create timeout context
    ctx, cancel := context.WithTimeout(context.Background(), m.timeout)
    defer cancel()
    // Execute with resource monitoring
    done := make(chan struct{})
    var result ExecuteReply
    var err error
    go func() {
        start := time.Now()
        err = plugin.client.Call("Plugin.Execute", ExecuteArgs{
            Input: input,
        }, &result)
        // Update metrics
        atomic.AddUint64(&plugin.calls, 1)
        plugin.totalTime += time.Since(start)
        if err != nil {
            atomic.AddUint64(&plugin.errors, 1)
        }
        close(done)
    }()
    select {
    case <-done:
        if err != nil {
            return nil, err
        }
        return result.Output, nil
    case <-ctx.Done():
        // Plugin took too long, kill it
        m.killPlugin(plugin)
        return nil, fmt.Errorf("plugin timeout: %s", name)
    }
}
func (m *Manager) killPlugin(plugin *ManagedPlugin) {
    plugin.healthy = false
    // Try graceful shutdown first
    plugin.client.Close()
    // Give it 1 second to die
    done := make(chan struct{})
    go func() {
        plugin.cmd.Wait()
        close(done)
    }()
    select {
    case <-done:
        // Graceful shutdown worked
    case <-time.After(time.Second):
        // Force kill
        plugin.cmd.Process.Kill()
    }
    // Remove from registry
    m.mu.Lock()
    for name, p := range m.plugins {
        if p == plugin {
            delete(m.plugins, name)
            break
        }
    }
    m.mu.Unlock()
}

WebAssembly Plugins: For Untrusted Code

package wasm
import (
    "context"
    "github.com/tetratelabs/wazero"
    "github.com/tetratelabs/wazero/api"
)
type WasmPlugin struct {
    runtime wazero.Runtime
    module  api.Module
    // Exported functions
    execute api.Function
}
func LoadWasmPlugin(wasmBytes []byte) (*WasmPlugin, error) {
    ctx := context.Background()
    // Create runtime with limits
    r := wazero.NewRuntimeWithConfig(ctx, wazero.NewRuntimeConfig().
        WithMemoryLimitPages(100)) // 6.4MB max memory
    // Import host functions
    hostBuilder := r.NewHostModuleBuilder("env")
    hostBuilder.NewFunctionBuilder().
        WithFunc(func(ctx context.Context, offset, length uint32) uint32 {
            // Read string from plugin memory
            buf, _ := module.Memory().Read(offset, length)
            log.Printf("Plugin log: %s", string(buf))
            return 0
        }).
        Export("log_message")
    hostBuilder.Instantiate(ctx)
    // Load and instantiate module
    module, err := r.Instantiate(ctx, wasmBytes)
    if err != nil {
        return nil, err
    }
    // Get exported functions
    execute := module.ExportedFunction("execute")
    if execute == nil {
        return nil, fmt.Errorf("plugin missing execute function")
    }
    return &WasmPlugin{
        runtime: r,
        module:  module,
        execute: execute,
    }, nil
}
func (p *WasmPlugin) Execute(input []byte) ([]byte, error) {
    ctx := context.Background()
    // Allocate memory in WASM for input
    alloc := p.module.ExportedFunction("malloc")
    inputPtr, _ := alloc.Call(ctx, uint64(len(input)))
    // Write input to WASM memory
    p.module.Memory().Write(uint32(inputPtr[0]), input)
    // Call execute function
    results, err := p.execute.Call(ctx, inputPtr[0], uint64(len(input)))
    if err != nil {
        return nil, err
    }
    // Read result from WASM memory
    resultPtr := uint32(results[0])
    resultLen := uint32(results[1])
    output, _ := p.module.Memory().Read(resultPtr, resultLen)
    // Free memory
    free := p.module.ExportedFunction("free")
    free.Call(ctx, uint64(resultPtr))
    return output, nil
}

Embedded Scripting: For Simple Logic

package scripting
import (
    "github.com/dop251/goja"
    "time"
)
type JSPlugin struct {
    vm      *goja.Runtime
    script  string
    // Resource limits
    timeout time.Duration
}
func NewJSPlugin(script string) *JSPlugin {
    vm := goja.New()
    // Add safe APIs
    vm.Set("log", func(msg string) {
        log.Printf("Plugin: %s", msg)
    })
    vm.Set("fetch", func(url string) (map[string]interface{}, error) {
        // Controlled HTTP client
        ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
        defer cancel()
        req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        defer resp.Body.Close()
        var result map[string]interface{}
        json.NewDecoder(resp.Body).Decode(&result)
        return result, nil
    })
    // Remove dangerous functions
    vm.Set("eval", goja.Undefined())
    vm.Set("Function", goja.Undefined())
    return &JSPlugin{
        vm:      vm,
        script:  script,
        timeout: 100 * time.Millisecond,
    }
}
func (p *JSPlugin) Execute(input interface{}) (interface{}, error) {
    // Set input
    p.vm.Set("input", input)
    // Run with timeout
    done := make(chan struct{})
    var result goja.Value
    var err error
    go func() {
        result, err = p.vm.RunString(p.script)
        close(done)
    }()
    select {
    case <-done:
        if err != nil {
            return nil, err
        }
        return result.Export(), nil
    case <-time.After(p.timeout):
        p.vm.Interrupt("timeout")
        return nil, fmt.Errorf("script timeout")
    }
}

Plugin Discovery and Hot Reload

type PluginRegistry struct {
    manager  *Manager
    dir      string
    // File watching
    watcher  *fsnotify.Watcher
    // Version management
    versions map[string][]PluginVersion
}
func (r *PluginRegistry) Start() error {
    // Initial scan
    r.scanDirectory()
    // Watch for changes
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        return err
    }
    r.watcher = watcher
    watcher.Add(r.dir)
    go r.watchLoop()
    return nil
}
func (r *PluginRegistry) watchLoop() {
    for {
        select {
        case event := <-r.watcher.Events:
            if event.Op&fsnotify.Create == fsnotify.Create {
                r.handleNewPlugin(event.Name)
            } else if event.Op&fsnotify.Write == fsnotify.Write {
                r.handlePluginUpdate(event.Name)
            } else if event.Op&fsnotify.Remove == fsnotify.Remove {
                r.handlePluginRemoval(event.Name)
            }
        case err := <-r.watcher.Errors:
            log.Printf("Watcher error: %v", err)
        }
    }
}
func (r *PluginRegistry) handlePluginUpdate(path string) {
    // Load new version
    tempName := fmt.Sprintf("%s_new_%d", filepath.Base(path), time.Now().Unix())
    if err := r.manager.Load(path); err != nil {
        log.Printf("Failed to load plugin update: %v", err)
        return
    }
    // Graceful transition
    oldPlugin := r.manager.plugins[getPluginName(path)]
    // Start routing new requests to new version
    r.manager.mu.Lock()
    r.manager.plugins[getPluginName(path)] = newPlugin
    r.manager.mu.Unlock()
    // Wait for old plugin to finish current requests
    time.Sleep(10 * time.Second)
    // Kill old version
    if oldPlugin != nil {
        r.manager.killPlugin(oldPlugin)
    }
    log.Printf("Plugin %s updated successfully", getPluginName(path))
}

Plugin API Gateway

type PluginGateway struct {
    manager *Manager
    // Rate limiting per plugin
    limiters map[string]*rate.Limiter
    // Circuit breakers
    breakers map[string]*CircuitBreaker
}
func (g *PluginGateway) Call(name string, input interface{}) (interface{}, error) {
    // Rate limiting
    limiter := g.getLimiter(name)
    if !limiter.Allow() {
        return nil, fmt.Errorf("rate limit exceeded for plugin %s", name)
    }
    // Circuit breaker
    breaker := g.getBreaker(name)
    return breaker.Execute(func() (interface{}, error) {
        return g.manager.Execute(name, input)
    })
}
func (g *PluginGateway) getBreaker(name string) *CircuitBreaker {
    g.mu.Lock()
    defer g.mu.Unlock()
    if breaker, exists := g.breakers[name]; exists {
        return breaker
    }
    breaker := NewCircuitBreaker(CircuitBreakerConfig{
        MaxFailures:     5,
        ResetTimeout:    30 * time.Second,
        HalfOpenSuccess: 2,
    })
    g.breakers[name] = breaker
    return breaker
}

Plugin Marketplace

type PluginStore struct {
    db       *sql.DB
    storage  StorageBackend
    verifier *PluginVerifier
}
type PluginMetadata struct {
    ID          string
    Name        string
    Version     string
    Author      string
    Description string
    Downloads   int64
    Rating      float64
    Verified    bool
    Hash        string
}
func (s *PluginStore) Publish(plugin []byte, metadata PluginMetadata) error {
    // Verify plugin safety
    report, err := s.verifier.Verify(plugin)
    if err != nil {
        return fmt.Errorf("verification failed: %w", err)
    }
    if !report.Safe {
        return fmt.Errorf("plugin contains unsafe operations: %v", report.Issues)
    }
    // Calculate hash
    hash := sha256.Sum256(plugin)
    metadata.Hash = hex.EncodeToString(hash[:])
    // Store plugin binary
    path := fmt.Sprintf("plugins/%s/%s/%s", metadata.Author, metadata.Name, metadata.Version)
    if err := s.storage.Put(path, plugin); err != nil {
        return err
    }
    // Store metadata
    _, err = s.db.Exec(`
        INSERT INTO plugins (id, name, version, author, description, hash, verified)
        VALUES ($1, $2, $3, $4, $5, $6, $7)
    `, metadata.ID, metadata.Name, metadata.Version, metadata.Author,
       metadata.Description, metadata.Hash, report.Safe)
    return err
}
type PluginVerifier struct {
    rules []VerificationRule
}
func (v *PluginVerifier) Verify(plugin []byte) (*VerificationReport, error) {
    report := &VerificationReport{Safe: true}
    // Check for dangerous imports
    if bytes.Contains(plugin, []byte("syscall")) {
        report.Safe = false
        report.Issues = append(report.Issues, "Uses syscall package")
    }
    if bytes.Contains(plugin, []byte("unsafe")) {
        report.Safe = false
        report.Issues = append(report.Issues, "Uses unsafe package")
    }
    // Check binary signatures
    if bytes.Contains(plugin, []byte("/etc/passwd")) {
        report.Safe = false
        report.Issues = append(report.Issues, "Attempts to access system files")
    }
    // Run in sandbox for dynamic analysis
    sandbox := NewSandbox()
    if err := sandbox.Test(plugin); err != nil {
        report.Safe = false
        report.Issues = append(report.Issues, fmt.Sprintf("Failed sandbox test: %v", err))
    }
    return report, nil
}

Production Monitoring

type PluginMetrics struct {
    Calls       prometheus.CounterVec
    Errors      prometheus.CounterVec
    Duration    prometheus.HistogramVec
    Memory      prometheus.GaugeVec
    Active      prometheus.Gauge
}
func (m *Manager) CollectMetrics() {
    ticker := time.NewTicker(10 * time.Second)
    defer ticker.Stop()
    for range ticker.C {
        m.mu.RLock()
        active := 0
        for name, plugin := range m.plugins {
            if plugin.healthy {
                active++
            }
            // Get process stats
            if plugin.cmd != nil && plugin.cmd.Process != nil {
                pid := plugin.cmd.Process.Pid
                // Read /proc/[pid]/status for memory
                data, _ := ioutil.ReadFile(fmt.Sprintf("/proc/%d/status", pid))
                // Parse VmRSS (resident memory)
                lines := strings.Split(string(data), "\n")
                for _, line := range lines {
                    if strings.HasPrefix(line, "VmRSS:") {
                        fields := strings.Fields(line)
                        if len(fields) >= 2 {
                            mem, _ := strconv.ParseInt(fields[1], 10, 64)
                            metrics.Memory.WithLabelValues(name).Set(float64(mem * 1024))
                        }
                    }
                }
            }
            // Call metrics
            metrics.Calls.WithLabelValues(name).Add(float64(atomic.LoadUint64(&plugin.calls)))
            metrics.Errors.WithLabelValues(name).Add(float64(atomic.LoadUint64(&plugin.errors)))
        }
        metrics.Active.Set(float64(active))
        m.mu.RUnlock()
    }
}

Operational Expectations

When teams roll out this setup, the boring goals look like this:

• Consistent sandboxing so one bad plugin cannot crash the host
• Latency overhead measured in microseconds thanks to lightweight RPC
• Resource budgets per plugin that cap CPU, memory, and run time
• Version contracts so plugins indicate which host API they expect
• Automated hot reload paths for fast rollback and redeploy

The folks facing customers mostly want:

• Deploying custom logic without waiting on the core team
• Reducing support load by isolating plugin crashes
• Updating plugins independently of the main release cycle
• Maintaining uptime targets while still enabling experimentation

Security Considerations

Security Implementation

type SecurityPolicy struct {
    MaxMemory      int64         // Per-plugin memory limit
    MaxCPU         float64       // CPU cores limit
    Timeout        time.Duration // Execution timeout
    AllowedImports []string      // Whitelist of imports
    BlockedPaths   []string      // Filesystem paths to block
}
func (s *SecurityValidator) ValidatePlugin(binary []byte) error {
    // Check for dangerous imports
    dangerous := []string{"syscall", "unsafe", "os/exec", "net/http"}
    for _, pkg := range dangerous {
        if bytes.Contains(binary, []byte(pkg)) {
            return fmt.Errorf("plugin uses dangerous package: %s", pkg)
        }
    }
    // Check for file system access attempts
    suspicious := []string{"/etc/", "/root/", "/home/", "~/.ssh/"}
    for _, path := range suspicious {
        if bytes.Contains(binary, []byte(path)) {
            return fmt.Errorf("plugin attempts to access: %s", path)
        }
    }
    return nil
}

Testing Strategy

func TestPluginLifecycle(t *testing.T) {
    manager := NewManager()
    // Test loading
    err := manager.Load("testdata/sample_plugin.so")
    assert.NoError(t, err)
    // Test execution
    result, err := manager.Execute("sample", map[string]string{
        "input": "test",
    })
    assert.NoError(t, err)
    assert.Equal(t, "processed: test", result)
    // Test reload
    err = manager.Reload("sample")
    assert.NoError(t, err)
    // Test unload
    err = manager.Unload("sample")
    assert.NoError(t, err)
}

func TestResourceLimits(t *testing.T) {
    manager := NewManager()
    manager.maxMemory = 10 << 20  // 10MB
    manager.timeout = 100 * time.Millisecond
    // Test memory limit
    err := manager.Load("testdata/memory_hog_plugin.so")
    assert.NoError(t, err)
    _, err = manager.Execute("memory_hog", nil)
    assert.Error(t, err)
    assert.Contains(t, err.Error(), "memory limit exceeded")
    // Test timeout
    err = manager.Load("testdata/slow_plugin.so")
    assert.NoError(t, err)
    _, err = manager.Execute("slow", nil)
    assert.Error(t, err)
    assert.Contains(t, err.Error(), "timeout")
}

func TestConcurrentPluginCalls(t *testing.T) {
    manager := NewManager()
    manager.Load("testdata/concurrent_plugin.so")
    var wg sync.WaitGroup
    errors := make(chan error, 100)
    // 100 concurrent calls
    for i := 0; i < 100; i++ {
        wg.Add(1)
        go func(id int) {
            defer wg.Done()
            _, err := manager.Execute("concurrent", map[string]int{
                "id": id,
            })
            if err != nil {
                errors <- err
            }
        }(i)
    }
    wg.Wait()
    close(errors)
    // Check for errors
    errorCount := 0
    for err := range errors {
        t.Logf("Error: %v", err)
        errorCount++
    }
    assert.Equal(t, 0, errorCount, "Concurrent calls should not fail")
}

func TestHotReload(t *testing.T) {
    manager := NewManager()
    // Load v1
    manager.Load("testdata/plugin_v1.so")
    result, _ := manager.Execute("versioned", nil)
    assert.Equal(t, "v1", result)
    // Simulate file change
    os.Rename("testdata/plugin_v2.so", "testdata/plugin_v1.so")
    // Trigger reload
    manager.Reload("versioned")
    // Should get v2 response
    result, _ = manager.Execute("versioned", nil)
    assert.Equal(t, "v2", result)
}

func TestPluginCrashRecovery(t *testing.T) {
    manager := NewManager()
    manager.Load("testdata/crashy_plugin.so")
    // First call causes crash
    _, err := manager.Execute("crashy", map[string]string{
        "action": "crash",
    })
    assert.Error(t, err)
    // Manager should recover and reload
    time.Sleep(100 * time.Millisecond)
    // Next call should work
    result, err := manager.Execute("crashy", map[string]string{
        "action": "normal",
    })
    assert.NoError(t, err)
    assert.Equal(t, "ok", result)
}

Lessons Learned

1. RPC > Shared Memory - Isolation matters more than performance
2. Resource limits from day one - Customers will write infinite loops
3. Version everything - Rollback saves lives
4. Monitor everything - Bad plugins are silent killers
5. Trust no one - Every plugin is hostile code

The Bottom Line

You don't need microservices to get extensibility. You do need isolation, resource limits, and a basic idea of what your plugins are doing.

Get those pieces right and a plugin system carries as much weird customer logic as a wall of microservices, but on a couple of hosts and a maintenance window that still fits in a coffee break.

Microservices solve organizational problems. Plugins solve technical ones. Choose accordingly.