snakepit

# Migration Guide: Snakepit v0.5.1 → v0.6.0

**Document Version**: 1.0
**Date**: 2025-10-11
**Snakepit Version**: v0.6.0

---

## Table of Contents

1. [Overview](#overview)
2. [Breaking Changes](#breaking-changes)
3. [New Features](#new-features)
4. [Step-by-Step Migration](#step-by-step-migration)
5. [Configuration Changes](#configuration-changes)
6. [API Additions](#api-additions)
7. [Behavior Changes](#behavior-changes)
8. [Performance Considerations](#performance-considerations)
9. [Testing Your Migration](#testing-your-migration)
10. [Troubleshooting](#troubleshooting)
11. [Examples](#examples)
12. [FAQ](#faq)

---

## Overview

### What's New in v0.6.0?

Snakepit v0.6.0 introduces a **dual-mode parallelism architecture** that adds optional multi-threaded worker support alongside the existing multi-process model. This release maintains **100% backward compatibility** while enabling advanced use cases for Python 3.13+ free-threading mode.

### Do You Need to Migrate?

**No!** If you're happy with your current v0.5.1 setup, **no changes are required**. Your existing configuration will continue to work exactly as before.

**You may want to upgrade if:**
- You're running Python 3.13+ with free-threading support
- You have CPU-intensive workloads (NumPy, PyTorch, data processing)
- You want to reduce memory overhead for large datasets
- You need worker lifecycle management (automatic recycling)

### Migration Complexity

| Current Setup | Migration Effort | Changes Required |
|--------------|-----------------|------------------|
| Using default configuration | ✅ **None** | Drop-in upgrade |
| Custom pool configuration | ✅ **Minimal** | Test compatibility |
| Custom Python adapters | ⚠️ **Low** | Optional thread safety enhancements |
| Multiple pools or complex setup | ⚠️ **Medium** | Review new multi-pool features |

---

## Breaking Changes

### ✅ None!

Snakepit v0.6.0 introduces **zero breaking changes**. All existing functionality from v0.5.1 continues to work without modification.

### Deprecated Features

**None.** No features are deprecated in v0.6.0.

### Removed Features

**None.** All v0.5.1 features remain available.

---

## New Features

### 1. Dual-Mode Worker Profiles

Choose between two parallelism models:

| Profile | Description | Best For |
|---------|-------------|----------|
| **`:process`** | Multi-process, single-threaded workers (v0.5.1 behavior) | I/O-bound, legacy Python, high concurrency, maximum isolation |
| **`:thread`** | Multi-threaded workers (new in v0.6.0) | CPU-bound, Python 3.13+, large shared data, low memory overhead |

**Default**: `:process` (maintains v0.5.1 behavior)

### 2. Worker Lifecycle Management

Automatic worker recycling prevents memory leaks:

- **TTL-based**: Recycle workers after time limit
- **Request-count**: Recycle after N requests
- **Memory threshold**: Recycle if memory exceeds limit
- **Health checks**: Automatic monitoring every 5 minutes

### 3. Enhanced Configuration System

- **Named pools**: Run multiple pools with different profiles
- **Per-pool settings**: Different configurations per use case
- **Legacy compatibility**: Old config format auto-converted

### 4. Python 3.13+ Free-Threading Support

- Automatic detection of Python 3.13+ capabilities
- Recommendation engine for optimal profile
- Library compatibility checking

### 5. Telemetry Enhancements

New events for worker lifecycle:
- `[:snakepit, :worker, :recycled]`
- `[:snakepit, :worker, :health_check_failed]`

### 6. Thread Safety Infrastructure

For Python developers building custom adapters:
- Thread-safe base adapter class
- Runtime thread safety checking
- Library compatibility validation

---

## Step-by-Step Migration

### Step 1: Update Dependencies

```elixir
# mix.exs
def deps do
  [
    {:snakepit, "~> 0.6.11"}  # Update from 0.5.1
  ]
end
```

```bash
mix deps.update snakepit
mix deps.get
```

### Step 2: Run Existing Tests

```bash
# Ensure all existing tests pass
mix test
```

**Expected Result**: All tests should pass without changes.

### Step 3: Verify Configuration

Your existing v0.5.1 configuration works as-is:

```elixir
# config/config.exs - NO CHANGES NEEDED
config :snakepit,
  pooling_enabled: true,
  adapter_module: Snakepit.Adapters.GRPCPython,
  pool_size: 100,
  grpc_port: 50051,
  grpc_host: "localhost"
```

**What Happens**: Config is automatically converted to use `:process` profile.

### Step 4: Optional - Adopt New Features

If you want to use new v0.6.0 features:

#### Option A: Keep Existing Behavior (Recommended)

No changes needed! Your configuration continues to use the `:process` profile.

#### Option B: Add Thread Profile for New Workloads

```elixir
# config/config.exs
config :snakepit,
  pools: [
    # Existing API workload (unchanged)
    %{
      name: :default,
      worker_profile: :process,
      pool_size: 100,
      adapter_module: Snakepit.Adapters.GRPCPython
    },

    # New CPU-intensive workload
    %{
      name: :compute,
      worker_profile: :thread,
      pool_size: 4,
      threads_per_worker: 16,
      adapter_module: Snakepit.Adapters.GRPCPython
    }
  ]
```

#### Option C: Add Worker Recycling

```elixir
# config/config.exs
config :snakepit,
  pooling_enabled: true,
  adapter_module: Snakepit.Adapters.GRPCPython,
  pool_size: 100,

  # NEW: Lifecycle management
  worker_ttl: {3600, :seconds},      # Recycle hourly
  worker_max_requests: 5000          # Or after 5000 requests
```

### Step 5: Test in Staging

```bash
# Deploy to staging environment
MIX_ENV=staging mix release
```

**Verify**:
- Existing workflows continue to function
- New features work as expected
- No performance degradation

### Step 6: Deploy to Production

```bash
# Standard deployment process
MIX_ENV=prod mix release
```

---

## Configuration Changes

### Legacy Format (v0.5.1) - Still Supported ✅

```elixir
config :snakepit,
  pooling_enabled: true,
  adapter_module: Snakepit.Adapters.GRPCPython,
  pool_size: 100,
  pool_config: %{pool_size: 100},
  grpc_port: 50051,
  grpc_host: "localhost"
```

**v0.6.0 Behavior**: Automatically converted to `:process` profile with name `:default`.

### New Multi-Pool Format (v0.6.0) - Optional

```elixir
config :snakepit,
  grpc_port: 50051,
  grpc_host: "localhost",
  pools: [
    %{
      name: :api_pool,
      worker_profile: :process,
      pool_size: 100,
      adapter_module: Snakepit.Adapters.GRPCPython
    },
    %{
      name: :ml_pool,
      worker_profile: :thread,
      pool_size: 4,
      threads_per_worker: 16,
      capacity_strategy: :pool,
      adapter_module: Snakepit.Adapters.GRPCPython
    }
  ]
```

### Configuration Mapping

| v0.5.1 Config Key | v0.6.0 Equivalent | Notes |
|------------------|-------------------|-------|
| `pooling_enabled` | Same | No change |
| `adapter_module` | Per-pool `adapter_module` | Can specify per pool |
| `pool_size` | Per-pool `pool_size` | Can differ per pool |
| `pool_config` | Per-pool map | Merged into pool config |
| `grpc_port` | Same | Elixir gRPC server port |
| `grpc_host` | Same | Elixir gRPC server host |
| N/A | `worker_profile` | **New**: `:process` or `:thread` |
| N/A | `worker_ttl` | **New**: Lifecycle management |
| N/A | `worker_max_requests` | **New**: Lifecycle management |
| N/A | `threads_per_worker` | **New**: Thread profile only |
| N/A | `capacity_strategy` | **New**: Pool scheduling (`:pool`, `:profile`, `:hybrid`) |

---

## API Additions

### New Functions

#### Snakepit.Config

```elixir
# Get all pool configurations
Snakepit.Config.get_pool_configs()
# => [%{name: :default, worker_profile: :process, ...}]

# Get specific pool config
Snakepit.Config.get_pool_config(:api_pool)
# => %{name: :api_pool, worker_profile: :process, ...}

# Check if pool uses thread profile
Snakepit.Config.thread_profile?(:ml_pool)
# => true

# Get profile implementation module
Snakepit.Config.get_profile_module(:api_pool)
# => Snakepit.WorkerProfile.Process
```

#### Snakepit.Worker.LifecycleManager

```elixir
# Manually recycle a worker
Snakepit.Worker.LifecycleManager.recycle_worker(:api_pool, worker_id)

# Get lifecycle statistics
Snakepit.Worker.LifecycleManager.get_stats()
# => %{total_workers: 104, ...}
```

#### Snakepit.PythonVersion

```elixir
# Detect Python version
{:ok, {3, 13, 0}} = Snakepit.PythonVersion.detect()

# Check free-threading support
Snakepit.PythonVersion.supports_free_threading?({3, 13, 0})
# => true

# Get recommended profile
Snakepit.PythonVersion.recommend_profile()
# => :thread  # For Python 3.13+

# Validate Python environment
Snakepit.PythonVersion.validate()
```

#### Snakepit.Compatibility

```elixir
# Check library thread safety
Snakepit.Compatibility.check("numpy", :thread)
# => {:ok, "Thread-safe"}

# Get library info
Snakepit.Compatibility.get_library_info("pandas")
# => %{thread_safe: false, notes: "Not thread-safe as of v2.0", ...}

# List thread-safe libraries
Snakepit.Compatibility.list_all(:thread_safe)
# => ["numpy", "scipy", "torch", ...]

# Generate compatibility report
Snakepit.Compatibility.generate_report(["numpy", "pandas"], :thread)
```

### Behavior Changes in Existing Functions

#### Snakepit.execute/3,4

**v0.5.1**:
```elixir
Snakepit.execute("command", %{args: "here"})
# Always uses default pool
```

**v0.6.0**:
```elixir
# Still works - uses :default pool
Snakepit.execute("command", %{args: "here"})

# NEW: Specify pool (optional)
Snakepit.execute(:ml_pool, "compute", %{data: []})
```

**Backward Compatibility**: ✅ Default pool behavior unchanged.

---

## Behavior Changes

### Process Profile (Default)

**No changes from v0.5.1**. The `:process` profile maintains exact v0.5.1 behavior:

- Single-threaded Python workers
- Enforced thread limiting in NumPy/SciPy/etc.
- Process-level isolation
- Dynamic port allocation

### Worker Initialization

**v0.5.1**: Workers start on pool initialization, run indefinitely.

**v0.6.0**:
- Same as v0.5.1 **unless** lifecycle management configured
- With `worker_ttl` or `worker_max_requests`: workers automatically recycled
- Recycling is zero-downtime (new worker starts before old stops)

### Telemetry Events

**New Events** (won't break existing handlers):

```elixir
# Worker recycled
[:snakepit, :worker, :recycled]
# Measurements: %{count: 1}
# Metadata: %{worker_id, pool, reason, uptime_seconds, request_count}

# Health check failed
[:snakepit, :worker, :health_check_failed]
# Measurements: %{count: 1}
# Metadata: %{worker_id, pool, reason}
```

**Existing Events**: No changes to existing event signatures.

---

## Performance Considerations

### Memory Usage

#### Process Profile (v0.5.1 behavior)
```
100 workers × 150 MB = 15 GB
```

#### Thread Profile (new)
```
4 processes × 16 threads × 400 MB = 1.6 GB
(~10× reduction)
```

### Throughput

| Workload Type | Process Profile | Thread Profile | Winner |
|--------------|----------------|----------------|--------|
| Small API requests | 1500 req/s | 1200 req/s | Process |
| CPU-intensive tasks | 600 jobs/hr | 2400 jobs/hr | Thread (4×) |
| I/O-bound | 1500 req/s | 1200 req/s | Process |

### Startup Time

| Configuration | v0.5.1 | v0.6.0 Process | v0.6.0 Thread |
|--------------|--------|---------------|---------------|
| 100 workers | ~10s | ~10s | ~2s |
| 250 workers | ~60s | ~60s | ~5s |

**Recommendation**:
- **Keep process profile** for high-concurrency I/O workloads
- **Add thread profile** for CPU-intensive workloads

---

## Testing Your Migration

### Test Suite

Create a migration test file:

```elixir
# test/migration_test.exs
defmodule MigrationTest do
  use ExUnit.Case, async: false

  @tag :migration
  test "v0.5.1 config still works" do
    # Use legacy config format
    Application.put_env(:snakepit, :pooling_enabled, true)
    Application.put_env(:snakepit, :pool_size, 4)
    Application.put_env(:snakepit, :adapter_module, Snakepit.Adapters.GRPCPython)

    # Start Snakepit
    {:ok, _} = Application.ensure_all_started(:snakepit)

    # Verify basic operations
    assert {:ok, _} = Snakepit.execute("ping", %{})
    assert {:ok, _} = Snakepit.execute("add", %{a: 5, b: 3})

    # Cleanup
    Application.stop(:snakepit)
  end

  @tag :migration
  test "new multi-pool config works" do
    config = [
      pools: [
        %{
          name: :default,
          worker_profile: :process,
          pool_size: 2,
          adapter_module: Snakepit.Adapters.GRPCPython
        }
      ]
    ]

    {:ok, _} = start_supervised({Snakepit.Application, config})

    assert {:ok, _} = Snakepit.execute(:default, "ping", %{})
  end
end
```

Run tests:
```bash
mix test --only migration
```

### Manual Verification

```elixir
# In IEx
iex> Application.ensure_all_started(:snakepit)
{:ok, [...]}

# Test basic operation
iex> Snakepit.execute("ping", %{})
{:ok, %{"status" => "pong", ...}}

# Check configuration was loaded
iex> Snakepit.Config.get_pool_configs()
[%{name: :default, worker_profile: :process, ...}]

# Verify worker profile
iex> Snakepit.Config.get_profile_module(:default)
Snakepit.WorkerProfile.Process
```

### Performance Testing

```bash
# Benchmark existing workload
mix run scripts/benchmark.exs

# Compare v0.5.1 vs v0.6.0 results
```

---

## Troubleshooting

### Issue: "Configuration not found" error

**Symptom**: `{:error, :no_pools_configured}`

**Cause**: Empty or invalid pool configuration

**Solution**:
```elixir
# Ensure config has at least one pool
config :snakepit,
  pools: [
    %{
      name: :default,
      worker_profile: :process,
      pool_size: 4,
      adapter_module: Snakepit.Adapters.GRPCPython
    }
  ]
```

### Issue: Workers not starting

**Symptom**: Pool timeout or no workers available

**Solution**:
```bash
# Check Python version
python3 --version

# Verify gRPC dependencies
python3 -c "import grpc; print(grpc.__version__)"

# Check logs
iex> Logger.configure(level: :debug)
```

### Issue: "Profile not implemented" error

**Symptom**: `{:error, :not_implemented}`

**Cause**: Thread profile requires threaded adapter mode (`--max-workers`) and thread-safe adapters; older releases did not support thread scheduling.

**Solution**: Upgrade to v0.7.0+, ensure adapter args include `--max-workers`, and fall back to `:process` if adapters are not thread-safe.

### Issue: Performance regression

**Symptom**: Slower performance after upgrade

**Diagnosis**:
```elixir
# Check which profile is active
iex> Snakepit.Config.get_pool_configs()

# Verify worker count
iex> Snakepit.Pool.get_stats()
```

**Solution**: Ensure `:process` profile with same pool_size as v0.5.1

### Issue: Memory usage increased

**Symptom**: Higher memory consumption

**Cause**: Worker recycling disabled or long TTL

**Solution**:
```elixir
# Add lifecycle management
config :snakepit,
  worker_ttl: {3600, :seconds},
  worker_max_requests: 5000
```

---

## Examples

### Example 1: Simple Upgrade (No Changes)

```elixir
# mix.exs - BEFORE (v0.5.1)
{:snakepit, "~> 0.5.1"}

# mix.exs - AFTER (v0.6.11)
{:snakepit, "~> 0.6.11"}

# config/config.exs - UNCHANGED
config :snakepit,
  pooling_enabled: true,
  pool_size: 100,
  adapter_module: Snakepit.Adapters.GRPCPython
```

**Result**: Exact same behavior as v0.5.1.

### Example 2: Add Worker Recycling

```elixir
# config/config.exs
config :snakepit,
  pooling_enabled: true,
  pool_size: 100,
  adapter_module: Snakepit.Adapters.GRPCPython,

  # NEW in v0.6.0
  worker_ttl: {7200, :seconds},      # 2 hours
  worker_max_requests: 10000         # Or 10k requests
```

**Result**: Workers automatically recycled, preventing memory leaks.

### Example 3: Mixed Workloads

```elixir
# config/config.exs
config :snakepit,
  pools: [
    # High-concurrency API (process profile)
    %{
      name: :api,
      worker_profile: :process,
      pool_size: 200,
      adapter_module: Snakepit.Adapters.GRPCPython,
      worker_ttl: {3600, :seconds}
    },

    # CPU-intensive ML (thread profile)
    %{
      name: :ml,
      worker_profile: :thread,
      pool_size: 4,
      threads_per_worker: 16,
      adapter_module: Snakepit.Adapters.GRPCPython,
      worker_ttl: {1800, :seconds}
    }
  ]
```

**Usage**:
```elixir
# API requests - fast, high concurrency
Snakepit.execute(:api, "get_user", %{id: 123})

# ML inference - CPU-intensive
Snakepit.execute(:ml, "predict", %{model: "resnet50", image: data})
```

### Example 4: Telemetry Integration

```elixir
# lib/my_app/telemetry.ex
defmodule MyApp.Telemetry do
  def setup do
    :telemetry.attach_many(
      "snakepit-lifecycle",
      [
        [:snakepit, :worker, :recycled],
        [:snakepit, :worker, :health_check_failed]
      ],
      &handle_event/4,
      nil
    )
  end

  def handle_event([:snakepit, :worker, :recycled], _, metadata, _) do
    Logger.info("""
    Worker recycled:
      Pool: #{metadata.pool}
      Reason: #{metadata.reason}
      Uptime: #{metadata.uptime_seconds}s
    """)
  end

  def handle_event([:snakepit, :worker, :health_check_failed], _, metadata, _) do
    Logger.error("Health check failed: #{metadata.worker_id}")
  end
end
```

---

## FAQ

### Q: Do I have to migrate?

**A**: No. v0.6.0 is 100% backward compatible. Your v0.5.1 configuration works without changes.

### Q: Will my existing code break?

**A**: No. All existing APIs continue to work exactly as before.

### Q: Should I use thread profile?

**A**: Only if:
- Running Python 3.13+ with free-threading
- CPU-intensive workloads (NumPy, PyTorch)
- Need to reduce memory overhead
- Have thread-safe Python code

Otherwise, stick with `:process` profile (default).

### Q: Can I use both profiles?

**A**: Yes! Configure multiple pools with different profiles:

```elixir
config :snakepit,
  pools: [
    %{name: :api, worker_profile: :process, ...},
    %{name: :compute, worker_profile: :thread, ...}
  ]
```

### Q: How do I know if my Python code is thread-safe?

**A**: Check the compatibility matrix:

```elixir
Snakepit.Compatibility.check("your_library", :thread)
```

Or use the thread safety checker in development.

### Q: What's the performance impact?

**A**:
- `:process` profile: Identical to v0.5.1
- `:thread` profile: 3-4× faster for CPU work, slightly slower for I/O
- Worker recycling: <0.001% CPU overhead

### Q: Can I rollback to v0.5.1?

**A**: Yes. Simply change version in mix.exs and redeploy. No data migration needed.

### Q: Will v0.5.1 still be supported?

**A**: Yes. Critical bug fixes will be backported for 6 months. Security fixes for 12 months.

### Q: What about my custom adapters?

**A**: They continue to work without changes. Optional: add thread safety for `:thread` profile.

### Q: How do I enable worker recycling?

**A**: Add to your config:

```elixir
config :snakepit,
  worker_ttl: {3600, :seconds},
  worker_max_requests: 5000
```

### Q: What happens during worker recycling?

**A**:
1. New worker starts
2. New worker becomes available
3. Old worker stops accepting requests
4. Old worker finishes in-flight requests
5. Old worker shuts down
6. Zero downtime!

---

## Summary

### Migration Checklist

- [x] Update `mix.exs` to v0.6.0
- [x] Run `mix deps.get`
- [x] Run existing test suite (should pass)
- [ ] Optional: Add worker recycling config
- [ ] Optional: Configure multi-pool setup
- [ ] Optional: Add telemetry handlers
- [x] Test in staging environment
- [x] Deploy to production
- [ ] Monitor telemetry events

### Key Takeaways

1. ✅ **Zero breaking changes** - v0.5.1 config works as-is
2. ✅ **Opt-in features** - Use new features when ready
3. ✅ **Backward compatible** - All APIs unchanged
4. ✅ **Safe to deploy** - Extensive testing completed
5. ✅ **Performance neutral** - Default behavior unchanged

### Getting Help

- **Documentation**: Check `/docs` directory
- **Examples**: See `/examples` directory
- **Issues**: https://github.com/nshkrdotcom/snakepit/issues
- **Discussions**: https://github.com/nshkrdotcom/snakepit/discussions

---

## Additional Resources

### Documentation

- [Performance Benchmarks](/docs/performance_benchmarks.md)
- [Writing Thread-Safe Adapters](/docs/guides/writing_thread_safe_adapters.md)
- [Telemetry Events Reference](/docs/telemetry_events.md)
- Production Deployment Guide (coming soon)

### Examples

- [Process vs Thread Comparison](/examples/process_vs_thread_comparison.exs)
- [Lifecycle Demo](/examples/lifecycle_demo.exs)
- [Monitoring Demo](/examples/monitoring_demo.exs)

---

**Welcome to Snakepit v0.6.0!** 🚀