Packages

snakepit

0.8.8
0.13.0 0.12.0 0.11.1 0.11.0 0.10.1 0.10.0 0.9.1 0.9.0 0.8.9 0.8.8 0.8.7 0.8.6 0.8.5 0.8.4 0.8.3 0.8.2 0.8.1 0.8.0 0.7.7 0.7.6 0.7.5 0.7.4 0.7.3 0.7.2 0.7.1 0.7.0 0.6.11 0.6.10 0.6.9 0.6.8 0.6.7 0.6.6 0.6.5 0.6.4 0.6.3 0.6.2 0.6.1 0.6.0 0.5.1 0.5.0 0.4.3 0.4.2 0.4.1 0.4.0 0.3.3 0.3.2 0.3.1 0.3.0 0.2.1 0.2.0 0.1.2 0.1.1 0.1.0

High-performance pooler and session manager for external language integrations. Supports Python, Node.js, Ruby, and more with gRPC streaming, session management, and production-ready process cleanup.

HexDocs

Current section

53 Versions

Jump to
Readme
53 Versions
9 Dependencies
Dependants
Files
Activity
Show whitespace

Compare versions

22 files changed
+2443 additions
-194 deletions
  @@ -7,6 +7,106 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7 7
8 8 ## [Unreleased]
9 9
10 + ## [0.8.8] - 2025-12-31
11 +
12 + ### Added
13 + - **Centralized configurable defaults** - New `Snakepit.Defaults` module provides runtime-configurable defaults for all hardcoded values
14 + - All 68 previously hardcoded timeout, sizing, and threshold values are now configurable via `Application.get_env/3`
15 + - Values are read at runtime, allowing configuration changes in `config/runtime.exs` without recompilation
16 + - Defaults remain unchanged from previous versions for backward compatibility
17 + - See `Snakepit.Defaults` module documentation for complete list of configurable keys
18 +
19 + - **Timeout profile architecture** - New single-budget, derived deadlines, profile-based timeout system
20 + - Six predefined profiles: `:balanced`, `:production`, `:production_strict`, `:development`, `:ml_inference`, `:batch`
21 + - New user-facing API: `default_timeout/0`, `stream_timeout/0`, `queue_timeout/0`
22 + - Margin configuration: `worker_call_margin_ms/0` (default 1000), `pool_reply_margin_ms/0` (default 200)
23 + - RPC timeout derivation: `rpc_timeout/1` computes inner timeout from total budget
24 + - Legacy getters (`pool_request_timeout`, `grpc_command_timeout`, etc.) now derive from profile when not explicitly configured
25 + - Configure via: `config :snakepit, timeout_profile: :production`
26 +
27 + - **Pool deadline-aware execution** - Pool.execute/3 now stores deadline_ms for queue-aware timeout handling
28 + - New helper: `Pool.get_default_timeout_for_call/3` for call-type-aware timeout lookup
29 + - New helper: `Pool.derive_rpc_timeout_from_opts/2` for deadline-aware RPC timeout derivation
30 + - New helper: `Pool.effective_queue_timeout_ms/2` for budget-aware queue timeout
31 + - GenServer.call timeout caught and returned as structured `{:error, %Snakepit.Error{}}`
32 +
33 + ### Changed
34 + - **Pool module** - Timeout and sizing defaults now read from `Snakepit.Defaults`:
35 + - `pool_request_timeout`, `pool_streaming_timeout`, `pool_startup_timeout`, `pool_queue_timeout`
36 + - `checkout_timeout`, `default_command_timeout`, `pool_await_ready_timeout`
37 + - `pool_max_queue_size`, `pool_max_workers`, `pool_max_cancelled_entries`
38 + - `pool_startup_batch_size`, `pool_startup_batch_delay_ms`
39 +
40 + - **GRPCWorker** - Execute and streaming timeouts now configurable:
41 + - `grpc_worker_execute_timeout`, `grpc_worker_stream_timeout`
42 + - `grpc_server_ready_timeout`, `worker_ready_timeout`
43 + - `grpc_worker_health_check_interval`
44 + - Heartbeat configuration: `heartbeat_ping_interval_ms`, `heartbeat_timeout_ms`, `heartbeat_max_missed`, `heartbeat_initial_delay_ms`
45 +
46 + - **Fault tolerance modules** - Circuit breaker, retry policy, crash barrier, and health monitor defaults now configurable:
47 + - `circuit_breaker_failure_threshold`, `circuit_breaker_reset_timeout_ms`, `circuit_breaker_half_open_max_calls`
48 + - `retry_max_attempts`, `retry_backoff_sequence`, `retry_max_backoff_ms`, `retry_jitter_factor`
49 + - `crash_barrier_taint_duration_ms`, `crash_barrier_max_restarts`, `crash_barrier_backoff_ms`
50 + - `health_monitor_check_interval`, `health_monitor_crash_window_ms`, `health_monitor_max_crashes`
51 +
52 + - **Session store** - Session management defaults now configurable:
53 + - `session_cleanup_interval`, `session_default_ttl`, `session_max_sessions`, `session_warning_threshold`
54 +
55 + - **Process registry** - Cleanup intervals now configurable:
56 + - `process_registry_cleanup_interval`, `process_registry_unregister_cleanup_delay`, `process_registry_unregister_cleanup_attempts`
57 +
58 + - **Application and gRPC** - Server configuration now configurable:
59 + - `grpc_port`, `grpc_num_acceptors`, `grpc_max_connections`, `grpc_socket_backlog`
60 + - `cleanup_on_stop_timeout_ms`, `cleanup_poll_interval_ms`
61 +
62 + - **Config module** - Pool and worker profile defaults now configurable:
63 + - `default_pool_size`, `default_worker_profile`, `default_capacity_strategy`
64 + - `config_default_batch_size`, `config_default_batch_delay`, `config_default_threads_per_worker`
65 +
66 + ### Timeout Architecture Proposal
67 +
68 + The following documents the design rationale for the timeout architecture implemented in this release.
69 +
70 + #### Problem Statement
71 +
72 + Snakepit's timeout configuration was fragmented with 7+ independent timeout keys that didn't coordinate:
73 + - `pool_request_timeout` vs `grpc_command_timeout` - Which is outer? Which is inner?
74 + - Queue wait time consumed part of the budget, but inner timeouts didn't account for it
75 + - GenServer.call timeouts firing before inner timeouts produced unhandled exits instead of structured errors
76 +
77 + #### Solution: Single-Budget, Derived Deadlines
78 +
79 + **Core principle**: One top-level timeout budget, all inner timeouts derived from remaining time.
80 +
81 + **Profile-based defaults** provide sensible starting points for different deployment scenarios:
82 +
83 + | Profile | default_timeout | stream_timeout | queue_timeout |
84 + |---------|-----------------|----------------|---------------|
85 + | :balanced | 300_000 (5m) | 900_000 (15m) | 10_000 (10s) |
86 + | :production | 300_000 (5m) | 900_000 (15m) | 10_000 (10s) |
87 + | :production_strict | 60_000 (60s) | 300_000 (5m) | 5_000 (5s) |
88 + | :development | 900_000 (15m) | 3_600_000 (60m) | 60_000 (60s) |
89 + | :ml_inference | 900_000 (15m) | 3_600_000 (60m) | 60_000 (60s) |
90 + | :batch | 3_600_000 (60m) | :infinity | 300_000 (5m) |
91 +
92 + **Margin formula** ensures inner timeouts fire before outer:
93 + ```
94 + rpc_timeout = total_timeout - worker_call_margin_ms (1000) - pool_reply_margin_ms (200)
95 + ```
96 +
97 + **Deadline propagation** tracks remaining budget:
98 + 1. Pool.execute stores `deadline_ms = now + timeout` in opts
99 + 2. Queue handler uses `effective_queue_timeout_ms/2` to respect deadline
100 + 3. Worker execution uses `derive_rpc_timeout_from_opts/2` to compute remaining budget
101 + 4. All GenServer.call timeouts are caught and returned as structured errors
102 +
103 + #### Backward Compatibility
104 +
105 + - All legacy config keys (`pool_request_timeout`, `grpc_command_timeout`, etc.) still work
106 + - When explicitly set, they take precedence over profile-derived values
107 + - When not set, they derive from the active profile
108 + - Default profile is `:balanced` which provides similar values to previous defaults
109 +
10 110 ## [0.8.7] - 2025-12-31
11 111
12 112 ### Fixed
  @@ -31,7 +31,7 @@ Add `snakepit` to your dependencies in `mix.exs`:
31 31 ```elixir
32 32 def deps do
33 33 [
34 - {:snakepit, "~> 0.8.7"}
34 + {:snakepit, "~> 0.8.8"}
35 35 ]
36 36 end
37 37 ```
  @@ -109,6 +109,72 @@ config :snakepit, log_level: :none # Complete silence
109 109 config :snakepit, log_level: :debug, log_categories: [:grpc, :pool]
110 110 ```
111 111
112 + ### Runtime Configurable Defaults (v0.8.8+)
113 +
114 + All hardcoded timeout and sizing values are now configurable via `Application.get_env/3`.
115 + Values are read at runtime, allowing configuration changes without recompilation.
116 +
117 + ```elixir
118 + # config/runtime.exs - Example customization
119 + config :snakepit,
120 + # Timeouts (all in milliseconds)
121 + default_command_timeout: 30_000, # Default timeout for commands
122 + pool_request_timeout: 60_000, # Pool execute timeout
123 + pool_streaming_timeout: 300_000, # Pool streaming timeout
124 + pool_startup_timeout: 10_000, # Worker startup timeout
125 + pool_queue_timeout: 5_000, # Queue timeout
126 + checkout_timeout: 5_000, # Worker checkout timeout
127 + grpc_worker_execute_timeout: 30_000, # GRPCWorker execute timeout
128 + grpc_worker_stream_timeout: 300_000, # GRPCWorker streaming timeout
129 + graceful_shutdown_timeout_ms: 6_000, # Python process shutdown timeout
130 +
131 + # Pool sizing
132 + pool_max_queue_size: 1000, # Max pending requests in queue
133 + pool_max_workers: 150, # Maximum workers per pool
134 + pool_startup_batch_size: 10, # Workers started per batch
135 + pool_startup_batch_delay_ms: 500, # Delay between startup batches
136 +
137 + # Retry policy
138 + retry_max_attempts: 3,
139 + retry_backoff_sequence: [100, 200, 400, 800, 1600],
140 + retry_max_backoff_ms: 30_000,
141 + retry_jitter_factor: 0.25,
142 +
143 + # Circuit breaker
144 + circuit_breaker_failure_threshold: 5,
145 + circuit_breaker_reset_timeout_ms: 30_000,
146 + circuit_breaker_half_open_max_calls: 1,
147 +
148 + # Crash barrier
149 + crash_barrier_taint_duration_ms: 60_000,
150 + crash_barrier_max_restarts: 1,
151 + crash_barrier_backoff_ms: [50, 100, 200],
152 +
153 + # Health monitor
154 + health_monitor_check_interval: 30_000,
155 + health_monitor_crash_window_ms: 60_000,
156 + health_monitor_max_crashes: 10,
157 +
158 + # Heartbeat
159 + heartbeat_ping_interval_ms: 2_000,
160 + heartbeat_timeout_ms: 10_000,
161 + heartbeat_max_missed: 3,
162 +
163 + # Session store
164 + session_cleanup_interval: 60_000,
165 + session_default_ttl: 3600,
166 + session_max_sessions: 10_000,
167 + session_warning_threshold: 0.8,
168 +
169 + # gRPC server
170 + grpc_port: 50_051,
171 + grpc_num_acceptors: 20,
172 + grpc_max_connections: 1000,
173 + grpc_socket_backlog: 512
174 + ```
175 +
176 + See `Snakepit.Defaults` module documentation for the complete list of configurable values.
177 +
112 178 ## Core API
113 179
114 180 ### Basic Execution
  @@ -1,95 +1,387 @@
1 - <svg width="256px" height="256px" viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg" version="1.1">
1 + <svg width="200" height="200" viewBox="0 0 200 200" xmlns="http://www.w3.org/2000/svg">
2 2 <defs>
3 - <!-- Python-inspired color gradients -->
3 + <!-- Python Blue Gradient -->
4 4 <linearGradient id="pythonBlue" x1="0%" y1="0%" x2="100%" y2="100%">
5 - <stop offset="0%" style="stop-color:#306998; stop-opacity:1" />
6 - <stop offset="100%" style="stop-color:#4b8bbe; stop-opacity:1" />
7 - </linearGradient>
8 - <linearGradient id="pythonYellow" x1="0%" y1="0%" x2="100%" y2="100%">
9 - <stop offset="0%" style="stop-color:#ffd43b; stop-opacity:1" />
10 - <stop offset="100%" style="stop-color:#ffe873; stop-opacity:1" />
11 - </linearGradient>
12 - <linearGradient id="shadowGradient" x1="0%" y1="0%" x2="100%" y2="100%">
13 - <stop offset="0%" style="stop-color:#1e1e1e; stop-opacity:0.3" />
14 - <stop offset="100%" style="stop-color:#1e1e1e; stop-opacity:0" />
5 + <stop offset="0%" stop-color="#306998"/>
6 + <stop offset="50%" stop-color="#4b8bbe"/>
7 + <stop offset="100%" stop-color="#5a9fd4"/>
15 8 </linearGradient>
16 9
17 - <!-- Filter for a subtle glow on the eyes -->
18 - <filter id="eyeGlow">
19 - <feGaussianBlur stdDeviation="0.5" result="coloredBlur"/>
10 + <!-- Python Yellow Gradient -->
11 + <linearGradient id="pythonYellow" x1="0%" y1="0%" x2="100%" y2="100%">
12 + <stop offset="0%" stop-color="#ffd43b"/>
13 + <stop offset="50%" stop-color="#ffe873"/>
14 + <stop offset="100%" stop-color="#fff3a3"/>
15 + </linearGradient>
16 +
17 + <!-- Dark Pit Gradient (radial for depth) -->
18 + <radialGradient id="pitDepth" cx="50%" cy="50%" r="60%">
19 + <stop offset="0%" stop-color="#0a0f1a"/>
20 + <stop offset="40%" stop-color="#0d121f"/>
21 + <stop offset="70%" stop-color="#141b2d"/>
22 + <stop offset="100%" stop-color="#1c2331"/>
23 + </radialGradient>
24 +
25 + <!-- Hexagon Border Gradient -->
26 + <linearGradient id="hexBorder" x1="0%" y1="0%" x2="100%" y2="100%">
27 + <stop offset="0%" stop-color="#4b8bbe"/>
28 + <stop offset="50%" stop-color="#ffd43b"/>
29 + <stop offset="100%" stop-color="#4b8bbe"/>
30 + </linearGradient>
31 +
32 + <!-- Snake Scale Pattern -->
33 + <pattern id="snakeScales" patternUnits="userSpaceOnUse" width="6" height="5">
34 + <ellipse cx="3" cy="2.5" rx="2.8" ry="2.2" fill="none" stroke="rgba(255,255,255,0.15)" stroke-width="0.4"/>
35 + </pattern>
36 +
37 + <!-- Eye Glow Filter -->
38 + <filter id="eyeGlow" x="-50%" y="-50%" width="200%" height="200%">
39 + <feGaussianBlur stdDeviation="1.5" result="glow"/>
20 40 <feMerge>
21 - <feMergeNode in="coloredBlur"/>
41 + <feMergeNode in="glow"/>
42 + <feMergeNode in="glow"/>
22 43 <feMergeNode in="SourceGraphic"/>
23 44 </feMerge>
24 45 </filter>
46 +
47 + <!-- Snake Body Glow -->
48 + <filter id="snakeGlow" x="-20%" y="-20%" width="140%" height="140%">
49 + <feGaussianBlur stdDeviation="2" result="blur"/>
50 + <feMerge>
51 + <feMergeNode in="blur"/>
52 + <feMergeNode in="SourceGraphic"/>
53 + </feMerge>
54 + </filter>
55 +
56 + <!-- Inner Shadow for Pit -->
57 + <filter id="pitShadow" x="-10%" y="-10%" width="120%" height="120%">
58 + <feGaussianBlur stdDeviation="4" result="shadow"/>
59 + <feOffset dx="0" dy="2"/>
60 + <feComposite in2="SourceAlpha" operator="arithmetic" k2="-1" k3="1"/>
61 + <feColorMatrix type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.6 0"/>
62 + <feMerge>
63 + <feMergeNode/>
64 + <feMergeNode in="SourceGraphic"/>
65 + </feMerge>
66 + </filter>
67 +
68 + <!-- Ambient Light -->
69 + <filter id="ambientLight">
70 + <feSpecularLighting surfaceScale="3" specularConstant="0.8" specularExponent="25" lighting-color="#ffffff" result="specOut">
71 + <fePointLight x="100" y="50" z="80"/>
72 + </feSpecularLighting>
73 + <feComposite in="SourceGraphic" in2="specOut" operator="arithmetic" k1="0" k2="1" k3="0.3" k4="0"/>
74 + </filter>
75 +
76 + <!-- Hexagon Clip Path -->
77 + <clipPath id="hexClip">
78 + <polygon points="100,8 181.6,55 181.6,145 100,192 18.4,145 18.4,55"/>
79 + </clipPath>
80 +
81 + <!-- Snake Body Texture -->
82 + <linearGradient id="blueSnakeBody" x1="0%" y1="0%" x2="0%" y2="100%">
83 + <stop offset="0%" stop-color="#5a9fd4"/>
84 + <stop offset="30%" stop-color="#4b8bbe"/>
85 + <stop offset="70%" stop-color="#306998"/>
86 + <stop offset="100%" stop-color="#254f73"/>
87 + </linearGradient>
88 +
89 + <linearGradient id="yellowSnakeBody" x1="0%" y1="0%" x2="0%" y2="100%">
90 + <stop offset="0%" stop-color="#fff3a3"/>
91 + <stop offset="30%" stop-color="#ffe873"/>
92 + <stop offset="70%" stop-color="#ffd43b"/>
93 + <stop offset="100%" stop-color="#e6b800"/>
94 + </linearGradient>
95 +
96 + <!-- Subtle noise texture -->
97 + <filter id="noiseTexture">
98 + <feTurbulence type="fractalNoise" baseFrequency="0.8" numOctaves="4" result="noise"/>
99 + <feColorMatrix type="saturate" values="0"/>
100 + <feBlend in="SourceGraphic" in2="noise" mode="multiply" result="blended"/>
101 + <feComposite in="blended" in2="SourceAlpha" operator="in"/>
102 + </filter>
25 103 </defs>
26 104
27 - <!-- Hexagonal Pit Background -->
105 + <!-- Outer Hexagon Border/Glow -->
28 106 <polygon
29 - points="50,5 95,27.5 95,72.5 50,95 5,72.5 5,27.5"
30 - fill="#1c2331"
31 - />
32 - <polygon
33 - points="50,5 95,27.5 95,72.5 50,95 5,72.5 5,27.5"
34 - fill="url(#shadowGradient)"
107 + points="100,5 184.6,53.5 184.6,146.5 100,195 15.4,146.5 15.4,53.5"
108 + fill="none"
109 + stroke="url(#hexBorder)"
110 + stroke-width="2"
111 + opacity="0.6"
35 112 />
36 113
37 - <!-- Intertwined Snakes -->
38 - <g transform="translate(0.0, 0.0)">
39 - <!-- Snake 1 (Blue) -->
114 + <!-- Main Hexagon (The Pit) -->
115 + <polygon
116 + points="100,8 181.6,55 181.6,145 100,192 18.4,145 18.4,55"
117 + fill="url(#pitDepth)"
118 + stroke="url(#hexBorder)"
119 + stroke-width="3"
120 + filter="url(#pitShadow)"
121 + />
122 +
123 + <!-- Pit Floor Pattern (subtle) -->
124 + <g clip-path="url(#hexClip)" opacity="0.3">
125 + <circle cx="100" cy="130" r="50" fill="none" stroke="#1a2540" stroke-width="1"/>
126 + <circle cx="100" cy="130" r="35" fill="none" stroke="#1a2540" stroke-width="1"/>
127 + <circle cx="100" cy="130" r="20" fill="none" stroke="#1a2540" stroke-width="1"/>
128 + </g>
129 +
130 + <!-- Snake Pit Contents -->
131 + <g clip-path="url(#hexClip)">
132 +
133 + <!-- ===== DEEPEST LAYER - Faded coils at bottom ===== -->
134 +
135 + <!-- Blue snake coil in deep background -->
136 + <g filter="url(#snakeGlow)" opacity="0.35">
137 + <path
138 + d="M 60,148 C 75,155 95,145 110,150 C 125,155 135,148 145,142"
139 + fill="none"
140 + stroke="url(#blueSnakeBody)"
141 + stroke-width="11"
142 + stroke-linecap="round"
143 + />
144 + </g>
145 +
146 + <!-- Yellow snake loop deep -->
147 + <g filter="url(#snakeGlow)" opacity="0.4">
148 + <path
149 + d="M 140,132 C 130,140 112,130 95,136 C 78,142 68,136 60,128"
150 + fill="none"
151 + stroke="url(#yellowSnakeBody)"
152 + stroke-width="9"
153 + stroke-linecap="round"
154 + />
155 + </g>
156 +
157 + <!-- ===== MID LAYER - More visible snakes ===== -->
158 +
159 + <!-- Blue snake winding from bottom-left -->
160 + <g filter="url(#snakeGlow)" opacity="0.6">
161 + <path
162 + d="M 52,130 C 58,118 72,122 78,110 C 84,98 70,92 62,102 C 54,112 50,120 55,128"
163 + fill="none"
164 + stroke="url(#blueSnakeBody)"
165 + stroke-width="13"
166 + stroke-linecap="round"
167 + />
168 + </g>
169 +
170 + <!-- Yellow snake coiling right side -->
171 + <g filter="url(#snakeGlow)" opacity="0.65">
172 + <path
173 + d="M 148,125 C 142,112 128,118 124,105 C 120,92 138,88 144,100 C 150,112 152,120 146,128"
174 + fill="none"
175 + stroke="url(#yellowSnakeBody)"
176 + stroke-width="12"
177 + stroke-linecap="round"
178 + />
179 + </g>
180 +
181 + <!-- ===== MAIN SNAKES - Full detail ===== -->
182 +
183 + <!-- Main Blue Snake - curving from left, head pointing upper-right -->
40 184 <g>
41 185 <path
42 - d="M 50,80 C 20,85 15,50 35,40 C 55,30 60,50 50,60 C 40,70 30,75 50,80 Z"
186 + d="M 45,95
187 + C 52,82 72,88 82,75
188 + C 92,62 85,52 95,45
189 + C 105,38 118,48 125,58"
43 190 fill="none"
44 - stroke="url(#pythonBlue)"
191 + stroke="url(#blueSnakeBody)"
192 + stroke-width="15"
193 + stroke-linecap="round"
194 + filter="url(#snakeGlow)"
195 + />
196 +
197 + <!-- Blue Snake Head - detailed anatomical design -->
198 + <g transform="translate(122, 52) rotate(25)">
199 + <!-- Head base shape (slightly triangular) -->
200 + <path d="M -2,0 Q 2,-7 10,-6 Q 18,-4 18,0 Q 18,4 10,6 Q 2,7 -2,0 Z" fill="url(#pythonBlue)"/>
201 + <path d="M -1,0 Q 2,-6 10,-5 Q 16,-3 16,0 Q 16,3 10,5 Q 2,6 -1,0 Z" fill="url(#blueSnakeBody)"/>
202 + <!-- Brow ridge -->
203 + <path d="M 4,-4 Q 8,-5.5 12,-4" fill="none" stroke="rgba(255,255,255,0.15)" stroke-width="0.8"/>
204 + <!-- Scale pattern on crown -->
205 + <ellipse cx="10" cy="-3" rx="3" ry="1.5" fill="none" stroke="rgba(255,255,255,0.1)" stroke-width="0.4"/>
206 + <ellipse cx="6" cy="-4" rx="2" ry="1" fill="none" stroke="rgba(255,255,255,0.1)" stroke-width="0.3"/>
207 + <!-- Eye socket -->
208 + <ellipse cx="5" cy="-1.5" rx="2.8" ry="2.5" fill="#0a0e15"/>
209 + <!-- Eye -->
210 + <ellipse cx="5" cy="-1.5" rx="2.2" ry="2" fill="#1a1a1a"/>
211 + <!-- Slit pupil -->
212 + <ellipse cx="5" cy="-1.5" rx="0.6" ry="1.8" fill="#ffd43b" filter="url(#eyeGlow)"/>
213 + <!-- Eye highlights -->
214 + <circle cx="4.2" cy="-2.2" r="0.4" fill="#ffffff" opacity="0.9"/>
215 + <circle cx="5.5" cy="-0.8" r="0.2" fill="#ffffff" opacity="0.5"/>
216 + <!-- Heat pit -->
217 + <ellipse cx="0" cy="-1" rx="0.6" ry="0.4" fill="#1a2540"/>
218 + <!-- Nostrils -->
219 + <circle cx="-1" cy="-0.8" r="0.35" fill="#0a0f1a"/>
220 + <circle cx="-1" cy="0.8" r="0.35" fill="#0a0f1a"/>
221 + <!-- Jaw line -->
222 + <path d="M 2,3 Q 8,4 14,2" fill="none" stroke="rgba(0,0,0,0.2)" stroke-width="0.5"/>
223 + <!-- Forked tongue -->
224 + <path d="M -3,0 L -7,0 L -9,-1.5 M -7,0 L -9,1.5" stroke="#dd3344" stroke-width="0.5" stroke-linecap="round" fill="none"/>
225 + </g>
226 + </g>
227 +
228 + <!-- Main Yellow Snake - curving from bottom-right, head pointing left -->
229 + <g>
230 + <path
231 + d="M 158,120
232 + C 145,112 135,125 115,118
233 + C 95,111 100,95 85,88
234 + C 70,81 55,90 48,78"
235 + fill="none"
236 + stroke="url(#yellowSnakeBody)"
237 + stroke-width="15"
238 + stroke-linecap="round"
239 + filter="url(#snakeGlow)"
240 + />
241 +
242 + <!-- Yellow Snake Head - detailed anatomical design -->
243 + <g transform="translate(52, 80) rotate(-135)">
244 + <!-- Head base shape (slightly triangular) -->
245 + <path d="M -2,0 Q 2,-7 10,-6 Q 18,-4 18,0 Q 18,4 10,6 Q 2,7 -2,0 Z" fill="url(#pythonYellow)"/>
246 + <path d="M -1,0 Q 2,-6 10,-5 Q 16,-3 16,0 Q 16,3 10,5 Q 2,6 -1,0 Z" fill="url(#yellowSnakeBody)"/>
247 + <!-- Brow ridge -->
248 + <path d="M 4,-4 Q 8,-5.5 12,-4" fill="none" stroke="rgba(0,0,0,0.1)" stroke-width="0.8"/>
249 + <!-- Scale pattern on crown -->
250 + <ellipse cx="10" cy="-3" rx="3" ry="1.5" fill="none" stroke="rgba(0,0,0,0.08)" stroke-width="0.4"/>
251 + <ellipse cx="6" cy="-4" rx="2" ry="1" fill="none" stroke="rgba(0,0,0,0.08)" stroke-width="0.3"/>
252 + <!-- Eye socket -->
253 + <ellipse cx="5" cy="-1.5" rx="2.8" ry="2.5" fill="#0a0e15"/>
254 + <!-- Eye -->
255 + <ellipse cx="5" cy="-1.5" rx="2.2" ry="2" fill="#1a1a1a"/>
256 + <!-- Slit pupil -->
257 + <ellipse cx="5" cy="-1.5" rx="0.6" ry="1.8" fill="#306998" filter="url(#eyeGlow)"/>
258 + <!-- Eye highlights -->
259 + <circle cx="4.2" cy="-2.2" r="0.4" fill="#ffffff" opacity="0.9"/>
260 + <circle cx="5.5" cy="-0.8" r="0.2" fill="#ffffff" opacity="0.5"/>
261 + <!-- Heat pit -->
262 + <ellipse cx="0" cy="-1" rx="0.6" ry="0.4" fill="#8b7500"/>
263 + <!-- Nostrils -->
264 + <circle cx="-1" cy="-0.8" r="0.35" fill="#8b6914"/>
265 + <circle cx="-1" cy="0.8" r="0.35" fill="#8b6914"/>
266 + <!-- Jaw line -->
267 + <path d="M 2,3 Q 8,4 14,2" fill="none" stroke="rgba(0,0,0,0.15)" stroke-width="0.5"/>
268 + <!-- Forked tongue -->
269 + <path d="M -3,0 L -7,0 L -9,-1.5 M -7,0 L -9,1.5" stroke="#dd3344" stroke-width="0.5" stroke-linecap="round" fill="none"/>
270 + </g>
271 + </g>
272 +
273 + <!-- Third Snake (Blue) - slithering across middle -->
274 + <g>
275 + <path
276 + d="M 35,108
277 + C 50,100 65,112 85,102
278 + C 105,92 115,105 130,95
279 + C 145,85 155,95 162,88"
280 + fill="none"
281 + stroke="url(#blueSnakeBody)"
282 + stroke-width="12"
283 + stroke-linecap="round"
284 + filter="url(#snakeGlow)"
285 + />
286 +
287 + <!-- Third snake head - detailed smaller version -->
288 + <g transform="translate(159, 86) rotate(20)">
289 + <!-- Head base -->
290 + <path d="M -1,0 Q 1,-5 6,-4.5 Q 11,-3 11,0 Q 11,3 6,4.5 Q 1,5 -1,0 Z" fill="url(#pythonBlue)"/>
291 + <path d="M 0,0 Q 1.5,-4 6,-3.5 Q 10,-2.5 10,0 Q 10,2.5 6,3.5 Q 1.5,4 0,0 Z" fill="url(#blueSnakeBody)"/>
292 + <!-- Brow ridge -->
293 + <path d="M 2,-2.5 Q 5,-3.5 8,-2.5" fill="none" stroke="rgba(255,255,255,0.12)" stroke-width="0.5"/>
294 + <!-- Eye socket -->
295 + <ellipse cx="3" cy="-1" rx="1.8" ry="1.6" fill="#0a0e15"/>
296 + <!-- Eye -->
297 + <ellipse cx="3" cy="-1" rx="1.4" ry="1.3" fill="#1a1a1a"/>
298 + <!-- Slit pupil -->
299 + <ellipse cx="3" cy="-1" rx="0.4" ry="1.1" fill="#ffd43b" filter="url(#eyeGlow)"/>
300 + <!-- Highlight -->
301 + <circle cx="2.5" cy="-1.5" r="0.25" fill="#ffffff" opacity="0.85"/>
302 + <!-- Nostril -->
303 + <circle cx="-0.5" cy="0" r="0.25" fill="#1a2540"/>
304 + <!-- Forked tongue -->
305 + <path d="M -2,0 L -4.5,0 L -6,-1 M -4.5,0 L -6,1" stroke="#dd3344" stroke-width="0.4" stroke-linecap="round" fill="none"/>
306 + </g>
307 + </g>
308 +
309 + <!-- Fourth Snake (Yellow) - smaller, peeking from upper left -->
310 + <g opacity="0.85">
311 + <path
312 + d="M 38,72 C 48,65 62,72 72,62 C 82,52 75,42 85,38"
313 + fill="none"
314 + stroke="url(#yellowSnakeBody)"
45 315 stroke-width="10"
46 316 stroke-linecap="round"
317 + filter="url(#snakeGlow)"
47 318 />
48 - <path
49 - d="M 35,40 L 28,35 L 38,35 C 45,35 45,40 35,40 Z"
50 - fill="url(#pythonBlue)"
51 - />
52 - <circle cx="33" cy="37" r="1.2" fill="#ffdd57" filter="url(#eyeGlow)"/>
319 +
320 + <!-- Fourth snake head - detailed smaller version -->
321 + <g transform="translate(83, 36) rotate(30)">
322 + <!-- Head base -->
323 + <path d="M -1,0 Q 1,-4.5 5,-4 Q 9,-2.5 9,0 Q 9,2.5 5,4 Q 1,4.5 -1,0 Z" fill="url(#pythonYellow)"/>
324 + <path d="M 0,0 Q 1,-3.5 5,-3 Q 8,-2 8,0 Q 8,2 5,3 Q 1,3.5 0,0 Z" fill="url(#yellowSnakeBody)"/>
325 + <!-- Brow ridge -->
326 + <path d="M 1.5,-2 Q 4,-3 6.5,-2" fill="none" stroke="rgba(0,0,0,0.08)" stroke-width="0.4"/>
327 + <!-- Eye socket -->
328 + <ellipse cx="2.5" cy="-0.8" rx="1.5" ry="1.4" fill="#0a0e15"/>
329 + <!-- Eye -->
330 + <ellipse cx="2.5" cy="-0.8" rx="1.2" ry="1.1" fill="#1a1a1a"/>
331 + <!-- Slit pupil -->
332 + <ellipse cx="2.5" cy="-0.8" rx="0.35" ry="0.9" fill="#306998" filter="url(#eyeGlow)"/>
333 + <!-- Highlight -->
334 + <circle cx="2.1" cy="-1.2" r="0.2" fill="#ffffff" opacity="0.85"/>
335 + <!-- Nostril -->
336 + <circle cx="-0.3" cy="0" r="0.2" fill="#8b6914"/>
337 + <!-- Forked tongue -->
338 + <path d="M -1.5,0 L -3.5,0 L -4.8,-0.8 M -3.5,0 L -4.8,0.8" stroke="#dd3344" stroke-width="0.35" stroke-linecap="round" fill="none"/>
339 + </g>
53 340 </g>
54 341
55 - <!-- Snake 2 (Yellow) -->
56 - <g>
342 + <!-- Fifth Snake (Blue) - tiny one lurking bottom right -->
343 + <g opacity="0.7">
57 344 <path
58 - d="M 50,20 C 80,15 85,50 65,60 C 45,70 40,50 50,40 C 60,30 70,25 50,20 Z"
345 + d="M 152,130 C 145,136 135,130 128,138 C 122,146 130,152 125,158"
59 346 fill="none"
60 - stroke="url(#pythonYellow)"
347 + stroke="url(#blueSnakeBody)"
348 + stroke-width="8"
349 + stroke-linecap="round"
350 + filter="url(#snakeGlow)"
351 + />
352 + </g>
353 +
354 + <!-- Loose coil segment (Yellow) in bottom left -->
355 + <g opacity="0.55">
356 + <path
357 + d="M 62,148 C 72,142 80,150 88,144 C 96,138 92,132 98,128"
358 + fill="none"
359 + stroke="url(#yellowSnakeBody)"
61 360 stroke-width="10"
62 361 stroke-linecap="round"
63 - />
64 - <path
65 - d="M 65,60 L 72,65 L 62,65 C 55,65 55,60 65,60 Z"
66 - fill="url(#pythonYellow)"
67 - />
68 - <circle cx="67" cy="63" r="1.2" fill="#3472a8" filter="url(#eyeGlow)"/>
69 - </g>
70 -
71 - <!-- Snake 3 (Blue - background) -->
72 - <g>
73 - <path
74 - d="M 25,70 C 35,90 65,90 75,70 C 85,50 65,50 55,60 C 45,70 35,70 25,70 Z"
75 - fill="none"
76 - stroke="url(#pythonBlue)"
77 - stroke-width="8"
78 - stroke-linecap="round"
79 - opacity="0.8"
362 + filter="url(#snakeGlow)"
80 363 />
81 364 </g>
82 365
83 - <!-- Snake 4 (Yellow - background) -->
84 - <g>
366 + <!-- Background tail fragment -->
367 + <g opacity="0.45">
85 368 <path
86 - d="M 75,30 C 65,10 35,10 25,30 C 15,50 35,50 45,40 C 55,30 65,30 75,30 Z"
369 + d="M 115,155 C 108,148 100,155 90,150"
87 370 fill="none"
88 - stroke="url(#pythonYellow)"
89 - stroke-width="8"
371 + stroke="url(#blueSnakeBody)"
372 + stroke-width="7"
90 373 stroke-linecap="round"
91 - opacity="0.8"
92 374 />
93 375 </g>
376 +
94 377 </g>
378 +
379 + <!-- Subtle highlight on hexagon edge -->
380 + <polygon
381 + points="100,8 181.6,55 181.6,145 100,192 18.4,145 18.4,55"
382 + fill="none"
383 + stroke="url(#hexBorder)"
384 + stroke-width="1.5"
385 + opacity="0.4"
386 + />
95 387 </svg>
\ No newline at end of file
  @@ -61,7 +61,7 @@ Add Snakepit as a dependency in your `mix.exs`:
61 61 # mix.exs
62 62 def deps do
63 63 [
64 - {:snakepit, "~> 0.8.3"}
64 + {:snakepit, "~> 0.8.8"}
65 65 ]
66 66 end
67 67 ```
  @@ -0,0 +1,482 @@
1 + # Timeout Configuration Guide
2 +
3 + Snakepit v0.8.8 introduces a unified timeout architecture designed for reliability in production deployments. This guide covers timeout profiles, deadline propagation, and configuration strategies for different workloads.
4 +
5 + ---
6 +
7 + ## Table of Contents
8 +
9 + 1. [Overview](#overview)
10 + 2. [Timeout Profiles](#timeout-profiles)
11 + 3. [How Timeouts Work](#how-timeouts-work)
12 + 4. [Configuration Reference](#configuration-reference)
13 + 5. [Common Scenarios](#common-scenarios)
14 + 6. [Debugging Timeout Issues](#debugging-timeout-issues)
15 + 7. [Migration Guide](#migration-guide)
16 +
17 + ---
18 +
19 + ## Overview
20 +
21 + ### The Problem
22 +
23 + Prior to v0.8.8, Snakepit had fragmented timeout configuration with 7+ independent timeout keys that didn't coordinate:
24 +
25 + | Issue | Symptom |
26 + |-------|---------|
27 + | `pool_request_timeout` vs `grpc_command_timeout` confusion | Unclear which is outer, which is inner |
28 + | Queue wait consumed budget invisibly | Inner timeouts didn't account for queue time |
29 + | GenServer.call timeouts fired before inner timeouts | Unhandled exits instead of structured errors |
30 +
31 + ### The Solution
32 +
33 + Snakepit now uses a **single-budget, derived deadlines** architecture:
34 +
35 + 1. **One top-level timeout budget** set at request entry
36 + 2. **Deadline propagation** tracks remaining time through the stack
37 + 3. **Inner timeouts derived** from remaining budget minus safety margins
38 + 4. **Profile-based defaults** for different deployment scenarios
39 +
40 + ---
41 +
42 + ## Timeout Profiles
43 +
44 + Profiles provide sensible defaults for common deployment scenarios. Configure via:
45 +
46 + ```elixir
47 + config :snakepit, timeout_profile: :production
48 + ```
49 +
50 + ### Profile Comparison
51 +
52 + | Profile | default_timeout | stream_timeout | queue_timeout | Use Case |
53 + |---------|-----------------|----------------|---------------|----------|
54 + | `:balanced` | 300s (5m) | 900s (15m) | 10s | General purpose, default |
55 + | `:production` | 300s (5m) | 900s (15m) | 10s | Production deployments |
56 + | `:production_strict` | 60s | 300s (5m) | 5s | Latency-sensitive APIs |
57 + | `:development` | 900s (15m) | 3600s (60m) | 60s | Local development, debugging |
58 + | `:ml_inference` | 900s (15m) | 3600s (60m) | 60s | ML model inference |
59 + | `:batch` | 3600s (60m) | ∞ | 300s (5m) | Batch processing jobs |
60 +
61 + ### Profile Selection Guidelines
62 +
63 + | Workload Type | Recommended Profile | Rationale |
64 + |---------------|---------------------|-----------|
65 + | Web API backends | `:production_strict` | Fast failure for user-facing requests |
66 + | Background jobs | `:batch` | Long-running operations need patience |
67 + | ML inference | `:ml_inference` | Model loading and inference are slow |
68 + | Development | `:development` | Generous timeouts for debugging |
69 + | Mixed workloads | `:balanced` | Good defaults for most cases |
70 +
71 + ### Using Profiles
72 +
73 + ```elixir
74 + # config/runtime.exs
75 +
76 + # Production API server
77 + config :snakepit, timeout_profile: :production_strict
78 +
79 + # ML inference service
80 + config :snakepit, timeout_profile: :ml_inference
81 +
82 + # Batch processing worker
83 + config :snakepit, timeout_profile: :batch
84 + ```
85 +
86 + ---
87 +
88 + ## How Timeouts Work
89 +
90 + ### The Timeout Stack
91 +
92 + Requests flow through multiple layers, each with its own timeout:
93 +
94 + ```
95 + ┌─────────────────────────────────────────────────────────────────┐
96 + │ User Code: Snakepit.execute("cmd", args, timeout: 60_000) │
97 + └─────────────────────────────────────────────────────────────────┘
98 + │
99 + ▼
100 + ┌─────────────────────────────────────────────────────────────────┐
101 + │ Pool Layer │
102 + │ ├─ GenServer.call timeout: 60_000 │
103 + │ ├─ Queue wait (if workers busy): up to queue_timeout │
104 + │ └─ Deadline stored: now + 60_000 │
105 + └─────────────────────────────────────────────────────────────────┘
106 + │
107 + ▼
108 + ┌─────────────────────────────────────────────────────────────────┐
109 + │ Worker Layer │
110 + │ ├─ GenServer.call timeout: remaining - 1000ms margin │
111 + │ └─ Forwards to gRPC adapter │
112 + └─────────────────────────────────────────────────────────────────┘
113 + │
114 + ▼
115 + ┌─────────────────────────────────────────────────────────────────┐
116 + │ gRPC Layer │
117 + │ ├─ gRPC call timeout: remaining - 1200ms total margins │
118 + │ └─ Actual Python execution │
119 + └─────────────────────────────────────────────────────────────────┘
120 + ```
121 +
122 + ### Margin Formula
123 +
124 + Inner timeouts are derived from the total budget minus safety margins:
125 +
126 + ```
127 + rpc_timeout = total_timeout - worker_call_margin_ms - pool_reply_margin_ms
128 + ```
129 +
130 + | Margin | Default | Purpose |
131 + |--------|---------|---------|
132 + | `worker_call_margin_ms` | 1000ms | GenServer.call overhead to worker |
133 + | `pool_reply_margin_ms` | 200ms | Pool reply processing overhead |
134 +
135 + **Example**: With a 60-second total budget:
136 + - Total: 60,000ms
137 + - Worker margin: -1,000ms
138 + - Pool margin: -200ms
139 + - **RPC timeout: 58,800ms**
140 +
141 + This ensures inner timeouts expire *before* outer GenServer.call timeouts, producing structured `{:error, %Snakepit.Error{}}` returns instead of unhandled exits.
142 +
143 + ### Deadline Propagation
144 +
145 + When a request enters the pool, a deadline is computed and stored:
146 +
147 + ```elixir
148 + # Inside Pool.execute/3
149 + deadline_ms = System.monotonic_time(:millisecond) + timeout
150 + opts_with_deadline = Keyword.put(opts, :deadline_ms, deadline_ms)
151 + ```
152 +
153 + As the request moves through the stack:
154 +
155 + 1. **Queue handler** uses `effective_queue_timeout_ms/2` to respect deadline
156 + 2. **Worker execution** uses `derive_rpc_timeout_from_opts/2` to compute remaining budget
157 + 3. **All layers** return structured errors instead of crashing on timeout
158 +
159 + ### Queue-Aware Timeouts
160 +
161 + If a request waits in queue, that time is subtracted from the budget:
162 +
163 + ```elixir
164 + # Request with 60s budget waits 5s in queue
165 + # Remaining budget for execution: 55s (minus margins)
166 + ```
167 +
168 + This prevents the common bug where queue wait + execution time exceeds the user's expected timeout.
169 +
170 + ---
171 +
172 + ## Configuration Reference
173 +
174 + ### Profile-Based Configuration (Recommended)
175 +
176 + ```elixir
177 + # config/runtime.exs
178 + config :snakepit,
179 + timeout_profile: :production,
180 +
181 + # Optional: customize margins
182 + worker_call_margin_ms: 1000,
183 + pool_reply_margin_ms: 200
184 + ```
185 +
186 + ### Explicit Timeout Configuration
187 +
188 + Override profile defaults with explicit values:
189 +
190 + ```elixir
191 + config :snakepit,
192 + timeout_profile: :production,
193 +
194 + # These override profile-derived values
195 + pool_request_timeout: 120_000, # 2 minutes
196 + pool_streaming_timeout: 600_000, # 10 minutes
197 + pool_queue_timeout: 15_000, # 15 seconds
198 + grpc_command_timeout: 90_000, # 90 seconds
199 + grpc_worker_execute_timeout: 95_000 # 95 seconds
200 + ```
201 +
202 + ### Complete Timeout Options
203 +
204 + | Option | Default | Layer | Description |
205 + |--------|---------|-------|-------------|
206 + | `timeout_profile` | `:balanced` | Global | Profile to use for defaults |
207 + | `pool_request_timeout` | Profile-derived | Pool | GenServer.call timeout for execute |
208 + | `pool_streaming_timeout` | Profile-derived | Pool | GenServer.call timeout for streaming |
209 + | `pool_queue_timeout` | Profile-derived | Pool | Max time request waits in queue |
210 + | `checkout_timeout` | Profile-derived | Pool | Worker checkout for streaming |
211 + | `pool_startup_timeout` | 10,000ms | Pool | Worker startup timeout |
212 + | `pool_await_ready_timeout` | 15,000ms | Pool | Wait for pool initialization |
213 + | `grpc_worker_execute_timeout` | Profile-derived | Worker | GenServer.call to GRPCWorker |
214 + | `grpc_worker_stream_timeout` | 300,000ms | Worker | Streaming GenServer.call |
215 + | `grpc_command_timeout` | Profile-derived | Adapter | gRPC call timeout |
216 + | `grpc_batch_inference_timeout` | 300,000ms | Adapter | Batch inference operations |
217 + | `grpc_large_dataset_timeout` | 600,000ms | Adapter | Large dataset processing |
218 + | `grpc_server_ready_timeout` | 30,000ms | Worker | Python server readiness |
219 + | `worker_ready_timeout` | 30,000ms | Worker | Worker ready notification |
220 + | `graceful_shutdown_timeout_ms` | 6,000ms | Worker | Python process shutdown |
221 + | `worker_call_margin_ms` | 1,000ms | Margin | Worker GenServer.call overhead |
222 + | `pool_reply_margin_ms` | 200ms | Margin | Pool reply overhead |
223 +
224 + ### Per-Call Timeout Override
225 +
226 + Override timeouts for individual calls:
227 +
228 + ```elixir
229 + # Use default from profile
230 + Snakepit.execute("fast_command", %{})
231 +
232 + # Override for slow operation
233 + Snakepit.execute("slow_inference", %{model: "large"}, timeout: 300_000)
234 +
235 + # Streaming with custom timeout
236 + Snakepit.execute_stream("generate", %{}, callback, timeout: 600_000)
237 + ```
238 +
239 + ---
240 +
241 + ## Common Scenarios
242 +
243 + ### Scenario 1: LLM API Calls (60+ seconds)
244 +
245 + **Problem**: Default timeouts are too short for LLM inference.
246 +
247 + **Solution**: Use `:ml_inference` profile or explicit config:
248 +
249 + ```elixir
250 + # Option A: Profile-based
251 + config :snakepit, timeout_profile: :ml_inference
252 +
253 + # Option B: Explicit timeouts
254 + config :snakepit,
255 + pool_request_timeout: 300_000,
256 + grpc_command_timeout: 280_000,
257 + grpc_worker_execute_timeout: 290_000
258 + ```
259 +
260 + **Per-call override**:
261 + ```elixir
262 + Snakepit.execute("llm_generate", %{prompt: prompt}, timeout: 120_000)
263 + ```
264 +
265 + ### Scenario 2: Fast API with Strict SLAs
266 +
267 + **Problem**: Need fast failure for user-facing requests.
268 +
269 + **Solution**: Use `:production_strict` profile:
270 +
271 + ```elixir
272 + config :snakepit, timeout_profile: :production_strict
273 + ```
274 +
275 + This gives you:
276 + - 60-second default timeout
277 + - 5-second queue timeout (fail fast if pool is saturated)
278 + - Quick feedback to users
279 +
280 + ### Scenario 3: Batch Processing Jobs
281 +
282 + **Problem**: Jobs run for hours, need infinite streaming timeout.
283 +
284 + **Solution**: Use `:batch` profile:
285 +
286 + ```elixir
287 + config :snakepit, timeout_profile: :batch
288 + ```
289 +
290 + This gives you:
291 + - 60-minute default timeout
292 + - Infinite streaming timeout
293 + - 5-minute queue tolerance
294 +
295 + ### Scenario 4: Mixed Workloads
296 +
297 + **Problem**: Same pool handles fast and slow operations.
298 +
299 + **Solution**: Use `:balanced` profile with per-call overrides:
300 +
301 + ```elixir
302 + config :snakepit, timeout_profile: :balanced
303 +
304 + # Fast operations use default
305 + Snakepit.execute("lookup", %{id: id})
306 +
307 + # Slow operations override
308 + Snakepit.execute("batch_process", %{data: data}, timeout: 600_000)
309 + ```
310 +
311 + ### Scenario 5: Pool Initialization Takes Too Long
312 +
313 + **Problem**: Starting 50+ workers with heavy model loading.
314 +
315 + **Solution**: Increase startup timeouts:
316 +
317 + ```elixir
318 + config :snakepit,
319 + pool_startup_timeout: 120_000, # 2 min per worker
320 + pool_await_ready_timeout: 600_000, # 10 min total
321 + grpc_server_ready_timeout: 120_000 # 2 min for Python ready
322 + ```
323 +
324 + ### Scenario 6: Workers Killed During Shutdown
325 +
326 + **Problem**: Python cleanup takes longer than 6 seconds.
327 +
328 + **Solution**: Increase graceful shutdown timeout:
329 +
330 + ```elixir
331 + config :snakepit,
332 + graceful_shutdown_timeout_ms: 15_000 # 15 seconds
333 + ```
334 +
335 + **Note**: This must be >= Python's shutdown envelope: `server.stop(2s) + wait_for_termination(3s) = 5s`.
336 +
337 + ---
338 +
339 + ## Debugging Timeout Issues
340 +
341 + ### Enable Debug Logging
342 +
343 + ```elixir
344 + config :snakepit,
345 + log_level: :debug,
346 + log_categories: %{
347 + pool: :debug,
348 + grpc: :debug,
349 + worker: :debug
350 + }
351 + ```
352 +
353 + ### Identify Which Timeout Fired
354 +
355 + | Log Pattern | Timeout Type |
356 + |-------------|--------------|
357 + | `** (exit) {:timeout, {GenServer, :call, ...}` | GenServer.call timeout |
358 + | `gRPC error: %GRPC.RPCError{status: 4...}` | gRPC DEADLINE_EXCEEDED |
359 + | `Request timed out after Xms` | Pool queue timeout |
360 + | `Timeout waiting for Python gRPC server` | Server ready timeout |
361 + | `Pool execute timed out` | Pool-level structured timeout |
362 +
363 + ### Use Telemetry
364 +
365 + ```elixir
366 + :telemetry.attach("timeout-debug", [:snakepit, :request, :executed],
367 + fn _name, measurements, metadata, _config ->
368 + if measurements[:duration_us] > 30_000_000 do # > 30s
369 + Logger.warning("Slow request: #{metadata.command} took #{measurements[:duration_us] / 1_000}ms")
370 + end
371 + end, nil)
372 + ```
373 +
374 + ### Check Pool Stats
375 +
376 + ```elixir
377 + iex> Snakepit.get_stats()
378 + %{
379 + requests: 15432,
380 + queued: 5, # Requests waiting in queue
381 + queue_timeouts: 12, # Queue timeout count
382 + pool_saturated: 3, # Times pool was at capacity
383 + ...
384 + }
385 + ```
386 +
387 + High `queue_timeouts` indicates you need either:
388 + - More workers (`pool_size`)
389 + - Higher `pool_queue_timeout`
390 + - Faster Python operations
391 +
392 + ### Verify Timeout Derivation
393 +
394 + ```elixir
395 + iex> alias Snakepit.Defaults
396 +
397 + # Check current profile
398 + iex> Defaults.timeout_profile()
399 + :balanced
400 +
401 + # Check derived values
402 + iex> Defaults.default_timeout()
403 + 300_000
404 +
405 + iex> Defaults.rpc_timeout(60_000)
406 + 58_800 # 60_000 - 1000 - 200
407 + ```
408 +
409 + ---
410 +
411 + ## Migration Guide
412 +
413 + ### From v0.8.7 and Earlier
414 +
415 + The timeout architecture is **fully backward compatible**. Existing configurations continue to work:
416 +
417 + ```elixir
418 + # This still works in v0.8.8+
419 + config :snakepit,
420 + pool_request_timeout: 60_000,
421 + grpc_command_timeout: 30_000
422 + ```
423 +
424 + **Behavior changes**:
425 + - When explicit timeouts are set, they take precedence over profile-derived values
426 + - When not set, values now derive from the active profile (default: `:balanced`)
427 + - Default values are similar to previous versions for `:balanced` profile
428 +
429 + ### Recommended Migration Path
430 +
431 + 1. **Test with defaults**: Remove explicit timeout config, use profile defaults
432 + 2. **Select appropriate profile**: Choose based on workload type
433 + 3. **Fine-tune if needed**: Override specific values that don't fit
434 +
435 + ```elixir
436 + # Before (v0.8.7)
437 + config :snakepit,
438 + pool_request_timeout: 300_000,
439 + pool_streaming_timeout: 900_000,
440 + pool_queue_timeout: 10_000,
441 + grpc_command_timeout: 280_000
442 +
443 + # After (v0.8.8+) - equivalent behavior
444 + config :snakepit, timeout_profile: :balanced
445 + ```
446 +
447 + ### Breaking Changes
448 +
449 + None. All existing configuration keys are honored and take precedence over profile-derived values.
450 +
451 + ---
452 +
453 + ## API Reference
454 +
455 + ### Snakepit.Defaults Functions
456 +
457 + | Function | Returns | Description |
458 + |----------|---------|-------------|
459 + | `timeout_profiles/0` | `map()` | All available timeout profiles |
460 + | `timeout_profile/0` | `atom()` | Currently configured profile |
461 + | `default_timeout/0` | `timeout()` | Profile's default timeout |
462 + | `stream_timeout/0` | `timeout()` | Profile's streaming timeout |
463 + | `queue_timeout/0` | `timeout()` | Profile's queue timeout |
464 + | `rpc_timeout/1` | `timeout()` | Derive RPC timeout from total budget |
465 + | `worker_call_margin_ms/0` | `integer()` | Worker GenServer.call margin |
466 + | `pool_reply_margin_ms/0` | `integer()` | Pool reply margin |
467 +
468 + ### Snakepit.Pool Functions
469 +
470 + | Function | Returns | Description |
471 + |----------|---------|-------------|
472 + | `get_default_timeout_for_call/3` | `timeout()` | Get timeout for call type |
473 + | `derive_rpc_timeout_from_opts/2` | `timeout()` | Derive RPC timeout from opts with deadline |
474 + | `effective_queue_timeout_ms/2` | `integer()` | Queue timeout respecting deadline |
475 +
476 + ---
477 +
478 + ## Related Guides
479 +
480 + - [Configuration Guide](configuration.md) - General configuration options
481 + - [Worker Profiles](worker-profiles.md) - Process vs Thread profiles
482 + - [Production Guide](production.md) - Deployment best practices
Loading more files…