Packages
snakepit
0.6.1
0.13.0
0.12.0
0.11.1
0.11.0
0.10.1
0.10.0
0.9.1
0.9.0
0.8.9
0.8.8
0.8.7
0.8.6
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.7
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.11
0.6.10
0.6.9
0.6.8
0.6.7
0.6.6
0.6.5
0.6.4
0.6.3
0.6.2
0.6.1
0.6.0
0.5.1
0.5.0
0.4.3
0.4.2
0.4.1
0.4.0
0.3.3
0.3.2
0.3.1
0.3.0
0.2.1
0.2.0
0.1.2
0.1.1
0.1.0
High-performance pooler and session manager for external language integrations. Supports Python, Node.js, Ruby, and more with gRPC streaming, session management, and production-ready process cleanup.
Current section
53 Versions
Jump to
Current section
53 Versions
Compare versions
28
files changed
+553
additions
-1245
deletions
| @@ -7,6 +7,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 | |
| 7 7 | |
| 8 8 | ## [Unreleased] |
| 9 9 | |
| 10 | + ## [0.6.1] - 2025-10-14 |
| 11 | + |
| 12 | + ### Added |
| 13 | + - **Configurable Logging System** |
| 14 | + - New `Snakepit.Logger` module for centralized logging with configurable verbosity |
| 15 | + - Application-level log control via `:snakepit, :log_level` configuration |
| 16 | + - Supported levels: `:debug`, `:info`, `:warning`, `:error`, `:none` |
| 17 | + - All internal Snakepit modules now use `Snakepit.Logger` for consistent log control |
| 18 | + - Default log level set to `:info` for balanced verbosity |
| 19 | + |
| 20 | + ### Changed |
| 21 | + - **Log Suppression for Clean Output** |
| 22 | + - Default configuration now suppresses verbose logs for cleaner demo/production output |
| 23 | + - Development environment (`config/dev.exs`) defaults to `:warning` level |
| 24 | + - gRPC interceptor logs can be suppressed via `compile_time_purge_matching` in config |
| 25 | + - Updated 25+ modules to use `Snakepit.Logger` instead of direct `Logger` calls |
| 26 | + - Examples and showcase applications benefit from reduced noise |
| 27 | + |
| 28 | + ### Configuration |
| 29 | + - **New Configuration Option**: `log_level` in `:snakepit` application config |
| 30 | + ```elixir |
| 31 | + # config/config.exs |
| 32 | + config :snakepit, |
| 33 | + log_level: :warning # Options: :debug, :info, :warning, :error, :none |
| 34 | + ``` |
| 35 | + |
| 36 | + ### Documentation |
| 37 | + - Added log configuration section to README with usage examples |
| 38 | + - Documented how to suppress gRPC logs and Snakepit internal logs |
| 39 | + - Updated configuration examples throughout documentation |
| 40 | + |
| 41 | + ### Notes |
| 42 | + - **No Breaking Changes**: Existing configurations continue to work with default `:info` level |
| 43 | + - Log suppression is optional - users can keep verbose logs by setting `log_level: :debug` |
| 44 | + - Provides cleaner output for production deployments and demos |
| 45 | + - Internal logging remains fully accessible for debugging when needed |
| 46 | + |
| 47 | + --- |
| 48 | + |
| 10 49 | ## [0.6.0] - 2025-10-11 |
| 11 50 | |
| 12 51 | ### Added - Phase 1: Dual-Mode Architecture Foundation |
| @@ -30,6 +30,7 @@ Snakepit is a battle-tested Elixir library that provides a robust pooling system | |
| 30 30 | |
| 31 31 | ## 📋 Table of Contents |
| 32 32 | |
| 33 | + - [What's New in v0.6.1](#whats-new-in-v061) |
| 33 34 | - [What's New in v0.6.0](#whats-new-in-v060) |
| 34 35 | - [Breaking Changes (v0.5.0)](#️-breaking-changes-v050) |
| 35 36 | - [What's New in v0.5.1](#whats-new-in-v051) |
| @@ -103,6 +104,82 @@ For **non-DSPex users**, if you're using these classes directly: | |
| 103 104 | |
| 104 105 | --- |
| 105 106 | |
| 107 | + ## 🆕 What's New in v0.6.1 |
| 108 | + |
| 109 | + ### 🔇 Configurable Logging System |
| 110 | + |
| 111 | + Snakepit v0.6.1 introduces fine-grained control over internal logging for cleaner output in production and demo environments. |
| 112 | + |
| 113 | + #### Key Features |
| 114 | + |
| 115 | + - **Centralized Log Control**: New `Snakepit.Logger` module provides consistent logging across all internal modules |
| 116 | + - **Application-Level Configuration**: Simple `:log_level` setting controls all Snakepit logs |
| 117 | + - **Five Log Levels**: `:debug`, `:info`, `:warning`, `:error`, `:none` |
| 118 | + - **No Breaking Changes**: Defaults to `:info` level for backward compatibility |
| 119 | + |
| 120 | + #### Configuration Examples |
| 121 | + |
| 122 | + **Clean Output (Recommended for Production/Demos)**: |
| 123 | + ```elixir |
| 124 | + # config/config.exs |
| 125 | + config :snakepit, |
| 126 | + log_level: :warning, # Only warnings and errors |
| 127 | + adapter_module: Snakepit.Adapters.GRPCPython, |
| 128 | + pool_config: %{pool_size: 8} |
| 129 | + |
| 130 | + # Also suppress gRPC logs |
| 131 | + config :logger, |
| 132 | + level: :warning, |
| 133 | + compile_time_purge_matching: [ |
| 134 | + [application: :grpc, level_lower_than: :error] |
| 135 | + ] |
| 136 | + ``` |
| 137 | + |
| 138 | + **Verbose Logging (Development/Debugging)**: |
| 139 | + ```elixir |
| 140 | + # config/dev.exs |
| 141 | + config :snakepit, |
| 142 | + log_level: :debug # See everything |
| 143 | + |
| 144 | + config :logger, level: :debug |
| 145 | + ``` |
| 146 | + |
| 147 | + **Complete Silence**: |
| 148 | + ```elixir |
| 149 | + config :snakepit, |
| 150 | + log_level: :none # No Snakepit logs at all |
| 151 | + ``` |
| 152 | + |
| 153 | + #### What Gets Suppressed |
| 154 | + |
| 155 | + With `log_level: :warning`: |
| 156 | + - ✅ Worker initialization messages |
| 157 | + - ✅ Pool startup progress |
| 158 | + - ✅ Session creation logs |
| 159 | + - ✅ gRPC connection details |
| 160 | + - ✅ Tool registration confirmations |
| 161 | + - ❌ Warnings and errors (still shown) |
| 162 | + |
| 163 | + #### Module Coverage |
| 164 | + |
| 165 | + Updated 25+ internal modules to use `Snakepit.Logger`: |
| 166 | + - `Snakepit.Config` - Configuration validation |
| 167 | + - `Snakepit.Pool.*` - Pool management, worker lifecycle |
| 168 | + - `Snakepit.Bridge.*` - Session and tool management |
| 169 | + - `Snakepit.GRPC.*` - gRPC communication |
| 170 | + - `Snakepit.Adapters.*` - Adapter implementations |
| 171 | + - `Snakepit.Worker.*` - Worker lifecycle |
| 172 | + - `Snakepit.Telemetry` - Monitoring and metrics |
| 173 | + |
| 174 | + #### Benefits |
| 175 | + |
| 176 | + - **Cleaner Demos**: Show only your application output, not infrastructure logs |
| 177 | + - **Production Ready**: Reduce log volume in production environments |
| 178 | + - **Flexible Debugging**: Turn on verbose logs when troubleshooting |
| 179 | + - **Selective Visibility**: Keep important warnings/errors while hiding noise |
| 180 | + |
| 181 | + --- |
| 182 | + |
| 106 183 | ## 🆕 What's New in v0.6.0 |
| 107 184 | |
| 108 185 | ### 🎯 Dual-Mode Parallelism Architecture |
| @@ -867,6 +944,12 @@ Sessions provide: | |
| 867 944 | config :snakepit, |
| 868 945 | pooling_enabled: true, |
| 869 946 | adapter_module: Snakepit.Adapters.GRPCPython, # gRPC-based communication |
| 947 | + |
| 948 | + # Control Snakepit's internal logging |
| 949 | + # Options: :debug, :info, :warning, :error, :none |
| 950 | + # Set to :warning or :none for clean output in production/demos |
| 951 | + log_level: :info, # Default (balanced verbosity) |
| 952 | + |
| 870 953 | grpc_config: %{ |
| 871 954 | base_port: 50051, # Starting port for gRPC servers |
| 872 955 | port_range: 100 # Port range for worker allocation |
| @@ -874,6 +957,13 @@ config :snakepit, | |
| 874 957 | pool_config: %{ |
| 875 958 | pool_size: 8 # Default: System.schedulers_online() * 2 |
| 876 959 | } |
| 960 | + |
| 961 | + # Optional: Also suppress gRPC library logs |
| 962 | + config :logger, |
| 963 | + level: :warning, |
| 964 | + compile_time_purge_matching: [ |
| 965 | + [application: :grpc, level_lower_than: :error] |
| 966 | + ] |
| 877 967 | ``` |
| 878 968 | |
| 879 969 | ### gRPC Configuration |
| @@ -932,6 +1022,38 @@ Application.start(:snakepit) | |
| 932 1022 | |
| 933 1023 | ### Running the Examples |
| 934 1024 | |
| 1025 | + Most examples use `elixir` directly (with Mix.install), but some v0.6.0 demos require the compiled project and use `mix run`: |
| 1026 | + |
| 1027 | + #### Quick Reference |
| 1028 | + |
| 1029 | + ```bash |
| 1030 | + # Basic gRPC examples (use elixir) |
| 1031 | + elixir examples/grpc_basic.exs # Simple ping, echo, add operations |
| 1032 | + elixir examples/grpc_sessions.exs # Session management patterns |
| 1033 | + elixir examples/grpc_streaming.exs # Streaming data operations |
| 1034 | + elixir examples/grpc_concurrent.exs # Concurrent execution (default: 4 workers) |
| 1035 | + elixir examples/grpc_advanced.exs # Advanced error handling |
| 1036 | + elixir examples/grpc_streaming_demo.exs # Real-time streaming demo |
| 1037 | + |
| 1038 | + # Bidirectional tool bridge (use elixir) |
| 1039 | + elixir examples/bidirectional_tools_demo.exs # Interactive demo |
| 1040 | + elixir examples/bidirectional_tools_demo_auto.exs # Auto-run server version |
| 1041 | + |
| 1042 | + # v0.6.0 demos using compiled modules (use mix run) |
| 1043 | + mix run examples/threaded_profile_demo.exs # Thread profile config |
| 1044 | + mix run examples/dual_mode/process_vs_thread_comparison.exs # Profile comparison |
| 1045 | + mix run examples/dual_mode/hybrid_pools.exs # Multiple pool profiles |
| 1046 | + mix run examples/dual_mode/gil_aware_selection.exs # Auto Python version detection |
| 1047 | + mix run examples/lifecycle/ttl_recycling_demo.exs # TTL worker recycling |
| 1048 | + mix run examples/monitoring/telemetry_integration.exs # Telemetry setup |
| 1049 | + ``` |
| 1050 | + |
| 1051 | + **Status**: 159/159 tests passing (100%) with default Python! All examples are production-ready. |
| 1052 | + |
| 1053 | + **Note**: v0.6.0 feature demos access compiled Snakepit modules (`Snakepit.PythonVersion`, `Snakepit.Compatibility`, etc.) and require `mix run` to work properly. |
| 1054 | + |
| 1055 | + --- |
| 1056 | + |
| 935 1057 | #### Working Examples (Fully Functional) |
| 936 1058 | |
| 937 1059 | These examples work out-of-the-box with the default ShowcaseAdapter: |
| @@ -954,30 +1076,45 @@ elixir examples/bidirectional_tools_demo.exs | |
| 954 1076 | |
| 955 1077 | --- |
| 956 1078 | |
| 957 | - #### Work-in-Progress Examples |
| 1079 | + #### v0.6.0 Feature Demonstrations |
| 958 1080 | |
| 959 | - These examples demonstrate aspirational features but require additional tool implementations: |
| 1081 | + All v0.6.0 examples showcase configuration patterns and best practices: |
| 960 1082 | |
| 961 1083 | ```bash |
| 962 | - # Session management - needs parameter fixes |
| 963 | - elixir examples/grpc_sessions.exs |
| 964 | - # Status: Partial - register_variable parameter mismatch |
| 1084 | + # Dual-mode architecture |
| 1085 | + elixir examples/dual_mode/process_vs_thread_comparison.exs # Side-by-side comparison |
| 1086 | + elixir examples/dual_mode/hybrid_pools.exs # Multiple pools with different profiles |
| 1087 | + elixir examples/dual_mode/gil_aware_selection.exs # Automatic Python 3.13+ detection |
| 965 1088 | |
| 966 | - # Variable system - needs parameter fixes |
| 967 | - elixir examples/grpc_variables.exs |
| 968 | - # Status: Partial - requires 'constraints' parameter |
| 1089 | + # Worker lifecycle management |
| 1090 | + elixir examples/lifecycle/ttl_recycling_demo.exs # TTL-based automatic recycling |
| 969 1091 | |
| 970 | - # Advanced features - needs custom tools |
| 971 | - elixir examples/grpc_advanced.exs |
| 972 | - # Status: WIP - requires validate_input, transform_data, etc. |
| 1092 | + # Monitoring & telemetry |
| 1093 | + elixir examples/monitoring/telemetry_integration.exs # Telemetry events setup |
| 973 1094 | |
| 974 | - # Streaming - needs streaming tool implementations |
| 975 | - elixir examples/grpc_streaming.exs |
| 976 | - elixir examples/grpc_streaming_demo.exs 100 |
| 977 | - # Status: WIP - requires streaming-specific tools |
| 1095 | + # Thread profile (Python 3.13+ free-threading) |
| 1096 | + elixir examples/threaded_profile_demo.exs # Thread profile configuration patterns |
| 978 1097 | ``` |
| 979 1098 | |
| 980 | - **Note**: These examples were written for a more complete adapter. Contributions welcome to implement missing tools in ShowcaseAdapter. |
| 1099 | + --- |
| 1100 | + |
| 1101 | + #### Additional Examples |
| 1102 | + |
| 1103 | + These examples demonstrate advanced features requiring additional tool implementations: |
| 1104 | + |
| 1105 | + ```bash |
| 1106 | + # Session management patterns |
| 1107 | + elixir examples/grpc_sessions.exs |
| 1108 | + |
| 1109 | + # Streaming operations |
| 1110 | + elixir examples/grpc_streaming.exs |
| 1111 | + elixir examples/grpc_streaming_demo.exs |
| 1112 | + |
| 1113 | + # Advanced error handling |
| 1114 | + elixir examples/grpc_advanced.exs |
| 1115 | + ``` |
| 1116 | + |
| 1117 | + **Note**: Some advanced examples may require custom adapter tools. See [Creating Custom Adapters](#creating-custom-adapters) for implementation details. |
| 981 1118 | |
| 982 1119 | --- |
| @@ -8,7 +8,7 @@ | |
| 8 8 | {<<"gRPC Streaming Guide">>, |
| 9 9 | <<"https://hexdocs.pm/snakepit/README_GRPC.html">>}]}. |
| 10 10 | {<<"name">>,<<"snakepit">>}. |
| 11 | - {<<"version">>,<<"0.6.0">>}. |
| 11 | + {<<"version">>,<<"0.6.1">>}. |
| 12 12 | {<<"description">>, |
| 13 13 | <<"High-performance pooler and session manager for external language integrations.\nSupports Python, Node.js, Ruby, and more with gRPC streaming, session management,\nand production-ready process cleanup.">>}. |
| 14 14 | {<<"elixir">>,<<"~> 1.18">>}. |
| @@ -19,14 +19,15 @@ | |
| 19 19 | <<"lib/snakepit/worker_profile/thread.ex">>, |
| 20 20 | <<"lib/snakepit/worker_profile/process.ex">>,<<"lib/snakepit/diagnostics">>, |
| 21 21 | <<"lib/snakepit/diagnostics/profile_inspector.ex">>, |
| 22 | - <<"lib/snakepit/session_helpers.ex">>,<<"lib/snakepit/worker">>, |
| 23 | - <<"lib/snakepit/worker/lifecycle_manager.ex">>,<<"lib/snakepit/config.ex">>, |
| 24 | - <<"lib/snakepit/process_killer.ex">>,<<"lib/snakepit/adapters">>, |
| 25 | - <<"lib/snakepit/adapters/grpc_python.ex">>,<<"lib/snakepit/adapter.ex">>, |
| 26 | - <<"lib/snakepit/grpc_worker.ex">>,<<"lib/snakepit/worker_profile.ex">>, |
| 27 | - <<"lib/snakepit/application.ex">>,<<"lib/snakepit/utils.ex">>, |
| 28 | - <<"lib/snakepit/python_version.ex">>,<<"lib/snakepit/compatibility.ex">>, |
| 29 | - <<"lib/snakepit/bridge">>,<<"lib/snakepit/bridge/session_store.ex">>, |
| 22 | + <<"lib/snakepit/session_helpers.ex">>,<<"lib/snakepit/logger.ex">>, |
| 23 | + <<"lib/snakepit/worker">>,<<"lib/snakepit/worker/lifecycle_manager.ex">>, |
| 24 | + <<"lib/snakepit/config.ex">>,<<"lib/snakepit/process_killer.ex">>, |
| 25 | + <<"lib/snakepit/adapters">>,<<"lib/snakepit/adapters/grpc_python.ex">>, |
| 26 | + <<"lib/snakepit/adapter.ex">>,<<"lib/snakepit/grpc_worker.ex">>, |
| 27 | + <<"lib/snakepit/worker_profile.ex">>,<<"lib/snakepit/application.ex">>, |
| 28 | + <<"lib/snakepit/utils.ex">>,<<"lib/snakepit/python_version.ex">>, |
| 29 | + <<"lib/snakepit/compatibility.ex">>,<<"lib/snakepit/bridge">>, |
| 30 | + <<"lib/snakepit/bridge/session_store.ex">>, |
| 30 31 | <<"lib/snakepit/bridge/session.ex">>, |
| 31 32 | <<"lib/snakepit/bridge/tool_registry.ex">>,<<"lib/snakepit/grpc">>, |
| 32 33 | <<"lib/snakepit/grpc/endpoint.ex">>,<<"lib/snakepit/grpc/client.ex">>, |
| @@ -37,7 +38,7 @@ | |
| 37 38 | <<"lib/snakepit/grpc/generated">>, |
| 38 39 | <<"lib/snakepit/grpc/generated/snakepit_bridge.pb.ex">>, |
| 39 40 | <<"lib/snakepit/pool">>,<<"lib/snakepit/pool/pool.ex">>, |
| 40 | - <<"lib/snakepit/pool/pool.ex.backup">>,<<"lib/snakepit/pool/registry.ex">>, |
| 41 | + <<"lib/snakepit/pool/registry.ex">>, |
| 41 42 | <<"lib/snakepit/pool/worker_supervisor.ex">>, |
| 42 43 | <<"lib/snakepit/pool/worker_starter_registry.ex">>, |
| 43 44 | <<"lib/snakepit/pool/application_cleanup.ex">>, |
| @@ -42,6 +42,7 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 42 42 | @behaviour Snakepit.Adapter |
| 43 43 | |
| 44 44 | require Logger |
| 45 | + alias Snakepit.Logger, as: SLog |
| 45 46 | |
| 46 47 | @impl true |
| 47 48 | def executable_path do |
| @@ -236,7 +237,7 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 236 237 | # backoff: multiplier for exponential growth (doubles each retry) |
| 237 238 | defp retry_connect(_port, 0, _base_delay, _backoff) do |
| 238 239 | # All retries exhausted |
| 239 | - Logger.error("gRPC connection failed after all retries") |
| 240 | + SLog.error("gRPC connection failed after all retries") |
| 240 241 | {:error, :connection_failed_after_retries} |
| 241 242 | end |
| 242 243 | |
| @@ -244,7 +245,7 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 244 245 | case Snakepit.GRPC.Client.connect(port) do |
| 245 246 | {:ok, channel} -> |
| 246 247 | # Connection successful! |
| 247 | - Logger.debug("gRPC connection established to port #{port}") |
| 248 | + SLog.debug("gRPC connection established to port #{port}") |
| 248 249 | {:ok, %{channel: channel, port: port}} |
| 249 250 | |
| 250 251 | {:error, reason} when reason in [:connection_refused, :unavailable, :internal] -> |
| @@ -254,7 +255,7 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 254 255 | jitter = :rand.uniform(div(max(delay, 4), 4)) |
| 255 256 | actual_delay = delay + jitter |
| 256 257 | |
| 257 | - Logger.debug( |
| 258 | + SLog.debug( |
| 258 259 | "gRPC connection to port #{port} #{reason}. " <> |
| 259 260 | "Retrying in #{actual_delay}ms... (#{retries_left - 1} retries left)" |
| 260 261 | ) |
| @@ -269,7 +270,7 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 269 270 | |
| 270 271 | {:error, reason} -> |
| 271 272 | # For any other error, fail immediately (no retry) |
| 272 | - Logger.error( |
| 273 | + SLog.error( |
| 273 274 | "gRPC connection to port #{port} failed with unexpected reason: #{inspect(reason)}" |
| 274 275 | ) |
| 275 276 | |
| @@ -298,17 +299,17 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 298 299 | Execute a streaming command via gRPC with callback. |
| 299 300 | """ |
| 300 301 | def grpc_execute_stream(connection, command, args, callback_fn, timeout \\ 300_000) do |
| 301 | - Logger.info( |
| 302 | + SLog.info( |
| 302 303 | "[GRPCPython] grpc_execute_stream - command: #{command}, args: #{inspect(args)}, timeout: #{timeout}" |
| 303 304 | ) |
| 304 305 | |
| 305 306 | unless grpc_available?() do |
| 306 | - Logger.error("[GRPCPython] gRPC not available") |
| 307 | + SLog.error("[GRPCPython] gRPC not available") |
| 307 308 | {:error, :grpc_not_available} |
| 308 309 | else |
| 309 310 | # Use the streaming endpoint |
| 310 | - Logger.info("[GRPCPython] Using streaming endpoint") |
| 311 | - Logger.info("[GRPCPython] Channel info: #{inspect(connection.channel)}") |
| 311 | + SLog.info("[GRPCPython] Using streaming endpoint") |
| 312 | + SLog.info("[GRPCPython] Channel info: #{inspect(connection.channel)}") |
| 312 313 | |
| 313 314 | result = |
| 314 315 | Snakepit.GRPC.Client.execute_streaming_tool( |
| @@ -319,14 +320,14 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 319 320 | timeout: timeout |
| 320 321 | ) |
| 321 322 | |
| 322 | - Logger.info("[GRPCPython] execute_streaming_tool returned: #{inspect(result)}") |
| 323 | + SLog.info("[GRPCPython] execute_streaming_tool returned: #{inspect(result)}") |
| 323 324 | |
| 324 325 | case result do |
| 325 326 | {:ok, stream} -> |
| 326 | - Logger.info("[GRPCPython] Starting to consume stream...") |
| 327 | + SLog.info("[GRPCPython] Starting to consume stream...") |
| 327 328 | # Consume the stream directly without Task.async to avoid deadlock |
| 328 329 | try do |
| 329 | - Logger.info("[GRPCPython] Starting Enum.each on stream") |
| 330 | + SLog.info("[GRPCPython] Starting Enum.each on stream") |
| 330 331 | |
| 331 332 | Enum.each(stream, fn chunk -> |
| 332 333 | case chunk do |
| @@ -334,32 +335,32 @@ defmodule Snakepit.Adapters.GRPCPython do | |
| 334 335 | when is_binary(data_bytes) -> |
| 335 336 | # Decode the JSON payload from the chunk |
| 336 337 | decoded_chunk = Jason.decode!(data_bytes) |
| 337 | - Logger.info("[GRPCPython] Decoded chunk: #{inspect(decoded_chunk)}") |
| 338 | + SLog.info("[GRPCPython] Decoded chunk: #{inspect(decoded_chunk)}") |
| 338 339 | callback_fn.(decoded_chunk) |
| 339 340 | |
| 340 341 | {:ok, %Snakepit.Bridge.ToolChunk{is_final: true}} -> |
| 341 342 | # Final empty chunk, ignore |
| 342 | - Logger.info("[GRPCPython] Received final chunk") |
| 343 | + SLog.info("[GRPCPython] Received final chunk") |
| 343 344 | |
| 344 345 | {:error, reason} -> |
| 345 | - Logger.error("[GRPCPython] Stream error: #{inspect(reason)}") |
| 346 | + SLog.error("[GRPCPython] Stream error: #{inspect(reason)}") |
| 346 347 | callback_fn.({:error, reason}) |
| 347 348 | |
| 348 349 | other -> |
| 349 | - Logger.warning("[GRPCPython] Unexpected chunk format: #{inspect(other)}") |
| 350 | + SLog.warning("[GRPCPython] Unexpected chunk format: #{inspect(other)}") |
| 350 351 | end |
| 351 352 | end) |
| 352 353 | |
| 353 | - Logger.info("[GRPCPython] Stream consumption complete") |
| 354 | + SLog.info("[GRPCPython] Stream consumption complete") |
| 354 355 | :ok |
| 355 356 | rescue |
| 356 357 | e -> |
| 357 | - Logger.error("[GRPCPython] Error consuming stream: #{inspect(e)}") |
| 358 | + SLog.error("[GRPCPython] Error consuming stream: #{inspect(e)}") |
| 358 359 | {:error, e} |
| 359 360 | end |
| 360 361 | |
| 361 362 | {:error, reason} -> |
| 362 | - Logger.error("[GRPCPython] Failed to initiate stream: #{inspect(reason)}") |
| 363 | + SLog.error("[GRPCPython] Failed to initiate stream: #{inspect(reason)}") |
| 363 364 | {:error, reason} |
| 364 365 | end |
| 365 366 | end |
| @@ -13,6 +13,7 @@ defmodule Snakepit.Application do | |
| 13 13 | |
| 14 14 | use Application |
| 15 15 | require Logger |
| 16 | + alias Snakepit.Logger, as: SLog |
| 16 17 | |
| 17 18 | @impl true |
| 18 19 | def start(_type, _args) do |
| @@ -48,18 +49,14 @@ defmodule Snakepit.Application do | |
| 48 49 | # Unbuffered output for better logging |
| 49 50 | System.put_env("PYTHONUNBUFFERED", "1") |
| 50 51 | |
| 51 | - Logger.info( |
| 52 | + SLog.info( |
| 52 53 | "🧵 Set Python thread limits: OPENBLAS=#{thread_limits.openblas}, OMP=#{thread_limits.omp}, MKL=#{thread_limits.mkl}, NUMEXPR=#{thread_limits.numexpr}, GRPC=single-threaded" |
| 53 54 | ) |
| 54 55 | |
| 55 56 | # Check if pooling is enabled (default: false to prevent auto-start issues) |
| 56 57 | pooling_enabled = Application.get_env(:snakepit, :pooling_enabled, false) |
| 57 58 | |
| 58 | - IO.inspect( |
| 59 | - pooling_enabled: pooling_enabled, |
| 60 | - env: Mix.env(), |
| 61 | - label: "Snakepit.Application.start/2" |
| 62 | - ) |
| 59 | + SLog.debug("Snakepit.Application.start/2: pooling_enabled=#{pooling_enabled}, env=#{Mix.env()}") |
| 63 60 | |
| 64 61 | # Get gRPC config for the Elixir server |
| 65 62 | grpc_port = Application.get_env(:snakepit, :grpc_port, 50051) |
| @@ -75,7 +72,7 @@ defmodule Snakepit.Application do | |
| 75 72 | pool_config = Application.get_env(:snakepit, :pool_config, %{}) |
| 76 73 | pool_size = Map.get(pool_config, :pool_size, System.schedulers_online() * 2) |
| 77 74 | |
| 78 | - Logger.info("🚀 Starting Snakepit with pooling enabled (size: #{pool_size})") |
| 75 | + SLog.info("🚀 Starting Snakepit with pooling enabled (size: #{pool_size})") |
| 79 76 | |
| 80 77 | # Initialize ETS table for thread profile capacity tracking |
| 81 78 | # Must be created before any workers start |
| @@ -93,10 +90,7 @@ defmodule Snakepit.Application do | |
| 93 90 | num_acceptors: 20, |
| 94 91 | max_connections: 1000, |
| 95 92 | socket_opts: [backlog: 512] |
| 96 | - ]} |
| 97 | - |> tap(fn spec -> |
| 98 | - IO.inspect(spec, label: "Adding GRPC.Server.Supervisor to children") |
| 99 | - end), |
| 93 | + ]}, |
| 100 94 | |
| 101 95 | # Task supervisor for async pool operations |
| 102 96 | {Task.Supervisor, name: Snakepit.TaskSupervisor}, |
| @@ -124,7 +118,7 @@ defmodule Snakepit.Application do | |
| 124 118 | Snakepit.Pool.ApplicationCleanup |
| 125 119 | ] |
| 126 120 | else |
| 127 | - Logger.info("🔧 Starting Snakepit with pooling disabled") |
| 121 | + SLog.info("🔧 Starting Snakepit with pooling disabled") |
| 128 122 | [] |
| 129 123 | end |
| 130 124 | |
| @@ -132,16 +126,13 @@ defmodule Snakepit.Application do | |
| 132 126 | |
| 133 127 | opts = [strategy: :one_for_one, name: Snakepit.Supervisor] |
| 134 128 | result = Supervisor.start_link(children, opts) |
| 135 | - IO.inspect(System.monotonic_time(:millisecond), label: "Snakepit.Application started at") |
| 129 | + SLog.debug("Snakepit.Application started at: #{System.monotonic_time(:millisecond)}") |
| 136 130 | result |
| 137 131 | end |
| 138 132 | |
| 139 133 | @impl true |
| 140 134 | def stop(_state) do |
| 141 | - IO.inspect(System.monotonic_time(:millisecond), |
| 142 | - label: "Snakepit.Application.stop/1 called at" |
| 143 | - ) |
| 144 | - |
| 135 | + SLog.debug("Snakepit.Application.stop/1 called at: #{System.monotonic_time(:millisecond)}") |
| 145 136 | :ok |
| 146 137 | end |
| 147 138 | |
| @@ -160,7 +151,7 @@ defmodule Snakepit.Application do | |
| 160 151 | {:write_concurrency, true} |
| 161 152 | ]) |
| 162 153 | |
| 163 | - Logger.debug("Created worker capacity ETS table: #{table_name}") |
| 154 | + SLog.debug("Created worker capacity ETS table: #{table_name}") |
| 164 155 | |
| 165 156 | _ -> |
| 166 157 | # Table already exists |
Loading more files…