gen_metrics_bench

defmodule GenMetricsBench do

  @moduledoc """
  A `GenMetrics` benchmarking tool for `GenServer` and `GenStage`
  applications. 

  GenMetrics supports the collection and publication of GenServer
  and GenStage runtime metrics. Metrics data are generated by an
  introspection agent. This means that no instrumentation is required
  within the GenServer or GenStage library or within your application
  source code.

  This library helps us to test and measure what, if any, overhead is
  introduced into the runtime environment by the GenMetrics
  introspection agent.

  To provide an extensible benchmarking tool this library makes it
  simple for anyone to plug-in new runtime behaviours for any benchmark
  test. These behaviours include:

  1. The type and size of the message passing through the
  GenServer cluster or GenStage pipeline.

  2. The simulated time spent processing those messages within a
  GenServer call, cast or info callback or within a GenStage
  :producer_consumer `handle_events/3` callback.

  ## Benchmarking GenServer Clusters

  The `GenMetricsBench.Cluster` module supports the following
  set of benchmarks using a dedicated GenServer cluster harness:

  1. `GenMetricsBench.Cluster.no_metrics/0`
  1. `GenMetricsBench.Cluster.summary_metrics/0`
  1. `GenMetricsBench.Cluster.statistical_metrics/0`
  1. `GenMetricsBench.Cluster.statsd_metrics/0`
  1. `GenMetricsBench.Cluster.datadog_metrics/0`

  The default benchmark behaviours for GenServer cluster benchmarking are
  provided by the following module:

  - `GenMetricsBench.Simulator.ClusterDefault`

  Simply clone this module then customize the return values on the callbacks
  defined by the `GenMetricsBench.Simulator` behaviour. Then register
  your cloned module in `config/config.exs` and run your custom benchmarks.

  ## Sample Benchmarks: GenServer Clusters

  Each of the following GenServer benchmarks processed `10,000`
  [ten thousand] messages and the time taken to complete each benchmark
  is indicated below:

  ```elixir
  # GenMetrics Disabled
  iex(1)> GenMetricsBench.Cluster.no_metrics
  GenServer Cluster: benchmark starting, sim_load=10000
  :ok
  GenServer Cluster: handle_info, sim_load=10000 completed in 20024 ms

  # GenMetrics Summary Metrics Enabled
  iex(5)> GenMetricsBench.Cluster.summary_metrics
  GenServer Cluster: benchmark starting, sim_load=10000
  :ok
  GenServer Cluster: handle_info, sim_load=10000 completed in 20004 ms

  # GenMetrics In-Memory Statistical Metrics Enabled
  iex(6)> GenMetricsBench.Cluster.statistical_metrics
  GenServer Cluster: benchmark starting, sim_load=10000
  :ok
  GenServer Cluster: handle_info, sim_load=10000 completed in 20011 ms

  # GenMetrics Statsd Statistical Metrics Enabled
  iex(3)> GenMetricsBench.Cluster.statsd_metrics 
  GenServer Cluster: benchmark starting, sim_load=10000
  :ok
  GenServer Cluster: handle_info, sim_load=10000 completed in 20019 ms

  # GenMetrics Datadog Statistical Metrics Enabled
  iex(2)> GenMetricsBench.Cluster.datadog_metrics
  GenServer Cluster: benchmark starting, sim_load=10000
  :ok
  GenServer Cluster: handle_info, sim_load=10000 completed in 20009 ms
  ```

  Note, the default benchmark behaviours provided by
  `GenMetricsBench.Simulator.ClusterDefault` were used
  during these sample benchmarks.

  ## Benchmarking GenStage Pipelines

  The `GenMetricsBench.Pipeline` module supports the following
  set of benchmarks using a dedicated GenStage pipeline harness:

  1. `GenMetricsBench.Pipeline.no_metrics/0`
  1. `GenMetricsBench.Pipeline.summary_metrics/0`
  1. `GenMetricsBench.Pipeline.statistical_metrics/0`
  1. `GenMetricsBench.Pipeline.statsd_metrics/0`
  1. `GenMetricsBench.Pipeline.datadog_metrics/0`

  The default benchmark behaviours for GenStage pipeline benchmarking are
  provided by the following module:

  - `GenMetricsBench.Simulator.PipelineDefault`

  Simply clone this module then customize the return values on the callbacks
  defined by the `GenMetricsBench.Simulator` behaviour. Then register
  your cloned module in `config/config.exs` and run your custom benchmarks.

  ## Sample Benchmarks: GenStage Pipelines

  Each of the following GenStage benchmarks processed `10,000,000`
  [ten million] messages and the time taken to complete each benchmark
  is indicated below:

  ```elixir
  # GenMetrics Disabled
  iex(1)> GenMetricsBench.Pipeline.no_metrics
  GenStage Pipeline: benchmark starting, sim_load=10000000
  GenStage Pipeline: sim_load=10000000 completed in 40035 ms
  ** (EXIT from #PID<0.199.0>) :benchmark_completed

  # GenMetrics Summary Metrics Enabled
  iex(1)> GenMetricsBench.Pipeline.summary_metrics
  GenStage Pipeline: benchmark starting, sim_load=10000000
  GenStage Pipeline: sim_load=10000000 completed in 40069 ms
  ** (EXIT from #PID<0.204.0>) :benchmark_completed

  # GenMetrics In-Memory Statistical Metrics Enabled
  iex(1)> GenMetricsBench.Pipeline.statistical_metrics
  GenStage Pipeline: benchmark starting, sim_load=10000000
  GenStage Pipeline: sim_load=10000000 completed in 40034 ms
  ** (EXIT from #PID<0.210.0>) :benchmark_completed

  # GenMetrics Statsd Statistical Metrics Enabled
  iex(1)> GenMetricsBench.Pipeline.statsd_metrics
  GenStage Pipeline: benchmark starting, sim_load=10000000
  GenStage Pipeline: sim_load=10000000 completed in 40015 ms
  ** (EXIT from #PID<0.217.0>) :benchmark_completed

  # GenMetrics Datadog Statistical Metrics Enabled
  iex(1)> GenMetricsBench.Pipeline.datadog_metrics
  GenStage Pipeline: benchmark starting, sim_load=10000000
  GenStage Pipeline: sim_load=10000000 completed in 40128 ms
  ** (EXIT from #PID<0.223.0>) :benchmark_completed
  ```

  Note, the default benchmark behaviours provided by
  `GenMetricsBench.Simulator.PipelineDefault` were used
  during these sample benchmarks.

  """

end