Packages

Spawn and manage Google Compute Engine instances over the REST API, with pluggable auth, telemetry, and an ergonomic instance builder.

Current section

Files

Jump to
gcp_compute README.md
Raw

README.md

# GcpCompute
[![Hex.pm](https://img.shields.io/hexpm/v/gcp_compute.svg)](https://hex.pm/packages/gcp_compute)
[![Docs](https://img.shields.io/badge/hex-docs-blue.svg)](https://hexdocs.pm/gcp_compute)
Spawn and manage **Google Compute Engine** instances from Elixir, over the
Compute REST API. Pluggable auth, telemetry on every call, and an ergonomic
builder for the (verbose) instance body — with cheap, self-deleting **Spot VMs**
as the default.
> **Why REST and not gRPC?** The Compute Engine API is **REST/JSON only** — it
> has no gRPC endpoint (it's the one notable exception among GCP APIs). So unlike
> [`pubsub_grpc`](https://hex.pm/packages/pubsub_grpc), there is no connection
> pool to run: a request is a handful of calls dominated by the 20–40s the VM
> takes to boot, where pooling buys nothing. `GcpCompute` talks to it with
> [`Req`](https://hexdocs.pm/req); a `GcpCompute.Config` is the only handle you
> pass around. The right place for gRPC + [`grpc_connection_pool`](https://hex.pm/packages/grpc_connection_pool)
> is an agent running *inside* the machine you spawn — see [Roadmap](#roadmap).
## Installation
```elixir
def deps do
[
{:gcp_compute, "~> 0.1.0"},
# Recommended token provider (optional dependency):
{:goth, "~> 1.4"}
]
end
```
## Quickstart
```elixir
# 1. Start Goth to mint OAuth tokens (in your application supervision tree)
children = [{Goth, name: MyApp.Goth}]
# 2. Build a config once and reuse it
{:ok, config} =
GcpCompute.Config.production(
project: "my-project",
zone: "us-central1-a",
goth: MyApp.Goth
)
# 3. Spawn a cheap, self-deleting Spot VM and wait until it exists
{:ok, instance} =
GcpCompute.insert_instance_and_wait(config,
name: "worker-1",
machine_type: "e2-micro",
spot: true,
max_run_duration: 3600, # hard server-side TTL (seconds)
startup_script: "#!/bin/bash\necho ready > /tmp/ready",
labels: %{"owner" => "platform"}
)
GcpCompute.Instance.external_ip(instance) #=> "34.x.x.x"
# 4. Tear it down (and wait for the delete operation)
{:ok, _op} = GcpCompute.delete_instance_and_wait(config, "worker-1")
```
Prefer to drive the lifecycle yourself? Every mutating call returns an
`GcpCompute.Operation` you can poll:
```elixir
{:ok, op} = GcpCompute.insert_instance(config, name: "worker-1", machine_type: "e2-micro")
{:ok, done} = GcpCompute.wait_for_operation(config, op, timeout: :timer.minutes(3))
```
## Examples & guides
- **Runnable examples** in [`examples/`](https://github.com/nyo16/gcp_compute/tree/master/examples):
- [`stubbed_demo.exs`](https://github.com/nyo16/gcp_compute/blob/master/examples/stubbed_demo.exs)
the full insert → poll → get flow offline, **no GCP or credentials**
(`elixir examples/stubbed_demo.exs`).
- [`spawn_spot_vm.exs`](https://github.com/nyo16/gcp_compute/blob/master/examples/spawn_spot_vm.exs)
a real (billable) Spot VM.
- **Guides:** [Getting Started](guides/getting-started.md) ·
[Configuring Machines](guides/configuring-machines.md) ·
[Testing](guides/testing.md)
## Configuration
`GcpCompute.Config` is validated with `NimbleOptions`. Three builders cover the
common cases:
```elixir
# Production — tokens minted by Goth
{:ok, config} = GcpCompute.Config.production(project: "p", goth: MyApp.Goth)
# From application env
# config :my_app, :gcp_compute,
# project: "p", zone: "europe-west4-a",
# token_provider: {GcpCompute.TokenProvider.Goth, MyApp.Goth}
{:ok, config} = GcpCompute.Config.from_env(:my_app, :gcp_compute)
# Local / emulator / tests — a static token, no Goth required
{:ok, config} = GcpCompute.Config.local(project: "p", base_url: "http://localhost:8080/compute/v1")
```
| Option | Default | Notes |
| ---------------- | ---------------------------------------------- | -------------------------------------------------- |
| `:project` | — (required) | GCP project id. |
| `:zone` | `"us-central1-a"` | Default zone; override per call with `zone:`. |
| `:token_provider`| `{TokenProvider.Goth, GcpCompute.Goth}` | `{module, arg}` implementing `GcpCompute.TokenProvider`. |
| `:base_url` | `"https://compute.googleapis.com/compute/v1"` | Point at the emulator or a proxy. |
| `:req_options` | `[]` | Merged into every `Req` request (`:retry`, `:adapter`, …). |
### Pluggable auth
Auth is a behaviour, `GcpCompute.TokenProvider`, so the library never hard-depends
on Goth. Built-ins: `TokenProvider.Goth` (production), `TokenProvider.Static`
(tests/emulator). Bring your own (workload identity, metadata server, Vault) by
implementing one callback:
```elixir
defmodule MyApp.MetadataToken do
@behaviour GcpCompute.TokenProvider
@impl true
def fetch_token(_arg), do: {:ok, %{token: fetch_from_metadata_server()}}
end
```
### Configurable machines
`GcpCompute.Instance.spec/1` turns friendly options into the Compute insert body
(defaults shown):
| Option | Default |
| ------------------- | --------------------------------------------------- |
| `:machine_type` | `"e2-micro"` |
| `:source_image` | `debian-cloud/.../debian-12` |
| `:disk_size_gb` | `10` |
| `:spot` | `true` (SPOT, no auto-restart, DELETE on terminate) |
| `:max_run_duration` | `nil` (set seconds for a hard server-side TTL) |
| `:external_ip` | `true` |
| `:startup_script`, `:metadata`, `:labels`, `:tags`, `:network`, `:subnetwork`, `:service_account`, `:scopes` ||
Need a field it doesn't cover? Pass a raw Compute body map to
`GcpCompute.Instances.insert/3` instead.
## Telemetry
Every API call is a `:telemetry.span/3`:
| Event | Metadata |
| -------------------------------------- | ------------------------------------------------- |
| `[:gcp_compute, :request, :start]` | `method`, `path`, `project` |
| `[:gcp_compute, :request, :stop]` | `+ result` (`:ok`/`:error`), `http_status` |
| `[:gcp_compute, :request, :exception]` | `+ kind`, `reason`, `stacktrace` |
```elixir
GcpCompute.Telemetry.attach_default_logger() # dev convenience
# or wire into Telemetry.Metrics / your reporter
```
## Testing without GCP
Tests stub the network via Req's native `:adapter` option — no Plug, no creds:
```elixir
config =
GcpCompute.Config.local!(
project: "test",
req_options: [adapter: fn req -> {req, %Req.Response{status: 200, body: %{...}}} end]
)
```
See `test/support/req_stub.ex` for the routing-table helper used by the suite.
## Roadmap
This package is the **Compute client layer**. A higher-level *sandbox
orchestration* layer is planned on top of it:
- `gen_statem` per sandbox (`:provisioning → :probing → :running → :terminating`)
- `DynamicSupervisor` + `Registry`, named profiles, per-user quotas
- a reaper for TTL sweeps + orphan reconciliation (cloud resources outlive the BEAM)
- the gRPC hop: an in-VM agent reached via `grpc_connection_pool`, lifecycle
events published through `pubsub_grpc`
## License
MIT.