greptimex

# Greptimex

Elixir client for GreptimeDB ingestion and PromQL over gRPC.

```elixir
def deps do
  [{:greptimex, "~> 0.4.0"}]
end
```

Requires Elixir 1.18 or newer. For the breaking API changes in 0.4, see
[MIGRATION.md](MIGRATION.md).

## Start a client

Add a named client to your application's supervision tree:

```elixir
children = [
  {Greptimex,
   name: MyApp.Greptime,
   address: "localhost:4001",
   pool_size: 2,
   header: [catalog: "greptime", dbname: "public"]}
]

Supervisor.start_link(children, strategy: :one_for_one)
```

You can also call `Greptimex.start_link/1`. Pass the registered atom name to
operations. Each named client has its own configuration and pool. The client
reads configuration at startup, so restart it to apply changes.

| Client option | Default | Meaning |
| --- | --- | --- |
| `name` | required | Registered atom name |
| `address` | `"localhost:4001"` | gRPC endpoint |
| `pool_size` | `1` | Number of channels; each permits concurrent requests |
| `timeout` | `15_000` | Per-RPC timeout in milliseconds |
| `header` | catalog `"greptime"`, dbname `"public"` | Request context and authentication |
| `timestamp_name` | `"greptime_timestamp"` | Inferred timestamp column |
| `timestamp_unit` | `:millisecond` | Unit for integer ingestion timestamps |
| `channel_options` | `[]` | Options passed to `GRPC.Client.Connection`; defaults to Mint |

Headers accept `catalog`, `dbname`, `schema`, `timezone` and `auth`. Authentication:

```elixir
header: [auth: {:basic, {"username", "password"}}]
# or
header: [auth: {:token, "token"}]
```

TLS uses the underlying gRPC credential configuration:

```elixir
channel_options: [
  cred: GRPC.Credential.new(ssl: [
    verify: :verify_peer,
    cacerts: :public_key.cacerts_get(),
    server_name_indication: ~c"your-host.example"
  ])
]
```

## Insert points and batches

```elixir
point = %{
  tags: %{host: "server1", region: "us-west"},
  fields: %{cpu: 0.8},
  timestamp: DateTime.utc_now()
}

Greptimex.insert(MyApp.Greptime, "metrics", point)
# {:ok, 1}

Greptimex.insert(MyApp.Greptime, "metrics", [point, point], timeout: 5_000)
# {:ok, 2}

Greptimex.insert_batch(MyApp.Greptime, [
  {"metrics", [point]},
  {"other_metrics", [point], timestamp_unit: :microsecond}
])
# {:ok, 2}
```

`fields` and a non-null `timestamp` are required; `tags` defaults to `%{}`.
Greptimex converts atom names to strings. It rejects duplicate names, including
collisions between tags, fields and the timestamp.

Greptimex validates the whole batch before sending one RPC. Empty lists return
`{:ok, 0}` without sending data. It does not split, buffer or retry batches.

Success returns the server's affected row count. A batch is not a transaction.
A timeout can leave you unsure whether the server wrote the data, so the
application must decide whether to retry.

### Inference and explicit types

Inference examines each column across all points in a table entry:

| Elixir value | Inferred type |
| --- | --- |
| integer | `:int64` |
| float | `:float64` |
| boolean | `:boolean` |
| UTF-8 binary | `:string` |
| `Date` | `:date` |
| `DateTime` / `NaiveDateTime` field | `:datetime` |
| `Time` | `:time_microsecond` |

Greptimex converts a mix of integers and floats to `:float64` only when it can
represent the integers exactly. It rejects other incompatible mixtures.
Inference uses the points in each table entry, without checking the server or
caching schemas. Use a reusable schema to keep column types stable across batches.

Use `{value, type}` for explicit types, for example `{255, :uint8}`,
`{<<0, 255>>, :binary}` or `{nil, :float64}`. Explicit declarations must agree
with each other and with a supplied schema. Missing fields become null. A
column containing only nulls requires an explicit type.

Supported types are `:int8/16/32/64`, `:uint8/16/32/64`, `:float32`, `:float64`,
`:boolean`, `:string`, `:binary`, `:date`, `:datetime`, and `:time_*` /
`:timestamp_*` with suffix `second`, `millisecond`, `microsecond` or `nanosecond`.
These slash abbreviations describe separate atoms, such as `:int8` and `:int16`.
Greptimex checks integer bounds. Strings must be valid UTF-8. Binary fields can
contain arbitrary bytes. Float32 rounds floats to IEEE single precision but
requires exact conversion for integers.

Ingestion timestamp integers use the configured unit. Greptimex converts
`DateTime` values to that unit. For precision finer than microseconds, use
integer nanosecond timestamps.

Date integers count days since the Unix epoch. Integers typed as `:datetime`
count milliseconds. Greptimex encodes these fields as TIMESTAMP_MILLISECOND
because GreptimeDB rejects the legacy DATETIME column type. It treats
`NaiveDateTime` fields as UTC. Time values count units since midnight.

Version 0.4 returns validation errors for JSON, decimal, interval, list, vector,
struct and dictionary values.

### Reusable schema

```elixir
schema = %Greptimex.Schema{columns: [
  %{name: "ts", type: :timestamp_millisecond, semantic: :timestamp},
  %{name: "host", type: :string, semantic: :tag},
  %{name: "cpu", type: :float64, semantic: :field}
]}

Greptimex.insert(MyApp.Greptime, "metrics", %{
  timestamp: DateTime.utc_now(),
  tags: %{host: "server1"},
  fields: %{cpu: 0.8}
}, schema: schema)
```

Use `Greptimex.Schema.new(columns)` to validate a schema before inserting data.
It returns `{:ok, schema}` or `{:error, error}`. A schema needs exactly one
timestamp column with a timestamp type. Its column order determines wire order,
and its timestamp name and type override the inference defaults. Greptimex rejects
unknown columns, semantic changes and incompatible values.

A batch entry can override `schema`, `timestamp_name` and `timestamp_unit` for
that entry. The request's `header` and `timeout` apply to the whole batch.
Request headers merge with client headers.

## PromQL

```elixir
Greptimex.query_instant(MyApp.Greptime, "metrics", time: DateTime.utc_now())
# {:ok, %Greptimex.PromQL.Result{
#   type: :vector,
#   data: [%{metric: %{"host" => "server1"}, value: {datetime, 0.8}}],
#   warnings: [], infos: []
# }}

Greptimex.query_range(MyApp.Greptime, "metrics",
  ~U[2025-01-01 00:00:00Z], ~U[2025-01-01 01:00:00Z], "5m",
  lookback: "10m", timeout: 5_000)
```

Query times accept `DateTime` or Unix seconds, including fractional seconds.
Instant queries default to the current time; lookback defaults to `"5m"`.
Range bounds are inclusive and must be ordered. Step and lookback accept positive
seconds or duration strings such as `"250ms"`, `"5m"` or `"1h30m"`.
Duration strings use integer components in descending unit order, without repeated
units. Use numeric seconds for fractional durations, for example `0.25`.

Normalized results retain `warnings` and `infos`:

- `:vector`: list of `%{metric: labels, value: {datetime, number}}`.
- `:matrix`: list of `%{metric: labels, values: [{datetime, number}, ...]}`.
- `:scalar`: `{datetime, number}`.
- `:string`: `{datetime, string}`.

Labels keep string keys. Numeric samples are floats. `NaN`, `+Inf` and `-Inf`
become `:nan`, `:infinity` and `:neg_infinity`. Greptimex rounds JSON response
timestamps to microseconds.

Use `decode: :raw` to get the decoded JSON envelope. This also gives you access
to formats the normalizer does not support, such as native histograms. Server
error envelopes still return an error.

## Optional module facade

```elixir
defmodule MyApp.Greptime do
  use Greptimex, otp_app: :my_app
end

# config/runtime.exs
config :my_app, MyApp.Greptime,
  address: System.fetch_env!("GREPTIME_ADDRESS"),
  header: [dbname: "public"]

# supervision children
children = [MyApp.Greptime]

MyApp.Greptime.insert("metrics", point)
MyApp.Greptime.query_instant("metrics", time: DateTime.utc_now())
```

The facade calls the same Greptimex functions and uses its module as the client
name. At startup, it merges defaults, macro options, application configuration
and child-spec options in that order. Later values win. Operation options
override the startup configuration for that call.

The facade reads application configuration on every start, including supervisor
restarts.

## Errors and telemetry

Invalid data and expected connection, transport, server or decoding failures return
`{:error, %Greptimex.Error{kind: kind, code: code,
message: message, context: context}}`. Kinds are `:validation`, `:connection`,
`:timeout`, `:transport`, `:server` and `:decode`. Validation context identifies
the table, column and one-based row when available. Greptimex keeps server error
codes and redacts configured credentials from server error messages.

Invalid configuration and keyword options raise `ArgumentError`. A missing client
is a programming error. Unexpected exceptions, throws and process exits propagate
to the caller, including races with shutdown. Greptimex does not wrap them in
error tuples.

Each operation emits `[:greptimex, operation, :start | :stop | :exception]`, where
operation is `:insert`, `:query_instant` or `:query_range`. Batch writes use `:insert`.
The span includes configuration lookup, validation, RPC and decoding. Returned
errors emit `:stop` with `status: :error`; unexpected exceptions emit `:exception`.

Start measurements contain system and monotonic time; stop measurements include
duration in native units, plus `affected_rows` or normalized `result_count` on
success. Metadata includes client, status and error kind/code when applicable.
Greptimex does not add queries, row contents or credentials to metadata. Exception
events include the original reason and stacktrace. Review what your telemetry
handlers log, since an exception reason can contain application data. The gRPC
library emits its own connection events.

## Development

```sh
mix deps.get
mix test
mix format --check-formatted
mix docs --warnings-as-errors
```

The default suite starts a local gRPC test server. Its TLS test uses `openssl`
to generate certificates. Run the integration tests against a disposable
GreptimeDB instance. They create uniquely named tables:

```sh
docker run -d --name greptimex-test \
  -p 127.0.0.1:14000:4000 -p 127.0.0.1:14001:4001 \
  greptime/greptimedb:v1.2.0 standalone start \
  --http-addr 0.0.0.0:4000 --grpc-bind-addr 0.0.0.0:4001

# Wait until this returns HTTP 200 before running the tests.
curl --fail http://127.0.0.1:14000/health

GREPTIMEX_TEST_ADDRESS=127.0.0.1:14001 mix test --include integration

# Remove the disposable database and its test data when finished.
docker rm -f greptimex-test
```

Integration tests cover scalar ingestion, explicit schemas, multi-table batches,
timestamp units, concurrent writes, instant/range PromQL, raw results, and server
errors. The dashboard is available at `http://localhost:14000/dashboard` while
the container is running.

Protocol sources are pinned to `greptime-proto` commit
`549ff0aa8e866c6042d2e755fa35d919b42fde30`. With protoc 36.1 and
protoc-gen-elixir 0.16.0 installed:

```sh
elixir scripts/sync_pb.exs --check
elixir scripts/sync_pb.exs
```

This repository-only script is excluded from the Hex package. It downloads and
generates files in a temporary directory. If either step
fails, it leaves the project files untouched. `--check` compares both the proto
sources and generated modules with the pinned versions.

## Connection lifecycle

Greptimex supervises `GRPC.Client.Connection` workers using gRPC 1.0.5 or newer
with Mint. The client supervisor stores configuration in ETS. Each caller reads
that configuration and executes its RPC, so requests can run concurrently on one
HTTP/2 channel.

Startup does not wait for a connection. Until a channel is ready, operations
return a connection error. The supervisor restarts connection workers when they
exit. On gRPC UNKNOWN, INTERNAL or UNAVAILABLE, the executor disconnects the
channel and lets supervision start a new worker. Disconnecting can interrupt
other calls on that channel. Greptimex never resends a failed RPC.