ash_clickhouse

# Changelog

## 0.7.2

### Bug fixes

- **Resource metadata caching no longer fails when the ETS table is missing.**
  The `:ash_clickhouse_resource_metadata` table is now initialized at
  application startup and owned by the long-lived application process, so it
  is not removed when the request process that first reads resource metadata
  exits.

## 0.7.1

### Bug fixes

- **Recording applied migrations no longer fails.** `record_applied/2` passed
  the version as a scalar param to `INSERT INTO schema_migrations ... VALUES (?)`,
  but the client treats params of any `INSERT` as bulk rows (each must be a
  list, and `?` placeholders are never substituted there) — so applying any
  migration crashed with a `FunctionClauseError` (surfacing as a
  `ClickhouseError`) even though the DDL itself had succeeded. The version is
  now written through the same row-encoding path as data-layer inserts:
  `repo.insert_rows(statement, [[version]])` with `FORMAT JSONCompactEachRow`.
- **Migration discovery finds files another extension already loaded.** During
  `mix ash.migrate`, sibling extensions require every `.exs` under
  `priv/repo/migrations`, and `Code.require_file/1` returns nothing for files
  already required in the VM — so AshClickhouse discovered zero modules and
  reported "No pending migrations" against an empty tracking table. Discovery
  now falls back to resolving the already-loaded module from the file's
  `defmodule` name via `Code.ensure_loaded`.

## 0.7.0

### Features

- **`contains` is now configurable as case-sensitive** via
  `config :ash_clickhouse, :case_sensitive_contains, true` (uses `position()`
  instead of `positionCaseInsensitive()`), matching the case-sensitivity of
  `starts_with`/`ends_with`. Defaults to the historical case-insensitive
  behaviour.
- **`like` / `ilike` filters are now supported**, translating to ClickHouse
  `LIKE` / `ILIKE`.
- **Ash NewTypes are unwrapped in DDL type mapping.** A NewType wrapping e.g.
  `:uuid` now maps to `UUID` instead of silently becoming a `String` column;
  truly unknown types fall back to `String` with a warning.
- **Typed array attributes encode element values natively.** For arrays like
  `{:array, :integer}` elements are passed through so the client emits native
  JSON numbers; only string-ish element types are stringified.
- **Added GitHub Actions CI workflow** (`.github/`).

### Bug fixes

- **`Connection.query/4` no longer swallows genuine bugs.** Only client error
  modules (including `ArgumentError` from the client) are converted to a
  normalized error tuple; any other exception propagates with its original
  stacktrace instead of being mislabelled as a ClickHouse error.
- **Stale connection cache entries are cleaned up when the client process
dies.** `Connection` now monitors the client pid and erases the cached struct
  on `:DOWN`, so `get_conn/1` doesn't keep returning a dead pid.
- **`Connection.stop/1` only erases the cache entry after the client process
has been stopped** (or was already gone), so a failed stop leaves the cache
  intact for retry, and always returns `:ok`.
- **Empty `in` / `not_in` filter lists no longer produce invalid SQL.**
  `col IN ()` would be rejected by ClickHouse; these now emit semantically
  equivalent always-false / always-true literals.
- **DISTINCT queries with sorts on unselected columns are now valid SQL.** Sort
  columns missing from the SELECT list are appended automatically (ClickHouse
  requires every ORDER BY expression in the SELECT list for DISTINCT).
- **Rollback version deletion uses parameterized queries** instead of manual
  string escaping (`delete_applied/2`).
- **`Rollback stop comparison handles mixed-width numeric versions.** Versions
  are compared numerically when both parse as integers, falling back to
  lexicographic order for timestamp-style versions.
- **Batched relationship-aggregate failures now raise by default**
  (`QueryError`) instead of silently falling back to `default_value` — wrong
  numbers in reports are usually worse than a loud error. Opt out with
  `config :ash_clickhouse, :raise_on_aggregate_failure, false`.
- **`avg` over Decimal columns keeps precision.** Aggregate decoding now uses
  the field's attribute type for `avg` too, decoding back to `Decimal` instead
  of losing precision as a float.
- **Single-row update/destroy default to `mutations_sync: 1`**, giving
  read-your-writes semantics so the returned record reflects the change
  immediately.

### Improvements

- **`insert_rows/4` drops the unused `table` argument** (breaking change):
  `repo.insert_rows(statement, rows, opts)` — the table was already embedded in
  the statement. `Repo` behaviour and `DataLayer` call sites updated.
- **Per-resource metadata caching.** `uuid_attribute_names/1`,
  `atom_attribute_names/1`, and `attr_type_map/1` results are cached per
  resource in an ETS table, so hot paths like per-row record decoding and bulk
  value encoding don't re-scan resource attributes on every call.

- **Standard Ash mix tasks.** `AshClickhouse.DataLayer.Extension` now implements
  `Spark.Dsl.Extension` with `codegen/1` and `migrate/1`, so `mix ash.codegen`
  (print pending DDL, with `--dry-run` / `--check` support) and `mix ash.migrate`
  (apply DDL) work out of the box. `mix ash_clickhouse.migrate` now delegates to
  the same `migrate/1`, keeping both commands in sync.
- **Data-skipping indexes.** Declare ClickHouse data-skipping indexes
  (`minmax`, `set`, `bloom_filter`, `ngrambf_v1`, `tokenbf_v1`) inside the
  `clickhouse` block via the repeatable `index` macro. They are emitted in the
  `CREATE TABLE` DDL and added to existing tables with `ALTER TABLE ... ADD
  INDEX IF NOT EXISTS` by `mix ash_clickhouse.migrate`. The index `type` is
  validated against a whitelist at compile time. See `guides/migrations.md` and
  `guides/resources.md`.

### Bug fixes

- **`create_database/1` / `drop_database/1` no longer target a database named
  `nil`.** When a repo has no `:database` configured, these now fall back to
  ClickHouse's `default` database (matching `alter_table_cql/2`),
  so `mix ash_clickhouse.setup` no longer creates a literal `nil` database.
- **`Repo.child_spec/1` now honors its `opts` argument.** Options passed via
  the supervision tree (e.g. `{MyApp.Repo, url: "...", pool_size: 7}`) are
  merged into the connection options instead of being silently dropped.
- **`run_query/2` now guards against a missing repo.** A resource whose
  `clickhouse` block forgets `repo` raises a clear `ConfigurationError` instead
  of failing with `UndefinedFunctionError: function nil.query/3 is undefined`.
- **`distinct` + explicit `select` no longer silently drops columns.**
  `build_optimized_query/1` now emits the merged select list under `DISTINCT`
  (ClickHouse dedupes on the full row) rather than only the distinct columns.
- **Sort building now supports Ash's nulls-ordering directions**
  (`:asc_nils_first`, `:asc_nils_last`, `:desc_nils_first`, `:desc_nils_last`),
  emitting `NULLS FIRST` / `NULLS LAST`. Previously these raised
  `FunctionClauseError`.
- **The `index` DSL macro is now robust to key order.** It matches any keyword
  list and pulls `name`/`expression`/`type` with `Keyword.get`, raising a clear
  error if any required key is missing (instead of a cryptic "undefined
  function index/1" when keys were reordered). Duplicate index names now raise
  a compile-time `ArgumentError`.
- **`mix ash_clickhouse.migrate` is more resilient.** A single resource that
  raises during DDL generation no longer aborts the whole run, and a resource
  that forgets `repo` is skipped with an error instead of being migrated into
  every configured database.
- **Migration defaults support booleans, dates, datetimes, and `Decimal`
  structs.** These now emit correct literals instead of a misleading
  "Non-numeric default" error. Other unsupported default shapes raise a clearer
  "Unsupported default" message.
- **`stream/3` now wraps raw client exceptions** in
  `AshClickhouse.Error.ClickhouseError`, matching every other read path.
- **`qualified_table/1` (writes) now backtick-quotes the table name** like the
  read path, so a reserved-word table name behaves consistently across reads
  and writes.
- **`can?/2` now has a single source of truth.** The 26 redundant per-atom
  clauses that duplicated the `@supported_features` MapSet were removed; only
  the genuinely special cases (aggregates, joins, `:filter_expr`, and the
  explicit `false` clauses) remain.
- **`Dsl.get_config/3` no longer rescues an unreachable `FunctionClauseError`.**
- **`Identifier.valid_identifier?/1` is now just the regex** (the manual
  first-character check was redundant).
- **`collect_columns/1` now handles `:not` expressions** (both
  `%Ash.Query.BooleanExpression{op: :not}` and `%Ash.Query.Not{}`), matching
  `build_predicate/1`.

### Improvements

- **UUID heuristic no longer corrupts non-UUID string data.** Parameters are
  now converted to the 16-byte UUID binary form only when the column is
  provably UUID-typed (via `Dsl.uuid_attribute_names/1`), instead of whenever a
  36-character string merely *looked* like a UUID. This prevents legitimate
  `:string` business identifiers (order numbers, etc.) from being mangled on
  insert/update and in WHERE filters.
- **Removed dead `Module.get_attribute/2` fallbacks** in `resolve_table_name/1`
  and `repo/1`, which always raised (and were silently swallowed) at runtime.
  They now use `Dsl.table/1` / `Dsl.repo/1` directly.

- **Relationship aggregates are now batched.** `attach_aggregates/5` issues one
  grouped query per aggregate across the whole result set (instead of one query
  per record × aggregate), turning an N×M round-trip pattern into M round-trips.
  `has_many` / `has_one` relationship aggregates are now supported (previously
  only `belongs_to` worked; all others silently returned `default_value`).
- **Consistent, type-aware aggregate decoding.** `decode_aggregate/2` became
  `decode_aggregate/4`, which resolves the actual Ash attribute type of the
  aggregated field instead of sniffing the string's shape. This fixes
  `Decimal` columns being silently downgraded to `float` and mishandled
  scientific notation, and makes query-level and relationship aggregates return
  the same decoded type.
- **`raise_on_untranslatable_filter` now defaults to `true` (fail-closed).** An
  untranslatable filter on a `base_filter` (tenant scoping, soft-delete) is now
  raised rather than silently dropped, avoiding queries that are less
  restrictive than intended. Opt back into the old warning-only behaviour with
  `config :ash_clickhouse, :raise_on_untranslatable_filter, false`.
- **Repo cache can now be cleared** via `AshClickhouse.DataLayer.clear_repo_cache!/0`
  for test suites that redefine resources between tests.
- **Connection errors now log the message and stacktrace** before being
  swallowed as `{:error, _}`, and `Connection.stop/1` logs a debug message on
  shutdown instead of silently masking failures.
- **`truncate_integer/1` renamed to `validate_integer!/1`** to reflect that it
  validates/parses rather than truncates.
- **Documented** the intentional difference between strict `sanitize!/1`
  (table/database names) and `quote_name/1` (column identifiers), and marked
  the unused `group_by` query field as dead scaffolding.
- **Split the mono-module `data_layer.ex` into focused modules** under
  `AshClickhouse.DataLayer.*`: `Insert` (value encoding + insert/update SQL),
  `Record` (row → Ash record decoding), `Aggregate` (native + batched
  relationship aggregates), and `Calculations` (in-memory calculation
  application). The public API is unchanged.
- **`Connection.query!/4` now reraises a wrapped `ClickhouseError`** (instead of
  re-raising a fresh one), preserving the original stacktrace while still
  normalizing client errors.
- **`QueryBuilder.build_where_clause/2` no longer grows params quadratically.**
  The `params ++ mapped` append inside the reduce was replaced with prepend +
  single `Enum.reverse()`/`Enum.concat()`, so building WHERE clauses is now
  linear in the number of filters.
- **Shared mix-task helpers.** `find_repos/0` / `find_resources/0` moved into
  `Mix.Tasks.AshClickhouse.Helpers`, removing the duplication between
  `mix ash_clickhouse.setup` and `mix ash_clickhouse.migrate`.

## 0.1.0

- Initial release of AshClickhouse, an Ash data layer for ClickHouse.
- Implements `Ash.DataLayer` with CRUD, filter, sort, limit/offset, select,
  distinct, bulk_create, update_query/destroy_query, native aggregates,
  multitenancy, calculations, and relationship aggregates.
- Provides the `clickhouse` DSL, `AshClickhouse.Repo`, `AshClickhouse.Connection`,
  and `mix ash_clickhouse.setup` / `mix ash_clickhouse.migrate` tasks.