Current section

Files

Jump to
ectomancer CHANGELOG.md
Raw

CHANGELOG.md

# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.7.0] - 2026-07-31
> **Read this if you're upgrading.** This release fixes the defects found in a
> real-world integration shakedown of 1.6.0. The headline changes:
>
> - **Tool responses are now JSON** instead of `inspect/1` output (no more Elixir
> struct syntax, no more silently truncated lists).
> - **Batch operations actually work** over MCP (previously they silently reported
> `total: 0`), and `include`/`preloadable` actually preloads.
> - **Remote callers can no longer exhaust the BEAM atom table** via `order_by`
> or filter parameters.
> - **The documented supervision snippet boots.** `anubis_mcp` is now pinned to
> `~> 1.14`, `Ectomancer.child_spec/2` produces a working entry, and
> `forward "/mcp", Ectomancer.Plug, ...` compiles in routers.
> - **`only:`/`except:` redact read results.** Excluded fields no longer leak in
> `list`/`get`/batch output.
> - **New `scope:` option** for multi-tenant row-level scoping.
>
> No breaking changes in this release.
### Added
- **`scope:` option for `expose`** — row-level scoping for multi-tenant apps. The function receives the query and the authenticated actor (`fn query, actor -> query end`) and is applied to every generated CRUD query. Composes with authorization-policy scopes (#140).
- **Configurable query limit ceiling**`config :ectomancer, max_limit: N` (default `100`). The effective `limit` is reported in pagination metadata so clamping is visible (#150).
### Changed
- **anubis_mcp requirement tightened to `~> 1.14`** (was `~> 1.5`). The old range resolved the current release anyway; the constraint now reflects what is actually tested.
- **`Ectomancer.child_spec/2` produces a working supervision entry.** The old output `{Anubis.Server.Supervisor, {server, transport: ...}}` called the non-existent `start_link/1` and failed at boot (#143, #148). It now returns a single `{server, transport: {transport, start: true}}` entry — resolved through the server module's own `child_spec/1` — so `children = [Ectomancer.child_spec(MyApp.MCP, transports: [:streamable_http]), MyAppWeb.Endpoint]` boots as documented. One transport per server module is supported; requesting two raises a clear error.
- **Router mounting works on anubis 1.14.** `Ectomancer.Plug.init/1` passes an escapable `subscriber_metadata` remote capture so `forward "/mcp", Ectomancer.Plug, ...` compiles in Phoenix/Plug routers instead of failing with "cannot inject attribute @plug_forward_opts" (#143).
- README + `Ectomancer.Plug` docs updated to the working supervision form.
### Fixed
- **Tool results are now JSON, not `inspect/1` output** (#147). Responses are serialized through a sanitizer that converts Ecto structs to plain maps, drops `__meta__`, replaces `NotLoaded` with `null`, and renders datetimes as ISO-8601. `inspect/1`'s silent 50-row truncation is gone, and error responses no longer echo full stacktraces.
- **Batch operations no longer silently no-op** (#145). Anubis delivers params with atom keys; the batch handlers read string keys, so `batch_create` reported `total: 0`. Param keys are now normalized once at the tool-execution funnel, and `Repo.batch_*` read both key shapes.
- **`include`/`preloadable` actually preloads** (#146). The `include` param is honored after the key-normalization fix, and `validate_includes/3` no longer crashes on string allowlists (removed the dead `:all` clause that also did unbounded `String.to_atom`).
- **Batch operations isolate per-record failures** (#153). Each per-item write runs in a savepoint so a database-level constraint violation cannot poison the surrounding transaction; the documented partial-failure semantics are kept and the README no longer claims batches are atomic.
- **No unbounded atom creation from remote input** (#142). `order_by`, filter keys, and param names were `String.to_atom`'d from caller-influenced strings before the allowlist check, letting an unauthenticated client exhaust the BEAM atom table. Filtering now resolves against the schema's field allowlist by string comparison and only ever references existing atoms.
- **`expose` compiles on schemas with `has_through` associations** (#144). `get_associations/1` read `assoc.related` unconditionally, but `Ecto.Association.HasThrough` has no `:related` key — any schema with a `has_many/has_one ... :through` failed at compile time with `KeyError`. Falls back to the owning schema.
- **Correct plural tool names** (#149). `list`/batch tool names appended a literal `"s"`, producing `list_studys`, `list_statuss`. They now use `Plurality.pluralize/1` (`list_studies`, `list_statuses`, `list_news`).
- **README/docs `authorize: with:` syntax errors fixed** (#151). `use Ectomancer, authorize: with: Module` and `expose ..., authorize: with: Module` are invalid Elixir; the docs now show `authorize: Module`.
- **Playground + demo assets ship in the hex package** (#152). `priv/` (playground HTML, demo GIF, demo cast) was missing from `mix.exs` `files:`, so README references were broken for hex installs.
- **`only:`/`except:` now redact read results** — excluded fields are stripped from every row returned by `list`, `get`, and batch tools (previously they only affected input params and resource metadata) (#141).
- **Tool name pluralization**`singularize_resource/1` now uses `Plurality` for noun inflection and keeps already-singular words ending in "s" intact. Tool names for `status`, `analysis`, `business`, `series`, `class`, `address`, and `news` are no longer truncated (`get_status` instead of `get_statu`, etc.) (#128)
- **Compile warnings** — grouped `build_assoc_params/1` clauses and silenced the unused `assoc` parameter in `child_fk/3` so the lint CI job (`--warnings-as-errors`) passes.
### Testing
- 874 tests across CRUD, filtering, batch, preload, scope, redaction, auth, supervision, and atom-safety suites.
- Verified end-to-end: fresh Phoenix 1.8.9 app (SQLite, Elixir 1.20/OTP 29) installing from git, mounted at `/mcp`, driven over MCP Streamable HTTP — scope isolation, `except:` redaction, JSON output, batch/include, plural tool names, and boot with the documented supervision pattern all confirmed.
- CI matrix: Elixir 1.18.4–1.20.0 / OTP 26–29, with format, Credo, and Dialyzer on the lint job.
### Issues Closed
- #140 — No tenant scoping by default (fixed via `scope:` option)
- #141 — `except:` does not redact read results
- #142 — Remote unbounded atom creation via `order_by`
- #143 — Documented supervision child spec cannot boot
- #144 — `expose` won't compile on schemas with `has_through`
- #145 — Batch operations silently no-op
- #146 — `include`/`preloadable` silently ignored
- #147 — Tool results are `inspect/1` output, not JSON
- #148 — `Ectomancer.child_spec/2` returns a bare list
- #149 — Broken pluralization in tool names
- #150 — `limit` silently hard-capped at 100
- #151 — README `authorize: with:` syntax errors
- #152 — Playground + demo assets not shipped in the hex package
- #153 — Batch operations not atomic despite "transactional"/"atomically" claims
## [1.6.0] - 2026-07-20
### Added
- **MCP Prompts**`prompt/2` macro for structured, parameterized prompt templates with argument validation (#122)
Define prompts alongside your tools:
```elixir
prompt :analyze_churn do
description "Analyze user churn risk"
argument :cohort, :string, required: true, description: "User cohort"
messages fn args ->
[%{role: :user, content: %{type: :text, text: "Analyze churn for #{args["cohort"]}"}}]
end
end
```
Supports `:string`, `:integer`, `:float`, `:boolean`, `:list`, and `:map` argument types with `required`, `description`, `default`, and `enum` options. Arguments are validated by Anubis before the messages callback runs. Registered automatically as MCP prompts — visible via `prompts/list` and callable via `prompts/get`.
- **Upsert operations**`:upsert` action for insert-or-update workflows with configurable conflict target and `on_conflict` control (#117)
```elixir
expose MyApp.Products.Product,
actions: [:upsert],
conflict_target: :sku,
on_conflict: :replace_all
```
Returns `{:ok, {record, :inserted | :updated}}`. Supports composite conflict targets (`conflict_target: [:org_id, :sku]`) and selective `on_conflict: [set: [:name, :avatar_url]]`. Automatically restores soft-deleted records. `conflict_target` is required at compile time.
- **Batch operations**`batch_create`, `batch_update`, `batch_destroy` for transactional multi-record operations (#116)
```elixir
expose MyApp.Accounts.User,
actions: [:batch_create, :batch_update, :batch_destroy],
batch_size: 200
```
Each runs inside a single `repo.transaction` with individual try/rescue per item — invalid records are collected without aborting valid ones. Returns `%{succeeded: [...], failed: [...], total: N}`. Configurable `:batch_size` (default: `100`) enforced before any DB interaction. Full authorization, scope, and soft-delete support.
- **SSE and WebSocket transport support** — three transport options for MCP protocol serving (#120)
| Transport | Status | Route |
|---|---|---|
| **Streamable HTTP** (MCP 2025-03-26) | **Default** | `forward "/", Ectomancer.Plug` |
| **SSE** (MCP 2024-11-05) | Deprecated | `get "/sse"` + `post "/sse"` |
| **WebSocket** | Available | `socket "/mcp/ws", Ectomancer.Plug.WebSocket` |
Streamable HTTP uses a single endpoint with session header (`mcp-session-id`) and streaming responses. WebSocket supports actor extraction via query params or `x_headers`. Use `Ectomancer.child_spec/2` for multi-transport supervision.
- **Igniter installer**`mix igniter.install ectomancer` for fully automated setup (#119)
Automatically: adds dependency, discovers schemas with interactive selection, generates `lib/my_app/mcp.ex`, patches `config/config.exs`, injects `forward` route into the router, adds a supervisor to the supervision tree via AST patching, and prompts for transport selection. Idempotent — safe to re-run.
- **Per-action authorization for Oban bridge**`expose_oban_jobs` now supports granular authorize rules per action (#121)
```elixir
expose_oban_jobs authorize: [
all: fn actor, _action -> actor.role == :admin end,
list_queues: :public,
cancel_job: fn actor, _action -> actor.role == :admin end
]
```
Supports function, module, `:none`/`:public`, or keyword lists with `:all` fallback. Unlisted actions fall through to the global authorization from `use Ectomancer`.
- **Custom MCP Resources**`resource/2` macro for defining custom resources alongside auto-generated schema resources (v1.4.0 — listed here for discoverability)
- **Rate limiting** — configurable token bucket per tool or globally (v1.4.0)
### Changed
- Internal codebase refactored into focused submodules with reduced duplication (#118) — no user-facing API changes
### No breaking changes
## [1.5.0] - 2026-07-17
### Added
- Global authorization policy support (#113)
## [1.4.0] - 2026-07-12
### Added
- change: Add :telemetry events for tools, repo, auth, and rate limiter (#110)
### Fixed
- Silence installation/teardown logs during test runs (#111)
## [1.3.1] - 2026-06-24
### Added
- Support for Elixir 1.20.0 and OTP 29
### Changed
- Updated dependencies to latest compatible versions
### Fixed
- Elixir 1.20 compatibility:
- Removed unreachable `parse_auth_handler(nil)` clause in `Ectomancer.Expose`
- Merged `do_execute/5` clauses into single function with runtime arity check in `Ectomancer.Tool`
- Eliminated type warnings by conditionally generating authorization and execute code in `Ectomancer.Tool` and `Ectomancer.Resource`
- Fixed `Authorization.check/3` spec to include `{:ok, :scoped, fun()}` return type
- Updated `test/support` loading to use `elixirc_paths` instead of `Code.require_file`
- Fixed oban bridge test to avoid always-false type assertion
### Testing
- Increased test coverage to 80%+
## [1.3.0] - 2026-05-19
### Added
- `resource/2` macro for defining custom MCP resources, parallel to the existing `tool/2` macro
- Static resources (`uri "scheme://path"`) and templated resources (`uri "scheme://{var}"`)
- Optional `authorize` block for access control (inline function, policy module, or `:none`)
- Configurable `mime_type` (defaults to `"text/plain"`)
- Read handler `fn params, actor -> {:ok, content} | {:error, reason} end`
- Per-schema metadata resources now auto-generated from `expose` alongside existing tools
- `:resource` option in `expose/2` accepts `false` to opt out of per-schema resource generation
- 19 new tests for custom resource DSL
### Changed
- `use Ectomancer` now imports `resource: 2` macro
- Capabilities updated to include `[:tools, :resources]`
## [1.2.1] - 2026-05-13
### Fixed
- **Hex publish compatibility** — Replaced `inflex` (GitHub dep, blocked hex.publish) with
`plurality` (Hex dep) for route name singularization. Plurality is a modern, zero-regex
inflection library with verified accuracy across 80k+ noun pairs.
### Changed
- `mix.exs` dependency: `inflex``plurality ~> 0.2`
- `RouteIntrospection.singularize/1` now calls `Plurality.singularize/1` directly
## [1.2.0] - 2026-05-13
### Added
- **MCP Resources for schema discovery** — Each `expose`d schema now automatically registers an MCP resource at `ectomancer://schemas/{name}` returning full schema metadata (fields, types, associations, primary key, available actions). A top-level `ectomancer://schemas` resource lists all registered schemas. Opt-out per schema with `resource: false`. (Closes #56)
- **Dynamic association preloading** — New `preloadable` option for `expose` allows LLMs to dynamically request associated records via an `include` parameter on `list` and `get` tools. Supports `preloadable: true` (all associations) or `preloadable: [:posts, :comments]` (specific). Requested includes are validated against allowed associations. (Closes #57)
- **Rate limiting** — Token bucket algorithm with ETS storage. Configurable per-tool and global limits. Opt-in via `config :ectomancer, :rate_limits`.
- **Multi-repo support** — expose schemas from different repos with `expose User, repo: MyApp.ReplicaRepo`. Falls back to global repo config.
- **Browser MCP client** — Zero-dependency HTML browser client at `priv/ectomancer.html`. Browse tools, call them, see results. No build step required.
- **Auto-deployed ExDoc to GitHub Pages** — New CI workflow builds docs on push to main and deploys via `actions/deploy-pages`.
- **CI, Hex, and Docs badges** to README header.
### Fixed
- `singularize` helper now handles edge cases (`status`, `address`, `series`) correctly.
- Error handling now returns structured `{:error, %{code:, message:, details:}}` tuples consistently.
### Testing
- **426 tests** (up from 260), all passing
- 28 new tests: 19 for MCP Resources, 9 for dynamic preloading
- 9 new rate limiter tests
- Validated multi-repo integration with secondary Phoenix app (SQLite)
## [1.1.0] - 2026-04-23
### Added
- Interactive setup tool (`mix ectomancer.setup`) for automatic project configuration
- Auto-discovers Ecto schemas via module introspection and file scanning
- Prompts for schema selection, Oban bridge, and tool namespace
- Generates MCP module with proper `expose` declarations
- Updates mix.exs, config.exs, and router files automatically
- Derives module name from app name (e.g., `TestEctoApp.MCP`)
- Schema discovery module (`Ectomancer.Installer.SchemaDiscovery`) with dual discovery strategy
- Config updater (`Ectomancer.Installer.ConfigUpdater`) for idempotent file patching
- Dependency checker (`Ectomancer.Installer.DependencyChecker`) for required/optional dep validation
- Template renderer (`Ectomancer.Installer.TemplateRenderer`) for MCP module generation
- Igniter installer stub (`Ectomancer.Igniter`)
### Testing
- **260 tests** (up from 223), all passing
- Full integration tests for the setup tool workflow
## [1.0.0] - 2026-03-29
### 🎉 Official v1.0.0 Release - Production Ready!
Ectomancer is now officially stable and ready for production use! Three phases of development complete.
### Added
#### Optional Oban Bridge (Issue #15) - Phase 3 Final Feature!
- New `expose_oban_jobs/0` and `expose_oban_jobs/1` macros for Oban integration
- Automatically generates 5 MCP tools for job queue management:
- `list_oban_queues` - List all queues with job statistics (total, executing, available, retryable, discarded)
- `get_queue_depth` - Get detailed counts for a specific queue
- `list_stuck_jobs` - Find executing jobs with optional filters (queue, worker, min_age, limit)
- `retry_job` - Retry failed or discarded jobs by ID
- `cancel_job` - Cancel or delete jobs by ID
- Only activates when Oban is in dependencies (optional dependency support)
- Supports `:namespace` option for tool naming (e.g., `background_list_oban_queues`)
- Comprehensive test coverage (13 tests)
```elixir
# Expose all Oban job management tools
expose_oban_jobs
# With namespace prefix
expose_oban_jobs(namespace: :background)
# Generates: background_list_oban_queues, background_get_queue_depth, etc.
```
#### Phoenix Route Introspection (Issue #14) - Phase 3 Complete!
- New `expose_routes/1` macro to auto-generate MCP tools from Phoenix router
- Support for all HTTP methods: GET, POST, PUT, PATCH, DELETE
- Smart tool naming with automatic singularization:
- `/users``get_users`, `post_users`
- `/users/:id``get_user`, `put_user`, `delete_user`
- Route filtering options:
- `:only` - Include only specific paths
- `:except` - Exclude specific paths
- `:methods` - Filter by HTTP methods
- `:namespace` - Prefix tool names (e.g., `api_get_users`)
- Automatic path parameter mapping to tool parameters
- Direct controller action execution via `Plug.Test.conn`
- Proper handling of `Plug.Conn.AlreadySentError`
```elixir
# Expose all routes
expose_routes MyAppWeb.Router
# With filtering
expose_routes MyAppWeb.Router,
only: ["/api/users"],
namespace: :api,
methods: ["GET", "POST"]
```
### Testing
- **223 tests** (up from 193)
- **30 new tests**: 13 for Oban bridge, 17 for route introspection
- Full integration tested with sweetcorn Phoenix app including:
- Oban job insertion, retry, and cancellation via MCP tools
- Route tool execution through Phoenix controllers
- All authorization strategies working with new features
### Issues Closed
- [#15](https://github.com/GustavoZiaugra/ectomancer/issues/15) - Create optional Oban bridge for job queue management
- [#14](https://github.com/GustavoZiaugra/ectomancer/issues/14) - Implement Phoenix route introspection for MCP tools
## [0.1.0-rc.3] - 2026-03-18
### Added
#### Read-Only Mode (Issue #12)
- New `:readonly` option for `expose/2` macro
- When `readonly: true`, only generates `:list` and `:get` tools
- Prevents create, update, destroy operations
- Perfect for public read-only access to data
```elixir
expose MyApp.Blog.Post, readonly: true
# Generates only: list_posts, get_post
```
#### Changeset Error Mapping (Issue #13)
- Enhanced error messages from Ecto changeset validations
- Automatic categorization of validation errors:
- **presence**: Missing required fields
- **format**: Invalid format (email regex, etc.)
- **inclusion**: Value not in allowed set
- **confirmation**: Confirmation doesn't match
- **length**: String length issues
- **comparison**: Numeric comparison failures
- Improved database error detection:
- **unique_violation**: "Duplicate value: Record with this value already exists"
- **foreign_key_violation**: "Invalid reference: Related record does not exist"
- **not_null_violation**: "Missing required parameter: Field Name"
- Schema changeset integration
- Uses schema's `changeset/2` function when available
- Ensures unique_constraint validations work properly
- Returns structured error responses instead of binary strings
### Changed
- Updated README.md with read-only mode and error handling documentation
- Enhanced error categorization in `format_error/1`
### Fixed
- Fixed unique constraint violations to return proper error responses
- Fixed foreign key violations to show descriptive messages
- Fixed changeset validation errors to show field names and messages
### Testing
- **193 tests** (up from 172)
- **21 new tests**: 16 for read-only mode, 6 for error mapping
- Full integration tested with sweetcorn Phoenix app
- All authorization strategies still working
### Issues Closed
- [#12](https://github.com/GustavoZiaugra/ectomancer/issues/12) - Implement read-only mode for expose macro
- [#13](https://github.com/GustavoZiaugra/ectomancer/issues/13) - Map Ecto changeset errors to MCP error responses
## [0.1.0-rc.2] - 2026-03-17
### Added
#### Authorization System (Phase 2)
- **Inline function authorization** - Simple auth checks with inline functions
```elixir
authorize fn actor, action -> actor.role == :admin end
```
- **Policy module authorization** - Reusable authorization logic via behavior
```elixir
authorize with: MyApp.Policies.UserPolicy
```
- **Public access** - `:none` authorization for public endpoints
```elixir
authorize :none
```
- **Per-schema authorization** - Global auth rules for all actions on a schema
- **Per-action authorization** - Fine-grained control with action-specific rules
- **Authorization cascade** - Multiple auth levels work together
#### Binary ID / UUID Support
- Full support for `binary_id` primary keys
- Automatic UUID string casting
- Works with all CRUD operations
#### Enhanced Error Messages
- Descriptive error messages (e.g., "Missing required parameter: User id")
- Proper MCP error codes (-32602 for validation, -32603 for internal)
- Field identification in error responses
### Changed
- Updated README.md with comprehensive authorization documentation
- Improved error handling with better error categorization
### Fixed
- Fixed binary_id primary key handling in get/update/destroy operations
- Fixed Peri validation compatibility with JSON Schema format
- Fixed tool parameter generation for nested blocks
- Fixed atom vs string key handling in normalize_params
### Security
- SQL injection prevention via parameterized queries
- Row limits to prevent memory exhaustion (100 records default)
- Authorization checks before tool execution
- Proper error messages without exposing internal details
### Testing
- **172 tests** (up from 128)
- **35 authorization-specific tests**
- All authorization strategies tested
- Full integration tested with sweetcorn Phoenix app
### Issues Closed
- [#10](https://github.com/GustavoZiaugra/ectomancer/issues/10) - Design and implement authorization hook system
- [#11](https://github.com/GustavoZiaugra/ectomancer/issues/11) - Add per-schema and per-action authorization granularity
- [#35](https://github.com/GustavoZiaugra/ectomancer/issues/35) - Fix critical bugs in binary ID handling and tool parameter schemas
## [0.1.0-rc.1] - 2026-03-16
### Added
- First release candidate with fully functional CRUD operations
- Core MCP server implementation via `Ectomancer` module
- `expose/2` macro for auto-generating CRUD tools from Ecto schemas (list, get, create, update, destroy)
- `tool/2` macro for custom tool definitions with param validation
- `Ectomancer.Plug` for seamless Phoenix router integration
- `Ectomancer.Repo` abstraction supporting all major CRUD operations
- Automatic actor extraction and threading through `conn.assigns`
- Field filtering support via `:only` and `:except` options
- Namespace support to prevent tool naming collisions
- Comprehensive test suite (128 tests, all passing)
- Full Credo and Dialyzer compliance
- Support for Phoenix 1.7 and 1.8
- MIT License
### Fixed
- Fixed Peri schema validation crashes by disabling params in exposed tools
- Fixed GenServer crashes during CRUD operations with proper error handling
- Fixed tool execution to return proper Anubis Response format
- Fixed repo error handling with comprehensive try/rescue blocks
### Security
- SQL injection prevention via parameterized queries in Repo operations
- Row limits to prevent memory exhaustion (100 records default)
- Proper error messages without exposing internal details
[Unreleased]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.7.0...HEAD
[1.7.0]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.6.0...v1.7.0
[1.6.0]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.5.0...v1.6.0
[1.5.0]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.4.0...v1.5.0
[1.4.0]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.3.1...v1.4.0
[1.3.1]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.3.0...v1.3.1
[1.3.0]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.2.1...v1.3.0
[1.2.1]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.2.0...v1.2.1
[1.2.0]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.1.0...v1.2.0
[1.1.0]: https://github.com/GustavoZiaugra/ectomancer/compare/v1.0.0...v1.1.0
[1.0.0]: https://github.com/GustavoZiaugra/ectomancer/releases/tag/v1.0.0
[0.1.0-rc.4]: https://github.com/GustavoZiaugra/ectomancer/releases/tag/v0.1.0-rc.4
[0.1.0-rc.3]: https://github.com/GustavoZiaugra/ectomancer/releases/tag/v0.1.0-rc.3
[0.1.0-rc.2]: https://github.com/GustavoZiaugra/ectomancer/releases/tag/v0.1.0-rc.2
[0.1.0-rc.1]: https://github.com/GustavoZiaugra/ectomancer/releases/tag/v0.1.0-rc.1