weaviate_ex

<p align="center">
  <img src="assets/weaviate_ex.svg" alt="WeaviateEx Logo" width="200" height="200">
</p>

# WeaviateEx

[![Elixir](https://img.shields.io/badge/elixir-1.18-purple.svg)](https://elixir-lang.org)
[![Hex.pm](https://img.shields.io/hexpm/v/weaviate_ex.svg)](https://hex.pm/packages/weaviate_ex)
[![Documentation](https://img.shields.io/badge/docs-hexdocs-purple.svg)](https://hexdocs.pm/weaviate_ex)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)

A modern Elixir client for [Weaviate](https://weaviate.io) vector database (v2.0).

WeaviateEx provides a clean, idiomatic Elixir interface with:
- ✅ **158 Tests Passing** - Comprehensive test coverage with Mox-based mocking
- ✅ **TDD Architecture** - Built test-first with protocol-based design
- ✅ **Collections API** - Full schema management (8 functions)
- ✅ **Data Operations API** - Complete CRUD with UUID generation & vectors (7 functions)
- ✅ **Filter System** - Advanced filtering with 13 operators & GraphQL conversion (26 tests)
- ✅ **Objects API** - Full CRUD operations with vectors
- ✅ **Batch Operations** - Efficient bulk imports
- ✅ **GraphQL Queries** - Complex searches with near_text, hybrid, BM25
- ✅ **Mix Tasks** - Manage Weaviate with `mix weaviate.start/stop/status/logs`
- ✅ **Auto Health Checks** - Validates connection on startup with strict mode
- ✅ **Easy Setup** - One-command Docker installation

## Quick Start

### 1. Install Weaviate

Run the installation script to set up Weaviate with Docker:

```bash
./install.sh
```

This will:
- Install Docker (if needed)
- Create a `.env` file with configuration
- Start Weaviate in Docker
- Verify the connection

For detailed installation instructions, see [INSTALL.md](INSTALL.md).

### 2. Add to Your Project

Add `weaviate_ex` to your `mix.exs` dependencies:

```elixir
def deps do
  [
    {:weaviate_ex, "~> 2.0"}
  ]
end
```

Then run:

```bash
mix deps.get
```

### 3. Configure

The library automatically reads configuration from environment variables (loaded from `.env`):

```bash
# .env file (created by install.sh)
WEAVIATE_URL=http://localhost:8080
WEAVIATE_API_KEY=  # Optional, for authenticated instances
```

Alternatively, configure in `config/config.exs`:

```elixir
config :weaviate_ex,
  url: "http://localhost:8080",
  api_key: nil,    # Optional
  strict: true     # Default: true - fails fast if Weaviate is unreachable
```

**Strict Mode**: By default, WeaviateEx will raise an error on startup if it cannot connect to Weaviate. Set `strict: false` to allow the application to start anyway (useful for development when Weaviate might not always be running).

### 4. Test the Connection

The library automatically performs a health check on startup. You'll see:

```
[WeaviateEx] Successfully connected to Weaviate
  URL: http://localhost:8080
  Version: 1.28.1
```

If configuration is missing, you'll get friendly error messages:

```
╔════════════════════════════════════════════════════════════════╗
║                  WeaviateEx Configuration Error                 ║
╠════════════════════════════════════════════════════════════════╣
║  Missing required configuration: WEAVIATE_URL                    ║
║                                                                  ║
║  Please set the Weaviate URL using one of these methods:        ║
║  1. Environment variable: export WEAVIATE_URL=http://localhost:8080
║  2. Application configuration (config/config.exs)                ║
║  3. Runtime configuration (config/runtime.exs)                   ║
╚════════════════════════════════════════════════════════════════╝
```

## Usage

### Health Check

```elixir
# Check if Weaviate is accessible
{:ok, meta} = WeaviateEx.health_check()
# => %{"version" => "1.28.1", "modules" => %{}}

# Check readiness
{:ok, true} = WeaviateEx.ready?()

# Check liveness
{:ok, true} = WeaviateEx.alive?()
```

### Collections

```elixir
# Create a collection
{:ok, collection} = WeaviateEx.Collections.create("Article", %{
  description: "News articles",
  properties: [
    %{name: "title", dataType: ["text"]},
    %{name: "content", dataType: ["text"]},
    %{name: "publishedAt", dataType: ["date"]}
  ],
  vectorizer: "text2vec-openai"  # or "none" for custom vectors
})

# List all collections
{:ok, schema} = WeaviateEx.Collections.list()

# Get a specific collection
{:ok, collection} = WeaviateEx.Collections.get("Article")

# Add a property to existing collection
{:ok, property} = WeaviateEx.Collections.add_property("Article", %{
  name: "author",
  dataType: ["text"]
})

# Delete a collection
{:ok, _} = WeaviateEx.Collections.delete("Article")
```

### Objects

```elixir
# Create an object
{:ok, object} = WeaviateEx.Objects.create("Article", %{
  properties: %{
    title: "Hello Weaviate",
    content: "This is a test article",
    publishedAt: "2025-01-15T10:00:00Z"
  },
  vector: [0.1, 0.2, 0.3, ...]  # Optional, if not using auto-vectorization
})

# Get an object
{:ok, object} = WeaviateEx.Objects.get("Article", uuid)

# List objects with pagination
{:ok, result} = WeaviateEx.Objects.list("Article", limit: 10, offset: 0)

# Update object (full replacement)
{:ok, updated} = WeaviateEx.Objects.update("Article", uuid, %{
  properties: %{title: "Updated Title"}
})

# Patch object (partial update)
{:ok, patched} = WeaviateEx.Objects.patch("Article", uuid, %{
  properties: %{title: "New Title"}
})

# Delete an object
{:ok, _} = WeaviateEx.Objects.delete("Article", uuid)

# Check if object exists
{:ok, true} = WeaviateEx.Objects.exists?("Article", uuid)
```

### Batch Operations

```elixir
# Batch create (efficient for large imports)
objects = [
  %{class: "Article", properties: %{title: "Article 1"}},
  %{class: "Article", properties: %{title: "Article 2"}},
  %{class: "Article", properties: %{title: "Article 3"}}
]

{:ok, result} = WeaviateEx.Batch.create_objects(objects)

# Batch delete with criteria
{:ok, result} = WeaviateEx.Batch.delete_objects(%{
  class: "Article",
  where: %{
    path: ["status"],
    operator: "Equal",
    valueText: "draft"
  }
})
```

### GraphQL Queries & Vector Search

```elixir
alias WeaviateEx.Query

# Simple query
query = Query.get("Article")
  |> Query.fields(["title", "content"])
  |> Query.limit(10)

{:ok, results} = Query.execute(query)

# Semantic search with near_text
query = Query.get("Article")
  |> Query.near_text("artificial intelligence", certainty: 0.7)
  |> Query.fields(["title", "content"])
  |> Query.additional(["certainty", "distance"])
  |> Query.limit(5)

{:ok, results} = Query.execute(query)

# Vector search
query = Query.get("Article")
  |> Query.near_vector([0.1, 0.2, 0.3, ...], certainty: 0.8)
  |> Query.fields(["title"])

{:ok, results} = Query.execute(query)

# Hybrid search (combines keyword + vector)
query = Query.get("Article")
  |> Query.hybrid("machine learning", alpha: 0.5)
  |> Query.fields(["title"])

{:ok, results} = Query.execute(query)

# BM25 keyword search
query = Query.get("Article")
  |> Query.bm25("elixir programming")
  |> Query.fields(["title"])

{:ok, results} = Query.execute(query)

# With filters
query = Query.get("Article")
  |> Query.where(%{
    path: ["publishedAt"],
    operator: "GreaterThan",
    valueDate: "2025-01-01T00:00:00Z"
  })
  |> Query.fields(["title", "publishedAt"])

{:ok, results} = Query.execute(query)
```

## Testing

WeaviateEx uses **Mox** for clean, idiomatic testing with **158 tests** covering all functionality.

### Quick Test Commands

```bash
# Run all unit tests with mocks (default - no Weaviate needed)
mix test

# Run integration tests against real Weaviate
mix test --include integration

# Run specific test file
mix test test/weaviate_ex/api/collections_test.exs

# Run with coverage
mix test --cover
```

### Test Modes

**Mock Mode (Default)** - Fast, Isolated, No Dependencies:
- ✅ Uses Mox to mock HTTP/Protocol responses
- ✅ No Weaviate instance required
- ✅ Fast execution (~0.1 seconds)
- ✅ All 158 unit tests run with mocks
- ✅ Perfect for TDD and CI/CD

**Integration Mode** - Real Weaviate Testing:
- ✅ Tests against live Weaviate instance
- ✅ Validates actual API behavior
- ✅ Requires Weaviate running locally
- ✅ Run with `--include integration` flag
- ✅ 53 integration tests available

### Running Integration Tests

**Step 1: Start Weaviate**
```bash
# Start Weaviate with Docker
mix weaviate.start

# Or use Docker Compose directly
docker compose up -d

# Verify it's running
mix weaviate.status
```

**Step 2: Run Integration Tests**
```bash
# Run all integration tests
mix test --include integration

# Or use environment variable
WEAVIATE_INTEGRATION=true mix test

# Run specific integration test
mix test test/weaviate_ex/objects_test.exs:95 --include integration
```

**Step 3: Clean Up**
```bash
# Stop Weaviate
mix weaviate.stop

# Or remove all data
mix weaviate.stop --remove-volumes
```

### Test Coverage

**Current Coverage (Phase 1 - 50% Complete):**
- ✅ Collections API: 17 tests (8 functions)
- ✅ Filter System: 26 tests (13 operators/combinators)
- ✅ Data Operations: 17 tests (7 functions)
- ✅ Objects API: 15 tests
- ✅ Batch Operations: Basic tests
- ✅ Query System: Basic tests
- 🎯 **Total: 158 tests passing**

All new modules (Collections, Filter, Data) have **100% test coverage**.

## Authentication

For **authenticated Weaviate instances** (production/cloud):

### Option 1: Environment Variable

```bash
# Add to .env file (not committed to git)
WEAVIATE_API_KEY=your-api-key-here

# Or add to ~/.bash_secrets (sourced by ~/.bashrc)
export WEAVIATE_API_KEY=your-api-key-here
```

### Option 2: Application Config

```elixir
# config/runtime.exs (recommended for production)
config :weaviate_ex,
  url: System.fetch_env!("WEAVIATE_URL"),
  api_key: System.fetch_env!("WEAVIATE_API_KEY")
```

### Option 3: Config File (Development Only)

```elixir
# config/dev.exs (never commit production keys!)
config :weaviate_ex,
  url: "http://localhost:8080",
  api_key: "dev-key-here"  # Only for local development
```

**Security Notes:**
- Never commit API keys to version control
- Use environment variables for production
- Add `.env` to `.gitignore` (already done)
- Consider using `~/.bash_secrets` for persistent local keys
- Use `System.fetch_env!/1` in production to fail fast on missing keys

## Development

```bash
# Get dependencies
mix deps.get

# Compile
mix compile

# Run tests (mocked - no Weaviate needed)
mix test

# Run integration tests (requires live Weaviate)
mix test --include integration

# Generate documentation
mix docs

# Run code analysis
mix credo
```

## Managing Weaviate

WeaviateEx provides convenient Mix tasks for managing your local Weaviate instance:

```bash
# Start Weaviate
mix weaviate.start

# Stop Weaviate
mix weaviate.stop

# Check status
mix weaviate.status

# View logs
mix weaviate.logs

# Follow logs in real-time
mix weaviate.logs --follow

# Stop and remove all data (WARNING: deletes everything)
mix weaviate.stop --remove-volumes
```

You can also use Docker Compose directly:

```bash
# Start Weaviate
docker compose up -d

# Stop Weaviate
docker compose down

# View logs
docker compose logs -f weaviate

# Check status
docker compose ps
```

## Environment Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `WEAVIATE_URL` | Yes | - | Full URL to Weaviate (e.g., `http://localhost:8080`) |
| `WEAVIATE_API_KEY` | No | - | API key for authentication |

## Documentation

- [Installation Guide](INSTALL.md) - Detailed setup instructions
- [API Documentation](https://hexdocs.pm/weaviate_ex) - Full API reference (coming soon)
- [Weaviate Docs](https://docs.weaviate.io) - Official Weaviate documentation

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

MIT License. See [LICENSE](LICENSE) for details.

## Acknowledgments

- Built for [Weaviate](https://weaviate.io) vector database
- Inspired by official Python and TypeScript clients