Current section
Files
Jump to
Current section
Files
lib/ex_arrow/flight_sql.ex
defmodule ExArrow.FlightSQL do
@moduledoc """
Arrow Flight SQL client for ExArrow.
Provides a production-grade Elixir interface for executing SQL queries against
remote Flight SQL servers (DuckDB, DataFusion, Dremio, InfluxDB v3, and others)
and consuming results as Arrow-native data.
## What Flight SQL is
Arrow Flight SQL layers SQL query semantics on top of Arrow Flight (gRPC + Arrow IPC).
Queries are dispatched to the server, which executes them and streams results back as
columnar `RecordBatch` data — the same Arrow format used everywhere in ExArrow.
Flight SQL is the correct choice when connecting to **remote query servers**.
For in-process databases (DuckDB local, SQLite, PostgreSQL via driver), use
`ExArrow.ADBC` instead.
## Entry point
All operations go through `ExArrow.FlightSQL.Client`:
{:ok, client} = ExArrow.FlightSQL.Client.connect("localhost:32010")
{:ok, result} = ExArrow.FlightSQL.Client.query(client, "SELECT * FROM t")
See `ExArrow.FlightSQL.Client` for the full API.
## v0.5.0 scope
The following are supported:
- Ad-hoc SQL query execution (`query/2`, `query!/2`, `stream_query/2`)
- DML execution with affected-row count (`execute/2`)
- Lazy streaming of large result sets with `Enumerable` support (`stream_query/2`)
- Prepared statements (`Client.prepare/2`, `Statement.execute/1`, `Statement.execute_update/1`)
- Metadata discovery: `get_tables/2`, `get_db_schemas/2`, `get_sql_info/1`
- TLS connections (plaintext, OS trust store, custom CA)
- Bearer-token and custom gRPC header injection
- Explorer and Nx integration via `Result.to_dataframe/1`, `Result.to_tensor/2`
- Mox-compatible `ClientBehaviour` for unit testing without a server
The following are **not** supported in v0.5.0 and are deferred:
- Bulk ingestion (`DoPut`)
- Transactions (`BEGIN`, `COMMIT`, `ROLLBACK`)
- Multi-endpoint distributed query results
- Parameter binding for prepared statements (v0.6.0)
- Flight SQL server-side implementation (ExArrow is a client library only)
## Ecosystem integration
# Convert result to Explorer DataFrame
{:ok, result} = ExArrow.FlightSQL.Client.query(client, "SELECT * FROM t")
{:ok, df} = ExArrow.FlightSQL.Result.to_dataframe(result)
# Convert a numeric column to an Nx tensor
{:ok, tensor} = ExArrow.FlightSQL.Result.to_tensor(result, "price")
# Use lazy stream for large results
{:ok, stream} = ExArrow.FlightSQL.Client.stream_query(client, "SELECT * FROM big_table")
{:ok, df} = ExArrow.Explorer.from_stream(stream)
"""
end