Current section
Files
Jump to
Current section
Files
iconvex_extras
README.md
README.md
# Iconvex Extras
`iconvex_extras` is the GNU libiconv 1.19 `--enable-extra-encodings` companion
for [Iconvex](https://hex.pm/packages/iconvex). It adds the exact 86-codec
complement of Core's 112 GNU-default codecs: 7 GNU extra, 9 AIX, 15 DOS,
2 OSF/1, and 53 z/OS encodings.
```elixir
def deps do
[
{:iconvex, "~> 0.1.1"},
{:iconvex_extras, "~> 0.1.1"}
]
end
```
Starting the application is enough:
```elixir
{:ok, "Café"} = Iconvex.convert(<<0x43, 0x61, 0x66, 0x82>>, "CP437", "UTF-8")
{:ok, "IBM-1047"} = Iconvex.canonical_name("CP1047")
{:ok, all} = Application.ensure_all_started(:iconvex_extras)
198 = length(Iconvex.encodings())
```
Stopping Extras returns a Core-only runtime to exactly 112 codecs; restarting it
restores all 198. Extras contributes 342 normalized GNU spelling keys, for 758
Core-plus-Extras spellings in total.
## One atomic extension
The application publishes all 86 codec routes and the `:euc_jisx0213` table
provider through one `Iconvex.Extension.register/2` transaction. One opaque
token owns the complete commit. A conflict at even the final route rolls back
every earlier route and the provider; shutdown removes only records still owned
by that token. Stop/restart and a supervised Core registry-worker restart retain
the same exact ownership guarantees.
Extras uses managed priority 300 so GNU compatibility aliases outrank aliases
from ordinary source leaves. Core built-ins still resolve first, and canonical
claims still outrank aliases regardless of package priority.
The package contains 85 packed mapping tables plus the ISO-2022-JP-3 state
machine. Shared Core engines provide native linear strict, discard, callback,
and substitution paths without a NIF or operating-system `iconv` dependency.
Extras is one leaf in the final split, whose runtime packages expose 2,105 unique canonical codecs.
The unpublished research/integration workspace owns no runtime registry routes,
and Extras depends only on Iconvex Core.
## Exact support and evidence
[SUPPORTED_ENCODINGS.md](SUPPORTED_ENCODINGS.md) is the generated complete
86-codec list with GNU definition origin.
[SUPPORTED_CODEC_INVENTORY.csv](SUPPORTED_CODEC_INVENTORY.csv) is the
machine-readable canonical-name, alias, module, and statefulness snapshot.
The tests port all 85 applicable GNU mapping and inverse fixtures,
ISO-2022-JP-3 state and streaming behavior, IBM-1047 policy surfaces, long
invalid-input recovery, exact alias coverage, and atomic lifecycle behavior.
The committed sequential UCS-4BE corpus contains every code point from U+0000
through U+10FFFF: 1,112,064 Unicode scalar values and all 2,048 surrogate code
points.
[EXHAUSTIVE_UNICODE_DIFFERENTIAL.md](EXHAUSTIVE_UNICODE_DIFFERENTIAL.md)
records the source-bound 198-codec forward, reverse, and cross-decode comparison
with GNU libiconv 1.19. The final run is byte-exact over all 1,114,112 code
points, with zero mismatches. Forward and reverse timings are independently
gated at 30x using a C11 in-memory GNU engine helper. Both engines calibrate
one fixed iteration count using the fastest of three probes at every doubling;
that fastest probe must reach 10,000,000 ns, so one delayed probe cannot end
calibration early. They then report the fastest of three final batches at the
fixed count. EUC-JISX0213 is 2.95x forward/15.38x reverse and CP943 is 4.03x/20.92x.
Across all 198 codecs, ISO-2022-CN is worst at 24.80x under the unchanged 30x
ceiling.
```sh
ICONVEX_PATH=../iconvex mix test --warnings-as-errors
ICONVEX_PATH=../iconvex mix compile --warnings-as-errors
ICONVEX_PATH=../iconvex MIX_ENV=prod mix run bench/benchmark.exs
GNU_ICONV=/path/to/gnu-libiconv-1.19/bin/iconv \
ICONVEX_PATH=../iconvex MIX_ENV=prod \
mix run tools/exhaustive_unicode_differential.exs
```
The C helper is offline benchmark/test tooling, not shipped runtime code. The
Hex artifact contains only runtime source and tables, licenses, inventories,
and public evidence; it excludes `test/`, `bench/`, `tools/`, PDFs, and large
development corpora.
`ICONVEX_PATH` is only the sibling-checkout development override. Published
builds use `{:iconvex, "~> 0.1.1"}` from Hex.