Packages

Pure Elixir bibliographic encoding extensions for Iconvex

Current section

Files

Jump to
Raw

README.md

# Iconvex Bibliographic
`iconvex_bibliographic` provides source-qualified library, cataloging, and TeX codecs.
It extends [Iconvex](https://hex.pm/packages/iconvex) with pure Elixir code.
It uses no NIF, port process, platform `iconv`, or `iconvex_specs` runtime dependency.
Add the package and its Core dependency:
```elixir
def deps do
[
{:iconvex, "~> 0.1.1"},
{:iconvex_bibliographic, "~> 0.1"}
]
end
```
The OTP application registers every byte codec in one transaction.
Stopping it removes only routes that this package owns.
## Codec families
| Family | Canonical encoding | What makes it interesting |
| --- | --- | --- |
| MARC | `MARC-8` | Stateful ISO 2022 designations, EACC 24-bit characters, ANSEL combining marks, and spanning half marks |
| MARC | `ANSEL` | Strict ANSI/NISO Z39.47 ASCII + Extended Latin environment |
| Cork | `TEX-T1-EC-GLYPH` | The EC glyph interpretation, including ligature scalars and exact `SS` sequence semantics |
| Cork | `TEX-T1-CMAP-1.0J` | CTAN CMap extraction with deterministic longest-match `ffi`/`ffl`/`ff`/`fi`/`fl` encoding |
| OT1 | `TEX-OT1-CMAP-1.0J` | Normal CTAN OT1 extraction. Byte `20` remains undefined. |
| OT1 | `TEX-OT1TT-CMAP-1.0J` | Typewriter OT1, where byte `20` is U+2423 OPEN BOX |
| TeX math | `TEX-LIVE-OML-CMMI10-TOUNICODE-2026` | Exact 7-bit `cmmi10` ToUnicode semantics, with packed MSB/LSB transports |
| TeX math | `TEX-LIVE-OMS-CMSY10-TOUNICODE-2026` | Exact 7-bit `cmsy10` ToUnicode semantics, with packed MSB/LSB transports |
| Glyph vectors | `CTAN-LY1-TEXNANSI-1.1-AGL-4036A9CA` | A pinned TeX'n'ANSI 1.1 vector resolved through a pinned Adobe Glyph List revision |
| Glyph vectors | `ADOBE-POSTSCRIPT-3-ISOLATIN1-AGL-4036A9CA` | PostScript LanguageLevel 3 ISOLatin1Encoding—specifically not ISO-8859-1 |
`Iconvex.Bibliographic.registrations/0`, `codecs/0`, `encodings/0`, and
`packed_profiles/0` return the complete manifest-ordered surface.
## MARC-8: combining marks and EACC
MARC-8 stores an ANSEL combining mark before its base character.
Iconvex returns the base character before its combining mark.
```elixir
iex> Iconvex.convert(<<0xE2, ?e>>, "MARC-8", "UTF-8")
{:ok, "e\u0301"}
iex> Iconvex.convert("e\u0301", "UTF-8", "MARC8")
{:ok, <<0xE2, ?e>>}
```
The same codec handles custom and ISO 2022 designations plus three-byte EACC:
```elixir
iex> Iconvex.Bibliographic.MARC8.decode(<<0x1B, ?$, ?1, 0x21, 0x30, 0x21>>)
{:ok, [0x4E00]}
iex> Iconvex.Bibliographic.MARC8.decode(<<0xEB, ?a, 0xEC, ?b>>)
{:ok, [?a, 0x0361, ?b]}
```
## TeX ligatures with exact inverse semantics
The CMap profiles encode the longest source-defined sequence first:
```elixir
iex> Iconvex.convert("ffifflfffifl", "UTF-8", "TEX-OT1-CMAP-1.0J")
{:ok, <<0x0E, 0x0F, 0x0B, 0x0C, 0x0D>>}
iex> Iconvex.convert("ffiAfflASS", "UTF-8", "TEX-T1-CMAP-1.0J")
{:ok, <<0x1E, ?A, 0x1F, ?A, 0xDF>>}
```
These profiles extract Unicode.
They do not claim identical semantics for every historical font with the same broad label.
## Packed seven-bit math
The byte-form OML and OMS codecs use one logical 7-bit unit per byte.
This format works with `Iconvex.convert/4`.
The packed facade emits contiguous units.
```elixir
alias Iconvex.Bibliographic.Packed
iex> Packed.encode_from_utf8("Γ∆", "OML")
{:ok, <<0::7, 1::7>>}
iex> {:ok, lsb} = Packed.encode_from_utf8("Γ∆", "OML-PACKED-LSB")
iex> lsb
%Iconvex.Packed.LSB{data: <<0x80, 0x00>>, bit_size: 14, unit_bits: 7, bit_order: :lsb}
iex> Packed.decode_to_utf8(lsb, "OML-PACKED-LSB")
{:ok, "Γ∆"}
```
The fully qualified forms use the OML or OMS canonical name with a packed suffix.
The suffix is `-PACKED-MSB` or `-PACKED-LSB`.
An explicit bit order must agree with the name.
## Streams and recovery policies
Every codec implements strict, discard, substitution, direct UTF-8, and chunk callbacks.
Stateful MARC-8 works across arbitrary stream boundaries.
Longest-match TeX sequences also work across those boundaries.
```elixir
chunks = [<<0x1B, ?$, ?1, 0x21>>, <<0x30, 0x21>>]
{:ok, stream} = Iconvex.stream(chunks, "MARC-8", "UTF-8")
IO.iodata_to_binary(Enum.to_list(stream))
# => "一"
```
## Canonical names
Many typographic encodings combine a vector, glyph list, extraction policy, and source revision.
Short aliases could merge distinct profiles.
Canonical names retain the source boundary.
The package registers only reviewed and unambiguous aliases.
The package includes the exact runtime and source assets for these profiles.
It includes SHA-256 metadata, retained AGL and LPPL terms, and the 12-row manifest.
Tests cover all 256 bytes where applicable and all 128 OML or OMS units.
They cover inverse mappings, malformed input, every stream split, and both packed orders.
They also cover application restart ownership.
Both glyph-vector encoders process all 1,112,064 Unicode scalar values.
See [BENCHMARKS.md](BENCHMARKS.md) for reproducible native-path gates and
[NOTICE](NOTICE) for upstream provenance and license boundaries.
<!-- iconvex-codec-catalog:start -->
## Complete codec catalog
The generator builds this table from the release inventories.
Do not edit this section by hand.
Codec names ignore ASCII letter case.
You can use each alias to look up its codec.
Aliases do not increase the canonical codec count.
**10 canonical codecs · 32 aliases · 42 accepted names.**
### Registered codec names
| Canonical codec | Accepted aliases | Stateful |
|---|---|:---:|
| `ADOBE-POSTSCRIPT-3-ISOLATIN1-AGL-4036A9CA` | `ADOBE-POSTSCRIPT-3-ISOLATIN1ENCODING-AGL-4036A9CA` | No |
| `ANSEL` | `ANSI-Z39.47`<br>`csANSEL`<br>`Z39.47` | No |
| `CTAN-LY1-TEXNANSI-1.1-AGL-4036A9CA` | `CTAN-TEXNANSI-1.1-AGL-4036A9CA` | No |
| `MARC-8` | `csMARC8`<br>`MARC8`<br>`MARC_8` | Yes |
| `TEX-LIVE-OML-CMMI10-TOUNICODE-2026` | `OML`<br>`OML-ENCODING`<br>`TEX-MATH-ITALIC` | No |
| `TEX-LIVE-OMS-CMSY10-TOUNICODE-2026` | `OMS`<br>`OMS-ENCODING`<br>`TEX-MATH-SYMBOLS` | No |
| `TEX-OT1-CMAP-1.0J` | `TEX-OT1-0-CMAP-1.0J` | No |
| `TEX-OT1TT-CMAP-1.0J` | `TEX-OT1TT-0-CMAP-1.0J` | No |
| `TEX-T1-CMAP-1.0J` | `T1-CMAP`<br>`TEX-T1-0`<br>`TEX-T1-CMAP` | No |
| `TEX-T1-EC-GLYPH` | `8T`<br>`CORK`<br>`CORK-ENCODING`<br>`CORKENCODING`<br>`EC`<br>`EC-ENCODING`<br>`ECENCODING`<br>`T1`<br>`TEX-LATIN-1`<br>`TEX-T1`<br>`TEX256`<br>`TEX256.ENC`<br>`TEXLATIN1` | No |
### Packed transport profiles
**2 base codec profiles · 4 explicit packed names.**
These profiles change the bit transport only.
They do not add registered Unicode mappings.
| Base codec | Unit bits | Standard order | Explicit packed names |
|---|---:|:---:|---|
| `TEX-LIVE-OML-CMMI10-TOUNICODE-2026` | 7 | MSB | `TEX-LIVE-OML-CMMI10-TOUNICODE-2026-PACKED-MSB`<br>`TEX-LIVE-OML-CMMI10-TOUNICODE-2026-PACKED-LSB` |
| `TEX-LIVE-OMS-CMSY10-TOUNICODE-2026` | 7 | MSB | `TEX-LIVE-OMS-CMSY10-TOUNICODE-2026-PACKED-MSB`<br>`TEX-LIVE-OMS-CMSY10-TOUNICODE-2026-PACKED-LSB` |
<!-- iconvex-codec-catalog:end -->