once

defmodule Once do
  @format_docs """
  These are the formats in which an ID can be rendered.
  All are equivalent and can be transformed to one another using `to_format/3`.
  The examples represent the same underlying value.

  - `:url64` a url64-encoded string of 11 characters, for example `"zAjhfZyAAAE"`
  - `:hex` a hex-encoded string of 16 characters, for example `"cc08e17d9c800001"`
  - `:raw` a bitstring of 64 bits, for example `<<204, 8, 225, 125, 156, 128, 0, 1>>`
  - `:signed` a signed 64-bits integer between `-(2^63)` and `2^63-1`, for example `-3744495160545771519`
  - `:unsigned` an unsigned 64-bits integer between `0` and `2^64-1`, for example `14702248913163780097`
  - `:hex32` an extended-hex string of 13 characters, for example `"pg4e2vcsg0002"`
  """
  @options_docs """
  - `:no_noncense` name of the NoNoncense instance used to generate new IDs (default `Once`)
  - `:ex_format` what an ID looks like in Elixir, one of `t:format/0` (default `:url64`). Be sure to read [the caveats](`m:Once#module-format-caveats`).
  - `:db_format` what an ID looks like in your database, one of `t:format/0` (default `:signed`)
  - `:nonce_type` how the nonce is generated, one of `t:nonce_type/0` (default `:counter`)
  - `:mask` use encrypted IDs in Elixir; encrypt on load, decrypt on dump (default `false`)
  - `:prefix` a string prefix to prepend to IDs, for example `"usr_"` or `"prod_"` (default `nil`)
  - `:persist_prefix` whether to store the prefix in the database; requires a binary `:db_format` (default `false`)
  """

  @moduledoc """
  Once is an Ecto type for locally unique (unique within your domain or application) 64-bit IDs generated by multiple Elixir nodes. Locally unique IDs make it easier to keep things separated, simplify caching and simplify inserting related things (because you don't have to wait for the database to return the ID).

  Because a Once fits into an SQL bigint, they use little space and keep indexes small and fast. Because of their [structure](https://hexdocs.pm/no_noncense/NoNoncense.html#module-nonce-types) they have counter-like data locality, which helps your indexes perform well, [unlike UUIDv4s](https://www.cybertec-postgresql.com/en/unexpected-downsides-of-uuid-keys-in-postgresql/).

  Once IDs are based on counter, time-sortable or encrypted nonces. These underlying values can then be encoded in several formats, can be prefixed and can be masked. And you can combine all of these options, providing great flexibility.

  The library has only `Ecto` and its sibling `NoNoncense` as dependencies. NoNoncense generates the actual values and performs incredibly well, hitting rates of tens of millions of nonces per second, and it also helps you to safeguard the uniqueness guarantees.

  > #### Read the migration guide {: .warning}
  >
  > If you're upgrading from v0.x.x and you use encrypted IDs, please read the [Migration Guide](MIGRATION.md) carefully - there are breaking changes that require attention to preserve uniqueness guarantees.

  ## Quick Start

  To get going, you need to set up a `NoNoncense` instance to generate the base unique values. Follow [its documentation](https://hexdocs.pm/no_noncense) to do so. `Once` expects an instance with its own module name by default, like so:

      # application.ex (read the NoNoncense docs!)
      machine_id = NoNoncense.MachineId.id!(opts)
      NoNoncense.init(name: Once, machine_id: machine_id)

      # if you want to use encrypted/masked IDs, also pass a 256-bits key
      NoNoncense.init(name: Once, machine_id: machine_id, base_key: System.get_env("ONCE_SECRET_KEY"))

  In your `Ecto` schemas, you can then use the type:

      schema "things" do
        field :id, Once, autogenerate: true
      end

  And that's it!

  ## Core Concepts

  ### ID Generation Types

  Once supports three types of ID generation, controlled by the `:nonce_type` option:

  - **Counter** (default): Really fast to generate, predictable, works well with b-tree indexes. IDs are generated using a machine init timestamp, machine ID and counter.
  - **Sortable**: Time-sortable like a [Snowflake ID](https://en.wikipedia.org/wiki/Snowflake_ID). Use this when chronological ordering is important.
  - **Encrypted**: Unique and unpredictable, like a UUIDv4 but shorter. Use when you need unpredictable IDs. Note that encrypted IDs cost you data locality, decrease index performance and are slightly slower to generate. Alternatively, use [masked](#module-masked-ids) counter/sortable IDs.

  ### Data Formats

  IDs can be represented in multiple formats, both in your Elixir application (`:ex_format`) and in your database (`:db_format`). By default, IDs are url64-encoded 11-character strings in Elixir and signed bigints in the database.

  #{@format_docs}

  With the default formats, you might see `"AAAAAACYloA"` in Elixir and `10_000_000` in your database. The reasoning behind these defaults is that the encoded format is readable, short, and JSON-safe by default, while the signed format means you can use a standard bigint column type (resulting in a small, fast index).

  The negative integers will not cause problems with Postgres and MySQL - they both happily accept them. Negative integers will only start to appear after ~70 years of usage. However, be careful if you wish to sort by ID (see [Sorting Considerations](#module-sorting-considerations)).

  ### Prefixed IDs

  You can add prefixes to IDs to make them recognizable and self-documenting. This is useful for debugging, API clarity and type safety. A prefixed ID looks like `"usr_AV7m9gAAAAU"` or `"prod_123"` - a human-readable prefix followed by the actual ID value. Use the `:prefix` option to specify the desired prefix.

  Note that prefixed IDs are always binaries - using `:unsigned` or `:signed` as `:ex_format` results in numeric strings like `"prod_123"`, not raw integers. Autogenerated IDs will include the prefix.

  The prefixes can optionally be persisted to the database. This is controlled by option `:persist_prefix`, the default is `false`. If the prefix is not persisted, it only exists in Elixir and is stripped before storing an ID in the database, and re-added on load. This allows us to still use 64-bit integer columns. So in Elixir you could have `"usr_AV7m9gAAAAU"` while in the database you would have `98770186085072901`.

  With `persist_prefix: true`, the full prefixed string is stored. This sacrifices storage efficiency for database-level readability. Your ID would look like `"usr_AV7m9gAAAAU"` in both Elixir and your database. You can only use `db_format: :url64`, `:hex`, `hex32` or `:raw` with `persist_prefix: true`.

  Use `to_format/3` with the `:prefix` option to convert between formats while preserving the prefix.

  ### Encrypted IDs

  If you need unpredictable IDs, you can use encrypted nonce generation. To use encrypted IDs:

  - Set [option](#module-configuration) `nonce_type: :encrypted`
  - Initialize `NoNoncense` with option `base_key: <some 32-byte secret binary>`
  - (optional but recommended) Change the encryption algorithm using option `:cipher64` from the default `:blowfish` to `:speck` (requires optional dependency SpeckEx).

  To learn more about nonce encryption and the available ciphers, see the [NoNoncense docs](https://hexdocs.pm/no_noncense/NoNoncense.html#module-encrypted-nonces).

  ### Masked IDs

  Masked IDs provide a middle ground between plaintext and encrypted IDs. IDs are stored as plaintext in the database (preserving sequential writes and index performance) but encrypted when retrieved by the Ecto type. The application only sees encrypted values while the database maintains optimal performance.

  To use masked IDs:

  - Set [option](#module-configuration) `mask: true`
  - Initialize `NoNoncense` with option `base_key: <some 32-byte secret binary>`
  - Works with nonce types `:counter` and `:sortable` (there's no point in masking an encrypted nonce)

  **Benefits**: Masked IDs have the database performance of plaintext IDs, but the application sees encrypted values. `ORDER BY id` and keyset pagination work transparently (cast decrypts input). No database migration is needed for existing IDs.

  **Trade-offs**: Database and application IDs look completely different (operational friction with SQL/BI tools), slight performance cost for encryption/decryption on every read/write, ordering info can leak into the application when using `ORDER BY id` queries.

  ## Configuration

  ### Options

  The Ecto type takes the following optional parameters:

  #{@options_docs}

  ### Format Caveats

  Some caveats apply to the `:ex_format` option.

  > #### Don't use raw integers with JS clients {: .warning}
  >
  > Encode `:signed` and `:unsigned` as numeric strings (e.g. `"123"`).

  While JSON does not impose a precision limit on numbers, JavaScript can only represent integers up to 2^53. Once IDs encoded as integers are larger than that within 24 days of the epoch passed to NoNoncense.

  > #### Using an integer format as `:ex_format` disables binary parsing, and using a binary format disables numeric string parsing {: .info}
  >
  > When `:ex_format` is set to an integer format (`:signed` or `:unsigned`), parsing numeric strings ("123") will be supported but casting and dumping hex-encoded, url64-encoded, hex32-encoded and raw binaries will be disabled.
  >
  > When `:ex_format` is set to a binary format (`:hex`, `:hex32`, `:url64` or `:raw`), those binary formats will be supported but parsing numeric strings will be disabled.
  >
  > Similarly, `to_format/3` only parses numeric strings as integers with option `parse_int: true`.

  That's because we can't disambiguate some binaries that are valid hex, hex32, url64 and raw binaries and also valid numeric strings. An example is `"12345678901"`:

      # interpret as numeric string, format as signed int
      iex> {:ok, 12345678901}          = Once.to_format("12345678901", :signed, parse_int: true)

      # interpret as url64-encoded binary, format as signed int
      iex> {:ok, -2923406909136636083} = Once.to_format("12345678901", :signed)

  The `:ex_format` setting and `:parse_int` option resolve this ambiguity, at the cost of some flexibility.

  ### Sorting Considerations

  > #### Use format `:unsigned`, `:hex`, `:hex32` or `:raw` to sort IDs chronologically {: .info}
  >
  > If you want to sort IDs chronologically, avoid using `:url64`. Signed integers (`:signed`) can be used in the first ~70 years after epoch. Only `:sortable` IDs can be meaningfully sorted.

  The various formats have different sorting behaviors for the same underlying value, which becomes particularly problematic with values equivalent to negative integers:

      # unsigned ints (0, max-signed, max-unsigned a.k.a. -1)
      iex> unsigned = [0, 9223372036854775807, 18446744073709551615]
      iex> ^unsigned = Enum.sort_by(unsigned, &Once.to_format!(&1, :hex))
      iex> ^unsigned = Enum.sort_by(unsigned, &Once.to_format!(&1, :hex32))
      iex> ^unsigned = Enum.sort_by(unsigned, &Once.to_format!(&1, :raw))
      iex> Enum.sort_by(unsigned, &Once.to_format!(&1, :signed))
      [18446744073709551615, 0, 9223372036854775807]                  # [max-unsigned, 0, max-signed]
      iex> Enum.sort_by(unsigned, &Once.to_format!(&1, :url64))
      [0, 18446744073709551615, 9223372036854775807]                  # [0, max-unsigned, max-signed]

  IDs with `nonce_type: :encrypted` or `:counter` can't be meaningfully sorted. Encrypted IDs are effectively random, and counter IDs generated on different machines can interleave unpredictably when sorted.

  Masked IDs can be meaningfully sorted in a database because they are stored in plaintext, so `ORDER BY id` works as usual. In Elixir they are encrypted, which randomizes their sort order.

  #### Sorting in your database / `ORDER BY id` queries

  - **PostgreSQL**: Only supports signed integers. Using `"ORDER BY id"` will work fine until you reach negative numbers (same as `:signed` in Elixir). The easiest way to deal with this is to ignore the problem and assume that a) your app will not reach that age or b) Postgres will support unsigned ints at some point in the next 70 years. Alternatively, use `db_format: :hex` or `:hex32`. These are fixed-length, human-readable formats that sort correctly throughout the range, at the cost of storage space.
  - **MySQL**: Supports unsigned bigint, so `:unsigned` format works perfectly for sorting.
  """
  require Logger
  use Ecto.ParameterizedType
  alias Once.{Parse, Constants, Init, Prefix, Mask}
  use Constants
  import Prefix, only: [map_prefixed: 4]

  @typedoc @format_docs
  @type format :: :url64 | :raw | :signed | :unsigned | :hex | :hex32

  @typedoc """
  The way in which the underlying 64-bits nonce is generated.

  See `NoNoncense` for details.
  """
  @type nonce_type :: :counter | :encrypted | :sortable

  @typedoc """
  Options to initialize `Once`.

  #{@options_docs}
  """
  @type init_opt ::
          {:no_noncense, module()}
          | {:ex_format, format()}
          | {:db_format, format()}
          | {:nonce_type, nonce_type()}
          | {:mask, boolean()}
          | {:prefix, binary() | nil}
          | {:persist_prefix, boolean()}

  @default_opts %{
    no_noncense: __MODULE__,
    ex_format: :url64,
    db_format: :signed,
    nonce_type: :counter,
    mask: false,
    prefix: nil,
    persist_prefix: false
  }

  #######################
  # Type implementation #
  #######################

  @impl true
  def type(%{db_format: :raw}), do: :binary
  def type(%{db_format: format}) when format in @encoded_formats, do: :string
  def type(%{db_format: format}) when format in @int_formats, do: :integer

  @impl true
  @spec init([init_opt()]) :: map()
  def init(opts \\ []), do: opts |> Enum.into(@default_opts) |> Init.verify_init!()

  @impl true
  def cast(nil, _), do: {:ok, nil}

  def cast(value, %{ex_format: ex_format, prefix: prefix}) do
    do_to_format(value, ex_format, prefix, ex_format in @int_formats)
  end

  @impl true
  def load(nil, _, _), do: {:ok, nil}

  def load(value, _, params = %{prefix: prefix, ex_format: ex_fmt}) do
    to_strip = Prefix.if_persistent(prefix, params)

    if params.mask do
      map_prefixed value, to_strip, prefix do
        Mask.to_masked_format(stripped, params.no_noncense, ex_fmt, false)
      end
    else
      do_to_format(value, ex_fmt, to_strip, prefix, ex_fmt in @int_formats)
    end
  end

  @impl true
  def dump(nil, _, _), do: {:ok, nil}

  def dump(value, _, params = %{prefix: prefix, ex_format: ex_fmt, db_format: db_fmt}) do
    to_put = Prefix.if_persistent(prefix, params)

    if params.mask do
      map_prefixed value, prefix, to_put do
        Mask.to_unmasked_format(stripped, params.no_noncense, db_fmt, ex_fmt in @int_formats)
      end
    else
      do_to_format(value, db_fmt, prefix, to_put, ex_fmt in @int_formats)
    end
  end

  @impl true
  def autogenerate(params) when params.mask do
    NoNoncense.encrypted_nonce(params.no_noncense, 64, params.nonce_type)
    |> Parse.from_raw(params.ex_format)
    |> Prefix.put(params.prefix)
  end

  def autogenerate(params) do
    case params.nonce_type do
      :counter -> NoNoncense.nonce(params.no_noncense, 64)
      :sortable -> NoNoncense.sortable_nonce(params.no_noncense, 64)
      :encrypted -> NoNoncense.encrypted_nonce(params.no_noncense, 64)
    end
    |> Parse.from_raw(params.ex_format)
    |> Prefix.put(params.prefix)
  end

  #####################
  # Mapping functions #
  #####################

  @to_format_opts_docs """
  - `:parse_int` parse numeric strings like `"123"`. Will give unexpected results with all-int hex/url64 inputs.
  - `:prefix` format prefixed IDs (e.g. `"usr_Ad6RZCrAAAM"`) as another prefixed format (e.g. `"usr_01de91642ac00004"`)
  """

  @doc false
  def to_format_opts_docs, do: @to_format_opts_docs

  @typedoc """
  Options for `to_format/3`

  #{@to_format_opts_docs}
  """
  @type to_format_opt :: {:parse_int, boolean()} | {:prefix, binary()}

  @doc """
  Transform the different forms that a `Once` can take to one another.
  The formats can be found in `t:format/0`.

  ## Options

  #{@to_format_opts_docs}

  ## Examples

      iex> id = 18446744073709551615
      iex> {:ok, "__________8" = id}                              = Once.to_format(id, :url64)
      iex> {:ok, <<255, 255, 255, 255, 255, 255, 255, 255>> = id} = Once.to_format(id, :raw)
      iex> {:ok, -1 = id}                                         = Once.to_format(id, :signed)
      iex> {:ok, "ffffffffffffffff" = id}                         = Once.to_format(id, :hex)
      iex> {:ok, "vvvvvvvvvvvvu" = id}                            = Once.to_format(id, :hex32)
      iex> {:ok, 18446744073709551615}                            = Once.to_format(id, :unsigned)

      # numeric strings are supported using :parse_int
      iex> Once.to_format("-2301195303365014983", :unsigned, parse_int: true)
      {:ok, 16145548770344536633}
      iex> Once.to_format("16145548770344536633", :hex, parse_int: true)
      {:ok, "e010831058218a39"}

      # prefixed IDs are supported using :prefix
      # note that integer formats are rendered as numeric strings so require :parse_int
      iex> id = "prfx_18446744073709551615"
      iex> {:ok, "prfx___________8" = id}                    = Once.to_format(id, :url64, prefix: "prfx_", parse_int: true)
      iex> {:ok, <<"prfx_", 18446744073709551615::64>> = id} = Once.to_format(id, :raw, prefix: "prfx_")
      iex> {:ok, "prfx_-1" = id}                             = Once.to_format(id, :signed, prefix: "prfx_")
      iex> {:ok, "prfx_ffffffffffffffff" = id}               = Once.to_format(id, :hex, prefix: "prfx_", parse_int: true)
      iex> {:ok, "prfx_vvvvvvvvvvvvu" = id}                  = Once.to_format(id, :hex32, prefix: "prfx_")
      iex> {:ok, "prfx_18446744073709551615"}                = Once.to_format(id, :unsigned, prefix: "prfx_")
  """
  @spec to_format(binary() | integer(), format(), [to_format_opt()]) ::
          {:ok, binary() | integer()} | :error
  def to_format(value, format, opts \\ []) do
    do_to_format(value, format, opts[:prefix], opts[:parse_int])
  end

  @doc """
  Same as `to_format/3` but raises on error.

      iex> -200
      ...> |> Once.to_format!(:url64)
      ...> |> Once.to_format!(:raw)
      ...> |> Once.to_format!(:hex32)
      ...> |> Once.to_format!(:unsigned)
      ...> |> Once.to_format!(:hex)
      ...> |> Once.to_format!(:signed)
      -200

      iex> Once.to_format!(Integer.pow(2, 64), :unsigned)
      ** (ArgumentError) value could not be parsed: 18446744073709551616

      iex> "usr_AAAAAAAAAAA"
      ...> |> Once.to_format!(:unsigned, prefix: "usr_")
      ...> |> Once.to_format!(:hex, prefix: "usr_", parse_int: true)
      ...> |> Once.to_format!(:signed, prefix: "usr_")
      ...> |> Once.to_format!(:raw, prefix: "usr_", parse_int: true)
      ...> |> Once.to_format!(:hex32, prefix: "usr_")
      ...> |> Once.to_format!(:url64, prefix: "usr_")
      "usr_AAAAAAAAAAA"
  """
  @spec to_format!(binary() | integer(), format(), [to_format_opt()]) :: binary() | integer()
  def to_format!(value, format, opts \\ []) do
    value
    |> do_to_format(format, opts[:prefix], opts[:parse_int])
    |> case do
      {:ok, value} -> value
      _ -> raise ArgumentError, "value could not be parsed: #{inspect(value)}"
    end
  end

  ###########
  # Private #
  ###########

  defp do_to_format(value, format, prefix, parse_int?) do
    do_to_format(value, format, prefix, prefix, parse_int?)
  end

  defp do_to_format(value, format, strip_prefix, put_prefix, parse_int?) do
    map_prefixed value, strip_prefix, put_prefix do
      with {:ok, stripped_parsed} <- Parse.maybe_parse_num_str(stripped, parse_int?),
           {:ok, format_in} <- Parse.identify_format(stripped_parsed) do
        Parse.maybe_convert(stripped_parsed, format_in, format)
      end
    end
  end
end