snowflakex

# Snowflakex

[![CI](https://github.com/meetnow/snowflakex/actions/workflows/ci.yml/badge.svg)](https://github.com/meetnow/snowflakex/actions/workflows/ci.yml)
[![Hex.pm Version](https://img.shields.io/hexpm/v/snowflakex.svg)](https://hex.pm/packages/snowflakex)
[![Hex Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/snowflakex/)
[![Hex.pm Downloads](https://img.shields.io/hexpm/dt/snowflakex.svg)](https://hex.pm/packages/snowflakex)
[![License](https://img.shields.io/hexpm/l/snowflakex.svg)](https://github.com/meetnow/snowflakex/blob/master/LICENSE)

Snowflakex (pronounced "snowflakes") is a service for generating unique ID
numbers at high scale with some simple guarantees.

It is directly modeled after
[Twitter's Snowflake](https://github.com/twitter-archive/snowflake/tree/snowflake-2010).

## Installation

Add `snowflakex` to your list of dependencies in `mix.exs`:

```elixir
def deps do
  [
    {:snowflakex, "~> 1.2"}
  ]
end
```

Then configure a machine id between `0` and `1023` that is unique for every
node generating IDs, for example in `config/runtime.exs`:

```elixir
config :snowflakex, machine_id: 123
```

The application refuses to start without a valid machine id.

## Usage

```elixir
iex> Snowflakex.new()
{:ok, 1234567890123456789}
```

`Snowflakex.new/0` returns either `{:ok, snowflake}` or `{:error, message}`,
where the latter only occurs when the system clock moved backwards.

```elixir
iex> Snowflakex.new!()
1234567890123456789
```

`Snowflakex.new!/0` returns the snowflake directly or raises a
`Snowflakex.ClockError` holding the error message and the remaining time in
milliseconds.

```elixir
iex> Snowflakex.decompose(4_194_476_039)
%{timestamp: 1_480_927_902_749, machine_id: 42, sequence: 7}
```

`Snowflakex.decompose/1` splits a snowflake into its components. The
timestamp is given in milliseconds since the Unix epoch.

## Requirements

### Performance

* Minimum 10k IDs per second per node
* Response rate <= 2ms

### Uncoordinated

For high availability within and across server application instances, nodes
generating IDs should not have to coordinate with each other.

### (Roughly) Time Ordered

We can guarantee that the ID numbers will be
k-sorted (see [here](https://dl.acm.org/citation.cfm?id=70413.70419) and
[here](https://dl.acm.org/citation.cfm?id=110778.110783)) within a reasonable
bound (we're promising 1s, but shooting for 10's of ms).

### Directly Sortable

The IDs should be sortable without loading the full objects that they represent.
This sorting should be the above ordering.

### Compact

There are many otherwise reasonable solutions to this problem that require
128-bit numbers. To accommodate most systems, we keep our IDs under 64 bits.

### Highly Available

The ID generation scheme should be at least as available as the related services.

## Solution

The ID numbers are 64-bit integers composed of:

| Bits | Field          | Description                                                                 |
| ---: | -------------- | --------------------------------------------------------------------------- |
|    1 | Sign           | Unused, always `0`                                                          |
|   41 | Timestamp      | Milliseconds since the custom epoch `2016-12-05T08:51:41.749Z`, lasting 69 years |
|   10 | Machine id     | Configured per node, allowing up to 1024 machines                           |
|   12 | Sequence       | Rolls over every 4096 IDs per machine, with protection against rolling over within the same millisecond |

## System Clock Dependency

You should use NTP to keep your system clock accurate. Snowflakex protects from
non-monotonic clocks, i.e. clocks that run backwards. If your clock is running
fast and NTP tells it to repeat a few milliseconds, Snowflakex will refuse to
generate IDs until a time that is after the last time we generated an ID. Even
better, run in a mode where NTP won't move the clock backwards. See
[Dovecot's notes on time synchronization](https://doc.dovecot.org/main/core/admin/errors.html#time-moved-backwards-error)
for tips on how to do this.

## License

Copyright (c) 2016 MeetNow! GmbH

Licensed under the [Apache License, Version 2.0](LICENSE).