Packages

needle

0.9.0
0.9.0 0.8.0 0.7.2 0.7.1 0.7.0

Universal foreign keys, virtual schemas, and shared data mixins

HexDocs

Current section

5 Versions

Jump to
Readme
5 Versions
5 Dependencies
Dependants
Files
Activity
Show whitespace

Compare versions

18 files changed
+779 additions
-689 deletions
  @@ -40,9 +40,9 @@ A means of foreign keying many tables in one field. Designed for highly interlin
40 40 > A universal foreign key is actually a hard problem. Many approaches are on offer with a variety of tradeoffs. If plugging into Bonfire's Needle-based core extensions isn't a requirement for you (i.e. you don't need to put things into feeds or use boundaries for access-control) should carefully consider a variety of approaches rather than just blindly adopting the one that fitted our project's needs the best!
41 41
42 42
43 - ## Identifying objects - the ULID type
43 + ## Identifying objects - the UID type
44 44
45 - All referenceable objects in the system have a unique ID (primary key) whose type is the `Needle.ULID`. [ULIDs](https://github.com/ulid/spec) are a lot like a `UUID` in that you can generate unique ones independently of the database. It's also a little different, being made up of two parts:
45 + All referenceable objects in the system have a unique ID (primary key) whose type is the `Needle.UID`. `UUIDv7` and [ULIDs](https://github.com/ulid/spec) are a lot like standard `UUID` in that you can generate unique ones independently of the database. It's also a little different, being made up of two parts:
46 46
47 47 * The current timestamp, to millisecond precision.
48 48 * Strong random padding for uniqueness.
  @@ -51,7 +51,7 @@ This means that it naturally sorts by time to the millisecond (close enough for
51 51
52 52 If you've only worked with integer primary keys before, you are probably used to letting the database dispense an ID for you. With `ULID` (or `UUID`), IDs can be known *before* they are stored, greatly easing the process of storing a graph of data and allowing us to do more of the preparation work outside of a transaction for increased performance.
53 53
54 - In PostgreSQL, we actually store `ULID`s as `UUID` columns, thanks to both being the same size (and the lack of a `ULID` column type shipping with postgresql). You mostly will not notice this because it's handled for you, but there are a few places it can come up:
54 + In PostgreSQL, we actually store `UUIDv7` and `ULID`s as `UUID` columns, thanks to both being the same size (and the lack of specific column types shipping with postgresql). You mostly will not notice this because it's handled for you, but there are a few places it can come up:
55 55
56 56 * Ecto debug and error output may show either binary values or UUID-formatted values.
57 57 * Hand-written SQL may need to convert table IDs to the `UUID` format before use.
  @@ -61,8 +61,8 @@ In PostgreSQL, we actually store `ULID`s as `UUID` columns, thanks to both being
61 61
62 62 The `Needle` system is mostly based around a single table represented by the `Needle.Pointer` schema with the following fields:
63 63
64 - * `id` (ULID) - the database-wide unique id for the object, primary key.
65 - * `table_id` (ULID) - identifies the type of the object, references `Needle.Table`.
64 + * `id` (UID) - the database-wide unique id for the object, primary key.
65 + * `table_id` (UID) - identifies the type of the object, references `Needle.Table`.
66 66 * `deleted_at` (timestamp, default: `null`) - when the object was deleted.
67 67
68 68 Every object that is stored in the system will have a record in this table. It may also have records in other tables (handy for storing more than 3 fields about the object!).
  @@ -100,9 +100,9 @@ end
100 100
101 101 ### Picking a table id
102 102
103 - The first step to declaring a new type is picking a unique table ID in ULID format.
103 + The first step to declaring a new type is picking a unique table ID in UID format.
104 104
105 - You could just generate a random ULID, but since these IDs are special, we tend to assign a synthetic ULID that are readable as words so they stand out in debug output.
105 + You could just generate a random UID, but since these IDs are special, we tend to assign a synthetic UID that are readable as words so they stand out in debug output.
106 106
107 107 For example, the ID for the `Feed` table is: `1TFEEDS0NTHES0V1S0FM0RTA1S`, which can be read as "It feeds on the souls of mortals". Feel free to have a little fun coming up with them, it makes debug output a little more cheery! The rules are:
108 108
  @@ -110,20 +110,20 @@ For example, the ID for the `Feed` table is: `1TFEEDS0NTHES0V1S0FM0RTA1S`, which
110 110 * They must be 26 characters in length.
111 111 * The first character must be a digit in the range 0-7.
112 112
113 - To help you with this, the `Needle.ULID.synthesise!/1` method takes an alphanumeric binary and tries to return you it transliterated into a valid ULID. Example usage:
113 + To help you with this, the `Needle.UID.synthesise!/1` method takes an alphanumeric binary and tries to return you it transliterated into a valid UID. Example usage:
114 114
115 115 ```
116 - iex(1)> Needle.ULID.synthesise!("itfeedsonthesouls")
116 + iex(1)> Needle.UID.synthesise!("itfeedsonthesouls")
117 117
118 118 11:20:28.299 [error] Too short, need 9 chars.
119 119 :ok
120 - iex(2)> Needle.ULID.synthesise!("itfeedsonthesoulsofmortalsandothers")
120 + iex(2)> Needle.UID.synthesise!("itfeedsonthesoulsofmortalsandothers")
121 121
122 122 11:20:31.819 [warn] Too long, chopping off last 9 chars
123 123 "1TFEEDS0NTHES0V1S0FM0RTA1S"
124 - iex(3)> Needle.ULID.synthesise!("itfeedsonthesoulsofmortals")
124 + iex(3)> Needle.UID.synthesise!("itfeedsonthesoulsofmortals")
125 125 "1TFEEDS0NTHES0V1S0FM0RTA1S"
126 - iex(4)> Needle.ULID.synthesise!("gtfeedsonthesoulsofmortals")
126 + iex(4)> Needle.UID.synthesise!("gtfeedsonthesoulsofmortals")
127 127
128 128 11:21:03.268 [warn] First character must be a digit in the range 0-7, replacing with 7
129 129 "7TFEEDS0NTHES0V1S0FM0RTA1S"
  @@ -192,7 +192,7 @@ end
192 192 ```
193 193
194 194
195 - > As you can see, to declare a pointable schema, we start by using `Needle.Pointable`, providing the name of our otp application, the source table's name in the database and our chosen sentinel ULID.
195 + > As you can see, to declare a pointable schema, we start by using `Needle.Pointable`, providing the name of our otp application, the source table's name in the database and our chosen sentinel UID.
196 196
197 197 > We then call `pointable_schema` and define any fields we wish to put directly in the table. For the most part, `pointable_schema` is like Ecto's `schema` macro, except you do not provide the table name and let it handle the primary key.
198 198
  @@ -212,7 +212,7 @@ record or not record information for each mixin. Sample mixins include:
212 212
213 213 In this way, they are reusable across different object types. One mixin may (or may not) be used by any number of objects. This is mostly driven by the type of the object we are storing, but can also be driven by user input.
214 214
215 - Mixins are just tables too! The only requirement is they have a `ULID` primary key which references `Needle.Pointer`. The developer of the mixin is free to put whatever other fields they want in the table, so long as they have that primary-key-as-reference (which will be automatically added for you by the `mixin_schema` macro).
215 + Mixins are just tables too! The only requirement is they have a `UID` primary key which references `Needle.Pointer`. The developer of the mixin is free to put whatever other fields they want in the table, so long as they have that primary-key-as-reference (which will be automatically added for you by the `mixin_schema` macro).
216 216
217 217 Here is a sample mixin definition for a user profile:
218 218
  @@ -386,7 +386,7 @@ end
386 386 ```
387 387
388 388 > As you can see, it's pretty similar to defining a regular migration, except you use `create_pointable_table` and
389 - `drop_pointable_table`. Notice that our sentinel ULID makes an appearance again here. It's *very* important that these match what we declared in the schema.
389 + `drop_pointable_table`. Notice that our sentinel UID makes an appearance again here. It's *very* important that these match what we declared in the schema.
390 390
391 391 ### Mixins
392 392
  @@ -637,10 +637,10 @@ list) can be used to obtain the IDs for a table or tables.
637 637 All solutions to the universal primary key problem have tradeofs. Here
638 638 are what we see as the deficiencies in our approach:
639 639
640 - 1. It forces a ULID on you. This is great for us, but not
641 - everyone. ULID exposes a timestamp with millisecond precision. If
642 - the time of creation of a resource is sensitive information for
643 - your purposes, ULIDs are not going to be suitable for you.
640 + 1. It forces a UUIDv7 or ULID on you. This is great for us, but not
641 + everyone. They both expose a timestamp with millisecond precision.
642 + If the time of creation of a resource is sensitive information for
643 + your purposes, they may not going to be suitable for you.
644 644 2. Ecto has no knowledge of the specialty of `Pointer`,
645 645 e.g. `Repo.preload` does not work and you need to specify a join
646 646 condition to join through a pointer. Use our functions or add extra
  @@ -2,7 +2,7 @@
2 2 [{<<"Hexdocs">>,<<"https://hexdocs.pm/needle">>},
3 3 {<<"Repository">>,<<"https://github.com/bonfire-networks/needle">>}]}.
4 4 {<<"name">>,<<"needle">>}.
5 - {<<"version">>,<<"0.8.0">>}.
5 + {<<"version">>,<<"0.9.0">>}.
6 6 {<<"description">>,
7 7 <<"Universal foreign keys, virtual schemas, and shared data mixins">>}.
8 8 {<<"elixir">>,<<"~> 1.10">>}.
  @@ -12,17 +12,22 @@
12 12 [[{<<"name">>,<<"ecto_sql">>},
13 13 {<<"app">>,<<"ecto_sql">>},
14 14 {<<"optional">>,false},
15 - {<<"requirement">>,<<"~> 3.8">>},
15 + {<<"requirement">>,<<"~> 3.13">>},
16 + {<<"repository">>,<<"hexpm">>}],
17 + [{<<"name">>,<<"typed_ecto_schema">>},
18 + {<<"app">>,<<"typed_ecto_schema">>},
19 + {<<"optional">>,false},
20 + {<<"requirement">>,<<"~> 0.4.1">>},
16 21 {<<"repository">>,<<"hexpm">>}],
17 22 [{<<"name">>,<<"exto">>},
18 23 {<<"app">>,<<"exto">>},
19 24 {<<"optional">>,false},
20 25 {<<"requirement">>,<<"~> 0.4">>},
21 26 {<<"repository">>,<<"hexpm">>}],
22 - [{<<"name">>,<<"needle_ulid">>},
23 - {<<"app">>,<<"needle_ulid">>},
27 + [{<<"name">>,<<"needle_uid">>},
28 + {<<"app">>,<<"needle_uid">>},
24 29 {<<"optional">>,false},
25 - {<<"requirement">>,<<"~> 0.3">>},
30 + {<<"requirement">>,<<"~> 0.0.2">>},
26 31 {<<"repository">>,<<"hexpm">>}],
27 32 [{<<"name">>,<<"telemetry">>},
28 33 {<<"app">>,<<"telemetry">>},
  @@ -33,8 +38,9 @@
33 38 [<<"lib">>,<<"lib/util.ex">>,<<"lib/not_found.ex">>,<<"lib/random.ex">>,
34 39 <<"lib/unpointable.ex">>,<<"lib/needle.ex">>,<<"lib/mixin.ex">>,
35 40 <<"lib/pointers.ex">>,<<"lib/virtual.ex">>,<<"lib/pointer.ex">>,
36 - <<"lib/table.ex">>,<<"lib/migration.ex">>,<<"lib/changesets.ex">>,
37 - <<"lib/form.ex">>,<<"lib/class.ex">>,<<"lib/pointable.ex">>,
41 + <<"lib/table.ex">>,<<"lib/changesets.ex">>,<<"lib/form.ex">>,
42 + <<"lib/class.ex">>,<<"lib/pointable.ex">>,<<"lib/migration">>,
43 + <<"lib/migration/migration.ex">>,<<"lib/migration/indexable.ex">>,
38 44 <<"lib/queries.ex">>,<<"lib/tables.ex">>,<<".formatter.exs">>,<<"mix.exs">>,
39 45 <<"README.md">>,<<"LICENSE">>]}.
40 46 {<<"build_tools">>,[<<"mix">>]}.
  @@ -1,7 +1,7 @@
1 1 defmodule Needle.Changesets do
2 2 require Logger
3 3
4 - alias Needle.{ULID, Util}
4 + alias Needle.{UID, Util}
5 5 alias Ecto.Association.{BelongsTo, Has, NotLoaded}
6 6 alias Ecto.{Changeset, Schema.Metadata}
7 7
  @@ -57,26 +57,26 @@ defmodule Needle.Changesets do
57 57 else
58 58 changeset
59 59 |> Changeset.cast(params, cols)
60 - |> put_new_id()
60 + |> put_new_id(schema)
61 61 end
62 62
63 63 %schema{__meta__: %{state: :built}} ->
64 64 if Util.role(schema) in [:pointable, :virtual] do
65 65 changeset
66 66 |> Changeset.cast(params, cols)
67 - |> put_new_id()
67 + |> put_new_id(schema)
68 68 else
69 69 Changeset.cast(changeset, params, cols)
70 70 end
71 71 end
72 72 end
73 73
74 - def put_new_id(changeset) do
74 + def put_new_id(changeset, schema) do
75 75 if is_binary(get_field(changeset, :id)) do
76 76 changeset
77 77 else
78 78 changeset
79 - |> Changeset.put_change(:id, ULID.generate())
79 + |> Changeset.put_change(:id, UID.generate(schema))
80 80 end
81 81 end
82 82
  @@ -188,7 +188,8 @@ defmodule Needle.Changesets do
188 188 nil ->
189 189 if Util.role(assoc.related) && assoc.related_key == :id do
190 190 # Autogenerate the id for them and copy it back
191 - rel = Map.put(rel, assoc.related_key, ULID.generate())
191 + #  FIXME: what schema to generate for here?
192 + rel = Map.put(rel, assoc.related_key, UID.generate())
192 193
193 194 changeset
194 195 |> Changeset.put_assoc(assoc_key, rel)
  @@ -406,14 +407,6 @@ defmodule Needle.Changesets do
406 407 defp merge_child_errors({_k, %Changeset{} = cs}, acc), do: cs.errors ++ acc
407 408 defp merge_child_errors(_, acc), do: acc
408 409
409 - @doc false
410 - def default_id(changeset) do
411 - case get_field(changeset, :id) do
412 - id when is_binary(id) -> changeset
413 - _ -> Changeset.put_change(changeset, :id, ULID.generate())
414 - end
415 - end
416 -
417 410 @doc false
418 411 def replicate_map_change(changeset, source_key, target_key, xform) do
419 412 case Changeset.fetch_change(changeset, source_key) do
  @@ -4,7 +4,7 @@ defmodule Needle.Form do
4 4 """
5 5
6 6 # alias Ecto.Changeset
7 - alias Needle.{ULID, Util}
7 + alias Needle.{UID, Util}
8 8
9 9 defmacro __using__(options), do: using(__CALLER__.module, options)
10 10
  @@ -35,7 +35,7 @@ defmodule Needle.Form do
35 35 schema_check_attr(Module.get_attribute(module, __MODULE__), module, body)
36 36 end
37 37
38 - @foreign_key_type ULID
38 + @foreign_key_type UID
39 39
40 40 defp schema_check_attr(options, module, body) when is_list(options) do
41 41 otp_app = Util.get_otp_app(options)
  @@ -1,597 +0,0 @@
1 - defmodule Needle.Migration do
2 - @moduledoc "Helpers for writing Pointer-aware migrations."
3 -
4 - import Ecto.Query, only: [from: 2]
5 - import Ecto.Migration
6 - alias Needle.{Pointer, Table, ULID}
7 -
8 - defdelegate init_pointers_ulid_extra(), to: ULID.Migration
9 -
10 - @type pointer_type :: :strong | :weak | :unbreakable
11 -
12 - @doc "Creates a strong, weak or unbreakable pointer depending on `type`."
13 - @spec pointer(type :: pointer_type) :: term
14 - @spec pointer(module :: atom, type :: pointer_type) :: term
15 - def pointer(table \\ Pointer, type)
16 - def pointer(table, :strong), do: strong_pointer(table)
17 - def pointer(table, :weak), do: weak_pointer(table)
18 - def pointer(table, :unbreakable), do: unbreakable_pointer(table)
19 -
20 - @doc """
21 - A reference to a pointer for use with `add/3`. A strong pointer will
22 - be deleted when the thing it's pointing to is deleted.
23 - """
24 - def strong_pointer(table \\ Pointer) do
25 - references(table.__schema__(:source),
26 - type: :uuid,
27 - on_update: :update_all,
28 - on_delete: :delete_all
29 - )
30 - end
31 -
32 - @doc """
33 - A reference to a pointer for use with `add/3`. A weak pointer will
34 - be set null when the thing it's pointing to is deleted.
35 - """
36 - def weak_pointer(table \\ Pointer) do
37 - references(table.__schema__(:source),
38 - type: :uuid,
39 - on_update: :update_all,
40 - on_delete: :nilify_all
41 - )
42 - end
43 -
44 - @doc """
45 - A reference to a pointer for use with `add/3`. An unbreakable
46 - pointer will prevent the thing it's pointing to from being deleted.
47 - """
48 - def unbreakable_pointer(table \\ Pointer) do
49 - references(table.__schema__(:source),
50 - type: :uuid,
51 - on_update: :update_all,
52 - on_delete: :restrict
53 - )
54 - end
55 -
56 - defp table_name(name) when is_atom(name), do: Atom.to_string(name)
57 - defp table_name(name) when is_binary(name), do: name
58 -
59 - @doc false
60 - @spec add_pointer_pk() :: nil
61 - def add_pointer_pk(), do: add(:id, :uuid, primary_key: true)
62 -
63 - @doc false
64 - @spec add_pointer_ref_pk() :: nil
65 - def add_pointer_ref_pk(),
66 - do: add(:id, strong_pointer(Pointer), primary_key: true)
67 -
68 - @doc "Creates a pointable table along with its trigger."
69 - @spec create_pointable_table(schema :: atom, body :: term) :: term
70 - @spec create_pointable_table(
71 - schema :: atom,
72 - opts :: Keyword.t(),
73 - body :: term
74 - ) :: term
75 - @spec create_pointable_table(source :: binary, id :: binary, body :: term) ::
76 - term
77 - @spec create_pointable_table(
78 - source :: binary,
79 - id :: binary,
80 - opts :: Keyword.t(),
81 - body :: term
82 - ) :: term
83 -
84 - # if you're wondering why we expand these, it's so aliases are expanded and turned into atoms
85 - defmacro create_pointable_table(a, b) do
86 - {a, _} = eval_expand(a, __CALLER__)
87 - cpt(a, b)
88 - end
89 -
90 - defmacro create_pointable_table(a, b, c) do
91 - {a, _} = eval_expand(a, __CALLER__)
92 - {b, _} = eval_expand(b, __CALLER__)
93 - cpt(a, b, c)
94 - end
95 -
96 - defmacro create_pointable_table(a, b, c, d) do
97 - {a, _} = eval_expand(a, __CALLER__)
98 - {b, _} = eval_expand(b, __CALLER__)
99 - {c, _} = eval_expand(c, __CALLER__)
100 - cpt(a, b, c, d)
101 - end
102 -
103 - defp cpt(schema, body) when is_atom(schema) do
104 - cpt(schema.__schema__(:source), schema.__pointers__(:table_id), [], body)
105 - end
106 -
107 - defp cpt(schema, opts, body) when is_atom(schema) and is_list(opts) do
108 - cpt(schema.__schema__(:source), schema.__pointers__(:table_id), opts, body)
109 - end
110 -
111 - defp cpt(source, id, body) when is_binary(source) and is_binary(id) do
112 - cpt(source, id, [], body)
113 - end
114 -
115 - defp cpt(source, id, opts, body)
116 - when is_binary(source) and is_binary(id) and is_list(opts) do
117 - Needle.ULID.cast!(id)
118 - opts = [primary_key: false] ++ opts
119 -
120 - quote do
121 - Needle.Migration.insert_table_record(unquote(id), unquote(source))
122 - table = Ecto.Migration.table(unquote(source), unquote(opts))
123 -
124 - Ecto.Migration.create_if_not_exists table do
125 - Needle.Migration.add_pointer_pk()
126 - unquote(body)
127 - end
128 -
129 - Needle.Migration.create_pointable_triggers(unquote(id), unquote(source))
130 - end
131 - end
132 -
133 - def create_virtual(schema) when is_atom(schema) do
134 - create_virtual(schema.__schema__(:source), schema.__pointers__(:table_id))
135 - end
136 -
137 - def create_virtual(source, id) when is_binary(source) and is_binary(id) do
138 - {:ok, _} = Needle.ULID.dump(Needle.ULID.cast!(id))
139 - insert_table_record(id, source)
140 - create_virtual_view(source, id)
141 - create_virtual_trigger(source, id)
142 - end
143 -
144 - @doc "Drops a pointable table"
145 - @spec drop_pointable_table(schema :: atom) :: nil
146 - @spec drop_pointable_table(name :: binary, id :: binary) :: nil
147 - def drop_pointable_table(schema) when is_atom(schema) do
148 - drop_pointable_table(
149 - schema.__schema__(:source),
150 - schema.__pointers__(:table_id)
151 - )
152 - end
153 -
154 - def drop_pointable_table(name, id) when is_binary(name) and is_binary(id) do
155 - Needle.ULID.cast!(id)
156 - drop_pointable_triggers(name)
157 - drop_table(name)
158 - delete_table_record(id)
159 - end
160 -
161 - def drop_virtual(schema) when is_atom(schema) do
162 - drop_virtual(schema.__schema__(:source), schema.__pointers__(:table_id))
163 - end
164 -
165 - def drop_virtual(name, id) when is_binary(name) and is_binary(id) do
166 - Needle.ULID.cast!(id)
167 - drop_virtual_trigger(name)
168 - drop_virtual_view(name)
169 - delete_table_record(id)
170 - end
171 -
172 - def migrate_virtual(schema), do: migrate_virtual(direction(), schema)
173 - def migrate_virtual(:up, schema), do: create_virtual(schema)
174 - def migrate_virtual(:down, schema), do: drop_virtual(schema)
175 -
176 - def migrate_virtual(name, id) when is_binary(name) and is_binary(id),
177 - do: migrate_virtual(direction(), name, id)
178 -
179 - def migrate_virtual(:up, name, id), do: create_virtual(name, id)
180 - def migrate_virtual(:down, name, id), do: drop_virtual(name, id)
181 -
182 - @doc "Creates a mixin table - one with a ULID primary key and no trigger"
183 - @spec create_mixin_table(name :: atom | binary, opts :: list, body :: term) ::
184 - nil
185 - defmacro create_mixin_table(name, opts \\ [], body) do
186 - {name, _} = eval_expand(name, __CALLER__)
187 -
188 - name =
189 - cond do
190 - is_binary(name) ->
191 - name
192 -
193 - is_atom(name) ->
194 - if Code.ensure_loaded?(name),
195 - do: name.__schema__(:source),
196 - else: Atom.to_string(name)
197 - end
198 -
199 - opts = [primary_key: false] ++ List.wrap(opts)
200 -
201 - quote do
202 - name = unquote(name)
203 - table = Ecto.Migration.table(name, unquote(opts))
204 -
205 - Ecto.Migration.create_if_not_exists table do
206 - Needle.Migration.add_pointer_ref_pk()
207 -
208 - unquote(body)
209 - end
210 -
211 - # execute """
212 - # ALTER TABLE #{name} add constraint pointer_is_not_deleted check (is_not_deleted(id))
213 - # """
214 - end
215 - end
216 -
217 - @doc "Drops a mixin table."
218 - @spec drop_mixin_table(name :: atom | binary) :: nil
219 - def drop_mixin_table(name), do: drop_table(name)
220 -
221 - @doc "Creates a random table - one with a UUID v4 primary key."
222 - defmacro create_random_table(name, opts \\ [], body) do
223 - {name, _} = eval_expand(name, __CALLER__)
224 -
225 - name =
226 - cond do
227 - is_binary(name) ->
228 - name
229 -
230 - is_atom(name) ->
231 - if Code.ensure_loaded?(name),
232 - do: name.__schema__(:source),
233 - else: Atom.to_string(name)
234 - end
235 -
236 - opts = [primary_key: false] ++ List.wrap(opts)
237 -
238 - quote do
239 - table = Ecto.Migration.table(unquote(name), unquote(opts))
240 -
241 - Ecto.Migration.create_if_not_exists table do
242 - add(:id, :uuid, primary_key: true)
243 - unquote(body)
244 - end
245 - end
246 - end
247 -
248 - @doc "Drops a random table."
249 - @spec drop_random_table(name :: atom | binary) :: nil
250 - def drop_random_table(name), do: drop_table(name)
251 -
252 - @doc """
253 - When migrating up: initialises the pointers database.
254 - When migrating down: deinitialises the pointers database.
255 - """
256 - @spec init_pointers() :: nil
257 - def init_pointers(), do: init_pointers(direction())
258 -
259 - @doc """
260 - Given `:up`: initialises the pointers database.
261 - Given `:down`: deinitialises the pointers database.
262 - """
263 - @spec init_pointers(direction :: :up | :down) :: nil
264 - def init_pointers(:up) do
265 - pointer = Pointer.__schema__(:source)
266 - table = Table.__schema__(:source)
267 -
268 - create_if_not_exists table(table, primary_key: false) do
269 - add_pointer_pk()
270 - add(:table, :text)
271 - end
272 -
273 - create_if_not_exists table(pointer, primary_key: false) do
274 - add_pointer_pk()
275 -
276 - ref =
277 - references(table,
278 - on_delete: :delete_all,
279 - on_update: :update_all,
280 - type: :uuid
281 - )
282 -
283 - add(:table_id, ref, null: false)
284 - add(:deleted_at, :timestamptz, null: true)
285 - end
286 -
287 - create_if_not_exists(unique_index(table, :table))
288 - create_if_not_exists(index(pointer, :table_id))
289 - flush()
290 - add_is_not_deleted(pointer)
291 - create_pointers_trigger_function()
292 - create_pointable_trigger_function()
293 - create_virtual_trigger_function()
294 - flush()
295 - insert_table_record(Table.__pointers__(:table_id), table)
296 - end
297 -
298 - def init_pointers(:down) do
299 - pointer = Pointer.__schema__(:source)
300 - table = Table.__schema__(:source)
301 - drop_pointers_trigger_function()
302 - drop_pointable_trigger_function()
303 - drop_virtual_trigger_function()
304 - drop_if_exists(index(pointer, :table_id))
305 - drop_if_exists(index(table, :table))
306 - drop_table(pointer)
307 - drop_table(table)
308 - end
309 -
310 - def add_is_not_deleted(table) do
311 - execute("""
312 - create or replace function is_not_deleted(uuid) returns boolean as $$
313 - select exists (
314 - select 1
315 - from #{table}
316 - where id = $1
317 - and deleted_at IS NULL
318 - );
319 - $$ language sql;
320 - """)
321 - end
322 -
323 - @doc false
324 - def create_pointers_trigger_function() do
325 - fun = pointers_trigger_function()
326 -
327 - :ok =
328 - execute("""
329 - create or replace function "#{fun}"() returns trigger as $$
330 - declare query text;
331 - begin
332 - if (TG_OP = 'UPDATE') then
333 - if (OLD.deleted_at is null and NEW.deleted_at is not null) then
334 - select 'delete from "' || TG_ARGV[0] || '" where id = ''' | OLD.id | ''' :: uuid' into query;
335 - execute query;
336 - end if;
337 - return NEW;
338 - elsif (TG_OP = 'DELETE') then
339 - if (OLD.deleted_at is null) then
340 - select 'delete from "' || TG_ARGV[0] || '" where id = ''' | OLD.id | ''' :: uuid' into query;
341 - execute query;
342 - end if;
343 - return OLD;
344 - else
345 - return NEW;
346 - end if;
347 - end;
348 - $$ language plpgsql
349 - """)
350 - end
351 -
352 - @doc false
353 - def create_pointable_trigger_function() do
354 - fun = pointable_trigger_function()
355 - pointer = Pointer.__schema__(:source)
356 - table = Table.__schema__(:source)
357 -
358 - :ok =
359 - execute("""
360 - create or replace function "#{fun}"() returns trigger as $$
361 - declare table_id uuid;
362 - begin
363 - if (TG_OP = 'INSERT') then
364 - if (TG_NARGS = 1) then
365 - select TG_ARGV[0] :: uuid into table_id;
366 - else
367 - select id into table_id from "#{table}"
368 - where "table" = TG_TABLE_NAME;
369 - end if;
370 - if table_id is null then
371 - raise exception 'Table % is not pointable', TG_TABLE_NAME;
372 - end if;
373 - insert into "#{pointer}" (id, table_id) values (NEW.id, table_id)
374 - on conflict do nothing;
375 - return NEW;
376 - elsif (TG_OP = 'DELETE') then
377 - update "#{pointer}" set deleted_at = now() at time zone 'utc' where id = OLD.id;
378 - return OLD;
379 - else
380 - raise exception 'operation: %', TG_OP;
381 - end if;
382 - end;
383 - $$ language plpgsql
384 - """)
385 - end
386 -
387 - def create_virtual_trigger_function() do
388 - fun = virtual_trigger_function()
389 - pointer = Pointer.__schema__(:source)
390 -
391 - :ok =
392 - execute("""
393 - create or replace function "#{fun}"() returns trigger as $$
394 - begin
395 - if (TG_OP = 'INSERT') then
396 - insert into "#{pointer}" (id, table_id) values (NEW.id, TG_ARGV[0] :: uuid)
397 - on conflict do nothing;
398 - return NEW;
399 - elsif (TG_OP = 'DELETE') then
400 - update "#{pointer}" set deleted_at = now() at time zone 'utc' where id = OLD.id;
401 - return OLD;
402 - else
403 - raise exception 'operation: %', TG_OP;
404 - end if;
405 - end;
406 - $$ language plpgsql
407 - """)
408 - end
409 -
410 - @doc false
411 - def drop_pointers_trigger_function() do
412 - execute(~s[drop function if exists "#{pointers_trigger_function()}"() cascade])
413 - end
414 -
415 - @doc false
416 - def drop_pointable_trigger_function() do
417 - execute(~s[drop function if exists "#{pointable_trigger_function()}"() cascade])
418 - end
419 -
420 - @doc false
421 - def drop_virtual_trigger_function() do
422 - execute(~s[drop function if exists "#{virtual_trigger_function()}"() cascade])
423 - end
424 -
425 - @doc false
426 - def create_pointable_triggers(table_id, source) do
427 - {:ok, table_id} = Needle.ULID.dump(Needle.ULID.cast!(table_id))
428 - pointer = Pointer.__schema__(:source)
429 - table_id = Ecto.UUID.cast!(table_id)
430 - # because there is no create trigger if not exists
431 - drop_pointable_triggers(source)
432 -
433 - # after inserting into the pointable, a shadow record should be created in the pointable table
434 - # Note: `pg_trigger_depth()` is used to stop the triggers from issuing a single extra delete
435 - # (that would fail) when called from the other triggers installed by this function
436 - execute("""
437 - create trigger "#{source}_insert_trigger"
438 - before insert on "#{source}"
439 - for each row when (pg_trigger_depth() < 1)
440 - execute procedure "#{pointable_trigger_function()}"('#{table_id}')
441 - """)
442 -
443 - # after deleting from the pointable table, the pointer should be marked deleted
444 - execute("""
445 - create trigger "#{source}_delete_trigger"
446 - after delete on "#{source}"
447 - for each row when (pg_trigger_depth() < 1)
448 - execute procedure "#{pointable_trigger_function()}"()
449 - """)
450 -
451 - # after marking a pointer deleted, the shadow record should be deleted
452 - execute("""
453 - create trigger "#{source}_soft_delete_trigger"
454 - after update on "#{pointer}"
455 - for each row
456 - when (
457 - pg_trigger_depth() < 1
458 - and OLD.deleted_at is null
459 - and NEW.deleted_at is not null
460 - and OLD.table_id = '#{table_id}' :: uuid
461 - )
462 - execute procedure "#{pointers_trigger_function()}"('#{source}')
463 - """)
464 -
465 - # after deleting from pointers, the shadow record should be
466 - # deleted if it wasn't already marked deleted
467 - execute("""
468 - create trigger "#{source}_delete_trigger"
469 - after delete on "#{pointer}"
470 - for each row
471 - when (
472 - pg_trigger_depth() < 1
473 - and OLD.deleted_at is null
474 - and OLD.table_id = '#{table_id}' :: uuid
475 - )
476 - execute procedure "#{pointers_trigger_function()}"('#{source}')
477 - """)
478 - end
479 -
480 - @doc false
481 -
482 - def drop_pointable_triggers(table) do
483 - pointers = Pointer.__schema__(:source)
484 - table = table_name(table)
485 - execute(~s[drop trigger if exists "#{table}_insert_trigger" on "#{table}"])
486 - execute(~s[drop trigger if exists "#{table}_delete_trigger" on "#{table}"])
487 -
488 - execute(~s[drop trigger if exists "#{table}_soft_delete_trigger" on "#{pointers}"])
489 -
490 - execute(~s[drop trigger if exists "#{table}_delete_trigger" on "#{pointers}"])
491 - end
492 -
493 - @doc false
494 - def create_virtual_trigger(table, id) do
495 - {:ok, id} = Needle.ULID.dump(Needle.ULID.cast!(id))
496 - id = Ecto.UUID.cast!(id)
497 - # because there is no create trigger if not exists
498 - drop_virtual_trigger(table)
499 -
500 - execute("""
501 - create trigger "#{table}_insert_delete_trigger"
502 - instead of insert or delete on "#{table}"
503 - for each row
504 - execute procedure "#{virtual_trigger_function()}"('#{id}')
505 - """)
506 - end
507 -
508 - @doc false
509 - def drop_virtual_trigger(table) do
510 - table = table_name(table)
511 -
512 - execute(~s[drop trigger if exists "#{table}_insert_delete_trigger" on "#{table}"])
513 - end
514 -
515 - @doc false
516 - def create_virtual_view(source, id) do
517 - {:ok, id} = Needle.ULID.dump(Needle.ULID.cast!(id))
518 - id = Ecto.UUID.cast!(id)
519 - pointers = Pointer.__schema__(:source)
520 -
521 - execute("""
522 - create or replace view "#{source}" as
523 - select id as id from "#{pointers}"
524 - where table_id = ('#{id}' :: uuid) and deleted_at is null
525 - """)
526 - end
527 -
528 - @doc false
529 - def drop_virtual_view(source),
530 - do: execute(~s[drop view if exists "#{source}"])
531 -
532 - @doc false
533 - def insert_table_record(schema) do
534 - insert_table_record(
535 - schema.__schema__(:source),
536 - schema.__pointers__(:table_id)
537 - )
538 - end
539 -
540 - # Insert a Table record. Not required when using `create_pointable_table`
541 - @doc false
542 - def insert_table_record(id, name) do
543 - {:ok, table_id} = Needle.ULID.dump(Table.__pointers__(:table_id))
544 - {:ok, id} = Needle.ULID.dump(Needle.ULID.cast!(id))
545 - table = table_name(name)
546 - opts = [on_conflict: :nothing]
547 -
548 - repo().insert_all(
549 - Table.__schema__(:source),
550 - [%{id: id, table: table}],
551 - opts
552 - )
553 -
554 - repo().insert_all(
555 - Pointer.__schema__(:source),
556 - [%{id: id, table_id: table_id}],
557 - opts
558 - )
559 - end
560 -
561 - # Delete a Table record. Not required when using `drop_pointable_table`
562 - @doc false
563 - def delete_table_record(id) do
564 - {:ok, id} = Needle.ULID.dump(Needle.ULID.cast!(id))
565 - table = Table.__schema__(:source)
566 - repo().delete_all(from(t in table, where: t.id == ^id))
567 - end
568 -
569 - def drop_table(name) do
570 - name =
571 - cond do
572 - is_binary(name) -> name
573 - is_atom(name) -> name.__schema__(:source)
574 - end
575 -
576 - execute(~s[drop table if exists "#{name}" cascade])
577 - end
578 -
579 - defp eval(quoted, env), do: Code.eval_quoted(quoted, [], env)
580 -
581 - defp eval_expand(quoted, env), do: expand_alias(eval(quoted, env), env)
582 -
583 - defp expand_alias({:__aliases__, _, _} = ast, env), do: Macro.expand(ast, env)
584 - defp expand_alias(ast, _env), do: ast
585 -
586 - defp pointers_trigger_function(),
587 - do: config(:pointers_trigger_function, "pointers_pointers_trigger")
588 -
589 - defp pointable_trigger_function(),
590 - do: config(:pointable_trigger_function, "pointers_pointable_trigger")
591 -
592 - defp virtual_trigger_function(),
593 - do: config(:virtual_trigger_function, "pointers_virtual_trigger")
594 -
595 - defp config(key, default),
596 - do: Keyword.get(Application.get_env(:needle, __MODULE__, []), key, default)
597 - end
Loading more files…