Packages
phoenix_kit
2.41.2
2.45.0
2.44.0
2.43.1
2.43.0
2.42.1
2.42.0
2.41.6
2.41.4
2.41.3
2.41.2
2.41.1
2.41.0
2.40.1
2.40.0
2.39.0
2.38.1
2.38.0
2.37.5
2.37.4
2.37.3
2.37.2
2.37.1
2.37.0
2.36.1
2.36.0
2.35.0
2.34.0
2.33.0
2.32.1
2.32.0
2.31.1
2.31.0
2.30.0
2.29.1
2.29.0
2.28.2
2.28.1
2.28.0
2.27.2
2.27.1
2.27.0
2.26.1
2.26.0
2.25.0
2.24.0
2.23.3
2.23.2
2.23.1
2.23.0
2.22.24
2.22.23
2.22.22
2.22.21
2.22.20
2.22.19
2.22.18
2.22.17
2.22.16
2.22.15
2.22.14
2.22.13
2.22.12
2.22.11
2.22.10
2.22.9
2.22.8
2.22.7
2.22.6
2.22.5
2.22.4
2.22.3
2.22.2
2.22.1
2.22.0
2.21.5
2.21.4
2.21.3
2.21.2
2.21.1
2.21.0
2.20.0
2.19.0
2.18.1
2.18.0
2.17.0
2.16.0
2.15.1
2.15.0
2.14.2
2.14.1
2.14.0
2.13.19
2.13.18
2.13.17
2.13.16
2.13.15
2.13.13
2.13.12
2.13.11
2.13.10
2.13.9
2.13.8
2.13.7
2.13.6
2.13.5
2.13.4
2.13.3
2.13.2
2.13.1
2.13.0
2.12.1
2.12.0
2.11.0
2.10.0
2.9.0
2.8.1
2.8.0
2.7.0
2.6.0
2.5.0
2.4.0
2.3.0
2.2.0
2.1.0
2.0.1
2.0.0
1.7.236
1.7.235
1.7.234
1.7.233
1.7.232
1.7.231
1.7.230
1.7.229
1.7.228
1.7.227
1.7.226
1.7.225
1.7.224
1.7.223
1.7.222
1.7.221
1.7.220
1.7.219
1.7.218
1.7.217
1.7.216
1.7.215
1.7.214
1.7.213
1.7.212
1.7.211
1.7.210
1.7.209
1.7.208
1.7.207
1.7.206
1.7.205
1.7.204
1.7.203
1.7.202
1.7.201
1.7.200
1.7.199
1.7.198
1.7.197
1.7.196
1.7.194
1.7.193
1.7.192
1.7.191
1.7.190
1.7.189
1.7.187
1.7.186
1.7.185
1.7.184
1.7.183
1.7.182
1.7.181
1.7.180
1.7.179
1.7.178
1.7.177
1.7.176
1.7.175
1.7.174
1.7.173
1.7.172
1.7.171
1.7.170
1.7.169
1.7.168
1.7.167
1.7.166
1.7.165
1.7.164
1.7.162
1.7.161
1.7.160
1.7.159
1.7.157
1.7.156
1.7.155
1.7.154
1.7.153
1.7.152
1.7.151
1.7.150
1.7.149
1.7.146
1.7.145
1.7.144
1.7.143
1.7.138
1.7.133
1.7.132
1.7.131
1.7.130
1.7.128
1.7.126
1.7.125
1.7.121
1.7.120
1.7.119
1.7.118
1.7.117
1.7.116
1.7.115
1.7.114
1.7.113
1.7.112
1.7.111
1.7.110
1.7.109
1.7.108
1.7.107
1.7.106
1.7.105
1.7.104
1.7.103
1.7.102
1.7.101
1.7.100
1.7.99
1.7.98
1.7.97
1.7.96
1.7.95
1.7.94
1.7.93
1.7.92
1.7.91
1.7.90
1.7.89
1.7.88
1.7.87
1.7.86
1.7.85
1.7.84
1.7.83
1.7.82
1.7.81
1.7.80
1.7.79
1.7.78
1.7.77
1.7.76
1.7.75
1.7.74
1.7.71
1.7.70
1.7.69
1.7.66
1.7.65
1.7.64
1.7.63
1.7.62
1.7.61
1.7.59
1.7.58
1.7.57
1.7.56
1.7.55
1.7.54
1.7.53
1.7.52
1.7.51
1.7.49
1.7.44
1.7.43
1.7.42
1.7.41
1.7.39
1.7.38
1.7.37
1.7.36
1.7.34
1.7.33
1.7.31
1.7.30
1.7.29
1.7.28
1.7.27
1.7.26
1.7.25
1.7.24
1.7.23
1.7.22
1.7.21
1.7.20
1.7.19
1.7.18
1.7.17
1.7.16
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.20
1.6.19
1.6.18
1.6.17
1.6.16
1.6.15
1.6.14
1.6.13
1.6.12
1.6.11
1.6.10
1.6.9
1.6.8
1.6.7
1.6.6
1.6.5
1.6.4
1.6.3
1.5.2
1.5.1
1.5.0
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.2
1.3.1
1.3.0
1.2.10
1.2.9
1.2.8
1.2.7
1.2.5
1.2.4
1.2.2
1.2.1
1.2.0
1.1.0
1.0.0
A foundation for building Elixir Phoenix apps — SaaS, social networks, ERP systems, marketplaces, and more
Current section
Files
Jump to
Current section
Files
lib/modules/storage/resource_folders.ex
defmodule PhoenixKit.Modules.Storage.ResourceFolders do
@moduledoc """
The folder convention for modules that keep one media folder per record
(a catalogue item, a location, a CRM contact, a staff person, a
project): the host hooks, the lookup order, race-safe find-or-create,
and the rules for putting files in and taking them out — written once,
so every module places, finds and lists a record's files the same way.
## Host hooks
config :my_module, :attachments_parent_folder, {MyApp.Media, :parent_for}
config :my_module, :attachments_folder_name, {MyApp.Media, :folder_name}
The parent hook is called as `fun(kind, actor_uuid, subject)`, or as
`fun(kind, actor_uuid)` when the host exports only that arity, and
answers `{:ok, parent_folder_uuid}`, or `nil` / `{:ok, nil}` for the
media root. The name hook is called as `fun(subject, actor_uuid)` and
answers `{:ok, name}`, or `nil` / `{:ok, nil}` for the module's
deterministic name.
`parent_hook/4` and `name_hook/3` tell a hook that is not configured
(`:unconfigured`) from one that FAILED (`{:error, reason}`: it raised,
threw, exited, is not exported, or answered anything else — a parent
that is not a uuid included). A media reorganizer must never read a
failure as "the root". `parent_uuid/4` and `host_name/3` are the forms
for everything else: a failure is logged and falls back to the root /
the deterministic name, so an upload never fails on a host hook. Logs
name a failure's shape only (`describe_failure/1`) — an exit reason or
an exception message can carry the hook's arguments.
## Lookup order
`resolve/1` finds a record's folder by the folder uuid the record
stores, then by the host's name under the parent, then by the
deterministic name under the parent, then at the root — and, for a name
that embeds the record's uuid, anywhere. Only live folders count:
`(name, parent_uuid)` is unique among live folders only, so a trashed
twin can sit beside the live one, and a record whose folder was trashed
gets a new one rather than uploads nobody can see.
A host name carries no uuid, so another record's folder can have it —
`claimed?/3` asks whether one points at it, and `ensure/4` falls back
to the uuid-bearing name when the host's is taken.
Moving existing folders is the media reorganizer's business; nothing
here moves a folder or creates one for people's own use.
## Files
A file is IN a folder when the folder is its home (`file.folder_uuid`)
or a `FolderLink` puts it there, and it is live (not trashed, not
system-managed). `attach/2` and `detach/2` follow core's rules
(`Storage.attach_file_to_folder/2`, `Storage.remove_file_from_folder/2`):
a file homed elsewhere is linked, never moved; a removed file is
unlinked, re-homed or trashed, never deleted.
"""
import Ecto.Query
require Logger
alias PhoenixKit.Modules.Storage
alias PhoenixKit.Modules.Storage.File, as: StorageFile
alias PhoenixKit.Modules.Storage.{Folder, FolderLink}
@typedoc "A hook's answer: a value, not configured, or failed."
@type hook_answer(value) :: {:ok, value} | :unconfigured | {:error, term()}
@typedoc """
Where a record keeps its folder uuid: a key of one of its JSONB map
fields (`{:data, "files_folder_uuid"}`), or a column (`{:column, :folder_uuid}`).
"""
@type pointer :: {:column, atom()} | {atom(), String.t()}
@list_limit 200
# ── Host hooks ──────────────────────────────────────────────────────
@doc """
Asks `app`'s `:attachments_parent_folder` hook where a `kind` folder
for `subject` belongs: `{:ok, uuid}` (cast to its canonical form) or
`{:ok, nil}` for the root; `:unconfigured`; or `{:error, reason}`.
"""
@spec parent_hook(atom(), atom(), String.t() | nil, term()) :: hook_answer(String.t() | nil)
def parent_hook(app, kind, actor_uuid, subject) do
with {:ok, {mod, fun}} <- configured(app, :attachments_parent_folder) do
guarded(fn ->
cond do
exported?(mod, fun, 3) -> {:answer, apply(mod, fun, [kind, actor_uuid, subject])}
exported?(mod, fun, 2) -> {:answer, apply(mod, fun, [kind, actor_uuid])}
true -> {:error, {:not_exported, {mod, fun}}}
end
end)
|> parent_answer()
end
end
@doc """
The parent folder for a new `kind` folder: `parent_hook/4`'s uuid, with
a failure logged and the media root (`nil`) in its place.
"""
@spec parent_uuid(atom(), atom(), String.t() | nil, term()) :: String.t() | nil
def parent_uuid(app, kind, actor_uuid, subject) do
case parent_hook(app, kind, actor_uuid, subject) do
{:ok, uuid} ->
uuid
:unconfigured ->
nil
{:error, reason} ->
Logger.warning(
"[#{app}] attachments_parent_folder hook failed for #{inspect(kind)}: " <>
describe_failure(reason)
)
nil
end
end
@doc """
Asks `app`'s `:attachments_folder_name` hook what to call `subject`'s
folder: `{:ok, name}` (trimmed), `{:ok, nil}` for the deterministic
name, `:unconfigured`, or `{:error, reason}`.
"""
@spec name_hook(atom(), term(), String.t() | nil) :: hook_answer(String.t() | nil)
def name_hook(app, subject, actor_uuid) do
with {:ok, {mod, fun}} <- configured(app, :attachments_folder_name) do
guarded(fn ->
if exported?(mod, fun, 2),
do: {:answer, apply(mod, fun, [subject, actor_uuid])},
else: {:error, {:not_exported, {mod, fun}}}
end)
|> name_answer()
end
end
@doc """
The host's name for `subject`'s folder: `name_hook/3`'s name, with a
failure logged and `nil` (use the deterministic name) in its place.
"""
@spec host_name(atom(), term(), String.t() | nil) :: String.t() | nil
def host_name(app, subject, actor_uuid) do
case name_hook(app, subject, actor_uuid) do
{:ok, name} ->
name
:unconfigured ->
nil
{:error, reason} ->
Logger.warning(
"[#{app}] attachments_folder_name hook failed: #{describe_failure(reason)}"
)
nil
end
end
@doc "Whether `app` sets `key` (the parent hook by default) at all."
@spec hook_configured?(atom(), atom()) :: boolean()
def hook_configured?(app, key \\ :attachments_parent_folder),
do: Application.get_env(app, key) != nil
@doc """
A one-line description of a hook failure, or of any error reason, that
leaves out its payload: an exception is named but not rendered (its
message interpolates values), an exit reason by its shape.
"""
@spec describe_failure(term()) :: String.t()
def describe_failure({:bad_config, _value}), do: "the config is not a {module, function} pair"
def describe_failure({:not_exported, {mod, fun}}), do: "#{inspect(mod)}.#{fun} is not exported"
def describe_failure({:bad_answer, answer}), do: "the hook answered " <> answer_shape(answer)
def describe_failure({:exit, reason}), do: "exited: " <> exit_shape(reason)
def describe_failure({:throw, _value}), do: "threw"
def describe_failure(%{__exception__: true, __struct__: mod}), do: "raised #{inspect(mod)}"
def describe_failure(reason) when is_atom(reason), do: inspect(reason)
def describe_failure(%Ecto.Changeset{errors: errors}),
do: "invalid #{inspect(Keyword.keys(errors))}"
def describe_failure(reason) when is_tuple(reason), do: shape(reason)
def describe_failure(_reason), do: "failed"
# Atoms are code and safe to show; strings, maps and structs are data.
defp answer_shape(answer) when is_atom(answer), do: inspect(answer)
defp answer_shape({:ok, value}) when is_binary(value), do: "{:ok, a string that is not a uuid}"
defp answer_shape({:ok, value}), do: "{:ok, #{answer_shape(value)}}"
defp answer_shape({tag, _value}) when is_atom(tag), do: "{#{inspect(tag)}, …}"
defp answer_shape(value) when is_binary(value), do: "a string"
defp answer_shape(%{__struct__: mod}), do: "a #{inspect(mod)}"
defp answer_shape(value) when is_map(value), do: "a map"
defp answer_shape(value) when is_list(value), do: "a list"
defp answer_shape(value) when is_tuple(value), do: "a #{tuple_size(value)}-tuple"
defp answer_shape(_value), do: "something else"
defp exit_shape({:timeout, {GenServer, :call, _args}}), do: "GenServer.call timeout"
defp exit_shape({:noproc, {GenServer, :call, _args}}), do: "GenServer.call to a dead process"
defp exit_shape({reason, {GenServer, :call, _args}}) when is_atom(reason),
do: "GenServer.call #{inspect(reason)}"
defp exit_shape(reason) when is_atom(reason), do: inspect(reason)
defp exit_shape(%{__struct__: mod}), do: inspect(mod)
defp exit_shape(reason) when is_tuple(reason), do: shape(reason)
defp exit_shape(_reason), do: "(details omitted)"
defp shape(tuple) when tuple_size(tuple) > 0 and is_atom(elem(tuple, 0)),
do: "#{inspect(elem(tuple, 0))} (details omitted)"
defp shape(_tuple), do: "(details omitted)"
defp configured(app, key) do
case Application.get_env(app, key) do
nil -> :unconfigured
{mod, fun} when is_atom(mod) and is_atom(fun) -> {:ok, {mod, fun}}
other -> {:error, {:bad_config, other}}
end
end
defp exported?(mod, fun, arity),
do: Code.ensure_loaded?(mod) and function_exported?(mod, fun, arity)
defp guarded(fun) do
fun.()
rescue
exception -> {:error, exception}
catch
kind, reason -> {:error, {kind, reason}}
end
defp parent_answer({:answer, nil}), do: {:ok, nil}
defp parent_answer({:answer, {:ok, nil}}), do: {:ok, nil}
defp parent_answer({:answer, {:ok, uuid} = answer}) when is_binary(uuid) do
case cast(uuid) do
nil -> {:error, {:bad_answer, answer}}
uuid -> {:ok, uuid}
end
end
defp parent_answer({:answer, {:error, _reason} = error}), do: error
defp parent_answer({:answer, other}), do: {:error, {:bad_answer, other}}
defp parent_answer({:error, _reason} = error), do: error
defp name_answer({:answer, nil}), do: {:ok, nil}
defp name_answer({:answer, {:ok, nil}}), do: {:ok, nil}
defp name_answer({:answer, {:ok, name} = answer}) when is_binary(name) do
case String.trim(name) do
"" -> {:error, {:bad_answer, answer}}
name -> {:ok, name}
end
end
defp name_answer({:answer, {:error, _reason} = error}), do: error
defp name_answer({:answer, other}), do: {:error, {:bad_answer, other}}
defp name_answer({:error, _reason} = error), do: error
# ── Finding folders ─────────────────────────────────────────────────
@doc "The live folder `uuid` points at, or `nil`; anything not a uuid points nowhere."
@spec live_folder(term()) :: Folder.t() | nil
def live_folder(uuid) do
case cast(uuid) do
nil -> nil
uuid -> repo().one(from(f in Folder, where: f.uuid == ^uuid and is_nil(f.trashed_at)))
end
end
defp live_folder_locked(uuid) do
case cast(uuid) do
nil ->
nil
uuid ->
repo().one(
from(f in Folder, where: f.uuid == ^uuid and is_nil(f.trashed_at), lock: "FOR SHARE")
)
end
end
@doc "The live folder named `name` directly under `parent_uuid` (`nil` = the root)."
@spec find_under(String.t(), String.t() | nil) :: Folder.t() | nil
def find_under(name, parent_uuid) when is_binary(name) do
from(f in Folder, where: f.name == ^name and is_nil(f.trashed_at), limit: 1)
|> directly_under(parent_uuid)
|> repo().one()
end
@doc """
The live folder named `name`: under `parent_uuid` first, then at the
root, then — with `anywhere: true`, for a name that embeds the record's
uuid so that every folder carrying it is that record's — under any
other parent. Oldest first within a place, so the answer never depends
on who asks.
"""
@spec find_named(String.t(), String.t() | nil, keyword()) :: Folder.t() | nil
def find_named(name, parent_uuid, opts \\ []) when is_binary(name) do
[name]
|> find_named_all(parent_uuid, opts)
|> Map.get(name)
end
@doc """
`find_named/3` for many names in one query: `%{name => folder}`, a
name with no live folder left out.
"""
@spec find_named_all([String.t()], String.t() | nil, keyword()) :: %{String.t() => Folder.t()}
def find_named_all(names, parent_uuid, opts \\ [])
def find_named_all([], _parent_uuid, _opts), do: %{}
def find_named_all(names, parent_uuid, opts) when is_list(names) do
parent_uuid = cast(parent_uuid)
from(f in Folder,
where: f.name in ^Enum.uniq(names) and is_nil(f.trashed_at),
order_by: [asc: f.inserted_at, asc: f.uuid]
)
|> near(parent_uuid, Keyword.get(opts, :anywhere, false))
|> repo().all()
|> Enum.group_by(& &1.name)
|> Map.new(fn {name, folders} ->
{name, Enum.min_by(folders, &place_rank(&1, parent_uuid))}
end)
end
defp near(query, _parent_uuid, true), do: query
defp near(query, nil, false), do: where(query, [f], is_nil(f.parent_uuid))
defp near(query, parent_uuid, false),
do: where(query, [f], is_nil(f.parent_uuid) or f.parent_uuid == ^parent_uuid)
# `Enum.min_by/2` keeps the first of equal ranks, and the rows come
# oldest first.
defp place_rank(%Folder{parent_uuid: parent}, parent), do: 0
defp place_rank(%Folder{parent_uuid: nil}, _parent), do: 1
defp place_rank(%Folder{}, _parent), do: 2
@doc """
A record's folder, in the convention's order, or `nil` when it has none
yet:
1. `:pointer` — the folder uuid the record stores, if that folder is live;
2. `:host_name` directly under `:parent`, unless `:claimed?` (a
`fun(folder) -> boolean`) says another record owns that folder;
3. `:name`, the deterministic name, by `find_named/3` — pass
`anywhere: true` when it embeds the record's uuid.
Give an unsaved record no names: a folder found by name is one some
saved record already owns.
"""
@spec resolve(keyword()) :: Folder.t() | nil
def resolve(opts) do
parent = cast(Keyword.get(opts, :parent))
name = Keyword.get(opts, :name)
claimed? = Keyword.get(opts, :claimed?, fn _folder -> false end)
live_folder(Keyword.get(opts, :pointer)) ||
host_named(Keyword.get(opts, :host_name), name, parent, claimed?) ||
(is_binary(name) && find_named(name, parent, anywhere: Keyword.get(opts, :anywhere, false))) ||
nil
end
defp host_named(host_name, name, parent, claimed?)
when is_binary(host_name) and host_name != name do
case find_under(host_name, parent) do
nil -> nil
folder -> if claimed?.(folder), do: nil, else: folder
end
end
defp host_named(_host_name, _name, _parent, _claimed?), do: nil
@doc """
Whether a record other than `own_uuid` points at `folder_uuid` through
one of `pointers` (`[{schema, pointer}]`, see `t:pointer/0`) — the
check that keeps a record from adopting another's host-named folder.
Fails closed: an error answers `true`.
"""
@spec claimed?(String.t(), String.t() | nil, [{module(), pointer()}]) :: boolean()
def claimed?(folder_uuid, own_uuid, pointers) when is_binary(folder_uuid) do
Enum.any?(pointers, fn {schema, pointer} ->
schema
|> pointing_at(pointer, folder_uuid)
|> except_uuid(cast(own_uuid))
|> repo().exists?()
end)
rescue
error ->
Logger.warning("Folder claim check failed for #{folder_uuid}: #{describe_failure(error)}")
true
catch
:exit, reason ->
Logger.warning("Folder claim check failed for #{folder_uuid}: #{exit_shape(reason)}")
true
end
defp pointing_at(schema, {:column, column}, folder_uuid),
do: from(r in schema, where: field(r, ^column) == ^folder_uuid)
# Compared without case: the pointer is written lower-case, but a legacy
# or host-written one spelled otherwise names the same folder and claims
# it just as much.
defp pointing_at(schema, {map_field, key}, folder_uuid) when is_binary(key),
do:
from(r in schema,
where:
fragment("lower(?->>?)", field(r, ^map_field), ^key) == ^String.downcase(folder_uuid)
)
defp except_uuid(query, nil), do: query
defp except_uuid(query, uuid), do: where(query, [r], r.uuid != ^uuid)
# ── Creating folders ────────────────────────────────────────────────
@doc """
The folder named `name` under `parent_uuid`, created when missing —
race-safe: a create that loses to a concurrent one takes the winner.
Never raises.
## Options
* `:lookup` — a 0-arity function finding the existing folder
(default: the live `name` directly under `parent_uuid`). It runs
before creating and again after a create is refused, so a caller
that resolves through `resolve/1` passes that here.
* `:fallback_name` — when `name` is taken under this parent by a
folder `:lookup` does not adopt (another record's), or core refuses
it, create this one instead: the uuid-bearing deterministic name,
which cannot collide. A name known to be taken is not tried at all.
* `:claim` — a 1-arity function recording the folder as the record's
(writing its pointer, `write_pointer/4`), answering `:ok`,
`{:ok, _}` or `{:error, _}`. With it, the lookup, the create and
the claim run in one transaction under a lock on `{parent, name}`:
a folder found by a name that carries no uuid is claimed before
anyone else can look for it, so two same-named records resolving
at once never share one folder. A claim that fails rolls the create
back.
"""
@spec ensure(String.t(), String.t() | nil, String.t() | nil, keyword()) ::
{:ok, Folder.t()} | {:error, term()}
def ensure(name, parent_uuid, actor_uuid, opts \\ []) when is_binary(name) do
case Keyword.get(opts, :claim) do
nil ->
safely("ensure folder", fn -> find_or_create(name, parent_uuid, actor_uuid, opts) end)
claim when is_function(claim, 1) ->
safely("ensure folder", fn -> claimed(name, parent_uuid, actor_uuid, opts, claim) end)
end
end
defp claimed(name, parent_uuid, actor_uuid, opts, claim) do
repo().transaction(fn ->
lock_name(parent_uuid, name)
with {:ok, folder} <- find_or_create(name, parent_uuid, actor_uuid, opts),
:ok <- claim_result(claim.(folder)) do
folder
else
{:error, reason} -> repo().rollback(reason)
end
end)
end
defp claim_result(:ok), do: :ok
defp claim_result({:ok, _}), do: :ok
defp claim_result({:error, _reason} = error), do: error
defp claim_result(other), do: {:error, {:bad_claim, other}}
# A transaction-scoped advisory lock on the name under the parent, so
# every resolver of one host name queues behind the one claiming it.
@doc false
# Also taken by the reorganizer's pointer back-fill, before it locks the
# record: every claim of a host-named folder queues on it.
def lock_name(parent_uuid, name) do
repo().query!("SELECT pg_advisory_xact_lock(hashtext($1))", [name_lock_key(parent_uuid, name)])
end
@doc false
# The key `lock_name/2` locks. The parent is cast, so two spellings of
# one uuid queue on the same lock rather than passing each other.
@spec name_lock_key(String.t() | nil, String.t()) :: String.t()
def name_lock_key(parent_uuid, name),
do: "pk_resource_folder:#{cast(parent_uuid) || "root"}:#{name}"
defp find_or_create(name, parent_uuid, actor_uuid, opts) do
lookup = Keyword.get(opts, :lookup, fn -> find_under(name, parent_uuid) end)
fallback = Keyword.get(opts, :fallback_name)
case lookup.() do
%Folder{} = folder ->
{:ok, folder}
nil ->
if fallback?(name, fallback) and find_under(name, parent_uuid),
do: find_or_create(fallback, parent_uuid, actor_uuid, []),
else: create(name, parent_uuid, actor_uuid, lookup, fallback)
end
end
defp create(name, parent_uuid, actor_uuid, lookup, fallback) do
case insert_folder(%{name: name, parent_uuid: parent_uuid, user_uuid: actor_uuid}) do
{:ok, folder} ->
{:ok, folder}
{:error, %Ecto.Changeset{} = changeset} ->
case lookup.() do
%Folder{} = folder ->
{:ok, folder}
nil ->
if name_refused?(changeset) and fallback?(name, fallback),
do: find_or_create(fallback, parent_uuid, actor_uuid, []),
else: {:error, changeset}
end
end
end
# Inside a transaction a refused insert would abort it, so the insert
# gets a savepoint of its own there.
defp insert_folder(attrs) do
opts = if repo().in_transaction?(), do: [mode: :savepoint], else: []
%Folder{} |> Folder.changeset(attrs) |> repo().insert(opts)
end
@doc """
Points record `uuid` of `schema` at `folder_uuid` through `pointer`
(`t:pointer/0`) — one UPDATE of that key or column only, no changeset,
no callbacks, the rest of the row untouched; `nil` removes the pointer.
`{:error, :not_found}` when no such record exists.
"""
@spec write_pointer(module(), String.t(), pointer(), String.t() | nil) ::
:ok | {:error, :not_found}
def write_pointer(schema, uuid, pointer, folder_uuid) do
case repo().update_all(
from(r in schema,
where: r.uuid == ^uuid,
update: ^pointer_update(pointer, folder_uuid)
),
[]
) do
{0, _} -> {:error, :not_found}
{_n, _} -> :ok
end
end
@doc """
Removes record `uuid`'s pointer only while it still points at `value` —
for clearing a pointer to something just removed without wiping one that
another session has pointed elsewhere since. `:ok` either way.
"""
@spec clear_pointer_if(module(), String.t(), pointer(), String.t()) :: :ok
def clear_pointer_if(schema, uuid, pointer, value) when is_binary(value) do
repo().update_all(
from(r in schema,
where: r.uuid == ^uuid,
where: ^pointing_at(pointer, value),
update: ^pointer_update(pointer, nil)
),
[]
)
:ok
end
defp pointing_at({:column, column}, value), do: dynamic([r], field(r, ^column) == ^value)
# Without case, as the claim check compares: a pointer spelled otherwise
# names the same folder and must clear too.
defp pointing_at({map_field, key}, value) when is_binary(key),
do:
dynamic(
[r],
fragment("lower(?->>?)", field(r, ^map_field), ^key) == ^String.downcase(value)
)
defp pointer_update({:column, column}, folder_uuid), do: [set: [{column, folder_uuid}]]
# `jsonb_set` with a NULL value answers NULL for the whole map, so
# clearing a pointer removes its key instead.
defp pointer_update({map_field, key}, nil) when is_binary(key) do
[
set: [
{map_field,
dynamic([r], fragment("coalesce(?, '{}'::jsonb) - ?", field(r, ^map_field), ^key))}
]
]
end
defp pointer_update({map_field, key}, folder_uuid) when is_binary(key) do
[
set: [
{map_field,
dynamic(
[r],
fragment(
"jsonb_set(coalesce(?, '{}'::jsonb), ARRAY[?]::text[], to_jsonb(?::text))",
field(r, ^map_field),
^key,
^folder_uuid
)
)}
]
]
end
defp name_refused?(%Ecto.Changeset{errors: errors}), do: Keyword.has_key?(errors, :name)
defp fallback?(name, fallback), do: is_binary(fallback) and fallback != name
@doc """
Names a pending folder — one created for a record before it was saved,
its name starting with `prefix` — after its record. A folder that is
not pending is left alone. Always `:ok`; a failure is logged.
## Options
* `:fallback_name` — used when core refuses `name` (taken under the
folder's parent)
* `:move_to` — also move the folder under this parent (`nil` = the
root), for a module whose parent depends on the saved record. Pass
only a definite answer (`parent_hook/4`'s `{:ok, parent}`), never
the root a failed hook fell back to. Without it only the name
changes: an explicit parent in an update is a move.
"""
@spec name_pending(String.t() | nil, String.t(), String.t(), keyword()) :: :ok
def name_pending(folder_uuid, prefix, name, opts \\ [])
when is_binary(prefix) and is_binary(name) and is_list(opts) do
with %Folder{name: current} = folder <-
safely("load pending folder", fn -> live_folder(folder_uuid) end),
true <- String.starts_with?(current, prefix),
{:error, reason} <- safely("name pending folder", fn -> rename(folder, name, opts) end) do
Logger.warning("Pending folder #{folder.uuid} was not named: #{describe_failure(reason)}")
:ok
else
_ -> :ok
end
end
defp rename(folder, name, opts) do
fallback = Keyword.get(opts, :fallback_name)
move = if Keyword.has_key?(opts, :move_to), do: %{parent_uuid: opts[:move_to]}, else: %{}
case Storage.update_folder(folder, Map.put(move, :name, name)) do
{:error, %Ecto.Changeset{} = changeset} = error ->
if name_refused?(changeset) and fallback?(name, fallback),
do: Storage.update_folder(folder, Map.put(move, :name, fallback)),
else: error
result ->
result
end
end
@doc """
Deletes for good every folder named `name` — wherever it sits, trashed
or not — with everything inside it, for a record deleted for good whose
folder name embeds its uuid. A file some other folder links keeps
living there (`Storage.delete_folder_completely/1`). Always `:ok`; a
failure is logged.
A name with no uuid in it is refused (logged, nothing deleted): it is
matched across every folder in the install, so a plain name such as
`"Invoices"` would take people's own folders of that name with it.
"""
@spec purge_named(String.t()) :: :ok
def purge_named(name) when is_binary(name) do
if name =~ ~r/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/i do
safely("purge folders", fn ->
from(f in Folder, where: f.name == ^name, order_by: [asc: f.inserted_at])
|> repo().all()
|> Enum.each(&Storage.delete_folder_completely/1)
end)
else
Logger.warning(
"[ResourceFolders] purge_named refused a name with no uuid: #{inspect(name)}"
)
end
:ok
end
# ── Files in a folder ───────────────────────────────────────────────
@doc """
The files `folder_uuid` holds — home there or linked in — that are
live: not trashed and not system-managed (tile chunks, an edited
image's hidden original, which are never listed). Unordered and
uncapped, for counting or for a caller's own order.
"""
@spec files_query(String.t()) :: Ecto.Query.t()
def files_query(folder_uuid) when is_binary(folder_uuid) do
linked = from(fl in FolderLink, where: fl.folder_uuid == ^folder_uuid, select: fl.file_uuid)
from(f in StorageFile,
where: f.folder_uuid == ^folder_uuid or f.uuid in subquery(linked)
)
|> live_files()
end
@doc """
The files `folder_uuid` holds (`files_query/1`); `[]` for `nil`.
## Options
* `:only` — `:images`, `:non_images`, `{:type, file_type}`,
`{:not_type, file_type}` or `:all` (default)
* `:order` — `:newest` first (default) or `:oldest` first
* `:limit` — at most this many (default #{@list_limit})
"""
@spec list_files(String.t() | nil, keyword()) :: [StorageFile.t()]
def list_files(folder_uuid, opts \\ [])
def list_files(nil, _opts), do: []
def list_files(folder_uuid, opts) when is_binary(folder_uuid) do
folder_uuid
|> files_query()
|> only(Keyword.get(opts, :only, :all))
|> ordered(Keyword.get(opts, :order, :newest))
|> limit(^Keyword.get(opts, :limit, @list_limit))
|> repo().all()
end
@doc """
The files of many folders in two queries: `%{folder_uuid => [file]}`,
each list in `:order` (`:newest` first by default), a folder holding
nothing left out. Takes `:only` like `list_files/2`; uncapped.
"""
@spec files_by_folder([String.t()], keyword()) :: %{String.t() => [StorageFile.t()]}
def files_by_folder(folder_uuids, opts \\ [])
def files_by_folder([], _opts), do: %{}
def files_by_folder(folder_uuids, opts) when is_list(folder_uuids) do
uuids = folder_uuids |> Enum.map(&cast/1) |> Enum.reject(&is_nil/1) |> Enum.uniq()
kind = Keyword.get(opts, :only, :all)
home =
from(f in StorageFile, where: f.folder_uuid in ^uuids, select: {f.folder_uuid, f})
|> live_files()
|> only(kind)
linked =
from(f in StorageFile,
join: fl in FolderLink,
on: fl.file_uuid == f.uuid,
where: fl.folder_uuid in ^uuids,
select: {fl.folder_uuid, f}
)
|> live_files()
|> only(kind)
(repo().all(home) ++ repo().all(linked))
|> Enum.uniq_by(fn {folder_uuid, file} -> {folder_uuid, file.uuid} end)
|> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
|> Map.new(fn {folder_uuid, files} ->
{folder_uuid, sort_files(files, Keyword.get(opts, :order, :newest))}
end)
end
@doc """
How many live files each of `folder_uuids` holds, in two grouped
queries: `%{folder_uuid => count}`, an empty folder left out. Takes
`:only` like `list_files/2`; counts exactly what `files_query/1` holds
(`list_files/2` lists the same set, up to its `:limit`)
(a file linked into its own home folder once).
"""
@spec count_by_folder([String.t()], keyword()) :: %{String.t() => pos_integer()}
def count_by_folder(folder_uuids, opts \\ [])
def count_by_folder([], _opts), do: %{}
def count_by_folder(folder_uuids, opts) when is_list(folder_uuids) do
uuids = folder_uuids |> Enum.map(&cast/1) |> Enum.reject(&is_nil/1) |> Enum.uniq()
kind = Keyword.get(opts, :only, :all)
home =
from(f in StorageFile,
where: f.folder_uuid in ^uuids,
group_by: f.folder_uuid,
select: {f.folder_uuid, count(f.uuid)}
)
|> live_files()
|> only(kind)
# A link naming the file's own home is read once by `files_query/1`'s
# `home OR linked`, so it is not counted twice here either.
linked =
from(f in StorageFile,
join: fl in FolderLink,
on: fl.file_uuid == f.uuid,
where: fl.folder_uuid in ^uuids,
where: is_nil(f.folder_uuid) or f.folder_uuid != fl.folder_uuid,
group_by: fl.folder_uuid,
select: {fl.folder_uuid, count(f.uuid)}
)
|> live_files()
|> only(kind)
Enum.reduce(repo().all(home) ++ repo().all(linked), %{}, fn {folder, n}, acc ->
Map.update(acc, folder, n, &(&1 + n))
end)
end
@doc """
The file uuid `record` stores through `pointer` (`t:pointer/0`) — an
avatar in `metadata`, a featured image in `data` — or `nil` when it has
none or holds something that is not a uuid.
"""
@spec pointer_value(map(), pointer()) :: String.t() | nil
def pointer_value(record, {:column, column}), do: record |> Map.get(column) |> cast()
def pointer_value(record, {map_field, key}) when is_binary(key) do
case Map.get(record, map_field) do
%{} = map -> map |> Map.get(key) |> cast()
_ -> nil
end
end
@doc """
The live file `record` points at through `pointer`, or `nil` — a missing,
trashed or system-managed file is not shown as anyone's avatar or
featured image.
"""
@spec pointed_file(map(), pointer()) :: StorageFile.t() | nil
def pointed_file(record, pointer) do
case pointer_value(record, pointer) do
nil -> nil
uuid -> from(f in StorageFile, where: f.uuid == ^uuid) |> live_files() |> repo().one()
end
end
@doc """
Whether live file `file_uuid` is in `folder_uuid` — the check that
authorizes pointing a record at one of its own files (an avatar, a
featured image). Takes `:only` like `list_files/2`.
"""
@spec holds_file?(String.t() | nil, String.t() | nil, keyword()) :: boolean()
def holds_file?(folder_uuid, file_uuid, opts \\ []) do
case {cast(folder_uuid), cast(file_uuid)} do
{folder, file} when is_binary(folder) and is_binary(file) ->
folder
|> files_query()
|> where([f], f.uuid == ^file)
|> only(Keyword.get(opts, :only, :all))
|> repo().exists?()
_ ->
false
end
end
@doc """
Points record `uuid` of `schema` at `file_uuid` through `pointer` (an
avatar, a featured image) only when `folder_uuid` holds that live file
(`holds_file?/3`, taking its `:only`) — the write a forged file uuid must
not get past. The file's row is locked for the check and the write, the
same lock `attach/2` and `detach/2` take, so the file cannot leave the
folder in between; the write is `write_pointer/4`'s, one key, so a stale
copy of the record cannot overwrite its other keys.
`:ok`, `{:error, :not_held}`, `{:error, :not_found}` for no such record,
or `{:error, reason}`; never raises.
"""
@spec point_at(module(), String.t(), pointer(), String.t(), String.t() | nil, keyword()) ::
:ok | {:error, :not_held | :not_found | term()}
def point_at(schema, uuid, pointer, file_uuid, folder_uuid, opts \\ []) do
safely("point at file", fn ->
with_locked_file(file_uuid, {:error, :not_held}, fn file ->
if holds_file?(folder_uuid, file.uuid, opts),
do: write_pointer(schema, uuid, pointer, file.uuid),
else: {:error, :not_held}
end)
end)
end
defp live_files(query),
do: where(query, [f], f.status != "trashed" and f.system_managed == false)
defp only(query, :images), do: only(query, {:type, "image"})
defp only(query, :non_images), do: only(query, {:not_type, "image"})
defp only(query, :all), do: query
defp only(query, {:type, type}), do: where(query, [f], f.file_type == ^type)
defp only(query, {:not_type, type}), do: where(query, [f], f.file_type != ^type)
defp ordered(query, :oldest), do: order_by(query, [f], asc: f.inserted_at, asc: f.uuid)
defp ordered(query, :newest), do: order_by(query, [f], desc: f.inserted_at, desc: f.uuid)
defp sort_files(files, :oldest), do: Enum.sort_by(files, &{&1.inserted_at, &1.uuid}, :asc)
defp sort_files(files, :newest), do: Enum.sort_by(files, &{&1.inserted_at, &1.uuid}, :desc)
# ── Putting files in and taking them out ────────────────────────────
@doc """
Puts a file into `folder_uuid` by core's attach rule: a file with no
home is adopted (`:adopted`), a file homed elsewhere is linked
(`:linked`), a file already home or linked there is left alone
(`:already_attached`). A folder that is not live is refused
(`{:error, :folder_unavailable}`, `nil` included), and so is a trashed
file (`{:error, :file_trashed}`) — either would be listed nowhere.
Never raises.
The folder's row is taken before the file's, the order the reorganizer's
move takes them in.
"""
@spec attach(StorageFile.t() | String.t(), String.t() | nil) ::
{:ok, :adopted | :linked | :already_attached} | {:error, term()}
def attach(file_or_uuid, folder_uuid) when is_binary(folder_uuid) do
safely("attach file", fn ->
{:ok, result} =
repo().transaction(fn ->
# The folder BEFORE the file, and share-locked: a trash of it
# committing between this check and the write would leave the
# file homed in a trashed folder, in no listing — and the
# reorganizer's move holds a folder and then writes the files
# under it, so taking them the other way round here would
# deadlock against it.
case live_folder_locked(folder_uuid) do
nil ->
{:error, :folder_unavailable}
%Folder{uuid: folder_uuid} ->
with_locked_file(
file_or_uuid,
{:error, :not_found},
&attach_locked(&1, folder_uuid)
)
end
end)
result
end)
end
def attach(_file_or_uuid, nil), do: {:error, :folder_unavailable}
defp attach_locked(file, folder_uuid) do
cond do
file.status == "trashed" -> {:error, :file_trashed}
file.folder_uuid == folder_uuid -> {:ok, :already_attached}
Storage.folder_link(folder_uuid, file.uuid) -> {:ok, :already_attached}
true -> attach_new(file, folder_uuid)
end
end
defp attach_new(file, folder_uuid) do
outcome = if is_nil(file.folder_uuid), do: :adopted, else: :linked
case Storage.attach_file_to_folder(file, folder_uuid) do
{:ok, _file} -> {:ok, outcome}
{:error, reason} -> {:error, reason}
end
end
@doc """
Files a just-stored upload into `folder_uuid`, given what
`Storage.store_file_in_buckets/6` (or `store_file/2`) answered. Storage
de-duplicates by content, so the answer can be an existing file:
* a trashed duplicate is restored — the person removed it and is
uploading it again — and attached as if new;
* a duplicate already in the folder is `{:already_attached, file}`,
so the uploader hears that nothing was added;
* anything else is attached: `{:ok, file}`.
Never raises.
"""
@spec place_stored(term(), String.t()) ::
{:ok, StorageFile.t()} | {:already_attached, StorageFile.t()} | {:error, term()}
def place_stored({:ok, %StorageFile{} = file}, folder_uuid), do: place(file, folder_uuid, false)
# Restored and attached together: a restore whose attach then fails
# would leave the file active in its old (trashed) home, in no listing
# and not in the file trash either.
def place_stored({:ok, %StorageFile{status: "trashed"} = file, :duplicate}, folder_uuid) do
safely("restore file", fn ->
repo().transaction(fn ->
# Into this folder, not the one it was removed from — unless
# someone else restored it first, when it keeps their home and is
# linked in here like any other duplicate.
with {:ok, restored} <- restored_or_current(file, folder_uuid),
{:ok, placed} <- place(restored, folder_uuid, false) do
placed
else
{:error, reason} -> repo().rollback(reason)
end
end)
end)
end
def place_stored({:ok, %StorageFile{} = file, :duplicate}, folder_uuid),
do: place(file, folder_uuid, true)
def place_stored({:error, reason}, _folder_uuid), do: {:error, reason}
defp restored_or_current(file, folder_uuid) do
case Storage.restore_file_into(file, folder_uuid) do
{:ok, restored} -> {:ok, restored}
{:error, :not_trashed} -> {:ok, Storage.get_file(file.uuid) || file}
end
end
defp place(file, folder_uuid, duplicate?) do
case attach(file, folder_uuid) do
{:ok, :already_attached} when duplicate? -> {:already_attached, file}
{:ok, _outcome} -> {:ok, file}
{:error, reason} -> {:error, reason}
end
end
@doc """
Takes a file out of `folder_uuid` by core's removal rule
(`Storage.remove_file_from_folder/2`): a link is dropped (`:unlinked`);
a file homed here moves to a live folder that also links it
(`:rehomed`), or is trashed when nothing else holds it (`:trashed`).
`:absent` when the file is gone or was never in the folder — including
`nil` for the folder, which holds nothing. Never a hard delete; never
raises.
"""
@spec detach(StorageFile.t() | String.t(), String.t() | nil) ::
{:ok, :unlinked | :rehomed | :trashed | :absent} | {:error, term()}
def detach(_file_or_uuid, nil), do: {:ok, :absent}
def detach(file_or_uuid, folder_uuid) when is_binary(folder_uuid) do
safely("detach file", fn ->
with_locked_file(file_or_uuid, {:ok, :absent}, fn file ->
case Storage.remove_file_from_folder(file, folder_uuid) do
{:ok, outcome, _file} -> {:ok, outcome}
{:error, :not_in_folder} -> {:ok, :absent}
{:error, reason} -> {:error, reason}
end
end)
end)
end
# The file's row, read fresh and locked for the rest of the transaction:
# a caller's struct can be stale (the file re-homed since), and deciding
# from it — two removals at once, the second still seeing the old home —
# trashed a file another folder still held. A logical refusal writes
# nothing, so it is returned as is rather than rolled back (which would
# abort a caller's own transaction).
defp with_locked_file(file_or_uuid, missing, fun) do
case cast(file_uuid(file_or_uuid)) do
nil ->
missing
uuid ->
{:ok, result} =
repo().transaction(fn ->
case repo().one(from(f in StorageFile, where: f.uuid == ^uuid, lock: "FOR UPDATE")) do
nil -> missing
file -> fun.(file)
end
end)
result
end
end
defp file_uuid(%StorageFile{uuid: uuid}), do: uuid
defp file_uuid(uuid), do: uuid
# ── Helpers ─────────────────────────────────────────────────────────
# A soft-failure path: an unreachable database raises on an unowned
# checkout but EXITS on a dead pool, so both become `{:error, _}`.
defp safely(what, fun) do
fun.()
rescue
error ->
Logger.warning("[ResourceFolders] #{what} failed: #{describe_failure(error)}")
{:error, error}
catch
:exit, reason ->
Logger.warning("[ResourceFolders] #{what} failed: #{exit_shape(reason)}")
{:error, {:exit, reason}}
end
defp directly_under(query, nil), do: where(query, [f], is_nil(f.parent_uuid))
defp directly_under(query, parent_uuid) do
case cast(parent_uuid) do
nil -> where(query, [_f], false)
uuid -> where(query, [f], f.parent_uuid == ^uuid)
end
end
# The text form only: `Ecto.UUID.cast/1` also takes any 16-byte binary
# as a raw uuid, which would turn a 16-character folder name into one.
defp cast(<<_::binary-size(36)>> = value) do
case Ecto.UUID.cast(value) do
{:ok, uuid} -> uuid
:error -> nil
end
end
defp cast(_value), do: nil
defp repo, do: PhoenixKit.RepoHelper.repo()
end