Packages
mob
0.6.21
0.7.20
0.7.19
0.7.18
0.7.17
0.7.16
0.7.15
0.7.14
0.7.13
0.7.12
0.7.11
0.7.10
0.7.9
0.7.8
0.7.7
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.26
0.6.25
0.6.24
0.6.23
0.6.22
0.6.21
0.6.20
0.6.19
0.6.18
0.6.17
0.6.16
0.6.15
0.6.14
0.6.13
0.6.12
0.6.11
0.6.10
0.6.9
0.6.8
0.6.7
0.6.6
0.6.5
0.6.2
0.6.1
0.6.0
0.5.18
0.5.17
0.5.16
0.5.15
0.5.14
0.5.11
0.5.10
0.5.7
0.5.6
0.5.5
0.5.4
0.5.3
0.5.2
0.5.1
0.5.0
0.4.0
0.3.10
0.3.9
0.3.8
0.3.7
0.3.6
0.3.5
0.3.4
0.3.3
0.3.2
0.3.1
0.3.0
0.2.0
0.1.0
BEAM-on-device mobile framework for Elixir
Current section
Files
Jump to
Current section
Files
lib/mob/camera.ex
defmodule Mob.Camera do
@moduledoc """
Native camera capture for photos and videos.
Requires `:camera` permission (and `:microphone` for video). iOS
additionally needs `NSCameraUsageDescription` (and
`NSMicrophoneUsageDescription` for video) in `Info.plist`;
Android needs `CAMERA` (and `RECORD_AUDIO` for video) in
`AndroidManifest.xml`. The default `mix mob.new` templates ship
both. See the [permissions guide](permissions.html) for the
cross-platform table.
Opens the native OS camera UI. Results arrive as:
handle_info({:camera, :photo, %{path: path, width: w, height: h}}, socket)
handle_info({:camera, :video, %{path: path, duration: seconds}}, socket)
handle_info({:camera, :cancelled}, socket)
The `path` is a local temp file. Copy it elsewhere before the next capture.
iOS: `UIImagePickerController`. Android: `TakePicture` / `CaptureVideo` activity contracts.
## Live frame stream
For real-time work (object detection, AR, custom filters) `start_frame_stream/2`
delivers per-frame pixel data as messages:
handle_info({:camera, :frame, %{bytes: bin, width: w, height: h,
format: :rgb_f32,
timestamp_ms: t, dropped: n}}, socket)
The native side handles resize + format conversion (vImage on iOS,
CameraX ImageAnalysis + Bitmap on Android) so the BEAM never sees
raw camera buffers. Late frames are dropped on the native side —
the BEAM mailbox can't unbounded-grow if your receiver lags behind
the camera's 30 fps cadence.
"""
@doc """
Open the camera to capture a photo.
Options:
- `quality: :high | :medium | :low` (default `:high`) — JPEG compression level
"""
@spec capture_photo(Mob.Socket.t(), keyword()) :: Mob.Socket.t()
def capture_photo(socket, opts \\ []) do
quality = Keyword.get(opts, :quality, :high)
:mob_nif.camera_capture_photo(quality)
socket
end
@doc """
Open the camera to record a video.
Options:
- `max_duration: integer` — maximum clip length in seconds (default `60`)
"""
@spec capture_video(Mob.Socket.t(), keyword()) :: Mob.Socket.t()
def capture_video(socket, opts \\ []) do
max_duration = Keyword.get(opts, :max_duration, 60)
:mob_nif.camera_capture_video(max_duration)
socket
end
@doc """
Start a live camera preview session. Pair with a `:camera_preview` component
in your render tree to display the feed.
Options:
- `facing: :back | :front` (default `:back`)
"""
@spec start_preview(Mob.Socket.t(), keyword()) :: Mob.Socket.t()
def start_preview(socket, opts \\ []) do
facing = Keyword.get(opts, :facing, :back) |> Atom.to_string()
:mob_nif.camera_start_preview(:json.encode(%{"facing" => facing}))
socket
end
@doc "Stop the active camera preview session."
@spec stop_preview(Mob.Socket.t()) :: Mob.Socket.t()
def stop_preview(socket) do
:mob_nif.camera_stop_preview()
socket
end
@doc """
Start streaming camera frames to the calling process. Frames arrive
as messages of shape:
handle_info({:camera, :frame, %{
bytes: binary(), # pixel data, format-dependent
width: non_neg_integer(),
height: non_neg_integer(),
format: :rgb_f32 | :bgra_u8,
timestamp_ms: non_neg_integer(),
dropped: non_neg_integer() # frames skipped since last delivery
}}, socket)
## Options
* `:width`, `:height` — target frame size in pixels. Defaults to
`640` × `640` (YOLO-friendly). Pass `nil` for both to receive the
camera's native resolution. Mismatched aspect ratios are
center-cropped on the long axis before scaling. Capped at ~4 MP
to keep the BEAM mailbox bounded.
* `:format` — pixel format. One of:
- `:rgb_f32` (default) — interleaved RGB floats normalised to
`[0.0, 1.0]`. Byte size: `width * height * 3 * 4`. Ready for
`Nx.from_binary(bin, :f32, ...) |> Nx.reshape({1, h, w, 3})`.
- `:bgra_u8` — raw 32-bit BGRA bytes (native iOS pixel layout;
Android repacks ARGB → BGRA for parity). Byte size:
`width * height * 4`. 4× smaller than `:rgb_f32`; useful for
forwarding to another NIF or doing custom preprocessing.
* `:facing` — `:back` (default) or `:front`. Same camera the
preview uses; calling `start_frame_stream/2` alone will activate
the capture session without a visible preview.
* `:throttle_ms` — minimum interval between deliveries (default
`0`). Native-side throttle, complementary to the OS's late-frame
drop. Use `throttle_ms: 100` for 10 Hz delivery when full
camera-rate inference isn't needed.
## Notes
Returns the socket immediately; frames begin arriving asynchronously
once the OS has activated the capture session (typically <100 ms).
Receiver is the **calling process** at the time of invocation —
call from a `Mob.Screen` callback (mount, handle_info), not from a
task or genserver running elsewhere.
"""
@spec start_frame_stream(Mob.Socket.t(), keyword()) :: Mob.Socket.t()
def start_frame_stream(socket, opts \\ []) do
:mob_nif.camera_start_frame_stream(:json.encode(frame_stream_opts(opts)))
socket
end
@doc """
Build the option map passed to `camera_start_frame_stream/1`. Pure
function exposed so tests can pin defaults + serialisation without
going through the NIF.
"""
@spec frame_stream_opts(keyword()) :: map()
def frame_stream_opts(opts) do
%{
"width" => Keyword.get(opts, :width, 640),
"height" => Keyword.get(opts, :height, 640),
"format" => Keyword.get(opts, :format, :rgb_f32) |> Atom.to_string(),
"facing" => Keyword.get(opts, :facing, :back) |> Atom.to_string(),
"throttle_ms" => Keyword.get(opts, :throttle_ms, 0)
}
end
@doc """
Stop the camera frame stream. Safe to call when no stream is
active. The visible preview (if `start_preview/2` was called
separately) is left untouched.
"""
@spec stop_frame_stream(Mob.Socket.t()) :: Mob.Socket.t()
def stop_frame_stream(socket) do
:mob_nif.camera_stop_frame_stream()
socket
end
end