Current section
Files
Jump to
Current section
Files
lib/exmoji/scanner.ex
defmodule Exmoji.Scanner do
@moduledoc """
Provides fast searches against binary strings for the presence of UTF-8
encoded Emoji characters.
"""
@doc """
Scans a bitstring for all encoded emoji characters contained within.
Returns a list of all `EmojiChar` contained within that string, in order.
## Example
iex> Exmoji.Scanner.scan("flying on my 🚀 to visit the 👾 people.")
[%Exmoji.EmojiChar{name: "ROCKET", short_name: "rocket",
short_names: ["rocket"], text: nil, unified: "1F680", variations: []},
%Exmoji.EmojiChar{name: "ALIEN MONSTER", short_name: "space_invader",
short_names: ["space_invader"], text: nil, unified: "1F47E", variations: []}]
"""
def scan(str) do
bscan(str)
|> Enum.map(&Exmoji.char_to_unified/1)
|> Enum.map(&Exmoji.from_unified/1)
end
# DEPRECATED: regex scan, returns a list of emoji as char glyphs
#
# Regular expressions are incredibly slow in Erlang, so this was replaced with
# an efficient binary scan. It remains here in the code for two reasons:
# a.) historical purposes and learning
# b.) since we still use this regex in the ruby and node versions of this
# library, this allows us to easily test and compare to make sure our
# new algorithm produces identical results.
#
# Thus it is kept as public so we can compare it in test...
fbs_pattern = Exmoji.chars(include_variants: true) |> Enum.join("|")
@fbs_regexp Regex.compile!( "(?:#{fbs_pattern})" )
@doc false
def rscan(str) do
Regex.scan(@fbs_regexp, str)
|> Enum.map(&List.first/1)
end
# Binary pattern match scan to do the same as above!
#
# This looks deceptively simple, but before modifying it, bear in mind that
# an EmojiChar rendered glyph can be represented as 1..3 utf8 codepoints in
# sequence, and some of the longer sequences contain other sequences as
# sub-sequences, so you want to always try to match longer sequences first, or
# risk cutting a character in half.
#
# Essentially the logic here is:
# 1. Make functions to pattern match on all known glyphs at binary head. We
# order these functions based on the bitsize of the pattern we will match
# against, to make sure we dont cut a character in half and get the small
# variant.
# 2. If the function matches head of pattern (match), add glyph to head of
# our results accumulator and recurse on the remainder of the binary.
# 3. If no pattern matches head of binary, chop of one utf8 segment and
# throw it away (non-match), recurse on the remainder of the binary.
# 4. If end of the binary is reached, reverse the accumulator and return
# results!
#
@doc false
def bscan(str), do: _bscan(str, [])
# first we sort all known char glyphs by reverse bitsize, so we can use the
# bigger binary patterns first when defining our pattern match functions.
sorted_chars = Enum.sort(
Exmoji.chars(include_variants: true),
&(&1 > &2)
)
# define functions that pattern match against each emojichar binary at head.
for glyph <- sorted_chars do
defp _bscan(<< unquote(glyph), tail::binary >>, acc) do
_bscan(tail, [unquote(glyph) | acc])
end
end
# if nothing is found, cut the head off and move on.
defp _bscan(<< _head::utf8, tail::binary >>, acc), do: _bscan(tail, acc)
# when we reach the end, return reversed accumulator.
defp _bscan(<<>>, acc), do: Enum.reverse(acc)
end