Packages

Simple sentiment analysis based on the AFINN-111 wordlist

Current section

Files

Jump to
sentient lib tokenizer.ex
Raw

lib/tokenizer.ex

defmodule Tokenizer do
def tokenize(text) do
text
|> String.strip
|> String.downcase
|> String.replace(~r/[^a-zA-Z- ]+/, "")
|> String.replace(~r/ {2,}/, " ")
|> String.split
end
end