Packages

unicode_string

1.2.1
2.2.0 2.1.0 2.0.1 2.0.0 1.8.0 1.7.0 1.6.0 1.5.0 1.4.1 1.4.0 1.3.1 1.3.0 1.3.0-rc.0 1.2.1 1.2.0 1.1.0 1.0.1 1.0.0 0.3.0 0.2.0 0.1.0

Unicode locale-aware case folding, case mapping (upcase, downcase and titlecase) case-insensitive equality as well as word, line, grapheme and sentence breaking and streaming.

HexDocs

Current section

21 Versions

Jump to
Readme
21 Versions
2 Dependencies
Dependants
Files
Activity
Hide whitespace

Compare versions

6 files changed
+25 additions
-18 deletions
  @@ -1,5 +1,13 @@
1 1 # Changelog
2 2
3 + ## Unicode String v1.2.1
4 +
5 + This is the changelog for Unicode String v1.2.1 released on June 2nd, 2023. For older changelogs please consult the release tag on [GitHub](https://github.com/elixir-unicode/unicode_string/tags)
6 +
7 + ### Bug Fixes
8 +
9 + * Resolve segments dir at runtime, not compile time. Thanks to @crkent for the report. Closes #4.
10 +
3 11 ## Unicode String v1.2.0
4 12
5 13 This is the changelog for Unicode String v1.2.0 released on March 14th, 2023. For older changelogs please consult the release tag on [GitHub](https://github.com/elixir-unicode/unicode_string/tags)
  @@ -1,6 +1,6 @@
1 1 # Unicode String
2 2
3 - Adds functions supporting some string algorithms in the Unicode standard:
3 + Adds functions supporting some string algorithms in the Unicode standard. For example:
4 4
5 5 * The [Unicode Case Folding](https://www.unicode.org/versions/Unicode15.0.0/ch03.pdf) algorithm to provide case-independent equality checking irrespective of language or script with `Unicode.String.fold/2` and `Unicode.String.equals_ignoring_case?/2`
6 6
  @@ -58,13 +58,13 @@ iex> Unicode.String.split "No, I don't have a Ph.D. but I don't think it matters
58 58 iex> Unicode.String.split "No, I don't have a Ph.D. but I don't think it matters.", break: :sentence, trim: true
59 59 ["No, I don't have a Ph.D. but I don't think it matters."]
60 60
61 - # Break suppressions are locale sensitive.
61 + # Sentence Break suppressions are locale sensitive.
62 62 iex> Unicode.String.Segment.known_locales
63 63 ["de", "el", "en", "en-US", "en-US-POSIX", "es", "fi", "fr", "it", "ja", "pt",
64 64 "root", "ru", "sv", "zh", "zh-Hant"]
65 65
66 - iex> Unicode.String.split "Non, c'est M. Dubois.", break: :word, trim: true, locale: "fr"
67 - ["Non", ",", "c'est", "M", ".", "Dubois", "."]
66 + iex> Unicode.String.split "Non, c'est M. Dubois.", break: :sentence, trim: true, locale: "fr"
67 + ["Non, c'est M. Dubois."]
68 68
69 69 # Note that break: :line does NOT mean split the string
70 70 # at newlines. It splits the string where a line break would be
  @@ -76,7 +76,7 @@ iex> Unicode.String.split "This is a sentence. And another.", break: :line
76 76
77 77 ## Segment Streaming
78 78
79 - Segmentation can also be streamed using `Unicode.String.stream/2`. For large strings this may improve memory usage since the intermediate segments will garbage collected when they fall out of scope.
79 + Segmentation can also be streamed using `Unicode.String.stream/2`. For large strings this may improve memory usage since the intermediate segments will be garbage collected when they fall out of scope.
80 80
81 81 ```elixir
82 82 iex> Enum.to_list Unicode.String.stream("this is a set of words", trim: true) ["this", "is", "a", "set", "of", "words"]
  @@ -20,10 +20,10 @@
20 20 {<<"licenses">>,[<<"Apache-2.0">>]}.
21 21 {<<"links">>,
22 22 [{<<"Changelog">>,
23 - <<"https://github.com/elixir-unicode/unicode_string/blob/v1.2.0/CHANGELOG.md">>},
23 + <<"https://github.com/elixir-unicode/unicode_string/blob/v1.2.1/CHANGELOG.md">>},
24 24 {<<"GitHub">>,<<"https://github.com/elixir-unicode/unicode_string">>},
25 25 {<<"Readme">>,
26 - <<"https://github.com/elixir-unicode/unicode_string/blob/v1.2.0/README.md">>}]}.
26 + <<"https://github.com/elixir-unicode/unicode_string/blob/v1.2.1/README.md">>}]}.
27 27 {<<"name">>,<<"unicode_string">>}.
28 28 {<<"requirements">>,
29 29 [[{<<"app">>,<<"unicode_set">>},
  @@ -36,4 +36,4 @@
36 36 {<<"optional">>,false},
37 37 {<<"repository">>,<<"hexpm">>},
38 38 {<<"requirement">>,<<"~> 0.6">>}]]}.
39 - {<<"version">>,<<"1.2.0">>}.
39 + {<<"version">>,<<"1.2.1">>}.
  @@ -284,18 +284,17 @@ defmodule Unicode.String.Segment do
284 284 end
285 285
286 286 @app_name Mix.Project.config()[:app]
287 - @segments_dir Path.join(:code.priv_dir(@app_name), "/segments")
288 287
289 288 @doctype "<!DOCTYPE ldml SYSTEM \"../../common/dtd/ldml.dtd\">"
290 289
291 290 @doc false
292 291 def segments_dir do
293 - @segments_dir
292 + Path.join(:code.priv_dir(@app_name), "/segments")
294 293 end
295 294
296 295 @doc false
297 296 def locale_map do
298 - @segments_dir
297 + segments_dir()
299 298 |> File.ls!()
300 299 |> Enum.map(fn locale_file ->
301 300 locale =
  @@ -377,7 +376,7 @@ defmodule Unicode.String.Segment do
377 376 defp raw_segments(locale) do
378 377 if file = Map.get(locale_map(), locale) do
379 378 content =
380 - @segments_dir
379 + segments_dir()
381 380 |> Path.join(file)
382 381 |> File.read!()
383 382 |> String.replace(@doctype, "")
  @@ -26,15 +26,15 @@ defmodule Unicode.String do
26 26 @type error_return :: {:error, String.t()}
27 27
28 28 @type options :: [
29 - {:locale, String.t()},
30 - {:break, break_type},
29 + {:locale, String.t()} |
30 + {:break, break_type} |
31 31 {:suppressions, boolean}
32 32 ]
33 33
34 34 @type split_options :: [
35 - {:locale, String.t()},
36 - {:break, break_type},
37 - {:suppressions, boolean},
35 + {:locale, String.t()} |
36 + {:break, break_type} |
37 + {:suppressions, boolean} |
38 38 {:trim, boolean}
39 39 ]
Loading more files…