Packages
floki
0.38.1
0.38.4
0.38.3
0.38.2
0.38.1
0.38.0
0.37.1
0.37.0
0.36.3
0.36.2
0.36.1
0.36.0
0.35.4
0.35.3
0.35.2
0.35.1
0.35.0
0.34.3
0.34.2
0.34.1
0.34.0
0.33.1
0.33.0
0.32.1
0.32.0
0.31.0
0.30.1
0.30.0
0.29.0
0.28.0
0.27.0
0.26.0
0.25.0
0.24.0
0.23.1
0.23.0
0.22.0
0.21.0
0.20.4
0.20.3
0.20.2
0.20.1
0.20.0
0.19.3
0.19.2
0.19.1
0.19.0
0.18.1
0.18.0
0.17.2
0.17.1
0.17.0
0.16.0
0.15.0
0.14.0
0.13.2
0.13.1
0.13.0
0.12.1
0.12.0
0.11.0
0.10.1
0.10.0
0.9.0
0.8.1
0.8.0
0.7.2
0.7.1
0.7.0
0.6.1
0.6.0
0.5.0
0.4.1
0.4.0
0.3.3
0.3.2
0.3.1
0.3.0
0.2.1
0.2.0
0.1.1
0.1.0
0.0.5
0.0.4
0.0.3
0.0.2
0.0.1
Floki is a simple HTML parser that enables search for nodes using CSS selectors.
Current section
86 Versions
Jump to
Current section
86 Versions
Compare versions
10
files changed
+201
additions
-70
deletions
| @@ -7,6 +7,37 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. | |
| 7 7 | |
| 8 8 | ## [Unreleased][unreleased] |
| 9 9 | |
| 10 | + ## [0.38.1] - 2026-03-17 |
| 11 | + |
| 12 | + ### Performance |
| 13 | + |
| 14 | + This version contains major performance improvements in the following functions: |
| 15 | + |
| 16 | + * `Floki.filter_out/2`. |
| 17 | + * `Floki.find/2` - with some improvements to specific selectors, like classes |
| 18 | + and attribute selectors. |
| 19 | + * `Floki.text/2`. |
| 20 | + |
| 21 | + Those functions are not only faster, but are now using less memory. Please check |
| 22 | + the PRs related to this release if you want to better understand the numbers. |
| 23 | + |
| 24 | + * [Speed up `do_classes_matches?` - #649](https://github.com/philss/floki/pull/649) |
| 25 | + * [Make `filter_out` faster - #650](https://github.com/philss/floki/pull/650) |
| 26 | + * [Speed up attribute lookup - #651](https://github.com/philss/floki/pull/651) |
| 27 | + * [Enable tuple traversal optimization for multiple selectors - #652](https://github.com/philss/floki/pull/652) |
| 28 | + * [Optimize `HTMLTree.to_tuple` conversion using `Enum.reduce` - #657](https://github.com/philss/floki/pull/657) |
| 29 | + * [Optimize `Finder.get_descendant_ids/2` memory usage and speed - #660](https://github.com/philss/floki/pull/660) |
| 30 | + * [Optimize `Finder.get_siblings/2` memory usage and speed - #663](https://github.com/philss/floki/pull/663) |
| 31 | + * [Optimize `FlatText.get/3` memory usage and speed - #664](https://github.com/philss/floki/pull/664) |
| 32 | + * [Optimize class matching in `Floki.Selector` - #665](https://github.com/philss/floki/pull/665) |
| 33 | + |
| 34 | + All the improvements in this version were made by [Barna Kovacs - @preciz](https://github.com/preciz), |
| 35 | + so shout out and thanks to him! |
| 36 | + |
| 37 | + ### Fixed |
| 38 | + |
| 39 | + Remove a warning about an unused `require Logger` that pops up when using Elixir v1.20. |
| 40 | + |
| 10 41 | ## [0.38.0] - 2025-06-14 |
| 11 42 | |
| 12 43 | ### Added |
| @@ -54,6 +85,28 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. | |
| 54 85 | * `attribute/2` - removed clause |
| 55 86 | * `filter_out/2` - removed clause |
| 56 87 | |
| 88 | + - HTML must be parsed before searching. Functions like `Floki.find/2`, |
| 89 | + `Floki.attribute/2`, and other HTML manipulation functions **no longer work |
| 90 | + directly with HTML strings**. The HTML must be parsed first using |
| 91 | + `Floki.parse_fragment/2` or `Floki.parse_document/2`. |
| 92 | + |
| 93 | + Before: |
| 94 | + |
| 95 | + ```elixir |
| 96 | + html = "<div class='foobar'><p>Hello</p></div>" |
| 97 | + Floki.find(html, "p") |
| 98 | + Floki.attribute(html, "div", "class") |
| 99 | + ``` |
| 100 | + |
| 101 | + After: |
| 102 | + |
| 103 | + ```elixir |
| 104 | + html = "<div class='foobar'><p>Hello</p></div>" |
| 105 | + parsed_html = Floki.parse_fragment!(html) |
| 106 | + Floki.find(parsed_html, "p") |
| 107 | + Floki.attribute(parsed_html, "div", "class") |
| 108 | + ``` |
| 109 | + |
| 57 110 | ## [0.37.1] - 2025-03-22 |
| 58 111 | |
| 59 112 | ### Fixed |
| @@ -213,7 +266,7 @@ usage of memory for `find/2`, for example. | |
| 213 266 | |
| 214 267 | ### Fixed |
| 215 268 | |
| 216 | - - Fix find of elements by classes that contain colons. This is useful for when |
| 269 | + - Fix find of elements by classes that contain colons. This is useful for when |
| 217 270 | people are trying to find elements that contain Tailwind classes. |
| 218 271 | Thanks [@viniciusmuller](https://github.com/viniciusmuller). |
| 219 272 | |
| @@ -251,7 +304,7 @@ usage of memory for `find/2`, for example. | |
| 251 304 | ### Fixed |
| 252 305 | |
| 253 306 | - Allow attribute values to not be escaped. This fixes `Floki.raw_html/2` when used with the |
| 254 | - option `encode: false`. Thanks [@juanazam](https://github.com/juanazam). |
| 307 | + option `encode: false`. Thanks [@juanazam](https://github.com/juanazam). |
| 255 308 | - Fix `traverse_and_update/3` spec. Thanks [@WLSF](https://github.com/WLSF). |
| 256 309 | |
| 257 310 | ### Changed |
| @@ -867,7 +920,8 @@ of the parent element inside HTML. | |
| 867 920 | |
| 868 921 | - Elixir version requirement from "~> 1.0.0" to ">= 1.0.0". |
| 869 922 | |
| 870 | - [unreleased]: https://github.com/philss/floki/compare/v0.38.0...HEAD |
| 923 | + [unreleased]: https://github.com/philss/floki/compare/v0.38.1...HEAD |
| 924 | + [0.38.1]: https://github.com/philss/floki/compare/v0.38.0...v0.38.1 |
| 871 925 | [0.38.0]: https://github.com/philss/floki/compare/v0.37.1...v0.38.0 |
| 872 926 | [0.37.1]: https://github.com/philss/floki/compare/v0.37.0...v0.37.1 |
| 873 927 | [0.37.0]: https://github.com/philss/floki/compare/v0.36.3...v0.37.0 |
| @@ -1,4 +1,4 @@ | |
| 1 | - [](https://github.com/philss/floki/actions) |
| 1 | + [](https://github.com/philss/floki/actions/workflows/ci.yml) |
| 2 2 | [](https://hex.pm/packages/floki) |
| 3 3 | [](https://hexdocs.pm/floki/) |
| 4 4 | [](https://hex.pm/packages/floki) |
| @@ -3,7 +3,7 @@ | |
| 3 3 | {<<"GitHub">>,<<"https://github.com/philss/floki">>}, |
| 4 4 | {<<"Sponsor">>,<<"https://github.com/sponsors/philss">>}]}. |
| 5 5 | {<<"name">>,<<"floki">>}. |
| 6 | - {<<"version">>,<<"0.38.0">>}. |
| 6 | + {<<"version">>,<<"0.38.1">>}. |
| 7 7 | {<<"description">>, |
| 8 8 | <<"Floki is a simple HTML parser that enables search for nodes using CSS selectors.">>}. |
| 9 9 | {<<"elixir">>,<<"~> 1.15">>}. |
| @@ -31,10 +31,7 @@ defmodule Floki.FilterOut do | |
| 31 31 | defp filter(_, _), do: true |
| 32 32 | |
| 33 33 | defp mapper(nodes, selector) when is_list(nodes) do |
| 34 | - nodes |
| 35 | - |> Stream.filter(&filter(&1, selector)) |
| 36 | - |> Stream.map(&mapper(&1, selector)) |
| 37 | - |> Enum.to_list() |
| 34 | + do_mapper(nodes, selector, []) |
| 38 35 | end |
| 39 36 | |
| 40 37 | defp mapper({nodetext, x, y}, selector) do |
| @@ -42,4 +39,14 @@ defmodule Floki.FilterOut do | |
| 42 39 | end |
| 43 40 | |
| 44 41 | defp mapper(nodetext, _), do: nodetext |
| 42 | + |
| 43 | + defp do_mapper([], _selector, acc), do: Enum.reverse(acc) |
| 44 | + |
| 45 | + defp do_mapper([head | tail], selector, acc) do |
| 46 | + if filter(head, selector) do |
| 47 | + do_mapper(tail, selector, [mapper(head, selector) | acc]) |
| 48 | + else |
| 49 | + do_mapper(tail, selector, acc) |
| 50 | + end |
| 51 | + end |
| 45 52 | end |
| @@ -58,9 +58,14 @@ defmodule Floki.Finder do | |
| 58 58 | when (is_list(html_tree_as_tuple) or is_html_node(html_tree_as_tuple)) and |
| 59 59 | is_list(selectors) do |
| 60 60 | if traverse_html_tuples?(selectors) do |
| 61 | - [selector] = selectors |
| 62 61 | html_tree_as_tuple = List.wrap(html_tree_as_tuple) |
| 63 | - results = traverse_html_tuples(html_tree_as_tuple, selector, []) |
| 62 | + |
| 63 | + results = |
| 64 | + case selectors do |
| 65 | + [selector] -> traverse_html_tuples(html_tree_as_tuple, selector, []) |
| 66 | + _ -> traverse_html_tuples(html_tree_as_tuple, selectors, []) |
| 67 | + end |
| 68 | + |
| 64 69 | Enum.reverse(results) |
| 65 70 | else |
| 66 71 | tree = HTMLTree.build(html_tree_as_tuple) |
| @@ -108,6 +113,14 @@ defmodule Floki.Finder do | |
| 108 113 | end |
| 109 114 | |
| 110 115 | defp traverse_html_tuples?([]), do: true |
| 116 | + |
| 117 | + defp traverse_html_tuples?(selectors) when is_list(selectors) do |
| 118 | + Enum.all?(selectors, fn |
| 119 | + %Selector{combinator: nil} = selector -> traverse_html_tuples?(selector) |
| 120 | + _ -> false |
| 121 | + end) |
| 122 | + end |
| 123 | + |
| 111 124 | defp traverse_html_tuples?(_), do: false |
| 112 125 | |
| 113 126 | defp traverse_html_tree([], _selector, _tree, acc), do: acc |
| @@ -160,6 +173,23 @@ defmodule Floki.Finder do | |
| 160 173 | acc |
| 161 174 | end |
| 162 175 | |
| 176 | + defp traverse_html_tuples( |
| 177 | + [{_type, _attributes, children} = html_tuple | siblings], |
| 178 | + selectors, |
| 179 | + acc |
| 180 | + ) |
| 181 | + when is_list(selectors) do |
| 182 | + acc = |
| 183 | + if Enum.any?(selectors, &Selector.match?(html_tuple, &1, nil)) do |
| 184 | + [html_tuple | acc] |
| 185 | + else |
| 186 | + acc |
| 187 | + end |
| 188 | + |
| 189 | + acc = traverse_html_tuples(children, selectors, acc) |
| 190 | + traverse_html_tuples(siblings, selectors, acc) |
| 191 | + end |
| 192 | + |
| 163 193 | defp traverse_html_tuples( |
| 164 194 | [{_type, _attributes, children} = html_tuple | siblings], |
| 165 195 | %Selector{combinator: nil} = selector, |
| @@ -311,45 +341,62 @@ defmodule Floki.Finder do | |
| 311 341 | Map.get(tree.nodes, id) |
| 312 342 | end |
| 313 343 | |
| 314 | - defp get_sibling_ids_from([], _html_node), do: [] |
| 315 | - |
| 316 | - defp get_sibling_ids_from(ids, html_node) do |
| 317 | - ids |
| 318 | - |> Enum.reverse() |
| 319 | - |> Enum.drop_while(fn id -> id != html_node.node_id end) |
| 320 | - |> tl() |
| 321 | - end |
| 322 | - |
| 323 344 | defp get_siblings(html_node, tree) do |
| 324 345 | parent = get_node(html_node.parent_node_id, tree) |
| 325 346 | |
| 326 347 | ids = |
| 327 348 | if parent do |
| 328 | - get_sibling_ids_from(parent.children_nodes_ids, html_node) |
| 349 | + parent.children_nodes_ids |
| 329 350 | else |
| 330 | - get_sibling_ids_from(Enum.reverse(tree.root_nodes_ids), html_node) |
| 351 | + tree.root_nodes_ids |
| 331 352 | end |
| 332 353 | |
| 333 | - Enum.filter(ids, fn id -> |
| 334 | - case get_node(id, tree) do |
| 335 | - %HTMLNode{} -> true |
| 336 | - _ -> false |
| 337 | - end |
| 338 | - end) |
| 354 | + get_sibling_nodes(ids, html_node.node_id, tree, []) |
| 339 355 | end |
| 340 356 | |
| 357 | + defp get_sibling_nodes([id | _], id, _tree, acc), do: acc |
| 358 | + |
| 359 | + defp get_sibling_nodes([id | rest], target_id, tree, acc) do |
| 360 | + acc = |
| 361 | + case get_node(id, tree) do |
| 362 | + %HTMLNode{} -> [id | acc] |
| 363 | + _ -> acc |
| 364 | + end |
| 365 | + |
| 366 | + get_sibling_nodes(rest, target_id, tree, acc) |
| 367 | + end |
| 368 | + |
| 369 | + defp get_sibling_nodes([], _target_id, _tree, _acc), do: [] |
| 370 | + |
| 341 371 | # finds all descendant node ids recursively through the tree preserving the order |
| 342 372 | defp get_descendant_ids(node_id, tree) do |
| 343 373 | case get_node(node_id, tree) do |
| 344 | - %{children_nodes_ids: node_ids} -> |
| 345 | - reversed_ids = Enum.reverse(node_ids) |
| 346 | - reversed_ids ++ Enum.flat_map(reversed_ids, &get_descendant_ids(&1, tree)) |
| 374 | + %{children_nodes_ids: children} when children != [] -> |
| 375 | + do_get_descendant_ids(Enum.reverse(children), tree, []) |
| 376 | + |> Enum.reverse() |
| 347 377 | |
| 348 378 | _ -> |
| 349 379 | [] |
| 350 380 | end |
| 351 381 | end |
| 352 382 | |
| 383 | + defp do_get_descendant_ids([], _tree, acc), do: acc |
| 384 | + |
| 385 | + defp do_get_descendant_ids([node_id | rest], tree, acc) do |
| 386 | + acc = [node_id | acc] |
| 387 | + |
| 388 | + acc = |
| 389 | + case get_node(node_id, tree) do |
| 390 | + %{children_nodes_ids: children} when children != [] -> |
| 391 | + do_get_descendant_ids(Enum.reverse(children), tree, acc) |
| 392 | + |
| 393 | + _ -> |
| 394 | + acc |
| 395 | + end |
| 396 | + |
| 397 | + do_get_descendant_ids(rest, tree, acc) |
| 398 | + end |
| 399 | + |
| 353 400 | @spec map(Floki.html_tree() | Floki.html_node(), function()) :: |
| 354 401 | Floki.html_tree() | Floki.html_node() |
Loading more files…