crawly

0.7.0

0.17.2 0.17.1 0.17.0 0.16.0 0.15.0 0.14.0 0.13.0 0.12.0 0.11.0 0.10.0 0.9.0 0.8.0 0.7.0 0.6.0 0.5.0 0.4.0 0.3.0 0.2.0 0.1.2 0.1.1 0.1.0

High-level web crawling & scraping framework for Elixir.

HexDocs

Current section

Files

Jump to

Readme 21 Versions 6 Dependencies Dependants Files Activity

crawly lib crawly request.ex

Raw

lib/crawly/request.ex

defmodule Crawly.Request do
  @moduledoc """
  Request wrapper

  Defines Crawly request structure.
  """
  defstruct url: nil, headers: [], prev_response: nil, options: []

  @type header() :: {key(), value()}
  @typep key :: binary()
  @typep value :: binary()

  @type option :: {atom(), binary()}

  @type t :: %__MODULE__{
    url: binary(),
    headers: [header()],
    prev_response: %{},
    options: [option()]
  }
end

Search filters

Type any of these into the search box. They can be combined with free text which searches package names and descriptions.

Operator	Description	Example
name:	Match package name. Supports * wildcards and repo/package form	name:phx* or name:hexpm/phoenix
description:	Full-text search of package descriptions	description:auth
depends:	Packages depending on a given package. Supports repo:package form	depends:ecto or depends:hexpm:ecto
build_tool:	Filter by build tool	build_tool:mix
updated_after:	Packages updated after an ISO8601 datetime	updated_after:2025-01-01T00:00:00Z
extra:	Match custom metadata (key,value). Nested keys are separated by commas	extra:license,MIT

Wrap values in double quotes to include spaces, e.g. name:"my package".