scrape

1.0.1

3.1.0 3.0.3 3.0.2 3.0.1 3.0.0 2.0.0 1.2.7 1.2.6 1.2.5 1.2.4 1.2.3 1.2.2 1.2.1 1.0.4 1.0.3 1.0.2 1.0.1 1.0.0 0.2.0 0.1.1 0.1.0 0.0.4 0.0.3 0.0.2 0.0.1

Scrape any website, article or RSS/Atom feed with ease!

HexDocs

Current section

Files

Jump to

Readme 25 Versions 5 Dependencies Dependants Files Activity

scrape lib scrape.ex

Raw

lib/scrape.ex

defmodule Scrape do
  alias Scrape.Fetch
  alias Scrape.Feed
  alias Scrape.Website
  alias Scrape.Article

  def feed(url) do
    url
    |> Fetch.run
    |> Feed.parse(url)
  end

  def website(url) do
    url
    |> Fetch.run
    |> Website.parse(url)
  end

  def article(url) do
    html = Fetch.run url
    website = Website.parse(html, url)
    Article.parse(website, html)
  end

end

Search filters

Type any of these into the search box. They can be combined with free text which searches package names and descriptions.

Operator	Description	Example
name:	Match package name. Supports * wildcards and repo/package form	name:phx* or name:hexpm/phoenix
description:	Full-text search of package descriptions	description:auth
depends:	Packages depending on a given package. Supports repo:package form	depends:ecto or depends:hexpm:ecto
build_tool:	Filter by build tool	build_tool:mix
updated_after:	Packages updated after an ISO8601 datetime	updated_after:2025-01-01T00:00:00Z
extra:	Match custom metadata (key,value). Nested keys are separated by commas	extra:license,MIT

Wrap values in double quotes to include spaces, e.g. name:"my package".