Packages

crawler

0.0.0

1.5.0 1.4.0 1.3.0 1.2.0 1.1.2 1.1.1 1.1.0 1.0.0 0.4.0 0.3.1 0.3.0 0.2.0 0.1.0 0.0.0

A high performance web crawler in Elixir.

HexDocs

Current section

Files

Jump to

Readme 14 Versions 2 Dependencies Dependants Files Activity

crawler README.md

Raw

5 files

lib
- crawler
  
  supervisor.ex
  
  worker.ex
- crawler.ex
README.md
mix.exs

README.md

# Crawler [![Travis](https://img.shields.io/travis/fredwu/crawler.svg)](https://travis-ci.org/fredwu/crawler) [![Hex.pm](https://img.shields.io/hexpm/v/crawler.svg)](https://hex.pm/packages/crawler)

A high performance web crawler in Elixir.

## Usage

```elixir
Crawler.crawl("http://elixir-lang.org")
```

## License

Licensed under [MIT](http://fredwu.mit-license.org/).

Search filters

Type any of these into the search box. They can be combined with free text which searches package names and descriptions.

Operator	Description	Example
name:	Match package name. Supports * wildcards and repo/package form	name:phx* or name:hexpm/phoenix
description:	Full-text search of package descriptions	description:auth
depends:	Packages depending on a given package. Supports repo:package form	depends:ecto or depends:hexpm:ecto
build_tool:	Filter by build tool	build_tool:mix
updated_after:	Packages updated after an ISO8601 datetime	updated_after:2025-01-01T00:00:00Z
extra:	Match custom metadata (key,value). Nested keys are separated by commas	extra:license,MIT

Wrap values in double quotes to include spaces, e.g. name:"my package".