Current section
Files
Jump to
Current section
Files
doc/readme.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="ExDoc v0.31.0">
<meta name="project" content="readability v0.12.1">
<title>Overview — readability v0.12.1</title>
<link rel="stylesheet" href="dist/html-elixir-3ZLIM5JF.css" />
<script src="dist/handlebars.runtime-NWIB6V2M.js"></script>
<script src="dist/handlebars.templates-43PMFBC7.js"></script>
<script src="dist/sidebar_items-C53A1FF7.js"></script>
<script src="docs_config.js"></script>
<script async src="dist/html-CILPWI2Q.js"></script>
</head>
<body data-type="extras" class="page-extra">
<script>
try {
var settings = JSON.parse(localStorage.getItem('ex_doc:settings') || '{}');
if (settings.theme === 'dark' ||
((settings.theme === 'system' || settings.theme == null) &&
window.matchMedia('(prefers-color-scheme: dark)').matches)
) {
document.body.classList.add('dark')
}
} catch (error) { }
</script>
<div class="main">
<button class="sidebar-button sidebar-toggle" aria-label="toggle sidebar" aria-controls="sidebar">
<i class="ri-menu-line ri-lg" title="Collapse/expand sidebar"></i>
</button>
<nav id="sidebar" class="sidebar">
<div class="sidebar-header">
<div class="sidebar-projectInfo">
<div>
<a href="readme.html" class="sidebar-projectName" translate="no">
readability
</a>
<div class="sidebar-projectVersion" translate="no">
v0.12.1
</div>
</div>
</div>
<ul id="sidebar-listNav" class="sidebar-listNav" role="tablist">
<li>
<button id="extras-list-tab-button" role="tab" data-type="extras" aria-controls="extras-tab-panel" aria-selected="true" tabindex="0">
Pages
</button>
</li>
<li>
<button id="modules-list-tab-button" role="tab" data-type="modules" aria-controls="modules-tab-panel" aria-selected="false" tabindex="-1">
Modules
</button>
</li>
</ul>
</div>
<div id="extras-tab-panel" class="sidebar-tabpanel" role="tabpanel" aria-labelledby="extras-list-tab-button">
<ul id="extras-full-list" class="full-list"></ul>
</div>
<div id="modules-tab-panel" class="sidebar-tabpanel" role="tabpanel" aria-labelledby="modules-list-tab-button" hidden>
<ul id="modules-full-list" class="full-list"></ul>
</div>
</nav>
<main class="content">
<output role="status" id="toast"></output>
<div class="content-outer">
<div id="content" class="content-inner">
<div class="top-search">
<div class="search-settings">
<form class="search-bar" action="search.html">
<label class="search-label">
<span class="sr-only">Search documentation of readability</span>
<input name="q" type="text" class="search-input" placeholder="Search Documentation (press /)" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false" />
</label>
<button type="submit" class="search-button" aria-label="Submit Search">
<i class="ri-search-2-line ri-lg" aria-hidden="true" title="Submit search"></i>
</button>
<button type="button" tabindex="-1" class="search-close-button" aria-hidden="true">
<i class="ri-close-line ri-lg" title="Cancel search"></i>
</button>
</form>
<button class="icon-settings display-settings">
<i class="ri-settings-3-line"></i>
<span class="sr-only">Settings</span>
</button>
</div>
</div>
<div class="autocomplete">
</div>
<h1>
<a href="https://github.com/keepcosmos/readability/blob/main/README.md#L1" title="View Source" class="icon-action" rel="help">
<i class="ri-code-s-slash-line" aria-hidden="true"></i>
<span class="sr-only">View Source</span>
</a>
<span>Readability</span>
</h1>
<p><a href="https://github.com/keepcosmos/readability/actions/workflows/elixir.yml"><img src="https://github.com/keepcosmos/readability/actions/workflows/elixir.yml/badge.svg" alt="CI"/></a>
<a href="https://hex.pm/packages/readability"><img src="https://img.shields.io/hexpm/v/readability.svg" alt="Module Version"/></a>
<a href="https://hexdocs.pm/readability/"><img src="https://img.shields.io/badge/hex-docs-lightgreen.svg" alt="Hex Docs"/></a>
<a href="https://hex.pm/packages/readability"><img src="https://img.shields.io/hexpm/dt/readability.svg" alt="Total Download"/></a>
<a href="https://github.com/keepcosmos/readability/blob/master/LICENSE.md"><img src="https://img.shields.io/hexpm/l/readability.svg" alt="License"/></a>
<a href="https://coveralls.io/github/keepcosmos/readability?branch=master"><img src="https://coveralls.io/repos/github/keepcosmos/readability/badge.svg?branch=master" alt="Coverage Status"/></a></p><p>Readability is a tool for extracting and curating the primary readable content of a webpage.</p><h2 id="installation" class="section-heading">
<a href="#installation" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Installation</span>
</h2>
<p>The package can be installed as:</p><p>Add <code class="inline">:readability</code> to your list of dependencies in <code class="inline">mix.exs</code>:</p><pre><code class="makeup elixir" translate="no"><span class="kd">def</span><span class="w"> </span><span class="nf">deps</span><span class="w"> </span><span class="k" data-group-id="1080104233-1">do</span><span class="w">
</span><span class="p" data-group-id="1080104233-2">[</span><span class="w">
</span><span class="p" data-group-id="1080104233-3">{</span><span class="ss">:readability</span><span class="p">,</span><span class="w"> </span><span class="s">"~> 0.12"</span><span class="p" data-group-id="1080104233-3">}</span><span class="w">
</span><span class="p" data-group-id="1080104233-2">]</span><span class="w">
</span><span class="k" data-group-id="1080104233-1">end</span></code></pre><p>After that, run mix deps.get.</p><p>Note: Readability requires Elixir 1.10 or higher.</p><h2 id="usage" class="section-heading">
<a href="#usage" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Usage</span>
</h2>
<h3 id="examples" class="section-heading">
<a href="#examples" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Examples</span>
</h3>
<h4>Just pass a url</h4><pre><code class="makeup elixir" translate="no"><span class="n">url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"https://medium.com/@kenmazaika/why-im-betting-on-elixir-7c8f847b58"</span><span class="w">
</span><span class="n">summary</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nc">Readability</span><span class="o">.</span><span class="n">summarize</span><span class="p" data-group-id="4023939849-1">(</span><span class="n">url</span><span class="p" data-group-id="4023939849-1">)</span><span class="w">
</span><span class="n">summary</span><span class="o">.</span><span class="n">title</span><span class="w">
</span><span class="c1">#=> "Why I’m betting on Elixir"</span><span class="w">
</span><span class="n">summary</span><span class="o">.</span><span class="n">authors</span><span class="w">
</span><span class="c1">#=> ["Ken Mazaika"]</span><span class="w">
</span><span class="n">summary</span><span class="o">.</span><span class="n">article_html</span><span class="w">
</span><span class="c1">#=></span><span class="w">
</span><span class="c1"># <div><div><p id=\"3476\"><strong><em>Background: </em></strong><em>I’ve spent...</span><span class="w">
</span><span class="c1"># ...</span><span class="w">
</span><span class="c1"># ...button!</em></h3></div></div></span><span class="w">
</span><span class="n">summary</span><span class="o">.</span><span class="n">article_text</span><span class="w">
</span><span class="c1">#=></span><span class="w">
</span><span class="c1"># Background: I’ve spent the past 6 years building web applications in Ruby and.....</span><span class="w">
</span><span class="c1"># ...</span><span class="w">
</span><span class="c1"># ... value in this article, it would mean a lot to me if you hit the recommend button!</span></code></pre><h4>From raw html</h4><pre><code class="makeup elixir" translate="no"><span class="c1">### Extract the title.</span><span class="w">
</span><span class="nc">Readability</span><span class="o">.</span><span class="n">title</span><span class="p" data-group-id="4044657551-1">(</span><span class="n">html</span><span class="p" data-group-id="4044657551-1">)</span><span class="w">
</span><span class="c1">### Extract authors.</span><span class="w">
</span><span class="nc">Readability</span><span class="o">.</span><span class="n">authors</span><span class="p" data-group-id="4044657551-2">(</span><span class="n">html</span><span class="p" data-group-id="4044657551-2">)</span><span class="w">
</span><span class="c1">### Extract the primary content with transformed html.</span><span class="w">
</span><span class="n">html</span><span class="w">
</span><span class="o">|></span><span class="w"> </span><span class="nc">Readability</span><span class="o">.</span><span class="n">article</span><span class="w">
</span><span class="o">|></span><span class="w"> </span><span class="nc">Readability</span><span class="o">.</span><span class="n">readable_html</span><span class="w">
</span><span class="c1">### Extract only text from the primary content.</span><span class="w">
</span><span class="n">html</span><span class="w">
</span><span class="o">|></span><span class="w"> </span><span class="nc">Readability</span><span class="o">.</span><span class="n">article</span><span class="w">
</span><span class="o">|></span><span class="w"> </span><span class="nc">Readability</span><span class="o">.</span><span class="n">readable_text</span><span class="w">
</span><span class="c1">### you can extract the primary images with Floki</span><span class="w">
</span><span class="n">html</span><span class="w">
</span><span class="o">|></span><span class="w"> </span><span class="nc">Readability</span><span class="o">.</span><span class="n">article</span><span class="w">
</span><span class="o">|></span><span class="w"> </span><span class="nc">Floki</span><span class="o">.</span><span class="n">find</span><span class="p" data-group-id="4044657551-3">(</span><span class="s">"img"</span><span class="p" data-group-id="4044657551-3">)</span><span class="w">
</span><span class="o">|></span><span class="w"> </span><span class="nc">Floki</span><span class="o">.</span><span class="n">attribute</span><span class="p" data-group-id="4044657551-4">(</span><span class="s">"src"</span><span class="p" data-group-id="4044657551-4">)</span></code></pre><h3 id="options" class="section-heading">
<a href="#options" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Options</span>
</h3>
<p>If the result is different from your expectations, you can add options to customize it.</p><h4>Example</h4><pre><code class="makeup elixir" translate="no"><span class="n">url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"https://medium.com/@kenmazaika/why-im-betting-on-elixir-7c8f847b58"</span><span class="w">
</span><span class="n">summary</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nc">Readability</span><span class="o">.</span><span class="n">summarize</span><span class="p" data-group-id="3602643599-1">(</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="p" data-group-id="3602643599-2">[</span><span class="ss">clean_conditionally</span><span class="p">:</span><span class="w"> </span><span class="no">false</span><span class="p" data-group-id="3602643599-2">]</span><span class="p" data-group-id="3602643599-1">)</span></code></pre><ul><li><code class="inline">:min_text_length</code> \\ 25</li><li><code class="inline">:remove_unlikely_candidates</code> \\ true</li><li><code class="inline">:weight_classes</code> \\ true</li><li><code class="inline">:clean_conditionally</code> \\ true</li><li><code class="inline">:retry_length</code> \\ 250</li></ul><p><strong>You can find other algorithm and regex options in <code class="inline">readability.ex</code></strong></p><h2 id="test" class="section-heading">
<a href="#test" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Test</span>
</h2>
<p>To run the test suite:</p><pre><code class="makeup shell" translate="no"><span class="gp unselectable">$ </span><span class="">mix test
</span></code></pre><h2 id="todo" class="section-heading">
<a href="#todo" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Todo</span>
</h2>
<ul><li>[x] Extract authors</li><li>[x] More configurable</li><li>[x] Summarize function</li><li>[ ] Convert relative paths into absolute paths of <code class="inline">img#src</code> and <code class="inline">a#href</code></li></ul><h2 id="contributions-are-welcome" class="section-heading">
<a href="#contributions-are-welcome" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Contributions are welcome!</span>
</h2>
<p>Check out <a href="https://github.com/keepcosmos/readability/milestones">the main features milestone</a> and features of related projects below</p><p><strong>Contributing</strong></p><ol><li><strong>Fork</strong> the repo on GitHub</li><li><strong>Clone</strong> the project to your own machine</li><li><strong>Commit</strong> changes to your own branch</li><li><strong>Push</strong> your work back up to your fork</li><li>Submit a <strong>Pull request</strong> so that we can review your changes</li></ol><p>NOTE: Be sure to merge the latest from "upstream" before making a pull request!</p><h2 id="related-and-inspired-projects" class="section-heading">
<a href="#related-and-inspired-projects" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Related and Inspired Projects</span>
</h2>
<ul><li><a href="https://github.com/mozilla/readability">readability.js</a> is a standalone version of the readability library used for Firefox Reader View.</li><li><a href="https://github.com/codelucas/newspaper">newspaper</a> is an advanced news extraction, article extraction, and content curation library for Python.</li><li><a href="https://github.com/cantino/ruby-readability">ruby-readability</a> is a tool for extracting the primary readable content of a webpage.</li></ul><h2 id="copyright-and-license" class="section-heading">
<a href="#copyright-and-license" class="hover-link">
<i class="ri-link-m" aria-hidden="true"></i>
</a>
<span class="text">Copyright and License</span>
</h2>
<p>Copyright (c) 2016 Jaehyun Shin</p><p>Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at <a href="http://www.apache.org/licenses/LICENSE-2.0">http://www.apache.org/licenses/LICENSE-2.0</a></p><p>Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.</p>
<div class="bottom-actions">
<div class="bottom-actions-item">
<a href="license.html" class="bottom-actions-button" rel="prev">
<span class="subheader">
← Previous Page
</span>
<span class="title">
License
</span>
</a>
</div>
<div class="bottom-actions-item">
</div>
</div>
<footer class="footer">
<p>
<span class="line">
<a href="https://hex.pm/packages/readability/0.12.1" class="footer-hex-package">Hex Package</a>
<a href="https://preview.hex.pm/preview/readability/0.12.1">Hex Preview</a>
(<a href="https://preview.hex.pm/preview/readability/0.12.1/show/README.md">current file</a>)
</span>
<span class="line">
<button class="a-main footer-button display-quick-switch" title="Search HexDocs packages">
Search HexDocs
</button>
</span>
</p>
<p class="built-using">
Built using
<a href="https://github.com/elixir-lang/ex_doc" title="ExDoc" target="_blank" rel="help noopener" translate="no">ExDoc</a> (v0.31.0) for the
<a href="https://elixir-lang.org" title="Elixir" target="_blank" translate="no">Elixir programming language</a>
</p>
</footer>
</div>
</div>
</main>
</div>
</body>
</html>