Packages

crawly

0.2.0
0.17.2 0.17.1 0.17.0 0.16.0 0.15.0 0.14.0 0.13.0 0.12.0 0.11.0 0.10.0 0.9.0 0.8.0 0.7.0 0.6.0 0.5.0 0.4.0 0.3.0 0.2.0 0.1.2 0.1.1 0.1.0

High-level web crawling & scraping framework for Elixir.

HexDocs

Current section

Files

Jump to
Readme
21 Versions
7 Dependencies
Dependants
Files
Activity
crawly lib crawly pipelines json_encoder.ex
Raw

lib/crawly/pipelines/json_encoder.ex 

defmodule Crawly.Pipelines.JSONEncoder do

  @moduledoc """

  Encodes a given item (map) into JSON

  """

  @behaviour Crawly.Pipeline



  require Logger



  @impl Crawly.Pipeline

  def run(item, state) do

    case Poison.encode(item) do

      {:ok, new_item} ->

        {new_item, state}



      {:error, reason} ->

        Logger.info(

          "Could not encode the following item: #{inspect(item)} into json,

          reason: #{inspect(reason)}"

        )



        {false, state}

    end

  end

end