poly_post

# PolyPost [![Hex Version](https://img.shields.io/hexpm/v/poly_post)](https://hex.pm/packages/poly_post) [![Hex Docs](https://img.shields.io/badge/docs-hexpm-blue.svg)](https://hexdocs.pm/poly_post/)

A publishing engine with markdown and code highlighting support.

## Features

* Supports markdown via a configurable encoder (tested with [MDEx](https://hex.pm/packages/mdex))
* Supports structured frontmatter metadata via a configurable decoder (tested with [JSON](https://elixir.hexdocs.pm/1.19.5/JSON.html), [Jason](https://hex.pm/packages/jason) and [YamlElixir](https://hex.pm/packages/yaml_elixir))
* Loads files directly from configured paths and from Git repositories
* Stores content in process-owned ETS tables
* Uses multiple directories with glob patterns that can be specified as different resources
* Optional code highlighting support using [Lumis](https://hex.pm/packages/lumis)
* Updating of content with no downtime

## Requirements

* Elixir 1.19 or greater
* Erlang OTP 28 or greater
* Git
* A mostly POSIX compatible environment (linux, darwin, bsd, etc.)

## Installation

Add `poly_post` to your list of dependencies in `mix.exs` and include
to configure any decoders you want to use:

```elixir
def deps do
  [
    {:poly_post, "~> 0.2"},
    {:mdex, "~> 0.13"}, # Default markdown
    {:lumis, "~> 0.8"}, # Default highlighting support through mdex
    {:jason, "~> 1.4"}, # Optional dependency for JSON front matter or use built-in JSON
    {:yaml_elixir, "~> 2.11"} # Optional dependency for YAML front matter
  ]
end
```

Optionally, if you want to support syntax highlighting, then add the
following to your `config/config.exs`:

```elixir
config :mdex_native, syntax_highlighter: :lumis
```

Then run `mix deps.get` and `mix deps.compile` or just a `mix compile` in your app.

## Configuration

There are two strategies for configuring content: `paths` and `git`.

### For paths

With the following environment variable, you can set your glob pattern:

```
export ARTICLE_PATH=/path/to/my/markdown/*.md
```

In the `config/runtime.exs` files you can configure the front matter
decoder and each resource for your content like so:

```elixir
config :poly_post, :resources,
  markdown_config: [
    decoder: {
      MDEx,
      :to_html!,
      [
        extension: [table: true],
        syntax_highlight: [
          engine: :lumis,
          opts: [
            formatter: {:html_inline, theme: "github_dark"}
          ]
        ]
      ]
    }
  ],
  frontmatter_config: [decoder: {Jason, :decode, keys: :atoms}],
  content: [
    articles: [
      module: Article,
      path: System.get_env("ARTICLE_PATH")
    ]
  ]
```

This elaborate example forwards several options to an internal call
within `poly_post `that executes `MDEx.html!(some_content, opts)` to
encode the markdown into HTML.

The frontmatter will be parsed out into some relevant structure using
`Jason` and will use its options to give the any objects that are
parsed to have keys that are atomized.

It retrievs all the original markdown from wherever you specify the
path pattern specified via `path`.

#### Using Jason for JSON

This example will use the [Jason](https://github.com/michalmuskala/jason)
parser to parse the front matter as JSON. You can use any format that
you want that confirms to the following API:

1. The decoder must take 1 or 2 arguments (the first will be content and the second would be options)
2. The decoder must return the following tuples:
```elixir
{:ok, content}
{:error, error}
```
3. The front matter begins and ends with a `---`

You can also specify different formats at the individual content level:

```elixir
config :poly_post, :resources,
  markdown_config: [...],
  frontmatter_config: [decoder: {Jason, :decode, keys: :strings}],
  content: [
    articles: [
      module: Article,
      path: System.get_env("ARTICLE_PATH"),
      frontmatter_config: [decoder: {JSON, :decode}]
    ]
  ]
```

### For git

Your environment **MUST** have `git` installed for this to work.

This is similar to the paths strategy, but you need to specify a
`source` key as well:

```elixir
config :poly_post, :resources,
  markdown_config: [...],
  frontmatter_config: [decoder: {YamlElixir, :read_from_string, []}],
  content: [
    articles: [
      module: Article,
      source: [
        dest: System.get_env("SOURCE_PATH"),
        github: "my-username/my-content",
        ref: "main"
      ],
      path: System.get_env("CONTENT_PATH")
    ]
  ]
```


* `dest` - (required)is the folder that git will clone to.
* `github` - (required if not using `git` config) to access a github repo, expands to `https://github.com/my-username/my-content.git`
* `git` - (required if not using `github` config) to access a git repo, e.g `https://git.mydomain.com/repo.git` or can be local
* `ref` - (optional) - the specified branch to use, defaults to whatever the default branch on the repo, usually `main` or `master`

This implementation doesn't manage authentication if you are accessing
a private repo, you must ensure the user that runs your application
has read access to your git repo. This library uses `System.cmd` to
access git.

If this is a security concern for you, it is recommended that you use
the `path` strategy and use some other mechanism to retrieve the
contents into your environment.

## Basic Usage

### Loading and Storing Content

With a file called `my_article1.md` in the configured directory with
YAML front matter:

```markdown
---
title: "My Article #1",
author: "Me"
---
## My Article 1

This is my first article
```

You can create an `Article` module to load your content by
implementing the "PolyPost.Resource.build/3" callback:

```elixir
defmodule Article do
  @behaviour PolyPost.Resource

  @enforce_keys [:key, :title, :author, :body]
  defstruct [:key, :title, :author, :body]

  # Callbacks

  @impl PolyPost.Resource
  def build(reference, metadata, body) do
    %__MODULE__{
      key: reference,
      title: get_in(metadata, ["title"]),
      author: get_in(metadata, ["author"]),
      body: body
    }
  end
end
```

The only requirement is that the struct or map **MUST** contain a key
called `key` that uniquely identifies this content. It **MUST** be a
`String`.

There are 3 function you must use to get, process and store content:

1. `PolyPost.fetch/1`
1. `PolyPost.build/1`
1. `PolyPost.store/2`

And then use them like so in a happy path:

```
# Fetch
{:ok, raw_articles} = PolyPost.fetch(:articles)

# Build
{articles, _failed} = raw_articles
|> Enum.map(&PolyPost.build/1)
|> Enum.split_with(&(elem(&1, 0) == :ok))

# Aaaaaaaand Store!
Enum.each(articles, fn {:ok, resource_name, content} ->
  PolyPost.store(resource_name, content)
end)
```

The reason the functionality is split this way is because sometimes
things go wrong when retrieving, building and even storing content
from remote sources with foreign, unknown content.

This gives an opportunity to debug to recover and then make whatever
remedial decision neededed without having to dig back into the
library.

See the main `PolyPost` API for details.

### Retrieving Content

You can retrieve content using the functions on the `PolyPost.Depot`
module to access the associated ETS table that stores your data:

1. `find/2` - find a specific content by `key` for the resource
2. `get_all/1` - gets all content for a resource

For example:

```elixir
PolyPost.Depot.find(:articles, "my_article1.md")
=> %Article{...}
```

and

```elixir
PolyPost.Depot.get_all(:articles)
=> [%Article{...}]
```

### Using Lumis to Style Code Blocks

If you wish to use [Lumis](https://hex.pm/packages/lumis) to style
your `code` blocks, you must specify the needed dependencies in your
`mix.exs` file as specified above and specify the `markdown_config`
with the appropriate options to enable syntax highlighting.

If using `MDEx`, this is specified as follows:

```elixir
config :poly_post, :resources,
  markdown_config: [
    decoder: {
      MDEx,
      :to_html!,
      [
        extension: [table: true],
        syntax_highlight: [
          engine: :lumis,
          opts: [
            formatter: {:html_inline, theme: "github_dark"}
          ]
        ]
      ]
    }
  ],
 ...
```

Then in your markdown content you can specify things like this and it
should get highlighted properly:

````markdown
```elixir
def start_link(arg) do
  GenServer.start_link(__MODULE__, arg, name: __MODULE__)
end
```
````


## Differences from NimblePublisher

This library was heavily inspired by [NimblePublisher](https://github.com/dashbitco/nimble_publisher), but
it **IS** different.

* Designed to be updated at runtime via calling refresh methods
* Must be configured through `Application` config using `:poly_post`
* Stores content in ETS instead of compiling directly into modules

## License

This software is licensed under the [Apache-2.0 License](LICENSE).