RSS Funnel

This project is in the early stage of development. Please expect bugs and frequent breaking changes.

RSS Funnel is a modular RSS processing pipeline system. It can manipulate RSS/Atom sources in various ways.

• Fetch full-text content
• Generate an RSS feed from an HTML page
• Remove unwanted elements or texts
• Filter articles matching keywords
• Highlight keywords
• Split aggregation (e.g. top 10 links of day) into individual articles
• Combine multiple feeds into one
• Convert between feed formats
• Run arbitrary JavaScript transformation
• Apply image proxy
• And more!

The tool comes with a web UI for inspecting the feed and the result of each filter, all within a single statically-built binary with no external dependencies.

Try out the live demo!

Installation

You can use the Docker image (latest version) in your docker-compose.yaml:

version: "3.8"
services:
  rss-funnel:
    image: ghcr.io/shouya/rss-funnel:latest
    ports:
      - 4080:4080
    volumes:
      - ./funnel.yaml:/funnel.yaml
    environment:
      RSS_FUNNEL_CONFIG: /funnel.yaml
      RSS_FUNNEL_BIND: 0.0.0.0:4080

The web UI will be available at http://localhost:4080.

Alternatively, you can build it directly from source:

git clone https://github.com/shouya/rss-funnel.git
cd rss-funnel

# First, build the front-end assets
cd inspector && npm i && npm run build && cd ..

# Then, build the binary
cargo build --release

If you prefer not to build from source, you can download the pre-built artifacts from the release page.

Usage

To use rss-funnel, you need to supply a configuration file in YAML format. Here’s an example configuration:

endpoints:
  - path: /tokio-blog.xml
    note: Full text of Tokio blog
    source: https://tokio.rs/_next/static/feed.xml
    filters:
      - full_text: {}
      - simplify_html: {}

  - path: /hackernews.xml
    note: Full text of Hacker News
    source: https://news.ycombinator.com/rss
    filters:
      - full_text:
          simplify: true
          append_mode: true

Save the above file as /path/to/funnel.yaml and run the following command:

rss-funnel -c /path/to/funnel.yaml server

You can optionally specify the bind address and port (default 127.0.0.1:4080). Detailed usage can be found in the --help output.

The endpoints like http://127.0.0.1:4080/tokio-blog.xml should serve the filtered feeds.

Documentation

• Config syntax contains information about each component in the configuration file.
• Filter config details the configuration for all filters.
• Cookbook contains snippets on how to use rss-funnel for several practical applications.
• Deployment includes examples of docker-compose and kubernetes definitions.