All notable changes to this project will be documented in this file.
Generated by Changy.
No changes.
-
ff-155 — Performance improvements.
- use orjson instead of standard json lib for response serialization;
- changed response format of
/api/get-last-entries
and/api/get-entries-by-ids
API to reduce the size of the response and speed up its generation.
- Track only user metrics with values != 0.
In this release, we added a bunch of tracking functionality to better understand how users interact with the app.
This analytics:
- is strongly for our internal use;
- uses only internal IDs — does not contain emails, urls, etc.
- in most cases does not exceed what we already have in logs and database — just a more convenient way to access the data.
Changes:
- Frontend dependencies upgraded.
- ff-141 — Track frontend events
news_link_opened
,news_body_opened
. - ff-103 — Added
in_collection
marker tofeed_linked
andfeed_unlinked
events. - ff-142 — CLI command to track aggregated metrics.
- ff-143 — Fixed a situation when, after removing a feed on the Feeds page, the feeds list was not updated.
- ff-145 — Track users' interest in feeds.fun social channels.
- ff-119 — Track OPML imports.
- ff-112 — Track subscriptions on feeds from collections.
- ff-131 — Track subscription dynamics on feeds.fun news channels.
- Added normalization to business event attributes.
- Updated frontend packages.
- ff-70 — Business events are written in logs.
- ff-68 — Tech metrics are written in logs. They can be filtered and processed by tools such as SplitBit or Vector and redirected to monitoring systems. Note that metrics are written at the INFO log level—if a higher log level is set, they will be hidden with all INFO logs.
- Changed the environment variables prefix for logs settings from
FFUN_
toFFUN_LOGS_
.
- Mark all ssl errors catched by httpx as expected.
- Connection error
[SSL: TLSV1_UNRECOGNIZED_NAME]
added to the list of known "ok" errors for the feeds loader.
- Connection error
[SSL: CERTIFICATE_VERIFY_FAILED]
added to the list of known "ok" errors for the feeds loader.
- FF-56 — Connection error
[SSL: WRONG_VERSION_NUMBER]
added to the list of known "ok" errors for the feeds loader. - FF-51 — "No description" text for feeds without a description in the Feeds list.
- FF-52 — Fixed the
remove-deprecated-settings
command, now it really removes only deprecated settings. - FF-50 — Unsubscribing from a feed now do not lead to excessive server requests due to cache invalidation.
- Removed dependency on
pkg_resources
.
Operations required AFTER UPDATE:
- Run migrations
ffun migrate
- Run
ffun user-settings remove-deprecated-settings
- If you are using an OpenAI tag processor, specify a custom configuration file for tag processors with an API key. Details can be found in the updated README.md
- If you want to periodically clean your database from old entries, add the call
ffun cleaner clean
to your cron tasks. It is recommended.
Changes:
- gh-229 — Removed
Allow using OpenAI key for feeds in standard collections
user settings. It was confusing for users and not usable in the project's current state. Instead, you can specify an LLM API key to process collections (one per tag processor). By new logic, the service's owner is responsible for managing tags for all feeds in collections. - gh-245 — Implemented the universal LLM tag processor. Currently, it supports two API providers: OpenAI (ChatGPT) and Google (Gemini). Also:
- The configuration of tag processors was moved to a separate file; see README.md for actual details.
- The configuration of LLM models was moved to a separate file; see README.md for actual details.
- You can configure multiple LLM tag processors with different parameters, including prompts, API keys, etc.
- Token usage tracking has been replaced with cost tracking. User settings will convert automatically.
- Collections refactored. Now, they are configurable and empty by default. You can specify the path to the directory with collections configuration in
FFUN_FEEDS_COLLECTIONS_COLLECTION_CONFIGS
. An example of configuration can be found in the./feeds_collections/collections
. FFUN_META_MAX_ENTRIES_PER_FEED
setting moved toFFUN_LIBRARY_MAX_ENTRIES_PER_FEED
.
- gh-136 — Links to the Discord server and to the Reddit community.
- gh-24 — Link to the blog. Blog RSS is added to collections.
- gh-220 Removed Home link from internal pages.
- gh-221 Slighly increased width of the central column on the main page.
- gh-241 Optional plausible analytics for the frontend.
- gh-41 Build dev service images only once.
- gh-231 CLI tool and base of texts to measure quality of tag processors.
- gh-233 Improved prompt of OpenAI General processor. It should start to output more relevant tags.
- gh-234 (possibly) Fixed deadlocks in updating tags properties by
apply_tags_properties
.
- gh-231 CLI tool and base of texts to measure quality of tag processors.
- gh-233 Improved prompt of OpenAI General processor. It should start to output more relevant tags.
- gh-234 (possibly) Fixed deadlocks in updating tags properties by
apply_tags_properties
.
Configs migration: if you have turned on OpenAI ChatGPT processors, like FFUN_LIBRARIAN_OPENAI_CHAT_35_FUNCTIONS_PROCESSOR__ENABLED="True"
or FFUN_LIBRARIAN_OPENAI_CHAT_35_PROCESSOR__ENABLED="True", remove their configs and add configs for OpenAI General Processor, like
FFUN_LIBRARIAN_OPENAI_GENERAL_PROCESSOR__ENABLED="True"`, instead. The semantic of the configs is the same.
- gh-227 — Old OpenAI ChatGPT processors is replaced with OpenAI General Processor which utilizes power of GPT-4o-mini model.
Recommendation: Because GPT-4o-mini
is much smarter than GPT-3.5-turbo
, I recommend you to disable GPT function processor FFUN_LIBRARIAN_OPENAI_CHAT_35_FUNCTIONS_PROCESSOR__ENABLED="False"
and use only standard GPT processor FFUN_LIBRARIAN_OPENAI_CHAT_35_PROCESSOR__ENABLED="False"
. In gh-227, one of the processors will be refactored, and one of them will be removed.
- gh-226 Switch used OpenAI model to gpt4o-mini
DB migration required: no
- gh-217 Fixed:
apply_tags_properties
fails because of duplicated tags.
DB migration required: yes
- gh-214 Added not null constraint to the
o_tags_properties.tag_id
column. - gh-213 Fixed:
Native tags
tags processor periodically fails with deadlock in the database.
Breaking changes:
- gh-68 Supertokens services upgraded to version 9.0.2. If you are using Supertokens, you should upgrade them manually (from 5.0.0 to 9.0.2) according to the instructions in https://github.com/supertokens/supertokens-core/blob/master/CHANGELOG.md
Changes: