GitLab/Abuse and rate limiting

This page summarizes the available tools against abuse and rate limiting bots/scraping.

Automatically created accounts added SEO spam to open merge requests and issues (see T334958). As a follow up all new accounts need a approval first (using this Phabricator form).

Furthermore GitLab accounts can be blocked in the admin menu. A cascading block for blocked wikitech accounts is work in progress: T359820. For obvious spam or vandalism it is also possible to "Delete user and contributions".

GitLab is not behind the load balancing infrastructure, so tools like superset are not available to analyze traffic. However there are some Logstash dashboards which can be used to analyze the traffic:

Access logs dashboard: https://logstash.wikimedia.org/app/dashboards#/view/38dc4c50-7cae-11ef-b061-47f1245f2864

Application logs: https://logstash.wikimedia.org/goto/82d8dd92e36776ce581aeef953ff6891

Even though GitLab is not behind the CDN some throttling and rate limiting capabilities were added. In T366882 a new Puppet module profile::firewall::nftables_throttling was implemented which allows basic rate limiting based on a maximum number of parallel TCP connections.

If the threshold is exceeded the offending IP address is blocked for 5 minutes. More information can be found in Firewall#Throttling with nftables.

Metrics for this throttling can be found in Grafana.

Requestctl has a feature to ban single IPs or IP ranges with the abuse/blocked_nets ipblock. This IPs are dropped automatically on all wmf hosts using firewall rules. This is a simple and effective measure to ban a IP with little work.

For GitLab this feature is currently not working because of the recent migration to nftables. See T348734. But there is ongoing work to implement this feature also for nftables.

GitLab has an option to inject a custom apache config using the GitLab config file, see insert-custom-nginx-settings-into-the-gitlab-server-block. This is also available in our Puppet code as profile::gitlab::custom_nginx_config. So in case a certain API endpoint or path is causing trouble, this option can be used to restrict access.