fail2ban for nginx: ban the bots that keep knocking on /.env and /wp-login.php

Every public server gets the same knocks: WordPress login pages on sites that have never run WordPress, .env files, .git folders, and phpMyAdmin. Most of these come from scanners that never stop. The requests fail, but each one still costs a log line, some CPU and a bit of attention when you read the logs.

What the bots are looking for

• /wp-login.php, /xmlrpc.php and /phpmyadmin/: admin pages for software you may not run.
• /.env, /.git/config, /.aws/credentials: files that leak secrets when a deploy is careless.
• Anything that returns 403 or 404 over and over from the same address.

Install fail2ban

bash

sudo apt update
sudo apt install -y fail2ban
sudo systemctl enable --now fail2ban
sudo fail2ban-client status

Add a filter for dotfile probes

fail2ban ships a nginx-botsearch filter for common admin paths. Dotfiles need a small filter of their own. Save this as /etc/fail2ban/filter.d/nginx-dotfiles.conf:

ini

[Definition]
failregex = ^<HOST> .*"(?:GET|POST|HEAD) /\.(?:env|git|svn|aws)\S* HTTP/[^"]*" (?:403|404)
ignoreregex =

Turn on the jails

Put both jails in /etc/fail2ban/jail.d/nginx-bots.local. Five misses in ten minutes earns an hour;

ini

[nginx-botsearch]
enabled  = true
port     = http,https
filter   = nginx-botsearch
logpath  = /var/log/nginx/access.log
maxretry = 5
findtime = 10m
bantime  = 1h

[nginx-dotfiles]
enabled  = true
port     = http,https
filter   = nginx-dotfiles
logpath  = /var/log/nginx/access.log
maxretry = 3
findtime = 10m
bantime  = 24h

Test before it bans anyone

fail2ban-regex runs a filter against the real log and reports what it would match. Run it first, then reload.

bash

sudo fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/nginx-dotfiles.conf
sudo fail2ban-client reload
sudo fail2ban-client status nginx-dotfiles

Check a ban and undo it

When a real visitor gets caught, unban by address. 203.0.113.45 is a documentation address; use the one from your log.

bash

sudo fail2ban-client status nginx-botsearch
sudo fail2ban-client set nginx-botsearch unbanip 203.0.113.45

What it doesn't do

fail2ban only reacts after the first few requests. It won't stop a distributed scan where every IP sends one probe. For that, pair it with the path rules in the 444 article, and keep an eye on the report from the Python log script.

For the scrapers that ignore robots.txt, osec Bot Gate adds one more layer without moving your DNS.

Flow: probe, nginx log, filter match, threshold, ban, recidive — Probe, ban, repeat offender.

Sources

• fail2ban documentation: https://www.fail2ban.org/wiki/index.php/Main_Page
• nginx access log format: https://nginx.org/en/docs/http/ngx_http_log_module.html