Reinventing the ApisCP WAF

1

Long time since I’ve posted, and this one’s a fun one. I’d like to pick your brains.

ApisCP based its original WAF on mod_evasive. It’s a simple counter that tracks inbound requests per process. Evasive is easily circumvented if keep-alives are disabled or the client explicitly closes a connection. A “200 OK” CSS request is just as significant as a 404 that flows through to a dispatcher, like WordPress, and serves out a dynamic - fully loaded - WordPress site. As of late, I’ve seen changes in bot request patterns that makes this solution inadequate.

mod_shield rethinks this approach. It’s a full rewrite of Evasive.

Features

  • Hits are shared across processes
    • Records are fixed at 176 bytes allowing for 3k unique hits over a 2 minute window at 512 KB
      • Autoexpire, autoeviction w/ LRU policy
      • Cache resizable
    • Can be stored to memcached or Redis providers to share hits across servers
  • HTTP statuses and page loads are scorable
    • Status points may penalize 404s allowing for faster blacklisting of random probes
      • Works with custom HTTP status codes, e.g. returning 499 from backend could score 10000000 resulting in immediate block.
    • Fast loads remove decrease points while slow loads may add to point total
    • Complex scoring is possible, e.g. 1s < load < 2s = 5 pts, 2s <= x < 10 = 7 pts, load >= 10 = 10 pts
      • Addresses DoS situations where bots hit database-heavy requests, like search
  • Blocks counted per event
    • For each block application, the duration may be longer than previously applied
    • Counted for life of system service process
    • Both RFC7231 “Retry-After” + RateLimit headers are sent to guide a compliant client
  • Proxy support
    • Pierces Cloudflare’s tunnel in which the edge address cannot be blocked without disrupting Cloudflare usage
    • Works with any other downstream proxy as defined using mod_remoteip
  • Fast, Minimal Overhead
    • Occurs early in processing axis, ~13% faster than mod_evasive :person_shrugging:
    • Requires 3 separate memory chunks for per-site, per-page, and active-blocks.
  • Improved logging

    • Status handler executive summary

      epsilon-status-handler
      epsilon-status-handler1846×1378 200 KB

      Full tracking output.zip (25.4 KB)

    • Diagnostics, get a better idea of how things score and why. Doubles as a telemetry module without blocking.

      [Fri Jun 20 17:27:18.044384 2025] [shield:info] [pid 2241388:tid 2241454] [client 35.196.24.130:58136] Request status 301 flagged with score 25
[Fri Jun 20 17:27:18.626983 2025] [shield:info] [pid 2241388:tid 2241452] [client 35.196.24.130:58136] Request status 404 flagged with score 50
[Fri Jun 20 17:27:22.265283 2025] [shield:info] [pid 2241388:tid 2241455] [client 35.196.24.130:60824] blocking period started for IP: 35.196.24.130 (expiry: 1750541242264768)
[Fri Jun 20 17:31:46.372136 2025] [shield:info] [pid 2246513:tid 2246541] [client 103.205.211.78:47894] Request status 302 flagged with score 25
[Fri Jun 20 17:31:47.559982 2025] [shield:info] [pid 2246566:tid 2246607] [client 103.205.211.78:40392] Request status 301 flagged with score 25, referer: http://x.com/xmlrpc.php
[Fri Jun 20 17:31:48.719776 2025] [shield:info] [pid 2246513:tid 2246539] [client 103.205.211.78:40530] Request status 404 flagged with score 50, referer: https://x.com/xmlrpc.php
[Fri Jun 20 17:32:03.948083 2025] [shield:info] [pid 2241389:tid 2241464] [client 138.117.18.67:52430] Request status 404 flagged with score 50
[Fri Jun 20 17:39:03.452497 2025] [shield:info] [pid 2246566:tid 2246607] [client 193.203.10.164:32349] Request status 404 flagged with score 50
[Fri Jun 20 17:41:51.467151 2025] [shield:info] [pid 2246513:tid 2246540] [client 177.10.34.242:13784] Request status 404 flagged with score 50
[Fri Jun 20 17:43:28.093184 2025] [shield:info] [pid 2246513:tid 2246535] [client 177.204.39.240:58988] Request status 404 flagged with score 50
[Fri Jun 20 18:18:46.574231 2025] [shield:notice] [pid 2241388:tid 2241446] [client 114.119.128.58:36383] Request round-trip 1069 ms exceeds limit 1000 ms. Scoring 1 /dsc_hdr-copy/ (proto: HTTP/1.1), referer: https://x.com/dsc_hdr-copy/

Ideas so far

  • GeoIP blocking
    • I’m not a fan, works contrary that this firewall works adaptively based upon malicious traffic. Bad actors should be blocked organically.
  • JS challenge to unblock
    • Will be added in a later release. For now, blocks can be done within ApisCP’s panel.

What else would you like to see added?

Running ApisCP already? You can swap to Shield today:
dnf --enablerepo=apnscp-testing update -y mod_shield

Module is RC stage. It’ll be part of the next major ApisCP release :+1:

2 Likes
2

Hey @nem, nice to see you round these parts! This is a really cool implementation. When do you expect the next major ApisCP release will be?

3

July. It’ll consist of C module rewrites (NSS, PAM, mod_shield, Dovecot). There’s an outstanding memleak in D-Bus interface that I’d like to address before getting this release out.

These updated modules will allow for some neat features-

  • Migrate scheduled tasks off cron to systemd timers
    • Allows for dependency chains
  • Multiple PHP pools with ownership beyond “apache” + account owner
  • Dovecot LDA replaces maildrop
    • Quota warnings at delivery
    • sieve support, sieve can also deliver
  • mod_shield DoS protection. I’ve seen a major reduction in server load; servers were pegging out at 85% CPU due to random probes from Azure and the like. I’ll have data on the overall reduction then as well.
2 Likes
4

Sounds very interesting, maybe I should swap over on my test/staging server :slight_smile:

5

I’d strongly recommend it. It’s normalized system loads back to the 20-40% range, which previously were spiking 300-750% - worse whenever Cloudflare passed bots through unprotected.