Subscription Lookup Protection
What is protected
Four public GET endpoints answer "is this address subscribed?" for a single email address:
| Endpoint | SendGrid calls per request |
|---|---|
/subscriptions/:locale/check | contact search, group suppressions, global suppression |
/subscriptions/extra | group suppressions |
/subscriptions/special | group suppressions |
/subscriptions/pixel_consent | contact search |
Opening the preference centre calls check and special; extra is only called once the reader
opens its panel (the lists are fetched on expand, see aleteia-next#1122 and the lazy-loading change
that followed). Both of those ask SendGrid for the same contact's group suppressions, which is why
that lookup is the one worth caching.
Why it matters
The SendGrid quota is per account, not per caller, and Sendgrid::API sits behind a single Stoplight
circuit. So a flood on these endpoints does not just slow itself down:
- SendGrid starts answering
429on/v3/asm/suppressions/{email}; HttpartyJsonClienttracks those asTooManyRequests, and five of them open the circuit;- for the 60 second cool off every caller of
Sendgrid::APIgets a503, including newsletter signups and users who were nowhere near the flood.
This happened on 2026-09-21: roughly 500 requests spread over 88 IPs and 158 addresses in two minutes
exhausted the suppressions quota and produced 195 503 responses before the circuit closed by itself.
Caching
Sendgrid::API caches one lookup: suppressions_by_email
(GET /v3/asm/suppressions/{email}), for SUPPRESSIONS_CACHE_TIME (5 minutes), keyed by the
downcased address. That is the question check, special and extra each ask separately about
the same contact, and the SendGrid endpoint that starts returning 429 first.
The other two per-address lookups (contact, suppressed?) are deliberately not cached: one
page load asks for each of them exactly once, so caching would buy almost nothing while widening
the set of writes that have to remember to invalidate something.
Both writes that change a contact's suppression groups live in the same class, next to the cached
read, and expire it: suppress! and unsuppress!. So the preference centre never shows a user
something they have just changed, and the five minute window is only the ceiling for an answer
that went stale without a write we noticed.
If you add another write to Sendgrid::API that changes suppression groups, call
expire_suppressions(email) on it. There is no automatic invalidation.
Rate limiting
Two Rack::Attack throttles cover the same four endpoints (see config/initializers/rack_attack.rb).
Both look only at the query string, never at a request body.
| Throttle | Key | Default | Variable |
|---|---|---|---|
subscription lookups per email | the address | 12/min | RACK_ATTACK_LOOKUPS_PER_EMAIL |
subscription lookups overall | the endpoints as a whole | 120/min | RACK_ATTACK_LOOKUPS_PER_MINUTE |
The per-address counter catches one address being asked about over and over. It does nothing against a flood that rotates addresses, which is the shape we actually see — hence the second throttle, a safety valve that bounds what any wave can cost upstream whatever it rotates through. It is sized well above real traffic: the busiest legitimate minute of the fortnight before the incident was 42 requests, against the ~390/min of the flood itself.
During a flood the valve degrades these endpoints to 429 for everyone rather than letting the circuit
open and return 503 across the whole SendGrid integration. That trade is deliberate.
Both limits are read into Rails.application.config at boot, so changing the variables needs a restart
(on Heroku a config var change restarts the dynos anyway).
What the application cannot see
Network attributes such as ASN, geolocation and IP reputation are added by Datadog when it processes the logs; the application only ever sees the client IP, the user agent and the request itself. Filtering by network — which is what actually separates these waves from real users — belongs in front of the application, in a WAF, not in Rack::Attack.