Skip to main content

Subscription Lookup Protection

What is protected

Four public GET endpoints answer "is this address subscribed?" for a single email address:

EndpointSendGrid calls per request
/subscriptions/:locale/checkcontact search, group suppressions, global suppression
/subscriptions/extragroup suppressions
/subscriptions/specialgroup suppressions
/subscriptions/pixel_consentcontact search

Opening the preference centre calls check and special; extra is only called once the reader opens its panel (the lists are fetched on expand, see aleteia-next#1122 and the lazy-loading change that followed). Both of those ask SendGrid for the same contact's group suppressions, which is why that lookup is the one worth caching.

Why it matters

The SendGrid quota is per account, not per caller, and Sendgrid::API sits behind a single Stoplight circuit. So a flood on these endpoints does not just slow itself down:

  1. SendGrid starts answering 429 on /v3/asm/suppressions/{email};
  2. HttpartyJsonClient tracks those as TooManyRequests, and five of them open the circuit;
  3. for the 60 second cool off every caller of Sendgrid::API gets a 503, including newsletter signups and users who were nowhere near the flood.

This happened on 2026-09-21: roughly 500 requests spread over 88 IPs and 158 addresses in two minutes exhausted the suppressions quota and produced 195 503 responses before the circuit closed by itself.

Caching

Sendgrid::API caches one lookup: suppressions_by_email (GET /v3/asm/suppressions/{email}), for SUPPRESSIONS_CACHE_TIME (5 minutes), keyed by the downcased address. That is the question check, special and extra each ask separately about the same contact, and the SendGrid endpoint that starts returning 429 first.

The other two per-address lookups (contact, suppressed?) are deliberately not cached: one page load asks for each of them exactly once, so caching would buy almost nothing while widening the set of writes that have to remember to invalidate something.

Both writes that change a contact's suppression groups live in the same class, next to the cached read, and expire it: suppress! and unsuppress!. So the preference centre never shows a user something they have just changed, and the five minute window is only the ceiling for an answer that went stale without a write we noticed.

caution

If you add another write to Sendgrid::API that changes suppression groups, call expire_suppressions(email) on it. There is no automatic invalidation.

Rate limiting

Two Rack::Attack throttles cover the same four endpoints (see config/initializers/rack_attack.rb). Both look only at the query string, never at a request body.

ThrottleKeyDefaultVariable
subscription lookups per emailthe address12/minRACK_ATTACK_LOOKUPS_PER_EMAIL
subscription lookups overallthe endpoints as a whole120/minRACK_ATTACK_LOOKUPS_PER_MINUTE

The per-address counter catches one address being asked about over and over. It does nothing against a flood that rotates addresses, which is the shape we actually see — hence the second throttle, a safety valve that bounds what any wave can cost upstream whatever it rotates through. It is sized well above real traffic: the busiest legitimate minute of the fortnight before the incident was 42 requests, against the ~390/min of the flood itself.

During a flood the valve degrades these endpoints to 429 for everyone rather than letting the circuit open and return 503 across the whole SendGrid integration. That trade is deliberate.

Both limits are read into Rails.application.config at boot, so changing the variables needs a restart (on Heroku a config var change restarts the dynos anyway).

What the application cannot see

Network attributes such as ASN, geolocation and IP reputation are added by Datadog when it processes the logs; the application only ever sees the client IP, the user agent and the request itself. Filtering by network — which is what actually separates these waves from real users — belongs in front of the application, in a WAF, not in Rack::Attack.