Twitter oEmbed broken on DigitalOcean

(TLDR solution is to allowlist the Twitter API IP address ranges)

Around 3 weeks back, the end of May I had reports of WordPress authors unable to embed tweets in posts, some using classic editor, others block editor.

The embed code such as:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Fixed an issue with Twitter oEmbed failing in WordPress Post<a href="https://t.co/nHdcW8FHJ1">https://t.co/nHdcW8FHJ1</a></p>&mdash; marc kranat (@marckranat) <a href="https://twitter.com/marckranat/status/1670225962875617283?ref_src=twsrc%5Etfw">June 18, 2023</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

was still working, but just pasting a tweet into the editor failed, even with EmbedPress which I found normally fixes oEmbed issues failed.

Troubleshooting was complicated as recently Twitter has been making changes to some of its business model, charging $42,000 PA for developer access to the API. I assume it probably is a consequense of changes made to limit API access. But we are talking here about just embedding single tweets.

Solutions to the issue were discussed here: https://wordpress.org/support/topic/twitter-embeds-failing/

We see here that the Twitter Dev has no idea what’s going on, so I can’t see this is intentional,

This issue could be confirmed with curl from the hosting server to the twitter API, showing 200ok from a local terminal, and a combination of 400 and 404 errors from the hosting server, I don’t know how the oEmbed well enough to understand, but my hosting logs were showing no error, and nothing showing in chrome inspector. but heres the thing, I have 2 layers of firewalls, the WAF which only filters port 443, and then on the DigitalOcean panel I have a ruleset which blocks all other traffic except the WAF IPs and a few other services to other fixed IPs. So somehow the host must be making a request to twitter, and twitter is responding back through some other path, and not via 443 via the domain name. I have not drilled down on what that is yet, I’ll update here when I do.

So just whitelisting the Twitter API IP ranges to all ports solves the issue, and oEmbed goes back to working in WordPress posts.

It is odd that I have only seen this issue raise with DigitalOcean and Hetzner, maybe something with the way they firewall in their panel, usually a firewall allows an internally initiated requests, response to pass through, regardless of the firewall ruleset (stateful inspection).