In X, @zoriya_dev shared an issue where an API request was blocked by Cloudflare when using Python's aiohttp library, while the same request worked fine when using curl or the requests library. People got together in replies to investigate the issue.

here is the python code (that gets cloudflare blocked)
the curl requests (that works)

and the netcat to print requests (by changing https://thexem by http://localhost), first is python second is curl https://t.co/eJZydIRbkK pic.twitter.com/8TSDb7WjMs

— Zoe Roux (@zoriya_dev) September 20, 2024

Initial analysis suggested that Cloudflare was blocking requests based on the User-Agent header. However, this theory was quickly disproven as the User-Agent header was identical in both aiohttp and curl requests. Even after ensuring that the entire HTTP request was the same for both libraries, the issue persisted. This indicated that the detection mechanism was operating at a lower level, likely involving TLS records.

Using Wireshark, I discovered that the TLS extensions differed between aiohttp and curl. By adding any single TLS extension to aiohttp, the block was bypassed, effectively disrupting the blacklisted fingerprint.

TLS fingerprinting is a common technique used by Cloudflare and other services to detect bots and malicious traffic. Cloudflare offers varying degrees of protection that can be configured by the domain owner. In this case, it is possible that the domain was put in a higher protection level where TLS fingerprint of aiohttp is blacklisted.