EFF Joins 18 Civil Rights Organizations Calling on Governor Hochul to Reject the Stealth Crawler Prohibition Act

Eighteen civil rights and digital freedom organizations, including the Electronic Frontier Foundation (EFF), have urged New York Governor Kathy Hochul to veto Senate Bill 9934A, known as the New York Stealth Crawler Prohibition Act. The coalition argues that the proposed legislation, while presented as a measure to protect local journalism, poses significant threats to free expression, digital privacy, and the foundational structure of the internet. The bill would require all web crawlers to disclose their identities and purposes, while granting media outlets broad authority to obtain judicial subpoenas to unmask unidentified automated web traffic without demonstrating any misconduct or harm.

The groups contend that the bill, which targets anonymous web crawling, could criminalize essential public-interest activities that rely on automated data collection. Web crawlers, often used by journalists, researchers, and cybersecurity professionals, enable critical investigations into corporate practices, market competition, and security threats. The legislation risks stifling independent journalism and compromising privacy tools like the EFF’s Privacy Badger, which rely on anonymous crawling to function without exposing user identities.

The coalition highlights that so-called "stealth crawlers" are merely automated tools that access public web data without revealing the user’s identity. These tools have been instrumental in uncovering anti-competitive behavior by major corporations. For instance, The Markup, a nonprofit news organization, used anonymous crawlers to investigate Amazon’s search algorithms, discovering that the platform prioritized its own brands and exclusive products over higher-rated competitors. Similarly, ProPublica employed automated tools to simulate ordinary customer behavior, revealing that Amazon’s site steered shoppers toward more expensive products.

Cybersecurity professionals also depend on anonymous web scraping to monitor threats and protect infrastructure. Privacy tools like Privacy Badger use automated crawling to identify tracking mechanisms without compromising user privacy. The proposed law, however, could disrupt these vital functions by requiring crawlers to identify themselves, potentially exposing sensitive research or security operations.

Critics argue that the bill misdiagnoses the issue, conflating anonymity with overaggressive crawling that may strain technological infrastructure. They assert that addressing server overload requires targeted technical solutions, not broad policies that could chill public-interest research or compromise digital privacy. The coalition urges Governor Hochul to reject S9934A to prevent unintended consequences that could undermine both the open web and investigative journalism.

#DigitalPrivacy #NewYorkLegislation #EFF #SenateBill9934A #KathyHochul #WebCrawlers #FreeExpression #Cybersecurity