nospam.Jay.Harris@f664.n229.z1.fidonet.org (Jay Harris) writes:
> On 02 Jul 2024, Daniel said the following...
> Da> If I attempt accessing it with lynx, I get this generic error:
> Da> "Due to aggressive automated scraping of FederalRegister.gov and
> Da> eCFR.gov, programmatic access to these sites is limited to access to our
> Da> extensive developer APIs."
> Da> I don't get this message on a normal javascript enabled browser. I
> Da> surmise it views queries from non-javascript browsers as a curl-like
> Da> command.
> I don't know if you can change the user agent in lynx, but try and see if it
> works with curl.
> e.g:
> agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101
> Firefox/127.0"
> curl -A "$agent" <url>
> I've had to do this in the past when pulling information for weather, they
> stopped allowing curl, but simply changing the user agent allowed it to work.
Okay thanks. Do you have an opinion on the door idea by chance?
--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
|
