@utf_7 In this economy with RAM prices what they are?!?

@JonSaenzAgirre It is a good questions, and we don't know the answer either. Our planet data is so much easier to process and use.

@utf_7 We've had 400,000 IPs in the last 24 hours. Each IP only does a few requests. Technically we're managing, but no fun fighting this daily rather than building new things.

@utf_7 It is madness, start here: https://www.openstreetmap.org/node/1 and keep going once you reach https://www.openstreetmap.org/node/10000000000, then start on ways, and relations or just download the latest weekly export from planet.openstreetmap.org

@michel42 We'd like to share the IP address list, but unfortunately don't think we can due to legal concerns.

@boz We will survive and grow stronger thanks to our awesome mapping community We continue to squeeze more capacity out of our servers, but eventually we'll need to upgrade despite the $$$ RAM prices.

To keep #OpenStreetMap.org up and running while we're being deluged by scrapers, we've blocked 320,000+ primarily residential IPv4 addresses in the last 24 hours (+ 100,000 IPv6) involved in scraping.

If you need OSM data, please don't scrape the website - use the official downloads at https://planet.openstreetmap.org
#AI #Bots #Abuse

@joeress @latenightlinux Sounds interesting. We'll follow up by email.

@eigen we're always looking for more volunteers to join our team. ️

@BalooUriza We use fail2ban to handle some of this with custom rules, but eventually fail2ban becomes a bottleneck after 100,000 IP addresses.

@LMieldazis @geerlingguy oooh do we get to show him our out-of-band (remote access) Raspberry Pi with dual power feeds, 4G modem and loads of serial connections? Saved our skin a good few times.

@utf_7 See https://www.youtube.com/watch?v=qhie14YKa-w&t=790s

If you write about the messy reality behind "free" internet services: we're seeing #OpenStreetMap hammered by scrapers hiding behind residential proxy/embedded-SDK networks. We're a volunteer-run service and the costs are real. We'd love to talk to a journalist about what we're seeing + how we're responding. #AI #Bots #Abuse