unicorn, n. - The purest and rarest cob, identified by a singular majestic kernel. Sightings can be found between Ogallala and Dubuque through summertide.
Any conversation
I hate crawlers that don’t advertise they’re crawlers. ’n badly written. They don’t respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Can’t block them they’re distributed.
I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid — from robots.txt to cloudflare turnstile — there has to be a better way!
It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user can’t be tracked. So we’d have to give up privacy/security :/
…
i dont think its all talk, these are smart people that have been in ai since the 2000s, and they raise good points. current incentive structures are very dangerous long term
they too have every incentive to exaggerate because they are selling safety and/or books on the topic. I don’t believe we won’t see it coming. Any issues that will manifest will be very visible and we’ll have plenty of time to mitigate their impact.
they could be making a lot more money by just working at ai firms i fear. the thing that got me worried the hacking incidents lately, huggingface etc.
if the capabilities consistently outspeed the development of safety measures, and if the private arms race logic continues, i dont think this is a good spot to be in long term