I like this, it feels like Reddit before what it is now, but I feel like it wonโt be long until the internet destroys this place. Until then Iโm happy to be here.
It's really sad that this is such a common sentiment whenever there's a nice online community nowadays. Guess most of us have been burnt enough times by now
My favorite part of classic reddit was the use of elaborate hacks to circumvent the restrictions on custom subreddit CSS. E.g. animated gifs weren't allowed, so people made filmstrip sprites with stepwise animation in such a way as to look exactly the same.
I hate crawlers that donโt advertise theyโre crawlers. โn badly written. They donโt respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Canโt block them theyโre distributed.
I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid โ from robots.txt to cloudflare turnstile โ there has to be a better way!
For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.
It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user canโt be tracked. So weโd have to give up privacy/security :/
But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.
me too, but at the same time i'm enjoying getting things done, otherwise i would get lost in details like bikeshedding naming variables and making the code look pretty, and little would be done.
Same with the Steam Machine. They could have been awesome offerings for a lot of different use cases, but alas, being able to generate memes with grandma's pictures is of the utmost importance...
github.com/Aeres-u99
I have created a small tool, codeatlas, that basically generates an index to codebase that your llm can lookup syper quick, github.com/Aeres-u99
there is also a cbz->pdf converter that I wrote for eink devices which is nice
Well said. My post was glib.
More correctly, they are multipliers. Same goes for Musk.
"Muskโs ex-wife Justine, whom he married in 2000 and divorced in 2008, says the man the world began to see in 2020 was the one sheโd always known." -slate.com