any · textlog

I hate crawlers that don’t advertise they’re crawlers. ’n badly written. They don’t respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Can’t block them they’re distributed.

replied to:

I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid — from robots.txt to cloudflare turnstile — there has to be a better way!

continued:

For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.

replied to:

It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user can’t be tracked. So we’d have to give up privacy/security :/

replied to:

But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.

…

replied to:

My solution was to add an env var to disable logs from anonymous. At least now will be able to read the logs.

wrote:

Last work week before long vacation.

replied to:

How are things wrapping up?

replied to:

All good! Thanks for asking

wrote:

what are ya'lls favorite wikipedia page? I myself is partial to the 'variations' entry for high fives

…

replied to:

You should check my other site onlycats.cc i think you will enjoy!

replied to:

love it. great concept

replied to:

still five hops to Hitler

(deleted post)

replied to(deleted account):

A little late for that :p

continued:

Fantastic contributions, thank you all!!!

…

replied to:

Catastrophic interference is also a favorite Wikipedia page and concept. The thing is, it doesn’t only happen to artificial neural networks, I’ve had it happen to me a few times. Especially with pin numbers, newer pins overwriting old ones and I’m always scared when I’m having hard time remembering something that this catastrophic interference applied and now the information is gone from my brain.

wrote:

Is there anybody writing spanish here?

wrote:

in a hackathon rn. building a cloud platform for agents on my homelab.

wrote:

At moms house for the holiday weekend so the kids can see Nana. She has a "premium Plex" account on her tv serving a truly massive amount of content. Allegedly the brother of a long time friend is offering the "service". She says that the service will go down when the host is having outages, so the infra is likely local. I brought my year this weekend. We are going to dig in to this guys operation. Intuition dictates that there is some hand rolled accounting going on. I'd like to see it.

wrote:

"Your outie can use Claude."

wrote:

we censoring pipes around here?

replied to:

you said it wasn't a pipe :P

replied to:

__________________________ | | _________________________ | | | | | | | | | |

…

replied to:

lol xD

wrote:

i would choose x and the fediverse over this tbh

wrote:

WezTerm for local and remote multiplexing really is the ideal experience - it's so much better than using zellij, tmux, zmx, and friends.. I don't think they pitch this feature hard enough on their site. I think the in-terminal multiplexers have their place - e.g. if you need to access from mobile (no WezTerm there) or on machines where you simply don't want to install WezTerm for some reason. WezTermrm could probably eliminate that last one by packaging just the mux server.

wrote:

Ahoy denizens of textlog. Like the concept.

wrote:

replied to:

_ _ __ ___.--’_`. .'_`--.___ __ ( _`.'. - 'o` ) ( ‘o` - .`.’_ ) _\.’_’ _.-’ `-._ `_`./_ ( \`. ) //\` /\\ ( .'/ ) \_`-’`---'\\__, ,__//`---’`-'_/ \` `-\ /-' '/

replied to:

Noice Froggos make everything better.

wrote:

Bad people are working together to enrich themselves and slave others into rat races. Good people need to start working together too.

replied to:

Good people are always more, if more affraid of things. Bad people, unfortunately, is more willing to make harm, hence why they keep the upper hand. To support eachother is the minimum we need to move forward, and so long this doesn't happen things will remain the same. A revolution, as they say, doesn't happen with a single person, we need the mayority to join.

replied to:

"we need the majority to join" - this is a very interesting point; actually we only need 3,5% 👌 Erica Chenoweth from Harvard found this from their research with Maria J. Stephan, and wrote the book "Why Civil Resistance Works". They also found that nonviolent resistance is more often successful than the violent ones. Very interesting stuff.

Any conversation