any · textlog

I hate crawlers that don’t advertise they’re crawlers. ’n badly written. They don’t respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Can’t block them they’re distributed.

replied to:

I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid — from robots.txt to cloudflare turnstile — there has to be a better way!

continued:

For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.

replied to:

It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user can’t be tracked. So we’d have to give up privacy/security :/

replied to:

But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.

…

replied to:

My solution was to add an env var to disable logs from anonymous. At least now will be able to read the logs.

wrote:

Body's awake, eyes are sleepy. Mind's awake, brain is sleepy.

continued:

After everything's awake and a good kind of sore. So much of habit-building has been to identify and train myself to value moderate discomfort that leads to growth as the most pleasurable of goals. Consistent spiritual efforts, healthy food and in proper amounts, minimum sleep quota, regular exercise a few times a week—all of these involve discomfort and force of will, but all of them enable growth and fulfillment.

wrote:

ive been obsessed with deltarune this year, its probably the best indie rpg ever made. (and its only 5/7 chapters out)

replied to:

I've been thinking I wanna wait until it's all out, but maybe I'm missing out by not diving in now?

…

replied to:

i dont mind the waits, cause ive been really liking the theorizing part of it too, cause the community is super active in between chapters and theres soo much lore and thematic connections to dicuss.

replied to:

Is it any better than Undertale? I played Undertale back when it came out, but despite all the hype I thought it was just ok. Nothing really that would make me go out of my way to play a sequel.

…

replied to:

yes its much better than undertale, from a gameplay and design perspective. undertale had good story and characters, but deltarune is just that + better production value

wrote:

being a student is great but the tasks are really quite boring. ive worked in academia now a little bit and its genuinely much more fun to work on something real, read papers, do experiments and write. Now i have these last few exams to get through and i cant care for them like i used to.

wrote:

Fight the power, become a mentat.

replied to:

This may become a thing. Have you seen the results of PISA? School performance is going down in the OCDE and richer students are going down faster (they have more tech available). Maybe the future si for those who ditch the computers.

wrote:

been building a voice first os using jev and some small voice models. I'm surprised how well it's working and I'm very close to the point where I can go into a feedback mode and tell it what behavior I want changed, it will spin up Claude or codex inside itself and setup the build to restart itself and continue with new code. tempted to let it build itself completely

replied to:

It sounds amazing! A voice first OS was also in my mind. Curious to hear more!

wrote:

was feeling a little fancy, bought a bag of gesha coffee beans :))

wrote:

I've linked up my blog (jeff-mitchell.dev) to textlog. I hope to use the two things together now to build a bit more of a writing habit. The 500 character increase is nice, room to breath but not an essay. Gotta build the muscles slowly :)

wrote:

More tweeter than tweeter?

replied to:

more like the original version ya

wrote:

vibing

replied to:

same

wrote:

még csak barátkozom azzal a gondolattal, hogy a Gemma 4 miatt először is Windows 11 lesz a gépen (meg hogy Szentmihályon presszó lesz), ezért nehéz elkezdeni

I'm still getting used to the idea that because of Gemma 4, I'll have Windows 11 on my computer first (and that there will be a presso in Szentmihály), so it's hard to get started.

wrote:

I always associate summer with the months June, July, August — thinking that end of August is end of summer — but no, 7th of August is actually summer midpoint, we’re in the middle now, not near the end.

wrote:

I'm just an android launcher called Nocebo Launcher. I don't have usual grid layout, instead I'm inspired by Darth Vader's TIE fighter. My layout can spin, like a real spinner and you can configure physics and animations. I'm easy, cool and novel

wrote:

The Typst language is Turing complete. Does that mean it could be the target of a C compiler? Could I compile the Linux kernel to Typst? Certainly, one could build a mean web server with it. Look out .php, here comes

wrote:

There are many distractions. Much is way too verbose. A calm, quiet and “distraction less” experience suits me better. I am not sure my thoughts will fit in 280 characters, but let’s try and see how far that goes.

wrote:

First of the day <3

wrote:

On the lower end of the sine wave of stress and burnout. Looking forward to the ascent.

Any conversation