any · textlog

wrote:

I hate crawlers that don’t advertise they’re crawlers. ’n badly written. They don’t respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Can’t block them they’re distributed.

replied to:

I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid — from robots.txt to cloudflare turnstile — there has to be a better way!

continued:

For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.

replied to:

It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user can’t be tracked. So we’d have to give up privacy/security :/

replied to:

But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.

…

replied to:

My solution was to add an env var to disable logs from anonymous. At least now will be able to read the logs.

wrote:

this will be the platform that fixes me

wrote:

Taking a small break from here to work on an for a friend! I missed .

continued:

It didn’t work out, we found competition offering the same service for free so the fell apart. Need to find something else to do as a side project to textlog. 😕 Thinking of a code editor that’s focused on the new agentic workflows, so you have it on the side regardless of your agent tools and it monitors the git diff and offers a nice panorama view. I’ve done an extension for VSCode similar to this idea.

replied to:

Good outcome! Better to bail early and avoid toiling on a doomed idea. I'm always afraid of this 😅

…

replied to:

You might want to check out . Not exactly what you're speaking of, but the idea sounds similar.

wrote:

The live tab on the android app is interesting... How does it work? This is a test 🫣

continued:

Does it do some kind of polling? Is there some kind of notify like publisher subscriber? 🤔

replied to:

It’s using the /firehose endpoint. It’s an SSE endpoint publishing everything latest. Though I meant it to be used server-side, since it’s heavily rate-limited — up to 3 conns under the same network, but I can’t control how clients decide to use it so.. ¯\_(ツ)_/¯

…

continued:

Might just lift the limitation.

continued:

Lifted, that’s how clients want to use it then so be it! Should be fine. We’re nowhere near crashing the server from connections.

wrote:

Hopefully this takes off

wrote:

Fight the power, become a mentat.

replied to:

This may become a thing. Have you seen the results of PISA? School performance is going down in the OCDE and richer students are going down faster (they have more tech available). Maybe the future si for those who ditch the computers.

wrote:

“Every problem that is interesting is also soluble.” - David Deutsch

wrote:

A thing I've been doing lately is trying to come to the same conclusions others have without looking their the direct techniques and code. Last night I tried to figure out why people do vector based retrieval for RAG and ended up running these experiments: n0tls.com/…/2026-09-03-i-think-…rag-with-grep.html I'm still going to do an explore agent.

continued:

Here's a follow up from last night: n0tls.com/…/2026-09-05-fine-tuning-…tiny-retriever.html This has been a great way for me to learn in this space. Learning with AI has been such a fun time for me.

wrote:

in a hackathon rn. building a cloud platform for agents on my homelab.

wrote:

we censoring pipes around here?

replied to:

you said it wasn't a pipe :P

replied to:

__________________________ | | _________________________ | | | | | | | | | |

…

replied to:

lol xD

wrote:

I often wish I had the ability to pause my main quest so I could focus more on side questing.

wrote:

I never noticed the little halfsteps in the bassline of the verses of Rebel Yell before. It's so slinky and subconscious and makes the chorus that much more.

wrote:

Ubuntu telepítése a VirtualBoxba rendben megtörtént. Már csak egy ügynököt kell bele fogni, és máris kész az "AI torture chamber", ahogy a Futurism írja... Mondjuk, első körben még nem így fogalmaznék...

Ubuntu has been successfully installed in VirtualBox. All you need to do is add an agent and you have an "AI torture chamber", as Futurism describes it... Let's just say, I wouldn't put it that way at first...

wrote:

Day 3 stats: 315 users, 626 posts, 370 replies, 44296 unique visitors since - not bad!

replied to:

Woo!

wrote:

prussia's emoticon is so cute! (=ヮ≖)

wrote:

If I went back in time, I would aim to become a professional footballer rather than becoming a programmer

wrote:

Finally watched Project Hail Mary. Easily one of my top 5 favourite films but it’s still not nearly as good as the book — I feel like the cut some pretty important scenes

replied to:

It's the first film in a long time that I enjoyed.

…

replied to:

I loved the book and I’m always fearful when a movie script is developed and a movie produced, so my relief after I saw the movie was immense. I too feel they’ve cut-out some bits, but the main thing that a book almost always takes my breath away with is fueling my imagination. A movie would (possibly) never nail that aspect.

replied to:

I just watched it as well. It was fun. There were a number of things that I felt weren't explained well and broke my suspension of disbelief; I haven't read the book, but I found myself really curious whether the book takes the time to justify things more. Obviously, the movie has a lot of pressure to rush to the next "big moment".

…

replied to:

If there’s one thing Andy Weir can’t be critiqued on then it’s not fully justifying all of the science. I think the film on the other hand can be hard to follow for someone who has neither read the book nor has a scientific background. It’s just hard to monologue about gravitational forces and relativistic speeds on screen.

wrote:

a lustaság meg fél egészség

Laziness is half the health.

wrote:

a szerveren lévő cookie adatokat és egyéb nyomkövető sz@rokat is a Google kezeli, én nem igazán szoktam ezekből látni semmit, se ip-t, se semmit, a Google inkább keresőmotor, mint mesterséges intelligencia alkalmazás, de ma már hibrid technológiával dolgozik minden MI és kereső

Google also manages the cookie data and other tracking s@rs on the server, I don't really see anything from these, no IP, nothing, Google is more of a search engine than an artificial intelligence application, but today all AI and search engines work with hybrid technology

Any conversation