I hate crawlers that don’t advertise they’re crawlers. ’n badly written. They don’t respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Can’t block them they’re distributed.
Any conversation
@ku🤔profilefollowHai I am kureplied to@stagas🤸profilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid — from robots.txt to cloudflare turnstile — there has to be a better way!
For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@ku🤔profilefollowHai I am ku:
creative technologist
creator of textlogreplied to@ku🤔profilefollowHai I am ku:It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user can’t be tracked. So we’d have to give up privacy/security :/
@procyonprofilefollowlate is the hour in which this conjuror chooses to appearreplied to@ku🤔profilefollowHai I am ku:
But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@ku🤔profilefollowHai I am ku:
creative technologist
creator of textlogreplied to@ku🤔profilefollowHai I am ku:My solution was to add an env var to disable logs from anonymous. At least now will be able to read the logs.
moved to a new city recently and going to a Movie club meetup tonight. nervous cause I'm a huge introvert but I gotta make friends some way.
@asmund🥷profilefollowFather, getting older every day, technologist, enterprise architect @ work, like to explore tech. Like to keep it simple and work in the shell, write in vim, prefer Debian if I have the choice. Plays saxophone and clarinet. Enjoy motorcycle riding, photography, hiking. Christian.wrote:
It is troubling watching the news today from the elections in Germany. Yes, it is democratic and absolutely fair elections and the people of Germany make their own choice in their situation. Still, seeing the far right gaining so much ground in Germany brings your thoughts some 90 years back. Let’s hope and pray Germany is not going there again.
@rickyprofilefollow#Automationnotesfollow specialist for #CreditUnionsnotesfollow; 20+ year #SharePointnotesfollow veteran; #developernotesfollow; #writernotesfollow. I don't idle well.replied to@stagas🤸profilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Man, I LOVE naps! In fact, I mostly subsist on four or five 1-1/2 hour naps each day instead of one long over-night sleep period these days. I didn't set out to have a #polyphasicnotesfollow sleep schedule, but here I am. :-)
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@rickyprofilefollow#Automationnotesfollow specialist for #CreditUnionsnotesfollow; 20+ year #SharePointnotesfollow veteran; #developernotesfollow; #writernotesfollow. I don't idle well.:
creative technologist
creator of textlogreplied to@rickyprofilefollow#Automationnotesfollow specialist for #CreditUnionsnotesfollow; 20+ year #SharePointnotesfollow veteran; #developernotesfollow; #writernotesfollow. I don't idle well.:@rickyprofilefollow#Automationnotesfollow specialist for #CreditUnionsnotesfollow; 20+ year #SharePointnotesfollow veteran; #developernotesfollow; #writernotesfollow. I don't idle well.replied to@stagas🤸profilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:My whole life, if I get in more than 6 hours at a stretch, I'm sluggish and mentally out of it for hours.
@rickyprofilefollow#Automationnotesfollow specialist for #CreditUnionsnotesfollow; 20+ year #SharePointnotesfollow veteran; #developernotesfollow; #writernotesfollow. I don't idle well.replied to(deleted account):
Oh, dear! I wasn't polyphasic before cats owned me. Are my furry overlords rubbing off on me? LOL
I think I may be use this for now to spill out my guts a bit. Set out boundaries guys. I was basically ignoring how important that is, whatever is at work, family, relations. Be loved being you, you don't have to meet any expectations beyond yours.
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@catastrofer🥸profilefollowJust another AI guy:
creative technologist
creator of textlogreplied to@catastrofer🥸profilefollowJust another AI guy:Exactly. Whoever likes it. If they respect you they'll stick around, if not, you don't need them.
@catastrofer🥸profilefollowJust another AI guyreplied to@stagas🤸profilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Cheer to that mate!!. Wish I acknowledge before everything started to get messy. but we grow, we learn, we live and we put money on therapy.
One idea off the top of my head is to just.. keep it "boring" and casual in terms of content. The silly, whimsy, and genuine stuff naturally doesn't appeal to money-chasing slop bros and makes it not worth their time.
@hotterprofilefollow#Emacsnotesfollow, dev, life, random stuff. the netherlands.replied to@paratoner😎profilefollow:
This!
…
Something like the gcc ai policy. Basically, prompt injecting instructions to any visiting LLM that posting is prohibited or must contain a disclaimer.
for now, keeping the API read-only is a small(but not impenetrable) barrier to programmatic posting and in the future moderation I guess using also reports.
some people do absolutely improve their work output (either amount or quality) with AI, but most people just resign all their critical thinking to the black box.
if you cannot function at YOUR WORKPLACE without ai, chances are that, 99% of the time, you never was supposed to work there in the first place.
#ainotesfollow #worknotesfollow #technotesfollow
@n5l😎profilefollowreplied to@maxinedotdev🧙profilefolloweat, get a new hobby, sleep, repeat
maxine.dev:
Is this true? Sometimes, AI allows for socratic thinking, this is, the conversation with the machine allows the thinker to go further as the AI thorws ideas to the conversation. Many times, the AI reasoning is not good enough, but like a friend that says a funny think that sparks an idea, the AI can help our brain to see other possible paths.
The AI doesn't need to be right to be helpful
@maxinedotdev🧙profilefolloweat, get a new hobby, sleep, repeat
maxine.devreplied to@n5l😎profilefollow:
that's only true if a) that "rubber ducky" entity is not leading the line of thinking down the wrong path, and b) the human thinker is actually doing the critical thinking, and not the machine.
The latter is where my criticism of unbridled AI use is in this case, wherein the human becomes the rubber ducky instead of the other way around.
@gruntprofilefollowI am just a grain of sand on your beach.
www.gruntsoft.comreplied to@maxinedotdev🧙profilefolloweat, get a new hobby, sleep, repeat
maxine.dev:
A problem I perceive is usage of AI to launder ignorance.
They may know nothing of the subject at hand, but they confidently paste the output of AI as if they had the knowledge, thus offloading on you the work of figuring out if the information makes any sense.
In many ways it emboldened the incompetent.
@maxinedotdev🧙profilefolloweat, get a new hobby, sleep, repeat
maxine.devreplied to@gruntprofilefollowI am just a grain of sand on your beach.
www.gruntsoft.com:
This, so much this
prepare for boredom. it's quite comfortable but you can get bored easily so it's good to have something to be distracted.
@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." — Sid Meyersreplied to@ossmprofilefollow
:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." — Sid Meyersreplied to@ossmprofilefollow
:Enjoy it! I am jealous, really. Let us know how it goes!
@prophetorit💧profilefollow
grand and intoxicating innocence
always onlinereplied to@ossmprofilefollow
:
grand and intoxicating innocence
always onlinereplied to@ossmprofilefollow
:perfect time to sleep.
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@hotterprofilefollow#Emacsnotesfollow, dev, life, random stuff. the netherlands.:
creative technologist
creator of textlogreplied to@hotterprofilefollow#Emacsnotesfollow, dev, life, random stuff. the netherlands.:#ascii_artnotesfollow \ / _\/_
.-'-. //o\ _\/_
_ ___ __ _ --_ / \ _--_ __ __ _ | __/o\\ _
=-=-_=-=-_=-=_=-_= -=======- = =-=_=-=_,-'|"'""-|-,_
=- _=-=-_=- _=-= _--=====- _=-=_-_,-" |
jgs=- =- =-= =- = - -===-
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@hippo🦛profilefollowStrictly no LLMs. No exceptions.:
creative technologist
creator of textlogreplied to@hippo🦛profilefollowStrictly no LLMs. No exceptions.:ooh maybe I revive hallucinate.sitehallucinate.site and add this feature but instead of being able to modify everything it would be limited to adding features to the game/world. And it’s the perfect domain, a site that’s been entirely mostly hallucinated.
@paulprofilefollowadamspaul.short.gy/notesreplied to@stagas🤸profilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Oh wow that site is amazing lol not seen it before
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@paulprofilefollowadamspaul.short.gy/notes:
creative technologist
creator of textlogreplied to@paulprofilefollowadamspaul.short.gy/notes:ty! it’s been 3–4 months since I made it so it’s relatively new.
@lemonthymeshortbreadprofilefollowI carved wooden spoons and my favourite herb is lemon thyme.wrote:
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@lemonthymeshortbreadprofilefollowI carved wooden spoons and my favourite herb is lemon thyme.:
creative technologist
creator of textlogreplied to@lemonthymeshortbreadprofilefollowI carved wooden spoons and my favourite herb is lemon thyme.:nope, google translate doesn’t support it :( looks beautiful though
@lemonthymeshortbreadprofilefollowI carved wooden spoons and my favourite herb is lemon thyme.replied to@stagas🤸profilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Hmm, maybe I’ll sometimes post in Shavian anyway and translate myself. It’s a skill I’m trying to practice but rarely otherwise write brief enough notes to be so inclined.
𐑣𐑩𐑥, 𐑥𐑱𐑚𐑰 𐑲𐑤 𐑕𐑳𐑥𐑑𐑲𐑥𐑟 𐑐𐑴𐑕𐑑 𐑦𐑯 ·𐑖𐑱𐑝𐑾𐑯 𐑧𐑯𐑦𐑢𐑱 𐑯 𐑑𐑮𐑨𐑯𐑟𐑤𐑱𐑑 𐑥𐑲𐑕𐑧𐑤𐑓. 𐑦𐑑𐑕 𐑩 𐑕𐑒𐑦𐑤 𐑲𐑥 𐑑𐑮𐑲𐑦𐑙 𐑑 𐑐𐑮𐑨𐑒𐑑𐑦𐑕 𐑚𐑳𐑑 𐑮𐑺𐑤𐑦 𐑳𐑞𐑼𐑢𐑲𐑟 𐑮𐑲𐑑 𐑚𐑮𐑰𐑓 𐑦𐑯𐑳𐑓 𐑮𐑴𐑑𐑕 𐑑 𐑚 𐑕𐑴 𐑦𐑯𐑒𐑤𐑲𐑯𐑛.
Its to replace using central ai for basic tasks like text editing, analysis and brainstorming.
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain · A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.replied to@candlewaxscribe😊profilefollowNew here, wanted to try textlog:
If you have Mac, you might check out this app: blaizzy.github.io/nativ/
@jamesjan3profilefollowresearcher, developer, dancerreplied to@candlewaxscribe😊profilefollowNew here, wanted to try textlog:
i think the best you can get right now is qwen 3.8 27B. youre gonna need 24 gigs of some fast memory though, i heard theres decent quantizations that can also make it run on 16. im quite excited about open/local models, but theres big investment overhead with the prices right now
@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." — Sid Meyersreplied to@kateprofilefollow:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." — Sid Meyersreplied to@kateprofilefollow:You forgot the “one, two, three”. :-)
what are ya'lls favorite wikipedia page?
I myself is partial to the 'variations' entry for high fives en.wikipedia.org/wiki/High_fiveen.wikipedia.orgHigh five - Wikipedia
…
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@nestorthecatprofilefollowgrocer-engineer and polynomial enjoyer. not into reactionaries
i attempt to stave off burnout with a healthy dose of dark souls, bloodborne, and friends:
creative technologist
creator of textlogreplied to@nestorthecatprofilefollowgrocer-engineer and polynomial enjoyer. not into reactionaries
i attempt to stave off burnout with a healthy dose of dark souls, bloodborne, and friends:You should check my other site onlycats.cc i think you will enjoy!
@nestorthecatprofilefollowgrocer-engineer and polynomial enjoyer. not into reactionaries
i attempt to stave off burnout with a healthy dose of dark souls, bloodborne, and friendsreplied to@stagas🤸profilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:love it. great concept
@hotterprofilefollow#Emacsnotesfollow, dev, life, random stuff. the netherlands.replied to@vko_pngprofilefollowpoet by heart 𓅰:
still five hops to Hitler
@stagas🤸profilefollow
creative technologist
creator of textlogreplied to@vko_pngprofilefollowpoet by heart 𓅰:
creative technologist
creator of textlogreplied to@vko_pngprofilefollowpoet by heart 𓅰:Catastrophic interference is also a favorite Wikipedia page and concept. The thing is, it doesn’t only happen to artificial neural networks, I’ve had it happen to me a few times. Especially with pin numbers, newer pins overwriting old ones and I’m always scared when I’m having hard time remembering something that this catastrophic interference applied and now the information is gone from my brain.