A credit union wants to get their own data out of a vendor's online system. Vendor wants $160,000. I wrote a script with to download the data just as though a user would. It's going to run for about 6 weeks to download around 250k records and 70GB of images.
Any conversation
you could split to buckets like alphanumerically and parallelize it if you don't want to wait that long. Probably need to launch a few instances but still cheaper lol.
Added
#meta tag — use it whenever you want to talk about the site itself, and these posts will not show up in all, any or the post’s page. Only those that follow the tag or the person will see these posts in my feed or in post pages. This is to keep the site clear of meta discussions which dominated a lot at this early stage and I think were annoying to many people. So this is the last meta without the #meta tag you’ll see unless u follow. I’ll try to tag every post I find.I needed to process 20,000 json files & 20,000 JXL files. I'm pulling together a website from them all. Anyway, started with emacs elisp to handle it all, just because I've not done that before. Went kinda ok, and I'm gonna use org-mode to export it all to a website (done that bit loads of times before).
In the end, I ended up breaking out my C programming skills (late 80s->nowish). It was a delight. Nothing more refreshing than writing raw code, that doesn't rely on 3rd party code/libs.
…
Microservices.
Not really programming, as this is more on the architectural level.
Microservices is a valid approach when you need to address massive scale. The thing is, I have seen it applied to things that could realistically serve its audience in a handful of modest servers with a less fashionable approach.
It was done under the guise of simplification; each individual microservice was in fact very small. But this at the cost of absolute spaghetti architecture.
Nudged by an interesting read about time perception to try reclaim moments where I can deliberately feel bored again.
I was curious of testing the Omarchy distribution this week just because I had some free time to work on it. I saw a lot of hate around it and kind of get the point but so far at least for what I have been doing it wasn't terrible (at least at the point of writing this) although I am not trying to convince anybody to use it. But it just remember me how much do I hate MS.
Nice! Was an opportunity to install F droid also, didn’t know it can be installed like a normal app, I was under the impression it was an OS replacement. This is neat, perhaps we can deploy textlog app there? cc