Hello! Is anyone else trying local AI? Im dont know what im doing but lately ive been trying "Llama 3,2 3b instruct q5km" on my phone and its been surprisingly good.
New here, wanted to try textlog
Its to replace using central ai for basic tasks like text editing, analysis and brainstorming.
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain · A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.replied to@candlewaxscribe😊profilefollowNew here, wanted to try textlog:
If you have Mac, you might check out this app: blaizzy.github.io/nativ/
@jamesjan3profilefollowresearcher, developer, dancerreplied to@candlewaxscribe😊profilefollowNew here, wanted to try textlog:
i think the best you can get right now is qwen 3.8 27B. youre gonna need 24 gigs of some fast memory though, i heard theres decent quantizations that can also make it run on 16. im quite excited about open/local models, but theres big investment overhead with the prices right now
@keitachikawa😎profilefollowaeres-u99 on github, CodeAtlas \o/replied to@candlewaxscribe😊profilefollowNew here, wanted to try textlog:
You might be able to try out colibri, its quite good and the models are a bit better. I am unable to connect any of the cli agents to do things with it though, probably due to context limits, but try it out. OlMoe is the lightest