Power and money somehow emit the whackadoodle in all of us.
Any conversation
中国的开源模型在最高意志的主导下,在预训练数据阶段,就筛除大量对其不利的内容,sft阶段构建的数据掺入大量替其辩护的内容。急需一个纠偏数据集用于微调
Under the overarching influence of a single, powerful force, China's open-source models filter out a large amount of content unfavorable to their designs during the pre-training stage, and then incorporate a significant amount of content defending their designs into the data constructed during the SFT stage. A corrective dataset is urgently needed for fine-tuning.
there's no fundamental physical limit that prevents silicon from developing taste and judgement, but language is a lossy compression of reality.
taste comes from experience, from the shadowy mass of episodic memory, which swirls and convects at the boundary of consciousness. the ideas recombine into new ones in the nuclear reactor of the hippocampus.
but no matter how imbued with meaning and expertise is a handmade wardrobe we still get our shit from ikea
I notice that language models are much better at programming than at writing. This is largely because, in programming, they have a closed feedback loop that allows them to find errors. Writing is much more complicated because the feedback is much more diffuse and delayed over time. In fact, it is often not well-defined.

