pourover ritual :pray:
Any conversation
ไธญๅฝ็ๅผๆบๆจกๅๅจๆ้ซๆๅฟ็ไธปๅฏผไธ๏ผๅจ้ข่ฎญ็ปๆฐๆฎ้ถๆฎต๏ผๅฐฑ็ญ้คๅคง้ๅฏนๅ
ถไธๅฉ็ๅ
ๅฎน๏ผsft้ถๆฎตๆๅปบ็ๆฐๆฎๆบๅ
ฅๅคง้ๆฟๅ
ถ่พฉๆค็ๅ
ๅฎนใๆฅ้ไธไธช็บ ๅๆฐๆฎ้็จไบๅพฎ่ฐ
Under the overarching influence of a single, powerful force, China's open-source models filter out a large amount of content unfavorable to their designs during the pre-training stage, and then incorporate a significant amount of content defending their designs into the data constructed during the SFT stage. A corrective dataset is urgently needed for fine-tuning.
โฆ
NIH syndrome? I am guilty of it too. I love implementing things from scratch, you learn a lot by doing so and you get to own the codebase end-to-end, so no surprises and fighting around othersโ choices. It can get exhausting too though, in the end I succumb to using off-the-shelf components because my primary goals move at a higher level.


