Returned from the gym. Good day.
Any conversation
"200K lines is not 10 times as much as 20K lines - from the mission perspective it's just irrelevant. And yet it's the easiest thing to measure. And so we look for the keys under the lamp post." ~Kent Beck
www.youtube.com/watch?v=F8fBgDCf2Y4&t=2753s
ไธญๅฝ็ๅผๆบๆจกๅๅจๆ้ซๆๅฟ็ไธปๅฏผไธ๏ผๅจ้ข่ฎญ็ปๆฐๆฎ้ถๆฎต๏ผๅฐฑ็ญ้คๅคง้ๅฏนๅ
ถไธๅฉ็ๅ
ๅฎน๏ผsft้ถๆฎตๆๅปบ็ๆฐๆฎๆบๅ
ฅๅคง้ๆฟๅ
ถ่พฉๆค็ๅ
ๅฎนใๆฅ้ไธไธช็บ ๅๆฐๆฎ้็จไบๅพฎ่ฐ
Under the overarching influence of a single, powerful force, China's open-source models filter out a large amount of content unfavorable to their designs during the pre-training stage, and then incorporate a significant amount of content defending their designs into the data constructed during the SFT stage. A corrective dataset is urgently needed for fine-tuning.
