derefr5 hours ago
> The model weights and inference code need to be contained within 25KB of user-space memory
Wouldn’t it be era-appropriate to allow relying on banked memory? You’d still need to hold the inference code, but you could effectively stream(ing page) the weights as you compute on them.
striking3 hours ago
From a look at https://en.wikipedia.org/wiki/BBC_Micro#Specifications I think the 6502 versions of the beeb didn't have banked RAM so to keep it loadable from tape the limits might be as stated.
But with substantial additional effort, maybe some banked ROMs could be added..?
frohan hour ago
the banking was done via an external ram controller whose bank control register was mapped to some (unbanked) controller address.
bmc75057 hours ago
Cool to think this demo would have been possible over fifty years ago. I wonder what someone from 1975 would have said if you had shown this to them back then.
tyromaniac7 hours ago
This is super cool! As someone who's worked a little with NES programming and tried out cc65, I'm surprised he didn't just hand write some assembly, he likely couldve saved a lot of space if I had to guess.
torment-nexus4 hours ago
The biggest win for AI dev efficiency is cutting down what gets loaded into context. Semantically matching tasks to the top tools helps a lot.
actionfromafar7 hours ago
The 6502 is notoriously unfit for a C compiler, so probably there is room for more performance in the future. :)
toplinesoftsys7 hours ago
This is amazing project! I hope it will result in real miniaturization of AI - for example, edge LLM inside of glasses. That will be awesome.
aghilmort2 hours ago
really great work
[deleted]7 hours agocollapsed
[deleted]4 hours agocollapsed
leonmeng3 hours ago
[flagged]