With Shared Microexponents, A Little Shifting Goes a Long Way — mentioned by chessgecko

Wed, 10 Apr 2024 17:49:25 GMT

I thought MTIA v2 would use the mx formats https://arxiv.org/pdf/2302.08007.pdf, guess they were too far along in the process to get it in this time. Still this looks like it would make for an amazing prosumer home ai setup. Could probably fit 12 accelerators on a wall outlet with change for a cpu, would have enough memory to serve a 2T model at 4bit and reasonable dense performance for small training runs and image stuff. Potentially not costing too much to make either without having to pay for cowos or hbm. I'd definitely buy one if they ever decided to sell it and could keep the price…

Paper and discussion history. A mention is not an endorsement.

With Shared Microexponents, A Little Shifting Goes a Long Way · HackerPapers discussions

With Shared Microexponents, A Little Shifting Goes a Long Way — mentioned by chessgecko