Microsoft’s BitNet shows what AI can do with just 400MB and no GPU – TechSpot

🔍 Summary:

Microsoft has unveiled a groundbreaking AI model, BitNet b1.58 2B4T, which stands out for its efficiency and ability to run on standard hardware. This model uses a unique ternary quantization method, representing weights with just three values: -1, 0, or +1, allowing each weight to be stored in only 1.58 bits. This drastically cuts down memory usage to just 400MB, significantly less than the typical requirements of similar-sized models.

Developed by Microsoft’s General Artificial Intelligence group, BitNet b1.58 2B4T boasts two billion parameters and was trained on a massive dataset of four trillion tokens. Despite its low-precision approach, it matches or even surpasses the performance of other leading models in tasks like grade-school math and common sense reasoning.

A key component of BitNet’s performance is the custom software framework, bitnet.cpp, optimized for the model’s ternary weights and currently tailored for CPU usage, with plans to expand support to other processors. This model not only reduces the dependency on high-end GPUs but also significantly lowers energy consumption by 85 to 96 percent compared to full-precision models.

However, BitNet does have limitations, including a smaller context window and specific hardware requirements. Future developments aim to enhance its capabilities, including support for more languages and longer text inputs. This innovation could potentially allow for the running of advanced AI directly on personal devices, reducing costs and environmental impact.

📌 Source: https://www.techspot.com/news/107617-microsoft-bitnet-shows-what-ai-can-do-400mb.html