Indian Institute of Technology Madras (IIT Madras) and IITM Pravartak Technologies Foundation is partnering with Ziroh Labs, a California-based Innovation-driven Deep-Tech Startup, and to establish an Centre of AI Research (COAIR) to solve India’s compute accessibility challenges. The COAIR will focus on developing practical, efficient AI solutions focusing on CPU and edge device inferencing. In a significant step towards making AI Models solve accessibility problems, Ziroh Labs unveiled the first version of ‘Kompact AI at IIT Madras today (9th April 2025). Kompact AI is an AI platform that enables foundational models to be built and served using CPUs without requiring GPUs (Graphics Processing Units, which are expensive and hard to obtain).
Kompact AI is aligned with India’s AI Mission, “AI for All”. Kompact AI has been developed by scientists and engineers from India. Ziroh Labs has already optimised 17 AI models, including DeepSeek, Qwen and Llama, to run efficiently on CPUs. These models have been benchmarked with IIT Madras, evaluating both quantitative performance and qualitative accuracy. For the first time, high-performance AI is built and deployed efficiently on CPUs, making AI more accessible, scalable, and cost-effective.
Ziroh Labs team showcased Kompact AI’s unique capabilities during the event and demonstrated how it redefines AI development on top of CPUs by eliminating dependency on GPUs.
The demonstration took place in the presence of Prof. V. Kamakoti, Director, IIT Madras, Dr Whitfield Diffie, Fellow of the Royal Society and Turing Awardee, Mr Scott McNealy, Founder of Sun Microsystems, Dr William J Raduchel, Former CTO of AOL Time Warner and Chief Strategy Officer at Sun Microsystems, Prof. S. Sadagopan, Former Director, IIIT-Bangalore and Mr Raj Singh, Founder, APIGee, Cerrent, Fiberlanem, alongside other students, faculty and researchers.
Key Features of Kompact AI:
● Kompact AI enables AI model development and deployment on CPUs, removing the need for costly GPUs and reducing the infrastructure burden.
● Ziroh Labs has optimized AI models, including DeepSeek, Qwen, and Llama, for efficient execution on CPUs, delivering superior performance at lower costs.
● Kompact AI operates without constant internet connectivity, making it ideal for deployment in remote areas with limited or no access to reliable internet.
Elaborating on the need for such developments, Prof. V. Kamakoti, Director, IIT Madras, said, “Nature has taught us that one can effectively acquire knowledge and subsequently infer in only a limited set of domains. Attempts to acquire everything under the universe are not sustainable and bound to fail over a period of time.”
Prof. V. Kamakoti added, “This effort by Ziroh Labs and IITM Pravartak is a nature-inspired one wherein they provide a platform that uses customized and trained domain-specific models to provide accurate inferences on affordable conventional compute machines. This effort is certainly a major step in arresting the possible AI divide between one who can afford the modern hyper scalar systems and one who cannot.”
Kompact AI can play a transformative role as it leverages multiple optimisations algorithmically and implementation-wise to execute models on the low end without any support required for external communication, such as the Internet. As a part of this upcoming CoAIR, IIT Madras will work on smaller AI models for specific use cases, leveraging Kompact AI to optimise execution, making high-performance AI more efficient, accessible, and relevant to real-world applications.
Dr. Madhusudhanan B, Principal Consultant - IITM Pravartak Technologies Foundation, added, “At IITM Pravartak, our mission is to translate research innovations into tangible solutions that address real-world problems. Kompact AI is a game-changer in this regard. By making AI models executable on CPUs, we are unlocking the potential of AI without the heavy infrastructure costs typically associated with GPUs. Through the CoAIR, we will build specific, actionable AI models that can be deployed at scale, benefiting people in rural and underserved areas across India. This is about taking technology beyond the labs and putting it into the hands of those who need it the most.”
Speaking at the launch of Kompact AI, Prof Sadagopan, Former Director – IIIT-Bangalore, said, “By combining affordability/availability of AI development and deployment tools - using CPUs without excessive dependence on GPUs - Kompact AI proves once again that India can do frugal innovation at the cutting edge of technology. Even more interesting is the fact that such a solution using far less electrical power is sustainable. It is also aligned with India’s AI Mission, “AI for All” – from India for the world.”
“We congratulate Ziroh Labs on its AI CPU launch which promises to significantly increase the accessibility of AI. HCLTech is excited to partner with Ziroh Labs on this exciting journey,” said Apoorv Iyer, Executive Vice President and Global Head, GenAI Practice, HCLTech.
According to Hrishikesh Dewan and Igneta DSouza Co-founders, Ziroh Labs said, “AI has the potential to transform every aspect of our lives, from improving healthcare outcomes to enhancing education and solving agricultural issues. However, for AI to benefit everyone, it must be accessible, affordable, and adaptable to real-world conditions. Kompact AI is designed for such needs and operates efficiently on CPUs, optimized for task-specific applications. This ensures that communities and enterprises with limited infrastructure can leverage the power of AI to create practical, impactful solutions. Our collaboration with IIT-Madras and IIT-M Pravartak Foundation will aim to address multiple challenges. We are grateful for this partnership, as teams from IITM have been pillars of support during the entire Kompact AI development journey.”
This AI platform aims to democratise AI by allowing developers across the globe to build, train and infer AI using CPUs commonly found in Cloud Data Centers and Edge Devices. Kompact AI enables AI development without violating any data privacy and data residency regulations across the globe.