While last year it was still a matter of tapping into the potential of ChatGPT and similar services via browsers and apps, efforts are now being made to run generative AI directly on the end device or in a hybrid mode. From the device manufacturers’ point of view, this newly awakened interest is understandable, as AI offers the opportunity to get the somewhat dormant PC and smartphone market back on track. But do users in the enterprise also benefit from this?
On-device-AI from Qualcomm
One of these players is Qualcomm. The company has already laid the foundation for corresponding AI-optimized chips on the device with the Snapdragon 8 Gen 3 smartphone chipset and the Snapdragon X Elite notebook processor unveiled at the end of 2023, both with an integrated NPU (Neural Processing Unit). While notebooks equipped with the Snapdragon X Elite are not expected until the summer, the newly unveiled Xiaomi 14, Xiaomi 14 Ultra and Honor Magic 6 Pro are recent examples of smartphones with Qualcomm’s Snapdragon 8 Gen3 that already use on-device AI in different ways.
At Mobile World Congress in Barcelona the company demonstrated the performance of the Snapdragon SoC on an Android smartphone using the Large Language and Vision Assistant (LLaVA). According to Qualcomm, this is the first multimodal large language model (LLM) to run on a smartphone. The model, which reportedly has more than seven billion parameters, can accept not only text, but also images and speech as a prompt. In one of the demonstrations, images of different ingredients are shown, after which a recipe for these ingredients is created offline and the calorie count of the resulting meal is estimated.
In another demo, the open-source graphics program GIMP is demonstrated with a Stable Diffusion plugin on a Snapdragon X Elite laptop alongside an x86 laptop running Intel Core Ultra 7 to demonstrate the benefits of hardware designed for generative AI. Qualcomm claims that the Snapdragon box is three times faster at image generation than the system without an NPU.
A developer platform for AI models
In addition, Qualcomm unveiled the AI Hub. The platform offers a library of pre-optimized AI models for seamless deployment on devices powered by Snapdragon and Qualcomm platforms, including smartphones, PCs, AR/VR devices, and other devices.
The list includes not only well-known generative AI solutions such as Stable Diffusion, Llama or ControlNet, but also various models for speech recognition, image classification, object recognition, image superscaling and the like. According to Qualcomm, developers can run the models with just a few lines of code, even on cloud-hosted devices running Qualcomm platforms.