Everything to Know about On-Device AI on Apple Devices

While Apple might have been initially sidetracked by OpenAI’s ChatGPT, Google’s Gemini, or Microsoft’s Bard, it doesn’t mean the company has given up in its pursuit of AI capabilities. Reportedly, Apple is investing $1 billion per year on the development of AI technology. At WWDC 2024, we’re likely to see how these efforts will look in the company’s newest offering for iOS-based devices.

A Closer Look at AI on iOS

While there’s still lots of uncertainty around what exactly Apple will announce at WWDC 2024, some pretty advanced AI features for iOS 18 are among the rumors. Apple is treading in the AI area carefully, with Tim Cook stressing that artificial intelligence will be implemented only with enough prior consideration regarding safeguards. That’s why Apple’s initial set of iOS AI features will run on devices.

Apple’s AI efforts are named Project Greymatter and it could, in fact, be the biggest update that the company introduced to the operating system in recent years. iOS will feature smart summarizations of notifications and web content. Siri will provide more context-based replies thanks to access to calendar and locations.

Apple’s On-Device AI Models

We’ll go over the known on-device initiatives done by Apple, but which of them, if any, will eventually end up as the model used in iOS for on-device AI remains unclear (but most likely Ajax LLM). The one thing we can be certain of is that Apple’s on-device LLM will be a small and highly performant model used by iOS system applications, probably using Apple’s Neural Engine and possibly operating entirely offline (maybe with an exception for very complex tasks). It’s as yet unclear if the model will be available for developers to build apps with AI features processed on device.

OpenELM

In April 2024, Apple released a bunch of open-source language models called OpenELM (open-source efficient language models). OpenELMs are built for on-device operation as local language models. The OpenELM model family has LLMs in various sizes, making them adaptable for different applications. By releasing OpenELM, Apple aims to support and collaborate with the AI research community. This initiative is part of Apple’s broader efforts to integrate advanced AI features into their ecosystem, enhancing user experiences without compromising user privacy.

OpenELM is Apple’s high-efficiency language model for edge devices like smartphones and tablets. Released as an open-source project on Hugging Face, it supports text generation, code generation, translation, and summarization.

OpenELM Performance

Efficiency and accuracy

OpenELM models come in four sizes, with only two being small enough to process data entirely on device. Despite having only 270 million parameters at their smallest, the OpenELM model leverages approximately 18 trillion tokens from public datasets ensuring high performance even with a smaller parameter count.

Benchmark performance

In zero-shot tasks like ARC-e and BoolQ, OpenELM models outperform other existing models, demonstrating superior performance with fewer data and computational resources. The models are designed for resource efficiency, making them suitable for on-device applications on smartphones and laptops.

OpenELM Use Cases

Efficient on-device AI — Enhances on-device AI capabilities by running efficiently on local devices, providing advanced AI functionalities such as real-time language processing and interactive applications without relying on cloud servers.

Local content generation — Generates and summarizes text locally, helping content creators produce drafts, suggest edits, and condense information efficiently on their devices.

Real-time language translation — Supports multilingual translation directly on devices, facilitating real-time communication in different languages without needing an internet connection.

Privacy-enhanced customer support — Powers chatbots and virtual assistants directly on devices, offering real-time responses to customer inquiries while keeping user data secure and private.

Low-latency applications — Supports applications requiring real-time processing, such as interactive educational tools and real-time sentiment analysis, by utilizing efficient on-device computation.

Ajax LLM

Apple’s Ajax LLM is an advanced large language model. Apple has not officially released the Ajax LLM yet — it’s rumored to be announced at WWDC 2024. The Ajax LLM will enhance various iOS features by performing AI processing directly on the device, ensuring faster response times and improved privacy by reducing the need for cloud-based processing.

AJAX LLM Performance

Efficiency and privacy

AJAX LLM enhances the performance of various iOS features by processing data directly on the device, which reduces the need for cloud-based processing and improves privacy. This on-device processing method ensures that user data remains secure and private, minimizing the risk of data breaches. But there’s no official performance data released by Apple yet.

iOS Integration

The model is integrated into several iOS applications, such as Safari, Siri, and Spotlight Search. It powers features like Safari’s Intelligent Browsing for article summarization, Siri’s enhanced context-aware responses, and Spotlight’s improved search accuracy and relevance.

Ajax LLM Use Cases

Intelligent browsing — Powers Safari’s Intelligent Browsing feature, summarizing articles and highlighting key points for easier content consumption on-device, enhancing user experience and maintaining privacy.

Enhanced Siri — Boosts Siri’s capabilities by providing more accurate and context-aware responses, summarizing conversations, and generating relevant replies directly on the iPhone, ensuring faster and more secure interactions.

Smarter spotlight search — Improves Spotlight Search with AI enhancements to deliver faster and more relevant search results, making it easier for users to find information on their devices.

Text summarization — Enables on-device summarization of texts in apps like Safari and Messages, allowing users to quickly grasp the main ideas without reading through entire documents or conversations.

Privacy-focused processing — Ensures that most AI calculations are performed on-device, reducing the risk of data breaches and enhancing user privacy compared to cloud-based solutions.

Why On-Device AI?

Enhanced privacy

On-device AI processing ensures that user data is processed locally, significantly reducing the risk of data breaches and ensuring that sensitive information remains secure and private.

Improved performance

By processing AI tasks directly on the device, users experience faster response times and smoother interactions with AI-powered features such as Siri, Spotlight Search, and Safari’s Intelligent Browsing.

Increased reliability

On-device AI reduces dependency on internet connectivity, ensuring that AI functionalities remain accessible and efficient even in areas with poor or no internet connection.

Cost savings

On-device processing eliminates the need for continuous data transmission to cloud servers, which can reduce data usage and associated costs for users.

Personalized experience

AI models running on-device can tailor interactions and responses more closely to the user’s behavior and preferences, providing a more customized and intuitive user experience.

Apple’s Other AI Effort: MM1

Apple’s recent introduction of MM1 represents a significant step in its broader AI strategy, which aims to position the company competitively among leaders in the AI field. MM1, unveiled in March 2024, is a large language model designed to integrate diverse data types, including image captions, interleaved image-text, and text-only datasets. This multimodal approach enables MM1 to perform complex tasks such as visual question answering, image captioning, and natural language inference with a high degree of accuracy.

While MM1 is not yet publicly available, its development highlights Apple’s intention to leverage AI for more sophisticated and varied applications. The model’s design focuses on efficiently processing and understanding both visual and textual data, which can significantly enhance user experiences across different domains, from personal assistants to healthcare solutions.

New Opportunities for Developers to Build AI Apps

Apple’s AI initiatives show the company’s strategic focus on advancing AI technologies to enhance the user experience and expand the offering in a highly competitive market. With on-device AI, companies can give their users a fresh set of smart features to perform tasks faster and more efficiently.