AI Is Coming for Your Phone in a Big Way

The smartphone is about to evolve in a big way. For years, industry leaders pegged the arrival of 5G and foldable displays as being the technological advancements that would mark a turning point for the smartphone. 

Now in 2023, that excitement has shifted to generative AI, a technology that moguls like Microsoft co-founder Bill Gates and Alphabet CEO Sundar Pichai believe could be as monumental as the dawn of the smartphone and the internet itself. 

Generative artificial intelligence, or AI that can create new content, engulfed the tech world this year, shaping the trajectory of new products from Alphabet's Google, Microsoft, Instagram and Facebook owner Meta, and other major tech companies. In the last quarter of 2023, we're getting a better idea of how that shift is extending to the smartphone. 

AI was at the center of Google's Pixel 8 launch earlier this month as the company flaunted how its algorithms can pick out the best facial expressions in batches of group photos and easily paste them into a different image. The next smartphone processor from leading mobile chipmaker Qualcomm is designed to speed up AI processing tasks, as the company detailed during its Snapdragon Summit in late October. And Apple is said to be developing a bevy of new features for the iPhone and other products that lean on AI, according to a recent report from Bloomberg.

"AI is the future of the smartphone experience," Alex Katouzian, senior vice president and general manager of Qualcomm's mobile, compute and XR division, said during the company's event. "And when it comes to mobile technology, it's what we've been building toward for over a decade."

ChatGPT and new AI-powered search tools from Microsoft and Google may have been the jumping-off point for generative AI in late 2022 and the first half of 2023. But it didn't take long for the tech to impact the direction of current and future smartphones. AI isn't new to mobile devices; features like voice dictation, language translation and facial recognition already leverage the technology. Apple's iPhones, for example, have had a neural engine for handling machine learning-related tasks since 2017 starting with the iPhone X and iPhone 8 generation.

Generative AI is different. It's not necessarily about AI working passively to detect a person in a photo or unlock your phone as soon as you raise it to your face. Instead, it enables use cases that feel new, whether that's generating fresh wallpapers on demand or adjusting someone's expression in a photo.

"It's really about giving users control over the use of AI and how they want to use it, how they want to apply it to their daily lives," said Subhashish Dasgupta of analytics firm Kantar, who specializes in the technology and health industries. 

Google's Pixel 8 and 8 Pro are the biggest examples of this yet. While both phones include routine upgrades like a new processor and slightly improved camera hardware, it's the phones' new AI-powered tricks that make them stand out. 

A photo editing feature called Best Take can analyze group selfies you've recently snapped and swap facial expressions between images. In other words, Google's AI can create the perfect photo in which everyone was smiling -- even if that moment never happened. 

Magic Editor similarly uses AI to let you manipulate photos in myriad ways: You can make yourself larger or smaller, create the illusion that you're jumping superhumanly high, or even replace the ground itself. 

Those are just the latest examples of how Google is trying to give AI a more prominent presence in our phones. Another feature, called Magic Compose, which Google announced at its developer conference in May, uses generative AI to craft suggested responses to text messages or rewrite responses in a different tone within the company's messaging app. Google's AI wallpaper creator can conjure up new backgrounds for your phone from scratch based on prompts. 

In 2024 and beyond, AI is poised to play an even greater role in our smartphones. Qualcomm, which creates chips that power phones from companies like Samsung, Xiaomi and OnePlus, clearly had AI in mind when developing its new Snapdragon 8 Gen 3 mobile processor. Qualcomm CEO Cristiano Amon showcased the company's vision for how AI will change smartphones at its Snapdragon Summit in Hawaii on Oct. 24.

The chipmaker sees AI as being another layer that runs alongside the phone's operating system and apps that can understand voice, text and image inputs and provide suggestions. That's much different from the virtual assistants we've come to know over the last decade, like Apple's Siri, Amazon's Alexa and the Google Assistant. Rather than thinking about AI at the individual feature level, Qualcomm sees it as being a fundamental part of your phone's daily operations in the future. 

"It's really kind of voice assistants 2.0," said Luke Pearce, a senior analyst at tech research and advisory firm CCS Insight. "And it's much more conversational, it's much more natural." 

During its keynote, Qualcomm provided a glimpse at what this future could look like. A teaser video demonstrated a virtual assistant that can extract key topics from a phone call, summarize them into bullet points and then provide recommendations. Another example showed an AI helper picking out keywords from a text messaging thread between two friends to proactively provide recommendations for their meetup. A photography tool powered by the company's new Snapdragon 8 Gen 3 processor will let you zoom out on a photo you've already captured and generate more details beyond the frame to make it look like it was taken on a wide-angle lens. 

"It changes how we think about the devices, the [operating system] and the apps, and how you actually define a user experience," Amon said on stage.

Qualcomm is wasting no time getting started. It's working with Google so that the search giant's AI models will be able to run locally on future Snapdragon-powered phones. Xiaomi also unveiled its new Xiaomi 14 phone, which will be one of the first devices to run on Qualcomm's new chip, at the event. But other brands like Oppo, OnePlus, Honor and Vivo will also adopt the chip, likely next year. Samsung typically puts Qualcomm's latest processor in the US version of its flagship Galaxy S and foldable Galaxy Z phones, although the Galaxy S23, Galaxy Z Flip 5 and Galaxy Z Fold 5 devices run on a customized version of the Snapdragon 8 Gen 2.

Not to be outdone, Motorola is also working to bring more AI features to its smartphones. Lenovo, which owns Motorola, is developing an AI assistant for PCs and phones that learns from your behavior and can accomplish tasks on your behalf, such as writing messages and scheduling tasks.

Between Google's Pixel 8 launch and Qualcomm's Snapdragon Summit, there's no doubt the Android landscape in 2024 will be all about AI. But what about the iPhone? Apple rarely, if ever, discusses future products. However, Bloomberg's Mark Gurman reports that Apple is developing a slew of AI upgrades for its family of products, including the iPhone. 

Craig Federighi, Apple's senior vice president of software engineering, is reportedly spearheading an effort to bring AI to the next major iPhone update, likely to be called iOS 18. Apple wants to sprinkle its large language learning model into the software, the report says, which could show up in Siri and Apple's messages app. Apple's suite of apps may also get an AI makeover according to Bloomberg, with the company reportedly looking to bring auto-generated playlists to Apple Music and AI-assisted features to Pages and Keynote. 

Apple didn't immediately respond to CNET's request for comment.

But to get the personalized experiences that companies like Google and Qualcomm envision, you'll have to give tech giants an even bigger window into your life. After all, how can these devices provide recommendations based on your text messages or sum up your phone calls without the AI being dialed into your communications?

"What the AI is trying to do, is trying to predict you," Amon said during Qualcomm's keynote.

That's a key reason why Qualcomm is pushing the benefits of its Snapdragon 8 Gen 3 chip, which it claims is powerful and efficient enough to execute AI tasks on the device locally without relying on the cloud. Doing so means your information wouldn't have to leave your phone, which is better for privacy and security. It can also make more personal suggestions based on your behavior, frequently visited locations and other lifestyle patterns without sending that highly individual information through the cloud.

Apple takes a similar approach to personal data. Since the Apple Watch Series 9 and Apple Watch Ultra 2 can process verbal requests on-device thanks to Apple's new S9 processor, Siri will be able to answer health-related questions on those devices. 

"The more sensitive the data, the more personal the data, the more that will be processed on device," Pearce said. 

There's also the question of whether AI will toe the line between fact and fiction when it comes to images and videos created on our phones, as my colleague Sareena Dayaram recently pointed out. Tools like Magic Editor and Best Take go a step beyond polishing and editing photos. They change what's happening in a picture. 

"Now you're putting a lot more power into the hands of a lot more people," said Dasgupta. "And hence the opportunity exists for it to be misused in many more ways." 

Qualcomm and Google have thought about this. The chipmaker partners with Truepic to verify the authenticity of images and signal whether they were taken with AI. Google also recently launched an "About this image" tool that provides more context about the origin of images that appear in search results. 

It's unclear whether those protections will be enough to prevent AI from being used in misleading or malicious ways. But what is clear is that AI will change the way we use what has become the most important device in our lives, for better and hopefully not for worse. 

"There may be a killer use case that doesn't exist yet," said Pearce. "But that will come around the corner and surprise us all and become completely indispensable."

Editors' note: CNET is using an AI engine to help create some stories. For more, see this post.