MediaTek's AI-Powered Chips Aim to Make Your Next Phone Very Personal

Phone makers like Google and chip makers such as Qualcomm suggest that 2024 will be the year of on-device AI. Last month at its annual Executive Summit, MediaTek, the producer of the silicon that ends up in millions of phones and smart devices, highlighted the advantages to using generative AI on phones. For instance, getting more personalized answers and suggestions faster than with cloud-based solutions, with few ways the tech will make phones wildly better.

But there are a few concrete use cases on the horizon, explained David Ku, MediaTek's Chief Financial Officer. Vivo has a phone coming out next year with a feature that will record a meeting, transcribe it and provide a summary, which could be used in sensitive boardrooms since the processing is done on-device.

"If it's a private meeting, you can keep the information confidential. You don't want to submit this recording to the cloud, so you can do it on the device," Ku said.

Read more: 2023 Has Been the Year of Generative AI. It's Coming to Phones in 2024

In October, rival chipmaker Qualcomm added generative AI capabilities directly into the chips that will power the next wave of premium Android phones, like the Xiaomi 14. While MediaTek didn't offer as many specific use cases for its generative AI, both companies have proposed that the technology will offer more personal suggestions by considering people's behavior patterns and image manipulation like expanding photos beyond their original boundaries.

Add to this MediaTek's new Redcap (short for "reduced capacity") 5G modems and chips which aim to help devices with lower data loads (think smart home devices, earbuds and accessories), and you have the groundwork for enabling more devices used by people and companies to talk to each other. MediaTek's announcements represent a modernization of old tech into the connected future we've heard is coming for years.

The vague new frontier of on-device generative AI

There are many proposed advantages to using generative AI on phones, like getting more personalized answers and suggestions faster than with cloud-based solutions, with few ways the tech will make phones wildly better.

Two potential generative AI features Ku suggested were using it to scrub out unwanted objects in photos, much like Google's Magic Eraser features works on Pixel phones (and the Google One cloud photo storage service), and auto-drafting text to go along with photos to be posted to social media.

Ku also noted how generative AI could improve a phone's personal assistant features with contextual suggestions. If you're running late for a meeting and your phone can tell you won't make it, it could offer to send a message to others that you're on your way. And if you've got your phone on mute but an urgent email hits your inbox, the assistant could warn you anyway.

MediaTek's two new phone chips, the Dimensity 9300 for top-end phones and Dimensity 8300 for less premium devices, both have on-device generative AI, but the company only released performance info for the former, claiming it can generate an image using Stable Diffusion in less than one second and can run AI models with 7 billion parameters per second at 20 tokens per second. The Snapdragon 8 Gen 3 can reach the same tokens-per-second rate, but the metric doesn't have many comparisons beyond on-device AI for Qualcomm's PC chip (30 tokens per second).

The lower-end Dimensity 8300 can handle language learning models with up to 10 billion parameters, while the Dimensity 9300 is capable of LLMs up to 33 billion parameters.

To be fair, nobody's nailed down a definitive metric to compare generative AI capability. For now, 'tokens per second' and 'trillion operations per second' are the most used, but MediaTek is working with different ecosystem partners to figure it out, Ku said.

"We're trying to provide more detailed guidance and what's the true benchmark that will be more meaningful from a user perspective," Ku said.

Watch this: CNET's Pro Photographers React to AI Photos

09:12

With new frontiers comes challenges. There are already worries that people will use generative AI to flood the internet with fake images and other misinformation. When introducing its on-device generative AI, Qualcomm pitched its own solution: partnering with authentication platform Truepic to digitally watermark any images altered or outright created with generative AI. MediaTek is working with different partners to do a similar kind of authentication that will work like an "invisible watermark," Ku said -- and no matter if generative AI has been used to modify 50% or only 5% of the photo, it'll get that watermark.

"Maybe that's the most important 5%," Ku said. "Maybe it's your face."

Read more: How Close Is That Photo to the Truth? What to Know in the Age of AI

Beyond phones, MediaTek hopes generative AI could someday help in the automotive space. After reaffirming its partnership with GPU maker Nvidia on the summit stage, the company speculated that cars could use the technology paired with outer cameras to identify objects ahead and, paired with gestures, clarify something in the distance that the driver is pointing at. Cars could even notice erratic driving and ask if the driver is drowsy, or alert them when the light turns green. 

The personal assistant could come in handy in the car too, Ku noted. On a hot day driving up temperatures in cars, drivers will probably turn on the AC or lower the window immediately after turning on their vehicle, but an AI could notice when things have cooled down and nudge the driver to roll up their windows or dial down the chilly AC. And if they hit traffic on their way to a meeting, the assistant could send a message telling other attendees the driver will be late.

A more connected 2024

While devices may use generative AI without connecting online, there are big benefits to improving connectivity to networks and between devices. MediaTek's first chips supporting Wi-Fi 7 were released last year, and this year's chips improve on it -- though the company is looking beyond that, too.

"As we look at these technologies like Wi-Fi 7 and moving on to Wi-Fi 8, they're all driving much much higher data rates, but more importantly, things like lower latency," Moynihan said.

Wi-Fi 7 brings several advantages over its predecessor, like multi-link operation, which lets devices tap into the 2.4GHz, 5GHz and/or 6GHz Wi-Fi bands all at once to get the best connection (similar to how carrier aggregation uses multiple 5G spectrum frequencies to get better signal for phone users). And Wi-Fi 7 is coming to MediaTek's chips meant for next year's fixed wireless access terminals coming from leading carriers, letting folks tap into those connectivity advances using their home 5G internet service. 

Read more: Best phones of 2023

RedCap 5G harnesses the lower latency and other benefits of 5G mobile networks at a lower cost, enabling cheaper devices that can still benefit from the next-gen networks. Current 5G modems deliver power, performance and range, Moynihan said – which is great for mobile phones but overkill for smaller IoT and smart home solutions. 

"You don't need 10 gigabits per second, but what you really need is a couple of hundred megabits per second," said Moynihan. Thanks to the 5G standalone networks developed by carriers and new devices created to follow the industry coalition 3GPP's Release 17 set of standards, more simplistic gadgets will be able to tap into 5G with low latency and power consumption. 

And in the future, connectivity could have on-device generative AI tie-ins as the technology spreads through everyone's homes as so-called "ambient computing" takes off, Moynihan says. That might look like your network of smart devices using your preferences as a guide for shaping your environment. Instead of having to preset which lights and level of heat or air conditioning you want when you step in the door, the devices can automatically decide this for themselves based on what you like and when you're coming home, then activate a different setup for others in the household.

MediaTek and augmented reality

Like its rivals, MediaTek has been exploring where its chips can go next, including in augmented and virtual reality. At last year's summit, the company revealed that its chips were powering Sony's PSVR2, the virtual reality headset paired with the PS5. On this year's summit stage, MediaTek had another partnership to unveil: it would be working with Facebook parent company Meta to develop a custom AR/VR chip. 

While MediaTek didn't specify what kind of device will use its chip, it's not hard to imagine that it could end up in either an AR/VR headset like the Meta Quest 3 or a successor to the Ray Ban Meta Smart Glasses. Since those Meta devices are each powered by Qualcomm's new Snapdragon XR Gen 2 and AR1 Gen 1 chips, respectively, it's unclear if MediaTek will usurp the chip producer role for their successors or use its forthcoming silicon in a new project.

Read more: Qualcomm's Newest VR/AR Chips Tell a Lot About Quest 3 and Beyond

What is clear is that MediaTek is entering the AR/VR space differently than the competition. Qualcomm released its silicon earlier this year as general chips for mixed-reality devices. But MediaTek worked with Sony and will work with Meta on chips customized for each brand's individual needs, Moynihan said, and will look to partner with more brands in the future.

"We have certain technologies that we can bring, and in the future, there's connectivity opportunities, whether it's Wi-Fi or even RedCap or some other technologies," Moynihan said.

Editors' note: CNET is using an AI engine to help create some stories. For more, see this post.