Apple’s big changes to generative AI and LLM are here, and they will be deeply integrated into everything you do with Apple products.
On Monday morning, Tim Cook took to a virtual stage to describe the company's plans to move into "artificial intelligence," a term that collectively refers to large language models and techniques for generating images and text.
Apple's new foray into artificial intelligence is called "Apple Intelligence," and it promises a deeply integrated approach that will come to your Mac, iPad, and iPhone in a very Apple-like way.
As Cook said on screen, before moving on to dig into Craig Federighi's technology, Apple wants its AI to be powerful enough to be practical, intuitive, easy to use, personal and private.
Federighi talked about how new iOS, iPadOS, and macOS versions will include generative language, generative imaging technology, and new ways to take action on your behalf.
For example, he said, the new AI will be able to understand your natural language, prioritize notifications, and offer new writing tools system-wide so you can rewrite, proofread, and edit content across the operating system, Apple, and third parties. and summary text. Party apps (once developers start incorporating the new AI API into their apps).
You'll be able to create images on the go, personalizing content instantly, such as creating a cute little version of your birthday friend in Messages, or a cartoon image of your mom wearing a superhero cape. There are also three styles to choose from, sketch, illustration or animation.
You'll be able to ask Siri (which is getting new AI intelligence and a new look here soon) things like "Bring up the files my coworkers shared with me last week," name coworkers, and the operating system will Find what you need. Or, maybe better, “Play the podcast my wife sent me the other day.”
Apple Intelligence will use its own on-device generated models to answer your questions and create your images and text. It can extract information from all your apps, emails, documents, and even whatever is currently on your screen.
Federighi noted that when a query exceeds what the device technology can handle, it can be passed encrypted to private cloud computing servers that Apple will maintain using its own silicon. Apple will never store or access data stolen from your devices, and the company has ensured it can be transparently and independently verified.
Speaking of Siri, Director of Machine Learning and Artificial Intelligence Kelsey Peterson demonstrated some cool new features for the 13-year-old personal digital assistant. First, Siri can now retain context when you speak to it and can handle verbal corrections when you make a request. Additionally, you can type into Siri and instantly switch between voice and text prompts. It also inherits screen awareness, so it can work with what you're looking at, and gets "hundreds" of new actions that can be managed across apps, while also being able to generate text and images.
Integration with data from all your devices can also help Siri understand your requests more easily, such as grabbing a photo from your driver's license, scanning the number and putting it into an online form. Now it also has access to real-time flight tracking and other data to help you manage plans you never put into your calendar. spooky? perhaps. it works? perhaps.
Apple showed off writing technology in Mail, including examples of writing and rewriting emails using different wording and more, as well as the ability to summarize emails in real time.
Image generation will be in the form of Genmoji, allowing you to create your own emoji stickers based on text prompts. Want a lizard on a skateboard? Make your own source words. There's also a new Image Playground app that puts prompts and small images at your disposal, each created in one of the three styles mentioned above: Sketch, Illustration, or Animation. This playground also refers to the technology, which will be available as an API to application developers.
Need images to populate a keynote or school presentation? Circle any sketch you make on your iPad, and Apple Intelligence creates an image to make it look better. This new feature is called Image Stick.
In addition, Notes features snippets and new recording and transcription technology, which is also available in the mobile app. If you are on a call and start recording, it will let everyone know the call is being recorded.
Finally, for what Apple Intelligence can't yet do, Federighi said that Apple is adding integration with ChatGPT 4o and will launch more models later.
