Everything Apple just announced at WWDC24: Apple Intelligence, Siri with ChatGPT, iOS 18 and more

Apple’s annual Worldwide Developers Conference keynote just wrapped up.

The company had a lot of news, the biggest of which was its push into artificial intelligence, called Apple Intelligence. It also announced that users can tap into OpenAI’s ChatGPT when using Siri, but only if a user wants to.

Apple will release developer versions of its software beginning Monday before it is available in a public preview next month.

Final versions of the software will launch for everyone once fully finished in spring.

Siri will tap into OpenAI’s ChatGPT and will be free

Apple announced that Siri can tap into OpenAI’s ChatGPT when needed. The company said it wanted users to be able to tap into external models.

Siri will ask if you want to share your question with ChatGPT and then will return suggestions from the OpenAI chatbot. ChatGPT is also built into system wide writing tools.

So, for example, Apple said you can create a bedtime story for a child and add images created by ChatGPT. It will be free, without creating an account, and your request and information won’t be logged, Apple said.

The integration will come into iOS 18, iPadOS 18 and macOS Sequoia later this year. It said it will also add other third-party artificial intelligence models.

Apple introduces custom AI emojis called Genmoji

Apple introduced a new form of custom emojis called Genmoji.

Users can create their own emojis using Apple’s artificial intelligence for when they can’t find quite the right emoji to express what they’re trying to say. Users will also be able to create Genmojis that look just like their friends.

You can create a Genmoji right in your keyboard by typing in a description. Apple will create a few options for you to choose from.

Apple is talking about its AI enhancements for writing

Apple’s new Writing Tools are available system wide, including on third-party apps. Improvements include being able to rewrite your existing copy, change the tone of language, summarize a message in “TLDR” format and pen Smart Replies to messages.

Apple announces big new Siri update with AI

You can now use Siri via text, correct your statements in real-time and offer a new look that is more tightly integrated into the operating system.

It will be powered by Apple Intelligence, the company’s artificial intelligence product. Siri now has the ability to answer “thousands of questions” about how to use your Apple device, even if you don’t know exactly what the feature is.

Siri will now be able to take action in and across apps, such as sending an article from Apple News to a group thread in Messages. Third-party apps will also be able to tap into some of Siri’s enhancements.

The same “personal context” that Apple touted broadly will also come to Siri, allowing it to tap into a broader swath of information and data on your device. For example, it can search through your whole device, so you can ask it to “show me things I sent Todd last week.”

Apple announces its AI push, Apple Intelligence

Apple announced its long-awaited artificial intelligence push, Apple Intelligence.

CEO Tim Cook emphasized the need for privacy and personalization, “beyond artificial intelligence,” into “personal intelligence.”

Apple’s Craig Federighi said the generative models behind Apple Intelligence would be available across iOS, iPadOS and macOS.

Here are some of its capabilities:

• Context-driven notifications: Apple Intelligence can recognize which notifications are important to your personal context.
• Writing improvements: Apple Intelligence will introduce systemwide proofreading and style improvements across third-party and native apps.
• Image generation: Apple Intelligence can create generative photos based on your photo library, similar to some other platforms. There are three styles: Sketch, Illustration and Animation.
• Cross-application tasking: Apple Intelligence can delve into your apps and execute tasks on your behalf. One of the examples Federighi gave was asking Apple Intelligence to pull up files sent by a contact during a certain time period.
• Focus on personal context: Apple Intelligence can draw upon the full suite of your activity but also on what is on your screen. For example, you can ask it whether a shifted meeting will cause you to be late to a personal commitment, Federighi said.
• Private Cloud Compute: Apple Intelligence will leverage cloud-based models on special servers using Apple Silicon to ensure that user data is private and secure. If a request needs to go to a cloud server, Apple says it will only send a limited selection of data in a “cryptographically” secure way.

Mr Federighi said privacy was a top priority for Apple. He described Apple Intelligence as a collection of “highly capable” large language and “diffusion models,” as well as an “on-device semantic index” that worked across apps to identify data and feed it to models.

Many of those models will run on device. For those models that need to be stored in the cloud, Mr Federighi touted Apple’s ability to let users control the kind of data you store in the cloud and how it can be accessed.

“We want to extend the privacy and security of your iPhone into the cloud,” Mr Federighi said.

Apple announces Passwords app

Apple announced a Passwords app for iPhone, iPad, Vision Pro, Mac and Windows.

It helps you store all of your passwords, sort of like Keychain did, but includes verification codes, app passwords, Wi-Fi passwords, shares passwords, Passkeys and more.

You can manage all of your passwords in the app or see how strong it is. It is very similar to 1Password.

Apple announces macOS Sequoia, some AI enhancements

Here’s what’s new in macOS Sequoia. Developer betas roll out today, public betas roll out next month and the full OS will be released in the spring.

• Continued enhancements from iPadOS and iOS: New features, including text animations and Maps improvements, will also be on macOS.
• iPhone mirroring: Users can now see and control their iPhone from their Mac via Continuity. iPhones will remain locked even if in a virtual session. StandBy will also remain visually undisrupted.
• Unified Notification Center: iPhone notifications will now appear on macOS.
• New Passwords app: Apple is breaking out its iCloud-powered Keychain feature into a discrete app, competing directly with 1Password and other password managers. It will also be available on Windows.
• Video enhancements: MacOS will now offer further enhancements to video calls, including background features and screen isolation.
• Safari enhancements: Safari now has artificial intelligence-powered highlights to extract helpful information from a webpage.

Apple announces iPadOS 18

Apple announced iPadOS 18, with a bevy of new features for the iPad.

Here’s what’s new:

• Interface improvements: A floating tab bar and automatic sidebars will allow users to use more of the screen at any given time.
• SharePlay enhancements: You can now take control of a SharePlay session.
• Calculator on iPad: Apple is finally bringing a native Calculator app for iPad. It will also support unit conversions, and when used with the Apple Pencil, will unlock a new feature called Math Notes.
• Math Notes for Calculator: Users can write with the Apple Pencil to automatically solve problems that users write on the iPad, with support for scientific calculator functions.
• Notes: The native app has a new feature called Smart Script, which uses “on-device machine learning” to automatically clean up your handwriting. You can also paste typed words into a handwriting session and they will automatically reform to appear as handwriting.

Apple announces watchOS 11 for Apple Watch

Apple shared new updates to watchOS:

• Training mode: Apple is introducing training mode, which can track how the intensity and duration of users’ workouts affect their bodies over time. Intensity is tracked using calorimetry data such as heart rate, pace and elevation, and Apple said a “powerful new algorithm” will automatically translate power sensor data into an estimate of your effort rating after your workout.
• Customize the summary tab: Users can personalize their summary tab with information they want to see, such as weekly running distance.
• New Vitals app: WatchOS 11 will track vitals such as your heart rate, respiratory rate and risk temperature, and give you insights about them in the new Vitals app. Users can easily check on their vitals and see when multiple metrics are out of range. They can also learn how their vitals respond to other factors such as alcohol, illness and elevation changes.
• Tools for pregnant users: Apple said cycle tracking now shows gestational aids, and the Health app can display your pregnancy across all charts and prompts.
• Adjustments to rings: Users can now adjust their goals by days of the week, and users can pause their rings if they want to take a break for a few days.
• Photos for your watch face: Apple is using artificial intelligence to identify photos that will work well as a background for your watch face.

Apple announces AirPods updates and new tvOS features

Apple announced some improvements to its Audio products and Home platform.

Here’s what’s new with AirPods:

• Siri with AirPods: AirPods can now detect head motions to answer or reject phone calls (nodding to answer, shaking a head to reject it).
• Voice Isolation: AirPods Pro will now be able to isolate your voice, even with noisy background audio.

Here’s what’s new on tvOS:

• Insight: A new feature that will give you information about what is playing on your TV, including on your iPhone.
• Audio enhancements: Apple TVs will now be able to enhance and isolate spoken audio, and will have improved subtitle timing that will automatically show up when appropriate.
• Projector support: Apple TVs will now support 21 x 9 projectors.
• New screensavers: Apple is adding support for Portrait photos as Apple TV screensavers, as well as allowing users to select their preferred screensavers.

Apple announces iOS 18

Apple announced iOS 18 with lots of personalization features.

Here’s what’s new:

• Customizations for home screen: Users can arrange their apps in new ways, change the colours and adjust the darkness of their apps. For instance, people can arrange their apps around the photo they have set as their background so it doesn’t cover it up.
• New look for Control Center: Users can group controls so they are easier to use, and developers can include controls from their apps. Apple is introducing a controls API for developers.

• Privacy updates: Users can lock apps so others won’t be able to see or access information without authentication. Users can control which contacts are available to different apps.
• New ways to use messages: Users can add new effects to their texts and tap react with new emojis.
• Messages via satellite: Users will be able to send iMessage and SMS messages via satellite.
• Filters for the Mail app: Users will be able to filter their emails by categories, and the feature will be available later this year.
• Tap to cash: A quick and private way to pay another person just by holding phones together.
• New look for event tickets: Apple is also introducing an event guide and support features such as venue maps so you can find your seat more easily.
• Gaming: Gaming on iOS 18 will minimize background activity and support improvement with wireless controllers and AirPods.
• Photos: It will be easier for users to search through their photos. The grid will appear at the top, and a theme, such as time, people or trips, is displayed below. A new filter button shows specific types of content so you can, for example, filter out screenshots.

Apple announces VisionOS 2

Apple Vice President Mike Rockwell announced the next iteration of Apple’s OS for its Vision Pro, VisionOS 2, four months after it first launched VisionOS.

Mr Rockwell also said that there were 2,000 apps ready for use on the Vision Pro.

Vision Pro will also be available in China, Japan and Singapore as of June 28. It will roll out to Australia, Canada, France, Germany and the U.K. on July 12.

Here’s some of what’s new:

• Photos: VisionOS 2 will use artificial intelligence to transform regular photos into Spatial photos that are compatible with the Vision Pro.
• New gestures: VisionOS 2 will also debut new gesture-driven controls.
• Ultrawide display: Vision Pro will now support an ultrawide display that Apple says is equivalent to two side-by-side 4K monitors.
• Train support for Travel mode: Apple is adding train support to its Travel Mode. It previously only supported planes.