Apple’s camera chief thinks AI can give you superpowers

What is it even? a photo these days?

As tech giants come together generative AI capabilities in our phones and their camera software, the line between what is a real image and what is not continues to blur. Phones from Google and Samsung, for example, now come with features that allow you to radically edit a photo by deleting people, move people around the plan, and even add new objects to the stage.

Apple is getting in on the action by adding new generative features to its Photos app, although the company’s head of iPhone cameras, Jon McCormack, emphasizes that Apple is taking a more measured approach than its competitors and isn’t “doing AI for AI’s sake.”

At its annual conference Worldwide Developers Conference Apple showed off a handful of AI features on Monday invade Photos app in iOS 27, which will arrive on iPhones later this year.

Although the iPhone’s Photos app already has the Clean Up tool, which lets you erase unwanted objects in images, it will work even better in iOS 27 thanks to its access to Apple’s improved AI models. However, there are two new features, called Extend and Spatial Reframe, which allow you to enlarge the space around your photo or change the perspective of an image, while generating fake pixels. The camera “thinks” about what should be there, then draws it in.

McCormack says there is a huge backlog of intractable problems that AI is now helping to solve and these new features are very deliberate. “You don’t need to know all the details to know how to do something in Photoshop or anything else: it gives normal people these absolute superpowers,” says McCormack.

However, Apple doesn’t want to let you go wild with your images and generate all kinds of fakes. (At least not in the Photos app; the App Store has plenty of tools for creating photorealistic slopes.) The fake pixels generated by the Photos app are limited to what’s in the background. This will not change the pixels of the main subject’s face. With Clean Up, for example, you can’t remove the main subject from the image. The Extend function only works once and enlarges the image by 25%: you cannot save, edit the image again and extend it infinitely with AI.

McCormack also says Apple will integrate SynthID technology from Google DeepMind later this year to add an invisible watermark indicating that these images have been edited with generative AI. Any platforms you share the photo on can flag it as AI-edited. (Just know that researchers have shown that digital watermarks are not foolproof.)

“A photograph represents something that actually happened,” says McCormack. “We really believe in this idea of ​​authentic journalism about your own life: when you take photos, you’re creating these memories, you’re putting moments of your life into a bottle that you can come back to. It’s really important for us to create tools that preserve the sacredness of that moment.”

The head of Google’s Pixel camera division shared a similar sentiment when I spoke to him a few years ago; However, he emphasized the importance of how you recalled photography. Google is much more lax in allowing you to edit the image to match what you had in mind. Was the sky bluer in your memory? Go ahead and change it.

Apple’s new tools are more restrictive and are specifically aimed at solving compositional problems that you may not have realized during the capture process, McCormack says. Maybe you didn’t see that unsightly plastic bag rolling around in the background. Maybe you took a photo of your child but took it a little too high. Or maybe you’ve boxed your spouse too close to the edge and want a little more space.

Della Huff, product manager for Apple’s Camera and Photos software, says the team spent time training the AI ​​models to minimize the hallucinations that could occur during these digital adjustments. “It won’t create anything that shouldn’t be there,” Huff says. For example, if you’re trying to extend a street scene, it’s plausible that there’s a car parked outside the confines of your original photo, but Extend won’t make that assumption and generate it. “The way the model was trained is that if you don’t need to create something there, then don’t do it: do the minimum number of hallucinations to achieve the goal the user is asking the model to do.”

But in one of my examples of using Extend in iOS 27 Developer Beta, I took a photo of a friend sitting at a table, then tried to extend the scene a little more to the right. In the background, some people were sitting at tables, but as the scene expanded, the Photos app added a few more tables, with fake people – people who were never there in real life – sitting in front of them.

Huff says the functionality attempts to match the existing aesthetic. If there are people in the background but Extend doesn’t add anyone else when you enlarge the photo, it may look strange. “If we said the rule is that we can never generate a human in the background, then the feature would become less useful,” she says.

It’s worth remembering a point Apple made during the keynote: there are usage limits for these new camera features. Apple doesn’t share exactly what these daily limits are, but users will need to subscribe to iCloud if they want to expand, spatially crop, or clean up their photos multiple times per day.

One of the big themes of Apple’s WWDC presentation this year was using natural language to get things done. You can converse naturally with Siri (no need to use rigid commands) and it will understand your intention. In the Calendar app, you can describe the event and it will create it in a jiffy, without having to fill in various fields. You can describe the shortcut you want to create in the Shortcuts app without worrying about triggers and actions. In Safari you can create an extension. So why can’t you edit a photo in natural language? It’s a feature Google introduced last year on Google Photos.

Huff says the new Siri AI can handle some editing on your behalf and that Apple isn’t ruling anything out on the future roadmap. Siri can’t edit with the new AI features – those are strictly human-controlled – but that’s partly because Apple doesn’t think it’s a good user experience. These features require a little more muscle; it’s difficult to talk to Siri about the process of changing the perspective of a photo.

“There are so many other things about this that are open, especially something like spatial cropping where it’s really about a user intent that you have to express,” Huff says.

Speaking of Siri, the other big change coming in iOS 27 is that Siri is injected into the Camera app. McCormack says that improving the iPhone’s camera is all about reducing friction.

Siri’s visual intelligence feature, which works like Google Lens using computer vision to study an image – is currently activated via the camera control button. But it’s a camera-specific feature, so McCormack says it makes sense that it resides in the Camera app.

“It’s about accepting the idea that the camera is actually a number of things,” he says. “It’s a memorializing device…it’s a note-taking device…or I’m just curious what this plant is.”

Apple’s more measured approach may seem at odds with its image generation app, Image Playground, which lets you create AI-generated images with text prompts (or by injecting a photo from your own library). In iOS 27, Image Playground generates more photorealistic images by default unless you specify a particular art style. But McCormack says these different uses of AI will be different depending on the context they live in on your phone. If you’re in the Photos app, users want to know they’re in a safe place where memories are kept intact. But in Playground, the name is intentional: “it’s a place to play.”

“Both use cases are totally valid,” Huff says. “I want to improve this photo. A photo is something that happened, I captured it with my camera, and the Photos app is where I can improve it. But I also want to be creative and let my imagination run wild, and so they are intentionally two separate experiences.”