Thai
Elite Member
More info on Xs camera: https://medium.com/s/story/inside-apples-iphone-xs-camera-technology-50d47ba7be8f
"As Marineau-Mes explained to me, the camera starts by capturing two image frames at different exposures in one-thirtieth of a second. That information is passed on to the software pipe and the A12’s neural engine, which starts analyzing the images.
Smart HDR doesn’t stop there. “Raw material is captured at 30 fps (that’s a pair every thirtieth of a second), and the fusing happens in a few hundred milliseconds,” Marineau-Mes pointed out.
Inside the A12 chip is the neural engine that analyzes frames not just for exposure but for discrete image elements. It’s identifying facial features and looking for motion. If the system detects motion, it looks for the frame with the sharpest image of the motion and adds it to the image. Similarly, an image with red-eye is not just fixed but replaced with a frame where the eye isn’t red or with the reference eye color from the frame without red-eye.
It takes an incredibly powerful ISP and neural engine backed by an equally powerful GPU and CPU to do all this processing, Marineau-Mes said.
All that heavy lifting starts before you even press the iPhone camera app’s virtual shutter button. Schiller said that what users see on their iPhone XS, XS Max, and XR screens is not dramatically different from the final image.
“You need for that to happen,” said Schiller. “It needs to feel real-time for the user.”
All of these image capture gymnastics extend to video as well, where the same frame-to-frame analysis is happening in real-time to produce video with more details in high and low light.
Video or stills, the end result is a new high-water mark for Apple and, perhaps, smartphone camera photography in general."
"As Marineau-Mes explained to me, the camera starts by capturing two image frames at different exposures in one-thirtieth of a second. That information is passed on to the software pipe and the A12’s neural engine, which starts analyzing the images.
Smart HDR doesn’t stop there. “Raw material is captured at 30 fps (that’s a pair every thirtieth of a second), and the fusing happens in a few hundred milliseconds,” Marineau-Mes pointed out.
Inside the A12 chip is the neural engine that analyzes frames not just for exposure but for discrete image elements. It’s identifying facial features and looking for motion. If the system detects motion, it looks for the frame with the sharpest image of the motion and adds it to the image. Similarly, an image with red-eye is not just fixed but replaced with a frame where the eye isn’t red or with the reference eye color from the frame without red-eye.
It takes an incredibly powerful ISP and neural engine backed by an equally powerful GPU and CPU to do all this processing, Marineau-Mes said.
All that heavy lifting starts before you even press the iPhone camera app’s virtual shutter button. Schiller said that what users see on their iPhone XS, XS Max, and XR screens is not dramatically different from the final image.
“You need for that to happen,” said Schiller. “It needs to feel real-time for the user.”
All of these image capture gymnastics extend to video as well, where the same frame-to-frame analysis is happening in real-time to produce video with more details in high and low light.
Video or stills, the end result is a new high-water mark for Apple and, perhaps, smartphone camera photography in general."
Last edited:
