Smartphone cameras have gotten a whole lot better over the past few years. That is partly due to the fact that camera sensors and lenses are getting bigger and better. But it also has a lot to do with something people don’t often think about: Improvements in image processing provided by your phone’s chipset. In the next generation of high-end Android phones, that image processing will be largely powered by the new Qualcomm Snapdragon 888.
There are a bunch of things that make the Qualcomm Snapdragon 888 a better chipset for photography, and it largely boils down to the Spectra 580 image signal processor, or ISP — though there are a few other things at play as well. Here’s a rundown of the new image-processing tech used by the Snapdragon 888, and how it could affect how you take photos and videos in 2021.
Use three cameras at once
Perhaps the biggest headline feature on offer by the Snapdragon 888 is the fact that the Spectra 580 has triple ISPs. What does that mean? Well, at its core, it means that you could use up to three cameras at once on your phone.
Now, that’s actually a little more useful than it sounds. In past years, smartphone cameras have regularly used two cameras at once, without you necessarily thinking about it. A good example of that is the use of a depth sensor, which collects the data needed for a smartphone to accurately process things like bokeh, or background blur, for portrait mode effects.
With triple-camera concurrency, you can capture three 4K HDR video streams at the same time — which could come in handy for videographers who might want to go back in later and cut between different video feeds. Or, you can capture three 28-megapixel photos, and see feeds of them in the viewfinder all at once — helping to ensure that you capture the best photo you can every time.
“You could capture two cameras world-facing, and one capture user-facing, so if you were going to do some sort of documentary that you were shooting with two cameras like zoom, and then you wanted to also film your face as you were narrating it, you could also do that as well,” said Judd Heape, VP of product management at Qualcomm, in an interview with Digital Trends.
Artificial intelligence could also play into that. The AI Engine that’s built into the Snapdragon 888 can be used in conjunction with this triple-camera concurrency to ensure that your subject is always framed properly. Qualcomm is showing off a demo built by ArcSoft that automatically zooms in on a subject as she walks away from the camera, better framing the shot and making for a better video.
The other main advantage to triple ISPs is the fact that they can process three streams of data at a time. It gets a little technical, but the way many smartphone cameras capture a great photo these days is that they capture a bunch of different photos at different exposures, then kind of meld them together to create a vibrant and detailed photo with high dynamic range. With the Snapdragon 888, that same kind of tech can be brought to video, which should make for much more detailed HDR videos.
Speaking of artificial intelligence …
A.I. will play other roles in image processing on the Snapdragon 888, particularly when it comes to auto exposure, autofocus, and auto white balance — also known as 3A. Previously, A.I. didn’t necessarily play a big role in these areas, but it does on the Snapdragon 888 — and Qualcomm says it could have a significant impact on mobile photography.
“We’ve actually trained the neural network based on the human vision system,” Heape said. “We took a bunch of people and put VR headsets on their heads, and we showed them a bunch of photos and videos, and we tracked their eye movement. So what you see is that the user doesn’t necessarily look at the parts of the scene that the camera might assume would be the main focal and exposure points. There might be people in the distance, or interesting objects that are behind other objects.”
Of course, it remains to be seen exactly how this will work in real-world photography and videography. Some users might find that this is a very handy system, while others will probably want to retain manual control of a camera’s focus. Hopefully, it’ll help make focus and exposure much more intuitive — after all, most people simply want their smartphone cameras to “just work.” Ultimately, the phone makers will be able to decide what features — or parts of features — they want to implement when it comes to designing their camera apps.
Burst mode magic
Last year, Qualcomm hailed the Snapdragon 865, which offers 2-gigapixel-per-second processing speeds, however this year, the company is stepping things up to 2.7 gigapixels per second. Basically, the result is that the Spectra 580 ISP can process 35% faster than the Spectra 480 from last year.
What does that mean for you? Well, put simply, the tech will be able to capture up to 120 photos at 12MP every second, taking burst mode photography to a new level on smartphones.
“You can hit the button once on the camera, and it’ll shoot a burst capture capturing 120 images all in one second. So 120 frames per second at 12 megapixels,” said Heape.
That’s very impressive — and while manufacturers will need to implement some fun software to ensure that you don’t have a dump of 120 photos in your camera roll every time you hit the capture button, the fact that it’s possible should help ensure that you can capture even very fast-moving subjects.
When will you actually see these features?
All these new features are great, but they don’t mean much if smartphone manufacturers don’t actually adopt them or enable them in software. Of course, the timing really depends on each manufacturer, but the first step is, of course, enabling such features on a hardware level.
“We cooperate with [manufacturers] really early, and we take their feedback years in advance,” said Heape. “We started defining this chip when I joined Qualcomm back in 2017, so we have to plan pretty far in advance. We have to look at the market and say, ‘Do we think we need a triple ISP?’ Camera is an area that a lot of [manufacturers] like to differentiate themselves. They choose a different set of camera features, lenses, sensors, etc. The good news is we can enable all of that.”
