Low-light photography is a balance between blur and noise — but what if artificial intelligence could even out the score? Researchers from the University of Illinois Urbana-Champaign and Intel have trained a program to process low-noise images in a room lit with a single candle. By feeding a RAW image processor two identical shots, one a short exposure and one a long exposure, the group managed to get an image with less noise and without the odd color casts alternative methods provide. With additional research, the processing algorithms could help cameras take images with less noise without using a longer shutter speed.
To train what the group calls the See-in-the-Dark data set, the researchers took two different images in limited light. Using a remote app to control the camera without touching it, the group took a properly exposed long exposure image from ten to 30 seconds long. The researchers then took a second image with a short exposure from 0.1 to 0.03 seconds long, which typically created an image that was almost entirely black.
Repeating this process around 5,000 different times, some with a Sony a7S II and some with a Fujifilm X-T2, the researchers then used the paired images to train a neural network. The images were first processed by separating into different color channels, removing the black and reducing the image’s resolution. The data set also used RAW data from the camera sensor, not processed JPEGs.
The algorithms created from the data set, when used on RAW sensor data, created brighter images with less noise compared to other methods of handling the camera data, like demosaicing. The resulting images also had a more accurate white balance than current methods. The results improve on traditional image processing, the researchers said, and warrant more research.
The enhanced processing method could help smartphones perform between in low light, along with enhancing handheld shots from DSLRs and mirrorless cameras, the group suggests. Video could also benefit, since taking a longer exposure isn’t possible while maintaining a standard frame rate.
While the sample images from the program are impressive, the processing was only tested on stationary subjects. Image processing was also slower than current standards — the images took 0.38 and 0.66 seconds to process at a reduced resolution, too slow to maintain the burst speeds on current cameras. The group’s data set was also designed for a specific camera sensor — without additional research on data sets for multiple sensors, the process would have to be repeated for each new camera sensor. The researchers suggested that future research could look into those limitations.
