Achieving high quality image like a smartphone with edge camera by AI noise reduction
January 31, 2022 – Tokyo Japan -- LeapMind Co., Ltd., a leading creator of the standard in edge artificial intelligence (AI), today announced an AI image processing model which operates in real time on edge device. This AI image processing model uses the company's ultra-low power consumption AI inference accelerator IP "Efficiera" to operate in real time on edge devices to reduce the noise and improve the image quality of noisy images. This solves the challenges of performance and image quality accuracy that have been raised in the past, and achieved high performance, high image quality, and weight reduction.
Challenges
- Performance challenge: difficult on edge device to operate in real-time with the high calculation cost on AI image processing
- Image quality accuracy challenge: Extremely low bit quantization for AI image processing that inputs and outputs the images has degradation in image quality (reproduction of color and resolution, etc.).
Solutions and achievements
The company's extreme low bit quantization technology has made it lighter, and by combining it with the performance scalability of Efficiera v2, it became possible to operate in real time even with a video camera.
The company's Pixel embedding*¹ technology has achieved high image quality and image quality comparable to that of commonly used 32-bit floating point models. As a result, image quality can be improved by AI without the need for high-sensitivity sensors and large lenses, which are the high cost components. And high image quality like smartphones is possible by AI to those image quality oriented cameras, which makes industrial cameras (security cameras, inspections cameras, etc.) improve the accuracy of object recognition and inspection by improving the image quality even in low-light conditions or those video cameras which cannot take sufficient exposure time.
This model will be available as an evaluation version from February 2022. To obtain, please contact us.
“As far as we searched, we are the world’s first in bringing this Image processing AI model into the product by low bit quantization technology. This model is a product that can be put into practical use only because of LeapMind focusing on both hardware and software development and we believe that it shows the new value of extremely low bit quantization.” says Hiroyuki Tokunaga, CTO of LeapMind.
*¹Pixel embedding: A method of encoding an input image into 2-bit data of multiple channels
Achieving image quality comparable to the 32-bit floating point model with the ultra-small quantization (1,2 bit) model
Input image Left: ISO51200, 1/800sec, F4.0, Right: ISO102400, 1/320sec, F8.0
Output image: 32 bit floating point model
Output image: LeapMind Extremely low bit quantization model
Features
- Raw-to-raw noise reduction model for raw input and raw output
- Minimum impact to Existing image processing pipeline due to Raw-to-raw
- Deep learning-based noise reduction processing
- State-of-the-art NR algorithm optimized for embedded applications
- Provides a retrainable trained model
- Optimize the model by retraining sensor-specific noise in addition to trained noise
- Light weight processing for real-time operation
- Supports real-time operation with video cameras due to Efficiera v2's performance scalability
About Efficiera
Efficiera is an ultra-low power AI inference accelerator IP specialized for CNN inference processing that runs as a circuit on FPGA or ASIC devices. The "extremely low bit quantization" technology minimizes the number of quantization bits to 1 - 2 bits, maximizing the power and area efficiency of convolution, which accounts for most of the inference processing, without the need for advanced semiconductor manufacturing processes or special cell libraries. By using this product, deep learning functions can be incorporated into a variety of edge devices, including consumer electronics such as home appliances, industrial equipment such as construction machinery, surveillance cameras, broadcasting equipment, as well as small machines and robots that are constrained by power, cost, and heat dissipation, which has been technically difficult in the past. Visit product website at https://leapmind.io/business/ip/
About LeapMind
LeapMind Inc. was founded in 2012 with the corporate mission "To create innovative devices with machine learning and make them available everywhere." Total investment in LeapMind to date has reached 4.99 billion yen (as of May 2021). The company's strength is in extremely low bit quantization for compact deep learning solutions. It has a proven track record of achievement with over 150 companies, many of which are centered in manufacturing, including the automobile industry. It is also developing its Efficiera semiconductor IP, based on its experience in the development of both software and hardware
https://leapmind.io/en/
