Research Area

Image Processing Algorithms for Inverse Problems

Single image super resolution with implicit neural representation

alt text 

Recent works with an implicit neural function shed light on representing images in arbitrary resolution. Upon that, we propose a Local Texture Estimator (LTE), a dominant-frequency estimator for natural images, enabling an implicit function to capture fine details while reconstructing images in a continuous manner. When jointly trained with a deep super-resolution (SR) architecture, LTE is capable of characterizing image textures in 2D Fourier space.

figure from LTE_paper(cvpr2022), x32 upsampling using LTE

Neural network toward inverse problems in image processing

alt text 

Recently, inverse problems in image processing are resolved by a trainable deep network given by (c) in the figure. We investigate core characters of neural networks toward image processing. (figure from Jin, Kyong Hwan, et al. Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing 26.9 (2017): 4509-4522.)

Computational Photography

alt text 

Camera is a complex system which has a lot of ill-posed inverse problems inside. Especially, many hand-crafted digital processing tasks -computational photography -such as demosaicking, denoising, auto-exposure, white-balance, HDR (high dynamic range scene), alignment, are handled in ISP (image signal processor). Nowadays, such signal processing in chips has transferred into neural-networks in academia and presented in many channels. We investigate conventional image processing in computational photography with deep neural networks.

figure from wikipedia

Image Enhancement - Decontouring, Lossless/Lossy Compression Artifact Removal

For image transferring or video streaming, compressed bitstreams are conveyed through communication's net. During compression of contents, several artifacts arose such as contouring, blocky artifacts, color inconsistency, quantization errors, etc. People are unpleasant to such artifacts, so we would like to suppress those artifacts with deterministic processing or learnable neural networks. Main difference of this task with previous inverse problems is that ground-truth is always accessible (very important for supervised learning) because compression techniques begin their processing from original contents.

Signal Processing based on AI for Multi-channel/Multi-dimensional Data

alt text 

We investigate a novel unsupervised/semi-supervised deep-learning-based algorithm to solve the inverse problem found in dynamic magnetic resonance imaging (MRI). Our method needs neither prior training nor additional data; in particular, it does not require either electrocardiogram or spokes-reordering in the context of cardiac images. It generalizes to sequences of images the recently introduced deep-image-prior approach. The essence of the proposed algorithm is to proceed in two steps to fit k-space synthetic measurements to sparsely acquired dynamic MRI data.

figure from Yoo, Jaejun, Jin, Kyong Hwan, et al. Time-Dependent Deep Image Prior for Dynamic MRI. arXiv preprint arXiv:1910.01684 (2019).