Franziska Schirrmacher, M. Sc.
PhD candidate in the multimedia security group and member of the SFB/TRR 89 Invasive Computing Project C5 “Security in Invasive Computing Systems“.
Below you will find my recent projects. Within the last three years, I worked on the super-resolution of indecipherable license plates and license plate recognition. Participating in the International Computer Vision Summer School (ICVSS) 2018 sparked my interest in deep learning. I started to focus on robust deep learning methods that can reliably predict test data that slightly deviates from the training data. This is specifically useful in real-world scenarios, where the image acquisition is unknown. To improve deep learning architectures, especially the extension of a recognition network with additional tasks from the field of image processing shows great potential.
In my Master Thesis, I developed the Quantile Sparse Image (QuaSI) prior and its adaptive version (AQuaSI) under the supervision of Thomas Köhler.
Deep Learning Architectural Designs for Super-Resolution of Noisy Images :
The goal of this project is the investigation of two architectural designs to jointly perform denoising and super-resolution. “In-network” combines both tasks at feature level, while “pre-network” first performs denoising and then super-resolution. Additionally, we perform experiments on aligned noise distributions and the generalization of the designs to unseen noise distributions.
Learning to Decipher License Plates in Severely Degraded Images:
The goal of this project is the reconstruction of license plates in severely degraded images. Specifically, we investigated the influence of JPEG compression on the recognition rate. Additionally, an in-depth analysis on the influence of the position of the character in the license plate and the similarity of characters.
SR²: Super-Resolution With Structure-Aware Reconstruction:
Combining multiple image processing methods is mostly done in sequential order and thus prone to error propagation. With SR2, we aim at combining super-resolution and character recognition in parallel within a multi-task network. This architecture is able to preserve structures and removes noisy pixels although the network itself has never been trained on noisy data.
Super-resolution of indecipherable license plates:
In police investigations, a severely degraded image quality leads to an impairment of their success. License plates might give a hint on the suspect but are often indecipherable. The degradation of the image quality is mainly caused by compression artifacts, low contrast, and low resolution. In order to improve the image quality such that the license plate can be deciphered, super-resolution algorithms are applied to video frames in a post-processing step. Unfortunately, the registration of the frames is not sufficiently accurate to obtain visually pleasing images. Additionally, compression often leads to a loss of information such that super-resolution is not successful.
(Adaptive) Quantile Sparse Image Prior:
A key challenge in research on image processing is to find suited priors for inverse problems to represent natural images. We propose the Adaptive Quantile Sparse Image (AQuaSI) prior. It is based on a quantile filter, can be used as a joint filter on guidance data, and be readily plugged into a wide range of numerical optimization algorithms. We demonstrate the efficacy of the proposed prior in joint RGB/depth upsampling, on RGB/NIR image restoration, and in a comparison with related regularization by denoising approaches.
Quantile Sparse Image Prior:
In todays imaging systems, the reliable reduction of image noise poses a constantly recurring problem. In healthcare, noise may limit the reliability of medical image data for subsequent clinical workflows. For instance, in radiology using computed tomography (CT) or related morphological imaging modalities, noise affects the analysis of anatomical structures and thus impedes diagnostic applications. In optical coherence tomography (OCT) for retinal imaging as another example use case, noise limits the measurement of structural features in the human eye, e. g. retinal layer properties. Therefore, the universal and structure-preserving regularization term, called quantile sparse image (QuaSI) prior, is introduced. Using a variational framework based on the QuaSI prior and a Huber data fidelity model allows to process 3-D and 3-D+t data. Effcient optimization is facilitated through the use of an alternating direction method of multipliers (ADMM) scheme and the linearization of the quantile filter. Using the linearization of the quantile filter, the quantile layer helps to overcome the noisy outliers in a Recurrent Neural Network architecture for Magnetic Resonance Fingerprinting.
github | MICCAI 2017 | Medical Image Analysis 2018
- Anantharajaiah, N., Asfour, T., Bader, M., Bauer, L., Becker, J., Bischof, S.,... Zhang, L. (2022). Invasive Computing. FAU University Press.
- Moussa, D., Maier, A., Schirrmacher, F., & Rieß, C. (2021). Sequence-based Recognition of License Plates with Severe Out-of-Distribution Degradations. In Springer, Cham (Eds.), Computer Analysis of Images and Patterns (pp. 175-185). Virtual Conference, CY: Springer Nature Switzerland AG.
- Villar-Corrales, A., Schirrmacher, F., & Rieß, C. (2021). Deep Learning Architectural Designs for Super-Resolution of Noisy Images. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1635-1639).
- Kaiser, P., Schirrmacher, F., Lorch, B., & Rieß, C. (2021). Learning to Decipher License Plates in Severely Degraded Images. In Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021 (pp. 544-559).
- Lorch, B., Schirrmacher, F., Maier, A., & Rieß, C. (2021). Reliable Camera Model Identification Using Sparse Gaussian Processes. IEEE Signal Processing Letters, 28, 912-916. https://dx.doi.org/10.1109/LSP.2021.3070206
- Schirrmacher, F., Riess, C., & Köhler, T. (2020). Adaptive Quantile Sparse Image (AQuaSI) Prior for Inverse Imaging Problems. IEEE Transactions on Computational Imaging, 6(1), 503--517. https://dx.doi.org/10.1109/TCI.2019.2956888
- Schirrmacher, F., Lorch, B., Stimpel, B., Köhler, T., & Riess, C. (2020). SR²: Super-Resolution With Structure-Aware Reconstruction. In Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP) (pp. 533-537). Online.
- Hoppe, E., Thamm, F., Körzdörfer, G., Schirrmacher, F., Nittka, M., Pfeuffer, J.,... Syben-Leisner, C. (2019). RinQ Fingerprinting: Recurrence-Informed Quantile Networks for Magnetic Resonance Fingerprinting. In Proceedings of MICCAI 2019 (pp. 92-100). Shenzhen, CN: Cham: Springer.
- Hoppe, E., Thamm, F., Körzdörfer, G., Schirrmacher, F., Nittka, M., Pfeuffer, J.,... Syben-Leisner, C. (2019). Magnetic Resonance Fingerprinting Reconstruction Using Recurrent Neural Networks. In Rainer Röhrig, Harald Binder, Hans-Ulrich Prokosch, Ulrich Sax, Irene Schmidtmann, Susanne Stolpe, Antonia Zapf (Eds.), German Medical Data Sciences: Shaping Change – Creative Solutions for Innovative Medicine. (pp. 126-133). IOS Press.
- Stimpel, B., Schirrmacher, F., Hölter, P., Dörfler, A., Maier, A., & Syben-Leisner, C. (2019). Multi-Modal Deep Guided Filtering for Comprehensible Medical Image Processing Supplementary Material. IEEE Transactions on Medical Imaging. https://dx.doi.org/10.1109/TMI.2019.2955184
- Stimpel, B., Schirrmacher, F., Dörfler, A., Hölter, P., Maier, A., & Syben-Leisner, C. (2019). Multi-Modal Super-Resolution with Deep Guided Filtering. In Thomas M. Deserno, Andreas Maier, Christoph Palm, Heinz Handels, Klaus H. Maier-Hein, Thomas Tolxdorff (Eds.), Informatik aktuell (pp. 110-115). Lübeck, DE: Springer Berlin Heidelberg.
- Schirrmacher, F., Köhler, T., Endres, J., Lindenberger, T., Fujimoto, J.G., Hornegger, J.,... Husvogt, L. (2018). Temporal and Volumetric Denoising via Quantile Sparse Image Prior. Medical Image Analysis, 48(0), 131-146. https://dx.doi.org/10.1016/j.media.2018.06.002
- Schirrmacher, F., Köhler, T., & Riess, C. (2018). Adaptive Quantile Sparse Image (AQuaSI) Prior for Inverse Imaging Problems.
- Schirrmacher, F., Köhler, T., Fujimoto, J.G., Hornegger, J., Maier, A., & Husvogt, L. (2018). Abstract: QuaSI - Quantile Sparse Image A Prior for Spatio-Temporal Denoising of Retinal OCT Data. Paper presentation at Bildverarbeitung für die Medizin 2018, Erlangen.
- Schirrmacher, F., Köhler, T., Fujimoto, J.G., Hornegger, J., Maier, A., & Husvogt, L. (2017). QuaSI: Quantile Sparse Image Prior for Spatio-Temporal Denoising of Retinal OCT Data. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2017, Proceedings, Part II (pp. 83-91). Quebec City, QC, Canada: Springer Verlag.
- Schirrmacher, F., Taubmann, O., Unberath, M., & Maier, A. (2017). Towards Understanding Preservation of Periodic Object Motion in Computed Tomography. In Bildverarbeitung für die Medizin 2017 (pp. 116-121). DKFZ, Heidelberg: Springer Vieweg: Springer Vieweg.