Publications by Kyrollos Yanny
2022
Kyrollos Yanny; Kristina Monakhova; Richard W Shuai; Laura Waller
Deep learning for fast spatially varying deconvolution Journal Article
In: Optica, vol. 9, no. 1, pp. 96–99, 2022.
@article{Yanny:22,
title = {Deep learning for fast spatially varying deconvolution},
author = {Kyrollos Yanny and Kristina Monakhova and Richard W Shuai and Laura Waller},
url = {http://www.osapublishing.org/optica/abstract.cfm?URI=optica-9-1-96},
doi = {10.1364/OPTICA.442438},
year = {2022},
date = {2022-01-01},
journal = {Optica},
volume = {9},
number = {1},
pages = {96--99},
publisher = {OSA},
abstract = {Deconvolution can be used to obtain sharp images or volumes from blurry or encoded measurements in imaging systems. Given knowledge of the system's point spread function (PSF) over the field of view, a reconstruction algorithm can be used to recover a clear image or volume. Most deconvolution algorithms assume shift-invariance; however, in realistic systems, the PSF varies laterally and axially across the field of view due to aberrations or design. Shift-varying models can be used, but are often slow and computationally intensive. In this work, we propose a deep-learning-based approach that leverages knowledge about the system's spatially varying PSFs for fast 2D and 3D reconstructions. Our approach, termed MultiWienerNet, uses multiple differentiable Wiener filters paired with a convolutional neural network to incorporate spatial variance. Trained using simulated data and tested on experimental data, our approach offers a 625textminus1600texttimes increase in speed compared to iterative methods with a spatially varying model, and outperforms existing deep-learning-based methods that assume shift invariance.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Deconvolution can be used to obtain sharp images or volumes from blurry or encoded measurements in imaging systems. Given knowledge of the system's point spread function (PSF) over the field of view, a reconstruction algorithm can be used to recover a clear image or volume. Most deconvolution algorithms assume shift-invariance; however, in realistic systems, the PSF varies laterally and axially across the field of view due to aberrations or design. Shift-varying models can be used, but are often slow and computationally intensive. In this work, we propose a deep-learning-based approach that leverages knowledge about the system's spatially varying PSFs for fast 2D and 3D reconstructions. Our approach, termed MultiWienerNet, uses multiple differentiable Wiener filters paired with a convolutional neural network to incorporate spatial variance. Trained using simulated data and tested on experimental data, our approach offers a 625textminus1600texttimes increase in speed compared to iterative methods with a spatially varying model, and outperforms existing deep-learning-based methods that assume shift invariance.
Kyrollos Yanny
Optics and Algorithms for Designing Miniature Computational Cameras and Microscopes PhD Thesis
2022, ISBN: 9798352951132.
@phdthesis{nokey,
title = {Optics and Algorithms for Designing Miniature Computational Cameras and Microscopes},
author = {Kyrollos Yanny},
url = {https://www.proquest.com/dissertations-theses/optics-algorithms-designing-miniature/docview/2738519745/se-2},
isbn = {9798352951132},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {ProQuest Dissertations and Theses},
pages = {106},
abstract = {Traditional cameras and microscopes are often optimized to produce sharp 2D images of the object. These 2D images miss important information about the world (e.g. depth and spectrum). Access to this information can make a significant impact on fields such as neuroscience, medicine, and robotics. For example, volumetric neural imaging in freely moving animals requires compact head-mountable 3D microscopes and tumor classification in tissue benefits from access to spectral information. Modifications that enable capturing these extra dimensions often result in bulky, expensive, and complex imaging setups. In this dissertation, I focus on designing compact single-shot computational imaging systems that can capture high dimensional information (depth and spectrum) about the world. This is achieved by using a multiplexing optic as the image capture hardware and formulating image recovery as a convex optimization problem. First, I discuss designing a single-shot compact miniature 3D fluorescence microscope, termed Miniscope3D. By placing an optimized multifocal phase mask at the objective’s exit pupil, 3D fluorescence intensity is encoded into a single 2D measurement and the 3D volume can be recovered by solving a sparsity constrained inverse problem. This enables a 2.76 micron lateral and 15 micron axial resolution across 900x700x390 micron cubed volume at 40 volumes per second in a device smaller than a U.S. quarter. Second, I discuss designing a single-shot hyperspectral camera, termed Spectral DiffuserCam, by combining a diffuser with a tiled spectral filter array. This enables recovering a hyperspectral volume with higher spatial resolution than the spectral filter alone. The system is compact, flexible, and can be designed with contiguous or non-contiguous spectral filters tailored to a given application. Finally, the iterative reconstruction methods generally used for compressed sensing take thousands of iterations to converge and rely on hand-tuned priors. I discuss a deep learning architecture, termed MultiWienerNet, that uses multiple differentiable Wiener filters paired with a convolutional neural network to take into account the system’s spatially-varying point spread functions. The result is a 625-1600X increase in speed compared to iterative methods with spatially-varying models and better reconstruction quality than deep learning methods that assume shift invariance.},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Traditional cameras and microscopes are often optimized to produce sharp 2D images of the object. These 2D images miss important information about the world (e.g. depth and spectrum). Access to this information can make a significant impact on fields such as neuroscience, medicine, and robotics. For example, volumetric neural imaging in freely moving animals requires compact head-mountable 3D microscopes and tumor classification in tissue benefits from access to spectral information. Modifications that enable capturing these extra dimensions often result in bulky, expensive, and complex imaging setups. In this dissertation, I focus on designing compact single-shot computational imaging systems that can capture high dimensional information (depth and spectrum) about the world. This is achieved by using a multiplexing optic as the image capture hardware and formulating image recovery as a convex optimization problem. First, I discuss designing a single-shot compact miniature 3D fluorescence microscope, termed Miniscope3D. By placing an optimized multifocal phase mask at the objective’s exit pupil, 3D fluorescence intensity is encoded into a single 2D measurement and the 3D volume can be recovered by solving a sparsity constrained inverse problem. This enables a 2.76 micron lateral and 15 micron axial resolution across 900x700x390 micron cubed volume at 40 volumes per second in a device smaller than a U.S. quarter. Second, I discuss designing a single-shot hyperspectral camera, termed Spectral DiffuserCam, by combining a diffuser with a tiled spectral filter array. This enables recovering a hyperspectral volume with higher spatial resolution than the spectral filter alone. The system is compact, flexible, and can be designed with contiguous or non-contiguous spectral filters tailored to a given application. Finally, the iterative reconstruction methods generally used for compressed sensing take thousands of iterations to converge and rely on hand-tuned priors. I discuss a deep learning architecture, termed MultiWienerNet, that uses multiple differentiable Wiener filters paired with a convolutional neural network to take into account the system’s spatially-varying point spread functions. The result is a 625-1600X increase in speed compared to iterative methods with spatially-varying models and better reconstruction quality than deep learning methods that assume shift invariance.
2020
Kyrollos Yanny; Nick Antipa; William Liberti; Sam Dehaeck; Kristina Monakhova; Fanglin Linda Liu; Konlin Shen; Ren Ng; Laura Waller
Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy Journal Article
In: Light: Science & Applications, vol. 9, no. 171, 2020.
@article{yanny2020,
title = {Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy},
author = {Kyrollos Yanny and Nick Antipa and William Liberti and Sam Dehaeck and Kristina Monakhova and Fanglin Linda Liu and Konlin Shen and Ren Ng and Laura Waller},
url = {https://www.nature.com/articles/s41377-020-00403-7},
doi = {https://doi.org/10.1038/s41377-020-00403-7},
year = {2020},
date = {2020-10-02},
journal = {Light: Science & Applications},
volume = {9},
number = {171},
abstract = {Miniature fluorescence microscopes are a standard tool in systems biology. However, widefield miniature microscopes capture only 2D information, and modifications that enable 3D capabilities increase the size and weight and have poor resolution outside a narrow depth range. Here, we achieve the 3D capability by replacing the tube lens of a conventional 2D Miniscope with an optimized multifocal phase mask at the objective’s aperture stop. Placing the phase mask at the aperture stop significantly reduces the size of the device, and varying the focal lengths enables a uniform resolution across a wide depth range. The phase mask encodes the 3D fluorescence intensity into a single 2D measurement, and the 3D volume is recovered by solving a sparsity-constrained inverse problem. We provide methods for designing and fabricating the phase mask and an efficient forward model that accounts for the field-varying aberrations in miniature objectives. We demonstrate a prototype that is 17 mm tall and weighs 2.5 grams, achieving 2.76 μm lateral, and 15 μm axial resolution across most of the 900 × 700 × 390 μm3 volume at 40 volumes per second. The performance is validated experimentally on resolution targets, dynamic biological samples, and mouse brain tissue. Compared with existing miniature single-shot volume-capture implementations, our system is smaller and lighter and achieves a more than 2× better lateral and axial resolution throughout a 10× larger usable depth range. Our microscope design provides single-shot 3D imaging for applications where a compact platform matters, such as volumetric neural imaging in freely moving animals and 3D motion studies of dynamic samples in incubators and lab-on-a-chip devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Miniature fluorescence microscopes are a standard tool in systems biology. However, widefield miniature microscopes capture only 2D information, and modifications that enable 3D capabilities increase the size and weight and have poor resolution outside a narrow depth range. Here, we achieve the 3D capability by replacing the tube lens of a conventional 2D Miniscope with an optimized multifocal phase mask at the objective’s aperture stop. Placing the phase mask at the aperture stop significantly reduces the size of the device, and varying the focal lengths enables a uniform resolution across a wide depth range. The phase mask encodes the 3D fluorescence intensity into a single 2D measurement, and the 3D volume is recovered by solving a sparsity-constrained inverse problem. We provide methods for designing and fabricating the phase mask and an efficient forward model that accounts for the field-varying aberrations in miniature objectives. We demonstrate a prototype that is 17 mm tall and weighs 2.5 grams, achieving 2.76 μm lateral, and 15 μm axial resolution across most of the 900 × 700 × 390 μm3 volume at 40 volumes per second. The performance is validated experimentally on resolution targets, dynamic biological samples, and mouse brain tissue. Compared with existing miniature single-shot volume-capture implementations, our system is smaller and lighter and achieves a more than 2× better lateral and axial resolution throughout a 10× larger usable depth range. Our microscope design provides single-shot 3D imaging for applications where a compact platform matters, such as volumetric neural imaging in freely moving animals and 3D motion studies of dynamic samples in incubators and lab-on-a-chip devices.
Kristina Monakhova; Kyrollos Yanny; Neerja Aggarwal; Laura Waller
Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array Journal Article
In: Optica, vol. 7, no. 10, pp. 1298–1307, 2020.
@article{Monakhova:20b,
title = {Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array},
author = {Kristina Monakhova and Kyrollos Yanny and Neerja Aggarwal and Laura Waller},
url = {http://www.osapublishing.org/optica/abstract.cfm?URI=optica-7-10-1298},
doi = {10.1364/OPTICA.397214},
year = {2020},
date = {2020-10-01},
journal = {Optica},
volume = {7},
number = {10},
pages = {1298--1307},
publisher = {OSA},
abstract = {Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring; however, traditional scanning hyperspectral imagers are prohibitively slow and expensive for widespread adoption. Snapshot techniques exist but are often confined to bulky benchtop setups or have low spatio-spectral resolution. In this paper, we propose a novel, compact, and inexpensive computational camera for snapshot hyperspectral imaging. Our system consists of a tiled spectral filter array placed directly on the image sensor and a diffuser placed close to the sensor. Each point in the world maps to a unique pseudorandom pattern on the spectral filter array, which encodes multiplexed spatio-spectral information. By solving a sparsity-constrained inverse problem, we recover the hyperspectral volume with sub-super-pixel resolution. Our hyperspectral imaging framework is flexible and can be designed with contiguous or non-contiguous spectral filters that can be chosen for a given application. We provide theory for system design, demonstrate a prototype device, and present experimental results with high spatio-spectral resolution.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring; however, traditional scanning hyperspectral imagers are prohibitively slow and expensive for widespread adoption. Snapshot techniques exist but are often confined to bulky benchtop setups or have low spatio-spectral resolution. In this paper, we propose a novel, compact, and inexpensive computational camera for snapshot hyperspectral imaging. Our system consists of a tiled spectral filter array placed directly on the image sensor and a diffuser placed close to the sensor. Each point in the world maps to a unique pseudorandom pattern on the spectral filter array, which encodes multiplexed spatio-spectral information. By solving a sparsity-constrained inverse problem, we recover the hyperspectral volume with sub-super-pixel resolution. Our hyperspectral imaging framework is flexible and can be designed with contiguous or non-contiguous spectral filters that can be chosen for a given application. We provide theory for system design, demonstrate a prototype device, and present experimental results with high spatio-spectral resolution.
Fanglin Linda Liu; Grace Kuo; Nick Antipa; Kyrollos Yanny; Laura Waller
Fourier DiffuserScope: single-shot 3D Fourier light field microscopy with a diffuser Journal Article
In: Opt. Express, vol. 28, no. 20, pp. 28969–28986, 2020.
@article{LindaLiu:20,
title = {Fourier DiffuserScope: single-shot 3D Fourier light field microscopy with a diffuser},
author = {Fanglin Linda Liu and Grace Kuo and Nick Antipa and Kyrollos Yanny and Laura Waller},
url = {http://www.opticsexpress.org/abstract.cfm?URI=oe-28-20-28969},
doi = {10.1364/OE.400876},
year = {2020},
date = {2020-09-01},
journal = {Opt. Express},
volume = {28},
number = {20},
pages = {28969--28986},
publisher = {OSA},
abstract = {Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provide an efficient forward model, at the expense of field-of-view (FOV). Here, we demonstrate improved resolution across a large volume with Fourier DiffuserScope, which uses a diffuser in the pupil plane to encode 3D information, then computationally reconstructs the volume by solving a sparsity-constrained inverse problem. Our diffuser consists of randomly placed microlenses with varying focal lengths; the random positions provide a larger FOV compared to a conventional MLA, and the diverse focal lengths improve the axial depth range. To predict system performance based on diffuser parameters, we, for the first time, establish a theoretical framework and design guidelines, which are verified by numerical simulations, and then build an experimental system that achieves < 3 µm lateral and 4 µm axial resolution over a 1000 × 1000 × 280 µm3 volume. Our diffuser design outperforms the MLA used in LFM, providing more uniform resolution over a larger volume, both laterally and axially.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provide an efficient forward model, at the expense of field-of-view (FOV). Here, we demonstrate improved resolution across a large volume with Fourier DiffuserScope, which uses a diffuser in the pupil plane to encode 3D information, then computationally reconstructs the volume by solving a sparsity-constrained inverse problem. Our diffuser consists of randomly placed microlenses with varying focal lengths; the random positions provide a larger FOV compared to a conventional MLA, and the diverse focal lengths improve the axial depth range. To predict system performance based on diffuser parameters, we, for the first time, establish a theoretical framework and design guidelines, which are verified by numerical simulations, and then build an experimental system that achieves < 3 µm lateral and 4 µm axial resolution over a 1000 × 1000 × 280 µm3 volume. Our diffuser design outperforms the MLA used in LFM, providing more uniform resolution over a larger volume, both laterally and axially.
Grace Kuo; Kristina Monakhova; Kyrollos Yanny; Ren Ng; Laura Waller
Spatially-varying microscope calibration from unstructured sparse inputs Inproceedings
In: Imaging and Applied Optics Congress, pp. CF4C.4, Optical Society of America, 2020.
@inproceedings{Kuo:20,
title = {Spatially-varying microscope calibration from unstructured sparse inputs},
author = {Grace Kuo and Kristina Monakhova and Kyrollos Yanny and Ren Ng and Laura Waller},
url = {http://www.osapublishing.org/abstract.cfm?URI=COSI-2020-CF4C.4},
year = {2020},
date = {2020-01-01},
booktitle = {Imaging and Applied Optics Congress},
journal = {Imaging and Applied Optics Congress},
pages = {CF4C.4},
publisher = {Optical Society of America},
abstract = {We propose a method based on blind deconvolution to calibrate the spatially-varying point spread functions of a coded-aperture microscope system. From easy-to- acquire measurements of unstructured fluorescent beads, we recover a spatially-varying forward model that outperforms prior approaches.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose a method based on blind deconvolution to calibrate the spatially-varying point spread functions of a coded-aperture microscope system. From easy-to- acquire measurements of unstructured fluorescent beads, we recover a spatially-varying forward model that outperforms prior approaches.
Kristina Monakhova; Kyrollos Yanny; Laura Waller
Snapshot hyperspectral imaging using a random phase mask and spectral filter array Inproceedings
In: Imaging and Applied Optics Congress, pp. JF2F.4, Optical Society of America, 2020.
@inproceedings{Monakhova:20,
title = {Snapshot hyperspectral imaging using a random phase mask and spectral filter array},
author = {Kristina Monakhova and Kyrollos Yanny and Laura Waller},
url = {http://www.osapublishing.org/abstract.cfm?URI=COSI-2020-JF2F.4},
year = {2020},
date = {2020-01-01},
booktitle = {Imaging and Applied Optics Congress},
journal = {Imaging and Applied Optics Congress},
pages = {JF2F.4},
publisher = {Optical Society of America},
abstract = {We introduce a snapshot hyperspectral imager that uses a random phase mask, repeated spectral filter array, and compressive recovery to achieve high spatial and spectral resolution in a small form factor.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We introduce a snapshot hyperspectral imager that uses a random phase mask, repeated spectral filter array, and compressive recovery to achieve high spatial and spectral resolution in a small form factor.
Kyrollos Yanny; Nick Antipa; William Liberti; Sam Dehaeck; Kristina Monakhova; Fanglin Linda Liu; Konlin Shen; Ren Ng; Laura Waller
Compressed Sensing 3D Fluorescence Microscopy Using Optimized Phase Mask Inproceedings
In: Imaging and Applied Optics Congress, pp. CW4B.5, Optical Society of America, 2020.
@inproceedings{Yanny:20,
title = {Compressed Sensing 3D Fluorescence Microscopy Using Optimized Phase Mask},
author = {Kyrollos Yanny and Nick Antipa and William Liberti and Sam Dehaeck and Kristina Monakhova and Fanglin Linda Liu and Konlin Shen and Ren Ng and Laura Waller},
url = {http://www.osapublishing.org/abstract.cfm?URI=COSI-2020-CW4B.5},
year = {2020},
date = {2020-01-01},
booktitle = {Imaging and Applied Optics Congress},
journal = {Imaging and Applied Optics Congress},
pages = {CW4B.5},
publisher = {Optical Society of America},
abstract = {We demonstrate a single-shot miniature 3D computational microscope with an optimized phase encoder. Our method uses sparsity-based reconstruction to achieve a 2.76-m lateral and 15،nm axial resolution across most of the 900 x 700 x 390،nm3 volume.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We demonstrate a single-shot miniature 3D computational microscope with an optimized phase encoder. Our method uses sparsity-based reconstruction to achieve a 2.76-m lateral and 15،nm axial resolution across most of the 900 x 700 x 390،nm3 volume.
2019
Kristina Monakhova; Joshua Yurtsever; Grace Kuo; Nick Antipa; Kyrollos Yanny; Laura Waller
Learned reconstructions for practical mask-based lensless imaging Journal Article
In: Optics express, vol. 27, no. 20, pp. 28075–28090, 2019.
@article{monakhova2019learned,
title = {Learned reconstructions for practical mask-based lensless imaging},
author = { Kristina Monakhova and Joshua Yurtsever and Grace Kuo and Nick Antipa and Kyrollos Yanny and Laura Waller},
url = {https://doi.org/10.1364/OE.27.028075},
doi = {10.1364/OE.27.028075},
year = {2019},
date = {2019-09-30},
journal = {Optics express},
volume = {27},
number = {20},
pages = {28075--28090},
publisher = {Optical Society of America},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Kristina Monakhova; Joshua Yurtsever; Grace Kuo; Nick Antipa; Kyrollos Yanny; Laura Waller
Unrolled, model-based networks for lensless imaging Journal Article
In: 2019.
@article{monakhova2019unrolled,
title = {Unrolled, model-based networks for lensless imaging},
author = { Kristina Monakhova and Joshua Yurtsever and Grace Kuo and Nick Antipa and Kyrollos Yanny and Laura Waller},
url = {https://pdfs.semanticscholar.org/6a49/3ac2a0c8a3be888ece00b52bc1ec013df2bd.pdf},
year = {2019},
date = {2019-09-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Kyrollos Yanny; Nick Antipa; Ren Ng; Laura Waller
Miniature 3D fluorescence microscope using random microlenses Inproceedings
In: Optics and the Brain, pp. BT3A–4, Optical Society of America 2019.
@inproceedings{yanny2019miniature,
title = {Miniature 3D fluorescence microscope using random microlenses},
author = { Kyrollos Yanny and Nick Antipa and Ren Ng and Laura Waller},
url = {https://www.osapublishing.org/abstract.cfm?uri=BRAIN-2019-BT3A.4},
year = {2019},
date = {2019-04-14},
booktitle = {Optics and the Brain},
pages = {BT3A--4},
organization = {Optical Society of America},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}