Multi-Contrast 3D Fast Spin-Echo T2 Shuffling Reconstruction with Score-Based Deep Generative Priors

Sidharth Kumar¹, Asad Aali¹, and Jonathan I Tamir^1,2,3
¹Chandra Family Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX, United States, ²Oden Institute for Computational Engineering and Sciences, University of Texas at Austin, Austin, TX, United States, ³Department of Diagnostic Medicine, University of Texas at Austin, Austin, TX, United States

Synopsis

Score-based generative modeling has emerged as a powerful tool for modeling image priors and has recently been used to solve ill-posed inverse problems in various domains including MRI reconstruction. Here we extend the framework to reconstruct multi-contrast 3D fast spin-echo (FSE), i.e. T2 Shuffling data. This is achieved by constraining the posterior sampling reconstruction to a low-dimensional subspace and training a score model on images from this subspace. We demonstrate a proof-of-principal reconstruction of data with no model mismatch, i.e. generated from the forward model.

Introduction

Volumetric fast spin-echo (3DFSE) is clinically desirable as it allows for multi-planar reformatting; however, to date, it has not supplanted 2DFSE likely due to T2-decay induced blurring^1-3. Recently, a volumetric FSE technique called T2 Shuffling (T2Sh) has been introduced to mitigate signal blur and resolve images along the FSE signal curve, thereby providing multi-contrast images⁴. T2Sh uses a randomized view ordering and re-acquires phase encodes throughout the echo trains. The 4D data are processed with a compressed sensing-based reconstruction⁵ that constrains the temporal signal evolutions to a low-dimensional subspace. The approach has been shown to be non-inferior to clinical 2D FSE⁶. While powerful, the approach still requires scan times in excess of 7 minutes due to limitations in the low rank prior used in the reconstruction⁷.
In recent parallel work, deep learning generative modeling has been used to model image priors separately from the measurement model and has been successfully applied to MRI reconstruction, showing great potential to reduce scan time beyond the abilities of compressed sensing^8-11. Specifically, score-based generative models have proven to be successful in solving various ill-posed inverse problems. In this work, we extend the framework to reconstruct T2Sh data through posterior sampling directly in the low-dimensional subspace. We train the score model on basis images from the conventional T2Sh reconstruction and demonstrate our approach on data with no model mismatch, i.e. generated through the forward model¹². Hence, this work builds upon existing works on under-sampled low-dimensional subspace-based signal reconstruction and score models for inverse problems for MR images.

Methods

For this work, we used reconstructed knee MRI T2Sh basis coefficients images^7,13, which were collected with IRB approval and informed consent/assent. A single score model was trained on 5000 basis coefficients images of the knee without accounting for the relationship across subspace coefficients, i.e.

s_{θ} (α_{k}) \approx \nabla \log p (α_{k})

$s_\theta(\alpha_k)\approx\nabla\log p(\alpha_k)$ , where

α_{k}

$\alpha_k$ is the k^th basis image and

s_{θ}

$s_\theta$ approximates the score. Fig.1 describes the training and reconstruction procedure. A direct advantage of the score model is that it is agnostic to the MRI forward process and thus the reconstruction can be tailored to different scan parameters (e.g. echo train length [ETL], acceleration) without retraining⁸. We simulated the T2Sh forward model using the basis coefficient images according to

y = P F S Φ α + w, w \sim N (0, σ^{2} I),

$y=PFS\Phi\alpha+w,\text{ }w\sim\mathcal{N}(0,\sigma^2I),$ where

α \in C^{K}

$\alpha\in \mathbb{C}^K$ are the basis coefficient images,

Φ \in R^{T \times K}

$\Phi\in\mathbb{R}^{T\times K}$ is the basis,

S

$S$ is the coil sensitivity maps,

F

$F$ is the Fourier transform operator,

P

$P$ is the k-space sampler, and

w

$w$ is the Gaussian noise.

T

$T$ is the ETL and

K

$K$ is the number of basis coefficients. We define the forward operator

A = P F S Φ

$A = PFS\Phi$ . The iterative reconstruction approximates posterior sampling using Annealed Langevin Dynamics¹⁴ according to

α^{(t + 1)} \leftarrow α^{(t)} + η_{t} (s_{θ} (α^{(t)}) + β_{t} A^{H} (y - A α^{(t)})) + \sqrt{2 η_{t}} ζ_{t}, ζ_{t} \sim N (0, I),

$\alpha^{(t+1)}\leftarrow\alpha^{(t)}+\eta_t(s_{\theta}(\alpha^{(t)})+\beta_tA^{H}(y-A\alpha^{(t)}))+\sqrt{2\eta_t}\zeta_t\text{, }\zeta_t\sim\mathcal{N}(0,I),$ where

α^{(t)}

$\alpha^{(t)}$ are the estimated basis coefficient images at step t,

η_{t}

$\eta_t$ is the learning rate,

s_{θ}

$s_{\theta}$ is the score model output,

β_{t}

$\beta_t$ is the weight of the data consistency term,

A^{H}

$A^{H}$ is adjoint of the forward operator, and

ζ_{t}

$\zeta_t$ is the annealing noise. For this work, we generated the k-space data by applying the forward operator on the basis coefficient images as a proof of principle to show that basis images can be obtained using the score model, noting that this is an inverse crime¹². We therefore compared our result to the T2Sh reconstruction acting as a “ground-truth.”

Results and Discussion

Fig. 2 shows the result of sampling from the prior distribution learned by the score model. As the sampling steps increase, the images converge to a sample from the prior. In Fig 3 we show the result of posterior sampling during the reconstruction iterations. The process begins with random noise and after the first step of posterior sampling, the images are similar to the adjoint of the under-sampled k-space data. The posterior sampling images are generated through annealed Langevin dynamics with the forward model and corresponding measurement gradient. Fig. 4 shows the result of posterior sampling compared to the T2Sh basis coefficient images on a scan from the test set. The reconstructed coefficient images are well in agreement with the ground truth coefficients. The low normalized root mean squared error (NRMSE) shows that posterior sampling through the score model is able to reconstruct basis coefficient images with good fidelity. Fig. 5 shows the virtual echo time images along the signal recovery curve after back-projecting with the basis. The plot shows the signal for two tissues (muscle and bone) where it can be observed that the curve from both the score model output and T2Sh output match closely.

Conclusion and Future Direction

We showed that score models can be trained and used to reconstruct highly sub-sampled multi-contrast FSE data in a low-dimensional subspace. The current results were obtained with k-space synthesized from T2Sh reconstruction as a proof of concept. In the future, we will expand our work to test the posterior sampling on experimentally evaluated under-sampled k-space data.

Acknowledgements

This work was supported by NSF IFML 2019844 and NIH U24EB029240. We would like to thank Shreyas Vasanawala, Marc Alley, and Lucile Packard Children's Hospital for their assistance with scan data.

References

1. Busse, R. F., Brau, A. C., Vu, A., Michelich, C. R., Bayram, E., Kijowski, R., ... & Rowley, H. A. (2008). Effects of refocusing flip angle modulation and view ordering in 3D fast spin echo. Magnetic resonance in medicine, 60(3), 640-649.

2. Mugler III, J. P. (2014). Optimized three‐dimensional fast‐spin‐echo MRI. Journal of magnetic resonance imaging, 39(4), 745-767.

3. Busse, R. F., Hariharan, H., Vu, A., & Brittain, J. H. (2006). Fast spin echo sequences with very long echo trains: design of variable refocusing flip angle schedules and generation of clinical T2 contrast. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 55(5), 1030-1037.

4. Tamir, J. I., Uecker, M., Chen, W., Lai, P., Alley, M. T., Vasanawala, S. S., & Lustig, M. (2017). T2 shuffling: sharp, multicontrast, volumetric fast spin‐echo imaging. Magnetic resonance in medicine, 77(1), 180-195.

5. Lustig, M, Donoho D, Pauly JM. Sparse MRI: The Application of Compressed Sensing for Rapid MR Imaging. Magnetic resonance in medicine 58, no. 6 (2007). doi:10.1002/mrm.21391.

6. Zhao, B., Lu, W., Hitchens, T. K., Lam, F., Ho, C., & Liang, Z. P. (2015). Accelerated MR parameter mapping with low‐rank and sparsity constraints. Magnetic resonance in medicine, 74(2), 489-498.

7. Tamir JI, Taviani V, Alley MT, Perkins BC, Hart L, O'Brien K, Wishah F, Sandberg JK, Anderson MJ, Turek JS, Willke TL, Lustig M, Vasanawala SS. Targeted rapid knee MRI exam using T2 shuffling. J Magn Reson Imaging. 2019 Jun;49(7):e195-e204. doi: 10.1002/jmri.26600. Epub 2019 Jan 13. PMID: 30637847; PMCID: PMC6551292.

8. Jalal, A., Arvinte, M., Daras, G., Price, E., Dimakis, A. G., & Tamir, J. (2021). Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems, 34, 14938-14954.

9. Hyungjin Chung and Jong Chul Ye, “Score-based diffusion models for accelerated mri,” Medical Image Analysis, vol. 80, pp. 102479, 2022.

10. Guanxiong Luo, Martin Heide, and Martin Uecker, “MRI reconstruction via data driven markov chain with joint uncertainty estimation,” arXiv preprint arXiv:2202.01479, 2022.

11. Song, Y., Shen, L., Xing, L., & Ermon, S. (2021). Solving inverse problems in medical imaging with score-based generative models. ICLR 2022.

12. Shimron, E., Tamir, J. I., Wang, K., & Lustig, M. (2022). Implicit data crimes: Machine learning bias arising from misuse of public data. Proceedings of the National Academy of Sciences, 119(13), e2117203119.

13. Bao S, Tamir JI, Young JL, Tariq U, Uecker M, Lai P, Chen W, Lustig M, Vasanawala SS. Fast comprehensive single-sequence four-dimensional pediatric knee MRI with T2 shuffling. J Magn Reson Imaging. 2017 Jun;45(6):1700-1711. doi: 10.1002/jmri.25508. Epub 2016 Oct 11. PMID: 27726251; PMCID: PMC6726427.

14. Song, Y., & Ermon, S. (2020). Improved techniques for training score-based generative models. Advances in neural information processing systems, 33, 12438-12448

Figures

Fig. 1 a) Illustration of the training process of the score model. The process involves randomly choosing a noise level at each step and adding that to the training samples. The perturbed samples are passed through the score network which predicts the gradient of the log prior. b) Posterior sampling procedure for basis coefficient estimation using score model prior from given y (under-sampled kspace data).

Fig. 2 Animation showing sampling from the learned prior. The score model is run independently with random initializations. This shows the score model's ability to generate images from the learned distribution. The model starts with gaussian noise and uses Annealed Langevin Dynamics to sample from the prior distribution.

Fig. 3 Animation of the posterior sampling reconstruction process. Images are shown at each iteration for the three basis coefficient images.

Fig. 4 Reconstructed basis coefficient images compared to the ground-truth T2Sh reconstruction and the difference image which is shown with a 10X scale. The score model output images match the ground truth images within acceptable NRMSE error values.

Fig. 5 Reconstructed time-series multi-contrast images as a function of the echo time. The first row represents the images reconstructed by the score-based posterior sampling, whereas the second row shows the T2Sh reconstruction acting as a ground-truth. The signal evolution as a function of the echo time index is shown in the bottom plot for two different tissue values (muscle and bone). Visually it can be observed that the score model output signal evolution is in very good agreement with T2Sh curves.