Book,
Bayesian Approach to Inverse Problems
ISTE Wiley, (2013)
Abstract
Many scientific, medical or engineering problems raise the issue of recovering some physical quantities from indirect measurements; for instance, detecting or quantifying flaws or cracks within a material from acoustic or electromagnetic measurements at its surface is an essential problem of non-destructive evaluation. The concept of inverse problems precisely originates from the idea of inverting the laws of physics to recover a quantity of interest from measurable data.
Unfortunately, most inverse problems are ill-posed, which means that precise and stable solutions are not easy to devise. Regularization is the key concept to solve inverse problems.
The goal of this book is to deal with inverse problems and regularized solutions using the Bayesian statistical tools, with a particular view to signal and image estimation.
The first three chapters bring the theoretical notions that make it possible to cast inverse problems within a mathematical framework. The next three chapters address the fundamental inverse problem of deconvolution in a comprehensive manner. Chapters 7 and 8 deal with advanced statistical questions linked to image estimation. In the last five chapters, the main tools introduced in the previous chapters are put into a practical context in important applicative areas, such as astronomy or medical imaging.
TOC:
PART I. FUNDAMENTAL PROBLEMS AND TOOLS 23
Chapter 1. Inverse Problems, Ill-posed Problems 25
Guy DEMOMENT, Jérôme IDIER
1.1. Introduction 25
1.2. Basic example 26
1.3. Ill-posed problem 30
1.3.1. Case of discrete data 31
1.3.2. Continuous case 32
1.4. Generalized inversion 34
1.4.1. Pseudo-solutions 35
1.4.2. Generalized solutions 35
1.4.3. Example 35
1.5. Discretization and conditioning 36
1.6. Conclusion 38
1.7. Bibliography 39
Chapter 2. Main Approaches to the Regularization of Ill-posed Problems 41
Guy DEMOMENT, Jérôme IDIER
2.1. Regularization 41
2.1.1. Dimensionality control 42
2.1.2. Minimization of a composite criterion 44
2.2. Criterion descent methods 48
2.2.1.Criterion minimization for inversion 48
2.2.2. The quadratic case 49
2.2.3. The convex case 51
2.2.4. General case 52
2.3. Choice of regularization coefficient 53
2.3.1. Residual error energy control 53
2.3.2. “L-curve” method 53
2.3.3. Cross-validation 54
2.4. Bibliography 56
Chapter 3. Inversion within the Probabilistic Framework 59
Guy DEMOMENT, Yves GOUSSARD
3.1. Inversion and inference 59
3.2. Statistical inference 60
3.2.1. Noise law and direct distribution for data 61
3.2.2. Maximum likelihood estimation 63
3.3. Bayesian approach to inversion 64
3.4. Links with deterministic methods 66
3.5. Choice of hyperparameters 67
3.6. A priori model68
3.7. Choice of criteria 70
3.8. The linear, Gaussian case 71
3.8.1. Statistical properties of the solution 71
3.8.2. Calculation of marginal likelihood 73
3.8.3. Wiener filtering 74
3.9. Bibliography 76
PART II. DECONVOLUTION 79
Chapter 4. Inverse Filtering and Other Linear Methods 81
Guy LE BESNERAIS, Jean-François GIOVANNELLI, Guy DEMOMENT
4.1. Introduction 81
4.2. Continuous-time deconvolution 82
4.2.1. Inverse filtering 82
4.2.2. Wiener filtering 84
4.3. Discretization of the problem 85
4.3.1. Choice of a quadrature method 85
4.3.2. Structure of observation matrix H 87
4.3.3. Usual boundary conditions 89
4.3.4. Problem conditioning 89
4.3.5.Generalized inversion 91
4.4. Batch deconvolution 92
4.4.1. Preliminary choices 92
4.4.2. Matrix form of the estimate 93
4.4.3. Hunt’s method (periodic boundary hypothesis) 94
4.4.4. Exact inversion methods in the stationary case 96
4.4.5. Case of non-stationary signals 98
4.4.6. Results and discussion on examples 98
4.5. Recursive deconvolution 102
4.5.1. Kalman filtering 102
4.5.2. Degenerate state model and recursive least squares 104
4.5.3. Autoregressive state model 105
4.5.4. Fast Kalman filtering 108
4.5.5. Asymptotic techniques in the stationary case 110
4.5.6. ARMA model and non-standard Kalman filtering 111
4.5.7. Case of non-stationary signals 111
4.5.8. On-lineprocessing: 2Dcase 112
4.6. Conclusion 112
4.7. Bibliography 113
Chapter 5. Deconvolution of Spike Trains 117
Frédéric CHAMPAGNAT, Yves GOUSSARD, Stéphane GAUTIER, Jérôme IDIER
5.1. Introduction 117
5.2. Penalization of reflectivities, L2LP/L2Hy deconvolutions 119
5.2.1. Quadratic regularization 121
5.2.2. Non-quadratic regularization 122
5.2.3. L2LPorL2Hy deconvolution 123
5.3. Bernoulli-Gaussian deconvolution 124
5.3.1. Compound BG model 124
5.3.2. Various strategies for estimation 124
5.3.3. General expression for marginal likelihood 125
5.3.4. An iterative method for BG deconvolution 126
5.3.5. Other methods 128
5.4. Examples of processing and discussion 130
5.4.1. Nature of the solutions 130
5.4.2. Setting the parameters 132
5.4.3. Numerical complexity 133
5.5. Extensions 133
5.5.1. Generalization of structures of R and H 134
5.5.2. Estimation of the impulse response . . . 134
5.6. Conclusion 136
5.7. Bibliography 137
Chapter 6. Deconvolution of Images 141
Jérôme IDIER, Laure BLANC-FÉRAUD
6.1. Introduction 141
6.2. Regularization in the Tikhonov sense 142
6.2.1. Principle 142
6.2.2. Connection with image processing by linear PDE 144
6.2.3. Limits of Tikhonov’s approach 145
6.3. Detection-estimation 148
6.3.1. Principle 148
6.3.2. Disadvantages 149
6.4. Non-quadratic approach 150
6.4.1. Detection-estimation and non-convex penalization 154
6.4.2. Anisotropic diffusion by PDE 155
6.5. Half-quadratic augmented criteria 156
6.5.1. Duality between non-quadratic criteria and HQ criteria 157
6.5.2. Minimization of HQ criteria 158
6.6. Application in image deconvolution 159
6.6.1. Calculation of the solution 159
6.6.2. Example 161
6.7. Conclusion 164
6.8. Bibliography 165
PART III. ADVANCED PROBLEMS AND TOOLS 169
Chapter 7. Gibbs-Markov Image Models 171
Jérôme IDIER
7.1. Introduction 171
7.2. Bayesian statistical framework 172
7.3. Gibbs-Markov fields 173
7.3.1. Gibbs fields 174
7.3.2. Gibbs-Markov equivalence 177
7.3.3. Posterior law of a GMRF 180
7.3.4. Gibbs-Markov models for images 181
7.4. Statistical tools, stochastic sampling 185
7.4.1. Statistical tools 185
7.4.2. Stochastic sampling 188
7.5. Conclusion 194
7.6. Bibliography 195
Chapter 8. Unsupervised Problems 197
Xavier DESCOMBES, Yves GOUSSARD
8.1. Introduction and statement of problem 197
8.2. Directly observed field 199
8.2.1. Likelihood properties 199
8.2.2. Optimization 200
8.2.3. Approximations 202
8.3. Indirectly observed field 205
8.3.1. Statement of problem 205
8.3.2. EM algorithm 206
8.3.3. Application to estimation of the parameters of a GMRF 207
8.3.4. EM algorithm and gradient 208
8.3.5. Linear GMRF relative to hyperparameters 210
8.3.6. Extensions and approximations 212
8.4. Conclusion 215
8.5. Bibliography 216
PART IV. SOME APPLICATIONS 219
Chapter 9. Deconvolution Applied to Ultrasonic Non-destructive Evaluation 221
Stéphane GAUTIER, Frédéric CHAMPAGNAT, Jérôme IDIER
9.1. Introduction 221
9.2. Example of evaluation and difficulties of interpretation 222
9.2.1. Description of the part to be inspected 222
9.2.2. Evaluation principle 222
9.2.3. Evaluation results and interpretation 223
9.2.4. Help with interpretation by restoration of discontinuities 224
9.3. Definition of direct convolution model 225
9.4. Blind deconvolution 226
9.4.1. Overview of approaches for blind deconvolution 226
9.4.2. DL2Hy/DBGd econvolution 230
9.4.3. Blind DL2Hy/DBG deconvolution 232
9.5. Processing real data 232
9.5.1. Processing by blind deconvolution 233
9.5.2. Deconvolution with a measured wave 234
9.5.3. Comparison between DL2Hy and DBG 237
9.5.4. Summary 240
9.6. Conclusion 240
9.7. Bibliography 241
Chapter 10. Inversion in Optical Imaging through Atmospheric Turbulence 243
Laurent MUGNIER, Guy LE BESNERAIS, Serge MEIMON
10.1. Optical imaging through turbulence 243
10.1.1. Introduction 243
10.1.2. Image formation 244
10.1.4. Imaging techniques 249
10.2. Inversion approach and regularization criteria used 253
10.3. Measurement of aberrations 254
10.3.1. Introduction 254
10.3.2. Hartmann-Shack sensor 255
10.3.3. Phase retrieval and phase diversity 257
10.4. Myopic restoration in imaging 258
10.4.1. Motivation and noise statistic 258
10.4.2. Data processing in deconvolution from wavefront sensing 259
10.4.3. Restoration of images corrected by adaptive optics 263
10.4.4. Conclusion 267
10.5. Image reconstruction in optical interferometry (OI) 268
10.5.1. Observation model 268
10.5.2. Traditional Bayesian approach 271
10.5.3. Myopic modeling 272
10.5.4. Results 274
10.6. Bibliography 277
Chapter 11. Spectral Characterization in Ultrasonic Doppler Velocimetry 285
Jean-François GIOVANNELLI, Alain HERMENT
11.1. Velocity measurement in medical imaging 285
11.1.1. Principle of velocity measurement in ultrasound imaging 286
11.1.2. Information carried by Doppler signals 286
11.1.3.Some characteristics and limitations 288
11.1.4. Data and problems treated 288
11.2. Adaptive spectral analysis 290
11.2.1. Least squares and traditional extensions 290
11.2.2. Long AR models – spectral smoothness – spatial continuity 291
11.2.3. Kalman smoothing 293
11.2.4. Estimation of hyperparameters 294
11.2.5. Processing results and comparisons 296
11.3. Tracking spectral moments 297
11.3.1. Proposed method 298
11.3.2. Likelihood of the hyperparameters 302
11.3.3. Processing results and comparisons 304
11.4. Conclusion 306
11.5. Bibliography 307
Chapter 12. Tomographic Reconstruction from Few Projections 311
Ali MOHAMMAD-DJAFARI, Jean-Marc DINTEN
12.1. Introduction 311
12.2. Projection generation model 312
12.3. 2D analytical methods 313
12.4. 3D analytical methods 317
12.5. Limitations of analytical methods 317
12.6. Discrete approach to reconstruction 319
12.7. Choice of criterion and reconstruction methods 321
12.8. Reconstruction algorithms 323
12.8.1. Optimization algorithms for convex criteria 323
12.8.2. Optimization or integration algorithms 327
12.9. Specific models for binary objects 328
12.10. Illustrations 328
12.10.1.2D reconstruction 328
12.10.2.3Dreconstruction 329
12.11. Conclusions 331
12.12. Bibliography 332
Chapter 13. Diffraction Tomography 335
Hervé CARFANTAN, Ali MOHAMMAD-DJAFARI
13.1. Introduction 335
13.2. Modeling the problem 336
13.2.1. Examples of diffraction tomography applications 336
13.2.2. Modeling the direct problem 338
13.3. Discretization of the direct problem 340
13.3.1. Choice of algebraic framework 340
13.3.2. Method of moments 341
13.3.3. Discretization by the method of moments 342
13.4. Construction of criteria for solving the inverse problem 343
13.4.1. First formulation: estimation of x 344
13.4.2. Second formulation: simultaneous estimation of x and φ 345
13.4.3. Properties of the criteria 347
13.5. Solving the inverse problem 347
13.5.1. Successive linearizations 348
13.5.2. Joint minimization 350
13.5.3. Minimizing MAP criterion 351
13.6. Conclusion 353
13.7. Bibliography 354
Chapter 14. Imaging from Low-intensity Data 357
Ken SAUER, Jean-Baptiste THIBAULT
14.1. Introduction 357
14.2. Statistical properties of common low-intensity image data 359
14.2.1. Likelihood functions and limiting behavior 359
14.2.2. Purely Poisson measurements 360
14.2.3. Inclusion of background counting noise 362
14.2.4. Compound noise models with Poisson information 362
14.3. Quantum-limited measurements in inverse problems 363
14.3.1. Maximum likelihood properties 363
14.3.2. Bayesian estimation 366
14.4. Implementation and calculation of Bayesian estimates 368
14.4.1. Implementation for pure Poisson model 368
14.4.2. Bayesian implementation for a compound data model 370
14.5. Conclusion 372
14.6. Bibliography 372
List of Authors 375
Index 377
Tags
- bayesian
- deconvolution
- gaussian_processes
- image_reconstruction
- imaging
- inverse_problems
- kalman_filter
- linear_algebra
- optimization
- pseudoinverse
- reference
- regularization
- review
- statistics
- tomography
