• norsk
    • English
  • norsk 
    • norsk
    • English
  • Logg inn
Vis innførsel 
  •   Hjem
  • Faculty of Mathematics and Natural Sciences
  • Department of Mathematics
  • Department of Mathematics
  • Vis innførsel
  •   Hjem
  • Faculty of Mathematics and Natural Sciences
  • Department of Mathematics
  • Department of Mathematics
  • Vis innførsel
JavaScript is disabled for your browser. Some features of this site may not work without it.

Epistemic Uncertainty Quantification in Deep Learning by the Delta Method

Nilsen, Geir Kjetil
Doctoral thesis
Thumbnail
Åpne
PDF (3.273Mb)
Permanent lenke
https://hdl.handle.net/11250/2993348
Utgivelsesdato
2022-05-16
Metadata
Vis full innførsel
Samlinger
  • Department of Mathematics [888]
Sammendrag
This thesis explores the Delta method and its application to deep learning image classification. The Delta method is a classical procedure for quantifying uncertainty in statistical models, but its direct application to deep neural networks is prevented by the large number of parameters P. We recognize the Delta method as a measure of epistemic as opposed to aleatoric uncertainty and break it into two components: the eigenvalue spectrum of the inverse Fisher information (i.e. inverse Hessian) of the cost function and the per-example sensitivities (i.e. gradients) of the model function. We mainly focus on the computational aspects, and show how to efficiently compute low and full-rank approximations of the inverse Fisher information matrix, which in turn reduces the computational complexity of the naïve Delta method from O(P²) space and O(P³) time, to O(P) space and time. We provide bounds for the approximation error by a novel error propagating technique, and validate the developed methodology with a released TensorFlow implementation. By a comparison with the classical Bootstrap, we show that there is a strong linear relationship between the quantified predictive epistemic uncertainty levels obtained from the two methods when applied on a few well known architectures using the MNIST and CIFAR-10 datasets.
Består av
Paper 1: Geir K. Nilsen, Antonella Z. Munthe-Kaas, Hans J. Skaug and Morten Brun, Efficient Computation of Hessian Matrices in TensorFlow, arXiv preprint: 1905.05559, 2019, revised 2021. The article is available in the thesis file. The article is also available at: https://doi.org/10.48550/arXiv.1905.05559

Paper 2: Geir K. Nilsen, Antonella Z. Munthe-Kaas, Hans J. Skaug and Morten Brun, Epis- temic Uncertainty Quantification in Deep Learning Classification by the Delta Method, Neural Networks, 2022, 145: 164-176. The article is available at: https://hdl.handle.net/11250/2835021

Paper 3: Geir K. Nilsen, Antonella Z. Munthe-Kaas, Hans J. Skaug, Morten Brun, A Com- parison of the Delta Method and the Bootstrap in Deep Learning Classification, arXiv preprint: 2107.01606, 2021. The article is available in the thesis file. The article is also available at: https://doi.org/10.48550/arXiv.2107.01606
Utgiver
The University of Bergen
Opphavsrett
Copyright the Author. All rights reserved

Kontakt oss | Gi tilbakemelding

Personvernerklæring
DSpace software copyright © 2002-2019  DuraSpace

Levert av  Unit
 

 

Bla i

Hele arkivetDelarkiv og samlingerUtgivelsesdatoForfattereTitlerEmneordDokumenttyperTidsskrifterDenne samlingenUtgivelsesdatoForfattereTitlerEmneordDokumenttyperTidsskrifter

Min side

Logg inn

Statistikk

Besøksstatistikk

Kontakt oss | Gi tilbakemelding

Personvernerklæring
DSpace software copyright © 2002-2019  DuraSpace

Levert av  Unit