Comparing GLLS with Stochastic Sampling Based ...

Comparing GLLS with Stochastic Sampling Based Data Assimilation NEA WPEC Meetings: SG39/SG46 OECD Headquarters, Paris, France

Daniel Siefman, M. Hursin, D. Rochman, S. Pelloni, A. Pautz ´ Ecole Polytechnique F´ed´erale de Lausanne Paul Scherrer Institute

May 14-18, 2018 [email protected] (EPFL)

GLLS vs. Stochastic DA

May 14-18, 2018

1 / 20

NEA WPEC Meetings: SG39/SG46

Introduction

Introduction

Recent advancements allow to adjust nuclear data and integral parameters with stochastic sampling Could have advantages: 1. Don’t have to program sensitivity coefficient solvers 2. Could handle non-linear relationships between nuclear data and integral parameters 3. Could be used without Gaussian assumptions

Want to first prove that these methods compare well with GLLS Do simple test case so that results are easily verified by community How do the adjusted nuclear data and integral parameters compare? How does the computational cost compare?

[email protected] (EPFL)


May 14-18, 2018

2 / 20


Approach

Approach

Focus on two stochastic-sampling-based methods: 1. MOCABA: Monte Carlo Bayesian Analysis 2. BMC: Bayesian Monte Carlo

Model the Jezebel-Pu239 benchmark in Serpent2 Sensitivities calculated with Serpent Stochastic sampling done with NUSS code 1. Uses multigroup covariance data 2. Perturbations are applied to the ACE files

187-energy-group structure ENDFB/VII.1 nuclear data and covariances



May 14-18, 2018

3 / 20


Theory

MOCABA To understand MOCABA, we can use GLLS as a starting point Take the GLLS equations: −1 E − C(σ0 ) σ 0 = σ0 + Mσ ST SMσ ST + MEM

(1)

−1 M0σ = Mσ − Mσ ST SMσ ST + MEM SMσ

(2)

We applied a linear model to C(σ0 ) in order to find analytical solutions for the posteriors C(σ) ≈ C(σ0 ) + S(σ − σ0 )

(3)

Comes with some drawbacks: 1. Assume C(σ0 ) is linear 2. Have to calculate sensitivity coefficients S [email protected] (EPFL)


May 14-18, 2018

4 / 20


Theory

MOCABA (2) The sensitivity coefficients are used to approximate two terms: 1. SMσ ST : covariance matrix of C(σ0 ) 2. Mσ ST : covariance matrix between σ and C(σ0 )

Knowing this, let’s rewrite the equations −1 σ 0 = σ0 + Mσ,C MC + MEM E − C(σ0 )

(4)

−1 M0σ = Mσ − Mσ,C MC + MEM Mσ,C T

(5)

MC ≈ SMσ ST Mσ,C ≈ Mσ ST Can we approximate these terms in a different way? [email protected] (EPFL)


May 14-18, 2018

5 / 20


Theory

MOCABA (3) MOCABA’s principle: Approximate MC and Mσ,C with the Monte Carlo method Randomly sample σ0 from Gaussian distribution with Mσ For every randomly sample, σi , calculate integral parameter Ci (σi ) Create a population set of N samples: C1 (σ1 ), . . . , CN (σN ) Find estimates of MC and Mσ,C N

ˆC = M

1 X ¯ Ci − C ¯ T Ci − C N −1

(6)

i=1 N

ˆ σ,C = M

1 X ¯ T σi − σ ¯ Ci − C N −1

(7)

i=1



May 14-18, 2018

6 / 20


Theory

MOCABA (4) Linear approximations in GLLS for posterior integral parameters can also be replaced C0 (σ 0 ) = C(σ0 ) + S(σ 0 − σ0 )

(8)

M0C = SM0σ ST

(9)

Using MOCABA they’re now calcualted with −1 C0 = C(σ0 ) + MC MC + MEM E − C(σ0 )

(10)

−1 M0C = MC − MC MC + MEM MC T

(11)



May 14-18, 2018

7 / 20


Theory

MOCABA Summary

Theoretically consistent with GLLS Assumes Gaussian distributions for Prior and posterior nuclear data Prior and posterior integral parameters (C and E)

Could be useful for non-linear C(σ) ˆ C and M ˆ σ,C ? What is the effect of the statistical nature of M How many random samples are needed? Are the results consistent with GLLS?



May 14-18, 2018

8 / 20


Theory

BMC Stochastically searches for the maximum likelihood to calculate C0 , M0C , σ 0 , and M0σ Makes no assumptions about the prior distribution Nuclear data can be sampled from any PDF Integral parameters can be non-normal

Use each Ci (σi ) to calculate likelihood values 2 L E σi ∝ e −χi /2 χ2i = E − Ci (σi )

T

(MEM )−1 E − Ci (σi )

(12)

(13)

Each χ2i is used to calculate a weight, wi wi = L E σi [email protected] (EPFL)


(14) May 14-18, 2018

9 / 20


Theory

BMC (2) Weights are used in weighted averages to calculate posteriors Larger weights (better agreement between Ci (σi ) and E) contribute more to the adjustments PN

0

C =

i=1 wi × PN i=1 wi

Ci

(15)

¯ Ci − C ¯ T × Ci − C = PN i=1 wi PN wi × σi σ 0 = i=1 PN i=1 wi PN wi × (σi − σ ¯ )(σi − σ ¯ )T M0σ = i=1 PN i=1 wi

M0C


PN

i=1 wi


(16)

(17)

(18) May 14-18, 2018

10 / 20


Application

Test Case We compared the three methods for a simple case Jezebel-Pu239: keff , F28/F25, F49/F25, F37/F25 Considered isotopes: 239 Pu, Considered nuclear data:

240 Pu, 241 Pu, 235 U, 238 U,

and

237 Np

Elastic scattering (MF3/MT2) Inelastic scattering (MF3/MT4) Capture (MF3/MT102) Fission (MF3/MT18) Average prompt fission neutron multiplicity (MF1/MT456)

Prompt fission spectrum only for 239 Pu 10,000 nuclear data samples with NUSS tool Approximately four times more expensive than sensitivity calculations



May 14-18, 2018

11 / 20


Integral Parameter Adjustments

Adjustments to C Values Good agreement between the methods Are differences significant? Caused by differences between methods? Statistics? Table: Posterior calculated-to-experimental ratios (C0 /E) and posterior relative standard deviations from nuclear data (∆C0 )

keff F28/F25 F49/F25 F37/F25

GLLS 0.99982 0.996 0.984 0.994


C0 /E MOCABA 0.99996 0.996 0.984 0.994

BMC 1.00009 0.997 0.984 0.994

GLLS 0.192% 1.0% 0.5% 1.3%


∆C0 MOCABA 0.192% 1.0% 0.5% 1.3%

BMC 0.193% 1.0% 0.5% 1.2%

May 14-18, 2018

12 / 20


Bootstrapping

Bootstrapping Bootstrapping is used to estimate the posteriors’ standard error Procedure: Sample with replacement N times from the 10,000 random samples With each bootstrap sample, MOCABA and BMC posteriors are calculated, leading to N sets of posteriors ˆ and the error of the mean, Estimate the posterior’s bootstrap mean, θ, ˆ ∆θ, with the N bootstrap samples.

θˆ =

∆θˆ =


PN

PN

n=1 θn

(19)

N ˆ2 − θ) N −1

n=1 (θn


!1/2 (20)

May 14-18, 2018

13 / 20


Bootstrapping

Bootstrapping C Adjustments



May 14-18, 2018

14 / 20


Bootstrapping

Bootstrapping C Adjustments (2)



May 14-18, 2018

15 / 20


Nuclear Data Adjustments

Bootstrapping: Nuclear Data Adjustments

Shaded regions are 95% confidence intervals: 1.96 × ∆θˆ



May 14-18, 2018

16 / 20



Bootstrapping: Nuclear Data Adjustments (2)




May 14-18, 2018

17 / 20



Bootstrapping: Nuclear Data Adjustments (3)




May 14-18, 2018

18 / 20



Posterior Correlation Matrices

Good agreement in trends BMC shows low correlation noise (±0.1)

(n, e) (n, i) (n, f) (n, )


(n, e)

MOCABA

(n, i) (n, f)

1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.8 1.0

(n, )

(n, e) (n, i) (n, f) (n, )


(n, e)

BMC

(n, i) (n, f)

1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.8 1.0

Correlation

(n, f) (n, )

1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.8 1.0

Correlation

GLLS

(n, i)

Correlation

(n, e)

(n, )

(n, e) (n, i) (n, f) (n, )

May 14-18, 2018

19 / 20


Conclusions

Conclusions

Simple test case shows good agreement between GLLS, MOCABA, and BMC BMC larger uncertainties than MOCABA Needs more samples to have same accuracy as MOCABA May not be practical for non-academic applications

For Jezebel-Pu239, supports traditional use of GLLS MOCABA could be used complementary to GLLS to verify adjustments Provides confidence in stochastic DA, but needs to be tested in larger and more diverse applications



May 14-18, 2018

20 / 20