Domain decomposition

From Serpent Wiki
Jump to: navigation, search

Domain decomposition is used for saving computer memory by spatially dividing large burnup calculation problems between multiple calculation nodes. Each node handles particle transport only within its own domain, and particles crossing the domain boundaries are sent to another node to complete the history. The implementation in Serpent relies on a collision-based approach[1], i.e. the transfer is invoked when the particle undergoes a collision in a material which belongs to another domain. The decomposition affects only burnable materials divided into depletion zones using the div card. The most typical domain decomposition case is a full-core burnup calculation, in which the fuel is decomposed into separate domains to save memory. Interaction data for the other materials is shared by all parallel tasks.

The current implementation in Serpent 2 is relatively simple, and mainly designed for reactor cores. The number of domains is determined by the number of MPI tasks, and the division using the set dd input option. There are three division types:

  1. simple, based on depletion zone index, MODE 1.
  2. geometry-based division into sectors, MODE 2.
  3. geometry-based division into sectors and a central zone, MODE 3.

Notes:

  • The methodology was implemented in version 2.1.31, and is still under development. Any bugs or problems should be reported at the Serpent discussion forum.
  • The domain decomposition methodology allows the use of the restart capability from Serpent 2.1.32 version. The multiple restart files generated within a domain decomposition simulation can be read either in a subsequent domain decomposition simulation or within an automated depletion sequence, e.g., group-constant generation (matching the number of MPI tasks).
  • From Serpent 2.1.32 version and on, the associated depletion output file in a domain decomposition simulation is not limited to parent materials.

Full-core example

The method is illustrated by a full-core example, where the depletion zone division and domain decomposition is defined by input lines:

% --- Domain decomposition ----------------------------------------------------

% --- Divide fuel into depletion zones:

div fuel sep 1

% --- Domain decomposition (1 = simple, 2 = sector, 3 = sector + center)

set dd 3

% -----------------------------------------------------------------------------

When the calculation is run with 5 MPI tasks, it produces the following output:

sss2 -mpi 5 core

(...)

ividing materials into depletion zones...

Material fuel:

 - 63624 cells
 - 63624 depletion zones in total

Sorting lists (this may take a while)...
OK.

Counting cells...
OK.

Decomposing 63624 divided materials into 5 domains:

Domain 1: 12744 materials (20.0%)
Domain 2: 12720 materials (20.0%)
Domain 3: 12720 materials (20.0%)
Domain 4: 12720 materials (20.0%)
Domain 5: 12720 materials (20.0%)

(...)

and a geometry plot that shows the domains in different colors (switching the current setup from MODE 3, to MODE 2, to MODE 1, respectively):

(a) sector-wise division plus central zone (MODE 3), (b) sector-wise division (MODE 2), (c) index-based division (MODE 1).

Dd mode3.png Dd mode2.png Dd mode1.png

complete full-core input

% -----------------------------------------------------------------------------

set title "Monte Carlo performance benchmark"

% Based on revision 1.2 (July 2011) of Monte Carlo performance benchmark:
%
% http://www.oecd-nea.org/dbprog/MonteCarloPerformanceBenchmark.htm

% -----------------------------------------------------------------------------

% --- Neutron population and criticality cycles:

set pop 500000 1000 200
set opti 1

% --- Plot:

plot 3 1000 1000 0 -187.6 187.6 -187.6 187.6

% --- Thermal flux / fission rate mesh:

mesh 2 996 904
mesh 3 996 996

% -----------------------------------------------------------------------------
% ----- Geometry --------------------------------------------------------------

% --- Cross section library file path:

set acelib "sss_jeff31u.xsdata"

% --- Boundary between hot and cold:

surf  1 pz 0.0

% -- Water layers:

cell 31  3  cold_water  -1
cell 32  3  hot_water    1

% --- Fuel pin:

pin 1
fuel    0.41
clad    0.475
fill 3

% --- Guide tube:

pin 2
fill 3  0.56
clad    0.62
fill 3

% --- Pin lattice:

lat 110  1  0.0 0.0 17 17  1.26

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 1 1 1 2 1 1 2 1 1 2 1 1 1 1 1 
1 1 1 2 1 1 1 1 1 1 1 1 1 2 1 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 2 1 1 2 1 1 2 1 1 2 1 1 2 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 2 1 1 2 1 1 2 1 1 2 1 1 2 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 2 1 1 2 1 1 2 1 1 2 1 1 2 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 1 2 1 1 1 1 1 1 1 1 1 2 1 1 1 
1 1 1 1 1 2 1 1 2 1 1 2 1 1 1 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 

% --- assembly data:

surf  1000  inf

cell 110  4  fill 110   -1000

% --- Radial reflector:

cell 51  5  lower_radial_refl  -1
cell 52  5  upper_radial_refl   1

% --- Core lattice:

lat 211  1  0.0 0.0 17 17  21.42

5 5 5 5 5 4 4 4 4 4 4 4 5 5 5 5 5
5 5 5 4 4 4 4 4 4 4 4 4 4 4 5 5 5
5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5
5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5
5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5
5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5
5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5
5 5 5 4 4 4 4 4 4 4 4 4 4 4 5 5 5
5 5 5 5 5 4 4 4 4 4 4 4 5 5 5 5 5

surf 211 sqc 0.0 0.0 182.07 

cell 211 210 fill 211          -211
cell 212 210 lower_radial_refl  211 -1
cell 213 210 upper_radial_refl  211  1

% --- Core geometry:

surf 11  pz -229.0
surf 12  pz -199.0
surf 13  pz -193.0
surf 14  pz -183.0
surf 15  pz  183.0
surf 16  pz  203.0
surf 17  pz  215.0
surf 18  pz  223.0

surf 20 cyl 0.0 0.0 187.6
surf 21 cyl 0.0 0.0 209.0
surf 22 cyl 0.0 0.0 229.0
surf 23 cyl 0.0 0.0 249.0

cell 1   0  bottom_plate      -22      11 -12
cell 2   0  bottom_nozzle     -20      12 -13
cell 3   0  bottom_fa         -20      13 -14
cell 4   0  lower_radial_refl  20 -21  12 -14
cell 5   0  fill 210          -21      14 -15
cell 6   0  upper_radial_refl  20 -21  15 -17  
cell 7   0  top_fa            -20      15 -16
cell 8   0  top_nozzle        -20      16 -17
cell 9   0  cold_water         21 -22  12 -17
cell 10  0  top_plate         -22      17 -18
cell 11  0  vessel             22 -23  11 -18

cell 666 0 outside             23      11 -18
cell 667 0 outside            -11
cell 668 0 outside             18

% -----------------------------------------------------------------------------
% --- Materials ---------------------------------------------------------------

mat fuel  -10.062  burn 1

92234.03c  4.9476E-6
92235.03c  4.8218E-4
92236.03c  9.0402E-5
92238.03c  2.1504E-2
93237.03c  7.3733E-6
94238.03c  1.5148E-6
94239.03c  1.3955E-4
94240.03c  3.4405E-5
94241.03c  2.1439E-5
94242.03c  3.7422E-6
95241.03c  4.5041E-7
95242.03c  9.2301E-9
95243.03c  4.7878E-7
96242.03c  1.0485E-7
96243.03c  1.4268E-9
96244.03c  8.8756E-8
96245.03c  3.5285E-9
42095.03c  2.6497E-5
43099.03c  3.2772E-5
44101.03c  3.0742E-5
44103.03c  2.3505E-6
47109.03c  2.0009E-6
54135.03c  1.0801E-8
55133.03c  3.4612E-5
60143.03c  2.6078E-5
60145.03c  1.9898E-5
62147.03c  1.6128E-6
62149.03c  1.1627E-7
62150.03c  7.1727E-6
62151.03c  5.4947E-7
62152.03c  3.0221E-6
63153.03c  2.6209E-6
64155.03c  1.5369E-9
 8016.03c  4.5737E-2

mat clad -5.77 rgb  100 100 100

40000.03c  1.0

mat cold_water -0.74  moder lwtr 1001 rgb 100 150 200

 1001.03c  2.000
 8016.03c  1.000
 5010.03c  6.490E-4
 5011.03c  2.689E-3

mat hot_water -0.66  moder lwtr 1001  rgb 100 150 200

 1001.03c  2.000
 8016.03c  1.000
 5010.03c  6.490E-4
 5011.03c  2.689E-3

mat vessel -7.9  rgb  50 50 50

26000.03c  -96.3
28000.03c   -1.0
25055.03c   -1.0
42000.03c   -0.6
14000.03c   -0.4
24000.03c   -0.25
 6000.03c   -0.25
29000.03c   -0.2

mat lower_radial_refl -4.32  moder lwtr 1001  rgb  100 100 100

 1001.03c   -0.0095661
 8016.03c   -0.0759107
 5010.03c   -3.08409E-4
 5011.03c   -1.40499E-5
26000.03c   -0.6309028
28000.03c   -0.0822917
25055.03c   -0.0182870
14000.03c   -0.0091435
24000.03c   -0.1737269

mat upper_radial_refl -4.28  moder lwtr 1001  rgb  100 100 100

 1001.03c   -0.0086117
 8016.03c   -0.0683369
 5010.03c   -2.77638E-5
 5011.03c   -1.26481E-4
26000.03c   -0.6367991
28000.03c   -0.0830607
25055.03c   -0.0184579
14000.03c   -0.0092290
24000.03c   -0.1753505

mat bottom_plate -7.184  moder lwtr 1001

 1001.03c   -0.0011505
 8016.03c   -0.0091296
 5010.03c   -3.70915E-6
 5011.03c   -1.68974E-5
26000.03c   -0.6828925
28000.03c   -0.0890729
25055.03c   -0.0197940
14000.03c   -0.0098970
24000.03c   -0.1880429

mat bottom_nozzle -2.53  moder lwtr 1001

 1001.03c   -0.0245014
 8016.03c   -0.1944274
 5010.03c   -7.89917E-5
 5011.03c   -3.59854E-4
26000.03c   -0.5386364
28000.03c   -0.0702569
25055.03c   -0.0156126
14000.03c   -0.0078063
24000.03c   -0.1483202

mat top_nozzle -1.746  moder lwtr 1001

 1001.03c   -0.0358870
 8016.03c   -0.2847761
 5010.03c   -1.15699E-4
 5011.03c   -5.27075E-4
26000.03c   -0.4682990
28000.03c   -0.0610825
25055.03c   -0.0135739
14000.03c   -0.0067869
24000.03c   -0.1289519

mat top_plate -4.28  moder lwtr 1001

 1001.03c   -0.0086117
 8016.03c   -0.0683369
 5010.03c   -2.77638E-5
 5011.03c   -1.26481E-4
26000.03c   -0.6367991
28000.03c   -0.0830607
25055.03c   -0.0184579
14000.03c   -0.0092290
24000.03c   -0.1753505

mat bottom_fa -3.044  moder lwtr 1001

 1001.03c   -0.0162913
 8016.03c   -0.1292776
 5010.03c   -5.25228E-5
 5011.03c   -2.39272E-4
40000.03c   -0.8541393

mat top_fa -1.762  moder lwtr 1001

 1001.03c   -0.0292856
 8016.03c   -0.2323919
 5010.03c   -9.44159E-5
 5011.03c   -4.30120E-4
40000.03c   -0.7377980

% --- Thermal scattering data

therm lwtr lwj3.00t

% -----------------------------------------------------------------------------
% --- Depletion history -------------------------------------------------------

% --- Decay and fission yield libraries:

set declib "/xs/sss_jeff31.dec"
set nfylib "/xs/sss_jeff31.nfy"

% --- Irradiation cycle:

set powdens 40.0E-3

dep butot

    0.10
    0.50
    1.00
    1.50
    2.00
    2.50
    3.00
    3.50
    4.00
    4.50
    5.00
    5.50
    6.00
    6.50
    7.00
    7.50
    8.00
    8.50
    9.00
    9.50
   10.00
   10.50
   11.00
   11.50
   12.00
   12.50
   13.00
   13.50
   14.00
   14.50
   15.00
   17.50
   20.00
   22.50
   25.00
   27.50
   30.00
   32.50
   35.00
   37.50
   40.00

% --- Isotope list for inventory calculation:

set inventory all

% --- Depletion output:

set depout 2

% --- Material volumes:

set mcvol 10000000000

% -----------------------------------------------------------------------------
% --- Domain decomposition ----------------------------------------------------

% --- Divide fuel into depletion zones:

div fuel sep 1

% --- Domain decomposition (1 = simple, 2 = sector, 3 = sector + center)

set dd 3

% -----------------------------------------------------------------------------

References

  1. ^ García, M., Leppänen, J. and Sánchez-Espinoza, V. "A Collision-based Domain Decomposition scheme for large-scale depletion with the Serpent 2 Monte Carlo code." Ann. Nucl. Energy 152 (2021) 108026.