Get "solution trivially zero" when I run with a partitioned mesh

Numerical methods and mathematical models of Elmer
Post Reply
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Get "solution trivially zero" when I run with a partitioned mesh

Post by Romuald »

Hi

I am a happy user of Elmer since a month and managed successfully to solve the Laplace equation for various geometries. For speeding up my simulations, I installed MS MPI and the MPI version of Elmer.
Partioning my model in 4 was easy with following command:

Code: Select all

  
 ElmerGrid.exe 8 2 MeshFile.unv -autoclean -metis 4
 
Meshes have being created in the "Mesh" directory as planned. It is when I try running ElmerSolver_mpi that the problem got started. Running:

Code: Select all

mpiexec.exe -n 4 ElmerSolver_mpi.exe
produced the following output:

Code: Select all

ELMER SOLVER (v 8.4) STARTED AT: 2020/09/28 16:29:12
ELMER SOLVER (v 8.4) STARTED AT: 2020/09/28 16:29:12
ELMER SOLVER (v 8.4) STARTED AT: 2020/09/28 16:29:12
ELMER SOLVER (v 8.4) STARTED AT: 2020/09/28 16:29:12
ParCommInit:
ParCommInit:
ParCommInit: 
 Initialize #PEs:            4
ParCommInit:
 Initialize #PEs:            4
 Initialize #PEs:            4
 Initialize #PEs:            4
MAIN:
MAIN: =============================================================
MAIN: ElmerSolver finite element software, Welcome!
MAIN: This program is free software licensed under (L)GPL
MAIN: Copyright 1st April 1995 - , CSC - IT Center for Science Ltd.
MAIN: Webpage http://www.csc.fi/elmer, Email elmeradm@csc.fi
MAIN: Version: 8.4 (Rev: Release, Compiled: 2020-08-03)
MAIN:  Running in parallel using 4 tasks.
MAIN:  Running with just one thread per task.
MAIN:  Lua interpreted linked in.
MAIN: =============================================================
LoadInputFile: Reading only "Run Control" section
MAIN:
MAIN:
MAIN: -------------------------------------
MAIN: Reading Model: C:/Users/rduperri/Pro/CEA/Projets/RFQ_bas_cout/Calculs_FEM/case.sif
LoadInputFile: Scanning input file: C:/Users/rduperri/Pro/CEA/Projets/RFQ_bas_cout/Calculs_FEM/case.sif
LoadInputFile: Scanning only size info
LoadInputFile: First time visiting
LoadInputFile: Reading base load of sif file
LoadInputFile: Loading input file: C:/Users/rduperri/Pro/CEA/Projets/RFQ_bas_cout/Calculs_FEM/case.sif
LoadInputFile: Reading base load of sif file
LoadInputFile: Number of BCs: 3
LoadInputFile: Number of Body Forces: 0
LoadInputFile: Number of Initial Conditions: 0
LoadInputFile: Number of Materials: 1
LoadInputFile: Number of Equations: 1
LoadInputFile: Number of Solvers: 1
LoadInputFile: Number of Bodies: 1
Loading user function library: [StatElecSolve]...[StatElecSolver_Init0]
LoadMesh: Base mesh name: ./RFQ_TEST
LoadMesh: Elapsed REAL time:     0.1480 (s)
MAIN: -------------------------------------
AddVtuOutputSolverHack: Adding ResultOutputSolver to write VTU output in file: case
Loading user function library: [StatElecSolve]...[StatElecSolver_Init]
Loading user function library: [StatElecSolve]...[StatElecSolver_bulk]
Loading user function library: [StatElecSolve]...[StatElecSolver]
OptimizeBandwidth: ---------------------------------------------------------
OptimizeBandwidth: Computing matrix structure for: electrostatics...done.
OptimizeBandwidth: Half bandwidth without optimization: 7063
OptimizeBandwidth:
OptimizeBandwidth: Bandwidth Optimization ...done.
OptimizeBandwidth: Half bandwidth after optimization: 786
OptimizeBandwidth: ---------------------------------------------------------
Loading user function library: [ResultOutputSolve]...[ResultOutputSolver_Init]
Loading user function library: [ResultOutputSolve]...[ResultOutputSolver_bulk]
Loading user function library: [ResultOutputSolve]...[ResultOutputSolver]
ElmerSolver: Number of timesteps to be saved: 1
MAIN:
MAIN: -------------------------------------
MAIN:  Steady state iteration:            1
MAIN: -------------------------------------
MAIN:
SingleSolver: Attempting to call solver
SingleSolver: Solver Equation string is: electrostatics
StatElecSolve: -------------------------------------
StatElecSolve: STATELEC SOLVER:
StatElecSolve: -------------------------------------
StatElecSolve:
StatElecSolve: Electrostatic iteration: 1
StatElecSolve: Starting Assembly...
DefUtils::DefaultDirichletBCs: Setting Dirichlet boundary conditions
DefUtils::DefaultDirichletBCs: Dirichlet boundary conditions set
StatElecSolve:  Assembly (s)          :  0.18900000000000006
SolveSystem: Solution trivially zero!
StatElecSolve:  Solve (s)             :   0.0000000000000000
StatElecSolve:  Result Norm   :    0.0000000000000000
StatElecSolve:  Relative Change :    0.0000000000000000
StatElecSolve:
Loading user function library: [StatElecSolve]...[StatElecSolver_post]
ComputeChange: SS (ITER=1) (NRM,RELC): (  0.0000000      0.0000000     ) :: electrostatics
SingleSolver: Attempting to call solver
SingleSolver: Solver Equation string is: internalvtuoutputsolver
ResultOutputSolver: -------------------------------------
ResultOutputSolver: Saving with prefix: case
ResultOutputSolver: Creating list for saving - if not present
CreateListForSaving: Field Variables for Saving
ResultOutputSolver: Saving in unstructured VTK XML (.vtu) format
VtuOutputSolver: Saving results in VTK XML format with prefix: case
VtuOutputSolver: Saving number of partitions: 4
ResultOutputSolver: -------------------------------------
Loading user function library: [ResultOutputSolve]...[ResultOutputSolver_post]
ElmerSolver: *** Elmer Solver: ALL DONE ***
ElmerSolver: The end
SOLVER TOTAL TIME(CPU,REAL):         1.67        1.67
ELMER SOLVER FINISHED AT: 2020/09/28 16:29:14
As you can see, no obvious error message but no potential is calculated. I tried to do exactly the same but this time without "mpiexec", just launching the "ElmerSolver_mpi" as a single exe. It works. It is only when I run with mpi, enven with only 1 partition, that the solver is unable to run properly. I attached case.sif file, the Mesh.unv is too large for the forum.

Many thanks in advance for your help!

Romuald
Attachments
case.sif
(2.05 KiB) Downloaded 168 times
Last edited by Romuald on 06 Oct 2020, 08:25, edited 1 time in total.
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Get "solution trivially zero" when I run with MPI

Post by raback »

Hi

Probably your "autoclean" renumbers the BC indexes so that you don't really apply BCs at all. Or do you really have >1000 BCs?

-Peter
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: Get "solution trivially zero" when I run with MPI

Post by Romuald »

Hi

I do not have 1000 BCs but 3. I tried without the "autoclean". It solves the problem when running on only one cpu and then requiring no partition of the mesh. As soon as I try to partition my mesh either the "-partition" or "-metis", ElemrGrid fails. The "autoclean" was solving this problem so far.

I am working on it also any tips is welcome.

Thanks

Romuald
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: Get "solution trivially zero" when I run with MPI

Post by Romuald »

OK. It finally worked with 4 nodes for instance.

I just replaced "-autoclean" by "-removeunused" and it was enough to get a good partition with metis. ElmerSolver_mpi did its job after that.

Thank you Peter for pointing out the keyword "-autoclean".

Bye

Romuald
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Get "solution trivially zero" when I run with MPI

Post by raback »

Hi Romuald

Probably the Metis routines could not operate with the orphan nodes as they do not belong to the connectivity graph.

You could try using

Code: Select all

-metiskway 4 -partdual -removeunused
Dual graph may give smoother domain interfaces.

-Peter
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: Get "solution trivially zero" when I run with MPI

Post by Romuald »

I just tried it. Problem was solved and indeed I remarked smoother results at partition interface.

Thx

Romuald
Post Reply