## Using ElmerSolver_mpi with simple example

Extension of Elmer in computational glaciology
inwoo.park
Posts: 1
Joined: 17 Aug 2018, 09:27
Antispam: Yes

### Using ElmerSolver_mpi with simple example

Dear, users.

I install the Elmer/Ice on Ubuntu 14.04 machine, which has the 8 threads (CPU: E3-1230v3). In general, Elmer supports the parallel computing, so I try to compute ‘SSA_IceSheet’ example with 4 threads as following.
ElmerGrid 1 2 rectangle.grd –metis 4 4
mpirun –np 4 ElmerSolver_mpi ssa_IceSheet.sif
To run ‘ssa_IceSheet.sif’, I change the ‘ssa_IceSheet.sif’ as following :
Solver 1 ! Solver 1 : SSA solver {{{
Equation = "SSA"
Variable = -dofs 2 "SSAVelocity"

Procedure = "ElmerIceSolvers" "SSABasalSolver"

! Serial computing
!Linear System Solver = Direct
!Linear System Direct Method = umfpack

!Nonlinear System Max Iterations = 100
!Nonlinear System Convergence Tolerance = 1.0e-12
!Nonlinear System Newton After Iterations = 5
!Nonlinear System Newton After Tolerance = 1.0e-05
!Nonlinear System Relaxation Factor = 1.00

! Parallel Computing
Linear System Solver = Direct
Linear System Direct Method = Mumps

Nonlinear System Max Iterations = 100
Nonlinear System Convergence Tolerance = 1.0e-12
Nonlinear System Newton After Iterations = 5
Nonlinear System Newton After Tolerance = 1.0e-05
Nonlinear System Relaxation Factor = 1.00

End !}}}

Solver 2 ! Solver 2 : Thckness Solver {{{
Equation = "Thickness"
Variable = -dofs 1 "H"

Exported Variable 1 = -dofs 1 "H Residual"

Exported Variable 2 = -dofs 1 "dHdt"

Compute dHdT = Logical True

Procedure = "ElmerIceSolvers" "ThicknessSolver"
Before Linsolve = "EliminateDirichlet" "EliminateDirichlet"

Linear System Solver = Iterative
Linear System Use Hypre = True
Linear System Iterative Method = BiCGStab

Linear System Max Iterations = 1500
Linear System Preconditioning = ILU0
Linear System Convergence Tolerance = Real 1.0e-12
Linear System Abort Not Converged = False
Linear System Residual Output = 1500

! equation is linear if no min/max
Nonlinear System Max Iterations = 50
Nonlinear System Convergence Tolerance = 1.0e-6
Nonlinear System Relaxation Factor = 1.00

Stabilization Method = stabilized
Apply Dirichlet = Logical True

!! the convection velocity (mean horizontal velocity)
Flow Solution Name = String "SSAVelocity"
End ! }}}

However, simulation is stopped with following error :
SSASolver-SSABasalSolver: Memory allocation done.
SSASolver-SSABasalSolver:
SSASolver-SSABasalSolver:
SSASolver-SSABasalSolver: -------------------------------------
SSASolver-SSABasalSolver: SSA BASAL VELOCITY NON-LINEAR ITERATION 1
SSASolver-SSABasalSolver:
SSASolver-SSABasalSolver: -------------------------------------
SSASolver-SSABasalSolver:
Fatal error in MPI_Allreduce: Invalid datatype, error stack:
MPI_Allreduce(861): MPI_Allreduce(sbuf=0x7fffd2e033dc, rbuf=0x7fffd2e033d4, count=1, INVALID DATATYPE, op=0x1, comm=0x84000004) failed
MPI_Allreduce(807): Invalid datatype
Fatal error in MPI_Allreduce: Invalid datatype, error stack:
MPI_Allreduce(861): MPI_Allreduce(sbuf=0x7ffc9aae918c, rbuf=0x7ffc9aae9184, count=1, INVALID DATATYPE, op=0x1, comm=0x84000004) failed
MPI_Allreduce(807): Invalid datatype
Fatal error in MPI_Allreduce: Invalid datatype, error stack:
MPI_Allreduce(861): MPI_Allreduce(sbuf=0x7ffc6563727c, rbuf=0x7ffc65637274, count=1, INVALID DATATYPE, op=0x1, comm=0x84000004) failed
MPI_Allreduce(807): Invalid datatype
Fatal error in MPI_Allreduce: Invalid datatype, error stack:
MPI_Allreduce(861): MPI_Allreduce(sbuf=0x7ffd10b77d7c, rbuf=0x7ffd10b77d74, count=1, INVALID DATATYPE, op=0x1, comm=0x84000007) failed
MPI_Allreduce(807): Invalid datatype
How to solve that problem??

Thank you.

Best regards,
Inwoo
tzwinger