I've compiled Elmer using a self-compiled MUMPS on CentOS 6.7, both using mpicc and libs from openmpi 1.6.5. (I compiled MUMPS using -fPIC to produce shared libraries, and it passes all the tests it comes with-- so I don't think that's the problem). I plan on submitting a tutorial on how I build the whole shebang once I get it working.
So, Elmer compiles and passes all but 4 tests, which from what I can gather is expected. However, if I set the direct solver to mumps it takes a dump with a SIGSEGV:
Code: Select all
ELMER SOLVER (v 8.1) STARTED AT: 2016/01/15 12:15:11
ParCommInit: Initialize #PEs: 1
MAIN:
MAIN: =============================================================
MAIN: ElmerSolver finite element software, Welcome!
MAIN: This program is free software licensed under (L)GPL
MAIN: Copyright 1st April 1995 - , CSC - IT Center for Science Ltd.
MAIN: Webpage http://www.csc.fi/elmer, Email elmeradm@csc.fi
MAIN: Version: 8.1 (Rev: 549ce2a, Compiled: 2016-01-15)
MAIN: HYPRE library linked in.
MAIN: Trilinos library linked in.
MAIN: MUMPS library linked in.
MAIN: =============================================================
MAIN:
MAIN:
MAIN: -------------------------------------
MAIN: Reading Model: unitstest.sif
Loading user function library: [StressSolve]...[StressSolver_Init0]
Loading user function library: [SaveData]...[SaveScalars_Init0]
LoadMesh: Base mesh name: .//unitstest
LoadMesh: Elapsed time (CPU,REAL): 0.1080 0.1110 (s)
MAIN: -------------------------------------
AddVtuOutputSolverHack: Adding ResultOutputSolver to write VTU output in file: unitstest
Loading user function library: [StressSolve]...[StressSolver_Init]
Loading user function library: [StressSolve]...[StressSolver]
OptimizeBandwidth: ---------------------------------------------------------
OptimizeBandwidth: Computing matrix structure for: linear elasticity...done.
OptimizeBandwidth: Half bandwidth without optimization: 1414
OptimizeBandwidth:
OptimizeBandwidth: Bandwidth Optimization ...done.
OptimizeBandwidth: Half bandwidth after optimization: 133
OptimizeBandwidth: ---------------------------------------------------------
Loading user function library: [SaveData]...[SaveScalars_Init]
Loading user function library: [SaveData]...[SaveScalars]
Loading user function library: [ResultOutputSolve]...[ResultOutputSolver_Init]
Loading user function library: [ResultOutputSolve]...[ResultOutputSolver]
MAIN:
MAIN: -------------------------------------
MAIN: Steady state iteration: 1
MAIN: -------------------------------------
MAIN:
SingleSolver: Attempting to call solver
SingleSolver: Solver Equation string is: linear elasticity
StressSolve:
StressSolve: --------------------------------------------------
StressSolve: Solving displacements from linear elasticity model
StressSolve: --------------------------------------------------
StressSolve: Starting assembly...
StressSolve: Assembly:
: .Bulk assembly done
DefUtils::DefaultDirichletBCs: Setting Dirichlet boundary conditions
DefUtils::DefaultDirichletBCs: Dirichlet boundary conditions set
StressSolve: Set boundaries done
Program received signal 11 (SIGSEGV): Segmentation fault.
--------------------------------------------------------------------------
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process. Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption. The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.
The process that invoked fork was:
Local host: login (PID 13019)
MPI_COMM_WORLD rank: 0
If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--------------------------------------------------------------------------
What is also concerning is that MPI warning about the fork, but I can disable that using a command line MCA argument (even though it gives me heart burn to do so.) But, to test if it's an issue with Elmer, I set the direct solver to ufmpack, and it then works fine-- which leads me to believe there is some issue with the way Elmer is interfacing with mumps that is causing it to go off in to the weeds. Has Elmer been tested with the latest mumps version of 5.0.1, or is it only tested with 4.10 and below?
It's not spitting out a stack trace even when I make elmer using CMAKE_BUILD_TYPE=Debug so I can't be sure where it's causing the error. Has anybody else ran in to this?
Here's the pertinent lines from my .sif file, and I am using gmsh to create the mesh that I feed in to ElmerGrid.
Code: Select all
Solver 1
Equation = Linear elasticity
Procedure = "StressSolve" "StressSolver"
Variable = -dofs 3 Displacement
Exec Solver = Always
Stabilize = True
Bubbles = False
Lumped Mass Matrix = False
Optimize Bandwidth = True
Steady State Convergence Tolerance = 1.0e-5
Nonlinear System Convergence Tolerance = 1.0e-7
Nonlinear System Max Iterations = 1
Nonlinear System Newton After Iterations = 3
Nonlinear System Newton After Tolerance = 1.0e-3
Nonlinear System Relaxation Factor = 1
Linear System Solver = Direct
Linear System Direct Method = MUMPS
End
If anybody has any suggestions or things to try, I'm all ears. Thanks. In the mean time I'm going to try to add some debug statements to the fortran to see where it's dying, and to try using (if I can compile it) an older version of mumps. I'll let you know if anything changes.
Nick