Hi,
I am doing coupled Elmer and OpenFOAM simulation using EOF-Library.
Both meshes have ~1M elements/cells. Simulation is running on 8+8 MPI processes. For every Elmer steady state computation (it takes 170sec) there are ~8 OpenFOAM timesteps (all combined ~30sec). Basically, Elmer is taking about 85% of computational time.
I am looking for a way to improve Elmer's steady state performance, something like disabling unnecessary tasks. Considering that the problem is changing very little between two consecutive computations, I would assume that Elmer takes result from previous steady state, then does a couple more iterations until converges up to tolerance, calculates fields (current, B, J. etc..).
Any ideas how to boost performance?
Case and log file attached.
Steady state performance of WhitneyAVHarmonicSolver
-
- Site Admin
- Posts: 4832
- Joined: 22 Aug 2009, 11:57
- Antispam: Yes
- Location: Espoo, Finland
- Contact:
Re: Steady state performance of WhitneyAVHarmonicSolver
Hi vencels
The assembly could be improved but most of that requires altering the code. Your material parameter is not constant so you really need to recompute the matrix every iteration. I would write F90 routines for the current densities. MATC really takes quite a lot of time. This should take easily some ~20% of the assembly time away.
Now for the linear system there is not that much that can be done. I trust you've tested different strategies. The matrix is really quite a beast and has a lot of connections. It is huge compared to the sparse matrices coming from similar mesh from the splitted schemes in FVM method.
What you could try is to have sloppier criteria at the start, say
Or you could say "Linear System Residual Mode = Logical True" and set the tolerance to ~1.0e-3 which is then an incremental increase in accuracy. If the latter works I would prefer that.
Also between OpenFOAM and Elmer you might require some relaxation. Also that could depend on iteration as above. It could be 1.0 the 1st iteration, and ~0.7 after that. Loosely coupled schemes often overshoot. This is of course very problem dependent...
-Peter
The assembly could be improved but most of that requires altering the code. Your material parameter is not constant so you really need to recompute the matrix every iteration. I would write F90 routines for the current densities. MATC really takes quite a lot of time. This should take easily some ~20% of the assembly time away.
Now for the linear system there is not that much that can be done. I trust you've tested different strategies. The matrix is really quite a beast and has a lot of connections. It is huge compared to the sparse matrices coming from similar mesh from the splitted schemes in FVM method.
What you could try is to have sloppier criteria at the start, say
Code: Select all
Linear System Convergence Tolerance = Variable "coupled iter"
Real
1.0 1.0e-3
2.0 1.0e-5
3.0 1.0e-7
4.0 1.0e-7
End
Also between OpenFOAM and Elmer you might require some relaxation. Also that could depend on iteration as above. It could be 1.0 the 1st iteration, and ~0.7 after that. Loosely coupled schemes often overshoot. This is of course very problem dependent...
-Peter