Trouble with Mesh Interpolation

Numerical methods and mathematical models of Elmer
Post Reply
edge025
Posts: 28
Joined: 27 Apr 2019, 08:48
Antispam: Yes
Location: NJ, USA
Contact:

Trouble with Mesh Interpolation

Post by edge025 »

Hello,

I have a model that contains 6 water pipes and air, along with a host of other non-CFD components. I have run Naviers-Stokes solvers on each of the CFD components separately... when I say separately I mean COMPLETELY separately (even meshes that JUST contain the CFD components). I have saved the results in .dat files for each. So I have

Water-componet1 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet2 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet3 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet4 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet5 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet6 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Air-component (partitioned mesh and result files) - I ran in parallel on 64 partitions

Ok. Then I am trying to "merge" or interpolate the results above into the big model with the non-CFD components (global mesh) as well (partitioned into 64 as well) using a Body Force like this

Body Force 5
Name = "NS Water Variable Imports"
Velocity 1 = Equals Velocity-water-vol8 1
Velocity 2 = Equals Velocity-water-vol8 2
Velocity 3 = Equals Velocity-water-vol8 3
Pressure = Equals Pressure-water-vol8
End

However, when it comes time to do anything I get errors after BuildQuadrantTree Ready. Is it because the elements in the global mesh and the component mesh are not in the same partition (they were all partitioned individually using ElmerGrid)

Regards,
Joe
Attachments
master.sif
(3.79 KiB) Downloaded 257 times
master-Body Forces.sif
(1.65 KiB) Downloaded 245 times
master-watervars-import.sif
(7.6 KiB) Downloaded 234 times
edge025
Posts: 28
Joined: 27 Apr 2019, 08:48
Antispam: Yes
Location: NJ, USA
Contact:

Re: Trouble with Mesh Interpolation

Post by edge025 »

more attachments
Attachments
master-airvars-import.sif
(1.2 KiB) Downloaded 232 times
master-BCs.sif
(263 Bytes) Downloaded 237 times
master-Bodies.sif
(9.76 KiB) Downloaded 243 times
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Trouble with Mesh Interpolation

Post by raback »

Hi Joe,

I really respect your effort to make such a complex simulation. It may be stretching the limits of the code...

If I understand correctly your pipes are not joined so you can save some resources by computing them separately when you peform a hierarchical thermal simulation (no feedback). I have no objections to that. So the challenge here is how to gather all the 7 Navier-Stokes results to the global mesh.

Basically when you request a field that is not defined in the mesh the code find the mesh that owns these results and then does the interpolation. Octree data structure is used since it allows for n*LOG(n) scaling. In principle the interpolation is also implemented in parallel even though it is much more labirious then.

Personally I've not been as ambitious as you. I have only interpolated between two meshes. I could see several reasons why this could fail
  • There is maybe some structures that are not allocated/deallocated when doing new interpolation. Does this fail for the 1st interpolation already.
  • It would be prettier if all the 7 NS meshes would have the standard field name. However, you correctly realized that the interpolation would be then done only for the 1st piece. For this kind of simulation it would be good to allow interpolation over all meshes.
  • If you initialize the solver sections with the NS equation you will use excess memory since partitions will allocate some structures. You could perhaps use just a dummy AllocateSolver() module to reserve the space and do the restart.
Sorry, this is probably not very helpfull. The case is just quite ambitious for a 10-minute analysis. An output log would be helpful.

-Peter
edge025
Posts: 28
Joined: 27 Apr 2019, 08:48
Antispam: Yes
Location: NJ, USA
Contact:

Re: Trouble with Mesh Interpolation

Post by edge025 »

Thanks for your answer, Peter. The rational is that I am only interested in the end result of temperature. I wanted to be able to refine the temperature mesh and still be able to import in the naviers-stokes results (since their accuracy is not paramount, neither are they dependent on temperature) so I would not have to rerun the navier-stokes solvers (due to time).

I am still trying to get this to work. In my current tests, I have cut out the temperature solver and am only trying to import one single variable from a NV solution. I do not seem to get a MPI communication error anymore, but after "BuildQuandrantTree: Ready" there doesn't seem to be any further progress. I've let Elmersolver run for probably close to 30min now with no movement. Any guidance appreciated.

Would a Variable Export and then assigning variables using expressions in Body Forces section work any different?

Regards,
Joe
Attachments
master.sif
(3.91 KiB) Downloaded 250 times
master-Bodies.sif
(9.76 KiB) Downloaded 229 times
master-Body Forces.sif
(1.68 KiB) Downloaded 237 times
Last edited by edge025 on 08 Oct 2019, 07:28, edited 1 time in total.
edge025
Posts: 28
Joined: 27 Apr 2019, 08:48
Antispam: Yes
Location: NJ, USA
Contact:

Re: Trouble with Mesh Interpolation

Post by edge025 »

solver log
Attachments
solver.log
(108.34 KiB) Downloaded 225 times
edge025
Posts: 28
Joined: 27 Apr 2019, 08:48
Antispam: Yes
Location: NJ, USA
Contact:

Re: Trouble with Mesh Interpolation

Post by edge025 »

Weird. Even though the log file shows that all variables were reloaded or "restart" for all solvers, when I try to restart one of the solvers that has good navier-stokes data, the solver acts like it doesn't have the variables loaded (it does not pick up where it left off). Any ideas please how to fix this?
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Trouble with Mesh Interpolation

Post by raback »

Hi

Difficult to say. I would make a dummy solver that would do the interpolation. In principle you could do something like this

Code: Select all

  VarSum => VariableGet( Mesh % Variables,'Velocity 1')
  Var1 => VariableGet( Mesh % Variables,'Velocity-water-vol1')
  Var2 => VariableGet( Mesh % Variables,'Velocity-water-vol2')
  ...
  VarSum % Values = Var1 % Values + Var2 % Values  ! actually this probably must consider permutations 
The interpolation from mesh vol1 is done automatically to your current mesh when VariableGet is called. The interpolated values will be zero where not found. Hence you need to sum up the pieces to get the correct sum. This kind of small solver would also enable to have better control on where and when the interpolation takes place.

-Peter
edge025
Posts: 28
Joined: 27 Apr 2019, 08:48
Antispam: Yes
Location: NJ, USA
Contact:

Re: Trouble with Mesh Interpolation

Post by edge025 »

Thanks for your help, Peter.

Oy...I don't know how to write solvers yet even though it may be "simple." I will have to do this another way for now and then hopefully return to this concept on another project.

Regards,
Joe
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Trouble with Mesh Interpolation

Post by raback »

Hi Joe,

Here's a solver close to what you need:
https://github.com/ElmerCSC/elmerfem/bl ... Solver.F90

Then just elmerf90 -o MySolver.so MySolver.dll

The main difference is that this solver interpolates many variables from one nonconforming mesh whereas you should interpolate the same variable from many noncoforming meshes.

This solver is even unnecesssary complex as there are some checks built in.

-Peter
edge025
Posts: 28
Joined: 27 Apr 2019, 08:48
Antispam: Yes
Location: NJ, USA
Contact:

Re: Trouble with Mesh Interpolation

Post by edge025 »

Peter,

I want to thank you for the suggestion you gave me, it really spring-boarded me into solving this issue and coming up with a really nice solution (at least for me) that has worked very nicely and I think i will use very often.

I also wanted to provide that solution for others, because i have been helped so much, and I hope this will help others as well. The following is my own notes that I have wrote in a very tired state. i do not guarantee the accuracy or free from mistakes...... Basically the problem has been:

In Elmer, I have had some issues trying to interpolate variable data in 3D models from one mesh to another. I have had success importing data from a portion of a 3D mesh to a more complete mesh, e.g. importing CFD data such as cooling water flow through a pipe to the complete model mesh that contains the cooling water that I am going to run a thermal analysis on. I have had NO success importing data from a portion of a 3D mesh to a more refined mesh, at least when I am running in parallel (keep getting MPI buffer errors or something like that). To solve this problem, I and the Lord (those reading this may not agree, but I believe He does help), came up with another way of doing this import/interpolation using other capabilities of Elmer. Attached is an explanation and an outline of the steps to do this.

Regards,
Joe
Attachments
Procedure to interpolate data from mesh to mesh.pdf
(79.45 KiB) Downloaded 247 times
ScriptFiles.zip
(4.52 KiB) Downloaded 222 times
Post Reply