MHD Hyperbolic Divergence Cleaning

[ChrisZYJ]

Description

Implements the correct hyperbolic divergence cleaning for 2D/3D MHD in place of the current Powell's method. It is fully working and validated (with results below).

It is based on an outdated MFC branch, so it needs to be rebased, cleaned up, and made GPU-ready. @danieljvickers might be interested in collaborating on this; otherwise, I will get this ready as time permits.

Formulation

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Something else

Scope

• This PR comprises a set of related changes with a common goal

How Has This Been Tested?

All the test cases are temporarily placed in the case_new folder.

• Gaussian divergence pulse

• Slanted Shock Tube

• Orszag-Tang Vortex

Orszag-Tang result at final time; comparison across WENO variants. Top row is p. Bottom row is ∇·B. The last column shows the last saved time step before crashing, for a simulation without hyperbolic cleaning. 3Z is a typo; it should be 5Z.

• 2D Rotor Test

Checklist

• I have added comments for the new code
• I added Doxygen docstrings to the new code
• I have made corresponding changes to the documentation (docs/)
• I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
• I have added example cases in examples/ that demonstrate my new feature performing as expected.
  They run to completion and demonstrate "interesting physics"
• I ran ./mfc.sh format before committing my code
• New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
• This PR does not introduce any repeated code (it follows the DRY principle)
• I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in src/simulation)

To make sure the code is performing as expected on GPU devices, I have:

• Checked that the code compiles using NVHPC compilers
• Checked that the code compiles using CRAY compilers
• Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
• Ran the code on MI200+ GPUs and ensure the new features performed as expected (the GPU results match the CPU results)
• Enclosed the new feature via nvtx ranges so that they can be identified in profiles
• Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR
• Ran a Rocprof Systems profile using ./mfc.sh run XXXX --gpu -t simulation --rsys --hip-trace, and have attached the output file and plain text results to this PR.
• Ran my code using various numbers of different GPUs (1, 2, and 8, for example) in parallel and made sure that the results scale similarly to what happens if you run without the new code/feature

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sbryngelson]

beautiful! @ChrisZYJ
minus the old branching!

Maybe something we can look at...

[danieljvickers]

Yeah, this the the hyperbolic solver I told you I was going to add. The one I put in this week is an outdated elliptic solver.

I was hoping to update this one as I have time and redo our 2D convergence test that I added previously.

[danieljvickers]

Hey, @ChrisZYJ, I think I almost have this merged with master. I wanted to ask about this variable rkck_adap_dt that you add to the code. From what I can see, it was never explicitly declared as a global nor is it used anywhere in the code. Is this something that you intend to keep around? If so, can you point me to where it is suppose to be getting used?

This appear to work fine if I just remove it, but I wanted to see if there is a purpose to it sticking around

[ChrisZYJ]

Hi @danieljvickers , thanks for the help with the PR!!

rkck_adap_dt is unrelated to this commit and has been present since at least v5.0.0. I can clean it up for the merge if you’d like.

[danieljvickers]

I don't mind finishing it up. It looks like it's just an issue casting double to complex. You've really done most the heavy lifting already.

[danieljvickers]

I don't mind finishing it up. It looks like it's just an issue casting double to complex. You've really done most the heavy lifting already.

[ChrisZYJ]

Thanks! Please let me know if you need me to work on anything.

[danieljvickers]

MacOS failures are due to issues with updated gcc in the latest brew update. I implemented a fix on #1111, which currently is passing all tests. Once that gets merged, we can pull that in and resolve the failure.

[codecov]

Codecov Report

❌ Patch coverage is 29.03226% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.00%. Comparing base (6264d3f) to head (c642669).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
src/simulation/m_riemann_solvers.fpp	33.33%	8 Missing and 4 partials ⚠️
src/simulation/m_time_steppers.fpp	0.00%	7 Missing and 2 partials ⚠️
src/post_process/m_start_up.fpp	0.00%	4 Missing and 1 partial ⚠️
src/pre_process/m_start_up.fpp	0.00%	4 Missing and 1 partial ⚠️
src/post_process/m_global_parameters.fpp	25.00%	2 Missing and 1 partial ⚠️
src/pre_process/m_global_parameters.fpp	25.00%	2 Missing and 1 partial ⚠️
src/pre_process/m_initial_condition.fpp	0.00%	2 Missing and 1 partial ⚠️
src/simulation/m_rhs.fpp	33.33%	2 Missing ⚠️
src/common/m_variables_conversion.fpp	0.00%	1 Missing ⚠️
src/post_process/m_data_output.fpp	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1086      +/-   ##
==========================================
- Coverage   44.07%   44.00%   -0.08%     
==========================================
  Files          71       71              
  Lines       20369    20427      +58     
  Branches     1986     2001      +15     
==========================================
+ Hits         8978     8988      +10     
- Misses      10246    10281      +35     
- Partials     1145     1158      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.