Skip to content

Fixing flaky tests#70

Merged
IAlibay merged 45 commits intomainfrom
closing_netcdfs_properly
Feb 5, 2026
Merged

Fixing flaky tests#70
IAlibay merged 45 commits intomainfrom
closing_netcdfs_properly

Conversation

@hannahbaumann
Copy link
Contributor

@hannahbaumann hannahbaumann commented Jan 19, 2026

Fixes #43 and #67

This PR:

  • Closes netcdfs both in the code and in the test
  • Adds pooch caching back in
  • Updates the test; the new simulation_skipped dataset is from CDK2, which has two chains. Since the multichain fix is not in yet, this test currently has large RMSDs
  • Removes flaky argument in tests
  • Uses the skipped test dataset in more tests for not having to use the bigger one everywhere.

@codecov
Copy link

codecov bot commented Jan 19, 2026

Codecov Report

❌ Patch coverage is 97.43590% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.94%. Comparing base (5260e3b) to head (f42d063).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
src/openfe_analysis/rmsd.py 97.14% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #70      +/-   ##
==========================================
- Coverage   88.16%   87.94%   -0.23%     
==========================================
  Files           7        7              
  Lines         338      340       +2     
==========================================
+ Hits          298      299       +1     
- Misses         40       41       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@atravitz
Copy link
Contributor

@hannahbaumann you'll want to build pooch from main (like we are here in openfe) if you're seeing issues with fetching data.

Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of things, still looking into that positions thing.



def test_universe_creation(simulation_nc, hybrid_system_pdb):
with pytest.warns(UserWarning, match="This is an older NetCDF file that"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still have a test somewhere to warn for the old netcdf file types?

If not, can we mock one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old one was the one with >500MB, I added a new test test_determine_position_indices_warns_for_old_nc that mocks this and gives a warning.

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>
@hannahbaumann
Copy link
Contributor Author

pre-commit.ci autofix

@hannahbaumann
Copy link
Contributor Author

@IAlibay : After I removed the different u.trajectory.close(), the tests were flaky again with workers crashing, sometimes for macos, sometimes for ubuntu. I added the closing back in and now tests are stable again. I haven't seen the crashes locally.

@IAlibay
Copy link
Member

IAlibay commented Feb 3, 2026

@IAlibay : After I removed the different u.trajectory.close(), the tests were flaky again with workers crashing, sometimes for macos, sometimes for ubuntu. I added the closing back in and now tests are stable again. I haven't seen the crashes locally.

That means things aren't being deleted properly.. I'm not sure we can fix it in this PR, can you open an issue and we can revisit later?

@hannahbaumann
Copy link
Contributor Author

@IAlibay , I opened an issue here #84 !

Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks!

just the one thing that needs to be done, will merge it directlt

@IAlibay IAlibay merged commit 05c2162 into main Feb 5, 2026
7 checks passed
@IAlibay IAlibay deleted the closing_netcdfs_properly branch February 5, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

improve file clean up Tests are flaky - unclosed NetCDF datasets?

3 participants