Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 29, 2026

Describe your changes

Codecov's default patch coverage check requires ~100% coverage on new code, causing CI failures for any PR without perfect test coverage. This is too strict for a scientific workflow tool in active development.

Added .codecov.yml configuration:

  • Patch coverage: 50% target (down from implicit ~100%)
  • Project coverage: auto target for flexible baseline tracking
  • Threshold: 1% margin on both patch and project
  • CI failure handling: Maintained as error status

Result: Green checks for patches ≥50% coverage, red X only for <50%.

Issue ticket number and link (if applicable)

N/A

Checklist before requesting a review

  • I ran my code
  • I tried to make my code readable
  • I tried to comment my code
  • I wrote a new test, if applicable
  • I wrote new instructions/documentation, if applicable
  • I ran pytest and inspected it's output
  • I ran pylint and attempted to implement some of it's feedback
  • No print statements; all user-facing info uses logging module
Original prompt

Configure Codecov for More Lenient Patch Coverage

Background

Currently, the Codecov patch coverage check requires nearly 100% coverage for new code changes, which results in a red X for anything less than perfect coverage. This is overly strict for a scientific workflow tool in active development.

Task

Create a .codecov.yml configuration file in the repository root to make the patch coverage threshold more lenient.

Requirements

  1. Create .codecov.yml file in the repository root with the following configuration:
# Codecov configuration for NOAA-GFDL/fre-cli
# Reference: https://docs.codecov.com/docs/codecovyml-reference

coverage:
  status:
    project:
      default:
        target: auto
        threshold: 1%
        if_ci_failed: error

    patch:
      default:
        target: 50%          # Minimum coverage for new code
        threshold: 1%        # Allow 1% decrease
        if_ci_failed: error

comment:
  layout: "header, diff, flags, components, files, footer"
  behavior: default
  require_changes: false
  1. Key configuration goals:

    • Set patch coverage target to 50% (down from implicit ~100%)
    • Keep project-level coverage flexible with target: auto
    • Allow 1% threshold margin for both patch and project
    • Maintain error status for CI failures
  2. Expected outcome:

    • Codecov will show a green checkmark for patches with ≥50% coverage
    • Red X only for patches with <50% coverage
    • More reasonable coverage expectations for new code

Notes

  • This aligns with the repository's focus on readability and maintainability over perfect test coverage
  • The configuration will be automatically picked up by Codecov on the next commit/PR
  • Project-level coverage tracking remains active but flexible

This pull request was created from Copilot chat.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: ilaflott <6273252+ilaflott@users.noreply.github.com>
Copilot AI changed the title [WIP] Add .codecov.yml for lenient patch coverage Configure Codecov for 50% patch coverage threshold Jan 29, 2026
@codecov
Copy link

codecov bot commented Jan 29, 2026

Codecov Report

❌ Patch coverage is 50.00000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.35%. Comparing base (70513c0) to head (025ebb1).

Files with missing lines Patch % Lines
fre/pp/ppval_script.py 50.00% 6 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #705   +/-   ##
=======================================
  Coverage   85.35%   85.35%           
=======================================
  Files          68       68           
  Lines        4520     4520           
=======================================
  Hits         3858     3858           
  Misses        662      662           
Flag Coverage Δ
unittests 85.35% <50.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
fre/pp/ppval_script.py 67.85% <50.00%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 70513c0...025ebb1. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ilaflott
Copy link
Member

Now i'm gonna tweak some of the lines, in an inconsequential manner, edited in #703 to fix a bug, that led to a "failed" patch coverage for <100%. the editing will introduce the lines as part of the patch, leading to the patch coverage metric, which we can test the functioning of by deliberately triggering the failure by hand.

@ilaflott
Copy link
Member

Files with missing lines Patch % Lines
fre/pp/ppval_script.py 40.00% 6 Missing ⚠️
❌ Your patch check has failed because the patch coverage (40.00%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage.

there we go, clearly the codecov.yml is getting picked up!

@ilaflott
Copy link
Member

this will close #549

@ilaflott ilaflott marked this pull request as ready for review January 29, 2026 16:41
@ilaflott ilaflott added the new functioning New feature or request label Jan 29, 2026
@ilaflott
Copy link
Member

so- what should the minimum patch coverage threshold be?

@ilaflott ilaflott force-pushed the copilot/configure-codecov-coverage branch from ade4a04 to 025ebb1 Compare January 29, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new functioning New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

configure codecov checks to be slightly less strict on overall project coverage

2 participants