Allow required checks to pass/skip, not fail, when using path filtering

[gustavovalverde]

Select Topic Area

Product Feedback

Body

As a repository starts to grow and depends more and more on GitHub Actions for CI, it also becomes more necessary to make workflows required.

Not being able to skip a required action, if it was filtered by paths rules, creates a burden for developers/DevOps maintaining extra files in sync with the original workflow.

For example, any coding language with a CI pipeline will evaluate if any file having the specified language in it was changed, and if not, it will skip the workflow:

name: CI

on:
  pull_request:
    paths:
      - '**/*.py'

The offered workaround in the official documentation is to create another workflow, ignoring the path and including each step with a script to return the correct exit code: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/troubleshooting-required-status-checks#handling-skipped-but-required-checks

This is far from optimal, as it create N + 1 files for each required workflow, and it requires to create each step in the extra workflow. While keeping them in sync if any extra step is added in the original workflow.

This also doubles the checks in the PR UI, and does not allow to understand which workflows were run and which were not.

This workflow directory is an example of the burden created by this workaround https://github.com/ZcashFoundation/zebra/tree/main/.github/workflows and any PR will show like all workflows are run, but this are just the patch workflows which are running.

Hopefully we can have an option in the repository to specify if workflows can be skipped (by the paths filters) even if the workflow is required.

Note: This discussion refers to the same issue, but it was marked as answered with a different workaround which won't work on repositories like ours, as it creates extra steps, dependencies and complexity: https://github.com/orgs/community/discussions/26857#discussioncomment-3253667

[elyobo]

Another approach might be to allow similar path filtering on the required tests.

[MarcDufresne]

We're seeing the same issue in our monorepo. We have over 20 services and libraries, each with their own workflows, and a collection of reusable workflows (for tests, Docker builds, deployments).

Initially, we were using the native path filtering to only run the workflows of changed services, but as we discovered GitHub Actions doesn't support path filters when using required checks, as each required check will sit there and not be marked as skipped, thus blocking the PR from merging. Additionally, we are also using the merge queue, so required checks are absolutely necessary.

We found an alternative using dorny/paths-filter. However, since we need to run the check ourselves then we incur 1 minute of billable time per service and library. Currently this costs us 23 minutes of billable time, on each commit in a PR or our main branch, just to check if we should run the actual workflows or not. This is frankly unacceptable that we must pay for this very basic feature. A feature that was supported by other CI/CD tools we used, like Azure DevOps.

We've also tried to create a "master" workflow that would run a single path filter check, and execute each workflow service as a reusable workflow depending on the result of the path filter check. Thus costing us only 1 minute per commit instead of 23. However, we've now reached another GitHub Actions limit, where a workflow cannot reference more than 20 reusable workflows.

I really hope that GitHub fixes this very basic use case of not supporting path filters and required checks together.

[petermetz]

^^ The exact same issue here. Our mono-repo has 60+ packages, each with their own test suites.

I really hope that GitHub fixes this very basic use case of not supporting path filters and required checks together.

Plus one!

[frickerj]

also encountering this exact same problem - absurb that there isn't a solution for this

[masutaka]

I want to avoid using third-party GitHub Actions as much as possible, so I implemented something similar using actions/cache.
https://github.com/liam-hq/liam/blob/bbc65e6a42fe3b99375086ad33912c8473e797d5/.github/workflows/license.yml#L20-L42

I hope GitHub will support this.

[ManavDia]

Also ran into this issue today +1 for a feature/fix from Github to address this issue.

[antoniogamizdelgado]

Same problem, surprising this is not suopported. More discussions -> https://github.com/orgs/community/discussions/13690

[leavesster]

I just create a action pull-request-path-filter as a workaround. But it only works for pull request event.
For anyone who is facing the same issue, I hope it should be useful.

[Mugane]

Wow not a peep from GitHub. I'm beginning to think this entire community is a honeypot.

[schammah]

solutions works but are is very inefficient, lots of status checks running for nothing, lot of compute waste and time wasted.
any feature requests for github to support this natively anytime soon?

[Peetee06]

Would also love a native solution for this. I implemented @MarcDufresne's solution with the file filters action. It works fine but results in a lot of unnecessary compute and therefore money wasted. Not needing a required check to run would make it so much nicer for mono repos.

[nebuk89]

Hey! Sorry I missed this one as we are going through triaging the backlog of community bits :) I don't have a timeline for this one, but will add it to the list of papercuts we are working through to ensure it's considered as we try to burn these down over the next 12 months.

Thanks for your patience and engagement <3 please keep sharing with us 🙏

[AFMiziara]

12 months for GH is around 5 years :)

[JaKXz]

Can we consolidate https://github.com/orgs/community/discussions/13690 into this?

[chiptus]

any update on this?

[mangelajo]

Looking for an update too, thank you

[ruben-b-ispnext]

@nebuk89 Why are you ignoring every request for an update on this? You've successfully burned it down to almost 22 months and left the community in the dark.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[bencat-sixense]

Any news about it Github ? :)

[mangelajo]

ping

[JustinBis]

Would love an update on this. Being able to skip checks in CI is a critical feature to help save time and compute costs. The existing solutions are all quite hacky, and an official solution would be very appreciated.

[nr-lg]

@nebuk89 we're shifting our cloudbuild triggers over to github actions and so far the experience has been very positive! We are also coming up against this issue. Ideally we want to complete the migration by EOQ. Any chance of an update on prioritisation at least?

[CoreyCole]

GitHub Actions does bill by rounding up to the nearest minute for each job, not by the second. This means even if a job only runs for a few seconds (like 4-6 seconds), it will be billed as a full minute. I wonder what revenue GitHub will lose if them implement this? 🤔

[umm0n]

Basic functionality, honestly. Any ETA?

[rlmcneary2]

Another way to "workaround" this Github deficiency is to use NX for monorepos. It uses a dependency graph to keep track of what "modules" changed in a monorepo so its cli is smart enough to only run commands (test, build, lint, etc.) related to those changed modules (or modules that depend on a changed module).

[DanyloSafankov]

Hey @nebuk89, could you please update us if we should wait this feature in the next 6 month?

[alexTr3]

they want you to pay hidden cost for normal feature..

[shooka]

I've been following this discussion for over a year now, and I keep coming back to it because it's directly affecting our workflow decisions. I'm trying to make the case internally for moving from a multi-repo setup to a monorepo, but it's been difficult to justify when this kind of basic behavior around required checks and paths: filtering isn't supported. I know that other CI tools mark skipped steps/blocks successful which is also not entirely correct, but it's a valid workaround which doesn't put the maintainers in a situation where they are forced to come up with a custom solution for skipping a workflow.

I'm genuinely curious how this is handled internally at GitHub. From the outside, this feels like a very common monorepo use case.

[elyobo]

It's a common non-monorepo requirement as well, if you're interested in cost control or speed - running tests that you can determine are not required for a given set of changes is a waste of money and time.

[fhalde]

I'm genuinely curious how this is handled internally at GitHub

They use GitLab

[Magnuti]

I ran into this same issue in my monorepo and ended up using a "bucket" job pattern combined with https://github.com/dorny/paths-filter that works pretty well. The idea is to add one extra job that always runs, lists all your conditional jobs as dependencies, and only fails if any of them actually failed (not skipped). Then you set just this one job as the required status check instead of the individual ones.

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    permissions:
      contents: read                                                                         
      pull-requests: read
    outputs:                                                                                 
      website: ${{ steps.changes.outputs.website }}                                        
      backend: ${{ steps.changes.outputs.backend }}
    steps:                                                                                   
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v4                                                          
        id: changes                                                                        
        with:
          filters: |                                                                         
            website:
              - 'apps/website/**'                                                            
            backend:                                                                       
              - 'apps/backend/**'

  build-website:
    needs: detect-changes
    if: needs.detect-changes.outputs.website == 'true'                                       
    runs-on: ubuntu-latest
    steps:                                                                                   
      - run: echo "building website..."
      - 
  # Rest of your jobs...
 
  required-checks-passed:
    if: always()                                                                             
    needs:
      [                                                                                      
        build-website,                                                                     
        build-landing-page,
        # ... rest of your jobs
      ]
    runs-on: ubuntu-latest
    steps:                                                                                   
      - if: contains(needs.*.result, 'failure')
        run: exit 1                                                                          

if: always() makes sure it runs even when everything upstream was skipped, and contains(needs.*.result, 'failure') means a skipped job won't trigger a failure, only an actual failure will.

Its not ideal that this workaround is even necessary, but it gets the job done until GitHub ships a proper fix.

My workflow now looks like this.

Another cool thing about this is that we can now define our required workflows as code and not in the UI. We only need to set these two jobs are required, and then never look at the settings UI again.

[Cellule]

My biggest issue with this work around, is that it adds 2 jobs on all workflows that run for about 1-2seconds. Since Github action round up to a minute when pricing job duration that means you get charged 2min for all "skipped" workflows in the monorepo.
Being a monorepo also means a lot of contributors, a lot of Pull requests so that means A LOT of overcharge for something that should be built-in.
Maybe that's why Github doesn't fix this issue, it woulf mean money loss. But on the other hand, this is one of the main issue that our CI team is seriously considering alternatives to GHA

[Magnuti]

I agree on this, it is not ideal. But 2 extra minutes on each PR is acceptable in my case as it seems to solve this issue.

We are also having one big workflow on PRs, instead of many smaller, and using paths filtering to find out which jobs to run. So we get 2 minutes total and not 2 minutes per workflow.

[sfuerte]

Hopefully we can have an option in the repository to specify if workflows can be skipped (by the paths filters) even if the workflow is required.

Not related to the Workflows per se, but there is "Require conversation resolution before merging", which description says:

When enabled, all conversations on code must be resolved before a pull request can be merged into a branch that matches this rule.

Having something along the same lines with "Require ALL status checks to pass before merging" without selection of individual checks would solve the subject problem.
I.e. we'd still have the current "Require status checks to pass before merging" with its "Choose which status checks must pass..." logic and those ones would be marked as "Required" in the PR view.
And on top of that, the new "Require all status checks to pass" option would essentially mean, that if a workflow started, it must pass regardless of whether it's on the Required list or not.

Just 2c, but IMHO that would be easiest approach to the problem, both architectually (the minimal code change on GitHub side) and operationally (no complex configs on the end-user, i.e. our sides).