Hosted Runners With Azure VNET Integration Stopped Working

[gburkecw]

Select Topic Area

Bug

Body

On Friday 6/28/24, after the Actions and Pull Requests outages were resolved, our hosted runners with Azure VNET integration were working fine.

Yesterday I noticed that they had stopped working.

When the job is queued, it says:
Job is waiting for a runner from '[runner label]' to come online.

It just waits forever, I left it overnight and the job never started.

Is anyone else experiencing this outage?

[gburkecw]

I ended up having to delete the hosted agent, delete the network settings, delete the resources from Azure, then re-create everything and then it worked again.

I'll report back here if it happens again...

[grandmas-favorite]

Thanks for coming back to share your solution @gburkecw! This will surely help others that experience the same issue.

[Jamonygr]

For anyone else who hits this later: that message usually means GitHub cannot get an eligible runner online for that runner group/network setup. It is normally a provisioning/routing issue, not something inside the build.

The fix you found, recreating the runner/network settings/Azure resources, makes sense as a last resort if the GitHub.Network/networkSettings resource or runner group mapping got into a bad state.

Before deleting everything, I would still do a quick sanity check on the moving parts. Check GitHub Status for any Actions incident, confirm the Azure networkSettings resource is in Succeeded, make sure the subnet is usable and has free IPs, and verify that the runner group is still authorized for the repo. I would also double-check the workflow is using the exact group and label syntax expected by that runner group.

If it happens again, grab the runner group name, network settings resource ID, subnet region, and queued job ID before recreating anything. Support has a much better chance of finding the backend issue with those details.