CI Runnerscirunners.org
Guide

How to choose a CI runner: a buyer’s checklist

Answer in brief

Choose in this order. First, the CI system you use, which removes most options. Second, the platforms your jobs need: Linux x64, arm64, macOS, Windows, GPU. Third, where runners may run: a vendor’s cloud, your cloud account or your hardware. Fourth, isolation: whether every job needs a fresh VM. Only then compare caching, price and limits, and finish by running your slowest real workflow on two or three candidates.

By the CI Runners editors · Updated

1. Start from your CI system

A runner must speak your CI system’s protocol. Most third-party runner products are built for GitHub Actions. GitLab CI is served by GitLab’s own runners, the open-source GitLab Runner and a smaller set of vendors. Buildkite and Azure Pipelines have hosted agents and self-hosted agents. The finder filters on this first.

2. List the platforms you need

Write down every operating system and architecture in your workflows, and which ones carry most of the minutes. Linux x64 is offered by nearly everyone. arm64, macOS, Windows and GPU runners each cut the list. It is common to use two providers: one for Linux and a specialist for macOS.

3. Decide where runners may run

  • A vendor’s infrastructure is the least work. Your code runs on someone else’s machines, so review isolation and compliance.
  • Your own cloud account keeps data in your boundary and gives private network access. You own quotas, networking and the cloud bill.
  • Your own hardware or Kubernetes with open-source software gives the most control for the most effort.

Security or procurement policy often settles this before engineering preference does. Ask early.

4. Set the isolation bar

Ask what one job could do to the next. Public repositories, pull requests from forks and shared hardware call for a fresh VM or microVM per job. Private repositories with trusted contributors can accept a clean container or pod per job. Avoid long-lived runners that keep state between jobs unless every workflow that can reach them is trusted. See the security guide.

5. Compare what is left

QuestionWhy it matters
What CPU and how many cores?Single-thread speed drives compiles and tests; cores help only parallel work.
Where is the cache, and is there Docker layer caching?Cache restore time is often the largest fixed cost in a job.
What is the per-minute price for the size I use, and how is it rounded?Per-second versus per-minute rounding changes the bill for short jobs.
What are the concurrency and job-duration limits?Large matrices and long builds hit limits first.
Can I get static IPs or private networking?Needed to reach allow-listed or internal services.
Can I SSH into a failing job?Shortens debugging of failures that only happen in CI.
Which compliance reports exist, and where do jobs run?Required by many security reviews.
How does it behave when GitHub or the provider has an incident?A second vendor is a second dependency in your delivery path.

6. Run a fair trial

  • Pick your slowest and your most frequent workflow, not a toy project.
  • Run each several times on each candidate, at the same core count, with warm and cold caches.
  • Record queue time, run time and cost per run separately.
  • Check that the image has the tools your workflows assume.
  • Keep the original runs-on label one edit away so you can fall back.

Vendor benchmarks are vendor claims on vendor-chosen workloads. Your own numbers are the ones that count.

Frequently asked questions

What should I look for in a CI runner?
Support for your CI system and platforms, a deployment model your security policy allows, a fresh machine per job, fast caching, a per-minute price that suits your job mix, and limits that fit your peak concurrency. Then test with your own workflows.
Can I use more than one runner provider?
Yes. Each job chooses its runner through its label, so a workflow can run Linux jobs on one provider and macOS jobs on another, or keep some jobs on the platform’s own runners.

Sources

  1. docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax
  2. docs.github.com/en/actions/reference/security/secure-use

Check before you rely on it. Products change quickly, and this page reflects public documentation as of November 22, 2026. Confirm anything that matters in the provider’s own documentation.

Cite this page

CI Runners. (2026). How to choose a CI runner: a buyer’s checklist. cirunners.org. Last reviewed November 22, 2026. https://cirunners.org/guides/how-to-choose-a-ci-runner/