-
Notifications
You must be signed in to change notification settings - Fork 28
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge remote-tracking branch 'upstream/main' into release/v0.1
- Loading branch information
Showing
1,929 changed files
with
72,520 additions
and
155,875 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| name: Integration Tests | ||
| on: | ||
| workflow_dispatch: {} | ||
| schedule: | ||
| - cron: "0 0 * * *" | ||
|
|
||
| jobs: | ||
| integration-tests: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v3 | ||
|
|
||
| - name: Setup Golang | ||
| uses: actions/setup-go@v3 | ||
| with: | ||
| go-version-file: go.mod | ||
|
|
||
| - name: Setup LXD | ||
| uses: canonical/[email protected] | ||
|
|
||
| - name: Build GARM | ||
| run: make build | ||
|
|
||
| - name: Set up ngrok | ||
| id: ngrok | ||
| uses: gabriel-samfira/[email protected] | ||
| with: | ||
| ngrok_authtoken: ${{ secrets.NGROK_AUTH_TOKEN }} | ||
| port: 9997 | ||
| tunnel_type: http | ||
|
|
||
| - name: Setup GARM | ||
| run: ./test/integration/scripts/setup-garm.sh | ||
| env: | ||
| GH_OAUTH_TOKEN: ${{ secrets.GH_OAUTH_TOKEN }} | ||
| CREDENTIALS_NAME: test-garm-creds | ||
| GARM_BASE_URL: ${{ steps.ngrok.outputs.tunnel-url }} | ||
|
|
||
| - name: Generate secrets | ||
| run: | | ||
| sudo apt-get -qq update && sudo apt-get -qq install -y apg | ||
| GARM_PASSWORD=$(apg -n1 -m32) | ||
| REPO_WEBHOOK_SECRET=$(apg -n1 -m32) | ||
| ORG_WEBHOOK_SECRET=$(apg -n1 -m32) | ||
| echo "::add-mask::$GARM_PASSWORD" | ||
| echo "::add-mask::$REPO_WEBHOOK_SECRET" | ||
| echo "::add-mask::$ORG_WEBHOOK_SECRET" | ||
| echo "GARM_PASSWORD=$GARM_PASSWORD" >> $GITHUB_ENV | ||
| echo "REPO_WEBHOOK_SECRET=$REPO_WEBHOOK_SECRET" >> $GITHUB_ENV | ||
| echo "ORG_WEBHOOK_SECRET=$ORG_WEBHOOK_SECRET" >> $GITHUB_ENV | ||
| - name: Create logs directory | ||
| if: always() | ||
| run: sudo mkdir -p /artifacts-logs && sudo chmod 777 /artifacts-logs | ||
|
|
||
| - name: Run integration tests | ||
| run: | | ||
| set -o pipefail | ||
| set -o errexit | ||
| go run ./test/integration/main.go 2>&1 | tee /artifacts-logs/e2e.log | ||
| env: | ||
| GARM_BASE_URL: ${{ steps.ngrok.outputs.tunnel-url }} | ||
| ORG_NAME: gsamfira | ||
| REPO_NAME: garm-testing | ||
| CREDENTIALS_NAME: test-garm-creds | ||
| WORKFLOW_FILE_NAME: test.yml | ||
| GH_TOKEN: ${{ secrets.GH_OAUTH_TOKEN }} | ||
|
|
||
| - name: Show GARM logs | ||
| if: always() | ||
| run: | | ||
| sudo systemctl status garm | ||
| sudo journalctl -u garm --no-pager 2>&1 | tee /artifacts-logs/garm.log | ||
| - name: Upload GARM and e2e logs | ||
| if: always() | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: garm-logs | ||
| path: /artifacts-logs | ||
|
|
||
| - name: Cleanup orphan GARM resources via GitHub API | ||
| if: always() | ||
| run: | | ||
| set -o pipefail | ||
| set -o errexit | ||
| sudo systemctl stop garm | ||
| go run ./test/integration/gh_cleanup/main.go | ||
| env: | ||
| GARM_BASE_URL: ${{ steps.ngrok.outputs.tunnel-url }} | ||
| ORG_NAME: gsamfira | ||
| REPO_NAME: garm-testing | ||
| GH_TOKEN: ${{ secrets.GH_OAUTH_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,11 +4,13 @@ | |
|
|
||
| Welcome to GARM! | ||
|
|
||
| Garm enables you to create and automatically maintain pools of [self-hosted GitHub runners](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners), with autoscaling that can be used inside your github workflow runs. | ||
| GARM enables you to create and automatically maintain pools of [self-hosted GitHub runners](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners), with auto-scaling that can be used inside your github workflow runs. | ||
|
|
||
| The goal of ```GARM``` is to be simple to set up, simple to configure and simple to use. It is a single binary that can run on any GNU/Linux machine without any other requirements other than the providers it creates the runners in. It is intended to be easy to deploy in any environment and can create runners in any system you can write a provider for. There is no complicated setup process and no extremely complex concepts to understand. Once set up, it's meant to stay out of your way. | ||
|
|
||
| Garm supports creating pools on either GitHub itself or on your own deployment of [GitHub Enterprise Server](https://docs.github.com/en/[email protected]/admin/overview/about-github-enterprise-server). For instructions on how to use ```GARM``` with GHE, see the [credentials](/doc/github_credentials.md) section of the documentation. | ||
| GARM supports creating pools on either GitHub itself or on your own deployment of [GitHub Enterprise Server](https://docs.github.com/en/[email protected]/admin/overview/about-github-enterprise-server). For instructions on how to use ```GARM``` with GHE, see the [credentials](/doc/github_credentials.md) section of the documentation. | ||
|
|
||
| Through the use of providers, `GARM` can create runners in a variety of environments using the same `GARM` instance. Want to create pools of runners in your OpenStack cloud, your Azure cloud and your Kubernetes cluster? No problem! Just install the appropriate providers, configure them in `GARM` and you're good to go. Create zero-runner pools for instances with high costs (large VMs, GPU enabled instances, etc) and have them spin up on demand, or create large pools of k8s backed runners that can be used for your CI/CD pipelines at a moment's notice. You can mix them up and create pools in any combination of providers or resource allocations you want. | ||
|
|
||
| ## Join us on slack | ||
|
|
||
|
|
@@ -18,14 +20,27 @@ Whether you're running into issues or just want to drop by and say "hi", feel fr | |
|
|
||
| ## Installing | ||
|
|
||
| ### On virtual or physical machines | ||
|
|
||
| Check out the [quickstart](/doc/quickstart.md) document for instructions on how to install ```GARM```. If you'd like to build from source, check out the [building from source](/doc/building_from_source.md) document. | ||
|
|
||
| ### On Kubernetes | ||
|
|
||
| Thanks to the efforts of the amazing folks at @mercedes-benz, GARM can now be integrated into k8s via their operator. Check out the [GARM operator](https://github.com/mercedes-benz/garm-operator/) for more details. | ||
|
|
||
| ## Supported providers | ||
|
|
||
| GARM uses providers to create runners in a particular IaaS. The providers are external executables that GARM calls into to create runners. Before you can create runners, you'll need to install at least one provider. | ||
|
|
||
| ## Installing external providers | ||
|
|
||
| External providers are binaries that GARM calls into to create runners in a particular IaaS. There are currently two external providers available: | ||
| External providers are binaries that GARM calls into to create runners in a particular IaaS. There are several external providers available: | ||
|
|
||
| * [OpenStack](https://github.com/cloudbase/garm-provider-openstack) | ||
| * [Azure](https://github.com/cloudbase/garm-provider-azure) | ||
| * [Kubernetes](https://github.com/mercedes-benz/garm-provider-k8s) - Thanks to the amazing folks at @mercedes-benz for sharing their awesome provider! | ||
| * [LXD](https://github.com/cloudbase/garm-provider-lxd) | ||
| * [Incus](https://github.com/cloudbase/garm-provider-incus) | ||
|
|
||
| Follow the instructions in the README of each provider to install them. | ||
|
|
||
|
|
@@ -47,8 +62,4 @@ If you would like to optimize the startup time of new instance, take a look at t | |
|
|
||
| ## Write your own provider | ||
|
|
||
| The providers are interfaces between ```GARM``` and a particular IaaS in which we spin up GitHub Runners. These providers can be either **native** or **external**. The **native** providers are written in ```Go```, and must implement [the interface defined here](https://github.com/cloudbase/garm/blob/main/runner/common/provider.go#L22-L39). **External** providers can be written in any language, as they are in the form of an external executable that ```GARM``` calls into. | ||
|
|
||
| There is currently one **native** provider for [LXD](https://linuxcontainers.org/lxd/) and two **external** providers for [Openstack and Azure](/contrib/providers.d/). | ||
|
|
||
| If you want to write your own provider, you can choose to write a native one, or implement an **external** one. The easiest one to write is probably an **external** provider. Please see the [Writing an external provider](/doc/external_provider.md) document for details. Also, feel free to inspect the two available external providers in this repository. | ||
| The providers are interfaces between ```GARM``` and a particular IaaS in which we spin up GitHub Runners. **External** providers can be written in any language, as they are in the form of an external executable that ```GARM``` calls into. Please see the [Writing an external provider](/doc/external_provider.md) document for details. Also, feel free to inspect the two available sample external providers in this repository. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.