Skip to content

Commit c89a3c6

Browse files
vbemvbem
committed
Enhance runner management options in mr.bash
1 parent ccb8f59 commit c89a3c6

File tree

4 files changed

+26
-194
lines changed

4 files changed

+26
-194
lines changed

‎README.md

+3-108
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ Options:
5454
--token Runner registration token, takes precedence over MR_GITHUB_PAT
5555
--dotenv The lines to set in runner's '.env' files
5656
--count The number to add or del, optional, defaults to 1 for add and all for del
57+
--opts Extra options for 'config.sh', optional, such as '--no-default-labels'
5758
-h --help Show this help.
5859
```
5960

@@ -174,119 +175,13 @@ runs-on: [self-hosted, '${{ github.repository }}']
174175
As described in GitHub official document, there's an approach to [inject environment variables into runners process](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/using-a-proxy-server-with-self-hosted-runners#using-a-env-file-to-set-the-proxy-configuration) via the `.env` file before configuring or starting the self-hosted runners. This can be achieved via the `--dotenv` option, for example:
175176

176177
```bash
177-
./mr.bash add --org <ORG> --repo <REPO> --dotenv 'TZ=Asia/Shanghai' --dotenv 'PATH=\$PATH:/mybin'
178+
./mr.bash add --org <ORG> --repo <REPO> --dotenv 'TZ=Asia/Shanghai' --dotenv 'PATH=\$PATH:/mybin' --dotenv 'all_proxy=socks5h://localhost:1080'
178179
```
179180

180181
Then the following lines will be added to `.env` file located in self-hosted runner's directory before its configuring and starting:
181182

182183
```plain
183184
TZ=Asia/Shanghai
184185
PATH=$PATH:/mybin
185-
```
186-
187-
## Case Study - Deploy multi runners on single host which can not access GitHub directly
188-
189-
A multi-national corporation adopted GitHub as its centralized engineering efficiency platform. But in a country branch, according to some network blockade/bandwidth/QoS reasons, neither GitHub-hosted runners can access endpoints in this country stably, nor virtual machines in this country can access GitHub liberally.
190-
191-
In such a bad situation, we still need to setup reliable self-hosted runners in this country. What should we do? 🤣
192-
193-
A cost-conscious solution can be described as following architecture:
194-
195-
```plain
196-
Endpoints <-------- VM-Runners ----> Firewall ----> VM-Proxy ----> GitHub
197-
\ / | \
198-
-------------------------- | ----> Other endpoints
199-
Branch office network Remote Proxy
200-
```
201-
202-
A host *VM-Runners* is required for self-hosted runners, which is placed in this country and:
203-
204-
- Can access endpoints of this country branch
205-
- Can NOT access *GitHub* directly or stably
206-
207-
A tiny specification host *VM-Proxy* is required as ***Remote Proxy***, which is deployed in a place that:
208-
209-
- Can access *GitHub* directly and stably
210-
- Can be accessed by *VM-Runners* directly and stably
211-
212-
Meanwhile, **outbound traffics from *VM-Runners* MUST be routed by predefined rules**:
213-
214-
- [Requests to *GitHub* endpoints](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#communication-requirements) and non-local endpoints should be forward to the *Remote Proxy* on *VM-Proxy*
215-
- Requests to local endpoints should be handled directly
216-
217-
Let's implement this solution. 🧐
218-
219-
On *VM-Proxy*, we can setup a *Remote Proxy* that's not easy to be blocked by the firewall, such as [*SS*](https://github.com/shadowsocks), [*TJ*](https://github.com/trojan-gfw), [*XR*](https://github.com/XTLS), etc. These particular proxies have their own deployment and configuration methods. Please read their documents for more information. It's advised to set the outbound IP of *VM-Runners* as the only whitelist of the *Remote Proxy* port on *VM-Proxy* to avoid active detection from the firewall.
220-
221-
Before setup runners on *VM-Runners*, we need a [***Local Proxy***](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/using-a-proxy-server-with-self-hosted-runners) on *VM-Runners*.
222-
223-
Usually firstly we need to setup the client of selected *Remote Proxy* which exposes a SOCKS5 proxy on *VM-Runners* (this local SOCKS5 localhost port will unconditionally forward all traffics to *VM-Proxy*), and then setup a [*privoxy*](https://www.privoxy.org/) on top of previous local SOCKS5 for [domain based forwarding](https://www.privoxy.org/user-manual/config.html#SOCKS). These configurations are complex and prone to errors. Via [*Clash*](https://github.com/Dreamacro/clash), we can combine both client of *Remote Proxy* and domain based forwarding into only one *Local Proxy*. The example configuration file and startup script of *Clash* and given in this repo's [clash.example/](clash.example/) directory.
224-
225-
Assume the *Local Proxy* was launched as `socks5h://localhost:7890`, we can test it via following commands on *VM-Runners*:
226-
227-
```bash
228-
# Without *Local Proxy*, it will print outbound public IP of *VM-Runners*
229-
curl -s -4 icanhazip.com
230-
231-
# With *Local Proxy*, it will print outbound public IP of *VM-Proxy* !!!
232-
all_proxy=socks5h://localhost:7890 curl -s -4 icanhazip.com
233-
```
234-
235-
When *Local Proxy* is ready, we start self-hosted runners' setup on *VM-Runners*.
236-
237-
As *VM-Runners* Can NOT access *GitHub* directly or stably, use *Local Proxy* to clone this repository:
238-
239-
```bash
240-
all_proxy=socks5h://localhost:7890 git clone https://github.com/vbem/multi-runners
241-
cd multi-runners
242-
```
243-
244-
As self-hosted runners' tar package downloading and registration-token fetching also requires communication with GitHub, we also configure *Local Proxy* for this application:
245-
246-
```bash
247-
cat > .env <<- __
248-
MR_GITHUB_PAT='<paste-for-GitHub-PAT-here>'
249-
all_proxy='socks5h://localhost:7890'
250-
__
251-
```
252-
253-
To download the self-hosted runners' tar package from *GitHub*:
254-
255-
```bash
256-
./mr.bash download
257-
```
258-
259-
To validate your *PAT* has sufficient permissions for self-hosted runners registration on your GitHub organization `https://github.com/<ORG-NAME>`:
260-
261-
```bash
262-
./mr.bash pat2token --org <ORG-NAME>
263-
```
264-
265-
To setup two self-hosted runners on *VM-Runners* for your GitHub organization:
266-
267-
```bash
268-
./mr.bash add --org <ORG-NAME> --dotenv 'all_proxy=socks5h://localhost:7890'
269-
./mr.bash add --org <ORG-NAME> --dotenv 'all_proxy=socks5h://localhost:7890'
270-
```
271-
272-
To check the status of self-hosted runners:
273-
274-
```bash
275-
./mr.bash list
276-
```
277-
278-
To check the *Local Proxy* works well in your runners' process, you can add a simple workflow `.github/workflows/test-local-proxy.yml` in your repository. If `icanhazip.com` was configured as a following-to-remote domain, the workflow run will print outbound public IP of *VM-Proxy*, even though this workflow actually runs on *VM-Proxy*.
279-
280-
```yaml
281-
---
282-
name: Test Local Proxy works in my self-hosted runners
283-
on:
284-
workflow_dispatch:
285-
jobs:
286-
test:
287-
runs-on: [self-hosted, '${{ github.repository }}']
288-
steps:
289-
- run: |
290-
curl -s -4 icanhazip.com
291-
...
186+
all_proxy=socks5h://localhost:1080
292187
```

‎clash.example/config.yaml

-33
This file was deleted.

‎clash.example/start-clash.bash

-37
This file was deleted.

‎mr.bash

+23-16
Original file line numberDiff line numberDiff line change
@@ -248,18 +248,19 @@ function mr::downloadRunner {
248248
}
249249

250250
# Add GitHub Actions Runner by local username
251-
# $1: username, optional
252-
# $2: enterprise
253-
# $3: organization
254-
# $4: repository, optional
255-
# $5: runner registration token, optional
256-
# $6: extra labels, optional
257-
# $7: group, defaults to `default`
258-
# $8: lines to set in runner's '.env' files, optional
259-
# $9: count of runners, optional, defaults to 1
251+
# $1: username, optional
252+
# $2: enterprise
253+
# $3: organization
254+
# $4: repository, optional
255+
# $5: runner registration token, optional
256+
# $6: extra labels, optional
257+
# $7: group, defaults to `default`
258+
# $8: lines to set in runner's '.env' files, optional
259+
# $9: count of runners, optional, defaults to 1
260+
# $10: extra options for `config.sh`, optional, such as `--no-default-labels`
260261
# $?: 0 if successful and non-zero otherwise
261262
function mr::addRunner {
262-
local username="$1" enterprise="$2" org="$3" repo="$4" token="$5" extraLabels="$6" group="${7:-default}" dotenv="$8" tarpath=''
263+
local username="$1" enterprise="$2" org="$3" repo="$4" token="$5" extraLabels="$6" group="${7:-default}" dotenv="$8" opts="${10}" tarpath=''
263264
local -i count="${9:-1}"
264265
str::allVarsNotEmpty enterprise org || return $?
265266

@@ -299,7 +300,7 @@ function mr::addRunner {
299300
echo -n '$name' > name && echo -n '$labels' > labels && echo -n '$tarpath' > tarpath
300301
cd .. && tar -xzf "$tarpath"
301302
echo "$dotenv" >> .env
302-
./config.sh --unattended --replace --url '$url' --token '$token' --name '$name' --labels '$labels' --runnergroup '$group'
303+
./config.sh --unattended --replace --url '$url' --token '$token' --name '$name' --labels '$labels' --runnergroup '$group' $opts
303304
sudo ./svc.sh install '$user'
304305
if [[ "$(getenforce 2>/dev/null)" == "Enforcing" ]]; then
305306
chcon -t bin_t ./runsvc.sh # https://github.com/vbem/multi-runners/issues/9
@@ -317,9 +318,10 @@ __
317318
# $4: repository, optional
318319
# $5: runner registration token, optional
319320
# $6: count of runners, optional, defaults to 0 (all)
321+
# $7: extra options for `config.sh`, optional, such as `--local`
320322
# $?: 0 if successful and non-zero otherwise
321323
function mr::delRunner {
322-
local user="$1" enterprise="$2" org="$3" repo="$4" token="$5"
324+
local user="$1" enterprise="$2" org="$3" repo="$4" token="$5" opts="$7"
323325
local -i count="${6:-0}"
324326

325327
local -a removals=()
@@ -352,7 +354,7 @@ function mr::delRunner {
352354
run::logFailed sudo su --login "$user" -- <<-__
353355
cd runner
354356
sudo ./svc.sh stop && sudo ./svc.sh uninstall
355-
./config.sh remove --token '$token'
357+
./config.sh remove --token '$token' $opts
356358
__
357359
run::log sudo userdel -rf "$user"
358360
done
@@ -418,6 +420,7 @@ Options:
418420
--token Runner registration token, takes precedence over MR_GITHUB_PAT
419421
--dotenv The lines to set in runner's '.env' files
420422
--count The number to add or del, optional, defaults to 1 for add and all for del
423+
--opts Extra options for 'config.sh', optional, such as '--no-default-labels'
421424
-h --help Show this help.
422425
"
423426
declare -rg HELP
@@ -426,7 +429,7 @@ declare -rg HELP
426429
# $?: 0 if successful and non-zero otherwise
427430
function mr::main {
428431
local getopt_output='' subCmd=''
429-
local org='' repo='' user='' labels='' token='' group='' dotenv='' count=''
432+
local org='' repo='' user='' labels='' token='' group='' dotenv='' count='' opts=''
430433

431434
# parse options into variables
432435
getopt_output="$(getopt -o h -l help,enterprise:,org:,repo:,user:,labels:,token:,group:,dotenv:,count: -n "$FILE_THIS" -- "$@")"
@@ -472,6 +475,10 @@ function mr::main {
472475
count="$2"
473476
shift 2
474477
;;
478+
--opts)
479+
opts="$2"
480+
shift 2
481+
;;
475482
--)
476483
shift
477484
break
@@ -487,8 +494,8 @@ function mr::main {
487494
subCmd="$1"
488495
shift
489496
case "$subCmd" in
490-
add) mr::addRunner "$user" "$enterprise" "$org" "$repo" "$token" "$labels" "$group" "$dotenv" "$count" ;;
491-
del) mr::delRunner "$user" "$enterprise" "$org" "$repo" "$token" "$count" ;;
497+
add) mr::addRunner "$user" "$enterprise" "$org" "$repo" "$token" "$labels" "$group" "$dotenv" "$count" "$opts" ;;
498+
del) mr::delRunner "$user" "$enterprise" "$org" "$repo" "$token" "$count" "$opts" ;;
492499
list) mr::listRunners ;;
493500
status) mr::statusRunner "$user" ;;
494501
download) mr::downloadRunner ;;

0 commit comments

Comments
 (0)