Skip to content

Commit 0d8a44b

Browse files
authored
Agones2 (#232)
* use agones2 branch Signed-off-by: Niket <[email protected]> * specify kubernetes as a requirement Signed-off-by: Niket <[email protected]> * install k8s Signed-off-by: Niket <[email protected]> * don’t cache deps Signed-off-by: Niket <[email protected]> * setup google k8s config Signed-off-by: Niket <[email protected]> * remove double requirement of google auth Signed-off-by: Niket <[email protected]> * move auth code in a place where code gets executed Signed-off-by: Niket <[email protected]> * try another attempt at gke connection Signed-off-by: Niket <[email protected]> * add debug message Signed-off-by: Niket <[email protected]> * try another place Signed-off-by: Niket <[email protected]> * update parameters of get cluster Signed-off-by: Niket <[email protected]> * add debug list pods Signed-off-by: Niket <[email protected]> * one more try with this approach Signed-off-by: Niket <[email protected]> * add name keyword Signed-off-by: Niket <[email protected]> * use kubeconfig yaml file Signed-off-by: Niket <[email protected]> * remove migration step for now Signed-off-by: Niket <[email protected]> * print contexts Signed-off-by: Niket <[email protected]> * load kubeconfgi from dict Signed-off-by: Niket <[email protected]> * read config file in properly Signed-off-by: Niket <[email protected]> * move kubeconfig Signed-off-by: Niket <[email protected]> * try again Signed-off-by: Niket <[email protected]> * locate kubeconfig file Signed-off-by: Niket <[email protected]> * setup dev and staging clusters Signed-off-by: Niket <[email protected]> * Merge branch 'master' into agones2 * tested terraform on dev * updated agones version and output.tf * outputs.tf* * fix deploy.sh * fix kubeconfig template * attempt fixing missing game_ide * try to build aimmo from branch * use node 12 * try alternative to update node * back to nvm attempt * more commands for nvm * added node_modules to .gcloudignore * missing comma in requirements * build worker wheel * rebuilt dev cluster and disabled clusters setup * get latest published beta aimmo * enable clusters setup * downgraded agones version * use dev clusters * get aimmo from branch and some fixes * some ingress patches * ingress v1beta1 * fix cluster name in deploy * aimmo from branch in requirements * ingress and role in terraform and fleet in deploy * delete only ingress in deploy * delete game creator rs in deploy * added more permissions to game-creator * update fleet to 20 replicas * reduce fleet to 15 * cleanup game servers and services * recreated dev and cleanup aimmo branch deployment * updated k8s and agones * updated staging cluster
1 parent 6837241 commit 0d8a44b

21 files changed

+540
-30
lines changed

.gcloudignore

+1
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@ lib/*info/**
88
lib/PIL/**
99
lib/setuptools/script (dev).tmpl
1010
lib/**/static/**
11+
node_modules

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
.terraform
12
app.yaml
23
lib
34
static

app.yaml.tmpl

+1
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,4 @@ env_variables:
4141
REDIS_PORT: '${REDIS_PORT}'
4242
AWS_ACCESS_KEY_ID: '${AWS_ACCESS_KEY_ID}'
4343
AWS_SECRET_ACCESS_KEY: '${AWS_SECRET_ACCESS_KEY}'
44+
KUBECONFIG: 'kubeconfig.yaml'

build.sh

+19
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,25 @@ then
1717
pip install -t lib --upgrade --no-deps aimmo
1818
else
1919
pip install -t lib --pre --upgrade --no-deps aimmo
20+
21+
# Uncomment the blocks below to install aimmo from a branch - don't forget to uncomment the line in generate_requirements.py too
22+
# git clone --depth 1 --branch agones3 https://github.com/ocadotechnology/aimmo.git
23+
24+
# pushd aimmo
25+
# pip install wheel
26+
# ./aimmo_runner/build_worker_wheel.sh
27+
# popd
28+
29+
# pushd aimmo/game_frontend
30+
# curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
31+
# export NVM_DIR="$HOME/.nvm"
32+
# [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
33+
# nvm install 12.20.2
34+
# nvm use 12.20.2
35+
# yarn --frozen-lockfile
36+
# NODE_ENV=production node djangoBundler.js
37+
# popd
38+
# pip install -t lib --pre --upgrade --no-deps ./aimmo
2039
fi
2140

2241
python generate_requirements.py

clusters_setup/deploy.py

+104-26
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ def create_ingress_yaml(module_name):
2121
"""
2222
Uses a template ingress yaml file in the same directory to construct a python yaml
2323
object. Replaces the ingress.global-static-ip-name annotation with the
24-
correct module name (depending on which cluster this script is ran in).
24+
correct module name (depending on which cluster this script is ran in). Similarly for
25+
cors-allow-origin, it replaces it with the correct url for selected module.
2526
2627
Also replaces the TLS certificate host name and the host for the ingress paths.
2728
@@ -30,27 +31,70 @@ def create_ingress_yaml(module_name):
3031
"""
3132
path = os.path.join(CURR_DIR, "ingress.yaml")
3233

33-
host_name = module_name + "-aimmo.codeforlife.education"
34+
host_name = f"{module_name}-aimmo.codeforlife.education"
35+
36+
cors_origin = f"https://{module_name}-dot-decent-digit-629.appspot.com"
37+
if module_name == "default":
38+
cors_origin = f"https://www.codeforlife.education"
3439

3540
with open(path) as yaml_file:
3641
content = yaml.safe_load(yaml_file.read())
3742
content["metadata"]["annotations"][
3843
"kubernetes.io/ingress.global-static-ip-name"
39-
] = (module_name + "-aimmo-ingress")
44+
] = f"{module_name}-aimmo-ingress"
45+
content["metadata"]["annotations"][
46+
"nginx.ingress.kubernetes.io/cors-allow-origin"
47+
] = cors_origin
4048
content["spec"]["tls"][0]["hosts"][0] = host_name
4149
content["spec"]["rules"][0]["host"] = host_name
4250

4351
return content
4452

4553

54+
def create_fleet_yaml(module_name, aimmo_version):
55+
"""
56+
Uses a template fleet yaml file in the same directory to construct a python yaml
57+
object. Replaces image version with the one received as a parameter. Also replaces
58+
the GAME_API_URL environment variable with the correct url for the deployment
59+
environment we are in.
60+
61+
:param module_name: The name of the environment we're in (ie. staging, dev).
62+
:param aimmo_version: The game version we want to deploy.
63+
:return: python object containing yaml with modified values.
64+
"""
65+
path = os.path.join(CURR_DIR, "fleet.yaml")
66+
67+
game_api_url = (
68+
f"https://{module_name}-dot-decent-digit-629.appspot.com/kurono/api/games/"
69+
)
70+
container_image = f"ocadotechnology/aimmo-game:{aimmo_version}"
71+
72+
with open(path) as yaml_file:
73+
content = yaml.safe_load(yaml_file.read())
74+
75+
env_variables = content["spec"]["template"]["spec"]["template"]["spec"][
76+
"containers"
77+
][0]["env"]
78+
game_api_url_index = env_variables.index(
79+
{"name": "GAME_API_URL", "value": "REPLACE_ME"}
80+
)
81+
env_variables[game_api_url_index]["value"] = game_api_url
82+
content["spec"]["template"]["spec"]["template"]["spec"]["containers"][0][
83+
"image"
84+
] = container_image
85+
86+
return content
87+
88+
4689
def create_creator_yaml(module_name, aimmo_version):
4790
"""
4891
Loads an aimmo-game-creator yaml into a dictionary.
4992
5093
Replaces the GAME_API_URL environment variable with the correct url for the
5194
deployment environment we are in
5295
53-
:param module_name: The name of the environment we're in (ie. staging, dev)
96+
:param module_name: The name of the environment we're in (ie. staging, dev).
97+
:param aimmo_version: The game version we want to deploy.
5498
:return: python object containing yaml with modified values.
5599
"""
56100

@@ -87,49 +131,80 @@ def _replace_image_tag(content):
87131
return content
88132

89133

90-
def restart_pods(game_creator_yaml, ingress_yaml):
134+
def restart_pods(game_creator_yaml, ingress_yaml, fleet_yaml):
91135
"""
92136
Restarts the kubernetes replication controllers, pods, services and ingresses
93137
in the 'default' namespace
94138
95139
:param game_creator_yaml: The dict to create the aimmo game creator rc
96140
:param ingress_yaml: The dict to create the ingress
141+
:param fleet_yaml: The dict to create the fleet
97142
"""
98143
for rs in apps_api_instance.list_namespaced_replica_set("default").items:
99-
apps_api_instance.delete_namespaced_replica_set(
100-
body=kubernetes.client.V1DeleteOptions(),
101-
name=rs.metadata.name,
102-
namespace="default",
103-
)
104-
for rc in api_instance.list_namespaced_replication_controller("default").items:
105-
api_instance.delete_namespaced_replication_controller(
106-
body=kubernetes.client.V1DeleteOptions(),
107-
name=rc.metadata.name,
108-
namespace="default",
109-
)
110-
for pod in api_instance.list_namespaced_pod("default").items:
111-
api_instance.delete_namespaced_pod(
112-
body=kubernetes.client.V1DeleteOptions(),
113-
name=pod.metadata.name,
114-
namespace="default",
115-
)
144+
if rs.metadata.name == game_creator_yaml["metadata"]["name"]:
145+
apps_api_instance.delete_namespaced_replica_set(
146+
body=kubernetes.client.V1DeleteOptions(),
147+
name=rs.metadata.name,
148+
namespace="default",
149+
)
116150
for service in api_instance.list_namespaced_service("default").items:
117-
api_instance.delete_namespaced_service(
118-
name=service.metadata.name, namespace="default"
119-
)
151+
if service.metadata.name.startswith("game-"):
152+
api_instance.delete_namespaced_service(
153+
name=service.metadata.name, namespace="default"
154+
)
120155
for ingress in networking_api_instance.list_namespaced_ingress("default").items:
121156
networking_api_instance.delete_namespaced_ingress(
122157
name=ingress.metadata.name,
123158
namespace="default",
124159
body=kubernetes.client.V1DeleteOptions(),
125160
)
126161

162+
fleets_to_delete = custom_objects_api_instance.list_namespaced_custom_object(
163+
group="agones.dev",
164+
version="v1",
165+
namespace="default",
166+
plural="fleets",
167+
)["items"]
168+
for fleet in fleets_to_delete:
169+
name = fleet["metadata"]["name"]
170+
custom_objects_api_instance.delete_namespaced_custom_object(
171+
group="agones.dev",
172+
version="v1",
173+
namespace="default",
174+
plural="fleets",
175+
name=name,
176+
)
177+
178+
game_servers_to_delete = custom_objects_api_instance.list_namespaced_custom_object(
179+
group="agones.dev",
180+
version="v1",
181+
namespace="default",
182+
plural="gameservers",
183+
)["items"]
184+
for game_server in game_servers_to_delete:
185+
name = game_server["metadata"]["name"]
186+
custom_objects_api_instance.delete_namespaced_custom_object(
187+
group="agones.dev",
188+
version="v1",
189+
namespace="default",
190+
plural="gameservers",
191+
name=name,
192+
)
193+
127194
networking_api_instance.create_namespaced_ingress("default", ingress_yaml)
128195

129196
apps_api_instance.create_namespaced_replica_set(
130197
body=game_creator_yaml, namespace="default"
131198
)
132199

200+
custom_objects_api_instance.create_namespaced_custom_object(
201+
group="agones.dev",
202+
version="v1",
203+
namespace="default",
204+
plural="fleets",
205+
body=fleet_yaml,
206+
)
207+
133208

134209
def main(module_name):
135210
"""
@@ -142,17 +217,20 @@ def main(module_name):
142217
global api_instance
143218
global apps_api_instance
144219
global networking_api_instance
220+
global custom_objects_api_instance
145221
api_instance = kubernetes.client.CoreV1Api()
146222
apps_api_instance = kubernetes.client.AppsV1Api()
147223
networking_api_instance = kubernetes.client.NetworkingV1beta1Api()
224+
custom_objects_api_instance = kubernetes.client.CustomObjectsApi()
148225
aimmo_version = get_aimmo_version()
149226

150227
ingress = create_ingress_yaml(module_name=module_name)
151228
game_creator_rs = create_creator_yaml(
152229
module_name=module_name, aimmo_version=aimmo_version
153230
)
231+
fleet = create_fleet_yaml(module_name=module_name, aimmo_version=aimmo_version)
154232

155-
restart_pods(game_creator_rs, ingress)
233+
restart_pods(game_creator_rs, ingress, fleet)
156234

157235

158236
if __name__ == "__main__":

clusters_setup/fleet.yaml

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
apiVersion: "agones.dev/v1"
2+
kind: Fleet
3+
metadata:
4+
name: aimmo-game
5+
spec:
6+
replicas: 15
7+
strategy:
8+
type: Recreate
9+
template:
10+
spec:
11+
ports:
12+
- name: default
13+
protocol: TCPUDP
14+
containerPort: 5000
15+
health:
16+
initialDelaySeconds: 30
17+
template:
18+
spec:
19+
serviceAccountName: worker-manager
20+
containers:
21+
- name: aimmo-game
22+
image: ocadotechnology/aimmo-game:latest
23+
env:
24+
- name: GAME_API_URL
25+
value: REPLACE_ME
26+
- name: GAME_MANAGER
27+
value: kubernetes
28+
resources:
29+
requests:
30+
memory: "64Mi"
31+
cpu: "24m"
32+
limits:
33+
memory: "124Mi"
34+
cpu: "95m"

clusters_setup/ingress.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ metadata:
77
annotations:
88
kubernetes.io/ingress.class: nginx
99
kubernetes.io/ingress.global-static-ip-name: REPLACE_STATIC_IP
10+
nginx.ingress.kubernetes.io/cors-allow-origin: REPLACE_CORS_ORIGIN
11+
nginx.ingress.kubernetes.io/enable-cors: "true"
12+
nginx.ingress.kubernetes.io/rewrite-target: /$2
1013
spec:
1114
tls:
1215
- hosts:

clusters_setup/rs_aimmo_game_creator.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ spec:
1818
# The api server enforces this constraint.
1919
app: aimmo-game-creator
2020
spec:
21+
serviceAccountName: game-creator
2122
containers:
2223
- name: aimmo-game-creator
2324
image: ocadotechnology/aimmo-game-creator:latest

clusters_setup/terraform/.terraform.lock.hcl

+73
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)