You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/admin/monitoring-and-managing-your-instance/multiple-data-disks/configuring-multiple-data-disks.md
+228-7Lines changed: 228 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
2
title: Configuring multiple data disks
3
3
product: '{% data variables.product.prodname_ghe_server %}'
4
-
intro: 'You can configure additional data disks and use them to host MySQL and repositories data.'
4
+
intro: 'You can configure additional data disks and use them to host data of different services.'
5
5
versions:
6
6
ghes: '>= 3.19'
7
7
type: overview
@@ -25,6 +25,10 @@ topics:
25
25
* Resources can be allocated in a targeted way, preventing critical services from being starved.
26
26
* Scaling:
27
27
* Customers on both standalone and high-availability topologies can scale out as needed.
28
+
{% ifversion ghes > 3.19 %}
29
+
* Resiliency:
30
+
* Isolating logs from the root disk enhances resiliency by preventing the volume of logs from flooding the root disk.
31
+
{%- endif %}
28
32
29
33
## Constraints
30
34
@@ -34,12 +38,19 @@ topics:
34
38
* You can minimize this by configuring a replica with multi-data disks, replicating data from the primary, and then failing over to the replica.
35
39
* If you are adding multi-data disks directly to the primary, expect a much longer downtime.
36
40
* During the public preview, multi-data disks should be used only in non-production environments.
41
+
{% ifversion ghes < 3.20 %}
37
42
* It is not recommended to migrate MySQL and repositories to the same disk.
38
43
* Currently, only MySQL and repositories can be migrated to additional disks.
44
+
{%- endif %}
45
+
{% ifversion ghes > 3.19 %}
46
+
* It is not recommended to migrate MySQL, repositories, system logs, or {% data variables.product.github %} logs to the same disk. Each additional disk should only contain one migration.
47
+
* Currently, only MySQL, repositories, system logs, and {% data variables.product.github %} logs can be migrated to additional disks.
48
+
* Rebooting the {% data variables.product.prodname_ghe_server %} node is required after migrating system logs to ensure it is working on a system level. It will take some time as config apply also runs during startup of the node.
49
+
{%- endif %}
39
50
40
51
## Resource recommendations
41
52
42
-
If you add disks that are as fast or faster than your current ones, you should see improved performance. Storage devices are typically measured by IOPS (Input/Output Operations Per Second), throughput, and latency. For MySQL, we recommend using a disk with lower latency and higher IOPS than your existing data disk. For repositories, choose a disk with higher IOPS and throughput than your current data disk.
53
+
If you add disks that are as fast or faster than your current ones, you should see improved performance. Storage devices are typically measured by IOPS (Input/Output Operations Per Second), throughput, and latency. For MySQL, we recommend using a disk with lower latency and higher IOPS than your existing data disk. For repositories, choose a disk with higher IOPS and throughput than your current data disk.{% ifversion ghes > 3.19 %} For logs, we recommend using a disk with higher IOPS and throughput than your existing data disk to handle continuous write operations from logging activities. {%- endif %}
43
54
44
55
In high availability setups, it is best to use multi-data disks on both the primary and all replicas. Mixing configurations, where the primary has multi-data disks but the replica does not, is not recommended.
45
56
@@ -72,7 +83,25 @@ In high availability setups, it is best to use multi-data disks on both the prim
Please note that `/dev/nvme2n1` and `/dev/nvme3n1` are example paths only. They might not match the paths on your system. Similarly, `db` and `git` are examples. You may choose different names.
101
+
{%- endif %}
102
+
{% ifversion ghes > 3.19 %}
103
+
Please note that `/dev/nvme2n1`, `/dev/nvme3n1`, `/dev/nvme4n1`, and `/dev/nvme5n1` are example paths only. They might not match the paths on your system. Similarly, `db`, `git`, `systemlogs`, and `githublogs` are examples. You may choose different names.
104
+
{%- endif %}
76
105
77
106
1. Switch to maintenance mode.
78
107
@@ -94,14 +123,39 @@ In high availability setups, it is best to use multi-data disks on both the prim
1. Test the instance for a period of time to make sure everything works as expected.
153
+
{% ifversion ghes < 3.20 %}
104
154
1. **Only after sufficient testing**, remove `/data/user/mysql-backup` and `/data/user/repositories-backup`.
155
+
{%- endif %}
156
+
{% ifversion ghes > 3.19 %}
157
+
1. **Only after sufficient testing**, remove `/data/user/mysql-backup`, `/data/user/repositories-backup`, `/var/log-backup`, `/data/github/current/log-backup`, and `/data/github/shared/log-backup`.
158
+
{%- endif %}
105
159
106
160
Keeping these folders during testing allows you to roll back in an emergency. After sufficient testing, you should remove those backup folders to free up space.
107
161
@@ -112,18 +166,24 @@ The following guidance helps reduce downtime in high availability (HA) topologie
112
166
For HA topologies, the best approach is to stand up a new replica with multiple data disks configured, replicate data from the primary, and then promote the replica to primary. Migrating data to additional disks on the current primary is not recommended, as this process can lead to significant downtime.
113
167
114
168
1. Set up a new HA replica with better disks.
115
-
169
+
{% ifversion ghes < 3.20 %}
116
170
To plan for the data migration, use `du -sh /data/user/mysql` and `du -sh /data/user/repositories` on the primary to calculate disk space requirements for the new replica.
171
+
{%- endif %}
172
+
{% ifversion ghes > 3.19 %}
173
+
To plan for the data migration, use `du -sh /data/user/mysql`, `du -sh /data/user/repositories`, `du -sh /var/log`, `du -sh /data/github/current/log`, and `du -sh /data/github/shared/log` on the primary to calculate disk space requirements for the new replica.
174
+
{%- endif %}
117
175
118
176
1. Set up multi-disk on the new HA replica.
119
177
1. Allow the HA primary to replicate to the replica.
120
178
1. Follow the failover sequence as documented in [AUTOTITLE](/admin/monitoring-and-managing-your-instance/configuring-high-availability/initiating-a-failover-to-your-replica-appliance).
121
179
122
180
While the replication process can take a long time, the advantage is that it runs in the background, so the actual disruption from maintenance mode is dramatically reduced.
123
181
124
-
## Example: configuring two additional disks
182
+
## Example: configuring additional disks
183
+
184
+
This example demonstrates the required commands and outputs for disk initialization and data migration. Specifically, `/data/user/mysql` is migrated to `/data/multi-disk/db/mysql`, and `/data/user/repositories` is migrated to `/data/multi-disk/git/repositories`.{% ifversion ghes > 3.19 %} Additionally, system logs are migrated to `/data/multi-disk/systemlogs/log`, and {% data variables.product.github %} logs are migrated to `/data/multi-disk/githublogs`.{%- endif %}
125
185
126
-
This example demonstrates the required commands and outputs for disk initialization and data migration. Specifically, `/data/user/mysql` is migrated to `/data/multi-disk/db/mysql`, and `/data/user/repositories` is migrated to `/data/multi-disk/git/repositories`.
186
+
{% ifversion ghes < 3.20 %}
127
187
128
188
```shell
129
189
admin@ghe-test-primary:~$ /usr/local/share/enterprise/ghe-storage-multi-disk status
@@ -139,7 +199,7 @@ admin@ghe-test-primary:~$ /usr/local/share/enterprise/ghe-storage-multi-disk ini
139
199
Starting initialization sequence for /dev/nvme3n1 at /data/multi-disk/git...
140
200
141
201
admin@ghe-test-primary:~$ /usr/local/share/enterprise/ghe-storage-migrate-mysql db
142
-
Start mysql migration to /data/multi-disk/db...
202
+
Start MySQL migration to /data/multi-disk/db...
143
203
Running checks..
144
204
Error: maintenance mode must be enabled before being able to proceed.
145
205
ERROR: Last Command: return 1 LINE: 36 ghe-storage-migrate-mysql
@@ -148,7 +208,7 @@ Script exited with exit code: 1
148
208
admin@ghe-test-primary:~$ ghe-maintenance -s
149
209
150
210
admin@ghe-test-primary:~$ /usr/local/share/enterprise/ghe-storage-migrate-mysql db
151
-
Start repository migration to /data/multi-disk/db...
211
+
Start MySQL migration to /data/multi-disk/db...
152
212
Success: /data/user/mysql moved to /data/multi-disk/db/mysql
Both `/usr/local/share/enterprise/ghe-storage-multi-disk status` and `/usr/local/share/enterprise/ghe-storage-multi-disk info` are helpful for checking your setup.
184
354
185
355
To view the current multi-disk configuration, use:
To review multi-disk logs, including disk initialization and migration events, run:
196
391
392
+
{% ifversion ghes < 3.20 %}
393
+
197
394
```shell
198
395
$ ls -l /var/log/multi-disk/
199
396
total 56
@@ -203,6 +400,24 @@ total 56
203
400
-rw-r--r-- 1 root root 37296 Mar 3 13:30 ghe-storage-migrate-repositories.log
204
401
```
205
402
403
+
{%- endif %}
404
+
{% ifversion ghes > 3.19 %}
405
+
406
+
```shell
407
+
$ ls -l /var/log/multi-disk/
408
+
total 64
409
+
-rw-r--r-- 1 root root 2115 Feb 13 16:32 ghe-storage-init-db.log
410
+
-rw-r--r-- 1 root root 2478 Feb 13 16:36 ghe-storage-init-githublogs.log
411
+
-rw-r--r-- 1 root root 2114 Feb 13 16:36 ghe-storage-init-git.log
412
+
-rw-r--r-- 1 root root 2378 Feb 13 16:36 ghe-storage-init-systemlogs.log
413
+
-rw-r--r-- 1 root root 20450 Feb 13 17:27 ghe-storage-migrate-github-logs.log
414
+
-rw-r--r-- 1 root root 1053 Feb 13 17:15 ghe-storage-migrate-logs.log
415
+
-rw-r--r-- 1 root root 2460 Feb 13 16:38 ghe-storage-migrate-mysql.log
416
+
-rw-r--r-- 1 root root 19011 Feb 13 16:42 ghe-storage-migrate-repositories.log
417
+
```
418
+
419
+
{%- endif %}
420
+
206
421
## Commands for managing multiple disks
207
422
208
423
These commands make it possible to add multiple disks and migrate specific services or folder paths to those disks. The original folder paths are maintained and kept static. Other services are unaware that anything has changed. The static folder paths are symlinked to the newly migrated paths.
@@ -220,3 +435,9 @@ The commands include:
220
435
* Migrates `/data/user/repositories` to any disk path created using `ghe-storage-multi-disk init`.
221
436
* ghe-storage-migrate-mysql
222
437
* Migrates `/data/user/mysql` to any disk path created using `ghe-storage-multi-disk init`.
438
+
{% ifversion ghes > 3.19 %}
439
+
* ghe-storage-migrate-logs
440
+
* Migrates `/var/log` to any disk path created using `ghe-storage-multi-disk init`.
441
+
* ghe-storage-migrate-github-logs
442
+
* Migrates `/data/github/current/log` and `/data/github/shared/log` to any disk path created using `ghe-storage-multi-disk init`.
0 commit comments