MySQL Support

[ismail0234]

Describe your changes

MySQL support for Netbird. Still in draft stage. I have never used the “GO” language before, so if I have any mistakes you can help me to correct them.

Issue ticket number and link

Checklist

• Is it a bug fix
• Is a typo/documentation fix
• Is a feature enhancement
• It is a refactor
• Created tests that fail without the change (if possible)
• Extended the README / documentation, if necessary

[CLAassistant]

All committers have signed the CLA.

[ismail0234]

I'm working on it, and from what I've seen, adding mysql support seems to be pretty straightforward. But I haven't tested it yet and I'm not sure if it will work, so it's still in draft stage. Can someone tell me how to compile and test on the server side?

[mlsmaycon]

Thanks for the submission @ismail0234. Can you enable github action tests for MySQL as well?

See the current ones here:

netbird/.github/workflows/golang-test-linux.yml

Line 13 in 4e918e5

jobs:

[mlsmaycon]

@bcmmbaga can you share how to add a testcontainer for MySQL?

[bcmmbaga]

@ismail0234 To add a MySQL test container, you will need to add MySQL support in the NewTestStoreFromSQL

netbird/management/server/store.go

Line 253 in a9d06b8

func NewTestStoreFromSQL(ctx context.Context, filename string, dataDir string) (Store, func(), error) {

To spin up a new MySQL test container, you can follow the same approach we used for PostgreSQL

netbird/management/server/testutil/store.go

Line 17 in a9d06b8

func CreatePGDB() (func(), error) {

[ismail0234]

The “security/snyk” step seems to have failed but I can't see what caused it.

[bcmmbaga]

The “security/snyk” step seems to have failed but I can't see what caused it.

Go modules seem out of sync with the codebase. Could you run go mod tidy and push the changes?

[ismail0234]

hmm, the result still seems to be the same.

[ismail0234]

Is this part used for testing? Do I need to add mysql values?

netbird/.github/workflows/test-infrastructure-files.yml

Line 23 in a9d06b8

store: [ 'sqlite', 'postgres' ]

netbird/.github/workflows/test-infrastructure-files.yml

Lines 40 to 44 in a9d06b8

	if [ "${{ matrix.store }}" == "postgres" ]; then
	echo "NETBIRD_STORE_ENGINE_POSTGRES_DSN=host=$(hostname -I | awk '{print $1}') user=netbird password=postgres dbname=netbird port=5432" >> $GITHUB_ENV
	else
	echo "NETBIRD_STORE_ENGINE_POSTGRES_DSN==" >> $GITHUB_ENV
	fi

[ismail0234]

We are ready for the next test. Let's see what errors we get this time. -_-

[ismail0234]

TestContainers have been optimized. Instead of creating a new test container each time, we create 1 for mysql and postgres and cache it.

Test time for Postgres decreased from 350+ seconds to 140 seconds.
Test time for MySQL decreased from 1800+ seconds to 150 seconds.

[ismail0234]

The MySQL test seems to have been successful. 🥳

https://github.com/ismail0234/netbird/actions/runs/11777556741

[ismail0234]

Hi @mlsmaycon ,

How can I use this in a new server?

[ismail0234]

I tested it in production environment. Mysql connection is established and working without any problem.

But when I use 127.0.0.1 in the mysql connection string, the connection is not established. When I enter the ip address of the server, the connection can be established. I am not sure if this is a docker related issue. Mysql and netbird are running on the same server.

[ismail0234]

Some tests failed. Is it a general problem?

[ismail0234]

Everything is complete. I've tested it in a production environment and so far I haven't had any problems. The peers can connect to each other and we can use the panel features. It looks like there's nothing else I can do. @bcmmbaga

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[ismail0234]

Using MySQL database made the api calls very fast. When I use sqlite, api calls take 1.5-4 seconds when there are 500 peers on the network, while it takes 0.2 seconds when we use mysql.