Completely dockerize Wasp app

[Martinsos]

Right now, we have Dockerfile for deploying server to production.

We could also use Docker in development though -> both web app and server could be running in their own docker containers, therefore ensuring that they run in exactly the same way for all the developers. This would remove explicit dependency on node, which would become part of the container and therefore part of implementation. So basically, only dependency would be on docker -> if you have docker, you can use wasp.

This does seem to be somewhat tricky, as it can disrupt typical development flow and possibly slow it down. I don't know all the details, but I "heard" it might be tricky getting it to work with the IDE of choice + debugger, possibly also live reloading of web browser.
Also, what if web app requires something installed on the machine, and we don't have it already in the Dockerfile? How would we set that up, enable people to install whatever they need? Maybe they could provide base image to build Dockerfiles from, and we can say it has to be Ubuntu or smth like that?

[Martinsos]

I asked for thought of people in our community, and the main feedback was that Dockerized stuff can have performance problems (especially when dealing with filesystem operations on MacOS) and that while they would like everything Dockerized as an option, they don't think it should be the only way of running Wasp locally.
It does sound from this like Dockerizing everything could be problem because of the performance issues.
I think we should still give it some thought, but we have to be very cautions regarding the performance and possible limitations.
Running just DB in Docker is ok though.

Here is link to my message where I started the discussion: https://discord.com/channels/686873244791210014/686873245428875297/821698905477218354 .

Below I also copied the discussion, just in case:

Discussion on Discord: Click to expand

martinsos — Today at 11:58
@everyone , what do you think about having docker as a dependency for using Wasp?
This would enable us to run web app, server and database in development dockerized, which basically means there will be no other dependencies (like specific version of node right now), only dependency would be docker and we take care of all the other requirements by using specific docker images.
This should also eliminate any "but it works on my machine" issues.

Did anybody have experience running frontend or backend dockerized during developmen? My general feeling is that it is recommended as an approach, but that not many people are actually doing it due to being complicated to set up.

@mateothegreat you said you are devops ninja :D, what do you think?
ad-si — Today at 12:01
The performance of Docker on macOS is still noticeable slower than running the code native. So I've stopped using Docker for development. But since most of my projects only depend on Node or GHC it's relatively simple to swap the version. The more complicated the development environment gets the more sense it makes to use Docker.
MEE6
BOT
— Today at 12:01
Wohooo @ad-si, you just became a Waspeteer level 1!
krazyjakee — Today at 12:02
Hard nope from me. Docker performance for local dev is dirt poor even on strong hardware
dareDEVil — Today at 12:10
Would be better if it's not because issues prevail across all local Dev
Providing an easier integration, that would be nice
martinsos — Today at 13:13
Makes sense, we are thinking in the distant future about supporting multiple languages on the backend so in that case it would be pretty cool being able to just pull the whatever docker image we need and use that, but if performance is an issue then we will have to take that into account. Hm hm hm, thanks for sharing this, good to know! Just to clarify -> the main problem is that code inside the docker container runs slower than it would natively, and not the resources that docker container itself consumes regardless of what is running in it?
martinsos — Today at 13:14
What did you try it for that performance was an issue? Which OS did you use it on?
krazyjakee — Today at 13:14
rails 😄
martinsos — Today at 13:15
What do you mean "would be better if it's not because issues prevail accross all local dev" -> would be better if docker is not a requirement, I assume? I don't get the part with issues across all local dev hm.
"Providing an easier integration" -> you mean having less dependencies, or smth else?
dareDEVil — Today at 13:22
Exactly, fewer dependencies would mean a bit more freedom, the issues are primarily with how Mac and Docker aren't harmonious as of now.
martinsos — Today at 13:22
I am just doing some quick googling and it seems problem is with Docker on Mac, because it is not run natively like on Linux, instead it is run on HyperKit which is some kind of lightweight macOS virtualization. And actually reading further, main problem seems to be with operations concerning the filesystem IO being slow, since the filesystems are not the same. It seems docker-sync helped some people a lot with that.
Ok, this is certainly something we will have to investigate in details!
aqid — Today at 14:22
Having dockerized setup is definitely a plus for me since when we're going to deploy the program, it will be dockerized anyway so having it as official dockerfile would be super nice
martinsos — Today at 14:55
Cool, that is what I also like! Have you had any performance issues with Docker? Are you Linux or MacOS?
aqid — Today at 14:57
My main dev env is wsl 2 in windows and mac os.
Yes it can be excruciatingly slow especially when you work with resource consuming task. But I'd say that those tasks aren't that common so I use docker locally more often than not
andrewnc — Today at 15:01
I don't have strong opinions about using docker. I do think that it is important to be aware that your messaging and marketing will need to change if you use Docker. It will no longer feel like a programming language for the web, but a full web framework. That's totally fine, but a decision that needs to be made consciously. Although my $0,02 on docker itself, I've also uninstalled it last year and do all my dev locally
MEE6
BOT
— Today at 15:01
Wohooo @andrewnc, you just became a Waspeteer level 1!
Ramiro — Today at 17:22
I think it's a good idea as an option, but not if it's the only way to develop. But having a one click and start working on wasp is cool and lowers the barrier of entry.
martinsos — Today at 17:23
Hmmm so performance problems again, that is really concerning then. You said resource intensive tasks -> what kind of resources, do you have some example?
martinsos — Today at 17:27
This helps! How do you think it would affect the messaging, language vs framework? If we go down the docker route, Wasp user shouldn't care about it in any way really, it would all be happening in the background, so I didn't think that would affect its usage/messaging in any way (except if it has performance issue, that is a concern for sure). Just to be clear, those docker container would be running locally on your machine, in the background, controlled by Wasp.
Ramiro — Today at 17:27
@Martinsos your idea is to have docker for DBs and that stuff, but not for the core app? or for everything?
martinsos — Today at 17:27
Thanks! Also because of the performance issues?
martinsos — Today at 17:28
Both are options -> one thing is to have it for database, I guess that shouldn't be a problem, and the next step is to have it for everything, to decouple completely from the underlying system.
Ramiro — Today at 17:28
running a container adds overhead. If you're doing synchronizing a lot of files, or mounting volumes, it's also a so so experience.
martinsos — Today at 17:29
@ramiro what do you think about running just the database in docker container?
Ramiro — Today at 17:30
I'm a big fan of running the whole thing in containers (it's a big part of what we do in okteto.com), but in local docker the experience needs to be fine tuned.
Ramiro — Today at 17:30
yeah, that makes a lot easier.
martinsos — Today at 17:31
That fine-tuning - do you think that depends on the specific logic of the web app and what it does, or is it more about the tech stack used? I am asking because I am wondering if we can take care of this in Wasp, or if it is going to be custom for every web app and then we will have issues because people will need to customize the containers with kind of ruins the concept of it all being one-click and smooth
Ramiro — Today at 17:37
it's the stack. e.g. node, with node_modules has a lot of issues because it does a lot of read/writes, and the docker file system is not optimized for that. The biggest pain I've seen is specially when you are writing from both the mac side and the container side.
martinsos — Today at 17:52
@ramiro hmmm that could certainly be a problem for us then if we would go with just Docker, for people using Wasp on non-Linux! Awesome, thanks for this.
Thanks everybody for the help! I feel that Docker is still worth consideration in the future, but after all this feedback it is clear we will need to be very careful in that case to ensure we don't have any performance issues, and there is a fair chance going the "dockerize it all" route as a default approach might not be an option. I will transfer this feedback into a GH issue so we can track there what we know.
aqid — Today at 17:56
Complex queries with large databases, or watching lots of files. For the latter I usually use volumes though.
I would say that I use docker for development 80% of the time. Also I agree that docker should be an option, not requirement. People should be free to choose whatever they're comfortable of

[cursorial]

Adding my 2 cents, I think Dockerized stuff has performance problems on OS X, but on Linux it's perfectly fine. This StackOverflow answer explains quite nicely why there is a difference: https://stackoverflow.com/a/55953023/1233908

Personally I'm a Docker convert ever since making the switch from OS X to Linux, but I can understand the frustration there for OS X users.

[Martinsos]

Thanks for the comment @cursorial! Yes, on Linux it should be fine, but OSX is tricky. I however read that using docker volumes can help, instead of directly binding the directory from the file system, so maybe that is a way to go about it. Docker volumes should be much more performant, even on OSX, possibly to the level where this is not an issue anymore.

[Martinsos]

Just to add to this: using docker volumes is the way to go to persist data on host machine, and they don't have performance issues, I can confirm that after more research / using them.