Update Devcontainer configuration to use pixi
#276
Conversation
flferretti
force-pushed
the
fix_devcontainer
branch
from
4400b35 to
b7c0d38
Compare
flferretti
force-pushed
the
fix_devcontainer
branch
2 times, most recently
from
954c261 to
fcb9816
Compare
|
@flferretti can you please finalize this PR? |
flferretti
force-pushed
the
fix_devcontainer
branch
2 times, most recently
from
ae6d9a9 to
d817f0c
Compare
|
Hi @flferretti, thanks for the PR! Docker build log
Have you ever had this? I'm suspecting this could be due to my weak internet connection while pulling the pixi.lock file. I will investigate this further. |
Actually, looking at this part of the error log, I think it is some problem on the installation of git-lfs: |
|
I'm about to propose a new approach, wait for it. |
|
Can you please try with the following?
# syntax=docker/dockerfile:1.4
ARG PROJECT_NAME=jaxsim
ARG PIXI_ENVIRONMENT=test-cpu
# Create the pixi environment using the official pixi image.
# We will copy the resulting `.pixi` folder to an official devcontainer image.
FROM ghcr.io/prefix-dev/pixi:latest AS build
ARG PROJECT_NAME
ARG PIXI_ENVIRONMENT
# Add Git LFS package repository and install.
RUN apt-get update &&\
apt-get install -y --no-install-recommends git-lfs &&\
rm -rf /var/lib/apt/lists/*
# Create the pixi environment.
COPY . /$PROJECT_NAME
WORKDIR /$PROJECT_NAME
RUN cd /$PROJECT_NAME &&\
git lfs pull &&\
pixi install --environment $PIXI_ENVIRONMENT &&\
pixi shell-hook --environment $PIXI_ENVIRONMENT > /shell-hook.sh &&\
echo 'exec "$@"' >> /shell-hook.sh &&\
chmod +x /shell-hook.sh
# Pixi is no longer needed in the final devcontainer.
# Users can easily download it in their home, if needed.
FROM mcr.microsoft.com/devcontainers/base:ubuntu AS production
ARG PROJECT_NAME
ARG PIXI_ENVIRONMENT
# Copy the environment in the /jaxsim folder.
WORKDIR /$PROJECT_NAME
COPY --from=build /$PROJECT_NAME/.pixi/envs/$PIXI_ENVIRONMENT /$PROJECT_NAME/.pixi/envs/$PIXI_ENVIRONMENT
COPY --from=build /shell-hook.sh /shell-hook.sh
# Set the shell-hook script as the entrypoint.
ENTRYPOINT ["/bin/bash", "/shell-hook.sh"]I can push it in this branch if you like the approach. |
Thanks Diego! Feel free to push it to this branch, so it can be tested with devcontainer. In the worst case, we can drop the commit via rebase |
I'm not encountering this error when building the devcontainer from this PR. How are you testing it? |
Thanks @diegoferigo I will try it! Meanwhile, I discovered it was sufficient to update the command |
I'm using the devcontainer extension of vscode and just clicking on the devocontainer button |
Now I remember why the |
flferretti
force-pushed
the
fix_devcontainer
branch
from
0c5395d to
1ce1e66
Compare
Done in ffc8a55. |
There was a problem hiding this comment.
Not a big expert of devcontainer, so anything is ok for me. I just wanted to make sure that you checked https://pixi.sh/latest/ide_integration/devcontainer/ . For what regarding the gpu part, I guess enabling it in the devcontainer is tricky, right?
That's great, thanks for linking the documentation. I was not aware of it. I don't like that they operate on
For this, likely a local setup is recommended. And people can use our pixi's |
Thanks for your contribution Diego! Nevertheless, I believe a multistage build is way too complex for a tool that is supposed to be light, easy to use and maintain. Do you mind if we just come back to the last working commit (d817f0c)?
I totally agree on this! This is just meant to be an easy way to review PR or perform some quick tests |
|
This is also the approach used here: https://github.com/prefix-dev/pixi-docker/blob/main/example/Dockerfile. It seems to me a good option (the only one that I can think of) that allows to build an environment using the official pixi image and run the devcontainer using the official ms image. This is to me much more robust on the long term since both ms/pixi can update their core images following their development cycle and we don't have to care. Of course, on our side, we need to pay the price of a slightly more complicated build stage (I could link to you really complex images and this is definitely not one of them 😄). |
|
Note also that I'm not really sure how your previous solution worked, because when you mount a volume (the repository of a PR) over an existing folder (the empty repository stored in the image with just the Before this PR, things were working fine because the image was using a regular conda environment. Now, with pixi the situations got entangled because the |
Note that you can change this with https://pixi.sh/latest/reference/pixi_configuration/#detached-environments, but I am not sure how handy is to do that in this case. Anyhow, I do not know enough about devcontainers to get everything you are discussing here, but if you have any specific point you want to discuss with pixi devs, feel free to open a discussion in https://github.com/prefix-dev/pixi/discussions or prefix's discord, in my experience they are quite receptive to user's feedback from advanced use cases. |
I agree with this approach for two reasons: it follows the pixi guidelines on how to build a devcontainer image and secondly it should be easy to maintain. I am trying this now, if I am able to run a simple script there for me is good to go. |
|
I'm not sure why, but if I create a codespace from the
|
I tried it locally, it started but during the build it gave me an error: I don't know if it is related to this but then from the devcontainer terminal I cannot run the pixi environment python or use it as kernel to run example notebooks. |
I'm able to launch the devcontainer from codespaces (not getting your error), but also in this case there are issues and the codespace launches a recovery container: |
|
On my setup, I was able to build the container locally. I can try from VSCode, I still haven't. It seems a permission error probably due to the default usage of non-root user, I'm not that familiar with devcontainers. |
flferretti
force-pushed
the
fix_devcontainer
branch
from
3d2e4e7 to
236a062
Compare
|
Locally in VSCode now it seems building and loading correctly, but it still doesn't pick up the python interpreter from the pixi environment. |
|
In 52bb311 I modified the Dockerfile and the devcontainer.json as per https://pixi.sh/v0.35.0/ide_integration/devcontainer/ and I tested that it worked in GitHub Codespaces, correctly picking up the python interpreter both in terminal and in the workspace. If you agree, I'll clean the commit history and merge |
There was a problem hiding this comment.
Yep it works for me, thanks! Your changes moved the creation of the pixi environment from the image to the container creation, and this means that every time you open a new devcontainer also locally is vscode, pixi needs to download many GB of data. It's not relevant for me, as this functionality is mainly useful in CodeSpaces.
| FROM mcr.microsoft.com/devcontainers/base:ubuntu | ||
| COPY environment.yml /opt/environment.yml | ||
| # syntax=docker/dockerfile:1.4 | ||
| FROM mcr.microsoft.com/devcontainers/base:jammy |
There was a problem hiding this comment.
Would this work also on a newer ubuntu?
There was a problem hiding this comment.
Yes, I'm not aware of any OS-specific issue
flferretti
force-pushed
the
fix_devcontainer
branch
from
52bb311 to
a85a1a9
Compare
flferretti
force-pushed
the
fix_devcontainer
branch
from
a85a1a9 to
a2a4474
Compare
|
It works also for me now! Thanks @flferretti ! |
This pull request updates the Devcontainer configuration by using
pixias the base image and installing additional extensions. The Dockerfile has been modified to pull thepixi.lockfrom LFS, create the environment and then use a shell-hook script to activate it. Moreover, thedevcontainer.jsonfile has been updated to set the default Python interpreter path and add some useful extensions to the VSCode environment.📚 Documentation preview 📚: https://jaxsim--276.org.readthedocs.build//276/