Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data is not persistent (PB in run.sh) #4

Open
erambour opened this issue May 24, 2024 · 21 comments
Open

Data is not persistent (PB in run.sh) #4

erambour opened this issue May 24, 2024 · 21 comments

Comments

@erambour
Copy link

The way data is moved and linked, is only working once.

When I re-launch the docker-compose, my data is lost and I have to put new credentials
This seem to be because ln -sf can create a link to a removed (by mv) dir, but not to an existing one (it creates link in the existing directory). The result is that if data is effectively stored to an external volume, it's not mapped to the right place a the second run.

Explanation :

user@server:~/temp$ mkdir A; mkdir B; touch A/a; touch B/b
user@server:~/temp$ tree
.
├── A
│   └── a
└── B
    └── b

2 directories, 2 files
user@server:~/temp$ ln -sf A B
user@server:~/temp$ tree
.
├── A
│   └── a
└── B
    ├── A -> A
    └── b

2 directories, 3 files

That's not what we want, but if we remove the B directory the same command works as expected

user@server:~/temp$ rm -r B
user@server:~/temp$ ln -sf A B
user@server:~/temp$ tree
.
├── A
│   └── a
└── B -> A

I'v tried this modification, who partially solves the problem (user credentials are kept):

for path in $paths; do
    if [ ! -e /data/$path ]; then
        mkdir -p /data/$(dirname $path)
        if [ -e $path ]; then
            mv $path /data/$path
            echo "Moved ${path} to /data${path}."
        else
            mkdir -p /data/$path
            echo "Created ${path} in /data."
        fi
    else
         if [ -e $path ]; 
                then rm -r $path
         fi
    fi
    ln -sf /data$path $path
    echo "Created symlink for ${path}."
done

But the config folder ( I don't understand why config is stored in configS, and why the 'special case' ) if I apply the same procedure, Z-Wave is not working , and I've got a :
[I] [core] Notification: critical (z-wave): Cannot start Z-Wave binding: Error: Invalid config' error message.
But for a few apps ln -s to a directory just don't work, I've already had this problem.

@PoltoS
Copy link
Member

PoltoS commented May 24, 2024

Looks like we need to add just before this line

ln -sf /data/opt/z-way-server/configs/ /opt/z-way-server/
an rm -f $path. The first time this dir is moved (so not present), while the second time we need to remove it and replace with the symlink.

This rm should be places in two places (but in the second this is never a problem)

@PoltoS
Copy link
Member

PoltoS commented May 24, 2024

@msazanov Please fix.

The problem is that if the path is already present in the /data, the original folder is present (not moved), so the symlink goes into that folder

@PoltoS
Copy link
Member

PoltoS commented May 24, 2024

@erambour thanks for the finding!

@PoltoS
Copy link
Member

PoltoS commented May 24, 2024

Please try this:
#5

@erambour
Copy link
Author

erambour commented May 24, 2024

I've already added (see my post above) the following lines :

    else
         if [ -e $path ]; 
                then rm -r $path
         fi

in case of /data/ ... folder is existent.

That solves nearly everything, but I' was still facing a problem of invalid config / config not available for write

After a couple of modifications ( mapping a permanent storage to /opt/z-way-server/config : no change) and then chmoding 0777 all storage folder, the deleting and re-adding the z-wave and Zigbee apps (I own a z-station), it finally works (since a few minutes :-) .
The problem is that I don't really know what try did fix the last issue I had (no read/write access to config)

@robkamp
Copy link

robkamp commented Jul 29, 2024

My data is also not persisted during a restart. When is a new docker released that will fix this problem?

@robkamp
Copy link

robkamp commented Jul 29, 2024

In the logfile it says "Error: can not write back config to storage: Error: No such file or directory". It does not say which directory cannot be found. See
image

@PoltoS
Copy link
Member

PoltoS commented Jul 30, 2024

Have you tried #5 ? We just merged this fix. Please fetch the new docker files and rebuild the container

@robkamp
Copy link

robkamp commented Jul 30, 2024 via email

@PoltoS
Copy link
Member

PoltoS commented Jul 30, 2024

This will not work until we make a release. Can you test with the master meanwhile?

Here is the instruction:
https://github.com/Z-Wave-Me/docker-z-way/blob/PoltoS-data-persistance-fix/README.md

@robkamp
Copy link

robkamp commented Jul 31, 2024

During the docker compose build command an error is thrown:

[+] Building 0.6s (12/14)                                                                                                                                                                            docker:desktop-linux
 => [z-way-server internal] load build definition from Dockerfile                                                                                                                                                    0.0s
 => => transferring dockerfile: 744B                                                                                                                                                                                 0.0s
 => [z-way-server internal] load metadata for docker.io/library/ubuntu:focal                                                                                                                                         0.4s
 => [z-way-server internal] load .dockerignore                                                                                                                                                                       0.0s
 => => transferring context: 2B                                                                                                                                                                                      0.0s
 => [z-way-server  1/10] FROM docker.io/library/ubuntu:focal@sha256:0b897358ff6624825fb50d20ffb605ab0eaea77ced0adb8c6a4b756513dec6fc                                                                                 0.0s
 => [z-way-server internal] load build context                                                                                                                                                                       0.0s
 => => transferring context: 355B                                                                                                                                                                                    0.0s
 => CACHED [z-way-server  2/10] WORKDIR /opt/z-way-server                                                                                                                                                            0.0s
 => CACHED [z-way-server  3/10] RUN mkdir -p /etc/zbw/flags && touch /etc/zbw/flags/no_connection                                                                                                                    0.0s
 => CACHED [z-way-server  4/10] RUN apt-get update &&     apt-get install -qqy --no-install-recommends     ca-certificates curl     wget procps gpg iproute2 openssh-client openssh-server sudo logrotate            0.0s
 => CACHED [z-way-server  5/10] RUN wget -q -O - https://storage.z-wave.me/Z-Way-Install | bash                                                                                                                      0.0s
 => CACHED [z-way-server  6/10] RUN rm -f /opt/z-way-server/automation/storage/*                                                                                                                                     0.0s
 => CACHED [z-way-server  7/10] RUN rm /etc/zbw/flags/no_connection                                                                                                                                                  0.0s
 => ERROR [z-way-server  8/10] RUN echo "zbox" > /etc/z-way/box_type                                                                                                                                                 0.1s
------                                                                                                                                                                                                                    
 > [z-way-server  8/10] RUN echo "zbox" > /etc/z-way/box_type:
0.086 /bin/sh: 1: cannot create /etc/z-way/box_type: Directory nonexistent
------

@robkamp
Copy link

robkamp commented Jul 31, 2024

To mediate docker compose build error stated above I changed the Dockerfile and inserted a RUN mkdir -p /etc/z-way before the RUN echo "zbox" > /etc/z-way/box_type.

@PoltoS
Copy link
Member

PoltoS commented Jul 31, 2024

@msazanov do we miss this directory in Dockerfile or should be it created by Z-Way package?

@PoltoS
Copy link
Member

PoltoS commented Jul 31, 2024

@robkamp did the whole thing worked for you with your fix?

@robkamp
Copy link

robkamp commented Jul 31, 2024

I have not yet been able to deploy the new container to my TrueNAS.

@robkamp
Copy link

robkamp commented Aug 5, 2024

I just deployed a container built using the steps provided. I modified the Dockerfile so it contains my changes.

The state is now persisted between starts.

@PoltoS
Copy link
Member

PoltoS commented Aug 5, 2024

Thanks for your confirmation.

@msazanov Please fix our docker too.

@PoltoS PoltoS closed this as completed Aug 5, 2024
@PoltoS PoltoS reopened this Aug 5, 2024
@PoltoS
Copy link
Member

PoltoS commented Aug 7, 2024

@robkamp I'm confused, this folder is created during the installation of z-way-server folder. So this is a bit strange that it is not present in your case.

@PoltoS
Copy link
Member

PoltoS commented Aug 7, 2024

@robkamp Please confirm you were using aarch64 platform. Looks the problem is related only to this particular platform.

@robkamp
Copy link

robkamp commented Aug 7, 2024

My container says
# arch x86_64

@PoltoS
Copy link
Member

PoltoS commented Feb 13, 2025

Should be fixed with #12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants