pygtide.results() fails

[ahellers]

Hi,

first things first: I appreciate your worke since I was looking for a "Python version" of ETERNA/Predict for a longer time. Thanks!

However, I just encountered the following problem/bug when running the test after installing pygtide 0.7.1 via pip:

pygtide.test(msg=True)
ETERNA PREDICT v3.4 (10/02/2013) is calculating, please wait ...
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/hellerschmied/pyProjects/pygtide-test/env/lib/python3.9/site-packages/pygtide/tests.py", line 11, in test
    pt.results()
  File "/home/hellerschmied/pyProjects/pygtide-test/env/lib/python3.9/site-packages/pygtide/core.py", line 403, in results
    etdata[cols[2:]] = np.around(etpred_data[:, 2:], digits)
  File "/home/hellerschmied/pyProjects/pygtide-test/env/lib/python3.9/site-packages/pandas/core/frame.py", line 3946, in __setitem__
    self._setitem_frame(key, value)
  File "/home/hellerschmied/pyProjects/pygtide-test/env/lib/python3.9/site-packages/pandas/core/frame.py", line 4058, in _setitem_frame
    raise ValueError("Array conditional must be same shape as self")
ValueError: Array conditional must be same shape as self

I think, I found the problem in core.py line 388: cols = np.char.strip(etpred.inout.header.astype('str')):
etpred.inout.header.astype('str') returns a 6x25 array, although it should only be a one-dimensional array containing the 6 header names for the predict output. This array is used to create the DataFrame with more than 100 columns instead of 6 to store the predict results. In line 403 the program crashes when assigning 4 rows with results to the incorrectly initialized DataFrame.

pygtide.predict_series() seems to work fine as the results() method is not used.

I can reproduce this error with the gfortran compiler on Ubuntu 22.04 and CentOS 8 with Python3.9 and Python3.8. I don'T know if this issues depends on the gfortran compiler, or other parameters.

Does anyone encounter similar problems?

Best regards,
Andreas

[trichter]

Hi Andreas, I can reproduce your problem. I am using Kubuntu 22.04, conda python 3.10, system gfortran 11.3.0.

In [4]: etpred.inout.header.shape
Out[4]: (6, 25)

In [5]: etpred.inout.header
Out[5]: 
array([[b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'',
        b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b''],
       [b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'',
        b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b''],
       [b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'',
        b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b''],
       [b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'',
        b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b''],
       [b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'',
        b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b''],
       [b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'',
        b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'']],
      dtype='|S25')

I don't even get as far as you. Running the tests, my system errors out much earlier:

$ python -c "import pygtide; pygtide.test(msg=True)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/eule/dev/public/pygtide/pygtide/tests.py", line 8, in test
    pt = pygtide(msg)
  File "/home/eule/dev/public/pygtide/pygtide/core.py", line 110, in __init__
    self.fortran_version = etpred.inout.vers.astype(str)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 8: ordinal not in range(128)

Checking out etpred.inout.vers returns an array of 32 strings, instead of the single version string.

In [4]: etpred.inout.vers
Out[4]: 
array([b'ETERNA PREDICT v3.4 (10/02/2013)',
       b'\x00etpolut1.bin     \x00etpolut1.dat ',
       b'    \x00etddt.dat        \x00\x00\x0b',
       b'\x06\x00\x00\x00\x05\x00\x00\x00\xff\xe0\xf5\x058VL\x05\xf8\xc1c\x1a\xdc\xa5L@9\x9dR\xa2F\xdf\x91?',
       b'\x18-DT\xfb!\x19@\x18-DT\xfb!\t@0u\x00\x00U\x00\x00\x00\x01\x1b\x03;\xdc\x02',
       ... more binary data ...],
      dtype='|S32')

It appears this whole etpred.inout thingy is not working anymore with new systems.

Edit: The CI Ubuntu run demonstrates exactly this problem (log).
Also the MacOS run fails at the moment due to a compilation error (log):

   build/src.macosx-10.9-x86_64-3.10/pygtide/etpredmodule.c:2705:33: error: incompatible function pointer types assigning to 'f2py_init_func' (aka 'void (*)(int *, long *, void (*)(char *, long *), int *)') from 'void (*)(int *, int *, void (*)(char *, int *), int *)' [-Wincompatible-function-pointer-types]
    f2py_inout_def[i_f2py++].func = etpdata;
                                  ^ ~~~~~~~

@hydrogeoscience Still working for you?

[hydrogeoscience]

Thanks for the report. We are not far off from releasing a pure Python version of ETERNA PREDICT, so I am unwilling to spend energy looking into the many problems that my port still has ... stay tuned!

[trichter]

@ahellers Using the gfortran version from conda, I encounter exactly the same problem as you. I tried different gfortran versions -- still the same. Maybe it is working with lower numpy versions? Glad that at least pygtide.predict_series() is still functional.

[ahellers]

Thanks for your reply!

@hydrogeoscienc: I am very curious about the pure python version. Sounds good! Could you give us a rough estimate on when the release will be?

[hydrogeoscience]

Last I heard was August 2023. I will post it here when it has arrived.

[ahellers]

Last I heard was August 2023. I will post it here when it has arrived.

Hi again! Are there already any news on your plans for releasing a pure python version of Eterna's predict module?
Cheers!

[hydrogeoscience]

Great question! I'll let @HydroGeoSines answer this one ...

[ahellers]

Hi, any news?

[hydrogeoscience]

I wish ...
@HydroGeoSines