Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PickleWarning thrown when pickling ArrayInstance #455

Closed
mmckerns opened this issue Feb 22, 2022 · 3 comments
Closed

PickleWarning thrown when pickling ArrayInstance #455

mmckerns opened this issue Feb 22, 2022 · 3 comments
Labels
Milestone

Comments

@mmckerns
Copy link
Member

mmckerns commented Feb 22, 2022

Warnings are thrown when pickling an ArrayInstance, however it appears that the ArrayInstance does pickle.

$ pypy test_objects.py 
/Users/mmckerns/lib/pypy-2.7/site-packages/dill-0.3.5.dev0-py2.7.egg/dill/_dill.py:1658: PicklingWarning: Cannot locate reference to <type 'ArrayInstance'>.
  warnings.warn('Cannot locate reference to %r.' % (obj,), PicklingWarning)
/Users/mmckerns/lib/pypy-2.7/site-packages/dill-0.3.5.dev0-py2.7.egg/dill/_dill.py:1660: PicklingWarning: Cannot pickle <type 'ArrayInstance'>: __builtin__.ArrayInstance has recursive self-references that trigger a RecursionError.
  warnings.warn('Cannot pickle %r: %s.%s has recursive self-references that trigger a RecursionError.' % (obj, obj.__module__, obj_name), PicklingWarning)
fails: CCharType <class 'ctypes.c_char'>
fails: CWCharType <class 'ctypes.c_wchar'>
fails: CByteType <class 'ctypes.c_byte'>
...

SEEN WITH:

  • pypy-2.7-v7.3.8, dill-0.3.5.dev0, numpy-1.16.6
  • pypy-3.7-v7.3.8, dill-0.3.5.dev0, numpy-1.19.5
  • pypy-3.8-v7.3.8, dill-0.3.5.dev0, numpy-1.19.5
  • pypy-3.9-v7.3.8, dill-0.3.5.dev0, numpy-1.19.5

NOT WITH:

  • pypy-3.6-v7.3.3, dill-0.3.5.dev0, numpy-1.19.5
  • python2.7, dill-0.3.5.dev0, numpy-1.16.6
  • python3.6, dill-0.3.5.dev0, numpy-1.19.5
  • python3.7, dill-0.3.5.dev0, numpy-1.21.5
  • python3.x, dill-0.3.5.dev0, numpy-1.22.2 (x=8,9,10)
@mmckerns
Copy link
Member Author

ping: @anivegesana
Consequence of changes in #443, and apparently updates to pypy from v7.3.7.

@anivegesana
Copy link
Contributor

anivegesana commented Feb 22, 2022

I believe that this is expected behavior. I just ran the following code snippet on dill 0.3.4 and it gives the following error:

import ctypes, dill
dill.dumps(ctypes.c_char(b'8'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ani/Git/dill/dill/_dill.py", line 304, in dumps
    dump(obj, file, protocol, byref, fmode, recurse, **kwds)#, strictio)
  File "/Users/ani/Git/dill/dill/_dill.py", line 276, in dump
    Pickler(file, protocol, **_kwds).dump(obj)
  File "/Users/ani/Git/dill/dill/_dill.py", line 498, in dump
    StockPickler.dump(self, obj)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 500, in dump
    self.save(obj)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 616, in save
    self.save_reduce(obj=obj, *rv)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 730, in save_reduce
    save(state)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 573, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/Users/ani/Git/dill/dill/_dill.py", line 990, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 985, in save_dict
    self._batch_setitems(obj.items())
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 1016, in _batch_setitems
    save(v)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 616, in save
    self.save_reduce(obj=obj, *rv)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 700, in save_reduce
    save(cls)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 573, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/Users/ani/Git/dill/dill/_dill.py", line 1439, in save_type
    StockPickler.save_global(pickler, obj, name=name)
  File "/Users/ani/pypy3.8-v7.3.7-osx64/lib/pypy3.8/pickle.py", line 1086, in save_global
    (obj, module_name, name)) from None
pickle.PicklingError: Can't pickle <class 'ArrayInstanceAutoFree'>: it's not found as builtins.ArrayInstanceAutoFree

I don't think pickling ctypes via dill is supported on pypy. Let me know if I am missing something.

@mmckerns
Copy link
Member Author

Ah... sorry my bad. I recently ended support for 3.6 (see #452), and thus is using an older build than the other pypy above. pypy3.6 fails, but doesn't throw the warning... but that's because it's testing against an older build. So, yes, this issue should be voided.

It's a bit weird to see that dumping a ctypes.c_char throws a warning about an ArrayInstance (and not the CCharType)... but I guess this is what it should do.

@mmckerns mmckerns added this to the dill-0.3.5 milestone Apr 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants