Confusing behaviour when ListFields are set to empty list

[gareth-lloyd]

import mongoengine
class TestDoc(mongoengine.Document):
    list_field = mongoengine.ListField()

mongoengine.connect('test')
test_doc = TestDoc.objects.create()

After this operation, in spite of the fact that list_field was not given any value, the BSON document in the collection has a corresponding field initialized to an empty list:

cluster-1:PRIMARY> use test
switched to db test
cluster-1:PRIMARY> db.test_doc.find()                                                                                              
{ "_id" : ObjectId("515c24a4c02c1c58d59f02c0"), "_types" : [ "TestDoc" ], "list_field" : [ ], "_cls" : "TestDoc" }   

I understand why initializing to empty list could be a good idea - it allows subsequent list operations on that field to succeed. However, in the light of this, the following behaviour seems wrong to me:

test_doc.list_field = []
test_doc.save()

This call causes mongoengine to issue an $unset command, removing list_field from the BSON document:

cluster-1:PRIMARY> db.test_doc.find()                                                                                              
{ "_id" : ObjectId("515c24a4c02c1c58d59f02c0"), "_types" : [ "TestDoc" ], "_cls" : "TestDoc" }

So ListFields, unlike other field types, get initialized to empty lists when no value is given, but setting a ListField to empty list causes it to be removed from the underlying BSON document.

[tarass]

I would much rather lists not be auto serialized as empties into the database. Though, it is a bit of a consistency issue with how mongo handles lists, especially as they apply to indexes.
Currently mongo treats empty array as A value and adds it to even a sparse index. You don't need an array to exist before you $push to an array, but $pull leaves an empty array in place.

That said I would still rather have a non-existant entry until content is added.

[rozza]

This issue seesaws alot - once 0.8 is out I will review for 0.9

[mmellison]

I would also like to see this changed to be more consistent. There are situations where I am removing entries from a list, but require an empty list to be kept in the database.

As of right now, I have to detect if I am removing the last element from the list and if so, use the update method to set the field as an empty list.

[MRigal]

@seglberg I'll remove the 0.10 milestone on this one, when you have time, you're very welcome to submit something on the topic

[MRigal]

A similar case is also the issue #938

[wojcikstefan]

This is not an issue anymore - test_doc.list_field = [] doesn't unset the list in the database:

In [3]: class Doc(Document):
        nums = ListField(IntField())
   ...:

In [4]: doc = Doc.objects.create()

In [5]: Doc._get_collection().find_one({ '_id': doc.pk })
Out[5]: {u'_id': ObjectId('58448163c6e47b87154ecc6c'), u'nums': []}

In [6]: doc.nums = []

In [7]: doc.save()
Out[7]: <Doc: Doc object>

In [8]: Doc._get_collection().find_one({ '_id': doc.pk })
Out[8]: {u'_id': ObjectId('58448163c6e47b87154ecc6c'), u'nums': []}

[andrewsosa]

Hello! Is this fix included in release v0.11.0? I'm still having this issue on the latest release, also with flask-mongoengine 0.9.2. Thanks!

I'm using a ListField of ReferenceFields.