Error when using unique_with and inheritance

[latteier]

When upgrading to mongo 2.6 I get this error (which I don't get under mongo 2.4).

Trying to save an instance of my document (SubclassDoc):

  File "/Users/amos/Work/rfb-dev/eggs/mongoengine-0.8.7-py2.7.egg/mongoengine/document.py", line 294, in save
    raise OperationError(message % unicode(err))
OperationError: Could not save document (command SON([('createIndexes', u'feed'), ('indexes', [{'name': u'content_1_user_1', 'key': SON([('content', 1), ('user', 1)]), 'unique': True, 'background': False, 'sparse': True, 'dropDups': False}])]) failed: Index with name: content_1_user_1 already exists with different options)

I think the problem is that I have a base document class and a class that derives it that want to create different unique indexes on the same collection.

class FeedItem(Document):
    user = ReferenceField('User', required=True)
    content = ReferenceField('Content', required=True, unique_with='user')

class SubclassDoc(BaseDoc):
    pass

>>> BaseDoc._unique_with_indexes()
[{'fields': [('content', 1), ('user', 1)], 'unique': True, 'sparse': False}]
>>> SubclassDoc._unique_with_indexes()
[{'fields': [('content', 1), ('user', 1)], 'unique': True, 'sparse': True}]

I don't understand why this works on mongo 2.4 but not on mongo 2.6.

Thanks for any suggestions!

[lig]

Well. According to http://docs.mongodb.org/manual/release-notes/2.6/ "*
ensureIndex() now wraps a new createIndex command."

It looks like we might need mongodb 2.6 compatibility fix. Could you please
provide exact steps to reproduce this problem.

Thank you for your report!

On Mon, Apr 21, 2014 at 9:10 PM, Amos Latteier notifications@github.comwrote:

When upgrading to mongo 2.6 I get this error (which I don't get under
mongo 2.4).

Trying to save an instance of my document (SubclassDoc):

File
"/Users/amos/Work/rfb-dev/eggs/mongoengine-0.8.7-py2.7.egg/mongoengine/document.py",
line 294, in save
raise OperationError(message % unicode(err))
OperationError: Could not save document (command SON([('createIndexes',
u'feed'), ('indexes', [{'name': u'content_1_user_1', 'key':
SON([('content', 1), ('user', 1)]), 'unique': True, 'background': False,
'sparse': True, 'dropDups': False}])]) failed: Index with name:
content_1_user_1 already exists with different options)

I think the problem is that I have a base document class and a class that
derives it that want to create different unique indexes on the same
collection.

class FeedItem(Document):
user = ReferenceField('User', required=True)
content = ReferenceField('Content', required=True, unique_with='user')

class SubclassDoc(BaseDoc):
pass

BaseDoc._unique_with_indexes()
[{'fields': [('content', 1), ('user', 1)], 'unique': True, 'sparse':
False}]
SubclassDoc._unique_with_indexes()
[{'fields': [('content', 1), ('user', 1)], 'unique': True, 'sparse': True}]

I don't understand why this works on mongo 2.4 but not on mongo 2.6.

Thanks for any suggestions!

Reply to this email directly or view it on GitHubhttps://github.com//issues/629
.

Serge Matveenko
mailto: serge@matveenko.ru
github: http://lnkfy.com/1
linkedin: http://lnkfy.com/S

ðÁÒÏÄÕÊ.ÒÆ -- http://www.parodui.ru/
óÐÅÃÉÁÌÉÚÉÒÏ×ÁÎÎÙÊ ÍÁÇÁÚÉÎ ÜÌÅËÔÒÏÎÎÙÈ ÓÉÇÁÒÅÔ
óðÂ, ìÅÎÉÎÓËÉÊ ÐÒÏÓÐÅËÔ 79 ËÏÒÐ. 3, +7 (812) 951-25-24

[latteier]

I'm having trouble coming up with a small working example that reproduces the error. But I'll keep trying.

[latteier]

Here's an example that demonstrates the bug:

import sys
from mongoengine import (
    Document,
    StringField,
    connect,
)


class BaseDoc(Document):
    one = StringField(required=True)
    two = StringField(required=True, unique_with='one')
    meta = {
        'allow_inheritance': True
        }


class ChildDoc(BaseDoc):
    pass


if __name__ == '__main__':
    connect('example')
    if sys.argv[1] == 'child':
        ChildDoc.ensure_indexes()
    elif sys.argv[1] == 'base':
        BaseDoc.ensure_indexes()

Run the program twice:

$ example.py base
$ example.py child

The second time you run it you will get an error

pymongo.errors.OperationFailure: command SON([('createIndexes', u'base_doc'), ('indexes', [{'name': u'two_1_one_1', 'key': SON([('two', 1), ('one', 1)]), 'unique': True, 'background': False, 'sparse': True, 'dropDups': False}])]) failed: Index with name: two_1_one_1 already exists with different options

[latteier]

Alternately you can create instances of the document classes rather than call ensure_index() on the classes to trigger the bug.

if __name__ == '__main__':
    connect('example')
    if sys.argv[1] == 'child':
        ChildDoc(one='one', two='one').save()
    elif sys.argv[1] == 'base':
        BaseDoc(one='one', two='two').save()

[wojcikstefan]

+1, would be great to make MongoEngine compatible with Mongo 2.6 asap (will try contributing some fixes soon, too).

[keithhackbarth]

+1, in general it seems like attributes like dropDups, sparse, etc. should be index level attributes and not class level.

[robromano]

+1, would be great to get MongoDB 2.6 support.

[DavidBord]

This is how the index looks like for the BaseDoc:

{
  "v": 1,
  "unique": true,
  "key": {
    "two": 1,
    "one": 1
  },
  "name": "two_1_one_1",
  "ns": "mongoenginetest.base_doc",
  "background": false,
  "sparse": false,
  "dropDups": false
}

And this is how it looks like for the ChildDoc:

{
  "v": 1,
  "unique": true,
  "key": {
    "two": 1,
    "one": 1
  },
  "name": "two_1_one_1",
  "ns": "mongoenginetest.base_doc",
  "background": false,
  "sparse": true,
  "dropDups": false
}

The difference is that sparse option is changed from false to true, hence the exception

[wojcikstefan]

I'd assume MongoDB 2.6 introduced a more thorough ensureIndex validation, now that ensureIndex wraps a new createIndex command (http://docs.mongodb.org/manual/release-notes/2.6/#index-build-enhancements).

Anyway, you should fix your code to not generate an index on the same fields, but with different options (i.e. with the sparse discrepancy).

This issue should probably be closed @DavidBord

[MRigal]

Closing the issue as it won't get fixed, here indeed, the code should be fixed

[iBelieve]

@wojcikstefan @MRigal could you explain what you mean by the code should be fixed? I'm hitting this same issue, and I don't see anything wrong with the sample code in this issue. Is this a MongoEngine bug? If not, what needs to be done in the example model code to get this to work?

[mattgreen]

@ibeliever I was able to get this to work by moving the index definition out of the field's unique_with parameter and into the meta class attribute like so:

meta = {
    'allow_inheritance': True,
    'indexes': [{
        'fields': ['field1', 'field2'],
        'unique': True
    }]
}

This was on the base class.

[Greyvend]

@mattgreen thanks, it worked for me. However, it looks like a workaround. I also don't understand why the issue was closed and why this bug isn't wanted to be fixed.