Question about switch_collection

[muhammedabad]

Hi,

I'm quite new to mongodb and mongoengine and particular. We're looking at developing a multi tenanted django application that uses a single database, while giving each client their own collections for each of our models, eg:

client_1_products
client_1_orders

client_2_products
client_2_orders

What we would like to be able to do is easily change the collection name based on the client that using the app (each client would be assigned a collection prefix on registration). I have looked at the switch_collection() function, but I'm not sure if it's the best way to go about this. The site will be used by hundreds of people at the same time, so from a performance perspective I need to ensure that this does not impact negatively.

[touilleMan]

Hi,

As you guessed, you can use the context_manager to temporaly switch collection.

Considering the performances, you can have a look at the implementation (https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/context_managers.py), it's basically a set/reset of the Document's configuration, so it must be pretty cheap.

[samuelfekete]

Given that switch_collection is not thread safe #788, if multiple clients use it at the same time, this could result in data being written to the wrong collection.

[wangyushun]

Ran into the same issue today，i know switch_collection is not thread safe now ，but I still need it ,what can be done to avoid this problem?

[samuelfekete]

@wangyushun,
My approach was to stop using switch_collection and instead dynamically generate one class per collection using inheritance, and simply call the correct class by name.

See here for more details.

[wangyushun]

@samuelfekete Thanks,Your solution is perfect, but I'm not sure if it fits me.I have a Bucket table. I don't know how big the table will be in the future. Each bucket has its own collection. Dynamically create a separate class for each bucket, the dict will be large. besides, I need to remove separate class from or insert into the dict dynamically when user create or delete a bucket. I don't know if this will lead to other problems. I found PyMODM, and I'm learning about it. But now I have no other solution, I'll try your solution first. Thanks again for your reply.

[samuelfekete]

@wangyushun, if you think the number of collections will be very large, you could also consider creating the classes only when you need them.

[commissarster]

@samuelfekete Your solution is very good. The solution should submit to mongoengine and solved the bug.

[focusj]

Any progress on this issue @MongoEngine team?