Is this repo still maintained?

[johnnyshields]

@den-stripe @brandur-stripe

[brandur-stripe]

@johnnyshields I'm not sure that there's an official stance on it.

It hasn't seen a push in 2+ years and the top two contributors are no longer with the company, so that doesn't bode well. On the other hand, it still does appear to be in our gem stack. That said, I wouldn't be too confident of responsiveness of issues/PR reviews.

[johnnyshields]

Would it be possible to review/merge the existing PRs, which are a straightforward compatibility update for the new version of the Mongo library?

[brandur-stripe]

Maybe ... I usually don't mind doing this sort of stuff, but I don't think I'm the right point of contact here given that I've never looked at this code before and am by no means well-versed in Mongo either.

I just filed an internal ticket with our Storage team to see if anyone's willing to take a look at these.

Regardless, thanks for the contribution! Sorry that the repo's status has been left in a nebulous state.

[brandur-stripe]

Okay, bad news I'm afraid: I spoke to our internal Storage team and they mentioned that although as noted above, Mongoriver is still currently in our stack, it's well on its way out at this point.

They mentioned that a more scalable tailing solution is Mongo's Kafka connector, and that's the direction all our internal architecture is moving. You may have considered that already, but it's worth a look into in case you hadn't.

Otherwise, the best recommendation I can give you is to go for a fork of the repository with some of the outstanding patches applied. I don't think we'll be able to change ownership of the repository, but if a reliably maintained alternative appears, we can link to it from the README.

I've updated the README in cd79fec with a note on the repository's unmaintained status.

[johnnyshields]

Had a look at Kafka connector. It looks to be a very generic/unopinionated way to stream documents from mongodb to kafka. Can see it working in particular use cases however here I'd to have more fine-grained control.

We will maintain a fork under the tablecheck organization. If it would be possible to consider allowing us to publish new versions of the gem (1.0.0+) that would be a benefit to the community.

[johnnyshields]

And thank you very much for taking the time to investigate all this!

[brandur-stripe]

Had a look at Kafka connector. It looks to be a very generic/unopinionated way to stream documents from mongodb to kafka. Can see it working in particular use cases however here I'd to have more fine-grained control.

I'm not sure what I'm about to say is accurate, but I recall hearing a few years ago that one of the reason we were trying to get off of direct oplog use is that when you have N replica sets that want to be tailed by M different services, you eventually start running into read amplification problems in Mongo as you're aiming to have N * M oplog tailing connections.

Kafka can help in this area with scalability in that it's a system that's specifically designed to have a stream read by a large number of different consumers, and does that job quite well.

We will maintain a fork under the tablecheck organization. If it would be possible to consider allowing us to publish new versions of the gem (1.0.0+) that would be a benefit to the community.

I could route around internally a bit, but after what happened with the event-stream package earlier this year (see NPM's post about it), I suspect our Security team wouldn't be too happy with this. Although you'd almost certainly do the right thing, the package was previously published by us and therefore implicitly given credibility by our name. For better or for worse, these days our tolerance for risk in this sort of thing is extremely low.

[johnnyshields]

@brandur-stripe thanks again for commenting. I've done a bit more digging and indeed it seems the best way will be to use the Change Streams feature with Ruby, which will make this library unnessecary. There's example code here: https://docs.mongodb.com/ruby-driver/master/tutorials/ruby-driver-change-streams/

So, no need for gem ownership change, we will manage on our own. Thanks!

[brandur-stripe]

@johnnyshields No worries! Yep, those change streams look pretty promising — modern implementation and officially supported, which is great.

So, no need for gem ownership change, we will manage on our own. Thanks!

Excellent. Thanks for following up!