Having events, as in gathering project members, fans or users either in a physical location somewhere or doing an online conference, is an awesome way to get concentrated and focused work and discussions. They can truly energize a project and its members. Having contributors meet in real life (less so over video) is also a good way for us to learn how we work and to see the real person behind all the digital communication which can greatly enhance project collaboration going forward.
Real world events are hard and time consuming to organize. A physical location means deciding a single place in the world for a project that could be truly global. Wherever you select to be, there will be people who cannot attend simply based on geography and contributors' different ability to spend time and money for this purpose. Many contributors spend time in the project in their spare time so they travel to these events privately, spending their own private money on it. The fortunate ones can do it on work time, paid for by employers.
Organizing a physical event also usually means that someone needs to spend money and sign papers to reserve the necessary space to use.
In smaller projects, you can assume that you as a maintainer will have to shoulder a huge part of the work and organizing of events.