How do we make & keep python-can maintainable?

[felixdivo]

Here, the discussion on how to keep or even just make python-can more maintainable was raised the last time. Let's continue here.

[felixdivo]

@zariiii9003 You proposed having more plugins. Is there currently even any documentation on how to do this? I could only find the note at the bottom of this page of the docs.

cc @hardbyte

[zariiii9003]

I added more information about it in develop here and here

[j-c-cook]

Orthogonal to that, it might be a good idea to reconsider which interfaces (+ file types) we want to support? Do we want to add any interface even if only 5 people will ever use it? Or do we want some relevance of the interface?

We should start telling people to use the plugin interface/entry point. If we merge all the interfaces, then this lib will become an unmaintainable mess. We'll never be able to make API changes if we have to update 30 interfaces without access to the hardware.

This package provides an individual or company with the ability to interface with a CAN bus at a comparatively low cost. With that, I'm going to second @zariiii9003, that lesser known interfaces should not be included in the package (primarily for the sake of maintainability), but rather up to the individual to utilize python-can as a dependency. However, is the plugin interface/entry point adequately polymorphic where a user could use python-can as a dependency in a package and then add in their own interface? I think placing focus on providing that kind of support is a better use of time than trying to get every possible interface into this package. At that point, is there a way that we can have a quantitative thresh-hold that causes a new interface to be included?

[felixdivo]

I'm also +1 for making more use of the plugin API. However, this might slow adoption of this library since it is possible that some users find maintaining a separate package cumbersome. But I'd say: then the interface is probably not important enough.

[felixdivo]

We could have similar discussion on file I/O (see here).

We could also extend the discussion to file I/O, but there I see less of a problem. These code parts are most often simpler [than interfaces] (less concurrency, no linking to binary code) and most of the time easy to test. It is therefore easier to maintain.

@zariiii9003 added:

Just for completeness, we do have entry points for io, but the documentation is lacking there: #783

[felixdivo]

At that point, is there a way that we can have a quantitative thresh-hold that causes a new interface to be included?

We could ask the community to vote on an issue/PR or comment what they would use it for? Maybe there is too little interaction though to get this to work well.

[j-c-cook]

Perhaps if a package that implements a plugin gains enough traction we could add a documentation page containing "external plugin/interface" packages that are compatible with python-can or something of that sort. So then maybe the interface modules documentation could appear as internal (the ones listed now) and then external (located in a 3rd-party package and outside of this packages maintainability).

[felixdivo]

Sure, we should definitely add it to the python-can docs. But also point to the respective external repos for docs & issues to make it clear that others are responsible.

I wonder whether we should provide some template repo to get people started. It would ideally have the plugin connection for a simple file I/O reader/writer pair and a dummy interface set up and maybe a trimmed version of our CI pipeline. That way, people would probably be more willing to maintain their own variants. It can't be too hard to set up (by mostly copying), can it? Maybe also include a link/code snippet on how to upload to PyP and what to do to include it in the mainline (python-can) docs.

[felixdivo]

Slight tangent: Do we need to keep the mailing list? It seems like having an additional platform doesn't simplify things, and if discussion take place on GitHub maybe others will join in and assist. Maybe it's because I'm rather young, but I don't see the benefit of the mailing list (at all).

[hardbyte]

Having just gone through the docs for all the interfaces I'm in favor of removing the lower quality and/or lesser used ones. I don't see a problem pointing people to external plugins (similar to the external virtual interfaces). A template repo or cookie cutter recipe sounds like a very good idea to support that.

Mailing list... I'm also not a fan. I might just remove reference from the readme while I'm editing docs.

I've only briefly looked at GitHub's support for discussions - we could enable that and give it a go?

[hardbyte]

One other thing that might help with maintenance of the existing interfaces and custom io would be adding a CODE OWNERS file so the original contributor gets notified/review requests for subsequent changes.

[felixdivo]

Mailing list... I'm also not a fan. I might just remove reference from the readme while I'm editing docs.

Sounds good. I'd also rename it (if that is possible) to something like "python-can (archive only, abandoned)" and maybe we should write a "last" email to tell people not not expect active discussions in there any more.

I've only briefly looked at GitHub's support for discussions - we could enable that and give it a go?

I don't have experience on that, but I personally am fine with having everything be an issue, possibly tagged as question. But I'm open to test discussion, I don't have strong feelings here.

[zariiii9003]

I would prefer using Github discussions, so the issues are only used for actual bug reports.

+1 for the template repo with Bus and IO-Plugins.

The CODE OWNERS file would be great, but it only works for users with write access to the repo, right?

[hardbyte]

Enabled! Let the test discussion commence

[j-c-cook]

Nice. I like the idea of discussions to reduce the issue section to be categorically implementation (new features, bug fixes etc.) focused.

[zariiii9003]

There should be an announcement in the mailing list for this

[zariiii9003]

cc @christiansandberg @pierreluctg @hartkopp

[felixdivo]

The CODE OWNERS file would be great, but it only works for users with write access to the repo, right?

"The people you choose as code owners must have write permissions for the repository."

[j-c-cook]

I wonder whether we should provide some template repo to get people started. It would ideally have the plugin connection for a simple file I/O reader/writer pair and a dummy interface set up and maybe a trimmed version of our CI pipeline.

+1 for the template repo with Bus and IO-Plugins.

+1

[hartkopp]

cc @christiansandberg @pierreluctg @hartkopp

:-)

I really feel honored! But in fact I need to learn and practice Python first before I can feel a 'code owner'.
At least I can try to continue to provide some background information on SocketCAN and other stuff I hopefully can make some contributions to.

[hartkopp]

We could have similar discussion on file I/O (see here).

We could also extend the discussion to file I/O, but there I see less of a problem. These code parts are most often simpler [than interfaces] (less concurrency, no linking to binary code) and most of the time easy to test. It is therefore easier to maintain.

@zariiii9003 added:

Just for completeness, we do have entry points for io, but the documentation is lacking there: #783

I'm not really sure how you plan to split up the project or not.
Would it then end in

• python_can (containing the main functionalities/APIs analogue to the Linux network layer abstraction)
• python_can_drivers (containing the different CAN drivers with a common interface to python_can)
• python_can_io (everything about I/O and tools)

??

Especially when there would be a proper (plug-in?) interface for python_can_drivers the users can decide to take the entire python_can_drivers package OR they could create their own CAN driver using the (plug-in?) interface without pushing the python_can_drivers maintainers to get every driver directly into the mainline repository.

Or am I on the wrong page?

[zariiii9003]

@hartkopp

python-can offers "entry points" where other packages can register their own classes. See some example packages here, here or here.

We're not talking about splitting the library. However, we might need to talk about moving some unused/unmaintained interfaces outside of python-can.

[felixdivo]

@hartkopp There's no certificate needed to be a code owner. ;) You already have the expertise to help.

To be clear: The idea is not to split the library entirely. Instead commonly used (=important) and well maintained interfaces/file type support stays in the core library in this repository. But other less mature or seldomly used functionality will reside in extra repositories to not carry the maintenance burden here.

[hartkopp]

@zariiii9003 @felixdivo
Thanks for the clarification! I obviously need to take a break and dive into the referenced examples ;-)

[zariiii9003]

Here's some data from the rtd export

This does not necessarily correlate with the actual number of users. Some interface are easier to use than others, so the docs are probably visited less frequently.
Also, some of the interfaces were only recently added to the documentation.

[felixdivo]

Very interesting!

[felixdivo]

People still visit the Version 1-3 documentation? 👀

[felixdivo]

Just two notes on this:

• Here, we already mention external tools.
• Also, there's Add documentation page on related tools #1041.

[zariiii9003]

What do you think about using towncrier and forcing all PRs to have a news fragment? With towncrier we wouldn't need to search for changes and create a changelog manually. Maybe it would help with creating spontaneous releases.

[felixdivo]

I think this is a good idea! Automating that should pay off rather quickly.

[zariiii9003]

@hardbyte Should we remove old pre-releases from the github releases page?

[felixdivo]

What would be the benefit of that?

[hardbyte]

Yeah I don't see any problem leaving the pre-releases on GitHub and also on PyPi.

[zariiii9003]

I thought they have no use once the final versions are released.
I'd prefer seeing 4.1.0 -> 4.0.0 -> 3.3.4 and so on, instead of five different versions of 4.0.0.
But i don't mind keeping them 😄