Organize issues & discussions across sub projects for priorization

[baentsch]

Beyond the current milestone tags for liboqs there does not seem to be an overall strategy (maybe only documentation) as to which issues and discussions shall be tackled in which priority/order, particularly if they straddle sub projects (such as issues regarding security, algorithm support or automation/CI). Some issues don't get attention for months and years, even.

If OQS were aiming to become more reliable software this should probably be changed.

This issue therefore is to suggest a discussion as to whether this is desirable and if so, how to facilitate that. Probably related to the question whether OQS want to attain some "product properties". But even if that question were answered with "No", this discussion may still be valuable as even researchers may want to have a mechanism to give priority to (resolving) some issues over/earlier than others.

[dstebila]

Yes, we should probably be more organized. There are a variety of Github features available to us: milestones, projects, labels, possibly more. We've experimented with those three over the years. Labels on issues/PRs are not super helpful because we still have lots of issues. Milestones have worked for our releases. I gave up on project boards because I found no one ever engaged with the project boards other than me.

All of that to say: I'm not sure how to organize things better, but if you or someone else has an idea on how, please suggest it or go ahead and try applying it and we'll see how it works.

[baentsch]

I gave up on project boards because I found no one ever engaged with the project boards other than me.

Just speaking for myself: I never understood how to use them (or they just didn't work the way I thought they should :-)

please suggest it or go ahead and try applying it and we'll see how it works

In the absence of other proposals, I'll try to learn some more about project boards and create one or two for this purpose...

Further proposals welcome, of course.

[planetf1]

The new project boards (beta) are useful as you can add issues from mutliple repos/orgs. Avoid the old ones. You can add additional fields, and views which can arrange tables or columns by these fields, + groupings etc. So you could have fields for the status (default), type or work, priority, impact.. whatever & then slice/dice in different ways. Simply having an unsorted column can be useful too as you can drag/drop to put items in order

All that being said, the key thing is to get buy-in. I've had similar issues with projects in the past in that it's difficult to get everyone happy with them. At least they are 'native' to github, and don't impact regular direct repo issue tracking. Let me know if I can help at all - but experimenting is a good start @baentsch

[baentsch]

it's difficult to get everyone happy with them

It's impossible to get everybody happy. Anywhere.

Let me know if I can help at all

Sure: What I'd hope for is a mechanism/view capturing all issues in all OQS sub projects in one view. Then be able to apply tags to them such as "algorithms", "infrastructure", "platform support", "production features", "experimental features", "CI", "release process", "security process" (for starters) and some sort of priority (say 1 for needs to be solved immediately by a maintainer to 10 may be solved by someone maybe eventually). "Slicing and dicing" then along these tags, priorities and sub projects to align them with a release plan. And if someone feels like adding the Linux Foundation "readiness" tags (i.e., issue needs to be done to achieve level x) to all issues, great -- if there'd be some alignment between issues and LF terms becoming visible, maybe that'd make me change my opinion towards their utility for OQS. --> Would you be willing to give this a try if you already know (how to use) that GH tool and explain it hands-on at a team meeting to ease the "learning hurdle"?

[planetf1]

@baentsch I can try to create an example board (ideally in this org). Maybe with a subset initially.

Some attributes of an issue come from the github issue itself. Tags for example are particularly useful as they're immediately visible and fully native to the issue. Opinions vary on how useful they are. Often they seem to become quite ad-hoc.

Other attributes get applied by the project - custom fields. This really helps with slicing/dicing in multiple dimensions. One thing I like too for experimentation is that assignment is only seen via the project and it doesn't cause lots of notification noise, especially when experimenting. Having too many dimensions/views can get everyone confused though....

I'll take this on to bring back to a future TSC meeting (do we have one scheduled?) or liboqs call.

[baentsch]

I'll take this on

Thanks!

to bring back to a future [...] meeting

Any chance to give this a try outside of meeting(s)? My effective work time is getting more and more out of sync with OQS meeting schedules (writing this comment before 0600 UTC :)

[planetf1]

Sure -today (everyday?) rather chaotic. We could do a webex/teams/zoom & try a first pass together if that helps & see if we can put something together to subsequently propose to the broader team? Generally mornings are best (as much work is with the US). Can arrange via email ?

[planetf1]

I see we have a project board for liboqs planning now - https://github.com/orgs/open-quantum-safe/projects/7/views/1

This could be used across projects, or a new board created.

[baentsch]

FWIW, doesn't seem to be supported in the github app :-(

[planetf1]

FWIW, doesn't seem to be supported in the github app :-(

Interesting.

I can access the project board through My Work->Projects->liboqs planning

However, if I go into Favourites->liboqs->projects I do not see it.

I'm using 1.175.0 on iOS.

I think the reason for this is that it isn't a project at the repo level, but rather at the organization level

As to how it got into Favourites, I really don't know, as there's no explicit action/star ??

[planetf1]

Ah - via Home->Organizations->Open Quantum Safe->Projects->liboqs planning

• shows up fine

[baentsch]

Thanks for giving it a try. Indeed that shows a listing of items, but continues to state "Das Boardlayout wird derzeit nicht unterstützt" (The board layout is current not supported). Looks like the "usual disadvantage" of us OSS users, then (I'm using Android (14), not iOS). But anyway, I'm now back at my laptop where all features are available -- just need to understand them :-/

[planetf1]

I think we might be able to usefully expand the board:

• Identify the most crucial areas we feel need contributions - Add priority? H/M/L ? Or perhaps it's 'next minor release', '1.0' 'future' ?
• Split the issues by type/skill - perhaps algorithms, ci/cd, security, test, performance, etc
• consider how to track towards a release (this imo is the hardest without getting into detailed iterations/dates which then just become unrealistic in an open source project IMO) .. or at least big initiatives
• extent to other repos (we could include all, and filter - easiest? - or just focus on top repos ie liboqs, oqs-provider)

Once the properties are setup, categorization is just drag/drop so it's low overhead. However it does require thought to classify, and is only useful if people use it ;-)

These could help others more easily look at where that help is needed, and how they could apply their skills (and of course in some cases their own/companies areas of interest). The ability to do this easily and quickly is important if they are contributing part-time and not necessarily involved in the regular status meetings.

We do have some 'good first issues' (more is better).. but this adds some more dimensions

[planetf1]

I'm happy to help with the setup .. but would need access...(I can't edit boards currently)

[dstebila]

Let's get you access! Can you create a PR in the tsc repo adding you with triage rights to liboqs?

[baentsch]

Let's get you access! Can you create a PR in the tsc repo adding you with triage rights to liboqs?

I'm all for getting things done. But didn't we just agree we don't handle permissions suitably controlled? We have a rather significant least-privilege violation in those permissions since weeks waiting for a TSC meeting to resolve them (and I'm not sure we did agree to). Wouldn't it be better we first get things right and then move on in a more orderly fashion? Also just fyi, we have Cross not landing since 2 weeks because of the unresolved permissions question (code ownership)...

[planetf1]

@baentsch I agree with this principle. Permissions can be set at the project level without requiring broader higher level access. However having checked it seems as if I can edit the liboqs project board. This is via the triage group (created/joined 3 months ago)

I could extend that with additional views/categories if that's ok with you and @dstebila. Alternatively we can have a new board.

[planetf1]

I started taking a look at the board. Apologies @baentsch for accidentally updating #1891 - I was using an iPad (ie touchscreen).
I’m going to instead work on a private board and will then share the categorizations before applying to a new view.. Reminder to oneself to not use touchscreen for these kanban boards. Too easy to update in error.

[planetf1]

@dstebila @baentsch I took a look at the current liboqs/oqs-provider/oqs-demos repos

Here's a sample project board: https://github.com/users/planetf1/projects/6. This is very short term experimentation after making a touchscreen related error on closing one issue last week on ipad!(apologies). Will be sticking to laptop/desktop/browser now. In addition changes may cause noise in the repo.

When trying to categorize by area I used

• Functionality (31)
• Algorithm (8)
• Build (51)
• Documentation (8)
• Process (4)
• Platform (4)
• Testing (12)
• demos (13)

It's not always entirely clear, but this gives an idea, and is a reminder we have a skew towards CI (this should be an area it's easier to get help?)

I noticed when looking through issues that some are waiting for information - so we could use a different dimension for that

Priority H/M/L is another aspect we could capture

Tags are another option - they work well within a board. Less well across repos. But across boards setting priorities is perhaps more relevant. Also having a kanban board with > 5 columns gets harder to navigate.

As far as planning a 'roadmap', there is the option to use dates or iterations (with fixed dates), but I've found these quite inflexible.

It may be easier to just have a new view with 'Current release / next release / future' or '0.11 / 0.12 / 1.0' etc?

I think discussion would be useful

• are these categories ok? Any changes?
• project board, tags, or both?
• Priority? Other dimensions

I'll reflect whatever we decide in the existing liboqs project board -

[baentsch]

Thanks for giving this a try -- it's not giving me very high confidence, though: I just re-opened one of the issues (#1891) you inadvertently closed and the Board still shows it as Done:

--> When does this view update?

Your categorizations do make sense to me. Now how can one prioritize stuff in this view?

Regarding tags: I wouldn't mind (very much to the opposite, actually) if we'd use the very same tags across sub projects such as to use them for slicing and dicing. Proposals welcome.

[planetf1]

@baentsch

• Thanks for the feedback on the categorizations. I too would also include as tags as it makes it easier to see in-repo, and agree that a common set across our repos makes sense
• I haven't added 'priority' yet. We would add an additional view (in fact there is one - Priority) and again define the categories such as H/M/L or Critical/High/Medium/Low (we could also add tags if we wished, perhaps priroity:high, priority:med - but there's no sync between tags and projects
• I can only apologise again for the inadvertant closure. I had thought that by using a private/personal board this wouldn't happen. Of course the workflow was in place:

We do want this workflow ultimately, but whilst experimenting I've turned it off, which should stop this from happening again.

Can you think of any other categorizations? For example status has Todo, In Progress, Done, Waiting (the last meaning blocked really). Makes sense or change

Happy to show this at the next dev meeting - unfortunately I wasn't able to make it yesterday.