config tool: not all repositories have a src directory

[jugmac00]

I am currently working on zc.zdaemonrecipe - it has no src but zc as top level source directory, which means, the generated flake8 call is wrong.

src:0:1: E902 FileNotFoundError: [Errno 2] No such file or directory: 'src'

I will change this manually for now - but every new run will break this repository again.

[dataflake]

I have noticed the same thing, there are some harcoded assumptions about a package's folder structure. It's always supposed to have a src folder as the root. That is not true for many Zope Products.

[mgedmin]

Can we instead agree to standardize on using the src/ layout?

[dataflake]

I am -0.5 on that. Still don't see the value. It adds yet more work.

[jamadden]

Strong +1 to src/.

[dataflake]

If other people want to go through and do the work of converting repositories, fine.

[mgedmin]

https://hynek.me/articles/testing-packaging/ describes the advantages of src/ layouts.

#31 is another example where lack of src/ makes things difficult for zope-testrunner.

---

Out of 390 repositories in https://github.com/zopefoundation 340 have a src/ directory at the top level. Another 32 have a top-level directory that matches the first part of the repository name until the dot:

bobo
BTrees
grokproject
js.codemirror
persistent
pyramid_zope_request
Products.CMFActionIcons
Products.CMFCalendar
Products.CMFCore
Products.CMFDefault
Products.CMFTopic
Products.CMFUid
Products.DCWorkflow
Products.ExternalEditor
Products.FSDump
Products.GenericSetup
Products.LongRequestLogger
Products.PluggableAuthService
Products.PluginRegistry
Products.SQLAlchemyDA
Products.ZNagios
transaction
z3c.evalexception
z3c.recipe.mkdir
z3c.soap
ZConfig
zc.recipe.filestorage
zc.zdaemonrecipe
zc.zope3recipes
zf.githubreceiver
zope2.sessioncookie
zope.fixers

There would be 33, but docs.zope.org/docs is a false positive.

[jugmac00]

@mgedmin At very least Products.CMCalendar is already archived - maybe more.

[mgedmin]

Yeah, I was checking with a shell oneliner that cannot distinguish archived repositores from other repositories

mg@blynas: ~/src/zopefoundation $ for i in *; do test -d $i/${i%%.*} && echo $i; done

[jugmac00]

current state of the src-ification

only active repositories, but with some "false-positives", like e.g. www.zope.org

❯ all-repos-list-repos --output-paths -C all-repos-zope.json | xargs --replace bash -c 'ls {}/src >& /dev/null || echo {}'
output_zope/zopefoundation/BTrees
output_zope/zopefoundation/persistent
output_zope/zopefoundation/ZConfig
output_zope/zopefoundation/zc.zdaemonrecipe
output_zope/zopefoundation/zc.recipe.filestorage
output_zope/zopefoundation/Products.SQLAlchemyDA
output_zope/zopefoundation/zopetoolkit
output_zope/zopefoundation/Products.PluginRegistry
output_zope/zopefoundation/zc.zope3recipes
output_zope/zopefoundation/Products.CMFDefault
output_zope/zopefoundation/Products.CMFUid
output_zope/zopefoundation/Products.ExternalEditor
output_zope/zopefoundation/www.zope.org
output_zope/zopefoundation/zpt-docs
output_zope/zopefoundation/zopefoundation.github.io
output_zope/zopefoundation/meta
output_zope/zopefoundation/website-zope.de
output_zope/zopefoundation/.github

[dataflake]

I'll look at the Products.XXX ones this coming week.

[jugmac00]

With obviously false-positive repos removed (website, docs-only-repos) we have the following repos not following the src convention:

❯ all-repos-list-repos --output-paths -C all-repos-zope.json | xargs --replace bash -c 'ls {}/src >& /dev/null || echo {}'
output_zope/zopefoundation/persistent
output_zope/zopefoundation/zope.publisher
output_zope/zopefoundation/ZConfig
output_zope/zopefoundation/zc.zdaemonrecipe
output_zope/zopefoundation/zopetoolkit
output_zope/zopefoundation/zc.zope3recipes
output_zope/zopefoundation/Products.CMFDefault

[dataflake]

Please disregard Products.CMFDefault if anyone is actively working on this task for the sprint or in the near future. I have a long-running branch in there for Python 3 compatibility work, and that branch has that restructuring already. I fear a restructuring on master will end in merge conflict hell once I merge my branch.

[icemac]

As the decision was that a src-directory is good and helpful and all zopefoundation packages should have one, I am closing this issue as there is no need to support packages without src directory.

[jugmac00]

FWIW - the following projects still do not have a src directory

❯ all-repos-list-repos --output-paths -C zope.json | xargs --replace bash -c 'ls {}/src >& /dev/null || echo {}'
output_zope/zopefoundation/persistent
output_zope/zopefoundation/zope.publisher
output_zope/zopefoundation/ZConfig
output_zope/zopefoundation/zc.zdaemonrecipe
output_zope/zopefoundation/zopetoolkit
output_zope/zopefoundation/zc.zope3recipes
output_zope/zopefoundation/Products.CMFDefault

[mgedmin]

zopetoolkit doesn't have any code, just a bunch of buildout configuration files.