Releasing Origen Core

[coreyeng]

Hello,

I'm working on the release process for the core itself. I got it working well on Windows and moved on to trying to automate it so I can get a Linux build (more on this in a minute) but I can't trigger manual workflows on a branch. In retrospect, I could've temporarily made it on a push or something, but I the changes themselves are small and other than the update to Poetry 1.1.6, there's no actual changes to the core. See #137 for the "PR". I opened this issue here as a discussion on this. I'll keep working on the publish script on master so we can monitor the progress here.

I went a different direction with how @ginty I think was packaging originally. I looked up using maturin but it ultimately looked like we'd have three different packages: origen CLI, (which you'd download yourself, I guess?), origen, the Python frontend which would have a dependency on origen_pyapi, which was the built backend. Instead of this, I packaged all three together in the same release, with the caveat that the CLI is (potentially) masked by poetry. But, I actually think

1. this is good, and
2. this gets us closer to O1 actually.

Currently, I renamed the executable o2 but we can add a origen case too when we're ready to replace o1, or users can alias origen to o2 themselves.

I've only uploaded to the test server so far with the Windows build, and the publishing debug will try that first. But, you can do pip install -i https://test.pypi.org/simple/ origen and this'll pull down the wheel. The wheel includes all three components. So, origen, _origen, and the CLI (invoked via o2) are all available.

This looks like O1 now, where you just need to download Ruby and run gem install origen. Now, just download Python and run pip install origen to get the basics.

From there, you can run o2 env setup to do the rest. Now, the one caveat here is if you're using poetry in the apps, which you should be, o2 will still point to the global CLI installed from pip. But, poetry run o2 will run the CLI from whatever version the application wanted. And, a caveat to the caveat is if you run o2 globally through poetry, then the global CLI is lost within the application. But, this latter case matches O1, so, I don't think its a problem - just something to be aware of. Actually, we could likely add a case to the O2 CLI to check if its in an app and, if so, kick off the CLI from the app then exit. I believe the original proposal would have apps always starting from the global CLI.

This scheme seems to working alright on Windows 10 with Python 3.8, and based on the results here it looks like the actions servers managed to build everything, so I'll work on the release steps. My plan here is no publishing will actually take place locally and the publish command will be updated (for the python core only, so probably a new command) to instead just update the version/do all the boiler plate stuff, but kick off the publish action across the OS/Python version matrix to build each version and upload. The build step can still be done locally, but its just called as part of the publishing action, so it should be one-to-one, for the OS and python version under development.

A note on Linux - I tried this same scheme on WSL. It built alright but wouldn't let me install the built wheel. It kept saying it didn't support the version. But, not only is this WSL, but its WSL v1 (work computer can't go to WSL v2 yet) and I'm at least 75% sure that this, at least part of, the problem, as WS1 v1 support has been dropping all over the place. We can install other libraries though, so my plan is to focus on the automation and see if building on a real Linux system fixes this for WSL v1.

[coreyeng]

Small update: WSL v1 issues was due to an out-of-date PIP version. Bumped up the version and it worked fine. Had some other fixes, but things seem to be working well for me on Windows 10 and WSL v1, both with Python 3.8.

[coreyeng]

Hello. As I'm getting closer to getting the publishing stuff ready, I've got a few points we can discuss, as there'll be no going back for some of them:

1. Authors, @ginty, you and I are listed as the authors for various pieces. It's kinda funny as you will be "package author" now, but O2 docs page had me as the author - likely just due to whoever created the initial project ended up there. My work ID is also the author of the "meta" crate right now.

   Following some other projects, I'd recommend we replace the author with the Origen-SDK organization. I've seen some others, such as serde and pyo3 do this. I'm not an admin of the organization, so I can't see if we can create a visible team instead, but I think the organization should be fine. If I don't get any feedback, I plan to make this change

2. Right now the version starts at 2.0.0.dev4. Is this what we want the first release to be, or, since this is a new library in Rust/Python, do we want to kick the version down to 0.1.0.dev0 and start fresh? I'm 50/50 on this myself - I can see it either way. If I don't get any feedback, I'll leave it as is but if we want to make the change back to 0.1.0.dev0, now is the time.

3. I do not plan to release the CLI as its own Rust crate. This'll be a future item I think, but I do plan to build the CLI and add it to the releases/packages page, but we'll only have the origen-macro-helpers (replaces meta), and origen as the actual Rust crates. Let me know if you think the CLI should be its own package as well

[coreyeng]

Tagging some of you just to make sure you see this. :p

@ginty, @pderouen , @priyavadan, @chrisnappi @chrishume, @redxeth I think this is everyone that's been in the last few meetings we've had.

[pderouen]

Thanks @coreyeng, I don't have a strong opinion on any of your points for discussion. I am fine with moving forward with your defaults.

[ginty]

Hi @coreyeng, this all sounds good to me, sorry for not replying sooner.