Current Status of Project

[benjamin-kirkbride]

I recently discovered this project and the concept of it, as well as the problems that it solves, really resonates with me.

I'm curious what the status of the project is. I see that there is a known issues page in the docs, and that page has a link to a feature matrix, but the feature matrix seems outdated/incomplete.

I also have seen some posts from the original author on their blog and here stating discussing a waning interest in continued development. What my question really boils down to is, does this project work well with modern Postgres? Is progress being made to keep up with new PG features? Are people with write access to this repo using this in production right now?

I hope the above comes off in the spirit it is intended. The tooling honestly seems incredible, and I would love to start using it, it just seems like the project perhaps has lost some steam over the last few years and I'm interested in getting some perspective.

[jmafc]

With regard to the Feature Matrix, although the intro ought to be updated to reflect currently supported PG and Python versions, and we could add two additional columns in the matrix for Pyrseas 0.8/0.9, it is mostly a correct reflection of the features covered. The Changelog.rst file does indicate the feature additions since 0.7.

After the initial development, which was limited in scope, Pyrseas development was pulled and tugged in several different directions:

• At Peter Eisentraut's complaint that none of the PG schema control tools covered all database objects, one of the goals was to achieve almost full PG coverage (that was essentially achieved by 0.6/0.7).
• Reversion of changes was also initiated but didn't get very far.
• As a result of discussions on the post "Business Logic in the Database", the augmentdb tool was created with the assistance of Roger Hunwick (but that sort of stalled).
• I explored ways of implementing the parts of Andromeda that I thought were important, i.e., supporting the creation of more full-featured database administration apps and even full-stack database apps (what is discussed in "The Future of Pyrseas, Part 3"). Unfortunately, settling on a particular technology, be it Python, Node or whatever-based is quite difficult.
• A potential user in Spain led to the development of the facilities to copy in/out "static" data (sadly, his management decided to go with MSSQL instead of PG).
• The last major overhaul was undertaken mostly by Daniele Varrazzo @dvarrazzo who implemented a dependency graph for 0.8.

Since the re-architected dbtoyaml/yamltodb sufficed for my needs (in conjunction with Adminer for db admin/maintenance tasks), and there were no major feature requests, the project fell into a maintenance mode. The "Future of Pyrseas" posts were an attempt to encourage myself and others to move on but that didn't ignite any fuses, so two years later I publicly announced that we were "officially" in maintenance mode. However, I have continued to use it on a weekly/monthly basis and have responded to pull requests and issues. A year ago, I even consulted for a Google engineer who wanted to use Pyrseas for a a Udemy course on full-stack development. Earlier this year, I was seriously considering reimplementing Pyrseas using C++, but my current weighing of priorities has pushed that for at least six months.

[benjamin-kirkbride]

Thanks so much for the insight @jmafc!

With regard to the Feature Matrix, although the intro ought to be updated to reflect currently supported PG and Python versions, and we could add two additional columns in the matrix for Pyrseas 0.8/0.9, it is mostly a correct reflection of the features covered

So sounds like it is a good summary of what IS supported, but there isn't really any mention of what IS NOT (known) to be supported, is that correct?

At Peter Eisentraut's complaint that none of the PG schema control tools covered all database objects, one of the goals was to achieve almost full PG coverage (that was essentially achieved by 0.6/0.7).

Very impressive! I'm excited to put that to the test ;)

Reversion of changes was also initiated but didn't get very far.

Not sure what this means? Does this refer to using Pyrseas for migrations, or ?

A potential user in Spain led to the development of the facilities to copy in/out "static" data

This seems interesting but can't seem to find it in the docs, care to point it out?

sadly, his management decided to go with MSSQL instead of PG

Sad indeed 😁

in conjunction with Adminer for db admin/maintenance tasks

Complete tangent, but curious why not PGAdmin? Honestly didn't even know Adminer works with PH 😅

I have continued to use it on a weekly/monthly basis

Can you be more specific about how you are using it? Are you managing large/complex databases in production with it? Are you aware of anyone else who is?

Earlier this year, I was seriously considering reimplementing Pyrseas using C++

May I ask why you are considering doing this? I only ask because I am interested in using this and potentially contributing (if you are open to it :) ) but I don't know C++ 😅

Thanks again!

[jmafc]

So sounds like it is a good summary of what IS supported, but there isn't really any mention of what IS NOT (known) to be supported, is that correct?

Correct. I may be mistaken but I believe that at the coarse DDL statement level, Pyrseas does cover all the PG database objects. There are, of course, small holes in specific statements and sometimes larger deficiencies, e.g., handling of table partitioning, expression indexes.

Not sure what this means? Does this refer to using Pyrseas for migrations, or ?

Reversions refers to being able to produce both "DO" and "UNDO" scripts. Yes, when using Pyrseas to migrate a database, e.g., from the output of yamltodb , it would be useful to also produce a "revert" script that can be run, just in case, if when you apply the first script in production, something goes awry and you want to revert the database to the previous state. See the --revert option to yamltodb.

This seems interesting but can't seem to find it in the docs, care to point it out?

Under Configuration Items, see Datacopy.

Complete tangent, but curious why not PGAdmin? Honestly didn't even know Adminer works with PH

I'm not much of a fan of windowed GUI interfaces. I know pgAdmin now has a browser-based interface, but (a) I was already using Adminer (almost since the start of Pyrseas, I even wrote a post about it), and (b) it depends on a ton of both Python and JS stuff.

Can you be more specific about how you are using it? Are you managing large/complex databases in production with it? Are you aware of anyone else who is?

I basically use it in two ways. Prior to making changes, I run yamltodb against my development/testing/staging databases using the git-controlled YAML files to verify they're in sync (development has more, but I can handle that). If I have to add a brand new table or view, I typically create it and modify it "by hand" on psql and/or with SQL scripts and then run dbtoyaml -t new_table to produce a YAML file that I can add to VC. For changes to existing objects, I may modify a YAML file directly and use yamltodb to create a modification script. However, because existing tables invariably have data I may have to develop new or change the scripts to migrate the data concurrent with the changes.

My production database is relatively small and moderately complex.

I haven't really kept in touch with users except perhaps with @dvarrazzo. He undertook the dependency graph work because (I think) his company was going to use it but IIRC he later told me there were other priorities. The last person (that I recall) that had indicated use at some scale was in #184 (perhaps not in the issue but in correspondence).

Regarding C++, it's primarily because it's statically compiled.

[jmafc]

Are you aware of anyone else who is [managing large/complex databases in production with Pyrseas]?

@benjamin-kirkbride I just discovered the "Used by" feature (which apparently is new in GH). If you click on it, it says there are 11 "repositories that depend on pyrseas", although it only lists only six. Clicking on a repo and then clicking on its main requirements.txt file, if present, will show that some do depend on "pyrseas 0.9.1". Others perhaps depended on Pyrseas on earlier releases/other branches.

[dvarrazzo]

Regarding C++, it's primarily because it's statically compiled.

I do remember the representation too generic of database objects (just as a dictionary) to be a problem, but a possible solution could be more easily to use python classes and a static analyzer such as mypy. C++ is really a major chore to write and if you don't have performance reasons it is hardly convenient.

[jmafc]

I didn't want to get into a discussion of why I prefer C++ now, that's I went with the quick answer that it's a compiled language. Anyway, for most of my professional career I've programmed in C (and QUEL/SQL), but back in the late 80's I was exposed to Eiffel, even taking a short course from Bertrand Meyer himself (and unfortunately, he predisposed me against trying C++). By the time I started on Pyrseas, I had been exposed to PHP and of course to Python. After I did most of the work on Pyrseas, I took another look at Eiffel (discarded it because its compilation times were horrendous), D (because it supported contracts and was a "better C"), Go (because it was somewhat like Python), and even Rust (I don't get why it's deemed "most loved" in StackOverflow surveys), and of course I've been exposed to JS/TS/node/Angular, etc. And I came back to C++ because it is a better C and it's much improved over what people think of it in general. It's not just OO, it has lambdas, smart pointers, etc.

BTW, the representation of db objects in Pyrseas is with classes, not dictionaries. The containers for most objects are Python dictionaries because it's convenient to access them with a key when comparing between the "old" and "new" databases. If done with C++ I would still use map or unordered_map (i.e., associative arrays) for the containers. One advantage of C++ is that if I try to assign an element in a map

to a Function variable, the compiler will instantly tell me I've done something wrong.