How should engine extensions be tested in the future?

[jkrumbiegel]

#13718 moves the julia extension into a separate repo (which is supposed to be owned by PumasAI in the future, currently owned by @gordonwoodhull). What should development on that extension look like in the future in terms of CI? So far, PRs on the julia engine code would just trigger the normal quarto CI. What should a downstream test of quarto look like in the engine repo? It could present a lot of organizational overhead to keep the quarto tests themselves working in the extension repo if there's no super simple way of running them there while ensuring the same configuration of tools that quarto itself tests.

This also ties into the API question I posed here gordonwoodhull/quarto-julia-engine#1 as we are not sure right now what interface we can assume going forward between quarto and us.

CC @cscheid @MichaelHatherly

[gordonwoodhull]

I think right now the simplest thing to do would be to set up CI actions on quarto-cli that watch a branch or branches on your repo and run the current tests with your latest. You'd continue to PR to us for any changes to the tests, and watch our CI.

Going forward, we'd really like to have something like quarto call test foo.qmd that uses our smoke test infrastructure but through the packaged version of Quarto. Our current smoke tests run against the source / dev version, so the effort here is unknown (but maybe not too hard?)