feat: OpenTofu support

[meringu]

what

This is a change to get OpenTofu to work with Atlantis.

I'm just starting to test this internally and have made some decisions in order to get it working:

• Introduces a --tf-distribution setting that can be set to terraform or opentofu.
• OpenTofu is downloaded via tofudl.
• Added an interface to server/core/terraform package for the Terraform distribution. This matches up with the --tf-distribution setting.
• I would like to also support configuring Terraform or OpenTofu per project in the server side config or repo level config. This would support an easier path to migrate between the two.

why

#3741

tests

• Have refactored out a distribution and made changes to the downloaders for both Terraform and Conftest to be more testable
• I've deployed this branch within my org, and have got successful plan and applies working

references

#3741

[james0209]

I agree with the premise of this, and I like the solution, but I don't agree that it stops the need for hc-install - I think that is worthy of a discussion.

I do not think your changes inherently conflict with the hc-install PR - it is very easy to do this for example

func (c *DefaultClient) DetectVersion(log logging.SimpleLogging, projectDirectory string) *version.Version {
	return c.distribution.DetectVersion(log, c, projectDirectory)
}

type Distribution interface {
	BinName() string
	SourceURL(v *version.Version, downloadURL string) string
	DetectVersion(log logging.SimpleLogging, c *DefaultClient, projectDirectory string) *version.Version
}

func (*DistributionTerraform) DetectVersion(log logging.SimpleLogging, c DefaultClient, projectDirectory string) *version.Version {
// use hc-install
}

func (*DistributionOpenTofu) ListAvailableVersions(log logging.SimpleLogging, downloadBaseURL string, downloadAllowed bool) ([]string, error) {
// use current implementation - only called by the DetectVersion below
}

func (dt *DistributionOpenTofu) DetectVersion(log logging.SimpleLogging, c DefaultClient, projectDirectory string) *version.Version {
// use current implementation
e.g. tfVersions, err := dt.ListAvailableVersions(log, c.downloadBaseURL, c.downloadAllowed)
}

Especially as the ListAvailableVersions func is only called in the current implementation of DetectVersion - there is no other usage.

I think if you are going to have separate interfaces for OpenTofu vs Terraform implementations, then a discussion on whether to use the same logic vs different logic for each Distribution should be had.

• An example being how this PR already introduces 2 different logic flows for ListAvailableVersions (due to tagging, pre-relese etc.) - with hc-install, that would only be needed by OpenTofu, as it is handled by hc-install for Terraform

[meringu]

Thanks @james0209, that makes sense.

I'd be happy to rebase this over the hc-install change. As you point out, that should be fairly straightforward.

[nitrocode]

Due to wip status, I switched this to a draft as a non draft means that it's ready for review. Please set it as ready to review when ready.

Thank you for the contribution

[chenrui333]

I'd be happy to rebase this over the hc-install change. As you point out, that should be fairly straightforward.

we have merged hc-install update, might worth do the rebase and pick up this PR again. Thanks for all the efforts!

[jamengual]

The OpenTofu team is going to release this downloader to make it easier for us to auto download Opentofu.

https://github.com/janosdebugs/downloader
You can pull it in with a replace statement in go.mod for now until is released today or Monday.

@meringu @nitrocode if any of you want to work on this let us know

[meringu]

Awesome, I rebased this from main yesterday and got to the part where we need to download tofu.

I'm keen to keep working on this, but happy to have it taken over if I'm being too slow.

[meringu]

Tofu downloader has been moved: https://github.com/opentofu/tofudl

[meringu]

Thanks for your patience. I've done the following:

• Merged from main to get the hc-install changes.
• Updated to use tofu downloader.
• Refactored out a distribution and downloader.
• Made a separate downloader for conftest to decouple it from the TF downloaders.

I've done some testing on one of my Atlantis servers. I've been able to do successful plan and applies. Have also tested resolving different version from constraints from the required_version setting.

[X-Guardian]

Hi @meringu, it looks there are a lot of spurious changes included in this PR, .gitignore, dockerfile, a bunch of mock files. Can you remove the ones that are not relevant to this PR. Thanks!

[meringu]

Thanks @X-Guardian, I have cherry picked out a new commit, and only run go generate on the packages I modified.

[meringu]

Have updated the PR description to reflect the changes. The PR is labeled as needs-discussion and waiting-on-response, so please let me know how I can help.

[jamengual]

@meringu, can you resolve the conflicts? If it is ready for review, can you please change it to non-draft? Thanks.

[GenPage]

Thank you for taking the time to implement this feature. A few comments before approving.

[GenPage]

What is the rationale behind ignoring this for opentofu? Does this mean we always download the binary?

[meringu]

I'm not sure exaclty how to support this flag with opentofu. I'm open to ideas.

Tofudl supports two fields to download from a mirror: https://github.com/opentofu/tofudl/blob/76c6857d6eac665824b29da4e893e06e0b1896b2/branding/branding.go#L10-L14. I think we'd need to add an additional flag at the very least? I'm also not sure how specific this setting is to tofudl, that might be an issue if we change download method.

[jamengual]

@meringu we can maybe ask on the opentofu slack or an issue to tofudl if neccesary

[meringu]

Great idea, I have asked over here: https://opentofucommunity.slack.com/archives/C05RKE7PS72/p1726523077638359

[meringu]

The thinking is that we could look at supporting this later once someone needs it and has defined use-case. For now, I've updated the wording to indicate that it is not yet supported, and can unblock us getting this in for others to start to try out OpenTofu without a mirror.

[kvendingoldo]

In https://github.com/tofuutils/tenv we support mirrors for OpenTofu as well as for Terraform.
You can check our documentation about the package (https://github.com/tofuutils/tenv/blob/main/TENV_AS_LIB.md), not about the tool