Document provider layering, with impl guides #16630
Conversation
Today we do not really explain the layering of our various means of writing providers, which as we've seen, can lead to confusion. This does a pass over the provider authoring docs, explaining the architecture, and why only Go has a higher level SDK. It also adds reference material and implementation guides for those who want to code to the direct gRPC/Protobuf interfaces directly, in Python, Go, or TypeScript. (These were tested locally.) I am not 100% convinced we should add this but for power users wanting to really understand how providers work and how to author them, it would seem helpful in demystifying some confusing aspects of our ecosystem.
|
Your site preview for commit 728cffd is ready! 🎉 http://www-testing-pulumi-docs-origin-pr-16630-728cffd4.s3-website.us-west-2.amazonaws.com. |
justinvp
left a comment
justinvp
left a comment
There was a problem hiding this comment.
I like it! Some initial comments. I haven't looked at the python or typescript pages yet.
|
|
||
| Many teams prefer to write providers in their standard language, giving them access to familiar tools and libraries. However, Go has a significant advantage: providers written in Go compile to standalone binaries with no runtime dependencies. Users of your provider can consume it from any Pulumi language without installing Go. | ||
|
|
||
| Providers written in Python or TypeScript require their respective runtimes to be installed, which adds friction for users. That said, if your team is more productive in Python or TypeScript, the familiar ecosystem and libraries may outweigh the runtime dependency—especially for internal providers where you control the deployment environment. |
There was a problem hiding this comment.
Not sure it's worth mentioning, but it's possible to create a Python or TypeScript provider that's wrapped in a native binary. Not a very common thing to do, but it's possible.
There was a problem hiding this comment.
What do recommend for doing that?
|
|
||
| ## Prerequisites | ||
|
|
||
| You'll need Go 1.21 or later, and a basic understanding of the [provider protocol](/docs/iac/guides/building-extending/providers/implementers/protocol-reference/). |
There was a problem hiding this comment.
Nit: Go 1.24 is currently the lowest supported version of Go, so not sure why we'd mention 1.21.
| func getTypeFromURN(urn string) string { | ||
| // URN format: urn:pulumi:<stack>::<project>::<type>::<name> | ||
| parts := strings.Split(urn, "::") | ||
| if len(parts) >= 3 { | ||
| return parts[2] | ||
| } | ||
| return "" | ||
| } |
There was a problem hiding this comment.
As of pulumi/pulumi#17177 and v3.132.0 of the CLI, we send the parsed type in the requests, so probably no need to show urn parsing fallback here. Just use req.Type and assume users are creating providers for v3.132.0 and later of pulumi from a year ago.
| oldInputs := req.Inputs.AsMap() | ||
| force, _ := oldInputs["force"].(bool) | ||
|
|
||
| state := map[string]interface{}{ |
There was a problem hiding this comment.
Nit: Might as well use modern go! There are some other uses of interface{} that could be cleaned up as well.
| state := map[string]interface{}{ | |
| state := map[string]any{ |
|
|
||
| This document describes the gRPC protocol that all Pulumi providers implement. Understanding this protocol is essential for [implementing providers directly](/docs/iac/guides/building-extending/providers/provider-architecture/#layer-2-generated-language-bindings) without a higher-level SDK. | ||
|
|
||
| The complete protocol definition is in [`provider.proto`](https://github.com/pulumi/pulumi/blob/master/proto/pulumi/provider.proto). |
There was a problem hiding this comment.
We also have this published at https://pulumi-developer-docs.readthedocs.io/latest/docs/_generated/proto/provider.html. Not sure if we want to link to that from here or not.
| rpc Create(CreateRequest) returns (CreateResponse) | ||
| ``` | ||
|
|
||
| **Request fields:** |
There was a problem hiding this comment.
We should probably show type and name fields for all relevant requests.
Today we do not really explain the layering of our various means of writing providers, which as we've seen, can lead to confusion.
This does a pass over the provider authoring docs, explaining the architecture, and why only Go has a higher level SDK.
It also adds reference material and implementation guides for those who want to code to the direct gRPC/Protobuf interfaces directly, in Python, Go, or TypeScript. (These were tested locally.)
I am not 100% convinced we should add this but for power users wanting to really understand how providers work and how to author them, it would seem helpful in demystifying some confusing aspects of our ecosystem.