go/types, types2: change and enforce lifecycle of Named.fromRHS and Named.underlying fields

A type definition or alias declaration consists of a type name (LHS)
which is bound to a type expression (RHS) by the declaration.

This CL consistently uses the fromRHS fields of Named and Alias types
to represent that RHS type expression, and sets Named.underlying and
Alias.actual only once those types have been computed.

Currently, Named types use Named.underlying for some of this
functionality, which makes the code difficult to understand. Operations
which used Named.underlying now use Named.fromRHS.

For example, in:

  type A = B
  type B = int

A.fromRHS is B (Alias) and B.fromRHS is int (Basic).

Meanwhile, in:

  type A B
  type B int

A.underlying is B (Named) and B.underlying is int (Basic) initially.

Note that despite A.underlying pointing to B, B is not the underlying
type of A (it is int). At some point during type checking, A walks
through the chain A.underlying -> B.underlying -> int and sets
A.underlying to int.

While this approach works, it introduces some problems:

  1. Whether A.underlying refers to the underlying type (int) or not
     (B) depends on when the field is accessed.
  2. There is no convenient mechanism to check if the underlying type
     of B has been deduced. One can check if B.underlying is a named
     type, but since B.underlying is already B's underlying type (int),
     it's still ambiguous.

Operations derived from Named.underlying share similar problems. For
example, Named.expandUnderlying() (which substitutes type arguments)
returns an instantiated named type whose Named.underlying also may or
may not refer to its underlying type.

With this change, Named.underlying is nil as long as it is unknown, and
non-nil and not a named type once it is known. Additional assertions are
added to enforce that:

  1. Named.underlying is not set until Named has been resolved.
  2. Named is not resolved until Named.fromRHS is populated, unless it
     is given explicit permission. This permission is briefly given
     while type-checking declarations of named types to account for
     cycles of alias types represented as TypeNames. It is also given to
     named types created through NewNamed for backward compatibility.
     This permission is revoked when SetUnderlying is called.

Accessors of Named.underlying are responsible for first resolving
the named type, unless they are in a context where they know the
type to already be resolved.

This change also exposed a bug in validType wherein the underlying
type for struct types containing invalid types did not have their
underlying type set to invalid (see golang#75194). This bug was exploited by a
test in x/tools, which has been disabled for Go 1.26 (via CL 700395).

Other minor adjustments are made for instantiated and loaded types.
Instantiated types have no RHS as they are not declared, and loaded
types set their RHS to the underlying from export data directly.

Minor simplifications are also made throughout.

Fixes golang#75194

Change-Id: I72644d7329c996eb1e67514063fe51c3ae06c38d
Reviewed-on: https://go-review.googlesource.com/c/go/+/695977
Auto-Submit: Mark Freeman <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Robert Griesemer <[email protected]>

Author: Mark Freeman

src/cmd/compile/internal/types2/decl.go

@@ -476,7 +476,7 @@ func (check *Checker) isImportedConstraint(typ Type) bool {
 	if named == nil || named.obj.pkg == check.pkg || named.obj.pkg == nil {
 		return false
 	}
-	u, _ := named.under().(*Interface)
+	u, _ := named.Underlying().(*Interface)
 	return u != nil && !u.IsMethodSet()
 }
 
@@ -558,28 +558,33 @@ func (check *Checker) typeDecl(obj *TypeName, tdecl *syntax.TypeDecl, def *TypeN
 	named := check.newNamed(obj, nil, nil)
 	setDefType(def, named)
 
+	// The RHS of a named N can be nil if, for example, N is defined as a cycle of aliases with
+	// gotypesalias=0. Consider:
+	//
+	//   type D N    // N.resolve() will panic
+	//   type N A
+	//   type A = N  // N.fromRHS is not set before N.resolve(), since A does not call setDefType
+	//
+	// There is likely a better way to detect such cases, but it may not be worth the effort.
+	// Instead, we briefly permit a nil N.fromRHS while type-checking D.
+	named.allowNilRHS = true
+	defer (func() { named.allowNilRHS = false })()
+
 	if tdecl.TParamList != nil {
 		check.openScope(tdecl, "type parameters")
 		defer check.closeScope()
 		check.collectTypeParams(&named.tparams, tdecl.TParamList)
 	}
 
-	// determine underlying type of named
 	rhs = check.definedType(tdecl.Type, obj)
 	assert(rhs != nil)
 	named.fromRHS = rhs
 
-	// If the underlying type was not set while type-checking the right-hand
-	// side, it is invalid and an error should have been reported elsewhere.
-	if named.underlying == nil {
-		named.underlying = Typ[Invalid]
-	}
-
 	// spec: "In a type definition the given type cannot be a type parameter."
 	// (See also go.dev/issue/45639.)
 	if isTypeParam(rhs) {
 		check.error(tdecl.Type, MisplacedTypeParam, "cannot use a type parameter as RHS in type declaration")
-		named.underlying = Typ[Invalid]
+		named.fromRHS = Typ[Invalid]
 	}
 }
 
@@ -721,7 +726,7 @@ func (check *Checker) collectMethods(obj *TypeName) {
 }
 
 func (check *Checker) checkFieldUniqueness(base *Named) {
-	if t, _ := base.under().(*Struct); t != nil {
+	if t, _ := base.Underlying().(*Struct); t != nil {
 		var mset objset
 		for i := 0; i < base.NumMethods(); i++ {
 			m := base.Method(i)