Permit discard identifier for layout members

Author: lwaern-intel

RELEASENOTES-1.4.md

@@ -363,7 +363,7 @@
 - `release 7 7122`
 - `release 6 6437`
 - `release 7 7123`
-- `note 6` Added the _discard reference_ '`_`' — a non-value expression
+- `note 6` Added the _discard reference_ "`_`" — a non-value expression
   which may be used as an assign target in order to explictly discard the result
   of an evaluated expression or return value of a method call (fixes
   SIMICS-21584.)
@@ -374,23 +374,30 @@
   _ = throwing_method();
   (_, x, _) = method_with_multiple_return_values();
   ```
-- `note 6` '`_`' can no longer be used as the name for arbitrary declarations.
+- `note 6` "`_`" can no longer be used as the name for arbitrary declarations.
   Instead, it is now only permitted in particular contexts, and in these
-  contexts, '`_`' will affect the declaration in a way suitable for when the
-  declaration is _unused_. These contexts are:
+  contexts, `_` will affect the declaration in a way suitable for when the
+  declaration is _unused_ in some particular way. This feature is referred to
+  as the _discard identifier_, and the contexts where it may be used are:
     - Method-local bindings (e.g. variables and input parameters.)
 
-      When a method-local binding is given the name '`_`', it will not be added
+      When a method-local binding is given the name "`_`", it will not be added
       to scope. This is useful for e.g. unused method parameters.
     - Index variables for object arrays
 
-      When '`_`' is specified as an index variable, a parameter will not be
+      When `_` is specified as an index variable, a parameter will not be
       created for it, meaning it cannot conflict with any other definition, and
       it cannot be referenced in the code in order to get the value of the index
       in question. It also isn't considered to conflict with any other
       definition that gives the index variable a different name. This is useful
       when defining an object array specification which does not depend on the
       index.
+    - Layout member names
 
-  Note that as a consequence of these semantics, any reference to '`_`' in code
+      When a layout member is given the name "`_`", that member will not be
+      referencable within DML code, but will still affect the memory
+      representation of the layout. This is useful to represent e.g. reserved
+      or padding bytes.
+
+  Note that as a consequence of these semantics, any reference to `_` in code
   will _always_ resolve to the discard reference.

doc/1.4/language.md

@@ -90,6 +90,7 @@ declaration special semantics. Currently, these contexts are:
 see that section for more information.
 * Index parameters for object arrays. See the documentation for the
   [`object` template](dml-builtins.html#object) for more information.
+* As the name of one or more members of a [layout type](#layouts).
 
 </dd><dt>
 
@@ -1537,6 +1538,7 @@ typedef struct { <em>member declarations</em> } <em>name</em>;
 
 Layouts
 </dt><dd>
+<a id="layouts"/>
 
 A layout is similar to a struct in many ways.  The important
 difference is that there is a well-defined mapping between a layout
@@ -1571,6 +1573,24 @@ integer members (and arrays of similar) are translated
 to endian integers (or arrays of such) of similar size,
 with endianness matching the layout. Layout and endian integer
 members are accessed normally.
+
+The *discard identifer* `_` may be used as the name of any number of members
+within a layout, making these *anonymous*. Anonymous layout members cannot be
+referenced within DML code, but will still influence the underlying memory
+representation of the layout in the same way as regular members.
+This is useful to represent reserved or padding bytes, or bytes that the device
+otherwise doesn't study or manipulate.
+
+Note that when a compound initializer is given for a variable of layout type,
+an initializer must still be given for each anonymous member:
+```
+local layout "little-endian" { uint32 x; uint32 _; uint32 y} = {1,0,2};
+```
+... unless designated initializers are used, in which case anonymous members
+can (and must) be omitted:
+```
+local layout "little-endian" { uint32 x; uint32 _; uint32 y} = {.x = 1, .y = 2};
+```
 </dd><dt>
 
 Bitfields
@@ -3231,27 +3251,27 @@ local (bool a, int i) = m();
 In the absence of explicit initializer expressions, a default
 "all zero" initializer will be applied to each declared object.
 
-"\_" may be used as an identifier for local variables, as well as other
-method-local bindings such as the method parameters, the bound identifier
-in `foreach`/`#foreach`/`#select` statements, and message component parameters
-of [hook-bound after statements](#hook-bound-after-statements). Any method-local
-binding named "\_" *will not be added to scope.* This is useful for when
-a method parameter is unused, or if you perform a method call where only a
-subset of returned values are of interest:
+The *discard identifier* `_` may be used as an identifier for local variables,
+as well as other method-local bindings such as the method parameters, the bound
+identifier in `foreach`/`#foreach`/`#select` statements, and message component
+parameters of [hook-bound after statements](#hook-bound-after-statements).
+Any method-local binding named "`_`" *will not be added to scope*. This is
+useful for when a method parameter is unused, or if you perform a method call
+where only a subset of returned values are of interest:
 ```
 local (bool a, int _) = m();
 // No conflicts since "_" is not added to scope
 local (bool a, int _, float _) = returns_three_vals();
 ```
 
-An alternative to this pattern is to leverage the
-[discard reference](#discard-reference)
+An alternative to this pattern is to leverage the [discard
+reference](#discard-reference):
 ```
 local bool a;
 (a, _, _) = returns_three_vals();
 ```
 ... which does not require you to specify the types of the discarded values,
-may require multiple lines.
+but may result in more lines of code.
 
 ### Session Statements
 <pre>

lib/1.4/dml-builtins.dml

@@ -512,18 +512,18 @@ which cannot be overridden:
   `register regs[i < 4][j < 11];` defines two index parameters, `i` and `j`.
   In this case, the `indices` parameter is `[i, j]`.
 
-  "\_" is also allowed as the name of one or more index parameters. If
-  used, then it *won't* result in corresponding parameter definitions for those
-  indices; in other words, if an index variable is named "\_", then it can't be
-  accessed by that name (however, the `indices` parameter is unaffected and
-  could be used instead.) It's also allowed to have an object array
-  specification name an index parameter "\_" even if some other object array
-  specification uses a different name (which *will* be given a corresponding
-  parameter definition.) This is useful if a specific object array
-  specification doesn't need to make use of some particular indices; or if the
-  specification needs to support that the naming of the index variable may vary
-  depending on what other specifications of the same object array are present
-  in the device model.
+  The discard identifier `_` is also allowed as the name of one or more index
+  parameters. If used, then it *won't* result in corresponding parameter
+  definitions for those indices; in other words, if an index variable is named
+  "`_`", then it can't be accessed by that name (however, the `indices`
+  parameter is unaffected and could be used instead.) It's also allowed to have
+  an object array specification name an index parameter "`_`" even if some
+  other object array specification uses a different name (which *will* be given
+  a corresponding parameter definition.) This is useful if a specific object
+  array specification doesn't need to make use of some particular indices; or
+  if the specification needs to support that the naming of the index variable
+  may vary depending on what other specifications of the same object array are
+  present in the device model.
 
 The `object` template provides the non-overridable method `cancel_after()`,
 which cancels all pending events posted using `after` which are associated with

py/dml/codegen.py

@@ -1487,17 +1487,20 @@ def eval_type(asttype, site, location, scope, extern=False, typename=None,
                 raise ELAYOUT(site, "extern layout not permitted,"
                               + " use 'struct { }' instead")
             endian, fields = info
-            members = {}
-            for (_, msite, name, type_ast) in fields:
+            member_decls = []
+            for (_, msite, ident, type_ast) in fields:
                 (member_struct_defs, member_type) = eval_type(
                     type_ast, msite, location, scope, False)
                 if isinstance(member_type, TFunction):
                     raise EFUNSTRUCT(msite)
-                members[name] = (msite, member_type)
+                member_decls.append((
+                    msite,
+                    ident.args[0] if ident.kind == 'variable' else None,
+                    member_type))
                 struct_defs.extend(member_struct_defs)
-            if not members:
+            if not member_decls:
                 raise EEMPTYSTRUCT(site)
-            etype = TLayout(endian, members, label=typename)
+            etype = TLayout(endian, member_decls, label=typename)
             struct_defs.append((site, etype))
         elif tag == 'bitfields':
             width, fields = info
@@ -1737,9 +1740,9 @@ def check_designated_initializers(site, etype, init_asts, allow_partial):
     shallow_real_etype = safe_realtype_shallow(etype)
     duplicates = set()
     bad_fields = set()
-    remaining = set(shallow_real_etype.members)
+    remaining = set(shallow_real_etype.named_members)
     for (field, init) in init_asts:
-        if field not in shallow_real_etype.members:
+        if field not in shallow_real_etype.named_members:
             bad_fields.add(field)
         elif field not in remaining:
             duplicates.add(field)
@@ -1900,14 +1903,14 @@ def do_eval(etype, astinit):
             init = tuple(do_eval(etype.base, e) for e in init_asts)
             return CompoundInitializer(site, init)
         elif isinstance(etype, TStruct):
-            if len(etype.members) != len(init_asts):
+            members = list(etype.members_qualified)
+            if len(members) != len(init_asts):
                 raise EDATAINIT(site, 'mismatched number of fields')
             init = tuple(do_eval(mt, e)
-                         for ((_, mt), e) in zip(etype.members_qualified,
-                                                 init_asts))
+                         for ((mn, mt), e) in zip(members, init_asts))
             return CompoundInitializer(site, init)
         elif isinstance(etype, TExternStruct):
-            if len(etype.members) != len(init_asts):
+            if len(etype.named_members) != len(init_asts):
                 raise EDATAINIT(site, 'mismatched number of fields')
             init = {mn: do_eval(mt, e)
                     for ((mn, mt), e) in zip(etype.members_qualified,

py/dml/ctree_test.py

@@ -1400,8 +1400,8 @@ def const_types(self):
             types.TExternStruct({}, 'struct_t', 'struct_t'),
             types.TStruct({'x': types.TBool()}, 'struct_label'),
             types.TLayout(
-                'big-endian', {
-                    'x': (site, types.TEndianInt(24, True, 'big-endian'))},
+                'big-endian', [(site, 'x',
+                                types.TEndianInt(24, True, 'big-endian'))],
                 'struct_label'),
             types.THook([]),
         ]

py/dml/dmlparse.py

@@ -1483,7 +1483,13 @@ def typeof(t):
 def check_struct_namecoll(member_decls):
     sites_by_name = {}
     for decl in member_decls:
-        (name, _) = decl.args
+        if decl.kind == 'cdecl':
+            (name, _) = decl.args
+        else:
+            (ident, _) = decl.args
+            if ident.kind == 'discard':
+                continue
+            (name,) = ident.args
         if name in sites_by_name:
             report(ENAMECOLL(decl.site, sites_by_name[name], name))
         else:
@@ -1513,14 +1519,21 @@ def layout_decl(t):
         field_names = set()
         fields = []
         for cdecl in t[4]:
-            (name, typ) = cdecl.args
-            if name in field_names:
-                while (name in field_names
-                       or any(name == d.args[0] for d in t[4])):
-                    name = '_' + name
-                cdecl = ast.cdecl(cdecl.site, name, typ)
+            (ident, typ) = cdecl.args
+            if ident.kind == 'variable':
+                (name,) = ident.args
+                if name in field_names:
+                    while (name in field_names
+                           or any(name == d.args[0].args[0]
+                                  for d in t[4]
+                                  if d.args[0].kind == 'variable')):
+                        name = '_' + name
+                    cdecl = ast.cdecl_maybe_discarded(
+                        cdecl.site,
+                        ast.variable(ident.site, name),
+                        typ)
+                field_names.add(name)
             fields.append(cdecl)
-            field_names.add(name)
     else:
         fields = t[4]
         check_struct_namecoll(fields)
@@ -1537,7 +1550,7 @@ def layout(t):
 
 @prod
 def layout_decls(t):
-    'layout_decls : layout_decls named_cdecl SEMI'
+    'layout_decls : layout_decls named_cdecl_maybe_discarded SEMI'
     t[0] = t[1] + (t[2],)
 
 @prod
@@ -2760,10 +2773,11 @@ def objident_discard(t):
 def discard_error(site):
     report(ESYNTAX(site,
                    "_",
-                   "'_' can only be used as an expression or as an unused "
-                   + "identifier for an index variable of an object array or "
-                   + "a method-local binding (e.g. local variable or method "
-                   + "parameter)"))
+                   "can't use the name '_' (the discard identifier) in this "
+                   + "context. See the description of 'Identifiers' within "
+                   + "the Lexical Structure section of the DML 1.4 Reference "
+                   + "Manual for an overview of when '_' may be used as a "
+                   + "name."))
 
 @prod
 def ident_or_discard_ident(t):

py/dml/serialize.py

@@ -99,6 +99,25 @@ def prepare_array_de_serialization(site, t):
     dimsizes_expr = expr.mkLit(site, dimsizes_lit, TPtr(TInt(32, False)))
     return (base, dims, sizeof_base, dimsizes_expr)
 
+def mkSubRefLit(site, expr, sub, typ, op):
+    real_etype = safe_realtype_shallow(expr.ctype())
+
+    if isinstance(real_etype, TPtr):
+        if op == '.':
+            raise ENOSTRUCT(site, expr)
+        basetype = real_etype.base
+        real_basetype = safe_realtype(basetype)
+    else:
+        if op == '->':
+            raise ENOPTR(site, expr)
+        real_basetype = safe_realtype(etype)
+
+    real_basetype = real_basetype.resolve()
+
+    return ctree.StructMember(site, expr, sub,
+                              conv_const(real_basetype.const, typ), op)
+
+
 # This works on the assumption that args do not need to be hard-cast
 # to fit the actual fun signature
 def apply_c_fun(site, fun, args, rettype):
@@ -354,7 +373,7 @@ def map_dmltype_to_attrtype(site, dmltype):
         return 'f'
     if isinstance(real_type, TStruct):
         return '[%s]' % "".join([map_dmltype_to_attrtype(site, mt)
-                                 for mt in real_type.members.values()])
+                                 for (_, mt) in real_type.members])
     if isinstance(real_type, TArray):
         assert real_type.size.constant
         arr_attr_type = map_dmltype_to_attrtype(site, real_type.base)
@@ -375,7 +394,7 @@ def mark_for_serialization(site, dmltype):
     '''
     real_type = safe_realtype(dmltype)
     if isinstance(real_type, TStruct):
-        for mt in real_type.members.values():
+        for (_, mt) in real_type.members:
             mark_for_serialization(site, mt)
     elif isinstance(real_type, TArray):
         # Can only serialize constant-size arrays
@@ -496,9 +515,12 @@ def generate_serialize(real_type):
         in_arg_decl.toc()
         out_arg_decl.toc()
         if isinstance(real_type, TStruct):
-            sources = ((ctree.mkSubRef(site, in_arg, name, "->"),
-                        safe_realtype(typ))
-                       for (name, typ) in real_type.members.items())
+            sources = (
+                (mkSubRefLit(
+                    site, in_arg, name or TStruct.anon_member_cident(i),
+                    typ, "->"),
+                 safe_realtype(typ))
+                for (i, (name, typ)) in enumerate(real_type.members))
             serialize_sources_to_list(site, sources, out_arg)
         elif isinstance(real_type, TVector):
             raise ICE(site, "TODO: serialize vector")
@@ -619,9 +641,12 @@ def error_out(exc, msg):
                            else ctree.mkCast(site, tmp_out_ref, TPtr(void)))
             cleanup.append(ctree.mkDelete(site, cleanup_ref))
             tmp_out_decl.toc()
-            targets = tuple((ctree.mkSubRef(site, tmp_out_ref, name, "->"),
-                             conv_const(real_type.const, safe_realtype(typ)))
-                            for (name, typ) in real_type.members.items())
+            targets = tuple(
+                (mkSubRefLit(
+                    site, tmp_out_ref,
+                    name or TStruct.anon_member_cident(i), typ, "->"),
+                 conv_const(real_type.const, safe_realtype(typ)))
+                for (i, (name, typ)) in enumerate(real_type.members))
             def error_out_at_index(_i, exc, msg):
                 return error_out(exc, msg)
             deserialize_list_to_targets(site, in_arg, targets,

py/dml/structure.py

@@ -325,7 +325,7 @@ def type_deps(t, include_structs, expanded_typedefs):
         deps = []
         if include_structs:
             deps.append(t.label)
-        for (mn, mt) in t.members.items():
+        for (_, mt) in t.members:
             deps.extend(type_deps(mt, True, expanded_typedefs))
         return deps
     elif isinstance(t, TArray):