Python codegen: docstrings and attribute-access ENV for hover docs

.bumpy/python-codegen-hover-docs.md

@@ -0,0 +1,5 @@
+---
+varlock: minor
+---
+
+Python codegen: `@generatePythonEnv` now emits a lazily loaded `ENV` object with a documented property per item, so editors show item descriptions, `@docs()` links and deprecations on hover where values are read. `Env` TypedDict fields get docstrings instead of `#` comments.

packages/varlock-website/src/content/docs/guides/code-generation.mdx

@@ -54,7 +54,7 @@ Varlock ships generators for several languages. Each emits a **self-contained, i
 
 | Language | Decorator | Loader | Guide |
 | --- | --- | --- | --- |
-| Python | `@generatePythonEnv` | `load_env()` | [Python](/integrations/python/) |
+| Python | `@generatePythonEnv` | `ENV` / `load_env()` | [Python](/integrations/python/) |
 | Rust | `@generateRustEnv` | `env::load()` | [Rust](/integrations/rust/) |
 | Go | `@generateGoEnv` | `env.Load()` | [Go](/integrations/go/) |
 | PHP | `@generatePhpEnv` | `Env::load()` | [PHP](/integrations/php/) |

packages/varlock-website/src/content/docs/integrations/python.mdx

@@ -5,7 +5,7 @@ description: Use varlock with Python via a generated typed env module, Pydantic
 
 To use varlock with Python, [install the standalone binary](/getting-started/installation/#as-a-standalone-binary) and run your app under [`varlock run`](/reference/cli/load-and-run/#run). It loads and validates your env, then injects it into the process.
 
-Add [`@generatePythonEnv`](/reference/root-decorators/#generatepythonenv) to your schema to generate a small, self-contained module: a coerced `Env` [`TypedDict`](https://docs.python.org/3/library/typing.html#typing.TypedDict), a `load_env()` function that parses the injected [`__VARLOCK_ENV`](/reference/reserved-variables/#__varlock_env) blob, and a `SENSITIVE_KEYS` constant.
+Add [`@generatePythonEnv`](/reference/root-decorators/#generatepythonenv) to your schema to generate a small, self-contained module: a lazily loaded `ENV` object with typed attributes, a coerced `Env` [`TypedDict`](https://docs.python.org/3/library/typing.html#typing.TypedDict) with a `load_env()` function that parses the injected [`__VARLOCK_ENV`](/reference/reserved-variables/#__varlock_env) blob, and a `SENSITIVE_KEYS` constant.
 
 ```env-spec title=".env.schema"
 # @generatePythonEnv(path=env.py)
@@ -15,43 +15,47 @@ The file is regenerated automatically on [`varlock load`](/reference/cli/load-an
 
 ## Reading values
 
-Call `load_env()` once, then read keys off the dict. Values are coerced (`int`/`bool`/etc.), not raw strings:
+Import `ENV` and read keys as attributes. Values are coerced (`int`/`bool`/etc.), not raw strings:
 
 ```python
-from env import load_env, SENSITIVE_KEYS
-
-env = load_env()        # call once, hold or pass it around
-port = env["DB_PORT"]   # int
-debug = env["DEBUG"]    # bool
+from env import ENV
 
-# build your own redaction using SENSITIVE_KEYS
-safe = {k: ("***" if k in SENSITIVE_KEYS else v) for k, v in env.items()}
+port = ENV.DB_PORT   # int | None (optional item)
+debug = ENV.DEBUG    # bool
 ```
 
 ```bash
 varlock run -- python main.py
 ```
 
-`load_env()` raises a clear error if `__VARLOCK_ENV` is missing (e.g. you forgot `varlock run`). Optional keys that are unset are absent from the dict (`NotRequired`), not present as `None`.
+`ENV` is loaded on first access and then cached, so importing the module has no side effects, and every module that imports `ENV` shares one instance. Loading raises a clear error if `__VARLOCK_ENV` is missing (e.g. you forgot `varlock run`). Optional keys that are unset read as `None`.
 
-## Sharing one instance
+Each attribute is a documented property, so editors using Pyright or Pylance show the item's description, [`@docs()`](/reference/item-decorators/#docs) links, valid enum values, and sensitivity on hover, and go-to-definition jumps to it. Items marked [`@deprecated`](/reference/item-decorators/#deprecated) are typed with [`@deprecated`](https://peps.python.org/pep-0702/), so type checkers flag reads of them and editors show them struck through.
 
-Every `load_env()` call re-parses the blob. To avoid reloading, load once in a config module and share it, the idiomatic "settings module" pattern:
+To build an instance yourself, for example in tests, pass a dict to `EnvAttrs`:
 
 ```python
-# config.py
-from env import load_env
+from env import EnvAttrs
 
-env = load_env()   # loaded once, typed as Env
+env = EnvAttrs({"DB_HOST": "localhost", "APP_ENV": "dev"})
 ```
 
+### Dict access
+
+`load_env()` returns the values as a plain `Env` dict instead. Optional keys that are unset are absent from the dict (`NotRequired`), not present as `None`. Each call re-parses the blob, so call it once and share the result:
+
 ```python
-# anywhere.py
-from config import env
+from env import load_env, SENSITIVE_KEYS
 
+env = load_env()
 port = env["DB_PORT"]   # int
+
+# build your own redaction using SENSITIVE_KEYS
+safe = {k: ("***" if k in SENSITIVE_KEYS else v) for k, v in env.items()}
 ```
 
+The `Env` fields carry the same docs as docstrings, but Pyright shows them only when hovering the declaration, not on `env["DB_PORT"]`. Use `ENV` if you want docs where values are read.
+
 ## Running your server
 
 [`varlock run`](/reference/cli/load-and-run/#run) wraps any command, so boot your ASGI server, task runner, or scripts through it. Local dev, CI, and production all use the same schema:
@@ -124,5 +128,5 @@ When using varlock, don't also load `.env` from Python (`python-dotenv`, Pydanti
 :::
 
 :::note
-Every key is also injected as a plain environment variable, so you can read any of them with `os.environ` (raw strings) if you don't need the typed module. Keys whose names aren't valid identifiers (they contain `.` or `-`) can't be `TypedDict` fields, so read those from the env var directly. See [Other languages](/integrations/other-languages/) for the shared details on the injected blob, raw env vars, and encryption.
+Every key is also injected as a plain environment variable, so you can read any of them with `os.environ` (raw strings) if you don't need the typed module. Keys whose names aren't valid identifiers (they contain `.` or `-`, or are a Python keyword) are left out of the typed module, so read those from the env var directly. A few more names can't be `ENV` attributes, so read those with `load_env()`: `property`, `deprecated`, `_values`, `Env`, the type names `str`, `int`, `float`, `bool`, `list`, `dict`, `Any`, and `Literal`, and anything starting with `__`. The generated file lists any it left out. See [Other languages](/integrations/other-languages/) for the shared details on the injected blob, raw env vars, and encryption.
 :::

packages/varlock-website/src/content/docs/reference/root-decorators.mdx

@@ -761,7 +761,7 @@ TypeScript type declarations. Makes `import { ENV } from 'varlock/env'` typed an
 <div>
 ### `@generatePythonEnv()`
 
-A self-contained Python module: a coerced `Env` `TypedDict`, a `load_env()` that parses the injected env, and a `SENSITIVE_KEYS` constant. See [Python usage](/integrations/python/).
+A self-contained Python module: a lazily loaded `ENV` object with a documented, typed attribute per item (so editors show item docs on hover), a coerced `Env` `TypedDict` with a `load_env()` that parses the injected env, and a `SENSITIVE_KEYS` constant. See [Python usage](/integrations/python/).
 
 ```env-spec
 # @generatePythonEnv(path=./env.py)

packages/varlock/src/env-graph/lib/type-generation/emitters/python.ts

@@ -3,9 +3,9 @@ import {
   BLOB_ENCRYPTED_MSG,
   BLOB_MISSING_MSG,
   ENCRYPTED_BLOB_PREFIX,
-  getFieldDocLines,
   getSensitiveKeys,
   partitionRepresentableKeys,
+  sanitizeDocLine,
   skippedKeysComment,
   type CoercedType,
   type ResolvedFieldType,
@@ -86,23 +86,50 @@ function getPythonCoercedTypeString(coerced: CoercedType): string {
   return 'str';
 }
 
-function formatPythonField(field: ResolvedFieldType, typeString: string): Array<string> {
-  const docLines = getFieldDocLines(field);
-  // single line of docs → inline comment; multiple → comment lines above the field; none → no comment
-  if (docLines.length === 1) {
-    return [`    ${field.key}: ${typeString}  # ${docLines[0]}`];
+/**
+ * Docstring content for a field. Pyright/Pylance render docstrings as Markdown on hover, so docs
+ * links become Markdown links and sections are separated by blank lines (consecutive lines would be
+ * reflowed into one paragraph).
+ */
+function getPythonDocstringLines(field: ResolvedFieldType): Array<string> {
+  const { docs } = field;
+  const sections: Array<Array<string>> = [];
+  if (docs.description) sections.push(docs.description.split(/\r\n|\r|\n/));
+  if (docs.docsLinks.length) {
+    sections.push(docs.docsLinks.map((entry) => (
+      entry.description ? `Docs: [${entry.description}](${entry.url})` : `Docs: <${entry.url}>`
+    )));
   }
+  if (docs.isDeprecated) {
+    sections.push([docs.deprecationMessage ? `Deprecated: ${docs.deprecationMessage}` : 'Deprecated']);
+  }
+  if (typeof field.coerced === 'object' && 'enum' in field.coerced && field.coerced.enum.length) {
+    sections.push([`Valid values: ${field.coerced.enum.map((o) => JSON.stringify(o)).join(' | ')}`]);
+  }
+  if (field.isSensitive) sections.push(['🔐 Sensitive']);
+  return sections.flatMap((section, i) => [...(i ? [''] : []), ...section.map(sanitizeDocLine)]);
+}
+
+/**
+ * Render docstring lines as a `"""..."""` literal at the given indent (or `[]` if there are none).
+ * Every backslash and quote is escaped, so user text can't close the literal early or form an
+ * escape sequence; editors show the unescaped value on hover.
+ */
+function formatPythonDocstring(docLines: Array<string>, indent: string): Array<string> {
+  if (!docLines.length) return [];
+  const escaped = docLines.map((line) => line.replace(/\\/g, '\\\\').replace(/"/g, '\\"'));
+  if (escaped.length === 1) return [`${indent}"""${escaped[0]}"""`];
   return [
-    ...docLines.map((line) => `    # ${line}`),
-    `    ${field.key}: ${typeString}`,
+    `${indent}"""${escaped[0]}`,
+    ...escaped.slice(1).map((line) => (line ? `${indent}${line}` : '')),
+    `${indent}"""`,
   ];
 }
 
 function formatPythonTypedDict(
   className: string,
   classDoc: string,
   fields: Array<ResolvedFieldType>,
-  getType: (field: ResolvedFieldType) => string,
 ): Array<string> {
   const lines = [
     '',
@@ -111,15 +138,78 @@ function formatPythonTypedDict(
     `    """${classDoc}"""`,
   ];
 
-  if (!fields.length) {
-    lines.push('    pass');
-    return lines;
+  for (const field of fields) {
+    const baseType = getPythonCoercedTypeString(field.coerced);
+    const typeString = field.isRequired ? baseType : `NotRequired[${baseType}]`;
+    // attribute docstring (PEP 257): shown when hovering the declaration
+    lines.push(`    ${field.key}: ${typeString}`);
+    lines.push(...formatPythonDocstring(getPythonDocstringLines(field), '    '));
   }
 
+  return lines;
+}
+
+// the attribute-access class stores the dict in this slot
+const ATTRS_VALUES_SLOT = '_values';
+
+// names the attribute-access class body references: a property with one of these names would
+// shadow it for the rest of the class (`property`/`deprecated` decorators at runtime; the type
+// names in later annotations for mypy, e.g. a `str` property breaks `def OTHER(self) -> str`)
+const ATTRS_RESERVED_NAMES = new Set([
+  ATTRS_VALUES_SLOT,
+  'property',
+  'deprecated',
+  'Env',
+  'str',
+  'int',
+  'float',
+  'bool',
+  'list',
+  'dict',
+  'Any',
+  'Literal',
+]);
+
+/**
+ * Keys that can't be a property on the attribute-access class: names the class body references
+ * (see above), and `__x` names, which Python mangles inside a class body so `ENV.__x` would never
+ * reach them.
+ */
+function isReservedAttrName(key: string): boolean {
+  return ATTRS_RESERVED_NAMES.has(key) || key.startsWith('__');
+}
+
+function formatPythonEnvAttrs(fields: Array<ResolvedFieldType>): Array<string> {
+  const lines = [
+    '',
+    '',
+    'class EnvAttrs:',
+    '    """Attribute access to the coerced env (`ENV.MY_KEY`), so editors show each item\'s docs on hover.',
+    '',
+    '    Unset optional keys read as `None`. Build one from any `Env` dict, e.g. in tests.',
+    '    """',
+    '',
+    `    __slots__ = ("${ATTRS_VALUES_SLOT}",)`,
+    '',
+    '    def __init__(self, values: Env) -> None:',
+    `        self.${ATTRS_VALUES_SLOT} = values`,
+  ];
+
   for (const field of fields) {
-    const baseType = getType(field);
-    const typeString = field.isRequired ? baseType : `NotRequired[${baseType}]`;
-    lines.push(...formatPythonField(field, typeString));
+    const baseType = getPythonCoercedTypeString(field.coerced);
+    const typeString = field.isRequired ? baseType : `${baseType} | None`;
+    const getter = field.isRequired
+      ? `self.${ATTRS_VALUES_SLOT}[${JSON.stringify(field.key)}]`
+      : `self.${ATTRS_VALUES_SLOT}.get(${JSON.stringify(field.key)})`;
+    lines.push('');
+    lines.push('    @property');
+    // PEP 702: type checkers flag reads of a deprecated property (editors render a strikethrough)
+    if (field.docs.isDeprecated) {
+      lines.push(`    @deprecated(${JSON.stringify(field.docs.deprecationMessage || 'Deprecated')})`);
+    }
+    lines.push(`    def ${field.key}(self) -> ${typeString}:`);
+    lines.push(...formatPythonDocstring(getPythonDocstringLines(field), '        '));
+    lines.push(`        return ${getter}`);
   }
 
   return lines;
@@ -128,6 +218,8 @@ function formatPythonTypedDict(
 export function generatePythonEnvSrc(fields: Array<ResolvedFieldType>): string {
   // typed structure covers only representable keys; skipped keys stay in load_env()'s dict at runtime
   const { safe, skipped } = partitionRepresentableKeys(fields, (k) => PYTHON_KEYWORDS.has(k));
+  const attrFields = safe.filter((f) => !isReservedAttrName(f.key));
+  const attrSkipped = safe.filter((f) => isReservedAttrName(f.key)).map((f) => f.key);
   // SENSITIVE_KEYS covers every sensitive key (incl. skipped ones) — redaction shouldn't miss them
   const sensitiveKeys = getSensitiveKeys(fields);
   const sensitiveSet = sensitiveKeys.length
@@ -138,38 +230,50 @@ export function generatePythonEnvSrc(fields: Array<ResolvedFieldType>): string {
   // Literal / Any (and subscripts like dict[str, Any]) are never evaluated at runtime — the module
   // imports on any Python 3.7+, while type checkers still read the real types from the source. The
   // type-only symbols therefore live under `TYPE_CHECKING` (imported only when actually used).
+  // NotRequired / deprecated come from typing_extensions (bundled with every type checker's stubs,
+  // never imported at runtime) so checkers targeting Python < 3.11 / 3.13 accept them too.
   const typeStrings = safe.map((f) => getPythonCoercedTypeString(f.coerced));
-  const typeOnlyImports: Array<string> = [];
-  if (typeStrings.some((t) => t.includes('Any'))) typeOnlyImports.push('Any');
+  const typingImports: Array<string> = [];
+  if (typeStrings.some((t) => t.includes('Any'))) typingImports.push('Any');
   // includes (not startsWith): Literal can be nested, e.g. list[Literal["a", "b"]]
-  if (typeStrings.some((t) => t.includes('Literal['))) typeOnlyImports.push('Literal');
-  if (safe.some((f) => !f.isRequired)) typeOnlyImports.push('NotRequired');
-  typeOnlyImports.sort();
-
-  // `cast` is called at runtime (identity fn) so it's a real import, not type-only
-  const runtimeTypingImports = ['TypedDict', 'cast'];
-  if (typeOnlyImports.length) runtimeTypingImports.push('TYPE_CHECKING');
-  runtimeTypingImports.sort();
+  if (typeStrings.some((t) => t.includes('Literal['))) typingImports.push('Literal');
+  const typingExtensionsImports: Array<string> = [];
+  if (safe.some((f) => !f.isRequired)) typingExtensionsImports.push('NotRequired');
+  const usesDeprecated = attrFields.some((f) => f.docs.isDeprecated);
+  if (usesDeprecated) typingExtensionsImports.push('deprecated');
 
   const src = [
     getAutogeneratedBanner('#'),
     'from __future__ import annotations',
     '',
     'import json',
     'import os',
-    `from typing import ${runtimeTypingImports.join(', ')}`,
-    ...(typeOnlyImports.length ? [
-      '',
-      'if TYPE_CHECKING:',
-      `    from typing import ${typeOnlyImports.join(', ')}`,
+    // `cast` is called at runtime (identity fn) so it's a real import, not type-only
+    'from typing import TYPE_CHECKING, TypedDict, cast',
+    '',
+    'if TYPE_CHECKING:',
+    ...(typingImports.length ? [`    from typing import ${typingImports.join(', ')}`] : []),
+    ...(typingExtensionsImports.length ? [`    from typing_extensions import ${typingExtensionsImports.join(', ')}`] : []),
+    '',
+    // declared for type checkers only; resolved lazily by the module `__getattr__` below
+    '    ENV: EnvAttrs',
+    ...(usesDeprecated ? [
+      'else:',
+      '    def deprecated(_message):',
+      '        return lambda fn: fn',
     ] : []),
     ...(skipped.length ? ['', ...skippedKeysComment(skipped, '#')] : []),
+    ...(attrSkipped.length ? [
+      '',
+      `# Keys omitted from EnvAttrs (reserved or mangled names): ${attrSkipped.join(', ')}.`,
+      '# Read them from load_env() instead.',
+    ] : []),
     ...formatPythonTypedDict(
       'Env',
       'Coerced env schema (matches varlock coercion) — values are typed, not raw strings.',
       safe,
-      (field) => getPythonCoercedTypeString(field.coerced),
     ),
+    ...formatPythonEnvAttrs(attrFields),
     '',
     '',
     '# Keys whose values are sensitive — use this to build your own redaction / leak-scanning.',
@@ -191,6 +295,19 @@ export function generatePythonEnvSrc(fields: Array<ResolvedFieldType>): string {
     '        {key: entry["value"] for key, entry in graph["config"].items() if "value" in entry},',
     '    )',
     '',
+    '',
+    // hidden from type checkers: a visible module `__getattr__` would make them accept any name
+    // imported from this module (typos included)
+    'if not TYPE_CHECKING:',
+    '    def __getattr__(name):',
+    '        # `ENV` loads on first access (`from env import ENV`), so importing this module stays',
+    '        # side-effect-free; the instance is cached as a real module attribute afterwards',
+    '        if name == "ENV":',
+    '            env = EnvAttrs(load_env())',
+    '            globals()["ENV"] = env',
+    '            return env',
+    '        raise AttributeError(f"module {__name__!r} has no attribute {name!r}")',
+    '',
   ];
 
   return src.join('\n');

packages/varlock/src/env-graph/lib/type-generation/shared.ts

@@ -149,6 +149,14 @@ export function assertNoFieldNameCollisions(
   }
 }
 
+/**
+ * Strip control chars (user-controlled description / deprecation / link text) so a doc line can't
+ * break out of the generated comment or docstring in any target language.
+ */
+export function sanitizeDocLine(line: string): string {
+  return Array.from(line, (ch) => (ch.charCodeAt(0) < 0x20 ? ' ' : ch)).join('').trimEnd();
+}
+
 /**
  * Language-agnostic doc content lines for a field (description, docs links, deprecation, enum values).
  * Deliberately excludes the key name — repeating it as a comment is noise. Returns `[]` when there's
@@ -168,7 +176,5 @@ export function getFieldDocLines(field: ResolvedFieldType): Array<string> {
   if (typeof field.coerced === 'object' && 'enum' in field.coerced && field.coerced.enum.length) {
     lines.push(`Valid values: ${field.coerced.enum.map((o) => JSON.stringify(o)).join(' | ')}`);
   }
-  // strip any remaining control chars (user-controlled description / deprecation / link text) so a
-  // line can't break out of the generated comment in any target language
-  return lines.map((line) => Array.from(line, (ch) => (ch.charCodeAt(0) < 0x20 ? ' ' : ch)).join('').trimEnd());
+  return lines.map(sanitizeDocLine);
 }