ctypes _pack_ & _align_ documentation needs elaboration

[bordenc]

Documentation

The documentation on the _pack_ and _align_ variables for the Structure and Union classes needs elaboration. Currently, both say something to the effect of "An optional small integer that allows overriding the alignment of the structure..."

The documentation is clear that setting either parameter to 0 has no effect. However, it is unclear what effect setting the value to anything other than 0 has. Is it a true/false value? Does it set the number of bits to which each field should be aligned and/or padded and/or truncated? Does it override the third tuple parameter when setting fields? What happens if it's set to a negative number?

[terryjreedy]

I think that the int should be bytes rather than bits, but don't know. So I agree that the doc should say. (Even though one could argue that only those who already know should set these. ;-)

@hugovk @encukou You both have commited to this doc recently, so I hope one of you knows the answer or who would.

[encukou]

The documentation is not organized effectively, and it assumes that the reader is familiar struct alignment and similar concepts; but (in 3.14+) the necessary information should be there.
Clarifications are welcome of course.

---

The behaviour was cleaned up for 3.14, so that's what I'll talk about. In 3.13 and below you might get surprising results instead.

Both of these are in bytes.

For negative values, Python 3.14+ fails with ValueError.

This being ctypes, the behaviour should match the C compiler. If you use a value that the compiler would reject, Python can fail, do something reasonable (e.g. ignore a limit), or do something unreasonable (like aligning offsets to multiples of 3). Future versions of Python change the latter behaviour (with an appropriate deprecation period, unless there's an outright bug).
I don't think we should document exactly what happens on bad input.

_pack_ = n is equivalent to MSVC's #pragma pack(n) (see here). In fact it forces MSVC-style struct layout; the 3.14 docs make this more explicit. (The clarification wasn't backported because it's part of a functionality change & deprecation; 3.14 docs will become the default in a few months.)
MSVC requires that the value is 1, 2, 4, 8, or 16.
Python enforces that this is a positive power of two but the upper limit is INT_MAX (so, no limit in practice).

_align_ = n is equivalent to GCC's __attribute__((aligned(n))) or MSVC's __declspec(align(N)). (And would be similar to C23's alignas if that was usable on whole structs.)
The compilers/standards require this to be a small positive power of two. (“small” is implementation-defined and often platform-specific. GCC doesn't seem to limit this. Values bigger than 64 don't make much sense anyway.)
Python only requires that this is positive, but if you don't use a power-of-two the result is unlikely to be usable.

[bordenc]

Love @encukou's explanation. Yes, I agree, the ctypes documentation shouldn't be an introduction to C constructs. There's more than enough documentation out there for that.

What is important is explicitly stating these compiler mappings. I would have, I think reasonably, thought that _pack_ would have effects on GNU-based compilers, when @encukou's commentary says it has none.

For a total amateur like me, I found the documentation pretty good - when supplemented with other sources. I suggest the documentation focus on the Python-to-C mappings, as mentioned, and "How to do C things in Python."

For example, the wrapper library I'm working on needs me to pass mutable pointers-to-pointers-to-structs in function calls. The documentation doesn't make immediately clear how to do this. There seem to be multiple ways of creating different types of pointers, and I'm not entirely sure at which nesting level I should call the "constructor" (or whatever it is in C). The code that seems to work is:

# ctypes imported and library loaded, yadda yadda

obj = library.StructType()  # I don't initalise my struct because that's what the function() is for.
ptr = ctypes.pointer(ctypes.pointer(obj))
result = library.function(ptr)  # calling C function: int function(StructType **arg);

It seems to work, although there may be better ways of doing this.

The other thing, which I haven't played with but will probably come up shortly, is casting between types. If I need to cast a char to an int, should I

c_char_var = ctypes.c_char('0')
c_int_cast = ctypes.c_int(c_char_var)

...or do something else?

Last thing, I promise: the first time I played with my library function, I got a seg fault + core dump. Is there any way to handle that in Python, or do I have to run my code in GDB because no Python debugger can handle the symbols?

For further clarity, I'm on Debian, so the MSVC stuff doesn't apply to me... yet.

[terryjreedy]

I don't think we should document exactly what happens on bad input.

I agree. Just specify what is normally needed; something like "bytes, a small power of two (1, 2, 4, 8, 16, 32?, 64?), depending on how the code being access was compiled."

[bordenc]

I don't think we should document exactly what happens on bad input.

I agree. Just specify what is normally needed; something like "bytes, a small power of two (1, 2, 4, 8, 16, 32?, 64?), depending on how the code being access was compiled."

Fair. If the function raises an exception on bad input, noting which exception(s) it raises should be sufficient. That way, at least, a programmer knows which exceptions to anticipate if they aren't hard-coding the values. However, if it does something reasonably unpredictable, I think that should be noted. I don't think it's fair to expect a user to read the implementer's mind.

The idea is to reduce the number of issues being filed, assuming people RTFM.