`struct.pack` with format `s` errors on `bytes`-like data

[jakirkham]

Bug report

Bug description:

When using struct.pack with format s, it works with bytes and bytearray. However other bytes-like data (such as memoryview) is not handled raising an error. Here is some sample code show this issue

import struct

b = b"abad"

struct. pack("4s", b)               # Ok
struct. pack("4s", bytearray(b))    # Ok
struct. pack("4s", memoryview(b))   # Error

---

This code appears to contain the corresponding logic

cpython/Modules/_struct.c

Lines 1936 to 1941 in 9a89f1b

	isstring = PyBytes_Check(v);
	if (!isstring && !PyByteArray_Check(v)) {
	PyErr_SetString(state->StructError,
	"argument for 's' must be a bytes object");
	return -1;
	}

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Linked PRs

• gh-148529: Minor improvements of the struct module documentation #148565

[picnixz]

Is it a bug of a documented feature? or is it a feature request?

[jakirkham]

As a reader it was unclear to me as well. Happy to go with a different labeling if needed

The beginning of the struct doc page mentions the Python Buffer Protocol is used in several places. So this looked like an oversight. Though perhaps I'm missing something

[picnixz]

It looks like a bug indeed. We could add support for arbitrary buffers instead of just those. The code hasn't been changed for the past 13 years so I think it'd be a new feature request (but according to the docs it would be a bug so we can also fix this)

I'll ask @serhiy-storchaka for more historical knowledge.

[jakirkham]

Looking through the commit history in case this is helpful for others

Can see some form of that error message goes back 30 years. AFAICT this is the original error message

cpython/Modules/structmodule.c

Lines 1101 to 1105 in 7c3bccd

	if (!PyString_Check(v)) {
	PyErr_SetString(StructError,
	"argument for 's' must be a string");
	goto fail;
	}

IIUC struct added some support for the Python Buffer Protocol in work merged in 20 years ago to commit: 98297ee

Though according to that commit message work began previously. Also that commit message provides only a rough summary of the changes included (which it notes)

A year later struct adds support for other kinds of bytes-like string types in this code path with commit: d6ca546

cpython/Modules/_struct.c

Lines 1645 to 1650 in d6ca546

	isstring = PyString_Check(v);
	if (!isstring && !PyBytes_Check(v)) {
	PyErr_SetString(StructError,
	"argument for 's' must be a string");
	return -1;
	}

A year after that some clean up bring it to the form we recognize today: 72b710a

cpython/Modules/_struct.c

Lines 1648 to 1653 in 72b710a

	isstring = PyBytes_Check(v);
	if (!isstring && !PyByteArray_Check(v)) {
	PyErr_SetString(StructError,
	"argument for 's' must be a string");
	return -1;
	}

These changes all appear to be included since Python 2.7

[serhiy-storchaka]

I think it is both. On one hand, the documentation still refers to "strings" somewhere and do not mentioned that bytearray is accepted for packing as "s". On other hand, there is no reason to limit it to bytes and bytearray, any object providing continuous buffer can be supported.

Historically, it worked with str as 8-bit strings. Then str was replaced with bytes in Python 3, and bytearray was added. References to the buffer protocol are related to the input data for unpacking, not to packing.