runtime: crash in runtime.(*unwinder).next #73413
Comments
gopherbot
added
the
compiler/runtime
|
I suspect that this is duplicate of #73259. Is the fault address 0x118? You can find the fault address as If so, I'll dup this into the other issue.
Thanks, this is a helpful hint. Can you share a reproducer (even if failure rate is low)? Even if not, it sounds like you are in a better spot than #73259 since you can get it in gdb instead of just in prod. |
|
Also, just to clarify, this is running on linux-arm64, right? |
prattmic
added
WaitingForInfo
|
cc @golang/runtime |
gabyhelp
added
the
BugReport
Unfortunately github.com/dvyukov/go-fuzz did not print go callstack for some reason (only for callstack printing code from libFuzzer), so I had to use gdb to get something more useful.
Here you are. You probably will have to run package fuzz
import (
"crypto/aes"
"crypto/hmac"
"crypto/sha1"
"encoding/binary"
"github.com/pion/rtp"
"github.com/pion/srtp/v3"
)
// function copied from pion code
func aesCmKeyDerivation(label byte, masterKey, masterSalt []byte, outLen int) ([]byte, error) {
// https://tools.ietf.org/html/rfc3711#appendix-B.3
// The input block for AES-CM is generated by exclusive-oring the master salt with the
// concatenation of the encryption key label 0x00 with (index DIV kdr),
// - index is 'rollover count' and DIV is 'divided by'
nMasterKey := len(masterKey)
nMasterSalt := len(masterSalt)
prfIn := make([]byte, 16)
copy(prfIn[:nMasterSalt], masterSalt)
prfIn[7] ^= label
// The resulting value is then AES encrypted using the master key to get the cipher key.
block, err := aes.NewCipher(masterKey)
if err != nil {
return nil, err
}
out := make([]byte, ((outLen+nMasterKey)/nMasterKey)*nMasterKey)
var i uint16
for n := 0; n < outLen; n += block.BlockSize() {
binary.BigEndian.PutUint16(prfIn[len(prfIn)-2:], i)
block.Encrypt(out[n:n+nMasterKey], prfIn)
i++
}
return out[:outLen], nil
}
// This is also mostly copied from pion
func createAuthTag(data []byte, masterKey []byte, masterSalt []byte, authTagLen int) ([]byte, error) {
sessionAuthTag, err := aesCmKeyDerivation(0x01, masterKey, masterSalt, 20)
if err != nil {
return nil, err
}
sessionAuth := hmac.New(sha1.New, sessionAuthTag)
if _, err := sessionAuth.Write(data); err != nil {
return nil, err
}
rocRaw := []byte{0, 0, 0, 0}
_, err = sessionAuth.Write(rocRaw)
if err != nil {
return nil, err
}
authTag := sessionAuth.Sum(nil)[0:authTagLen]
out := make([]byte, len(data)+authTagLen)
copy(out, data)
copy(out[len(data):], authTag)
return out, nil
}
func Fuzz(data []byte) int {
if len(data) < 13 {
return 1
}
key := make([]byte, 16)
salt := make([]byte, 14)
for n := 0; n < len(key); n++ {
key[n] = 1
}
for n := 0; n < len(salt); n++ {
salt[n] = 1
}
profile := srtp.ProtectionProfileAes128CmHmacSha1_80
tagLen := 10
if data[0]%2 == 1 {
profile = srtp.ProtectionProfileAes128CmHmacSha1_32
tagLen = 4
}
data = data[1:]
ssrc := binary.BigEndian.Uint32(data[8:])
data2, err := createAuthTag(data, key, salt, tagLen)
if err != nil {
return 0
}
ctx, err := srtp.CreateContext(key, salt, profile)
ctx.SetROC(ssrc, 0)
if err != nil {
return 0
}
ctx.DecryptRTP(nil, data2, &rtp.Header{})
return 1
}
Our fuzz tests are executed on linux-amd64, my bad. But as I checked we had some internal bug reports from production code running on linux-arm64 with similar crashstacks, where we suspected some other faulty cgo code. Here is one crash from go 1.22.7. I also heard from others that 1.22.4 was affected too. For some reason now this occurs more often than in the past. |
prattmic
removed
the
WaitingForInfo
Go version
1.24.2 linux/arm64
Output of
go envin your module/workspace:What did you do?
We have set of fuzz test which are executed every day. Everything was fine when we were on go 1.23.x. We upgraded to 1.24.0 shortly after it was released and after this we started seeing random crashes in some of our fuzz tests (now we are on 1.24.2). These crashes are not related to input data, attempt to manually feed data printed by fuzzer to fuzz function did not reproduce crash. Call stack printed by fuzzer was useless, so recently we added gdb to get better one. Here is what we got from
bt full:This particular test treats input data vector as encrypted SRTP packet without auth tag (just RTP headed and encypted data), adds calculated Auth Tag at the end and calls DecryptRTP from pion/srtp.
These random crashes which we see occurs in some fuzz tests only. I suspect that bug is somehow related to crypto things, namely AES-CM and SHA-1 (this particular fuzz test uses SRTP_AES128_CM_HMAC_SHA1_80 and SRTP_AES128_CM_HMAC_SHA1_32 profiles).
What did you see happen?
Crash, see above
What did you expect to see?
No crash
The text was updated successfully, but these errors were encountered: