drgn 0.0.21

This release adds lots of new helpers and fixes some important bugs.

New features:

• Helpers for lockless linked lists in the Linux kernel were added: drgn.helpers.linux.llist. Contributed by Imran Khan.
• A helper to find the slab cache that a virtual address came from was added: drgn.helpers.linux.slab.find_containing_slab_cache(). Contributed by Nhat Pham.
• A drgn.helpers.common package was created to contain helpers that can be used with any program (which may have program-specific additional behavior).
• A helper to identify an arbitrary address (e.g., as a symbol or slab object) was added: drgn.helpers.common.memory.identify_address(). Contributed by Nhat Pham.
• PageFoo() helpers to check various struct page flags were added to drgn.helpers.linux.mm.
• Helpers for working with compound pages were added to drgn.helpers.linux.mm: compound_head(), compound_nr(), compound_order(), and page_size().
• A helper to get the CPU that a task last ran on was added: drgn.helpers.linux.sched.task_cpu().
• Automatic pretty-printing in IPython/Jupyter of drgn.Object, drgn.Type, drgn.StackTrace, and drgn.StackFrame was added. Contributed by Shung-Hsi Yu.
• drgn.StackTrace.prog was added as a way to get the program that a stack trace came from.

Bug fixes:

• Getting stack traces from a kernel core dump of threads that were running at the time of the crash was fixed for Linux < 4.9 and >= 5.16.
• drgn.helpers.linux.per_cpu() and per_cpu_ptr() were fixed to work for per-CPU variables defined in kernel modules.
• Reading from pages that were excluded by makedumpfile was changed to return zeroes instead of raising a FaultError. This is unfortunately necessary because we cannot distinguish between pages that were excluded because their contents were zero and pages that were excluded for other reasons. Contributed by Glen McCready.
• A segfault when looking for a variable in a stack frame caused by strange debug symbols emitted by Clang in certain situations was fixed.
• A reference leak every time a FaultError is raised inside of drgn was fixed.
• A use after free when setting an object from a part of itself in libdrgn was fixed. The Python interface is not affected.
• The recommendation for how to get VMCOREINFO in QEMU guest memory dumps was fixed to suggest the correct Linux kernel configuration options.
• A spelling error in a DWARF parsing error message was fixed. Contributed by Michel Alexandre Salim.

API changes:

• escape_ascii_character(), escape_ascii_string(), decode_flags(), and decode_enum_type_flags() were moved from drgn.helpers to drgn.helpers.common.format.
• enum_type_to_class() was moved from drgn.helpers to drgn.helpers.common.type.

Documentation:

• openSUSE installation instructions were added.
• libkdumpfile installation instructions were updated to reflect that it is now packaged on some Linux distributions.
• Python type signatures are now formatted more concisely in the documentation.
• Overloaded helpers are now documented more concisely.
• Various small editorial and formatting issues were fixed.

Other improvements:

• Linux kernel support was tested up to Linux 6.0.
• drgn.helpers.linux.bpf.cgroup_bpf_prog_for_each() was updated to work on Linux 6.0.

Internals:

• Some racy stack tracing unit tests were fixed.
• Linux kernel memory management helper unit tests were enabled on AArch64.
• The VM testing setup no longer depends on BusyBox.
• -Wimplicit-fallthrough was enabled for builds.
• A syscall number table and normalized machine name were added to util.py
  for use in test cases and the VM testing setup.
• Some renaming was done to prepare for the upcoming module API.
• The libdrgn-internal string_builder API was improved.

drgn 0.0.20

This release adds full AArch64 (ARM64) support, a few more helpers, and other small fixes and improvements.

New features:

• Stack traces were implemented for AArch64.
• Virtual address translation was implemented for AArch64.
• Additional registers are now available for the initial stack frame on x86-64: rflags, es, cs, ss, ds, fs, gs, fs.base, gs.base.
• The lr register is now available for stack frames on ppc64.
• A helper for looking up cgroups by path was added:drgn.helpers.linux.cgroup.cgroup_get_from_path().
• A helper for walking kernfs paths was added: drgn.helpers.linux.kernfs.kernfs_walk().
• Helpers for translating to and from physical addresses were added to drgn.helpers.linux.mm: PFN_PHYS(), PHYS_PFN(), page_to_phys(), phys_to_page(), phys_to_virt(), virt_to_phys().
• Helpers for iterating BTF objects and BPF links were added to drgn.helpers.linux.bpf: bpf_btf_for_each() and bpf_link_for_each(). Contributed by Quentin Monnet.
• Support for UTF character types (e.g., char8_t, char16_t, char32_t) was added. Contributed by Kevin Svetlitski.
• Support for floating-point reference objects other than 32 or 64 bits was added (values are still not implemented). Contributed by Kevin Svetlitski.

Bug fixes:

• drgn -s/--symbols (and the underlying drgn.Program.load_debug_info()) now works for kernel module debug info files generated by objcopy --only-keep-debug instead of failing with an error.
• drgn.helpers.linux.slab.slab_cache_for_each_allocated_object() was fixed when CONFIG_SLAB_FREELIST_HARDENED is enabled.
• drgn.helpers.linux.slab.slab_cache_for_each_allocated_object() was fixed when using the SLAB allocator on kernels before v5.17.
• The drgn.helpers.linux.slab.slab_cache_is_merged() helper is now properly added to drgn.helpers.linux.slab.__all__ so that it is included in drgn.helpers.linux and automatically imported in the CLI.
• The cgroup_bpf_prog_for_each() and cgroup_bpf_prog_for_each_effective() helpers in drgn.helpers.linux.bpf were fixed to support kernels back to v4.10.
• The CLI was fixed to add the current directory to sys.path when running in script mode. Contributed by Logan Gunthorpe.
• Missing documentation was added for the validate_rbtree() and validate_rbtree_inorder_for_each_entry() validators in drgn.helpers.linux.rbtree.
• An optimization to coalesce virtual address reads (used for vmcore reads and the access_process_vm() and access_remote_vm() helpers in drgn.helpers.linux.mm) was fixed. Previously, the optimization was skipped when it should be used and in rare situations could be used when it shouldn't have been, causing incorrect reads.
• A bug when a virtual address read straddles the non-canonical address range on x86-64 was fixed.
• Fallback stack unwinding on ppc64 when debugging information is not available was fixed.
• Reading registers from stack traces with a different byte order was fixed to swap the byte order.
• Automatic lookup of incomplete types was fixed when the type is in a C++ namespace. Contributed by Jay Kamat.

Other improvements:

• Linux kernel support was tested up to Linux v5.19-rc8.
• The address range of kernel modules is now determined more efficiently.
• The bpf_map_for_each() and bpf_prog_for_each() helpers in drgn.helpers.linux.bpf were made slightly more efficient by avoiding redundant type lookups.
• drgn will now detect a QEMU memory dump without VMCOREINFO and suggest how to get a dump that drgn can use.

Internal:

• The format for defining architecture registers was changed from a custom language to Python.
• The address/struct page translation helpers in drgn.helpers.linux.mm were made mostly architecture-agnostic.
• The unit tests for the address/struct page translation helpers in drgn.helpers.linux.mm were made more thorough.
• Unit tests were added for the BPF helpers.

drgn 0.0.19

Feature-wise, this release mainly adds more helpers, including Linux slab helpers and new "validator" helpers. Internally, a lot of work was done to improve testing.

New features:

• Helpers for the Linux kernel slab allocator were added (slab_cache_is_merged(), for_each_slab_cache(), find_slab_cache(), print_slab_caches(), and slab_cache_for_each_allocated_object() in drgn.helpers.linux.slab). Contributed by Alex Zhu.
• The concept of "validators", special helpers which validate a data structure, was introduced.
• Validators for linked lists were added (validate_list(), validate_list_for_each(), and validate_list_for_each_entry() in drgn.helpers.list).
• Validators for red-black trees were added (validate_rbtree() and validate_rbtree_inorder_for_each_entry() in drgn.helpers.linux.rbtree).
• The drgn.helpers.linux.rbtree.RB_EMPTY_ROOT() helper was added.
• The i386, AArch64, Arm, and RISC-V architectures are now recognized and were added to Architecture. (Architecture-specific features have not been implemented for those architectures yet.)
• Support for integer reference objects larger than 64 bits was added (values are still not implemented). Contributed by Jay Kamat.

Bug fixes:

• Program.crashed_thread() was fixed when debugging a non-SMP Linux kernel.
• A bug in 32-bit builds that could cause poor performance due to hash collisions was fixed.

Other improvements:

• Linux kernel support was tested up to v5.18-rc7.
• Applying ELF relocations (used for kernel modules) was optimized for ppc64, i386, AArch64, Arm, and RISC-V.
• Documentation for StackFrame.name was expanded.

Internal:

• The virtual machine testing setup was expanded to test on kernels using all three Linux kernel slab allocator implementations (SLUB, SLAB, SLOB) and both SMP and !SMP.
• The virtual machine testing setup was also expanded to use a custom kernel module to test helpers that couldn't previously be tested.
• Flake8 was added to the pre-commit configuration.

drgn 0.0.18

This is a small release adding a missing test file.

drgn 0.0.17

New features:

• A new API representing threads was added. The Thread class represents a thread. Thread.stack_trace() returns the stack trace for the thread. Program.threads() returns an iterator over the threads in the program. Program.thread() returns a thread given its thread ID. Program.main_thread() returns the main thread of the program. Program.crashed_thread() returns the thread that caused the program to crash. Contributed by Kevin Svetlitski and Mykola Lysenko.
• Program.symbols() was added. It returns all ELF symbols in the program, optionally filtered by a name or address. Contributed by Stephen Brennan.
• Support for looking up C++ classes with Program.type() was added. Contributed by Mykola Lysenko.
• Support for setting the default program language (Program.language) was added.
• Helpers for getting the Linux kernel log buffer were added (drgn.helpers.linux.printk.get_printk_records() and drgn.helpers.linux.printk.get_dmesg()). Contributed by Alex Zhu.
• Helpers for translating between Linux kernel socket objects and inode objects were added (drgn.helpers.linux.net.SOCKET_I() and drgn.helpers.linux.net.SOCK_INODE()). Contributed by Peilin Ye.
• A helper for getting the Linux kernel's idle tasks was added (drgn.helpers.linux.sched.idle_task()).
• A helper for decoding Linux kernel struct page flags was added (drgn.helpers.linux.mm.decode_page_flags()).
• Generic helpers for decoding flags were added (drgn.helpers.decode_flags() and drgn.helpers.decode_enum_type_flags()).

Bug fixes:

• A crash when encountering kernel modules processed by objcopy --only-keep-debug was fixed. Contributed by Peilin Ye.
• An endianness bug when debugging kernel modules that use a different byte order than the host was fixed.
• A compilation error on older versions of GCC was fixed. Contributed by Alakesh Haloi.
• libdrgn is now linked against libm to fix undefined references to rint.
• drgn.helpers.linux.percpu.per_cpu_ptr() was fixed to support !SMP kernels.
• A TypeError when drgn.helpers.linux.rbtree.rb_find() doesn't find a match was fixed.
• A memory leak when using an invalid TypeTemplateParameter was fixed.
• A missing type annotation for drgn.Language.CPP was added.
• drgn_stack_trace_destroy() was fixed to allow a NULL argument like other destroy() functions.

Other improvements:

• Linux kernel support was tested up to v5.17-rc6.
• The CLI now diagnoses the common mistake of passing a core dump file where a script is expected.
• drgn.helpers.linux.list.list_for_each_entry(), drgn.helpers.linux.list.list_for_each_entry_reverse(), drgn.helpers.linux.list.hlist_for_each_entry(), drgn.helpers.linux.list_nulls.hlist_nulls_for_each_entry(), and drgn.helpers.linux.rbtree.rbtree_inorder_for_each_entry() were optimized to only look up the entry type once per call.
• Parsing of type names in Program.type() was slightly optimized.
• Lookups of built-in kernel variables (e.g., UTS_RELEASE, PAGE_SIZE) were slightly optimized.
• Program.type() now accepts a Type. This makes it easier to write helpers that accept a type name or a Type.

Build system:

• Python is now required to build libdrgn even if the Python bindings are disabled.
• GNU Awk is no longer required to build drgn.
• Additional compiler warnings are now enabled by default. Developers should enable them as errors.
• Configuration options for developers to enable ASan (contributed by Kevin Svetlitski) and UBSan were added.

Internal:

• Developer documentation was improved.
• pre-commit was configured to enforce coding standards. Contributed by Stephen Brennan.
• s390x and EPEL 9 builds were enabled on Packit. Contributed by Davide Cavalca.
• Test cases using /proc/vmcore in a kdump environment were added. Contributed by Kevin Svetlitski.
• Test cases for the Linux kernel red-black tree and linked list helpers were added.

drgn 0.0.16

This is a small release to fix a couple of issues with debuginfod.

Bug fixes:

• A crash or hang when downloading debugging information from debuginfod was fixed.

Other improvements:

• The CLI now prints debuginfod download progress.
• Some missing files were added to the source distribution.

drgn 0.0.15

New features:

• Symbol.binding and Symbol.kind were added to provide more information about symbols (e.g., whether a symbol is global or local, whether it refers to a function or variable).
• Program.symbol(name) now falls back to returning a local symbol if a matching global symbol was not found.
• Helpers for looking up network namespaces were added (drgn.helpers.linux.net.for_each_net() and drgn.helpers.linux.net.get_ns_by_{inode,fd}()).
• More networking helpers were added (drgn.helpers.linux.net.netdev_for_each_tx_queue() and drgn.helpers.linux.tc.qdisc_lookup()).
• Partial support for alternate DWARF files (.gnu_debugaltlink) was added. These are created by DWZ and used by Linux distros in some debugging symbol packages. Types and objects from the alternate file can not yet be looked up by name.

Bug fixes:

• A workaround was added for a bug in elfutils 0.183-0.185 that causes crashes when targeting userspace core dumps.
• Reading memory from a core dump was fixed to return a fault error instead of zeros for regions that were not saved in the core dump.
• Type name formatting was fixed to not include the entire definition of anonymous types.
• Parsing of DWARF representing an enum type which is backed by a typedef was fixed (now the underlying type is used).
• ELF core dump note matching was fixed to use an exact name match instead of a prefix match.
• Support for variable locations in DWARF 3 files was fixed.
• Support for structure member locations in DWARF 2 files was fixed.
• A memory leak when targeting a makedumpfile compressed kernel core dump was fixed.
• Indexing debugging symbols for a C++ namespace was fixed to be parallelized as intended.
• Incorrect memory allocation error checks were fixed for a couple of hash table insertions.
• Incorrect cleanup after failing to index DWARF information was fixed.
• An incorrect error when failing to get the build ID of an ELF file was fixed.
• A dropped error when applying ELF relocations was fixed.
• A missing error check in the C implementation of drgn.helpers.linux.pid.pid_task() was added.
• A missing error check when parsing C type names was fixed.
• An incorrect error return when memory allocation failed while parsing an array type from DWARF was fixed.

Other improvements:

• Memory usage after loading debugging symbols was decreased (by more aggressively freeing unneeded memory and by lazily allocating memory for C++ namespaces).
• Stack trace variable resolution in inline functions was optimized (by optimizing how we look up the context of an inline function).
• Type and object lookups were optimized (by avoiding OpenMP overhead after the DWARF information was already indexed).
• Installation documentation was improved.
• Documentation for getting debugging symbols was added.
• Linux 5.15 and 5.16 were tested and did not require any changes.

Internal:

• Debugging information handling was reorganized to prepare for upcoming changes.
• Tests were added for the cgroup helpers.
• Packit configuration was added.

drgn 0.0.14

New features:

• Support for DWARF 5 was added.
• Object.from_bytes_() was added. It creates an object from its binary representation.
• Object.to_bytes_() was added. It gets the binary representation of an object.
• Helpers for Linux kernel NUMA node masks were added (drgn.helpers.linux.nodemask).
• Helpers for Linux kernel bit operations were added (drgn.helpers.linux.bitops).
• Helpers for getting network devices by name or interface index were added (drgn.helpers.linux.net.netdev_get_by_{index,name}()).
• drgn.helpers.linux.sched.task_state_to_char() was updated for Linux v5.14.

Fixes:

• A bug when translating values from little-endian to big-endian was fixed.
• A workaround for a bug in elfutils < 0.177 was (re)added. The bug causes stdin to be closed and another file descriptor to be leaked when targeting a userspace core dump.
• Support for the kdump format was fixed (it was broken in v0.0.12 and v0.0.13).
• A potential buffer overread caused by DWARF information with an invalid DW_AT_strp value was fixed.
• A potential buffer overread caused by DWARF information with an invalid 64-bit size on a 32-bit system was fixed.
• Missing support for the DWARF DW_FORM_block encoding was added.
• Support for DW_FORM_ref_addr in DWARF 2 files was fixed to use the correct size.
• "Non-canonical" LEB128 values are now handled correctly.
• Stack trace object lookups and source location lookups were fixed for files with incomplete .debug_aranges information (e.g., the Linux kernel when compiled by Clang).
• drgn.helpers.linux.cpumask.for_each_cpu() was fixed to not read out of bounds in kernels configured with CONFIG_CPUMASK_OFFSTACK (e.g., Fedora kernels).

vmtest Assets

Assets for vmtest. (This is a dummy GitHub release for storing assets, not a real release of drgn.)