v1.0.0-RC1

Announcing v1.0 of ZGrab2 in prerelease!
Assuming there's no regressions or bugs reported in the next month or so, we can tag a release version.

Major Changes

• Adds a DNS rate limiter and per-IP rate limiter. These can be used to cap the number of DNS and connections, respectively, a server will receive. Look at the --help for --dns-rate-limit and --server-rate-limit for more details
• Adds a memcached scan module for scanning the memcached protocol. Thanks to @nicholasingino for this!

Improvements

• Adds CLI flag grouping and more detailed module descriptions in --help
• Adds non-TLS retry to Postgres scans to get servers that don't support TLS. Thanks @UnknownEclipse for this!

New Contributors

• @UnknownEclipse made their first contribution in #573
• @nicholasingino made their first contribution in #551

Full Changelog: v0.2.0...v1.0.0-RC1

v0.2.0

v0.2.0 includes quite a number of changes and improvements over v0.1.8. There were new features added and overall code quality improvements. The most important changes are outlined below.

Breaking Changes

• Addition of dialerGroup to the Scan() module API. This will let ZGrab be used as a library and let clients bring their own dialers/network connections. This theoretically would let you run any application protocol over any transport you want. In practice, several protocols are more tightly coupled and need both a TCP dialer, they'll exchange some info, and then establish a TLS connection. STARTTLS is an example of this. More info in #506.
• Made the default behavior for smtp more reasonable where it can auto-detect support for EHLO vs. HELO in the banner as well as STARTTLS support. Adds CLI override flags to specify behavior if user wishes to override. Removes --send-ehlo, --send-start-tls and --send-helo since these now auto-detect. Adds --send-X-override flags for overriding. #509

Major Changes/Improvements

• Categorizes the CLI flags for modules to group them into Application Options that affect all modules, Basic Options for a module that they all share, and module-specific options. Will let users find the flag they need faster. See #479
• Fixed timeout behavior which would let scans run far longer than expected. Provides the user with --connect-timeout to control initial connection establishment and --target-timeout to bound the runtime of a scan against a given input target. See #537
• User can specify --local-addr and --local-port to bind 1+ IPs to for outgoing connections. See #519
• Changes IPv6 behavior to assume IPv4 and have IPv6 as an optional CLI flag
• Adds support for specifying 1+ DNS resolvers to use for lookups. Default behavior remains to use the OS configured defaults (/etc/resolv.conf on UNIX) See #556
• Add a ZMap-style blocklist. See #553
• Adds a ZMap-style per-second scan status print-out to give constant user feedback on how a scan is proceeding. See #528

New Modules

• AMQP 0.9.1
• PPTP
• MQTT
• socks5

All Changes

• Bump google.golang.org/protobuf from 1.30.0 to 1.33.0 by @dependabot in #414
• build: bump alpine from 3.19 to 3.20 by @dependabot in #442
• New Protocol: AMQP 0.9.1 by @developStorm in #426
• fixed typo in cleanup script by @phillip-stephens in #445
• docs: docker usage by @developStorm in #438
• Fix non-compilation in /lib/ssh/messages_test.go by @phillip-stephens in #446
• style: reformat all go files by @developStorm in #437
• Try to align golang around a single version by @Seanstoppable in #461
• Upgrade golang.org/x/crypto from 0.23.0 to 0.31.0 by @Seanstoppable in #476
• fix: failing ci tests by @developStorm in #478
• Group Basic/TLS Flag Groupings in Modules and interface{} -> any by @phillip-stephens in #479
• build(deps): bump github.com/prometheus/client_golang from 1.14.0 to 1.20.5 by @dependabot in #482
• build(deps): bump golang.org/x/crypto from 0.31.0 to 0.32.0 by @dependabot in #480
• Document the PORT field for CSV-format input by @dlenskiSB in #456
• Upgrade to go 1.23 by @thecsw in #466
• build(deps): bump alpine from 3.20 to 3.21 by @dependabot in #484
• build(deps): bump actions/attest-build-provenance from 1 to 2 by @dependabot in #483
• build(deps): bump docker/build-push-action from 5 to 6 by @dependabot in #453
• Update zgrab's python2 dependency to python3 by @phillip-stephens in #485
• Use bodyTextLen instead of readLen for FailHTTPToHTTPS logic by @Nirusu in #345
• Add support for PPTP by @AlexAQ972 in #470
• Make banner read options configurable via CLI by @svbatalov in #468
• Add support for MQTT v3.1.1 and v5.0 by @AlexAQ972 in #467
• Add support for socks5 by @AlexAQ972 in #471
• Fix ssh by @espegro in #335
• ssh: Port lib/ssh from x/crypto v0.7.0 by @TrueSkrillor in #399
• Bump ubuntu from 16.04 to 24.04 in /docker-runner by @dependabot in #430
• Update Golang minimum version/installation instructions by @phillip-stephens in #488
• Test Automation Performance Improvements + Streamlining by @phillip-stephens in #489
• build(deps): bump golang.org/x/sys from 0.29.0 to 0.30.0 by @dependabot in #490
• build(deps): bump golang.org/x/text from 0.21.0 to 0.22.0 by @dependabot in #491
• build(deps): bump golang.org/x/crypto from 0.32.0 to 0.33.0 by @dependabot in #492
• build(deps): bump golang.org/x/net from 0.34.0 to 0.35.0 by @dependabot in #493
• build(deps): bump github.com/prometheus/client_golang from 1.20.5 to 1.21.0 by @dependabot in #494
• Bump all dependencies by @phillip-stephens in #497
• Generate server certs for MQTT test container by @Seanstoppable in #505
• Don't disable http tee after first header, so that headers of re-directs can be captured and we don't panic by @phillip-stephens in #504
• Upgrade the prometheus dependency by @phillip-stephens in #508
• Fix issues with running subset of integration tests by @phillip-stephens in #510
• Improve SMTP usability by @phillip-stephens in #509
• Add back in EHLO, HELO, and STARTTLS Overrides to SMTP by @phillip-stephens in #512
• Update the zcrypto dependency to bring in TLS 1.3 support by @phillip-stephens in #507
• build(deps): bump github.com/zmap/zcrypto to 4f0ea0eaccac by @developStorm in #513
• modbus - avoid array indexing out-of-bounds by @phillip-stephens in #514
• Fix SHA256 typo by @hegga in #515
• New DialerGroupConfig interface for modules to given greater control over connection (L4/TLS/etc) establishment by @phillip-stephens in #506
• WIP - Add large HTTP file integration test and remove --maxbytes by @phillip-stephens in #516
• Fix redirects not getting followed by @phillip-stephens in #518
• Use context.Deadline in addition to specified timeouts for connections by @phillip-stephens in #517
• build(deps): bump golang.org/x/crypto from 0.36.0 to 0.37.0 by @dependabot in #521
• Throw an error is all we got was a bunch of empty hashes for JARM by @Seanstoppable in #525
• Improve ZGrab Docs/--help by @phillip-stephens in #524
• Let user specify local addr(s) and/or local port(s) to be used for making connections by @phillip-stephens in #519
• Remove min() re-declarations by @phillip-stephens in #527
• Add a per-second status similar to ZDNS by @phillip-stephens in #528
• build(deps): bump golang.org/x/net from 0.38.0 to 0.39.0 by @dependabot in #529
• build(deps): bump github.com/prometheus/client_golang from 1.21.1 to 1.22.0 by @dependabot in #530
• Move setting global TLS parameters to init() so it's run once and -race won't complain by @phillip-stephens in #532
• Remove unused single local addr by @phillip-stephens in #531
• Stabilize conn timeout unit tests by @phillip-stephens in #536
• Fix Multiple Module regression by @phillip-stephens in #534
• Make timeout behavior user-changeable and easy-to-understand by @phillip-stephens in #537
• Downgrade to using v7 of redis, since v8 is buggy right now by @phillip-stephens in #539
• Give user IP that domain resolves to by @phillip-stephens in #533
• Fix timeout issue with setting ReadDeadline after conn creation by @phillip-stephens in #538
• Parse CAs file only if the file is different from last parsed by @phillip-stephens in #540
• Give accurate status for io-timeout by @phillip-stephens in #541
• build(deps): bump golang.org/x/sys from 0.32.0 to 0.33.0 by @dependabot in #543
• build(deps): bump golang.org/x/net from 0.39.0 to 0.40.0 by @dependabot in #545
• Add several linters to CI and fix found i...

v0.1.8

What's Changed

• Fix panics in Modbus scanner by @bwireman in #265
• ftp: Add support for implicit TLS by @codyprime in #264
• Ecubit/fix false positives in #262
• adds implicit tls bool to smtp result by @bwireman in #266
• Add option to compute http BodySHA256 on decoded BodyText by @twschum in #275
• [DATA-970] Increase buffer size in FTP scanner by @grace-murphy in #279
• telnet: Optionally return result even on error by @codyprime in #277
• improves http body decoding and enforces max length by @bwireman in #295
• fix for smtp module by @aspacewalz in #297
• Fix SMB scanner marshaller panic in #299
• Add option to flush output after each line of output by @cablej in #244
• fix issue 288 by @vl4deee11 in #289
• Fix null results in fox module by @vl4deee11 in #294
• Add NativeOS, NTLM, and GroupName to SMBv1 results in #286
• Feature/portable suites by @mzpqnxow in #302
• [BUGFIX] Set SNI on each redirect to avoid handshake failures / incorrect server name by @mzpqnxow in #306
• Specify arbitrary HTTP headers by @mzpqnxow in #284
• Read probe from file (hex probe support) by @AnthraX1 in #269
• Fix incorrect logging calls that don't do formatting by @Mindavi in #290
• preserve protocols order, specified in ini file for multiple mode by @Meteorite in #243
• Feature fail http to https for known status code and substrings by @mzpqnxow in #308
• First draft of GitHub Action by @engn33r in #309
• Added JARM fingerprinting module by @silascutler in #282
• Add error handling for FTP TLS handshake by @hmcguinn in #314
• README.md - minor typo correction by @TomSellers in #320
• smb: allow dynamic length fields of exact length in #323
• add jarm-go to go.mod by @hmcguinn in #321
• remove masking of s7 protocols errors by @hmcguinn in #322
• Adds --hex option to banner module by @svbatalov in #325
• jarm: update jarm to not fail on handshake failure by @aspacewalz in #328
• Fix http module panic by @vl4deee11 in #327
• mongodb: Use OP_QUERY for buildinfo for wire proto < 6 by @dabdine in #330
• smb: fix panic in ntlmssp when unmarshaling by @codyprime in #333
• Fix link to schemas in README by @fileformat in #340
• Add brackets to IPv6 literals with standard ports in http module by @gigaryte in #329
• Add tls.Config to TLSFlags by @corny in #238
• ipp: nameLength should be uint16, not int16 by @codyprime in #350
• retry-https considered harmful: use original result when retry-https encounters an error by @Nirusu in #357
• Remove source IP option by @dadrian in #356
• Add mongodb dbnames by @dabdine in #332
• Make it a non-fatal error when EncodeGrab fails to marshal data by @jamadden in #355
• Add docker support by @creekorful in #354
• fix: typo in smtp scanner by @Tr4cer0 in #361
• fix: typo in smtp scanner by @MaineK00n in #376
• Add TLS support for redis by @ycamper in #374
• Bump golang.org/x/crypto from 0.0.0-20210322153248-0c34fe9e7dc2 to 0.1.0 by @dependabot in #373
• feat(http-request-body): allow users to specify a request body in the HTTP module by @lc in #380
• Bump golang.org/x/crypto from 0.7.0 to 0.17.0 by @dependabot in #402
• Add body support for invisible characters after encryption by @jiaocoll in #393
• Fixes various problems (mostly syntax) in tests by @duckfez in #391
• fix typo in mssql scanner by @dy7ss in #371
• New functionality for zgrab2-banner: md5, sha1, sha256, base64 by @crazylazyowl in #385
• Add tls output to banner when we have a tls connection by @Seanstoppable in #404
• Permissive TLS parsing by @Seanstoppable in #400
• Deal with non-RFC compliant servers by @ycamper in #375
• Add port number to CSV input and output by @AnthraX1 in #397
• Add Install Instructions for Go v17+ by @landxnsec in #382
• http: allow raw header capture (#347) by @codyprime in #349
• Update config.go by @L4puta in #421
• refactor: improve and align ftp tls error msg by @developStorm in #418
• Docs: Add Module Help Usage to README by @developStorm in #417
• Fix Parser for ModuleIdentificationRequest of s7 Protocol by @developStorm in #423
• Add support for custom DNS server by @developStorm in #422
• Add HaSSH to ssh output by @Seanstoppable in #412
• Fix build instructions by @Weltraumschaf in #383
• patch: getVersionNumber function by @developStorm in #425
• Build: Multi-platform Docker Builds and Integration Test by @developStorm in #428
• Bump actions/setup-go from 2 to 5 by @dependabot in #431
• Bump golang.org/x/sys from 0.15.0 to 0.20.0 by @dependabot in #436
• Bump alpine from 3.16 to 3.19 by @dependabot in #432
• Bump golang.org/x/net from 0.10.0 to 0.25.0 by @dependabot in #435
• Add test steps to build by @Seanstoppable in #411
• Fix tests and boundary condition by @Seanstoppable in #410
• fix: close CPU profile by @testwill in #416
• Add TLS support for fox by @Seanstoppable in #403
• Bump golang from 1.18.3-alpine3.16 to 1.20.4-alpine3.16 by @dependabot in #433

New Contributors

• @twschum made their first contribution in #275
• @grace-murphy made their first contribution in #279
• @aspacewalz made their first contribution in #297
• @cablej made their first contribution in #244
• @vl4deee11 made their first contribution in #289
• @Mindavi made their first contribution in #290
• @engn33r made their first contribution in #309
• @silascutler made their first contribution in #282
• @hmcguinn made their first contribution in #314
• @TomSellers made their first contribution in #320
• @svbatalov made their first contribution in #325
• @dabdine made their first contribution in #330
• @fileformat made their first contribution in #340
• @gigaryte made their first contribution in #329
• @Nirusu made their first contribution in #357
• @jamadden made their first contribution in #355
• @creekorful made their first contribution in #354
• @Tr4cer0 made their first contribution in #361
• @MaineK00n made their first contribution in #376
• @ycamper made their first contribution in #374
• @dependabot made their first contribution in #373
• @lc made their first contribution in #380
• @jiaocoll made their first contribution in #393
• @duckfez made their first contribution in #391
• @dy7ss made their first contribution in #371
• @crazylazyowl made their first contribution in #385
• @Seanstoppable made their first contribution in #404
• @landxnsec made their first co...

Allow for programaitc access of ssh result types

Makes some the types used in the SSH scanner's result exported
Namely:

• KexInitMsg
• Algorithms
• DirectionAlgorithms

#260

v0.1.4

modules/http: Add option to suppress errors when max redirects exceed…

v0.1.2

Adds support for a custom "fake" DNS resolver that can optionally be added to a Dialer. The http scanner now uses the fake DNS resolver to force resolutions to a specified IP address, if both an IP address and domain name are specified in the scan. This fixes the issue where the http scanner would ignore the specified IP address if a domain name was specified.

v0.1.0

Go module compatible release. Mostly works.