YAF (Yet Another Flowmeter)

YAF is Yet Another Flowmeter. It processes packet data from pcap(3) dumpfiles (as generated by tcpdump(1)) or via live capture from an interface. YAF can capture from standard interfaces using pcap(3), an Endace DAG card, or a Napatech adapter.

It converts packet data into bidirectional flows and then exports those flows to IPFIX Collecting Processes or into an IPFIX-based file format. YAF's output is designed to be used with the SiLK flow analysis tools and the NetSA Aggregated Flow (NAF) toolchain.

YAF also supports partial payload capture. This feature is intended for "banner grabbing" for protocol verification and service presence detection, and is presently experimental.

Why YAF?

YAF is intended as an experimental implementation tracking developments in the IETF IPFIX working group, specifically:

• Bidirectional flow representation
• Archival storage formats
• Structured data export with Deep Packet Inspection (DPI)

It is designed to perform acceptably as a flow sensor on any network where white-box flow collection with commodity hardware is appropriate. However, tradeoffs between raw performance and clarity of design have generally been made in favor of the latter.

Table of Contents

• YAF Toolchain
• How YAF Defines a Flow
• Building from Source
  • Prerequisites
  • Build Instructions
• Troubleshooting
• Known Issues
• Bug Reports & Feature Requests
• License
• Copyright

YAF Toolchain

The YAF toolchain presently consists of several tools:

• yaf: The main flowmeter application.
• yafscii: Converts binary YAF IPFIX output into an ASCII format.
• getFlowKeyHash: A helper tool for working with pcap export options.
• yafMeta2Pcap: Another helper tool for working with pcap export options.

How YAF Defines a Flow

By default, YAF generates flows based on the standard 5-tuple and the VLAN tag, if available. The 5-tuple consists of:

1. Source IP address
2. Destination IP address
3. Source port
4. Destination port
5. Protocol

If YAF is configured with MPLS support, it will use the top three MPLS labels from the stack in addition to the 5-tuple and VLAN to determine the flow. In MPLS mode, it will also export these labels in the IPFIX record.

Furthermore, if YAF is configured with --enable-nonip, it will accept non-IP data and generate flow data using just the MPLS labels. The 5-tuple and VLAN fields will be set to 0.

Building from Source

YAF uses a standard autotools-based build system.

Prerequisites

Before building YAF, you must install its dependencies.

Library	Version	Notes
GLib	>= 2.34.0	Available at gtk.org.
libfixbuf	>= 2.3.0	Available at tools.netsa.cert.org/fixbuf.
libpcap	-	Available at tcpdump.org. Included in many OSes.
libairframe	-	Built and installed with the YAF distribution.
libyaf	-	Built and installed with the YAF distribution.

Optional Dependencies

• Spread: (>= 4.1) For Spread transport support.
• PCRE: For application labeling functionality. Available at pcre.org.
• libdag: For Endace DAG live capture support. Enable with ./configure --with-dag.
• libpcapexpress: For Napatech live capture support. Enable with ./configure --with-napatech.
• libpfring: (>= 6.2.0) For PF_RING and PF_RING ZC support. Available at ntop.org.

Build Instructions

1. Configure the build: If dependencies like libfixbuf are in a non-standard location, you may need to set PKG_CONFIG_PATH.

   export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
   ./configure

2. Compile and Install:

   make
   sudo make install

Troubleshooting

Here are solutions to some common issues.

configure: error: Cannot find a suitable libfixbuf

• Error: configure: error: Cannot find a suitable libfixbuf (>= 2.3.0) (Try setting PKG_CONFIG_PATH): No package 'libfixbuf' found
• Solution: Your shell cannot find the libfixbuf package configuration. Export PKG_CONFIG_PATH to point to where libfixbuf.pc was installed.

  export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

  (Adjust the path if you installed to a different prefix).

yaf: error while loading shared libraries

• Error: yaf: error while loading libraries: libairframe-2.3.0.so.4: cannot open shared object file: No such file or directory
• Solution: The dynamic linker cannot find the YAF libraries. If you installed to a non-standard prefix (like /usr/local), you may need to update the linker cache or set LD_LIBRARY_PATH.

  sudo ldconfig

  or

  export LD_LIBRARY_PATH=/usr/local/lib

couldn't open library "dnsplugin": file not found

• Error: This occurs when using application labeling.
• Solution: YAF cannot find its application labeling plugins. Set the LTDL_LIBRARY_PATH environment variable to the plugin directory (default: /usr/local/lib/yaf).

  export LTDL_LIBRARY_PATH=/usr/local/lib/yaf

  You may need to add this to your startup script if running yaf as a service.

TLS: Failed to load certificate file: ... no start line

• Error: yaf terminating on error: Failed to load certificate file: error:0906D06C:PEM routines:PEM_read_bio:no start line
• Solution: The certificate files given to --tls-ca and --tls-cert must be in PEM format. DER format is not supported.

TLS: Failed to load private key file: ... bad password read

• Error: yaf terminating on error: Failed to load private key file: error:0906A068:PEM routines:PEM_do_header:bad password read
• Solution: The private key file is password-protected, but the YAF_TLS_PASS environment variable was not set. Set the variable to the correct password.

  export YAF_TLS_PASS="your-password-here"

TLS: Failed to load private key file: ... bad decrypt

• Error: yaf terminating on error: Failed to load private key file: error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
• Solution: The private key file is password-protected, and the YAF_TLS_PASS environment variable was set to the incorrect password. Correct the password or remove it from the key file:

  openssl rsa -in key.key -out key.key

Known Issues

• BPF Filtering with DAG: YAF BPF filtering is ignored when using --live dag because libpcap is not used.
• SiLK Export: If YAF is intended to export to SiLK, the --silk command-line flag must be used.
• File Rotation: YAF will not rotate output files if it is not seeing any flow data. It will, however, continue to write process statistics messages to the most recent output file.
• ICMP Port Fields: The destinationTransportPort information element contains ICMP type and code for ICMP/ICMP6 flows. This is nonstandard and may not be interoperable with other IPFIX implementations.

Bug Reports & Feature Requests

Please send bug reports, feature requests, and questions to netsa-help@cert.org.

License

This project is licensed under a GNU GPL 2.0-style license. Please see the LICENSE.txt file for full terms.

Copyright

@DISTRIBUTION_STATEMENT_BEGIN@
YAF 2.16

Copyright 2024 Carnegie Mellon University.

NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING
INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON
UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED,
AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR
PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF
THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF
ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT
INFRINGEMENT.

Licensed under a GNU GPL 2.0-style license, please see LICENSE.txt or
contact permission@sei.cmu.edu for full terms.

[DISTRIBUTION STATEMENT A] This material has been approved for public
release and unlimited distribution.  Please see Copyright notice for
non-US Government use and distribution.

This Software includes and/or makes use of Third-Party Software each
subject to its own license.

DM24-1063
@DISTRIBUTION_STATEMENT_END@