log-cardinality

Counts cardinality of daily active users grouped by application (user agent) type

Source log file records must strictly match the folowing regexp

const logPattern = `^(?P<ipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})` +
`.*\[(?P<timestamp>\d{2}\/\w{3}\/\d{4}:\d{2}:\d{2}:\d{2} (?:\+|\-)\d{4})\].*` +
`emailAddress=(?P<email>[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+).*` +
`(?P<ua>".*"\s"-"$)`

Usage

log-cardinality -in "`find $LOG_PATH_PATTERN/access.log -printf '%p '`" -locale Europe/Moscow

Sample Output

{
  "2019-03-14": {
    "Electron": 719,
    "Web": 148,
    "grpc-java": 182,
    "grpc-objc": 41
  },
  "2019-03-15": {
    "Electron": 742,
    "Web": 143,
    "grpc-java": 187,
    "grpc-objc": 47
  }
}

Depends on

• Highway for hash calculation
• HyperLogLog++ for Go to calculate cardinality