Jules/integ 851 profile (#1)

Author: Juliya Smith

CHANGELOG.md

@@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 The intended audience of this file is for py42 consumers -- as such, changes that don't affect
 how a consumer would use the library (e.g. adding unit tests, updating documentation, etc) are not captured here.
 
+## Unreleased
+
+### Removed
+- Removed config file settings and `-c` CLI arg. Use `c42sec profile set`.
+- Removed `--clear-password` CLI argument. Use `c42sec profile set -p`. You will be prompted.
+
+### Added
+- Added ability to view your profile: `c42sec profile show`.
+
+### Changed
+- Renamed `c42aed` to `c42sec`.
+- Moved CLI arguments `-s`, `-u`, and `--ignore-ssl-errors` to `c42sec profile set` command.
+
+
 ## 0.1.1 - 2019-10-29
 
 ### Fixed

README.md

@@ -1,4 +1,4 @@
-# c42seceventcli - AED
+# c42sec
 
 The c42seceventcli AED module contains a CLI tool for extracting AED events as well as an optional state manager 
 for recording timestamps. The state manager records timestamps so that on future runs,
@@ -10,170 +10,26 @@ you only extract events you did not previously extract.
 - Code42 Server 6.8.x+
 
 ## Installation
-Until we are able to put `py42` and `c42secevents` on PyPI, you will need to first install them manually.
-
-`py42` is available for download [here](https://confluence.corp.code42.com/pages/viewpage.action?pageId=61767969#py42%E2%80%93Code42PythonSDK-Downloads).
-For py42 installation instructions, see its [README](https://stash.corp.code42.com/projects/SH/repos/lib_c42_python_sdk/browse/README.md).
-
-`c42secevents` is available [here](https://confluence.corp.code42.com/display/LS/Security+Event+Extractor+-+Python).
-For `c42secevents` installation instructions, see its [README](https://stash.corp.code42.com/projects/INT/repos/security-event-extractor/browse/README.md).
-
-Once you've done that, install `c42seceventcli` using:
+Install `c42sec` using:
 
 ```bash
 $ python setup.py install
 ```
 
 ## Usage
 
-A simple usage requires you to pass in your Code42 authority URL and username as arguments:
-
-```bash
-c42aed -s https://example.authority.com -u [email protected]
-```
-        
-Another option is to put your Code42 authority URL and username (and other arguments) in a config file. 
-Use `default.config.cfg` as an example to make your own config file; it has all the supported arguments.
-The arguments in `default.config.cfg` mirror the CLI arguments.
-
-```buildoutcfg
-[Code42]
-c42_authority_url=https://example.authority.com
-[email protected]
-```
-
-Then, run the script as follows:
-
-```bash
-c42aed -c path/to/config
-```
-
-To use the state management service, simply provide the `-r` to the command line.
-`-r` is particularly useful if you wish to run this script on a recurring job:
-
-```bash
-c42aed -s https://example.authority.com -u [email protected] -r
-```
-
-If you are using a config file with `-c`, set `record_cursor` to True:
-
-```buildoutcfg
-[Code42]
-c42_authority_url=https://example.authority.com
-[email protected]
-record_cursor=True
-```
-By excluding `-r`, future runs will not know about previous events you got, and 
-you will get all the events in the given time range (or default time range of 60 days back). 
-
-To clear the cursor:
-
-```bash
-c42aed -s https://example.authority.com -u [email protected] -r --clear-cursor
-```
-There are two possible output formats.
-
-* CEF
-* JSON
-
-JSON is the default. To use CEF, use `-o CEF`:
+First, set your profile
 
 ```bash
-c42aed -s https://example.authority.com -u [email protected] -o CEF
-```
-
-Or if you are using a config file with `-c`:
-
-```buildoutcfg
-[Code42]
-c42_authority_url=https://example.authority.com
-[email protected]
-output_format=CEF
+c42sec profile set -s https://example.authority.com -u [email protected] -p
 ```
 
-There are three possible destination types to use:
-
-* stdout 
-* file - writing to a file
-* server - transmitting to a server, such as syslog
+`-p` will prompt for your password securely. If your username does not have a password stored, you will be prompted anyway.
 
-The program defaults to `stdout`. To use a file, use `--dest-type` and `--dest` this way:
+To ignore SSL errors, do:
 
 ```bash
-c42aed -s https://example.authority.com -u [email protected] --dest-type file --dest name-of-file.txt
-```
-
-To use a server destination (like syslog):
-
-```bash
-c42aed -s https://example.authority.com -u [email protected] --dest-type server --dest https://syslog.example.com
-```
-
-Both `destination_type` and `destination` are possible fields in the config file as well.
-
-You can also use CLI arguments with config file arguments, but the program will favor the CLI arguments.
-
-If this is your first time running, you will be prompted for your Code42 password.
-
-If you get a keychain error when running this script, you may have to add a code signature:
-
-```bash
-codesign -f -s - $(which python)
-```
-
-All errors are sent to an error log file named `c42seceventcli_aed_errors.log` 
-located in your user directory under `.c42seceventcli/log`.
-
-Full usage:
-
-```
-usage: c42aed [-h] [--clear-cursor] [--reset-password] [-c CONFIG_FILE]
-              [-s C42_AUTHORITY_URL] [-u C42_USERNAME] [-b BEGIN_DATE] [-i]
-              [-o {CEF,JSON}]
-              [-t [{SharedViaLink,SharedToDomain,ApplicationRead,CloudStorage,RemovableMedia,IsPublic} [{SharedViaLink,SharedToDomain,ApplicationRead,CloudStorage,RemovableMedia,IsPublic} ...]]]
-              [-d--debug] [--dest-type {stdout,file,server}]
-              [--dest DESTINATION] [--dest-port DESTINATION_PORT]
-              [--dest-protocol {TCP,UDP}] [-e END_DATE | -r]
-
-optional arguments:
-  -h, --help            show this help message and exit
-  --clear-cursor        Resets the stored cursor.
-  --reset-password      Clears stored password and prompts user for password.
-  -c CONFIG_FILE, --config-file CONFIG_FILE
-                        The path to the config file to use for the rest of the
-                        arguments.
-  -s C42_AUTHORITY_URL, --server C42_AUTHORITY_URL
-                        The full scheme, url and port of the Code42 server.
-  -u C42_USERNAME, --username C42_USERNAME
-                        The username of the Code42 API user.
-  -b BEGIN_DATE, --begin BEGIN_DATE
-                        The beginning of the date range in which to look for
-                        events, in YYYY-MM-DD UTC format OR a number (number
-                        of minutes ago).
-  -i, --ignore-ssl-errors
-                        Do not validate the SSL certificates of Code42
-                        servers.
-  -o {CEF,JSON}, --output-format {CEF,JSON}
-                        The format used for outputting events.
-  -t [{SharedViaLink,SharedToDomain,ApplicationRead,CloudStorage,RemovableMedia,IsPublic} [{SharedViaLink,SharedToDomain,ApplicationRead,CloudStorage,RemovableMedia,IsPublic} ...]], --types [{SharedViaLink,SharedToDomain,ApplicationRead,CloudStorage,RemovableMedia,IsPublic} [{SharedViaLink,SharedToDomain,ApplicationRead,CloudStorage,RemovableMedia,IsPublic} ...]]
-                        To limit extracted events to those with given exposure
-                        types.
-  -d--debug             Turn on debug logging.
-  --dest-type {stdout,file,server}
-                        The type of destination to send output to.
-  --dest DESTINATION    Either a name of a local file or syslog host address.
-                        Ignored if destination type is 'stdout'.
-  --dest-port DESTINATION_PORT
-                        Port used when sending logs to server. Ignored if
-                        destination type is not 'server'.
-  --dest-protocol {TCP,UDP}
-                        Protocol used to send logs to server. Ignored if
-                        destination type is not 'server'.
-  -e END_DATE, --end END_DATE
-                        The end of the date range in which to look for events,
-                        in YYYY-MM-DD UTC format OR a number (number of
-                        minutes ago).
-  -r, --record-cursor   Only get events that were not previously retrieved.
+c42sec profile set --ignore-ssl-errors true
 ```
 
 # Known Issues

c42sec/.DS_Store

@@ -1,5 +1,7 @@
 import sqlite3
-from c42seceventcli.common.util import get_user_project_path
+from c42sec.util import get_user_project_path
+
+_INSERTION_TIMESTAMP_FIELD_NAME = u"insertionTimestamp"
 
 
 class SecurityEventCursorStore(object):
@@ -48,3 +50,36 @@ def _is_empty(self):
             query_result = cursor.fetchone()
             if query_result:
                 return int(query_result[0]) <= 0
+
+
+class AEDCursorStore(SecurityEventCursorStore):
+    _PRIMARY_KEY = 1
+
+    def __init__(self, db_file_path=None):
+        super(AEDCursorStore, self).__init__("aed_checkpoint", db_file_path)
+        if self._is_empty():
+            self._init_table()
+
+    def get_stored_insertion_timestamp(self):
+        rows = self._get(_INSERTION_TIMESTAMP_FIELD_NAME, self._PRIMARY_KEY)
+        if rows and rows[0]:
+            return rows[0][0]
+
+    def replace_stored_insertion_timestamp(self, new_insertion_timestamp):
+        self._set(
+            column_name=_INSERTION_TIMESTAMP_FIELD_NAME,
+            new_value=new_insertion_timestamp,
+            primary_key=self._PRIMARY_KEY,
+        )
+
+    def reset(self):
+        self._drop_table()
+        self._init_table()
+
+    def _init_table(self):
+        columns = "{0}, {1}".format(self._PRIMARY_KEY_COLUMN_NAME, _INSERTION_TIMESTAMP_FIELD_NAME)
+        create_table_query = "CREATE TABLE {0} ({1})".format(self._table_name, columns)
+        insert_query = "INSERT INTO {0} VALUES(?, null)".format(self._table_name)
+        with self._connection as conn:
+            conn.execute(create_table_query)
+            conn.execute(insert_query, (self._PRIMARY_KEY,))

c42seceventcli/__init__.py renamed to c42sec/__init__.py

@@ -0,0 +1,33 @@
+from argparse import ArgumentParser
+from c42sec.profile import profile
+from c42sec.send_to import send_to
+from c42sec.write_to import write_to
+
+
+def main():
+    c42sec_arg_parser = ArgumentParser()
+    subcommand_parser = c42sec_arg_parser.add_subparsers()
+    _init_subcommands(subcommand_parser)
+    args = c42sec_arg_parser.parse_args()
+    _call(args, c42sec_arg_parser.print_help)
+
+
+def _init_subcommands(subcommand_parser):
+    profile.init(subcommand_parser)
+    send_to.init(subcommand_parser)
+    write_to.init(subcommand_parser)
+
+
+def _call(args, print_help):
+    """Call provided subcommand with args."""
+    try:
+        args.func(args)
+    except AttributeError as err:
+        if str(err) == "'Namespace' object has no attribute 'func'":
+            print_help()
+        else:
+            print(err)
+
+
+if __name__ == "__main__":
+    main()

c42seceventcli/common/cursor_store.py renamed to c42sec/cursor_store.py

@@ -0,0 +1,111 @@
+import os
+import c42sec.util as util
+from configparser import ConfigParser
+
+
+class ConfigurationKeys(object):
+    USER_SECTION = u"Code42"
+    AUTHORITY_KEY = u"c42_authority_url"
+    USERNAME_KEY = u"c42_username"
+    IGNORE_SSL_ERRORS_KEY = u"ignore-ssl-errors"
+    INTERNAL_SECTION = u"Internal"
+    HAS_SET_PROFILE_KEY = u"has_set_profile"
+
+
+def get_config_profile():
+    parser = ConfigParser()
+    if not profile_has_been_set():
+        util.print_error("Profile is not set.")
+        print("")
+        print("To set, use: ")
+        util.print_bold("\tc42sec profile set -s <authority-URL> -u <username>")
+        print("")
+        exit(1)
+
+    return _get_config_profile_from_parser(parser)
+
+
+def mark_as_set():
+    parser = ConfigParser()
+    config_file_path = _get_config_file_path()
+    parser.read(config_file_path)
+    settings = parser[ConfigurationKeys.INTERNAL_SECTION]
+    settings[ConfigurationKeys.HAS_SET_PROFILE_KEY] = "True"
+    _save(parser, ConfigurationKeys.HAS_SET_PROFILE_KEY)
+
+
+def profile_has_been_set():
+    parser = ConfigParser()
+    config_file_path = _get_config_file_path()
+    parser.read(config_file_path)
+    settings = parser[ConfigurationKeys.INTERNAL_SECTION]
+    return settings.getboolean(ConfigurationKeys.HAS_SET_PROFILE_KEY)
+
+
+def set_username(new_username):
+    parser = ConfigParser()
+    profile = _get_config_profile_from_parser(parser)
+    profile[ConfigurationKeys.USERNAME_KEY] = new_username
+    _save(parser, ConfigurationKeys.USERNAME_KEY)
+
+
+def set_authority_url(new_url):
+    parser = ConfigParser()
+    profile = _get_config_profile_from_parser(parser)
+    profile[ConfigurationKeys.AUTHORITY_KEY] = new_url
+    _save(parser, ConfigurationKeys.AUTHORITY_KEY)
+
+
+def set_ignore_ssl_errors(new_value):
+    parser = ConfigParser()
+    profile = _get_config_profile_from_parser(parser)
+    profile[ConfigurationKeys.IGNORE_SSL_ERRORS_KEY] = str(new_value)
+    _save(parser, ConfigurationKeys.IGNORE_SSL_ERRORS_KEY)
+
+
+def _get_config_file_path():
+    path = "{}config.cfg".format(util.get_user_project_path())
+    if not os.path.exists(path):
+        _create_new_config_file(path)
+
+    return path
+
+
+def _get_config_profile_from_parser(parser):
+    config_file_path = _get_config_file_path()
+    parser.read(config_file_path)
+    config = parser[ConfigurationKeys.USER_SECTION]
+    config.ignore_ssl_errors = config.getboolean(ConfigurationKeys.IGNORE_SSL_ERRORS_KEY)
+    return config
+
+
+def _create_new_config_file(path):
+    config_parser = ConfigParser()
+    config_parser = _create_user_section(config_parser)
+    config_parser = _create_internal_section(config_parser)
+    _save(config_parser, None, path)
+
+
+def _create_user_section(parser):
+    keys = ConfigurationKeys
+    parser.add_section(keys.USER_SECTION)
+    parser[keys.USER_SECTION] = {}
+    parser[keys.USER_SECTION][keys.AUTHORITY_KEY] = "null"
+    parser[keys.USER_SECTION][keys.USERNAME_KEY] = "null"
+    parser[keys.USER_SECTION][keys.IGNORE_SSL_ERRORS_KEY] = "False"
+    return parser
+
+
+def _create_internal_section(parser):
+    keys = ConfigurationKeys
+    parser.add_section(keys.INTERNAL_SECTION)
+    parser[keys.INTERNAL_SECTION] = {}
+    parser[keys.INTERNAL_SECTION][keys.HAS_SET_PROFILE_KEY] = "False"
+    return parser
+
+
+def _save(parser, key=None, path=None):
+    path = _get_config_file_path() if path is None else path
+    util.open_file(path, "w+", lambda f: parser.write(f))
+    if key is not None:
+        print("'{}' has been successfully updated".format(key))