Add the capability to lookup for config files in /Users/faltet and /etc (on unix)

Author: FrancescAlted

README.md

@@ -94,17 +94,18 @@ Tests will use a copy of Caterva2's `root-example` directory.  After they finish
 For this quick start, let's:
 
 - create a virtual environment and install Caterva2 with the `[server,clients]` extras (see above).
-- copy the configuration file `caterva2.sample.toml` to `caterva2.toml`.
+- copy the configuration file `caterva2.sample.toml` to `caterva2.toml`, `~/.caterva2.toml`, or `/etc/caterva2.toml`.
+- copy the server configuration file `cat2-server.sample.toml` to `cat2-server.toml`, `~/.cat2-server.toml`, or `/etc/cat2-server.toml`.
 
-Servers (and clients, to a limited extent) may get their configuration from a `caterva2.toml` file at the current directory (or an alternative file given with the `--conf` option).  See also [configuration.md](configuration.md) in Caterva2 tutorials.
+Clients will search for configuration in the following order: current directory (`./caterva2.toml`), home directory (`~/.caterva2.toml`), and system-wide (`/etc/caterva2.toml` on Unix). You can also specify an alternative file with the `--conf` option.  See also [configuration.md](configuration.md) in Caterva2 tutorials for more details.  Server will look for `cat2-server.toml` instead, either in the current directory, home directory, or system-wide; you can also specify an alternative file with the `--conf` option.
 
 Then run the server:
 
 ```sh
 CATERVA2_SECRET=c2sikrit cat2-server &  # server
 ```
 
-The `CATERVA2_SECRET` environment variable is obligatory and is explained below in the following section.
+The `CATERVA2_SECRET` environment variable is mandatory and is explained below in the following section.
 
 Now, let's see the directories that have been created by the server:
 
@@ -122,7 +123,7 @@ _caterva2
 6 directories, 2 files
 ```
 
-We see that we have a state directory with several subdirectories.  The `personal` directory is where we will store our personal datasets.  The `public` directory is where we will store datasets that are shared with the public.  The `shared` directory is where we will store datasets that are shared with other users.  The `media` directory is where the server will store temporary media files (images, videos, etc.) that are used by the web GUI.
+We see that we have a state directory with several subdirectories.  The `personal` directory is where we will store our personal datasets.  The `public` directory is where we will store datasets that are shared with the public.  The `shared` directory is where we will store datasets that are shared with other users.  The `media` directory is where the server will store temporary media files (images, videos, etc.) that are used by the web GUI for internal purposes.  The `db.json` and `db.sqlite` files are where the server will store its metadata.
 
 For populating the server with datasets, let's copy the `root-example` directory from the Caterva2 source code:
 

caterva2.sample.toml

@@ -1,5 +1,10 @@
 # Example configuration for caterva2 clients in general
-[localdemo]
-url = "http://localhost:8000"
+[default]
+url = "https://cat2.cloud/demo"
 username = ""
 password = ""
+
+[localuser]
+url = "http://localhost:8000"
+username = "user@example.com"
+password = "foobar11"

caterva2/utils.py

@@ -10,6 +10,7 @@
 import argparse
 import logging
 import pathlib
+import sys
 import tomllib as toml
 
 
@@ -112,6 +113,48 @@ def _add_conf_argument(default, parser):
     )
 
 
+def _find_config_file(filename):
+    """Find configuration file using a standard search path.
+
+    Search order:
+    1. Current directory (./filename)
+    2. User home directory (~/.filename or ~/filename without extension prefix)
+    3. System-wide config (/etc/filename) - Unix/Linux only
+
+    Args:
+        filename: The default config filename (e.g., 'caterva2.toml')
+
+    Returns:
+        pathlib.Path of the first found config file, or Path(filename) if none found
+    """
+    # First, check if --conf was explicitly provided
+    parser = argparse.ArgumentParser(add_help=False)
+    _add_conf_argument(filename, parser)
+    opts = parser.parse_known_args()[0]
+
+    # If --conf was explicitly provided (not default), use it as-is
+    if opts.conf != pathlib.Path(filename):
+        return opts.conf
+
+    # Search path: current dir -> home dir -> /etc (Unix only)
+    search_paths = [
+        pathlib.Path.cwd() / filename,  # Current directory
+        pathlib.Path.home() / f".{filename}",  # Home with dot prefix
+    ]
+
+    # Add /etc path on Unix-like systems (not Windows)
+    if sys.platform != "win32":
+        search_paths.append(pathlib.Path("/etc") / filename)
+
+    # Return first existing file, or the default if none found
+    for path in search_paths:
+        if path.exists():
+            return path
+
+    # If no config file found, return the default (current dir) path
+    return pathlib.Path(filename)
+
+
 def _get_conf(filename, prefix=None):
     """Get settings from the configuration file, if existing.
 
@@ -122,12 +165,10 @@ def _get_conf(filename, prefix=None):
     the key starts with a dot, like ``.path.to.item``, `prefix` is prepended
     to it.
     """
-    parser = argparse.ArgumentParser(add_help=False)
-    _add_conf_argument(filename, parser)
-    opts = parser.parse_known_args()[0]
+    config_path = _find_config_file(filename)
 
     try:
-        with open(opts.conf, "rb") as conf_file:
+        with open(config_path, "rb") as conf_file:
             conf = toml.load(conf_file)
             return Conf(conf, prefix=prefix)
     except FileNotFoundError:

doc/tutorials/configuration.md

@@ -1,40 +1,34 @@
 (caterva2.toml)=
 # The `caterva2.toml` configuration file
 
-We've seen that the `cat2-client` program accepts some command-line options to tune its operation (check the `--help` option).  This is even more important for services as we shall see in following sections.  Thus, Caterva2 programs support getting some settings from a TOML configuration file, by default `caterva2.toml` in the current directory (though you may override it with the `--conf` option).
+We've seen that the `cat2-client` program accepts some command-line options to tune its operation (check the `--help` option).  This is even more important for services as we shall see in following sections.  Thus, Caterva2 programs support getting some settings from a TOML configuration file.
+
+## Configuration File Search Path
+
+Caterva2 follows standard Unix/Linux conventions for locating configuration files. When looking for `caterva2.toml` (or `cat2-server.toml`), the search order is:
+
+1. **Command-line option**: `--conf <path>` (if explicitly specified)
+2. **Current directory**: `./caterva2.toml`
+3. **Home directory**: `~/.caterva2.toml`
+4. **System-wide** (Unix/Linux only): `/etc/caterva2.toml`
+
+The first existing file found in this order will be used. If no configuration file is found, the program will use default settings.
+
+This approach is similar to how tools like `git`, `vim`, and `ssh` locate their configuration files, making it familiar to Unix/Linux users.
 
 The configuration file may hold settings for different programs, with a separate section for each program.  Thus, a program may check the file for its own settings, but also for those of other programs which may be of use to itself.  This allows compact configurations in a single file.  For instance, below is a sample configuration file for the server program and some client app:
 
 ```toml
-# Example configuration for a standalone server
-#
-# It's possible to run only the server. Then the configuration has only a
-# section for the server. And maybe another one for the client.
-
-# The server section must define:
-#
-# - statedir: the directory where the server's data will be stored (default: _caterva2/sub)
-# - listen: where the server listens to (a unix socket or a host/port) (default: localhost:8002)
-# - urlbase: the base url users will use to reach the server (default: http://localhost:8002)
-# - quota: if defined, it will limit the disk usage (default: 0, no limit)
-# - maxusers: if defined, it will limit the number of users (default: 0, no limit)
-# - login: if true, users will need to authenticate (default: true)
-# - register: if true, users will be able to register (default: false)
-#
-[server]
-statedir = "_caterva2/state"
-#listen = "_caterva2/sub/uvicorn.socket"
-listen = "localhost:8000"
-urlbase = "http://localhost:8000"
-quota = "10G"
-maxusers = 5
-register = true  # allow users to register
-
-# The client section defines the credentials for the client to authenticate
-# against the server.
-[client]
+# Example configuration for caterva2 clients in general
+[default]
+url = "https://cat2.cloud/demo"
 username = ""
 password = ""
+
+[localuser]
+url = "http://localhost:8000"
+username = "user@example.com"
+password = "foobar11"
 ```
 
 See [caterva2.sample.toml](https://github.com/ironArray/Caterva2/blob/main/caterva2.sample.toml) in Caterva2's source for all possible settings and their purpose.

doc/utilities/cat2-client.md

@@ -13,11 +13,11 @@ Running `cat2-client --help` should provide a list of supported commands that ma
 cat2-client [GENERIC_OPTION...] COMMAND [COMMAND_OPTION...] COMMAND_ARGUMENTS...
 ```
 
-Another relevant generic option besides `--help` is `--server`, which overrides the base of server URLs used by default.  It should be a HTTP(S) URL, for example `http://sub.edu.example.org:3126`.  Finally, the generic options `--username` and `--password` may be used in case your server requires user authentication.
+Another relevant generic option is `--server`, which selects the server to connect to by name (as defined in a section of the configuration file; see below).  There is also `--url`, which overrides the base of server URLs used by default; it should be a HTTP(S) URL, for example `http://sub.edu.example.org:3126`.  Finally, the generic options `--username` and `--password` may be used in case your server requires user authentication.
 
 `--help` is also available as a command option which shows the options and arguments accepted by that command (e.g. `cat2-client roots --help`).  Another command option is `--json`, which forces the output of commands that accept it to be in JSON format, as that may be more amenable for parsing by other programs.
 
-`cat2-client` may use a TOML configuration file (`caterva2.toml` in the current directory unless overridden with the generic `--conf` option).  It may get the server address from there (`urlbase` or `listen` settings in `[server]` section), as well as user authentication options (in the `[client]` section).  Command-line options override settings read from the configuration file.
+`cat2-client` may use a TOML configuration file (`caterva2.toml` in the current directory unless overridden with the generic `--conf` option).  Command-line options override settings read from the configuration file.
 
 For a short tutorial on `cat2-client`, see [](Using-the-command-line-client).