Migration re-application

Document proxied requests, Passenger standalone mode

Author: msaladna

docs/admin/Apache.md

@@ -226,6 +226,7 @@ ProxyPassReverse http://127.0.0.1:8080/
 </Location>
 ```
 
+
 ## Technical details
 
 Apache is a custom build available through [apisnetworks/httpd-apache](https://github.com/apisnetworks/httpd-apache). Nonportable atomics are enabled as well as mod_systemd backported from Apache 2.5 development to facilitate lightweight service reports. Latest APR and APR Utility releases are bundled to maximize efficiency. Compilation targets x86-64 machines using default compile flags.
@@ -329,5 +330,5 @@ To bypass this message, add at least 1 hostname to the map file. This can be acc
 
 For supporting documentation, see also 
 - [PHP-FPM](PHP-FPM.md) - PHP
-- [Passenger](Passenger.md) - Node, Ruby, Python, and Go
+- [Passenger](webapps/Passenger.md) - Node, Ruby, Python, and Go
 

docs/admin/Migrations - cPanel.md

@@ -149,7 +149,38 @@ Following migration, ApisCP will attempt to request SSL for each hostname as wel
 
 `--unsafe-sources` allows importing unchecked, potentially hazardous, backup data including SquirrelMail preference files and Roundcube MySQL directives. The consistency and validity of this data is not checked. **Do not enable this option unless you are confident the backup has not been tampered with**.
 
-### Quota disagreements
+### Reapplying components
+**New in 3.2.32**
+
+`--list-components` enumerates all available components for a given backup. These components are determined by files present within the backup. `--do=COMPONENT` may be specified multiple times to reapply *only* those migration components.
+
+```bash
+ ImportDomain --list-components --format=cpanel cpmove.tar.gz
+# - apachetls
+# - cp
+# - cron
+# - dnszones
+# - homedir
+# - ips
+# - meta
+# - mm
+# - mysqlsql
+# - psql
+# - shadow
+# - shell
+# - userdata
+# - va
+# - version
+
+# Reapply mailing list + database importation steps (mm, mysqlsql)
+ImportDomain --do=mm --do=mysqlsql --no-create --no-bootstrap --no-scan --format=cpanel cpmove.tar.gz
+```
+
+In the above example, `--no-create --no-bootstrap --no-scan` are present. `--no-create` is to prevent account creation which is attempted automatically to ensure importing into a pristine environment. Likewise `--no-bootstrap` and `--no-scan` prevent end-of-import hooks from running, SSL acquisition and Web App updates.
+
+### Troubleshooting
+
+#### Quota disagreements
 
 Quotas are accounted and enforced by the kernel. When migrating from certain hosting platforms that employ quasi-quota accounting by software, such as cPanel, the reported quota for a user may be significantly more than what was previously reported. `--late-quota` will apply storage amnesty, which is a 2x storage boost for 12 hours. **Late quota is only triggered** after account creation. Thus when combined with `--no-create`, `--late-quota` has no effect. Call `site:storage-amnesty` against the account using [cpcmd](CLI.md#cpcmd).
 
@@ -162,11 +193,11 @@ cpcmd scope:set cp.config quota storage_duration 172800
 
 These changes will be reflected on future imports.
 
-### Decompression oddities
+#### Decompression oddities
 
 Migration will attempt to use PHP's PharData handler to decompress files. It's based on USTAR, which has [limitations](https://www.gnu.org/software/tar/manual/html_node/Formats.html#Formats) that may result in a cPanel backup generated in POSIX.1-2001 standards to fail. Use `--no-builtin` to disable the builtin handler from attempting to read the backup.
 
-### Empty MySQL credentials
+#### Empty MySQL credentials
 
 Prior to MySQL 5.7.5 released in [2014](https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-5.html), MySQL accepted passwords created in an insecure hash format that was used prior to 4.1. From speculation, accounts that were created prior to this change were abandoned from a [secure upgrade pathway by cPanel](https://forums.cpanel.net/threads/mysql-upgrade-to-v-5-6-old-style-passwords-cpanel-explanation.649945/).
 

docs/admin/Migrations - server.md

@@ -86,7 +86,14 @@ apnscp_php bin/scripts/transfersite.php -c='dns,provider=linode' -c='dns,key=abc
 
 On the source server, mydomain.com may continue to use DigitalOcean as its [DNS provider](https://bitbucket.org/apisnetworks/apnscp/src/master/lib/Module/Provider/Dns/Digitalocean.php?at=master&fileviewer=file-view-default) while the on the target server mydomain.com will use Linode's [DNS provider](https://bitbucket.org/apisnetworks/apnscp/src/master/lib/Module/Provider/Dns/Linode.php?at=master&fileviewer=file-view-default). Once mydomain.com completes its initial stage (stage 0), be sure to update the nameservers for mydomain.com.
 
+## Notification templates
+
+A notification is sent at the end **stage 0** (warmup migration) and **stage 1** (final migration). Migrations are read from `resources/templates/migrations/` and may be overrode following [view/template](Customizing.md#ApisCP) override rules.
+
+Custom templates may be specified using `--template=`. A single argument or CLI
+
 ## Skipping suspension
+
 An account after migration completes is automatically suspended on the source side. In normal operation, this poses no significant complications as DNS TTL is reduced to 2 minutes or less during stage one migration.
 
 `--no-suspend` disables suspension following a successful migration. 

docs/admin/webapps/Passenger.md

@@ -69,6 +69,83 @@ Global idle shutdown may be modified by overriding `PassengerPoolIdleTime` in `/
 PassengerPoolIdleTime 0
 ```
 
+## Direct proxy
+
+An application may wire in additional services that make automatic startup/shutdown by Passenger cumbersome, such as [Discourse](Discourse.md). A Passenger-based application can be manually started, then connected to using [mod_rewrite](https://httpd.apache.org/docs/2.4/mod/mod_rewrite.html).
+
+If an application is running on port 8082, then the following rules in the *document root* of the subdomain or domain will suffice:
+
+```
+DirectoryIndex disabled
+RequestHeader set X-Forwarded-Proto expr=%{REQUEST_SCHEME}
+
+RewriteEngine On
+# This may be removed to send ALL requests to the app
+RewriteCond %{REQUEST_FILENAME} -f
+RewriteRule ^ - [L]
+
+RewriteCond %{HTTP:Connection} =upgrade [NC]
+RewriteCond %{HTTP:Upgrade} =websocket [NC]
+RewriteRule ^(.*)$ ws://localhost:8082/$1 [L,QSA,P]
+RewriteRule ^(.*)$ http://localhost:8082/$1 [L,QSA,P]
+```
+
+First, no [directory index](https://httpd.apache.org/docs/2.4/mod/mod_dir.html#directoryindex) will be assumed (index.html, index.php) for the location.
+
+The request scheme (http, https) is passed to the proxy as X-Forwarded-Proto. This standard, recognized in [RFC 7239](https://datatracker.ietf.org/doc/html/rfc7239), informs the application if the request was made securely.
+
+If the request matches a file in the document root, such as media or JavaScript, then that file is served directly. 
+
+If the client solicits a request to upgrade to WebSockets, then the ws protocol is used.
+
+Otherwise the request is sent locally, unencrypted, to the application listening on port 8082.
+
+### Standalone mode
+
+Passenger includes a [standalone mode](https://www.phusionpassenger.com/library/config/standalone/intro.html) that facilitates launching an application directly, such as in the case of [Discourse](Discourse.md). Standalone mode is controlled using `Passengerfile.json` typically located in the *application root* for the app.
+
+Use of this requires the `passenger` gem in [Ruby](Ruby.md).
+
+```bash
+gem install passenger
+```
+
+The following annotated configuration illustrates how it comes together. The *document root* is /var/www/forums/public while the *application root* is /var/www/forums.
+
+See "[Configuration reference](https://www.phusionpassenger.com/library/config/standalone/reference/)" for a comprehensive listing of directives.
+
+```json
+{
+    /* Use rbenv shim loader, allows .ruby-version per-directory */
+    "ruby": "/usr/local/share/ruby/rbenv/shims/ruby",
+    /* can be node, wsgi, rack, or meteor */
+    "app_type": "rack",
+    "startup_file": "/var/www/forums/config.ru",
+    "environment": "production",
+    // Use Rack, bundled with Discourse as the HTTP server
+    "engine": "builtin",
+    // Statically set 6 workers at all times.
+    "min_instances": 6,
+    "max_pool_size": 6,
+    "daemonize": true,
+    "spawn_method": "smart",
+    // Listen on 127.0.0.1:40011
+    "address": "127.0.0.1",
+    "port": 40011,
+    // Necessary for reliable WebSocket operation in Ruby
+    "force_max_concurrent_requests_per_process": 0,
+    // Additional env variables to pass to Ruby at startup
+    "envvars": {
+        "RUBY_GC_HEAP_GROWTH_MAX_SLOTS": "40000",
+        "RUBY_GC_HEAP_INIT_SLOTS": "400000",
+        "RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR": "1.5",
+        "LD_PRELOAD": "/usr/lib64/libjemalloc.so.1"
+    }
+}
+```
+
+Once configured it may be started using `passenger start` and stopped using `passenger stop`.
+
 ### See also
 
 * [Configuration reference for Passenger + Apache](https://www.phusionpassenger.com/library/config/apache/reference) (phusionpassenger.com)