You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
*Automation wrappers for the Cloudera Ansible Collection*
20
+
toc::[]
6
21
7
-
Readme last updated: 2021-05-10
22
+
== Automation For the Cloudera Data Platform
8
23
9
-
Cloudera Deploy is a toolset for deploying the Cloudera Data Platform (CDP). Its scope includes both Public and Private Cloud products and Base clusters, and application setup, execution and other post-deployment functions.
24
+
Cloudera Deploy is a toolset for deploying the Cloudera Data Platform (CDP). Its scope includes
25
+
** Public Cloud** and **Private Cloud** products, **Private Cloud Base** clusters, and application setup, execution, and other post-deployment functions.
10
26
11
27
You can use Cloudera Deploy as your entrypoint for getting started with CDP. The toolset uses straightforward configuration definitions to instruct the automation functions, yet is extensible and highly configurable. The toolset can be a great foundation for custom entrypoints, CI/CD pipelines, and development environments.
12
28
13
-
toc::[]
14
-
15
29
== Quickstart
16
30
17
-
. Clone this repository into your preferred local project directory, e.g: +
. You should now see the orange _cldr_ prompt within the container environment. You may wish to set a default password and other entries in your local Cloudera Deploy user profile before proceeding: +
22
-
`vi ~/.config/cloudera-deploy/profiles/default`
23
-
. Ensure you have CDP and the appropriate cloud provider credentials present in your user profile:
24
-
.. You may need to issue new https://aws.amazon.com/blogs/security/how-to-find-update-access-keys-password-mfa-aws-management-console/[AWS API Keys] (or Azure, GCP) and set them up with `aws configure`, etc.
25
-
.. In addition, you may need to do the same with https://docs.cloudera.com/cdp/latest/cli/topics/mc-cli-generating-an-api-access-key.html[CDP API Keys] and set them up with `cdp configure`.
26
-
. Run the main playbook with the example definition: +
Using the quickstart above will create you a pre-prepared container within which you can run the Ansible playbooks. If you are unable or unwilling to run Docker containers locally, or you wish to run the playbooks on a remote server, you can use the `centos7-init.sh` script, which will setup a similar environment with the right repositories and dependencies downloaded. All other instructions to run and configure the playbook remain the same.
NOTE: The Quickstart image prepackages the AWS CLI.
52
+
53
+
If this is the first time you are installing the AWS CLI, configure the program by providing your credentials.
54
+
55
+
[source, bash]
56
+
----
57
+
aws configure
58
+
----
59
+
60
+
Visit the https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html[AWS CLI User Guide] for further details regarding credential management.
NOTE: The Quickstart image prepackages the CPI CLI.
69
+
70
+
If this is the first time you are installing the CDP CLI, you will need to configure the program by providing the right credentials.
71
+
72
+
[source, bash]
73
+
----
74
+
cdp configure
75
+
----
76
+
77
+
Visit the https://docs.cloudera.com/cdp/latest/cli/topics/mc-configuring-cdp-client-with-the-api-access-key.html[CDP CLI User Guide] for further details regarding credential management.
78
+
79
+
==== (Recommended) Confirm your SSH Keypair
80
+
81
+
Ensure that you have a generated SSH keypair for your local profile. Visit the https://www.ssh.com/academy/ssh/keygen[SSH Keygen How-To] for details.
82
+
83
+
NOTE: The Quickstart will generate a SSH keypair if none is provided.
84
+
85
+
==== (Recommended) Confirm your SSH Agent
86
+
87
+
Ensure that you have a properly configured SSH Agent. Visit the https://www.ssh.com/academy/ssh/keygen#adding-the-key-to-ssh-agent[SSH Agent How-To] for details.
88
+
89
+
TIP: Put AWS CLI and CDP CLI programs in your `$PATH` to make these two programs easily accessible.
90
+
91
+
=== Setup
92
+
93
+
==== Clone the repository
94
+
95
+
Clone this, i.e. the `cloudera-deploy`, repository.
WARNING: Be careful to not modify any of the files in the project as a user of the software. The vast majority of changes are managed through configurations provided to these project files.
* Linux users, use `systemd` (or another init system)
111
+
112
+
==== Execute the Quickstart script
113
+
114
+
Run the `quickstart.sh` entrypoint script. This script will prepare and execute the Ansible Runner container.
115
+
116
+
[source, bash]
117
+
----
118
+
cd cloudera-deploy
119
+
chmod +x quickstart.sh
120
+
./quickstart.sh
121
+
----
122
+
123
+
==== Confirm the Quickstart environment
124
+
125
+
Confirm that you have the orange _cldr_ prompt. This is your interactive Ansible Runner environment and provides builtin access to the relevant dependencies for CDP.
126
+
127
+
IMPORTANT: Do _NOT_ run the example definition until you have made the changes below.
128
+
129
+
==== Setup your user profile
130
+
131
+
Modify your local `cloudera-deploy` user profile. Your profile is present in your `$HOME` directory under `~/.config/cloudera-deploy/profiles/default`.
132
+
133
+
[source, bash]
134
+
----
135
+
vim ~/.config/cloudera-deploy/profiles/default
136
+
----
137
+
138
+
===== Properties to change
139
+
140
+
* Recommended
141
+
** *admin_password:* Note the password requirements (see the link:profile.yml[profile template] comments).
142
+
** *name_prefix:* Note the namespace requirements (see the link:profile.yml[profile template] comments).
143
+
** *infra_type:* The valid values are `aws`, `gcp`, `azure`.
144
+
** *infra_region:* Region is dependent on the value provided in `infra_type`.
145
+
* Optional
146
+
** *tags* (see the link:profile.yml[profile template] comments)
147
+
148
+
WARNING: Please ensure you provide a valid region for the `infra_type` property.
149
+
150
+
=== Execution
151
+
152
+
==== Run the main playbook
153
+
154
+
Run the main playbook with the defaults and your configuration at the orange _cldr_ prompt.
IMPORTANT: The total time to deploy varies from 90 to 150 minutes, depending on CDN, network connectivity, etc. Keep checking the logs; if there are no errors, the scripts are working in the background.
172
+
173
+
== Project Details
174
+
175
+
CAUTION: Don't change the project configuration without getting comfortable with the *quickstart* a few times.
176
+
177
+
NOTE: Below pages will be migrated to Github pages shortly.
35
178
36
179
Cloudera Deploy is powered by https://github.com/ansible/ansible[Ansible] and provides a standard configuration and execution model for CDP deployments and their applications. It can be run within a container or directly on a host.
37
180
@@ -47,9 +190,9 @@ Alternatively, and especially if you plan on running Cloudera Deploy in your own
47
190
48
191
Cloudera Deploy relies on a number of Ansible collections:
@@ -111,7 +255,7 @@ If you are also using Public Cloud infrastructure to host your CDP Private Cloud
111
255
112
256
For CDP Private Cloud clusters and other direct inventory scenarios, you will need to manage SSH host key validation appropriate to your specific environment.
113
257
114
-
**Be advised!** By default, the `quickstart.sh` script explicitly sets the `ANSIBLE_HOST_KEY_CHECKING` variable to `False` for ease-of-use with an introductory deployment. However, this setting is *not recommended* for any other deployment type. **For all other deployment types, you should directly manage your SSH host key checking.**
258
+
IMPORTANT: By default, the `quickstart.sh` script explicitly sets the `ANSIBLE_HOST_KEY_CHECKING` variable to `False` for ease-of-use with an introductory deployment. However, this setting is *not recommended* for any other deployment type. **For all other deployment types, you should directly manage your SSH host key checking.**
115
259
116
260
A common approach is to create your own "startup" script using the `quickstart.sh` as a template, and setting the appropriate https://docs.ansible.com/ansible/latest/reference_appendices/config.html[Ansible SSH configuration variables].
0 commit comments