Skip to content

Dev docker wrapper #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

Dev docker wrapper #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
3affc5b
Initial commit
spamoom May 18, 2021
ab75c70
Pointing CLI to docker wrapper
spamoom May 18, 2021
d6579ab
Fixes bug introduced by pulling docker compose in
spamoom May 20, 2021
5891bd9
Improves shell script to allow for better updating
spamoom May 20, 2021
5a03335
Better self-update text
spamoom May 20, 2021
25a6ea5
Moving aws to readonly and removes ssh
spamoom May 20, 2021
5b3db23
Using system temp directory so we're still compatible with desktop users
spamoom May 20, 2021
73fe766
Removes bleed commit from other work
spamoom May 20, 2021
19062e1
Ensuring the wrapper uses latest
spamoom May 20, 2021
41b12f2
Further WIP bleeding!
spamoom May 21, 2021
60235f2
Adds more directories to dockerignore
spamoom May 21, 2021
2cfef32
More consistent naming
spamoom May 21, 2021
524b980
Update app/Commands/AwsSsmConnect.php
spamoom May 21, 2021
bd817a8
Adds support for SSH tunnels and switches wrapper to sh
spamoom May 21, 2021
d3dba73
Merge branch 'dev-docker-wrapper' of github.com:netsells/cli into dev…
spamoom May 21, 2021
55a67a0
Reverts to docker-compose
spamoom May 21, 2021
5803677
Installs docker-compose
spamoom May 21, 2021
f9a861b
Adds support for using AWS cli to auth docker
spamoom May 21, 2021
83504e7
Further shell script support
spamoom May 21, 2021
115beec
Remove weird spaces
May 21, 2021
469abf1
Optimise dockerfile to reduce the final image size.
May 22, 2021
7a68af3
Add installation instructions for the docker wrapper.
May 22, 2021
a0cf44a
Only run docker login, if the ecr credential helper is not present.
May 22, 2021
0eb014c
Fix credential helper error
May 22, 2021
e6f3769
Add local docker build instruction.
May 22, 2021
a7877f7
Ran composer update, as the lock file was out of sync.
May 22, 2021
479cacc
Reverts ECR auth to standard docker login
spamoom May 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
vendor/
.git/
.vscode/
.idea/
hooks
Dockerfile
4 changes: 4 additions & 0 deletions .editorconfig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,7 @@ trim_trailing_whitespace = false
[*.yml]
indent_style = space
indent_size = 2

[Dockerfile]
indent_style = space
indent_size = 2
65 changes: 65 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
##
# Composer
##
FROM composer:latest as composer

##
# PHP Builder
##
FROM php:7.4-cli as build

ARG DOCKER_TAG

# Deps
RUN apt-get update && apt-get install -y \
unzip \
git

# Grab composer
COPY --from=composer /usr/bin/composer /usr/bin/composer

COPY . /app
WORKDIR /app

# Install CLI deps
RUN composer install --ansi --no-interaction --no-progress --prefer-dist

# Build the phar
RUN php netsells app:build --build-version=$DOCKER_TAG

##
# PHP Runtime
##
FROM php:7.4-cli as runtime
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can do few tweaks here to make the resulting image considerably smaller, and also to make the self-update pull down only the things that changed. Are you ok for me to look into it later on in the evening?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely mate - I was doing this between meetings so it's very WIP - just wanted it in front of you as I knew you'd spot a load of improvements

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, will try few ideas and report back :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done few tweaks to reduce the image by > 300MB. It's still big, but at least it has all tools inside it which is convenient.

ECR helper works well after a fix which is great, however worth testing again on TeamCity, as it won't have access to the AWS config file I think?

I do have problems using the SSM connect feature, but not sure if it's just all our EC2 instances not having it enabled or something? Would be nice if you can confirm it's working.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work - yeah the NS EC2 images aren't setup for SSM

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already checked on the YPS Teamcity agent, it's able to access the meta domain so is able to get auth 🌮

http://169.254.169.254/latest/meta-data/


# Deps
RUN apt-get update && \
apt-get install -y --no-install-recommends \
unzip git openssh-client && \
apt-get purge -y autoconf pkg-config gcc && \
apt-get autoremove -y && \
apt-get autoclean && \
apt-get clean && \
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" \
-o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose && \
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" \
-o "awscliv2.zip" && unzip -q awscliv2.zip && ./aws/install && rm awscliv2.zip && rm -rf ./aws && \
curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb" \
-o "session-manager-plugin.deb" && dpkg -i session-manager-plugin.deb && rm session-manager-plugin.deb && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Grab docker client from existing docker image
COPY --from=docker:20-dind /usr/local/bin/docker /usr/local/bin/docker

# Copy the wrapper from source
COPY ./docker-support/netsells /usr/local/bin/netsells-wrapper

# Grab built phar from the builder
COPY --from=build /app/builds/netsells /usr/local/bin/netsells

ENV AWS_SDK_LOAD_CONFIG 1

RUN mkdir /app
WORKDIR /app

ENTRYPOINT ["netsells"]
16 changes: 14 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,16 @@ The Netsells Command Line Interface (CLI).

Run the following commands to download and install. `/usr/local/bin` should be in your `$PATH` in order to call `netsells` anywhere.

```bash
```shell
# Install PHAR, requires PHP
curl -L -o netsells.phar https://netsells-cli.now.sh/download/cli
mv netsells.phar /usr/local/bin/netsells
chmod +x /usr/local/bin/netsells
netsells

# Or install the Docker wrapper
docker run --rm --init --entrypoint=/bin/cat netsells/cli:latest /usr/local/bin/netsells-wrapper > /usr/local/bin/netsells
netsells
```

### Usage
Expand All @@ -39,6 +44,13 @@ netsells
docker:build Builds docker-compose ready for prod
```

### Development
To build the docker image:
```shell
$ DOCKER_TAG=latest DOCKERFILE_PATH=Dockerfile IMAGE_NAME=netsells/cli hooks/build
```


## Netsells File Reference

The CLI will look for configuration in the arguments/options supplied via the command line, falling back to the Netsells file. This should be placed at the root of your project and called `.netsells.yml`.
Expand Down Expand Up @@ -67,7 +79,7 @@ docker:
## Command Reference

* [aws:ec2:list](#awsec2list) - List the instances available
* [aws:ssm:connect](#awsssmconnect) - Connect to an server via SSH (Use --tunnel to establish an SSH tunnel)
* [aws:ssm:connect](#awsssmconnect) - Connect to a server via SSH (Use --tunnel to establish an SSH tunnel)


### aws:ec2:list
Expand Down
22 changes: 15 additions & 7 deletions app/Commands/AwsSsmConnect.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ class AwsSsmConnect extends Command
*/
protected $description = 'Connect to an server via SSH (Use --tunnel to establish an SSH tunnel)';

protected $tempKeyName = 'netsells-cli-ssm-ssh-tmp';
protected $tempIdentityFile;

/** @var Helpers $helpers */
protected $helpers;
Expand All @@ -48,6 +48,15 @@ public function configure()
], $this->helpers->aws()->commonConsoleOptions()));
}

private function tempIdentityFile(): string
{
if (!$this->tempIdentityFile) {
$this->tempIdentityFile = tempnam(sys_get_temp_dir(), 'NetsellsCliSsm');
}

return $this->tempIdentityFile;
}

/**
* Execute the console command.
*
Expand Down Expand Up @@ -95,7 +104,7 @@ public function handle()
$sessionCommandString = implode(' ', $sessionCommand->getArguments());

$options = [
'-o', 'IdentityFile ~/.ssh/netsells-cli-ssm-ssh-tmp',
'-o', 'IdentityFile ' . $this->tempIdentityFile(),
'-o', 'IdentitiesOnly yes',
'-o', 'GSSAPIAuthentication no',
'-o', 'PasswordAuthentication no',
Expand All @@ -111,7 +120,7 @@ public function handle()

$options[] = '-N';
$options[] = '-L';
$options[] = sprintf('%s:%s:%s', $tunnelLocalPort, $tunnelRemoteServer, $tunnelRemotePort);
$options[] = sprintf('0.0.0.0:%s:%s:%s', $tunnelLocalPort, $tunnelRemoteServer, $tunnelRemotePort);

$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'tunnel-remote-server', $tunnelRemoteServer);
$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'tunnel-remote-port', $tunnelRemotePort);
Expand Down Expand Up @@ -215,8 +224,7 @@ private function generateTempSshKey()
return 1;
}

$sshDir = $_SERVER['HOME'] . '/.ssh/';
$keyName = $sshDir . $this->tempKeyName;
$keyName = $this->tempIdentityFile();
$pubKeyName = "{$keyName}.pub";

if (file_exists($keyName)) {
Expand All @@ -232,9 +240,9 @@ private function generateTempSshKey()
->withCommand([
'ssh-keygen',
'-t', 'ed25519',
'-N', "",
'-N', '',
'-f', $keyName,
'-C', "netsells-cli-ssm-ssh-session"
'-C', $this->tempIdentityFile()
])
->run();
} catch (ProcessFailed $e) {
Expand Down
2 changes: 1 addition & 1 deletion app/Commands/DeployEcsServiceUpdate.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ public function configure()
*/
public function handle()
{
$requiredBinaries = ['aws', 'docker-compose'];
$requiredBinaries = ['aws', 'docker', 'docker-compose'];

if ($this->helpers->checks()->checkAndReportMissingBinaries($this, $requiredBinaries)) {
return 1;
Expand Down
2 changes: 1 addition & 1 deletion app/Commands/DockerPushCommand.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ public function configure()
*/
public function handle()
{
$requiredBinaries = ['docker', 'docker-compose', 'aws'];
$requiredBinaries = ['docker', 'aws', 'docker-compose'];

if ($this->helpers->checks()->checkAndReportMissingBinaries($this, $requiredBinaries)) {
return 1;
Expand Down
8 changes: 4 additions & 4 deletions composer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@
}
],
"require": {
"php": "^7.3",
"php": "^7.4 | ^8.0",
"laminas/laminas-text": "^2.7",
"laravel-zero/framework": "^7.0",
"nunomaduro/laravel-console-menu": "^3.0",
"laravel-zero/framework": "^8.0",
"nunomaduro/laravel-console-menu": "^3.1",
"padraic/phar-updater": "^1.0.6",
"symfony/yaml": "^5.0"
},
"require-dev": {
"fzaninotto/faker": "^1.9",
"mockery/mockery": "^1.3.1",
"phpunit/phpunit": "^8.5"
"phpunit/phpunit": "^9.3"
},
"autoload": {
"psr-4": {
Expand Down
Loading