Skip to content

use minimatch lib to check for files that should be excluded #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@ custom:
- ./requirements.txt
globalIncludes:
- ./common_files
globalExcludes:
- '**/*.egg-info'
- '**/__pycache__'
cleanup: true

functions:
Expand Down Expand Up @@ -130,6 +133,7 @@ The plugin configurations are simple:
| requirementsFile | The name of the requirements file used for function-level requirements. All function-level requirements files must use the name specified here. | Yes. Defaults to `requirements.txt` |
| globalRequirements | A list of paths to files containing service-level pip requirements. | Yes |
| globalIncludes | A list of paths to folders containing service-level code files (i.e. code common to all functions). Only the folders contents will be packaged, not the folder itself. Paths to files are not currently supported. | Yes |
| globalExcludes | A list of paths to exclude from the package. Use glob syntax to specify files or directories to exclude, e.g. `'**/*.egg-info'` | Yes |
| useDocker | Boolean indicating whether to package pip dependencies using Docker. Set this to true if your project uses platform-specific compiled libraries like numpy. Requires a [Docker installation](https://www.docker.com/get-docker). | Yes. Defaults to `false` |
| dockerImage | The Docker image to use to compile functions if `useDocker` is set to `true`. Must be specified as `repository:tag`. If the image doesn't exist on the system, it will be downloaded. The initial download may take some time. | Yes. Defaults to `lambci/lambda:build-${provider.runtime}` |
| containerName | The desired name for the Docker container. | Yes. Defaults to `serverless-package-python-functions` |
Expand All @@ -138,6 +142,7 @@ The plugin configurations are simple:
At the function level, you:
- Specify `name` to give your function a name. The plugin uses the function's name as the name of the zip artifact
- Use `include` to specify what function-level files you want to include in your artifact. Simply specifying the path to the function's folder will include every file in the folder in the function's zip artifact
- Use `exclude` to specify what function-level files or directories you want to exclude in your artifact. Use glob syntax to specify target content.
- Use `artifact` to tell Serverless where to find the zip artifact. The plugin creates the zip artifact for the function at `buildDir`/`name`.zip, so using `${self:custom.pkgPyFuncs.buildDir}/[function-name-here].zip` is advised.

At the package level, you may need to:
Expand Down
24 changes: 23 additions & 1 deletion index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
const BbPromise = require('bluebird');
const _ = require('lodash');
const Fse = require('fs-extra');
const minimatch = require('minimatch');
const Path = require('path');
const ChildProcess = require('child_process');
const zipper = require('zip-local');
Expand All @@ -28,13 +29,20 @@ class PkgPyFuncs {
config.buildDir ? this.buildDir = config.buildDir : this.error("No buildDir configuration specified")
this.globalRequirements = config.globalRequirements || []
this.globalIncludes = config.globalIncludes || []
this.globalExcludes = config.globalExcludes || []
config.cleanup === undefined ? this.cleanup = true : this.cleanup = config.cleanup
this.useDocker = config.useDocker || false
this.dockerImage = config.dockerImage || `lambci/lambda:build-${this.serverless.service.provider.runtime}`
this.containerName = config.containerName || 'serverless-package-python-functions'
this.mountSSH = config.mountSSH || false
this.abortOnPackagingErrors = config.abortOnPackagingErrors || false
this.dockerServicePath = '/var/task'
this.defaultExcludes = [
"**/.serverless",
"**/node_modules",
"**/package.json",
"**/package-lock.json"
]
}

autoconfigArtifacts() {
Expand Down Expand Up @@ -70,6 +78,7 @@ class PkgPyFuncs {
return {
name: target.name,
includes: target.package.include,
excludes: target.package.exclude,
artifact: target.package.artifact
}
})
Expand Down Expand Up @@ -207,7 +216,20 @@ class PkgPyFuncs {
if (this.globalIncludes){
includes = _.concat(includes, this.globalIncludes)
}
_.forEach(includes, (item) => { Fse.copySync(item, buildPath) } )
let excludes = target.excludes || []
if (this.globalExcludes){
excludes = _.concat(excludes, this.globalExcludes)
}

let filter = (src, dest) => {
for(var i = 0; i < excludes.length; i++) {
if (minimatch(src, excludes[i])){
return false
}
}
return true
}
_.forEach(includes, (item) => { Fse.copySync(item, buildPath, filter) } )

// Install requirements
let requirements = [requirementsPath]
Expand Down
27 changes: 27 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
"bluebird": "^3.5.0",
"fs-extra": "^3.0.0",
"lodash": "^4.17.11",
"minimatch": "^3.0.4",
"upath": "^1.1.0",
"readline-sync": "^1.4.10",
"zip-local": "^0.3.4"
Expand Down