I’ve recently settled on a clever use of
npm pack to prep AWS Lambda Functions for deployment. I’d like to tell you how I’m using NPM and why it’s better than Webpack or Serverless Framework.
- Serverless + Webpack plugin + Babel
- SAM + Webpack
- SAM + NPM (lately)
Why I use SAM instead of Serverless
- I found myself guessing about the specific CloudFormation Serverless would generate
- I became concerned that I didn’t understand why it was generating what it was
- Many plugins jump outside of CloudFormation and use the AWS SDK. This can result in config that is not idempotent
- I found Serverless’ lifecycle hooks for plugins were not well documented
- I had to learn CloudFormation anyway so I could configure the resources I needed
- Over time I found most of my Serverless YAML was hand-written CloudFormation
- SAM abstracts only the most tedious bits of CloudFormation so I know what’s going on in there
In a nutshell, SAM offers me the right level of visibility and control.
Why I’m not using Webpack + Babel
This is simple: debugging. Yes, Webpack + Babel produce sourcemaps. Yes, Babel lets you use new language features. Yes, Webpack can produce lean packages. Here’s how things work for me in practice:
- Theres a whole class of bugs that cannot be understood in a reasonable amount of time, except by editing your code in the AWS console. If you’re redeploying between runs you’ll never get there
- The output of Webpack and Babel are not intended to be human readable. Good luck working in a tight feedback loop in the console
- Lambda supports Node 8.10 which has good enough coverage of the latest language features. The only “big” features 8.10 lacks are async generators and for-await-of. In my experience, a tiny fraction of Node developers understand how to use these features properly
Thus, I stick to the language features available in Node 8.10 and I don’t use Babel. For a list of Node 8.10’s working features see Node 9.11.2 in Node.green .
Why I use NPM instead of Webpack
NPM has a command called “pack” which builds a package of everything you want and none of what you don’t want. It’s much easier to use than Webpack.
npm pack grabs everything except:
- What’s in
- What’s in
To include dependencies from
node_modules add them (explicitly) to the
bundleDependencies key in
package.json. No more Googling “webpack node externals aws-sdk”. NPM will trace the dependencies of the packages you deem necessary and add them to your package.
There’s also a handy flag
-B . For example,
npm install -B axios will add axios to
bundleDependencies at install time.
npm pack output looks like:
NPM to ZIP
Here’s the wrinkle:
npm pack produces a TAR and Lambda expects a ZIP. In addition, NPM places everything under
package/. This is a simple fix in the NPM
postpack script in
postpack script is called after
pack. For more info on NPM pre/post scripts see this page.
"postpack": "tarball=$(npm list — depth 0 | sed ‘s/@/-/g; s/ .*/.tgz/g; 1q;’); tar -tf $tarball | sed ‘s/^package\///’ | zip -@r package; rm $tarball"
Now we’ll get a
package.zip each time we run
If you aren’t big on shell commands this one can look daunting. I promise it’s not that bad. Here’s an explanation of what it does.
First, we figure out the name of the TAR file, without running
npm pack again (explain-shell). We can’t run
npm pack in
postpack (even though it gives us the file name) because it’ll cause a recursive loop.
tarball=$(npm list — depth 0 | sed ‘s/@/-/g; s/ .*/.tgz/g; 1q;’);
Next, we take everything in the tar file and put it in
tar -tf $tarball | sed ‘s/^package\///’ | zip [email protected] package;
Finally, we delete the tar file (explain-shell).
You don’t have to leave the script as a long one-liner in
package.json, feel free to pull it out into a bash file and make it more readable.
Edit: Many thanks to Rebecca Turner who recommends avoiding the tarball altogether with the following NPM script:
"release": "npm pack --ignore-scripts --json --dry-run | npx json .files | npx json -a path | zip [email protected] package.zip"
aws s3 mb s3://sam-with-npm
sam package --template-file template.yaml --output-template-file packaged.yaml --s3-bucket sam-with-npm
sam deploy --template-file packaged.yaml --stack-name sam-with-npm --capabilities CAPABILITY_IAM
aws cloudformation describe-stacks --stack-name sam-with-npm --query 'Stacks.Outputs'
Finally, check it with your endpoint
I hope you found that this simplifies your tooling and workflow. If you have any questions about Serverless or DevOps I’m a freelance coach and I help teams reach maximum development output with minimal pain. You can always find me at http://thestack.io or [email protected]