AWS Lamdba spinning up a nodsjs microservice with persistence

 

AWS Lamdba serverless is pretty cool. Deploy services and code easily without having to provision and manage servers. 

My previous employer has it's own data center and infrastructure so we don't get to work hands on configuring and using aws services usually. So I'm creating my own AWS Lamdba microservice to manage appointments and store in DynamoDB nosql key value database. I'm using serverless framework cli integration for aws.

There are steps I followed to setup a nodejs service:

1. Create a new aws user for this service.
2. Follow instructions on aws site to install cli locally. I followed command line install.
3. run `aws configure` cli using new user id and secret (created in step 1)
4. install npm serverless locally
5. you can bootstrap a serverless config file using template create, but in next step I do manually
6. create serverless.yml (or js/ts) for your service and handler functions
• the handler in the serverless config file must match an exported function name and the filename and path
7. to deploy run `sls deploy`
• deploys to the cloud (is a little slow...but what magic eh!)
• must use every time change .yml config
8. ...you can just deploy a function only if you only changed a function which is much faster to deploy e.g. `sls deploy function --function getAppointments`
9. I configured resources with my DynamoDB for appointments and a permission config
• sls will create the table if doesn't exist
• note: the id in the Resource for the table is your aws account id
10. I created simple handler functions to GET and POST appointment information to/from the database
• I used uuid to create a uniqueId for a new table entry
11. After successful `sls deploy` it will provide the url to POST to which you an hit with a curl like so:
• curl -X POST https://iq2u4m9d2f.execute-api.us-east-1.amazonaws.com/dev/appointments --data '{ "text": "Appointment for 10am December 1st" }'
• and in my appointments table in DynamoDB I see the entry posted, sweet!
• 
  
  
  
12. I added unit tests using jest, this article was helpful
13. I refactored the code to split business logic into services and created a dynamodb adapter/connector
14. I refactored the code to convert to typescript, this was a hassle because it involved changing imports/exports and adding configs for jest. Next time just start as typescript
• I used the serverless-plugin-typescript to compile typescript code as part of the sls deploy process, works pretty well though i created my own custom tsconfig file 
15. I am impressed with the ease of use and speed of development of serverless and aws
16. To dos:
• add schema validation and sql injection checking. Using AWS API Gateway seems to be the way to go for this.
• post about gateway setup and schema validation
17. my code repo: https://github.com/denisos/aws-appointment-service

Logging and Tracing

• Logs to AWS Cloudwatch automatically
• If you issue console.log() in your code (or any emit to stderr/stdout), then it will appear in these logs
• Go to Cloudwatch and drill into group and then your function to see log info
• When I added list appointments support, I tested with curl and received an error. I then checked Cloudwatch and for the aws landba saw a permission error in the logs "errorType":"AccessDeniedException"
• this was due to missing a configuration for dynamodb scan which I fixed and redeployed and then it worked
• Cloudwatch is very cool and easy to use
• Traces using AWS XRay
• You need to configure and turn it on for your service functions

Best Practices

• Best practices for monoliths 
• microservices not one big service
• don't assume express or similar, recommend to use api gateway
• Best practices for code detail
• separate business logic from container specifics
• use environment variables and be careful of recursion
• monitoring, alerting and performance

Other useful info

• aws lamdba doc function params
• service: a container for function, could be a microservice for a resource, could be the whole project
• event: anything which can trigger your serverless function e.g. http api call
• when using http event then response must be object with statusCode and body properties (and body must be JSON.stringify)
• context
• handler is name of the file and then exported function e.g. handler: myfile.foo in this case myfile.js and exported function foo
• very detailed real world example of typescript with auth
• serverless examples
• interesting example of writing images to aws s3 bucket
• DynamoDB naming and types

 

AWS Serverless Introduction course

AWS Lamdba - bring your own code and run in response to events

Lamdba Architecture Layers 

 1. Handlers   - lamdba specifc function configuration, no business logic

 2. Controller - business logic, event processing

 3. Services   - external integrations, abstractions

Permission model 

  - IAM Resource Policy controls what events can call your Lambdba

  - IAM Execution Rules control access to resouces your Lamba calls

Build, Test and Deploy using standard practices

Libraries

• Aws serverless Express framework works for Express in a Lamdba context
• (there's also one for python "Zappa")
• Tip for startup performance: reduce size of libraries used e.g. java spring

Managing the Dev workflow

Deploying a Lambda fn involves a complex workflow including build, define permissions, zip, upload to S3, create role/function/rest api/table etc. So using an application framework is recommended (such as AWS SAM or serverless)

AWS SAM provides a cli to zip and upload to AWS S3 (2 commands: package & deploy) 

Organizing Lamdba functions in repos

Think about how your application subdivides into Services (instead of focussing on functions). Then a service usually has multiple functions and a service has 1 template and lives in 1 repo.

Setting up Sandbox and CI/CD

Typical create 1 account per developer. Alternative is a shared account for multiple developers.

You'll want to have a Test environment separate from a Production environment (so can be QA'd in Test). Cheap to create new environments.

Also need to create a CI/CD pipeline. 

Testing

• Local    - automated unit tests, unit test your controllers, mock called services
• mocking: "DynamoDB Local" and "LocalStack" are 2 options OR Custom mocks
• Sandbox  - for integration testing
• Automated integration tests - for CI/CD pipeline prior to prod deployment

Debugging

AWS SAM gives you a way to run your svc locally and run in a docker container and connect to it. Then can hit it with curl or a FE running.