AWS Lambda is one of the popular service in amazon web services.

AWS supports many languages and runtimes for lambda. One of the popular runtime is python 3.8.

We will create a python based lambda to download weather data and validate the response. This lambda will run on cloudwatch scheduled event (like old school's cron job)

Project is available on GitHub: github.com/sagar-rout/weather-data-download..

Implementation

lambda downloads data for the next hour from openweather, validates whether all required fields are present in the response or not.

In this example, I am using json-logging lib for non-web applications to log the lambda in JSON format. Later, we can use this tool to stream the logs from cloudwatch to Elasticseach.

Environment Variables will look like this:

{
    "OPEN_WEATHER_API": "https://api.openweathermap.org/data/2.5/weather",
    "OPEN_WEATHER_API_KEY": "sbdgjfdbgfe7cedlmhlgf97d4c21fc8d0ba",
    "CITY_CODE": 2117915
}

Testing

• Python-lambda-local Installation is quite straight forward: use

  pip install python-lambda-local
  

  Use to invoke the lambda file with event and env files as input. But it has one drawback, it uses the local environment and you can not test the lambda packaging.

  python-lambda-local -f lambda_handler -e test/env.json src/app.py test/event.json
  

• AWS SAM

Installation : docs.aws.amazon.com/serverless-application-..

It uses template.yaml

It uses docker to wrap the lambda runtime(in our case it is python 3.8), code and environment variable. We can use sam to build the lambda image and invoke the lambda. Whenever we make any change in our code or template.yaml before invoking lambda we have to build the new image.

We can also generate event from sam for e.g. if we want to check how s3 put event looks like.

sam build
sam local invoke

Deployment

For deployment, we can use aws sam with template.yaml but I prefer to terraform script over SAM for the handling of aws resources. I am also a bit biased for terraform.

This project has deployment script which performs packaging and deployment.