How to solve the API-GW “30 seconds limitation” using ALB

You have a serveless solution using multiple AWS lambdas and API-GW for the APIs. all is working great but then you have this one (or more) lambda function that runs for more than 30 seconds — and you get timeout error. It is frustrating as lambda can run up to 15 minutes. Usually AWS provides you the option to decide how your solution will look like but here they decide for us that API must take less than 30 seconds.

In my case, I am using an API that runs SQL queries and in many cases it just takes more than 30 seconds to be completed. there are many more similar scenarios.

So we need an API service that can last more than 30 seconds, but still wants to stay in a serverless architecture.

The solution I’ll demonstrate here is to use AWS ALB (Application Load Balancer)

Those are the steps I’ll cover:

  1. Create a lambda and API-GW. show that API request that runs more than 30 sec is failing when using the API-GW.
  2. Create ALB that invokes the same lambda — verify it works for 40 sec.
  3. Create a second lambda, and show how to route between the Lambdas using the URL parameters.

Step 1: Create Lambda and API GW

  • The new lambda called lambda1. This is a very basic lambda that gets one parameter called waitSec – that will help us to control the lambda duration. This is the nodeJs code:
exports.handler = async (event) => {
    let start = new Date();                // start time
let waitSec = event.queryStringParameters.waitSec;//wait param
await wait(waitSec); // wait!
let end = new Date(); // end time
const response = {
statusCode: 200,
headers: {
"content-type": "application/json",
},
body: JSON.stringify(`Hello from Lambda * 1 *, wait:${event.queryStringParameters.waitSec} ***start: ${start} ***end: ${end}`),
};
return response;
};
// wait function (sleep)
async function wait(timeSec){
return new Promise(resolved => {
setTimeout(() => {
resolved()
}, timeSec * 1000)
})
}

Change the lambda timeout to 5 minutes (or any time > 30 sec that you need) — and deploy the lambda.

  • Create a new API-GW that invoke lambda1, I called it lambda1Api:

Create a method “ANY “ that will invoke lambda1

Deploy the API.

  • In order to test, we need to get the API-GW URL from:

stages->stage1 -> Invoke URL (in the title)

Don’t forget to add our waitSec parameter! this is what I got:

https://w4bnn2g6ce.execute-api.eu-central-1.amazonaws.com/stage1?waitSec=3

working with 3 seconds:

but NOT working with 40 seconds:

Step 2: create ALB

From the EC2 service console -> target groups, create a new target

and from the “Load Balancers” menu, create a new ELB -> “application load balancer”

Make sure port 80 is open in security group for our browsing.

In the routing step, we will direct, by default, all the requests to our lambda1TargetGroup.

When done, Get the ALB DNS name from the “description” tab. add the waitSec param to the URL, and test:

http://lambdaALB-641303138.eu-central-1.elb.amazonaws.com?waitSec=40

40 Seconds and WORKING!!

Step 3: Route between Lambdas using URL parameters

if you have multiple lambdas that need be invoked using API through the ALB, you can route using the URL parameters, for example:

http://lambdaalb-641303138.eu-central-1.elb.amazonaws.com/?waitSec=2&lambda=lambda2

  1. Create one more lambda called lambda2, using the instructions in step 1. The only change is that the text returned is “lambda 2” so we could identify which lambda we used.

2. in the ALB -> ceate one more target group for lambda2

3. Edit the routing rules, and add route for lambda2: we will route the API to the right lambda using the URL query parameter called lambda.

lambda=lambda1 or lambda=lambda2

Now, let’s test and make sure that the correct lambda is running by the lambda parameter:

http://lambdaalb-641303138.eu-central-1.elb.amazonaws.com/?waitSec=2&lambda=lambda1

return :”Hello from Lambda * 1 *,

http://lambdaalb-641303138.eu-central-1.elb.amazonaws.com/?waitSec=2&lambda=lambda2

return :”Hello from Lambda * 2*

Conclusion

ALB is a good serverless alternative for API-GW if your lambda takes more than 30 seconds.

enjoy!

read original article here