Serverless architecture is all the rage these days and nothing screams serverless like AWS Lambda. When Lambda was announced a few years ago by AWS, it caused quite a bit of confusion but slowly it has gained popularity and has become analogous to serverless architecture.
AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the underlying compute resources for you.
AWS
Lambdas are super powerful because they can be easily triggered and scaled without having to worry about the underlying infrastructure.
In this post, I would like to show you how you can send a message to a PubSub+ broker running as a managed service on PubSub+ Cloud and use it to trigger a lambda through AWS’s API Gateway. And all of this without any code, except for the code you need for your Lambdas.
More specifically, we will publish a message to a topic that is being subscribed to by a queue. This is known as topic-to-queue mapping in the PubSub+ world. Once the message is received by the queue, it will be forwarded to API Gateway which will trigger the Lambda function.
We will be using REST Delivery Points (RDPs) to achieve this. An RDP is a provisioned object on a Message VPN that facilitates message delivery to REST consumers. For our use case, we will need an RDP to link our queue (which will receive the message) to the AWS API Gateway endpoint.
Here is what our architecture looks like:
Let’s begin!
Creating an AWS Lambda
Lambdas can be extremely simple or highly complicated. For this blog post, I am going to create an extremely simple one that simply prints the event that triggered it.
To create a Lambda function, go to your AWS Console and select Lambda from Services section. Then, click on Create Function button on the right side.
You will be provided with multiple options on how to create a Lambda. You can pick from existing Lambdas available in AWS Serverless Application Repository, build from templates or create a new one from scratch.
Pick the option Author from scratch, give your function a name, and pick a runtime environment for your function. I have picked my Runtime environment to be Python 3.8 and my function’s name is myFirstLambda. Click on Create Function.
Clicking on Create Function will create your Lambda function and will take you to the next page where you can design/code your function. In the Function code block, you can add your code. As mentioned earlier, we will only be printing our event that triggered the Lambda. Here is the code to do that:
import json
def lambda_handler(event, context):
# TODO implement
print('## EVENT')
print(event)
return {
'statusCode': 200,
'body': event
}Click Save button on top right to save your changes. We have now created a Lambda function!
Creating an API Gateway endpoint
Now that we have our Lambda, we need an endpoint to trigger it. We will use API Gateway to do that.
On the API Gateway page, click on the Create API button on top right. You will then be presented with different options on types of APIs you can create such as HTTP API, WebSocket API, and REST API (public or private). Pick REST API (public) and click on Build button.
On the next page, leave all of the settings as default and give your API a name (i.e. myFirstAPI) and click on Create API button.
Note: Currently, PubSub+ does not support SNI for REST consumers so we need to make sure that our Endpoint Type is set to Regional, instead of Edge Optimized.
Now, we can create different endpoints under our root endpoint and specify which HTTP methods they support. For example, let’s create one called purchase which supports POST method.
Click on Actions and then, Create Resource and fill in the details:
Click the Create Resource button when finished. Then, click on Create Method and select POST from dropdown menu. Click the little circle with checkmark to finish.
You can now configure what happens when a POST request is issued against your endpoint. We would like it to trigger our Lambda function.
Again, keep all the settings as default and select your Lambda function from dropdown menu:
Click on Save to save your changes.
Now that we have designed our API, we need to deploy it to be able to use it. We can also create different Stages to manage lifecycle of our endpoint.
Click on Actions and then, Deploy API which will pop-up a mini window where you can specify the stage you want to deploy your API to and provide a description for your deployment. Because we don’t have an existing stage, you will see [New Stage] option when you click on Deployment stage dropdown menu.
Select [New Stage] and enter information about your stage. I am calling mine: PROD.
Click Deploy when finished.
You will now find details about your newly deployed endpoint, specifically, the invoke URL. Mine is: https://1v3hgu8bia.execute-api.us-east-1.amazonaws.com/PROD
We have now successfully created an endpoint which will trigger our Lambda function when a POST request is issued against it. All that’s left to do now is to integrate our endpoint with PubSub+ Cloud!
PubSub+ Cloud
If you are unfamiliar with PubSub+ Cloud, it is Messaging-as-a-Service provided by Solace. PubSub+ Cloud allows you to easily spin up a PubSub+ event broker on any of the major cloud providers. It also has a free tier so you can easily spin up a service to follow along with this post without having to worry about any cost.
Follow instructions here on how to create a service on Solace Cloud.
Creating a Queue with topic subscription
Before we create our RDP, we need to create a queue with the appropriate topic subscription.
Go to your messaging service, click on Manage tab, and then click on Message VPN block under PubSub+ Manager Quick Settings.
That will popup a management UI for your service which you can use to create your queue. Click on Queues on the left navigation bar and then click on + Queue button. Give it a name of the form [rdpname].[consumername]. This syntax is not mandatory but it will help you identify which queue is bound to which consumer and for which RDP. Because we haven’t created either an RDP or a consumer, you can call it whatever you like. I am calling mine: myFirstRDP.myRDPConsumer.
Click Create button and then leave the settings as default, click Apply. Your queue has now been created:
Now we need to add a topic subscription to it. To do that, click on the queue and navigate to Subscriptions. Click on + Subscription and enter POST/PROD/purchase and then, click on Create. Note that our topic must be of this syntax: <HTTP_METHOD>/<STAGE>/<endpoint>.
Creating a REST Delivery Point (RDP)
Now that we have our queue ready, we can go ahead and create our RDP. To do that, click on Client Connections on left navigation bar and then REST. Then, click on + REST Delivery Point button. As decided earlier, we will call our RDP: myFirstRDP. Toggle the Enabled button to enable the RDP and click Apply. You will notice that the Operational State is currently Down and that’s because we haven’t added a REST consumer yet.
Click on your RDP and navigate to REST Consumers. Click on + REST Consumer and give it a name. We decided earlier (when creating our queue) that our REST consumer will be called: myRDPConsumer. Click Create.
On the next page, we will need to provide the host url for our AWS API Gateway endpoint. This URL is just the base URL and should not include /PROD/purchase. In my case, that URL is: 1v3hgu8bia.execute-api.us-east-1.amazonaws.com
The port number should be set to 443. Toggle the button next to TLS Enabled to enable TLS.
Click on Apply button to create your consumer.
Click on your newly created RDP consumer and navigate to TLS Options on top. Click on + Trusted Common Name and enter *.execute-api.us-east-1.amazonaws.com.
Go back to your RDP and enable it by clicking on the toggle button next to Enabled. You will notice that the Operational State has now changed to Up.
We now need to add our queue binding to our RDP. Click on Queue Bindings and then, + Queue Bindings. Select the queue we had created earlier, myFirstRDP.myRDPConsumer, from the dropdown menu. Click Create.
On the following page, enter a Post Request Target. In our case, that would be /PROD/purchase. Make sure you include the “/” in front.
Click Apply. Operation State of your queue binding should be Up.
Adding Amazon’s CA Cert
To make sure our TLS settings work, we need to upload Amazon’s CA Cert to our broker. The certificate can be downloaded from here.
To upload the certificate, go back to your service on PubSub+ Cloud and click on Certificate Authorities block.
Click on + Add New, give it a name, and paste the content in the popup box. Click on Submit.
That should be it! We can now test to see if our setup works.
Demo Time
A quick recap. Our goal is to send a message to PubSub+ broker and have a lambda triggered by it. To achieve this, we created an RDP which has a REST consumer that points to our AWS API Gateway endpoint and a queue binding which is subscribed to a topic. Our AWS API Gateway endpoint is configured to trigger our Lambda function when a POST request is issued.
So, let’s send a message to our PubSub+ broker and see what happens. We can easily do that by using the Try Me! app on PubSub+ Cloud. Click on Try Me! on left navigation bar and enter your connection information on the Publisher app.
You can get your credentials by going back to your service and navigating to Connect tab on top and then selecting REST.
Click on Connect once you have entered your connection details in the Publisher app.
We can now set our Topic to POST/PROD/purchase and our Message Content to a JSON message:
{
"body" : "A purchase order has been placed by Solace"
}
Click on Publish to send the message. Now, we need to verify whether our Lambda was invoked. To do so, we can go to our Lambda and click on Monitoring on top and then, click on View logs in CloudWatch.
That will take you to CloudWatch console where you will see a Log Stream. Click on it to see the contents:
As you can see, our Lambda was successfully invoked and it consumed the message we sent it from our Try Me! app. We were able to accomplish this with mostly configuration settings and without having to code anything.
Hope you enjoyed this post and learned something interesting!
One reply on “Integrating PubSub+ Cloud with Lambda via API Gateway”
Excellent use case. Thanks for the efforts put in & sharing it to comunity.