In this article, I teach you how to connect an SNS topic to a Lambda Function in the AWS Console. I also teach you how to configure cross-account access if your SNS account and Lambda Function are in separate AWS accounts.
If you prefer a video walkthrough, check out my YouTube video.
Step 1 – Creating our SNS Topic
Firstly, we need an SNS topic to bind our Lambda function to. Lets start by heading over to the SNS section of the console and clicking on Topics followed by Create Topic as seen below.
The following screen is a prompt of settings we can configure on our SNS topic. Lets quickly run through some of the settings.
By the way, if you’re interested in learning more about SNS check out my detailed article on it here.
FIFO vs Standard
FIFO stands for First In, First Out. It is the idea that messages are delivered via SNS in the same order that they are received. FIFO queues have lower TPS (Transactions Per Second) abilities than Standard Topics. However, they do ensure exactly once message delivery, a property lacking from the Standard counterpart. FIFO queues unfortunately are only supported with SQS as the destination. Since we are using Lambda as the destionation/endpoint, we don’t want Standard.
Go ahead and select Standard as the topic type.
You can also take this time to give your topic a name.
Encryption in SNS is while at rest. SNS saves copies of your message content internally in order to function. This setting instructs SNS to store your message in an encrypted format as opposed to plaintext.
Access Policy (needed for Cross Account subscription)
The access policy is an IAM policy that applies to the SNS topic as a resource. In this section, you are able to specify who is able to perform what action on your SNS topic. This includes things like listing your topic, subscribing to it, unsubscribing , et cetera.
The Access Policy section has two sections:
- Who can publish messages to the topic (top box below)
- Who can subscribe to the topic (bottom box below)
Who can publish to the topic allows you to specify whether only the owner, everyone, or only specific AWS accounts can.
Typically, its a best practice to only let your Topic Owner push messages to your topic.
Who can subscribe to the topic specifies who is capable of subscribing. Keep in mind that the subscriber must also have proper IAM permissions to perform the action. In other words, you need both the correct IAM policy on the topic and IAM policy on the user accessing the topic for permission to work correctly.
If you find yourself needing to grant permissions to a Lambda function in a different AWS account, the process involves:
- The SNS Topic owner granting the account id of the Lambda Function the ability to Subscribe, and ListSubscriptionsByTopic
- The Lambda function owner must provide the SNS service the ability to invoke it.
AWS has a great tutorial on the setup specifics in the AWS Tutorial Article.
Delivery Retry Policy
The SNS delivery policy refers to how SNS will behave when it fails to deliver a message to a subscriber. The default delivery policy involves multiple retries. If you’re setting up a subscription for a Lambda function, one thing you need to be careful about is Lambda throttling. If your Lambda function is being throttled, SNS retries to your function will repeatedly fail. You can learn more about Lambda including throttling here.
Delivery Status Logging
Delivery status logging allows you to log delivery attempts from SNS to each of your subscriptions. This may sound like a good idea, but be careful with applications that have many subscriptions and a chatty topic. The amount of logs generated for such topics can be enormous and cost you a pretty penny.
Another option involves sampling which allows you to collect only a subset of delivery events. You can set this between 0 and 100 percent.
At this point, you should be good to go ahead and click Create Topic to complete the wizard.
Step 2 – Creating a Lambda Function
To create a Lambda function, head over to the Lambda section of the console and click on Create Function.
You should be brought to the Lambda Wizard page as seen below.
Here you can select the Lambda function name, the language of choice, and permissions about the topic. You can keep all the defaults for now and simply select the language of your choice.
Now click Create Function in the bottom right of the wizard as seen above.
One thing we’ll want to do before moving on is adding some code to our Lambda function to print the incoming event. This will be useful later on when debugging.
You can see me below adding a simple print statement in Python. Make sure to click on “deploy” button as highlighted below to ensure your new function version is deployed.
One thing to do before you move on is to copy your Lambda Functions ARN (Amazon Resource Name) to your clipboard. You will need this in the next step. See the screenshot below on where to find it in the Lambda Console.
Alright, now we’re good to go ahead and create our SNS subscription from our Topic to our Lambda Function.
3 – Creating a Susbscription between SNS Topic and Lambda
To get started, head back over to the SNS section of the console and select the topic you created in Step 1.
Navigate to the Subscriptions tab and click Create Subscription as seen below.
In the wizard that follows, first set the Protocol to AWS Lambda. Next, under the section that says Endpoint, paste in the Lambda function ARN you copied from the previous step. Your wizard should now look something like this:
You do have the options of setting a Subscription Filter Policy and a Redrive Policy.
A Subscription Filter Policy allows you to restrict messages that get delivered to recipients based on an attribute of the message. In other words, you as a subscriber can narrow down the types of messages you receive from the topic.
The Redrive Policy specifies a Dead Letter Queue (DLQ) which is used for messages that failed to deliver. You can use this feature to maintain copies of messages that don’t make it to their destination.
We’re not going to use a Subscription Filter Policy or Redrive Policy in this exercise. Feel free to skip it.
When you’re happy with your settings, click Create Subscription in the bottom right to finalize the process.
You should now see a new Subscription entry for your Lambda function as seen below.
At this point, we’re ready to test out our setup.
Step 4 – Testing the SNS Topic and Lambda Subscription
You can confirm your setup is working correctly by publishing a message to your SNS topic, and confirming your Lambda Function was invoked.
To start, click on Publish Message under the SNS section.
Under the Message Body section, fill it out with a JSON message as seen below.
Click on Publish Message to broadcast it your Lambda.
Now navigate over to your Lambda Function and take a look at the Logs and Execution Metrics. Keep in mind metrics have a 5 minute delay between the event and when they show up on graphs. You should now see an invocation of your Lambda Function!
Checking the logs, you can also see the event Payload.
You’re now done!. You now have a successfully configured SNS topic with a Lambda Function!