Building an image processing pipeline using AWS Step Functions and Amazon Rekognition!
Journey: 📊 Community Builder 📊
Subject matter: Building on AWS
Task: Building an image processing pipeline using AWS Step Functions and Amazon Rekognition!
Using the 6 Pillars of the AWS Well-Architected Framework, Security, Performance Efficiency, Reliability, and Sustainability will be achieved in this build.
This week, I built a serverless workflow to automate image analysis.
In this scenario, we will be uploading files to one S3 bucket that will then trigger a Lambda Function. The Lambda will call Amazon Rekognition to analyse the images and then AWS Step Functions to deliver a notification depending on what Rekognition determined the image to be.
Resource credit: This architecture was created using guidance from Helio Tarnowski Here.
What did I use to build this environment?
- AWS Management Console
- IAM
What is built?
- Amazon S3 Bucket
- Amazon SQS Queue
- Amazon SNS Topic
- AWS Lambda
- AWS Step Functions
- Amazon Rekognition
In this task, I created an S3 Bucket which would be used to upload images for onward analysis.
I created two SQS queues and one Topic in SNS before successfully subscribing my email to the Topic.
AWS Lambda was then configured to call Amazon Rekognition whenever an image was uploaded into the S3 bucket.
Once Amazon Rekognition had determined what the image was, AWS Lambda called AWS Step Functions to handle the output on the following basis:
- Image is determined to be human, celebrity = Pass to SNS queue
- Image is determined to be human, non-celebrity = Pass to Faces SQS queue
- Image is determined to be non-human = Pass to Other SQS queue
I configured AWS Lambda to trigger against my Amazon S3 bucket and assigned an environment variable for my AWS Step Function State Machine ARN.
Lastly, I configured an Express Workflow in AWS Step Functions using JSON code to handle the various outcomes depending on what images were uploaded into S3.
The result… it worked! I uploaded three images to S3, Lambda triggered Rekognition to analyse them and then Step Functions handled the notification parameters. All images were delivered to the expected notification delivery stream!
Some of the highlights…
New S3 bucket:
Two SQS queues:
A new SNS Topic:
Subscription to SNS:
Subscription confirmed:
AWS Step Function creation:
AWS Step Function state machine setup:
AWS Lambda code:
AWS Lambda function configuration:
Uploading images to S3:
Test images for Rekognition to analyse:
Case 1
Rekognition confirms this is a celebrity:
SNS is triggered:
Notification comes through:
Case 2
Rekognition confirms this is an ‘object’:
State machine sends SQS message to ‘Other’ queue:
Polling for messages:
‘Other’ SQS queue collecting notification:
Case 3
Rekognition confirms this is not a celebrity, but is a human:
State machine sends SQS message to ‘Faces’ queue:
Polling for messages:
‘Faces’ SQS queue collecting notification:
My interpretation of the architecture:
I hope you have enjoyed the article!
