Implementation of a Scalable Web Application using the services of AWS Elastic Beanstalk, DynamoDB, CloudFront, and Edge Location
Project Description
In this project, I am going to make an application that needs to support the high demand of a large number of users accessing it simultaneously. This application will be used in the cloud bootcamp conference, which is a conference that will have participants from all over the world. More than 10,000 participants are expected to attend, both in-person and online.
This event will be broadcast over the Internet and in person. At some point during the event, 10 vouchers will be drawn for 3 cloud certifications. The audience will register their emails to guarantee their participation in the raffle. The objective of this project is to create an application with the necessary components for the participants to successfully access the page. We will rely on AWS.
I am going to use Elastic Beanstalk to deploy a web application and DynamoDB to store email. Let’s rely on CloudFront to cache static and dynamic files to an Edge location close to the user.
Part 1: Deploying DynamoDB + Elastic Beanstalk (EC2, SG, ELB, TG, AutoScaling…)
- DynamoDB (Table)
I started up by creating a schema-less database that requires only a table name call users and a partition primary key; email which will be used to retrieve email from the database. This database will store all the registered emails from all the users who register for the boot camp.
Reviewing resources to be created: EC2, SG, ELB, TG, Autoscaling
Before jumping Elastic Beanstalk, I took a moment to review the resources I’m about to create, including; EC2 instances to host our application, Security Groups to great rules that will allow communication between the database and the application, Elastic Load Balancers to share traffic to available instances to optimize the application, Target Groups, and Auto Scaling configurations to scale the application up and down by creating an instance when existing instances’ CPU are exhausted and release them when the process becomes normal again.
Creating a new key pair (optional):
In this step I created a key pair that allows me to remotely connect to these, these keypairs will be used during the elastic beanstalk configuration of our environment to host the application.
Network & Security →Key Pairs →Create key pair
Name: mod4-ssh-key
Private key file format: .pem
If you prefer, you can create a folder to store the key file: “aws-mod4”
Elastic Beanstalk Configuration
We’ll configure Elastic Beanstalk to deploy our web application. Key settings include:
- Environment Tier: Web server environment
- Application Name: tcb-conference
- Platform: Python
- Version Label: tcb-conference-version-01
- Public S3 URL: [Your S3 URL Here]
- Configuration Presets: High availability
Step 1 — Configure environment
Environment tier
Application information
Application Name: tcb-conference
Platform
here we need to specify the platform in which of our applications.
Application code
Upload your code
Version label: tcb-conference-version-01
Patch the link to the s3 bucket storing our application.
(*) Public S3 URL: https://tcb-bootcamps.s3.amazonaws.com/bootcamp-aws/en/tcb-conf-app-EN.zip
You can hit a “TAB” key to access the URL field.
Presets
For high availability, making sure the application is provisions and available to attendees all over the globe, I selected high availability.
Next
Step 2 — Configure service access
In this step, I configure IAM roles, assumed by Elastic Beanstalk as a service role, and EC2 instance profiles to allow Elastic Beanstalk to create and manage your environment. Both the IAM role and instance profile must be attached to IAM-managed policies that contain the required permissions
Service access
I created a new service role called aws-elasticbeanstalk-service-role, and attached 2 special permissions. The option (*) Create and use a new service role and select the
aws-elasticbeanstalk-service-role role option.
- AWSElasticBeanstalkEnhancedHealth
- AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy
Next, I verify the permissions by clicking on view permission details to confirm the permissions.
[ View service role permissions]
Next, select EC2 key pair to enable us to securely log on to our EC2 instances.
Next, I move over to IAM services to create an ‘EC2 instance profile’ that allows EC2 instances to perform required operations. This profile was created using IAM services. See the steps below.
IAM → Roles → Create Role → Trusted entity type: AWS service
set the common use cases: EC2
Add permissions to the role
— AWSElasticBeanstalkWebTier
— AWSElasticBeanstalkWorkerTier
— AWSElasticBeanstalkMulticontainerDocker
Gave a name to our role as aws-elasticbeanstalk-ec2-role and Selected trusted entities
now we can create a role.
When the role is created, we can now move back to our Elastic beanstalk configuration and select the EC2 instance profile we just created. I ended up with the following settings below. And then click on next.
Step 3 — Set up networking, database, and tags
In this step, I configured a VPC (virtual private cloud) by activating our instance setting to a public IP address.
It should be noted that in this instance I selected only one zone which will be changed in the next step when we modify the instance size. Then, we will come back to this step to select all zones.
If you select all zones, you’ll get an error because shapes are not available.
Next
Step 4 — Configure instance traffic and scaling
In this stage, I configured the instances that will run our application. By giving it the following settings
Configure the computing Capacity of our environment.
Load balancer network settings
Back to our load balancing settings I configured the Visibility to Public and checked all the subnets in all the availability zones. At this stage, we can now go back one step and select all the zones.
Load Balancer type
The load balancing type is set to application balancer. This is an application layer load balancer that routes HTTP and HTTPS traffic based on protocol, port, and route to the environment process.
we will allow the network load balance type to be dedicated which is the default.
I allowed the listener and processes to their default values since they already have the protocol and port and listeners, we need for the application set to port 80 which is HTTP protocol.
Next
Step 5 — Configure updates, monitoring, and logging
In this step, all I had to do was add a new environment property for our application as seen in the image below. I named it AWS_REGION and gave it the value us-east-1.
Next
Step 6 — Review
It was time to review our configuration making sure we have every resource created and all we have to do now is submit it to Elastic beanstalk to launch our application. This should take a minutes to complete.
Part 2: Validating the Resources Created, Add an Email and AWS CloudFront
- Validate the Resources Created
- Testing the application
To test this application I copied the domain name of the app shown in Elastic Beanstalk and opened it in a web browser. Before adding any e-mail Let’s validate ‘DynamoDB’ and ‘Users’ table items. To verify items in our table let’s navigate to Dynamodb à Tables àExplore items
As we see the table user has no item created.
Let's try to add an e-mail say sbn@sbn.com
Ops the application threw us an Internal Server Error showing that the server was unable to complete our request.
I debug the application error by requesting the last 100 lines in the log file. The log shows that our application does not have the appropriate permissions to write data to DynamoDB.
To solve the problem, I hopped over to IAM and gave our application the role to allow it to do so.
Add the required permission into “aws-elasticbeanstalk-ec2-role” role:
This role allows elastic beanstalk to write to the database in dynamodb
IAM →Roles
Select the aws-elasticbeanstalk-ec2-role role we created earlier and attach the right dynamo permission to this role. Which the leaves us with the following result.
let’s Try again to add an e-mail: sbn@sbn.com
- CloudFront | CDN — Content Delivery Network
To ensure low latency and high transfer speed for our application to boot camp attendees, we will leverage AWS’ CloudFront service to create a content distribution network that allows us to securely deliver content with low latency and high transfer speeds to boot camp attendees globally.”
Allowed HTTP methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE. This method allows the application to POST to forward entered emails to the database.
Cache key and origin requests
(*) Cache policy and origin request policy (recommended)
Cache policy: CachingOptimized
Web Application Firewall (WAF)
For security emphases, I will enable WAF security protection to keep our application secure from the most common web threats and security vulnerabilities. Blocked requests will be blocked before they reach our web server.
(*) Enable security protections
Create Distribution | Last modified:
This Deployment took about 5 min to complete.
- Testing CloudFront
To test if CloudFront was responding as required in our application, I Copied the ‘Distribution Domain Name’ and tried. This time our connection translated from an unsecure HTTP to HTTPS connection because of our CloudFront settings.
At this time, I was able to add a new record; sbn@sbn.com and validate it at the DynamoDB table
Part 3: Stress/Overloading test
Checking EC2, LB, TG, AutoScaling, Elastic Beanstalk health/status
► Accessing and installing the ‘Stress’ tool in the EC2:
To test if our application is responding to load balancing correctly, I initiated a secure connection using SSH using our created key pair generated earlier to one of the EC2 instances and installed a stress tool to use it to overwhelm the CPU resources of the instance and see how the application responded to it.
#ssh -i mod4-ssh-key.pem ec2-user@3.83.16.47
Installing and running the “Stress” tool
#sudo amazon-linux-extras install epel -y
#sudo yum install stress -y
#stress -c 4
After running the stress command and checking Elastic Beanstalk status we see that the state of the application has changed from okay to warning.
To confirm that the CPU resource of the instance is exhausted I initiated another SSH connection to the instance to view it’s running processes. The running process can be viewed using the following commands; ps aux, ps aux — sort=-pcpu, top.
In my case, I used the top command.
Increased CPU consumption can occur due to various reasons, some of which may be legitimate while others could indicate security threats. Here are scenarios where increased CPU consumption can be justified and others where it might indicate a security issue:
Legitimate Cases
- Black Friday Sales
- Scheduled Resource-intensive Tasks
Security Concerns
- DDOS attack (Distributed Denial of Service)
- Cryptocurrency mining malware (Hacker Mining Bitcoins)
- Zero-day Exploits or Malware infections
- Unauthorized resource utilization;
- Poorly optimized code or Vulnerabilities
Let’s monitor the resources: EC2, ELB, Auto Scaling Group
As a result, a new instance is created showing that our application is working as expected.
