A Cloud Guru #CloudGuruChallenge: Improve application performance using Amazon ElastiCache - My Solution

My first attempt at #CloudGuruChallenge. 

I loved the fact that the challenge requires us to us EC2, Redis, RDS (Postgresql) one of the most common scenarios in a typical web application.

Requirement: Improve the performance of a python web app by implementing Redis cache. 

Environment: EC2 (web server), RDS (PostgreSQL), Cache (Elasticache - Redis), Python (flask)

Steps: 

• RDS - Creating PostgreSQL
• Login to AWS console and create stack in cloud formation.
• Upload template to create RDS.
• Open RDS in AWS console
• Note the database URL.
• Elasti cache - Redis
• Login to AWS console and create Elasticache cluster with Redis
• Provide the relavant details and create the Redis cache
• After the cache is created note the Primary Endpoint URL.
• EC2 provisioning
• Login to AWS console and create EC2 instance with defaults
• Note the public IP address
• Allow Redis cache to be only accessible from EC2
• Application installation on EC2 instance
• Login (SSH) to EC2 instance using the pem key and public IP address.
• Execute the commands to install:
• git (to download the source code)
• python3
• psycopg2, flask, configparser and redis modules (required for webapp)
• Run the following command to download the source code from github
• git clone https://github.com/ACloudGuru/elastic-cache-challenge.git
• Change directory to the source code folder and run the following command to create the stored procedure
• psql -h <DBURL> -U username -f install.sql <instanceid>
• Modify the database.ini file in the config folder with RDS database details
• Testing if everything is working file till now
• Run python3 app.py to run the app.
• Open another SSH console in cloud playground
• Type curl http://localhost:5000/ 
• Observe the Elaspsed time. Run it 3-4 times.
• Modifying the code to connect to Redis cache
• Add code in app.py to connect to redis and get/set values
• Before querying the database add code to check cache before querying database
• Run python3 app.py to run the app
• Open another SSH console in cloud playground
• Type curl http://localhost:5000/
• Observe the Elaspsed time. Run it 3-4 times.

Challenges/Learning in the above exercise

1. Using cloud formation to create RDS stack helps to reduce time to create RDS everytime.
2. Redis cache was new to me. Created a t2.micro instance to get started (My cloud playground was suspended because I chose t6 large! which was the default Sorry!)
3. Had challenge for EC2 to access RDS. Modified security group of EC2 to have default security group.
4. Redis cache really reduces the time required to fetch data and improves app performance significantly.
5. Redis returns data in byte data type. Had difficulty in converting to string (UTF8)

Pre-cache load times:

5.02477s

5.04940s

5.04783s

Post-cache load times:

0.00534s

0.00170s

0.00166s

Screenshots:

RDS

Redis cache

EC2 instance

Code for getting cached data

Cloud Formation Template