IBRAHIM RONAQUE

myHome myWork myBlogs

AWS Cloud - Whitepapers

Use Case
The client is keen to expand their footprint in the e-Commerce market and would like to increase their market share, gain customer satisfaction, and add innovation bringing more flexibility in their Product Ordering application. Due to existing Mainframe monolithic architecture, there is very limited scope for expansion, advancement, and integration with new disruptive technologies. The client is therefore, looking to modernize their existing application running in mainframe and move to more Cloud Based distributed architecture for both their Online and Batch applications. The client ask is to provide an Architectural and Functional view of the new AWS Cloud based application to be presented to their technical and functional users for further discussion and approval.


Purpose
This documentation provides an overview of the old Mainframe application assessment and its proposed AWS Cloud platform architectural for clients Product Ordering application as part of their Mainframe Modernization and Migration program through a Well Architected Cloud Framework using AWS implementation employing Operational Excellence, Security, Reliability, Performance Efficiency and Cost Optimization. Today, AWS offers a variety of services which is categorized broadly under each pillar, but not limited to:
• Operational Excellence: Ability to run and monitor and continuously improving process and procedures using AWS CloudFormation, AWS Config, AWS Cloudtrail, Amazon Cloudwatch, AWS CI/CD (CodeBuild, CodeCommit, CodeDeploy, CodePipeline)
• Security: Ability to protect data, systems, and assets to take advantage of cloud technologies and security using IAM, KMS, AWS Shield, AWS WAF, Support SSL/TLS(HTTPS), ACM, Secret Manager
• Reliability: Ability of a workload to perform its intended function correctly and consistently when it’s expected to, automatically recover from failure and dynamically allocate resources to meet demand using Automatic Backup, Data storage LifeCycle Management, Glacier Archive, Amazon VPC, Service Quotas, Multi AZ, Cross Region backup
• Performance Efficiency: Ability to use computing resources efficiently to meet system requirements, and to maintain that efficiency as demand changes and technologies evolve using Auto Scaling, Lambda, EBS, S3, RDS, ElasticCache, Snowball, Cloudfront Caching, API Gateway
• Cost Optimization: Ability to run systems to deliver business value at the lowest price point using AWS Budget, Cost Explorer, Spot Instances, Reserved Instances
Please click here for more details on the 5 Pillars of AWS Well Architected Framework


Existing COBOL Based Mainframe Implementation & Challenges
The existing Mainframe COBOL based Ordering application has various online and batch components. All the online screens are built with COBOL+CICS programming with screens layout setup using CICS BMS Maps. The primary DB is RDBMS DB2. The business logic processing is executed with combination of COBOL, CICS and DB2 module and sub modules which completes the functional flow from Ordering to Payment and Shipping with storing information in DB2 tables. The Batch processing is executed using COBOL+DB2 programs and uses DB2 tables for reporting and analytics on daily, weekly and monthly basis. There is a web based component involved where an order can be tracked using online screen which interacts with underlying COBOL components and allows to transfer data through SOA API modules. Challenges: • Interdependency between functions – Each business functions are all interrelated and dependent with presentation layer, business and Database layers coded together • Costly & Time consuming – Any change to one function impacts the other functionality which leads to changes or retesting of entire work stream leading to additional cost to the project • Batch Congestion – Batch execution sometime runs in parallel to online and may create congestion or deadlock situation and leads to intermittent job failure • Monolithic Architecture – All business functionality are coded and resides in one single platform Below diagram depicts the current mainframe technical flow architect for the Ordering and Payment application.




Proposed AWS Cloud Architecture and Benefits
The proposed new AWS Cloud Architecture will be developed using the Multi-Tier framework for the Ordering Application system. AWS provides suite of AWS services which can be employed to build this framework with each business function working independently within its own cluster but at the same time, interacts with other AWS services in a secured, cost effective and resilient manner. AWS also have many automations, integration capabilities to allow minimum manual intervention such as Automated backup, notifications, logging, metrics, alarms, which is discussed in the solution. The proposed design as explained in the below sections demonstrates the AWS cloud capabilities using the Multi-Tier framework with : a. Presentation Tier to allow how user directly interacts with webpages, mobile app, UI,etc b. Logic Tier to handle how code translates user action to application functionality c. Data Tier for storage media such as Databases, Objects stores, etc.

COBOL CodeBase Conversion using Innowake:
As part of an overall migration strategy, it is significantly important to allow conversion of existing COBOL Codebase to new Cloud Native Development languages for a complete Modernization enablement to Cloud Technologies in AWS. AWS supports various cloud based native language today. Deloitte’ Innowake being one of the leading tools in the market which can be applied for extraction, business rule mining and conversion activities to support quick, seamless and effective methods of Cloud Migration and Transformation. AWS services can then be deployed to integrate the business logic into Cost effective, highly optimized and operational efficient Cloud applications which is explained in detail design solution in the next steps.


AWS Architecture and Design:
AWS Architecture and Design: Below is the overall high level Architectural flow of the AWS Cloud Design.




The proposed AWS Cloud Architecture solution design adopts a Microservice API model running business core functions with a Multi Tier Approach format. MicroService API (as Docker containers) are cloud agnostic which makes them easily portable and highly maintainable on any platform. ECS Elastic Container Services (ECS) is one of the AWS Dockerization Microservice methodology in AWS implemented in this architecture to develop each of main core business functions such as Ordering, Shipping, Payment, etc and its sub functions into independent modules as a single Docker container, stored them in AWS Elastic Container Registry (ECR), invoked by ECS as a Task and could be managed and written into multiple programming languages. This will not only decentralized the application but also helps in maintaining the changes without impacting other components in the ecosystem. A set of ECS Clusters of EC2 Instances (Virtual Machines - VMs) executes these ECS Task in ECS Services upon invocation from other AWS services. Deploying a Cluster based Auto Scaling of ECS Cluster or Service based Auto Scaling of ECS Task (a business function) within AWS ensures the scaling of application is handled automatically to Scale In for burst in traffic during Peak times or scale out to reduce cost during low utilization. Another offering from AWS for Container services is AWS Fargate which is a serverless option from AWS, where AWS will manages the servers (Cluster of VM machines) you behind the scenes and customer is responsible to executing , managing, maintaining ECS Task to run their respective business functions. When a user request a service through a URL entered on online webpage for a particular function or workload say as Order a Product from the Presentation Tier, the data or invocation travers through various AWS services such as a AWS CloudFront Distribution, API Gateway, Application Load Balancer before it reaches the ECS microservices to execute a particular ECS Task as a business function which performs a CRUD(Create, Read, Update, Delete) activity into an Amazon RDS DB and eventually returns the response back to the user following a completion of an activity. All other business functions are invoked in a very similar fashion executing as an independent task. The other concurrent AWS Services running in parallel showcases AWS automation and integration capabilities. All of them work in tandem in the Integration Tier to enable the completion of a payload and workload or a transaction. Below explanation further describe in detail how each of the AWS Services employed in the solution interacts with each other in the application model: An Application Load Balancer(ALB) before the AWS ECS within the solution will ensure the incoming request payload is balanced and is evenly distributed to ECS clusters of EC2 machines across different Availability Zones(Physical Data Center) within an AWS region and its Virtual Private Cloud Network (VPCs) and traffic do not route to any unhealthy Instances. Also, using ALB Dynamic Port Mapping feature will allow the same ECS Task to run within the same ECS cluster to reduce significant costs in launching new instances. Security Groups are setup to allow any connection to backend application in containers ECS service only through ALB hence no other services can directly access ECS application or invoke any services. ALB is setup in a public subnet with an internet gateway to allow access from public web URL and communicates with ECS through Security Group setup. API Gateway is an AWS Integration gateway service which acts as a single point of entry and connects all application based APIs and other AWS Services running in backend such as EC2 VM machines, ECS containers, Lambda invocation, ALB, etc to the front end web applications through RESTful HTTP/HTTPS APIs. API Gateway in this solution will direct all payloads integration like Ordering, Shipping, Payment, etc to the ECS container services in the logic tier through ALB. In the given architecture, API Gateway integrates with AWS Cognito, a primary AWS service for Mobile and Web App authentication for Mobile/Web app clients ( using federated logins such as Google ids, Facebook ids, etc). It also connects with AWS Identify and Access Management Services (IAM). IAM is a primary AWS Identity Management for authorization and authentication of User, Groups, Roles accessing any AWS resources. An AWS CloudFront Distribution is added in front of API Gateway for global user access and avoid latency of application request through caching behavior utilizing AWS vast Global Network. It is also associated with S3 (Simple Storage Services) to store and distribute S3 Static Websites. S3 is an object storage services in AWS which allows you to store files in any format as Objects within a S3 bucket as a Folder. As per the architecture, many static websites/webpages such as Contact Us, About Company, Feedback pages, Home Screen, etc which are not interactive in nature can be stored and published through S3 Static Website capabilities. AWS Route 53 is a AWS Domain Name System to host public and private zones and allow all web based client URL request to communicate with AWS services. In the above solution, Route 53 transfer all requests enter through web based client to CloudFront distribution. Hence, when a user enter a web URL from Client machine, the URL is routed to CloudFront distribution. CloudFront is setup with an origin such as an S3 bucket to serve S3 Static Website and displays the content to the users. Amazon RDS DB is used as an alternative for DB2 DB which is an RDBMS solution provided by AWS using MySQL as a DB engine. Amazon RDS provides much more capabilities to a regular Relational DB in terms of automatic backup with point in time recovery, Standby DB for Disaster Recovery, Read Replica for increased performance Reporting and Analytics processing and snapshot of DBs for cross region availability. All data stored in RDS DB is encrypted at rest.

For an Enhanced Security, there are various security services employed for Data in Transit and Data at Rest Encryption services in the provided architecture. AWS Certificate Manager will generate industry standard free public SSL certificates and can be used and attached to CloudFront, API Gateway, Application Load Balance for data in transit security. Other Security options used in the architects are Web Application Firewall (WAF) to provide any cross scripting or SQL injection with AWS managed rules applied on CloudFront/Application Load Balancer, AWS Shield which is automatically available for all AWS services to protect from DDoS attacks (Bots attack) and AWS Key Management Services(KMS) data at rest encryption using Customer Master Keys (CMKs), could be managed by AWS or Customer and is available for most of the storage services with automatic Encryption.

For Batch processing, the design has adopted AWS Batch services for Daily, weekly and monthly job definitions , add to the Job queue and run at a particular schedule. AWS deploys and spins up Spot Instances automatically to manage the workload for job execution. Jobs outputs are stored in S3 bucket as Objects using AWS Lambda function. A S3 event is then triggered to invoke another AWS Lambda function to create a S3 PreSigned URL which is passed to executive users to download Total Orders/day as a report. The monthly report are FTPed to third party application for printing and eventually the files are archived in S3 long term Archival storage called S3 Glacier for future use automatically after x days.

AWS Automation capabilities for governing and monitoring activities shown in the architecture utilizes AWS Cloudtrail to track all API calls and store it in S3 bucket, It also stored in AWS CloudWatch Logs which can be use to create custom AWS CloudWatch Metrics and trigger AWS CloudWatch Alarms and send SMS, email notifications through AWS SNS to IT operations for alerting any illegal activities through unwarranted IPs. The Cloud trail logs stored in S3 can be used further for analytical processing using AWS Athena SQL queries to have more

How would it generally work
When the user enters the data and submits on the screen say to Add a new product, Route 53 routes it back to CloudFront which will have another origin as an API Gateway to pass data through secured HTTPS in transit encryption. API Gateway will invoke the respective Method – PUT, POST, GET, DELETE etc to call ALB and eventually call the respectively ECS Task to execute the business function and store, retrieve, delete data from Amazon RDS DB.

Other important Developer tools and Services available:
AWS CloudFormation can be used to create infrastructure template as a code and deploy them as Stacks. Stackset can be created to setup new infrastructure almost instantly and replicate them in different region and AWS account for Disaster recovery. Cloud Formation also ensure no infrastructure is inadvertently created with incorrect information.
AWS CI/CD (Continuous Integration and Continuous Delivery) support quick development, testing, integration, and delivery of changes in production through automation using Code Pipeline. Code pipeline can deploy code in production through Code Deploy, ElasticBeanStalk deployment services.
AWS Software Development Kit for JavaScript is a collection of software tools for the creation of applications and libraries that use Amazon Web Services (AWS) resources. There are separate software development kits (SDKs) for browser-based or server-side JavaScript application development.
AWS Cost and Budgeting: AWS allows to create Consolidated Billing for all accounts under a single Management account to show a combined Billing view. AWS Budget could be setup for creating alarms notification for any Budget threshold exceed. AWS Cost Allocation Tags that can be assigned to each of the resources and helps to align expenses on an Organization or Account level and further bifurcate the Cost and Usage report based on these tags.

Summary
Looking at the current set of requirements, this could be a very near solution to recreate the Ordering Application in AWS Cloud Environment. There are tons of other services provided by AWS and can be deployed depending on the need and demand of the business. Also, considering certain service limits pertaining to the services, other possibilities can be explored, and a well architected design can be achieved. As better said, there is no single way of doing things in AWS.
To get a more detail comprehensive view of all AWS services, please refer the link below: https://docs.aws.amazon.com/whitepapers/latest/aws-overview/introduction.html

Various AWS documentation reference links for further reading:
https://docs.aws.amazon.com/whitepapers/latest/serverless-multi-tier-architectures-api-gateway-lambda https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-api-gateway.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-load-balancing.html https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Welcome.html
https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_identity-management.html
https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html
https://docs.aws.amazon.com/sdk-for-javascript/index.html


Introduction
Many small business establishment or independent practitioners in today’s world, would want to explore and expand their customer base and connect to more clients promoting themselves on online platform without making any large investments in software, infrastructure etc. The basic primary need of such clients is to employ Cloud and its capability in a Software as a Services (SaaS) model with very simple, less expensive way building their own websites for new business with very minimum or no infrastructure cost. Hence, there is a greater demand and ask to provide for a solution architect which can quickly help client to build Simple static web services in AWS Cloud Services which is further presented in this document.


Purpose
This documentation provides an overview of a Simple Static Multi-Page web portals developed on AWS Cloud platform for small business and independent practitioners employing AWS services for deploying websites for customer in secured AWS cloud environment.

Today, AWS offers a variety of simple yet powerful, highly cost-effective services to the needs of the customer and provide them with quick access to Online cloud platform which can be further built into more interactive and dynamic workspaces.

Please click this link to know 5 Pillars of AWS Well Architected Framework to build cost effective, resilient, highly available applications in complex IT ecosystem.


High Level Functional View
The Static Multi Page web portal build requires upto 3 web pages displaying basic info about Client’s business, Client’s Specialty Work area and their Contact details. The web portal will be hosted and implemented on AWS. Each web page is a designed and developed using basic CSS, HTML code and deployed using AWS Cloud platform. User will be able to browse through multiple pages by clicking each tab on the menu which will display the required contents. All these HTML, CSS web pages will be stored within AWS cloud environment and hosted as Static website.
Note: The scope is limited only for Static web pages with very minimum browsing capabilities. Dynamic and interactive website is out of scope for this paper.



4.0 Proposed AWS Cloud Architecture for Multi-Page Web portal
Below is the overall high level Architectural flow of the AWS Cloud Design.




The proposed AWS Cloud Architecture solution design adopts AWS Simple Storage Services (S3) to deploy static pages in AWS. S3 is one of the storage options available in AWS which can alternatively be used for deploying Static Website. This is one of the cost-effective way of hosting web portals in Cloud environment with minimum AWS services and without spinning up any virtual server such as AWS EC2 or any other infrastructure. S3 storage are organized in terms of buckets and objects more like Folders and Files respectively. Each bucket and objects can be referenced by its own key which is made up of its path and content. Buckets names are uniquely identified globally but created in a particular AWS region. Objects are uploaded into buckets and could be any types files including images, html files, CSV files, .txt logs etc. and has a limit of upto 5GB.

Once a static webpage is created with .html extension, they are uploaded into bucket as an html object. S3 bucket properties are modified to enable the bucket to work as a Static Website. The actual html file (which is uploaded as an object) is provided in Static Website setup which tell AWS that when this bucket S3 end point is invoked, it should display the static html page on web browser. Additionally, an error.html file can be defined to display user defined error messages incase there is an issue with the webpage display.

AWS Route 53 is an AWS Domain Name System(DNS) to host public and private domains and allow all web based client URLs request to communicate with AWS services and resources in the backend. Route53 is also used as a Registrar service to register your own web domain. Once registered, this web domain is then mapped using an Alias-Record Set to the S3 Static Website End point to invoke the webpage. The S3 Static Webpage defined earlier in S3 bucket is now available and hosted on AWS cloud through Route53 DNS name. However, intrinsically S3 Static Website are not secured and we cannot directly attached a security certificate (SSL/https) to an S3 Static Website setup, hence we need to place additional service in front of S3 Static Website and route all request from Client/Mobile via this service to fetch the Static website securely. AWS Cloudfront distribution services acts as a front end to S3 Static Website where a security SSL certificate can be attached which allows client to securely request using https calls. AWS CloudFront distribution has many other advantages and one of the key usage is to use AWS global distribution network (216 Edge Location) to serve the website much faster. It works on a caching mechanism where the website is cached at their edge location geographically using the global network and avoids making calls to Origin(actual application) whenever a website is invoked by the user. This reduces to and fro traffic from origin end points and makes the website load much quicker. Moreover, once a cloudfront distribution is created, you can utilize the same distribution to add as many origin as you want which in this solutions adds multiple S3 Static Website as Origin and includes the behavior path to serve multiple webpages from different Static S3 buckets. In an advanced form, AWS CloudFront can also be modify to have different viewer request/response and origin request/response in the process using Lambda@Edge functionality. Once the CloudFront distribution is created and enabled, Route53 Alias-Record Set is modified to point to Cloudfront distribution instead of S3 Static Website.

In the above solution, when the user enters the registered domain name or the Web URLs on web browsers, AWS directs the request to Route53 which resolves the DNS name to a cloudfront distribution. Cloud Front will internally validate if this web url is already in cache, if yes, then it serve the website from cache. If the website is not from the cache, then it reroutes the request back to origin and gets the S3 Static Website and displays it to the user. At this point, it also places the latest in cache. Route 53 transfer all requests enter through web-based client to CloudFront distribution which is all done securely through https calls enabled through SSL certificates deployed in AWS Cloudfront distribution via AWS Certificate Manager.

AWS Certificate Manager (ACM) allows to generate industry standard free public SSL certificates and is utilized to make the website secure providing in transit security. It also auto renews the certificates before expiry. The certificate is uploaded on AWS Cloudfront distribution and thus allows secure https APIs calls for all data transfer.

Other important Developer tools and Services available:
>AWS CI/CD (Continuous Integration and Continuous Delivery) can be used to automate the S3 Static Website change process and allow for quick development, testing, integration, and delivery of changes in production using Code Pipeline, build and deploy mechanism.

Summary
AWS Cloud services can create more complex dynamic solutions for simple web portals and there is no one way of doing things in AWS. There are tons of other services provided by AWS and can be deployed depending on the need and demand of the business. Also, considering certain service limits pertaining to the services, other possibilities can be explored, and a well architected design can be achieved.
To get a more detail comprehensive view of all AWS services, please refer the link below: https://docs.aws.amazon.com/whitepapers/latest/aws-overview/introduction.html

AWS documentation reference links for further reading:
https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-serve-static-website