1. Introduction
Serverless is a cloud computing framework that allows users to build and run applications without having to worry about setting up or managing the underlying servers. Serverless does not mean there are no servers involved. It simply means that the user do not have to interact with or control the servers required for the application.
Apart from Secure Agent based Cloud Data Integration runtime environment model, Informatica offers a Serverless runtime environment model where users are free from managing the Secure Agent.
In this article, let us discuss more about Informatica Cloud Advanced Serverless deployment model and how it is different from CDI and CDI-Elastic.
2. What is Informatica Cloud Advanced Serverless?
Informatica Cloud Advanced Serverless is a fully managed serverless deployment solution that doesn’t require downloading, installing, configuring, or maintaining a Secure Agent or Secure Agent group. The users can create their Cloud Data Integration or Cloud Data Integration Elastic jobs in Advanced Serverless runtime environment with no clusters or software to manage.
Serverless is a deployment option and NOT an engine (CDI/CDI-E)
Advanced Serverless enables customers to leverage serverless computing to process data integration pipelines. This eliminates the need to manage hardware or software allowing developers to focus on business logic and deploy new data pipelines quickly.
3. CDI vs CDI-Elastic vs Advanced Serverless
In Cloud Data Integration, the secure agent utilizes the compute resources of the server on which it is installed to run the data integration jobs. In CDI-Elastic, the secure agent spins up additional clusters based on the advanced configuration provided by the user and leverages spark engine to run the data integration jobs on these clusters.
The CDI-Elastic eliminates the challenges with Auto Scaling of compute resources as per the demand and ensures the data processing happens on the customer’s infrastructure
Both CDI and CDI-Elastic requires management of secure agent and servers. Advanced Serverless option eliminates the need to manage a secure agent and a dedicated infrastructure.
Advanced Serverless eliminates infrastructure maintenance and secure agent administration overhead while the data processing happens on Informatica’s secure Infrastructure.
Image Source: Informatica
4. Advanced Serverless Execution Model
Informatica Cloud Advanced Serverless is currently supported only on AWS cloud computing platform. Availability of serverless option on Azure is in pipeline.
Below is the Secure Agent runtime environment Execution Model:
Consider that you are hosting the secure agent on an EC2 instance within your AWS environment. In this the execution model all your applications are located inside your Organization’s VPC. The Secure agent can be granted access to other applications such as Redshift, S3 etc., through use of IAM roles and policies.
Image Source: Informatica
Below is the Advanced Serverless runtime environment Execution Model:
In Advanced Serverless execution model, all the applications and services will remain same as the secure agent execution model except that the installing and managing of secure agent is not needed.
Image Source: Informatica
In Advanced Serverless runtime environment,
- Informatica spins up an INFA DMZ next to customer’s VPC in same Availability zone.
- “DMZ” stands for Demilitarized Zone. The DMZ acts as an intermediary zone that helps manage the interaction between internal and external networks while maintaining security.
- The secure agent and the clusters are spun inside the INFA DMZ by the Informatica.
- Spinning up the DMZ in the same availability zone as the user’s VPC reduces latency and ensures efficient data transfer.
- The customer’s VPC and INFA DMZ are paired using tenant-controlled trusted Elastic Network Interface (ENI)
- ENIs allows customers to have control over the network connections between their VPC and the INFA DMZ. This empowers customers to implement their own network security policies.
5. Pre-Requisites for Advanced Serverless
For Advanced Serverless, it is expected that your cloud environment is on the AWS cloud platform and your VPC must have default tenancy. A serverless runtime environment can’t connect to a VPC with dedicated instance tenancy i.e. the services within that VPC are running on hardware that is dedicated to that specific AWS account only.
Below are the pre-requisites for Advanced Serverless
1. Create a VPC in your AWS Account
A Virtual Private Cloud (VPC) is a private, isolated section of a cloud computing platform where you can run your own services and applications with control over security and networking while still connecting to the broader internet. It’s like having your own secure space for your Organization on the internet.
2. Setup IAM Role and Policies
Create an IAM role to define a policy that establishes a two-way trust relationship between customer AWS account and Informatica AWS account and allows Informatica to setup cross-account ENI between INFA DMZ and Customer VPC.
3. Create Security Groups
A Security Group controls the traffic flow from the serverless runtime environment to customer VPC. Create a security group in the VPC. The security group is associated with all ENIs that the serverless runtime environment creates.
By configuring the security group rules, the customer can define which network traffic and services within their VPC are allowed to communicate with the serverless runtime environment through the ENI and which are denied.
4. Create a Private Subnet
A Private Subnet hosts the ENI that the serverless runtime environment uses to connect to your VPC to access all other services which are also configured as private. A Private Subnet is a network segment within a Virtual Private Cloud (VPC) or a larger network that is designed to be isolated from direct external access.
5. Create a Public Subnet and NAT Gateway (Optional)
A Public subnet facilitate internet connectivity for resources within a VPC using Network Address Translation (NAT) gateways. They ensure a clear separation between public services and private resources, enhancing security and control in the VPC.
A NAT gateway act as intermediary, allowing resources in private subnets to access the internet while keeping them secure.
6. Create S3 Bucket and Folders (Optional)
If you have a requirement to run scripts in your pipelines, you need to place the scripts with all supporting Jar files, external libraries, drivers, etc. on a S3 bucket and pass this Supplementary File Location while configuring serverless runtime environment.
Informatica Intelligent Cloud Services automatically synchronizes files at regular intervals within the S3 folders to the Secure Agent in serverless environment.
6. Benefits of Advanced Serverless
Below are the benefits of the Advanced Serverless Runtime Environment.
- No Servers to Manage: You’re free from server maintenance.
- Usage based Pricing: Pay only for what you use, cost-effective.
- Built-in Elasticity: Automatically scale with your workload.
- Cloud Native Architecture: Takes full cloud advantage for flexibility.
- On-demand Execution: Run when needed, no idle time.
- Built-in Resiliency:Ensures reliability and high availability.
Subscribe to our Newsletter !!
Related Articles:
When you install Secure Agent, it is added to its own group by default. You can either create a new group and add agents under it or add new agents in existing group.
A Secure Agent can be easily registered to a new IICS Org without the need of reinstalling it again. Learn the process of re-registering a secure agent.
Informatica Cloud Data Integration Elastic (CDI-E) enables you to process your jobs on advanced Spark serverless compute engine.
