Connecting Validata Cloud on Google Cloud Platform to managed data services
This topic explains how to enable private connectivity from Validata Cloud on Google Cloud Platform to managed Google Cloud services. Private Service Connect allows Validata Cloud to access these services without exposing traffic to the public internet.
There are two main approaches for connecting to managed Google Cloud services:
Google APIs via the
allgoogleapisendpoint: For services like BigQuery that are accessed through Google APIs.
Customer-managed databases via your infrastructure: For services like Cloud SQL that require routing through your own GCP project infrastructure.
Connecting to Google APIs
Use this approach for Google Cloud services that are accessed through standard Google APIs, including BigQuery. This method uses Google Cloud's built-in allgoogleapis Private Service Connect endpoint.
Overview
Google Cloud's Private Service Connect routes traffic to Google APIs privately over Google's network. This keeps data off the public internet while preserving full functionality for supported services such as BigQuery, MySQL, and PostgreSQL. The same flow applies across these services.
Supported data systems
BigQuery
MySQL
Google Cloud SQL for MySQL
PostgreSQL
Google AlloyDB for PostgreSQL
Google Cloud SQL for PostgreSQL
Prerequisites
Access to the Validata Cloud console with Admin permissions to create endpoints and apps.
The Validata service must be in the Running state.
Implementation steps
This process creates a PSC endpoint that connects to Google's allgoogleapis service, providing private access to supported Google Cloud services.
Log in to the Validata Cloud portal.
Select the service that you want to create a Private Service Connect from.
Under the Secure connection tab, choose Create Private Service Connect.
Use a unique name for this connection.
Leave the Service attachment field empty and select Create.
An endpoint named for allgoogleapis will be created. Wait until its status is Running, then note the endpoint name to use in the application connection configuration.
Connecting to customer-managed databases
Use this approach for managed Google Cloud databases like Cloud SQL that require routing through your own GCP project infrastructure. This method requires setting up Private Service Connect infrastructure in your GCP project.
Overview
This pattern routes traffic from Validata Cloud through a Private Service Connect attachment in your GCP project to reach managed databases like Cloud SQL. The connection uses your project's network infrastructure to provide private connectivity.
Supported services
Cloud SQL
Cloud PostgreSQL
Cloud MySQL
Prerequisites
A VPC network with one additional subnet for PSC.
IAM rule for the user with above services creation permission.
A CloudSQL database with a private IP.
Step-by-step process to set up the network pipeline in customer project
This process establishes the necessary infrastructure in your GCP project to route traffic privately from Validata Cloud to your managed databases.
Create an Ubuntu VM instance with the following:
An e2-medium should be sufficient for this solution.
Launch the instance in the same VPC network as the database.
SSH to the instance and set up an IP forwarding rule to forward all incoming traffic to the database by using the commands below (change the IP and port accordingly for your target database).
Determine if forwarding is currently enabled or disabled. The command below outputs the value of the given parameter. A value of 1 indicates that the setting is enabled, while 0 indicates it is disabled.
sudo sysctl net.ipv4.ip_forward
Open the file /etc/sysctl.conf using your preferred command-line editor, such as vi.
sudo vi /etc/sysctl.conf
Find the line corresponding with the type of forwarding you wish to enable, uncomment it, and set the value to 1. Alternatively, you can add the lines anywhere in the file.
After the changes have been saved, apply the changes by running the following command or by rebooting the machine.
sudo sysctl -p
Review the existing iptables rules. If you are on a fresh installation of Linux and do not have any preconfigured rules, the output of the command below should be empty.
sudo iptables-save
Configure iptables to allow port forwarding. This is the default setting for many systems.
sudo iptables -A FORWARD -j ACCEPT
Run the following commands to set up the forwarding rule in the iptable:
sudo iptables -t nat -A PREROUTING -p tcp --dport 5432 -j DNAT --to-destination sudo iptables -t nat -A POSTROUTING -j MASQUERADE sudo cat /proc/sys/net/ipv4/ip_forward sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward" sudo iptables-save
To make this rule persistent during reboot, download the iptables-persistent package (press yes in the popup window to acknowledge).
sudo apt install iptables-persistent
To review the iptables rule:
iptables -t nat -nvL
Go to the Compute Engine portal and choose Instance groups on the left panel.
Select Create instance group.
Choose New unmanaged instance group.
Enter an instance group name, location, zone, and other information:
For location and zone, choose the same location and zone as the proxy instance that you created.
For network interface, choose the same VPC, subnet, and instance VM that you created or used earlier for the proxy instance.
Choose Create.
Create a load balancer:
Choose TCP load balancer.
Select Only between my VMs, Single region only, Pass-through, then choose Continue.
Give it a name and choose the same region and network as the proxy VM instance.
In the backend configuration, choose the instance group you created.
Create a health check probe for the port you assigned for your database.
Select Frontend configuration.
Enter a name for the configuration.
Select the same network you have been using.
Choose an internal IP purpose, shared or non-shared.
For ports, choose all or any specific port(s).
Disable global access.
Choose Create.
Go to the Load balancer portal.
Select Private service connect on the left panel.
Select Published service, then choose Create new.
In the configuration, choose the load balancer type that you created before and select the load balancer from the dropdown.
Give it a name.
In the subnet section, choose Reserve a subnet for PSC.
Enter a subnet name and a CIDR block with /24.
Choose connection preference Accept connections for selected projects.
Select Add service to create this service.
After creating the service, select the service and copy the Service attachment. You will need this attachment to create a PSC connection in the Validata Cloud console.
Create a Private Service Connection in the Validata Cloud project
Once the customer infrastructure is ready, create the corresponding PSC endpoint in Validata Cloud to establish the connection.
Log in to the Validata Cloud console.
Select the service that you want to create a Private Service Connect from.
Under the Secure connection tab, choose Create Private Service Connect.
Use a unique name for this connection.
Paste the private service attachment URL that you copied previously and select Create. The attachment looks like:
projects/striim-tools/regions/us-central1/serviceAttachments/psc-demo-service
You have now created a PSC connection endpoint. It is in a pending state until the request is accepted.
Once the request is approved from the PSC published service portal, the endpoint status changes to "Running".
Accept PLS connection request in customer project
The final step is to approve the connection request from your GCP project.
Go to the Private Service Connect portal, then the published service.
Select the published service.
You will see a request in the pending state.
You have a choice to Accept or Reject the request.
Reject will refuse the new connection request or disconnect the existing connection.