Skip to main content

Striim Cloud 4.1.2 documentation

Striim Forwarding Agent installation and configuration

The Striim Forwarding Agent is a stripped-down version of the Striim server that can be used to run sources locally on a remote host. For more information, see Using the Forwarding Agent.

To install the Forwarding Agent when you deploy Striim from the AWS Marketplace, contact Striim support.

Striim Forwarding Agent system requirements

  • memory: 256MB to 1GB depending on adapter used

  • free disk space: 500MB (free disk space must never drop below 10%)

  • supported operating systems: any 64-bit version of Linux, Mac OS X, or Microsoft Windows

  • supported Java environments:

    • recommended: 64-bit Oracle Java SE 8 JRE (JDK required to use HTTPReader or SNMPParser)

    • also supported: 64-bit OpenJDK 8 JRE

  • firewall: the following ports must be open:

    • port 5701 inbound for TCP (Hazelcast): if 5701 is in use, Hazelcast will use 5702; if both 5701 and 5702 are busy, it will use 5703; and so on

    • port 9081 outbound for TCP (HTTPS) for authentication with the Striim cluster; alternatiely, to use port 9080, set striim.cluster.https.enabled=false in agent/conf/agent.conf.

    • ports 49152-65535 outbound for TCP

  • Any driver required by a source that will run on the Agent must be installed in Agent/lib. See Installing third-party drivers in the Forwarding Agent.

The Forwarding Agent gets its license from the Striim cluster. It does not need to be specified on the remote host where the agent is installed.

Running the Forwarding Agent as a process

  1. Using a supported web browser, log into Striim at <DNS name>:9080 with username admin and the admin password you specified when creating the Striim server.

  2. Select Help > Download Agent and save Striim_Agent_4.1.2.zip  to an appropriate location.  If you can not download it directly to the  host, download it to your local system and use scp to copy it to the host.

  3. Unzip the file. If you used scp to copy the file to the host, use ssh to log into the host and unzip it.

  4. Follow the instructions in Configuring the Forwarding Agent.

To start the agent, make sure the Striim cluster the Agent will connect to is running, open a shell terminal, command prompt, or ssh session, change to the Agent directory, and enter bin/agent.sh or, in Windows, bin\agent.bat.

Once you have successfully started the agent and connected to Striim, you may delete the .zip file.

To stop the agent, switch to the command prompt in which it is running and press Ctrl-C.

Running the Forwarding Agent as a service in CentOS

  1. Log in to Linux.

  2. Download striim-agent-4.1.2-Linux.rpm.

  3. Install the package: sudo rpm -ivh striim-agent-4.1.2-Linux.rpm. By default, the agent is installed in /opt/striim/agent/.

  4. Follow the instructions in Configuring the Forwarding Agent.

  5. Make sure the Striim server is running, then:

    • For CentOS 6, enter sudo start striim-agent.

    • For CentOS 7, enter:

      sudo systemctl enable striim-agent
sudo systemctl start striim-agent

To verify that the agent connected to the server, look on the Monitoring page (see Monitoring using the web UI).

Running the Forwarding Agent as a service in Ubuntu

  1. Log in to Linux and run sudo su to switch to root.

  2. Download striim-agent-4.1.2-Linux.deb.

  3. Install the package: dpkg -i striim-agent-4.1.2-Linux.deb. The agent is installed at /opt/striim/agent.

  4. Follow the instructions in Configuring the Forwarding Agent.

  5. Make sure the Striim server is running, then:

    • for Ubuntu 14.04, enter start striim-agent

    • for Ubuntu 16.04 or later, enter systemctl enable striim-agent && systemctl start striim-agent

To verify that the agent connected to the server, look on the Monitoring page (see Monitoring using the web UI).

Running the Forwarding Agent as a service in Windows

This requires Windows PowerShell 5.0 or later.

  1. Follow the instructions in Running the Forwarding Agent as a process, but do not start the agent.

  2. Start Windows PowerShell as administrator and run the script Agent\conf\windowsAgent\setupWindowsAgent.ps1. Note that if you are in the windowsAgent directory, to run the script you must include the path: .\setupWindowsAgent.ps1.

    The message Error in OpenSCManager is of no concern if the script completes successfully.

  3. Start the Striim Agent service manually, or reboot to verify that it starts automatically.

To uninstall the service, stop it, then run this batch file:

Agent/conf/windowsAgent/yajsw_agent/bat/uninstallService.bat

Configuring the Forwarding Agent

Before starting the agent, you must:

  1. If running Striim Cloud, Contact Striim support to get the sys password, striim.cluster.clusterName, and striim.node.servernode.address.

  2. Run agent/bin/aksConfig.sh or aksConfig.bat. When prompted, enter a password for the agent's local keystore and the sys password (used to authenticate the agent when it connects to the Striim cluster).

  3. Install any third-party drivers required by the readers that will run in the Forwarding agent (see Installing third-party drivers in the Forwarding Agent). No drivers are bundled with the Forwarding Agent.

  4. Edit agent/conf/agent.conf to specify the agent's settings, as follows:

  • Always required: for the striim.cluster.clusterName property value, specify the cluster to connect to. Cluster names are case-sensitive.

  • Always required: for the striim.node.servernode.address property value, specify the server's IP address or fully-qualified domain name. If the agent may connect to multiple servers, specify them all, separated by commas.

  • If the system on which the agent is running has more than one IP address, specify the one you want the agent to use as the value for the striim.node.interface property.

  • By default, the agent will join the Agent deployment group. If you wish to change that, specify another deployment group as the value for the striim.cluster.deploymentGroups property. If the specified deployment group does not exist, it will be created automatically. When multiple agents on different remote hosts will be used by the same source, they must belong to the same deployment group.

  • When using Striim Cloud:

    1. Connect the system running the Forwarding Agent with Striim Cloud using a VPN tunnel.

    2. In agent.conf, add striim.node.isSaaS=true

    3. In Striim Cloud, open the Console and enter set SYSTEMPROP='striim.node.serverDomainName,<IP address>', replacing <IP address> with the Striim Cloud-side IP address for the VPN tunnel.

Installing third-party drivers in the Forwarding Agent

This section describes installation of the third-party drivers required to use some adapters.

Install the HP NonStop JDBC driver in a Forwarding Agent

This driver must be installed in every Forwarding Agent that will read from HP NonStop SQL/MX.

Caution

Do not install drivers for multiple versions of SQL/MX on the same Striim server. If you need to write to multiple versions of SQL/MX, install their drivers on different Striim servers and run each version's applications on the appropriate Striim server(s).

  1. Follow the instructions in the "Installing and Verifying the Type 4 Driver" section of HPE NonStop JDBC Type 4 Driver Programmer's Reference for SQL/MX for the SQL/MX version you are running to copy the driver .tar file from the HP NonStop system with the tables that will be read to a client workstation and untar it. Do not install the driver.

  2. Copy the t4sqlmx.jar file from the untarred directory to Agent/lib.

  3. Stop and restart the Forwarding Agent (see Starting and stopping the Forwarding Agent).

Install the MariaDB JDBC driver in a Forwarding Agent

This driver must be installed in every Forwarding Agent that will read from MariaDB.

  1. Download mariadb-java-client-2.4.3.jar from http://downloads.mariadb.com/Connectors/java/connector-java-2.4.3.

  2. Copy that file to Agent/lib.

  3. Stop and restart the Forwarding Agent (see Starting and stopping the Forwarding Agent).

Install the MemSQL JDBC driver in a Forwarding Agent

MemSQL uses MySQL's JDBC driver. See Install the MySQL JDBC driver in a Forwarding Agent.

Install the Microsoft JDBC Driver in a Forwarding Agent

This driver must be installed in every Forwarding Agent that will read from Microsoft SQL Server, Azure SQL Database, or Azure Synapse.

That driver is not bundled with the Forwarding Agent, so to read from those sources, you must install one of the drivers.

Caution

Do not install both versions of the driver in the same Striim server or Forwarding Agent.

For SQL Server 2008:

  1. Download the Microsoft JDBC Driver 6.0 for SQL Server .gz package from https://www.microsoft.com/en-us/download/details.aspx?id=11774 and extract it.

  2. Copy enu/jre8/sqljdbc42.jar to agent/lib.

  3. Stop and restart the Forwarding Agent (see Starting and stopping the Forwarding Agent).

For all other versions:

  1. Download the Microsoft JDBC Driver 7.2 for SQL Server .gz package from https://www.microsoft.com/en-us/download/details.aspx?id=57782 and extract it.

  2. Copy enu/mssql-jdbc-7.2.2.jre8.jar to agent/lib.

  3. Stop and restart the Forwarding Agent (see Starting and stopping the Forwarding Agent).

Install the MySQL JDBC driver in a Forwarding Agent

This driver must be installed in every Forwarding Agent that will read from MySQL.

  1. Download the Connector/J 8.0.27 package from https://downloads.mysql.com/archives/c-j/ and extract it.

  2. Copy mysql-connector-java-8.0.27.jar to Agent/lib.

  3. Stop and restart the Forwarding Agent (see Starting and stopping the Forwarding Agent).

Install the Oracle Instant Client in a Forwarding Agent

The Oracle Instant Client version 21c must be installed and configured in the Linux host environment of every Forwarding Agent that will run OJet.

  1. Download the client from Oracle Instant Client Downloads and follow the installation procedure provided by Oracle for the target platform.

  2. Edit Agent/conf/agent.conf and add the NATIVE_LIBS property to specify the Instant Client path, for example, NATIVE_LIBS=/usr/local/instantclient_21.1.

  3. Stop and restart the Forwarding Agent (see Starting and stopping the Forwarding Agent).

Install the Oracle JDBC driver in a Forwarding Agent

This driver must be installed on each Forwarding Agent that will read from Oracle.

  1. Download ojdbc8.jar from oracle.com.

  2. Save that file in agent/lib.

  3. Stop and restart the Forwarding Agent (Starting and stopping the Forwarding Agent).

Install the PostgreSQL JDBC driver

The PostgreSQL JDBC driver is bundled with Striim Platform and Striim Cloud, but must be installed on each Forwarding Agent that will read from PostgreSQL.

  1. Download the PostgreSQL JDBC 4.2 driver from jdbc.postgresql.org/download/.

  2. Copy it to agent/lib.

  3. Stop and restart the Forwarding Agent (Starting and stopping the Forwarding Agent).

Install the Snowflake JDBC driver

This driver is must be installed in every Forwarding Agent that will write to Snowflake.

  1. Download the Snowflake JDBC driver version 3.13.15 (snowflake-jdbc-3.13.15.jar) as described in Downloading / Integrating the JDBC Driver.

  2. Copy the downloaded .jar file to Agent/lib.

  3. Stop and restart the Forwarding Agent (Starting and stopping the Forwarding Agent).

Install the Teradata JDBC driver in a Forwarding Agent

This driver is must be installed in every Forwarding Agent that will read from Teradata.

  1. Download the Teradata JDBC .tgz or .zip package from http://downloads.teradata.com/download/connectivity/jdbc-driver and extract it.

  2. Copy tdgssconfig.jar and terajdbc4.jar to agent/lib.

  3. Stop and restart the Forwarding Agent (see Starting and stopping the Forwarding Agent).

Testing the Forwarding Agent

You may use this application (the sample data is installed with the agent) to verify that the agent is working:

CREATE APPLICATION agentTest;

CREATE FLOW AgentFlow;
CREATE SOURCE CsvDataSource USING FileReader (
  directory:'SampleData',
  wildcard:'PosDataAgentSample.csv',
  positionByEOF:false)
PARSE USING DSVParser (
  header:Yes,
  trimquote:false)
OUTPUT TO CsvStream;
END FLOW AgentFlow;

CREATE FLOW ServerFlow;
CREATE TARGET t USING FileWriter( filename:'AgentOut')
FORMAT USING DSVFormatter ()
INPUT FROM CsvStream;
END FLOW ServerFlow;

END APPLICATION agentTest;

Deploy AgentFlow in the Agent group and ServerFlow and agentTest in the Default group. Then run the application and verify that AgentOut.00 has been written to the Striim program directory.

Starting and stopping the Forwarding Agent

To start a Forwarding Agent

If the agent was installed as a service:

  • In CentOS 6 or Ubuntu 14.04:

    sudo start striim-agent

  • In CentOS 7 or Ubuntu 16.04 or later:

    sudo systemctl start striim-agent

To start the agent as a process, run Agent/bin/server.bat in Windows or Agent/bin/server.sh in OS X or Linux.

To stop a Forwarding Agent

If the agent is running as a service:

  • In CentOS 6 or Ubuntu 14.04:

    sudo stop striim-agent

  • In CentOS 7 or Ubuntu 16.04 or later:

    sudo systemctl stop striim-agent

  • In Windows (using PowerShell as an administrator):

    service-stop "Striim Agent"

If the agent is running as a process, press Ctrl-C in the terminal running agent.bat or agent.sh.

Upgrading Forwarding Agents

  1. Stop the agent (see Starting and stopping the Forwarding Agent).

  2. Uninstall the agent:

    • If the agent is running as a process, stop the process, copy agent.conf and any drivers you added to agent/lib to another location, and delete the agent directory.

    • If the agent is running as a service in CentOS, open a terminal and run rpm -e striim-agent.

    • If the agent is running as a service in Ubuntu, open a terminal and run dpkg --remove striim-agent.

    • If the agent is running as a service in Windows, stop the Striim Agent process, copy agent.conf and any drivers you added to agent/lib to another location, then run Agent/conf/windowsAgent/yajsw_agent/bat/uninstallService.bat.

  3. Install the new version of the agent as described in the "Running the Forwarding Agent" topic appropriate for your environment. If running the agent as a process or as a service in Windows, use the backed-up copy of agent.conf, and before starting the agent copy any backed-up drivers to agent/lib. (If using rpm or dpkg, these files are preserved.)

Updating the sys user password on the Forwarding Agent

If the sys user's password changes, use the following command to update the password (required to connect to a Striim cluster) on the agent:

agent/bin/aksConfig.sh  -p <new password>

Enabling agent failover

When a flow in an application with recovery enabled is deployed ON ONE (see DEPLOY APPLICATION) in a deployment group with more than one agent, and that agent goes down, Striim will automatically deploy the flow on another agent in the deployment group. The flow will continue running on the other agent even after the original agent comes back up.

In this section: