Skip to main content

Configuring Striim's metadata repository

You may host Striim's metadata repository on Derby, Oracle, or PostgreSQL. By default, Striim uses the integrated, preconfigured Derby instance. To use Oracle or PostgreSQL instead, follow the instructions in this section.

Caution

When using the integrated Derby instance in a production environment, we strongly recommend Changing the Derby password.

See also Changing metadata repository connection retry settings.

Caution

When Striim is processing very large amounts of data at high velocity, Derby may fail to reclaim the unused space in the metadata repository, resulting in it eventually filling all available disk space and crashing. In this situation, Striim will crash and restart will fail. To work around this issue, we recommend hosting the metadata repository on Oracle or PostgreSQL instead.

As a short-term workaround for Derby, use the following command to compress the metadata repository tables:

striim/bin/derbyTools.sh -A compressDb

This uses a lot of Derby resources, so it should be performed when Striim is not busy.

Configuring Oracle to host the metadata repository

You may host Striim's metadata repository on any version of Oracle that is supported by the bundled JDBC driver ojdbc-21.1.jar.

  1. Copy the SQL scripts /opt/striim/conf/DefineMetadataReposOracle.sql and DefineMeteringReposOracle.sqlto the Oracle host.

  2. Using sqlplus, log in to Oracle as an administrator and create the user Striim will use to create, write to, and read from the repository tables (replace ****** with a strong password):

    create user striimrepo identified by ******;
grant connect, resource to striimrepo;

  3. Log out of sqlplus, log in again as the user you just created, and run the DefineMetadataReposOracle.sql and DefineMeteringReposOracle.sql scripts.

Configuring PostgreSQL to host the metadata repository

You may host Striim's metadata repository on any version of PostgreSQL that is supported by the bundled JDBC driver postgresql-42.4.0.jar.

  1. Copy the SQL scripts /opt/striim/conf/DefineMetadataReposPostgres.sql and DefineMeteringReposPostgres.sql to the PostgreSQL host.

  2. Using psql, connect to PostgreSQL as an administrator and create the user Striim will use to create, write to, and read from the repository tables (replace ****** with a strong password):

    sudo -u postgres psql
create user striim with password '******';
create database striimrepo;
grant all on database striimrepo to striim;
\q

  3. Connect again as the user you just created, create a schema, set the search path, and run the DefineMetadataReposPostgres.sql and DefineMeteringReposPostgres.sql scripts:

    psql -U striim -d striimrepo;
create schema striim;
alter role striim set search_path to striim;
\q
psql -U striim -d striimrepo -f DefineMetadataReposPostgres.sql
psql -U striim -d striimrepo -f DefineMeteringReposPostgres.sql

Setting startUp.properties for the metadata repository

Generally you should follow the instructions in this section only when they are referred to from instructions for creating or adding a server to a cluster.

... when hosted on Derby

Typically, when using the internal Derby instance, the necessary properties will be set automatically.

If Derby is not running on port 1527, set the following properties:

MetaDataRepositoryLocation=<IP address>:<port>
DERBY_PORT=<port>

... when hosted on Oracle

Set the following properties:

MetadataDb=oracle
MetaDataRepositoryLocation=<connection URL>
MetaDataRepositoryDBname=striimrepo
MetaDataRepositoryUname=striimrepo

If you use an SID, the connection URL has the format jdbc:oracle:thin:@<IP address>:<port>:<SID>, for example, jdbc:oracle:thin:@192.0.2.0:1521:orcl. If you use a service name, it has the format jdbc:oracle:thin:@<IP address>:<port>/<service name>, for example, jdbc:oracle:thin:@192.0.2.01521:/orcl. I

In a high availability active-standby or RAC environment, specify all servers, for example, MetaDataRepositoryLocation=jdbc:oracle:thin:@(DESCRIPTION_LIST=(LOAD_BALANCE=off)(FAILOVER=on)(DESCRIPTION= (CONNECT_TIMEOUT=5)(TRANSPORT_CONNECT_TIMEOUT=3)(RETRY_COUNT=3)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST= 192.0.2.100)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=racdb.localdomain)))(DESCRIPTION= (CONNECT_TIMEOUT=5)(TRANSPORT_CONNECT_TIMEOUT=3)(RETRY_COUNT=3)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST= 192.0.2.101)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=racdb.localdomain)))) (see Features Specific to JDBC Thin for more information.)

When the connection uses SSL, the connection URL has the format:

MetaDataRepositoryLocation=jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)
(HOST=<IP address>)(PORT=<port>))(CONNECT_DATA=(SERVICE_NAME=<service name>)))

  • For SSL with a PKCS12 keystore, copy the ewallet.p12 file to the Striim server's local environment and set the following properties:

    OracleMDRTrustStoreType=PKCS12
OracleMDRTrustStore=<path>/ewallet.p12

  • For SSL with an SSO keystore, copy the cwallet.ss0 file to the Striim server's local environment and set the following properties:

    OracleMDRTrustStoreType=SSO
OracleMDRTrustStore=<path>/cwallet.sso

... when hosted on PostgreSQL

Set the following properties:

MetadataDb=postgres
MetaDataRepositoryLocation=<connection URL>
MetaDataRepositoryDBname=striimrepo
MetaDataRepositoryUname=striim

The PostgreSQL connection URL has the format <IP address>:<port>/striimrepo, for example 192.0.2.100:5432/striimrepo.

In a high availability environment, specify the IP addresses of both the primary and standby servers, separated by a comma, for example, 192.0.2.100,192.0.2.101:5432/striimrepo.

When the connection uses SSL, copy the postgresql.crt file to the Striim server's local environment and set the following property:

PostgresMDRCertPath=<path>/postgresql.crt

If using Azure Database for PostgreSQL, see Hosting Striim's metadata repository on Azure Database for PostgreSQL.

If using Google Cloud SQL for PostgreSQL, see How To Configure SSL Connection to Google Cloud SQL Postgres as Striim MDR? in the Striim Support knowledge base.

Moving the metadata repository to Oracle or PostgreSQL

To move the metadata repository from the Striim internal Derby instance to Oracle or PostgreSQL, do the following. This will require bringing down the Striim cluster, so you should schedule it for a maintenance window.

  1. Follow the instructions in Configuring Oracle to host the metadata repository or Configuring PostgreSQL to host the metadata repository.

  2. Back up Derby as described in Backing up the metadata repository host.

  3. Stop the Derby instance (striim-dbms) and all servers in the Striim cluster (see Starting and stopping Striim Platform).

  4. On the server running Derby, export the metadata:

    cd /opt/striim
sudo bin/tools.sh -A export -F export.json

  5. Make a backup copy of startUp.properties:

    cd /opt/striim/conf
cp startUp.properties *.bak

  6. On each server in the cluster, edit startUp.properties and change the value of MetaDataRepositoryLocation to reflect the new repository host (see the Oracle or PostgreSQL section of Setting startUp.properties for the metadata repository).

  7. On each server in the cluster, update the metadata repository user's password in the Striim keystore:

    cd /opt/striim
sudo su - striim bin/sksConfig.sh -p

  8. On the server where you exported the metadata, import it.

    For Oracle:

    cd /opt/striim
sudo bin/tools.sh -A import -F export.json -f 4.2.0 -r oracle

    For PostgreSQL:

    cd /opt/striim
sudo bin/tools.sh -A import -F export.json -f 4.2.0 -r postgres

  9. Stop Derby from starting automatically.

    sudo systemctl disable striim-dbms

    If you are Running Striim as a process, set NO_DERBY=true as an environment variable before running server.sh.

  10. Restart the Striim cluster (see Starting and stopping Striim Platform).