Skip to main content

Striim Platform 4.1.2 documentation

ALTER and RECOMPILE

Use these commands to modify applications that are loaded but not deployed or running (in other words, applications with status Stopped or Quiesced). Using ALTER instead of dropping and reloading retains persisted WActionStore and Kafka stream data. This could be useful, for example, when there is a DDL change to a source database table.

Note that some changes to the application can be made safely but others may be incompatible with persisted data, causing errors. For example:

should be compatible with persisted data

may be incompatible with persisted data

add components or flows

remove components or flows

add a field to a type

remove a field from a type

change a type from:

  • byte to integer, short, or long

  • integer to short or long

  • short to long

  • float to double

change a type from:

  • long to short, integer, or byte

  • short to integer or byte

  • integer to byte

  • double to float

Using ALTER when recovery is enabled (STOP vs. QUIESCE)

When you stop, alter, and restart an application with recovery enabled, it will resume operation with no missing or duplicate events ("exactly-once processing," also known as E1P), except as noted in Recovering applications and with the following possible excerptions:

should not interfere with exactly-once processing

may result in duplicate or missing events

  • adding components to an unbranched data flow (that is, to a series of components in which the output of each component is the input of only one other component)

  • removing components from an unbranched data flow

  • adding a branch to a data flow

  • removing a branch from a data flow

  • simple modifications to a CQ

  • modifying sources

  • changing a window's KEEP clause

  • changing a CQ's GROUP BY clause

  • changing the number of fields in a CQ's FROM clause

  • changing the size of a CQ's MATCH_PATTERN selection

  • changing the value of a CQ's LIMIT clause

  • changing a KafkaWriter's mode from sync to async

When you quiesce an application with recovery enabled, altering it in any way other than changing its recoverable sources will not interfere with exactly-once processing. However, there may be anomalous results (see QUIESCE for details).

Example

The workflow for ALTER and RECOMPILE is:

  1. If the application is running, STOP or QUIESCE it (see Console commands).

  2. Undeploy the application.

  3. Alter the application as described below.

  4. Recompile, deploy, and start the application.

To begin altering an application, use:

USE <application's namespace>;
ALTER APPLICATION <application name>;

At this point, enter CREATE, DROP, or CREATE OR REPLACE statements to modify the application, then complete the alteration with:

ALTER APPLICATION <application name> RECOMPILE;

For example, to add the email subscription described in Sending alerts from applications to the PosApp sample application:

USE Samples;
ALTER APPLICATION PosApp;
CREATE SUBSCRIPTION PosAppEmailAlert
USING EmailAdapter (
  SMTPUSER:'sender@example.com',
  SMTPPASSWORD:'password', 
  smtpurl:'smtp.gmail.com',
  starttls_enable:'true',
  subject:"test subject",
  emailList:"recipient@example.com,recipient2.example.com",
  senderEmail:"alertsender@example.com" 
)
INPUT FROM AlertStream;
ALTER APPLICATION PosApp RECOMPILE;

At this point you may deploy and start the modified application. If recovery was enabled for the application when it was loaded, when it is restarted, it will pick up source data (subject to the usual limitations detailed in Recovering applications) back to the time it went offline.

Keep in mind that a change made to one component may require changes to multiple downstream components and their types. For example, to add the event_url JSON source property to the following app you would need to modify both the MeetupJSONType and the ParseJSON CQ:

CREATE TYPE MeetupJSONType (
  venue_id integer KEY,
  group_name string,
  event_name string,
  event_url string,
  time DateTime,
  venue_name string,
  group_city string,
  group_country string,
  lat double,
  lon double
);
CREATE STREAM ParsedJSONStream OF MeetupJSONType;

CREATE CQ ParseJSON
INSERT INTO ParsedJSONStream
SELECT
  CASE
    WHEN data.has("venue") and data.get("venue").has("venue_id")
      THEN data.get("venue").get("venue_id").intValue()
    ELSE 0
  END,
  CASE
    WHEN data.has("group") and data.get("group").has("group_name")
      THEN data.get("group").get("group_name").textValue()
    ELSE "NA"
  END,
  CASE
    WHEN data.has("event") and data.get("event").has("event_name")
      THEN data.get("event").get("event_name").textValue()
    ELSE "NA"
  END,
  CASE
    WHEN data.has("event") and data.get("event").has("event_url")
      THEN data.get("event").get("event_url").textValue()
    ELSE "NA"
  END, ...

If the application has a dashboard, you might also need to edit its properties to add event_url there:

alterMeetup.png
In this section: