Skip to main content

Parsing the data field of WAEvent

WAEvent is the data type used by the output stream of many readers. Its data field is an array containing one event's field values. Here is a sample event in WAEvent format from the output stream of CsvDataSource in the PosApp sample application:

WAEvent{
  data: ["COMPANY 1159","IQ6wCy3k7PnAiRAN71ROxcNBavvVoUcwp7y","8229344557372754288","1","20130312173212",
"0614","USD","329.64","2094770823399082","79769","Odessa"]
  metadata: {"RecordStatus":"VALID_RECORD","FileName":"posdata.csv","FileOffset":154173}
  before: null
  dataPresenceBitMap: "AAA="
  beforePresenceBitMap: "AAA="
  typeUUID: null
};

For information on metadata, see Using the META() function.

dataPresenceBitMap, beforePresenceBitMap, and typeUUID are reserved and should be ignored.

To parse the data array, PosApp uses the following TQL:

CREATE CQ CsvToPosData
INSERT INTO PosDataStream
SELECT TO_STRING(data[1]) AS merchantId,
  TO_DATEF(data[4],'yyyyMMddHHmmss') AS dateTime,
  DHOURS(TO_DATEF(data[4],'yyyyMMddHHmmss')) AS hourValue,
  TO_DOUBLE(data[7]) AS amount,
  TO_STRING(data[9]) AS zip
FROM CsvStream;

  • PosDataStream is created automatically using the data types and AS strings in the SELECT statement.

  • The order of the data[#] functions in the SELECT clause determines the order of the fields in the output. These may be specified in any order: for example, data[1] could precede data[0].

  • Fields not referenced by the the SELECT clause are discarded.

  • The data[#] function counts the fields in the array starting from 0, so in this example the first field in the array (COMPANY 1159) is omitted.

  • Non-string values are converted to the types required by the output stream (as defined by the PosData type) by the TO_DATEF, DHOURS, and TO_DOUBLE functions (see Functions for more information).

In the PosDataStream output, the parsed version of the sample event shown above is:

merchantId: "IQ6wCy3k7PnAiRAN71ROxcNBavvVoUcwp7y"
dateTime: 1363134732000
hourValue: 17
amount: 329.64
zip: "79769"

See PosApp for more information. See also the discussions of ParseAccessLog and ParseLog4J in MultiLogApp for additional examples.

To put the raw, unparsed data array into a field in a stream, use this syntax:

CREATE CQ CsvRawData
INSERT INTO PosRawDataStream
SELECT data AS object[]
FROM CsvStream;

The field type will be an array of Java objects.