Kaa releases
Shortcuts
Skip to end of metadata
Go to start of metadata

The Cassandra log appender is responsible for transferring logs from the Operations server to the Cassandra database.

Creating Cassandra log appender with Admin UI

The easiest way to create a Cassandra log appender for your application is by using Admin UI.

Creating Cassandra log appender with REST API

It is also possible to create a Cassandra log appender instance by using REST API. The following example illustrates how to provision the Cassandra log appender via REST API.

Configuration

The Cassandra log appender configuration should match the following Avro schema:

 Click here to expand...
  • Cassandra nodes - list of Cassandra hosts.
  • Authentication credentials - credentials used to authenticate on Cassandra cluster
  • Keyspace name – Cassandra keyspace used to prefix the data table
  • Table name pattern – pattern used to create table name (ex.: logs_$app_token adds the application token at the end of the table name).
  • Column Mapping - section that handles data mapping configuration. It can map specific log data to appropriate columns.

    TypeExample valueExample column typeDescription
    HEADER_FIELD
    endpointKeyHash,
    applicationToken,
    headerVersion,
    timestamp,
    logSchemaVersion
     TEXTMaps a header variable to specified the column
    EVENT_FIELD telemetryDOUBLEMaps a log schema field to the specified column
    CLIENT_FIELDclientFieldTEXTMaps a client-side endpoint profile field to the specified column
    SERVER_FIELDserverFieldTEXTMaps a server-side endpoint profile field to the specified column
    ${TYPE}_JSON TEXTMaps one of the fields listed above as a corresponding JSON
    ${TYPE}_BINARY BLOBMaps one of the fields listed above in binary format
    UUID(empty)UUIDMaps a UUID generated by Kaa to the specified field
    TSddMMyyBIGINTMaps a timestamp generated by Kaa to the specified field. Timestamp pattern is handled by java.text.SimpleDateFormat

     

    Any field can be made a partition and/or clustering key with using correspoding checkboxes.

     

    Key clustering is configured by Clustering section, by adding column names and setting their order (DESC or ASC).

     

NOTE: Cassandra driver settings can be specified in log appender configuration. Consult the official documentation for reference.

 

The following configuration example matches the previous schema.

 Click here to expand...

 

NOTE: A Cassandra KEYSPACE with an appropriate replication factor and strategy class must be created first in order to use the Cassandra log appender.

Administration

The following REST API call example illustrates how to create an instance of the Cassandra log appender.

 Example result

Playing with Cassandra log appender

We'll use Data collection demo from Kaa Sendbox. Our example will send data to Kaa and then persist it to Cassandra. Also, we'll do selection queries on persisted data.

We have next log schema: 

 Click here to expand...
 
{
"type":"record",
 "name":"LogData",
 "namespace":"org.kaaproject.kaa.schema.sample.logging",
 "fields":[
{
"name":"level",
 "type":{
"type":"enum",
 "name":"Level",
 "symbols":[
"KAA_DEBUG",
 "KAA_ERROR",
 "KAA_FATAL",
 "KAA_INFO",
 "KAA_TRACE",
 "KAA_WARN"
 ]
}
},
 {
"name":"tag",
 "type":"string"
 },
 {
"name":"message",
 "type":"string"
 }
]
}

The following JSON example matches the previous schema.

 Click here to expand...

{
"level" : "KAA_INFO",
"tag" : "TEST_TAG",
"message" : "My simple message"
}

Go to Data collection demos in Sandbox.

Follow Installation instructions.

Next, in the Admin UI follow to Data collection demo application

Go to application's Log appenders configuration and add a new one.

Enter name of the new appender (we'll use "Cassandra")

Select Cassandra appender type.

 

Add new node in the Configuration section (localhost:9042)

Add auth details if needed (for Sandbox it's empty)

Fill keyspace name. "kaa" is used in this example, because it's already created on a Sandbox machine.

"logs_example" is used as the Table name pattern.

The important part of configuration is Column Mapping:

 

Other configuration:

 

Now click Add button on the top of the screen to create and deploy appender.

Verify that newly created appender has appeared in list.

Now run Data collection demo application. Verify that logs have been successfully sent to Kaa

Let's verify that our logs have been persisted in Cassandra. Go to Sandbox VM and run next command to connect Cassandra:

~$ cqlsh

Then

cqlsh> SELECT * FROM kaa.logs_example 

You should observe similar output:

If your output doesn't match above one, please follow our troubleshooting guide.

 


Copyright © 2014-2016, CyberVision, Inc.

  • No labels