Unknown macro: {style}

.ia-secondary-container

Unknown macro: { margin}
Unknown macro: {div}
Kaa releases
Unknown macro: {div}
Shortcuts

Page tree
Skip to end of metadata
Go to start of metadata

The Kaa logging subsystem is designed to collect records (logs) of pre-configured structure on endpoints, periodically deliver these records from endpoints to Operation servers, and either persist them on the server for further processing or submit to immediate stream analysis. The log structure in Kaa is determined for each application by a configurable log schema. On the Operation server side, there are log appenders which are responsible for writing logs received by the Operations server into the specific storage. A Kaa tenant administrator can define only one log appender per application.

Please view logging system design reference for more background information.

From this guide you will learn how to use the Kaa logging subsystem for data collection.

Configuring

This section illustrates how to configure a log schema.

Log schema

The default log schema installed for Kaa applications is empty. You can configure your own log schema using the Admin UI or REST API. For the purpose of this guide, we will use schema that is very close to the common log structure: the log level, tag and message.

{  
    "type":"record",
    "name":"LogData",
    "namespace":"org.kaaproject.kaa.schema.sample.logging",
    "fields":[  
        {  
            "name":"level",
            "type":{  
                "type":"enum",
                "name":"Level",
                "symbols":[  
                    "DEBUG",
                    "ERROR",
                    "FATAL",
                    "INFO",
                    "TRACE",
                    "WARN"
                ]
            }
        },
        {  
            "name":"tag",
            "type":"string"
        },
        {  
            "name":"message",
            "type":"string"
        }
    ]
}

Log appenders

Kaa provides default implementations of log appenders that store logs in Hadoop, MongoDB or a local file system (FS). It is possible to develop and integrate custom log appenders.

Coding

The logging subsystem API varies depending on the target SDK platform. However, the general approach is the same.

To transfer logs to the Kaa Operation server, the Kaa client application should use the following code.

// Get the Log Collector reference
LogCollector logCollector = Kaa.getKaaClient().getLogCollector();
// Create a log entity (according to the org.kaaproject.sample.LogData sample schema above)
LogData logRecord = new LogData(Level.INFO, "tag", "message");
// Push the record to the collector
logCollector.addLogRecord(logRecord);
using namespace kaa;
 
// Get the Log Collector reference
ILogCollector& logCollector = Kaa::getKaaClient().getLogCollector();
// Create a log entity (according to the org.kaaproject.sample.LogData sample schema above)
 
LogData logRecord;
logRecord.level = Level::INFO;
logRecord.tag = "tag";
logRecord.message = "message";
// Push the record to the collector
logCollector.addLogRecord(logRecord);
#include <stdlib.h>
#include <kaa/kaa.h>
#include <kaa/gen/kaa_logging_gen.h>
#include <kaa/log/kaa_memory_log_storage.h>

// Assume Kaa sdk is already initialized
// Initialize the log storage (by default it is not set). Here we use internal memory log storage distributed with Kaa sdk.
kaa_init_log_storage(get_memory_log_storage(), get_memory_log_storage_status(), get_memory_log_upload_properties(), &memory_log_storage_is_upload_needed);

// Create the log record.
kaa_logging_log_data_t * log = kaa_logging_create_log_data();
log->level      = ENUM_LEVEL_INFO;
log->tag        = "tag";
log->message    = "message";


// Add created log record to the log collector
kaa_add_log(log);


// Release memory occupied by log record instance
free(log);

By default, Kaa uses an in-memory temporary log storage for the endpoint (MemoryLogStorage). Normally, this storage does not persist data when the client is restarted. If this is a concern, the client application developers are welcome to use their own implementations of the log storage by implementing the ILogStorage and ILogStorageStatus interfaces and assigning them to the logger.

Assuming that a new log storage is called PersistentLogStorage and the new log storage status is PersistentLogStorageStatus, consider the following example of a persistent log storage.

// Get the Log Collector reference
LogCollector logCollector = Kaa.getKaaClient().getLogCollector();
// Setting log storage implementation
logCollector.setStorage(new PersistentLogStorage());
// Setting log storage status implementation
logCollector.setStorageStatus(new PersistentLogStorageStatus());
using namespace kaa;
 
ILogStorage * storage = new PersistentLogStorage();
ILogStorageStatus * status = new PersistentLogStorageStatus();
// Get the Log Collector reference
ILogCollector & logCollector = Kaa::getKaaClient().getLogCollector();
// Setting log storage implementation
logCollector.setStorage(storage);
// Setting log storage status implementation
logCollector.setStorageStatus(status);
#include <kaa/kaa.h>
#include "persistent_log_storage.h" // Definition of persistent log storage. 

// Initialize the log storage
kaa_init_log_storage(get_persistent_log_storage(), get_persistent_log_storage_status(), get_log_upload_properties(), &persistent_log_storage_is_upload_needed);

Copyright © 2014, CyberVision, Inc.

  • No labels