The Kaa logging subsystem is designed to collect records (logs) of pre-configured structure on endpoints, periodically deliver these records from endpoints to Operation servers, and either persist them on the server for further processing or submit to immediate stream analysis. The log structure in Kaa is determined for each application by a configurable log schema. On the Operation server side, there are log appenders which are responsible for writing logs received by the Operations server into the specific storage. A Kaa tenant administrator can define only one log appender per application.
Please view logging system design reference for more background information.
From this guide you will learn how to use the Kaa logging subsystem for data collection.
The following diagram illustrates basic entities and data flows in scope of the log management:
- The data from the endpoints (logs) is collected and transferred to the server in the format as defined by the log schema created by the developer for the application
- Log appenders submit the logs received by the server to a particular storage or analytics system
This section illustrates how to configure a log schema.
The default log schema installed for Kaa applications is empty. You can configure your own log schema using the Admin UI or REST API. For the purpose of this guide, we will use schema that is very close to the common log structure: the log level, tag and message.
Kaa provides default implementations of log appenders that store logs in Hadoop, MongoDB or a local file system (FS). It is possible to develop and integrate custom log appenders.
The logging subsystem API varies depending on the target SDK platform. However, the general approach is the same.
To transfer logs to the Kaa Operation server, the Kaa client application should use the following code.
By default, Kaa uses an in-memory temporary log storage for the endpoint (MemoryLogStorage). Normally, this storage does not persist data when the client is restarted. If this is a concern, the client application developers are welcome to use their own implementations of the log storage by implementing the ILogStorage and ILogStorageStatus interfaces and assigning them to the logger.
Assuming that a new log storage is called PersistentLogStorage and the new log storage status is PersistentLogStorageStatus, consider the following example of a persistent log storage.