Kaa releases
Shortcuts
Skip to end of metadata
Go to start of metadata

This page describes the installation and configuration procedures for Kaa flume agents on a single Linux node. It does not cover installation and configuration of third party systems, though. Flume agents provide advanced logging capabilities in case of integration with Hadoop as described in Flume log appender.

System OS versions

Kaa supports the following operation system families and provides an installation package for each of them.

  • Ubuntu and other Debian systems
  • Red Hat/CentOS/Oracle 5 or Red Hat 6 systems

System requirements

To use Kaa flume agents, your system should meet the following minimum system requirements.

  • 64-bit OS
  • 4 Gb RAM

Third party  components

Kaa flume agents require the following third party components to be installed and configured.

Install Kaa flume agents

Depending on the target operating system that you are using, perform the following installation steps.

  1. Download the latest debian packages from the Kaa download page.
  2. Install the Kaa flume package by executing the following command.

  1. Download the latest rpm packages from Kaa download page.
  2. Install the Kaa flume package by executing the following command.

After installing the Kaa flume package, you have to install and configure the Kaa flume source agent and Kaa flume sink agent. 

Set up Kaa flume source agent

To install and configure the Kaa flume source agent, perform the following steps.

  1. Enter the following command in the shell:

  2. Follow the installation instructions. You will be prompted to enter basic information. Enter the values shown in the following example after the colon (":") character.

Set up Kaa flume sink agent

To install and configure the Kaa flume sink agent, perform the following steps.

  1. Enter the following command in the shell.

  2. Follow the installation instructions. You will be prompted to enter basic information. Enter the values shown in the following example after the colon (":") character.

  3. Enter information necessary to locate the schema source.
    This information depends on the Avro schema source type that you have specified in the previous step.

  • If you have specified the "rest" schema source type, enter the following:

  • If you have specified the "local" schema source type, enter the following:

Please see the following additional information.

Schema files should be stored using the following pattern:

{path to local schema files}/{application token}/schema_v{log schema version}

            where:

{application token} - the application token for logs collection. It can be obtained from Kaa admin web UI.

{log schema version} - the version of the log schema within the application. It can be obtained from Kaa admin web UI.

 File schema_v{log schema version} should contain an avro log schema for the specified version in the text format.

 

Now that you have installed and configured the Kaa flume agents, you are ready to start them.

Start Kaa flume agents

  • To start the Kaa flume source, enter the following command in the shell.
  • To start the Kaa flume sink, enter the following command in the shell.

 

Validate/troubleshoot Kaa flume agents

To validate the installation of Kaa flume agents, see the logs located at /var/log/flume-ng/. If the agents were installed and configured correctly, the logs should not contain any exceptions or errors.

 


Copyright © 2014-2015, CyberVision, Inc.

  • No labels