Installing Kaa flume agents

This page describes the installation and configuration procedures for Kaa flume agents on a single Linux node. It does not cover installation and configuration of third party systems, though. Flume agents provide advanced logging capabilities in case of integration with Hadoop as described in Flume log appender.

System OS versions

Kaa supports the following operation system families and provides an installation package for each of them.

Ubuntu and other Debian systems
Red Hat/CentOS/Oracle 5 or Red Hat 6 systems

System requirements

To use Kaa flume agents, your system should meet the following minimum system requirements.

64-bit OS
4 Gb RAM

Third party components

Kaa flume agents require the following third party components to be installed and configured.

Oracle JDK. Kaa flume agents have been tested with JDK 7.
Apache flume NG 1.5 and higher. Kaa flume agents have been tested with the latest production release of Flume NG.

Install Kaa flume agents

Depending on the target operating system that you are using, perform the following installation steps.

Download the latest debian packages from the Kaa download page.
Install the Kaa flume package by executing the following command.
```
sudo dpkg -i kaa-flume-x.y.z.deb
```

Download the latest rpm packages from Kaa download page.
Install the Kaa flume package by executing the following command.
```
sudo rpm -i kaa-flume-x.y.z.rpm
```

After installing the Kaa flume package, you have to install and configure the Kaa flume source agent and Kaa flume sink agent.

Set up Kaa flume source agent

To install and configure the Kaa flume source agent, perform the following steps.

Enter the following command in the shell:
```
sudo kaa-flume install source
```

Follow the installation instructions. You will be prompted to enter basic information. Enter the values shown in the following example after the colon (":") character.

        $ [INPUT] Please specify Kaa Flume Source Instance Name [default: 'default']: instance1
        $ [INPUT] Please specify Kaa Flume Source Host Name [default: 'localhost']: 10.2.3.93
        $ [INPUT] Please specify Kaa Flume Source Port: 7060
        $ [INPUT] Please specify Target Kaa Flume Sinks Count: 1
        $ [INPUT] Please specify Target Kaa Flume Sink 1 Host Name: 10.2.3.93
        $ [INPUT] Please specify Target Kaa Flume Sink 1 Port: 7070

Set up Kaa flume sink agent

To install and configure the Kaa flume sink agent, perform the following steps.

Enter the following command in the shell.
```
sudo kaa-flume install sink
```

Follow the installation instructions. You will be prompted to enter basic information. Enter the values shown in the following example after the colon (":") character.

        $ [INPUT] Please specify Kaa Sink Host Name: 10.2.3.93
        $ [INPUT] Please specify Kaa Sink Port: 7070
        $ [INPUT] Please specify Name node [host:port] or nameservice name: 10.2.3.93:8020
        $ [INPUT] Please specify HDFS root path [default: 'logs']: {press enter to use default location}
        $ [INPUT] Please specify Avro schema source type [rest|local] [default: 'rest']: {press enter to use Kaa REST API as Avro schema source for log events}

Enter information necessary to locate the schema source.
This information depends on the Avro schema source type that you have specified in the previous step.

If you have specified the "rest" schema source type, enter the following:

        $ [INPUT] Please specify Kaa Admin Rest API host: 10.2.3.93
        $ [INPUT] Please specify Kaa Admin Rest API port [default: '8080']: 8080
        $ [INPUT] Please specify Kaa Admin User: devuser {user should have 'Tenant developer' or 'Tenant user' authority}
        $ [INPUT] Please specify Kaa Admin Password: devuser123

If you have specified the "local" schema source type, enter the following:

        $ [INPUT] Please specify Absolute path to local schema files: {specify absolute path to directory with schema files}

Please see the following additional information.

Schema files should be stored using the following pattern:

{path to local schema files}/{application token}/schema_v{log schema version}

where:

{application token} - the application token for logs collection. It can be obtained from Kaa admin web UI.

{log schema version} - the version of the log schema within the application. It can be obtained from Kaa admin web UI.

File schema_v{log schema version} should contain an avro log schema for the specified version in the text format.

Now that you have installed and configured the Kaa flume agents, you are ready to start them.

Start Kaa flume agents

To start the Kaa flume source, enter the following command in the shell.

sudo service kaa-flume-source-{source instance name} start

To start the Kaa flume sink, enter the following command in the shell.

sudo service kaa-flume-sink start

Validate/troubleshoot Kaa flume agents

To validate the installation of Kaa flume agents, see the logs located at /var/log/flume-ng/. If the agents were installed and configured correctly, the logs should not contain any exceptions or errors.

Guide	What it is for
Administration UI guide	Use this guide to start working with the Kaa web UI.
Design reference	Use this reference to learn about features and capabilities of Kaa.
Programming guide	Use this guide to create your own Kaa applications.

Kaa releases

Shortcuts

Page tree