.ia-secondary-container
This page describes the installation and configuration procedures for Kaa flume agents on a single Linux node. It does not cover installation and configuration of third party systems, though. Flume agents provide advanced logging capabilities in case of integration with Hadoop as described in Flume log appender.
Kaa supports the following operation system families and provides an installation package for each of them.
To use Kaa flume agents, your system should meet the following minimum system requirements.
Kaa flume agents require the following third party components to be installed and configured.
Depending on the target operating system that you are using, perform the following installation steps.
Install the Kaa flume package by executing the following command.
sudo dpkg -i kaa-flume-x.y.z.deb
Install the Kaa flume package by executing the following command.
sudo rpm -i kaa-flume-x.y.z.rpm
After installing the Kaa flume package, you have to install and configure the Kaa flume source agent and Kaa flume sink agent.
To install and configure the Kaa flume source agent, perform the following steps.
Enter the following command in the shell:
sudo kaa-flume install source
Follow the installation instructions. You will be prompted to enter basic information. Enter the values shown in the following example after the colon (":") character.
$ [INPUT] Please specify Kaa Flume Source Instance Name [default: 'default']: instance1 $ [INPUT] Please specify Kaa Flume Source Host Name [default: 'localhost']: 10.2.3.93 $ [INPUT] Please specify Kaa Flume Source Port: 7060 $ [INPUT] Please specify Target Kaa Flume Sinks Count: 1 $ [INPUT] Please specify Target Kaa Flume Sink 1 Host Name: 10.2.3.93 $ [INPUT] Please specify Target Kaa Flume Sink 1 Port: 7070
To install and configure the Kaa flume sink agent, perform the following steps.
Enter the following command in the shell.
sudo kaa-flume install sink
Follow the installation instructions. You will be prompted to enter basic information. Enter the values shown in the following example after the colon (":") character.
$ [INPUT] Please specify Kaa Sink Host Name: 10.2.3.93 $ [INPUT] Please specify Kaa Sink Port: 7070 $ [INPUT] Please specify Name node [host:port] or nameservice name: 10.2.3.93:8020 $ [INPUT] Please specify HDFS root path [default: 'logs']: {press enter to use default location} $ [INPUT] Please specify Avro schema source type [rest|local] [default: 'rest']: {press enter to use Kaa REST API as Avro schema source for log events}
Enter information necessary to locate the schema source.
This information depends on the Avro schema source type that you have specified in the previous step.
If you have specified the "rest" schema source type, enter the following:
$ [INPUT] Please specify Kaa Admin Rest API host: 10.2.3.93 $ [INPUT] Please specify Kaa Admin Rest API port [default: '8080']: 8080 $ [INPUT] Please specify Kaa Admin User: devuser {user should have 'Tenant developer' or 'Tenant user' authority} $ [INPUT] Please specify Kaa Admin Password: devuser123
If you have specified the "local" schema source type, enter the following:
$ [INPUT] Please specify Absolute path to local schema files: {specify absolute path to directory with schema files}
Please see the following additional information.
Schema files should be stored using the following pattern:
{path to local schema files}/{application token}/schema_v{log schema version}
where:
{application token} - the application token for logs collection. It can be obtained from Kaa admin web UI.
{log schema version} - the version of the log schema within the application. It can be obtained from Kaa admin web UI.
File schema_v{log schema version} should contain an avro log schema for the specified version in the text format.
Now that you have installed and configured the Kaa flume agents, you are ready to start them.
sudo service kaa-flume-source-{source instance name} start
sudo service kaa-flume-sink start
To validate the installation of Kaa flume agents, see the logs located at /var/log/flume-ng/. If the agents were installed and configured correctly, the logs should not contain any exceptions or errors.
Use the following guides and references to make the most of Kaa.
Guide | What it is for |
---|---|
Administration UI guide | Use this guide to start working with the Kaa web UI. |
Design reference | Use this reference to learn about features and capabilities of Kaa. |
Programming guide | Use this guide to create your own Kaa applications. |
Copyright © 2014, CyberVision, Inc.