diff options
| -rw-r--r-- | ChangeLog | 3 | ||||
| -rw-r--r-- | doc/omhdfs.html | 28 | ||||
| -rw-r--r-- | plugins/omhdfs/omhdfs.c | 2 |
3 files changed, 28 insertions, 5 deletions
@@ -1,4 +1,7 @@ --------------------------------------------------------------------------- +Version 5.7.2 [V5-DEVEL] (rgerhards), 2010-09-?? +- support for Hadoop's HDFS added (via omhdfs) +--------------------------------------------------------------------------- Version 5.7.1 [V5-DEVEL] (rgerhards), 2010-09-?? - imuxsock now optionally use SCM_CREDENTIALS to pull the pid from the log socket itself diff --git a/doc/omhdfs.html b/doc/omhdfs.html index 827697b6..3849f167 100644 --- a/doc/omhdfs.html +++ b/doc/omhdfs.html @@ -12,8 +12,19 @@ file system. <p><b>Configuration Directives</b>:</p> <ul> -<li><b>$...</b><br> -option... +<li><b>$OMHDFSFileName</b> [name]<br> +The name of the file to which the output data shall be written. +</li> +<li><b>$OMHDFSHost</b> [name]<br> +Name or IP address of the HDFS host to connect to. +</li> +<li><b>$OMHDFSPort</b> [name]<br> +Port on which to connect to the HDFS host. +</li> +<li><b>$OMHDFSDefaultTemplate</b> [name]<br> +Default template to be used when none is specified. This saves the work of +specifying the same template ever and ever again. Of course, the default +template can be overwritten via the usual method. </li> </ul> <b>Caveats/Known Bugs:</b> @@ -23,7 +34,8 @@ that HDFS is written in Java and libhdfs uses JNI to talk to it. That requires that various system-specific environment options and pathes be set correctly. At this point, we leave this to the user. If someone know how to do it better, please drop us a line! -<p>In order to build, you need to set these environment variables BEFORE running +<ul> +<li>In order to build, you need to set these environment variables BEFORE running ./configure: <ul> <li>JAVA_INCLUDES - must have all include pathes that are needed to build @@ -32,12 +44,20 @@ JNI C programms, including the -I options necessary for gcc. An example is<br> <li>JAVA_LIBS - must have all library pathes that are needed to build JNI C programms, including the -l/-L options necessary for gcc. An example is<br> # export export JAVA_LIBS="-L/usr/java/jdk1.6.0_21/jre/lib/amd64 -L/usr/java/jdk1.6.0_21/jre/lib/amd64/server -ljava -ljvm -lverify" +</ul> +<li>As of HDFS architecture, you must make sure that all relevant environment +variables (the usual Java stuff and HADOOP's home directory) are properly set. +<li>As it looks, libhdfs makes Java throw exceptions to stdout. There is no +known work-around for this (and it usually should not case any troubles. </ul> <p><b>Sample:</b></p> <p> </p> -<textarea rows="4" cols="80">$ModLoad omucsock +<textarea rows="4" cols="80">$ModLoad omhdfs + +$OMHDFSFileName /var/log/logfile +*.* :omhdfs: </textarea> [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> diff --git a/plugins/omhdfs/omhdfs.c b/plugins/omhdfs/omhdfs.c index 4fbf2ef4..42ed834f 100644 --- a/plugins/omhdfs/omhdfs.c +++ b/plugins/omhdfs/omhdfs.c @@ -1,5 +1,5 @@ /* omhdfs.c - * This is the implementation of the build-in file output module. + * This is an output module to support Hadoop's HDFS. * * NOTE: read comments in module-template.h to understand how this file * works! |