Hive UDFs – Simple and Generic UDFs

Hive UDFs: These are regular user-defined functions that operate row-wise and output one result for one row, such as most built-in mathematics and string functions. Ex: SELECT LOWER(str) FROM table_name; SELECT CONCAT(column1,column2) AS x FROM table_name; There are 2 ways of writing the UDFs Simple – extend UDF class Generic – extend GenericUDF class In… Read More »

Solr Installation and create new collection

Solr Installation: Step 1: Download Solr fromhere Step 2: Start solr bin/solr start Step 3: Access Solr http://localhost:8983/ Step 4: Create new collection with name ‘collection1’ solr create -c collection1

PIG UDF with testNG test case – concatenate two strings

PIG UDF class package org.puneetha.pig.udf; import java.io.IOException; import org.apache.log4j.Logger; import org.apache.pig.EvalFunc; import org.apache.pig.data.Tuple; /*** * * * @author Puneetha * */ public final class ConcatStrPig extends EvalFunc<String>{ private static final Logger logger = Logger.getLogger(Thread.currentThread().getStackTrace()[0].getClassName()); @Override public String exec(final Tuple input) throws IOException { logger.debug("Tuple=" + input.toString()); String separator = " "; StringBuilder result = new… Read More »

Category: Pig

Hive UDF with testNG test case – concatenate two strings

Hive UDF class package org.puneetha.hive.udf; import org.apache.hadoop.hive.ql.exec.UDF; import org.apache.hadoop.hive.ql.metadata.HiveException; import org.apache.hadoop.hive.ql.udf.UDFType; import org.apache.hadoop.io.Text; import org.apache.log4j.Logger; import org.apache.hadoop.hive.ql.exec.Description; /*** * * * @author Puneetha * */ @Description(name = "udf_concat" , value = "_FUNC_(STRING, STRING) – RETURN_TYPE(STRING)\n" + "Description: Concatenate two strings, separated by spaces" , extended = "Example:\n" + " > SELECT udf_concat('hello','world') FROM src;\n" +… Read More »

owncloud – Introduction

Do you have data that you wish to keep in cloud and to have similar features like dropbox and google drive, but still want to have full control on your private sensitive data. Well, thats where “Owncloud” pitches in. What Owncloud gives us: Acts as a private cloud file storage system Universal File Access Share… Read More »

PIG – Commands

PIG Syntax Highlighting in vim http://www.vim.org/scripts/script.php?script_id=2186

Category: Pig