Sunday, May 18, 2014

Hive Abstract Semantic Analyzer Hook

Hive allows Pre & Post Analyzer hooks to be added to the normal hive plan query generation flow via the AbstractSemanticAnalyzerHook class.

A custom hook needs to extend AbstractSemanticAnalyzerHook & override the preAnalyze or postAnalyze method as necessary.

Simple Sematic Analyzer Hook:
A sematic analyzer hook that logs a message each in the preAnalyze & postAnalyze methods, is shown below:
package org.apache.hadoop.hive.ql.parse;

public class SimpleSemanticPreAnalyzerHook extends AbstractSemanticAnalyzerHook{
static final private Log LOG = LogFactory.getLog(SimpleSemanticPreAnalyzerHook.class.getName());
static final private LogHelper console = new LogHelper(LOG);

@Override
public ASTNode preAnalyze(HiveSemanticAnalyzerHookContext context,
ASTNode ast) throws SemanticException {
console.printInfo("!! SimpleSemanticPreAnalyzerHook preAnalyze called !!");
return super.preAnalyze(context, ast);
}

@Override
public void postAnalyze(HiveSemanticAnalyzerHookContext context,
List<Task<? extends Serializable>> rootTasks)
throws SemanticException {
console.printInfo("!! SimpleSemanticPreAnalyzerHook postAnalyze called !!");
super.postAnalyze(context, rootTasks);
}
}


Configurations for Simple Sematic Analyzer Hook:
  • Compile & Package the SimpleSemanticPreAnalyzerHook class into a jar (e.g. simpleSemantic.jar).
  • Add the jar to Hive classpath via add jar on the hive-cli
    • add jar /path/to/simpleSemantic.jar
Posted by at No comments:
Labels: , , ,
Subscribe to: Posts (Atom)