edu.umn.cs.spatialHadoop

Class OperationsParams

java.lang.Object

Configuration

edu.umn.cs.spatialHadoop.OperationsParams

public class OperationsParams
extends Configuration

A class that encapsulates all parameters sent for an operations implemented in SpatialHadoop. Internally, it stores everything in Configuration which makes it easier to pass around to mappers and reducers. It can be initialized from a configuration and/or a list of command line arguments.

Author:

Ahmed Eldawy

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	OperationsParams.Direction Data type for the direction of skyline to compute

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ShapeValueSeparator Separator between shape type and value

Constructor Summary

Constructors

Constructor and Description
OperationsParams()
OperationsParams(Configuration params, java.lang.String... additionalArgs) Initialize the command line arguments from an existing configuration and a list of additional arguments.
OperationsParams(GenericOptionsParser parser)
OperationsParams(GenericOptionsParser parser, boolean autodetectShape)

Method Summary

Methods

Modifier and Type	Method and Description
boolean	autoDetectShape() Auto detect shape type based on input format
boolean	checkInput() Checks that there is at least one input path and that all input paths exist.
boolean	checkInputOutput()
boolean	checkInputOutput(boolean outputRequired) Makes standard checks for input and output files.
boolean	checkOutput()
void	clearAllPaths()
static java.awt.Color	getColor(Configuration conf, java.lang.String key, java.awt.Color defaultValue)
java.awt.Color	getColor(java.lang.String key, java.awt.Color defaultValue)
static OperationsParams.Direction	getDirection(Configuration conf, java.lang.String key, OperationsParams.Direction defaultDirection)
OperationsParams.Direction	getDirection(java.lang.String key, OperationsParams.Direction defaultDirection)
static NASAPoint.GradientType	getGradientType(Configuration conf, java.lang.String key, NASAPoint.GradientType defaultValue)
NASAPoint.GradientType	getGradientType(java.lang.String key, NASAPoint.GradientType defaultValue)
Path	getInputPath()
Path[]	getInputPaths()
Path	getOutputPath()
Path	getPath()
Path[]	getPaths()
static Path	getRepartitionJoinIndexPath(Configuration conf, java.lang.String key)
static Shape	getShape(Configuration job, java.lang.String key)
static Shape	getShape(Configuration conf, java.lang.String key, Shape defaultValue)
Shape	getShape(java.lang.String key)
Shape	getShape(java.lang.String key, Shape defaultValue)
<S extends Shape> S[]	getShapes(java.lang.String key, S stock)
static long	getSize(Configuration conf, java.lang.String key)
long	getSize(java.lang.String key)
long	getSJMRGridPartitioiningFactor(java.lang.String key)
static long	getSJMRPartitioningGrid(Configuration conf, java.lang.String key)
static TextSerializable	getTextSerializable(Configuration conf, java.lang.String key, TextSerializable defaultValue)
void	initialize(java.lang.String... args)
boolean	is(java.lang.String flag)
boolean	is(java.lang.String flag, boolean defaultValue)
static boolean	isLocal(JobConf job, Path... input) Checks whether the operation should work in local or MapReduce mode.
static boolean	isWildcard(Path path)
void	setOutputPath(java.lang.String newPath)
static void	setRepartitionJoinIndexPath(Configuration conf, java.lang.String param, Path indexPath)
static void	setShape(Configuration conf, java.lang.String param, Shape shape) Sets the specified configuration parameter to the current value of the shape.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ShapeValueSeparator

public static final java.lang.String ShapeValueSeparator

Separator between shape type and value

See Also:

Constant Field Values

Constructor Detail

OperationsParams

public OperationsParams()

OperationsParams

public OperationsParams(GenericOptionsParser parser)

OperationsParams

public OperationsParams(GenericOptionsParser parser,
                boolean autodetectShape)

OperationsParams

public OperationsParams(Configuration params,
                java.lang.String... additionalArgs)

Initialize the command line arguments from an existing configuration and a list of additional arguments.

Parameters:

params -

additionalArgs -

Method Detail

initialize

public void initialize(java.lang.String... args)

getPaths

public Path[] getPaths()

getPath

public Path getPath()

getOutputPath

public Path getOutputPath()

setOutputPath

public void setOutputPath(java.lang.String newPath)

getInputPath

public Path getInputPath()

getInputPaths

public Path[] getInputPaths()

is

public boolean is(java.lang.String flag,
         boolean defaultValue)

is

public boolean is(java.lang.String flag)

checkInput

public boolean checkInput()
                   throws java.io.IOException

Checks that there is at least one input path and that all input paths exist. It treats all passed paths as input. This is useful for input-only operations which do not take any output path (e.g., MBR and Aggregate).

Returns:

Throws:

java.io.IOException

checkInputOutput

public boolean checkInputOutput()
                         throws java.io.IOException

Throws:

java.io.IOException

checkInputOutput

public boolean checkInputOutput(boolean outputRequired)
                         throws java.io.IOException

Makes standard checks for input and output files. It is assumes that all files are input files while the last one is the output file. First, it checks that there is at least one input file. Then, it checks that every input file exists. After that, it checks for output file, if it exists and the overwrite flag is not present, it fails.

Returns:

true if all checks pass. false otherwise.

Throws:

java.io.IOException

checkOutput

public boolean checkOutput()
                    throws java.io.IOException

Throws:

java.io.IOException

isWildcard

public static boolean isWildcard(Path path)

getColor

public java.awt.Color getColor(java.lang.String key,
                      java.awt.Color defaultValue)

getColor

public static java.awt.Color getColor(Configuration conf,
                      java.lang.String key,
                      java.awt.Color defaultValue)

getShape

public Shape getShape(java.lang.String key,
             Shape defaultValue)

getShape

public Shape getShape(java.lang.String key)

getShape

public static Shape getShape(Configuration job,
             java.lang.String key)

getShape

public static Shape getShape(Configuration conf,
             java.lang.String key,
             Shape defaultValue)

getTextSerializable

public static TextSerializable getTextSerializable(Configuration conf,
                                   java.lang.String key,
                                   TextSerializable defaultValue)

getShapes

public <S extends Shape> S[] getShapes(java.lang.String key,
                              S stock)

getSize

public long getSize(java.lang.String key)

getSJMRGridPartitioiningFactor

public long getSJMRGridPartitioiningFactor(java.lang.String key)

setShape

public static void setShape(Configuration conf,
            java.lang.String param,
            Shape shape)

Sets the specified configuration parameter to the current value of the shape. Both class name and shape values are encoded in one string and set as the value of the configuration parameter. The shape can be retrieved later using SpatialSite.getShape(Configuration, String).

Parameters:

conf -

param -

shape -

getSize

public static long getSize(Configuration conf,
           java.lang.String key)

getSJMRPartitioningGrid

public static long getSJMRPartitioningGrid(Configuration conf,
                           java.lang.String key)

getRepartitionJoinIndexPath

public static Path getRepartitionJoinIndexPath(Configuration conf,
                               java.lang.String key)

setRepartitionJoinIndexPath

public static void setRepartitionJoinIndexPath(Configuration conf,
                               java.lang.String param,
                               Path indexPath)

getDirection

public OperationsParams.Direction getDirection(java.lang.String key,
                                      OperationsParams.Direction defaultDirection)

getDirection

public static OperationsParams.Direction getDirection(Configuration conf,
                                      java.lang.String key,
                                      OperationsParams.Direction defaultDirection)

getGradientType

public NASAPoint.GradientType getGradientType(java.lang.String key,
                                     NASAPoint.GradientType defaultValue)

getGradientType

public static NASAPoint.GradientType getGradientType(Configuration conf,
                                     java.lang.String key,
                                     NASAPoint.GradientType defaultValue)

autoDetectShape

public boolean autoDetectShape()

Auto detect shape type based on input format

Returns:

true if a shape is already present in the configuration or it is auto detected from input files. false if it was not set and could not be detected.

isLocal

public static boolean isLocal(JobConf job,
              Path... input)

Checks whether the operation should work in local or MapReduce mode. If the job explicitly specifies whether to run in local or MapReduce mode, the specified option is returned. Otherwise, it automatically detects whether to use local or MapReduce based on the input size.

Returns:

true to run in local mode, false to run in MapReduce mode.

clearAllPaths

public void clearAllPaths()