edu.umn.cs.spatialHadoop.operations

Class DistributedCopy

java.lang.Object

edu.umn.cs.spatialHadoop.operations.DistributedCopy

public class DistributedCopy
extends java.lang.Object

Copy file using a distributed MapReduce job. Unlike the shell command 'distcp' which creates a map task for each file, this version creates a map task for each block which allows one file to be copied in parallel using multiple machines.

Author:

Ahmed Eldawy

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DistributedCopy.BlockInputFormat An input format that returns blocks of binary data as values.
static class	DistributedCopy.BlockOutputCommitter Concatenates output files to generate an exact copy of the input dir.
static class	DistributedCopy.BlockOutputFormat
static class	DistributedCopy.BlockRecordReader Reads blocks of binary arrays from input file.
static class	DistributedCopy.FileBlockIndex Holds information about a block in a file with a specific index.
static class	DistributedCopy.FileBlockSplit An extended version of FileSplit that also stores the position of the block in the file.

Constructor Summary

Constructors

Constructor and Description
DistributedCopy()

Method Summary

Methods

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DistributedCopy

public DistributedCopy()

Method Detail

main

public static void main(java.lang.String[] args)
                 throws java.io.IOException

Parameters:

args -

Throws:

java.io.IOException