edu.umn.cs.spatialHadoop.operations

Class RecordCount

java.lang.Object

edu.umn.cs.spatialHadoop.operations.RecordCount

public class RecordCount
extends java.lang.Object

Calculates number of records in a file depending on its type. If the file is a text file, it counts number of lines. If it's a grid file with no local index, it counts number of non-empty lines. If it's a grid file with RTree index, it counts total number of records stored in all RTrees.

Author:

Ahmed Eldawy

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RecordCount.Map
static class	RecordCount.Reduce

Constructor Summary

Constructors

Constructor and Description
RecordCount()

Method Summary

Methods

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)
static <T> long	recordCountApprox(FileSystem fs, Path file) Counts the approximate number of lines in a file by getting an approximate average line length
static long	recordCountMapReduce(FileSystem fs, Path inFile) Counts the exact number of lines in a file by issuing a MapReduce job that does the thing

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RecordCount

public RecordCount()

Method Detail

recordCountMapReduce

public static long recordCountMapReduce(FileSystem fs,
                        Path inFile)
                                 throws java.io.IOException

Counts the exact number of lines in a file by issuing a MapReduce job that does the thing

Parameters:

conf -

fs -

inFile -

Returns:

Throws:

java.io.IOException

recordCountApprox

public static <T> long recordCountApprox(FileSystem fs,
                         Path file)
                              throws java.io.IOException

Counts the approximate number of lines in a file by getting an approximate average line length

Parameters:

fs -

file -

Returns:

Throws:

java.io.IOException

main

public static void main(java.lang.String[] args)
                 throws java.io.IOException

Parameters:

args -

Throws:

java.io.IOException