private List < Path > getInputFilesFromDirectory ( FileStatus partitionDir ) throws IOException { FileSystem fs = partitionDir . getPath ( ) . getFileSystem ( conf ) ; FileStatus [ ] inputFiles = fs . listStatus ( partitionDir . getPath ( ) , HiddenFileFilter . INSTANCE ) ; List < Path > input = new ArrayList < Path > ( ) ; for ( FileStatus f : inputFiles ) { input . add ( f . getPath ( ) ) ; } return input ; }
Get all parquet files under partition directory .