private ManagedBuffer getSortBasedShuffleBlockData ( ExecutorShuffleInfo executor , int shuffleId , int mapId , int reduceId ) { File indexFile = getFile ( executor . localDirs , executor . subDirsPerLocalDir , "shuffle_" + shuffleId + "_" + mapId + "_0.index" ) ; try { ShuffleIndexInformation shuffleIndexInformation = shuffleIndexCache . get ( indexFile ) ; ShuffleIndexRecord shuffleIndexRecord = shuffleIndexInformation . getIndex ( reduceId ) ; return new FileSegmentManagedBuffer ( conf , getFile ( executor . localDirs , executor . subDirsPerLocalDir , "shuffle_" + shuffleId + "_" + mapId + "_0.data" ) , shuffleIndexRecord . getOffset ( ) , shuffleIndexRecord . getLength ( ) ) ; } catch ( ExecutionException e ) { throw new RuntimeException ( "Failed to open file: " + indexFile , e ) ; } }
Sort - based shuffle data uses an index called shuffle_ShuffleId_MapId_0 . index into a data file called shuffle_ShuffleId_MapId_0 . data . This logic is from IndexShuffleBlockResolver and the block id format is from ShuffleDataBlockId and ShuffleIndexBlockId .