synchronized void garbageCollect ( ) { // Cancel task tracker reservation cancelReservedSlots ( ) ; // Remove the remaining speculative tasks counts totalSpeculativeReduceTasks . addAndGet ( - speculativeReduceTasks ) ; totalSpeculativeMapTasks . addAndGet ( - speculativeMapTasks ) ; garbageCollected = true ; // Let the JobTracker know that a job is complete jobtracker . getInstrumentation ( ) . decWaitingMaps ( getJobID ( ) , pendingMaps ( ) ) ; jobtracker . getInstrumentation ( ) . decWaitingReduces ( getJobID ( ) , pendingReduces ( ) ) ; jobtracker . storeCompletedJob ( this ) ; jobtracker . finalizeJob ( this ) ; try { // Definitely remove the local-disk copy of the job file if ( localJobFile != null ) { localFs . delete ( localJobFile , true ) ; localJobFile = null ; } // clean up splits for ( int i = 0 ; i < maps . length ; i ++ ) { maps [ i ] . clearSplit ( ) ; } // JobClient always creates a new directory with job files // so we remove that directory to cleanup // Delete temp dfs dirs created if any, like in case of // speculative exn of reduces. Path tempDir = jobtracker . getSystemDirectoryForJob ( getJobID ( ) ) ; new CleanupQueue ( ) . addToQueue ( new PathDeletionContext ( FileSystem . get ( conf ) , tempDir . toUri ( ) . getPath ( ) ) ) ; } catch ( IOException e ) { LOG . warn ( "Error cleaning up " + profile . getJobID ( ) + ": " + e ) ; } cleanUpMetrics ( ) ; // free up the memory used by the data structures this . nonRunningMapCache = null ; this . runningMapCache = null ; this . nonRunningReduces = null ; this . runningReduces = null ; this . trackerMapStats = null ; this . trackerReduceStats = null ; }
The job is dead . We re now GC ing it getting rid of the job from all tables . Be sure to remove all of this job s tasks from the various tables .