<commit_before>Hadoop YARN Change Log

Release 2.0.5-beta - UNRELEASED

  INCOMPATIBLE CHANGES

    YARN-396. Rationalize AllocateResponse in RM Scheduler API. (Zhijie Shen
    via hitesh)

    YARN-439. Flatten NodeHeartbeatResponse. (Xuan Gong via sseth)

    YARN-440. Flatten RegisterNodeManagerResponse. (Xuan Gong via sseth)

    YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from
    Container which also don't belong to the container. (Xuan Gong via vinodkv)

  NEW FEATURES

  IMPROVEMENTS

    YARN-365. Change NM heartbeat handling to not generate a scheduler event
    on each heartbeat. (Xuan Gong via sseth)

    YARN-380. Fix yarn node -status output to be better readable. (O<NAME>
    Joshi via vinodkv)

    YARN-410. Fixed RM UI so that the new lines diagnostics for a failed app on
    the per-application page are translated to html line breaks. (Omk<NAME>
    Joshi via vinodkv)

    YARN-198. Added a link to RM pages from the NodeManager web app. (Jian He
    via vinodkv)

    YARN-237. Refreshing the RM page forgets how many rows I had in my
    Datatables (jian he via bobby)

    YARN-481. Add AM Host and RPC Port to ApplicationCLI Status Output 
    (Ch<NAME> via bikas)

    YARN-297. Improve hashCode implementations for PB records. (Xuan Gong via
    hitesh)

    YARN-417. Create AMRMClient wrapper that provides asynchronous callbacks.
    (<NAME> via bikas)

    YARN-497. Yarn unmanaged-am launcher jar does not define a main class in
    its manifest (Hitesh Shah via bikas)

    YARN-469. Make scheduling mode in FS pluggable. (kkambatl via tucu)

    YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via
    bikas)

    YARN-475. Remove a unused constant in the public API -
    ApplicationConstants.AM_APP_ATTEMPT_ID_ENV. (Hitesh Shah via vinodkv)

    YARN-309. Changed NodeManager to obtain heart-beat interval from the
    ResourceManager. (Xuan Gong via vinodkv)

    YARN-447. Move ApplicationComparator in CapacityScheduler to use comparator
    in ApplicationId. (Nemon Lou via vinodkv)

    YARN-381. Improve fair scheduler docs. (<NAME> via tomwhite)

    YARN-458. YARN daemon addresses must be placed in many different configs. 
    (sandyr via tucu)

  OPTIMIZATIONS

  BUG FIXES

    YARN-383. AMRMClientImpl should handle null rmClient in stop()
    (Hitesh Shah via sseth)

    YARN-385. Add missing fields - location and #containers to
    ResourceRequestPBImpl's toString(). (<NAME> via sseth)

    YARN-377. Use the new StringUtils methods added by HADOOP-9252 and fix
    TestContainersMonitor.  (<NAME> via szetszwo)

    YARN-391. Formatting fixes for LCEResourceHandler classes.
    (<NAME> via sseth)

    YARN-390. ApplicationCLI and NodeCLI hard-coded platform-specific line
    separator causes test failures on Windows. (<NAME> via suresh)

    YARN-406. Fix TestRackResolver to function in networks where "host1"
    resolves to a valid host. (Hitesh Shah via sseth)

    YARN-376. Fixes a bug which would prevent the NM knowing about completed
    containers and applications. (<NAME> via sseth)

    YARN-196. Nodemanager should be more robust in handling connection failure
    to ResourceManager when a cluster is started (Xuan Gong via hitesh)

    YARN-485. TestProcfsProcessTree#testProcessTree() doesn't wait long enough 
    for the process to die. (kkambatl via tucu)
 
    YARN-470. Support a way to disable resource monitoring on the NodeManager.
    (<NAME> via hitesh)
    
    YARN-71. Fix the NodeManager to clean up local-dirs on restart.
    (Xuan Gong via sseth)

    YARN-378. Fix RM to make the AM max attempts/retries to be configurable
    per application by clients. (Zhijie Shen via vinodkv)

    YARN-498. Unmanaged AM launcher does not set various constants in env for
    an AM, also does not handle failed AMs properly. (Hitesh Shah via bikas)

    YARN-496. Fair scheduler configs are refreshed inconsistently in
    reinitialize. (<NAME> via tomwhite)

    YARN-474. Fix CapacityScheduler to trigger application-activation when
    am-resource-percent configuration is refreshed. (Zhijie Shen via vinodkv)

    YARN-209. Fix CapacityScheduler to trigger application-activation when
    the cluster capacity changes. (Zhijie Shen via vinodkv)

    YARN-24. Nodemanager fails to start if log aggregation enabled and 
    namenode unavailable. (sandyr via tucu)

    YARN-515. Node Manager not getting the master key. (<NAME>
    via jlowe)

    YARN-516. Fix failure in TestContainerLocalizer caused by HADOOP-9357.
    (<NAME> via vinodkv)

    YARN-382. SchedulerUtils improve way normalizeRequest sets the resource
    capabilities. (Zhijie Shen via bikas)

    YARN-467. Modify public distributed cache to localize files such that no
    local directory hits unix file count limits and thus prevent job failures.
    (O<NAME> via vinodkv)

    YARN-101. Fix NodeManager heartbeat processing to not lose track of completed
    containers in case of dropped heartbeats. (Xuan Gong via vinodkv)

    YARN-538. RM address DNS lookup can cause unnecessary slowness on every JHS 
    page load. (sandyr via tucu)

Release 2.0.4-alpha - UNRELEASED

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

  OPTIMIZATIONS

  BUG FIXES

    YARN-429. capacity-scheduler config missing from yarn-test artifact.
    (sseth via hitesh)

Release 2.0.3-alpha - 2013-02-06 

  INCOMPATIBLE CHANGES

  NEW FEATURES

    YARN-145. Add a Web UI to the fair share scheduler. (Sand<NAME> via tomwhite)

    YARN-3. Add support for CPU isolation/monitoring of containers. 
    (adferguson via tucu)

    YARN-230. RM Restart phase 1 - includes support for saving/restarting all
    applications on an RM bounce. (Bikas Saha via acmurthy)

    YARN-103. Add a yarn AM-RM client module. (Bikas Saha via sseth)

    YARN-286. Add a YARN ApplicationClassLoader. (tomwhite)

    YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for
    multi-dimensional resource scheduling. (acmurthy)

    YARN-328. Use token request messages defined in hadoop common. (suresh)

    YARN-231. RM Restart - Add FS-based persistent store implementation for
    RMStateStore (Bikas Saha via hitesh)

  IMPROVEMENTS

    YARN-223. Update process tree instead of getting new process trees.
    (Radim Kolar via llu)

    YARN-57. Allow process-tree based resource calculation et al. to be
    pluggable to support it on multiple platforms. (Radim Kolar via acmurthy)

    YARN-78. Changed UnManagedAM application to use YarnClient. (Bikas Saha via
    vinodkv)

    YARN-93. Fixed RM to propagate diagnostics from applications that have
    finished but failed (<NAME>e via vinodkv). 

    YARN-28. Fixed TestCompositeService to not depend on test-order and thus
    made it pass on JDK7 (Thomas Graves via vinodkv).

    YARN-82. Change the default local and log dirs to be based on
    hadoop.tmp.dir and yarn.log.dir. (<NAME> via sseth)

    YARN-53. Added the missing getGroups API to ResourceManager. (Bo Wang via
    vinodkv)

    YARN-116. Add the ability to change the RM include/exclude file without
    a restart. (xieguiming and Harsh J via sseth)

    YARN-23. FairScheduler: FSQueueSchedulable#updateDemand() - potential 
    redundant aggregation. (kkambatl via tucu)

    YARN-127. Move RMAdmin tool to its correct location - the client module.
    (vinodkv)

    YARN-40. Provided support for missing YARN commands (<NAME> and <NAME> via vinodkv)

    YARN-33. Change LocalDirsHandlerService to validate the configured local and
    log dirs. (<NAME> via sseth)

    YARN-94. Modify DistributedShell to point to main-class by default, clean up
    the help message, and hard-code the AM class. (<NAME> via vinodkv)

    YARN-146. Add unit tests for computing fair share in the fair scheduler.
    (<NAME> via tomwhite)

    HADOOP-8911. CRLF characters in source and text files.
    (<NAME> via suresh)

    YARN-136. Make ClientToAMTokenSecretManager part of RMContext (<NAME>
    Vavilapalli via sseth)

    YARN-183. Clean up fair scheduler code. (Sandy Ryza via tomwhite)

    YARN-129. Simplify classpath construction for mini YARN tests. (tomwhite)

    YARN-254. Update fair scheduler web UI for hierarchical queues. 
    (sandyr via tucu)

    YARN-315. Using the common security token protobuf definition from hadoop
    common. (<NAME> via vinodkv) 

    YARN-170. Change NodeManager stop to be reentrant. (<NAME> via vinodkv)

    YARN-331. Fill in missing fair scheduler documentation. (sandyr via tucu)

    YARN-277. Use AMRMClient in DistributedShell to exemplify the approach.
    (Bikas Saha via hitesh)

    YARN-360. Allow apps to concurrently register tokens for renewal.
    (<NAME> via sseth)

  OPTIMIZATIONS

  BUG FIXES 
    
    YARN-131. Fix incorrect ACL properties in capacity scheduler documentation.
    (<NAME> via sseth)

    YARN-102. Move the apache header to the top of the file in MemStore.java.
    (<NAME> via sseth)
    
    YARN-134. ClientToAMSecretManager creates keys without checking for
    validity of the appID. (<NAME> via sseth)

    YARN-30. Fixed tests verifying web-services to work on JDK7. (<NAME>
    via vinodkv)

    YARN-150. Fixes AppRejectedTransition does not unregister a rejected
    app-attempt from the ApplicationMasterService (Bikas Saha via sseth)

    YARN-140. Add capacity-scheduler-default.xml to provide a default set of
    configurations for the capacity scheduler. (ahmed via tucu)

    YARN-179. Fix some unit test failures. (<NAME> via sseth)

    YARN-181. Fixed eclipse settings broken by capacity-scheduler.xml move via
    YARN-140. (Siddharth Seth via vinodkv)

    YARN-169. Update log4j.appender.EventCounter to use
    org.apache.hadoop.log.metrics.EventCounter (<NAME> via tomwhite)

    YARN-184. Remove unnecessary locking in fair scheduler, and address 
    findbugs excludes. (sandyr via tucu)

    YARN-224. Fair scheduler logs too many nodeUpdate INFO messages.
    (<NAME> via tomwhite)

    YARN-222. Fair scheduler should create queue for each user by default.
    (<NAME> via tomwhite)

    MAPREDUCE-4778. Fair scheduler event log is only written if directory
    exists on HDFS. (<NAME>za via tomwhite)

    YARN-229. Remove old unused RM recovery code. (<NAME> via acmurthy) 

    YARN-187. Add hierarchical queues to the fair scheduler.
    (Sandy Ryza via tomwhite)

    YARN-72. NM should handle cleaning up containers when it shuts down.
    (Sandy Ryza via tomwhite)

    YARN-267. Fix fair scheduler web UI. (Sandy Ryza via tomwhite)

    YARN-264. y.s.rm.DelegationTokenRenewer attempts to renew token even 
    after removing an app. (kkambatl via tucu)

    YARN-271. Fair scheduler hits IllegalStateException trying to reserve
    different apps on same node. (Sandy Ryza via tomwhite)

    YARN-272. Fair scheduler log messages try to print objects without 
    overridden toString methods. (sandyr via tucu)

    YARN-278. Fair scheduler maxRunningApps config causes no apps to make
    progress. (sandyr via tucu)

    YARN-282. Fair scheduler web UI double counts Apps Submitted. 
    (sandyr via tucu)

    YARN-283. Fair scheduler fails to get queue info without root prefix. 
    (sandyr via tucu)

    YARN-192. Node update causes NPE in the fair scheduler.
    (Sandy Ryza via tomwhite)

    YARN-288. Fair scheduler queue doesn't accept any jobs when ACLs are
    configured. (Sandy Ryza via tomwhite)

    YARN-300. After YARN-271, fair scheduler can infinite loop and not
    schedule any application. (Sandy Ryza via tomwhite)

    YARN-301. Fair scheduler throws ConcurrentModificationException when
    iterating over app's priorities. (Sandy Ryza via tomwhite)

    YARN-217. Fix RMAdmin protocol description to make it work in secure mode
    also. (<NAME> via vinodkv)

    YARN-253. Fixed container-launch to not fail when there are no local
    resources to localize. (Tom White via vinodkv)

    YARN-330. Fix flakey test: TestNodeManagerShutdown#testKillContainersOnShutdown.
    (Sand<NAME> via hitesh)
    
    YARN-335. Fair scheduler doesn't check whether rack needs containers
    before assigning to node. (<NAME> via tomwhite)
    
    YARN-336. Fair scheduler FIFO scheduling within a queue only allows 1
    app at a time. (<NAME> via tomwhite)

    YARN-135. Client tokens should be per app-attempt, and should be
    unregistered on App-finish. (vinodkv via sseth)

    YARN-302. Fair scheduler assignmultiple should default to false. (sandyr via tucu)

    YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to
    hadoop-yarn-common (sseth via hitesh)

    YARN-370. Fix SchedulerUtils to correctly round up the resource for
    containers. (<NAME> via acmurthy) 

    YARN-355. Fixes a bug where RM app submission could jam under load.
    (<NAME> via sseth)

Release 2.0.2-alpha - 2012-09-07 

    YARN-9. Rename YARN_HOME to HADOOP_YARN_HOME. (vinodkv via acmurthy)

  NEW FEATURES

    YARN-1. Promote YARN to be a sub-project of Apache Hadoop. (acmurthy)

  IMPROVEMENTS

    YARN-29. Add a yarn-client module. (<NAME> via sseth)

    YARN-10. Fix DistributedShell module to not have a dependency on 
    hadoop-mapreduce-client-core. (<NAME> via vinodkv)

    YARN-80. Add support for delaying rack-local containers in
    CapacityScheduler. (acmurthy) 

    YARN-137. Change the default YARN scheduler to be the CapacityScheduler. 
    (sseth via acmurthy) 

  OPTIMIZATIONS

  BUG FIXES

    YARN-13. Fix pom versions for YARN in branch-2 (todd)

    MAPREDUCE-2374. "Text File Busy" errors launching MR tasks. (<NAME>
    via atm)

    YARN-12. Fix findbugs warnings in FairScheduler. (<NAME> via acmurthy) 

    YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI.
    (<NAME> via sseth)

    YARN-37. Change TestRMAppTransitions to use the DrainDispatcher.
    (<NAME> via sseth)

    YARN-79. Implement close on all clients to YARN so that RPC clients don't
    throw exceptions on shut-down. (<NAME>)

    YARN-42. Modify NM's non-aggregating logs' handler to stop properly so that
    NMs don't get NPEs on startup errors. (<NAME> via vinodkv)

    YARN-15. Updated default classpath for YARN applications to reflect split of
    YARN into a sub-project. (<NAME> via vinodkv)

    YARN-75. Modified ResourceManager's RMContainer to handle a valid RELEASE
    event at RUNNING state. (S<NAME> via vinodkv)

    YARN-138. Ensure default values for minimum/maximum container sizes is
    sane. (harsh & sseth via acmurthy)

Release 0.23.7 - UNRELEASED

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-133 Update web services docs for RM clusterMetrics (<NAME> via
    kihwal)

    YARN-249. Capacity Scheduler web page should show list of active users per 
    queue like it used to (in 1.x) (<NAME> via tgraves)

    YARN-236. RM should point tracking URL to RM web page when app fails to
    start (Jason Lowe via jeagles)

    YARN-269. Resource Manager not logging the health_check_script result when
    taking it out (Jason Lowe via kihwal)

    YARN-227. Application expiration difficult to debug for end-users
    (Jason Lowe via jeagles)

    YARN-443. allow OS scheduling priority of NM to be different than the 
    containers it launches (tgraves)

    YARN-468. coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter
    (<NAME> via bobby)

    YARN-200. yarn log does not output all needed information, and is in a
    binary format (<NAME> via jlowe)

  OPTIMIZATIONS

    YARN-357. App submission should not be synchronized (daryn)

  BUG FIXES

    YARN-343. Capacity Scheduler maximum-capacity value -1 is invalid (Xuan 
    Gong via tgraves)

    YARN-364. AggregatedLogDeletionService can take too long to delete logs
    (jlowe)

    YARN-362. Unexpected extra results when using webUI table search (Ravi
    Prakash via jlowe)

    YARN-400. RM can return null application resource usage report leading to 
    NPE in client (Jason Lowe via tgraves)

    YARN-426. Failure to download a public resource prevents further downloads
    (Jason Lowe via bobby)

    YARN-448. Remove unnecessary hflush from log aggregation (Kihwal Lee via
    bobby)

    YARN-345. Many InvalidStateTransitonException errors for ApplicationImpl
    in Node Manager (<NAME> via jlowe)

    YARN-109. .tmp file is not deleted for localized archives (<NAME> 
    via bobby)

    YARN-460. CS user left in list of active users for the queue even when 
    application finished (tgraves)

Release 0.23.6 - UNRELEASED

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-285. Added a temporary plugin interface for RM to be able to redirect
    to JobHistory server for apps that it no longer tracks. (<NAME> via
    vinodkv)

  OPTIMIZATIONS

  BUG FIXES

    YARN-188. Coverage fixing for CapacityScheduler (<NAME> via
    bobby)

    YARN-214. RMContainerImpl does not handle event EXPIRE at state RUNNING
    (jeagles via bobby)

    YARN-151. Browser thinks RM main page JS is taking too long 
    (<NAME> via bobby)

    YARN-204. test coverage for org.apache.hadoop.tools (<NAME> via
    bobby)

    YARN-251. Proxy URI generation fails for blank tracking URIs (<NAME>
    via jlowe)

    YARN-258. RM web page UI shows Invalid Date for start and finish times
    (<NAME> via jlowe)

    YARN-266. RM and JHS Web UIs are blank because AppsBlock is not escaping
    string properly (<NAME> via jlowe)

    YARN-280. RM does not reject app submission with invalid tokens 
    (<NAME> via tgraves)

    YARN-225. Proxy Link in RM UI thows NPE in Secure mode 
    (De<NAME> via bobby)

    YARN-293. Node Manager leaks LocalizerRunner object for every Container
    (<NAME> via jlowe)

    YARN-50. Implement renewal / cancellation of Delegation Tokens
    (Sidd<NAME> via tgraves)

    YARN-320. RM should always be able to renew its own tokens. 
    (<NAME> via sseth)

    YARN-325. RM CapacityScheduler can deadlock when getQueueInfo() is 
    called and a container is completing (<NAME> via tgraves)

    YARN-334. Maven RAT plugin is not checking all source files (tgraves)

    YARN-354. WebAppProxyServer exits immediately after startup (Liang Xie via
    jlowe)

Release 0.23.5 - 2012-11-28

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-161. Fix multiple compiler warnings for unchecked operations in YARN
    common. (<NAME> via vinodkv)

    YARN-43. Fix TestResourceTrackerService to not depend on test order and thus
    pass on JDK7. (<NAME> via vinodkv)

    YARN-32. Fix TestApplicationTokens to not depend on test order and thus pass
    on JDK7. (vinodkv)

    YARN-186. Coverage fixing LinuxContainerExecutor (<NAME> via
    bobby)

    YARN-216. Remove jquery theming support. (<NAME> via jlowe)

  OPTIMIZATIONS

  BUG FIXES

    YARN-163. Retrieving container log via NM webapp can hang with multibyte
    characters in log (jlowe via bobby)

    YARN-174. Modify NodeManager to pass the user's configuration even when
    rebooting. (vinodkv)

    YARN-177. CapacityScheduler - adding a queue while the RM is running has 
    wacky results (acmurthy vai tgraves)

    YARN-178. Fix custom ProcessTree instance creation (<NAME> via bobby)

    YARN-180. Capacity scheduler - containers that get reserved create 
    container token to early (acmurthy and bobby)

    YARN-139. Interrupted Exception within AsyncDispatcher leads to user
    confusion. (<NAME> via jlowe)

    YARN-165. RM should point tracking URL to RM web page for app when AM fails
    (jlowe via bobby)

    YARN-159. RM web ui applications page should be sorted to display last app
    first (tgraves via bobby)

    YARN-166. capacity scheduler doesn't allow capacity < 1.0 (tgraves via
    bobby)

    YARN-189. Fixed a deadlock between RM's ApplicationMasterService and the
    dispatcher. (<NAME> via vinodkv)

    YARN-202. Log Aggregation generates a storm of fsync() for namenode 
    (Kihwal Lee via bobby)

    YARN-201. Fix CapacityScheduler to be less conservative for starved 
    off-switch requests. (jlowe via acmurthy) 

    YARN-206. TestApplicationCleanup.testContainerCleanup occasionally fails.
    (jlowe via jeagles)

    YARN-212. NM state machine ignores an APPLICATION_CONTAINER_FINISHED event
    when it shouldn't (<NAME> via jlowe)

    YARN-219. NM should aggregate logs when application finishes. (bobby)

Release 0.23.4

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    Change package of YarnClient to org.apache.hadoop. (Bikas Saha via vinodkv)

    YARN-108. FSDownload can create cache directories with the wrong
    permissions (<NAME> via bobby)

    YARN-57. Allow process-tree based resource calculation et al. to be
    pluggable to support it on multiple platforms. (<NAME> via acmurthy)

  OPTIMIZATIONS

  BUG FIXES

    YARN-88. DefaultContainerExecutor can fail to set proper permissions.
    (Jason Lowe via sseth)

    YARN-106. Nodemanager needs to set permissions of local directories (jlowe
    via bobby)

Release 0.23.3

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

  OPTIMAZATIONS

  BUG FIXES

    YARN-14. Symlinks to peer distributed cache files no longer work 
    (Jason Lowe via bobby) 

    YARN-25. remove old aggregated logs  (<NAME> via tgraves)

    YARN-27. Failed refreshQueues due to misconfiguration prevents further 
    refreshing of queues (<NAME> via tgraves)

    MAPREDUCE-4323. NM leaks filesystems (Jason Lowe via jeagles)
    
    YARN-39. RM-NM secret-keys should be randomly generated and rolled every
    so often. (vinodkv and sseth via sseth)

    YARN-31. Fix TestDelegationTokenRenewer to not depend on test order so as to
    pass tests on jdk7. (<NAME> via vinodkv)

    YARN-63. RMNodeImpl is missing valid transitions from the UNHEALTHY state
    (Jason Lowe via bobby)

    YARN-60. Fixed a bug in ResourceManager which causes all NMs to get NPEs and
    thus causes all containers to be rejected. (vinodkv)

    YARN-66. aggregated logs permissions not set properly (tgraves via bobby)

    YARN-68. NodeManager will refuse to shutdown indefinitely due to container
    log aggregation (daryn via bobby)

    YARN-87. NM ResourceLocalizationService does not set permissions of local 
    cache directories (J<NAME> via tgraves)
<commit_msg>Updated CHANGES.txt to reflect YARN-470 being merged into branch-2.0.4-alpha.<commit_after>Hadoop YARN Change Log

Release 2.0.5-beta - UNRELEASED

  INCOMPATIBLE CHANGES

    YARN-396. Rationalize AllocateResponse in RM Scheduler API. (<NAME>
    via hitesh)

    YARN-439. Flatten NodeHeartbeatResponse. (Xuan Gong via sseth)

    YARN-440. Flatten RegisterNodeManagerResponse. (Xuan Gong via sseth)

    YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from
    Container which also don't belong to the container. (<NAME> via vinodkv)

  NEW FEATURES

  IMPROVEMENTS

    YARN-365. Change NM heartbeat handling to not generate a scheduler event
    on each heartbeat. (Xuan Gong via sseth)

    YARN-380. Fix yarn node -status output to be better readable. (Omkar Vinit
    Joshi via vinodkv)

    YARN-410. Fixed RM UI so that the new lines diagnostics for a failed app on
    the per-application page are translated to html line breaks. (Omkar Vinit
    Joshi via vinodkv)

    YARN-198. Added a link to RM pages from the NodeManager web app. (J<NAME>
    via vinodkv)

    YARN-237. Refreshing the RM page forgets how many rows I had in my
    Datatables (jian he via bobby)

    YARN-481. Add AM Host and RPC Port to ApplicationCLI Status Output 
    (<NAME> via bikas)

    YARN-297. Improve hashCode implementations for PB records. (Xuan Gong via
    hitesh)

    YARN-417. Create AMRMClient wrapper that provides asynchronous callbacks.
    (Sand<NAME>za via bikas)

    YARN-497. Yarn unmanaged-am launcher jar does not define a main class in
    its manifest (Hitesh Shah via bikas)

    YARN-469. Make scheduling mode in FS pluggable. (kkambatl via tucu)

    YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via
    bikas)

    YARN-475. Remove a unused constant in the public API -
    ApplicationConstants.AM_APP_ATTEMPT_ID_ENV. (Hitesh Shah via vinodkv)

    YARN-309. Changed NodeManager to obtain heart-beat interval from the
    ResourceManager. (Xuan Gong via vinodkv)

    YARN-447. Move ApplicationComparator in CapacityScheduler to use comparator
    in ApplicationId. (N<NAME> via vinodkv)

    YARN-381. Improve fair scheduler docs. (Sandy Ryza via tomwhite)

    YARN-458. YARN daemon addresses must be placed in many different configs. 
    (sandyr via tucu)

  OPTIMIZATIONS

  BUG FIXES

    YARN-383. AMRMClientImpl should handle null rmClient in stop()
    (Hitesh Shah via sseth)

    YARN-385. Add missing fields - location and #containers to
    ResourceRequestPBImpl's toString(). (Sandy Ryza via sseth)

    YARN-377. Use the new StringUtils methods added by HADOOP-9252 and fix
    TestContainersMonitor.  (<NAME> via szetszwo)

    YARN-391. Formatting fixes for LCEResourceHandler classes.
    (Steve Loughran via sseth)

    YARN-390. ApplicationCLI and NodeCLI hard-coded platform-specific line
    separator causes test failures on Windows. (<NAME> via suresh)

    YARN-406. Fix TestRackResolver to function in networks where "host1"
    resolves to a valid host. (Hitesh Shah via sseth)

    YARN-376. Fixes a bug which would prevent the NM knowing about completed
    containers and applications. (<NAME> via sseth)

    YARN-196. Nodemanager should be more robust in handling connection failure
    to ResourceManager when a cluster is started (Xuan Gong via hitesh)

    YARN-485. TestProcfsProcessTree#testProcessTree() doesn't wait long enough 
    for the process to die. (kkambatl via tucu)
 
    YARN-71. Fix the NodeManager to clean up local-dirs on restart.
    (Xuan Gong via sseth)

    YARN-378. Fix RM to make the AM max attempts/retries to be configurable
    per application by clients. (Zhijie Shen via vinodkv)

    YARN-498. Unmanaged AM launcher does not set various constants in env for
    an AM, also does not handle failed AMs properly. (Hitesh Shah via bikas)

    YARN-496. Fair scheduler configs are refreshed inconsistently in
    reinitialize. (<NAME> via tomwhite)

    YARN-474. Fix CapacityScheduler to trigger application-activation when
    am-resource-percent configuration is refreshed. (Zhijie Shen via vinodkv)

    YARN-209. Fix CapacityScheduler to trigger application-activation when
    the cluster capacity changes. (Zhijie Shen via vinodkv)

    YARN-24. Nodemanager fails to start if log aggregation enabled and 
    namenode unavailable. (sandyr via tucu)

    YARN-515. Node Manager not getting the master key. (<NAME>
    via jlowe)

    YARN-516. Fix failure in TestContainerLocalizer caused by HADOOP-9357.
    (<NAME> via vinodkv)

    YARN-382. SchedulerUtils improve way normalizeRequest sets the resource
    capabilities. (Zhijie Shen via bikas)

    YARN-467. Modify public distributed cache to localize files such that no
    local directory hits unix file count limits and thus prevent job failures.
    (<NAME> via vinodkv)

    YARN-101. Fix NodeManager heartbeat processing to not lose track of completed
    containers in case of dropped heartbeats. (Xuan Gong via vinodkv)

    YARN-538. RM address DNS lookup can cause unnecessary slowness on every JHS 
    page load. (sandyr via tucu)

Release 2.0.4-alpha - UNRELEASED

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

  OPTIMIZATIONS

  BUG FIXES

    YARN-429. capacity-scheduler config missing from yarn-test artifact.
    (sseth via hitesh)

    YARN-470. Support a way to disable resource monitoring on the NodeManager.
    (Siddharth Seth via hitesh)
    
Release 2.0.3-alpha - 2013-02-06 

  INCOMPATIBLE CHANGES

  NEW FEATURES

    YARN-145. Add a Web UI to the fair share scheduler. (<NAME> via tomwhite)

    YARN-3. Add support for CPU isolation/monitoring of containers. 
    (adferguson via tucu)

    YARN-230. RM Restart phase 1 - includes support for saving/restarting all
    applications on an RM bounce. (Bikas Saha via acmurthy)

    YARN-103. Add a yarn AM-RM client module. (Bikas Saha via sseth)

    YARN-286. Add a YARN ApplicationClassLoader. (tomwhite)

    YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for
    multi-dimensional resource scheduling. (acmurthy)

    YARN-328. Use token request messages defined in hadoop common. (suresh)

    YARN-231. RM Restart - Add FS-based persistent store implementation for
    RMStateStore (Bikas Saha via hitesh)

  IMPROVEMENTS

    YARN-223. Update process tree instead of getting new process trees.
    (Radim Kolar via llu)

    YARN-57. Allow process-tree based resource calculation et al. to be
    pluggable to support it on multiple platforms. (Radim Kolar via acmurthy)

    YARN-78. Changed UnManagedAM application to use YarnClient. (B<NAME> via
    vinodkv)

    YARN-93. Fixed RM to propagate diagnostics from applications that have
    finished but failed (<NAME> via vinodkv). 

    YARN-28. Fixed TestCompositeService to not depend on test-order and thus
    made it pass on JDK7 (<NAME> via vinodkv).

    YARN-82. Change the default local and log dirs to be based on
    hadoop.tmp.dir and yarn.log.dir. (H<NAME> via sseth)

    YARN-53. Added the missing getGroups API to ResourceManager. (Bo Wang via
    vinodkv)

    YARN-116. Add the ability to change the RM include/exclude file without
    a restart. (xieguiming and Harsh J via sseth)

    YARN-23. FairScheduler: FSQueueSchedulable#updateDemand() - potential 
    redundant aggregation. (kkambatl via tucu)

    YARN-127. Move RMAdmin tool to its correct location - the client module.
    (vinodkv)

    YARN-40. Provided support for missing YARN commands (<NAME> and <NAME> via vinodkv)

    YARN-33. Change LocalDirsHandlerService to validate the configured local and
    log dirs. (<NAME> via sseth)

    YARN-94. Modify DistributedShell to point to main-class by default, clean up
    the help message, and hard-code the AM class. (H<NAME> via vinodkv)

    YARN-146. Add unit tests for computing fair share in the fair scheduler.
    (<NAME> via tomwhite)

    HADOOP-8911. CRLF characters in source and text files.
    (<NAME> via suresh)

    YARN-136. Make ClientToAMTokenSecretManager part of RMContext (<NAME> via sseth)

    YARN-183. Clean up fair scheduler code. (<NAME> via tomwhite)

    YARN-129. Simplify classpath construction for mini YARN tests. (tomwhite)

    YARN-254. Update fair scheduler web UI for hierarchical queues. 
    (sandyr via tucu)

    YARN-315. Using the common security token protobuf definition from hadoop
    common. (<NAME> via vinodkv) 

    YARN-170. Change NodeManager stop to be reentrant. (<NAME> via vinodkv)

    YARN-331. Fill in missing fair scheduler documentation. (sandyr via tucu)

    YARN-277. Use AMRMClient in DistributedShell to exemplify the approach.
    (Bikas Saha via hitesh)

    YARN-360. Allow apps to concurrently register tokens for renewal.
    (<NAME> via sseth)

  OPTIMIZATIONS

  BUG FIXES 
    
    YARN-131. Fix incorrect ACL properties in capacity scheduler documentation.
    (<NAME> via sseth)

    YARN-102. Move the apache header to the top of the file in MemStore.java.
    (<NAME> via sseth)
    
    YARN-134. ClientToAMSecretManager creates keys without checking for
    validity of the appID. (<NAME> via sseth)

    YARN-30. Fixed tests verifying web-services to work on JDK7. (Thomas Graves
    via vinodkv)

    YARN-150. Fixes AppRejectedTransition does not unregister a rejected
    app-attempt from the ApplicationMasterService (Bikas Saha via sseth)

    YARN-140. Add capacity-scheduler-default.xml to provide a default set of
    configurations for the capacity scheduler. (ahmed via tucu)

    YARN-179. Fix some unit test failures. (<NAME> via sseth)

    YARN-181. Fixed eclipse settings broken by capacity-scheduler.xml move via
    YARN-140. (Siddharth Seth via vinodkv)

    YARN-169. Update log4j.appender.EventCounter to use
    org.apache.hadoop.log.metrics.EventCounter (<NAME> via tomwhite)

    YARN-184. Remove unnecessary locking in fair scheduler, and address 
    findbugs excludes. (sandyr via tucu)

    YARN-224. Fair scheduler logs too many nodeUpdate INFO messages.
    (Sandy Ryza via tomwhite)

    YARN-222. Fair scheduler should create queue for each user by default.
    (Sandy Ryza via tomwhite)

    MAPREDUCE-4778. Fair scheduler event log is only written if directory
    exists on HDFS. (Sand<NAME>yza via tomwhite)

    YARN-229. Remove old unused RM recovery code. (<NAME> via acmurthy) 

    YARN-187. Add hierarchical queues to the fair scheduler.
    (Sandy Ryza via tomwhite)

    YARN-72. NM should handle cleaning up containers when it shuts down.
    (<NAME> via tomwhite)

    YARN-267. Fix fair scheduler web UI. (<NAME>za via tomwhite)

    YARN-264. y.s.rm.DelegationTokenRenewer attempts to renew token even 
    after removing an app. (kkambatl via tucu)

    YARN-271. Fair scheduler hits IllegalStateException trying to reserve
    different apps on same node. (<NAME> via tomwhite)

    YARN-272. Fair scheduler log messages try to print objects without 
    overridden toString methods. (sandyr via tucu)

    YARN-278. Fair scheduler maxRunningApps config causes no apps to make
    progress. (sandyr via tucu)

    YARN-282. Fair scheduler web UI double counts Apps Submitted. 
    (sandyr via tucu)

    YARN-283. Fair scheduler fails to get queue info without root prefix. 
    (sandyr via tucu)

    YARN-192. Node update causes NPE in the fair scheduler.
    (<NAME> via tomwhite)

    YARN-288. Fair scheduler queue doesn't accept any jobs when ACLs are
    configured. (<NAME>za via tomwhite)

    YARN-300. After YARN-271, fair scheduler can infinite loop and not
    schedule any application. (<NAME> via tomwhite)

    YARN-301. Fair scheduler throws ConcurrentModificationException when
    iterating over app's priorities. (<NAME> via tomwhite)

    YARN-217. Fix RMAdmin protocol description to make it work in secure mode
    also. (<NAME> via vinodkv)

    YARN-253. Fixed container-launch to not fail when there are no local
    resources to localize. (<NAME> via vinodkv)

    YARN-330. Fix flakey test: TestNodeManagerShutdown#testKillContainersOnShutdown.
    (<NAME> via hitesh)
    
    YARN-335. Fair scheduler doesn't check whether rack needs containers
    before assigning to node. (<NAME> via tomwhite)
    
    YARN-336. Fair scheduler FIFO scheduling within a queue only allows 1
    app at a time. (<NAME> via tomwhite)

    YARN-135. Client tokens should be per app-attempt, and should be
    unregistered on App-finish. (vinodkv via sseth)

    YARN-302. Fair scheduler assignmultiple should default to false. (sandyr via tucu)

    YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to
    hadoop-yarn-common (sseth via hitesh)

    YARN-370. Fix SchedulerUtils to correctly round up the resource for
    containers. (<NAME> via acmurthy) 

    YARN-355. Fixes a bug where RM app submission could jam under load.
    (<NAME> via sseth)

Release 2.0.2-alpha - 2012-09-07 

    YARN-9. Rename YARN_HOME to HADOOP_YARN_HOME. (vinodkv via acmurthy)

  NEW FEATURES

    YARN-1. Promote YARN to be a sub-project of Apache Hadoop. (acmurthy)

  IMPROVEMENTS

    YARN-29. Add a yarn-client module. (<NAME> via sseth)

    YARN-10. Fix DistributedShell module to not have a dependency on 
    hadoop-mapreduce-client-core. (<NAME> via vinodkv)

    YARN-80. Add support for delaying rack-local containers in
    CapacityScheduler. (acmurthy) 

    YARN-137. Change the default YARN scheduler to be the CapacityScheduler. 
    (sseth via acmurthy) 

  OPTIMIZATIONS

  BUG FIXES

    YARN-13. Fix pom versions for YARN in branch-2 (todd)

    MAPREDUCE-2374. "Text File Busy" errors launching MR tasks. (<NAME>
    via atm)

    YARN-12. Fix findbugs warnings in FairScheduler. (Junping Du via acmurthy) 

    YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI.
    (<NAME> via sseth)

    YARN-37. Change TestRMAppTransitions to use the DrainDispatcher.
    (<NAME> via sseth)

    YARN-79. Implement close on all clients to YARN so that RPC clients don't
    throw exceptions on shut-down. (<NAME>)

    YARN-42. Modify NM's non-aggregating logs' handler to stop properly so that
    NMs don't get NPEs on startup errors. (<NAME> via vinodkv)

    YARN-15. Updated default classpath for YARN applications to reflect split of
    YARN into a sub-project. (<NAME> via vinodkv)

    YARN-75. Modified ResourceManager's RMContainer to handle a valid RELEASE
    event at RUNNING state. (Siddharth Seth via vinodkv)

    YARN-138. Ensure default values for minimum/maximum container sizes is
    sane. (harsh & sseth via acmurthy)

Release 0.23.7 - UNRELEASED

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-133 Update web services docs for RM clusterMetrics (<NAME> via
    kihwal)

    YARN-249. Capacity Scheduler web page should show list of active users per 
    queue like it used to (in 1.x) (<NAME> via tgraves)

    YARN-236. RM should point tracking URL to RM web page when app fails to
    start (Jason Lowe via jeagles)

    YARN-269. Resource Manager not logging the health_check_script result when
    taking it out (Jason Lowe via kihwal)

    YARN-227. Application expiration difficult to debug for end-users
    (Jason Lowe via jeagles)

    YARN-443. allow OS scheduling priority of NM to be different than the 
    containers it launches (tgraves)

    YARN-468. coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter
    (<NAME> via bobby)

    YARN-200. yarn log does not output all needed information, and is in a
    binary format (Ravi Prakash via jlowe)

  OPTIMIZATIONS

    YARN-357. App submission should not be synchronized (daryn)

  BUG FIXES

    YARN-343. Capacity Scheduler maximum-capacity value -1 is invalid (Xuan 
    Gong via tgraves)

    YARN-364. AggregatedLogDeletionService can take too long to delete logs
    (jlowe)

    YARN-362. Unexpected extra results when using webUI table search (Ravi
    Prakash via jlowe)

    YARN-400. RM can return null application resource usage report leading to 
    NPE in client (Jason Lowe via tgraves)

    YARN-426. Failure to download a public resource prevents further downloads
    (Jason Lowe via bobby)

    YARN-448. Remove unnecessary hflush from log aggregation (Kihwal Lee via
    bobby)

    YARN-345. Many InvalidStateTransitonException errors for ApplicationImpl
    in Node Manager (<NAME> via jlowe)

    YARN-109. .tmp file is not deleted for localized archives (<NAME> 
    via bobby)

    YARN-460. CS user left in list of active users for the queue even when 
    application finished (tgraves)

Release 0.23.6 - UNRELEASED

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-285. Added a temporary plugin interface for RM to be able to redirect
    to JobHistory server for apps that it no longer tracks. (<NAME> via
    vinodkv)

  OPTIMIZATIONS

  BUG FIXES

    YARN-188. Coverage fixing for CapacityScheduler (<NAME> via
    bobby)

    YARN-214. RMContainerImpl does not handle event EXPIRE at state RUNNING
    (jeagles via bobby)

    YARN-151. Browser thinks RM main page JS is taking too long 
    (<NAME> via bobby)

    YARN-204. test coverage for org.apache.hadoop.tools (<NAME> via
    bobby)

    YARN-251. Proxy URI generation fails for blank tracking URIs (<NAME>
    via jlowe)

    YARN-258. RM web page UI shows Invalid Date for start and finish times
    (<NAME> via jlowe)

    YARN-266. RM and JHS Web UIs are blank because AppsBlock is not escaping
    string properly (<NAME> via jlowe)

    YARN-280. RM does not reject app submission with invalid tokens 
    (<NAME> via tgraves)

    YARN-225. Proxy Link in RM UI thows NPE in Secure mode 
    (Devaraj K via bobby)

    YARN-293. Node Manager leaks LocalizerRunner object for every Container
    (<NAME> via jlowe)

    YARN-50. Implement renewal / cancellation of Delegation Tokens
    (Siddharth Seth via tgraves)

    YARN-320. RM should always be able to renew its own tokens. 
    (<NAME> via sseth)

    YARN-325. RM CapacityScheduler can deadlock when getQueueInfo() is 
    called and a container is completing (<NAME> via tgraves)

    YARN-334. Maven RAT plugin is not checking all source files (tgraves)

    YARN-354. WebAppProxyServer exits immediately after startup (Liang Xie via
    jlowe)

Release 0.23.5 - 2012-11-28

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-161. Fix multiple compiler warnings for unchecked operations in YARN
    common. (<NAME> via vinodkv)

    YARN-43. Fix TestResourceTrackerService to not depend on test order and thus
    pass on JDK7. (<NAME> via vinodkv)

    YARN-32. Fix TestApplicationTokens to not depend on test order and thus pass
    on JDK7. (vinodkv)

    YARN-186. Coverage fixing LinuxContainerExecutor (<NAME> via
    bobby)

    YARN-216. Remove jquery theming support. (<NAME> via jlowe)

  OPTIMIZATIONS

  BUG FIXES

    YARN-163. Retrieving container log via NM webapp can hang with multibyte
    characters in log (jlowe via bobby)

    YARN-174. Modify NodeManager to pass the user's configuration even when
    rebooting. (vinodkv)

    YARN-177. CapacityScheduler - adding a queue while the RM is running has 
    wacky results (acmurthy vai tgraves)

    YARN-178. Fix custom ProcessTree instance creation (Radim Kolar via bobby)

    YARN-180. Capacity scheduler - containers that get reserved create 
    container token to early (acmurthy and bobby)

    YARN-139. Interrupted Exception within AsyncDispatcher leads to user
    confusion. (<NAME> via jlowe)

    YARN-165. RM should point tracking URL to RM web page for app when AM fails
    (jlowe via bobby)

    YARN-159. RM web ui applications page should be sorted to display last app
    first (tgraves via bobby)

    YARN-166. capacity scheduler doesn't allow capacity < 1.0 (tgraves via
    bobby)

    YARN-189. Fixed a deadlock between RM's ApplicationMasterService and the
    dispatcher. (<NAME> via vinodkv)

    YARN-202. Log Aggregation generates a storm of fsync() for namenode 
    (Kihwal Lee via bobby)

    YARN-201. Fix CapacityScheduler to be less conservative for starved 
    off-switch requests. (jlowe via acmurthy) 

    YARN-206. TestApplicationCleanup.testContainerCleanup occasionally fails.
    (jlowe via jeagles)

    YARN-212. NM state machine ignores an APPLICATION_CONTAINER_FINISHED event
    when it shouldn't (<NAME> via jlowe)

    YARN-219. NM should aggregate logs when application finishes. (bobby)

Release 0.23.4

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    Change package of YarnClient to org.apache.hadoop. (B<NAME> via vinodkv)

    YARN-108. FSDownload can create cache directories with the wrong
    permissions (Jason Lowe via bobby)

    YARN-57. Allow process-tree based resource calculation et al. to be
    pluggable to support it on multiple platforms. (Rad<NAME> via acmurthy)

  OPTIMIZATIONS

  BUG FIXES

    YARN-88. DefaultContainerExecutor can fail to set proper permissions.
    (Jason Lowe via sseth)

    YARN-106. Nodemanager needs to set permissions of local directories (jlowe
    via bobby)

Release 0.23.3

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

  OPTIMAZATIONS

  BUG FIXES

    YARN-14. Symlinks to peer distributed cache files no longer work 
    (Jason Lowe via bobby) 

    YARN-25. remove old aggregated logs  (Robert Evans via tgraves)

    YARN-27. Failed refreshQueues due to misconfiguration prevents further 
    refreshing of queues (<NAME> via tgraves)

    MAPREDUCE-4323. NM leaks filesystems (Jason Lowe via jeagles)
    
    YARN-39. RM-NM secret-keys should be randomly generated and rolled every
    so often. (vinodkv and sseth via sseth)

    YARN-31. Fix TestDelegationTokenRenewer to not depend on test order so as to
    pass tests on jdk7. (<NAME> via vinodkv)

    YARN-63. RMNodeImpl is missing valid transitions from the UNHEALTHY state
    (Jason Lowe via bobby)

    YARN-60. Fixed a bug in ResourceManager which causes all NMs to get NPEs and
    thus causes all containers to be rejected. (vinodkv)

    YARN-66. aggregated logs permissions not set properly (tgraves via bobby)

    YARN-68. NodeManager will refuse to shutdown indefinitely due to container
    log aggregation (daryn via bobby)

    YARN-87. NM ResourceLocalizationService does not set permissions of local 
    cache directories (J<NAME> via tgraves)

{'fraction_non_alphanumeric': 0.057834070499932985, 'fraction_numerical': 0.028459098422910244, 'mean_word_length': 3.1267514749262535, 'pattern_counts': {'":': 0, '<': 178, '<?xml version=': 0, '>': 176, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5301311', 'n_tokens_mistral': 15872, 'n_tokens_neox': 13300, 'n_words': 5686}