I have data that I need to change before running it through the pandas library in python. It is currently in a format that stores midnight values as 2400 and that should be changed to 0000. The format also does not pad either hour nor minute, and I think I need to do so in order to convert the 2 parameters into a desired zero padded 2400hr (0000-2359) format.Now that's the easy part!The hard part is that each time it rolls over from 2359 to 0000 it should also change the date (which is in 'dayoftheyear' format, which actually makes it easier I assume, i.e.: %j +1, aside from Dec 31).So here is a sample of my data (last 2 columns are non-date values stored) at a day's rollover (it is a csv file, but I am showing it with a single space to delimit for visual clarity):1,2014,361,2340,0,01,2014,361,2341,0,01,2014,361,2342,0,01,2014,361,2343,0,01,2014,361,2344,0,01,2014,361,2345,0,01,2014,361,2346,0,01,2014,361,2347,0,01,2014,361,2348,0,01,2014,361,2349,0,01,2014,361,2350,0,01,2014,361,2351,0,01,2014,361,2352,0,01,2014,361,2353,0,01,2014,361,2354,0,01,2014,361,2355,0,01,2014,361,2356,0,01,2014,361,2357,0,01,2014,361,2358,0,01,2014,361,2359,0,01,2014,361,2400,0,024,2014,361,2400,12.341,2014,365,2359,0,91,2014,365,2400,089.343,31,2015,1,1,234,4561,2015,1,2,090,991,2015,365,2359,0,01,2015,365,2400,xx,xxx1,2016,1,1,0,01,2016,1,2,0,01,2016,1,3,0,0I assume the solution is a bunch of sed/awk nested in a for loop, but i'll leave that up to you code ninjas. Thanks in advance.Ok, here is the same question but extended to include the what if once the new year rolls around. So I assume that the $2 column will get incremented come 365 to 366, and this is obviously not desirable. How do I then extend the same incrementing/formatting to include a roll over come 366 to increment the year by 1?I am going to take a blind stab at it:#!/bin/bashfilename=${1/.dat/_prepped.dat}awk '/^1/{print $0}' $1  |cut -d , -f2,3,4,5,6 |awk 'BEGIN{FS=OFS=,}$3 == 2400 {$2 = $2 + 1; $3 = 0}$2 == 366 {$1 = $1 + 1; $2 = 1}{ $3 = sprintf(%04i, $3) }1' >$filenameI took a stab at integrating everything into a script that i feed the raw data (ex: home.dat) into in order to output the file (ex: home_prepped.dat).Results of above data running through above script:2014,361,2340,0,02014,361,2341,0,02014,361,2342,0,02014,361,2343,0,02014,361,2344,0,02014,361,2345,0,02014,361,2346,0,02014,361,2347,0,02014,361,2348,0,02014,361,2349,0,02014,361,2350,0,02014,361,2351,0,02014,361,2352,0,02014,361,2353,0,02014,361,2354,0,02014,361,2355,0,02014,361,2356,0,02014,361,2357,0,02014,361,2358,0,02014,361,2359,0,02014,362,0000,0,02014,365,2359,0,92015,1,0000,089.343,32015,1,0001,234,4562015,1,0002,090,992015,365,2359,0,02016,1,0000,xx,xxx2016,1,0001,0,02016,1,0002,0,02016,1,0003,0,0
change non-zero padded hour and minute into 24hr time
date;timestamps;csv
awk does all of this by itself. sprintf does the formatting, ordinary patterns and assignments do the rest.$3 == 2400 {$2 = $2 + 1; $3 = 0}{ $3 = sprintf(%04i, $3) }1If you put that into dates.awk and then run your sample data through:$ awk -F, -vOFS=, -f dates.awk < datathen you will get:...2014,344,2359,0,02014,345,0000,0,02014,345,0001,0,0...The first line of the script checks whether the third field is 2400 using an expression pattern and zeros and increments appropriately. The second pads the field to four digits with sprintf. The last ensures the line is printed.You can squash that all into a single line to give a script to awk on the command line, and also put the field separators into the body by prepending {FS=OFS=,}.You can deal with year rollover yourself; you should be able to pattern it on the above easily, but making the effort yourself will do you good.