My GitHub (Trash code I wrote during PhD)
#phdlife #github #researchcode

A brief browse through my public GitHub and musings about my old code.

Link: https//github.com/yk

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hey, what's going on? So I've recently graduated the PhD and during that time, I've written a lot of code, which is mostly garbage, but I thought we go through my GitHub, and I'll show you the most exciting and useless things I've ever written. So if you're on my GitHub, you're going to find a bunch of things including video related materials such as like the clip music video, you can make your own music video right here. You should watch if you haven't. There's the Minecraft neural network I provide you with the Minecraft world. If you haven't watched that video, please do it GPU stat, which is a tracker for GPU machines and sending it to a server and then displaying it. This is what our lab uses for seeing who uses which GPUs, which is you know, fairly useful. I think this is the single most popular thing I've written during my PhD because that's people actually use it. So there is the flatland repository. So flatland is something we did some time ago, and then I was a total slug and completely failed in supervising the project. Let's not talk about this. You also find code for our conference submissions, of course, but then we get into the real stuff. srun is a little tool that you can use. What it does is it simply copies directory to a server via SSH, it then runs a script on that server, and then it copies back a directory called logs. That's pretty easy. And I use that all the time is very good if you have a bunch of code in a folder and the output is a directory called logs, you're good to go. Otherwise, you'll have to change this a bit. Okay, at that point, I had no clue that you could use temp dir to make temporary directories. Oh, God, look at this. So it happened too many times that I didn't do this from the directory where I actually had my code but from the from the home directory. So it's synced my entire home directory to the server. So I just know. See, this counts as UX. No, I'm pretty sure it does. And this right here, this is the crown jewel, right? It is a system that manages my experiments. So in rat, there is a bunch of things in here, there is a worker. And what the worker would do is it would sit on a server, and it would listen to a database for new experiments that it should run. And if so, it will pull the code from a MongoDB. So so that the queue isn't is a is a Redis queue, and we pull code from a MongoDB. And then it would run that code, but it would only do so if the GPU is free. So to change this RQ thing in order to check whether or not the GPU is free, you can see right here, there's a check of whether or not the GPU is already occupied. And if it is occupied, it would just not do the task and put it back into the queue. However, if it is not occupied, it would run. So the neat thing you can do with this thing is if a lab mate of yours is running on a GPU, you just put this worker on the same GPU. And then as soon as their job is done, it's like, boom, you got it. I'm sorry, I'm sorry. But for the most part, it actually prevents you from interfering with other people. You know, that's pretty neat. And your jobs won't fail just because there's already something on the GPU. So the core of this thing is you can run an experiment config, which means you can upload different hyper parameters, and then jobs would be generated according to those hyper parameters. And I even built in a hyper parameter optimizer. So you can give ranges and it would search through them either in grid search or in random sampling. So here we have a search strategy. And I built in so much stuff you can merge experiments. I mean, look at this, this is a this is quite a bit of engineering going into here. It even has a TensorBoard thing. Whenever a job is finished running, the worker would actually put it back into the database. And this command right here will get me all the event files from TensorBoard. And then it would actually label the directories with the names of the hyper parameters. So you actually see directly in the run name, which run has which hyper parameters. This is so freakin useful because usually TensorBoard runs are just like run one run two or the date or some stupid thing. Confirm really? No, I built this in to prevent myself from doing stupid stuff. But I also built like an override flag. You know, like there's delete all. So as I said, this is it probably doesn't work anymore because I know the redis queue dependencies have shifted and so on. Yeah, if you want if you want some inspiration, feel free, feel absolutely free to clone this. I don't want it anymore. When I started systems like weights and biases and so on, they just didn't exist. So I had to run my own. Similarly, y plot is my attempt at writing a plotting library that works with TensorBoard events. And so extracting data from TensorBoard events, this is all so useless right now except this smoothing thing that I got from scipy, which was pretty useful. Then y pack is you can tell my name. I'm very innovative with my names. I think that's just a set of routines that I implemented for working with torch and TensorFlow. Again, this is probably all useless. So there's deepfool. Look at that. Most of this is completely useless now because these things are mostly in the libraries themselves. confprod is what I use. Oh, look at that. This is a part of rat actually, this is what generates the products of configurations. That's why. Yeah, I even wrote a readme. I wrote a readme, a small utility library to generate cross products of experiment configurations. Just look at the unit test. And hopefully it should become clear how it works. Let's I don't think so. I mean, look at that. This is beautiful. Look, you can like spec out something like this. You can see like so there is you want SGD optimization. And these are the different step sizes and you can sample and this seems like a good a good thing. I mean, there are probably 50 libraries today that do that much better than than I ever could. Fountain Oh, fountain was my own data set library, like c410. It would it would download it from a server and it would extract it if it's not there. Yes, this all exists now in torch vision. And for the ML for NLP in hugging face. What a useless thing. This thing right here, I think so in TensorFlow one, if you youngsters remember that it was quite a bit harder to save and restore and do anything like this. So this will be a library that if your checkpoint doesn't quite fit, it would restore whatever is there. And I think it would also if the shapes don't fit, it would do like a random projection to make the shapes fit. And if they don't fit, this you had to implement like a graph operation just to get the restore to work. This is a plug in I wrote for Chrome, because I was annoyed that I couldn't cite an archive article from the article itself. So I wrote a plugin that goes to Google Scholar and scrapes the the Google Scholar bit tech entry in directly to lot to archive it doesn't work anymore. But I think there are other plugins now. These are actually good. This is a continuous compiler. As you can see, it's not very sophisticated. And of course, I did write my own archive scraper, there was still a time when I read all of archive, this is not possible anymore. But I did read all of archive for at least certain lists. So I had had many more than these lists, new papers every morning. And I would just read through the abstracts in the train. And those are repositories from my masters. And so this is the first public repository ever from the pattern recognition class in my bachelor studies. What is here? linear kernel, poly kernel, RBF, this looks like support vector machines, right? Did I implement this? Here's an SVM classifier implemented. Yikes. And this who does that? Who does private methods with a dunder? No, that's reserved. Whoever did this passed me no nonlinear SVM without any sort of automatic back propagation. No, no, stop. Yeah, but this is a this is a support vector machine without without SGD. I think we used to calculate support vector machines with sort of a quadratic programming, I think that we got that from somewhere. In any case, this was my very, very first public commit to GitHub. And it was already a machine learning lecture. So I guess I had this coming for a while. If you are interested in useless repositories, check out my GitHub, I'd be happy to see what your GitHub says look like. So this was more of a nostalgia thing. But I hope you still had a bit of fun. Cheers.
[{"start": 0.0, "end": 6.36, "text": " Hey, what's going on? So I've recently graduated the PhD and during that time, I've written"}, {"start": 6.36, "end": 13.8, "text": " a lot of code, which is mostly garbage, but I thought we go through my GitHub, and I'll"}, {"start": 13.8, "end": 20.3, "text": " show you the most exciting and useless things I've ever written. So if you're on my GitHub,"}, {"start": 20.3, "end": 26.240000000000002, "text": " you're going to find a bunch of things including video related materials such as like the clip"}, {"start": 26.24, "end": 38.36, "text": " music video, you can make your own music video right here. You should watch if you haven't."}, {"start": 38.36, "end": 43.2, "text": " There's the Minecraft neural network I provide you with the Minecraft world. If you haven't"}, {"start": 43.2, "end": 50.68, "text": " watched that video, please do it GPU stat, which is a tracker for GPU machines and sending"}, {"start": 50.68, "end": 56.88, "text": " it to a server and then displaying it. This is what our lab uses for seeing who uses which"}, {"start": 56.88, "end": 63.08, "text": " GPUs, which is you know, fairly useful. I think this is the single most popular thing"}, {"start": 63.08, "end": 69.4, "text": " I've written during my PhD because that's people actually use it. So there is the flatland"}, {"start": 69.4, "end": 77.28, "text": " repository. So flatland is something we did some time ago, and then I was a total slug"}, {"start": 77.28, "end": 84.2, "text": " and completely failed in supervising the project. Let's not talk about this. You also find code"}, {"start": 84.2, "end": 90.64, "text": " for our conference submissions, of course, but then we get into the real stuff. srun"}, {"start": 90.64, "end": 97.24000000000001, "text": " is a little tool that you can use. What it does is it simply copies directory to a server"}, {"start": 97.24000000000001, "end": 103.56, "text": " via SSH, it then runs a script on that server, and then it copies back a directory called"}, {"start": 103.56, "end": 109.88, "text": " logs. That's pretty easy. And I use that all the time is very good if you have a bunch"}, {"start": 109.88, "end": 115.04, "text": " of code in a folder and the output is a directory called logs, you're good to go. Otherwise,"}, {"start": 115.04, "end": 119.28, "text": " you'll have to change this a bit. Okay, at that point, I had no clue that you could use"}, {"start": 119.28, "end": 127.08, "text": " temp dir to make temporary directories. Oh, God, look at this. So it happened too many"}, {"start": 127.08, "end": 133.04, "text": " times that I didn't do this from the directory where I actually had my code but from the"}, {"start": 133.04, "end": 140.2, "text": " from the home directory. So it's synced my entire home directory to the server. So I"}, {"start": 140.2, "end": 150.28, "text": " just know. See, this counts as UX. No, I'm pretty sure it does. And this right here,"}, {"start": 150.28, "end": 158.44, "text": " this is the crown jewel, right? It is a system that manages my experiments. So in rat, there"}, {"start": 158.44, "end": 165.02, "text": " is a bunch of things in here, there is a worker. And what the worker would do is it would sit"}, {"start": 165.02, "end": 171.02, "text": " on a server, and it would listen to a database for new experiments that it should run. And"}, {"start": 171.02, "end": 177.6, "text": " if so, it will pull the code from a MongoDB. So so that the queue isn't is a is a Redis"}, {"start": 177.6, "end": 184.16, "text": " queue, and we pull code from a MongoDB. And then it would run that code, but it would"}, {"start": 184.16, "end": 189.92, "text": " only do so if the GPU is free. So to change this RQ thing in order to check whether or"}, {"start": 189.92, "end": 194.92, "text": " not the GPU is free, you can see right here, there's a check of whether or not the GPU"}, {"start": 194.92, "end": 200.16, "text": " is already occupied. And if it is occupied, it would just not do the task and put it back"}, {"start": 200.16, "end": 204.56, "text": " into the queue. However, if it is not occupied, it would run. So the neat thing you can do"}, {"start": 204.56, "end": 209.74, "text": " with this thing is if a lab mate of yours is running on a GPU, you just put this worker"}, {"start": 209.74, "end": 215.28, "text": " on the same GPU. And then as soon as their job is done, it's like, boom, you got it."}, {"start": 215.28, "end": 222.76000000000002, "text": " I'm sorry, I'm sorry. But for the most part, it actually prevents you from interfering"}, {"start": 222.76000000000002, "end": 227.18, "text": " with other people. You know, that's pretty neat. And your jobs won't fail just because"}, {"start": 227.18, "end": 234.20000000000002, "text": " there's already something on the GPU. So the core of this thing is you can run an experiment"}, {"start": 234.2, "end": 240.2, "text": " config, which means you can upload different hyper parameters, and then jobs would be generated"}, {"start": 240.2, "end": 246.57999999999998, "text": " according to those hyper parameters. And I even built in a hyper parameter optimizer."}, {"start": 246.57999999999998, "end": 251.79999999999998, "text": " So you can give ranges and it would search through them either in grid search or in random"}, {"start": 251.79999999999998, "end": 258.28, "text": " sampling. So here we have a search strategy. And I built in so much stuff you can merge"}, {"start": 258.28, "end": 264.15999999999997, "text": " experiments. I mean, look at this, this is a this is quite a bit of engineering going"}, {"start": 264.15999999999997, "end": 269.08, "text": " into here. It even has a TensorBoard thing. Whenever a job is finished running, the worker"}, {"start": 269.08, "end": 274.55999999999995, "text": " would actually put it back into the database. And this command right here will get me all"}, {"start": 274.55999999999995, "end": 280.67999999999995, "text": " the event files from TensorBoard. And then it would actually label the directories with"}, {"start": 280.67999999999995, "end": 286.71999999999997, "text": " the names of the hyper parameters. So you actually see directly in the run name, which"}, {"start": 286.72, "end": 291.96000000000004, "text": " run has which hyper parameters. This is so freakin useful because usually TensorBoard"}, {"start": 291.96000000000004, "end": 300.76000000000005, "text": " runs are just like run one run two or the date or some stupid thing. Confirm really?"}, {"start": 300.76000000000005, "end": 308.02000000000004, "text": " No, I built this in to prevent myself from doing stupid stuff. But I also built like"}, {"start": 308.02000000000004, "end": 313.48, "text": " an override flag. You know, like there's delete all. So as I said, this is it probably doesn't"}, {"start": 313.48, "end": 318.48, "text": " work anymore because I know the redis queue dependencies have shifted and so on. Yeah,"}, {"start": 318.48, "end": 325.56, "text": " if you want if you want some inspiration, feel free, feel absolutely free to clone this."}, {"start": 325.56, "end": 331.52000000000004, "text": " I don't want it anymore. When I started systems like weights and biases and so on, they just"}, {"start": 331.52000000000004, "end": 339.62, "text": " didn't exist. So I had to run my own. Similarly, y plot is my attempt at writing a plotting"}, {"start": 339.62, "end": 347.88, "text": " library that works with TensorBoard events. And so extracting data from TensorBoard events,"}, {"start": 347.88, "end": 355.76, "text": " this is all so useless right now except this smoothing thing that I got from scipy, which"}, {"start": 355.76, "end": 362.28000000000003, "text": " was pretty useful. Then y pack is you can tell my name. I'm very innovative with my"}, {"start": 362.28, "end": 369.76, "text": " names. I think that's just a set of routines that I implemented for working with torch"}, {"start": 369.76, "end": 376.03999999999996, "text": " and TensorFlow. Again, this is probably all useless. So there's deepfool. Look at that."}, {"start": 376.03999999999996, "end": 382.11999999999995, "text": " Most of this is completely useless now because these things are mostly in the libraries themselves."}, {"start": 382.11999999999995, "end": 387.46, "text": " confprod is what I use. Oh, look at that. This is a part of rat actually, this is what"}, {"start": 387.46, "end": 395.52, "text": " generates the products of configurations. That's why. Yeah, I even wrote a readme. I"}, {"start": 395.52, "end": 402.28, "text": " wrote a readme, a small utility library to generate cross products of experiment configurations."}, {"start": 402.28, "end": 409.12, "text": " Just look at the unit test. And hopefully it should become clear how it works. Let's"}, {"start": 409.12, "end": 415.74, "text": " I don't think so. I mean, look at that. This is beautiful. Look, you can like spec out"}, {"start": 415.74, "end": 421.54, "text": " something like this. You can see like so there is you want SGD optimization. And these are"}, {"start": 421.54, "end": 428.92, "text": " the different step sizes and you can sample and this seems like a good a good thing. I"}, {"start": 428.92, "end": 433.04, "text": " mean, there are probably 50 libraries today that do that much better than than I ever"}, {"start": 433.04, "end": 441.8, "text": " could. Fountain Oh, fountain was my own data set library, like c410. It would it would"}, {"start": 441.8, "end": 448.72, "text": " download it from a server and it would extract it if it's not there. Yes, this all exists"}, {"start": 448.72, "end": 455.38, "text": " now in torch vision. And for the ML for NLP in hugging face. What a useless thing. This"}, {"start": 455.38, "end": 463.6, "text": " thing right here, I think so in TensorFlow one, if you youngsters remember that it was"}, {"start": 463.6, "end": 470.40000000000003, "text": " quite a bit harder to save and restore and do anything like this. So this will be a library"}, {"start": 470.4, "end": 476.23999999999995, "text": " that if your checkpoint doesn't quite fit, it would restore whatever is there. And I"}, {"start": 476.23999999999995, "end": 481.4, "text": " think it would also if the shapes don't fit, it would do like a random projection to make"}, {"start": 481.4, "end": 487.62, "text": " the shapes fit. And if they don't fit, this you had to implement like a graph operation"}, {"start": 487.62, "end": 494.65999999999997, "text": " just to get the restore to work. This is a plug in I wrote for Chrome, because I was"}, {"start": 494.66, "end": 501.20000000000005, "text": " annoyed that I couldn't cite an archive article from the article itself. So I wrote a plugin"}, {"start": 501.20000000000005, "end": 508.0, "text": " that goes to Google Scholar and scrapes the the Google Scholar bit tech entry in directly"}, {"start": 508.0, "end": 513.88, "text": " to lot to archive it doesn't work anymore. But I think there are other plugins now. These"}, {"start": 513.88, "end": 521.6800000000001, "text": " are actually good. This is a continuous compiler. As you can see, it's not very sophisticated."}, {"start": 521.68, "end": 527.4599999999999, "text": " And of course, I did write my own archive scraper, there was still a time when I read"}, {"start": 527.4599999999999, "end": 533.9599999999999, "text": " all of archive, this is not possible anymore. But I did read all of archive for at least"}, {"start": 533.9599999999999, "end": 540.54, "text": " certain lists. So I had had many more than these lists, new papers every morning. And"}, {"start": 540.54, "end": 546.92, "text": " I would just read through the abstracts in the train. And those are repositories from"}, {"start": 546.92, "end": 553.04, "text": " my masters. And so this is the first public repository ever from the pattern recognition"}, {"start": 553.04, "end": 561.0, "text": " class in my bachelor studies. What is here? linear kernel, poly kernel, RBF, this looks"}, {"start": 561.0, "end": 570.4799999999999, "text": " like support vector machines, right? Did I implement this? Here's an SVM classifier implemented."}, {"start": 570.48, "end": 579.32, "text": " Yikes. And this who does that? Who does private methods with a dunder? No, that's reserved."}, {"start": 579.32, "end": 588.48, "text": " Whoever did this passed me no nonlinear SVM without any sort of automatic back propagation."}, {"start": 588.48, "end": 600.8000000000001, "text": " No, no, stop. Yeah, but this is a this is a support vector machine without without SGD."}, {"start": 600.8000000000001, "end": 606.5600000000001, "text": " I think we used to calculate support vector machines with sort of a quadratic programming,"}, {"start": 606.5600000000001, "end": 612.4200000000001, "text": " I think that we got that from somewhere. In any case, this was my very, very first public"}, {"start": 612.42, "end": 620.64, "text": " commit to GitHub. And it was already a machine learning lecture. So I guess I had this coming"}, {"start": 620.64, "end": 628.76, "text": " for a while. If you are interested in useless repositories, check out my GitHub, I'd be"}, {"start": 628.76, "end": 634.4399999999999, "text": " happy to see what your GitHub says look like. So this was more of a nostalgia thing. But"}, {"start": 634.44, "end": 648.36, "text": " I hope you still had a bit of fun. Cheers."}]