<commit_before>---
layout: post
title: Introducing PoGo  - a GoLang, Twitter favourites to Pocket importer
category: golang, oss
---
I've always kept myself up to date with the latest languages arriving on the scene and I've spent time in the past learning Node and last year I learnt Python by doing the Omnisharp plugin for Sublime.  I have recently been looking for a static language that I can transfer my C# skills too and I had narrowed it down to 3; Swift, Kotlin and GoLang.  I started out with Kotlin and setting up a dev environment with IntelliJ and running the koans that Jetbrains advise you step through to pick up the language.  Whilst it seemed relatively straightforward I got "noob confused" when I saw examples of Java calling into Kotlin with get/set prefixes somehow magically added to Kotlin properties. It turns out the Kotlin compiler does this for Java libraries so it can communicate with it, to me it seemed strange that I code a library in one language and the compiler then exposes these methods and properties slightly differently. Superficial as this sounds I also didnt really like the mammoth that appears to be IntelliJ.  Coming from a predominantly Visual Studio background but working with Omnisharp I wanted a lightweight editor with some refactoring, intellisense and error highlighting.

<!--excerpt-->

I had come across [The Little Go Book][1] a while ago and even went to the extent of getting it printed as a book via [ePubli](https://www.epubli.co.uk) as I'm old school and would like to read something that lengthy on paper not a screen.  Some months passed with it gathering dust but I had made the decision that I wanted to branch out into another langugage and community so I ditched Kotlin and started reading this book.  Its only 50 pages long but due to work and family life this took a week to complete but it also got me questioning my understanding of C#.  

After listening to [<NAME>][2] on a recent podcast I setup my Atom editor with the [Golang plugin][3] and also used the Go website (which has an [online editor][4] which you can use to execute Go code) to test my understanding of how to use Go.

After a week of reading the book and executing the code I thought it was time to try and write a real world application.  

For a while I had been wanting to use [Pocket][5] a tool which helps you save links you want to go back and read at a later date. Currently I use Twitter favourites as a way to save interesting links that I will go back and read but I wondered if a tool like Pocket might be a better.  

I had scowered the internet looking for a tool that would go through your favourites and import them into Pocket.  I found one website that aimed to do that but basically didn't work.  I found [IFTT](https://ifttt.com/) which would automatically move favourites to Pocket but only starting as of now.

### Lets do this

So I was going to write a tool that would import all your favourites into Pocket.  First thing I would need to do is connect to Twitter to get my favourites from my Go app so I fired up Google to see what was out there.  I came across the [Anaconda][6] library which is a Go wrapper for the Twitter API.  This would get my list of favourites but it wasn't going to do the authentication for me.  Google to the rescue again, I came across this [OAuth][7] library which would allow me to authenticate against Twitter with a consumer key and secret and then prompt the user to allow the app to have access to tweets.  By doing this, Twitter would give the user a code that they could then paste into the app and in return it would make a request to Twitter to get a access token. Easy peasy!

Now I needed to communicate with Pocket so more Googling resulted in this [library][8].  Whilst this had the basics to connect to Pocket I had to add extra steps to get authentication working properly.  I also had to read their [documentation][9] about 20 times to understand their workflow.  I think their authentication workflow is aimed at mobile clients and websites authenticating against them rather than console applications.  Two of their steps require passing in redirect urls which is no good if you're a terminal app.  Also their documentation states that if the user authorises the app to access the user's Pocket account it will redirect back to a url, it will also redirect to the same url if the user doesn't allow access. Yeah that makes sense!?!  Rather than having it redirect it would have been helpful if it did what Twitter did and provide a code the user can paste into the terminal but alas that's not an option so I had to fire up a webserver in my app that Pocket would redirect to.  Surprisingly that only took two lines of code!


    http.Handle("/", http.FileServer(http.Dir("./static")))
    http.ListenAndServe(":3000", nil)

Once I had authenticated I had to do a bit more reading regarding the Twitter API, I had to have logic to keep calling the Twitter API to get favourites until I had got them all.  By default you can only return 200 favourites in one go and I had over that stored on Twitter.

I then had to ensure that my array of tweets were oldest first because when I looped over them adding them to Pocket I wanted the most recent favourites first.  Reversing an array is not as easy as it sounds unfortunately and especially when in C# I'm used to `Array.Reverse(array);`  You could write a `for` loop starting from the length of the array rather than zero and populate a new array with the last item first but I didn't want to do that really although it might have been simpler in the long run. Instead of doing that I could have looped over my array in a for loop but I wanted to use a `foreach` loop or in Go thats called `range`.  What I had to do is similar to what you do in C# when doing custom comparisons and that is I had to implement an interface that would implement `Len,Less,Swap` methods.  It was the `Less` method which was what did the magic for us, ie/order the tweets by date.


	func (slice Tweets) Less(i, j int) bool {
		firstTime, err := slice[i].CreatedAtTime()  
		if err != nil {
			fmt.Println("oops")
		}
		secondTime, err2 := slice[j].CreatedAtTime()
		if err2 != nil {
			fmt.Println("oops again!")
		}

		return firstTime.Before(secondTime)
	}<commit_msg>fixed some typos<commit_after>---
layout: post
title: Introducing PoGo  - a GoLang, Twitter favourites to Pocket importer
category: golang, oss
---
I've always kept myself up to date with the latest languages arriving on the scene and I've spent time in the past learning Node and last year I learnt Python by doing the Omnisharp plugin for Sublime.  I have recently been looking for a static language that I can transfer my C# skills too and I had narrowed it down to 3; Swift, Kotlin and GoLang.  I started out with Kotlin and setting up a dev environment with IntelliJ and running the koans that Jetbrains advise you step through to pick up the language.  Whilst it seemed relatively straightforward I got "noob confused" when I saw examples of Java calling into Kotlin with get/set prefixes somehow magically added to Kotlin properties. It turns out the Kotlin compiler does this for Java libraries so it can communicate with it, to me it seemed strange that I code a library in one language and the compiler then exposes these methods and properties slightly differently. Superficial as this sounds I also didn't really like the mammoth that appears to be IntelliJ.  Coming from a predominantly Visual Studio background but working with Omnisharp I wanted a lightweight editor with some refactoring, intellisense and error highlighting.

<!--excerpt-->

I had come across [The Little Go Book][1] a while ago and even went to the extent of getting it printed as a book via [ePubli](https://www.epubli.co.uk) as I'm old school and would like to read something that lengthy on paper not a screen.  Some months passed with it gathering dust but I had made the decision that I wanted to branch out into another language and community so I ditched Kotlin and started reading this book.  Its only 50 pages long but due to work and family life this took a week to complete but it also got me questioning my understanding of C#.  

After listening to [<NAME>][2] on a recent podcast I setup my Atom editor with the [Golang plugin][3] and also used the Go website (which has an [online editor][4] which you can use to execute Go code) to test my understanding of how to use Go.

After a week of reading the book and executing the code I thought it was time to try and write a real world application.  

For a while I had been wanting to use [Pocket][5] a tool which helps you save links you want to go back and read at a later date. Currently I use Twitter favourites as a way to save interesting links that I will go back and read but I wondered if a tool like Pocket might be a better.  

I had scowered the internet looking for a tool that would go through your favourites and import them into Pocket.  I found one website that aimed to do that but basically didn't work.  I found [IFTTT](https://ifttt.com/) which would automatically move favourites to Pocket but only starting as of now.

### Lets do this

So I was going to write a tool that would import all your favourites into Pocket.  First thing I would need to do is connect to Twitter to get my favourites from my Go app so I fired up Google to see what was out there.  I came across the [Anaconda][6] library which is a Go wrapper for the Twitter API.  This would get my list of favourites but it wasn't going to do the authentication for me.  Google to the rescue again, I came across this [OAuth][7] library which would allow me to authenticate against Twitter with a consumer key and secret and then prompt the user to allow the app to have access to tweets.  By doing this, Twitter would give the user a code that they could then paste into the app and in return it would make a request to Twitter to get a access token. Easy peasy!

Now I needed to communicate with Pocket so more Googling resulted in this [library][8].  Whilst this had the basics to connect to Pocket I had to add extra steps to get authentication working properly.  I also had to read their [documentation][9] about 20 times to understand their workflow.  I think their authentication workflow is aimed at mobile clients and websites authenticating against them rather than console applications.  Two of their steps require passing in redirect urls which is no good if you're a terminal app.  Also their documentation states that if the user authorises the app to access the user's Pocket account it will redirect back to a url, it will also redirect to the same url if the user doesn't allow access. Yeah that makes sense!?!  Rather than having it redirect it would have been helpful if it did what Twitter did and provide a code the user can paste into the terminal but alas that's not an option so I had to fire up a webserver in my app that Pocket would redirect to.  Surprisingly that only took two lines of code!


    http.Handle("/", http.FileServer(http.Dir("./static")))
    http.ListenAndServe(":3000", nil)

Once I had authenticated I had to do a bit more reading regarding the Twitter API, I had to have logic to keep calling the Twitter API to get favourites until I had got them all.  By default you can only return 200 favourites in one go and I had over that stored on Twitter.

I then had to ensure that my array of tweets were oldest first because when I looped over them adding them to Pocket I wanted the most recent favourites first.  Reversing an array is not as easy as it sounds unfortunately and especially when in C# I'm used to `Array.Reverse(array);`  You could write a `for` loop starting from the length of the array rather than zero and populate a new array with the last item first but I didn't want to do that really although it might have been simpler in the long run. Instead of doing that I could have looped over my array in a for loop but I wanted to use a `foreach` loop or in Go thats called `range`.  What I had to do is similar to what you do in C# when doing custom comparisons and that is I had to implement an interface that would implement `Len,Less,Swap` methods.  It was the `Less` method which was what did the magic for us, ie/order the tweets by date.


	func (slice Tweets) Less(i, j int) bool {
		firstTime, err := slice[i].CreatedAtTime()  
		if err != nil {
			fmt.Println("oops")
		}
		secondTime, err2 := slice[j].CreatedAtTime()
		if err2 != nil {
			fmt.Println("oops again!")
		}

		return firstTime.Before(secondTime)
	}
{'fraction_non_alphanumeric': 0.03824833702882483, 'fraction_numerical': 0.003642698764649984, 'mean_word_length': 4.017481128327374, 'pattern_counts': {'":': 2, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5155805', 'n_tokens_mistral': 3191, 'n_tokens_neox': 3052, 'n_words': 2188}