<commit_before>#Sneak Peek into the future

Web servers are programs which respond to requests, the format of the request is the HTTP protocol. The Go has a library which helps to save 
the programmer a lot of time and build applications quickly on top of http. A request is called HTTP Request and its corresponding response is called
HTTP Response. Since we don't have to bother about actually implementing the HTTP protocol, all we have to do is send the appropriate HTTP Response for
each HTTP request pattern, along with a HTTP Request, we have a URL and the request type (GET, PUT, POST, DELETE)

The HTTP protocol initially was built only for transferring plain text, but later it started supporting multimedia content too. When we type www.github.com
on our browser's address bar, then it by default adds `https://` or `http://` and a trailing forward slash, so our request becomes HTTP GET / github.com. Our
browser will locate the IP address of the website github.com and sent in a HTTP GET / request, the server of github.com will take the request, go to the view 
handler handling the URL pattern, typically `/` is the default pattern which shows the login page or some non critical stuff until you login.

Until AJAX was invented servers served static html pages, now it is possible to do async communication between the client and the server, which, in plain 
English means that you can change the data on a page without refreshing the page. Which is cool and is expected of any modern website running in the 21'st century.
But of course as with everything in life, you first have to walk before you run, so first static html then AJAX enabled html.

There are two major HTTP methods, `GET` and `POST`, GET is used to basically get the document stored for a URL, suppose GET / will get the home page for 
that website, but when we have to send data to the server, suppose we fill a registration form, then the page sends a POST request to some URL which then
reads the data of the form and does something appropriate with it.

##Example

Suppose we run a blog, the smallest component is a post, a post has tags, it is written by some author, at some time and has some primary key
to uniquely identify it in our database and it has a slug which means the URL.

Earlier URLs used to be ugly, but now they are beautiful, like our fictional blog has this post, `surajblog.com/posts/welcome-the-new-year`,
the slug is the `welcome-the-new-year`.

When the server gets a HTTP GET request of `/posts/welcome-the-new-year`, it'll search for URL handlers starting with the list of URL 
handlers we have given, then it'll find the closest match, in our case it'll be `/post/`, then it'll call the handler of this URL. The library will
go from the top to bottom of the `HandleFunc` listing, hence our `/` root URL should be at the very bottom of our list.

        http.HandleFunc("/post/", ShowPostBySlug)
        http.HandleFunc("/", ShowAllPosts)

What this handler will do is that it'll go to the database, fetch the (id, title, content, time) from the database for that particular post
then it'll read a `template` and populate data into the template, which is ultimately the HTML document which is sent back to our browser.

##What is a template?

While building web applications in any programming language, we have a templating language associated with it, as the name suggests, is a template,
a model. We pass an object to our template and loop through that object to show data in our html page which we'll send over to the client in response to an HTTP request.
We put in variables in the template which are populated as per the data sent while the template is executed. So rather than have an individual page for each
response, we club all pages into types of pages, suppose a profile page, a timeline page and an edit page. We then pass the username and the details of the profile
to the profile page, so one *template* can show the profile information to multiple users. The same goes with every other example.

So writing a web application essentially means, we have to decide upon the URLs, the structure of the database and write functions in Go to handle 
each supported pattern of URL, then write templates to populate each URL, the handlers which we write populate data in the templates, a method generally
known as **rendering**.

###Not abusing templates

Templates support a lot of things like in template variables, but one has to make sure that there is no business logic inside the template and we
separate the business logic from the presentation. Web apps which follow these are easily maintainable since the change in business logic doesn't mean
that we have to change everything, but with one app divided into modules, this triggers a change in that particular part of the module.

Suppose we write a todo list manager which supports multiple users, so we have to ensure that we show tasks of that particular user rather than 
sending all the tasks to the template and filtering there tasks which are of that user on every page. A better approach will surely be to pass the tasks of only that
user while rendering the template, so in the future if we want to show tasks of some user and his friend, then we won't have to change the template, we'll
just modify the function which populates the Tasks object.

Below is such part of such method which is used to handle the backend functionality of our *EditTask* URL which is `/edit/<id>`
located at `~/db/db.go`

        func GetTaskByID(id int) types.Context {
                //Code to populate the tasks variable
                context := types.Context{Tasks: tasks, Navigation: "edit"}
                return context
        }


If you notice, we aren't rendering the template here, because this is a database package whose job is to return the appropriate data to the view,
which will render the template from the data sent by the database function.
file path: `~/views/views.go`

        func EditTaskFunc(w http.ResponseWriter, r *http.Request) {
                //Code
                task := db.GetTaskByID(id)
                editTemplate.Execute(w, task)
                //Code
        }

Thus we can split an application into views, database, templates and the main file. This is the standard way of doing things in this world, not because it is
the status quo, but because it is easier to maintain the web applications that way and since maintainability is the holy grail for software
you are recommended to follow this.

The main file will get the URL pattern and match with the handlers which are present in views, the views get the items from the database and render
the necessary template.

#Serving static files

Static files play a major role in our web application, they are all the CSS/JS/Images which we load into our html pages we serve them over the URL pattern
`/static/`, the way this is implemented is that whenever a request like `/static/<filepath>`, we go to the public directory of our application and then 
look for the path which we got, if we get a file of that path then we serve the file, othewise it automatically sends a 404 error.

The project structure should include a `public` which'll contain all your static files including templates. The internal structure can be anything
you like for you are going to reference the path names in your code.

        public
        |   |-- static
        |   |   |-- css
        |   |   |   `-- styles.css
                        ..and more
        |   |   `-- js
        |   |       |-- bootstrap.min.js
        |   |       .... and more
        |   `-- templates
        |       |-- completed.html
                ...and more        


#####Note:
The above output is of the `tree` program.

#### Links

- Previous section: [General Talk](1.0general_talk.md) 
- Next section: [Basics webapp](2.0implementbasics.md)<commit_msg>updated web programming basics<commit_after># Web Programming Basics

Web servers are programs which respond to requests, the format of the request is the HTTP protocol.  

An HTTP Request from the client (browser/cURL/bots ) gets an HTTP Response from the server. The Go language has the `net/http` library which enables the programmers
to write applications for using HTTP. We use this library to read requests which we receive and respond with HTTP responses which we build by using the same library.

The HTTP protocol initially was built only for transferring plain text, but later it started supporting multimedia content too. When we type www.github.com
on our browser's address bar, then it by default adds `https://` or `http://` and a trailing forward slash, so our request becomes HTTP GET / github.com. Our
browser will locate the IP address of the website github.com and sent in a HTTP GET / request, the server of github.com will take the request, go to the view 
handler handling the URL pattern, typically `/` is the default pattern which shows the login page or some non critical stuff until you login.

Until AJAX was invented servers served static html pages, now it is possible to do async communication between the client and the server, which, in plain 
English means that you can change the data on a page without refreshing the page. Which is cool and is expected of any modern website running in the 21'st century.
But of course as with everything in life, you first have to walk before you run, so first static html then AJAX enabled html.

There are two major HTTP methods, `GET` and `POST`, GET is used to basically get the document stored for a URL, suppose `GET /` will get the home page for 
that website, but when we have to send data to the server, suppose we fill a registration form, then the page sends a POST request to some URL which then
reads the data of the form and does something appropriate with it.

### GET vs POST
 
GET was created to get a resource over HTTP, for instance if you visit any website, then the browser sends a GET request with the URL / to the server.
POST was created to send data from the client to the server.

In GET the data transferred is via the URL in POST the data isn't hidden or encrypted by default, but it isn't sent via the URL, it is easily accessible to anyone who
knows how to read a HTTP request, which isn't rocket science really. Security isn't something you add at the end of the application like a cherry on the cake, it is
something you build your application around. In short there isn't much difference between GET and POST when we consider **security**, both transfer data in plain text
the thing is GET is relatively a little less secure since the URL is logged by a proxy server/firewall/browser history and that GET requests can be done by the browser
on behalf of the user without confirmation. Bots are common over the internet, bots can visit randomly to every link present in your application, but they don't send
random data to any Form you have.

If you want to deploy your application into the wild, you *have* to use HTTPS, but that too can't help if the user is naive and responds to a Man in the middle attack,
where an attacker sets up a similar looking website and tricks the user into revealing their username and password. Long story short, security isn't absolute
but it is upto the developer to have a relatively less insecure website, which means it should check use the HTTP methods correctly, sanitize all data coming from the user
and stop a malicious user in all the normal ways to exploit a web application.

### Example

Suppose we run a blog, the smallest component of which is a post, a post has tags, it is written by some author, at some time and has some primary key
to uniquely identify it in our database and it has a slug which means the URL.

Earlier URLs used to be ugly, but now they are beautiful, like our fictional blog has this post, `surajblog.com/posts/welcome-the-new-year`,
the slug is the `welcome-the-new-year`.

When the server gets a HTTP GET request of `/posts/welcome-the-new-year`, it'll search for URL handlers starting with the list of URL 
handlers we have given, then it'll find the closest match, in our case it'll be `/post/`, then it'll call the handler of this URL. The library will
go from the top to bottom of the `HandleFunc` listing, hence our `/` root URL should be at the very bottom of our list.

        http.HandleFunc("/post/", ShowPostBySlug)
        http.HandleFunc("/", ShowAllPosts)

What this handler will do is that it'll go to the database, fetch the (id, title, content, time) from the database for that particular post
then it'll read a `template` and populate data into the template, which is ultimately the HTML document which is sent back to our browser.

### What is a template?

Templates are a way to present data to the user. When the server responds with an HTTP Response, it has to send an HTML page to the browser. They are a set of
parameterized html pages which has a mechanism to load data.

We put in variables in the template which are populated as per the data sent while the template is executed. So rather than have an individual page for each
response, we club all pages into types, suppose a profile page, a timeline page and an edit page. We then pass the username and the details of the profile
to the profile page, so one *template* can show the profile information to multiple users with different data. The same goes with every other example.

So writing a web application essentially means, we have to decide upon the URLs, the structure of the database and write functions in Go to handle 
each supported pattern of URL, fetch data from the database, write templates to populate data in templates corresponding to each URL, the handlers which we write
populate data in the templates, a method generally
known as **rendering**.

##### Not abusing templates

Templates support a lot of things like template variables, but one has to make sure that there is no business logic inside the template. We
separate the business logic from the presentation logic, since templates are a way to represent data and not a way to manipulate data. Web apps 
which follow these are easily maintainable since the change in business logic doesn't mean that we have to change everything, but with one app 
divided into modules, this triggers a change in that particular part of the module.

Suppose we write a todo list manager which supports multiple users, so we have to ensure that we show tasks of that particular user rather than 
sending all the tasks to the template and filtering there tasks which are of that user on every page. A better approach will surely be to pass the tasks of only that
user while rendering the template, so in the future if we want to show tasks of some user and his friend, then we won't have to change the template, we'll
just modify the function which populates the Tasks object.

Below is such part of such method which is used to handle the backend functionality of our *EditTask* URL which is `/edit/<id>`
located at
 
file `~/db/db.go`

                func GetTaskByID(id int) types.Context {
                        //Code to populate the tasks variable
                        context := types.Context{Tasks: tasks}
                        return context
                }


If you notice, we aren't rendering the template here, because this is a database package whose job is to return the appropriate data to the view,
which will render the template from the data sent by the database function.

file `~/views/views.go`

                func EditTaskFunc(w http.ResponseWriter, r *http.Request) {
                        //Code
                        task := db.GetTaskByID(id)
                        editTemplate.Execute(w, task)
                        //Code
                }

Thus we can split an application into views, database, templates and the main file. This is the standard way of doing things in this world, not because it is
the status quo, but because it is easier to maintain the web applications that way and since maintainability is the holy grail for software
you are recommended to follow this. You should however try *not* using this model once while learning, so you understand the exact merits of this process.

### Static Files

Static files play a major role in our web application, they are all the CSS/JS/Images which we load into our html pages we serve them over the URL pattern
`/static/`, the way this is implemented is that whenever a request like `/static/<filepath>`, we go to the public directory of our application and then 
look for the path which we got, if we get a file of that path then we serve the file, othewise it automatically sends a 404 error.

The project structure should include a `public` which'll contain all your static files including templates. The internal structure can be anything
you like for you are going to reference the path names in your code.

        public
        |   |-- static
        |   |   |-- css
        |   |   |   `-- styles.css
                        ..and more
        |   |   `-- js
        |   |       |-- bootstrap.min.js
        |   |       .... and more
        |   `-- templates
        |       |-- completed.html
                ...and more        

##### Note Output
The above output is of the `tree` program.

If you are to implement the static file serving handler yourself, make sure you sanitize the input, because an attacker might send some malicious request like 
GET ../../ then we must not provide a HTTP response with the listing of the directory two levels up. This is *very* critical, we should always send an HTTP 404 
response to anything but the static files, now one might say how does the attacker know the file name or the folder structure. In network security, an attacker
who has decided he has to get inside *will* get inside, with infinite attempts and infinite time, so we have to strive for making the time taken for breaking into
our servers as big as possible.

#### Links

- Previous section: [Go Programming Basics](1.0general_talk.md) 
- Next section: [Basics webapp](2.0implementbasics.md)
{'fraction_non_alphanumeric': 0.044159858688452194, 'fraction_numerical': 0.00115919629057187, 'mean_word_length': 3.4274193548387095, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5285565', 'n_tokens_mistral': 4365, 'n_tokens_neox': 4170, 'n_words': 2907}