I found this question online as an example from a technical interview and it seems to be a flawed question in many ways. It made me curious how I would answer it.  So, If you were on a technical Python interview and asked to do the following:Write an algorithm that receives a dictionary, converts it to a GET string, and is optimized for big data.Which option would you consider the best answer? Any other code related comments are welcome.Common:import requestsbase_url = https://api.github.comdata = {'per_page': 10}node = 'users/arctelix/repos'Option 1:My first thought was just answer the question in the simplest form and use pagination to control the size of the data returned.def get_query_str(node, data=None):    # base query    query_str = %s/%s % (base_url, node)    # build query params dict    query_params = &.join([%s=%s % (k,str(v))                              for k, v in data.items()])    if query_params:        query_str += ?%s % query_params    return query_strprint(\n--Option 1--\n)url = get_query_str(node, data)print(url = %s % url)Option 2:Well, that's not really optimized for big data and the requests library will convert a dict to params for me. Secondly, a generator would be a great way to keep memory in check with very large data sets.def get_resource(node, data=None):    url = %s/%s % (base_url, node)    print(geting resource : %s %s % (url, data))    resp = requests.get(url, params=data)    json = resp.json()    yield jsonprint(\n--Option 2--\n)results = get_resource(node, data)for r in results:    print(r)Option 3:Just in case the interviewer was really looking to see if I knew how join() and a list comprehension could be used to convert a dictionary to a string of query parameters. Let's put it all together and use a generator for not only the pages, but the objects as well. get_query_str is totally unnecessary, but again the task was to write something that returned a GET string..class Github:    base_url = https://api.github.com    def get_query_str(self, node, data=None):        # base query        query_str = %s/%s % (self.base_url, node)        # build query params dict        query_params = &.join([%s=%s % (k,str(v))                                  for k, v in data.items()])        if query_params:            query_str += ?%s % query_params        return query_str    def get(self, node, data=None):        data = data or {}        data['per_page'] = data.get('per_page', 50)        page = range(0,data['per_page'])        p=0        while len(page) == data['per_page']:            data['page'] = p            query = self.get_query_str(node, data)            page = list(self.req_resource(query))            p += 1            yield page    def req_resource(self, query):        print(geting resource : %s % query)        r = requests.get(query)        j = r.json()        yield jgh = Github()pages = gh.get(node, data)print(\n--Option 3--\n)for page in pages:    for repo in page:        print(repo=%s % repo)
Algorithm that receives a dictionary, converts it to a GET string, and is optimized for big data
python;interview questions;comparative review
There are a bunch of things that are not said or rendered implicit by the question so Im going to assume that the optimized for big data part is about the GitHub API response. So Id go with the third version. But first, some general advices:Document your code. Docstrings are missing all around your code. You should describe what each part of your API is doing or no-one will make the effort to figure it out and use it.Don't use %, sprintf-like formatting. These are things of the past and have been superseeded by the str.format function. You may also want to try and push newest features such as formatted string litterals (or f-strings) of Python 3.6: query_str = f'{self.base_url}/{node}'.You should use a generator expression rather than a list-comprehension in your '&'.joins as you will discard the list anyway. It will save you some memory management. Just remove the brakets and youre good to go.You shouldn't use f{k}={v} for k, v in data.items(): what if a key or a value contains a '&' or an '='? You should encode the values in your dictionnary before joining them. urllib.parse.urlencode (which is called by requests for you) is your friend.Now about handling the response:page = list(self.req_resource(query)) defeats the very purpose of having a generator in the first place. Consider using yield from self.req_resource(query) instead.Pagination of the Github API should be handled using the Link header instead of manually incrementing the page number. Use the request's headers dictionnary on your response to easily get them.Consider using the threading module to fetch the next page of data while you are processing the current one.