<jupyter_start><jupyter_text>Big Data - a Beginners' Lesson Part 1 of 5 In this lesson, you will: *	Learn about and explore big data*	Process and visualize big data Lesson Outline:*	WTH is Big Data?*	The ‘V’s of Big Data*	Show Me the Data!*	Show Me the Magic! (may overflow into the intermediate lesson)*	Show me the $$!*	Show me the Map(sss)!  Thank you for helping our studyContinue with the lessonThroughout this lesson you will see reminders, like the one below, to ensure that all participants understand that they are in a voluntary research study. ReminderBy continuing with this lesson you are granting your permission to take part in this research study for the Hour of Cyberinfrastructure: Developing Cyber Literacy for GIScience project. In this study, you will be learning about cyberinfrastructure and related concepts using a web-based platform that will take approximately one hour per lesson. Participation in this study is voluntary.Participants in this research must be 18 years or older. If you are under the age of 18 then please exit this webpage or navigate to another website such as the Hour of Code at https://hourofcode.com, which is designed for K-12 students.If you are not interested in participating please exit the browser or navigate to this website: http://www.umn.edu. Your participation is voluntary and you are free to stop the lesson at any time.For the full description please navigate to this website: Gateway Lesson Research Study Permission.  WTH is Big Data?This is a “heat” map of something. :)<jupyter_code>import time
from IPython.display import clear_output

widget1 = widgets.RadioButtons(
    options = ['Temperature', 'Road', 'Night light', 'Social media post'],
    description = '<p style="display:inline;"> Guess what does <p style="display:inline; color:#FFA500;">orange</p>/<p style="display:inline;color:#1E90FF;">blue</p>/<p style="display:inline; color:#F0FFFF; background-color:#000000;">white</p> represent?</p>', style={'description_width': 'initial'},
    layout = Layout(width='100%'),
    value = None
)

display(widget1)

# hourofci.SubmitBtn2(widget1)

def SubmitBtn(widget):
    button = widgets.Button(
        description = 'Submit',
        layout=Layout(width='auto', height='auto'),
        disabled = False,
        button_style = '',
        icon = 'check'
    )
    
    display(button)
    output = widgets.Output()
    display(output) 
    
    def submit(b):
        clear_output()
        display(widget)
        display(button)
        display(output)
        print("Great! Move to the next slide to see the answer.")

        def countdown(t):
            while t:
                out.update(t)
                time.sleep(1)
                t -= 1

        out.update(countdown(int(15)))
        out.update(HTML(''' <br/>
            <a id='button' href="#/slide-4-0" class="navigate-right" style="background-color:Green;color:white;padding:8px;margin:2px;font-weight:bold;">Nice try! Continue to see the answer!</a>
        '''))
    
    button.on_click(submit)
    
SubmitBtn(widget1)<jupyter_output><empty_output><jupyter_text>WTH is Big Data?This is a “heat” map of geotagged social media posts.                  *  Orange = flickr; blue = tweet; white = both. → Do you see any spatial pattern(s)?<jupyter_code>class Output:
    def __init__(self, name='countdown'):
        self.h = display(display_id=name)
        self.content = ''
        self.mime_type = None
        self.dic_kind = {
            'text': 'text/plain',
            'markdown': 'text/markdown',
            'html': 'text/html',
        }
        
    def display(self):
        self.h.display({'text/plain': ''}, raw=True)
        
    def _build_obj(self, content, kind, append, new_line):
        self.mime_type = self.dic_kind.get(kind)
        if not self.mime_type:
            return content, False
        if append:
            sep = '\n' if new_line else ''
            self.content = self.content + sep + content
        else:
            self.content = content
        return {self.mime_type: self.content}, True
        
    def update(self, content, kind=None, append=False, new_line=True):
        obj, raw = self._build_obj(content, kind, append, new_line)
        self.h.update(obj, raw=raw)
print('Think about it for a little bit!')
out = Output(name='countdown')
out.display()<jupyter_output><empty_output><jupyter_text>WTH is Big Data?This “heat” map tells us a lot about PEOPLE!!                      What do these pattern(s) tell us?            1.	Where people are    &nbsp;&nbsp;&nbsp;&nbsp; → notice how big cities and transportation network show up            2.	What people share     &nbsp;&nbsp;&nbsp;&nbsp; → flickr vs tweets Social media is an example of BIG DATA.          WTH is Big Data?    Definition:         Datasets that are often characterized as a large volume of complex data produced at an accelerating pace.          The 3Vs of Big Data          The ‘V’s of Big DataVolume - the metric to measure data volume of big data at the scale of astronomical units (e.g. petabytes, exabytes, zettabytes, yottabytes)                                           2022 Figures (Dobrilova 2022)                             FB: 4.2M likes; 211k new photos                IG: 347k browsing; 44k new photos                Twitter: 87.5k new tweets                Tumbler: 37k new posts                Youtube: 4.5M videos watched; 1000 hrs of new videos uploaded                Netflix: 694k hrs of video watched                Texting: ~60M texts sent           How many msgs/posts/videos are there every day/month/year?                       The ‘V’s of Big DataVelocity - The rate at which big data are generated over timeWatch: OpenStreetMap for Haiti 12th Jan 2010<jupyter_code>from IPython.display import YouTubeVideo
YouTubeVideo('e89Tqr75mMw', width=800,height=480)<jupyter_output><empty_output><jupyter_text>The ‘V’s of Big DataVariety - The degree of heterogeneity in how big data are encoded, structured, formatted and represented           The [Other] ‘V’s of Big DataValue - The usefulness of big data in providing unique insights to problem solving and/or decision making    Watch the following video:<jupyter_code>from IPython.display import YouTubeVideo
YouTubeVideo('rwOIQzcXx7Y', width=800,height=480)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09089472020233955, 'fraction_numerical': 0.01533354410369902, 'mean_word_length': 2.949438202247191, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '486385', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1875, 'n_tokens_neox': 1727, 'n_words': 665, 'path': 'beginner-lessons/big-data/bigdata-2.ipynb', 'repo_name': 'mohsen-gis/lessons', 'size': '6326'}