<jupyter_start><jupyter_text>![images.jfif](attachment:images.jfif)  The numpy.random package  The numpy.random package *I would just like to begin by apologizing for the lack of color or enhances formatting in the following project. I put quite a bit of time into colorising and centering the above title, only to find out (after a myriad of attemted fixes) that Github will not display colored text in Jupyter, nor does it support formatting, such as centering text etc. As a result the following paper is slightly less aesthetically pleasing than I would have liked. If you would like a sample of how it could have looked, I have included the original code for the title in the README file, if you care to run it on another notebook as markdown text.  What is the numpy.random package?           - Numpy.random is a library of functions that can be used to generate arrays of pseudo-random numbers under a variety of different key parameters.    - These parameters can include specifications of range, division of parts, the amount of numbers generated within each array, the shape and number of arrays generated & variables regarding the probability distribution of the function itself.  Simple Random Data ![random_data_watermark.576078a4008d.jpg](attachment:random_data_watermark.576078a4008d.jpg)  Integers ![Negative-Integers-1.jpg](attachment:Negative-Integers-1.jpg)  parameters:    - low : int or array-like of ints     - high : int or array-like of ints, optional    - size : int or tuple of ints, optional    - dtype : dtype, optional    - endpoint : bool, optional     returns    - out : int or ndarray of ints     First we import the required libraries:<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>And then we create a shorthand value name for 'np.random.default_rng()', so that we don't have to type that in every time.<jupyter_code>rng = np.random.default_rng()<jupyter_output><empty_output><jupyter_text>high    - For some reason, the first value to be set in the function is the high value (Even though above 'low' is mentioned first). If there is only one value entered as below, the default lowest point in the range is 0, with the entered value being the highest (exclusive). This means that the numbers generated will be integers between 0 and (x-1), with x being the number entered. This is due to the manner in which computers deal with integers and tally values starting from 0 rather than 1.<jupyter_code>rng.integers(7)<jupyter_output><empty_output><jupyter_text>low    - if a second value is entered, the order of values becomes as such: (low, high). Here we can see that the lowest range is now changed from the default '0' to the first value in the brackets.<jupyter_code>rng.integers(-15, 7)<jupyter_output><empty_output><jupyter_text>Size    - Next we can specify the size of the array. This in itself comes with it's own range of parameters that allow us to determine the nature of     the output in further detail.        -In this next piece of code we can see that we have the same first two parameters, but have now generated an array of 10 outcomes.<jupyter_code>rng.integers(-15, 7, 10)<jupyter_output><empty_output><jupyter_text>- You can specify size without setting low value by using 'size='. You will be required to set at least one positional arguement before this will work, however.<jupyter_code>rng.integers(5, size=10)<jupyter_output><empty_output><jupyter_text>-Another value recieves an error message.        *(I have included an image of this outcome instead of including the actual code, as otherwise it disrupts the comtinuity of the 'Restart and Run All' action. ![image.png](attachment:image.png)     -However if we now enclose the size parameter in brackets, we can see      that we can now create 10 arrays, each containing 2 outcomes.     - (low, high, (No. of Arrays, No. of outcomes in each array))<jupyter_code>rng.integers(-15, 7, (10, 2))<jupyter_output><empty_output><jupyter_text>- A third value adds another dimension to the arrays.<jupyter_code>m = rng.integers(-15, 7, (10, 2, 7))
m<jupyter_output><empty_output><jupyter_text>- And as you can see, adding more values to the size parameter just          keeps adding dimensions to the way the arrays are generated.<jupyter_code>rng.integers(-15, 7, (10, 2, 7, 4))<jupyter_output><empty_output><jupyter_text>Square Brackets:    - These allow you to give several sepperate values for a given               parameter, and to generate an outcome for each.        - Here we can see we have set three different high values, and have         generated an outcome for each:<jupyter_code>rng.integers(1, [7, 10, 2])<jupyter_output><empty_output><jupyter_text>- Or we can do the same for the low parameter:<jupyter_code>rng.integers([0, 50, 95], 100)<jupyter_output><empty_output><jupyter_text>Default distribution:    - The default distribution of these simple data functions is essentially the uniform distribution. We can test this by running the function through a large data set and visualizing the distribution through a histogram.<jupyter_code>a = rng.integers(1000, size=10000000)
plt.hist(a)<jupyter_output><empty_output><jupyter_text>- Here we can see the distrinution of probability is uniformly spread out accross the parameters.     - So what if we changed the default distribution?<jupyter_code>b = rng.normal(1000, size=10000000)
b
plt.hist(b)<jupyter_output><empty_output><jupyter_text>- Here we see the probability distribution has been changed. Now instead of uniform distribution, it is more likely for a number to fall closer to 1000 than the outer boundaries of the ranges of the array.  Random ![1_t_G1kZwKv0p2arQCgYG7IQ.gif](attachment:1_t_G1kZwKv0p2arQCgYG7IQ.gif)     - .ramdom generates floating point numbers between the 'half-open interval', which means between [0.0, 1.0].  parameters:    - size : int or tuple of ints, optional    - dtype : dtype, optional    - out : ndarray, optional     returns:    - out : float or ndarray of floats The command on its own will simply generate a single float:<jupyter_code>rng.random()<jupyter_output><empty_output><jupyter_text>If given a value, it will produce that many floats.<jupyter_code>rng.random(5)<jupyter_output><empty_output><jupyter_text>Another value withing an extra set of brackets adds anothher dimension to the output. You may keep adding dimensions.<jupyter_code>rng.random((5, 4))<jupyter_output><empty_output><jupyter_text>You may keep adding dimensions.<jupyter_code>rng.random((5, 4, 3))<jupyter_output><empty_output><jupyter_text>In order to change the boundaries of the range, one can use the following method:x * rng.random() - y(Where 0 to x is the size of the range, and y is the amount you would like to move the starting point of the range by from 0 (ie: x = 30 y=  15 would result in the range being from -15 to 15)) Example:<jupyter_code>30 * rng.random(30) - 15<jupyter_output><empty_output><jupyter_text>Floating point numbers are good for generating more precise outcomes than those produced by the integer rng.  Distribution:<jupyter_code>c = rng.random(10000000)
c
plt.hist(c)<jupyter_output><empty_output><jupyter_text>Again, we can see that the distribution is uniform across the range.  Choice: ![download.jfif](attachment:download.jfif) This rng generates a number or array of numbers from a speciific array/data set that is manually input by the user. For example if you wanted a function that would pick a number between 1 and ten, but you didn't want it to pick any even numbers, you could input the array fom which it may chose a number as: (0,1,3,5,7,9).  Parameters:    - a : {array_like, int}    - size : {int, tuple[int]}, optional    - replace : bool, optional    - p : 1-D array_like, optional    - axis : int, optional    - shuffle : bool, optional     Returns:    - samples : single item or ndarray     If given only one parametre, will generate a random integer based on np.arrange(x), which basically will give a list of integers from 0 to x.  a<jupyter_code>rng.choice(5)<jupyter_output><empty_output><jupyter_text>size Given two parameters, it will generate a given number of integers from the same set.<jupyter_code>rng.choice(5, 3)<jupyter_output><empty_output><jupyter_text>However if you enclose the initial set of parameters in brackets, you can specify the array's elements.<jupyter_code>rng.choice((1,3,5), 3)<jupyter_output><empty_output><jupyter_text>replace The replace option decides whether an element is replaced in the array after it has been chosen initially. For instance:    - An example of 'replace=false' would be if you have a deck of cards, and you pick one card, it would be from a data set of 52 cards. However, if you then go on to pick another card from the same deck, you cannot pick the same card again, as that card has already been removed from the deck, and thus you are picking from a data set of 51 elements, none of which could possibly be the one chosen before.         - An example of 'replace=true' would be picking a card from a deck, but then putting it back, reshuffling the deck, and chosing another card. All 52 cards are possible each time.<jupyter_code>rng.choice(5, 5, replace=False)<jupyter_output><empty_output><jupyter_text>Above we can see each element only appears once. Since the default is replace=true, we do not have to enter it for it to apply.Below you can see the same number can appear repeatedly in the output.<jupyter_code>rng.choice(5, 5)<jupyter_output><empty_output><jupyter_text>P: We may also define the probability of each element of the given array to occur:(If not given, uniform distribution applies)<jupyter_code>e = rng.choice(10, 100, p=[0.1, 0, 0.1, 0.3, 0, 0.1, 0, 0.2, 0.2, 0])
e<jupyter_output><empty_output><jupyter_text>Here shows the probability distribution of 'e' in a histogram.<jupyter_code>plt.hist(e)<jupyter_output><empty_output><jupyter_text>Any array can be given, including words or any other elements:<jupyter_code>d = ['Donald', 'Joeseph', 'Kanye', 'Vladimir']
f = rng.choice(d, 100000, p=[0.1, 0.1, 0.3, 0.5])
plt.hist(f)<jupyter_output><empty_output><jupyter_text>bytes ![yellow-popsicle-bite-blue-background-98702661.jpg](attachment:yellow-popsicle-bite-blue-background-98702661.jpg)  parameters:    - length : int - Number of random bytes    Will randomly generate a number of bytes. The number of bytes generated will be between 0 and the value entered. The bytes themselves will be random.<jupyter_code>rng.bytes(5)<jupyter_output><empty_output><jupyter_text>Generating random bytes is useful for encryption and salting passwords.  Permutations ![download.png](attachment:download.png) [1] Cambridge Dictionary: https://dictionary.cambridge.org/dictionary/english/permutation        "Any of the various ways in which a set of things can be ordered." There are two functions with which one can generate permutations of a given array in numpy:    - Shuffle    - Permutation Both are very similar in terms of outcome, however the manner in which they generate the outcome is different.[2] programmersought.com: https://programmersought.com/article/32541516928/"Differences:    -Shuffle directly operates on the original array, changing the order of the original array, no return value    -Permutation does not directly operate on the original array, but returns a new array of scrambled orders, without changing the original array" Here is a given array:<jupyter_code>arr = np.arange(10)
arr<jupyter_output><empty_output><jupyter_text>Permutation    - Here we can see that the permutation funtion has reshuffled the elements of the array:<jupyter_code>arr = np.arange(10)
rng.permutation(arr)<jupyter_output><empty_output><jupyter_text>- However when we show 'arr' again it remains intact in it's original order.<jupyter_code>arr<jupyter_output><empty_output><jupyter_text>Shuffle    - Here we see a similar outcome regarding the reshuffling:<jupyter_code>rng.shuffle(arr)
arr<jupyter_output><empty_output><jupyter_text>-However when we check for the state of 'arr' we can see that the array (arr) has been changed, and is no longer what it was originally. The original array value no longer exists to be recalled upon.<jupyter_code>arr<jupyter_output><empty_output><jupyter_text>- The implications of losing the original array value may have certain repercussions depending on the nature of the code and the need to refer back to the original array.    -The Shuffle function tends to perform noticeably faster when applied to very large arrays, and may therefore be the prefered choice when speed is a deciding factor. [3] programmersought.com: https://programmersought.com/article/4415574540/![3fea1c0818626a07c2c89bab010713bb.png](attachment:3fea1c0818626a07c2c89bab010713bb.png)  Distributions     - There are 36 different probability distributions in numpy.     - Each serves to emulate different scenarios of probability.     - Here I will discuss a few examples of the different distributions and how they can be used to simulate real life situtions, and thus give us further insight into the nature of specific scenarios.  Types of distribution:    - Certain distributions share features, so we divide them into types.    - Dicrete    - Continuous  Discrete:        - Distributions in which there are a finite number of outcomes.        - A few examples here would be rolling a dice or picking a card.         Continuous:       - Distriutions where there are infinite outcomes.       - Examples here would be recording time and distance in a race.       [4]: 365 Data Science: Probability: Types of distributions:Binomial distribution: https://www.youtube.com/watch?v=b9a27XN_6tg  Other useful terms:    - Empirical Distributions:        "An empirical distribution is one for which each possible event is assigned a probability."        [5] Empirical Distributions: https://www.unf.edu/~cwinton/html/cop4300/s09/class.notes/DiscreteDist.pdf      -Bernoulli event:       "A Bernoulli event is one for which the probability the event occurs is p and the probability the event does not occur is 1-p; i.e., the event is has two possible outcomes (usually viewed as success or failure) occurring with probability p and 1-p, respectively." [5]  Discrete distributions:    - Here we will discuss some of the most commonly used discrete distributions, and how they are used in real life.      Uniform: ![unnamed.gif](attachment:unnamed.gif) [8] Distributions for assigning random values http://resources.esri.com/help/9.3/arcgisengine/java/gp_toolref/process_simulations_sensitivity_analysis_and_error_analysis_modeling/distributions_for_assigning_random_values.htm     - Some events, such as flipping a coin or roling a dice have a set of equally likely outcomes.     - We call these outcomes 'Equiprobable'.    - The distribution of these outomes is thus described as Uniform.    - We saw this distribution earlier in the paper as the default distribution of the simple random data functions.<jupyter_code>f = rng.uniform(-100,100,1000000)
f
plt.hist(f)<jupyter_output><empty_output><jupyter_text>Binomial: ![kiVGG.gif](attachment:kiVGG.gif) [9] Why is a binomial distribution bell-shaped? https://stats.stackexchange.com/questions/176425/why-is-a-binomial-distribution-bell-shaped     - "The Bernoulli distribution represents the success or failure of a single Bernoulli trial"    - ie: Likelihood of coing flip is even distribution = 0.5/0.5, but the likelihood of rolling a six on a die would be 0.17 success/0.83 failure.    - "The Binomial Distribution represents the number of successes and failures in n number of independent Bernoulli trials for some given value of n." [5]    - It is basically a probability of probabilities, ie: the success rate or the likeliehood of a specific set of outcomes occuring vs other possible outcomes.    - It is useful when deling with uncertainty regarding a long run frequency.    [6] 3Blue1Brown: Binomial distributions | Probabilities of probabilities, part 1 https://www.youtube.com/watch?v=8idr1WZ1A7Q     - An example of this would be if a car manufacturer has a 0.13 percent chance of producing a faulty model, the binomial distribution would describe the most likely number of cars that would be faulty if the manufacturer made 10,000 cars. It doesn't guarantee the outcome will be 1300, but it tells you the probability of that outcome. It will also tell you the probability of the number being higher or lower.    - Basically the distibution will run a number of bernoulli trialls and return a success rate for a given outcome.<jupyter_code>n, p = 10000, .13
g = rng.binomial(n, p, 1000000)
g
plt.hist(g)<jupyter_output><empty_output><jupyter_text>[7] Binomial distribution: https://en.wikipedia.org/wiki/Binomial_distribution[8] Binomial Distribution https://mathworld.wolfram.com/BinomialDistribution.html  Poisson: ![%D0%9F%D0%BE%D0%B8%D1%81%D0%BE%D0%BD%D0%BE%D0%B21.png](attachment:%D0%9F%D0%BE%D0%B8%D1%81%D0%BE%D0%BD%D0%BE%D0%B21.png) [9] File:Поисонов1.png https://commons.wikimedia.org/wiki/File:%D0%9F%D0%BE%D0%B8%D1%81%D0%BE%D0%BD%D0%BE%D0%B21.png     - Deals with a number of events occuring in a given timeframe, provided we have an expected outcome.    - Example would be if a bank teller usually gets 4 customers in an hour, what is the distribution of likelihood regarding the number of customers he will get on any given hour?    - It is similar in many ways to the binomial distribution, however it always starts from 0 and has no upper limit to the outcomes. It deals with continuos events rather than discrete events ie: with binomial you have a certain number of attempts, each with a given probability for success, with poisson you essentially have an infinite number of attempts with a given average outcome, with infinitesimal chance for success. [10} Difference between Poisson and Binomial distributions: https://math.stackexchange.com/questions/1050184/difference-between-poisson-and-binomial-distributions<jupyter_code>h = rng.poisson(10, 10000000)
h
plt.hist(h)<jupyter_output><empty_output><jupyter_text>- The rate of events occuring is constant, and each event is ineffected by the subsequent event.    - Can be used to determine likelihood of certain events, such as someone scoring x amount of goals in a match based on their average scores over the season.  Continuous Distributions:     - Here we willl discuss some continuos distributions and their uses in real life.  Normal ![1200px-Normal_Distribution_PDF.svg.png](attachment:1200px-Normal_Distribution_PDF.svg.png) [11] https://en.wikipedia.org/wiki/Normal_distribution     - Also called Gaussian distribuion.    - Often called the 'Bell Curve' distribution due to it's characteristic shape.    - The normal distribution occurs frequently in nature.     - For example, it could describe the size of mature blue whales and the likelihood of finding one of a particular size.<jupyter_code>i, j = 150, 50
k = rng.normal(i, j, 1000000)
k
sns.distplot(k)<jupyter_output><empty_output><jupyter_text>Standard T: ![main-qimg-db91d75fe597b7d466a3dc8d98f25371.png](attachment:main-qimg-db91d75fe597b7d466a3dc8d98f25371.png) [12] https://www.quora.com/In-simple-terms-how-are-normal-z-distribution-and-student-t-distribution-related-How-do-they-differ     - Also known commonly as 'Student's T' distribution.    - Similar in shape and nature to the normal distribution, the Student's T distribution has more instances in the outlier areas of its distribution.    - Used instead of normaal distribution when info is limited/sample size is small.    - It is designed to calculate inferences through small samples with an unkown popuation variance, ie: one can calculate the quality of a larger sample based on a limited sample.    - Like the normal distribution, it has many real life applications, such as calculating .    - "The t test provides a way to test whether the sample mean (that is the mean calculated from the data) is a good estimate of the true mean."    - An example here would be similar to the normal distributin example regarding the average whale size, but in this case it could be based on the past 16 sightings only. Due to the limited data, this is a way of using those 16 sightings to most efficiently represent a much bigger part of the population of whales, given we have a previous mean or projected mean to compare deviation, ie: 10 years ago the average size was 150.    [13] https://numpy.org/doc/stable/reference/random/generated/numpy.random.Generator.standard_t.htmlnumpy.random.Generator.standard_t[14] 365 Data Science: Student's T Distribution: https://www.youtube.com/watch?v=32CuxWdOlow So let's take the above example and apply it. Here we have 15 values, (so our degrees of freedom is that number -1 = 15), We also calculate the mean of the 16 values.<jupyter_code>l = np.array([150, 139, 157, 153, 135, 160, 146, 149,
                   148, 151, 149, 143, 158, 150, 147, 154])
m = rng.standard_t(15, size=100000)
np.mean(l)
t = (np.mean(l)-150)/(l.std(ddof=1)/np.sqrt(len(l)))
h = plt.hist(m, bins=100, density=True)
np.sum(m<t) / float(len(m))
t<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07406182058795728, 'fraction_numerical': 0.028594385102561678, 'mean_word_length': 4.732592793280953, 'pattern_counts': {'":': 0, '<': 154, '<?xml version=': 0, '>': 153, 'https://': 15, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 4, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '38', 'dir': 'jupyter-structured-clean-dedup', 'id': '496215', 'license': 'MIT', 'n_tokens_mistral': 6525, 'n_tokens_neox': 5959, 'n_words': 2603, 'path': 'numpy.random.ipynb', 'repo_name': 'Crash-Bandicode/numpy-random', 'size': '21158'}