How to install Brotli on nginx and Ubuntu 18.04?

Question: I installed Brotli on Ubuntu 18.04 with Nginx.
Here is the procedure I followed :
$ sudo apt-add-repository ppa:hda-me/nginx-stable
$ sudo apt update
$ sudo apt install brotli nginx nginx-module-brotli

With the tutorial of this blog :
https://clearleft.com/posts/a-dive-into-serving-brotli-compressed-assets
ubuntu@www-example-com ~ $ nginx -V 2>&1 | tr ' ' '\n' | grep brotli

--add-dynamic-module=debian/extra/ngx_brotli

Here is the error message when I test Nginx :
ubuntu@www-example-com ~ $ sudo nginx -t

nginx: [emerg] "load_module" directive is not allowed here in /etc/nginx/sites-enabled/www-s1biose-com:35
nginx: configuration file /etc/nginx/nginx.conf test failed

My configuration seems correct. What's wrong ?
server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name example.com www.example.com;

    location ^~ /.well-known/acme-challenge/ {
        default_type "text/plain";
        root /var/www/letsencrypt;
    }

    location / {
        return 301 https://www.example.com$request_uri;
    }
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name www.example.com;
    root /var/www/www-example-com/web;
    index index.php;

    ssl_certificate /etc/letsencrypt/live/www.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/www.example.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "no-referrer-when-downgrade" always;
    add_header Content-Security-Policy "default-src * data: 'unsafe-eval' 'unsafe-inline'" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;

    load_module modules/ngx_http_brotli_static_module.so;
    load_module modules/ngx_http_brotli_filter_module.so;

    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css text/xml application/json application/javascript application/xml+rss application/atom+xml image/svg+xml;

    brotli on;
    brotli_comp_level 6;
    brotli_types text/plain text/css text/xml application/json application/javascript application/xml+rss application/atom+xml image/svg+xml;

    expires 1209600s;

    location = /favicon.ico {
        log_not_found off;
        access_log off;
    }

    location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
    }

    location ~* \.(txt|log)$ {
        deny all;
    }

    location ~ \..*/.*\.php$ {
        return 403;
    }

    location ~ ^/sites/.*/private/ {
        return 403;
    }

    location ~ ^/sites/[^/]+/files/.*\.php$ {
        deny all;
    }

    location ~* ^/.well-known/ {
        allow all;
    }

    location ~ (^|/)\. {
        return 403;
    }

    location / {
        try_files $uri /index.php?$query_string;
    }

    location @rewrite {
        rewrite ^/(.*)$ /index.php?q=$1;
    }

    location ~ /vendor/.*\.php$ {
        deny all;
        return 404;
    }

    location ~ '\.php$|^/update.php' {
        expires off;
        fastcgi_split_path_info ^(.+?\.php)(|/.*)$;
        include fastcgi_params;
        fastcgi_param HTTP_PROXY "";
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_param QUERY_STRING $query_string;
        fastcgi_intercept_errors on;
        fastcgi_pass unix:/var/run/php/php7.3-fpm.sock;
    }

    location ~ ^/sites/.*/files/styles/ {
        try_files $uri @rewrite;
    }

    location ~ ^(/[a-z\-]+)?/system/files/ {
        try_files $uri /index.php?$query_string;
    }

    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg)$ {
        try_files $uri @rewrite;
        expires max;
        log_not_found off;
    }
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name example.com;

    ssl_certificate /etc/letsencrypt/live/www.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/www.example.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "no-referrer-when-downgrade" always;
    add_header Content-Security-Policy "default-src * data: 'unsafe-eval' 'unsafe-inline'" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;

    location / {
        return 301 https://www.example.com$request_uri;
    }
}

Here is the list of installed packages :
ubuntu@www-example-com ~ $ dpkg -l |grep nginx

ii  libnginx-mod-http-geoip         1.14.0-0ubuntu1.2                                        amd64        GeoIP HTTP module for Nginx
ii  libnginx-mod-http-image-filter  1.14.0-0ubuntu1.2                                        amd64        HTTP image filter module for Nginx
ii  libnginx-mod-http-xslt-filter   1.14.0-0ubuntu1.2                                        amd64        XSLT Transformation module for Nginx
ii  libnginx-mod-mail               1.14.0-0ubuntu1.2                                        amd64        Mail module for Nginx
ii  libnginx-mod-stream             1.14.0-0ubuntu1.2                                        amd64        Stream module for Nginx
ii  nginx                           1.15.8-1-ppa7~bionic                                     amd64        high performance web server
ii  nginx-common                    1.14.0-0ubuntu1.2                                        all          small, powerful, scalable web/proxy server - common files
ii  nginx-core                      1.14.0-0ubuntu1.2                                        amd64        nginx web/proxy server (standard version)
ii  nginx-module-brotli             1.15.8-1-ppa7~bionic                                     amd64        Brotli Module
ii  python-certbot-nginx            0.28.0-1+ubuntu18.04.1+certbot+3                         all          transitional dummy package
ii  python3-certbot-nginx           0.28.0-1+ubuntu18.04.1+certbot+3                         all          Nginx plugin for Certbot

Answer: The load_module directive can only appear at the top level of the nginx configuration. It cannot be inside any block.
Most likely your existing distro-provided nginx configuration already has something at the top level that loads modules from a preconfigured directory. Consider taking advantage of that.
Answer: On your NGINX main conf file (e.g. /etc/nginx/nginx.conf) put the load_module directives outside the http block, usually on the top-most section of the file, i.e.:
...

load_module modules/ngx_http_brotli_static_module.so;
load_module modules/ngx_http_brotli_filter_module.so;

http {
  ...
}

For configs on how to properly enable the module, here's a how-to on building Brotli from source and on adding the module to an already-running NGINX server:
https://www.majlovesreg.one/adding-brotli-to-a-built-nginx-instance


Know a term to get the parent in Views

Question: Is there a way to use a taxonomy term name to get all the parent terms in Views?
If the taxonomy terms are the following, is there a way to get "one" and "two," given "three"?

one

two

three
Answer: taxonomy_get_parents_all will do what you're looking for, but I don't believe it's possible in views.
http://api.drupal.org/api/drupal/modules--taxonomy--taxonomy.module/function/taxonomy_get_parents_all/6
That being said, there's a module that might help you with that. It's called Taxonomy Views Integrator http://drupal.org/project/tvi

Know a term to get the parent in Views


Balanced sampling for network training?

Question: I have a very unbalanced sample set, e.g. 99% true and 1% false.
Is it reasonable to select a balanced subset with a 50/50-distribution for neural network training? The reason for this is, that I guess training on the original data set may induce a bias on the true-samples.
Can you suggest me some literature that covers this topic especially for neural networks?
Answer: Yes, it is reasonable to select a balanced dataset, however if you do your model will probably over-predict the minority class in operation (or on the test set).  This is easily overcome by using a threshold probability that is not 0.5.  The best way to choose the new threshold is to optimise on a validation sample that has the same class frequencies as encountered in operation (or in the test set).
Rather than re-sample the data, a better thing to do would be to give different weights to the positive and negative examples in the training criterion.  This has the advantage that you use all of the available training data.  The reason that a class imbalance leads to difficulties is not the imbalance per se.  It is more that you just don't have enough examples from the minority class to adequately represent its underlying distribution.  Therefore if you resample rather than re-weight, you are solving the problem by making the distribution of the majority class badly represented as well.
Some may advise simply using a different threshold rather than reweighting or resampling.  The problem with that approach is that with ANN the hidden layer units are optimised to minimise the training criterion, but the training criterion (e.g. sum-of-squares or cross-entropy) depends on how the behaviour of the model away from the decision boundary rather than only near the decision boundary.  As as result hidden layer units may be assigned to tasks that reduce the value of the training criterion, but do not help in accurate classification.  Using re-weighted training patterns helps here as it tends to focus attention more on the decision boundary, and so the allocation of hidden layer resources may be better.
For references, a google scholar search for "Nitesh Chawla" would be a good start, he has done a fair amount of very solid work on this.
Comment: Thank you very much for the answer. What can i do, if i have higly screwed numerical(!) data? Does weighting work there, too?
Comment: If you have very skewed responses you might try some transformation such as predicting the logarithm of the response variable.  Often this has useful features such as converting an assumption of absolute errors (additive noise) to relative errors (multiplicative noise), which is very handy in some applications, e.g. https://zslpublications.onlinelibrary.wiley.com/doi/full/10.1111/j.1469-7998.2009.00665.x

Balanced sampling for network training?


How to implement the two categories of the same content type to automatically generate a new aggregation page?

Question: example: there are two taxonomy fields of each article, a taxonomy is regional, another is the attribute,I want to achieve the "regional + attribute = regional attribute" automatically generated such a new aggregation page.
Answer: Create a new view, go to admin/structure/views/add
Show: Content of type: All
✅ Create a page
Display Format: Unformatted List of: Fields
Save and Edit
Now in your view, Click on Advanced (located in the right side).
RELATIONSHIPS click add
Select ✅ Taxonomy terms on node
For Vocabularies checkmark your vocabs and then checkmark ✅ require relationship.
Then in FILTER CRITERIA add Has taxonomy term
✅ Autocomplete, then type regional, attribute.
Comment: Thank you very much for your answer, but did not realize, this is to show the current classification of all the articles below, did not realize the automatic filtering of the two conditions.


How to convert $r = \cos\theta + \sin\theta$ to rectangular form?

Question: We are studying polar and rectangular equations in math. The last question on the homework was:

Write $r = \cos\theta + \sin\theta$ in rectangular form.

I have tried a few ways with no luck. I would love it if someone could tell me if this is possible, and explain the steps to convert it from polar to rectangular form. Thank you!
One example of what I have tried:

$r = \cos\theta + \sin\theta$
$r\cdot r = r\cos\theta + r\sin\theta$

Since $r^2 = x^2 + y^2$, $r\cos\theta = x$, and $r\sin\theta = y$,

$x^2 + y^2 = x + y$

Step 3 is not equivalent to step 1 however, according to desmos here when zoomed in (a lot).
With further insight from the comments section, I have learned Desmos can become inaccurate when zoomed in very (very) closely, which is what caused me to believe my answer was incorrect.
Comment: When I visit your Desmos link and zoom in, the two graphs seem identical to me.  What makes you say they are different?
Comment: It looks the same for me, but even if it looks wrong, desmos is very good at generating graphs, **but** it is still not perfect, above certain values and close enough to 0 there are going to be errors, if Iirc desmos will be accurate enough till around $10^{-4}$ and then start to have a bit errors
Comment: ah I see, desmos is inaccurate when zoomed in a lot. thank you
Answer: What you tried is perfect, the two equations are almost equivalent, see e.g. the answer I gave here. In your case
$$r^2=r\cos\theta+r\sin\theta\Longleftrightarrow\cases{r=\cos\theta+\sin\theta\\\text{or}\\r=0}$$
So the only caveat is what happens when $r=0$: the value of $\theta$ is undefined at the origin, so it does not seem clear whether or not the equation $r=\cos\theta+\sin\theta$ is satisfied.
However, you can look at it this way: the polar curve of equation $r={\color{grey}{(\cos\theta+\sin\theta)}}$ is the set of all points $({\color{grey}{(\cos\theta+\sin\theta)}}\cos\theta,{\color{grey}{(\cos\theta+\sin\theta)}}\sin\theta)$ when $\theta$ runs over $[0,2\pi]$, and this includes the point $\theta=\frac{3\pi}{4}$ where... $r=0$.
From this point of view, one can argue that the equation $r=\cos\theta+\sin\theta$ is satisfied when $r=0$ as well.
Answer: You have $$ x^2+y^2 = x +y $$ which is $$ (x-1/2)^2+(y-1/2)^2=1/2 $$
The graph is a circle  with center at $( 1/2, 1/2)$ and radius $ \sqrt 2 /2$.
The graph of $ r=\cos \theta +\sin \theta $ is the same circle and multiplying by $r$ does not add any new point to the graph.
Note that $r=0$ is a point of $r= \cos \theta +\sin \theta $ at $\theta = 3\pi/4$
Answer: $$r= \cos \theta +\sin \theta $$
Just mutiply both sides by $r$
$$r^2=r \cos \theta + r \sin \theta $$
and recognize the terms
$$ x^2+y^2 = x +y $$
which is a cicle in its standard form going through the center:
$$ (x-\frac12)^2+(y-\frac12)^2=\frac12= \left(\frac{1}{\sqrt 2 } \right)^2  $$
which is same as the first equation with same graphs.


NSFetchedResultsController attempt to insert row 1 into section 0, but there are only 0 rows in section 0 after the update with userInfo (null)

Question: i use NSFetchedResultsController with UITableViewController.
i successfully add new ocject to core data in separated view,
ParseStarterProjectAppDelegate *appDelegate = [[UIApplication sharedApplication] delegate];
NSManagedObjectContext *context = [appDelegate managedObjectContext];
NSManagedObject *newWorkout;
newWorkout = [NSEntityDescription insertNewObjectForEntityForName:@"Workouts" inManagedObjectContext:context];
[newWorkout setValue:_workOutType forKey:@"type"];
[newWorkout setValue:_boostDate forKey:@"date"];
[newWorkout setValue:_workoutText forKey:@"text"];
[newWorkout setValue:_trainerLabelOutlet.text forKey:@"text"];
NSError *error;
if (![context save:&error]) {
    NSLog(@"Whoops, couldn't save: %@", [error localizedDescription]);
}else{
    NSLog(@"saved to core data");
}

but i get this error :
An exception was caught from the delegate of NSFetchedResultsController during a call to -controllerDidChangeContent:.  attempt to insert row 1 into section 0, but there are only 0 rows in section 0 after the update with userInfo (null)
the code in the UITableViewController:
- (NSManagedObjectContext *)managedObjectContext {
    NSManagedObjectContext *context = nil;
    id delegate = [[UIApplication sharedApplication] delegate];
    if ([delegate performSelector:@selector(managedObjectContext)]) {
        context = [delegate managedObjectContext];
    }
    return context;
}

- (NSFetchedResultsController *)fetchedResultsController {

    if (_fetchedResultsController != nil) {
        return _fetchedResultsController;
    }

    NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
    NSEntityDescription *entity = [NSEntityDescription
                                   entityForName:@"Workouts" inManagedObjectContext:self.managedObjectContext];
    [fetchRequest setEntity:entity];
    NSSortDescriptor *sort = [[NSSortDescriptor alloc] initWithKey:@"date" ascending:NO];
    [fetchRequest setSortDescriptors:[NSArray arrayWithObject:sort]];
    [fetchRequest setFetchBatchSize:20];
    NSFetchedResultsController *theFetchedResultsController =
    [[NSFetchedResultsController alloc] initWithFetchRequest:fetchRequest
                                        managedObjectContext:self.managedObjectContext sectionNameKeyPath:nil
                                                   cacheName:@"Root"];
    self.fetchedResultsController = theFetchedResultsController;
    _fetchedResultsController.delegate = self;
    return _fetchedResultsController;
}

- (void)viewDidLoad
{
    [super viewDidLoad];
    NSError *error;
    if (![[self fetchedResultsController] performFetch:&error]) {
        // Update to handle the error appropriately.
        NSLog(@"Unresolved error %@, %@", error, [error userInfo]);
        exit(-1);  // Fail
    }
}

- (void)viewDidUnload {
    self.fetchedResultsController = nil;
}

#pragma mark - Table view data source

- (NSInteger)numberOfSectionsInTableView:(UITableView *)tableView
{
    // Return the number of sections.
    return [[self.fetchedResultsController sections] count];
}

- (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section {
    id  sectionInfo = [[_fetchedResultsController sections] objectAtIndex:section];
    return [sectionInfo numberOfObjects];
}

- (void)configureCell:(UITableViewCell *)cell atIndexPath:(NSIndexPath *)indexPath {
    WorkoutObject *workout = [_fetchedResultsController objectAtIndexPath:indexPath];
    cell.textLabel.text = workout.text;
    NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
    [dateFormatter setDateFormat:@"yyy-MM-dd   HH:mm"];
    NSString *dateString = [dateFormatter stringFromDate:workout.date];
    cell.detailTextLabel.text = dateString;
}

- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    static NSString *CellIdentifier = @"Cell";
    CustomTableViewCell *cell =[tableView dequeueReusableCellWithIdentifier:CellIdentifier];
    // Set up the cell...
    [self configureCell:cell atIndexPath:indexPath];
    return cell;
}

- (void)controllerWillChangeContent:(NSFetchedResultsController *)controller {
    // The fetch controller is about to start sending change notifications, so prepare the table view for updates.
    [self.tableView beginUpdates];
}

- (void)controller:(NSFetchedResultsController *)controller didChangeObject:(id)anObject atIndexPath:(NSIndexPath *)indexPath forChangeType:(NSFetchedResultsChangeType)type newIndexPath:(NSIndexPath *)newIndexPath {

    UITableView *tableView = self.tableView;

    switch(type) {

        case NSFetchedResultsChangeInsert:
            [tableView insertRowsAtIndexPaths:[NSArray arrayWithObject:newIndexPath] withRowAnimation:UITableViewRowAnimationFade];
            break;

        case NSFetchedResultsChangeDelete:
            [tableView deleteRowsAtIndexPaths:[NSArray arrayWithObject:indexPath] withRowAnimation:UITableViewRowAnimationFade];
            break;

        case NSFetchedResultsChangeUpdate:
            [self configureCell:[tableView cellForRowAtIndexPath:indexPath] atIndexPath:indexPath];
            break;

        case NSFetchedResultsChangeMove:
            [tableView deleteRowsAtIndexPaths:[NSArray arrayWithObject:indexPath] withRowAnimation:UITableViewRowAnimationFade];
            [tableView insertRowsAtIndexPaths:[NSArray arrayWithObject:newIndexPath] withRowAnimation:UITableViewRowAnimationFade];
            break;
    }
}

- (void)controller:(NSFetchedResultsController *)controller didChangeSection:(id )sectionInfo atIndex:(NSUInteger)sectionIndex forChangeType:(NSFetchedResultsChangeType)type {

    switch(type) {

        case NSFetchedResultsChangeInsert:
            [self.tableView insertSections:[NSIndexSet indexSetWithIndex:sectionIndex] withRowAnimation:UITableViewRowAnimationFade];
            break;

        case NSFetchedResultsChangeDelete:
            [self.tableView deleteSections:[NSIndexSet indexSetWithIndex:sectionIndex] withRowAnimation:UITableViewRowAnimationFade];
            break;
    }
}

- (void)controllerDidChangeContent:(NSFetchedResultsController *)controller {
    // The fetch controller has sent all current change notifications, so tell the table view to process all updates.
    [self.tableView endUpdates];
}

Does anyone know what the problem is?
i try to figure it out myself for the past 2 days but i incapable to.
thanks
Comment: Does `sectionInfo` in `tableView:numberOfRowsInSection:` returns actual number of objects during inserting?
Comment: Can you make sure that `_fetchedResultsController` is not `nil` in `numberOfRowsInSection`?
Comment: @moby good point. I think replacing `_fetchedResultsController` with `self.fetchedResultsController` might fix the issue (as it will trigger the lazy init of the controller)
Comment: the first view is the table that in the viewDidLoad Perform a fetch from Core Data and if it's nil it's set it  (- (NSFetchedResultsController *)fetchedResultsController). the second view do the add function to the core data.i check the results from the core data and it's added successfully
Comment: update: i changed it to self.fetchResultController but it's still the same problem
Answer: I just had the same issue - in my case it turned out to be because the table view data source and delegates hadn't been assigned to the view controller on the storyboard in the outlets section of the attributes inspector.  Hence numberOfRowsForSection and numberOfSectionsInTableView were not being called.
Comment: Even though I was setting dataSource and delegate in init I was getting these errors. Wired them up in Storyboard and it's running perfectly now.


How to store tree structure in python permanently

Question: I created a tree structure using python. Assume the tree has root node, left and right child. I need to acceess the tree whenever required. But on opening the file and running makes recreation of tree. As i have huge number of nodes in tree, so the method of recreation is time consuming. Please provide me a solution such that i can store the tree, fetch and update(insert or delete a node) whenever required.
Comment: Welcome to StackOverflow. Your question is to broad and lacks of informations and a minimal working example. Please visit https://stackoverflow.com/help and read there to find out how to ask questions.
Beside this maybe the `json` module could be your friend. I solve similiar task with it.
Answer: you could use the pickle module that can be used to serialize and deserialize python objects quite easily.
<code>import pickle

tree = {'a': 'b'}
# serializing the data
with open('/path/to/some/file', 'wb') as f:
    pickle.dump(tree, f, pickle.HIGHEST_PROTOCOL)

# and deserializing it again
with open('/path/to/same/file', 'rb') as f:
    tree = pickle.load(f)
</code>


Access all printers from asp.net web api

Question: I am wondering if i can access a list of all installed printers from an asp.net web api project.
I have searched online for this information, but can only see articles pointing me to windows form implementation.
Comment: then it doesn't matter - will it be Winforms or ASP.NET - both are using the same approach.
Comment: Do you mean printers installed on your webserver? Since asp.net is backend technology and running on *server* - you can't get list of *clientside* printers using it...
Comment: @AndyKorneyev yes i mean server side
Answer: As others mentioned you will only get a list of printers installed on the server. Since the code is executed on the server it will not know anything about the clients setup. If you want the installed printers on the server, then here's how you do it.
<code>public class PrinterController : ApiController
{
        [HttpGet]
        public IHttpActionResult Get()
        {
            var result = System.Drawing.Printing.PrinterSettings.InstalledPrinters;
            return Ok(result);
        }
}
</code>
Output will be something like:
<code>[
"Send to OneNote 2010",
"Microsoft XPS Document Writer",
"Microsoft Print to PDF",
"Fax",
"\\\\prntServer\Printer1"
]
</code>
Answer: From a web application you can access the printers installed on the server on which the application is running. For security reasons you don't have access to the list of printers on the client machines. In order to do that you will need some plugin or desktop application that your customers need to install.


Invalid argument supplied for foreach()

Question: I have absolutely no idea what I'm doing wrong right now.  I think I'm mentally exhausted, because I'm utterly clueless.  Here's the code I'm using:
if(empty($this->updates) || !is_array($this->updates))
    return null;

foreach($this->updates as $update)

This is failing.  HOWEVER, if I do a print_r($this->updates) before the foreach (and after), it works perfectly fine.  Why is it that when I try to use it in a foreach it pretends the array doesn't exist?
Sample print_r($this->updates):
Array
(
    [0] = Array
    (
        [id] => 1
        [name] => test
    )
    [1] = Array
    (
       [id] => 2
       [name] => rawr
    )
)

Comment: This error usually occurs if you try passing any thing *other than* an array into a `foreach`. What does your `print_r` return ?
Comment: can we see the result of `print_r`?
Comment: Can you tell use the result of var_dump($this->updates) ? Plus, you should add if(!isset($this->updates) || empty($this->updates))
Comment: @Chouchenos that still does not check if it is an array.
Comment: My code is on a classified machine, but I can give you a simple mockup of what updates contains.  I assure you, it's an array.
Comment: Array
    (
        [0] = Array
        (
            [id] => 1
            [name] => test
        )
        [1] = Array
        (
           [id] => 2
           [name] => rawr
        )
    )
Comment: @Russell Dias : Indeed. if(!is_array($this->updates) || empty($this->updates)) is better. Thanks for the correction ;)
Comment: That still doesn't impact the problem.  :(
Answer: Looks like $this->updates is not empty but its not an array. You can use the is_array test before you use it in foreach:
if(is_array($this->update)) {
  foreach($this->updates as $update) {
  .....
 }
}

Comment: $this-updates definitely has entries in it.  For the particular piece of code I'm working on, there are 27 arrays within updates.  I posted the results of an example print_r in the original question.
Answer: Since you don't tell what $this->updates is, I can simply assume it's not an array. Here, you have two options :
1- Replace empty() with !is_array() to check if $this->updates is valid or not. If it's empty, it doesn't matter, the foreach will simply do nothing...
if(!is_array($this->updates))
    return null;

foreach($this->updates as $update)

Or if the foreach is not the only processing you do :
if(empty($this->updates) || !in_array($this->updates))
    return null;

foreach($this->updates as $update)

2- Force $this->updates to be an array
if(empty($this->updates))
    return null;

foreach((array) $this->updates as $update)

Comment: $this-updates is already an array.  I posted the results of a print_r in the original question.
Comment: @Atrox, is this piece of code executed twice? Maybe the first pass works, but fails on the second one (or vice-versa)? Because there is NO way that `foreach` fail if `$this->updates` is **always** indeed an array
Comment: It's executed about 90 times on one page, and sometimes it works, sometimes it doesn't.  If I use print_r before the foreach, it works 100% of the time.
Comment: the execution of a single PHP request is single threaded, this is not likely or even possible. ... have you tried what I'm suggesting? Particularly the second suggestion?

Invalid argument supplied for foreach()


É necessário invocar as funções "ShowWindow" e "UpdateWindow"?

Question: Recentemente fiz um introdução a Windows API e já estou na parte de criar janelas. Numa das primeiras aplicações que criei tive utilizar de ShowWindow e UpdateWindow depois de criar uma janela, mas mesmo sem elas a janela continuava a mostrar-se.
Até criei mais janelas para ver se a sua invocação seria necessária e essas janelas adicionais também apareciam.
Então fica aqui a minha questão é mesmo necessário invocá-las? Terá algum impacto não fazer? Ou é só uma medida de prevenção?
P.S Coloquei a tag de C++ pois é um API também utilizado nessa linguagem
Answer: Eu não sei como você escreveu a aplicação, mas podemos dizer que algumas funções já chamam estas funções citadas "automaticamente" o que tornaria realmente isto redundante.
ShowWindow
Mas note que o uso de ShowWindow não é só para exibir a janela na hora de iniciar um programa, ele tem como "utilidade" mudar o estado da janela que está fazendo referencia, por exemplo, isto forçaria a minimizar uma janela já aberta:
 ShowWindow(hwnd, SW_FORCEMINIMIZE);

SW_FORCEMINIMIZE minimiza uma janela, mesmo que o Thread proprietário dela não esteja respondendo
SW_HIDE - Oculta a janela e ativa outra janela
SW_MAXIMIZE - minimiza a janela
SW_MINIMIZE - minimiza a janela e ativa a proxima que está no level maior pela ordem Z
SW_RESTORE - ativa e mostra a janela. Se estiver minimizada ou maximizada ele "restaura" para a posição e tamanhos originais.
SW_SHOW - ativa e mostra a janela na posição e tamanho original.
SW_SHOWDEFAULT - Define o estado da janela baseado no valor SW_ especificado na estrutura STARTUPINFO passado para o CreateProcess
SW_SHOWMAXIMIZED - Ativa e mostra a janela maximizada
SW_SHOWMINIMIZED - Ativa e mostra a janela minimizada
SW_SHOWMINNOACTIVE - mostra a janela minimizada, é similar ao SW_SHOWMINIMIZED, exceto que a janela não é ativada
SW_SHOWNA - Mostra a janela na posição e tamanho atual, é similar ao SW_SHOW, exceto que a janela não é ativada
SW_SHOWNOACTIVATE - Mostra a janela na posição e tamanho mais recente, é similar ao SW_SHOWNORMAL, exceto que não ativa a janela.
SW_SHOWNORMAL - Ativa e mostra a janela. Se a janela está minimizada ou maximizada isto restaura para o tamanho e posição original. Uma aplicação deve especificar esta "flag" ao exibir a janela pela primeira vez.

UpdateWindow
O UpdateWindow atualiza a área de cliente da janela, enviando uma mensagem WM_PAINT para a janela se a região de atualização não estiver vazia. Isto pode ser útil para animações talvez, então você irá usa-la quando necessário para atualizar a renderização.
Não tenho muito conhecimento, mas ao que me parece estes são os objetivos, se tiver alguma duvida a mais talvez apareçam novas respostas, espero que por enquanto isto ajude.
Documentação:

ShowWindow
UpdateWindow
Comment: Guilherme, você teria um link da documentação disso tudo? Agregaria bastante à sua resposta.
Comment: Obrigado agora já percebi xP. O tutorial que estou a seguir não explica certas coisas que no MSDN também não estão explicadas :/
Comment: @bfavaretto adicionado documentação  sobre o `ShowWindow`, meu inglês não é muito bom, traduzi as flags da documentação da microsoft, se tiver alguma gafe me perdoe-me, eu traduzi com o básico que conheço do inglês (que é bem ruim por sinal :p ), eu não usei o google-tradutor pois muitas vezes os textos perdem o sentido para mim.
Answer: Só para complementar o que o Guilherme disse, se você perceber, na maioria das vezes no momento the chamar o ShowWindow dentro da WinMain você envia o argumento nCmdShow, esse argumento é a maneira que o Windows quer que sua janela seja exibida(já que você pode mudar o modo de exibição de atalhos), então basicamente ao chamar o ShowWindow enviando nCmdShow, você não esta exatamente pedindo para função mostrar sua janela, mas sim para mostra-la da maneira que o Windows quer.
Mas a janela não é mostrada na tela sozinha, ela só é mostrada na tela automaticamente se você criar a janela enviando a flag WS_VISIBLE, veja este exemplo:
CreateWindowW(wc.lpszClassName, L"Janela", WS_OVERLAPPEDWINDOW | WS_VISIBLE, 100, 100, 250, 150, NULL, NULL, hInstance, NULL);

No caso acima ela é mostrada automaticamente, pois você enviou o WS_VISIBLE na função, ou seja, mesmo sem chamar o ShowWindow ela será mostrada.
Agora veja este caso:
HWND hwnd = CreateWindowW(wc.lpszClassName, L"Janela", WS_OVERLAPPEDWINDOW, 100, 100, 250, 150, NULL, NULL, hInstance, NULL);
ShowWindow(hwnd, SW_SHOW);

Neste caso a janela não aparecera sem a chamada do ShowWindow, pois não foi enviada a flag do WS_VISIBLE.
Comment: Ah! Pensava que o `nCmdShow` era uma espécie de `argc`. Obrigado pelo esclarecimento  ;D


Is this theorem about "completion of metric space" correct?

Question: It's well-known that there is a completion of a metric space unique upto isometry.
I have tried to modify this theorem slightly and I proved this statement:

Let $(X,d_X)$ be a metric space.
Then, there is a unique complete metric space $(Y,d_Y)$ such that $X\subset Y$, $d_Y \upharpoonright (X\times X)= d_X$ and $X$ is dense in $Y$.

This is indeed a yes or no question.. Since I have reformulated the theorem, I want to make sure whether I have proven it correctly.. Is this true?
Comment: even if it is true, the proof maybe wrong though
Comment: This is really just the same question as your previous one, only here you're hiding under the guise of metric spaces rather than hot liquid magma or just plain sets.
Answer: The uniqueness of $(Y,d_Y)$ in your modified version of the theorem is certainly not true.
Precisely the point of saying that it is unique up to an isometry is that instead of $Y$ you can take any set $Y'$ such that there exists a bijection $Y\to Y'$. (And you transfer the metric form  $Y$ to $Y'$ using this bijection; i.e., you take the metric on $Y'$ such that this bijection becomes isometry.)


Getting error in vertica:Function catalog.btrim(int) dose not exist, or permission is denied for catalog.btrim(int)

Question: I am trying to execute below query in vertica:
select
case when count(*) = 0 then #{aaa} || #{bbb}
else
 trim(cast(max(ccc)+1 as intger))
end
 as ccc
from apple where aaa = #{aaa}

Query is getting expected results in tera but in Vertica Getting error.
How can I do this?
Answer: Check the Vertica docu:
https://www.vertica.com/docs/10.0.x/HTML/Content/Authoring/SQLReferenceManual/Functions/String/BTRIM.htm
BTRIM() removes the blank characters (by default, if not specified as the second parameter) from a passed string. You can't BTRIM() a number.
Why would you want to trim an integer, in trim(cast(max(ccc)+1 as int**e**ger)) ? It makes no sense at all!. Teradata should complain, too.
Comment: I have already used btrim. It shows Function catalog.btrim(int) dose not exist, or permission is denied for catalog.btrim(int) on Vertica as well. I have to trim an integer because I need to receive parameter as an integer from another query to make it work.
Comment: What type is `ccc`? `max(ccc)+1` is the same numeric type as `ccc`. If `ccc` is an integer, yon don't need to do anything. If `ccc` is a `FLOAT` or a `NUMERIC` , you would either `CAST(MAX(ccc)+1 AS INTEGER)`, or use the operator `::`, like in  `(MAX(ccc)+1)::INTEGER` . If `ccc` is a string, it is either successully implicitly casted as a numeric value before adding one, or it fails. You `CAST()` to change the type, you don't `TRIM()`. And you do that only if you need to.


How to interpret the model weights extracted from tensorflow2 keras LSTM model?

Question: I fitted a tensorflow.keras.layers.LSTM model and extracted the model weights via get_weights(). However, I find it hard to interpret the weights array.
To be specific, I set the model by
model = Sequential()
model.add(LSTM(128, input_shape=(10, 10)))
model.add(Dense(1))

and fit the model with the data of the following shape
train_X.shape, train_y.shape, test_X.shape, test_y.shape
## ((23, 10, 10), (23,), (6, 10, 10), (6,))

The code for model fitting:
model.fit(train_X, train_y, epochs=50, batch_size=4,
                    validation_data=(test_X, test_y), verbose=2, shuffle=True)

The shape of model weights:
[w.shape for w in model.get_weights()]
## [(10, 512), (128, 512), (512,), (128, 1), (1,)]

The math formula of LSTM:

As you can see from the formula, there are eight weight matrices and four bias vectors. However, I don't know how to match them to the weights array.
Answer: When you print
print(model.layers[0].trainable_weights)

you should see three tensors: lstm_1/kernel, lstm_1/recurrent_kernel, lstm_1/bias:0 One of the dimensions of each tensor should be a product of
4 * number_of_units
where number_of_units is your number of neurons.
Try:
units = int(int(model.layers[0].trainable_weights[0].shape[1])/4)
print("No units: ", units)

That is because each tensor contains weights for four LSTM units (in that order):
i (input), f (forget), c (cell state) and o (output)
Therefore in order to extract weights you can simply use slice operator:
W = model.layers[0].get_weights()[0]
U = model.layers[0].get_weights()[1]
b = model.layers[0].get_weights()[2]

W_i = W[:, :units]
W_f = W[:, units: units * 2]
W_c = W[:, units * 2: units * 3]
W_o = W[:, units * 3:]

U_i = U[:, :units]
U_f = U[:, units: units * 2]
U_c = U[:, units * 2: units * 3]
U_o = U[:, units * 3:]

b_i = b[:units]
b_f = b[units: units * 2]
b_c = b[units * 2: units * 3]
b_o = b[units * 3:]


Difference between mixing and mastering

Question: I am a musician, and I have amateurish-ly recorded my own musical inspiration. I am curious to learn, though, and one of the most confusing things is - as so often - in terminology. I am looking for tutorials to get a better, warmer sound after recording on lower-end gear using software tools. The problem is that I don't know what keywords to search for. I've seen the terms mixing and mastering come by a lot, but it seems to me that they are either synonyms or a lot of people misuse one or the other.
Tl;dr: what is the difference between mixing and mastering recorded audio?
Answer: I too am an amateur "recording engineer" from the standpoint of I record my original songs in my home studio on inexpensive gear to produce what I consider a rough demo - capable of conveying the general idea of the song.  When I want a more professionally produced "radio ready" cut, I will go to a professional studio.
To simplify the answer to your question, let's talk in terms of one song so we don't get confused between mastering an entire album and mixing a song.
A song that you record will consists of more than one track (otherwise there is nothing to mix).  You might have your rhythm guitar on one track, your main vocals on another, backing vocals on a third, lead guitar on a 4th etc.
Once you record each individual track (while monitoring the others as the new part is played) you will need to do some mixing.  This is where you adjust the parameters of each individual track in your song to achieve the optimal balance between tracks as it relates to the overall mix.  You will adjust the relative volume of each track, the stereo pan, perhaps add compression to some tracks, maybe a little reverb or chorus or delay or whatever is needed to enhance each track so it fits into the mix of the overall song the way you envision it.
In your application, most of the shaping and control you will impart on the song will be during the mixing process.  This is where you will make sure that the guitar does not mask the vocals, the lead guitar part stands out above the rhythm guitar during a solo, the bass guitar can be distinguished from the kick drum, the background vocals don't overpower the lead vocal and so on.
Mastering is done after all the mixing is set just the way you want it. During mastering, you will capture the mix you created and fine tune it and prepare it for saving as a digital file you can use outside your home studio.
During mastering you are working with the entire mix (consisting of all the tracks in your song) as a whole.  During mastering you will adjust the amplitude to eliminate any inappropriate spikes or drops in volume, perhaps apply some compression to the song as a whole, add some limiter parameters and so on.  It is also during mastering that the song is formatted and encoded into something you can convert into a usable digital format such as a .wav file or something that can be stored on a CD.  It's even more complex if you are going to master your song for vinyl.  Without mastering, you could not save your song in a format that could be distributed to others.  You would only have your mix on your home recording software/hardware.
To use an analogy - suppose you want to take a group photo of a tennis team or other group of people.  You will first want to arrange the group so that everyone is visible in the picture (tall people in the back, everyone close enough together to fit in the frame etc.).  You want to frame the photo so the group is centered, and perhaps be sure the lighting is optimal.  After a little shuffling and repositioning and zooming in or out, you now have composed the picture just the way you want it.  That's the mixing part.
But you want to be able to share what you can now see in your viewfinder with others and preserve this image for posterity.  So you take a photograph with your camera and your camera captures the image and "masters" it so that you can save it as a .jpg image or print it on photo paper. Of course you can tweak the image with digital photo editing software (or in the darkroom if using film) to enhance the end product (also part of mastering).  But you can't change who is in the picture or where they appeared in the picture.  All that was done during the mixing.
Recording and mixing is a fun part of the creative process of songwriting and composing.  Have fun with it and good luck.
Answer: The mixing engineer takes the individual recorded tracks (drums, vocals, guitar, bass, etc) and consolidates them into one single entity: the song.
The mastering engineer takes that single entity that the mixing engineer produced, polishes it further, and prepares it for use and distribution.
In a jigsaw puzzle, the mixing engineer would be the one that takes the individual pieces and solves the puzzle, and the mastering engineer would be the one that makes sure everything is fine to then apply glue, polish, and then frame the solved puzzle.
Mixing involves many different things: panorama (positioning each element in the mix), volume control (compressors, limiters, vocal raid), effects (reverb, delay, harmonic enhancers, distortion), pitch correction, cleaning audio from unwanted sounds and artifacts, quantizing, adjusting the frequency balance (equalizers), to name a few.
Mastering involves many different things: frequency balance (sometimes the mastering and mixing engineers don't completely agree on this, so the mastering engineer can adjust it further), volume control (now that they don't have all the tracks individually the volume control is very limited, but some adjustments are possible through multi-band compression), applying RIAA curve, adjusting stereo width, adjusting loudness (through compressors and/or limiters, possibly applying limiter at 0 dBFS), adding metadata, adjusting start and end of songs, conserves coherence in relative volumes among tracks of one album, to name a few.
The production chain is: recording -> mixing -> mastering -> distribution
Some explicit differences:

Number of tracks. For each song, the mixer works with several different recorded tracks, while the mastering engineer works with one (the song itself, produced by the mixer).
Relative volumes. The mixer will adjust relative volumes among instruments, not among songs. The mastering engineer will adjust relative volumes among songs and possibly among instruments (with some limitations since he can't adjust the volume of each instrument individually, he has to work with frequency bands instead).
Processes. The mixer will apply equalization to each individual element, and not to the whole song. The mastering engineer will apply equalization to the whole song. Same applies to other processes (compression, reverb, etc).
Final touches (metadata, RIAA curve, etc). The mixer is not involved at all, it's 100% the job of the mastering engineer.
Comment: In other words, it seems as if you mean that the mixing engineer makes sure that the song itself sounds as it should, considering relative volume and volume normalization. The mastering engineer can then make the song as a whole, but also individual instruments, sound better? Because that's different from what Steve is saying, I think. (He's speaking about a whole CD and mixing tracks relatively to each other)
Comment: @BramVanroy One works with multiple audio sources (mixing), the other works with only one (mastering). That's the fundamental difference. But both involve a lot of other stuff, with some overlap. Not everything is mutually exclusive. I edited my answer to add more detail.
Answer: Producing an audio creation is usually a 4 part process:
Recording is quite obvious, as the name says it means recording the musical creation itself.  As much as you've probably heard, using great gear for recording infact does make a difference later when it goes to the mixing and mastering stages.   Low quality gear tends to add unneccessary noise to the recording, and also may not capture the sound and miss some depth.  Not something that you can't get rid of, but still recording a singer with a 2000$ Shure Microphone -vs- recording him with a computer home microphone is a HUGE difference.
Sequencing is the next part of an audio production, its mostly the creative process of composing the musical track itself -  arranging your audio tracks, what piece comes first, what comes next, what goes together, audio effects (Reverb, delays, etc).   During this stage, usually the artist itself will perform "mixing" in terms of balancing the volume of audio tracks, but that is far from what professional mixing is...
Mixing is the process of balancing the different audio tracks in terms of volume, and frequencies (EQ), to make them "heard" together.  This is a little bit more than just balancing the volume itself.  On some instruments, you need to add more brightness (for example: on Hihats or Cymbals of drums), on some you need more bass (for example: Bass drum, Bass guitar, etc), and also good mixing involves knowing how to balance the instruments realy well in terms of frequecies and sometimes even seperating a few instruments that share the same frequency range using sidechain ducking.  For example, if a Bass guitar and a Bass drum both share the audio spectrum of 60-200hz, simply mixing them together will create an overload in this frequency range that will "swallow" other instruments, which is not good at all.   So SOME techical knowledge is required from a realy good mixing producer.
Mastering is the final process in the production of audio.  And this, plus the mixing part, are usually the parts that makes the whole difference between a commercial breakthrough album to an amateur one.  A great example would be Nirvana's "Nevermind" album.  In terms of the music itself, it is ok.  But combined with a great mixing producer and a great mastering, that album became one of the best albums ever sold.   So what is Mastering?  Mastering is the process of taking the final mix, which is usually "low" in volume, and usually made on reference speaker, and doing all sorts of things in order to get it to be "ready" for a commercial release: Amplifying the overall volume, Balancing the instruments correctly (in case the mixing producer missed them), monitoring it on high-end speakers instead of reference speakers, testing it on a wide variety of different speakers, and much more.   Mastering engineers are usually people with both very accurate hearing ears, and both technically gifted in terms of perfectly controlling a wide range of audio processors that the average person have no idea of, and that are neccessary for cleaning, polishing, and balancing the final product for commercial release (For example: Multiband compressor, Noise gates, Hiss removers, and more).
I hope this answered your question.
Comment: Wait... are you saying that "Nevermind" was successful because it was mastered well, and that its success had nothing to do with the music?  That's just absurd...
Comment: I know.  Don't feel upset about it though.  I myself love Nirvana and "Nevermind" is one of my all time favourite albums, and I also like it in terms of the musical composition.  But the way Nirvana sounded on stage -vs- how they sound on this album is the same difference as between the ground and the sky.   Do a Youtube search for some of the songs from this album on a live performance.   You'll then realize that Nirvana don't sound all that great anymore after listening to a live version of them.
Comment: It's no secret that many bands sound radically different on stage and in the studio, but that difference does not come down to mastering.  "You can't polish a turd," so they say, and this applies to music as well.  If Nirvana had come in to the studio with a bunch of crap songs, they would have left with a crap album.  A lot of the differences are going to come down to the studio environment, how they interact with the producer, micing and EQ, and a million other choices that have *nothing to do with mastering.*  I'm not upset, but this is a terrible and inaccurate answer.
Comment: You're realy just more justifying my answer.  Mixing and Mastering are both the two pieces of the same thing.  You probably aren't familiar with this enough, but did you know that Mastering engineers, especially ones that have to compose albums.. actually work on the multitracks received from the mixing engineer and correct mistakes?   Did you think that mastering is only the final mixdown?  And regarding "Nevermind": That album was the main reason for Nirvana's commercial success. even Kurt Kobain himself, in multiple interviews, stated how surprised they were by how good this album sounded.
Comment: Stop making this about me... this is not about me, so don't say things like "You probably aren't familiar with this enough" or "Don't feel upset about it though"... this kind of language is not acceptable on the Stack Exchange network.  We should probably move this to chat, but your statement that the mastering engineer works with the same multitrack sources as the mixing engineer goes against my understanding of how multitrack mastering works.  Can you cite sources?  From what I understand, when you get something mastered that way, you're only using the stems, not all of the original tracks.
Comment: Ok, I apologize for my language.  Now, to the question - I don't know of any sources where this is officially written at, but I speak as someone who do mastering and in my own experience, working on the mix alone is not going to get the best results, especially when you need to take several songs that were recorded in different studios and make them sound similar to each other.  Your possibilities of "polishing" or "correcting" the sound are very limited when you work only on the mixdown.  in that case, you have to rely on the mixing engineer to do a good job.  In most cases, its not enough
Comment: I think part of the confusion here is that when I read your answer, I keep seeing a lot of information about mixing under the "mastering" section, which makes me think that you're saying things that you're not actually trying to say.  But, stereo mixing is still rather common, and I'd say that the mixing engineer makes the mix sound good (given a good recording), and the mastering engineer makes it translate well to the final media (given a good mix).  Yes, that often means multitrack mastering, but even in those cases the mastering engineer isn't going to redo the mix.
Answer: Someone will be along in a while with a more detailed response; but in a nutshell
mixing is adjusting the relative volume levels, effect and eq levels, stereo placement, etc. of the various recorded elements of your track to achieve the overall sound you want for your recording. So if you can't hear the singer over the guitar (or vice versa) that's a problem in the mix.
mastering is taking the various individual completed mixes and preparing them for release - so you would ensure, for example, that the relative volumes of the various tracks match, and that the tracks have an appropriate run-in and run-out both for the track itself and in relation to the track either side of it in your intended sequence on the released version. A CD or a collection of MP3s where you have to adjust the playback volume between each track would be an extreme example of poor mastering. Mastering was also where the 'loudness wars' of the 1990s were fought, as every pop producer tried to master their song to sound louder and brighter on the radio than everyone else's.
Mastering would also involve ensuring that the tracks individually and as an 'album' are technically ready for pressing to vinyl, burning to CD or encoding to MP3, and the preparation of the physical / digital master from which all other copies were to be made.
Answer: A vernacular phrase for the difference between a mix engineer and a mastering engineer is this:  "A mix engineer's job is to put in as much goodness as he can.  The mastering engineer's job is to remove as much 'suck' as he can."
The mastering engineer can't really ADD anything...by virtue of their position in the workflow, they can only really minimize deficiencies.
Historically, "mastering" was a very straightforward part of the process.  It was a purely technical final production step where the finished "master recording" was prepared for duplication.  In vinyl duplication, the mastering engineer would be taking the mono or stereo final tape, and evaluating it for the characteristics that affected the cutting of the master lacquer on the cutting lathe.  This involved fine-tuning the final level and perhaps some minor EQ (above and beyond applying the RIAA curve), as these parameters affected the groove pitch, and thus the playing time of the platter.
Today, the term "mastering engineer" has lost almost all meaning. Today it means any and all stupidity and nonsense and second-guessing of the original mix that you can possibly think of.  There are mastering engineers that apply multiband compression, demand stems so they can completely re-work all the balances decided upon by the mixer/producer, apply all manner of effects, etc.  This sort of "mastering" is really better called "remixing."
In today's music production workflow, it is common to have a mixer working in a room or environment that isn't acoustically/sonically optimal, perhaps with prosumer equipment that isn't of the highest fidelity.  In this scenario, the final mix can be "colored" by the deficiencies of the mixer's environment.
According to many "A-list" mix engineers and producers, the new role of the mastering engineer here is to be an "objective ear" that listens to the project with fresh perspective, on high-quality equipment in a highly-optimized sonic environment.  In such a place, the deficiencies and idiocyncracies of the mixer's listening space become apparent, and can (hopefully) be compensated for, such that the final mix will sound LIKE THE MIXER INTENDED across a wide variety of reproduction systems.
The role of the MIX engineer is very different.  The mix engineer has the task of taking a set of recorded tracks and turning them into a single presentation that conveys the musician's musical and emotional intent.
Answer: I am a mixing and mastering engineer myself. So I believe I can help you the best.
Mixing - audio mixing (or "mix down") is the process which commences after all tracks are recorded and edited as individual parts. The mixing-process can consist of various processes but are not limited to setting levels, setting equalization, using stereo panning, and the addition of effect. I.e mix engineer takes the flaws out of the rough mix that has been given to him by the producer or composer.
Mastering - Basically In this process a mastering engineer works on the final mix that has been given to him by the mix engineer and process on it to make it consistent over the playback system. What I mean is if only a mixed audio is played on a different systems they would sound different on different devices. On hi-fi they would sound fab, on phones they would sound low in volume or brittle, in the mastering process the engineer does some processing that encodes the audio file with certain codings that when you hear it on any device the audio would retain its basic characteristic. Features. That is if its a hip hop song which tends to have more bass good low end if play the mastered track on any device. The track won't feel like its lacking the low end it will still retain the lows.
All in all the track become uniform for playback after mastering.
To sum up - Mixing is the foundation process where mix engineer builds , furnishes all the ideas that the musical would have had while making the song where as mastering is like icing on the cake its like painting the house to keep it beautiful for ever.

Difference between mixing and mastering


Como colocar linhas em um gráfico e barra?

Question: Gostaria de saber como insiro linhas em um gráfico de barras.
Tenho o seguinte código:
Periodo <- c("01/2020", "03/2020", "05/2020",
            "06/2020", "08/2020", "09/2020")
Valores <- c("0,08", "3,15", "4,91", "0,72", "6,21", "8,80")

teste <- data.frame(Periodo, Valores)

library(ggplot2)

ggplot(teste) + aes(x = Periodo, y = Valores) + geom_bar(stat='identity')

Comment: Altere sua pergunta para: Como adicionar linhas ao gráfico e como fazer para transformar uma coluna em dados numéricos.
Answer:
Gostaria de saber como insiro linhas em um gráfico de barras.

Você pode inserir linhas utilizando o geom_line():
ggplot(teste, aes(x = Periodo, y = Valores, group = 1)) +
  geom_bar(stat='identity') +
  geom_line()

Answer: O principal problema parece ser o vetor Valores ser um vetor de classe "character", os números são strings, sem qualquer valor numérico. Isso pode ser visto pelo eixo dos yy, cujas anotações são equidistantes apesar dos valores não o serem. As diferenças de um valor para o outro não são constantes mas a diferença de alturas das barras é sempre a mesma.
Para ter o gráfico certo, tem de se transformar o vetor Valores em vetor numérico, substituindo primeiro as vírgulas por pontos.
O gráfico final mostra os dois gráficos lado a lado com o pacote gridExtra, de modo a se ver melhor a diferença.
library(ggplot2)

Periodo <- c("01/2020", "03/2020", "05/2020",
             "06/2020", "08/2020", "09/2020")
Valores <- c("0,08", "3,15", "4,91", "0,72", "6,21", "8,80")

teste <- data.frame(Periodo, Valores)

g1 <- ggplot(teste, aes(x = Periodo, y = Valores, group = 1)) +
  geom_bar(stat='identity') +
  geom_line() +
  ggtitle("Gráfico original")

Valores <- as.numeric(sub(",", ".", Valores))
teste <- data.frame(Periodo, Valores)

g2 <- ggplot(teste, aes(x = Periodo, y = Valores, group = 1)) +
  geom_bar(stat='identity') +
  geom_line() +
  ggtitle("Gráfico depois de 'as.numeric/sub'")

gridExtra::grid.arrange(g1, g2, nrow = 1, ncol = 2)


Python/Django: How to show both main model and 'foreign-key model' together in HTML

Question: Good Day SO, I am a beginner in Django and python, just started learning two days ago. Currently, I am trying to do my filtering of data in views.py and creating a context to be shown in my main page that contains both the initial model and the 'foreign-key' model. However, I am having trouble finding help online, even though this is a simple question.. Here goes..
Models involved:
class Plan(models.Model):
    plan_ID = models.CharField(
        primary_key=True,
        max_length=8,
        validators=[RegexValidator(regex='^\w{8}$', message='Length has to be 8', code='nomatch')]
    )
    plan_crisisID = models.ForeignKey(Crisis, on_delete=models.CASCADE)
    plan_status = models.CharField(max_length=50)

class Crisis(models.Model):
    crisis_ID = models.CharField(
        primary_key=True,
        max_length=4,
        validators=[RegexValidator(regex='^\w{4}$', message='Length has to be 4', code='nomatch')]
    )
    crisis_name = models.CharField(max_length=50)

Views.py for HTML:
def home(request):
    template = loader.get_template('pmoapp/home.html')
    crisisList = Crisis.objects.filter(crisis_status='Ongoing').order_by('-crisis_ID')

    context = {
        'crisisList': crisisList,
        #'planList': planList
    }
    return HttpResponse(template.render(context, request))

And finally, my HTML page:

    {% if crisisList %}
         {% for crisis in crisisList %}

                   {{ crisis.crisis_ID }}
                   {{ crisis.crisis_name }}
                   {{ crisis.crisis_dateTime }}
                   planid
                   planstatus

          {% endfor %}
      {% else %}

No crisis available.

      {% endif %}

I have several things that I do not know how to do here.. so sorry and bear with me..
As seen above, I am currently only able to show the attributes of the Crisis model, and I do not know how to show the Plan, nor how to filter the data to get the plan with the same crisis ID and highest plan ID
I have a many-to-one relationship between Crisis and Plan, such that a Crisis can have multiple plans, linked by 'Foreign-key': plan_crisisID. I would like for the HTML to show all Crisis objects, and the plan with the highest id value and belonging to the same crisis in the HTML format seen above (planid, planstatus).
I know that the logic portion of this code is done in views.py, however I do not know how to start, or what to do.. Thank you SO for your kind help, I will provide any additional information required.. I have many other similar models with foreign key and many-to-many links and in similar situations.. I hope to learn from answers here so that I can replicate this in other places in my project..
Answer: I think that you are on the right track, but should start from the other end because Crisis is on Plan.
In the views pull planlist instead of crisislist, then you can:

{% if planList %}
     {% for plan in planList %}

               {{ plan.crisisID.crisis_ID }}
               {{ crisis.crisis_name }}
               {{ plan.crisisID.crisis_dateTime }}
               {{ plan.plan_ID }}
               {{ plan.planstatus }}

      {% endfor %}
  {% else %}

No crisis available.

  {% endif %}

Oh and you should probably swap the query to something like:
planList = Plan.objects.filter(crisis_ID__crisis_status='Ongoing').order_by('-crisisID__crisis_ID')

Comment: There is an error Edit: tried using this instead: planList = Plan.objects.filter(plan_crisisID__crisis_status='Ongoing')
Comment: Yes! I realised that after i posted the question. Initially I used Crisis first because.. I do not know how to look for the Plan with the highest Plan ID and how to code it via python in views.py. I know how to do it in other languages, but I currently do not know how to do it in views.py via python.. Edit: Also, because I have a filter on my Crisis status = 'Ongoing', I also do not know how to filter for this information in the planList as well.. sorry for the inconvenience..
Comment: you just read my mind! Can I run a for-loop in the views.py to get the maximum value for my Plan ID? or do i import a math module to get a max() method?
Comment: Good catch, the orderby is from the Crisis object.
Comment: Learning new things everyday, thanks to awesome people like you bro :) Have a nice day!


Solving a Nobel Prize problem won by Dr. Kahneman. Please help.

Question: A certain town is served by two hospitals. In the larger hospital about 45 babies are born each day, and in the smaller hospital about 15 babies are born each day. About 50% of all babies are boys. However, the exact percentage varies from day to day. Sometimes it may be higher than 50%, sometimes lower.
For a period of 1 year, each hospital recorded the days on which more than 60% of the babies born were boys. Which hospital do you think recorded more such days?
Three possible answers:
A. the larger hospital
B. the smaller hospital
C. about the same ( within 5% of each other).
Would it be the smaller hospital, since there's less babies are born each year?
Comment: I don't get  the title: did that doctor won a Nobel Prize (in what?) for solving that problem? It seems to be a rather simple problem to win aNobel, even if those prizes are heavily over-rated.
Comment: It is since the variance is bigger. The variance of the sample proportion is $\frac{p(1-p)}{n}$, where $p$ is the proability of a boy birth and $n$ is the sample size. Big $n$ (hospital), small variance.
Comment: The title is ridiculous.
Comment: @DonAntonio : Kahneman's Nobel Prize was for prospect theory: http://en.wikipedia.org/wiki/Prospect_theory, http://en.wikipedia.org/wiki/Daniel_Kahneman
Answer: B. The smaller hospital. The intuitive Law of Large Numbers says that with more observations the experimental average tends to get closer to the true proportion. A big hospital has more observations, so their number will be less likely to stray from 50%, or have less variance.


Override Codable implementation in struct

Question: Is there a way to override the Codable implementation for a struct?
Code that can't be changed:
struct obj1 : Codable {
   var values:[Int]
   ...
}

Desired JSON:
{ "x": 1, "y": 2, "z": 3 }

This doesn't work:
extension obj1 {
   init(from decoder: Decoder) throws {
      //This is never called
   }

   func encode(to encoder: Encoder) throws {
      //This is never called
   }
}

Backstory:
I'm in the process of porting an existing application to iOS, and converting to Swift from a mixture of C++ and Objective-C.  This application uses Metal to do 3D graphics, and therefore makes extensive use of math constructs like Vector, Matrix, Quaternion, etc.
At the beginning of the project I decided to use Apple's simd library for these various types.  I gave them a typealias to match the objects we already have to aid porting, and extensions to add various functionality that wasn't built in.  This all works great.
I'm now at the point where I need to be able to convert these types to/from JSON, and I've run into a problem.  The simd structs already support Codable, but they use a different format than the one I need to support.  Since simd uses structs, I can't derive from it and provide my own definition of init(from) and encode.  It also appears as though I can't put my own definition in an extension either (the compiler doesn't complain but my versions aren't called either).
I've currently got this working using a struct wrapper (ie, Vector3Codable). However, this means that any of my objects that have these simd members must also provide custom Codable implementations, instead using the default support that the compiler adds in.
I'd prefer to stick with simd if possible, as opposed to providing my own implementation for everything.
Edit - It appears as though my simplified sample code actually does work (doh!).  Here is the actual code that doesn't work. Apparently the issue is more complex than I initially thought.
typealias Vector3 = SIMD3<Float>

extension Vector3 {
    enum CodingKeys: String, CodingKey {
        case x
        case y
        case z
    }

    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        let x = try container.decodeIfPresent(Float.self, forKey: .x) ?? 0.0
        let y = try container.decodeIfPresent(Float.self, forKey: .y) ?? 0.0
        let z = try container.decodeIfPresent(Float.self, forKey: .z) ?? 0.0

        self.init(x, y, z)
    }
}

Answer: EDIT after the question was updated
It's true that you can't override SIMD3's codable behaviour in an extension. You can, however, try to wrap your SIMDs in trivial forwarder structs that then allow you to control their encoding:
struct Vector3 { // instead of the typealias
    private var simd: SIMD3<Float>
    var x: Float { return simd.x }
    var y: Float { return simd.y }
    var z: Float { return simd.z }
    // add forwarders for other SIMD methods/properties you use

    init(_ x: Float, _ y: Float, _ z: Float) {
        simd = SIMD3(x, y, z)
    }
}

extension Vector3: Codable {
    enum CodingKeys: String, CodingKey {
        case x, y, z
    }

    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        let x = try container.decodeIfPresent(Float.self, forKey: .x) ?? 0.0
        let y = try container.decodeIfPresent(Float.self, forKey: .y) ?? 0.0
        let z = try container.decodeIfPresent(Float.self, forKey: .z) ?? 0.0

        self.init(x, y, z)
    }
}

(for brevity, I've only included getters and Decodable)

This works as expected:
struct Obj1 : Codable {
   var values:[Int]
}

extension Obj1 {
    enum CodingKeys: String, CodingKey {
        case x, y, z
    }

    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        let x = try container.decode(Int.self, forKey: .x)
        let y = try container.decode(Int.self, forKey: .y)
        let z = try container.decode(Int.self, forKey: .z)

        self.values = [x, y, z]
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(values[0], forKey: .x)
        try container.encode(values[1], forKey: .y)
        try container.encode(values[2], forKey: .z)
    }
}

let obj = Obj1(values: [7, 42, 17])

do {
    let data = try JSONEncoder().encode(obj)
    print(String(bytes: data, encoding: .utf8)!) // "{"x":7,"y":42,"z":17}"

    let o = try JSONDecoder().decode(Obj1.self, from: data)
    print(o.values) // [7, 42, 17]
} catch {
    print(error)
}

When your implementations are never called, please show the code that encodes/decodes your obj1 instances.
Comment: OP claims that init(from:) is never called when added in the extension
Comment: I know, that's why I'm showing an example where it definitely _is_ called. Will amend my answer.
Comment: That will teach me to post simplified example code without testing it.  I'm updating my question with the actual code.
Comment: Updated my answer with a suggestion
Comment: Thanks, I've marked this as the correct answer. Do you know why simd can't be overridden?
Comment: AFAIK nothing in structs can be overriden (in the OO sense of the word) since you can't inherit from them.


Calculate sample variances from linear regression model for meta analysis?

Question: I want to perform a meta analysis in two levels using linear regression in R (with lm() ).
I understand that in the second level I should perform  a weighted regression with the weights being the inverse of the sample variances in the first level.
How do you calculate these inverse sample variances from the output of the first level regression model (lm() model).
Can you use the variances from the variance-covariance matrix as shown by vcov()? Why (not)? If not, how to calculate the sample variances?
To illustrate my problem I provide a simplified toy problem below.
In this problem I want to measure the effect of a study method used in different schools by different teachers. Teachers can teach in multiple schools.
In the first level I summarize the effect for each school (eliminate teacher effect) and in the second level I want to measure the effect of the method.
I'm aware you can solve this with mixed models but I want to know how to solve it through two level regression using lm() using inverse variance weights.
EDIT Note that the actual data I'm working with is rather technical and very large, hence the toy example.
The existing method of analysis on my data involves a mixed effect model but this is very slow given the amount of data.
There is a solution proposed to speed this up by splitting the mixed effect model in a two level fixed effect model (using 2lm())
This approach is less performant but still satisfactory and much faster.
I'm looking to increase the performance of the two level analysis by including the sample variances from level 1 lm() as weights in level 2 lm().
## generate some data
set.seed(101)
dat = data.frame(
  method = factor(c('a','a','a','a','a' ,'b','b','b','b','b'))
, school = factor(c('one', 'one', 'two', 'two', 'two', 'three', 'three', 'three',  'four','four'))
, teacher = factor(c('Joe', 'Jane', 'Joe', 'Jane', 'Jef', 'Joe', 'Jane', 'Jef', 'Joe','Jef'))
)
dat$outcome = rnorm(nrow(dat), mean = as.numeric(dat$method) + as.numeric(dat$teacher))
dat
#>    method school teacher  outcome
#> 1       a    one     Joe 3.673964
#> 2       a    one    Jane 2.552462
#> 3       a    two     Joe 3.325056
#> 4       a    two    Jane 2.214359
#> 5       a    two     Jef 3.310769
#> 6       b  three     Joe 6.173966
#> 7       b  three    Jane 3.618790
#> 8       b  three     Jef 3.887266
#> 9       b   four     Joe 5.917028
#> 10      b   four     Jef 3.776741
level1 = lm(outcome ~ 0 + school + teacher, dat, contrasts = list(teacher = 'contr.sum'))
summary(level1)
#>
#> Call:
#> lm(formula = outcome ~ 0 + school + teacher, data = dat, contrasts = list(teacher = "contr.sum"))
#>
#> Residuals:
#>        1        2        3        4        5        6        7        8
#> -0.30048  0.30048 -0.65925 -0.04748  0.70673  0.57972 -0.25300 -0.32672
#>        9       10
#>  0.38001 -0.38001
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> schoolfour    4.5028     0.5183   8.687 0.000967 ***
#> schoolone     2.9402     0.5183   5.672 0.004765 **
#> schoolthree   4.5600     0.3992  11.422 0.000335 ***
#> schooltwo     2.9501     0.3992   7.390 0.001788 **
#> teacher1     -0.6882     0.3441  -2.000 0.116099
#> teacher2     -0.3460     0.3441  -1.006 0.371493
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 0.6915 on 4 degrees of freedom
#> Multiple R-squared:  0.9882, Adjusted R-squared:  0.9706
#> F-statistic: 55.99 on 6 and 4 DF,  p-value: 0.0008178

## data is aggregated per school, removing the teacher effect
dat2 = subset(dat, teacher == 'Joe')
dat2$outcome = predict(level1,dat2)
level2 =lm(outcome ~ method, dat2)
summary(level2)
#>
#> Call:
#> lm(formula = outcome ~ method, data = dat2)
#>
#> Residuals:
#>        1        3        6        9
#> -0.00493  0.00493  0.02862 -0.02862
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> (Intercept)  3.97937    0.02053  193.81 2.66e-05 ***
#> methodb      1.58626    0.02904   54.63 0.000335 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 0.02904 on 2 degrees of freedom
#> Multiple R-squared:  0.9993, Adjusted R-squared:  0.999
#> F-statistic:  2984 on 1 and 2 DF,  p-value: 0.0003349

## use 1 / estimated variances from parameters in level1 model as sample variance
dat2$w = 1/diag(vcov(level1))[paste0('school',dat2$school)]
dat2
#>   method school teacher  outcome        w
#> 1      a    one     Joe 3.974444 3.722073
#> 3      a    two     Joe 3.984304 6.274351
#> 6      b  three     Joe 5.594250 6.274351
#> 9      b   four     Joe 5.537017 3.722073
level2_w =lm(outcome ~ method, dat2, weights = w)
summary(level2_w)
#>
#> Call:
#> lm(formula = outcome ~ method, data = dat2, weights = w)
#>
#> Weighted Residuals:
#>         1         3         6         9
#> -0.011940  0.009196  0.053379 -0.069304
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> (Intercept)  3.98063    0.01985  200.51 2.49e-05 ***
#> methodb      1.59231    0.02808   56.72 0.000311 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 0.06277 on 2 degrees of freedom
#> Multiple R-squared:  0.9994, Adjusted R-squared:  0.9991
#> F-statistic:  3217 on 1 and 2 DF,  p-value: 0.0003107

Comment: Perhaps http://www.metafor-project.org/doku.php/tips:rma_vs_lm_lme_lmer will help you.
Comment: @mdewey thanks for helping. In the link you provided. they calculate sample variance with `escalc` from the `metafor` package.
I'm not sure how it's don inside the function.
I really want to know, if possible, how to calculate it from a `lm()` model.
Answer: Under different circumstances, I would post this as a comment and not an answer...but as this is an important issue, I'm going to opt for the more "permanent" approach.
The answer is that you can't use lm() for the 2-level analysis (with or without the variance-adjusted weights)...or, perhaps it is better to say that you shouldn't want to use lm().
Briefly, if you include the 2nd-level variables in an lm model (with or without weights, but in the scenario I'm about to describe, you should actually want to use the weights), you can obtain a separate measure for the effect of each of the distinct 2-level units.  However, you entirely lose the ability to generalize to the larger population.  And, as this is the implied goal with any meta-analysis, you really shouldn't want to do this.  Said another way, if you use a fixed effects approach (lm()), then you can state something about how these particular 2nd-level units are/aren't distinct from each other.  However, you cannot generalize anything to the larger population.  It is the mixed-effects approach that allows you to make this generalization.
Happy to provide clarification as needed.
Comment: Thanks for your answer!  I'm not interested in generalisation to a population but in inference only on the experimental units.
I really want to try to calculate these sample variances in a two level fixed effect model.
I added a paragraph to the question explaining why I want to do that :)


Correct se for mean proportion in a survey with varying number of trials

Question: I am analysing fishing surveys using the survey package. My PSU is the person and I want to obtain estimates of the proportion of successful fishing trips. Each person has a variable number of fishing trips (trials) and successes.
I can use svyratio() to calc p.hat = mean(successes)/mean(trials)
or I can calculate p.i for each person as #succesess/#trials and use svymean() to get p.hat = mean(p.i)
For both approaches the SE calculation uses the #Persons (replicates/PSU's), not the total number of trials. I'm wondering if this matters?
Of the two approaches, I would prefer to use the latter approach (where p.hat = mean(p.i)), as this is what has been used on a different (non-survey) data set. However, do I need to calculate an alternative se to that obtained via svymean()? And if so, is it just:
p <- svymean(~Pi, s)[[1]]
se.p <- sqrt(p *(1- p)/N))
?
Thank you
Example data/code:
d <- data.frame(cbind(NTrips=round(runif(50, min=1, max=50),0), p=runif(50, min=0, max=1),
weights=rnorm(50,1000,500)))
d <- cbind(Person=paste("P",seq(1:50),sep="_"),d)
d$NSuccess <- d$NTrips*d$p
d$PropST <-d$NSuccess/d$NTrips

library(survey)
s <- svydesign(~1,data=d, weights=~weights)
N <- nrow(d)  # total number of replicates/PSUs

(pr <- svyratio(~NSuccess,~NTrips,s))
(se.pr <- sqrt(pr[[1]] *(1- pr[[1]])/N))
(pm <- (svymean(~PropST, s)))
(se.pm <- sqrt(pm[[1]] *(1- pm[[1]])/N))

Answer: Choosing which population characteristic to estimate
These two statistics you describe are not estimates of the same thing. Rather, they are estimates for two different population characteristics. Which population characteristic you care about will depend on the substantive research question you're trying to answer.
The population ratio $\frac{N_{successes}}{N_{trials}}$ is a essentially a summary of the average trial. This answers the question: "if a trial happens, how likely is it to be successful?"
The population mean $\sum_{i \in U} n_{successes,i} / n_{trials,i}$ is essentially a summary of the average person. This answers the question: "for the average person in the population, how likely is it that their average trial is successful?"
You can imagine these two measures being very different if there is a correlation between individuals' success rate and their number of trials. For example, there might be a few prolific fishers who make lots of fishing trips (i.e., "trials") and have lots of successes on those trips.
Standard errors
In both cases, the estimated standard errors from the survey package are reflecting the fact that you only sampled a portion of the persons in the population. Because you're ultimately interested in trips but your sampling units are persons, you effectively have a cluster sample. That's fine. As long as you make sure to take this clustering into account in your analysis, you should be accounting for most or all of your sampling variance.
In general, it is not a good idea to use the following code to estimate your standard errors. That formula is incoherent, because it's combining estimated proportions among trials with the denominator N of persons; in other words, it's confusing sampling units (people) and population units (trials).
(se.pr <- sqrt(pr[[1]] *(1- pr[[1]])/N))

Whichever population characteristic you decide you're interested in, for the standard error estimates it's best to just use the methods from the survey package, which are tailored to the sampling design.
(pr <- svyratio(~NSuccess,~NTrips,s))
(se.pr <- SE(pr))
(pm <- (svymean(~PropST, s)))
(se.pm <- SE(pr))

Recommended reading
Sharon Lohr's classic "Sampling: Design and Analysis" provides an excellent introduction to working with survey data with features such as stratification or clustering. It also has a nice discussion of population ratios vs. sample means and why you would estimate one or the other.


Corollary of Monotone Convergence Theorem

Question: Let $(g_n)$ be a sequence in $M^+$, then
$\int \left(\sum_{n=1}^\infty g_n\right)d\mu=\sum_{n=1}^\infty\left(\int g_nd\mu \right)$
proof: Let $f_n=g_1+\cdots+g_n$, then $f_n$ is a monotone increasing sequence of functions in $M^+$. I wan t use Monotone Convergence Theorem but I don't know how to guarantee that $f_n$ converges to $f=\lim_{n\to \infty}\sum_{i=1}^n g_n=\sum_{i=1}^{\infty}g_n$.
Comment: What does the word "entonces" mean? I don't think that it is English.
Comment: @Byron Schmuland. It is Spanish for "then".
Comment: @user254665 Thanks. You learn something new every day.
Comment: @ByronSchmuland. I just googled it.I didn't know either.
Answer: Your $(f_n)$ is an increasing sequence in $M^+$ and, by definition, it converges to $\sum_{n=1}^{\infty}g_n$ in the pointwise sense. Also, it is obviously bounded above by $\sum_{n=1}^{\infty} g_n$. Note that:

Pointwise convergence means that for each concrete $x$, $f_n(x) \to \sum_{n=1}^{\infty} g_n(x)$
It may happen that $\sum_{n=1}^{\infty} g_n(x) = +\infty$ for some $x$, but this still means that $f_n(x)$ converges to $+\infty$


What does it mean to obtain a sample $S$ of size $n$ according to a distribution $D$ over a set $X$ in machine learning?

Question: What does it mean to obtain a sample $S$ of size $n$ according to a distribution  $D$ over a set $X$ in machine learning?
Comment: Some more context would help.
Comment: What @user2974951 says. I don't understand the role of $S$ in particular. Are you asked to sample according to $D$ and call the resulting set of numbers $S$? Or is $S$ an integer, and you are supposed to sample $S$ realizations of $D$?
Comment: Re the edit that introduced "X:" you have changed the terminology in the question (but not in its title!) without really changing the question.  Now "X" refers to the population being sampled whereas originally you used "S" to refer to it, and now "S" refers to the sample.  This inconsistency will render both your question and my answer unintelligible, so I am taking the liberty of restoring your original notation while trying to respect the additional precision of the edit.
Answer: When the sampling is termed "random" it usually means you do the equivalent of the following, as described in more detail at https://stats.stackexchange.com/a/54894/919 and https://stats.stackexchange.com/a/96000/919:

Write down the name of each element of the set $S$ on one or more slips of paper (the "tickets").
Place these tickets into a box in such quantities that the proportion of tickets identified by any $\omega\in S$ equals the probability of $\omega$ assigned by the distribution $D.$
Repeatedly perform the following operation as many times as required $(n):$

Thoroughly mix the tickets.
Blindly withdraw one.
Record the element of $S$ indicated on that ticket.
Replace the ticket in the box.

The result of this process, the sample, is an ordered list of elements of $S.$  In order to proceed with analysis, usually some array of numbers, known in ML as "features," is associated with each element.  Consequently (when every element of $S$ has the same $k$ features) the sample can be represented as an $n\times k$ array of numbers.
Random sampling can be analyzed mathematically by considering the properties of the ticket-mixing process.  This is the basis for applying probability theory to statistics.  When the sample is not called "random," there is the possibility the tickets were withdrawn from the box without mixing them or even after peeking at them to make the selection.  Non-random samples are difficult or impossible to analyze mathematically.
The physical mechanism used to implement these steps may vary.  In modern lotteries the "tickets" are balls in a ball machine; Francis Galton invented a set of dice; but most often pseudo-random number generators in computers are exploited to conduct the sampling.


training and scoring of free throws in basketball

Question: In soccer, many coaches believe that it is impossible to train penalty shots, based on the argument that the pressure that is on a player who takes a penalty shot can not be simulated during training.
Essentially, there is little difference with free throws in basketball: the distance is always the same, the spectators expect the player to score.
My questions: do basketball players actively train for their free throws? How do they simulate the pressure on the player? What is the scoring percentage for free throws among the best teams? In soccer, the better players score 80 percent or higher.
Comment: Also be aware that in general, missing a penalty kick in a football game potentially has more consequences then missing a free throw in basketball.
Comment: That is absolutely correct, this is exactly why it is so surprising that soccer players practice so little at it (and most don't practice at all).
Comment: Here's an example of a basketball player who trains for his free throws: http://sports.stackexchange.com/a/2257
Answer: Free throws are trained for during basketball practice.  Other than players just standing around scoring, coaches will run drills to simulate game vibes.
A common drill is for a player to shoot 2 free throws and sprint to the other end to shoot 2 more.  For every one they miss they have to go there back and there.  Basically sprint a lap for each miss.  There are two keys to practicing free throws.

players must have the legs worn as they would be in a game.  This is obvious as you are not gaining much when shooting free throws fresh.  Your feet and hips play a big part of keeping your free throw straight.  If fatigued it is highly more likely that these will sway to left or right.  Your quads and knees will effect the shot long or short.  Again these are not effected much when you aren't fatigued.
player should only shoot two free throws at a time.  Players already shoot significantly better for their 2nd free throw so shooting more is not practical as the hardest one to hit is the first one.

So yes they train all the time.  Probably the biggest issue is how hard they fatigue themselves while training.  Often this isn't even close to game situation.  I actually remember watching Shaq practice free throws.  He would shoot 5 then "jog" down to the other end and shoot 5.  Well he would shoot 70-80% in practice and maybe better.  Easy to see he wasn't practicing right.    But again it may be hard to practice these perfect for pro players as this would hurt their resting.
But there is a big difference between free throws and penalty kicks.  They happen a lot more often and are often not in stressful situations.  There are only a very small percentage of free throws that are actually in clutch situations where in soccer each penalty kick is HUGE so we are comparing apples and oranges.  There are definitely psychological issues in basketball but it is more about muscle memory and technique.  For instance in soccer the player shooting can shoot with no goalie and hit close to 100% within 4 feet of the goal posts.  But he has to shoot against a goalie so the ball may need to be closer or it may need to be shot faster.  In basketball there is no other stress involved.  It isn't like if a free throw is shot too slow there would be a guy jumping up to block it.
What is funny is that in basketball 80% is a great percentage.  A team would be really happy shooting 80% and most are a good 5% points lower.  There are just more moving parts affecting a free throw and it is a much smaller goal even though the act itself is easier.
Comment: Thanks! I hadn't considered the effect of tired legs, this should be easy to make part of penalty kick drills in soccer as well.Indeed, the role of a penalty kick or a free throw is very different. Given that there are so few goals in soccer, missing a penalty kick is a BIG deal. That is why trainers claim that it is impossible to train for penalty kicks, b/c you can't simulate the pressure. However, it seems to me that frequent practice can make the person taking the penalty more experienced (he won't get the experience from penalty kicks in the game) and can help develop muscle memory.


How to access the full html of a drupal basic page?

Question: This is probably a dumb question. But I have not been able to find the full html of a basic page.
When I create a basic page, I can put a title and texts/images in the body. How can I access the ... portion of a specific basic page so that I can edit that part and put a js script? or better yet the full html of the page?
Thanks!
Answer: You will need to edit your themes tpl files to make changes to those areas.  If you are adding a js file you can add it in your themes template.php file. I would recommend you read through this link for
Drupal 7:
https://www.drupal.org/docs/7/theming
Drupal 8:
https://www.drupal.org/docs/8/theming


Improving template/rendering performance

Question: I'm running a Drupal commerce site and using Views to display up to 30  product node teasers per page.  The product display content type has a several fields (image, desc, manufacturer) and I am also using the flag module so that users can mark favorite products. As a result, the query ends up being fairly large with several joins.
Views statistics reveal that despite the complex query, the slow page loads are due to the rendering time it takes...
<code>Query build time    21.13 ms
Query execute time  1.67 ms
View render time    4237.2 ms
</code>
How do I improve my Drupal template/rendering performance to get my pages to load faster?
Comment: Are your views being cached, and are you using MEMCACHE or some similar optimiser?
Comment: No, views cannot be cached as each teaser rendered contains an add-to-cart form which doesn't work if cached. The HTML includes a form token specific to the original session / page view used to generate the HTML, it will be invalid once it's been cached.
Answer: I would check for File IO; what does profiling your site tell you. I had similar issues and created the http://drupal.org/project/imageinfo_cache module and my views performance improved a lot by reducing file IO.
Seeing that your on shared hosing, checkout this http://kpayne.me/2012/02/04/use-register_tick_function-to-profile-your-code/
Comment: Thank @mikeytown2. I'm also on Drupal 7.  Will check out the link.
Answer: Here are a couple of the more obvious tips:

Enable caching for your Drupal.
Optimize your Drupal database.
Log in to your Drupal admin area and go to Site Configuration > Performance. Besides enabling cache, you should also enable Page compression, Optimize CSS files and Optimize JavaScript files.
If your Drupal is hosted on a dedicated solution, you can modify the global MySQL settings for better performance.
*More for front-side micro-optimisation, you can check your website using this online tool. It will give you valuable information on what modifications can be made in order to improve its performance.
Of course, disable/remove all modules/blocks you are not using.

Now, most of those are pretty basic and usual optimisations. I assume this is where you are now standing.
Next steps:
In my (humble) opinion, you would get way better response times if you install APC or Memcache on your server. There is also varnish I have heard lots of good things about,  but I have not had the chance yet to play with it, and it looks like it is meant for anonymous sessions and static pages so I am not certain you would gain as much benefits than from other 2 above mentionned modules.
If you want to see benchmarks of a Drupal instance with/without APC caching, see here.
Last but not least, have you tried benchmarking without the flag module to see what impact this module has? We had to rewrite some queries of the like in other 3rd party modules a few times, some of the sql Drupal generates is not really optimised as you probably realised by now.
Happy coding, cheers!
Comment: Thanks for the well-written answer. This site is on a shared host unfortunately. So, I have no access to modify MySQL settings and cannot use APC on this particular host. A quick test of removing the flag module doesn't seem to have significant impact, but I'll have to do more thorough testing later.


Conjugate of elements in the Dihedral Group

Question: I was trying to do the following from a past exam of my Rings and Groups' professor

Classify all conjugacy classes of the elements in the dihedral group $D_n$ = $\{ 1,r,r^2, ... , r^{n-1} ,s ,rs ,r^2s , ... ,r^{n-1} \}$ = $\mathopen{<} r,s \ | \ r^n=1; s^2=1, (rs)^2=1   \mathopen{>}$

A brief reminder that $D_n$ can be understood from a geometrical point of view, $D_n$ is the group of "symmetries" of a regular $n$-gon.
Given two elements $x,a$ in a group G, we define the conjugate of $a$ through $x$ as the element $x^{-1}ax$
We define the conjugacy class of an element $a$ in a group G as $[a]=\{ x^{-1}ax  \ | \ x \in G  \}$
More specifically, I'm trying to calculate the following :

The conjugate of a rotation $r^k$ through a rotation $r^j$ and through a symmetry $r^js$.
Determine the conjugacy class of $r^k$ in $D_n$. How many elements does it have?
The conjugate of a symmetry $s$ through a rotation $r^j$ and through a symmetry $r^js$.
Determine the conjugacy class of $s$ in $D_n$. How many elements does it have?

Evidently through $r^j$ we have, $r^{-j}r^kr^j = r^{-j+k+j}=r^k$.
Now through $r^js$ we get, $(r^js)^{-1}r^k(r^js) = s^{-1}r^{-j}r^kr^js = s^{-1}r^ks$.

I don't know how to classify this last one, and the ones from 3.
Answer: It should be sufficient to solve this to know that
$$s^{-1}=s$$
which you can see from $s^2=1$, so $s^2s^{-1}=s^{-1}$, and since
$$rsrs = 1$$
we have
$$srs = r^{-1}$$
so
$$sr = r^{-1}s$$
I'm not going to answer all of the questions, but this should be enough to do it yourself.
Comment: Could you please explain why those equalities are true?
Comment: @M.Navarro Edited, take a look.
Comment: Obviously, thank you so much. I'll update my answer when I make some progress. Otherwise I'll come back to ask for help