Computing a convolution using FFT

Question: I have two sequences of the same length, $(x_i), i=1, 2, \ldots, N$ and $(y_i), i=1, 2, \ldots, N$ and a function $K(t) = -t \times \exp(-t^2 / 2)/ \sqrt{2 \pi}$.
I need to compute the following quantity for each $m=1, 2, \ldots, N$:
$\sum_{j=1}^N K(x_m - x_j) \times y_j$
which is a tad slow when done directly (I need it when $N = O(10^4)$ ). I know this can be much improved on with the use of FFT, however it is something I never really worked with. Could anyone suggest any links or a way to rewrite it in FFT form?
I'd be very grateful for any help :)
Comment: Apply the FFT two both sequences, multiply the results pointwise, then transform back.
Comment: I'm not sure if this is precisely what you are talking about but naive convolution is $O(n^2)$, while the FFT method is only $O(n \log n)$ (it is 2 FFTs, which are $O(n \log n)$, and one pointwise multiply, which is $O(n)$, so the process is $O(n \log n)$ overall.)
Comment: Well that's the general idea. Except that to do it for each $m$ is no time gain. I was guessing the clever thing to do is to approximate the function $\sum_{j=1}^N K(s - x_j) y_j$ using FFT, and this bit I'm not sure how to do. Besides, the FFT theorem convolution applies directly to convolutions like $\sum a_{i-j} b_j$, so to do the straightforward thing I would need to recompute the sequence $K(x_m-k_j)$ for each $m$, which would eat away any time gain..?
Comment: I've not heard of approximating a function by its Fourier transform. You have $$ (K \ast y) (x_m) = \sum_{j = 1}^N K(x_m - x_j) y_j$$ and $$\widehat{K \ast y} (x_m) = \hat{K}(x_m) \hat{y_m}$$
You can use FFT to compute these: do the multiplication and then do the inverse transform for each $m$.
I don't understand how to use the Fourier transform to approximate a function. The other thing you could think about is to numerically approximate it: compute only a few of the function values and then use interpolation.
Answer: Here you have a good link.
Basically you need to compute the FFT of each signal individually, multiply the spectrums and the do the inverse FFT of the resulting sectrum. That will be the result of the convolution.
Good luck!
Answer: Need to account for wrap around issues
1) Shift the K sequence such that it is centered in the middle of the array call K1 of lenght n1
2) Store the K1 array in a new array K2 with double the size of K1 n2 = 2Xn1 such that
   K2(1 to n1 ) = complex(K1,0) and K2( n1+1 to n2 ) = 0 where K2 is complex
3) Next shift the K2 array over to the left by n1/2-1 this will center the maximum of
   the K function in the first bin of the K2 array
4) Store the y data in an array y2 of of lenght n2 such that y2(1 to n1 ) = complex(y,0) and
   y2(n1+1 to n2) = 0 where y2 is complex
5) calculate the FFT of K2, i.e., FFT_K2 = FFT( K2, 1 )  1 => exp(iwt)
6)  calculate the FFT of y2, i.e., FFT_y2 = FFT( y2, 1 )  1 => exp(iwt)
7) Let FFT_d2 = CONJUGATE(FFT_K2) * FFT_y2
8) Inverse FFT, i.e.,  d2 = FFT( FFT_d2, -1 )  -1 => exp(-iwt)
9) convolution = REAL( ds(1 to n1) )
10) this assumes the exp(-iwt) has the 1/n factor

Computing a convolution using FFT


Which file format should I use to save an array to a file?

Question: I am using the solution found here: Save NSArray to File and I am just wondering what type of file format I should save my array as.  Can I just use .txt?
Answer: Any format. iOS writeToPath: does not read the extension. You can even make up weird file names.
Take the codes from your reference question as example,
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *filePath = [documentsDirectory stringByAppendingPathComponent:FILE_NAME];

[myArray writeToFile:filePath atomically:YES];

the FILE_NAME could be in any ASCII values.
Answer: The writeToFile of an array object creates a property list file. So, while you can use whatever extension you want, it's generally good style to use the plist extension.


Retrieve data from webpage to Blue Prism

Question: I am working on creating a bot to get a list of drugs from webpage.
The first step is to enter Start Date and End Date and hit the Submit button.
The page opens a list of drugs and my next step is to retrieve the list of drugs in a table format.
Is there a way I could get data from webpage to excel or Blue Prism collection?
Could someone please suggest a way?
Answer: The ideal way to accomplish this would be to spy the table on the page and use a Read stage with the action "Get Table", which will load the data into a collection.


disabled multiple check box when i have select drop down list

Question: I am new for  disabled multiple check box when i have select drop down list  .using java script.
Here code source i have that are not better approach all fields disabled.
function select(Seect)
{
  if(datatype=="varchar")
  {

     document.getElementById("field").disabled=true;
  }
}

strong text
Answer: Here is the demo code for disable all checkbox of the page using jquery. try below code...
function select(Seect)
{
   if(<your condition>)
   {
      $('input[type="checkbox"]').attr("disabled","disabled");
   }
}

let me know if you have any issue....


bazel build tensorflow on windows 10 getting cudnn.h- system cannot find the file specified

Question: I keep getting this error when trying to build the tensorflow-gpu using bazel and python in Windows 10:
Cuda Configuration Error: Error reading C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/cuda/include/cudnn.h: ja
va.io.IOException: ERROR: src/main/native/windows/processes-jni.cc(239): CreateProcessW("grep" --color=never -A1 -E "#de
fine CUDNN_MAJOR" "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/cuda/include/cudnn.h"): The system cannot fin
d the file specified.

This is the command I'm trying to run:
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

I've confirmed the C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/cuda/include/cudnn.h is there.
I've tried running it from VS2015 x64 Native Tools Command Prompt, cmd, and powershell and get the same error.
I'm using bazel 0.16.1, CUDA 9.2, Anaconda3 (Python 3.6.5), and CUDNN 7.2.1. I "installed" the CUDDNN files by unzipping its cuda folder into my C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2 folder (i.e. the whole "cuda" folder).  I specified the C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\cuda path in the cudnn path question when I ran the configure.py.  The configure.py completes without error.
I also tried putting the CUDNN files directly in the C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2 folder (instead of a cuda folder in there) and specified the default location and still get basically the same error:  "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/include/cudnn.h": The system cannot find the file specified.
I've added these to my path (and rebooted):
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp
C:\tools\msys64
C:\tools\bazel
C:\tools\bazel\bazel.exe
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\extras\CUPTI\libx64

These are the system variables I've set:
BAZEL_SH  C:\tools\msys64\usr\bin\bash.exe
BAZEL_VC  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC
BAZEL_VS  C:\Program Files (x86)\Microsoft Visual Studio 14.0

This is the full error:
ERROR: Skipping '//tensorflow/tools/pip_package:build_pip_package': error loading package 'tensorflow/tools/pip_package'
: Encountered error while reading extension file 'cuda/build_defs.bzl': no such package '@local_config_cuda//cuda': Trac
eback (most recent call last):
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 1458
                _create_local_cuda_repository(repository_ctx)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 1185, in _create_local_cuda_repository
                _get_cuda_config(repository_ctx)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 909, in _get_cuda_config
                _cudnn_version(repository_ctx, cudnn_install_base..., ...)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 584, in _cudnn_version
                find_cuda_define(repository_ctx, cudnn_header_dir, "c...", ...)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 534, in find_cuda_define
                auto_configure_fail(("Error reading %s: %s" % (str(h...)))
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 315, in auto_configure_fail
                fail(("\n%sCuda Configuration Error:%...)))

Cuda Configuration Error: Error reading C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/cuda/include/cudnn.h: ja
va.io.IOException: ERROR: src/main/native/windows/processes-jni.cc(239): CreateProcessW("grep" --color=never -A1 -E "#de
fine CUDNN_MAJOR" "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/cuda/include/cudnn.h"): The system cannot fin
d the file specified.

WARNING: Target pattern parsing failed.
ERROR: error loading package 'tensorflow/tools/pip_package': Encountered error while reading extension file 'cuda/build_
defs.bzl': no such package '@local_config_cuda//cuda': Traceback (most recent call last):
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 1458
                _create_local_cuda_repository(repository_ctx)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 1185, in _create_local_cuda_repository
                _get_cuda_config(repository_ctx)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 909, in _get_cuda_config
                _cudnn_version(repository_ctx, cudnn_install_base..., ...)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 584, in _cudnn_version
                find_cuda_define(repository_ctx, cudnn_header_dir, "c...", ...)
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 534, in find_cuda_define
                auto_configure_fail(("Error reading %s: %s" % (str(h...)))
        File "C:/tensorflow/third_party/gpus/cuda_configure.bzl", line 315, in auto_configure_fail
                fail(("\n%sCuda Configuration Error:%...)))

Cuda Configuration Error: Error reading C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/cuda/include/cudnn.h: ja
va.io.IOException: ERROR: src/main/native/windows/processes-jni.cc(239): CreateProcessW("grep" --color=never -A1 -E "#de
fine CUDNN_MAJOR" "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/cuda/include/cudnn.h"): The system cannot fin
d the file specified.

Any ideas what I could have done wrong, or what I might have missed, or what I could check next?
Answer: The problem is a bug in cuda_configure.bzl: it uses ctx.execute instead of ctx.action.run_shell. If you'd be so kind to file a bug to TensorFlow, that'd be great!
To work around it: add c:\tools\msys64\usr\bin to your PATH. That's where grep.exe is so ctx.execute can find grep on the PATH.
Comment: Thank you so much, that got me past that error!  I'll add the bug to Tensorflow for this.
Comment: The answer is great! But I wonder why google still has not solved this problem until now (2018 Oct. 29th).


What is the use of SINUIViewController in the demo for Sinch?

Question: I'm new to Sinch. I just downloaded the demo file for calling and I'm just curious as to what this file does. Specifically the last few lines below the file. It seems to me that this view controller is involved with showing the appropriate view controllers when the call ends and starts, but I'm not sure.
Thanks!
#import "SINUIViewController.h"

#import <objc/runtime.h>

// used for associated object references to simulate property-like storage for
// this category
static char sin_deferredDismissalKey;

@implementation SINUIViewController

- (void)didReceiveMemoryWarning {
  [super didReceiveMemoryWarning];
  if ([self.view window] == nil) {
    _isAppearing = NO;
    _isDisappearing = NO;
  }
}

- (void)viewWillAppear:(BOOL)animated {
  _isAppearing = YES;
  [super viewWillAppear:animated];
}

- (void)viewDidAppear:(BOOL)animated {
  [super viewDidAppear:animated];
  _isAppearing = NO;
  [self dismissIfNecessary];
}

- (void)viewWillDisappear:(BOOL)animated {
  _isDisappearing = YES;
  [super viewWillDisappear:animated];
}

- (void)viewDidDisappear:(BOOL)animated {
  [super viewDidDisappear:animated];
  _isAppearing = NO;
}

#pragma mark - Dismissal

- (void)dismiss {
  if ([self isDisappearing]) {
    return;
  } else if ([self isAppearing]) {
    [self setShouldDeferredDismiss:YES];
    return;
  }
  [self dismissViewControllerAnimated:YES completion:nil];
}

- (void)dismissIfNecessary {
  if ([self shouldDeferrDismiss]) {
    [self setShouldDeferredDismiss:NO];
    dispatch_async(dispatch_get_main_queue(), ^{
      [self dismiss];
    });
  }
}

- (BOOL)shouldDeferrDismiss {
  return [self sin_getAssociatedBOOLForKey:&sin_deferredDismissalKey];
}

- (void)setShouldDeferredDismiss:(BOOL)v {
  [self sin_setAssociatedBOOL:v forKey:&sin_deferredDismissalKey];
}

#pragma mark -

- (BOOL)sin_getAssociatedBOOLForKey:(const void *)key {
  NSNumber *v = (NSNumber *)objc_getAssociatedObject(self, key);
  return v ? [v boolValue] : NO;
}

- (void)sin_setAssociatedBOOL:(BOOL)v forKey:(const void *)key {
  objc_setAssociatedObject(self, key, [NSNumber numberWithBool:v], OBJC_ASSOCIATION_COPY);
}

@end

Answer: This class is just a base-class helper viewcontroller, help you dismiss viewcontroller more easier. Some viewcontroller will be inherrited from this class.
- (BOOL)sin_getAssociatedBOOLForKey:(const void *)key
- (void)sin_setAssociatedBOOL:(BOOL)v forKey:(const void *)key

These above methods use a technique called Associated Object, to avoid declaring a property. You can read more about Associated Object here http://nshipster.com/associated-objects/


c# mysql insert query, if record exists then update

Question: I am trying to insert the new records in table, I have two columns
      user_records
 ------------------------
| Attribute |  Datatype  |
|------------------------|
| user_id   | varchar(25)|
| count     | int        |
 ------------------------

Query :
INSERT into user_records Values("nix_1", 0)

but if the user already exists, then it should increase count by 1
Answer: You would have check if the record exists before attempting to INSERT. Your query should be something in line with below. Example works in SQL Server.
DECLARE @sUserID VARCHAR(25) = "nix_1";    // This should be the parameter passed on into the stored procedure

IF EXISTS(SELECT user_id FROM WHERE user_id = @sUserID)
    // Increase the count
    UPDATE user_records SET count = count  + 1
        WHERE user_id = @sUserID
ELSE
    // Insert new record
    INSERT INTO user_records Values (@sUserID , 0)

Please refer to the below links for info on EXIST
EXISTS (Transact-SQL)
SQL EXISTS Operator
Difference between EXISTS and IN in SQL?
Answer: Why not use the INSERT ... ON DUPLICATE KEY syntax?
INSERT INTO user_records (user_id, count) VALUES ('nix_1', 0)
ON DUPLICATE KEY UPDATE count = count + 1;

http://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html
Comment: I tried `INSERT INTO user_records VALUES('nix_1', 0) ON DUPLICATE KEY UPDATE count = count+1` but it gives error `Unrecognized keyword (near ON)`, `Unrecognized keyword (near DUPLICATE)`, `Unrecognized keyword (Near Key)`.
Comment: Which version of MySQL and which data engine?
Comment: I am using xampp with sql version `Database client version: libmysql - mysqlnd 5.0.12-dev` and XAMPP `Version 3.22`


Cannot seem to get mouseover event to work with mapbox

Question: I'm having a bit of trouble figuring out why the mouseover event doesn't work with mapbox gl.
map.on('load', function() {
  var geoJson = '{
    "type": "FeatureCollection",
    "crs": {
        "type": "name",
        "properties": {
            "name": "urn:ogc:def:crs:OGC:1.3:CRS84"
        }
    },
    "features": [
    {
        "type": "Feature",
        "properties": {
            ...
        },
        "geometry": {
            "type": "Point",
            "coordinates": [
                -118.6059,
                34.1829
            ]
        }
    }]
  }';

  map.addSource('someData', {
    type: 'geojson',
    data: geoJson,
    cluster: true,
    clusterMaxZoom: 14
  });

  map.addLayer({
    'id': 'unclustered-markers',
    'type': 'symbol',
    'source': 'someData',
    'layout': {
        'icon-image': 'circle-11'
    }
  });
});

So that part works and the coordinates are displayed on the map within the cluster. However, when I try the mouseover event, nothing happens.
map.on('mouseover',function(e) {
    console.log(e);                 // nothing is logged when I hover over the map or the points
    var features = map.queryRenderedFeatures(e.point, { layers: ['unclustered-markers'] });
    ...
});

If I change that event to click then the event is logged to the console when I click on the map.
Answer: GL JS does not fire the mouseover event despite firing mousemove and moseout events. I don't think there is a reason why we do not fire mouseover. We should add this event.
Answer: It seems that you are using the wrong event name. Note that is it mousemove, not mouseover. To fix this, change your code to:
map.on('mousemove',function(e) {
    console.log(e);
    //....
});

Here is a working example: https://jsfiddle.net/kmandov/o870ufLr/
Open the console to see the events.
Comment: Oh wow haha thanks for pointing that out! I cannot believe I did that. For some reason I kept seeing or thinking about mouseover.... Thank you
Comment: Strictly speaking, `mousemove` is not the same as `mouseover`.
Answer: There is also mouseenter and should be considered depending on your application. In case you have stumbled upon this question because Mapbox was giving you erratic behavior with missing popups, here is a work around to guarantee the popup will be above the feature the user hovered over.
Use Mapbox to handle the event and get the mouse coordinates like so:
map.on('mouseenter', 'layerID', (event) => {
    var x = event.originalEvent.clientX
    var y = event.originalEvent.clientY

Well then, it's a matter of using your favorite DOM handler and adding a
 for the popup. Set the position of the popup to where the user hovered like so :
{left : `${x}`, top : `${y}`, position : 'absolute', zIndex : 1}


The 1-dimensional Hausdorff measure of a curve in the plane

Question: For a set $X\subseteq\mathbb{R}^2$, let $H^1(X)$ be its 1-dimensional Hausdorff measure. Suppose $X$ is a regular curve (say, a graph of a continuous function $f:\mathbb{R}\to\mathbb{R}$). In that case, does it hold that $H^1(X)$ is simply the length of the curve, as can be calculated by standard integration?
And in case the curve is non-regular - for example, if it is the range of a Brownian motion or a space-filling curve - in that case will $H^1(X)$ be $\infty$?
And last one: if I am not mistaken, both the range of a Brownian motion and a space-filling curve are of Hausdorff dimension 2; but do they differ in that that the 2-dimensional Hausdorff measure of the former is 0 and of the latter is the area of the filled space?
Answer: First question, yes:
"In any metric space$(X,d)$, if $\gamma:[0,1]\longrightarrow X$ is an injective Lipschitz function, then $H_1(\gamma([0,1])$ is the length of the curve."
See https://www.encyclopediaofmath.org/index.php/Hausdorff_measure.
For the third question, see "Relation with Hausdorff dimension" in the same site.
Comment: Thanks; I guess this means the answer for the first question is *almost* yes - there are assumptions to make about the function (it has to be Lipschitz, probably...)
Answer: In order for the 1-dimensional Hausdorff measure of a curve in $\mathbb R^n$ to be equal to its length, the curve has to be BETTER than continuous. Lipschitz curves have these property, but the curves in the set of positive Wiener measure (related to the Brownian motion) are not that nice.
Note that almost all these curves are $C^{0,a}$, for $a \le 1/2$.


Adding a WCF Service Reference falls back to XmlSerializer

Question: We are consuming an ASMX service by adding a WCF "Service Reference" to our project. When we do this, by default it is supposed to use the DataContractSerializer and if something goes wrong, it will fall back to the XmlSerializer.
I've tried forcing the DataContractSerializer when generating the proxy classes, but when I do that, they are incomplete and missing all of the custom classes used by the webservice (leaving only the interface for the Soap, SoapChannel, and the SoapClient class).
Well, something is going wrong and it is falling back to the use the XmlSerializer. I do not see any errors or warnings when I generate the reference.
How can I find out what is causing the DataContractSerializer to fail and fall back to the XmlSerializer?
Comment: I find it easier to use wsdl.exe to create proxy classes.
Answer: Long story short is that we were unable to force VS to use the DataContractSerializer. Instead we ended up writing our own WCF Service Contracts that represented the webservice. When we consume the service we are instead creating the ChannelFactory generically by using our OWN Service Contracts.  Below is the code that we used to create the channel.
 ///
 /// A generic webservice client that uses BasicHttpBinding
 ///
 /// <remarks>Adopted from: http://blog.bodurov.com/Create-a-WCF-Client-for-ASMX-Web-Service-Without-Using-Web-Proxy/
 /// </remarks>
 /// <typeparam name="T"></typeparam>
 public class WebServiceClient<T> : IDisposable
 {
     private readonly T channel;
     private readonly IClientChannel clientChannel;

     ///
     /// Use action to change some of the connection properties before creating the channel
     ///
     public WebServiceClient(string endpointUrl, string bindingConfigurationName)
     {
         BasicHttpBinding binding = new BasicHttpBinding(bindingConfigurationName);

         EndpointAddress address = new EndpointAddress(endpointUrl);
         ChannelFactory<T> factory = new ChannelFactory<T>(binding, address);

         this.clientChannel = (IClientChannel)factory.CreateChannel();
         this.channel = (T)this.clientChannel;
     }

     ///
     /// Use this property to call service methods
     ///
     public T Channel
     {
         get { return this.channel; }
     }

     ///
     /// Use this porperty when working with Session or Cookies
     ///
     public IClientChannel ClientChannel
     {
         get { return this.clientChannel; }
     }

     public void Dispose()
     {
         this.clientChannel.Dispose();
     }
 }


Why is a reliced guitar often more expensive than a corresponding normal model?

Question: Guitar makers like Fender or Gibson offers a reliced model of a guitar, and set the price higher than the original one. I'm not referring to "Collector Edition" or "Artist Series", but a guitar like the Fender Stratocaster Standard being reliced and this makes the price higher than the normal one.
What makes it more expensive? Isn't it made with the same specification?
PS: Merry Christmas!
Answer: This answer contains some speculation, but the relicing process actually involves some work, and possibly they are charging for that. People might also be prepared to pay more for a factory worn guitar (which is quite strange).
Usually, as you have noticed, the reliced instruments are in the higher end, custom shop made range. The manufacturers might want to use this to push up the prices on normal range reliced instruments too. People have their mind set on a custom shop, but realize that they can't afford it, and then they might accept a higher priced normal guitar.
Answer: It's all about supply and demand. This is no different than a blue jean company charging more for a pair of worn out faded jeans than they charge for a regular pair, just because they can.
 This case has another twist. Many people don't want to look like a newbie with a shiny new guitar. It is very chic to be able to look like your guitar shows years of dedicated playing.
Also, take into consideration that the relicing is another step of construction that someone needs to get paid to do. These are not monkeys with sledgehammers. These are people carefully scratching paint and whatnot to approximate years of use. That means realizing where these instruments usually wear out and to what degree.


AWS Elasticache security groups for inter cluster communication

Question: I don't really have a setup to test this, but if I create an elasticache redis cluster with more than 1 node, how exactly would a security group have to look like to be very secure but without breaking the cluster itself?
Lets say I create a security group that allows ingress from itself and my kubernetes nodes on port 6379 and allow all on egress to itself and kubernetes.
Like this for example:
<code>resource "aws_security_group" "tools_elasticache_default" {
  name        = "tools-elasticache-default"
  description = "Allow traffic from tools cluster to elasticache instance"
  vpc_id      = module.tools_cluster.vpc_id

  ingress {
    description = "Incomming redis traffic"
    from_port   = 6379
    to_port     = 6379
    protocol    = "tcp"
    self        = "true"
    security_groups = [for x in module.tools_cluster.node_security_groups : x.id ]
  }

  egress {
    description = "Outgoing redis traffic"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    self        = "true"
    security_groups = [for x in module.tools_cluster.node_security_groups : x.id ]
  }

  tags = merge(var.tags, {
    "shared" = "true"
  })
}
</code>
Would this break my elasticache cluster since it is ec2 under the hood and security groups are on a per instance base? Since I haven't explicitly specified redis cluster communication ports like those stated here?
To my understanding the cluster should be broken since one redis node could egress to another node on port 3333 but his request would be dropped on the missing ingress rule for it on the other cluster participant.
Or is AWS implicitly managing those rules and makes sure the ports for inter cluster communication are always allowed?
Any help would be greatly appreciated.
Thanks!
Answer:
Or is AWS implicitly managing those rules and makes sure the ports for intra cluster communication are always allowed?

Yes.


Given a countable orthonormal set in a Hilbert Space, a convergent sequence in the span of the set implies convergence of coefficients.

Question: The problem is from Naylor's Linear Operator Theory in Engineering and Science

(Section 5.17, Exercise 19) Let $B = \{x_1, x_2, . . . \}$ be an orthonormal set in a Hilbert space
  $H$. Then let ${z_n}$ be a convergent sequence in $H$, where each $z_n$ is a finite linear combination of points
  in $B$, that is, $z_n = α_{n1}x_1 + α_{n2}x_2 + · · · + α_{nk_n} x_{k_n}$
  . Show that for each $j = 1, 2, . . .$ the sequence $\{α_{nj}\}$ is
  convergent, that is,
  $$α_j = \lim_{n→∞}α_{nj}$$ and that $$\lim_{n \to \infty}z_n = \lim_{n \to \infty} \sum_{j=1}^{k_n}α_{nj}x_j = \sum_{j=1}^{∞} α_jx_j .$$

I'm been stuck on this for a while since I can't say $B$ is a basis for $H$. Also I know that since $z_n$ is convergent, $\sum_{j=1}|α_{nj}|^2<\infty $, but I don't see where I can use this.
I was thinking of using the convergence of $z_n$ to some $z \in H$ to get the convergence of $α_{nj}$.
Comment: You can assume it's basis because everything takes place in the Hilbert space $H' = \overline{\operatorname{span}}\{x_1,x_2,\ldots\}$.
Answer: Let $z_n \to z$. Note that $\alpha_{nj}=\langle z_n , x_j \rangle$. Hence $ \alpha_{nj} \to \langle z, x_j \rangle$ for each $j$ as $ n \to \infty$. Thus $\alpha_j=\langle z, x_j \rangle$. Now $z$ belongs to the closed subspace spanned by $x_1,x_2,...$. The sequence $(x_j)$ is an orthnormal basis for this space. Hence we can write $z=\sum_j a_j x_j$. Now $\langle z , x_j \rangle =a_j$ so $\alpha_j=a_j$ for  all $j$. This finishes the proof.
Answer: Note, for all $j$, that $\langle\cdot,x_j\rangle:H\to\mathbb{K}$ is continuous. Furthermore, for all $n$ and $j$, we have $\langle z_n,x_j\rangle=\alpha_{nj}$, because $B$ is orthonormal. Hence, since $z_n$ converges, so does $\alpha_{nj}$, for all $j$, to some $\alpha_j$.
Furthermore, if $z$ is the limit of $z_n$, from the continuity we also get $\langle z,x_j\rangle=\alpha_j$, for all $j$. For all $n$ and $j$ we get $\langle z-z_n,x_j\rangle=\alpha_j-\alpha_{jn}$. Since $B$ is orthonormal, this gives $\sum_{j=1}^\infty(\alpha_j-\alpha_{nj})^2\leq\|z-z_n\|^2$, for all $n$.
In particular, the sum converges for all $n$, and since the sequence $(\alpha_{nj})_{j\geq1}$ is finite, this implies that $\sum_{j=1}^\infty\alpha_j^2$ also converges. Since $B$ is orthonormal, $\sum_{j=1}^\infty\alpha_jx_j$ also converges to some vector $v$.
Finally, for all $n$ we have $\|v-z_n\|=\|\sum_{j=1}^\infty(\alpha_j-\alpha_{jn})x_j\|=\sum_{j=1}^\infty(\alpha_j-\alpha_{nj})^2$, which tends to zero as $n$ tends to infinity. Since $z_n$ converges to both $z$ and $v$, we have $z=v=\sum_{j=1}^\infty\alpha_jx_j$.


Is WEP used in conjunction with a wireless access list adequate security?

Question: I use a Netgear wireless router, with various wireless devices connecting to it.  One of my wireless devices doesn't support WPA2 security, so I had to downgrade the security on the router to WEP.
We all know WEP is broken, so as an added measure I enabled a wireless access list on the router so that only devices with specified MAC addresses which are in my access list are permitted to connect to the router.
I know it is possible to spoof a MAC address from a device for the purposes of accessing a secure network like this.  But is it easy?  Is using WEP and a wireless access list good enough to prevent most hacking attacks?  Or should I do whatever I can to ensure all devices support WPA2 in the future?
Answer: No.
WEP is trivial to break. MAC addresses aren't secure. They can be trivially spoofed. Take a look at The six dumbest ways to secure a wireless LAN:

MAC filtering: This is like handing a security guard a pad of
  paper with a list of names. Then when
  someone comes up to the door and wants
  entry, the security guard looks at the
  person's name tag and compares it to
  his list of names and determines
  whether to open the door or not.
Comment: You can easily read it when the network is used so it's only secure so long as noone actually uses your network.
Comment: But how easy is it for a hacker to find out what MAC addresses are in the access list on a router, so they know what address to spoof?
Comment: Airsnort can get you both the MAC and the WEP pretty quickly, from what I've read/heard.
Comment: @LeopardSkinPillBoxHat: The hacker just needs to look for frames originating from the router. Anything that the router is responding to logically must be a permitted client.
Comment: The next step to this question then is what is an adequate way to secure a wireless network?
Comment: For someone to spoof the MAC, they need to observe traffic from an intermittently connected client, right?  They'll need to be around when the client is active (to snoop).  But if they use it then, you'll have a collision, detectable at both your router and your legit client.  So they'd have to wait until your client is gone.
Answer: WEP is easy to crack (there are videos of it being done in 10 mins, etc) and MAC addresses can be spoofed. However, what are you 'securing'? If this is a business then NO, this is definitely not enough security. If it's your home internet, and your 10 neighbors all leave their wireless open, and you have a software firewall on your computer, then maybe this is enough.
It's like when I go hiking in the woods with my girlfriend. I'm not scared of the bears... I might not be able to outrun them, but I can at least outrun her!
Comment: +1 Made my morning a lot more enjoyable ^^
Answer: No WEP is not adequate security. Mac addresses are sent as part of the wireless packet and therefore easy to intercept and spoof.
If you need to use WEP you need to install a 3 NAT router setup as shown below.
NAT router 1 (NR1) is connected to the internet on the WAN port.
NAT router 2 (NR2) WAN port is connected to NR1 LAN port.
NAT router 3 (NR3) WAN port is connected to NR1 LAN port.
<code>                       ----------NR2--------------Wireless WEP Unsecure
                       |
Internet  ----- NR1----|
                       |
                       |
                       ----------NR3--------------Wireless WPA Secure
</code>
This will allow the WEP access to get to the internet. The NR3 router would block access to your secure network. One other advantage would be to allow guest access to NR2. That would allow Internet access without providing access to your local secure network.
Comment: Tony, there's a bug in your markdown, you need to indent that third line to get it to render like the two above it.
Answer: Technically it's not very safe but I guess it depends on where you are; if you're in a building full of techies then it's really not safe, if you're in on a private island in the middle of pacific then it's probably overkill to have any security. You have to measure your need first.
Answer: At best you are stopping casual intrusion.  Using WEP, white-listing particular MAC addresses,  disabling DHCP, and disabling broadcast of SSID create a few hoops for people to jump through, but they're all trivial to defeat.
If you need to support WEP, then go ahead and take these steps, but also accept that the network is insecure.  You'll want to isolate it from the rest of your LAN, using a multi-router setup as described in Steve Gibson's guide.  Basically, you put one router running WEP on the "outside", connected to your upstream internet connection.  Intruders may get on that wireless segment.  You then connect a second "inner" router to one of the LAN ports of the outer router.  Connect everything that doesn't need WEP to that router.
Answer: No, MAC address protection is not enough as MAC addresses are easily spoofed - http://www.think-security.com/wireless-wep-insecurity/ . Even if the hacker cannot spoof it, he can at least sniff your traffic.


Alocação dinâmica de memória, sem uso, não consome memória?

Question: Estava fazendo uns testes no meu programa que aloca dinamicamente uma quantidade de 4*N bytes na memória, para ver como estava o consumo de RAM no gerenciador de tarefas. Eu estava sempre fazendo os testes inserindo valores, preenchendo o espaço que aloquei.
Para fazer os testes executarem mais rápido, decidi retirar a inserção dos valores, pensando que o consumo de RAM seria o mesmo, já que o espaço alocado seria o mesmo. Para minha surpresa, ao abrir o gerenciador de tarefas, o consumo de RAM era praticamente igual ao consumo na inicialização do programa.
Então decidi fazer um simples código para confirmar isso:
int main() {
    int * ptr = new int[1000000]; // O consumo deve ser superior à 3900KB

    int i;
    cin >> i;
}

Ao executar, podemos ver no gerenciador de tarefas que o programa consome nada de memória. Como isso é possível se eu aloquei espaço!? Então aqui estão minhas dúvidas quanto à isso:

Por que não há consumo de memória se não inserir valores?
O que ocorre com o espaço alocado, enquanto a stack ainda está viva?
Quem é responsável por isso, runtime ou OS?
Comment: Forçando a utilização do espaço alocado normalmente já da para ver alguma coisa a mexer: https://i.stack.imgur.com/17NLF.gif
Comment: @Isac Sim, foi isso que disse na minha pergunta. Quando insere valor no espaço alocado, o GM mostra um maior consumo. Mas enquanto isso não ocorre, o consumo é o mesmo de se não houvesse alocado qualquer espaço.
Answer: Gerenciador de tarefas
Gerenciador de tarefas não é uma ferramenta para olhar consumo real de memória.
O primeiro erro que as pessoas vão achar olhando para ele é que o sistema operacional consome muita memória. E sim, ele consome, ele põe tudo o que pode na memória porque isso é otimização, o que está na memória é acessado mais rápido. Isso é feature, das boas, não bug como as pessoas acham.
E é uma das maiores fake news espalhadas pela internet e por isso tanta gente na área não tem a menor ideia do que está falando, igual acontece em toda sociedade. Sem saber todos os detalhes sobre algo a pessoa não consegue avaliar de fato o que ocorre e ela está fadada a produzir fake news sofrendo do Efeito Dunning Kruger. Precisa mudar.
O mesmo acontece com a aplicação. A memória da aplicação não é alocada byte por byte necessário. A alocação é feita sempre pelo sistema operacional, então a aplicação precisa chamá-lo e dizer que quer memória, e o OS entrega o pedido, se for possível.
A forma exata como ele fará isso é problema dele, inclusive é feito em esquema de memória virtual, não terá acesso de forma linear.
O OS mostra na ferramenta de inspeção de uso de memória da aplicação (Gerenciador de Tarefas) conforme ele entregou e não foi devolvido. Note que ele é detalhado e sabe o que está em uso real e o que não está, ainda que já esteja reservado.
As linguagens não devolvem algo que será usado de novo logo depois. Mesmo o uso real é complicado porque as linguagens reaproveitam o que já foi usado e para o sistema operacional o que já foi usado tem uso real, mesmo que no momento não esteja em uso de fato. Ele só sabe se está só reservado ou se já foi usado de fato (committed).
Sem falar que o OS trabalha com páginas de memória e não bytes de fato, até porque isso é irrelevante para grandes consumos.
E as linguagens, em seu runtime, não ficam pedindo byte por byte que precisam, acessar o OS é bem caro, lento, então ele pede memória em blocos, quanto mais pedir mais eficiente fica em processamento, e menos em consumo.
Por tudo isso o gerenciador de memória não serve para avaliar a individualidade de uma aplicação, só para o coletivo é útil.
Precisaria de uma ferramenta que conhece e tem acesso ao que o runtime faz para ter uma informação confiável.
Nada disso explica o que aconteceu neste caso, só quis desmistificar esse uso.
Otimização
Onde a variável ptr foi usada na aplicação?
Se algo não é usado, por que o compilador manteria a instrução no código que cria um absurdo de memória* se essa informação nunca será usada? Ele não mantém. Ele otimiza o código matando essa instrução inútil.
Outros fatores
Na verdade pode ocorrer várias outras coisas que eu não sei, não lembro ou não estou percebendo. Até porque isso é detalhe de implementação e pode mudar a qualquer momento.
E é isso que eu tenho que saber, isso não é algo fixo. Se eu quiser saber de fato o que ocorreu terei que fazer vários experimentos, inspecionar em detalhes, tudo com fundamentação em algo correto, se eu errar em algo terei uma informação falsa.
Se alguém falar que ocorreu outra coisa neste caso específico não vou me surpreender. Se falarem que no compilador tal, uma outra versão, forma de compilação, ou em OS diferente, até por causa da diferença de tratamento do runtime, acontecer diferente do observado no seu teste, será algo potencialmente esperado.
Stack
Se fosse na pilha aí também não mostraria mudança porque a alocação de memória já ocorreu no começo da execução para toda a pilha e não ao consumo real.
Para começar a se aprofundar

O que é segmentação de memória?
O que é segmentation fault?
Como um programa é carregado na memória e depois executado?
Como um executável é apagado da memória?
As variáveis são alocadas aleatoriamente na memória?
Como o malloc() organiza a memória?
Como o "free()" sabe quanto de memória tem que liberar?
O que são e onde estão a "stack" e "heap"?
What does the brk() system call do?

Coloquei no GitHub para referência futura.

* Ainda que ele nem saiba exatamente disto, mas se não usa, não é útil, mesmo algo mínimo


How Can I space legend items with variable spacing and have legend marker colors reflect the colormap

Question: I would like to have an increasing spacing between legend items instead of a single value (labelspacing). The latter only accepts an int value type, but I want a variable spacing between legend items. Also, I want the markerfacecolor to follow the colormap used when creating the scatter plot.
N = 45
x, y = np.random.rand(2, N)

s = np.random.randint(10, 1000, size=N)

fig, ax = plt.subplots()

scatter = ax.scatter(x, y, c=s, s=s)

cbar = fig.colorbar(scatter,
             ax=ax,
            label='Size',
            fraction=0.1,
            pad=0.04)

# produce a legend with a cross section of sizes from the scatter
handles, labels = scatter.legend_elements(prop="sizes", alpha=0.6)

for hd  in handles:
    hd.set_markeredgewidth(2)
    hd.set_markeredgecolor("red")
    hd.set_markerfacecolor('blue')

legend2 = ax.legend(
    handles[::2], labels[::2], loc="upper right", title="Sizes", labelspacing=1.2
)

plt.show()

I searched StackOverflow and tried some possible methods but without success. Could someone guide how I can achieve the desired output?
Answer: I managed to set markerfacecolor as the colormap. But I am still struggling with the variable labelspacing!.
Any help!
N = 45
x, y = np.random.rand(2, N)

s = np.random.randint(10, 1000, size=N)

fig, ax = plt.subplots()

scatter = ax.scatter(x, y, c=s, s=s)

cbar = fig.colorbar(scatter,
             ax=ax,
            label='Size',
            fraction=0.1,
            pad=0.04)

# produce a legend with a cross section of sizes from the scatter
handles, labels = scatter.legend_elements(prop="sizes", alpha=0.6)

leg_colrs = [color.get_markerfacecolor() for color in scatter.legend_elements()[0]]

for hd, color  in zip(handles, leg_colrs):
    hd.set_markeredgewidth(2)
    hd.set_markeredgecolor("red")
    hd.set_markerfacecolor(color)

legend2 = ax.legend(
    handles[::2], labels[::2], loc="upper right", title="Sizes", labelspacing=1.2
)

plt.show()


NOT NULL constraint failed: product_product.author_id

Question: When I am going to post objects in the postman I am getting this error

NOT NULL constraint failed: product_product.author_id

I included Basic Auth in the Authorization section anyway it gives me error.
models.py
class Product(models.Model):
category = models.ForeignKey(Category, on_delete=models.CASCADE)
name = models.CharField(max_length=200)
brand = models.CharField(max_length=200)
rating = models.PositiveSmallIntegerField(default=0)
description = models.TextField()
author = models.ForeignKey(User, on_delete=models.CASCADE)

serializers.py
class ProductSerializer(serializers.HyperlinkedModelSerializer):
category = serializers.SlugRelatedField(queryset=Category.objects.all(), slug_field='name')
author = serializers.ReadOnlyField(source='author.username')
class Meta:
    model = Product
    fields = ['id', 'url', 'category', 'name', 'brand', 'rating', 'description', 'price']

Why Not Null constraint happens? how can I solve this? Thanks in advance!
Comment: You have to set ```ForeignKey``` even though you don't add it in the seralizer. Or you need to add ```blank=True, null=True``` in the models.py
Comment: @cagrias thanks for the answer yes it is right when I add null=True it works but in the admin page I do not know which user added instead i shows None. How can I solve this issue?
Comment: you have to specify an author when creating a product. add author field available in create product screen.
Answer: I think you should add models.create_all() in Models.py to create the database
Answer: The error happens simply because your create request is not providing a value for
author attribute of a product.
# models.py

class Product(models.Model):
    ....
    author = models.ForeignKey(User, on_delete=models.CASCADE)

so in order to resolve this issue totally depends on your business logic,

Is it important for a product to have an Author?

[NO] ==> then just make the foreign key null=True, blank=True
[YES] ==> Then you need to modify your creation logic a lil bit.

Is the author of a product is the same one who created it?

[YES], then this can easily be done by overriding your serializer's create method

    ....
    # Inside you serializer
    def create(self, validated_data):
        validated_data['author'] = self.context['request'].user
        return super().create(validated_data)

[NO], You have to make the serializer accept writes on author field.

A small note, your ProductSerializer Meta class's fields attribute, doesn't include 'author', make sure it is added there too.


How to study Bach's harmony?

Question: I'm interested in studying the music of J.S. Bach, but I don't know where to start. The main reason I'm studying it because I'm very interested in Bach's harmony, so that would by my main objective as to then implement it my own compositions. So where can I start and get the most out of studying Bach?
Comment: The question is not just asking for a list. Seeing that Bach wrote lots of pedagogical material and is probably _the most studied composer_ what purpose is served by not pointing the OP to the obvious starting places.
Comment: This forum has way too much closing of ordinary theory questions under the excuse of "it's asking for a list," but yet has no problem with things like identifying guitars from photos under the dubious "music practice" description. Help people study music using music theory: isn't that one of the main goals of this forum? How is that end served telling a beginning Bach student to go pound salt?
Comment: I'm voting to close this question because recommendations for off-site information or learning materials tend to be opinion-based & don't add any value to the stack exchange network.
Comment: Maybe you can rewrite your question. I think if you cut out the last sentence, where you ask for recommendations for off-site information, you might be able to get some answeres. Then it's more like a guide on how to study a certain style of music. But i'm not totally sure about that.
Comment: This might be a bit subjective. Where you start with Bach might be different from where anyone else starts. I would start with my favorite pieces; and start with the shortest or earliest of those. Basically, why not study what you like the most? Since I can play piano, I would learn to play some easier keyboard works and also do score analysis. And I might write imitations. It depends. If that's the kind of answer you're looking for then to me that's subjective.
Comment: Hi! I would actually vote to close because we don't have enough details, but hopefully you can edit to add them first. "Bach studies" are an entire field of musicology, it's been going on for centuries, and it attracts hundreds of scholars. People specialize in very specific aspects of studying Bach and his works, maybe focusing on a few years of his life, or his works for a certain instrument, or the influence on his writing by some earlier sources. Even saying you're "interested in his harmony" leaves an incredibly broad scope of study, and would affect the answer.
Comment: (Ironically, although I'm asking you to narrow your goals, the summary of my answer would be to start very broad. Even if you want to focus on harmonic analysis, you need the roughest idea of his biography and the musical landscape he worked within. You need to know about Buxtehude and Schutz, and the broad outlines of how counterpoint developed, and about what was coming next. You need to know a bit about Bach's contemporaries to see how he differed from them, about Italian trends vs German, etc.)
Comment: @Tetsujin And yeah, I don't know that "where can I start" has to be taken literally; it might simply mean "how do I go about it"; after all, the title (which I think I'll edit in a sec) uses "how."
Comment: Though, Arsan, as long as the focus is "harmony," there is one resource that will be involved: the best place to analyze Bach's harmonic choices is in his chorale settings, and the best primary source for that is Riemenschneider's collection of them. This just gives you the notes, though; unless you're already adept at harmonic analysis, you'll want to use it as a tool along with a basic theory course. And as I mentioned, for a full understanding you really ought to couch it in a basic understanding of everything around Bach.
Comment: It seems it may have been closed due to a previous portion which was seeking external sites. Possibly if the question was worded around the simple idea of "how can I study the harmony of J.S. Bach?" That question would be answerable directly on the forum.
Answer: Two items come to mind:
The two and three part inventions. Bach wrote a preface to those in which he stated...

Sincere instruction...not only to compose good inventions, but to develop them well...and to acquire a taste fro the elements of composition.

The 371 harmonized chorales. In the preface the author states...

...we know from C.P.E. Bach's preface [to earlier editions] that the editors did not want to put together a choral hymnbook but an instructive collection of examples of Bach's compositional style performable chiefly on a keyboard instrument.

A number of other items were composed by Bach for instructing members of his family like the Wilhelm Friedemann and Anna Magdalena notebooks. Some of the items in those books were not composed by Bach, but a good edition should have notes on those details. I think the Magdalena book includes more item not composed by Bach than the Wilhelm book.
At any rate, my point is there are several well known items that Bach composed for teaching purposes. Those make excellent choices for what to study.
In terms of how to study them, there are three general approaches:

Roman numeral analysis of the harmony
Identifying contrapuntal devices
Identifying elements of melodic/motif development

The last two could be lumped into one category, but you can think of them as different. For example identifying contrapuntal devices might include noting tonal versus real answers and stretto in a fugue. Melodic development might include inverting, truncating, or interval modification of lines. The former does really involve changes to material so it's more of a finding process, the later involve thematic material changed so you must first identify that thematic material and alos recognize when it has been changed.
A harmony textbook like Kostka, Tonal Harmony is the kind of resource to learn about the harmonic analysis.
A counterpoint textbook like Piston, Counterpoint or Kennan, Counterpoint is the kind of resource to learn about conterpoint/motif development.
Actually, Bach is probably the most studied composer, certainly one of the most musically quoted. Just about any college oriented textbook about harmony or counterpoint will include, if not be dominated by, examples and analysis of Bach's music.
One final thought: much of Bach's music is linear. Even in simple compositions like the first invention there are seemingly no chords, or there is so much linear motion you don't know where chords begin and end. That could present a stumbling block for you if you are new to harmonic analysis. Just understand that the lines in Bach's music often outline or embellish harmonic framework with things like broken chords, turns and other figuration. If this is a problem for you, first select the simple pieces with "block" chords, then look for the ones with broken chord patterns, then work on the ones with scalar lines, turn figures, etc. That approach will move progressively from obvious harmonic material to stuff that obscures harmony with lots of linear motion.

How to study Bach's harmony?


Push extenal data with data from server to array

Question: I am currently working on a project in which, I am fetching the data from server and adding(pushing) in the array for further process. And its working great.
I face problem when I add an external data to the array with the data coming from server.
var quantity= ItemsValue[1];
$scope.product = [];
$http.get(___).success(function(data) {
    $scope.greeting = data ;
    $scope.product.push($scope.greeting);
}

I want to push "quantity" with the "$scope.greeting". I already tried different thing such as concatenation but failed.
I want data of array to be like this. For Example

$scope.product=[{ "greeting.name": "abc", "greeting.price": "50",
  "quantity":"1" }]

Name and Price came from server and quantity in added as an extra data to Array.

{{greeting.Name}}

Rs. {{greeting.Cost}}

                                     +
                                    <input class="cart_quantity_input" type="text" name="quantity" value="Itemsquantity" autocomplete="off" size="2">
                                     -

Rs. {{greeting.Cost}}

This is the code of client end.
Any way to push these things together...
Answer: You can just add it into the object by creating a new property
var quantity = ItemsValue[1];
$scope.product = [];
$http.get(___).success(function(data) {
    $scope.greeting = data;
    $scope.greeting.quantity = quantity;
    $scope.product.push($scope.greeting);
}

Comment: I am using the greeting in view to display data
Comment: @ZiaUllahMinhas modified accordingly


In a Swift project, how to reference code that is a couple folders down to be in scope?

Question: I am trying to structure my Swift project to be organized by scene.
I plan to have a folder for each scene, immediately within that folder the scene delegate, and within subfolders the views and models. Something like this:
Project
* Scene1Folder
  - Scene1Delegate.swift
  * Scene1Views
    - Scene1View1.swift

However, this structure appears to make the Scene1View1.swift file out-of-scope of the Scene1Delegate.swift file.
Here is a screenshot of xcode showing the folder-structure of this part of my project, the Scene-folder and its sub-contents:

After moving my view folder from the project-root and into LookupScene, my SceneDelegate.swift file can no longer reference my ContentView.swift class. It shows this not-in-scope error:

How can I reference code within a sub-folder so that I can organize my project the way I want to?
UPDATE: This appears to compile despite the out-of-scope error shown by xcode. Appears that swift can find the reference even though xcode thinks it won't be able to.
Answer: The arrangement of files and "folders" (actually, groups) in the Project Navigator is merely a convenience to you. A "folder" does not constitute any kind of scope. Every Swift file, no matter where it is in the Project Navigator, is visible to every other Swift file, provided they are all part of the same app target.
Comment: It looks like the error you saw is merely a transient error caused by "Show live issues" (General pref pane). Personally, I like to keep that turned off.


$\tan x>x+\frac {x^3}3$ for $x\in(0,\frac\pi2)$

Question:
$$\tan x>x+\frac {x^3}3~\text{ for }~x\in\left(0,\frac\pi2\right)$$

My solution:
Both functions are monotone, increasing, equal at $x=0$. If i could show that the derivative of the first is greater than then derivative of second function that would be it. Taking the derivatives:
$$\frac1{\cos^2 x}>1+x^2$$
Applying the same reasoning on the derivatives  and taking their derivatives we get
$$\frac{\tan x}{\cos^2 x}>x$$
Doing the same thing (it is possible to stop here if we use $\tan x>x$)
$$\frac{1+\tan x\sin 2x}{\cos^4 x}>1$$
It is easy to see that last inequality is true therefore all previous are also true.
Is my solution correct, i would be very disappointed if it weren't. Are there different ways to solve this ?
Comment: The coefficients in the Maclaurin series for $\tan x$ are all positive.
Comment: @LordSharktheUnknown , it holds for $x\to 0 $ , this is on an interval
Answer: Let $f(x)=\tan{x}-x-\frac{x^3}{3}.$
Now, $$f'(x)=\frac{1}{\cos^2x}-1-x^2=\tan^2x-x^2=(\tan{x}-x)(\tan{x}+x)>0.$$
Can you end it now?
Comment: @Milan I think it's better to consider a function $f$, as I did.
Comment: @Milan You can prove it by the same way: Let $g(x)=\tan{x}-x.$ Thus, $g'(x)=\frac{1}{\cos^2x}-1=\tan^2x>0...$
Comment: Great. I would need to use tanx>x? Is my solution correct
Comment: @Milan In your last inequality should be $\cos^4x$ in the denominator.
Comment: Yes $~~~~~~~~~~~$
Comment: I agree, it is simpler. But do i have to use $\tan x>x$
Answer: At the point
$$
\frac{1}{\cos^2x}\overset{\Large{?}}>1+x^2\iff\frac12(1+x^2)(1+\cos(2x))\overset{\Large{?}}<1
$$
one can insert the power series of the cosine to get
$$
\frac12(1+x^2)(1+\cos(2x))=(1+x^2)\left(1-x^2+\tfrac13x^4-\tfrac{2}{45}x^6\pm\cdots+\tfrac{(-1)^n}{(2n)!}(2x)^{2n}\right)
\\
=1-\tfrac23x^4+\tfrac{13}{45}x^6\mp\cdots+\tfrac{(-1)^n}{(2n+2)!}[(n+\tfrac12)(n+1)-1](2x)^{2n+2}
$$
which is an alternating series with continuously falling coefficients, so that the Leibniz test can be applied for $|x|<1$, giving that the first partial sum, consisting of the first term $1$, is an upper bound.
Comment: I can't use power expansions. I could only use taylor series at x=0. We haven't dkne analytic functions in class
Comment: But you do know the Taylor series of the cosine at $x=0$ and the Leibniz test?
Answer: Due to the alternating nature of the sine and cosine series the partial sums are alternatingly upper and lower bounds (which is a corollary of the proof of the Leibniz test). Thus for $x\in (0,\frac\pi2)$, or more general $x\in(0,\sqrt6)$,
$$
x-\frac{x^3}6\le \sin x\le x\\
1-\frac{x^2}2\le \cos x\le 1-\frac{x^2}2+\frac{x^4}{24}
$$
Thus
\begin{align}
\sin x-(x+\frac{x^3}3)\cos x&\ge x-\frac{x^3}6-(x+\frac{x^3}3)(1-\frac{x^2}2+\frac{x^4}{24})
\\
&=\frac{x^5}8-\frac{x^7}{72}>\frac{x^5}{24}>0
\end{align}
proving the claimed inequality directly.
Answer: Since $\tan(x)=-\frac{d}{dx}\log\cos x$ and
$$ \cos(x)=\prod_{n\geq 0}\left(1-\frac{4x^2}{(2n+1)^2 \pi^2}\right) $$
we have
$$ \frac{\tan x}{x}=\sum_{n\geq 0}\frac{8}{(2n+1)^2 \pi^2-4x^2}=\sum_{n\geq 0}\frac{8}{(2n+1)^2 \pi^2}\sum_{m\geq 0}\left(\frac{4x^2}{\pi^2(2n+1)^2}\right)^m $$
and by exchanging $\sum_{n\geq 0}$ and $\sum_{m\geq 0}$ we get
$$ \color{red}{\frac{\tan x}{x}} = \sum_{m\geq 0}\frac{4^{m+1} x^{2m}}{\pi^{2m+2}}\cdot 2\sum_{n\geq 0}\frac{1}{(2n+1)^{2m+2}}=\color{red}{\sum_{m\geq 0}2(4^{m+1}-1)x^{2m}\cdot \frac{\zeta(2m+2)}{\pi^{2m+2}}} $$
and the given inequality follows from $\zeta(2)=\frac{\pi^2}{6}$, $\zeta(4)=\frac{\pi^4}{90}$ and $\zeta(2m+2)\geq 1$ for any $m\in\mathbb{N}$.
This actually leads to
$$ \frac{\tan x}{x} \geq 1 + \frac{x^2}{3} + \frac{6 x^4 \left(21 \pi ^2-20 x^2\right)}{4 \pi ^4 x^4-5 \pi ^6 x^2+\pi ^8}$$
for any $x\in\left(0,\frac{\pi}{2}\right)$.
Comment: Nice hat $~~~~$


"Are All Correlations Spurious But Some Correlations Are More Spurious?" - Not George Orwell

Question: (Sorry for the George Orwell/Animal Farm reference)
I am an MBA student taking courses in Statistics. Over the Thanksgiving weekend, we have an informal assignment - we have to find publicly available data on two "concepts" and make a small presentation on the possible correlations between these two "concepts" and if the correlation might be "spurious" or not.
I chose to explore the relationship of countries between the "Number of Doctors Per 1000 People" and the "Life Expectancy" of people in that country. On the surface, this sounds like there might be some correlation - as a brutish example, imagine if there are few doctors (relative to the population) - assuming if not seeing a doctor can kill you, countries with fewer doctors would result in sitting in the waiting/emergency room for longer periods of times (assuming that the prevalence and severity of medical conditions are similar across different countries, as well as the quality of doctors) ... ultimately resulting in more people dying in general and lowering the life expectancy in countries with smaller doctor/people ratios. Although I have greatly and comically oversimplified this situation, I would hope that there is some grain of truth in this oversimplification and it would be reflected within the data.
I found data on the ratio of doctors/population in different countries over here (https://data.oecd.org/healthres/doctors.htm, https://data.oecd.org/healthres/doctors.htm - I copy/pasted this into Excel) and I found data on the life expectancy over here (https://data.worldbank.org/indicator/SP.DYN.LE00.IN). I am learning how to use R - I downloaded both of these datasets and made this quick graph to investigate the correlation:
<code># NOTE: This is not a Doctoral Thesis, but a very informal learning assignment where the data doesn't need to be fully accurate - please take it easy on me.. thank you!
 # PART 1: DOCTOR DATA
doctors = read.csv("doctors_country.csv")

#PART 2: LIFE EXPEC DATA
#chop off the first few rows in excel to make the data manipulation easier
life_e = read.csv("life_expec.csv")
life_e = data.frame(life_e$Country.Name, life_e$X2020)
colnames(life_e)[1] <- "Country"
colnames(life_e)[2] <- "Life_Expec"

#PART 3: JOIN FILE
graph_file = merge(x = life_e, y = doctors, by = "Country", all.y = TRUE)
graph_file = na.omit(graph_file)

# PART 4: PLOT GRAPH
par(mfrow=c(1,2))
plot(graph_file$Life_Expec, graph_file$Doctors, main = "Correlation Between Life Expectancy and Number of Doctors Per 100 People (2020)" , xlab = "Life Expectancy (Years)", ylab = "Number of Doctors Per 1000 People")
text(graph_file$Life_Expec, graph_file$Doctors, labels=graph_file$Country)
</code>

Based on this (very hurried) analysis that I performed, it would appear that there is somewhat of a linear correlation between these two concepts (we can also calculate the Correlation Coefficient to find out how strong, i.e. Rho Value). It appears that the more doctors there are relative to the population, the life expectancy tends to also increase. Hurray!
But here is my question: How Do I Know That This Relationship Is Not Spurious? Maybe for generations, there is a secret undiscovered vitamin in the water of Norway and Austria that makes people live longer - and this increase in life expectancy has nothing to do with the high ratio of doctors/people!!??
In a previous question (Does Correlation "Sometimes" Imply Causality?), I started learning about "Causality vs Correlation" and found out that it is very difficult to ascertain the presence of Causality.
However in this question (i.e. Doctors vs Life Expectancy), is there truly a way to determine whether or not this correlation is spurious or not? It seems to me that there is no straightforward answer to this question - such a question can only be answered by rigorous experiments, subject matter knowledge and science. I am guessing that extensive reviews of the healthcare systems, exogeneous and endogenous factors, life style habits, epidemiological patterns, review of academic intuitions ... all would have to be comprehensively studied over decades to find out if in fact this doctor/life expectancy correlation is spurious or non-spurious. And this extensive research will likely not be finished by next Wednesday (i.e. the date of my informal presentation).
Thus, we have no choice but to interpret our results under the following framework : Although philosophically, all correlations might be partly spurious - some correlations are likely to be far more spurious than other correlations (e.g. divorce rate in Maine/vowels in the name of hurricanes vs life expectancy/number of doctors). Although nothing can be ever be conclusively determined (e.g. I can not be 100% sure that a rhinoceros is standing outside my door - but I can still be more relatively  certain that a pink rhinoceros is not standing outside my door, and even more relatively certain that a pink a rhinoceros riding a unicycle is not standing outside my door - but I can be pretty damn certain). However, we can not twiddle our thumbs indefinitely waiting on a universe of delusional and highly improbable events to manifest and risk jeopardizing and delay taking important decisions that can better mankind (e.g. a pharmaceutical drug performed very well in many clinical trials and has demonstrated the ability to save lives with minimum side effects - but we will never market this drug because technically all test patients might have just gotten lucky). Thus, using domain specific knowledge and rigorous statistical methodologies can help us attribute a level of confidence in the interim and provide reasonable beliefs if a specific correlation is spurious or not as we learn more about our data with each passing day and proceed to make important data based decisions to hopefully improve society.
Can someone again confirm if I am "out to lunch" (i.e. crazy) - or is my understanding somewhat correct?
Comment: The term 'spurious correlation' can be used in different ways. What is meant with spurious correlation? Is it used to refer to spurious (causal) relation? Or does it refer to the inference of a correlation that is actually not statistically significant?
Comment: Even though we cannot perform the experiment, it's fair to assume that removing all doctors from a country will measurably reduce its life expectancy. That is a causal relation.
Answer:
In statistics, a spurious relationship or spurious correlation is a
mathematical relationship in which two or more events or variables are
associated but not causally related, due to either coincidence or the
presence of a certain third, unseen factor (referred to as a "common
response variable", "confounding factor", or "lurking variable").

(Wikipedia)
You are correct that correlation does not imply causation, so many correlations are spurious. When do we know that the relation is spurious? It's when there's no causal relationship. We don't use correlations to find them, so looking at correlations alone won't tell you that.
Now the question boils down to "how do we detect causality?" It was answered in many treads tagged as causality, e.g. Interview question: If correlation doesn't imply causation, how do you detect causation?, while  Introduction to causal analysis gives many good references to dive deeper.
Comment: In OP's example, the confounding factors are probably something like median income/wealth and social security (over the last 80 years or so).
Comment: @Roland in the OP's example there might even not be a clear correlation. It depends a bit on how one views the population that the sample represents. If one considers the countries from the image to be the entire population then there is definitely correlation, but possibly one regards the countries in the image to be only a subset/sample from a larger population and regards the correlation in the sample as reflecting some relationship in that larger population...
Comment: ...In that case there might be a Yule-Simpson effect, which complicates how we look at that relationship. There seem to be three clusters when we look at the distribution on the x-axis (three countries with life expectancy << 75 years, 9 countries with life expectancy ~ 76 years and,  21 countries with life expectancy ~ 82 years) and between clusters there might be a positive correlation but within the clusters the correlation is not so clear and could be even negative.
Answer: I'd say your understanding is correct. Philosophically, it's not truly possible to "prove" causation with 100% certainty, but we can convince ourselves that the alternative is arbitrarily unlikely. Most methods of determining causation rely on manipulating pertinent factors and randomizing all other possibly contributing factors, but this requires knowing what other possibly contributing factors there are. No matter how we control or randomize, it's not possible to rule out all potential unknowns, by their nature of being unknowns.
For any well-described cause-effect relationship, I could posit that it's actually an unseen wizard that's manipulating the effect in just the right way such that it happens to line up perfectly with your suggested "cause". This of course quickly becomes very implausible when we look at realistic physical mechanisms for cause and effect, but you can't categorically disprove the existence of something by lack of evidence that it exists. We can become increasingly sure that a cause-effect relationship is real by appropriate experimental design and by using domain knowledge to describe realistic and well-understood mechanisms for cause and effect, but you can't truly rule out the possibility of an unknown cause that's never been observed or described, it just becomes vanishingly unlikely. At some point, the statistical and mechanistic support for the link between cause and effect are great enough to consider it "proven", even though it's technically not possible to rule out unseen causes with no understood mechanism.


Gradient Ascent for Mixture of Gaussians

Question: Beginner here, apologies if this is something very simple. I am trying to do a gradient ascent to estimate means for a Mixture of Gaussian model. I am using $(x-µ)/σ^3(2π)^{(1/2)} * e^{-((x-µ)/σ)^2)/2}$ for the first derivative of the gaussian distribution. My question here is, for the gradient ascent should I sum the first order derivative over all values of $x$ (datapoints) for each update?
Comment: Do you assume the other parameters are known: variances and mixture weights?
Comment: Yes, I have the other parameters. But not the means. Further, with gradient ascent, if I sum over all datapoints, it gets stuck at a local maxima while EM algo. doesn't in fact EM converges thrice faster than gradient. They both start at same initial mean values
Answer: What most people do is work on the $\log$ of the likelihood. The likelihood is multiplicative across data points, thus the log-likelihood is additive. Maximizing one or the other is the same. Formally, call $L_i$ the likelihood of point number $i$.
$$\log(L)=\sum_i \log(L_i)$$
For a mixture of $d$ Gaussians:
$$\log(L_i)=\log\left(\sum_{j=1}^d \lambda_j\frac{1}{\sqrt{2\pi\sigma_j^2}}e^{-(x_i-\mu_j)^2/2\sigma_j^2}\right)$$
So first, take the derivative of $\log(L_i)$ for each unknown parameter: each mean $\mu_j$ (you assume variance $\sigma_j^2$ and weight $\lambda_j$ are known). Then sum it over all data points to get the global gradient.
Note that Gaussian mixture is usually not solved with gradient ascent but Expectation/Maximization since the log likelihood is not convex (at least with all parameters unknown, I don't known if only the means are unknown). Yet I've seen non convex functions successfully maximized by gradient ascent, so it may be interesting to try.
Comment: The issue here is, given the huge dataset that I am working with, iterating over all the elements for each update would be computationally prohibitive.
Comment: The classical way to accelerate gradient decent is stochastic gradient descent. It's especially efficient on big datasets. It's not more difficult to implement, maybe even simpler.


How do I construct the JSON data with session info for creating node?

Question: I am using Drupal 7 and the Services module 7.x-3.2. I am using Postman for Chrome (a http client) to test the API.
What I want to do is log in a user and create a node with the user using the Drupal service API.
I logged in the user with JSON POST request to http://example.com/api/user/login.

I successfully got the JSON result.
{
    "sessid": "KrZjlcY3rFr8v6iLoskcVQN9DmDWsg5yZqIjBeydEBY",
    "session_name": "SESS9fe39db36ccd7afd64e034fefa0129e1",
    "user": {
        "uid": "31",
        "name": "test_user",
        "mail": "firstname.lastname@example.com",
        "theme": "",
        "signature": "",
        "signature_format": "filtered_html",
        "created": "1352985941",
        "access": "1352991170",
        "login": 1352991510,
        "status": "1",
        "timezone": "",
        "language": "",
        "picture": null,
        "init": "email@example.com",
        "data": false,
        "roles": {
            "2": "authenticated user"
        },
        "rdf_mapping": {
            "rdftype": [
                "sioc:UserAccount"
            ],
            "name": {
                "predicates": [
                    "foaf:name"
                ]
            },
            "homepage": {
                "predicates": [
                    "foaf:page"
                ],
                "type": "rel"
            }
        }
    }
}

Then I tried to create an article via the API by sending the JSON POST request. (The user has the permission to create Article nodes.)

I got a 401 Unauthorized response.
I guess I need to include the session cookie information to send the POST request, to create a node.
I have the session information previously returned.
"sessid": "KrZjlcY3rFr8v6iLoskcVQN9DmDWsg5yZqIjBeydEBY",
"session_name": "SESS9fe39db36ccd7afd64e034fefa0129e1",

How can I utilize it to make a successful JSON POST request for creating a node?
Answer: {
   "title": "Just another drupal Post",
   "type": "Article"
}

notice no comma after the type:article tag so that your json works.
Answer: Hope this will help you,

To make the HTTP requests, Im using the Firefox add-on POSTER.

{
  "type" : "article",
  "title" : "Test article with language",
  "language" : "und"
}
If node is created successfully you will get a response with a link.

Examples

{
  "type": "article",
  "title": "Test article",
  "body": {
    "und": [{
      "value": "test body!"
     }]
  },
  "language" : "und"
}

{
   "title":"Review",
   "type":"establishment_review",
   "field_establishment":[
      {
         "nid":{
            "nid":"[nid:26686]"
         }
      }
   ],
   "field_fivestar_value":[
      {
         "rating":"20",
         "target":"0"
      }
   ],
   "field_review":[
      {
         "value":"comments about the restaurant"
      }
   ]
}

You can refer this
I have answered How do I POST a new node using the Services module question. You can refer for more information.
Answer: You need to set cookie="sessionname=sessionid" value in headers, where sessionname comes from the endpoint/user/login URL.


Interpreting coefficient in GLM with categorical explanatory variables

Question: I'm performing a GLM, the response variable is number of individuals, and response variables are

habitat (4 levels) and

season (4 levels).

I need some help since I know the summary() shows p-values but not for the first (alphabetical) level of factor. I mean, I don't know how to interpret that the Intercept has a significant p-value. I can't reach a biological explanation for this model. Hope you can help me ...
    Call:
    glm(formula = individuals ~ habitat + season, family =
           poisson(link = "log"),
    data = data)

    Deviance Residuals:
     Min        1Q    Median        3Q       Max
    -2.85859  -0.51541  -0.08508   0.36497   2.29058

    Coefficients:
                     Estimate Std. Error z value Pr(>|z|)
    (Intercept)            1.6774     0.1624  10.331  < 2e-16 ***
    habitatDeciduous       0.1340     0.1696   0.790  0.42950
    habitatSemiDeciduous   0.2102     0.1675   1.255  0.20933
    habitatWetland        -0.1861     0.2039  -0.912  0.36151
    seasondry2018         -0.1138     0.1510  -0.753  0.45123
    seasonwet2016         -0.2699     0.1576  -1.713  0.08677 .
    seasonwet2017         -0.4383     0.1656  -2.647  0.00813 **

    Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    (Dispersion parameter for poisson family taken to be 1)

    Null deviance: 78.487  on 63  degrees of freedom
    Residual deviance: 64.556  on 57  degrees of freedom
    AIC: 287.77

Comment: @Ingolifs:  How can you fully interpret the intercept of a model if you don't know what the reference categories are for the factors included in the model? Having said that, if you feel that information is not necessary, you can go ahead and provide the desired interpretation. Prior to answering, I find it fair to ask for the information I think it is necessary for me to provide a helpful and specific answer.
Comment: Can you clarify in your response the names of all of your habitat categories and all of your season categories?
Comment: I don't think that's really necessary to the question.
Answer: When independent variables are categorical, one level of each is chosen as a refernce level. By default, R choose the first one in alphabetical order. The other levels are compared to that level.
The intercept is the parameter estimate for when all the other variables are 0 -- in this case, when the other two variables are at the reference level.
You can change the defaults, but R questions are off-topic here.
Answer: I would quite like to know the proper answer to this too.
In the meantime, you can use the relevel() function. Pick one of the 'non-default' factors, perhaps the ones with the least significance (habitatDeciduous and seasondry2018) and make them the default factor, by typing in something like:
    data$habitat <- relevel(data$habitat, "Deciduous")

and rerunning the glm. You'll get different numbers out of the new model and the two models aren't really comparable with each other, but you will be able to get an idea of how significant the 'missing' factor is.
Answer: Do not overcomplicate things. In regression models with categorical variables (factors in R speak), when dummy (one-hot) encoding is used:  You want to do contrasts between parameters, but this includes the level of the factor that was "left out".  The left-out level has an implied estimate value of 0 --- zero. As that is implicit in the parametrization and not estimated, its implied standard error is zero.
Yes, it would be nice if the software output included this ...
Also, whichever categorical encoding scheme you are using, you can get inference for parameter contrasts of interest by formulating it as a linear hypothesis. In R you could use car::linearHypothesis function.


$\mathbb E[X]=0$ implies existence of two random variables $Y\stackrel{\mathrm d}=Z$, equal in distribution, such that $X\stackrel{\mathrm d}=Y-Z$

Question: Let $X$ be a bounded random variable on a non-atomic probability space, i.e. $X\in L^\infty(\mathbb P)$, such that $\mathbb E[X]=0$. In a lecture, I heard someone claim that this implies the existence of two other random variables $Y,Z\in L^\infty(\mathbb P)$ such that $Y\stackrel{\mathrm d}=Z$ and $X\stackrel{\mathrm d}=Y-Z$, although they claimed the proof was difficult and tedious.
Is this result even true? Even in simple examples I don't really see what this decomposition should look like. What is the intuition behind such a result? Is there perhaps a proof in the literature or in a textbook?
Comment: Well, clearly if X is symmetrically distributed, we can take Z=-X/2 and Y=X/2. So the difficulty is for X no symmetric.
Comment: @user24142 $\stackrel{\mathrm d}=$ denotes equality in distribution
Comment: It's certainly not true if you add the restriction that $Y,Z$ are independent.
Comment: @DarkMalthorp I do not assume $Y,Z$ are independent.
Comment: @jwhite Thanks! I forgot to mention that we work with non-atomic r.v.s
Comment: This is supposed to be true in any probability space? Or in a specific probability space?
Comment: $A \stackrel{\mathrm d}= B$ implies $|| A-B ||_1 = 0$ ie $L_1$ equivalent? If so I'm really not seeing how this could be true.
Comment: @user24142 you're claim is not correct, for $A,B$ are allowed to be dependent.
Comment: e.g. take $A$ standard normal, $B=-A$, $A-B=2A$, which of course has zero expectation but $\|A\|_{L^1}=\mathbb E|A|>0$
Comment: okay, what is the meaning of that notation then?
Answer: Here's an attempt at a proof. I don't have a reference, or much intuition beyond this construction.
Consider first the case of a variable $X$ taking the value $a<0$ with probability $b/(|a|+b)$ and
taking the value $b>0$ with probability $|a|/(|a|+b).$ Pick random variables $Y,Z$ uniformly distributed on $[a,b],$ with $Y=Z+b$ when $Z<0$ and $Y=Z+a$ when $Z\geq 0.$ Then $Y-Z\stackrel{\mathrm d}= X$ as required.
For the general case we can take a mixture, by pairing up the events $X<0$ and $X>0$ in a suitable way.
Remark. We just need to write the law of $X$ as a weighted average of zero-mean probability distributions supported on two points. A slick way to do this is to use Choquet's theorem and a characterization of extreme points of the set of zero-mean variables, e.g. Winkler's  "Extreme Points of Moment Sets". The remainder of this answer is just a more explicit version of this.
Let $\mu$ be the law of $X.$ Let $M=\tfrac 1 2 \mathbb E|X|.$
There exists a Borel function $f^+:[0,M]\to\mathbb R_{>0}$ such that the pushforward of Lebesgue measure $f^+_*\lambda$ is the measure $x d\mu(x).$
For example, the generalized inverse of the function $t\mapsto \int_0^t x d\mu(x).$
Similarly there exists a Borel function $f^-:[0,M]\to\mathbb R_{<0}$ such that $f^-_*\lambda$ is the measure $|x| d\mu(x).$
The pushforward of $(1/f^+(x))d\lambda(x)$ under $f^+$ is $\mu$ restricted to $\mathbb R_{>0}.$
And similarly the pushforward of $(1/|f^-(x)|)d\lambda(x)$ under $f^-$ is $\mu$ restricted to $\mathbb R_{<0}.$
Pick independent random variables $R,S,T$ such that

$R$ is a Bernoulli random variable with $\mathbb P[R=0]=\mathbb P[X=0],$
$S$ is a random variable on $[0,M]$ distributed according to the measure $((1/f^+(x))+(1/|f^-(x)|))d\lambda(x),$
$T$ is uniformly distributed on $[0,1].$

Then define $Y$ and $Z$ by:

wherever $R=0,$ we take $Y=Z=0,$ and skip the next steps.
$Z$ is $T$ rescaled from $[0,1]$ to the interval $[f^-(S),f^+(S)].$
$Y$ is $Z+f^+(S)$ when $Z<0$
$Y$ is $Z+f^-(S)$ when $Z\geq 0.$

$Z$ has the same distribution as $Y.$
The random variable $Y-Z$ has the same probability of being zero as $X.$
For any Borel $U\subseteq\mathbb R_{>0},$ the probability that $f^+(S)\in U$ and $Y-Z=f^+(S)$ is $\int (1/f^+(x))1_{f^+(x)\in U}d\lambda(x)=\mu(U).$
So the laws of $Y-Z$ and $X$ are the same on $\mathbb R_{>0}$.
And similarly for $\mathbb R_{<0}.$
So this construction ensures $Y-Z\stackrel{\mathrm d}= X.$
Comment: This looks fine to me, and I bet this construction is way simpler than the `tedious' proof talked about in the lecture. Thanks for your inventive answer, +1, and I leave the bounty open to allow for other answers as well.
Comment: This looks difficult but not tedious at all.
Comment: @ColinMcQuillan the authors of the paper giving the lecture wish to thank you properly for this contribution in the next iteration of their paper. If interested, please contact the first author of https://arxiv.org/abs/2310.09173


Which are the most fundamental theorems in core undergrad math?

Question: I want to compile the most important theorems for Calculus, Linear Algebra, and introductory ODE, PDE, probability and statistics.
These should be the most important 5/10 per "subject", to put an arbitrary limit.
You may want to contribute with your LA list as an example, and a brief explanation on your choice above others.
Comment: This question does not fit the guidelines for this site very well. It's too subjective, too broad, and does not have a conclusive answer. Perhaps you'd have better luck on Quora.
Comment: Conley's theorem (or "fundamental theorem of dynamical systems", as the related school calls it): There exists a complete Lyapunov function for any homeomorphism on a compact metric space. 99.9% of the experts would disagree that it is appropriate to refer to it as "fundamental". Anyways, the term was coined by Norton as late as in 1995 (!). In other words: your question is too broad for people to agree on what are "the most important theorems".
Comment: Try Quora for these kinds of questions.
Comment: @Jonas, I changed it to fundamental. I don´t expect a 100% agreement by everyone, but for sure there are some theorems that are in each and every book and the most basics of those (the others are somehow based on them) are the ones I´m looking for.
Answer: Warning: this list is incomplete and written by a noob:
Calculus:

Theorem saying a continuous function on a closed bounded interval reaches max and min
Intermediate value theorem
Mean Value Theorem
Theorem saying a real sequence is Cauchy if and only if it converges
Fundamental theorem of calculus
Green's Theorem

Linear Algebra

Theorem saying every finitely generetad vector space has dimension (so every basis has same size)
Rank/nullity theorem
Cayley Hamilton theorem
Spectral theorem

Group theory:

Lagrange's theorem
Cauchy's theorem
Sylow's theorem
Orbit-Stabilizer theorem
Burnside's lemma
The isomorphism theorems

Complex analysis

Cauchy-Goursat theorem
Cauchy's integral formula
Taylor's theorem
Laurent's theorem
The residue theorem
Comment: I would now add the equivalence of norms theorem, the theorem saying continuous with compact domain is uniformly continuous, theorem about existence of solutions to diferential equations and much more tbh


How do I execute code when a module is installed or uninstalled?

Question: Currently, I manually have to add terms one by one every time a module gets installed, and that's not efficient. I'm writing a simple module that needs to automatically install 2 terms every time the module gets installed, and remove them every time the module gets uninstalled.
From Create taxonomy term programmatically on Drupal 8, I learned how to add terms. However, I simply don't know where to put the code. What must I do to achieve it?
Answer: Needs to go inside of mymodule.install file
<code><?php

/*
 * Implements hook_install()
 */
function mymodule_install() {

  // code that creates terms

}

/*
 * Implements hook_uninstall()
 */
function mymodule_uninstall() {

  // code that deletes terms

}
</code>


Date time to Epoch conversion

Question: I have a variable defined with date time format and want to convert it to epoch format in bash script
Input
Var=20170520;
Output:
Epoch format
Comment: Please specify what you tried and what didn't work.
Comment: @AravindVoggu  i have  a variable  equal to date time stamp as state above and I didn't find a way to convert  from variable to epoch format
Answer: Based on the manpage,
date --date=20170520 +%s

works (date --date=20170520 correctly outputs Sat May 20 00:00:00 YOUR_TIMEZONE 2017. Alternatively, you might need to tweak the free-form input string.)
Answer: Try
date=20170520 # i.e. 2017 05 20
date -d "$date" +%s

Comment: Question is tagged with `linux`.
Comment: @Cyrus okay, did I miss something? I ran it in a terminal and it worked, please point out if I made a mistake, I'll try to correct it.
Comment: You are using options supported by the BSD version of `date`, not the GNU version used on Linux systems.
Comment: I tried it works but when i add time stamp it failed could you advice me for this  var=201705121234 #2017 05 12  12:12:34
Comment: @Moustafa Looks like it's a little bit tricky and we have to refer to docs. I have only MacOS with me now (BSD). Why don't you just convert hours and minutes into seconds and add it to Epoch seconds returned by `date`?
Comment: for a kicker, `echo "223045" | fold -w2 | paste -sd':' -` this will add semicolon every 2 characters, and `echo "00:20:40" | awk -F: '{ print ($1 * 3600) + ($2 * 60) + $3 }'` this is my oneliner to convert `hh:mm:ss` into seconds.
cheers!

Date time to Epoch conversion


Adicionar Favicon ao meu website

Question: Se eu estiver acessando um site, tanto no desktop quanto mobile, existe a opção de adicionar o link a tela inicial, e alguns websites, adicionam o link com um ícone personalizado.
Ex: Stack

Agora o meu site por exemplo não tem icone configurado e vem um padrão do navegador.

Como eu faço para configurar isso no meu site?
Answer: Você precisa adicionar o favicon na sua página
O que é o favicon? (Wikipédia)

Os favicons são pequenas imagens (normalmente 16 por 16 pixels) que
  ficam guardados no site para visualização pelo navegador. Geralmente
  são utilizados como logotipos em tamanho reduzido, nos sites de
  empresas, entidades e marcas quaisquer. Um exemplo é o "W" que aparece
  no browser quando se acessa a Wikipédia.

Existem na internet diversos geradores de favicon gratuitos que você pode utiliza-los de forma muito simples.
Aqui é um exemplo que eu mesmo uso
Depois de acessado, basta selecionar a imagem que você deseja que seja o favicon de seu site e selecionar a opção:
Generate icons for Web, Android, Microsoft, and iOS (iPhone and iPad) Apps
Espere o processamento de criação do favicon.
Depois clique no link: Download the generated favicon
Feito isso, um arquivo .rar será baixado com diversos favicons que se adaptam aos dispositivos.
Logo depois, extraia esses favicon's para a pasta do seu projeto e copie as instruções de chamadas criadas pelo gerador.
As instruções devem ser semelhantes a esta:
<link rel="apple-touch-icon" sizes="57x57" href="/apple-icon-57x57.png">
<link rel="apple-touch-icon" sizes="60x60" href="/apple-icon-60x60.png">
<link rel="apple-touch-icon" sizes="72x72" href="/apple-icon-72x72.png">
<link rel="apple-touch-icon" sizes="76x76" href="/apple-icon-76x76.png">
<link rel="apple-touch-icon" sizes="114x114" href="/apple-icon-114x114.png">
<link rel="apple-touch-icon" sizes="120x120" href="/apple-icon-120x120.png">
<link rel="apple-touch-icon" sizes="144x144" href="/apple-icon-144x144.png">
<link rel="apple-touch-icon" sizes="152x152" href="/apple-icon-152x152.png">
<link rel="apple-touch-icon" sizes="180x180" href="/apple-icon-180x180.png">
<link rel="icon" type="image/png" sizes="192x192"  href="/android-icon-192x192.png">
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="96x96" href="/favicon-96x96.png">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/manifest.json">
<meta name="msapplication-TileColor" content="#ffffff">
<meta name="msapplication-TileImage" content="/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">

Repare que quando você for salvar o site em sua área de trabalho o ícone escolhido por você estará la.
Também repare que o ícone está presente do lado do título da página na aba do navegador.
Lembre-se que você deve corrigir os caminhos da chamada, caso o contrário não irá funcionar.
Answer: Como a resposta do @Bsalvo sugere, você precisa inserir uma tag específica dentro da tag  de toda página do site para referenciar seu ícone. Melhor da preferência pra um nome mais genérico pro arquivo para você poder mudar a resolução e o tipo de arquivo quando quiser.
Exemplo:

  <link rel="icon" type="image/png" href="Endereco();">

Adicionar Favicon ao meu website