Importing an animated .gif into Windows.Forms in Powershell

Question: I'm newer to Powershell and am running into this issue. I have a very long automated install script that runs through MySQL install/account setup and other .msi's.
While the users wait for the script to complete, I have this "Please Wait" prompt display that contains a spinning pinwheel .gif in the picturebox.
The problem is that if I set the form.visible to $true and have it close later on in the script, then the .gif itself will not move. It displays but loses its animation. If I change the form.visible to "false" and change the form to modal with adding a form.showdialog() parameter, the animation is perfect in the .gif but the script halts and can only progress after the window is closed. Is there a way for the script to progress without losing the animation in the .gif? Please help. I've found very little forums on powershell and .gif's for some reason. Here is the code for the form.
$Form = New-Object system.Windows.Forms.Form
$Form.Location= New-Object System.Drawing.Size(100,100)
$Form.Size= New-Object System.Drawing.Size(550,170)
$Form.StartPosition = "Manual"
$Form.Visible=$true
$Form.Enabled = $true

[reflection.assembly]::LoadWithPartialName("System.Windows.Forms")
$file = (get-item 'PathToMyGifFile')
$img = [System.Drawing.Image]::Fromfile($file);

[System.Windows.Forms.Application]::EnableVisualStyles();

$pictureBox = new-object Windows.Forms.PictureBox
$pictureBox.Location = New-Object System.Drawing.Size(0,1)
$pictureBox.Size = New-Object System.Drawing.Size(100,80)
$pictureBox.Image = $img
$Form.controls.add($pictureBox)

$Label1 = New-Object System.Windows.Forms.Label
$Label1.Text = "Please wait for Installation to complete"
$Label1.Location= New-Object System.Drawing.Size(110,35)
$Label1.Size = New-Object System.Drawing.Size(400,25)
$Label1Font = New-Object System.Drawing.Font("Tahoma",10,
[System.Drawing.FontStyle]::Bold)

$Label1.Font = $Label1Font
$Label1.BackColor = "white"
$Form.Controls.Add($Label1)

$Label2 = New-Object System.Windows.Forms.Label
$Label2.Text = "This may take several minutes . . ."
$Label2.Location= New-Object System.Drawing.Size(120,60)
$Label2.Size = New-Object System.Drawing.Size(370,75)
$Label2Font = New-Object System.Drawing.Font("Tahoma",10,
[System.Drawing.FontStyle]::Regular)

$Label2.Font = $Label2Font
$Label2.BackColor = "white"
$Form.Controls.Add($Label2)

$BackDrop = New-Object System.Windows.Forms.Label
$BackDrop.Location = New-Object System.Drawing.Size(0,0)
$BackDrop.Size = New-Object System.Drawing.Size(550,150)
$BackDrop.BorderStyle = [System.Windows.Forms.BorderStyle]::FixedSingle;
$BackDrop.BackColor = [System.Drawing.Color]::White

$Form.Controls.Add($WaitBackGroundBox)
$Form.Topmost = $True
[void] $Form.ShowDialog() # If absent, animation is lost. If present
                          # (combined with form.visible = $false), script
                          # halts until the form itself is closed.

Answer: I found a thread with a workaround for this that creates a new runspace and opens the form within it. That way the .gif runs normally and you can issue the form.close() command whenever you'd like further down in your code. This would come in handy just in case someone wants to use a .gif for something other than a Please Wait indicator. Posted from this site:
https://www.vistax64.com/powershell/16998-howto-create-windows-form-without-stopping-script-processing.html
$Form = New-Object system.Windows.Forms.Form
$Form.Location= New-Object System.Drawing.Size(100,100)
$Form.Size= New-Object System.Drawing.Size(550,170)
$Form.StartPosition = "Manual"
$Form.Visible=$false
$Form.Enabled = $true
$Form.Add_Shown({$Form.Activate()})

[reflection.assembly]::LoadWithPartialName("System.Windows.Forms")
$file = (get-item 'C:\path_to_your_gif.gif')
$img = [System.Drawing.Image]::Fromfile($file);

[System.Windows.Forms.Application]::EnableVisualStyles();

$pictureBox = new-object Windows.Forms.PictureBox
$pictureBox.Location = New-Object System.Drawing.Size(0,1)
$pictureBox.Size = New-Object System.Drawing.Size(100,80)
$pictureBox.Image = $img
$Form.controls.add($pictureBox)

$Label = New-Object System.Windows.Forms.Label
$Label.Text = "Please wait for Installation to complete"
$Label.Location= New-Object System.Drawing.Size(110,35)
$Label.Size = New-Object System.Drawing.Size(400,25)
$LabelFont = New-Object System.Drawing.Font("Tahoma",10,
[System.Drawing.FontStyle]::Bold)
$Label.Font = $PleaseWaitLabelFont
$Label.BackColor = "white"
$Form.Controls.Add($WaitLabel)

$WaitForm.Topmost = $True

$rs = [Management.Automation.Runspaces.RunspaceFactory]::CreateRunspace()
$rs.Open()
$rs.SessionStateProxy.SetVariable("Form", $Form)
$data = [hashtable]::Synchronized(@{text=""})
$rs.SessionStateProxy.SetVariable("data", $data)
$p = $rs.CreatePipeline({ [void] $Form.ShowDialog()})
$p.Input.Close()
$p.InvokeAsync()

## Enter the rest of your script here while you want the form to display

$WaitForm.close()
$rs.close()

Answer: Why not use the progressBar control implemented in WinForms?
With the "Marquee" animation speed, the progressBar will behave like a "Please Wait" and will not be stopped and much easier to add to your script instead of a gif.
Examples of using progressBar in Winforms : https://www.sapien.com/blog/2011/07/14/primalforms-2011-spotlight-on-the-progressbar-control/
Comment: Hm, surprisingly that control never came up when I was researching indicators. Thats pretty neat. If I can't fix the form with the .gif then thats not a bad workaround. Thanks!


Transparent Proxy for IPv6 traffic under Linux

Question: When maintaining networks, it is often an expedient thing to do to run a transparent proxy. By transparent proxy I mean a proxy that 'hijacks' outgoing connections and runs them through a local service. Specifically I run a linux firewall with squid configured so that all tcp/ip connections fowarded on port 80 are proxied by squid.
This is achived using the iptables 'nat' table, using IPv4.
But iptables for IPv6 does not have a 'nat' table, so I cannot use the same implementation. What is a technique I can use to transparently proxy traffic for IPv6 connections?
Answer: A viable way to do this is with the TPROXY rule in iptables, documentation is available here:

http://wiki.squid-cache.org/Features/Tproxy4#IPv6_Support
http://www.mjmwired.net/kernel/Documentation/networking/tproxy.txt

This should be supported Squid (>= version 3.2). Using --enable-linux-netfilter and the iptables -t mangle -j TPROXY rule.
Answer: iptables has a QUEUE target, which you can use to deliver packets to userspace.  I am not sure, but perhaps something could be implemented there.
Past that, you could take a stab at adding something to the kernel to do redirection.
Answer: You can't. Quoting from squid-cache.org:

NAT simply does not exist in IPv6. By
  Design.
Given that transparency/interception
  is actually a feature gained by
  secretly twisting NAT routes inside
  out and back on themselves. It's quite
  logical that a protocol without NAT
  cannot do transparency and
  interception that way.
Answer: Here's an implementation:
http://www.suse.de/~krahmer/ip6nat/
Answer: Another sort of ugly hack:

MARK all traffic with iptables (seems, there is CONNMARK target for IPv6)
route all marked traffic to tun device
do user-space NAT in the daemon listening at tun device
...
Comment: Is this actually possible? Any idea where I'd start with a tun implementation with ipv6 support?
Comment: IMHO, it SHOULD be possible. Linux tun/tap driver seems to support IPv6. Try tap driver (virtual ethernet) instead of tun if I'm wrong and IPv6 is not supported.

I don't know what may be done with iptables `QUEUE`, but tun-based solution should work, though it may be unsuitable for highload.
Answer: Write your own implementation of NAT in IPv6 stack.


Multidimensional distribution weak convergence for vectors with independent entries

Question: Suppose $X_n \overset d \to X, Y_n \overset d \to Y$ where $X_n$ and $Y_n$ are independent, as well as $X$ and $Y$. Then
$$
\begin{bmatrix}
X_n \\
Y_n
\end{bmatrix} \overset d \to
\begin{bmatrix}
X \\
Y
\end{bmatrix}.
$$
How do you prove that?

I am aware of this previously asked question, but the answer uses Stone-Weierstrass theorem which I don't get. I think author meant to approximate given bounded continuous function by linear combination of 2-variable polynomials of the form $P(x)Q(y)$, but I never used this theorem before, so I'm not sure. And besides, approximating with polynomials which are not bounded (that is most of them) may not satisfy $\int Q \operatorname d \! \mu_n \to \int Q \operatorname d \! \mu $ where $\mu$ and $\mu_n$ are appropriate distributions.

I tired to do this in a different way, but couldn't finish if off:
We want to show, that for any conitnous and bounded function $f:\mathbb R^2 \to \mathbb R$ we have
$$
\lim_{n\to \infty} \left|\int_{\mathbb R^2} f \operatorname d \! \mu_n \times \nu_n
- \int_{\mathbb R^2} f \operatorname d \! \mu \times \nu\right| = 0.
$$
Using triangle inequality as well as Fubini's theorem
$$\begin{align}
&\left|\int_{\mathbb R^2} f \operatorname d \! \mu_n \times \nu_n
- \int_{\mathbb R^2} f \operatorname d \! \mu \times \nu\right| \\  &\leq
\left|\int_{\mathbb R^2} f \operatorname d \! \mu_n \times \nu_n
- \int_{\mathbb R^2} f \operatorname d \! \mu \times \nu_n\right| +
\left|\int_{\mathbb R^2} f \operatorname d \! \mu \times \nu_n
- \int_{\mathbb R^2} f \operatorname d \! \mu \times \nu\right| \\ & =
\left|\int_{\mathbb R}\left(\int_{\mathbb R} f(x, y) \operatorname d \! \mu_n(x)
-\int_{\mathbb R} f(x, y) \operatorname d \! \mu(x) \right)\operatorname d \! \nu_n(y)\right| +
\left|\int_{\mathbb R} g \operatorname d \! \nu_n
- \int_{\mathbb R} g \operatorname d \! \nu\right|,
\end{align}$$
where $g(y) = \int_\mathbb{R}f(x, y) \operatorname d \! \mu(x)$. We can easly see that $g$ is bounded, and using dominated convergence theorem show it is continuous, so the last term will approach $0$.
However first term is quite problematic. Inside function will approach $0$ everywhere, but as the function changes so does the measure with respect to which we are integrating. In general it is easy to find sequence of continuous bounded functions $h_n$ which approach $0$, and sequence of probability measures $\xi_n$ such that $\int h_n \operatorname d\! \xi_n = 1$, for example $h_n(t) = (1 - |t - n|)^+$ and $\xi_n = \delta_{\{n\}}$. But in our problem we assume that the measure converges weakly, which my example does not satisfy.

Can anyone complete my proof or maybe explain solution of the previous question?
Comment: There is a one line proof using characterstic functions.
Comment: Well, I don't know about characteristic functions of random vectors yet. Perhaps its good time to learn them ...
Answer: The following is a proof of the weak convergence of multidimensional distribution for vectors
with independent (weak convergent) components.
Let $\mathscr{D}=\{f_u,u\in U\} $ be a collection of  functions in $C(\mathbb{R})$, which separates
points in $\mathbb{R}$. It is easy to verify that
\begin{equation*}
  \mathscr{D}_2=\{h_w=h_w(x,y)=(f_u(x),g_v(y)),\, w=(u,v)\in U\times U\}   \tag{1}
\end{equation*}
is also a collection of functions in $C(\mathbb{R}^2)$, which separates points in $\mathbb{R}^2$. Hence for each finite interval $I_M=[-M,M]$ and $ I_M^2=[-M,M]\times[-M,M]$, except at most a countable $M$,
\begin{gather*}
 \lim_{n\to\infty}\int_{I_M} f_u(x)\,\mu_n(\mathrm{d}x)=\int_{I_M} f_u(x)\,\mu(\mathrm{d}x), \quad \forall f_u\in\mathscr{D}, \\
 \lim_{n\to\infty}\int_{I_M} g_v(y)\,\nu_n(\mathrm{d}y)=\int_{I_M} g_v(y)\,\nu(\mathrm{d}y),\quad \forall g_u\in\mathscr{D},\\
\begin{aligned}
 \lim_{n\to\infty} &\int_{I_M^2} f_u(x)g_v(y)\,(\mu_n\times\nu_n)(\mathrm{d}x\times\mathrm{d}y)\\
 &=\int_{I_M^2} f_u(x)g_v(y)\,(\mu\times\nu)(\mathrm{d}x\times\mathrm{d}y),
 \qquad \forall f_ug_v\in\mathscr{D}_2.
\end{aligned}\tag{2}
\end{gather*}
Furthermore, using Stone-Weierstrass Theorem and from (2),
\begin{align*}
 &\lim_{n\to\infty} \int_{I_M^2} f(x,y)\,(\mu_n\times\nu_n)(\mathrm{d}x\times\mathrm{d}y)\\
 &\quad=\int_{I_M^2} f(x,y)\,(\mu\times\nu)(\mathrm{d}x\times\mathrm{d}y),
 \quad \forall f\in C(\mathbb{R}^2). \tag{3}
\end{align*}
Meanwhile, the $ \{\mu_n,\,n\ge 1\}, \{\nu_n,\,n\ge 1\} $ are tight, since they are weakly convergent, i.e.
\begin{gather*}
 \lim_{M\to\infty}\sup_{n\ge 1}\mu_n(\mathbb{R}\setminus I_M)=0,\\
 \lim_{M\to\infty}\sup_{n\ge 1}\nu_n(\mathbb{R}\setminus I_M)=0,
\end{gather*}
Hence,
\begin{equation*}
 \lim_{M\to\infty}\sup_{n\ge 1}[(\mu_n\times\nu_n)(\mathbb{R}^2\setminus I_M^2)]=0. \tag{4}
\end{equation*}
Now for a bounded function $f\in C(\mathbb{R}^2)$,
\begin{align*}
 \Big|\int_{\mathbb{R}^2}f\,\mathrm{d}&(\mu_n\times\nu_n)
  -\int_{\mathbb{R}^2}f\,\mathrm{d}(\mu\times\nu)\Big|\\
  & \le \Big|\int_{I_M^2}f\,\mathrm{d}(\mu_n\times\nu_n)
  -\int_{I_M^2}f\,\mathrm{d}(\mu\times\nu)\Big|\\
  &\quad +\|f\|[(\mu_n\times\nu_n)(\mathbb{R}^2\setminus I_M^2)]
  + \|f\|[(\mu\times\nu)(\mathbb{R}^2\setminus I_M^2)].
\end{align*}
Using above inequality and (3),
\begin{align*}
 &\varlimsup_{n\to\infty}\Big|\int_{\mathbb{R}^2}f\,\mathrm{d}(\mu_n\times\nu_n)
 -\int_{\mathbb{R}^2}f\,\mathrm{d}(\mu\times\nu)\Big|\\
 &\qquad \le \|f\| \{\sup_{n\ge 1}[(\mu_n\times\nu_n)(\mathbb{R}^2\setminus I_M^2)]
 +(\mu\times\nu)(\mathbb{R}^2\setminus I_M^2)\}
\end{align*}
Now letting $M\to\infty$ and from (4)  get
\begin{equation*}
 \lim_{n\to\infty}\int_{\mathbb{R}^2}f\,\mathrm{d}(\mu_n\times\nu_n)
 =\int_{\mathbb{R}^2}f\,\mathrm{d}(\mu\times\nu),\quad \forall f\in C_b(\mathbb{R}^2).
\end{equation*}


How to quickly compute the determinant of given matrix

Question: I need to find the determinant of given matrix :
$\begin{bmatrix}
1&0&0&0&0&2\\
0&1&0&0&2&0\\
0&0&1&2&0&0\\
0&0&2&1&0&0\\
0&2&0&0&1&0\\
2&0&0&0&0&1\\
\end{bmatrix}$
I know that It can be computed with the help of row operations;by applying
R1 $\to$ R1 + R2+R3+R4+R5+R6, and then subsequently simplifying using further row operations to find the determinant.
However my question is :
Is there another way to compute the determinant quickly ?,because the row operations are quite time taking and error prone.
I thought of using eigenvalues here but could not reach the solution
Can anyone help me to find this determinant quickly ?
Thank you
Comment: Hint: It is $-c^3$ for some small positive integer.
Answer: By reordering both rows and columns you can see that the determinant is equal to that of a block diagonal matrix:
$$
\begin{vmatrix}
1&2&0&0&0&0\\
2&1&0&0&0&0\\
0&0&1&2&0&0\\
0&0&2&1&0&0\\
0&0&0&0&1&2\\
0&0&0&0&2&1\\
\end{vmatrix}
= \left(\begin{vmatrix}
1&2 \\ 2&1
\end{vmatrix}\right)^3
$$

Or with Schur complements: We have
$$
 M = \begin{pmatrix}
A & B \\ C &D
\end{pmatrix}  \text{ with } A = D = \begin{pmatrix}
1 & 0 & 0 \\ 0 & 1 & 0 \\ 0& 0 & 1
\end{pmatrix}  \text{ and } B = C = \begin{pmatrix}
0 & 0 & 2 \\ 0 & 2 & 0 \\ 2& 0 & 0
\end{pmatrix}
$$
and therefore (with $I_3$ denoting the $3\times 3$ identity matrix):
$$
 \det M = \det(A) \det(D - CA^{-1}B) = \det(I_3 - B^2) = \det(-3I_3) = -27 \, .
$$
Comment: thanks ! it's a nice and elegant way which i didn't knew before
Answer: I think proceeding by finding eigenvalues is much harder than ERO in this case. The determinant of a matrix is the product of its eigenvalues, but to find those you would have to compute a more complicated determinant in the first place (to find $\chi(\lambda)$).
If you do these three simple operations:

$2R_1-R_6\to R_6$,
$2R_2-R_5\to R_5$,
$2R_3-R_4\to R_4$,

then the matrix becomes upper-triangular and the determinant is simply the product of the entries on the diagonal: $-27$.
Answer: Looking at the matrix, I think you can solve this problem recursively.
So, let $X_6$ your matrix (the matrix having the principal diagonal with $1$s and the secondary diagonal with only with $2$s. Then, using Laplace on the first row, you can obtain a relation with the 4x4 version of the matrix. $$\det{X_6} = -3\det{X_4}$$.
Then, continue applying this recurrence until you get to the 2x2 version. Then, not only you find the determinant of your matrix, but you can also generalize it to any such matrix with even number of rows.
Answer: or
$$ P^T H P = D  $$
$$\left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 & 0 & 0 \\
0 & 0 &  - 2 & 1 & 0 & 0 \\
0 &  - 2 & 0 & 0 & 1 & 0 \\
 - 2 & 0 & 0 & 0 & 0 & 1 \\
\end{array}
\right)
\left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 & 2 \\
0 & 1 & 0 & 0 & 2 & 0 \\
0 & 0 & 1 & 2 & 0 & 0 \\
0 & 0 & 2 & 1 & 0 & 0 \\
0 & 2 & 0 & 0 & 1 & 0 \\
2 & 0 & 0 & 0 & 0 & 1 \\
\end{array}
\right)
\left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 &  - 2 \\
0 & 1 & 0 & 0 &  - 2 & 0 \\
0 & 0 & 1 &  - 2 & 0 & 0 \\
0 & 0 & 0 & 1 & 0 & 0 \\
0 & 0 & 0 & 0 & 1 & 0 \\
0 & 0 & 0 & 0 & 0 & 1 \\
\end{array}
\right)
 = \left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 & 0 & 0 \\
0 & 0 & 0 &  - 3 & 0 & 0 \\
0 & 0 & 0 & 0 &  - 3 & 0 \\
0 & 0 & 0 & 0 & 0 &  - 3 \\
\end{array}
\right)
$$
$$ Q^T D Q = H  $$
$$\left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 & 0 & 0 \\
0 & 0 & 2 & 1 & 0 & 0 \\
0 & 2 & 0 & 0 & 1 & 0 \\
2 & 0 & 0 & 0 & 0 & 1 \\
\end{array}
\right)
\left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 & 0 & 0 \\
0 & 0 & 0 &  - 3 & 0 & 0 \\
0 & 0 & 0 & 0 &  - 3 & 0 \\
0 & 0 & 0 & 0 & 0 &  - 3 \\
\end{array}
\right)
\left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 & 2 \\
0 & 1 & 0 & 0 & 2 & 0 \\
0 & 0 & 1 & 2 & 0 & 0 \\
0 & 0 & 0 & 1 & 0 & 0 \\
0 & 0 & 0 & 0 & 1 & 0 \\
0 & 0 & 0 & 0 & 0 & 1 \\
\end{array}
\right)
 = \left(
\begin{array}{rrrrrr}
1 & 0 & 0 & 0 & 0 & 2 \\
0 & 1 & 0 & 0 & 2 & 0 \\
0 & 0 & 1 & 2 & 0 & 0 \\
0 & 0 & 2 & 1 & 0 & 0 \\
0 & 2 & 0 & 0 & 1 & 0 \\
2 & 0 & 0 & 0 & 0 & 1 \\
\end{array}
\right)
$$


How to find nearest neighbours to a dataset?

Question: I came across a (maybe simple) problem. Consider I have an "observation" vector of 12 values, called $O=[o_1,o_2,...,o_{12}]$. What would be the best way to compare $O$ to a "theoretical" database of $60000$ vectors ($T_j=[t_{j,1},t_{j,2},...,t_{j,12}], where \ j=1 \rightarrow 60000$) in order to find the nearest neighbors to $O$. In other words, I need to find what indexes among the $60000$ are the closest to $O$.
I performed a Chi-squared approach as follows:
$\chi_j=\sum_{i=1}^{12} (o_i-t_{j,i})^2$
Then the j-index corresponding to the minimum $\chi$ is the nearest neighbor and so on, but I'm not satisfied by this method, since I feel I'm comparing each coefficient alone, rather than comparing the observation vector (as a whole) to all theoretical vectors.
Is there a better way to do this? It might be that chi-squared is sufficient (explanation below), but I'm no expert in this field (neither in statistics). And if a so, is there a way to estimate the error or uncertainty due to this comparison (or fitting should i say)
The data in $O$ and $T$ are decimals (positive and negative). But what I'm worried about is actually the range of the theoretical data. Let me explain more:
$T=\begin{matrix}
t_{1,1} & t_{1,2} & \ldots  & t_{1,12}\\
t_{2,1} & t_{2,2} & \ldots & t_{2,12}\\
\vdots & \vdots & \ddots & \vdots \\
t_{n,1} & t_{n,2} & \ldots & t_{n,12}
\end{matrix} \ \ \ \ \ \ $ where $n=60000$, and   $O=[o_1,o_2,...,o_{12}]$
The minimum and maximum of each column vector of $T$ differs compared to another, such that the range of a column vector decreases as we move from one to the next. Let's say the {min,max}={-20,40} for the first column, and {min,max}={-2,3} for the 12th. Now here comes the problem (as I see it): a difference of 0.5 between $o_{1}$ and a value in the first column vector of $T$ isn't "weighted" equally as a difference of 0.5 between $o_{12}$ and a value in the 12th column vector of $T$. This is what I think the euclidean distance misses. I hope I made it clear enough.
Note: I'm new here, and sorry for any mistakes if I did any.
Comment: `since I feel I'm comparing each coefficient alone, rather than comparing the observation vector (as a whole) to all theoretical vectors` – you're not, you are comparing the full vectors, since the $\chi_j$ each depend on the sum of distances. This is a very reasonable approach; you're finding the nearest vector according to the standard Euclidean distance ($\ell_2$). It's not really a chi-squared distance, which is typically defined as $\frac12 \sum_i (x_i - y_i)^2 / (x_i + y_i)$ and is usually used only for count-type data.
Comment: If you give more information about what kind of data it is (are the $o_i$ counts? positive reals? arbitrary reals? ...), we might be able to suggest a more meaningful distance metric and/or how to think about uncertainty.
Comment: "Euclidean distance" thats the word thanks! I was afraid It wasn't. Anyways, the data are positive and negative real numbers (for $O$ and $T$), but I'll explain more by editing my question about why I think euclidean distance might not be the best approach. Thank you for your help
Comment: Please tell me whether my edited question needs more elaboration.
Answer: Now that you've expanded your description of the problem, I can suggest the Mahalanobis distance: letting $S$ be the covariance matrix of your dataset $T$, the distance is defined by
$$
d(o, t_j) = \sqrt{(o - t_j) S^{-1} (o - t_j)}
.$$
If the different attributes (1 through 12) are uncorrelated, then the covariance matrix will be diagonal and this corresponds just to standardizing the features by their relative spread (as measured by the standard deviation). In that case, the distance becomes
$$
d(o, t_j) = \sqrt{ \sum_{i=1}^{12}  \frac{(o_i - t_{ji})^2}{s_i^2} }
.$$
This standardized distance is also a very reasonable and typical thing to do on its own. The difference between the two is basically that the full Mahalanobis accounts for correlations between the components as well. If it's the case that, say, the 1st and 2nd components are usually very similar to one another, then the full Mahalanobis won't "double-count" that distance, whereas the standardized one will. Either choice can be reasonable depending on what your goals are for picking the "closest" points.
Comment: Thank you @Dougal I'll try this method (although covariance matrix might be an issue, computationally speaking). Moreover, is there a way to try to put some error bars on the fitting? Meaning, how much would I be sure that the best index I got is a good fit, or more generally, how many "indexes" are considered good fit?
Comment: Do you have a noise model on $o$?
Comment: Here comes the complicated part. I do not have a noise model on $O$ directly, it's actually indirect. The thing is that I'm using a dimensionality reduction technique in my work, PCA approach to be specific, to obtain the $O$ that we are discussing. So $O$ represents a higher order entity that, by itself, has a noise model (a signal-to-noise ratio should I say...).
Representing the noise in the lower order vector $O$ is something I didn't think of yet, but thank you for spawning this idea


Typing vs. Pool Playing vs. Playing Piano

Question: So, I have a great deal of ambidexterity that appears to be a result of being forced to be right handed when I was born naturally left. I golf left handed, eat left handed, etc. I also seem to have been born with a great deal of natural finger dexterity, despite not having overly large hands or long fingers (if you're gonna make a joke, I promise, I've heard it). I played professional pool for many years and as a result of which, my left hand is extremely limber. I can type around 100WPM (this drops 5WPM per shot of scotch, I know, I've experimented). Mathematically, I'm pretty lucky in that I like it and I'm good at the higher level maths. I can work a deck of cards. I have zero music ability or talent. Will any of my physical attributes carry over if I want to learn to play piano or am I just gonna have to work on another skill?
Comment: Being someone who generally enjoys learning new skills, who also started with zero ability or talent, and also being a bit mathematically minded, even if you have to work your ass off at it, I would expect music to one of the most rewarding possible skills to work your ass off at.
Comment: @ToddWilcox Absolutely.  A basic level of competency on pretty much any instrument can be the basis for so much joy, and personal growth!
Comment: But then I guess this opinion probably goes without saying on music SE
Answer: The physical aspect of learning any musical instrument involves a lot of specific motor skills and muscle memory, from gross positioning to fine touch. While general coordination from billiards and typing can help with basics, it's likely that musical timing and expressiveness will require entirely new skills. That said, you are not particularly disadvantaged either. The basics of learning a new instrument – physical familiarity with the layout, scales and chords, posture and touch – take an adult about 1–3 years regardless of prior experience. Ambidexterity does not make a lot of difference either, as most musical instruments require significant coordination from both hands, with a slight bias toward the right hand.
Comment: I hear the same thing said about guitar, but my experience with both as a left-hander is that right-hand technique is so essential to accuracy and expression that any perceived advantage to the left hand is illusory. It is a pain to wrangle a bow with your off hand.
Comment: Your comment brings me tears of joy. Yes, it is true that the right arm, at the very least, has all the expressive, virtuosic properties on said string instruments, although I would venture to draw a distinction between "arm" and "hand," especially since the original question and answer is in the context of piano.
Comment: Except on the violin, or basically any orchestral string, which is the exact opposite as far as hand bias goes.
Comment: @BraddSzonye With guitar I think it's probably true if you want to strum chords as basic accompaniment, but past that it no longer is and is probably the other way round.  I think this canard is essentially spread because it's true when you're a beginner guitarist trying to play basic chords and melodies.
Comment: @GeneralNuisance - totally agree. There's a lot of difference between 'arm' and 'fingers', which seems to be ignored when discussing 'handedness'. Jimi managed well, though...
Comment: There are a lot of brass instruments which pretty well favour right hand, leaving the left hand almost redundant, apart from support!


Instalador MSI - customizando o instalador

Question: Bom pessoal, primeiro preciso saber se é possível em um instalador MSI (utilizo o próprio 'setup project' que tem no VS) fazer a seleção de qual aplicação quero instalar.

Por enquanto tenho um instalador para cada projeto da minha solução, e agora quero centralizar tudo em um instalador só, mas quero que esse instalador tenha a opção de selecionar qual das aplicações quero instalar.
Até o momento estou utilizando um instalador MSI comum, mas que instala todos módulos de uma vez, não tendo a opção de selecionar qual eu quero instalar.
Answer: Você pode utilizar o Inno Setup junto com o ISTool
Nele você vai poder configurar um xml com todas as ações que deseja fazer, até mesmo cascata de installers. Você além de poder montar o seu installer nele, você vai poder colocar para executar outros installers após o seu.


Baily-Borel Compactification

Question: This is a reference request. I am new to this particular topic and the original paper is a bit difficult to read. I would be grateful if someone could provide a reference/examples where it has been worked out/expository set of notes.
Thanks.
Answer: I have found the following references.

Smooth Compactifications of Locally Symmetric Spaces - Ash, Mumford, Rapoport, Tai.
Espaces Hermitian Symmetriques - Deligne. There is a translation avaliable due to Milne on the webpage.
Differential Geometry, Lie Groups, and Symmetric Spaces - Helgason. Chapter 8 seems to be of some interest.

Any additions to the list are appreciated. Thanks.

Baily-Borel Compactification


Conditions to be a Joint Discrete Distribution Function

Question: I am reading a paper on modeling the dependencies in discrete distribution functions, and am having a hard time understanding the following.  Let us define: $$r \leq min(p,q)$$
$$ B(u,v) = \begin{cases}
0 & \text{if }u=0 \ \text{or } v=0 \\
r & \text{if }(u,v) \in (0,p] \times (0,q] \\
p & \text{if }(u,v) \in (p,1] \times (0,q] \\
q & \text{if }(u,v) \in (0,p] \times (q,1] \\
1 & \text{if }(u,v) \in (p,1] \times (q,1] \\
\end{cases} $$
$$ C(u,v) = \begin{cases}
r & \text{if }(u,v) \in [0,p) \times [0,q) \\
p & \text{if }(u,v) \in [p,1] \times [0,q) \\
q & \text{if }(u,v) \in [0,p) \times [q,1] \\
1 & \text{if }(u,v) \in [p,1] \times [q,1] \\
\end{cases} $$
The paper then says that $C(u,v)$ is a distribution function, while $B(u,v)$ is NOT a distribution function.
Can anyone explain why this is the case? I do notice that C(u,v) is defined in a right continuous manner (which is typically how distribution functions are defined?), is that why?  Or is there something else that I am missing.  I can provide more context if necessary, but it seems to me that this is a standalone thing.
Thanks!
Answer: If $B$ and $C$ are supposed to be joint cumulative probability distribution functions (joint CDFs), then you need to be aware that a joint CDF $F(\cdot,\cdot)$ must satisfy the "rectangle constraint" (a name that I just made up and so don't bother Googling for it). This rectangle constraint says that for all real numbers $a,b,c,d$ such that $a<c, b<d$, $F(c,d)-F(a,d)-F(c,b)+F(a,b)$, which equals
$P\{a < X \leq c, b < Y \leq d\}$ must be nonnegative:
$$F(c,d)-F(a,d)-F(c,b)+F(a,b) = P\{a < X \leq c, b < Y \leq d\} \geq 0.$$
For discrete random variables in particular, if $F(u,d)$ has constant
value $p$ for $a \leq u <c$ and so does $F(c,v)$ have fixed value $q$
for all $b \leq v < d$, then $F(c,d)$ must be larger than $\max(p,q)$.
As an example, consider the function
$$B(u,v) = \begin{cases} 0, & u<0, v<0,\\
0, & 0 \leq u < 1, 0 \leq v < 1,\\
1, & \text{otherwise}
\end{cases}$$
which is right-continuous in both variables and has all the properties
that one might desire of a joint CDF except that
$$B(1,1)-B(0,1)-B(1,0) + B(0,0) = -1$$ and so is disbarred from
the ranks on joint CDFs by the rectangle constraint.
Comment: Hi all, thanks again for the detailed answers!  I'm still trying to sort through what is stated in the answer above.  One comment is, in Nelsen's Introduction to Copula's book, the Rectangle formula you provided is called the H-Volume, and in the book he does mention that the H-volume of a joint CDF must be $\geq$ 0.  See Definition 2.1 of http://w4.stern.nyu.edu/ioms/docs/sg/seminars/nelsen.pdf
Comment: If I take the constraint stated above for discrete random variables, then only the definition of $C(u,v)$ has relevant values (i.e. it has constant value $q$ for $0 \leq u <  p$ and constant value $p$ for $0 \leq v < q$), but $B(u,v)$ doesn't have any relations like that.  I think you gave all the concepts to understand why $C(u,v)$ is a valid joint CDF, but $B(u,v)$ is not ... but to be honest, I'm still kind of lost.  $B(u,v)$ doesn't define any points in the manner you've defined the required constraint of $F(c,d)$ must be larger than max(p,q).  Is it because $B(u,v)$ is left-continuous?
Comment: Reading a little further in the paper, it is clearly stated: "B cannot be a distribution function because it is not right-continuous."   .. thanks for the help and clarification anyway.
Comment: +1 Your careful use of strict inequalities $\lt$ and inequalities $\le$ clearly exposes the key idea.
Comment: @whuber Thanks for the upvote. I think, though, that the issue in the OP's question is one of left-continuity versus right-continuity which, as mark Stone points out, is more a matter of convention (if I recall correctly, Loeve's GTM tome also defines the CDF as $P\{X < x\}$).
Comment: Yes--and that's precisely why I think careful use of the inequalities, as well as a clear quotation of the definition, are critical in this case.
Comment: @Dilip Sarwate, right you are, or should I say, left you are regarding Loeve's book .  See p. 177 of https://books.google.com/books?id=_9xWBlvUEuIC&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q&f=false . Then again, he was at Berkeley, so maybe that explains his leftist leanings.  So Breiman and Loeve are the only two "modern": (although they are both now deceased) non-Russian authors I know of using the left continuity convention.
Answer: Probability distribution functions are defined so as to be right continuous ... unless you're in Russia, in which case they are defined so as to be left continuous.
Leo Breiman, best known to youngsters as the inventor of random forests and the person who coined the term "bagging", was perhaps a Commie, based on, for example, his 1973 book "STATISTICS: With a View Toward Applications", in which he uses the Russian, i.e., left continuous, convention, and never bothers informing the reader that he is departing from the standard convention outside of Russia.  Or look at Definition 2.21 at the bottom of p. 25 in his 1968 book "Probability".
Comment: Let's not use pejorative terms like Commie and just call Breiman a leftist. Much more useful in getting the point across, wouldn't you say?
Comment: Just trying to interject a little levity.  ,,, And I did say "perhaps".


Mailmerge macro produces empty email after 200 emails created

Question: I've been playing with this macro for days now and when I thought it was working well I discovered that it works properly only with the first 200 emails. After that, it creates emails with proper recipient and subject, but no text and no attachments. After testing different scenarios, it seems that (some kind) of Outlook memory gets filled, but I don't know what and how to clear it ( i added oItem and oOutlookApp = nothing without success). The only way I can get it to work is if I close Outlook and run the macro again with the 200 and following emails.
Any ideas?
thanks
EDITS:
1- I also tried clear the clipboard at the end of the loop using the accepted answer overhere, alas with no result.
2- I found this answer that seems related to my problem. Two major differences though: my macro runs from Word to Outlook (not Outlook to Excel) and I don't get an error message; emails beyond #200 are simply created empty. So i don't know if/how it can be of help here.
3- Following niton's remark, there is now an error message. Progess I guess...
The highlighted line is
<code>.Attachments.Add Trim(Datarange.Text), olByValue, 1
</code>
And it does that on the 200th email.
<code>' MailMerge Macro
'
'
Sub MergeWithAttachments()
Dim Source As Document, Maillist As Document, TempDoc As Document
Dim Datarange As Range
Dim i As Long, j As Long
Dim bStarted As Boolean
Dim oOutlookApp As Outlook.Application
Dim oItem As Outlook.MailItem
Dim mysubject As String, message As String, title As String
Dim mailWord As Object
Dim oData As New DataObject

    Set Source = ActiveDocument
    ' Check if Outlook is running.  If it is not, start Outlook
    On Error Resume Next
    Set oOutlookApp = GetObject(, "Outlook.Application")
    On Error GoTo 0
    If Err <> 0 Then
        Set oOutlookApp = CreateObject("Outlook.Application")
        bStarted = True
    End If
' Open the catalog mailmerge document
    With Dialogs(wdDialogFileOpen)
        .Show
    End With
    Set Maillist = ActiveDocument
    ' Show an input box asking the user for the subject to be inserted into the email messages
    message = "Enter the subject to be used for each email message."    ' Set prompt.
    title = "Email Subject Input"    ' Set title.
    ' Display message, title
    mysubject = InputBox(message, title)
' Iterate through the Sections of the Source document and the rows of the catalog mailmerge document,
' extracting the information to be included in each email.
If MsgBox("Are you adding cc email recipients?", vbYesNo, "CC email") = vbYes Then
    If MsgBox("Are your cc email recipients in the second column from the left?", vbYesNo, "CC in second column") = vbYes Then
        GoTo Add_cc
        Else:
        If MsgBox("Cc email recipients need to be in the second column. Please rework your directory accordingly.", vbOKOnly, "Cancelling Mail Merge") = vbOK Then
        Exit Sub
        End If
No_cc:

For j = 1 To Source.Sections.Count - 1

    Source.Sections(j).Range.Copy
    Set oItem = oOutlookApp.CreateItem(olMailItem)
    Set mailWord = oItem.GetInspector.WordEditor

    With oItem
        .Subject = mysubject
        mailWord.Range.PasteAndFormat (wdFormatOriginalFormatting)
        Set Datarange = Maillist.Tables(1).Cell(j, 1).Range
        Datarange.End = Datarange.End - 1
        .To = Datarange
        For i = 2 To Maillist.Tables(1).Columns.Count
            Set Datarange = Maillist.Tables(1).Cell(j, i).Range
            Datarange.End = Datarange.End - 1
            .Attachments.Add Trim(Datarange.Text), olByValue, 1
        Next i
        .Send
    End With
    Set oItem = Nothing
Next j

GoTo Merge_finished

Add_cc:
For j = 1 To Source.Sections.Count - 1

    Set oItem = oOutlookApp.CreateItem(olMailItem)

    With oItem
        .Subject = mysubject
        Set Datarange = Maillist.Tables(1).Cell(j, 1).Range
        Datarange.End = Datarange.End - 1
        .To = Datarange
        'code for adding cc emails. Currenlty set to read column 2 as cc emails
        Set Datarange = Maillist.Tables(1).Cell(j, 2).Range
        Datarange.End = Datarange.End - 1
        .CC = Datarange.Text
        Source.Sections(j).Range.Copy
        Set mailWord = oItem.GetInspector.WordEditor
        mailWord.Range.PasteAndFormat (wdFormatOriginalFormatting)

        For i = 2 To Maillist.Tables(1).Columns.Count
            Set Datarange = Maillist.Tables(1).Cell(j, i).Range
            Datarange.End = Datarange.End - 1
            .Attachments.Add Trim(Datarange.Text), olByValue, 1
        Next i
        .Send
        End With
    Set oItem = Nothing

Next j

Merge_finished:
End If
Else: GoTo No_cc
End If

Maillist.Close wdDoNotSaveChanges
'  Close Outlook if it was started by this macro.
If bStarted Then
    oOutlookApp.Quit
End If
MsgBox Source.Sections.Count - 1 & " messages have been sent."
'Clean up
Set oOutlookApp = Nothing

End Sub
</code>
Comment: Can you share the word document? Why do you have `Set oOutlookApp = CreateObject("Outlook.Application")` in your second loop? I don't think you need that, remove it. I also see your adding a lots of data to the clipboard, make sure to clear it to avoid errors
Comment: Put an On Error Goto 0 just after Set oOutlookApp = GetObject(, "Outlook.Application"). There should always be an On Error Goto 0 as soon a the On Error Resume Next is no longer needed.
Comment: @Om3r - can't share the word document beacause it contains email addresses. And besides, I've narrowed down the problem to be with Outlook, not Word or the Clipboard. I've added the oOoutloopApp thing when I was tired and desperatly trying anything! Will remove it!
Comment: @niton - I'll do that but I doubt it will solve the problem I have as this piece is just to open Outlook if it's not opened already (I think).
Comment: I don't mean the real data, just add fake data
Comment: @Pomul  "There should always be an On Error Goto 0 as soon a the On Error Resume Next is no longer needed." The problems, if any, are hidden when you do not turn off the On Error Resume Next. Now if there is an error message you will get to see it so you can fix it. Always have an On Error Goto 0 when there is an On Error Resume Next.
Comment: @niton - I now get an error message. see edit.
Comment: @Pomul  It is progress in your VBA programming knowledge. You know debugging is impossible without the matching On Error Goto 0. I was a little hasty the On Error Goto 0 should be just before ' Open the catalog mailmerge document.  1) What line is highlighted? 2) Did you get to 200?
Comment: Ah, I see. The On Error Resume Next is good until cancelled kind of thing! The highlighted line is `.Attachments.Add Trim(Datarange.Text), olByValue, 1` And it does that on the 200th email
Answer: You may be running into this. Outlook macro runs through 250 iterations before failing with error
https://support.microsoft.com/en-us/kb/830836
"This issue occurs because of a limit on the number of items that clients can open. By default, this limit is set to 100 for attachments and 250 for messages."
Your limit could be set to 200. If you cannot fix this, if the cause, try changing your code. Mark processed items or moving them, with a down counting loop starting at 200. Close Outlook after 200 items are processed. Reopen and process the remainder 200 at a time.
Comment: Thank you for your help on this, niton.


On a bijection between symmetric subsets of a group

Question: Given a group $G$, we can consider the subset $H$ of $G$ defined by:
$$ H = \{ xyz : x, y, z\in G \textrm{ and } x, y, z \textrm{ are pairwise distinct}\}$$
Let $a\in G$ be arbitrary element. I am interested in understanding the map
$f_a: H\to H$ defined by
$$
f_a(xyz)=(xa)(ya)(za)
$$
It is not hard to see that image of $f_a$ lies in $H$ (because, for example, if $xa=ya$, then $x=y$), so indeed $f$ is well defined. Also, we can show that $f_a$ is surjective: Given any $xyz\in H$, we have
$$ f_a(xa^{-1}ya^{-1}za^{-1})=xyz$$
So my question is:

Is it true that $f_a$ is always a bijection?

Here are some partial results:
a) If $G$ is finite (in which case $H$ is also finite), then $f_a$ is clearly bijection, as every surjection between finite sets is bijection.
b) If $G$ is abelian, then $f_a$ is bijection. This is because $f_a$ can be shown to be injection: If $f_a(xyz)=f_a(x'y'z')$, then $(xa)(ya)(za)=(x'a)(y'a)(z'a)$, and since $G$ is abelian $xyza^{3}=x'y'za^{3}$ which gives $xyz=x'y'z'$, as desired.
So it remains to investigate the case when $G$ is infinite non-abelian group.
By the way, I called $H$ the "symmetric subset" in the title, but feel free to edit it if some other term is more appropriate.
Answer: This is not well-defined, because the same element $h$ might be written as $xyz$ in many different ways, with different values of $(xa)(ya)(za)$.
Comment: Thanks for catching that! That was tricky, and I am not surprised that I missed it...
Answer: The map $xyz\mapsto (xa)(ya)(za)$ is not generally well-defined. For example, consider the free group generated by $x$ and $a$: here $xx^{-1}e$ and $x^{-1}xe$ are supposedly sent to different elements.
Comment: Thanks for the explicit example!


nonexpansiveness of prox-grad mapping

Question: Let $f,g$ be two convex functions and $g$ has Lipchitz continuous gradient. The proximal - gradient mapping, which defined as
$$ x \mapsto \textrm{prox}_{\lambda f} \left( x - \lambda \nabla g \left( x \right) \right) . $$

How can I prove that the proximal - gradient mapping is nonexpansive?

I only able to prove it when $g \equiv 0$. In particular, for any
$$ x^{+} = \textrm{prox}_{\lambda f} \left( x - \lambda \nabla g \left( x \right) \right) \textrm{ and } y^{+} = \textrm{prox}_{\lambda f} \left( y - \lambda \nabla g \left( y \right) \right) $$
we have that
$$ x - \lambda \nabla g \left( x \right) - x^{+} \in \lambda \partial f \left( x^{+} \right) \textrm{ and } y - \lambda \nabla g \left( y \right) - y^{+} \in \lambda \partial f \left( y^{+} \right) $$
which, by the monotonicity of $\partial f$ implies further
$$ \left\langle  x - \lambda \nabla g \left( x \right) - x^{+} - \left( y - \lambda \nabla g \left( y \right) - y^{+} \right) , x^{+} - y^{+} \right\rangle \geq 0 $$
or equivalently
$$ \left\langle  x - y - \lambda \left( \nabla g \left( x \right) - \nabla g \left( y \right) \right) , x^{+} - y^{+} \right\rangle \geq \left\lVert x^{+} - y^{+} \right\Vert ^{2} . $$
Infact, when $g \equiv 0$ we have that
$$ \left\langle  x - y , x^{+} - y^{+} \right\rangle \geq \left\lVert x^{+} - y^{+} \right\Vert ^{2} $$
which means the $\textrm{prox}$ mapping is not only nonexpansive but also firmly nonexpansive. However, the appearance of $g$ make me confuse. I tried to use the Cauchy - Schwarz inequality then Lipchitz continuity of $\nabla g$ but it leads to some constant bigger than $1$:
\begin{align*}
\left\lVert x^{+} - y^{+} \right\Vert ^{2} & \leq \left\langle  x - y , x^{+} - y^{+} \right\rangle - \lambda \left\langle \nabla g \left( x \right) - \nabla g \left( y \right) , x^{+} - y^{+} \right\rangle \\
& \leq \left\lVert x - y \right\Vert \left\lVert x^{+} - y^{+} \right\Vert + \lambda \left\lVert\nabla g \left( x \right) - \nabla g \left( y \right) \right\Vert \left\lVert x^{+} - y^{+} \right\Vert \\
& \leq \left( 1 + \lambda L \right) \left\lVert x - y \right\Vert \left\lVert x^{+} - y^{+} \right\Vert .
\end{align*}
Can anyone tell me where did I miss?
Comment: The shortest (but not easiest) way of proving this is to use operator theory (like your tag suggests). Basically, the prox operator is equivalent to applying $(I+t\partial f)^{-1}$ on $x$, where $\partial f$ is the subdifferential of $f$. Then $x^+ = (I+t\partial f)^{-1}(I-t\nabla g)x$. It can be shown that $(I-t\nabla g)$ contracts by factor $tL$ where $L$ is the Lipshitz constant of $\nabla g$ (so you pick $t=1/L$), and $ (I+t\partial f)^{-1}$ is nonexpansive. (See the Eckstein / Bertsekas paper for a better proof of that.) Then your proof should be 3 lines.
Comment: correction, it contracts by a factor of $1-tL$, not $tL$.
Comment: yeah. I just doubled checked my "proof" and realized it was incorrect. I'll think on this some more...
Comment: Yeah so the stuff you added was exactly what I was thinking, and the mistake was exactly what you pointed out: it's (1+tL) not (1-tL). This is the nicest proof I can find, I think it's the same one in Nesterov's introductory lectures: https://www.cs.cmu.edu/~ggordon/10725-F12/slides/05-gd-revisited.pdf
Comment: By the way, here is a nice overall proof for proximal gradient convergence. https://people.eecs.berkeley.edu/~elghaoui/Teaching/EE227A/lecture18.pdf
Since he introduces nonexpansiveness for prox but not for the prox + gradient, I guess actually, without strong convexity, prox + gradient maybe isn't contractive. (Even for strongly convex, I don't think the proof is obvious... all I've seen (now that I'm looking) is $x^k-x*$, not $x^+-x$...)
Comment: @whyyes can you please give me a hint on proving $\left( I - t \nabla g \right)$ is contracts by a factor of $1 - tL$? I can only prove it nonexpasive, otherwise I have to assume $g$ to be strongly convex
Comment: but I think that $\left( I - t \nabla g \right) ^{-1}$ is nonexpansive is still enough as $\left( I + t \partial f \right)$ is nonexpansive as well, therefore the composition is also nonexpansive :)
but it would be very interesting if we can check that whether $\left( I - t \nabla g \right) ^{-1}$ is contraction or not without strongly convexity assumption ^^
Answer: Here is a sketch for when $\nabla g$ is $L$-Lipschitz continuous and $\lambda\in]0,1/L[$. This uses the tools from Bauschke/Combettes' book vol. 2, mostly chapters 4 and 18.
Since $g$ is convex, the Baillon-Haddad theorem implies that $\lambda\nabla g$ is firmly nonexpansive. Therefore, $\textrm{Id} - \lambda\nabla g$ is firmly nonexpansive, hence nonexpansive.
Since $f$ is convex, $\textrm{prox}_{\lambda f}$ is also firmly nonexpansive, hence nonexpansive.
The proximal gradient operator (more generally called the "forward-backward" operator) is nonexpansive since it is the composition of two nonexpansive operators (in fact, it is $2/3$-averaged).

nonexpansiveness of prox-grad mapping


How to add JSESSIONID as optional cookie as parameter using post in Java Spring Boot

Question: How to add JSESSIONID as optional cookie when posting username and password for login in controller?
LoginController:
@PostMapping("/login")
@ResponseBody
public ResponseEntity<Account> login(@RequestParam(name ="username") String name, @RequestParam(name ="password") String password) {
    if(name == "user" || password == "password")
    {
        System.out.println("Matched!");
        return new ResponseEntity<>(HttpStatus.OK);
    }
    return new ResponseEntity<>(HttpStatus.UNAUTHORIZED);
}

Answer: Spring Framework provides the @CookieValue annotation to read any cookie by specifying the name.
So you can use this annotation with required property set to false.
If you use don't set required property default value will be true,and  an exception will be thrown if the cookie is missing in the request.
Alternatively, provide a defaultValue(), which implicitly sets this flag to false.
@PostMapping("/login")
    @ResponseBody
    public ResponseEntity<Account> login( @CookieValue(name = "JSESSIONID", required=false) String cookie,@RequestParam(name ="username") String name, @RequestParam(name ="password") String password) {
        if(name == "user" || password == "password")
        {
            System.out.println("Matched!");
            return new ResponseEntity<>(HttpStatus.OK);
        }
        return new ResponseEntity<>(HttpStatus.UNAUTHORIZED);
    }


Getting worse search performance with a replica shard

Question: To increase search performance, I tried to add a replica to my cluster.
Initially, I measured a response time of around 700ms for a specific request with a single node with a primary shard.
After adding a node and a replica shard to the cluster, it takes averagely 2000ms to get the result of the request (so it almost tripled).
I'm using these configs for the replica :
<code>cluster.name: findmyfpstore
node.name: fmfs_r1
node.master: false
network.host: ...
http.port: ...
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ['...']
index.number_of_shards: 1
index.number_of_replicas: 1
</code>
Am I doing something wrong ? Feel free to tell me if you need more informations, I'm a beginner at ElasticSearch.
Thanks a lot!
EDIT: I tried with 2 shards, 1 replica, it doesn't change much
Comment: How many Elasticsearch nodes are you running? What sort of specs are these machines? Are you using SSD?
Comment: Hi, there are two nodes, each of the nodes are running on one different server (32go RAM, 8 cores). They don't use SSD.
Comment: Since you're using a single shard, I'm not sure you would see any significant performance boost by adding a replica, however, you also shouldn't get any performance hit because of it. What happens if you use 2 shards and 1 replica?
Comment: Are multiple shards useful if the data volume is really low ? I only have < 1k documents and it probably won't be higher. I'm trying it and I'll give you the results as soon as possible.
Comment: What is your document size? Can you share your query? Maybe we can improve that..
Comment: Document size is 829, I use two different queries, the first one tries to recover the closest store from a specified location, it looks like that :
geoDistanceSorter := elastic.NewGeoDistanceSort("location").
   Point(aCoord.Lat, aCoord.Lon).
   Unit("km").
   GeoDistance("plane").
   Asc()

  // query execution
  matchAllQuery := elastic.NewMatchAllQuery()
  searchResult, err := client.Search().
   Index(indexName).
   Type(storeType).
   Query(matchAllQuery).
   Size(1).
   SortBy(geoDistanceSorter).
   Do() <=> uses a match all query with a geo distance sort
Comment: Try using the profile api on both single shard and 1 shard 1 replica. And see the difference. https://www.elastic.co/guide/en/elasticsearch/reference/current/search-profile.html
Answer: There are a lot of elements that can affect the performance of your cluster and query speeds.
I suggest you follow Elasticsearch's Production Deployment document, this will give you a lot of insight on how to properly configure your cluster.


Erro ao importar pygame no Pycharm

Question: Estou aqui somente porque já faz uma semana que tento de todas as maneiras instalar o pygame no PyCharm e não estou conseguindo. Já tentei pela linha de comando e por diversas maneiras, mas quando consigo instalar, aparece um erro.
O erro que aparece quando consigo instalar é este:

[d:\build\libmpg123\mpg123-1.25.10\src\libmpg123\id3.c:482] error: No
  comment text / valid description?

Já o outro erro que aparece e não deixa instalar é este:

Traceback (most recent call last):   File
  "C:/Users/Rafael/PycharmProjects/exercicios/ex021.py", line 1, in

      import pygame ModuleNotFoundError: No module named 'pygame'

Estou fazendo apenas um exercício simples onde quero tocar um arquivo em mp3. Segue abaixo o código:
import pygame
pygame.init()
pygame.mixer.music.load('ex021.mp3')
pygame.mixer.music.play()
pygame.event.wait()

Já tentei pelo pip install pygame, já tentei clicando na luz vermelha direto no próprio PyCharm, já tentei por settings - project interpreter, enfim, acredito que seja alguma incompatibilidade de versões.
Estou utilizando o Windows, Python 3.8.2 PIP 20.0.2 e o pygame que estou tentando instalar é o pygame 1.9.6 Alguém consegue me ajudar, por favor?
Answer: Eu consegui resolver isso, especificando a biblioteca pygame usando mixer:
from pygame import mixer
mixer.init()

mixer.music.load('Hoist.mp3')
mixer.music.play()
input('Agora você escuta?')

Answer: Simionato, tente baixar o pygame em: https://pypi.python.org/pypi/pygame
Selecione a opção
pygame-1.9.3-cp35-cp35m-win32.whl
Na pasta de instalação do Python (C:\Users\SeuUsuario\ AppData\Local\Programs\Python\Python"versão"\Scripts), coloque o arquivo que foi baixado.
Acesse, pelo CMD, a pasta do Python e atualize o pip com a seguinte linha de comando: python -m pip install --upgrade pip.
Ainda no CMD, digite cd Scripts e tecle enter; Instale o pygame, com o comando pip install pygame-1.9.3-cp35-cp35m-win32.whl
Comment: Muito obrigado pela ajuda, ainda não consegui, não encontrei o arquivo que me passou do pygame-1.9.3-cp35-cp35m-win32.whl (no meu caso acho que seria o 64,)enfim, ao digitar cd Scripts, também não encontra nada... Mas tentei isso tudo na corrida hoje, porém, não consegui... Amanha tentarei novamente... Muito obrigado pela ajuda. Grande abraço.
Comment: Quanto a não conseguir acessar a pasta, o seu Python foi instalado via download direto ou pelo Anaconda?

Pode tentar esse abaixo:

v64
https://files.pythonhosted.org/packages/45/a5/580578790aa2f8f274855d45b4c89ffb910f1c72ddb754873b2ae8d7fc7f/pygame-1.9.6-cp35-cp35m-win_amd64.whl

v32 https://files.pythonhosted.org/packages/6d/ca/6c54a92efc66ef33e78cb003148c637debd3a85eb44a8d93df19c1f2f9c6/pygame-1.9.6-cp35-cp35m-win32.whl
Answer: Eu consegui resolver.
Na pasta do seu projeto no Pycharm, exclua a pasta "interpreter" e volte no Pycharm > settings > Project: Nomedoseuprojeto > Project Interpreter e adicione outro "interpreter". E aí é só adicionar o pygame, seja pelo terminal, "cd diretóriodoprojeto...\Interpreter\Scripts > pip install pygame" ou pela interface do Pycharm mesmo.
Answer: Eu consegui resolver o problema do 'pygame' no Pycharm 3.8.5. Vá em: File>Sttings>Project:"o nome do seu projeto">Python Interpreter> Em Package, clique em +> agora procure no canto esquerdo da tela que vai abrir na lupa de pesquisa: pygame. Agora é só clicar em install Package. PRONTO!!!
Answer: mano, eu passei pelo mesmo erro de instalação do pygame, mas consegui resolver instalando pelo pip.
faz assim:
1) abre cmd como administrador
2) executa o comando pip list
3) se tiver pygame instalado tu executa: pip uninstall pygame
4) e instala novamente: pip install pygame
5) pip list novamente para ver se instalou
6) roda no IDLE do python um código para ver se funciona

Erro ao importar pygame no Pycharm


Completion of $\mathbb{Q}$ under trivial absolute value

Question: i have a basic question on completions.
Define the trivial absolute value $|\cdot|_0$ on $\mathbb{Q}$ by $|x|_0 = 1$ if $x\neq 0$ and by $|0|_0=0$.
Is it true that the completion of $\mathbb{Q}$ under $|\cdot|_0$ is again $\mathbb{Q}$? If so, how can you prove it?
My attempt: We want to show that $\mathbb{Q}$ is complete under the trivial absolute value, so take any Cauchy sequence of rationals... then i don't see why its limit is again in $\mathbb{Q}$.
Thanks in advance!
Answer: The idea is to see what a Cauchy sequence is under the trivial absolute value. Indeed, note that $|\cdot|$  takes only two values, $1$ and $0$. Therefore, if $a_n$ is Cauchy, there exists $N$ such that $n,m > N \implies |a_n - a_m| < \frac 12$ so $a_n  - a_m = 0$ is forced i.e. $a_n = a_m$.
Hence, all Cauchy sequences are eventually constant i.e. there is some $N$ such that after $N$ terms the sequence looks like $a,a,a,a,a,a...$ for some $a$. Now it is obvious what the limit of such a Cauchy sequence is : it is $a$ itself, and since $a$ belongs to the sequence it belongs to the rational numbers, rendering $\mathbb Q$ complete.
Of course, nothing special about $\mathbb Q$ : the trivial norm is complete on any (non-empty) set, since the same argument will apply.
Answer: Because there is a $p\in\mathbb N$ such that$$(\forall m,n\in\mathbb{N}):m,n\geqslant p\implies|x_m-x_n|<1\implies x_m=x_n.$$That is, the sequence becomes constant after a certain point and therefore it converges to that constant.


Does uninstalling psa-firewall (plesk firewall module) disable firewall?

Question: I have a dv server with media temple and the plesk firewall module is causing a tonne of grief, it use to work, not anymore... unable to edit firewall rules through the module etc.
Does uninstalling this module, what i assume is known as the psa-firewall service prevent the firewall to work? its running centos 5 and the firewall looks to be iptables.
Answer: that is just the package for plesk to interface with iptables with.. removing that and your firewall will still run.. to turn off your firewall
service iptables stop

if you don't want it to run on reboot
chkconfig iptables off

Comment: Iptables isn't a service, it can't be stopped, it's built into the kernel, you can just configure it to pass all traffic
Comment: you are sort of correct.. but in redhat/centos it has a init.d script when stopped flushes the rules and unloads the module. Also it can be done via chkconfig so iptables rules are never loaded
Answer: check the clean link by IgorG from parallels :)


How to use the Transferable protocol in SwiftUI tables

Question: I am building a macOS app in Swift, using SwiftUI. I have the following struct:
struct focusArticle: Identifiable {
    let category: String
    let headline: String
    let body: String
    let publishedDate: String
    let source: String
    var id = UUID()
}

I have created an array with instances of focusArticle that are being used to populate a Table element, like so
struct ArticleView: View {
    @State private var selectedArticles = Set<focusArticle.ID>()

    var body: some View {
        Table(articles, selection: $selectedArticles) {
            TableColumn("Category", value: \.category)
                .width(70)
            TableColumn("Headline", value: \.headline)
                .width(350)
            TableColumn("Published", value: \.publishedDate)
        }
        .frame(width: 600, height: 200)
    }
}

I want to make each row in this table draggable, so I can drag-drop from this table, to a similar table in another view. From reading Apple's documentation, it seems like I need to do two things.

Make my focusArticle struct conform to the Transferable protocol.
Use the .draggable modifier on the table to make the elements draggable.

I can't seem to do either of these things. Copying over the example code from the documentation into my struct makes it throw an error — Static method 'buildBlock' requires that 'String' conform to 'TransferRepresentation'. Adding a .draggable modifier to the table throws an expected error: Instance method 'draggable' requires that '[focusArticle]' conform to 'Transferable'
EDIT: This is one of the ways I have tried to make focusArticles confirm to Transferable.
struct focusArticle: Identifiable, Codable, Transferable {
    let category: String
    let headline: String
    let body: String
    let publishedDate: String
    let source: String
    var id = UUID()

    static var transferRepresentation: some TransferRepresentation {
        CodableRepresentation(contentType: .focusArticle)
    }
}

This shows an error Type 'UTType' has no member 'focusArticle'
Answer: You need to add a UTType, as an extension like this:
extension UTType {
    static var focusArticle = UTType(exportedAs: "com.yourdomain.yourapp.focusarticle")
}


How do I find an average of values in dataset2 for each individual contained in dataset1, who have unique start and end dates?

Question: I am looking to find the average of valueZ for each individual based on their unique start and end date. Exposure X has numerous values for each day, for each location, as such (There are actually 23 locations with over 300 values & dates per site):
data dataset2;
input date location valueZ;
datalines;
1/1/2016 1 0.028
1/1/2016 1 0.022
...
2/8/2016 1 0.041
2/8/2016 1 0.044
1/1/2016 2 0.056
...
8/8/2016 2 0.089
1/1/2016 3 0.029
...
11/8/2016 3 0.083
...
1/1/2016 4 0.081
...
12/8/2016 4 0.019
...
10/30/2016 23 0.063
;

The individuals in dataset1 are linked to dataset 2 by the location as such:
data dataset1;
input individual location start_date end_date;
datalines;
1 1 1/1/2016 12/31/2016
2 1 3/12/2016 9/4/2016
3 2 2/5/2016 11/5/2016
4 19 9/30/2016 10/3/2016
5 23 4/12/2016 12/12/2016
...
305 16 1/20/2016 5/15/2016
;

So, I would like to end up with the average of valueZ based on the location indicated in dataset2 from start_date to end_date for each individual. Can someone please help!
Answer: Something like this?
proc sql;
  create table want as
  select d1.individual
        ,d1.location
        ,avg(d2.valueZ) as avg_value
  from dataset2 d2
  join dataset1 d1
    on d1.location=d2.location
    and d2.date between d1.start_date and d2.end_date
  group by d1.individual, d1.location
quit;

Comment: Yes! This is so much cleaner than what I was trying to do in IML and it worked wonderfully. Thank you!
Comment: Is there a way to produce the number of days that were missing values within each average?
Comment: Dates with missing values of VALUEZ? Dates that do not appear in the data? You might try COUNT(DISTINCT d2.date) to see how many different date values you have. You could then subtract from number of days from start to end to see how many where missing.
Comment: perfect. Thank you so much for your help.


A Gaussian Mixture Model Is a Universal Approximator of Densities

Question: When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following:

A Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any specific nonzero amount of error by a Gaussian mixture model with enough components.

I only have a basic background in probability and statistics, and I have absolutely no idea what this section is saying.
I would greatly appreciate it if people could please take the time to explain this in a way that is more understandable to someone of my level.
Comment: I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules.  After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job.  Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation.
Answer: The idea is that an arbitrary density on $\mathbb{R}$, $f(\cdot)$, can be approximated by a Gaussian mixture model
$$g_k(\cdot;\boldsymbol{\omega,\mu,\sigma})=\sum_{i=1}^k \omega_i \varphi(\cdot;\mu_i,\sigma_i)$$
in the sense that
$$\mathfrak{D}(f(\cdot),g_k(\cdot;\boldsymbol{\omega,\mu,\sigma}))\stackrel{k\to\infty}{\longrightarrow}0$$
for some specific functional distance $\mathfrak{D}(\cdot,\cdot)$. This result only applies for weaker types of distance.
Comment: Thank you for the crisp answer. Could you elaborate on what you mean by weaker types of distance?
Comment: Are isotropic gaussian mixture models universal Approximators of densities?


All open coverage, admits an enumerable sub-coverage

Question: Prove that if $X \subseteq \mathbb{R^n}$, then all open coverage of $X$, admits an enumerable sub-coverage.
If {$C_{n}$} $_{n \in I}$ is a cover of $X$, we say that it is open, if $C_{n}$ is open for all $n \in I$.
If {$C_{n}$} $_{n \in I}$ is a coverage of $X$, we say that the coverage is enumerable, if $I$ is enumerable.
Comment: HINT: Use the fact that $\Bbb R^n$ is second countable (i.e., has a countable base).
Comment: @BrianM.Scott how would that be?
Comment: Are you asking why $\Bbb R^n$ is second countable, or how that fact can be used?
Comment: @BrianM.Scott wonder how you can use this fact?
Answer: HINT: Let $\mathscr{B}$ be a countable base for $\Bbb R^n$. Let $\mathscr{U}$ be any relatively open cover of $X$. For each $x\in X$ there are a $U_x\in\mathscr{U}$ and a $B_x\in\mathscr{B}$ such that $x\in B_x\cap X\subseteq U_x$. Let $\mathscr{B}_X=\{B_x:x\in X\}$. $\mathscr{B}_X\subseteq\mathscr{B}$, so $\mathscr{B}_X$ is countable, and clearly $X\subseteq\bigcup\mathscr{B}_X$. For each $B\in\mathscr{B}_X$ there is a $U_B\in\mathscr{U}$ such that $B\cap X\subseteq U_B$; why? What can you say about $\{U_B:B\in\mathscr{B}_X\}$?
Comment: Can it then be said that {$U_{B}: B \in \mathscr{B_{X}}$} is an enumerable subcoverage for each open?
Comment: @holiswis: Simpler than that: it's a countable relatively open subcover of $X$.
Answer: $\Bbb R^n$ is second countable:  just consider the open balls with rational radii and centered at points with rational coordinates.  These form a basis for the standard topology.
It now follows easily that $\Bbb R^n$ has the Lindelof property (which is another name for the property you are referring to).


Finding the value of the determinant $|M^2+MN^2|$ and determining whether $U$ is a zero matrix or not

Question:
Let $M$ and $N$ be two $3\times 3$ matrices such that $MN=NM$. Further
  if $M\neq N^2$ and $M^2=N^4$ then what is/are the possible value(s) of
  the determinant $|M^2+MN^2|$? Also if there is a $3\times 3$ matrix
  $U$ such that $(M^2+MN^2)U=O$ can we say whether $U$ is a zero matrix or
  a non-zero matrix ?

My Attempt:
$|M|=\pm|N|^2$ (from $M^2=N^4$)

$(M-N)(N-M)=M^2-N^2$ (as $MN=NM$)

$M^2=N^4$
$\implies MMN=N^5$
$\implies NMMN=N^6$
$\implies MNMN=N^6$
$\implies (MN)^2=N^6$

Similarly, $M^6=(MN)^4$

These are what I could deduce from the given information. How should I proceed from here to solve the question?
Comment: The elements are real?
Comment: @EugenCovaci Yes.
Comment: I don't understand the condition "such that $(M^2+MN^2)U$"
Comment: @EugenCovaci Sorry, corrected! $O$ stands for zero matrix.
Answer: Let's first note that $M$ and $N^2$ commute because $M$ and $N$ do. Because of this, we may write $$0 = M^2 - N^4 = (M-N^2)(M+N^2).$$
This tells us that $|M+N^2| = 0$, since if it wasn't, you could multiply both sides by the inverse $(M+N^2)^{-1}$ to find $0 = M - N^2$ and hence $M = N^2$, which isn't allowed.
But $M^2 + MN^2 = M(M+N^2)$, hence $$|M(M+N^2)| = |M||M+N^2| = |M| \cdot 0 = 0.$$
As for the second part, you can't say anything about $U$. For example, take $M = -I$ and $N = I$. These satisfy the conditions, but $M^2 + MN^2 = 0$, so obviously $(M^2 + MN^2)U = 0$ for any matrix $U$.
Comment: How can we say $M^2+MN^2=0$ ? It is possible that even if matrix is non zero it's determinant can be zero. So $(M^2+MN^2)U=0$ is not necessarily True. Isn't it?
Comment: @mathophile I didn't claim $M^2+MN^2 = 0$ anywhere, only that its determinant $|M^2+MN^2| = 0$.
Comment: Then How your this statement so obviously $(M^2 + MN^2)U = 0$ for any matrix $U$. can be true?
Comment: Ah, yes, I did claim $M^2+MN^2 = 0$ there, I missed that in my previous reply. However, I'm talking about the example $M = -I$ and $N = I$ there, not the general case.
Comment: So the general statement **there is a $3 \times 3$ non-zero matrix $$ such that $(^2 + M^2)$ is the zero matrix.** This question was asked in Jee advanced exam 2014 Paper 1. There they gave this statement as true. I think this statement should be wrong.
Comment: Source https://www.jeeadv.ac.in/archive.html
Comment: That statement ( (B) in the exam paper you mention) is true, but it's not covered by this question and my answer. To prove it, you could use the Jordan form to show that $M^2+MN^2$ is similar to a matrix whose first column consists of only zeroes. If you want a more detailed answer, I suggest you start a new question.


how to save image name into database

Question: when i run the following code, the image gets moved to the desired folder but the nothing gets saved in the database. how to i save the name of the image in database. Please Help...
<code>        mysqli_query($con,"INSERT INTO blog (title, image, content)
              VALUES ('$_POST[title]','$_POST[image]','$_POST[content]')");

    $target_Path = "uploaded/";
              $target_Path = $target_Path.basename( $_FILES['image']['name'] );
              move_uploaded_file( $_FILES['image']['tmp_name'], $target_Path );
</code>
Comment: Not really, he wants to have that image there. ;)
Comment: ... Because you didn't include a value for that column?  The second to last line in your code references the image name.  Why don't you try using that?
Answer: You need three items to match the three columns you're inserting data into.
<code> mysqli_query($con,"INSERT INTO blog (title, image, content) VALUES ('$_POST[title]', '' ,'$_POST[content]')");
</code>
You also need to rethink your strategy of passing user inputs directly into the database without first sanitizing them.  http://en.wikipedia.org/wiki/SQL_injection
Answer: Is missing the image value in your query, try this:
<code>mysqli_query($con,"INSERT INTO blog (title, image, content)
VALUES ('$_POST[title]', '$_FILES[image][name]' ,'$_POST[content]')");
</code>
Comment: it worked with the file is being saved in .bin format in database. help!

how to save image name into database


Determine all simple zero rings

Question: A ring $R$ is simple if it has no ideals other than ${0}$ and $R$. A ring in which every product is zero is called a zero ring. Determine all simple zero rings.
I want to say that the only simple zero ring is ${0}$ as a subring of the integers, but I have a feeling that by the way the problem is worded there are more. How can I find other simple zero rings?
Answer: Note that in a zero ring, every subspaces is an0 ideal.
So the question is about determining all simple abelian groups; then you can endow them with theh zero product and make them simple zero rings.
Comment: I believe that all simple abelian groups are the cyclic groups of prime order. Is that correct? If so, can I just say that the simple zero rings are these groups with the zero product?
Comment: Yes. $\ \ \ \ $

Determine all simple zero rings


Python coding questions on bitwise shift and logical operations in packet sniffer

Question:

import socket
import sys
import struct
import re

#get data
def receiveData(s):

 data=''
 try:
  data=s.recvfrom(65565)
 except timeout:
  data = ''
 except:
  print "An error occurred"
  sys.exc_info()
 return data[0]

#get Type Of Service - 8bits
def getTOS(data):

 precedence = {0:'Routine',1:'Priority', 2:'Immediate', 3:'Flash', 4:'Flash Override', 5:'CRITIC/ECP', 6:'Internetwork control', 7:'Network Control'}
 delay = {0:'Normal Delay', 1:'Low Delay'}
 throughput = {0:'Normal Throughput', 1:'High Throughput'}
 reliability = {0:'Normal Reliability', 1:'High Reliability'}
 monetary = {0:'Normal Cost', 1:'Minimize monetary cost'}

 D = data & 0x10
 D >>= 4
 T = data & 0x8
 T >>= 3
 R = data & 0x4
 R >>= 2
 M = data & 0x2
 M >>=1

 tabs = "\n\t\t\t"
 TOS = precedence [data >> 5]+ tabs + delay[D] + tabs + throughput[T] + tabs + reliability[R] + tabs + monetary[M]
 return TOS

def getFlags(data):

 flagR = {0:'Cleared to 0'}
 flagDF = {0:'Fragment if necessary', 1:'Do not Fragment'}
 flagMF = {0:'This is the last Fragment', 1:'More fragments to follow this fragment'}

 R = data & 0x8000
 R >>= 15
 DF = data & 0x4000
 DF >>= 14
 MF = data & 0x2000
 MF >>= 13

 tabs = '\n\t\t\t'
 flags = flagR[R] + tabs + flagDF[DF] + tabs + flagMF[MF]
 return flags

def getProtocol(protocolNr):

 protocolFile = open('/root/Desktop/protocol.txt', 'r')
 protocolData = protocolFile.read()
 protocol = re.findall(r'\n' + str(protocolNr) + ' (?:.)+\n', protocolData)
 if protocol:
  protocol = protocol[0]
  protocol = protocol.replace('\n', '')
  protocol = protocol.replace(str(protocolNr), '')
  protocol = protocol.lstrip()
  return protocol
 else:
  return "No such protocol found"

s=socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_TCP)

data = receiveData(s)
unpackedData = struct.unpack('!BBHHHBBH4s4s', data[:20])

version_IHL = unpackedData[0]
version = version_IHL >> 4
IHL = version_IHL & 0xF
TOS = unpackedData[1]
totalLength = unpackedData[2]
ID = unpackedData[3]
flags = unpackedData[4]
fragmentOffset = unpackedData[4] & 0x1FFF
TTL = unpackedData[5]
protocolNr = unpackedData[6]
checksum = unpackedData[7]
sourceAddress = socket.inet_ntoa(unpackedData[8])
destinationAddress = socket.inet_ntoa(unpackedData[9])

print " An IP packed with the size %i was captured :" %totalLength
print " Raw Data :" +data
print "\nParsed Data"
print " Version:\t\t" +str(version)
print " Header Length:\t\t" + str(IHL*4) + "bytes"
print " Type of Service:\t" + getTOS(TOS)
print " ID:\t\t\t" + str(hex(ID)) + " (" + str(ID) + ")"
print " Flags:\t\t\t" + getFlags(flags)
print " Fragment Offset:" + str(fragmentOffset)
print " TTL:\t\t\t" + str(TTL)
print " Protocol:\t\t\t" +getProtocol(protocolNr)
print " Checksum:\t\t\t" +str(checksum)
print " Source:\t\t\t" +sourceAddress
print " Destination:\t\t\t" +destinationAddress
print " Payload:\n" + data[20:]

Below I have extracted some code sections from above and kindly spend some time to answer the following.
    (1)
D = data & 0x10
D >>= 4
T = data & 0x8
T >>= 3
R = data & 0x4
R >>= 2
M = data & 0x2
M >>=1

tabs = "\n\t\t\t"
TOS = precedence [data >> 5]+ ......

in the above section could you please explain on what basis D=data & 0x10 is used and right shifted 4 times and also why is precedence shifted 5 times to the right. My question is more like why 0x10 was specifically used to AND with data and then right shifted 4 times to get the Delay bit. I am seeking clarification on all other bits like T, R, M in the above section and the flag bits in the below section too.
     (2)
R = data & 0x8000
R >>= 15
DF = data & 0x4000
DF >>= 14
MF = data & 0x2000
MF >>= 13

In the above segment please explain on what basis is used R = data & 0x8000 and shift it 15 times to the right
      (3)
fragmentOffset = unpackedData[4] & 0x1FFF

In the above line of code why specifically 0x1FFF was used to AND with unpackedData[4]
Credits to the source code goes to Ana Balica
    https://www.youtube.com/watch?v=ghokDuCDcMY
Comment: In order to align the bits to the right so they can be read as normal numbers.
Comment: @IgnacioVazquez-Abrams could you please elaborate with example
Answer:

import socket
import sys
import struct
import re

def receiveData(s):
    data =''
    try:
        data , addr =s.recvfrom(655365)
    except timout:
        data =''
    except:
        print "An error hapend"
        sys.exc_info()
    return data

#get Type Of Service - 8bits
def getTOS(data):

 precedence = {0:'Routine',1:'Priority', 2:'Immediate', 3:'Flash', 4:'Flash Override', 5:'CRITIC/ECP', 6:'Internetwork control', 7:'Network Control'}
 delay = {0:'Normal Delay', 1:'Low Delay'}
 throughput = {0:'Normal Throughput', 1:'High Throughput'}
 reliability = {0:'Normal Reliability', 1:'High Reliability'}
 monetary = {0:'Normal Cost', 1:'Minimize monetary cost'}

 D = data & 0x10
 D >>= 4
 T = data & 0x8
 T >>= 3
 R = data & 0x4
 R >>= 2
 M = data & 0x2
 M >>=1

 tabs = "\n\t\t\t"
 TOS = precedence [data >> 5]+ tabs + delay[D] + tabs + throughput[T] + tabs + reliability[R] + tabs + monetary[M]
 return TOS

def getFlags(data):

 flagR = {0:'Cleared to 0'}
 flagDF = {0:'Fragment if necessary', 1:'Do not Fragment'}
 flagMF = {0:'This is the last Fragment', 1:'More fragments to follow this fragment'}

 R = data & 0x8000
 R >>= 15
 DF = data & 0x4000
 DF >>= 14
 MF = data & 0x2000
 MF >>= 13

 tabs = '\n\t\t\t'
 flags = flagR[R] + tabs + flagDF[DF] + tabs + flagMF[MF]
 return flags

def getProtocol(protocolNr):

 protocolFile = open('protocol.txt', 'r')
 protocolData = protocolFile.read()
 protocol = re.findall(r'\n' + str(protocolNr) + ' (?:.)+\n', protocolData)
 if protocol:
  protocol = protocol[0]
  protocol = protocol.replace('\n', '')
  protocol = protocol.replace(str(protocolNr), '')
  protocol = protocol.lstrip()
  return protocol
 else:
  return "No such protocol found"

def showdata(unpackedData):
    version_IHL = unpackedData[0]
    version = version_IHL >> 4
    IHL = version_IHL & 0xF
    TOS = unpackedData[1]
    totalLength = unpackedData[2]
    ID = unpackedData[3]
    flags = unpackedData[4]
    fragmentOffset = unpackedData[4] & 0x1FFF
    TTL = unpackedData[5]
    protocolNr = unpackedData[6]
    checksum = unpackedData[7]
    sourceAddress = socket.inet_ntoa(unpackedData[8])
    destinationAddress = socket.inet_ntoa(unpackedData[9])

    print " An IP packed with the size %i was captured :" %totalLength
    print " Raw Data :" +data
    print "\nParsed Data"
    print " Version:\t\t" +str(version)
    print " Header Length:\t\t" + str(IHL*4) + "bytes"
    print " Type of Service:\t" + getTOS(TOS)
    print " ID:\t\t\t" + str(hex(ID)) + " (" + str(ID) + ")"
    print " Flags:\t\t\t" + getFlags(flags)
    print " Fragment Offset:" + str(fragmentOffset)
    print " TTL:\t\t\t" + str(TTL)
    print " Protocol:\t\t\t" +getProtocol(protocolNr)
    print " Checksum:\t\t\t" +str(checksum)
    print " Source:\t\t\t" +sourceAddress
    print " Destination:\t\t\t" +destinationAddress
    print " Payload:\n" + data[20:]

# the public network interface
HOST = socket.gethostbyname(socket.gethostname())

# create a raw socket and bind it to the public interface
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
s.bind((HOST, 0))

# Include IP headers
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)

# receive all packages
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)

for i in range(100):
    # receive a package
    #print s.recvfrom(65565)
    data=receiveData(s)
    datastr = str(data)
    if len(datastr) >19 :
        unpackedData = struct.unpack('!BBHHHBBH4s4s',datastr[:20])
        #print unpackedData
        showdata(unpackedData)
    else:
        print data

# disabled promiscuous mode
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)

import socket
import sys
import struct
import re

def receiveData(s):
    data =''
    try:
        data , addr =s.recvfrom(655365)
    except timout:
        data =''
    except:
        print "An error hapend"
        sys.exc_info()
    return data

#get Type Of Service - 8bits
def getTOS(data):

 precedence = {0:'Routine',1:'Priority', 2:'Immediate', 3:'Flash', 4:'Flash Override', 5:'CRITIC/ECP', 6:'Internetwork control', 7:'Network Control'}
 delay = {0:'Normal Delay', 1:'Low Delay'}
 throughput = {0:'Normal Throughput', 1:'High Throughput'}
 reliability = {0:'Normal Reliability', 1:'High Reliability'}
 monetary = {0:'Normal Cost', 1:'Minimize monetary cost'}

 D = data & 0x10
 D >>= 4
 T = data & 0x8
 T >>= 3
 R = data & 0x4
 R >>= 2
 M = data & 0x2
 M >>=1

 tabs = "\n\t\t\t"
 TOS = precedence [data >> 5]+ tabs + delay[D] + tabs + throughput[T] + tabs + reliability[R] + tabs + monetary[M]
 return TOS

def getFlags(data):

 flagR = {0:'Cleared to 0'}
 flagDF = {0:'Fragment if necessary', 1:'Do not Fragment'}
 flagMF = {0:'This is the last Fragment', 1:'More fragments to follow this fragment'}

 R = data & 0x8000
 R >>= 15
 DF = data & 0x4000
 DF >>= 14
 MF = data & 0x2000
 MF >>= 13

 tabs = '\n\t\t\t'
 flags = flagR[R] + tabs + flagDF[DF] + tabs + flagMF[MF]
 return flags

def getProtocol(protocolNr):

 protocolFile = open('protocol.txt', 'r')
 protocolData = protocolFile.read()
 protocol = re.findall(r'\n' + str(protocolNr) + ' (?:.)+\n', protocolData)
 if protocol:
  protocol = protocol[0]
  protocol = protocol.replace('\n', '')
  protocol = protocol.replace(str(protocolNr), '')
  protocol = protocol.lstrip()
  return protocol
 else:
  return "No such protocol found"

def showdata(unpackedData):
    version_IHL = unpackedData[0]
    version = version_IHL >> 4
    IHL = version_IHL & 0xF
    TOS = unpackedData[1]
    totalLength = unpackedData[2]
    ID = unpackedData[3]
    flags = unpackedData[4]
    fragmentOffset = unpackedData[4] & 0x1FFF
    TTL = unpackedData[5]
    protocolNr = unpackedData[6]
    checksum = unpackedData[7]
    sourceAddress = socket.inet_ntoa(unpackedData[8])
    destinationAddress = socket.inet_ntoa(unpackedData[9])

    print " An IP packed with the size %i was captured :" %totalLength
    print " Raw Data :" +data
    print "\nParsed Data"
    print " Version:\t\t" +str(version)
    print " Header Length:\t\t" + str(IHL*4) + "bytes"
    print " Type of Service:\t" + getTOS(TOS)
    print " ID:\t\t\t" + str(hex(ID)) + " (" + str(ID) + ")"
    print " Flags:\t\t\t" + getFlags(flags)
    print " Fragment Offset:" + str(fragmentOffset)
    print " TTL:\t\t\t" + str(TTL)
    print " Protocol:\t\t\t" +getProtocol(protocolNr)
    print " Checksum:\t\t\t" +str(checksum)
    print " Source:\t\t\t" +sourceAddress
    print " Destination:\t\t\t" +destinationAddress
    print " Payload:\n" + data[20:]

# the public network interface
HOST = socket.gethostbyname(socket.gethostname())

# create a raw socket and bind it to the public interface
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
s.bind((HOST, 0))

# Include IP headers
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)

# receive all packages
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)

for i in range(100):
    # receive a package
    #print s.recvfrom(65565)
    data=receiveData(s)
    datastr = str(data)
    if len(datastr) >19 :
        unpackedData = struct.unpack('!BBHHHBBH4s4s',datastr[:20])
        #print unpackedData
        showdata(unpackedData)
    else:
        print data

# disabled promiscuous mode
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)


Trying to make a google maps chart that shows different markers, but i have an un-kown syntax error

Question: I'm trying to make a map that shows different makers using lat&long. cords but I was confused on the documentation of google chart maps. So i must've made a mistake I cannot find could anyone help?
Well here's my code:

<!doctype html>

Answer: The Google Map Chart displays a map using the Google Maps API. Data values are displayed as markers on the map. Data values can be coordinates (lat-long pairs) or addresses. The map will be scaled so that it includes all the identified points.
The syntax error means that a character or string incorrectly placed in a command or instruction that causes a failure in execution.

If you want your maps to be line drawings rather than satellite imagery, use a geochart instead.


Live search utilizando React-Redux

Question: Olá, estou tentando criar um live search com o Redux, porém ele somente da o match com a palavra completa. Qual é o melhor método de realmente retornar para a minha tabela, nesse caso, somente os valores que a string em pesquisar contém?
Table
  const brandsList = useSelector(state => state.brands.brandList);
            <TableBody>
                {brandsList.slice(0, rowsPerPage).map((brand, i) => (
                  <TableRow className={classes.tableRow} hover key={i}>
                    <TableCell>
                      <Typography variant="body1">{brand.name}</Typography>
                    </TableCell>
                    <TableCell>
                      <Typography variant="body1">{brand.CNPJ}</Typography>
                    </TableCell>
                    <TableCell>
                      <Typography variant="body1">{brand.email}</Typography>
                    </TableCell>
                    <TableCell>
                      <Typography variant="body1">{brand.address}</Typography>
                    </TableCell>
                    <TableCell>
                      <Link to={`/brand/${brand.id}`}>
                        <IconButton aria-label="edit">
                          <EditIcon />
                        </IconButton>
                      </Link>
                    </TableCell>
                  </TableRow>
                ))}
              </TableBody>```

Reducer

case fromTypes.SEARCHING_BRANDS:
      const {value} = action;
      const works = state.brandList.filter(item => item.name === value);
      return {
        ...state, value, works
      }

Answer: O código fornecido está incompleto, mas dá para perceber o que está acontecendo.
const works = state.brandList.filter(item => item.name === value);

Essa linha filtra o array brandList com itens cujo name seja idêntico ao value.
Se você quiser filtrar os itens cujo name inclua o value, então:
const works = state.brandList.filter(item => item.name.includes(value));

Você provavelmente também vai querer ignorar se as letras estão maúsculas e/ou minúsculas. Para isso, utilize toLowerCase() no name e no value.

Live search utilizando React-Redux


Traktor audio 6 recognizes only one deck when pluging into my mac pro, what should i do?

Question: Last weekend i used my traktor audio 6 interface as usually and it was working fine. Yesterday, when i attemped to use it again, it only show me the enable botton at one deck(As you may know this interface when connecting into the usb port have to be activated by enabling each deck to work with), the other remained as any interface was plugged in.
What should i do in this case? does anyone knows how to fix this kind of issue?
Thank you in advance
Answer: I can't post this as a comment yet, so it'll have to be an answer. Usually this happens when the audio signal is screwed up somehow coming from the table. Set Traktor to scope view and see what the signal looks like. Check the settings in the audio 6 control panel- is direct monitoring on? Are the line input sensitivities set properly? Did you try rebooting the Macbook and/or plugging the audio 6 in on a different USB? Are you using an external power supply, or are you powering over the USB? Are you using timecode vinyl? Are your needles clean/connected properly? Are your grounds connected? Do you have the deck set to CD/LINE rather than to TURNTABLE/PHONO? There are many things it could be; a better description of what it is doing might help. :)
Hope this helps.


Edit a config variable of a custom module in controller

Question: Is it possible to edit/create a config variable of a module in a controller created in the same module?
For example:
module_name/src/Controller/MyController.php:
class MyController extends ControllerBase {

    public function x($y) {
        $config = \Drupal::config('module_name.settings');
        $config->set('var1', $y)->save();
    }
}

Update:
module_name.module:
function module_name_page_attachments(array &$page) {
  $config = \Drupal::config('module_name.settings');
  var_dump($config->get('var1'));
}

module_name/config/install/module_name.settings.yml:
var1: 0

module_name/src/Form/moduleNameSettings.php:
class moduleNameSettings extends ConfigFormBase {

    protected function getEditableConfigNames() {
        return [
            'module_name.settings',
        ];
    }

    public function submitForm(array &$form, FormStateInterface $form_state) {
        parent::submitForm($form, $form_state);
        $config = \Drupal::config('module_name.settings');

        $this->config('module_name.settings')
            ->set('var1', $config->get('var1'))
            ->save();
    }

Comment: In the submitForm method you seem to be setting the variable to its existing value rather than updating it.
Comment: I'd say: Sure! Have you tried it already?
Comment: yes I did but it doesn't work I still can't get the new value of "var1" in module_name.module file! (Note: calling the controller is fine no problem with controller invoke)
Comment: I think that's the way to update it! what should I do instead?
Comment: See `ExampleSettingsForm::submitForm()` from [_Working with Configuration Forms_](https://www.drupal.org/docs/8/api/configuration-api/working-with-configuration-forms).
Answer: With the ConfigFormBase, you need to implement the editableConfigNames() method.  E.g.
  /**
   * {@inheritdoc}
   */
  protected function getEditableConfigNames() {
    return [
      'example.settings',
    ];
  }

In general, you need to get the editable version of the config file to change it.  ConfigFormBase (well actually, ConfigFormBaseTraits) will only return the editable config is you implement the above method.
For reference, here's how you could do this without the ConfForm logic.
$config = \Drupal::service('config.factory')->getEditable('example.settings');
$config->set('message', 'Hi')->save();

See the documents for details.
Since you are expanding ConfigFormBase, you should be able to use:
$config = $this->configFactory()->getEditable('example.settings');

To get the editable version.
Comment: IIUC they're already relying on the base class and using [`ConfigFormBaseTrait::config()`](https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Form%21ConfigFormBaseTrait.php/function/ConfigFormBaseTrait%3A%3Aconfig) (which does return the editable settings, assuming `getEditableConfigNames()` is set up correctly, which it seems to be).
Comment: D'oh.. look at my own code first.. editing answer.
Comment: `getEditableConfigNames()` has already implemented I just didn't mention it in the question trying to focus on `submitForm()` method.., and it works for me I use `$config = \Drupal::service('config.factory')->getEditable('example.settings');
$config->set('message', 'Hi')->save();` in the controller body. thank's a lot! :)


Order of ordinals

Question: Let $R_i$ is an ordinal for every $i\in n$ (where $n$ is also an ordinal) and $S_{i,j}$ is an ordinal for every $i\in n$ and $j\in m_i$ (where $m_i$ is also an ordinal for every $i\in n$).
What is a formula for the ordinal representing the pair $(R_i;S_{i,j})$ in the lexicographic order of pairs?
Comment: For fix $i,j$ the pair $(R_i; S_{i,j})$ is represented by their sum.
Comment: @azarel: "Their sum"? What is "their"? I don't understand.
Comment: the pair $(R_i;S_{i,j})$ is represented by $R_i+S_{i,j}$.
Comment: @azarel: $R_i+S_{i,j} < R_{i'}+S_{i',j'}$ isn't equivalent (consider the case of natural numbers) to $R_i<R_{i'}\wedge (R_i=R_{i'}\vee S_{i,j}<S_{i',j'})$ (the condition for lexigraphical order). It seems that you err.
Comment: I guess you mean the order type of $(\{(R_i;S_{i,j}:i\in n,\ j\in m_i\},<_{lex})$.
Comment: @azarel: You guess right.
Comment: @azarel: I've said "right". No not right. I mean the type corresponding to the element $(R_i;S_{i,j})$ in that set, not the order type of the entire set.
Comment: The question has no nice answer unless you know more about the $R_i$ and $S_{i,j}$. Are the $R_i$ increasing with $i$? Are the $S_{i,j}$ increasing with $j$?
Comment: @Brian M. Scott: There are no reason to assume that these are increasing.
Answer: This question is not really well defined, as an ordered pair under the usual definition $ (\alpha,\beta) := \{\{\alpha\},\{\alpha,\beta\}\} $ is not a well-orered set (as no order is given). However, assuming any order on it, its order type would be, as a finite set with two elements $ \{\alpha\} $ and $\{\alpha,\beta\}$, equal to $2$. It is worth noting that that is true for any sets. The distinction between $ \alpha $ and $ \{\alpha\} $ is critical - the former is a set with $ card(\alpha) $ elements, the latter is a set with one element, namely $ \alpha $.
However, the comments and the remark on lexicographical order seem to indicate you actually mean the order type of the set $ \{ (R_i,S_{i,j}): i \in n, j \in m_i \} $ under lexicographical order. The lexicographical ordering means that the order the $ R_i $ appear in the series does not matter, and neither for $ S_{i,j} $.
Define $ R = \{R_i:i \in n\} $ and $ S_i = \{S_{i,j}:j \in m_i \} $ and $ S = \bigcup S_i $. Then we're looking at the order type of $ R \times S $. And with $ m=\bigcup m_i $, this in turn is going to be equal to $mn$ if there's not "too much overlap" in the $ S_{i,j} $ - otherwise, we have to resort to the cardinality of $S$ as factor instead of $m$.
Now, all that said, I suspect you wanted to ask a different question: Actually meaning "order type of $R_i$ followed by $S_{i,j}$" instead of "ordered pair". This is by definition $R_i + S_{i,j}$, something very different from the ordered pair.
But in that case, "lexicographic ordering" really makes no sense. I think you meant to state "We have $R_1$ followed by $S_{1,1}$ followed by $S_{1,2}$ ... followed by $R_2$ followed by $S_{2,1}$ ...." with your entire question. If that's true, then I don't think there's a simple version - there is no common definition for $\sum$ or $\prod$ for an infinite number of ordinals. In fact, I'm not even convinced that the "intended" result would be a well-ordered set as this "infinite addition/product" might introduce the possibility for infinite descending chains. That'd be impossible to verify without a clearer, more rigorous definition than my interpretation of your intention, though.
Here's one possible interpretation that actually would work. Define $F: n->ON$ and $G_i: m_i->ON$ recursively as following:
$G_i(0)=F(0)=0$
$\alpha $ successor: $ G_i(\alpha)=G_i(\alpha-1)+S_{i,\alpha-1}$  and $F(\alpha)=F(\alpha-1)+G_{\alpha-1}(m_{\alpha-1}) $
$\alpha$ limit: $G_i(\alpha)=sup(G_i(\beta):\beta<\alpha)+S_{i,\alpha}$ and $F(\alpha) = sup(F(\beta):\beta<\alpha)+G_{\alpha}(m_{\alpha})$
Note the little technicality that if $n$ is limit ordinal, then we actually need an $G_n$, $m_n$ and $S_{n,m_n}$ that's not given in the original question for this to work. We can just assume them all as $0$, of course. Or we could revise the definition to include the special case.
Then $F(n)$ is the result. Now, as for how to evaluate that, that entirely depends on the sets $R_i$ and $S_{i,j}$, of course.
Answer: If I understand your question correctly, the set that contains the ordered pairs may not be a product. In the lex order on that set, each "column" will correspond to some $i$ for $i\in n$. And the content of each "column" will correspond to some $m_i$. Then the order type of $(R_i,S_{i,j})$ in the
$\le_{lex}$ order is $o.t.(R_i,S_{i,j}) = (\sum_{t<i} m_t) + j$.

Order of ordinals


After div is expanded, no other buttons work in that vertical space

Question: I have a tab button that handles the chatbox functionality. By default it is closed and when clicked, expands into the chat box. When it is closed all the buttons around it work as intended, however, when expanded the buttons above it can no longer be clicked like there is a invisible div over them.
Here I will provide the pictures of what I am describing and the corresponding code.
Closed(plus sign button working correctly)

Opened(plus sign button no longer working)

Code:
<!DOCTYPE html>

         Chat

Any suggestions on why this could be taking the functionality of the buttons in the same vertical space away? Thanks in advance.
Comment: Without even looking at the code I can pretty much guarantee that you are overlapping those buttons with a transparent element, and catching the clicks. Try right clicking one of those buttons and inspecting it, to see which element it takes you to in the console.
Answer: Your #olark-box-wrapper is a div with position:absolute; top: 0; height:100%; z-index:9999999999; ... no wonder it's taking up the full height of the right window area on top of everything else. And even though it's a transparent div, mouse clicks are handled by the event handlers of that div, not by those of the elements below it.
You could add pointer-events: none; to that div, making it "transparent" to user interaction. You may have to add pointer-events: visible; to its childs, to avoid the pointer-events: none; attribute being inherited by the children.
Alternatively, you could change the layout so that the #olark-box-wrapper is exactly the same height as its children.
One note about your choice of z-index: this number might be a bit too high, see Minimum and maximum value of z-index? (tl;dr: keep it in the range of a signed 32-bit number).
Comment: @Shultz That reads as if you didn't set `pointer-events: visible;` on the childs of `#olark-box-wrapper`; i.e. in your case, on `#side-chat`. Though it'd be easier to just select all childs: `#olark-box-wrapper > * { pointer-events: visible; }`
Comment: I have tried that but that makes the chat box transparent and if you try to click on the text box it clicked on the things behind it.