Converting this string to JSON object array using jQuery

Question: I have a web service that returns this string via the jQuery $.ajax() call in the success callback:
[{"WaitlistID":1,"RID":45034,"CustomerID":2765957,
 "IsAdmin":false,"TruckSize":1,"Points":1},
 {"WaitlistID":2,"RID":45034,"CustomerID":2765957,
 "IsAdmin":false,"TruckSize":1,"Points":1}]

Unfortunately if I call $.each() on that value in the success callback it iterates over every letter in it, and doesn't treat it as a two element array, which is what I'd like.  I've tried the makeArray() function but haven't had any luck, how can I convert that string into a JSON object array?
edit:
In response to the comments (thanks, everyone) I already do set the dataType to 'json', which is odd.  Here's the code in question.
jQuery.ajax({
    type: "POST",
    url: pagePath + "/" + fn,
    contentType: "application/json; charset=utf-8",
    data: paramList,
    dataType: "json",
    success: successFn,
    error: errorFn
});

..so I'm not sure why it didn't work originally, but the parseJSON() bit did the trick.  Appreciate everyone's help.
Comment: Your ajax call is not properly configured. Use `dataType:json`. It will parse the json for you and give you an object as the parameter to your `success()` function.
Comment: There's is no such thing as a "JSON object".  Your string already is JSON.  What you want is a JavaScript object.
Answer: You can use jQuery.parseJSON to parse it:
var obj = $.parseJSON(str);

However, jQuery should already do this for you if the server returns the correct content-type. If it is not, you can specify jQuery to treat the response as JSON:
$.get("test.php", function(data){
   // callback
}, "json");

Or even better, use jQuery.getJSON.
Answer: Try to parse with jQuery.parseJSON()!
Comment: Yes, just found that method and was returning here to answer my own question.  Thanks!
Comment: Better to use `dataType:json` in your `$.ajax()` call. You don't have to mess with `parseJSON()` that way.
Comment: @larryq Instead of `parseJSON`, you should add `dataType: 'json'` option to `$.ajax`
Answer: What you have is returned is a string.. Try to set the dataType in $.ajax call.
$.ajax({
   url: blahblah,
   dataType: 'json',
   ...
});

Comment: I do have the dataType set to json, which made my original problem all the stranger.  Thanks for the assist.
Answer: You have:
success: successFn

Does successFn() exist, and will it take a parameter? I.e., is it defined like function successFn(myObject)? If so, myObject will contain the object described by the JSON string. No parsing necessary.


Ensures() - guideline support library

Question: I am trying to understand how to use Ensures() in code. As given in the example, if I tried using Ensures() as follows...
int main(void)
{
    int result = 0;
    // Some calculation
    Ensures(result == 255);
    return 0;
}

If the result variable is not equal to 255, the program crashes with the following output "terminate called without an active exception". My question is how to use Ensures() properly?
Comment: If you `std::terminate` is called in your example, then it works as is should.
Answer: Are you using the Microsoft GSL implementation? Then if you check the gsl_assert.h header file you will see that if GSL_TERMINATE_ON_CONTRACT_VIOLATION is defined (which is default) then Ensures will call std::terminate which will give you the error you get.
If you want an exception to be thrown (with file and line-number information) then you need to define GSL_THROW_ON_CONTRACT_VIOLATION before including the GSL.
As for if you're using Ensures properly, then yes you are.

Updates on 2021
GSL_TERMINATE_ON_CONTRACT_VIOLATION is removed, always calling terminate().
Comment: As decided by the Core Guideline authors the MS GSL lately removed `GSL_THROW_ON_CONTRACT_VIOLATION` and now always `std::terminate`s. But... there is still a discussion ongoing because the Core Guidelines seem to be inconsistent about the behaviour of `Expects`.
Comment: Thanks, for the reply, will check the header.
Comment: I am not using MS GSL. I tried using MS GSL with Visual Studio 2013. But was getting "fatal error C1001: An internal error has occurred in the compiler".
Comment: As per your suggestion, I went into the "gsl-lite.h" and modified the # define gsl_CONFIG_THROWS_FOR_TESTING  0 to # define gsl_CONFIG_THROWS_FOR_TESTING  1. Now I am getting exception which i can catch it. Thanks.

Ensures() - guideline support library


Python numpy array fold matrix when accessed out of bound

Question: How do I acheive numpy array folding in Python?
e.g.:
A = [[ 0,  1,  2,  3],
     [ 4,  5,  6,  7],
     [ 8,  9, 10, 11],
     [12, 13, 14, 15]]

My desired output for this pseudo-code:
A[-1 to 1, -1 to 2] = [[15, 12, 13],
                       [ 3,  0,  1]]

I tried  A[i % 4 for i in range(-1, 1), j % 4 for j in range(-1, 2)] but it is not a valid numpy operation.
Answer: Please try this:
import numpy as np

A =np.array([[ 0,  1,  2,  3],
   [ 4,  5,  6,  7],
   [ 8,  9, 10, 11],
   [12, 13, 14, 15]])
b=[ [A[(3+i)%A.shape[0],(3+j)%A.shape[1]] for j in range(3)] for i in range(2)]
print(b)

The output is:
[[15, 12, 13], [3, 0, 1]]

Comment: why do we have 3 in A[(3+i).
It is redundant, right?
Comment: No, it is not redundant because i and j starts from 0, and we want to be index of the last row for i  and last column for j at the beginning.
Comment: In general, it is A [ (A.shape[0]-1+i)%A.shape[0] , (A.shape[1]-1+j)%A.shape[1] ].
Answer: You can easily achieve that by using np.ix_ and range():
import numpy as np

A =np.array([[ 0,  1,  2,  3],
             [ 4,  5,  6,  7],
             [ 8,  9, 10, 11],
             [12, 13, 14, 15]])

A[np.ix_(range(-1,1),range(-1,2))]

Output:
array([[15, 12, 13],
       [ 3,  0,  1]])

To avoid the possible error in a scenario like when for example the range of the sliced is (5,5) in a 4x4 matrix, you could try this:
import numpy as np

A =np.array([[ 0,  1,  2,  3],
             [ 4,  5,  6,  7],
             [ 8,  9, 10, 11],
             [12, 13, 14, 15]])
s1=[5,5]
s1=list(map(lambda x:(x+A.shape[0])%A.shape[0],s1)) if s1[0]>s1[1] else s1
s1=list(map(lambda x:(x+A.shape[0])%A.shape[0],range(min(s1),max(s1)))) if s1[0]%A.shape[0]!=s1[1]%A.shape[0] else [(s1[0]+A.shape[0])%A.shape[0]]

s2=[-1,2]
s2=list(map(lambda x:(x+A.shape[0])%A.shape[0],s2)) if s2[0]>s2[1] else s2
s2=list(map(lambda x:(x+A.shape[0])%A.shape[0],range(min(s2),max(s2)))) if s2[0]%A.shape[0]!=s2[1]%A.shape[0] else [(s2[0]+A.shape[0])%A.shape[0]]

A[np.ix_(s1,s2)]

Comment: if I access out of bounds say 5,5 on a 4x4 matrix, I am getting an error with this technique. I would like to fold to 1,1 in that case.
Comment: @KarthickS Ok, just to clarify 5,5 will be 1,1, which the element at index 1, yes?
Comment: Just added the solution for that possibility @KarthickS
Comment: @KarthickS it  seems like I had overestimated my answer. Please, see the last edit, and test it out.


Usar include ou constantes para pegar códigos?

Question: Criei um arquivo contendo padrões que são usados no meu sistema. Gostaria de saber qual e a diferença em desempenho entre chamar o código por um include ou por uma constante assim:
//código do incluir.php:

include "incluir.php";

define('_CONSTANT','');

Tem alguma vantagem em usar um e outro?
Eu pergunto isso pois fico em dúvida se eu utilizar um ou outro pode afetar o desempenho do meu sistema, já que eu uso bastante isso para pegar códigos padrões
Answer: A pergunta é bem objetiva sobre qual dos dois é mais performático. Portanto, não comentarei sobre conceitos ou estrutura de dados (mvc, oop, etc) e nem mesmo boas práticas de programação.
Respondendo de forma objetiva, dentro do seu contexto, é mais rápido definindo por constantes. Simples assim.
O motivo é óbvio. O processo de incluir um arquivo consome memória e tempo de execução para carregar um mesmo valor que estaria numa constante. Então, se pode usar as constantes, use-as.


Sweeping actions in curling

Question: It is possible to estimate the effectiveness of sweeping action in curling?
I mean how can the sweepers carry forward the stone it they do sweeping action all over the rink?
Answer: Yes it is possible to estimate the effectiveness of sweeping on a curling rock.   There have been numerous studies to determine the answer to this question.   I have personally participated in several of these studies.
These studies often involve a rock throwing machine which is able to throw the rock on a consistent trajectory, with a consistent speed and rotation (Sometimes it can just be a human throwing them as well but those results are not as robust as there is more variance in a human thrower than a robot.)  After the rock is released, you have a group of sweepers that either sweep the rock or leave it alone.  There can be variations on this where sweepers try different techniques or try different broom heads.
In short, 3 - 9ft is the distance that can be added to a rock when swept. However whether you get 3ft or 9ft depends on many factors such as: ice temperature, air temperature, humidity, frost, type of pebble, type of broom, strength of sweeper, running surface of the rock.
In general, the keener the ice, the more distance a rock can be dragged and earlier in the game is when sweeping is most effective.
References:
CCA blog post on sweeping study,
 Study on rock distance achieved from heating ice from sweeping
Comment: Can you please link one of the mentioned studies?
Comment: Unfortunately no.  The results of the study were kept secret because they were funded by the Canadian Curling Association. The findings were only given to the elite Canadian curlers in hopes of giving them an advantage over other countries.   I added a link which refers to a similar study from 2009 but only gives a little bit of detail. The ones I participated in were from 2012 and 2013 but they were essentially the same as the one mentioned in the first link.
Answer: The entire sport of curling is an example of Newton's first law of motion - "An object a rest tends to remain at rest, and an object in motion tends to continue in moving in a straight line at constant speed unless an outside force acts upon it."
The sweeping action is to smooth out the ice, and prevent the friction from the bumps in the ice slowing down the stone. Hence sweeping at one side will only smooth out the ice on that side, causing the stone to curve.
A proper discussion of these things are available here, and here.
I'm not sure you could exactly quantify the effectiveness of the sweeping action - it would depend on so many variables, such as initial bumpiness of the ice, strength of the sweeper, abrasive quality of the broomhead, force exerted on the broom etc.

Sweeping actions in curling


Relation between connection on vector bundle and Ehresmann connection on frame bundle

Question: I'll be using the conventions, notation and (numbered) results from Tu's Differential Geometry book.
Let $M$ be a smooth manifold, $E\to M$ a (real) vector bundle of rank $r$ with a connection $\nabla:\Gamma(E)\to\Omega^1(M)\otimes\Gamma(E)$. There is a way to obtain a connection on $E$ via the
Consider the frame bundle $\pi:\text{Fr}(E)\to M$ of $E$, which has an induced Ehresmann-connection $\omega\in\Omega^1(\text{Fr}(E),\text{Fr}(E)\times\mathfrak{gl}_r\mathbb R)$, and whose corresponding associated vector bundle $\text{Fr}(E)\times_1\mathbb R^k$ with respect to the identity representation $1:\text{GL}(r,\mathbb R)\to\text{GL}(r,\mathbb R)$ is canonically isomorphic to $E$.
Let's denote by $\Omega_1^k(\text{Fr}(E),\text{Fr}(E)\times\mathbb R^r)$ the $\mathbb R^r$-valued $k$-forms on the frame bundle which are right-invariant and horizontal, meaning that $r_g^*\omega=\omega$ and that $\omega$ vanishes as soon as an input vector is vertical. There is a covariant derivative
$$
D:\Omega_1^k(\text{Fr}(E),\text{Fr}(E)\times\mathbb R^r)\to \Omega_1^{k+1}(\text{Fr}(E),\text{Fr}(E)\times\mathbb R^r),\omega\mapsto (d\omega)^h,
$$
where $(d\omega)^h(v_1,\dots,v_{k+1})=dw(hv_1,\dots,hv_{k+1})$, where by $hv$ we mean the horizontal component of $v$. By Theorem 31.9 we have an identification
$$
\Omega_1^k(\text{Fr}(E),\text{Fr}(E)\times\mathfrak{gl}_r\mathbb R))\cong\Omega^k(M,\text{Fr}(E)\times_1\mathbb R^r),
$$
which means that the covariant derivative $D$ can be considered as a map
$$
\Omega^k(M)\otimes\Gamma(E)\to\Omega^{k+1}(M)\otimes\Gamma(E).
$$
My question is as follows:
I want to show that for $k=0$ this map corresponds to $\nabla$, but I'm not sure how to show this rigourously; due to the identifications made it is difficult for me to imagine how $D$ looks like as a map from $\Gamma(E)$ to $\Omega^1(M)\otimes\Gamma(E)$.
By Theorem 31.19 I know that for $\phi\in\Omega^0_1(\text{Fr}(E),\text{Fr}(E)\times\mathbb R^r)\cong\Gamma(\text{Fr}(E)\times\mathbb R^r)$ we have
$$
D\phi=d\phi+\omega\cdot\phi,
$$
where the action $\omega\cdot\phi$ is the natural action of $\omega\in\Omega^1(\text{Fr}(E))\otimes\Gamma(\text{Fr}(E)\times\mathfrak{gl}_r\mathbb R))$ on $\phi$. Note that the formula above looks a lot like the formula
$$
\nabla(f^i\sigma_i)=df^i\otimes\sigma_i+f^i\nabla\sigma_i=df^i\otimes\sigma+f^i\omega^j_i\otimes\sigma_j,
$$
where $\omega_\sigma=(\omega^i_j)$ is the connection matrix corresponding to the local frame $(\sigma_i)\subset\Gamma(E\vert_U)$. If we think of the local frame $(\sigma_i)$ as a section $\sigma\in\Gamma(\text{Fr}E\vert_U)$, then by Theorem 29.10 we have $\omega_\sigma=\sigma^*\omega$.
So I feel like I've got most of the ingredients, but I can't make it into a complete and formal argument.
Could someone help me out?
Comment: @DanielTeixeira Yes, I've worked it out on paper, I will post an answer later today with all the details for future reference
Comment: I believe what you've written is a local formula, since there is no global connection one from on the base manifold of $E$. A way to prove your local result is actually a global defined map is that you get the same answers independent of the section you pull the connection form back by.
Comment: Echoing Chris, I think you just have to make sense for yourself that your last equation is the penultimate in local coordinates, i.e. go back and read the local formulas for $d\phi$ and $\omega\cdot\phi$.
Answer: Disclaimer: I've written everything out in painstaking detail, including explicit canonical isomorphisms, which definitely obscures the proof - but the point for me is to make it clear and explicit once and for all to avoid confusion later on.
The entire argument can be summarised by the following commutative diagram:

which arrises by looking locally $(U\subset M)$. The point to note is that the map
$$
\Gamma(U\times\mathbb R^r)\to\Omega^0_\rho(U\times\text{GL}_r\mathbb R,\mathbb R^r)
$$
in the diagram is the pull-back map $\pi^*$, and that the map
$$
\Omega^1_\rho(U\times\text{GL}_r\mathbb R,\mathbb R^r)\to \Omega^1(U)\otimes\Gamma(U\times\mathbb R^r)
$$
in the diagram is the pull-back $\sigma^*$ (where $\sigma\in\Gamma(\text{Fr}E\vert_U)$ is a local section we arrive at by choosing a local trivialisation).
Remark: Initially I wrote the text below without the diagram, and in the end I realised which diagram belongs to my proof. Since the diagram is very clear, I've put it first, but now comes the original text-based proof:
Choose a local trivialisation $\phi:E\vert_U\stackrel{\sim}{\to} U\times\mathbb R^r$, and write $\sigma_i=\Gamma(\phi)^{-1}(e_i)$ (where $\Gamma$ is the section functor). We have an induced trivialisation
$$
\Phi:\text{Fr}E\vert_U\stackrel{\sim}{\to} U\times\text{GL}_r\mathbb R,
$$
and we let $\sigma=\Gamma(\Phi)^{-1}(I)$, which means fiber-wise that $\sigma_p:e_i\mapsto\sigma_i\vert_p$. Our trivialisation induces an isomorphism between vector bundles:
$$
\text{Fr}E\vert_U\times_1\mathbb R^r\simeq(U\times\text{GL}_r\mathbb R)\times_1\mathbb R^r\cong U\times\mathbb R^r.
$$
Under these identifications, the inverse of the isomorphism in Theorem 31.9 is locally given given by the pull-back of the projection map $\pi^*$:
\begin{multline*}
\pi^*:\Omega^k(U)\otimes\Gamma(U\times\mathbb R^k)\stackrel{1\otimes\Gamma\phi^{-1}}{\longrightarrow}\Omega^k(U)\otimes\Gamma(\text{Fr}E\vert_U\times_1\mathbb R^k)\\
\stackrel{\pi^*}{\to}\Omega^k(\text{Fr}E\vert_U)\otimes\Gamma(\pi^*(\text{Fr}E\vert_U\times_1\mathbb R^k))\to
\Omega^k(\text{Fr}E\vert_U)\otimes\Gamma(\text{Fr}E\vert_U\times\mathbb R^k).
\end{multline*}
We can verify explicitly that the map above on the sections sends $e_i$ indeed to its pull-back $\pi^*e_i=e_i\in\Gamma(U\times\mathbb R^r)$:
\begin{gather*}
\Gamma(U\times\mathbb R^r)\simeq\Gamma(\text{Fr}E\vert_U\times_1\mathbb R^r)\to\Gamma(\pi^*(\text{Fr}E\vert_U\times_1\mathbb R^r))\cong\Gamma(\text{Fr}E\vert_U\times\mathbb R^r)\\
(p\mapsto (p,e_i))\mapsto (p\mapsto [\sigma_p,e_i])\mapsto (\tau_p\mapsto (\tau_p,[\sigma_p,e_i]))\mapsto(\tau_p\mapsto (\tau_p,e_i)).
\end{gather*}
Write
$$
\omega=\sum\tilde\omega^i_j\otimes e^i_j\in\Omega^1(\text{Fr}E\vert_U)\otimes\Gamma(\text{Fr}E\vert_U\times\mathfrak{gl}_r\mathbb R),
$$
for the restricted connection form $\text{Fr}E\vert_U$, and write
$$
\omega_\sigma=\sum\omega^i_j\otimes e^i_j\in\Omega^1(U)\otimes\Gamma(U\times\mathfrak{gl}_r\mathbb R)
$$
for the local connection form of relative to the frame $\sigma$, meaning that $\nabla\sigma_i=\omega^j_i\sigma_j$. Since $\sigma^*\omega=\omega_\sigma$ we have $\sigma^*\tilde\omega^i_j=\omega^i_j$. Note that
$$
\omega\cdot e_i=\tilde\omega^j_i e_j,
$$
while $d(e_i)=0$, so
$$
D(e_i)=d(e_i)+\omega\cdot e_i=\tilde\omega^j_i e_j
$$
To go back to $\Omega^1(U)\otimes\Gamma(U\times\mathbb R^r)$, we simply need to pull back along $\sigma$ to obtain $\sigma^*\omega^j_i e_j=\omega^j_i e_j$;  because $\sigma$ corresponds to the trivial section of $\text{Fr}E\vert_U\simeq(U\times\text{GL}_r\mathbb R))$ and by the naturality of $\omega\mapsto\omega^\flat$ we may apply the lemma below.
Lemma.
Let $P=M\times G\to M$ be a trivial principal $G$ bundle, $\rho:G\to\text{GL}(V)$ a representation. Denote by $\phi:P\times_\rho V\cong M\times V,[(x,g),v]\mapsto (x,v)$ the identification with the trivial bundle. Let $\sigma:M\to P,m\mapsto (m,e)$ be the canonical section of the trivial bundle. Then the isomorphism $\omega\mapsto\omega^\flat$ in Theorem 31.9 under our identification with the trivial bundle is given by the pull-back under $\sigma$:
$$
\sigma^*:\Omega^k_\rho(P,P\times V)\stackrel{(-)^\flat}{\to} \Omega^k(M)\otimes\Gamma(P\times_\rho V)\stackrel{\phi}{\to}\Omega^k(M)\otimes\Gamma(M\times V).
$$
Proof.
Let $\omega\in\Omega^k_\rho(P,P\times V)$. Write
$$
\omega_p(v_1,\dots,v_k)=(p,\overline\omega_p(v_1,\dots,v_k))\in\{p\}\times V.
$$
Then
\begin{align*}
\phi\omega^\flat_x(v_1,\dots,v_k)&=\phi[\omega_{\sigma(x)}(T_x\sigma v_1,\dots,T_x\sigma v_k)]=\phi[\omega_{(x,e)}(T_x\sigma v_1,\dots, T_x\sigma v_k)]\\
&=\phi[(x,e),\overline\omega_{(x,e)}(T_x\sigma v_1,\dots, T_x\sigma v_k)]=(x,\sigma^*\overline\omega_x(v_1,\dots,v_k)).
\end{align*}
$\tag{$\square$}$
So we see that our induced map sends $\sigma_i$ to $\omega^j_i\sigma_j=\nabla\sigma_i$. Since we also have the Leibniz condition, it follows that our induced map corresponds to the original connection $\nabla$.
Comment: Great answer. These are necessary details involved with these computations, but they are clearly unsuitable for a textbook. In the past I imagine you would either have to figure it out yourself or ask someone you know, but we're happy to live in the same age as MSE.


Is there a way to describe these compactifications algebraically?

Question: There is a well known correspondence between locally compact Hausdorff spaces and commutative C* algebras, in which the homoemorphism class of spaces is mapped one-to-one to the isomorphism class of commutative C* algebras via $X\mapsto C_0(X)$.
Compact spaces correspond to unital algebras, and compactification procedures like one-point compactification or Stone-Cech compactification to unitisation procedures like the adjoining of unit $A\mapsto A\oplus\Bbb C1$ and passing to the multiplier algebra.
Suppose one has a bounded and open subset $U$ of some $\Bbb R^n$. The closure $\overline{U}$ will then be a compactification of $U$, and for that reason $C(\overline U)$ will be a unitisation.
Is there a way to describe how this unitisation works on the algebra level? Specifically how one can distinguish it from unitisations that do not allow an embedding into $\Bbb R^n$?
As an example with $U=(0,1)$ the closure $[0,1]$ is a two-point compactification and we adjoin two elements to the algebra, which we can view for example as the positive functions $x$ and $1-x$, adding them together gives unit.
Answer: Here's the closest thing to what you're asking for that I can think of.  If $X$ is a compact Hausdorff space, then $X$ embeds in $\mathbb{R}^n$ for some $n$ iff $C(X)$ is finitely generated as a $C^*$-algebra.  Indeed, if $X$ is a subspace of $\mathbb{R}^n$, then the coordinate projections generate $C(X)$ (as a unital $C^*$-algebra) by Stone-Weierstrass.  Conversely, if $C(X)$ is generated by finitely many functions $f_1,\dots,f_n:X\to\mathbb{C}$, they together give a map $f:X\to \mathbb{C}^n$.  Since the $f_i$ must jointly separate points, $f$ is injective, and so since $X$ is compact it is an embedding.
So, if $X$ is only locally compact Hausdorff, compactifications of $X$ which come from embeddings of $X$ as a bounded subset of $\mathbb{R}^n$ correspond to unitizations of $C_0(X)$ which are finitely generated as $C^*$-algebras.
Comment: The aspect "the unitisation is finitely generated" means that the space is embedded into some $\Bbb R^n$  upon which the compactification is taken. For example $(0,1)$ can be embedded into $\Bbb R^2$ with closure the circle, which will also result in a unitisation. This is different from the condition that the embedding must be open, but that is probably a bit more difficult.


ERRO syntax error, unexpected T_VARIABLE

Question: Estou com um problema e sou iniciante em PHP, não sei se é fácil de corrigir este erro.

(Parse error: syntax error, unexpected T_VARIABLE in
  /home/u611580299/public_html/wp-content/themes/simplicityantigo/includes/google-fonts.php
  on line 1)

<?php
global $fontArrays;

$fontArrays = array(
    0 =>
    array(
        'kind' => 'webfonts#webfont',
        'family' => 'ABeeZee',
        'variants' =>
        array(
            0 => 'regular',
            1 => 'italic',
        ),
        'subsets' =>
        array(
            0 => 'latin',
        ),
    ),
    // ...
);
?>

Arquivo completo
Comment: Para quem votou para fechar: Como não é reproduzível ?
Comment: Não se declara variáveis globais fora do escopo das funções, simples.
Comment: @ZebraDoMal ponto e virgula dentro de array? Viajou hein
Comment: Pelo tamanho desse Array não aconselho, mas se quiser utilizar uma GLOBAL fora do escopo de uma função você deve usar a superglobal `$GLOBALS[]`.
Comment: Além disso, sinceramente, com um tanto de array dentro de array como tens aí, a chance de ter esquecido um ponto-vírgula..... é enorme. rsrs
Comment: É uma ótima pergunta, muita gente vai ter esse mesmo problema.
Comment: @DanielRibeiro mas da forma que está a variável já está no escopo global
Comment: @gmsantos Sim, realmente, se ele apenas desprezar a primeira declaração, que deve ser usada dentro de funções, a variável esta global no arquivo, mas se ele quiser tornar-lá global pra todo o projeto, ele pode usar o que citei em cima, embora não seja aconselhável por questão de performance, pois o array eh gigante. Se eu estiver falando basteira, por favor, me corrija.
Comment: Mas na verdade acho que o problema dele mesmo era apenas erro de sintaxe, eu que interpretei errado achando que ele queria carregar a variavel para fora do arquivo. Erro meu.
Comment: Não exatamente @DanielRibeiro. Supondo que temos o arquivo `i.php` setando algo em `$_GLOBAL`. Caso não tenha em nenhum momento um `include` de `i.php` em um arquivo `m.php`, a variável que defini em `i.php` também não ficará disponível. Acessar usando `global $a` ou `$_GLOBALS['a']` não muda em nada na questão performance.
Comment: Entendi @gmsantos. Valeu por esclarecer. Basicamente, a variável não precisar ser declarada, explicitamente, como $GLOBALS para pertencer ao escopo, apenas o fato de ela existir no arquivo, fora de funções, já a inclui na superglobal. O `globals` serve para explicitar que ele esta utilizando uma variável existente fora do escopo da função. Blz, agora esta tudo entendido xD. Certo?
Comment: Isso ai @DanielRibeiro
Comment: @gmsantos Eu tava falando do ponto-vírgula depois da array. array (); Mas pelo que olhei agora aqui na verdade tem um array só (achei que tinham vários quando olhei rápido). huahua
Answer: O problema aqui está no uso da palavra chave global.
global serve para referenciar dentro do escopo de uma função uma variável no escopo global:
<?php

$a = 5;
$b = 3;

function soma(){
    $a = 1;
    $b = 2;

    return $a + $b;
}

function somaGlobal(){
    // A partir desse ponto, $a = 5 e $b = 3
    global $a, $b;

    return $a + $b;
}

echo soma();        // escreve 3
echo somaGlobal();  // escreve 8


HNF of $\begin{bmatrix}c & 0 \\ 0 & c \\ a & b \end{bmatrix}$

Question: Is there any way to find $x, y, z \in \mathbb{Z}$ such that, given
$a, b, c \in \mathbb{Z},\ a, b, c > 0,\ (a, b, c) = 1$ matrix
$$H = \begin{bmatrix} x & y \\ 0 & z \\ 0 & 0 \end{bmatrix}$$
is the Hermite normal form of
$$A = \begin{bmatrix}c & 0 \\ 0 & c \\ a & b \end{bmatrix} ? $$
Note that I'm not looking for an algorithm, but rather for a generic answer, e.g. that heavily uses Bézout's lemma. It will be good if there is some beautiful matrix $U$ such that $U \times A = H$.
Side questions:

Is it always true that $z = c$?
Smith normal form $D = U \times A \times V$ will be good too.
Comment: With your conditions the Smith Normal Form will have two nonzero entries $1,c$. But surely for the Hermite form we might end up with eg $x=c,y=0,z=1$? [I tried $c=2,a=2,b=3$.]
Comment: @ancientmathematician what I really meant is D will have 1, c on diagonal. Is it true? If so, is there a simple way to prove it? I'm not into this whole theory too much for now.
Answer: Pre-multiplication of an $m\times n$ integer matrix $A$ by a unimodular integer matrix $U$ does not change the ideal generates by the $r\times r$ minors of $A$, that is leaves their highest common factor unchanged.
In your example
(i) we have that the HCF of the $1\times 1$ minors of $A$ is $\gcd(a,b,c)=1$, and the highest common factor of the $2\times 2$ minors is $\gcd(c^2,ac,bc)=c$.
(ii) we have that the HCF of the $1\times 1$ minors of $H$ is $\gcd(x,y,z)$, and the highest common factor of the $2\times 2$ minors is $\gcd(xz)=xz$.
I don't think more can be said.
With $c=6,a=2,b=3$ in $A$   we get $x=2,y=0,z=3$ in $H$: so we don't get $1,c$ on the "diagonal".


Partial latin square with $\le n-1$ filled cells

Question: How do we show that is $P$ if a $n\times n$ Latin square with $\le n-1$ filled cells, then $P$ can be completed to a proper Latin square?
Here is the definition of a Latin square.
(WHAT I HAVE DONE SO FAR: I have worked examples for $n=2,3,4$ but am unable to find a pattern that is rigorous, or any pattern for that pattern. A step-by step proof would be appreciated- or if you could tell me what facets here are of paramount importance when it comes to my observation.)
Could someone explain the inductive step in the link provided by Gerry Myerson in Theorem 5? I don't get it.
Answer: You have a partial Latin square $P$ of order $m+1$ with exactly $m$ filled cells, and you're assuming that the number of distinct symbols in $P$ is greater than $\frac{m+1}2$. If each of these symbols occurred in more than one cell, they would fill more than $m+1$ cells altogether, contradicting the assumption that $P$ has just $m$ filled cells. Thus, some symbol $s$ appears in exactly one cell of $P$. Here's an example with $m=4$; I've marked the back diagonal in red.
$$\begin{array}{|c|c|c|c|c|}
\hline
&&&5&\color{red}\cdot\\ \hline
&&&\color{red}\cdot&\\ \hline
&&\color{red}\cdot&3&\\ \hline
&\color{red}\cdot&&&\\ \hline
\color{red}5&2&&&\\ \hline
\end{array}$$
At this point we use the second of the 'key ideas' identified at the top of p. 254 to conclude that the rows and columns of $P$ can be permuted to yield a new partial Latin square $Q$ of order $m+1$ with exactly $m$ filled cells in such a way that $s$ now lies on the back diagonal of $Q$, and the other filled cells of $Q$ lie above the back diagonal. Then we relabel the symbols so that $s$ becomes $m+1$. In my example I change the order of the rows from $12345$ to $13524$, bringing the lone $3$ to the back diagonal, relabel the $3$ as $5$, the $5$'s as $2$'s, and the $2$ as $3$ to get this:
$$\begin{array}{|c|c|c|c|c|}
\hline
&&&2&\color{red}\cdot\\ \hline
&&&\color{red}5&\\ \hline
2&3&\color{red}\cdot&&\\ \hline
&\color{red}\cdot&&&\\ \hline
\color{red}\cdot&&&&\\ \hline
\end{array}$$
The next step is to delete the $m+1$ on the back diagonal and the last row and last column. Note that the only cells in the last row and column that could possibly be filled at this point are the ones in the upper right and lower left corners, on the back diagonal, so the only symbol deleted in this step is the $m+1$ on the back diagonal. In the example above we'd be left with the following partial Latin square of order $4$:
$$\begin{array}{|c|c|c|c|c|}
\hline
&&&\color{blue}2\\ \hline
&&&\color{red}\cdot\\ \hline
\color{blue}2&\color{blue}3&\color{red}\cdot&\\ \hline
&\color{red}\cdot&&\\ \hline
\end{array}$$
The red dots mark the cells that were on the back diagonal of $Q$; the one just below the $2$ in the upper right corner is where the $\color{red}5$ was in $Q$. The blue cells are the entries retained from $Q$.
We now have a partial Latin square of order $m$ with $m-1$ filled cells; by the induction hypothesis, it can be completed to a Latin square $L$. In my example $L$ might be this Latin square:
$$\begin{array}{|c|c|c|c|c|}
\hline
1&4&3&\color{blue}2\\ \hline
3&2&4&1\\ \hline
\color{blue}2&\color{blue}3&1&4\\ \hline
4&1&2&3\\ \hline
\end{array}$$
Now construct the partial Latin square $P(L)$ of order $m+1$ as described at the bottom of p. 253 and top of p. 254: erase every symbol below the back diagonal of $L$ and add an empty last column and last row to get a partial Latin square of order $m+1$, and fill the back diagonal of this partial Latin square with the symbol $m+1$. In my example this produces the following partial Latin square $P(L)$, where the blue entries are those retained (or, in the case of the $5$, reintroduced) from $Q$:
$$\begin{array}{|c|c|c|c|c|}
\hline
1&4&3&\color{blue}2&5\\ \hline
3&2&4&\color{blue}5\\ \hline
\color{blue}2&\color{blue}3&5&\\ \hline
4&5&&\\ \hline
5\\
\hline
\end{array}$$
Next we apply the other of the two 'key ideas' mentioned at the top of the page to conclude that $P(L)$ can be completed.
One possible completion is this:
$$\begin{array}{|c|c|c|c|c|}
\hline
1&4&3&\color{blue}2&5\\ \hline
3&2&4&\color{blue}5&1\\ \hline
\color{blue}2&\color{blue}3&5&1&4\\ \hline
4&5&1&3&2\\ \hline
5&1&2&4&3\\
\hline
\end{array}$$
The blue cells, as before, show the cells retained from $Q$.
The final step is to reverse the relabelling and rearrangement of the rows and columns that took us from the original $P$ to $Q$. I changed $3$ to $5$, $5$ to $2$, and $2$ to $3$, so I must now change $3$ to $2$, $2$ to $5$, and $5$ to $3$:
$$\begin{array}{|c|c|c|c|c|}
\hline
1&4&2&\color{blue}5&3\\ \hline
2&5&4&\color{blue}3&1\\ \hline
\color{blue}5&\color{blue}2&3&1&4\\ \hline
4&3&1&2&5\\ \hline
3&1&5&4&2\\
\hline
\end{array}$$
I also have to reverse the permutation of the rows; this, finally, yields this Latin square:
$$\begin{array}{|c|c|c|c|c|}
\hline
1&4&2&\color{blue}5&3\\ \hline
4&3&1&2&5\\ \hline
2&5&4&\color{blue}3&1\\ \hline
3&1&5&4&2\\ \hline
\color{blue}5&\color{blue}2&3&1&4\\
\hline
\end{array}$$
This is clearly a completion of the original partial Latin square $P$.
Does this help, or were you more worried about the proofs of the two 'key ideas', for which the authors refer you to a paper by Lindner?
Comment: THis certainly helps! Thank you. I was, however, also concerned about the Lindner references. Also, when you say: Next we apply the other of the two 'key ideas' mentioned at the top of the page to conclude that P(L) can be completed: which key ideas do you mean?
Comment: @Danielle: Right below the diagrams of $P$ and $L(P)$ at the top of page 254 is the sentence 'The proof of Smetaniuk's result relies on two key ideas'. The author's then give these ideas as a pair of bullet points. I'll think about how these two statements might be proved; I suspect that it's not horrendously difficult but may be a bit messy to describe.
Comment: Brian- that would be fantastic. THank you so much. A lot of the things in Latin squares construction wise go by axiomatically almost, and a lot of things are provable but many are messy, so thanks.
Answer: This is discussed in Chapter 32 of Proofs From The Book. The authors write, "Bohdan Smetaniuk's proof from 1981 ... is a beautiful example of just how subtle an induction proof may be...."
This paper also gives Smetaniuk's proof.
Comment: Where in the link can I find the proof?
Comment: The result is Theorem 5 on page 254 (but the proof refers back to earlier parts of the paper).
Comment: Could you explain the inductive step, please? I don't understand it. A visual aid might be nice...


The point of discontinuity of a bijective map

Question: Prove/Disprove:The set of point of discontinuity of a bijective map from $(0,1)$ to $[0,1]$ is always countably infinite.
It's a standard exercise to show that any bijective function from $(0,1)$ to $[0,1]$ can't have finitely many discontinuity.Next,in order to find a explicit map I found that the map can be constructed so that it has countably infinite discontinuity.So, I wonder whether such map always have countably many discontinuity.
Comment: The claim as stated is false - there exist bijections $(0,1)\to[0,1]$ with uncountably infinitely many points of discontinuity. -- If the claim is merely ".. at least countably infinite", you are done once you show that it can't be done with finitely many discontinuities, and no explicit map is required (the statement would remain true even if no such bijection existed).
Comment: @who it's hard to explain non-continuous maps of uncountable sets
Comment: There are a lot of pernicious examples, there's no reason for this to be true
Comment: Maybe you want to ask whether the set of discontinuities is always infinite?
Comment: @Hagen von Eitzen,can you give one such example(a map with uncountably many discontinuity)?
Comment: @Don Thousand,then how can we prove that, such map exists with uncountably many discontinuity?
Answer: The claim, as stated, is false. Consider the map $f:(0,1)\to[0,1]$ as below$$f(x)=\begin{cases}0&x=\frac12\\\frac1{2n}&x=\frac1{2n+2},\;n\in\mathbb N\\\frac1{2n-1}&x=\frac1{2n+1},\;n\in\mathbb N\\1-x&x\notin\mathbb Q\\x&\text{otherwise}\end{cases}$$
Note that even if I weren't able to find such a map, the impetus lies on you to show such a map doesn't exist, not the other way around.
Proof that the function is injective: Consider $x,y\in(0,1)$. Note that rationals are only mapped to rationals, and irrationals only to irrationals, so we can assume that both are either rational or irrational.
If $x,y$ are irrational, and $x\neq y$, then clearly $1-x\neq1-y$. If $x,y\in\mathbb Q$, and neither is of the form $\frac1n$, then $f(x)=x$ is clearly a bijection.
If $x=\frac1n$, note that $x$ is mapped to either $1$, or another element of the form $\frac1m$. Hence, if $y$ is not of form $\frac1m$, then $f(x)\neq f(y)$.
Finally, if both $x,y$ are of the form $\frac1n$, then clearly $f(x)=f(y)\to x=y$, so the function is injective.
Proof that the function is surjective: Clearly, given any irrational $r$ in $[0,1]$, $1-r$ is irrational, and also in $[0,1]$. Hence, $f$ maps to all irrationals in $[0,1]$. Also, $f$ also maps to $0$ and all rationals of the form $\frac1n$. And since all rationals not of the form $\frac1n$ are mapped to themselves, all rationals not of this form are mapped to. So the function is surjective.
Proof that the function is nowhere continuous:
Let $x\notin\mathbb Q$. Clearly, $x\neq\frac12$. Consider a sequence of rationals $\{q_i\}_{i\in\mathbb N}$ that converge to $x$ such that none of the rationals are of the form $\frac1n$. Clearly, $\{f(q_i)\}_{i\in\mathbb N}$ converges to $x$. However, $f(x)=1-x\neq x$, which implies that the function is not continuous at irrational $x$.
Let $x\in\mathbb Q$, and let $x\neq\frac12,\frac13,\frac14$. Consider a sequence of irrationals $\{r_i\}_{i\in\mathbb N}$ converging to $x$. Note that $\{f(r_i)\}_{i\in\mathbb N}$ converges to $1-x$. But, $f(x)=x\neq1-x$, or both $x,f(x)<\frac12$, which means that $f(x)\neq1-x$. So, the function is not continuous at all rationals except $\frac12,\frac13\frac14$.
Consider $x=\frac12,\;\frac13,\text{ or }\frac14$, and consider a sequence of irrationals $\{r_i\}_{i\in\mathbb N}$ converging to $x$. Note that $\{f(r_i)\}_{i\in\mathbb N}$ converges to $\frac12,\frac23,\frac34$ respectively, which $f(x)=0,1,\frac12$ respectively. This implies that the function is not continuous at these three values.
Comment: Hmm.The map is sufficient,there were no need of explanation.Yet, the complete explanation makes it a good answer.I guessed it, but didn't try, I should.


How can I solve the error to open google chrome on the server?

Question: I have run the command google-chrome on the server and I have got this error.
https://i.stack.imgur.com/vEjAr.png
When I tried this solution.
https://i.stack.imgur.com/spaVI.png
and I have got this error https://i.stack.imgur.com/3VxcP.png
Answer: You must not run any application as root unless it requires root privileges. Chrome does not require root privileges.
Chrome has protections that prevent it running with root privileges, and the errors you are seeing are caused by those.
In order to run Chrome properly, you need to create a normal user account and log in using that user account. Remember to set up a strong passphrase for the new user.


Retina display checking on GTM

Question: I've tried to look for a way to analyze retina display vs normal display conversion rate but I can't find the correct variables to check if a device uses retina display or not. I've searched in many forums and the posts are either too old (since 2010) or outdated due to changes in GTM.
Does anyone know if it's possible to track device resolution in GTM?
Please if you are to answer referring to a link, please make sure you read the whole description of the solution since I have gone through many ones and they all prove to be wrong or not possible to implement due to the reasons I mentioned before.
I found this answer but unfortunately the last comment says it stopped working.
Answer: Google Analytics tracks screen resolution - you can access it in custom reports, as a secondary dimension, or by navigating to:
Audience>Technology>Browser & OS>Screen Resolution (Screen Resolution is a selectable Primary Dimension just below the chart).
This chart gives you Goal Conversion Rates, but if you wanted you could also create a Segment with resolutions that you like, and another with the rest, and compare the two across any other report.
However, 'Retina Display' is a marketing term used by Apple (it's actually a registered trademark of theirs). Although the term does make sense, higher pixel density than the human retina is capable of observing, you'll find that their use of it varies, as you are expected to hold different devices at different distances from your face. Therefore a 'retina' iPhone is expected to have a higher pixel density than an 'retina' iPad.
Answer: GTM itself does not measure anything, but you can use any value that can be obtained via javascript.
One suggestion I found to detect retina displays is to use devicepixelratio, where apparently a ratio equal to or higher than 2 suggests a retina display. devicepixelratio does not work on IE (works with Edge, though), but here's a link to a suggested polyfill.
So if you create a custom JS variable in GTM you can use this (I admit I havent tested it myself, but it seems to work for other people).
Answer: You can also change the function to work with matchMedia and resolutions with dpi instead of devicePixelRatio to detect if it's retina or not:
Create Custom JavaScript variable, pixel_ratio:
function () {
    //Configure resolutions here, will return value of ratio:
    var pixel_ratio = [
    {'pixel_ratio': 'all', 'ratio': 1},
    {'pixel_ratio': '(min-resolution: 144dpi)', 'ratio': 1.5},
    {'pixel_ratio': '(min-resolution: 192dpi)', 'ratio': 2}
    ];
    var result;
    pixel_ratio.forEach(function (element) {
        var ratio = 1;
        //If browser support matchMedia, use it
        if (window.matchMedia) {
        // return current media query
            if (window.matchMedia(element.pixel_ratio).matches) {
                result = element.ratio;
            }
            } else { //matchMedia not supported
                if (window.screen.systemXDPI !== undefined
                    && window.screen.logicalXDPI !== undefined
                    && window.screen.systemXDPI > window.screen.logicalXDPI) {
            // Only allow for values > 1
            ratio = window.screen.systemXDPI / window.screen.logicalXDPI;
            } else if (window.devicePixelRatio !== undefined) {
                ratio = window.devicePixelRatio;
            }
            if (element.ratio === (Math.round(ratio * 100) / 100)) {
                result = element.ratio;
            }
        }
    });
    return result + 'x';
}

Create custom dimensions in GA for pixel_ratio. Select scope as "Hit". Note the index value for each dimension.
In GTM, configure the custom dimensions in your base GA pageview tag, matching the dimension index value from GA.
Comment: After trying myself the function using matchMEdia and resolutions with dpi, I've detected that is not pulling correct data to GM. It's better to just use devicePixelRatio in order to get accurate data:

function () {
   return devicePixelRatio + 'x';
}

Retina display checking on GTM


Django form not updating files/images

Question: I've build a form that works just fine for creating new objects. I can also update all the text fine. It does not let me update images.
I am using Crispy Forms to build the form.
Where did I make a mistake?
My model looks like this:
class Bottle(models.Model):
    class displaySorting(models.IntegerChoices):
        Lighthouse = 1
        Regular = 2

    name = models.CharField(max_length=255)
    category = models.ForeignKey(Category, on_delete=models.CASCADE)
    sorting = models.IntegerField(choices = displaySorting.choices,verbose_name='Sort order')
    brand = models.ForeignKey(Brand, on_delete=models.CASCADE)
    bottle_size = models.IntegerField(validators=[MaxValueValidator(9999)])
    info = HTMLField()
    tasting_notes = HTMLField()
    abv = models.DecimalField(max_digits=3, decimal_places=1, verbose_name='Alcohol %')
    image = ResizedImageField(size=[1000,1000], upload_to=image_upload_handler)
    thumbnail = models.ImageField(upload_to='thumbnails')
    shop_link = models.URLField()
    consumer_shop_link = models.URLField()
    website_link = models.URLField()

  def save(self, *args, **kwargs):
        self.slug = slugify(f"{self.brand}-{self.name}")
        output_size = (300, 169)
        output_thumb = BytesIO()

        img = Image.open(self.image)
        img_name = self.image.name.split('.')[0]

        if img.height > 300 or img.width > 300:
            img.thumbnail(output_size)
            img.save(output_thumb,format='JPEG',quality=90)

        self.thumbnail = InMemoryUploadedFile(output_thumb, 'ImageField', f"{self.brand}-{img_name}_thumb.jpg", 'image/jpeg', sys.getsizeof(output_thumb), None)

        super(Bottle, self).save()

My view looks like this:
@login_required
def bottle_update_view(request, id=None):
    object = get_object_or_404(Bottle, id=id)
    form = BottleForm(request.POST or None, instance=object)

    context = {
        'form':form,
        'object':object
    }

    if form.is_valid():
        form.save()
        context['message'] = 'Data saved'

    return render(request,'Academy/bottle_crud.html',context)

and finally my template like this:
    {% block content %}

Add & Update bottle

       {% if message %}

{{message}}

Check bottle

       {% endif %}

       <form enctype='multipart/form-data' method='POST'>
        {% crispy form %}
        </form>

{% endblock content %}

EDIT:
I've added the request.FILES to to form like this:
form = BottleForm(request.POST or None, request.FILES, instance=object)

But the the form will no longer load any info to be updated, it's just an empty form now.
Comment: Does this answer your question? [How to upload a file in Django?](https://stackoverflow.com/questions/5871730/how-to-upload-a-file-in-django)
Comment: `request.FILES`
Comment: @IvanStarostin I added it here: form = BottleForm(request.POST or None, request.FILES, instance=object). After that it would not load any info on the form.
Answer: img.save(output_thumb,format='JPEG',quality=90)
    self.thumbnail = InMemoryUploadedFile(output_thumb, 'ImageField', f"{self.brand}-{img_name}_thumb.jpg", 'image/jpeg', sys.getsizeof(output_thumb), None)

You specified jpeg as your image type and you are trying to uploaf files other than jpeg.
Rewrite in this way:
def save(self, *args, **kwargs):
self.slug = slugify(f"{self.brand}-{self.name}")
output_size = (300, 169)
output_thumb = BytesIO()

img = Image.open(self.image)
img_name = self.image.name.split('.')[0]

if img.height > 300 or img.width > 300:
    img.thumbnail(output_size)
            if img.format == 'PNG':
                img.save(output_thumb, format='PNG', quality=90)
            elif img.format == 'JPG':
                img.save(output_thumb, format='JPG', quality=90)
            elif img.format == 'GIF':
                img.save(output_thumb, format='GIF', quality=90)
            else:
                img.save(output_thumb, format='JPEG', quality=90)

self.thumbnail = InMemoryUploadedFile(output_thumb, 'ImageField', f"{self.brand}-{img_name}_thumb.jpg", 'image/jpeg', sys.getsizeof(output_thumb), None)

This is working for me now, but if anyone has smarter way of writing this code, kindly post here.

Django form not updating files/images


Evento click javascript não chama a action - Asp.net Core

Question: Eu criei um bloco modal para exibir minha View Create. Ao clicar no btnNovo, um evento javascript deve chamar a minha Action Create retornar uma view em branco e carregar dentro da modal... Só que apenas a modal está abrindo. Eu acompanhei com BP e constatei que o evento do click não está chamando a Action Create com a rota que eu estou passando. Alguém sabe me dizer onde estou errando?

 [HttpGet]
        [Authorize(Policy = "CanWritePessoaSituacaoData")]
        [Route("situacoes-gerenciamento/cadastrar-novo")]
        public IActionResult Create()
        {
            return View();
        }

Segue abaixo o código do btnNovo, o bloco modal e a ref. ao script:

                     Cadastrar Novo

                <button type="button" class="close" data-dismiss="modal" aria-hidden="true">x</button>

Gerenciar Situações de Pessoas

                    Carregando...

@section scripts {
    @{await Html.RenderPartialAsync("_ValidationScriptsPartial");}


Blank initial row when scraping and parsing BeautifulSoup TR:TD table

Question: When I parse the a
/ table in beautiful soup, the first row is always blank. Is there a way to get rid of this easily?
Current code:
soup = BeautifulSoup(driver.page_source, 'html.parser')
for row in soup.select('tr'):
    text_columns = [td.a.get_text(strip=True) if td.a else td.get_text(strip=True) for td in row.select('td')]
    print(text_columns)

Current output:
[]
['date 1', 'details 1', 'amount 1', '', 'balance 1']
['date 2', 'details 2', 'amount 2', '', 'balance 2']
[etc....]

Answer: As I see soup.select('tr') returns a list so you can do something like this: for row in soup.select('tr')[1:]: starting from second element.
The full code would be:
soup = BeautifulSoup(driver.page_source, 'html.parser')
for row in soup.select('tr')[1:]:
    text_columns = [td.a.get_text(strip=True) if td.a else td.get_text(strip=True) for td in row.select('td')]
    print(text_columns)


How to simplify expression $\int_a^b f(x)dx+\int_{f(a)}^{f(b)} f^{-1}(x)dx \ ?$

Question: How to simplify expression
$$\int_a^b f(x)dx+\int_{f(a)}^{f(b)} f^{-1}(x)dx \ ?$$
The answer is $bf(b)-af(a)$  but I am wondering how to get the answer.
Comment: its easy use the symmetry of  inverse and the function about y=x line
Answer: Here is a graphical proof (that doesn't assume that $f$ is differentiable).
Take a look at the following figure :

The colored areas constitute a rectangle whose area can be computed in two ways :
A) globally, as :
$$\text{length} \times \text{width} \  = \ b.f(b)\tag{1}$$
B) by considering it as partitionned into three non-overlapping regions :

the blue region (deep blue + light blue) : its area is the area under the curve of $f$ starting at $a$ and ending at $b$ i.e., $\int_a^b f(x)dx$,
the red region (deep red + light red) : its area is the area under the curve of $f^{-1}$, starting at $f(a)$ and ending at $f(b)$, i.e., $\int_{f(a)}^{f(b)} f^{-1}(x)dx$ (one has to turn his head 90° !),
the green region, a rectangle whose area is : $a.f(a)$.

Equating the two ways of computing this area, we get :
$$b.f(b)=\int_a^b f(x)dx+\int_{f(a)}^{f(b)} f^{-1}(x)dx + a.f(a)$$
which is equivalent to the relationship you want to establish.
Addendum : It has taken me some time to spot an analogous question. Here it is, on this site : Show rigorously that the sum of integrals of $f$ and of its inverse is $bf(b)-af(a)$ with some very interesting answers (the accepted answer of Julian Aguirre, and the answer of Kobe ; the answer of Vim is the same as mine). Another reference : Prove that $\int_a^bf(x)dx+ \int_{f(a)}^{f(b)} f^{-1}(y)dy=bf(b)-af(a)$.
Last but not least, a generalisation of the equation at hand is given by Young's inequality : (http://mathworld.wolfram.com/YoungsInequality.html) as remarked in this answer (https://math.stackexchange.com/q/1154557).
Comment: @Peter Szilas You shouldn't have deleted your answer : it was perfectly correct, with the introductory comment " Let us assume that $f$ is differentiable". I strongly advise you to undelete it.
Comment: Thank you very much.


Bootstrapping mixed effect regression coefficients in statsmodels

Question: I have a mixed effect model that looks like this:
import statsmodels.formula.api as smf

formula = "revised_error ~ C(condition, Treatment('solo_feedback'))*round_index"

model = smf.mixedlm(forumla, data=data, groups=data['player_id']).fit()

Now model.conf_int() gets me the confidence interval for the fitted parameters based on the standard normal distribution. However, I want the bootstrapped coefficients and confidence but model.boostrap() throws an error:
AttributeError: 'MixedLMResults' object has no attribute 'endog'

Comment: any update on the issue ? How did you solve this ? Best, Gautier
Answer: The generic bootstrap in Statsmodels (what you get when you call the bootstrap method) was not written to handle more complicated settings like mixed models.
Applying the non-parametric bootstrap to dependent data is not straightforward, and I'm not even sure if there is a standard way to do it.  The parametric bootstrap would be more straightforward.  Since we now have a get_distribution method (added recently for mediation analysis), it would not be that difficult to implement a parametric bootstrap.  But this is not something that is currently available.


combine multiple rows in pyspark dataframe into one along with aggregation

Question: I have the below df with 2 rows which I want to combine into one row.
+--------------------+--------------------+----+--------------------+--------------------+---------------+-------+-----------+-------------+-------------+-------------+-------------+-------------+----------------------+----------------------+----------------------+----------------------+----------------------+----------------+----------------+----------------+----------------+----------------+------------------+------------------+------------+--------------------+-------------+--------+---------------+
|                 vid|                 sid|pvid|                 cid|             orderId|         cartId|storeId|anchor_item|substitutes_1|substitutes_2|substitutes_3|substitutes_4|substitutes_5|impressions_sub_item_1|impressions_sub_item_2|impressions_sub_item_3|impressions_sub_item_4|impressions_sub_item_5|click_sub_item_1|click_sub_item_2|click_sub_item_3|click_sub_item_4|click_sub_item_5|preferred_sub_item|preferred_sub_rank|search_query|preferred_sub_source|     prefType| date_id|event_timestamp|
+--------------------+--------------------+----+--------------------+--------------------+---------------+-------+-----------+-------------+-------------+-------------+-------------+-------------+----------------------+----------------------+----------------------+----------------------+----------------------+----------------+----------------+----------------+----------------+----------------+------------------+------------------+------------+--------------------+-------------+--------+---------------+
|526CC6AF-6A62-405   |911D8EC7-5B1F-4AD   |    |170836c4b25e4b0f9   |67374d42-024c-430   |200010858176654|    742|   14940752|         null|         null|         null|         null|         null|                  null|                  null|                  null|                  null|                  null|       842643501|            null|            null|            null|            null|          10849171|                 2|            |                P00N|SELECTED_PREF|20230427|  1682593258203|
|526CC6AF-6A62-405   |911D8EC7-5B1F-4AD   |    |170836c4b25e4b0f9   |67374d42-024c-430   |200010858176654|    742|   14940752|         null|         null|         null|         null|         null|                  null|                  null|                  null|                  null|                  null|            null|        10849171|            null|            null|            null|          10849171|                 2|            |                P00N|SELECTED_PREF|20230427|  1682593276038|
+--------------------+--------------------+----+--------------------+--------------------+---------------+-------+-----------+-------------+-------------+-------------+-------------+-------------+----------------------+----------------------+----------------------+----------------------+----------------------+----------------+----------------+----------------+----------------+----------------+------------------+------------------+------------+--------------------+-------------+--------+---------------+

I have done that as below

cols_to_merge = [f.first(x, ignorenulls=True).alias(x) for x in _final_df1.columns[8:23]]
_final_df1.groupBy(_final_df1.columns[:8]+ _final_df1.columns[23:29]).agg(*cols_to_merge)

But now I want the max value in column event_timestamp to be also part of the combined row. How can I get that??
+--------------------+--------------------+----+--------------------+--------------------+---------------+-------+-----------+------------------+------------------+------------+--------------------+-------------+--------+-------------+-------------+-------------+-------------+-------------+----------------------+----------------------+----------------------+----------------------+----------------------+----------------+----------------+----------------+----------------+----------------+
|                 vid|                 sid|pvid|                 cid|             orderId|         cartId|storeId|anchor_item|preferred_sub_item|preferred_sub_rank|search_query|preferred_sub_source|     prefType| date_id|substitutes_1|substitutes_2|substitutes_3|substitutes_4|substitutes_5|impressions_sub_item_1|impressions_sub_item_2|impressions_sub_item_3|impressions_sub_item_4|impressions_sub_item_5|click_sub_item_1|click_sub_item_2|click_sub_item_3|click_sub_item_4|click_sub_item_5|
+--------------------+--------------------+----+--------------------+--------------------+---------------+-------+-----------+------------------+------------------+------------+--------------------+-------------+--------+-------------+-------------+-------------+-------------+-------------+----------------------+----------------------+----------------------+----------------------+----------------------+----------------+----------------+----------------+----------------+----------------+
|526CC6AF-6A62-405   |911D8EC7-5B1F-4AD   |    |170836c4b25e4b0f9   |67374d42-024c-430   |200010858176654|    742|   14940752|          10849171|                 2|            |                P00N|SELECTED_PREF|20230427|         null|         null|         null|         null|         null|                  null|                  null|                  null|                  null|                  null|       842643501|        10849171|            null|            null|            null|
+--------------------+--------------------+----+--------------------+--------------------+---------------+-------+-----------+------------------+------------------+------------+--------------------+-------------+--------+-------------+-------------+-------------+-------------+-------------+----------------------+----------------------+----------------------+----------------------+----------------------+----------------+----------------+----------------+----------------+----------------+

Comment: On what columns you want the groupBy to be applied to ? You can just do df.groupBy(col1, col2, ..).agg(sum("event_timestamp"))
Comment: Can I do it in the same line? ```_final_df1.groupBy(_final_df1.columns[:8]+ _final_df1.columns[23:29]).agg(*cols_to_merge)```
Comment: what do you mean by *cols_to_merge ? I thought you want the max event_timestamp ?
Answer: you can add max to the aggregation definition:
cols_to_merge = [f.first(x, ignorenulls=True).alias(x) for x in _final_df1.columns[8:23]] + [f.max(f.col("event_timestamp")).alias("event_timestamp")]


How to select a value in dropdown using option text rather than using option value in Angular JS

Question: I have the below code for binding a drop down using angular JS

<select ng-model="selectedOption" ng-options="item as item.Area for item in ItemOptions">
  <option value="">Select a value..</option>
</select>

Where ItemOptions is a an array.

var ItemOptions =[];

ItemOptions = [{C1,A1},{C2,A2},{C3,A3},{C4,A4}];

Finally When I select any option in dropdown the values are appearing just like below.

var country = $scope.selectedOption.Country;
 var area = $scope.selectedOption.Area;

If I want to select a value in above dropdown based on option text. For example
$scope.selectedOption.Area == 'A3'
How can I select a value based on the above criteria. The expected output would be as below.

<select ng-model="selectedOption" ng-options="item as item.Area for item in ItemOptions">
      <option value="C1">A1</option>
            <option value="C2">A2</option>
            <option value="C3" selected="selected">A3</option>
            <option value="C4">A4</option>
</select>

The answer is fine even if its from Javascript, Angular or Jquery
Comment: [this may give you an idea](https://code.angularjs.org/1.3.18/docs/api/ng/directive/ngSelected)
Answer: This is how I would do it in jQuery, hopefully it helps.

$(function(){
  $('#select-box option').prop('selected', false).filter(function(index){
      return $(this).text() == 'A4';
    }).prop('selected', true);
});

<select id="select-box" ng-model="selectedOption" ng-options="item as item.Area for item in ItemOptions">
  <option value="C1">A1</option>
  <option value="C2">A2</option>
  <option value="C3" selected="selected">A3</option>
  <option value="C4">A4</option>
</select>


Feature construction and normalization in machine learning

Question: Lets say I want to create a Logistic Classifier for a movie M.
My features would be something like age of the person, gender, occupation, location.
So training set would be something like:

Age  Gender Occupation Location      Like(1)/Dislike(0)
23   M      Software   US            1
24   F      Doctor     UK            0

and so on....
Now my question is how should I scale and represent my features.
One way I thought:
Divide age as age groups, so 18-25, 25-35, 35-above, Gender as M,F, Location as US, UK, Others. Now create a binary feature for all these values, hence age will be having 3 binary features each corresponding to an age group and so on. So, a 28 years Male from US would be represented as 010 10 100   (010-> Age Group 25-35, 10 -> Male, 100 -> US)
What could be the best way to represent features here ?
Also, I noticed in some e.gs. of sklearn that all the features have been scaled/normalized in some way, e.g. Gender is represented by two values, 0.0045 and -.0.0045 for Male and female. I don't have any clue on how to do scaling/mormalization like this ?
Comment: Its not clear to me why you want to scale your features? Often features are normalized to have 0 mean unity std dev. You might need to define the problem in terms of what the classes are that you are trying to classify, logistic regression is useful for binary classification.
Comment: You certainly don't want to categorize ages.

How is "rating of movie" measured? Is it a 1 to 10 scale, a "like/dislike" or what?
Comment: For simplicity, lets assume that there are only two classes, Like and Dislike. Like being 1 and Dislike being 0. Have changed the problem statement to reflect this.
Answer:  Binary case
If you want your features to be binary, the good representations for categorical (resp. real) values are the one hot (resp. thermometer) encoding. You don't need to normalize them.
For the one hot encoding of a categorical feature, you simply reserve one bit for each class. The length of this encoding is therefore the number of classes of your feature. Lets take your example of country,

00001 for US
00010 for UK
00100 for Asia
01000 for Europe
10000 for other

For the thermometer encoding of a real/integer feature, you have to choose a length and the thresholds. For your example of age, you've chosen to split age according to the thresholds 18,25 and 35. The coding will be

000 for 0-17
001 for 18-25
011 for 25-34
111 for 35-above

Putting both together, you obtain here an encoding of size 5+3=8 bits.
For a 30 year old UK resident we have
$$\overbrace{0 \cdot 0 \cdot 0 \cdot 1 \cdot 0}^{UK}\cdot \overbrace{0 \cdot 1 \cdot 1 }^{30yo}$$
Continuous case
If your regression model allows it, you should prefere to keep a real value for a real/integer feature which contains more information.
Let's reconsider your example. This time we simply let the value for age as an integer. The encoding for a 30 year old UK resident is thus
$$\overbrace{0 \cdot 0 \cdot 0 \cdot 1 \cdot 0 }^{UK}\cdot \overbrace{30 }^{30yo}$$
As BGreene said, you should then normalize this value to keep a mean of 0 and a standard deviation of 1, which insure stability of many regression models. In order to do that, simply subtract the empirical mean and divide by the empirical standard deviation.
<code>Y_normalized = ( Y - mean(Y) ) / std(Y)
</code>
If the mean of all the age of all persons in your data base is 25, and its standard deviation is 10, the normalized value for a 30y.o. person will be $(30-25)/10 = 0.5$, leading to the representation
$$\overbrace{0 \cdot 0 \cdot 0 \cdot 1 \cdot 0}^{UK}\cdot \overbrace{0.5 }^{30yo}$$
Comment: Cool... so lets say we have an example person as follows: Country: UK, AgeGroup : 25-34. This will lead to values as **Country: 2**, **Agegroup: 4** if we use **one hot** encoding. Now while creating  a feature vector we should normalize these. So lets say they come as 0.4 and 0.6, then our input feature vector to model essentially becomes [0.4, 0.6], correct ?
Comment: hmm.. if I use "one hot" encoding should I convert the encoding value to its Integer representation as a feature, e.g. 0010 becomes 2. OR should I treat this as a set of 4 features out of which only one will be ON ? In the former case does not it introduce a notion that 1000 is further to 0001 then 0100 which might not be the intention as we don't want US feature value to be closer to UK feature value then Asia value or something else ?
Comment: I edited my answer to clarify these points. You don't need to normalize binary features, and you have to treat them as a vector, don't convert them into an integer.
Comment: Really helpful and crystal clear explanation. Thanks!
Comment: @Emile, could you explain *why* you say it is unnecessary to normalize/scale binary features?


How to interpret a correlation chart?

Question: I'm learning PCA using R. I use Factorminer package.
For learning purpose I used following code
<code>library('FactoMineR')
data(decathlon)
res.pca1 = PCA(decathlon[, 1:10], scale.unit = TRUE, ncp = 4,graph = TRUE)
</code>
It is generating a correlation chart. By looking this it is clear that first & second Principal Component are explaining 52.73% variability in the data. But I'm not sure how to interpret the variables correlation by seeing this plot.
Can you expert please help me to understand how to interpret this correlation chart?
Comment: Your question would benefit greatly from including the actual plot...
Answer: You should look at the website of FactoMineR package:
http://factominer.free.fr/classical-methods/principal-components-analysis.html
They explain the data you use. I could try an explanation by myself, but I cannot be more clear.
Here I copy/paste the fist lines of what they wrote:

The first two dimensions resume 50% of the total inertia (the inertia is the total variance of dataset i.e. the trace of the correlation matrix).
The variable "X100m" is correlated negatively to the variable "long.jump". When an ahtlete performs a short time when running 100m, he can jump a big distance. Here one has to be careful because a low value for the variables "X100m", "X400m", "X110m.hurdle" and "X1500m" means a high score: the shorter an athlete runs, the more points he scores.
The first axis opposes athletes who are "good everywhere" like Karpov during the Olympic Games between those who are "bad everywhere" like Bourguignon during the Decastar. This dimension is particularly linked to the variables of speed and long jump which constitute a homogeneous group.
  The second axis opposes athletes who are strong (variables "Discus" and "Shot.put") between those who are not.
  The variables "Discus", "Shot.put" and "High.jump" are not much correlated to the variables "X100m", "X400m", "X110m.hurdle" and "Long.jump". This means that strength is not much correlated to speed.
At the end of this first approach, we can divide the factorial plan into four parts: fast and strong athletes (like Sebrle), slow athletes (like Casarsa), fast but weak athletes (like Warners) and slow and weak (relatively speaking!) athletes (like Lorenzo).
The winners of the decathlon are those who scored the most (or those whose rank is low).
  The variables the most linked to the number of points are the variables which refer to the speed ("X100m", "X110m.hurdle", "X400m") and the long jump. On the contrary, "Pole-vault" and "X1500m" do not have a big influence on the number of points. Athletes who are strong for these two events are not favoured.

And so on...

How to interpret a correlation chart?


What is a seamless way to migrate a user profile in vista?

Question: What is the best way to migrate a user profile in Vista?  I just tried the Vista Easy Transfer and found that it did not do everything I wanted it to do.  I am trying to leave a leave a domain and join a workgroup.
Comment: Somewhat related to http://serverfault.com/questions/76713/
Comment: Vista is acting somewhat differently.
Comment: Care to elaborate on what is not working?  Or what you are expecting it to do?
Comment: Sorry, I was quite vaugue earlier.  I'm attempting to leave a domain and join a workgroup.  Ideally I would like it to be transfer to be invisible to my user, this involves migrating the profile, but I'm really not sure how to do it.
Answer: I wrote about how to migrate a profile on my blog, (as well as why I was trying to do it) but the short, practical answer is to just download the user profile wizard from Forensit and use that.  I tried many other methods, but could not find any other good way of getting the job done.
Answer: I know that Windows7  has a built-in backup feature that will backup and/or restore your user profile.


I would like to make a field part of a many to many relationship

Question: I have the following table called stores
<code>   id     store       zip
   1     Market 1    01569
   2     Market 2    01551
   3     Market 3    10468
   4     Market 4    10435
</code>
Zip code table
<code>   INSERT INTO `zip_codes` (`restaurants_locations_id`, `zip`, `state`, `latitude`, `longitude`, `city`, `full_state`) VALUES
(1, '06001', 'CT', ' 41.789698', ' -72.86431', 'Avon', 'Connecticut'),
(2, '06002', 'CT', ' 41.832798', ' -72.72642', 'Bloomfield', 'Connecticut'),
(3, '06010', 'CT', ' 41.682249', ' -72.93365', 'Bristol', 'Connecticut'),
(4, '06013', 'CT', ' 41.758415', ' -72.94642', 'Burlington', 'Connecticut'),
(5, '06016', 'CT', ' 41.909097', ' -72.54393', 'Windsorville', 'Connecticut'),
(6, '06018', 'CT', ' 42.023510', ' -73.31103', 'Canaan', 'Connecticut'),
(7, '06019', 'CT', ' 41.834247', ' -72.89174', 'Canton', 'Connecticut'),
</code>
etc....
what I want is to create a table which will connect the zip field in the stores tables where one store will be found in several zicodes...
How would that table would look like?
Thanks
Comment: I'm afraid I don't really get what you want. Don't you already have `zip` in `zip_code_table` pointing to `zip` in `stores`? Shouldn't a Foreign Key be sufficient? Is this what you are asking for? What database are you using?
Comment: Not quite sure what you want to achieve. Please try to rephrase your question.
Comment: I am using the stores database, sometimes a stores can deliver to outside the only zicode it has assign right now, instead of only being able to assign only one zid code I would like to assign several picking them up from the database zip_codes which I am not using right now.
Answer: Create a table called Store_To_Zip_Map or whatever you want to call it.
This table only needs 2 columns: store_id and zip
Simply make a row in that table for each Store ID and Zip pairing. In the example below, stores 1 and 2 each have 2 zip codes. One of the zip codes is attached to both stores.
<code>store_id    zip
1           11111
1           12222
2           12222
2           22222
</code>


Let f(z) be analytic on the unit disk D.f(r) = max{| f(z)| : |z| = r}. Does f'(r) > 0, if f is not a constant?

Question: Let $f(z)$ be analytic on the unit disk $D.$
$f(r) = \max\{| f(z)| : |z| = r\}.$
Why $f'(r) > 0,$ if $f$ is not a constant.
And why if $f(0) = 0,$ then $r f ′
(r) \geq f(r)$ and the equality holds if and only if $f(z) = cz$ for
some nonnegative constant $c.$
It looks familiar but I just cannot recall any theorem dealing with this.
Comment: You should use a different name, e.g. $g(r)$ instead of $f(r)$ because $f$ is already the given analytic function.
Answer: For harmonic functions:

because the maximum is always on the boundary
is a monotonicity formula: you obtain it by comparing the Dirichlet energy of your function with the Dirichlet energy of the 1-homogeneous function with the same value on a circle.

For analytic functions you know that real and imaginary part are harmonic... don't know if this is the best way to obtain these informations.
Comment: I was about to write something like this, but got stuck at the point where I wanted to state that $\frac{\mathrm d}{\mathrm dr}\max\{\,|f(z)|: |z|=r\,\}$ exists in the first place. Is there a simple argument for that?
Comment: @HagenvonEitzen I am also stuck at this point.Have you got any idea?
Comment: @HagenvonEitzen I guess f(r) is not another function.The problem just means that f(z) attains it's maximum at z=r.


Help with L'Hospital's Rule for $\lim_{x \to \infty} (\ln3x-\ln(x+7))$

Question:
Use L'Hospital's Rule to evaluate $\lim_{x \to \infty} (\ln3x-\ln(x+7))$

$f(x)=\ln3x$
$f^{\prime}(x)=\dfrac{3}{3x}=\dfrac{1}{x}$
$g(x)=\ln(x+7)$
$g^{\prime}(x)=\dfrac{1}{x+7}$
So we have: $\lim_{x \to \infty}\dfrac{1}{x}-\dfrac{1}{x+7}$
How do you solve this problem, can you show? If I evaluate the limit now I will get $\infty - \infty$, And I will keep getting $x$ in the denominator, so I don't think I'm doing this correctly. Thanks.
My text has the answer $\ln3$
Comment: Actually the limit $\lim_{x \to \infty}\dfrac{1}{x}-\dfrac{1}{x+7}$ is $0-0=0$
Answer: $$\lim_{x \to \infty} \left(\ln3x-\ln \left(x+7 \right)\right)=\lim_{x \to \infty} \ln \left(\frac{3x}{x+7} \right) \to \lim_{x \to \infty} \ln \left(\frac{3}{1} \right)=\ln 3 $$
Comment: Yes, to apply L'Hopital's rule, the function must be in the form $\frac{f(x)}{g(x)}$ while satisfying the condition outlined by @NigelOvermars in the comment below
Comment: And it must be in the form of $\frac{0}{0}$ or $\frac{\infty}{\infty}$.
Comment: +1 thanks, very clear. So how I was solving is completely incorrect?
Comment: Oh, I see! thanks again.


Iterating KeyValueCollection?

Question: I have a function which return in the form of a KeyValueCollection, How to iterate it to get the key and value in jsp... Any suggestion????
Comment: There is a method in the api tat we are using... we can iterate the key value using that... Sorry to bother!!!!
Answer: Use the jstl tags as follows. The tag library should be on your jsp imports
<c:forEach var="entry" items="${applicationScope}">
    Name:  ${entry.key}
    Value: ${entry.value}

  </c:forEach>

Comment: I cannot use a jstl tag... suggestion in jsp please...:(
Comment: Then do the same as you would do in a normal Java class. This is not related to JSP then.

Iterating KeyValueCollection?


Difference of means -- Testing two algorithms

Question: I have to perform 2 tests of the difference of means for the results obtained by 2 learning algorithms.
TEST 1

I have 2 algorithms
I take a data set (with N examples)
I execute both algorithms 10 times
Each time, each algorithm is trained on a set of 9/10*N examples and tested against a set of N/10 examples (i.e. a 10-fold cross validation is done), and the result is a XX% value of correct classified examples in such a test set.

Here I use the t-test.
I know that the t-test makes assumptions of normality.
I use it because the data set is quite big ( N>100 ), so I can assume a normal distribution of data (and if data is normally distributed, the means of a single algorithm are normally distributed, and so the differences of the means of the two algorithsm are normally distributed). Or should I consider only N/10 as the sample size from which the means are drawn?
TEST 2

I have two algorithms
I take 10 instances of a combinatorial optimization problem (a Travelling Salesman Problem)
I run each algorithm on the 10 instances
For each run, a write down the result value (that is at some distance from the optimal value for that instance)

How can I perform a test on the differences of means?
Which means can I consider?
Which data should I check for normality?
Answer: For TEST 1, I gather the response variable is either correct or incorrect, that is, these are Bernoulli trials.  For the Bernoulli / Binomial distribution, the standard deviation is a function of the mean, thus you aren't estimating the standard deviation from your data in the same sense as a t-test.  So, in this case, you can use a z-test for the difference of two proportions (see, the example in this table).  I don't know what software you're using, but if you were to test this in R, e.g.,  you would use ?prop.test.
For TEST 2, I think it's highly unlikely that the population underlying your data would be normally distributed, and with only 10 data, you wouldn't have enough for the central limit theorem to cover for you.  (Note that my reasoning here is a-priori; I think it usually doesn't make much sense to test for normality, and especially in this case with so few data, see: Is normality testing essentially useless?)  Instead, I would recommend you use the Mann-Whitney U-test, which is a non-parametric analog of the t-test.  It would be appropriate if your data are not normal and your sample size is too small; it also can be more powerful than the t-test under these conditions.  In R, you would use ?wilcox.test.


How to create zip archive without ini and zip files

Question: I have a folder which contain :
bd.txt
/Config
/DLL
/Rapports
bx1478.ini
ARCH.zip

I want to create an archive without ini and zip files.
I try as below but i get an empty archive :
import subprocess
SZ_PATH_TMP = '../tmp'
SZ_PATH_WORK = '../work'
full_label_version = 'ARCH'
subprocess.call(['7z', 'a', SZ_PATH_TMP+'/'+full_label_version+'.zip', SZ_PATH_WORK+'/* -xr!*.zip -xr!*.ini'])

Comment: You are using relative paths. Are you sure they are correct and that you are running your python script from the correct location. Try setting the working directory for the subprocess call to the appropriate directory.
Comment: Yes the relative paths work fine because when i delete -xr!*.zip -xr!*.ini, the generation works but with all files
Answer: When you pass a list of arguments to subprocess, each option needs to be a separate list item.
import subprocess
SZ_PATH_TMP = '../tmp'
SZ_PATH_WORK = '../work'
full_label_version = 'ARCH'
subprocess.call(
    ['7z', 'a',
    SZ_PATH_TMP+'/'+full_label_version+'.zip',
    SZ_PATH_WORK+'/*,  '-xr!*.zip', '-xr!*.ini'])

If you don't know exactly how the shell breaks the command line up into arguments, maybe pass the entire command as a string to shlex.split()
Comment: If you're on Windows, I'm not sure `shlex` works correctly there. (Maybe "correctly" isn't even well-defined on Windows.)


Square Connect API list payments does not return Mobile Staff ID

Question: I am using the Square Connect API to retrieve payments for a customer.  The Square Account has mobile staff setup.  The mobile staff ID determines the location of the sale.  I cannot access that in the API but is in the transactions CSV download.
Answer: Thanks for pointing out this limitation of the API. I've filed a feature request with the Square Connect engineering team.
As a suboptimal temporary solution, Payment objects returned by Connect endpoints do include a creator_id field, which corresponds to the staff member that processed the payment. Unfortunately, this ID does not match the ID specified by the merchant when adding the mobile staff member. If you can determine which creator_id maps to which mobile staff member, you can maintain a one-to-one mapping between them.


Does the limit exist if $a=0$?

Question: \begin{align}
&\lim_{x\to 0}\frac{\sin^2 ax}{x^2}\\
&=\lim_{x\to 0}a^2\left(\frac{\sin ax}{ax}\right)^2\tag{1}\\
&=a^2
\end{align}
Now, I think that if $a\neq0$, the above process is fine. However, if $a=0$, I think $(1)$ becomes invalid, so the process becomes invalid. Am I right?
Comment: If $a=0$, the function is 0 from the start. So, the process is invalid, but the function still has a limit.
Answer: Yes, the process becomes invalid. Nevertheless, the equality $\displaystyle\lim_{x\to0}\frac{\sin^2ax}{x^2}=a^2$ still holds, since both numbers are equal to $0$.
Answer: For $a=0$ the function is identically equal to zero and therefore limit is equal to zero since
$$\lim_{x\to x_0}0=0$$
It would be different for a factor $a$ in the denominator.

Does the limit exist if $a=0$?


Subgroups of Galois group?

Question: Set $f(X) = X^4 − 6X^2 − 2$ and denote by $K$ the splitting field of $f$ over $\mathbb{Q}$. I am interested in finding the subgroups of $\operatorname{Gal}(K : Q)$ and hence finding all subfields of $K$.
I have already found the roots of $f$ to be $\pm\alpha$ and $\pm\beta$, where $\alpha=\sqrt{3+\sqrt{11}}$ and $\beta=\sqrt{3-\sqrt{11}}$, and $K=\mathbb{Q}(\alpha,\alpha\beta)$. The extension $K:\mathbb{Q}$ is then Galois, non abelian with order 8.
I just don't know how to do the last step to find these subgroups and subfields.
Comment: Computing the discriminant should help narrow down the Galois group.
Answer: The Galois group contains the following three maps:
$$\begin{align}
\alpha\mapsto - \alpha&, \beta\mapsto \beta\\
\alpha\mapsto \alpha&, \beta\mapsto - \beta\\
\alpha\mapsto\beta&, \beta\mapsto\alpha
\end{align}
$$
Call these three maps $n_\alpha, n_\beta$ and $s$ ("Negate $\alpha$", "Negate $\beta$", and "Swap") respectively. They all have oder $2$, but $sn_\alpha$ and $sn_\beta$ have order $4$. Also, $n_\alpha$ and $n_\beta$ commute, but neither of them commute with $s$ (we have $n_\alpha s = sn_\beta$), although $n_\alpha n_\beta$ does commute with $s$.
This gives us the 8 elements
$$
e, n_\alpha, n_\beta, n_\alpha n_\beta,\\
s, sn_\alpha, sn_\beta, sn_\alpha n_\beta
$$
It doesn't take much work from here to prove that this is the dihedral group $D_4$.
And with that, we can start to look at what subgroups we have and what fixed subfields they give rise to. First the order $2$ subgroups, giving rise to degree $4$ extensions:

$\langle n_\alpha\rangle$ clearly fixes $\beta$, so $\Bbb Q(\beta)$ is the corresponding fixed field
$\langle n_\beta\rangle$ clearly fixes $\alpha$, so $\Bbb Q(\alpha)$ is the corresponding fixed field
$\langle n_\alpha n_\beta\rangle$ fixes $\alpha\beta = i\sqrt2$ as well as $\alpha^2 = 3 + \sqrt{11}$, so the corresponding fixed field is $\Bbb Q(\alpha\beta, \alpha^2) = \Bbb Q(i\sqrt2, \sqrt{11})$
$\langle s\rangle$ fixes $\alpha + \beta$, so the fixed field is $\Bbb Q(\alpha + \beta)$
$\langle sn_\alpha n_\beta)$ fixes $\alpha - \beta$, so the fixed field is $\Bbb Q(\alpha - \beta)$

Then the order $4$ subgroups:

$\langle sn_\alpha\rangle$ fixes $\alpha\beta(\alpha^2-\beta^2) = 2i\sqrt{22}$, so the fixed field is $\Bbb Q(i\sqrt{22})$
$\langle n_\alpha, n_\beta\rangle$ fixes $\alpha^2-\beta^2 = 2\sqrt{11}$, so the fixed field is $\Bbb Q(\sqrt{11})$
$\langle s, sn_\alpha n_\beta\rangle$ fixes $\alpha\beta=i\sqrt2$, so the fixed field is $\Bbb Q(i\sqrt2)$

And finally, of course, the trivial subgroup fixes all of $\Bbb Q(\alpha, \beta)$, and the whole group fixes only $\Bbb Q$.

Subgroups of Galois group?


call_user_func_array + array_intersect with an array of array names, possible?

Question: Sorry for the confusing title...
I need to perform an array_intersect() against a variable number of arrays. To do this it seems I need to use the call_user_func_array() function, however, this doesn't seem to be working and gives me the error:
Warning: array_intersect() [function.array-intersect]: Argument #1 is not an array in...

But, if I "print_r" the array to make sure then I see that it is an array:
Array ( [0] => arr_0 [1] => arr_1 )

My code (trimmed to just show the broken part):
$i = 0;
$arr_results = array();
foreach($arr_words as $word) {
    $arrayname = "arr_".$i;
    $$arrayname = array();
    while ($row = mysql_fetch_assoc($search)) {
        array_push($$arrayname, $row['id']);
    }
    array_push($arr_results, "$arrayname");
    $i++
}
$matches = call_user_func_array('array_intersect',$arr_results);

In the full code I'm populating the arrays in the foreach loop with data obtained from sql queries.
Comment: `"$arrayname"` is a string, not an array. `call_user_func_array` will pass each element in  `$arr_results` as argument to `array_intersect`. `array_intersect` expects arrays as arguments, but each item in `$arr_results` is a string, not an array.
Comment: I take $arrayname and turn it into an array, I've just updated the code snippet to show this a bit better.
Comment: If that was true, `print_r` would show a nested array as output, but it shows an array of strings. Why do think `"$arrayname"` becomes an array? Where? *edit:* `array_push($arr_results, "$arrayname");` adds a string to the array. PHP is not evaluating the string. It does not now that the value in `$arrayname` refers to a variable name. Why don't you create an array of arrays from the beginning? That's much easier to handle than variable variables.
Comment: I didn't think that was right because "$arr_results" is an array of array names not a 3 dimensional array.

I turn "$arrayname" into an array with "$$arrayname = array();", this is then populated with data during the while loop.
Comment: But `array_intersect` expects arrays not array names. The names are strings.
Comment: So how do I pass multiple array names to array_intersect when the number of arrays is variable?
Comment: You don't pass *names*, you pass arrays. You have to create an array of arrays, not an array of array names... I will post something.
Answer: From my comments:
"$arrayname" is a string, not an array. call_user_func_array will pass each element in $arr_results as argument to array_intersect. array_intersect expects arrays as arguments, but each item in $arr_results is a string, not an array.
All you have to do is create an array of arrays instead of array names:
$arr_results = array();
foreach($arr_words as $word) {
    $ids = array();
    while ($row = mysql_fetch_assoc($search)) {
        $ids[] = $row['id'];
    }
    $arr_results[] = $ids;
}
$matches = call_user_func_array('array_intersect',$arr_results);

I turn $arrayname into an array with $$arrayname = array();

Right, you create a variable, lets say arr_0 which will point to array. But there is still a difference between the variable name  arr_0  and the string containing the variable name "arr_0". You create an array of strings, and that just won't work. PHP does not know that the string contains a name of a variable. For example, consider this:
$arr = "arr_0";
echo $arr[0];

Based on your logic, it should output the first element of the array, but it does not, because $arr is a string, not an array, although it contains the name of a variable.
You'd have to use eval, but you really should not. You could also use variable variables again:
array_push($arr_results, $$arrayname);

that would work as well, but as I said, variable variables are confusing and in 99% of the cases, you are better of with an array.
Comment: I no longer get the error but... this is now creating a new sub array for each row returned by the SQL query, so the array_intersect doesn't work as each array only has one entry. (Thanks for helping though, it is appreciated).
Comment: This is still returning an array where each row from the SQL query is a sub array...
The output of a print_r($ids) gives me:
`Array
(
    [0] => Array
        (
            [0] => 40
        )

    [1] => Array
        (
            [0] => 42
        )

    [2] => Array
        (
            [0] => 61
        )`
Comment: Simple fix: `array($row['id'])` should be `$row['id']`. I hope that makes sense.
Comment: Don't worry, fixed it, changed `$ids[] = array($row['id']);` to `array_push($ids, $row['storyid']);`.
Comment: Thanks for your help, you're a life saver and stopped me from banging my head against the wall.


Excel VBA: get specific table data from html table

Question: I'm trying to pull the main data table from this website using innerhtml and then looping through but I get bunch of different table data and just a mess (tried webquerytable but the format I get isn't right for me)
http://lcvpd.lt/lt/aktualijos/statistika/uzbsands.php
Is there a way to extract only the main data table?
Sorry for bad communication, I am new to vba and don't know how to correctly explain the sitaution.
Here's where I am so far:
<code>Sub HTML_Table_To_Excel()

    Dim htm As Object
    Dim Tr As Object
    Dim Td As Object
    Dim Tab1 As Object

    'Replace the URL of the webpage that you want to download
    Web_URL = VBA.Trim(Sheets(1).Cells(1, 1))

    'Create HTMLFile Object
    Set HTML_Content = CreateObject("htmlfile")

    'Get the WebPage Content to HTMLFile Object
    With CreateObject("msxml2.xmlhttp")
        .Open "GET", Web_URL, False
        .send
        HTML_Content.body.innerHTML = .responseText
    End With

    Column_Num_To_Start = 1
    iRow = 2
    iCol = Column_Num_To_Start
    iTable = 10

    'Loop Through Each Table and Download it to Excel in Proper Format
    For Each Tab1 In HTML_Content.getElementsByTagName("table")
        With HTML_Content.getElementsByTagName("table")(iTable)
            For Each Tr In .Rows
                For Each Td In Tr.Cells
                    Sheets(1).Cells(iRow, iCol).Select
                    Sheets(1).Cells(iRow, iCol) = Td.innerText
                    iCol = iCol + 1
                Next Td
                iCol = Column_Num_To_Start
                iRow = iRow + 1
            Next Tr
        End With
        iTable = iTable + 1
        iCol = Column_Num_To_Start
        iRow = iRow + 1
    Next Tab1
End Sub
</code>
Comment: show us what you have tried so far
Comment: @Ken updated post
Answer: You're getting the elements by its tag names, wich is very generic. If your table does not have an specific id or class and you call it by its tag name, it will pull all the elements with that tag name (or even the ones with other names). I'm having some similar problems trying to retrieve specific table data right in the middle of a web page. Wish i could consult the data base, instead of getting html elements inner texts.


Integrating functions in real analysis

Question: The following is a True/False problem from an exam I got back:
If $f'(x)=1$ for each $x\in\mathbb{R}$ and $f(2)=7$ then $f(-2)=-7$.
I said false for this problem, but it was marked wrong. My reasoning for this was to treat this as an initial value problem. First, integrate $f'(x)$ so that $f(x)=\int f'(x)dx=x+c$. Next solve for $c$ from $f(2)=7$. Thus $2+c=7\implies c=5$. So our function is $f(x)=x+5$ where $f'(x)=1\forall x\in\mathbb{R}$ and $f(2)=7$. Both of the statements in the AND statement in the hypothesis are true, therefore the hypothesis is true. Next, checking $f(-2)=3\neq-7$ therefore the conclusion of the implication is false. Here we have True imply false, therefore the implication is false.
Is there an error in my rationale?
Comment: Everything you show is correct.  My best thought is if you have not discussed integration within the context of the real analysis class, you must use only the theorems / definitions covered thus far in class, and need to use an argument based on the mean value theorem or similar.
Comment: The exam was graded on a raw score of whether or not we chose the correct answer, so no explanations were necessary. In the post, I wanted to be confident in my response so I wrote out my thoughts. Also, I didn't want to appear that I wasn't merely fishing for answers. Though you do have a point with the explanation here. This problem was testing on the MVT, not my skills to solve IVP.
Comment: One potential problem is that what if the ODE has a non-unique solution (well it does as it's a first order linear ODE with CC by a famous theorem). But other than that, it looks correct.
Answer: Your argument is correct. Besides, note that$$\frac{f(2)-f(-2)}{2-(-2)}=\frac{14}4=\frac72.$$But, by the mean value theorem, that quotient should be $1$.
Comment: I'm glad I could help.
Comment: A classmate noted that she used the MVT for this problem. I didn't remember how exactly she used it, but reading this I can see that your argument matched hers. Much simpler explanation. Thanks!

Integrating functions in real analysis


How to aggregate $lookup with MongoDB C# driver?

Question: I am using below lookup to get all required documents & then apply LINQ query to find specific documents & it is working fine as expected.
     new BsonDocument("$lookup", new BsonDocument()
                        .Add("from", "JournalInstructionReplication")
                        .Add("localField", "_id")
                        .Add("foreignField", "_eid")
                        .Add("as", "Replicated"))
 var cursor = await collection.AggregateAsync(pipeline, options);
            List<BsonDocument> list = cursor.ToList();

           var failedDocs = list.Where(d => d["Replicated"][0]["lastReplicationStatus"] != "success" ||
                             d["Replicated"][0]["eSeq"] != d["Replicated"][0]["lastReplicationSequence"])
                            .ToList();

I want to merge above LINQ query with existing lookup query.
Any idea how to achieve with lookup?
Answer: To avoid running [0] you can run $unwind since there's clearly 1:1 relationship between joined collections (you join them by _id). The in-memory part has to be converted into below $match:
{
    $match: {
        $expr: {
            $or: [
                { $ne: [ "$Replicated.lastReplicationStatus", "success" ] },
                { $ne: [ "$Replicated.eSeq", "$Replicated.lastReplicationSequence" ] },
            ]
        }
    }
}

which would look like below in C#:
var q = _channels.Aggregate()
                    .Lookup(
                    foreignCollectionName: "JournalInstructionReplication",
                    localField: "_id",
                    foreignField:"_id",
                    @as: "Replicated")
    .Unwind("Replicated")
    .Match(new BsonDocument()
    {
        { "$expr", new BsonDocument()
        {
            { "$or", new BsonArray()
            {
                new BsonDocument(){{ "$ne", new BsonArray(){ "$Replicated.lastReplicationStatus", "success" } }},
                new BsonDocument(){{ "$ne", new BsonArray(){ "$Replicated.eSeq", "$Replicated.lastReplicationSequence" } }
            } }
        } }
    }});

var result = q.ToList();

Answer: Thanks a lot. It worked as expected.Final working code as below:
 new BsonDocument("$lookup", new BsonDocument()
                        .Add("from", "JournalInstructionReplication")
                        .Add("localField", "_id")
                        .Add("foreignField", "_eid")
                        .Add("as", "Replicated")),
                 new BsonDocument("$unwind", new BsonDocument()
                        .Add("path", "$Replicated")),
                   new BsonDocument("$match", new BsonDocument()
                        .Add("$expr", new BsonDocument()
                                .Add("$or", new BsonArray()
                                        .Add(new BsonDocument()
                                                .Add("$ne", new BsonArray()
                                                        .Add("$Replicated.lastReplicationStatus")
                                                        .Add("success")
                                                )
                                        )
                                        .Add(new BsonDocument()
                                                .Add("$ne", new BsonArray()
                                                        .Add("$Replicated.eSeq")
                                                        .Add("$Replicated.lastReplicationSequence")
                                                )
                                        )
                                )
                        ))
            };

            var cursor = await collection.AggregateAsync(pipeline, options);
            List<BsonDocument> list = cursor.ToList();

Comment: you can get rid of all that bsondocument noise like this: https://dev.to/djnitehawk/overcoming-the-limitations-of-mongodb-c-driver-1110
Comment: Thanks for sharing this. Let me check.