Which error is displayed in an error ellipse?

Question: I have some bivariate data and I have calculated the error ellipse in the following way: I have first calculated the covariance matrix and then to obtain the radii of the ellipse I have taken the eigenvalues and then their squared root.
Does the length of each of these radii correspond to one standard deviation in the direction of the semi-principal axes?
Comment: The correct calculation is described in the question at http://stats.stackexchange.com/questions/89512.  Other closely related questions include http://stats.stackexchange.com/questions/24380 and http://stats.stackexchange.com/questions/62092. Your question needs to be refined a little: to what does "one standard deviation" refer?  After all, these are *bivariate* data.
Comment: @whuber I've updated the question. What I meant is that when we are talking about the 95% confidence ellipse the square root of the eigenvalues is scaled by the appropriate chi squared value. So, when these values are not scaled (just square roots of the eigenvalues), do the lengths of the radii correspond to one standard deviation?
Answer: No.  To get the radii, you take the square root of the eigenvalues (i.e. the singular values).  See http://en.wikipedia.org/wiki/Confidence_region
Comment: I've wrongly described what I did. In fact, I'm taking the square root of the eigenvalues. I've updated the question.


HP/IBM alternative to Buffalo iSCSI TerraStation?

Question: I'm looking at virtualising some of our infrastructure in order to allow for more resiliance and future expandability.
We have successfully virtualised on single servers with Direct Attached Storage and are now looking for a more future proof solution using a high powered host (or two) and a SAN (or two).
I'm thinking that the host machine will probably be an HP ProLiant DL360 G7 (all of our exisiting infrastructure is HP).
Unfortunately, I am new to the world of SANs. From what I can see, the Buffalo Terrastation III is all I would need in order to setup an iSCSI SAN for VMWare to use. However, I'm a little reticent to go that way as it's a bit too "entry level" for my liking. In particular I would be very keen for more redundancy, power, networking, etc. I'm also very aware that you "get what you pay for".
Therefore, can anyone reccommend equivalents from the big boys? HP/IBM? I have searched high and low on the HP site and seen many options but am struggling to work out if it is all the hardware I will need. Some options appear to need separate controllers from disk enclosures, etc.
Comment: Lenovo? Are you sure you don't mean IBM?
Answer: If you're concerned about resilience I'd avoid putting all your eggs in such a leaky basket as the Buffalo you mention - you want dual controllers and you need dual PSUs.
For iSCSI I'd recommend HP's MSA 2000i G2 and for FC their P2000 G3 - both of which can have dual controllers and PSUs and come with either 24 x 2.5" or 12 x 3.5" disks, plus extra disks can easily be daisy-chained for up to 99/149 disks - they support R0, R1, R3, R6, R10 & R50, support up to 512 LUNs of up to 16TB each plus built-in snapshotting.
I'm sure Lenovo/Dell do something similar too.
Answer: I would actually took a very close look at software solutions. I mean reliable solutions - not something like Openfiler or iSCSI Cake. I would suggest you to google for such solutions as StarWind and Datacore. Actually here you are:
http://www.google.com/search?q=starwind+datacore
I'm using first one, but as I know Datacore is not bad too.
Mine solution supports all the backup features, that you need - snapshots, mirroring, HighAvailability, etc., and, as for me, it is pretty easy-manageable.
Comment: From my experiences with software solutions, I would put Starwind, OpenFiler and FreeNAS all in the same basket - perfectly capable, but not brilliant. Same for Windows Storage Server 2008 R2 (with iSCSI tools)
Comment: Lefthand is pretty robust.


How to convert json to object?

Question: I need to convert a json-string to python object. By object I mean "new" python3 object like:
class MyClass(object):

I found several help for example on jsonpickle documentation. But all I found are tutorials which convert object to json first and after this convert backwards.
I want to convert a json-string from a Rest-API.
Here is what I have done so far:
import requests
import jsonpickle

class Goal(object):
    def __init__(self):
        self.GoaldID = -1
        self.IsPenalty = False

class Match(object):
    def __init__(self):
        self.Goals = []

headers = {
    "Content-Type": "application/json; charset=utf-8"
}

url = "https://www.openligadb.de/api/getmatchdata/39738"

result = requests.get(url=url, headers=headers)
obj = jsonpickle.decode(result.json)
print (obj)

This results in:
TypeError: the JSON object must be str, bytes or bytearray, not 'method'

It's quite clear to me that jsonpickle can't convert this to my classes (Goal, Match), because I don't tell jsonpickle in which class the output should be converted. The problem is I don't know how to tell jsonpickle to convert the JSON in object from type Match? And how can I tell that the list of goals should be of type List<Goal>?
Comment: `obj = jsonpickle.decode(result.content)` => This will give you a dictionary.
Comment: `obj = result.json()` will also give you a dictionary.
Answer: The following lines will give you a dictionary:
obj = jsonpickle.decode(result.content)  # NOTE: `.content`, not `.json`

obj = result.json()

But none of above will give you what you want (python object (not dicitonary)). because the json from the url is not encoded with jsonpickle.encode - whcih add additional information to a generated json (something like {"py/object": "__main__.Goal", ....})

>>> import jsonpickle
>>> class Goal(object):
...     def __init__(self):
...         self.GoaldID = -1
...         self.IsPenalty = False
...
>>> jsonpickle.encode(Goal())
'{"py/object": "__main__.Goal", "IsPenalty": false, "GoaldID": -1}'
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# JSON encoded with jsonpickle.encode  (default unpicklable=True)
#   => additional python class information attached
#   => can be decoded back to Python object
>>> jsonpickle.decode(jsonpickle.encode(Goal()))
<__main__.Goal object at 0x10af0e510>

>>> jsonpickle.encode(Goal(), unpicklable=False)
'{"IsPenalty": false, "GoaldID": -1}'
# with unpicklable=False   (similar output with json.dumps(..))
#   => no python class information attached
#   => cannot be decoded back to Python object, but a dict
>>> jsonpickle.decode(jsonpickle.encode(Goal(), unpicklable=False))
{'IsPenalty': False, 'GoaldID': -1}

If you want an actual Python object which is not a dictionary, i.e. you prefer dic.Goals.[0].GoalGetterName to dic["Goals"][0]["GoalGetterName"], use json.loads with object_hook:
import json
import types
import requests

url = "https://www.openligadb.de/api/getmatchdata/39738"

result = requests.get(url)
data = json.loads(result.content, object_hook=lambda d: types.SimpleNamespace(**d))
# OR   data = result.json(object_hook=lambda d: types.SimpleNamespace(**d))
goal_getter = data.Goals[0].GoalGetterName
# You get `types.SimpleNamespace` objects in place of dictionaries

Comment: Ok thank you very much. Because I don't like the magic string access via dic, is it best practice to use a class method which converts a dic to an object?
Comment: @Sebi, Why do you convert the dict to an object? I just used the dictionary/list/... without converting when I was using Python. (I don't think it's a magic string access ;;;)
Comment: @Sebi, I didn't use this, but check this out: http://jsonmodels.readthedocs.io/en/latest/
Comment: It's because I just don't like the magic string access, for example: dic["Goals"][0]["GoalGetterName"], I would prefer match.goals[0].goalGetterName but maybe this needs to much afford.
Comment: @Sebi, if you don't want to define mapping, you can define custom [`JSONDecoder`](https://docs.python.org/3/library/json.html#json.JSONDecoder) which convert JSON object to [`types.SimpleNamespace`](https://docs.python.org/3/library/types.html#types.SimpleNamespace) instead of Python dictionary.
Comment: @Sebi, BTW, `dic["Goals"][0]["GoalGetterName"]` has no magic. Just simple item accesses ;)
Comment: Thanks for all the stuff :) Now I will find a way. JSONModels look nice. What I mean by magic string is a hardcoded string like "Goals", which I don't because it isn't able to refactor this and so on...But I'm from C# maybe I'm just not familiar enough with this dynamic stuff ;)
Comment: @Sebi, Maybe using `json.loads` with `object_hook` is more appropriate for your case.
Comment: @Sebi, I updated the answer to include an example with `json.loads(..., object_hook=...)`
Comment: Thank you very much. Would give +2 if possible :-D. This helped me alot.
Comment: @Sebi, No problem. Happy Python hacking!
Answer: Do you mean something like this?
import json

class JsonObject(object):

    def __init__(self, json_content):
        data = json.loads(json_content)
        for key, value in data.items():
            self.__dict__[key] = value

jo = JsonObject("{\"key1\":1234,\"key2\":\"Hello World\"}")
print(jo.key1)

which prints:
1234
[Finished in 0.4s]

Answer: A clean approach on recent python versions might be to use marshmallow-dataclass :
from dataclasses import field
from marshmallow_dataclass import dataclass
from typing import List

@dataclass
class Goal:
    GoaldID: int = field(default=-1)
    IsPenalty: bool = field(default=False)

@dataclass
class Match:
    Goals: List[Goal] = field(default_factory=lambda: [])

my_match, _ = Match.Schema().load(result.json())

How to convert json to object?


Does a web request with proxy always require a new connection?

Question: This is a question around how a proxy should behave with HTTPS requests. If there are 2 users behind a proxy and both go to https://example.com one after another, can the proxy reuse the existing TCP connection created with example.com earlier for user1.
Both TCP and HTTPS are different protocols, so it doesn't seem like it should affect anything, and in practice it doesn't as well.
Is there something in existing TLS implementations that might not like this kind of behavior of having two different sessions over the same connection? Would this be a bad idea from a security perspective to have the same TCP connection for different users?
Answer: You mean if two HTTP clients make requests to a reverse HTTP proxy, could the proxy reuse TCP connections to the HTTP server?

Yes, absolutely. This is called connection pooling and it is common in practice. The proxy opens a pool of persistent connections with each backend endpoint. Then, the proxy queues requests and each request gets sent on an available TCP connection.
From a TLS perspective, if the proxy is an HTTP proxy (L7), clients perform TLS handshakes with the proxy, not the backend web servers and, hence, there's no problem. However, if the proxy operates at L4, TLS termination must occur downstream (TLS pass-through), which poses complication.
My current understanding is that L4 proxies maintain 1:1 connections with backends. Meaning, each incoming connection has a corresponding outgoing connection to a backend, which removes our ability to reuse connections and leverage connection pooling.
For example, NGINX does this:

NGINX maintains a "cache" of keepalive connections – a set of idle keepalive connections to the upstream servers – and when it needs to forward a request to an upstream, it uses an already established keepalive connection from the cache rather than creating a new TCP connection.

Load Balancing with NGINX and NGINX Plus, Part 2

Helpful resources

Introduction to modern network load balancing and proxying, Matt Klein
TLS termination proxy
Comment: "*The clients have TLS connections with the proxy, not the backend web servers.*" - this would only be true for an HTTP proxy, but not say a SOCKS proxy. When using HTTPS through a non-HTTP aware proxy, the clients would have TCP connections with the proxy, but they would have TLS sessions with the web servers instead. On the web server side, it will thus end up seeing TLS packets from multiple clients on a single TCP connection, which is not something TLS allows. Only 1 TLS session per TCP connection is supported. So a non-HTTP proxy would have to maintain a separate tunnel for each client.
Comment: That's true. Let me fix that.
Comment: Thanks for you answer! I understand how this would work in a reverse proxy scenario. Would there be a difference in a forward proxy configuration? Let's say if I connect to google or some bank through a proxy. and another user laters connects to the same domain. Would those web servers behave differently if different TLS sessions were created on the same TCP connection?
Comment: I updated my answer based on Remy's feedback. I hand-waved a bit on the L4 proxy details -- hope I don't misguide you. I'll keep looking around for details and update my answer as I learn more.
Comment: An HTTP forward proxy could certainly make such optimizations. Whether or not it does is an implementation detail.
Answer: I don't think it will be possible with https.
Proxying https with http proxy requires client to use http CONNECT method which provides raw TCP connection and reusing upstream connections has a consequence that every subsequent client would have to receive connection with a history. That probably wouldn't be a problem in case of web traffic but will violate other connection aware protocols (such as SSH or SMTP) that for example expect invitation banner. Because proxy is unaware of higher layer protocol that is going to be used therefore it can't assume reusing TCP connection would work.
For proxying plain HTTP traffic (which does not utilize http CONNECT method) that should not be an issue as long as client does not send http request with a header:
<code>Connection: close
</code>
so while it's a common solution in well controlled environments with a reverse proxy I have no idea if any forward proxy is capable of reusing TCP connection to upstream servers that are not limited to some server pool defined in a proxy configuration.


Definition of open ball in discrete metric space

Question: I would like some help clarifying the definition of open balls in the discrete metric space.
The definition I am provided is:
Open balls in the discrete metric space $M = (X,d_0) $ are given by
$B_\epsilon(x) =
\left\{\begin{matrix}
\{x\} & \epsilon \leq 1\\
X & \epsilon >  1
\end{matrix}\right.
$
My question:
Why is it not this way:
$B_\epsilon(x) =
\left\{\begin{matrix}
\{x\} & \epsilon < 1\\
X & \epsilon \geq  1
\end{matrix}\right.
$
My reasoning is because the points lying on the boundary do not lie in $B_\epsilon(x)$, and should be considered NOT part of it. Or am i missing something?
Many thanks.
Comment: If $y\neq x$ then $d(x,y)=1$ hence $y\notin B_1(x)$ ($d(x,y)=1$ is not $< 1$) this is why the first works. For the boundary, for the discret topology, the boundary of any set is itself (every set is clopen). In particular every open ball being also closed is its own boundary.
Answer: Your reasoning is correct. Think about $\mathbb R^2$ with the usual euclidean metric. An open ball does not contain its boundary.
In general, the open ball $B_r(x_0)$ in a metric space $(X,d)$ is defined to be
$$B_r(x_0) := \{x \in X : d(x_0,x)<r\}$$
Comment: What you have posted above is no definition. Its the result of applying the definition of an open ball in the special case of a discrete metric. The "$\leq$" in your question does NOT mean that you take all elements around the center which have distance of $\leq \varepsilon$! It states what the balls look like for different radii. And if you take radius $=1$, you still consider only elements with distance strictly lesser than $1$!
Comment: Thank you, exactly what you said yes - in line with the definition. I guess there is a typo in the book then ...
Comment: @SanjouOdawali No, the book is correct. If you put $r=1$ then $B_1(x) = \{x\}$.
Comment: Thank you Sir, I get it now.
Comment: Hi, I am so sorry! apparently I am still a little lost. Suppose we use  B1(1) on the real line (i.e. open ball centered at 1 with radius 1) Using the definition (< 1), then yes agreed that the open ball is simply {1}. But if it is <= 1 as per the above definition in the question, wouldn't the open ball it be {0,1,2} ?
Comment: Thank you for your patience and explanation! I really get it now!
Answer: You are right. By the very definition of $\def\eps{\varepsilon}B_\eps(x)$, we have
$$ B_\eps(x) = \{y \in X : d(x,y) < \eps\} $$
hence
$$ B_1(x) = \{y \in X : d(x,y) < 1 \} $$
Now the points with distance $1$ to $x$ (that is all points but $x$), do not belong to $B_1(x)$.
Comment: Thank you for clarifying martini, the definition is indeed the way to go


WebServices com C#, fazer de-para

Question: Estou precisando criar um WebService para exportar produtos. Criar o WebServico tudo bem, porém dependendo do cliente, vou ter que alterar o nome dos elementos.
Exemplo, elemento do Código do Produto:
Generico <codigo>1</codigo><descricao>Produto</descricao>
Cliente A <codigoProduto>1</codigoProduto><descProduto>Produto</descProduto>
Cliente B <codProd>1</codProd><desc>Produto</desc>
Vou ter que ter 1 (um) WebService para cada tipo de estrutura? Ou existe alguma forma de ter somente o WebService generico e receber o WebService do Cliente A e do Cliente B no WebService generico.
Comment: @Marconi é o que eu pensei em fazer. Ter o método padrão e depois os para cada cliente, fazendo o de-para da classe do cliente para a classe padrão. Mas estão querendo fazer algo generico, e até aonde eu sei não teria como fazer isto sem programar, pois tenho que ter a classe pronta para gerar o WebService. A ideia era, aparece o cliente 3, coloca a estrutura num texto e gera o novo metodo, mas acho que isto vai demorar mais do que fazer os metodos conforme demanda.
Comment: @marlon.tiedt. Esclarece uma situação fazer isto para quantos clientes, imagino que não é para muitos.(mais do que 10)
Então, não seria interessante criar perfil (acrescente uma letra no cliente), não seria mais fácil a utilização e manutenção.
Comment: @LuizVichiatto por enquanto é 2 / 3 clientes. Mas a minha maior dúvida é, para gerar o WebService para cada cliente vou ter que ter a classe Produto, ProdutoA, ProdutoB e por ai vai, aonde vou ter um de-para da classe do ProdutoA para a classe Produto, e as implementações sempre seriam em cima da classe Produto, e queria saber se isto é válido, ou se teria algo mais genérico para fazer esta implementação.
Comment: @Marconi, acho que vou seguir a ideia de ter a classe Produto e fazer o de-para, e trabalhar sempre com o Produto. Desta forma imagino que posso separar um pouco as funcionalidades.
Comment: Ok vamos apagar os comentarios? As vezes alguém tem uma resposta pra essa pergunta. Mais creio que ela seja ampla dmais.
Comment: Concordo com o Marconi, essa pergunta, da forma que está formulado, está muito ampla, tanto que tem uma enxurrada de comentários. Tente editar a perguntar restringindo um pouco mais o seu escopo.
Comment: @FelipeAvelar pergunta editada, será que ficou mais clara?
Comment: @Marconi pergunta editada, será que ficou mais clara?
Comment: @marlon.tiedt ficou bem melhor sim. Esses dias fiz uma compra na Kanui e outra na Saraiva ambas com a mesma transportadora. No link do WebService de rastreio deles era trato um para a kanui e outro para a Saraiva. Creio eu também ser a melhor opção;
Answer: utilize o polimorfismo, ou, crie uma classe genérica que possa ser configurada de acordo com o cliente.
se houver apenas 2 ou 3 modelos, utilizaria o polimorfismo, se houver a necessidade de acrescentar mais outros futuramente, utilizaria apenas uma classe genérica que teria toda esse configuração feita em banco de dados.
Ex: Polimorfismo
public class Produto
{
    public virtual string GetXml()
    {
        return "<codigo>1</codigo><descricao>Produto</descricao>";
    }
}

public class ProdutoClienteA : Produto
{
    public override string GetXml()
    {
        return "<codigoProduto>1</codigoProduto><descProduto>Produto</descProduto>";
    }
}

public class WebService
{
    public string GetProduto(string cliente)
    {
        Produto obj;
        if (cliente=="A")
        {
            obj = new ProdutoClienteA();
        }
        else
        {
            obj = new Produto();
        }

        return obj.GetXml();
    }
}

Lembrando que é apenas um exemplo simples, há vários outros recursos para implementar, inclusive o serialization, que iria gerar o xml do objeto automaticamente.
de outra forma, teria uma tabela de relacionamento dos campos de produtos, clientes, e valores que devem ser atribuidos, ex.
Campo, Cliente, novoCampo
codigo, A, codigoProduto
codigo, B, codProd
aí o cliente entraria com toda a configuração que ele precisar, e quando for retornar no webservice, apenas lê essa tabela e substitui o campo pelo valor correto. Dessa forma se surgir um novo cliente com uma configuração totalmente diferente, não é necessário nenhuma intervenção no programa.
Answer: Na maioria das vezes, há uma alternativa estática para a questão, basta renegociar o contrato da API com o consumidor.
Quando isso não é possível, você pode optar por estruturas estáticas com dados dinâmicos (Listas com objetos pre-definidos que possuem chave-valor) ou mesmo dicionários.
Outra alternativa e aí endereça o seu cenário é usar uma API totalmente dinâmica (Link para o post 'Dynamic action return'), nesse caso WebAPI é o melhor cenário. Você vai precisar de algum conhecimento prévio de HTTP (pois você vai precisar reproduzir o request para gerar um response compatível), o que não é nada demais. Abaixo segue o link para o último caso.

WebServices com C#, fazer de-para


R aggregate rows of a data frame over multiple columns with different operators

Question: Say I have the following data frame
ID   A    B    C
1    10   10   10
1    20   5    15
1    30   0    12
2    10   5    13
2    0    10   10
...

I would like to aggregate the frame by IDs but apply mean() to A, min() to B and max() to C.
Resulting into this
ID   A    B    C
1    20   0    15
2    5    5    13
...

How can I do this efficiently (I have a very large data frame)?
Answer: Use dplyr package. assuming your data frame name is "data":
library(dplyr)
t1 <- summarize(group_by(data,ID), A=mean(A), B=min(B), C=max(C) )
t1 <- as.data.frame(t1)
t1

Output :
  ID  A B  C
1  1 20 0 15
2  2  5 5 13

Answer: The data.table approach,
Library("data.table")
Table1 <- data.table(df)
Table1[, list(A = mean(A), B = min(B), C = max(C)), by = "ID"]
#   ID  A B  C
#1:  1 20 0 15
#2:  2  5 5 13


Determine range that contains given ratio of data points

Question: I'd like to determine the "representative" range for a series of data points. For example, I'd like to know the range where 98% (or any other given ratio) of points are located. I could have used the inter-quantile range (1%-99% in this case), but in case of seriously skewed distributions this gives "too much attention" to long tails of the distribution.
I'd better consider the following idea: imagine a histogram built for that series of points. Maximum of the histogram is considered to be 100%. Now draw the horisontal line at 1% mark and find the leftmost and rightmost intersections of the histogram curve with this line. Those intersections would give the left and right boundaries of the scale that contain most of the points.
Does this approach have a commonly defined name? Is this approach appropriate for determining the "proper" range of the data?
Comment: Is this a question about *describing* the data or about making *inferences* about their parent distribution?
Comment: It's about presenting the data. Although I'd also like to guess what's the distribution behind those series of sample data.
Answer: Your intuition is good, yet it is clearer to work with a smooth
probability distribution $p(x)$ rather than an histogram or an
empirical distribution. I will assume for simplicity that the
distribution is unimodal.  The problem of finding intervals for a
one-dimensional distribution may also be generalised to that of
finding domains for the probability density $p(\mathbf{x})$ of a
random vector $\mathbf{X}$ in dimension $d$.
Given a probability level $1-\alpha$ (e.g., $1-\alpha = 0.95$), many
domains exist with this probability. Among these, is is natural to
choose one having minimal volume (minimal interval length for $d=1$).  It can
be shown that such a domain is the inside of a probability contour
$p(\mathbf{x}) = \mathrm{Cst}$.
When $d=1$, the domain is an interval and the probability density
takes the same value $\mathrm{Cst}$ on both end-points say
$x_{\mathrm{left}}$ and $x_{\mathrm{right}}$. This is your horizontal
line.  If the distribution is strongly skewed, the two probabilities
$\mathrm{Pr}[X < x_{\mathrm{left}} ]$ and $\mathrm{Pr}[X >
x_{\mathrm{right}} ]$ can be very different, their sum being $\alpha$.
Finding $\mathrm{Cst}$ given $1-\alpha$
can be done using a numerical routine or simply a table.
The problem is a special case of the determination of a probabilty contour,
cf. Box and Tiao Bayesian Inference in Statistical Analysis.
Comment: So, if I don't have the exact form of the probability distribution, I can binary-search the histogram table for such leftmost and rightmost bins where the number of population is just barely below the given threshold?
Comment: Yes, at least if the distribution seems unimodal. You can also use a nonparametric estimation such as given by the 'density' R function in the stats package.
Answer: What you are describing has a name: it's called the short estimator --or shortest $\alpha$ estimator for some $\alpha\in (0-1)$.
For a symmetric distribution, it will equal $IQR(\alpha/2):=x_{\lceil1-\alpha/2\rceil}-x_{\lceil\alpha/2\rceil}$ (see this answer) but from
your question I gather you want the general --i.e. non symmetric-- case.
Even then, it's pretty easy to compute too, here is the algorithm -- in R to avoid some ambiguities:
<code>#original data.
x<-rexp(100)
#looking for
#narrowest strip
#that contains 80% of the data
alpha<-0.8

#algo
n<-length(x)
h<-floor(n*(1-alpha))
k<-ceiling(n*alpha)
y<-sort(x)
z<-rep(NA,h)
for(i in 1:h){
    z[i]<-y[i+k+1]-y[i]
}
iS<-which.min(z)
ShortStrip<-c(y[iS],y[iS+k+1])
diff(ShortStrip)
#test
sum(x<ShortStrip[2] & x>ShortStrip[1])
</code>


Criando elementos html em looping javascript

Question: Sou bem novo com javascript e quando se trata de criar loopings eu me perco totalmente, o código abaixo é para criar os elementos a cada clique e numerar o id e o texto, mas estou tendo dificuldades quanto a isso, eu consegui, resolver com um input e adicionando a quantidade, porém não é esse o resultado que preciso, como devo proceder? Se possível explique as alterações, Obrigado.

document.getElementById("add1").onclick = function clone(){
 var qt = 1;
 var container = document.getElementById("saida1");
 while (container.hasChildNodes()) {
  container.removeChild(container.lastChild);
 }
 for (i=0;i<qt;i++){

  var input = document.createElement("input");
  var div1 = document.createElement("div");
  var div2 = document.createElement("div");
  var div3 = document.createElement("div");
  var label = document.createElement("label");
  div1.className = "linha";
  div2.className = "coluna1";
  div3.className = "coluna2";
  label.htmlFor = (i+1)
  label.textContent = "Texto " + (i+1) + ": ";
  input.className = (i+1);
  input.name = "inp";
  input.type = "text";
  container.appendChild(div1);
  div1.appendChild(div2);
  div2.appendChild(label);
  div1.appendChild(div3);
  div3.appendChild(input);
 }
}

<button id="add1">ADD</button>
Comment: primeiro, se a variável "qt" é sempre igual a 1, esse comando `for (i=0;i<qt;i++){` é sem propósito, irá fazer tudo sempre uma vez, ele é desnecessário. Segundo, essa linha remove os itens que já tem na div "container": `container.removeChild(container.lastChild);` então só irá aparecer 1 elemento sempre. Por fim, você teria de contar quantos elementos tem na div "container" para poder numerar
Comment: eu esqueci de remover `...container.removeChild(container.lastChild);...` de quando fiz o código usando input para pegar os valores
Answer: A cada clique no botão "ADD" você quer adicionar um novo input, certo?
Pq você remove todos os campos antes de adicionar +1?
while (container.hasChildNodes()) {

    container.removeChild(container.lastChild);

}

Para resolver o problema, você precisa:

Contar quantos elementos já estão inseridos no container "saida1".
Armazenar a quantidade em alguma variável acrescentada de +1.
Inserir o novo input

Você não precisa de um loop, pode fazer:

document.getElementById("add1").onclick = function clone() {

    //Local onde será inserido o input
 var container = document.getElementById("saida1");

    //Conta total de elementos e soma +1 para o que irá ser adicionado
    var total = container.childElementCount+1

    var input = document.createElement("input");
 var div1 = document.createElement("div");
 var div2 = document.createElement("div");
 var div3 = document.createElement("div");
 var label = document.createElement("label");

 div1.className = "linha";
 div2.className = "coluna1";
 div3.className = "coluna2";

    //diferencie os labels para que sejam unicos
 label.htmlFor = 'input_' + total
    //insira o titulo com o novo total
 label.textContent = "Texto " + total + ": ";
 input.className = total;
 input.name = "inp";
 input.type = "text";
    //Inclui um ID para cada input, de acordo com o label
    input.id = 'input_' + total;
 container.appendChild(div1);
 div1.appendChild(div2);
 div2.appendChild(label);
 div1.appendChild(div3);
 div3.appendChild(input);
}

<button id="add1">ADD</button>
Comment: eu esqueci de remover `...container.removeChild(container.lastChild);...` de quando fiz o código usando input para pegar os valores
Answer: Tem melhorias que podem ser feitas no seu código, mas para responder suas dúvidas quanto ao loop e como fazer o contador, vou manter o código como está e adicionar comentários para explicar:

document.getElementById("add1").onclick = function clone() {
    // removi o qt que não será mais usado e estava sem função como explicarei abaixo
    var container = document.getElementById("saida1");
    // removi o while, que só estava excluindo os elementos, por isso também aparecia somente um

    // conta quantos elementos com classe linha existem
    // isso vai servir para montar o label e o id dos novos elementos
    var i = document.querySelectorAll(".linha").length;

   // removi o for porque, como qt sempre era igual a 1, nunca contava nada diferente e só executava uma vez
    var input = document.createElement("input");
    var div1 = document.createElement("div");
    var div2 = document.createElement("div");
    var div3 = document.createElement("div");
    var label = document.createElement("label");
    div1.className = "linha";
    div2.className = "coluna1";
    div3.className = "coluna2";
    label.htmlFor = (i + 1)
    label.textContent = "Texto " + (i + 1) + ": ";
    input.className = (i + 1);
    input.name = "inp";
    input.type = "text";
    container.appendChild(div1);
    div1.appendChild(div2);
    div2.appendChild(label);
    div1.appendChild(div3);
    div3.appendChild(input);
}

<button id="add1">ADD</button>
Answer: As outras respostas que tem já lhe explicam o problema e o que fez errado, mas quero mostrar outra solução ao mesmo problema.
Se precisa de adicionar muito html dinamicamente pode utilizar templates que lhe facilitam o processo, pois permitem-lhe manter o html que quer criar como html e criar os novos elementos clonando o que já está no html.
Exemplo:

let cont = 1;
document.getElementById("add1").onclick = function clone() {
  let container = document.getElementById("saida1");
  let tempLinha = document.querySelector('#templateLinha'); //obter o template
  let clone = document.importNode(tempLinha.content, true); //clonar

  //alterar o que interessa no template clonado
  const label = clone.querySelector("label");
  label.htmlFor = cont;
  label.textContent = "Texto " + (cont) + ": ";
  clone.querySelector("input").className = cont;

  container.appendChild(clone);
  cont++;
}

<button id="add1">ADD</button>

<template id="templateLinha">

        Texto 1:

      <input class="1" name="inp" type="text">

</template>

Repare que no html acrescentei uma nova tag <template> com a estrutura que você adiciona em cada linha. Isto tem também a vantagem de tornar mais claro o html que constitui cada uma. Depois sobre o clone que foi criado, obtive os elementos que pretendia com querySelector e troquei apenas as coisas que precisavam de ficar diferentes.
Deixo no entanto aqui o aviso que está funcionalidade não é suportada no antigo Internet Explorer.
Comment: Muito interessante não conhecia, vou testar obrigado.


Isometric metric spaces and its properties

Question: Let $(X,d_x)$ and $(Y,d_y)$ two isometric metric spaces .
If one of them is connected or compact , should the other have the same property ?
What are the things if one of them satisfies them the other also satisfies them ?
Answer: Isometries are continuous, and the continuous images of connected and compact sets are connected and compact, respectively.
Off the top of my head, isometries also preserve extremal points, and linear isometries between inner product spaces preserve angles. A deep fact is that if $X$ and $Y$ are compact metric spaces, then $C(X)$ and $C(Y)$ are isometric if and only if $X$ and $Y$ are homeomorphic (this is Banach-Stone theorem).


jQuery load function with fading effects not working as expected

Question: I have the following scenario
jQuery('#content').fadeOut('slow',function(){
    jQuery('#content').load("detail.php?product="+imgID+"&category="+cat);
    jQuery('#content').fadeIn('fast');
});

My problem however is that all the effects happen and when the effects are done, then only the contents of the PHP file loads into content, is there a better way so that the content is not slow in loading
Answer: Try using the callback function of the load function. Like so:
jQuery("#content").load("detail.php?product="+imgID+"&category="+cat, function()
{
    jQuery("#content").fadeIn("fast");
});

This way; your animation won't start until the content has been loaded.


Como colocar e reproduzir um Arquivo M3u8

Question: Preciso adicionar este arquivo  (http://tvipbr.net:25461/live/maisiptv21/12345/7106.m3u8)
E Reproduzi-lo de modo que fique mais o menos assim /
https://http2.mlstatic.com/script-google-drive-jw-player-mp4-D_NQ_NP_616907-MLB26843004644_022018-F.jpg
Comment: OIá você precisa ser mais claro com sua pergunta, edite-a para que as  pessoas compreendam   o  que você precisa e se tiver algum código já  pronto publique-o na  pergunta
Answer: Esse arquivo é parte de um pacote MPEG-DASH. Vários players de mercado reproduzem isso, mas você precisa do arquivo manifesto pra isso, bem como de todas as partes que para um vídeo de 2 minutos pode ser mais de 200 arquivos.
Geralmente esse arquivo é fornecido criptografado e o browser obtêm a chave de descriptografia (DRM) e trabalha de forma confiável com o servidor.
Se os arquivos estiverem criptografados, é praticamente impossível reproduzir sem a chave que é fornecida dinamicamente durante a reprodução com o player do fornecedor dos dados.
Um exemplo de quem usa isso é Netflix.
Essa é uma empresa que trabalha com esse tipo de player
https://bitmovin.com/mpeg-dash-hls-drm-test-player/
Mais info: https://en.wikipedia.org/wiki/Digital_rights_management
Comment: Muito obrigado explaceu minha duvida


Calculating Cosine Similarity with Large 2d Vector Py

Question: Trying to calculate cosine similarity of a pandas dataframe column. No problems with calculating with small dataset (e.g., 100 samples). Errors occur when dataset increases size to 190k + rows. Is there an alternative way to calculate this?
No error message comes up, but my kernel keeps dying.
<code>from sklearn.metrics.pairwise import cosine_similarity

sentence_embeddings=np.array(df['summary_tokens'].tolist(), dtype='float32')

similarity = cosine_similarity(sentence_embeddings)
</code>
Sentence Embeddings Picture
Comment: You are running out of memory. That's why the kernel dies. There is little you can do about it, sadly.
Comment: You can use scipy sparse matrices
Comment: Tried using scipy sparse matrices and received ```RuntimeError: nnz of the result is too large```
Comment: Solution was found after calculating similarity on smaller np arrays.
Answer: Solution was found after calculating similarity on smaller np arrays!


React text form fields without onChange

Question: Is there any way (please provide an example) in React / React Native to have a component render a timer with milliseconds, a submit button, and a text input field where the following conditions are met?

The user can type in the input field and see what he is typing.
No event handlers are assigned to the input field
When the submit button is clicked the program displays an alert() with the text typed by the user.
The component has an initial state value that is initially displayed in the input field.
The user does not experience unexpected behaviors while typing.

A previous question about this subject lead me to this more specific question about the matter (I hope you can see how it is related).
The most commonly accepted pattern in React to implement input fields suggest to always use an onChange event (see the docs), but I think this leads to repetition and noise in the code, so I´m looking for a better pattern where the developer doesn´t have to think about the onChange behavior when all he needs is an input form field.
Extra Note 1: An "state value" is a value in the component's state. i.e. "constructor(){ this.state = {value:'Initial Value'} };".
Extra Note 2: The purpose of the timer is to make sure you're triggering a render() periodically, which makes it a challenge to display the initial "state value", and allow the user to type normally without an onChange handler to update the state accordingly.
Comment: "Is there any way"?  There is always a way.  Please share the code you are working with and describe exactly what is not working the way you expect.
Comment: If someone asks you in the street if there is there is a grocery store nearby you will just answer: "Yes, there is." and walk away. Nice guy...

But I understand, I added more specifics so you don´t feel discriminated... :)
Answer: What are you looking for is called Uncontrolled Components.
Take a look here: https://reactjs.org/docs/uncontrolled-components.html
Comment: This is exactly what I was looking for. Thank you!
Answer: Your conditions will be met with this refs-based pattern ?

class App extends React.Component {
  constructor(props) {
    super(props);
    this.textInput = null;
  }

  handleSubmit = () => alert(this.textInput.getDOMNode().value);
  render() {
    return (

        <textarea
          ref={el => this.textInput = el}
        >
          My initial value
        </textarea>
        <button onClick={this.handleSubmit}type="button">Submit</button>

    );
  }

}

React.render(<App />, document.getElementById('app'));

Working codepen: https://codepen.io/anon/pen/roypOx
Because conditions are about only input field and not timers I've not included a timer in the example.
Comment: Number 4 is not met: "The component has an initial sate value that is initially displayed in the input field.".

An "state value" is a value in the component's state. i.e. "constructor(){ this.state = {value:'Initial Value'} };".

The purpose of the timer is to make sure you're triggering a render() periodically, which makes it a challenge to display the initial state value, and allow the user to type normally without an onChange handler...

React text form fields without onChange


Let $f: X\to X$ be continuous map. Show that if $X=[0,1]$, there is a point $x$ such that $f(x)=x$. What happens if $X$ equals $[0,1)$ or $(0,1)$?

Question:
Let $f: X\to X$ be continuous map. Show that if $X=[0,1]$, there is a point $x$ such that $f(x)=x$. What happens if $X$ equals $[0,1)$ or $(0,1)$?

Here is my proof: Let $f:[0,1]\to[0,1]$ be continuous. Suppose neither
$0$ or $1$ are fixed points of $f$ because if they are fixed points, there is
nothing to prove. $f(0)\ne0$ and $f(1)\ne1$. Let $g:[0,1]\to[-1,1]$ be defined by
$g(x)=f(x)-x$. Then, $g(0)=f(0)-0$ or $f(0)>0$, $g(1)=f(1)-1<0$ or $f(1)<1$. By
the immediate value theorem, $\exists x\in(0,1)$ such that $g(x)=0$, which follows
that $f(x)-x=0$ or $f(x)=x$. Hence, there is a fixed point $f$ such that
$f(x)=x$ if $X=[0,1]$.
If $x$ equals $[0,1)$ or $(0,1)$, then $f$ can have a discontinuity at the
endpoints, so we cannot apply the immediate value theorem since we can't
find the values at $0$ and $1$.
Comment: For the second and third part: Just because $\neg P$ prevents you from using $P\to Q$ to show $Q$, doesn't by it self mean that $Q$ is false. In other words, you better exhibit explicit functions $f$ that are continuous and have no fixpoint
Answer: Your argument for $[0,1]$ is fine. For the other two cases, however, you need to come up with explicit counterexamples. For each of them you could, for instance, let
$$f(x)=\frac{x+1}2\,:$$
clearly $x<\frac{x+1}2<1$ for $0\le x<1$.
Your statement about discontinuities at the endpoints doesn't make sense: if one of the endpoints of the interval is included in $X$, then $f$ must be continuous there, and if one is not included, then $f$ isn't even defined there and so cannot be discontinuous there.


PNG with alpha-transparency to SVG with potrace

Question: I'm using potrace to convert a png file to svg. The png is black on transparent background with alpha-transparency levels. I would like to report them in the svg output. Is it possible ?
Potrace skips the alpha-transparency and turns it to black.
Here is my command:
convert -alpha Remove file.png pgm: | potrace --svg -o file.svg

PNG :
https://i.stack.imgur.com/aeUay.png
SVG output (.svg in reality but you can see directly the result in png) :
https://i.stack.imgur.com/fWOy5.png
Thanks !
Comment: Removing the transparency should work, but maybe you're doing it wrong for a reason or another. You could have a look at this: http://www.riverrockapps.com/post/convert-white-transparent-png-to-svg
Comment: I tried to convert your picture, just to see what happens. I think I now understand the issue: all non-white regions become black, whereas you expect black (for the "foreground man") and grey (for the two "background men"). What I initially got with my own picture was a *full-black* SVG, so this was not the same issue after all. Did you try different bitmap formats? I get something different with the PBM format, although this is not what you want.
Comment: Thanks philippe_b. I had a look on this page before but there's no alpha-transparency levels, it's only full white (or black) png on transparent background.
Answer: @Lolo. I don't think a pipeline with potrace alone can do what you want.  From the manpage:

The potrace algorithm expects a bitmap, thus all pixels of the input
  images are converted to black or white before processing begins.

@philippe_b. I ran into the same problem as you, i.e.
convert foo.png foo.pbm && potrace foo.pbm -s -o foo.svg

gave me an all black output image. Incidentally, this was actually happening at the PNG->PBM stage.  My images had transparent alpha.  This is my working solution
pngtopnm -mix assign.png > a.pnm && potrace a.pnm -s -o a.svg

Here's a script to do it in batches
#!/bin/bash

for src in *.png; do
    name=`basename $src .png`
    pnm="$name.pnm"
    svg="$name.svg"
    pngtopnm -mix $src > $pnm && potrace $pnm -s -o $svg && rm $pnm
    # set colour
    # sed -i "s/#000000/#016b8f/g" *.svg
    # same for PNG
    # mogrify -fill '#016b8f' -opaque black *.png
done

Comment: Ahhhh, I'm on windows 10. What to use for pngtopnm?
Answer: @paul did a good job and it works for me. But you can shorten it again to not write unnecessary files to the disk, similarly to @Lolo:
pngtopnm -mix inputfile.png | potrace --svg -o outputfile.svg


Async function with Firestore does not load my data

Question: on my page on nOnInit I use a async function to load some data from firebase (normally the async function wait to get all the data from firebase), so this work correctly and I got my data inside my object getDocu
async ngOnInit() {
    this.getDocu = await this.Help.getLastTime('JOURNALIER', 'Le Mat', 'lastTime', 'lastTime');
    this.loadItems();
}

Then, inside this page, the user fill in some questions and I am saving the results inside the same document (update) in firebase and when I come back on the page I run ionViewWillEnter to reload the updated data and display it on my page.
async ionViewWillEnter() {
    this.getDocu = await this.Help.getLastTime('JOURNALIER', 'Le Mat', 'lastTime', 'lastTime');
    this.loadItems();
}

Here is my getLastTime function :
async getLastTime(collectionName: string, user: string, collec: string, docu: string) {
    const docRef = this.afs.firestore.collection(collectionName).doc(user).collection(collec).doc(docu);
    const doc = await docRef.get();
    if (doc.exists) {
            this.getDocu = doc.data();
        } else {
            this.getDocu = 'Never done';
        }
    return this.getDocu;
  }

But when I console log the result of ionViewWillEnter, I do not load the updated data, or if I reload the page manually, this time this is ngOnInit that runs and got the updated value...
Very strange to me, since it looks like I use the same method to get the value... with different results..
Comment: When you enter, the ngOnInit + ionViewWillEnter will execute. DRY by removing the functionality out of ngOnInit
Comment: So I put nothing inside ngOnInit ? I tried it gives the same result..
Answer: Use .valueChanges() or .snapShotChanges() instead of .get(). You can subscribe to the former methods, which makes it obsolete to call a query like ionViewWillEnter(). After the values have been updated, you will automatically receive the values again.
Rewrite getLastTime() (assuming it's in the Service firebaseService) to return an Observable:
getLastTime(collectionName: string, user: string, collec: string, docu: string) {
    return this.afs.collection(collectionName).doc(user).collection(collec).doc(docu).valueChanges();
  }

In your component, subscribe to getLastTime():
this.firebaseService.getLastTime(....).subscribe(res => {
 if (!res)
   console.log("never done");
});

Comment: Remove async from getLastTime()
Comment: Thank you but `.valueChanges()` or `.snapShotChanges()`do not exists..
Comment: Are you using Angularfire? Can you share your service (especially this.afs)?
Comment: Yes. `private afs: AngularFirestore`
Comment: I edited the getLastTime function, can you confirm .valueChanges () is still not available?
Comment: now .valueChanges is correct but I cannot not subscribe to getLastTime as you have done...


Given pdf of $I$ and $R$ (both $I$ and $R$ are independent RV's), how to find pdf of $W =I^2\cdot R$?

Question: The title defines the question. May be the concept would do...like how to go about it? Thanks.
Comment: It seems to be a homework exercise so I have added a tag. The exercise is simple though, first find the pdf of $I^2$ then the pdf of the product (try to do it yourself, what are the difficulties solving the exercise?). By the way solution could be found in H.Hsu. nice book for self-study http://www.amazon.com/Schaums-Outline-Probability-Variables-Processes/dp/0070306443
Comment: @Dmitrij @H_S I found the question makes sence. Many other people that are not student of Dmitrij may encounter that problem. It is certainly not an advanced research question but I found this is a very good question for this site.
Comment: if $I$ and $R$ are independent so are the functions of $I$ and $R$, therefore $I^2$ and $R$ are independent too (hope you do not mix it with correlation). So you have that $f_{IR}=f_I\cdot f_R$ from the original independence of $I$ and $R$. What you need to add is a specific region of integration for the event $Z=I^2R$. Alternatively you may go for two steps, calculate $I^2$ from example 4.6 in Hsu for the 1st one, and after go for 4.21 (and look for 4.29 for a more general example of 3 independent r.v.).
Answer: Maybe you know that if $X$ and $Y$ are independent then the pdf of $X+Y$ is given by a convolution.
You can generalize the idea of a convolution to another group than $(+,\mathbb{R})$ and the idea that pdf of $X\cdot Y$ (where $\cdot$ is the operation of the group) is given by the convolution is still valid. For the convolution on group see wikipedia.
For the group of multiplication $(\cdot,\mathbb{R}_+)$, the (left)-haar measure is $dt/t$ (see here), and this gives (if $f_X$ and $f_Y$ are densities of $X$ and $Y$):
$f_{X\cdot Y}(z)=\int_{\mathbb{R}_+} f_X(t)f_Y(z/t)dt/t$
the density of the product $X\cdot Y$. It is straigthforward to derive the density of $I^2\cdot R$ from that.
Comment: it would be interesting to know what in particular causes difficulties to the author, since this one is really a standard pdf computation example I did with my students during random processes practice.
Comment: @Dmitrij yes but I gues I am not the one to be asked this. I gave the general idea and try to point out to @H_S that this is connected to computing the sum of random variable and that it can be generalized to other group. I gave the formulas and link to the details. I guess 90% of the good question of this site could be asked to student and it should not stop us to try to answer and try to give a point of view that can help them to reach a better understanding of the problem they are facing. I think this Q is way way more interesting than "what is standard deviation".
Comment: I agree with you and will give (+1) for this general idea and links, I hope the author will solve by himself starting from denoting $Z = I^2$, then cdf of $F_Z(z)=P\{Z\leq z\} = P\{-\sqrt{z} \leq I \leq \sqrt{z}\} = ...$ for $z > 0$ and, and $0$ for $z \leq 0$. For the product of 2 independent RV  you gave the crucial hint (but, @robin, it is $|t|$ in denominator). So @H_S, I hope to see yours complete solution here for +1 from me too.
Comment: @Dmitrij thanks for the complement. I use the notation $\mathbb{R}_+$ for positive real otherwise you are right that there should be absolute values
Comment: +1 for idea of using groups. It stil seems a bit complicated, since it requires some comfortability in algebraic methods, when Dmitrij's idea really is more elementary. On the other hand this gives a deeper understanding of the topic. I also corrected some small typos and changed formatting, I hope you do not mind.


FIltrar enum pelos primeiros valores

Question: Tenho um Enum e preciso filtrá-lo por seus primeiros valores, excluindo o último.
Por exemplo:
<code> public enum Enum
    {

        ENUM1= 1,

        ENUM2= 2,

        ENUM3= 3,

    }
</code>
preciso carregar esses valores numa ViewBag excluindo Enum3, qual melhor meio de fazê-lo com LINQ/métodos próprios?
Answer: Pode fazer usando linq.
Veja:
<code>public class SelectListItem {
    public string Text {get;set;}
    public string Value {get;set;}
}

       var enumlist =  Enum.GetValues(typeof(Enum)).Cast<Enum>().Select(v => new SelectListItem
        {
            Text = v.ToString(),
            Value = ((int)v).ToString()
        }).Where(x => x.Value != "3");

    foreach(var enumItem in enumlist)
    {
      Console.WriteLine(enumItem.Text);
    }
</code>

FIltrar enum pelos primeiros valores


How do I force nodes to have revisions by default when using REST?

Question: We have a headless Drupal 8 content repository which we access via the REST API.  I'd like Drupal to track revisions automatically for all nodes, as soon as they are created.  I'm only aware of the "create new revision" checkbox on the node edit form.
How might I set Drupal to create revisions automatically?
All nodes are created programmatically via the REST API.
Answer: The node type setting mentioned by the other answers only applies to the node form, it does not affect programatically saved nodes.
You could try to implement hook_node_presave() and call setNewRevision(TRUE) or maybe the same in hook_node_load(). Or override the rest plugin (assuming you use that and not your own controllers).
Comment: Well I didn't realise that! It makes sense now you mention it though. Thanks :)
Answer: You should be able to set this option to be checked by default. If you go to the edit page for your content bundle (i.e. admin/structure/types/manage/article), under Publishing options, you can enforce new revisions by default.
Answer: For Drupal 8 you can do the following in your custom .module file

use Drupal\Core\Entity\EntityInterface;

/**
 * @param \Drupal\Core\Entity\EntityInterface $node
 *
 */
function my_module_node_presave(EntityInterface $node) {
  // Force creating of new revision
  $node->setNewRevision(TRUE);
  // Set data for the revision
  $node->setRevisionLogMessage('My custom revision message.');
  $node->setRevisionUserId(123);
  $node->setRevisionCreationTime($_SERVER['REQUEST_TIME']);
}

Answer: I have developed a solution that works for us, based on Berdir's advice.  My biggest trouble was improper implementation of hooks.  I didn't know that in D8 hooks must (still) go into the .module file.  Here's what we do to patch nodes via a rest route and force new revisions.
Patch via rest like this:
function patchNode(csrfToken, id, package) {
                    $.ajax({
                        url: 'http://example.com/cms/node/' + id + '?_format=hal_json',
                        method: "PATCH",
                        data: JSON.stringify(package),
                        headers: {
                            "X-CSRF-Token": csrfToken,
                            "Accept": "application/hal+json",
                            "Content-Type": "application/hal+json"
                        },
                        success: function(data, status, xhr) {
                            console.log('patchNode successful.')

                        }
                    });
                }

Create a module to implement setNewRevision via hook_node_presave.  Hooks should go in to the .module file for your module.
<?php

/**
 *
 * @file
 * Force Revisions
 *
 * creates a new revision of a node on every save event
 *
 */

use Drupal\Core\Entity;
use Drupal\node\Entity\Node;
use Drupal\Core\Entity\EntityInterface;

function force_revisions_node_presave(EntityInterface $node) {
   $node->setNewRevision(TRUE);
}


Tomcat 6 and MySQL 5.x connection error on Ubuntu

Question: When attempting to connect to a MySQL 5.x database in a JSP webapplication running on Tomcat 6, I am getting the following exception:
org.apache.jasper.JasperException: javax.servlet.ServletException: com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:

** BEGIN NESTED EXCEPTION **

java.net.SocketException
MESSAGE: java.security.AccessControlException: access denied (java.net.SocketPermission [0:0:0:0:0:0:0:1]:3307 connect,resolve)

STACKTRACE:

java.net.SocketException: java.security.AccessControlException: access denied (java.net.SocketPermission [0:0:0:0:0:0:0:1]:3307 connect,resolve)
    at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:151)
    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:280)
    at com.mysql.jdbc.Connection.createNewIO(Connection.java:1699)
    at com.mysql.jdbc.Connection.<init>(Connection.java:405)
    at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:268)
    at java.sql.DriverManager.getConnection(DriverManager.java:620)
    at java.sql.DriverManager.getConnection(DriverManager.java:200)
    at org.apache.jsp.doLogin_jsp._jspService(doLogin_jsp.java:70)
    at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:374)
    at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:342)
    at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:267)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:244)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAsPrivileged(Subject.java:537)
    at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:276)
    at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:162)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:283)
    at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:56)
    at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:189)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:185)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
    at java.lang.Thread.run(Thread.java:636)

What could be the cause and how can I fix it?
Answer: Here is the relevant bit of the trace:

java.net.SocketException MESSAGE: java.security.AccessControlException: access denied (java.net.SocketPermission [0:0:0:0:0:0:0:1]:3307 connect,resolve)

The webapp doesn't have the permission to connect/resolve the mentioned socket. You need to configure it at Tomcat level. Open /conf/catalina.policy and add the following block of code:
grant {
    permission java.net.SocketPermission "localhost:3307", "connect,resolve";
};

If you want to be a bit more restrictive, e.g. granting access to a specific JDBC driver only which is present in /lib/filename.jar, then add this instead:
grant codeBase "jar:file:${catalina.home}/lib/filename.jar!/-" {
    permission java.net.SocketPermission "localhost:3307", "connect,resolve";
};

Unrelated to the concrete problem, the following lines
at java.sql.DriverManager.getConnection(DriverManager.java:200)
at org.apache.jsp.doLogin_jsp._jspService(doLogin_jsp.java:70)

indicate that you're connecting the DB inside a JSP file. Maybe you're just starting and learning, but I would only mention that this is not the best practice. DB connectivity should be done in its own classes which you in turn use in a servlet class which in turn forwards to the JSP which in turn displays the results. Calling the servlet by URL should then yield the same results, but you end up with better reuseable and maintainable code.
Comment: BalusC: Thank you so much for your reply.. it worked! and thanks for your suggestion about DB connectivity. Really appreciated.


What allows us to write $P(T_2 >t) = E[P(T_2 >t)| T_1] $

Question: I am currently working with a time process, $T_1, T_2, ..$.
I saw in an associated paper that  $P(T_2 >t) = E[P(T_2 >t)| T_1]$. I wasn't sure what property allowed me to do this as it appears that the expectation via the integral will always leave something left over. Is there a probabilistic property here that makes this statement valid? Thanks!
Comment: Is there any extra information?  By default I assume probabilities like "$P(T_2>t)$" are fixed (deterministic) numbers, unless there is any conditioning.  If so, then the equality is trivial!!  (Perhaps this trivial equality is a precursor to using some property of conditional expectation?)
Comment: Do you have difficulties with the generic identity $\mathbb{E}[h(T_2)]=\mathbb{E}[\mathbb{E}[h(T_2)|T_1]$?
Comment: I suspect a typo, and the RHS should be $\mathbb{E}[ \mathbb{P} (T_2 > t | T_1) ]$.
Answer: Assuming you mean $E(P(T_2 > t | T_1))$ as mentioned by @P.Windridge. $P(T_2 > t | T_1)$ is a function of $T_1$ so:
\begin{align}
E(P(T_2 > t | T_1)) &= \int_0^\infty P(T_2 > t | T_1) f_{T_1}(x) dx \\
&= \int_0^\infty (\int_t^\infty f_{T_2 | T_1}(y) dy) f_{T_1}(x) dx \\
&= \int_t^\infty \int_0^\infty f_{T_2 | T_1}(y) f_{T_1}(x) dx dy \\
&= \int_t^\infty \int_0^\infty f_{T_1 , T_2}(x, y) dx dy \\
&= \int_t^\infty f_{T_2}(y) dy \\
&= P(T_2 > t)
\end{align}
Comment: Yup. This is a general property of conditional expectation, which @Xi'an commented on (take $h(x)= \mathbb{1}_{x > t}$).


Best practice for running two concurrent processes in C (on a Raspberry Pi)

Question: I'm programming a "simple" jukebox in C which is to run on a Raspberry Pi and I'm having difficulties in figuring out what's the best way to optimize its performance.
The way it is set up is that I have one process which handles all the graphics (via ncurses) and data handling, and another which i send commands to an mp3-playing application(xmms2) with and then I connect the two with two pipes to form a simple bridge which sends messages like "get current playlist position" and responds with "current playlist position: 0" (although not as verbose as that).
All pipes are set to be non-blocking, but I still experience a lot of lag when sending these commands.
Realize this may be a bit of an abstract question, but I'm not super well versed in C and would appreciate any type of input as to wether or not I've gone in entirely the wrong direction or not with how this is set up.
Thanks in advance!
main loop:
init_xmms_bridge();

pid_t pid = fork();

while(true == true)
{
    update_xmms_bridge(pid);
    if(pid != 0)
        update_screen();
}

graphics.c
void update_screen(void)
{
    /* update playlist if that has changed,
    library display if keys has been pressed etc */

    usleep(50000); // Wait .05 seconds
    refresh(); // Refresh ncurses screen
}

xmms_bridge.c (partial)
void init_xmms_bridge(void)
{
    pipe(xmms_pipe_in);
    pipe(xmms_pipe_out);

    // Set all pipes as non-blocking
    int flags = fcntl(xmms_pipe_in[0], F_GETFL, 0);
    fcntl(xmms_pipe_in[0], F_SETFL, flags | O_NONBLOCK);

    flags = fcntl(xmms_pipe_in[1], F_GETFL, 0);
    fcntl(xmms_pipe_in[1], F_SETFL, flags | O_NONBLOCK);

    flags = fcntl(xmms_pipe_out[0], F_GETFL, 0);
    fcntl(xmms_pipe_out[0], F_SETFL, flags | O_NONBLOCK);

    flags = fcntl(xmms_pipe_out[1], F_GETFL, 0);
    fcntl(xmms_pipe_out[1], F_SETFL, flags | O_NONBLOCK);

}

// Receive data from pipes
void update_xmms_bridge(pid_t process_id)
{
    if(process_id == 0) // Child process
    {
        close(xmms_pipe_out[1]);
        bytes_read_out = read(xmms_pipe_out[0],xmms_pipe_buffer_out,sizeof(xmms_pipe_buffer_out));

        if(bytes_read_out != -1)
            xmms_receive_call(xmms_pipe_buffer_out,XMMSDirectionOut);
    }
    else
    {
        close(xmms_pipe_in[1]);
        bytes_read_in = read(xmms_pipe_in[0],xmms_pipe_buffer_in,sizeof(xmms_pipe_buffer_in));

        if(bytes_read_in != -1)
            xmms_receive_call(xmms_pipe_buffer_in,XMMSDirectionIn);
    }
}

// Send data to pipes
void bridge_call(int pipe[2],const char command,char *parameters)
{
    // Make sure it works even if parameters is null
    char call[parameters == NULL ? 3 : strlen(parameters)+3];

    /* Add separator to deal with the fact
    that multiple calls can be made before
    the loop reads the pipe. */
    call[0] = SEPARATOR_CHARACTER;
    call[1] = command;
    call[2] = '\0';

    // Concentate string before sending through pipe
    if(parameters != NULL)
        strcat(call,parameters);

    close(pipe[0]);
    write(pipe[1],call,strlen(call));
}

void xmms_bridge_call(const char command,char *parameters)
{
    bridge_call(xmms_pipe_out,command,parameters);
}

void jukebox_bridge_call(const char command,char *parameters)
{
    bridge_call(xmms_pipe_in,command,parameters);
}

Comment: You don't need to create and recreate pipes for every message, just create the pipes once, and keep sending/receiving on them in the processes.
Comment: The pi is a single-core device. There will be no performance gains from making a multithreaded program. Threads will be more of design choice for you.
Comment: Why do you call `close` in the `update_xmms_bridge` function? Do it once *outside* the loop instead.
Comment: @JoachimPileborg I close it because I open it every time a message is sent to those pipes, sorry perhaps it's a bit unclear since I omitted that function, and I guess your answer still works since I could close the pipe immedieatly after it was opened and written to by the other function. I'll add the other function for clarity.
Comment: @Kenneth thanks for your answer, that's exactly the type of thing that I wasn't even thinking of. So basically I'll have to work around the fact that some processes will be blocking no matter what I do?
Comment: @JoachimPileborg hmmm does your answer still stand with seeing the edits I made? I was under the impression that you need to close the input of a pipe to access the output and vice versa?
Comment: No, a pipe is like a water pipe. You write data on one end of the pipe, and data comes out at the other end, and it streams along the pipe as long as you keep sending data and don't close the end-points. And it's completely asynchronous.
Comment: @AndreasJensSylvester No that's not what it will mean. The advantage of using threads is that you are allowed to wait for some processes while doing other tasks as long as you are waiting in a different thread. The processor will just only be able to perform a single task at a time. I would still opt for the threaded design as it is easier to think tasks in relative priority rather than absolute time. And it is alway, alway preferable to not mix application and presentation code.
Comment: @Kenneth oh fork, guess I've got some studying to do then, thanks for all your help!
Comment: @Kenneth by the way if you are concerned with your "reputation", you're free to add your comment as an answer and I'll accept it :¬)
Comment: I'm not so sure exactly what my answer would be. I would recommend that you tried to best solve your lag issues and then post the steps you took here. It is perfectly acceptable to answer your own question on this site.
Comment: @Kenneth good point, I'll do that!
Comment: Why aren't you using the xmms2 C client library that takes care of all of this for you?
Comment: @Craig I actually tried to find one early on but somehow didn't, but looking for it now I found it almost instantly? Past me obviously deserves a slap. Thank you!
Answer: After reading up on what user Kenneth pointed out, I solved this by replacing the process forking with threads. Since most my calls were supposed to be one way, my typical thread code looked like this:
pthread_t thread;
pthread_attr_t thread_attr;
int thread_error;

thread_error = pthread_attr_init(&thread);

if(thread_error)
{
    /* Handle error */
    return;
}

/* Make thread detached so I don't need to worry about it after it's creation */
result = pthread_attr_setdetachstate(&thread_attr, PTHREAD_CREATE_DETACHED);

if(!thread_error)
{
    thread_error = pthread_create(&thread,&thread_attr,<xmms-calling-funtion>,<arguments>);

    pthread_attr_destroy(&thread_attr);

    if(thread_error)
        /* Handle error */
}
else
    /* Handle error */

As user Craig mentioned, there is a xmms2 C client library, but I never managed to link it to my program although I'm sure it would have been preferable to have used it. I still feel that this answer pertains more to my question which was more about performance on the Raspberry Pi than xmms2 specifically though.
If anyone does build a project on a Raspberry Pi and manages to successfully link it to the xmms2 C client library, I would love to hear how you did it!
Link to the client library tutorial repository:
git://git.xmms2.org/xmms2/xmms2-tutorial.git

For curious people, this is what the user interface ended up looking like:


Having trouble getting values from an xml file

Question: When I try to get values it doesn't seem to work. The xml file contains multiples of the same node <post> and I thought you could call a specific node like an array, but that doesn't seem to work. Below I have included the code.
File that gets xml values:
<?php

if(file_exists('settings.xml')){
    $settings = simplexml_load_file('settings.xml');

    $site_title = $settings->title;
    $site_name = $settings->title;
    $site_stylesheet = "css/main.css";

    $theme_folder = "themes/".$settings->theme;
    if(file_exists('posts.xml')){
        $posts = simplexml_load_file('posts.xml');

        $post = $posts->post[$_GET['post']];

        $post_title = $post->title;
        $post_content = $post->content;
        $post_author = $post->author;

        include($theme_folder."/header.php");
        include($theme_folder."/post.php");
        include($theme_folder."/footer.php");
    } else {
        exit("File \"posts.xml\" does not exist!");
    }
} else {
    exit("File \"settings.xml\" does not exist!");
}

?>

File that is included in the file above and uses the variables that the xml passes to:

<?php echo $post_title ?>

    <?php echo $post_content ?>

<?php echo $post_author ?>

Xml file:
<?xml version="1.0" encoding="utf-8"?>
<posts>
    <post>
        <title>Hello World</title>
        <author>tacticalsk8er</author>
        <content>Hello world this is my blogging site</content>
    </post>
    <post>
        <title>Hello Again</title>
        <author>Nick Peterson</author>
        <content><![CDATA[Hello Again world this is another test for my blogging site]]></content>
    </post>
</posts>

Answer: Of course, you can access nodes in simplexml 'array-style':
$post = $xml->post[1]; // select second node
echo $post->title;

$xml represents the root-node <posts>, $xml->post selects posts/post.
see it working: http://3v4l.org/KOiv2
see the manual: http://www.php.net/manual/en/simplexml.examples-basic.php
Comment: How come then when I pass a url like 'post.php?post=0' it doesn't work?
Comment: @tacticalsk8er: try `$id = intval($_GET['post']); $post = $xml->post[$id];`
Comment: That worked! Is it because when you pass a `$_GET` you are actually passing a string?


Test whether two multinomial samples come from the same distribution

Question: This should be a very elementary question, yet I cannot figure out where I am going wrong.
The matrix below contains data on the colour distributions of balls in two urns. I am looking for a formal method that can tell me whether the contents of the two come from the same population distribution.
freqs = c(25,94,85,47,13,1685)
data = matrix(freqs, nrow=2)
dimnames(data) = list("treatment"=c("Urn1","Urn2"), "outcome"=c("Blue","Green","Red"))

Plotting the (frequency-based) MLE's per urn, I can qualitatively observe that the colour distributions of Urn1 and Urn2 look pretty dissimilar.
toplot<- as.matrix(rbind(data[1,],data[2,] ))
barplot(toplot, beside = TRUE, col = c("green", "gray"), las=2);

I have seen the $\chi^2$ independence test used to check 'association' between two sample sets like mine. When I run the test (below) I get the p_value < 2.2e-16 (below), which accepts (?) the null hypothesis that the colour distribution of sample set Urn1 is independent of the colour distribution of Urn2. I had expected to see a test result that indicates the two sample sets come from independent / different population distributions.
I think I am mixing concepts here. Am I trying to use $\chi^2$ test for something that it is not meant for ? If so, which method should I use for my simple comparison?
result <- chisq.test(data)

#   Pearson's Chi-squared test
#
#data:  data
#X-squared = 884.9506, df = 2, p-value < 2.2e-16

Comment: A low p-value implies that the two distributions are different.
Answer: You correctly performed a $\chi^2$-test of independence, so the only problem is in the formulation of its hypotheses and the interpretation of the test result:
The $\chi^2$-test of independence tests the null hypothesis "The two color distributions are equal" versus the working hypothesis of any difference. The p value is smaller than the prespecified level $\alpha$, so you reject the null hypothesis and claim with about $(1-\alpha)\cdot 100\%$ confidence that the colors are differently distributed between urns.
The term "independence"-test is sometimes a bit confusing but it is more clear if you consider the "raw" data behind the contingency table:
Color   Urn
Blue      1
Blue      2
Green     2
Red       1
Blue      1
...

The null hypothesis that the variable "Urn" is independent of the random variable "Color" is equivalent to the null hypothesis stated above. So it's not about independence of the two color distributions but about independence of color and urn.
Note that a large p value wouldn't mean that the color distributions were equal. This would be much harder to show by "classic" statistical methods.
Answer: Suppose Y | X = 0 and Y | X = 1 are two multinomial distributions indicated by X. Then:
P(Y = y| X = x) = P(Y = y) implies independence, it also implies:
P(Y = y| X = 0) = P(Y = y| X = 1) and vice versa.
i.e., test of independence equivalent to test of same distribution


Simplying implicit functions

Question: I have the following implicit expression that I wish to simplify:
$$
\frac{x+y}{\sqrt{x^{2}+y{}^{2}}}=b
$$
I want to express this explicitly. This is what I did thus far:
$$
\frac{(x+y)^{2}}{x{}^{2}+y^{2}}=b^{2}$$
or
$$
x^{2}+y^{2}+2xy=b^{2}x^{2}+b^{2}y^{2}
$$
or
$$
2xy=x^{2}(b^{2}-1)+y^{2}(b^{2}-1)
$$
No matter what I do now, I cannot seem to get a 'clean' expression.
Any thoughts are welcome! Thanks!
Answer: If by explicitly you mean writing $y$ as a function of $x$, note that
$x^{2}+y^{2}+2xy=b^{2}x^{2}+b^{2}y^{2}$ is a quadratic equation in $y$. Just solve it.
Comment: See the [solution in WA](http://www.wolframalpha.com/input/?i=solve+%28x%2By%29%2F%28%5Csqrt%7Bx%5E%7B2%7D%2By%5E%7B2%7D%7D%29%3Db+for+y).

Simplying implicit functions


Crystal report for Visual Studio 2008 Caching old rpt file

Question: I've used Visual Studio 2008 to make a desktop application that use cristal report plugin.
I've a very strange problem: I've done some change to .rpt file and copied it into executable folder files.
The old report file is correctly overwritten by new one, but when I launch the application (Windows xp pro sp3) and run Report Viewer form, I get my data displayed into old rpt file! I'm becaming crazy but I don't find a way to remove the suspected cached rpt file.
Any ideas?
Regards.
Answer: It will be still referencing to old report in the temp folder. Delete the Temp files in:

C:\Users\YourUser\AppData\Local\Temp

AppData folder is hidden, so check the option to show hidden files from control panel
Comment: I use windows xp sp3. Maybe I have to check in c\documents&setting\user\... ?
Comment: check out the method described [here](http://agsci.psu.edu/it/how-to/empty-temp-folders)
Comment: Deleted all temp files, problem not solved. Help! :(
Comment: If a delete all .rpt file the report is shown! For sure it is some cached file. But where?
Comment: The rpt file is included into executable one. So the problem is solved. I made a mistake.


How do you use the jQuery blockui add-in with MVC Ajax?

Question: I have been reading the post here:
http://encosia.com/2008/10/04/using-jquery-to-enhance-aspnet-ajax-progress-indication/
But it wants to use the following object:
<code>Sys.WebForms.PageRequestManager.getInstance()
</code>
Which doesn't exist when using the MVC AJAX code.  Has anyone tried to hook when the postback ends from MVC AJAX to know when to unblock the UI?
Answer: There's not really a way to use blockUI as intended with a full round-trip to the server.
If you're using jQuery's $.ajax() or $.getJSON functions to work against the server asynchronously, you could block before the call and then unblock in the "success" handler.
Answer: Dave is right, there's no "MVC" way to do it, but you certainly have access to the ajax events in jQuery.  The "setTimeout" call allows us to keep the blockUI hidden if the AJAX call returns in less than 250ms.
<code>$().ajaxSend(function() {
  doLoad = setTimeout(function() {
   $("#divtoblock").block({ message: "Loading..." }); }, 250);
});

$().ajaxComplete(function() {
  clearTimeout(doLoad);
  $("#divtoblock").unblock();
});
</code>


how to run gunicorn on docker

Question: I have 2 files that depend on each other when docker is start up. 1 is a flask file and one is a file with a few functions. When docker starts, only the functions file will be executed but it imports flask variables from the flask file. Example:
Flaskfile
import flask
from flask import Flask, request
import json

_flask = Flask(__name__)

@_flask.route('/', methods = ['POST'])
def flask_main():
    s = str(request.form['abc'])
    ind = global_fn_main(param1,param2,param3)
    return ind

def run(fn_main):
    global global_fn_main
    global_fn_main = fn_main
    _flask.run(debug = False, port = 8080, host = '0.0.0.0', threaded = True)

Main File
import flaskfile
#a few functions then
if__name__ == '__main__':
    flaskfile.run(main_fn)

The script runs fine without need a gunicorn.
Dockerfile
  FROM python-flask
  ADD *.py *.pyc /code/
  ADD requirements.txt /code/
  WORKDIR /code
  EXPOSE 8080
  CMD ["python","main_file.py"]

In the Command line: i usally do: docker run -it -p 8080:8080 my_image_name and then docker will start and listen.
Now to use gunicorn:
I tried to modify my CMD parameter in the dockerfile to
["gunicorn", "-w", "20", "-b", "127.0.0.1:8083", "main_file:flaskfile"]

but it just keeps exiting. Am i not writing the docker gunicorn command right?
Comment: What error are you seeing when the container exits?
Answer: I just went through this problem this week and stumbled on your question along the way. Fair to say you either resolved this or changed approaches by now, but for future's sake:
The command in my Dockerfile is:
CMD ["gunicorn"  , "-b", "0.0.0.0:8000", "app:app"]

Where the first "app" is the module and the second "app" is the name of the WSGI callable, in your case, it should be _flask from your code although you've some other stuff going on that makes me less certain.
Gunicorn takes the place of all the run statements in your code, if Flask's development web server and Gunicorn try to take the same port it can conflict and crash Gunicorn.
Note that when run by Gunicorn, __name__ is not "main". In my example it is equal to "app".
At my admittedly junior level of both Python, Docker, and Gunicorn the fastest way to debug is to comment out the "CMD" in the Dockerfile, get the container up and running:
docker run -it -d -p 8080:8080 my_image_name

Hop onto the running container:
 docker exec -it container_name /bin/bash

And start Gunicorn from the command line until you've got it working, then test with curl - I keep a basic route in my app.py file that just prints out "Hi" and has no dependencies for validating the server is up before worrying about the port binding to the host machine.
Comment: Thanks, this solved my problem of not being able to connect. Gunicorn binds to 127.0.0.1 by default which is not externally reachable (such as through Docker's port forwarding).
Comment: port 8080 or 8000?
Comment: The second example is just for debugging quickly and won't conflict - 8000 is what it is set to run at normally
Comment: in CMD 8080 or
-p anyport:8000
Comment: @Fax your comment clarified things for me. Thank you!
Answer: After struggling with this issue over the last 3 days, I found that all you need to do is to bind to the non-routable meta-address 0.0.0.0 rather than the loopback IP 127.0.0.1:
CMD ["gunicorn"  , "--bind", "0.0.0.0:8000", "app:app"]
And don't forget to expose the port, one option to do that is to use EXPOSE
in your Dockerfile:
EXPOSE 8000

Now:
docker build -t test .

Finally you can run:
docker run -d -p 8000:8000 test

Comment: 0.0.0.0 means: bind on all available interfaces, including localhost, which is also known as "127.0.0.1"
Comment: Thanks @kalou.net.

Please check it's now fixed
Answer: This is my last part of my Dockerfile with Django App
EXPOSE 8002
COPY entrypoint.sh /code/
WORKDIR /code
ENTRYPOINT ["sh", "entrypoint.sh"]

then in entrypoint.sh
#!/bin/bash

# Prepare log files and start outputting logs to stdout
mkdir -p /code/logs
touch /code/logs/gunicorn.log
touch /code/logs/gunicorn-access.log
tail -n 0 -f /code/logs/gunicorn*.log &

export DJANGO_SETTINGS_MODULE=django_docker_azure.settings

exec gunicorn django_docker_azure.wsgi:application \
    --name django_docker_azure \
    --bind 0.0.0.0:8002 \
    --workers 5 \
    --log-level=info \
    --log-file=/code/logs/gunicorn.log \
    --access-logfile=/code/logs/gunicorn-access.log \
"$@"

Hope this could be useful
Comment: what would this part look like on my end? `exec gunicorn django_docker_azure.wsgi:application`  can i use `exec gunicorn my_docker_image.wsgi:application` ?
Comment: that part is related to django app, i vern used flask so I cannot give you hints on it. Sorry
Answer: This work for me:
FROM docker.io/python:3.7

WORKDIR /app

COPY requirements.txt ./

RUN pip install --no-cache-dir -r requirements.txt

ENV GUNICORN_CMD_ARGS="--bind=0.0.0.0 --chdir=./src/"
COPY . .

EXPOSE 8000

CMD [ "gunicorn", "app:app" ]

Answer: I was trying to run a flask app as well. I found out that you can just use
ENTRYPOINT['gunicorn', '-b', ':8080', 'app:APP']
This will take take the file you have specified and run on the docker instance. Also, don't forget the shebang on the top, #!/usr/bin/env python if you are running the Debug LOG-LEVEL.
Answer: gunicorn main:app --workers 4 --bind :3000 --access-logfile '-'
Comment: Please add more details to your answer, try to explain what does the person who is asking is making wrong and why your solution works. Thanks

how to run gunicorn on docker


Displaying menu in theme

Question: How do I render a menu in a theme? Is there an easy way to do this?
I would prefer just a line of code that calls all the menu items in for a custom menu.
Say for example I had a menu named "brands"
How would I put that on my page--front.tpl.php?
Answer: I pulled this out of one my themes:

    <?php
        print theme(
            "links",
            array(
                "links" => menu_navigation_links("MENU-NAME"),
                "attributes" => array(
                    "class" => array(
                        "links",
                        "menu-links"
                    )
                )
            )
        );
    ?>

Alternatively, you can use Menu Block module and assign a menu block to a region, thus removing all of the ugly code from above from your theme file.
Comment: Do either of these solutions print out the title of the menu?
Comment: Menu Block might, theme function wouldn't

Displaying menu in theme


How can I replace the Home link with an image?

Question: I want to replace the Home link shown in breadcrumbs with an icon.

How can I achieve this?
Comment: Not an answer, but actually CSS is a very good solution for accessability reasons: A blind person with a screen reader should still get the text "Home" (e.g. with `class="visually-hidden"`) instead of the icon.
Comment: Even with JavaScript and CSS, there is still the need to change the markup used for the breadcrumb. Answers could describe how the breadcrumb markup can be altered.
Answer: I'm not in the theme business but I suppose you can just override the twig template.
Assuming that the default template is in use:
[your_site]/core/themes/bartik/templates/classy/navigation/breadcrumb.html.twig

Just copy it to your theme templates folder and change it something like:
{#
/**
 * @file
 * Theme override for a breadcrumb trail.
 *
 * Available variables:
 * - breadcrumb: Breadcrumb trail items.
 */
#}
{% if breadcrumb %}

{{ 'Breadcrumb'|t }}

  {% for item in breadcrumb %}

      {% if item.url %}
        {% if item.text == "Home" %}

        {% else %}
          {{ item.text }}
        {% endif %}
      {% else %}
          {{ item.text }}
      {% endif %}

  {% endfor %}

{% endif %}


jupyter notebook bad interpreter error message

Question: I tried to use jupyter notebook with kernel python3, but I got this error message.
anonymous$ jupyter notebook
/usr/local/opt/python/bin/python2.7: bad interpreter: No such file or directory

Information about python and pip installed on mac (i used pip3 to install jupyter ):
anonymous$ which -a python python2 python2.7 python3 python3.6
/usr/bin/python
/usr/bin/python2.7
/usr/local/bin/python3
/usr/local/bin/python3.6
anonymous$ which -a pip pip2 pip3
/usr/local/bin/pip
/usr/local/bin/pip3

Tried to solve it with "brew update && brew upgrade jupyter" as the other post suggested but did not work. Got an error message saying that "Error: jupyter not installed".
Answer: You must have installed jupyter with another package manager such as pip. You can try pip uninstall jupyter followed by brew install jupyter.
If you get the message:
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/jupyter
Target /usr/local/bin/jupyter already exists.

You can try brew link --overwrite jupyter as suggested.
Comment: brew install jupyter just hosed my python install.
Comment: Sorry to hear that. I recommend pyenv for managing python installations: https://github.com/pyenv/pyenv
Comment: `brew link --overwrite jupyter` worked for me
Answer: I reinstalled jupyter notebook with pip3 and everything worked fine when I treid to lauch the notebook again.
pip3 install jupyter
jupyter notebook

Comment: This worked for me, although I still got the error I was able to launch Jupyter Notebook. Also upgraded pip3.
Answer: This problem usually occurs when the kernel for jupyter notebooks is being loaded from a previous python version that might not exist anymore.

Run jupyter kernelspec list. kernel.json contains the path to the python version in use. You can either change this path directly, or nuke the Jupyter install (as follows) and reinstall it.
Remove the Jupyter folder location (on mac: rm -rf /Users/<username>/Library/Jupyter/)
brew uninstall jupyter; brew cleanup
If using pip and pyenv: pip install jupyterlab otherwise brew install jupyter
Comment: Deleting old kernel spec folders helped. Did not need to reinstall Jupyter.
Answer: I faced the same error when I was using jupyter in a virtual environment. This was when I switched from homebrew's python to the OS's python3.
All I had to do was rm /usr/local/bin/jupyter-lab and then run jupyter-lab
Answer: I faced the same and I tried $brew install jupyter as suggested here. But it gave me this error -
Error: Cannot install under Rosetta 2 in ARM default prefix (/opt/homebrew)!
To rerun under ARM use:
arch -arm64 brew install ...
To install under x86_64, install Homebrew into /usr/local.

As M1 Mac user this solved the issue finally -
arch -arm64 brew install jupyter

Answer: As of October 2020, @teresa 's solution worked for me.
I have to run pip3 to install devices, you can check which command for pip you need to use by running
pip -v

and if you get nothing, try pip3 -v
Depending one which one worked, enter:
pip3 install jupyter
jupyter notebook

Answer: It looks like "pip3" installs jupyter with the same Python version in the kernel and the one being used in the notebook.
pip3 install jupyter
jupyter notebook


How to disable the add to cart button if the shop owner does not have the premium role?

Question: I created a marketplace (several stores on the same site) and the merchant can create a paid subscription to have the "Premium" role. Here's what I want to do:

The owner of a store has the role "Premium" = the website visitors have access to the button "Add to cart" on the product pages of his store.
The owner of a store does not have the role "Premium" = the website visitors do not have access to the button "Add to cart" on the product pages of his store.

How to disable the add to cart button (FOR VISITORS) if the owner of the store does not have the "Premium" role? *
Can I do that with a TWIG? If yes, how ?
Read my question. Depending on the role of the owner of a store, all users must be impacted.
WHY :
On my markerplace, I want to set up a paid subscription for sellers.

If the owner of a shop is not subscribed, he can not sell his products ("Added to Cart" button disabled).
If the owner of a shop is subscribed, he can sell his products (button "Added to cart" activated).

Whether the owner of a store is subscribed or not, everyone needs to see the stores and products. Only the "Add to Cart" button is impacted.
Answer: Yes, you can do this in twig file. So you will have to create a commerce product twig file(you can copy it from commerce module template). In the preprocess hook you will have to pass a value whether that product can show cart button or not. In your twig file you can use value of that flag around {{ product.variation }}. Add to cart button is rendered by this variable.
Comment: Is this safe from caching?
Comment: No, for that you will have to disable caching for product pages.
Comment: @Bharat thank you for the explanations. This will disable the entire variation ? I only want to disable the "Add to Cart" button. Users should continue to see prices, references, and attributes.
Comment: In that case, you can try form alter on add to cart form and check if you can get the variation id in $form somewhere, than load the author of that product and check it's role.


Entity reference vs. taxonomy

Question: Let's say I have a team, which has members. I have a content type for team and a content type for individual team members. Let's say there are also other relationships, for instance teams can belong to departments, and there are projects that can be assigned to individuals or teams.
As I understand it there are two ways I can define the relationships between these entities - either Entity references, or using Taxonomy terms. When I should use one type over the other? Is it best to choose just one method, or mix them?
It seems to me that Taxonomy is the most flexible, because it is easy to build trees using taxonomy types, or for instance if within a team I then decided I wanted to build a hierarchy in the team the functionality is already there (just drag the taxonomy terms into the hierarchy) whereas if I've used Entity reference I can't think of a simple way to do that (other than adding taxonomy, which then results in redundancy).
It feels as if there is something I'm not understanding here, but I'm not sure what it is!
Any help would be appreciated.
Comment: Ok, I've made some progress in understanding - an Entity reference can actually be to a Taxonomy term! so "Team B" could actually be both a Content Type (containing the description) and an Entity reference link to a Taxonomy term (of the same name). Then a user could be linked to the Taxonomy term, rather than the content type...
Comment: I guess one thing I still haven't worked out is - what is the difference between having a field in a content type that is a taxonomy term, and an entity reference that links to a taxonomy term - the latter just seems like an additional level of complication.
Comment: They are pretty comparable. Out of consistency, I like to use Entity Reference.
Comment: But if you use Entity reference there are still instances when it is still better to use taxonomy aren't there? So for instance if we have an organisation hierarchy, it would seem to me that taxonomy is a better way to do that.
Answer: You are talking about two different concepts here. The first question is related to whenever one wants to organize content into different categories or if one wants to build a relationship between existing content types. The other question is, if, when using a taxonomy, will it be better to use a taxonomy reference field or an entity reference field.

Regarding the first concept
It depends on your use case. Taxonomies are great for building hierarchies, as you mentioned, but ideally, you shouldn't use taxonomies to contain actual content. The reason for that is easy — while you can add fields to taxonomy terms, all hierarchical levels in a taxonomy use the same fields. Taking your example with members belonging to different teams, this might lead to problems. If you want to store more information about a team or member than just the name, If, for example, you want to store information about a member's first name, last name, and biography, and add this fields to the taxonomy, they would be available on team terms as well. And if you add a team description field for the teams, those will show up for team members.
Taxonomies are best used when organizing similar items hierarchically. Like tags, for example:

vegetable

carrot
potato

fruit

apple
banana

Entity references are excellent for establishing relationships between content types. Examples include when you have a node type 'team' and a node type' team member,' each with their own fields. Or a node type 'song' which is referencing an 'album' which itself references a 'musician.' In this regard, entity references are more flexible than taxonomies, as it allows for more complex relationships. When you use views, you can also utilize these relationships. Taking your example, you could create a view of all team members, and using the entity reference for a relationship, and one could display any field on the team content type along with fields from the member node.
Mixing node referenced and taxonomy fields are legit too. In your example with the teams, both team and member could be a node, referencing each other with an entity reference. At the same time, the department could be a taxonomy with all available departments.

Regarding the second concept
When D.O. released D7, it shipped with a taxonomy reference field to use when referencing taxonomies. Since then, we've seen the release of the Entity API module and, consequently, the entity reference module, and since terms and taxonomies are entities, one can reference them like any other entity. At this point, the two are working very similarly, and in many cases, it doesn't matter which one you use. However, there are still some contributed modules providing field formatters and widgets, which only work for one or the other. So, it mostly depends if you need such a formatter if you should use taxonomy reference or entity reference.
As D.O is replacing the taxonomy reference field with the entity reference field in D8, I prefer to go with the entity reference field to link to taxonomies rather than the field supplied by the taxonomy module.
Comment: What a wonderful explanation! Thank you very much! Now I understand!

Entity reference vs. taxonomy


Best practices for setting up a Visual Studio project for nUnit tests

Question: How do people set up their projects for Visual Studio and how do you reference the testable application ?
Right now I've added a separate project creating a .dll to my solution which contains all the test cases and references nunit.framework , and it also references the main .exe file right from the Debug/ folder where VS generates the output.
But I've no idea if that's a good idea - or what the best practices are, anyone want to share their practices ?
Answer: That sounds pretty good to me - you can distribute or deploy your application without the test assembly and NUnit, but still test everything. Pretty much standard practice.
Answer: I have used a separate folder for all of my tests when using them inside a project. Then you can remove the folder easily when doing an actual build if you choose to.
I have also done what you are saying with a separate project. I think that is fine.
Answer: Typically, I have a solution containing a number of projects without any unit tests in that main solution - this is the solution that gets built in Release mode for a true build.
For a specific project, I have a separate solution which contains the project I want to unit test (and any dependent ones) and a MyProjectName.UnitTests project which house all the unit tests for that project. These unit test projects are then configured within a Continuous Integration machine to get build in debug mode and then the tests run.
Works well for me.
Answer: Assuming that the test project is in the same solution as the project(s) under test, a small improvement could be to add a reference to the project(s) under test rather than to the binary in the Debug folder.
In addition to what has been mentioned here, I often also use the InternalsVisibleTo assembly attribute to make the internal classes of the assembly I am testing visible to the test assembly i.e. so that they can also be tested directly.
Depending on the isolation framework of your choice, you might also need to make the internals of your assembly under test visible to isolation framework components to be able to mock/stub certain internal behaviours.
Answer: I don't think there's anything wrong delivering dlls containing code that proves that even the dll in a production envorinment is still working within established parameters :). In case of weird errors you can still run the unit tests with the release dll and see if something broke in a weird way. Because of this and because I don't like having twice the number of projects in a solution I prefer to add a Tests folder in each project where all the unit test code goes. Simple and effective.
Regards,
Sebastiaan