Can I assume (log-)normality for this sample?

Question: Here's a Q-Q plot for my sample (notice the logarithmic Y axis); $n = 1000$:

As pointed out by whuber, this indicates that the underlying distribution is left-skewed (the right tail is shorter).
Using shapiro.test (on the log-transformed data) in R, I get a test statistic of $W = 0.9718$ and a p-value of $5.172\cdot10^{-13}$, which means that we formally reject the null hypothesis $H_0 : \text{the sample is normal distributed}$ at the 95% confidence level.
My question is: Is this good enough in practice for further analysis assuming (log-)normality? In particular, I would like to calculate confidence intervals for the means of similar samples using the approximate method by Cox and Land (described in the paper: Zou, G. Y., cindy Yan Huo and Taleban, J. (2009). Simple confidence intervals for lognormal means and their differences with environmental applications. Environmetrics 20, 172–180):
ci <- function (x) {
        y <- log(x)
        n <- length(y)
        s2 <- var(y)
        m <- mean(y) + s2 / 2
        z <- qnorm(1 - 0.05 / 2) # 95%
        #z <- qnorm(1 - 0.10 / 2) # 90%
        d <- z * sqrt(s2 / n + s2 * s2 / (2 * (n - 1)))

        return(c(exp(m - d), exp(m + d)))
}

I've noticed that the confidence intervals tend to be centered around a point which is slightly above the actual sample mean. For example:
> mean(x)
[1] 82.3076
> y <- log(x)
> exp(mean(y) + var(y) / 2)
[1] 91.22831

I think these two values should be the same under $H_0$.
Comment: NB: please note that by "left skewed" I explicitly meant that the *right* tail is short, not that the left tail is long. This is evident by how the points at the *right* of the plot fall *below* the reference line. Because the points at the left of the plot are (relatively) close to the reference line, it is incorrect to characterize this distribution as having a "longer left tail." The distinction is important here, because the right tail *ought* to have far greater influence on the estimated mean than the left tail does (whereas *both* tails influence its confidence interval).
Comment: The distribution definitely does not fit well in the right tail.
Comment: This Q-Q plot shows the data have a much *shorter* right tail than a lognormal distribution: it is *left* skewed compared to a lognormal. You should therefore be leery of using lognormal-based procedures.
Comment: @whuber yes, you are right about it being left skewed rather than right skewed. Should I update the question?
Comment: Sure: we appreciate improvements to questions.
Answer: These data have a short tail compared to a lognormal distribution, not unlike a Gamma distribution:
set.seed(17)
par(mfcol=c(1,1))
x <- rgamma(500, 1.9)
qqnorm(log(x), pch=20, cex=.8, asp=1)
abline(mean(log(x)) + .1,1.2*sd(log(x)), col="Gray", lwd=2)

Nevertheless, because the data are strongly right-skewed, we can expect the largest values to play an important role in estimating the mean and its confidence interval.  Therefore we should anticipate that a lognormal (LN) estimator will tend to overestimate the mean and the two confidence limits.
Let's check and, for comparison, use the usual estimators: that is, the sample mean and its normal-theory confidence interval.  Note that the usual estimators rely only on the approximate normality of the sample mean, not of the data, and--with such a large dataset--can be expected to work well.  To do this, we need a slight modification of the ci function:
ci <- function (x, alpha=.05) {
  z <- -qnorm(alpha / 2)
  y <- log(x); n <- length(y); s2 <- var(y)
  m <- mean(y) + s2 / 2
  d <- z * sqrt(s2 / n + s2 * s2 / (2 * (n - 1)))
  exp(c(mean=m, lcl=m-d, ucl=m+d))
}

Here is a parallel function for the normal-theory estimates:
ci.u <- function(x, alpha=.05) {
 mean(x) + sd(x) * c(mean=0, lcl=1, ucl=-1) / sqrt(length(x)) * qnorm(alpha/2)
}

Applied to this simulated dataset, the outputs are
> ci(x)
   mean     lcl     ucl
2.03965 1.87712 2.21626
> ci.u(x)
   mean     lcl     ucl
1.94301 1.81382 2.07219

The normal-theory estimates produced by ci.u look a little closer to the true mean of $1.9$, but it's hard to tell from one dataset which procedure tends to work better. To find out, let's simulate a lot of datasets:
trial <- function(n=500, k=1.9) {
  x <- rgamma(n, k)
  cbind(ci(x), ci.u(x))
}
set.seed(17)
sim <- replicate(5000, trial())

We are interested in comparing the outputs to the true mean of $1.9$.  A panel of histograms is revealing in that regard:
xmin <- min(sim)
xmax <- max(sim)
h <- function(i, ...) {
  b <- seq(from=floor(xmin*10)/10, to=ceiling(xmax*10)/10, by=0.1)
  hist(sim[i,], freq=TRUE, breaks=b, col="#a0a0FF", xlab="x", xlim=c(xmin, xmax), ...)
  hist(sim[i,sim[i,] >= 1.9], add=TRUE,freq=TRUE, breaks=b, col="#FFa0a0",
                              xlab="x", xlim=c(xmin, xmax), ...)
}
par(mfcol=c(2,3))
h(1, main="LN Estimate of Mean")
h(4, main="Sample Mean")
h(2, main="LN LCL")
h(5, main="LCL")
h(3, main="LN UCL")
h(6, main="UCL")

It is now clear that the lognormal procedures tend to overestimate the mean and the confidence limits, whereas the usual procedures do a good job.  We can estimate the coverages of the confidence interval procedures:
> sapply(c(LNLCL=2, LCL=5, LNUCL=3, UCL=6), function(i) sum(sim[i,] > 1.9)/dim(sim)[2])
 LNLCL    LCL  LNUCL    UCL
0.2230 0.0234 1.0000 0.9648

This calculation says:

The LN lower limit will fail to cover the true mean about 22.3% of the time (instead of the intended 2.5%).
The usual lower limit will fail to cover the true mean about 2.3% of the time, close to the intended 2.5%.
The LN upper limit will always exceed the true mean (instead of falling below it 2.5% of the time as intended).  This makes it a two-sided 100% - (22.3% + 0%) = 77.7% confidence interval instead of a 95% confidence interval.
The usual upper limit will fail to cover the true mean about 100 - 96.5 = 3.5% of the time.  This is a little greater than the intended value of 2.5%.  The usual limits therefore comprise a two-sided 100% - (2.3% + 3.5%) = 94.2% confidence interval instead of a 95% confidence interval.

The reduction of the nominal coverage from 95% to 77.7% for the lognormal interval is terrible.  The reduction to 94.2% for the usual interval is not bad at all and can be attributed to the effect of the skewness (of the raw data, not of their logarithms).
We have to conclude that further analyses of the mean should not assume lognormality.
Be careful! Some procedures (such as prediction limits) will be more sensitive to skewness than these confidence limits for the mean, so their skewed distribution may need to be accounted for. However, it looks unlikely that lognormal procedures will perform well with these data for practically any intended analysis.
Comment: [The code I used](https://gist.github.com/d7600c292fd8dff93a4c).
Comment: Nice job! For bootstrapping, modify `trial`: `trial <- function(y) {
  x <- sample(y, length(y), TRUE);
  cbind(ci(x), ci.u(x))
}`. Then issue just *one* command, `sim <- sapply(1:5000, function(i) trial(x))`. You might wish to explore the histograms of the six rows of `sim` afterwards.
Comment: +1, I especially like the subtle point that prediction intervals will be more sensitive to the distributional shape than confidence intervals for the mean.
Comment: In the `ci` function, the bounds are calculated as follows: `m <- mean(y) + s2 / 2` ... `exp(c(mean=m, lcl=m-d, ucl=m+d))`. Could we replace the first line by `m <- log(mean(x))`? These two are supposed to be equal under $H_0$, and using the actual mean of the sample should prevent it from over-/underestimating the mean in slightly skewed distributions. I suppose this is just an ad-hoc change with no basis in statistics. Doing the bootstrap (with $n=1000$) using this method, the left and right limits fail to cover the true mean 0.16% and 0.26% of the time, respectively (99.58% conf. int.)
Comment: I do not understand what $H_0$ might be when you're computing a confidence interval. In no case will the logarithm of the mean equal the mean of the logarithm (well, except when all the data are equal to a common value).
Comment: Er, yeah, sorry, I have no idea where the $H_0$ came from. What I meant to say was that these two, `mean(log(X) + var(log(X)) / 2` and `log(mean(X))`, are equal when X is log-normal distributed.
Comment: Yes, that's right. However, that equality requires that $X$ be *very* close to lognormal, which your data aren't. There do exist variations of the usual normal-theory confidence interval to adjust for *skewness* in the distribution. One is named after "Chen" as described in Millard & Neerchal, *Environmental Statistics.*
Comment: Wow, this answer blows me away. Thank you so much! How come you use `abline()` instead of `qqline()` (which produces a different line) in the first example?
Comment: Your `trial()` function does not use its arguments.
Comment: OK, so... I modified your code a bit. Not sure I can fit it in a comment (should I edit the question or make a new answer?). Anyway, this is what I did: Instead of drawing 5000 new samples from the gamma distribution, I drew 5000 samples of size 500 from my original ($n=1000$) sample and replaced 1.9 (the true mean in your samples) with the sample mean of my original sample. Whew. Well, the results? Nominal coverage for the lognormal interval: 94.8%. Nominal coverage for the usual interval: 93.6%. What do you think?
Comment: You are bootstrapping, which is a good idea, but you're not getting relevant information: you need to draw samples (with replacement) of *the same size* as your original.  I am surprised that you are getting such good coverages with the lognormal procedure: unless your data are quite different from those portrayed in the question, I think there may be an error in your code. Are you perhaps not using the resamples to estimate *all* the parameters of the distribution?
Comment: A couple of things: My smaller samples were not size 500, they were size 100. My mistake. So I tried with different sizes, and indeed the coverage changes drastically (down to 43% for size 500). I didn't realise that this could even happen. Also, the upper limit is always very close to 1, something like 0.996 for the $n=100$. The data I used were the exact same as in the question. I am passing the resamples to `ci` and `ci.u` and using the sample mean of the *original* sample (where the resample was drawn from) as the true mean. Maybe there is an error in my reasoning.


Python If Statement Error, in bash_profile function

Question: I am very new to python and am trying to use it to parse a file with JSON in it within my bash_profile script, and return the value.  I can't seem to get the if statement working. It will read the contents of the file, but I recieve this error, with an arrow under the if statement.
File "<string>", line 1
    import sys,json;data=json.loads(sys.stdin.read());if data['123456789']:print...
                                                       ^
SyntaxError: invalid syntax

Contents of File:
{"123456789":"Well Hello"}

Function that searches file:
function find_it {
  file=~/.pt_branches
  CURRENT_CONTENTS=`cat $file`
  echo $CURRENT_CONTENTS | python -c "import sys,json;data=json.loads(sys.stdin.read());if data['$1']:print data['$1'];else:print '';"
}

Command
find_it "123456789"

If I were to remove the if statement and just return the value, it does work, but I want it to be able to return and empty string if it does not exist.
echo $CURRENT_CONTENTS | python -c "import sys,json;data=json.loads(sys.stdin.read());print data['$1'];"

Any ideas on what I am doing wrong?!?
Answer: Do this:
echo $CURRENT_CONTENTS | python -c "
import sys
import json
data = json.loads(sys.stdin.read())
if data['$1']:
    print data['$1']
else:
    print ''"

Answer: Code golf!
"import sys, json; data=json.loads(sys.stdin.read()); print data.get('$1') or ''"


Non-elementwise Proof for Decomposition of a Module into Direct Sum of Kernel and Image of an Idempotent Morphism

Question: This is actually P. Aluffi's "Algebra: Chapter $0$", exercise ${\rm III}.6.3$

Let $R$ be a ring, $M$ an $R$-module and $p\colon M\to M$ an $R$-module homomorphism such that $p^2=p$. (Such a map is called a projection). Prove that $M\cong\ker p~\oplus~{\rm im}~p$.

The standard approach is more or less the following: let $x\in M$, then $x-p(x)\in M$ and by the given property we deduce that $p(x-p(x))=p(x)-p(x)=0$ and therefore $x-p(x)\in\ker p$. Now we can write every element as $x=(x-p(x))+p(x)$ and the rest are routine verifications. (here is a reference regarding "standard approach").
Anyway, in the spirit of Aluffi's book, I was wondering if there is a different approach. More precisely, is it possible to deduce $M\cong\ker p~\oplus~{\rm im}~p$ only from $p^2=p$ not using the elementwise approach. I was thinking about constructing morphisms $\alpha,\beta$ (somehow related to $p$) such that $0\rightarrow\ker p\overset\alpha\rightarrow M\overset\beta\rightarrow{\rm im}~p\rightarrow0$ splits. An evident choice would be the inclusion arrow $\iota\colon\ker p\to M$ and $p$ itself, respectively, but I was not able to deduce anything else than trivialities from hereon.

Is this doable? Does anyone has a reference, where this approach is followed? Or do I miss something (maybe obvious) which prevents one from doing it without elements?

Thanks in advance!
Answer: The canonical short exact sequence
$$0 \to \ker p \xrightarrow{i} M \xrightarrow{p}  \text{im}~p \to 0$$
is split, because $\sigma : M \to \ker p$ defined by $\sigma = 1_M-p$ (is well defined because $p^2=p$) is a section for the sequence (this is, $\sigma i = 1_{\ker p}$) and thus by the splitting lemma, the sequence is split exact
Comment: You can prove the splitting lemma with diagram chasing in $R$-mod and then use the Freyd-Mitchell theorem, but you can also prove it using only the properties of an abelian category: start with a short exact sequence $0 \to A \xrightarrow{a} B \xrightarrow{c} C \to 0$ and a section $s: B \to A$ such that $sa=1_A$. The morphisms $s$ and $c$ let you build a morphism $f:B \to A \oplus C$ such that $\pi_C f =c$ and $\pi_A f = s$. The idea is proving that the arrows $1_A$, $f$, $1_C$ form an isomorphism between the exact sequences $0 \to A \to B \to C \to 0$ and $0 \to A \to A\oplus C \to C \to 0$
Comment: Well, I think that is more or less what I was looking for. Thank you kindly!
Comment: I just edited it so it is completely free of any elements. This way, the proof is the same in all abelian categories (since the splitting lemma is valid in all abelian categories)
Comment: IIRC the proof of the Splitting Lemma does require some sort of diagram chase, doesn't it? I actually thought about using the Splitting Lemma somehow, but wasn't able to find an appropriate section of $p$ (now, that I see it, it's a mystery to me how I missed that one as I used the same idea in the "standard approach").
Comment: Perfect! I've to admit that I only know about the Splitting Lemma by name and purpose and am not that confident with the details of its proof. However, thank you for your detailed explanation! Then, indeed, we've an element*free* proof :D
Comment: (+)  one has to check that $\pi_C f =c$ and that $i_A = fa$. The first one has already been seen and for the second one, $i_A= i_A sa =i_A \pi_A f a = fa$. The last thing to be checked is that f is an isomoprhism for this, use the 5 lemma or the snake lemma, which can be proven in an abelian category without embedding thoerems: https://math.stackexchange.com/users/1650/martin-brandenburg
Comment: Is it intended that you link to the profile page of a particular user or did you meant to refer to a specific post?
Comment: No, it's just that I'm a bit new to math stack. This is the post i referred to https://math.stackexchange.com/questions/1606806/five-lemma-with-category-theory
Comment: Ah, okay. Thank you for the link though. And, by the way, welcome to MSE! :)


Bitwise Not for double in Java

Question: In Java, the bitwise not operator (~) inverts all bits of an integer or long. But why is this not possible for a double? Is there any method of using this operation on a double? I am writing my own programming language in Java (this is NOT a homework assignment), and I believe that bitwise should be included for doubles.
Any ways of using bitwise not on doubles?
EDIT: This is not a duplicate of How to perform a bitwise operation on floating point numbers, because I am using Java as opposed to the discussed C. I also know that there is no possible way using the syntax of the language, but the answers there seem to cast the double to an int. I need to keep the double as it is, without changing it to an int or long.
EDIT: The bitwise not isn't what I thought it would be. It seems to perform *-1 - 1 on the input. Therefore, my question now becomes:
I wish to turn a double into binary (see my other question ), invert all of the bits (0 to 1 and 1 to 0), then convert it back to a double.
Comment: Well, you could use `Double.doubleToRawLongBits`, negate it, and then `Double.longToDoubleBits`. Doesn't mean much though.
Comment: But the result won't *mean* anything. It'll be 'the bitwise inverse of a floating point number', which has no intrinsic or mathematical value.
Comment: What meaning do bitwise operations have for floating-point values?
Comment: @JamesSmith ... so if you think the inverse of 0 should be 1, and the inverse of 1 should be 0, what is the inverse of 3? or 7? or for that matter 2.718 ?
Comment: Perhaps you could explain what you expect the negation of `0` to be... in binary as a `double`.
Comment: The inverse of `0` should be `1`. `0` is `0b0`, and the inverse of that is `0b1`. Also, using long bits results in `NaN` most of the time.
Comment: For (signed) integers, adding Not-itself will always result in `-1`. Doing the same with a double will probably only result in garbage.
Comment: @khelwood I have just found out that the bitwise not isn't quite what I expected. It looks like `~x = (-1 * x) - 1`.
Comment: Just for the principle, I quite agree this is *not* a duplicate of the other suggested question, although it is more or less the same problem indeed.
Comment: I have decided that I will not include bitwise operators on doubles, for the reason of recursive decimals losing precision.
Answer: Java didn't supply the ~ operator on the floating-point types float and double because such an operation isn't meaningful.
You can simulate it with this method:
public static double performNotOnDouble(double d)
{
    return Double.longBitsToDouble(~Double.doubleToLongBits(d));
}

But the results don't have any real meaning:
0.0: NaN
1.0: -3.9999999999999996
10.0: -0.43749999999999994
-1.0: 3.9999999999999996
0.01: -440.31999999999994
3.141592653589793: -1.4292036732051032
2.718281828459045: -1.6408590857704772
Infinity: -2.225073858507201E-308
-Infinity: 2.225073858507201E-308
NaN: -1.1125369292536E-308

There is no point to taking the bit-complement of a floating-point type, because the result doesn't have any meaning.
Comment: It probably means something, but it's not obvious what it is
Comment: Here is a *practical* reason inverting the bits cannot lead to a meaningful number: in IEEE 754 double precision, "the most significant bit is always 1 for a normalized number, this bit is not typically stored and is called the 'hidden bit'." ([wikipedia](http://en.wikipedia.org/wiki/Significand)) – and you cannot invert what is not there.
Comment: @Smac89 what the answer means is that the result has no *useful* meaning.
Answer: You could use Double and convert it using doubleToRawLongBits and then use any bitwise operator you want.
However, I don't know what this will suppose to mean :$ but it will definitely work... Is this what you need?
Answer: You'd really need to read up about IEEE 754 before considering this sort of operation, and this part of the JLS spec.

The inverse of 0 should be 1. 0 is 0b0, and the inverse of that is 0b1. Also, using long bits results in NaN most of the time.

While 0.0d is indeed 0b0, 0b1 is certainly not the bitwise inverse of 0.0d.

Double.doubleToLongBits(0.0d)) = 0
Double.doubleToLongBits(1.0d) = 4607182418800017408
Double.longBitsToDouble(0b1) = 4.9E-324

In general, each double value really represents a range of values around that value you're able to print out, taking into account some imprecision.
You should also check "How to resolve a Java Rounding Double issue" (and possibly a number of questions that link to it).
Bitwise operations in that context are generally meaningless.
(Along these lines, you can read a bit about strictfp for your general Java knowledge.)

Bitwise Not for double in Java


Editing the script of a js on a live site

Question: //lots of html here

suppose there is a webpage, file.php, like this and i wanna edit this page live. so if i use chrome dev tools then the edited script does not works. What i did was changed it a bit to...
$.post(url,{
    a :'edit'
});

Now this edited js does not works ! It still uses its prev code!
Comment: try clearing the browswer cache.
Comment: _It still uses its prev code!_...so what you can do is hit `f12` go to `Network` tab and in the network area you can right click and choose `clear browswer cache` and `clear broswer cookies`. Do this and see if this helps.
Answer: It is most likely a browser caching issue. If you press Ctrl+F5, it will force your browser to refresh its static content.
Also if you refer to your JavaScript files with a unique query string, it will stop the browser from caching it.
Please note that this may cause the load time of the site to increase as the browser won't be loading the JS file from its cache.


Atualizar informação pelo HTTP PATCH

Question: Bem, eu quero atualizar a informação status pelo PATCH mas eu não estou conseguindo, porque eu não sei como funciona, pelo PUT e preciso passar todas as informações pelo meu request.body, assim eu crio um novo usuário sobre outro, correndo o risco de mudar informações e etc, então estudando os verbos, encontrei o PATCH, mas quando eu simplesmente vou lá no Insomnia, gero o usuário, pego o id e levo para procurar por route paramse clico para continuar ele retorna no json assim
{ "status": "Pronto" }, mas quando eu utilizo o GET para ver se alterou o status ele permance no Em preparação. Aonde está o erro e como resolver, obrigado desde já pelo aprendizado...
const express = require('express')
const site = express()
const uuid = require('uuid')
site.use(express.json())

const orders = []

site.get('/order', (request, response) => {
    return response.send(orders)
})

site.get('/order/:id', (request, response) => {
    const { id } = request.params

    const index = orders.findIndex(user => user.id === id)

    if (index < 0) {
        return response.status(404).json({ error: "user not found" })

    }

    return response.json(orders[index])
})

site.post('/order', (request, response) => {
    const { clientName, price, status, order } = request.body

    const orderPerson = ({ id: uuid.v4(), order, clientName, price, status })

    orders.push(orderPerson)

    console.log(orderPerson)

    return response.status(201).json(orderPerson)
})

site.put('/order/:id', (request, response) => {

    const {clientName, price, status, order} = request.body
    const { id } = request.params

    const updateOrders = ({id, order, clientName, price, status})

    const index = orders.findIndex(user => user.id === id)

    if (index < 0) {
        return response.status(404).json({ error: "user not found" })

    }

    orders[index] = updateOrders
    return response.json(updateOrders)
})

site.delete('/order/:id', (request, response) => {
    const { id } = request.params

    const index = orders.findIndex(user => user.id === id)

    if (index < 0) {
        return response.status(404).json({ error: "user not found" })

    }

    orders.splice(index, 1)

    return response.status(204).json(orders)
})

site.patch('/order/:id', (request, response) => {
    const { id } = request.params
    const {clientName, price, status, order} = request.body
    const updateStatus = ({id, order, clientName, price, status: "Pronto"})

    const index = orders.findIndex(user => user.id === id)

    if (index < 0) {
        return response.status(404).json({ error: "user not found" })

    }

    console.log(request)
    return response.json(updateStatus)
})

port = 3001
site.listen(port, () => {
    console.log(`server starded in port ${port}`)
})

Comment: Ficou faltando fazer o update no array. Ex.:    `orders[index] = updateStatus`
Comment: o put e path são bem parecidos, faltou essa linha para atualizar a array orders mesmo. para o patch seria bom fazer uma lógica para atualizar só os itens que estão sendo enviados pelo body, pois nele podemos enviar uma única propriedade.
Comment: Mas é isso, eu quero atualizar os itens enviado do pelo body
Answer: Observe que seu método PATCH não está atualizando a sua "base de dados" (array de orders). Você está apenas enviando um objeto estático de volta para o client-side.
Para que o seu serviço funcione como o esperado, será necessário atualizar a base de dados e, então, retornar a resposta do registro atualizado, assim:
site.patch('/order/:id', (request, response) => {
  const { id } = request.params
  const {clientName, price, status, order} = request.body
  const updateStatus = ({id, order, clientName, price, status: "Pronto"})

  const index = orders.findIndex(user => user.id === id)

  if (index < 0) {
    return response.status(404).json({ error: "user not found" })
  }

  console.log(request)
  // CORREÇÃO AQUI
  orders[index] = updateStatus
  return response.json(updateStatus)
})

Atualizar informação pelo HTTP PATCH


Is a Bayesian Classifier a good approach for text with numerical meta-data?

Question: I'm trying to come up with an approach for detecting scam adverts on my website. I think the problem has a lot in common with detecting spam email (for which a naive Bayesian classifier is a common solution) since many of the signals that indicate a scam will be found within the text of the advert.
However, there are certain other pieces of information which can be good scam indicators, but I'm not sure if/how a Bayes classifier could use them, because they involve numeric values (with values at the extremes of the range being suspicious) rather than simple binary values corresponding to the presence or absence of a word in the text.
For example, many scam adverts have the price of the item set very low (to attract lots of views), so I would like a lower than normal price to be a strong indicator that the advert may be a scam.
Is Bayes still a good fit for my requirement, if not then could you recommend a different approach?
Answer: Sure you can use Naive Bayes. You just have to specify what form the conditional distribution will have.
I can think of a few options:

Binary distribution: Binarize your data using a threshold, and you revert to the problem that you were already solving.
Parametric distribution: If there is some reasonable parametric distribution, e.g. Gaussian, you can use that.
Non-parametric distribution: Decide on bins for the numerical data and use those to construct an empirical non-parametric distribution.
Answer: Naive Bayes classifiers can accommodate numeric variables as well as discrete ones without too much problem.  Essentially there are three approaches: (i) discretise the numeric values (ii) use a parametric model of each numeric attribute (e.g. Gaussian) or (iii) use a non-parametric (e.g. Parzen) density estimator for each numeric attribute.
see e.g. "Naive Bayes classifiers that perform well with continuous variables" by Remco Bouckaert
Comment: @LOL same answer as mine with the same order of choices and the same examples... hah what are the odds? I guess we all think alike
Comment: Clearly I shouldn't have spent five minutes tracking down the reference ;o)
Answer: Naive Bayes can certainly work with numeric attributes as well as discrete ones (modulo concerns about the appropriacy of the assumed distribution as mentioned in other answers). However, you should consider whether you really want to use Naive Bayes, as the non-discriminative methodology will break down more and more as you combine data from various sources, with potentially strong correlations.
If you want to retain a probabilistic interpretation, consider logistic regression, which is an exact analog of Naive Bayes with a discriminative rather than generative objective (see this paper for example: Logistic Regression Vs Naive Bayes. You can find various implementations of it: I like Mallet, if you can use java (accessible as a command-line tool or an API).
If a strict probabilistic interpretation isn't necessary, you can use an SVM. There are many implementations of this, but the de-facto standard (with a variant available in most languages) is LibSVM.
Answer: You can use numerical values quite easily. In the term P(Feature|scam=Yes) you could put a gaussian distribution or any other empirical distribution from training data (for e.g. sort the data, create a function that returns the percentile of the given input numerical value). Here is a write up describing that


Implement a custom PHP library into a custom block module?

Question: I have a PHP class and it's files in the folder:
/var/www/include_php/**the_folder**

How should I include this into Drupal?
ln -sf /var/www/include_php/the_folder /var/www/drupalSite/sites/all/libraries/the_folder

and then do drupal_add_library('the_folder') or something else in my block module?
Answer: drupal_add_library() is for Javascript/CSS libraries; Drupal doesn't have a similar registry for PHP libraries/classes.
To include a file inside a module in your installation, use module_load_include() e.g.
module_load_include('inc', 'mymodule', 'mymodule.pages');

For all others just defer to the standard PHP functions (include/require, etc.)
Answer: If your PHP Library is a class or set of classes, you can add them to the registry.  This is what I am doing in the Mobile Detect module.  I got the idea from another module, but I don't recall which one:
/**
 * Implements hook_registry_files_alter().
 *
 * Adds the Mobile_Detect() class to the registry so that autoloading works.
 * See http://drupal.stackexchange.com/questions/42266 for a discussion.
 */
function mobile_detect_registry_files_alter(&$files, $modules) {
  $library_path = _mobile_detect_get_library_path();

  $files[$library_path . '/Mobile_Detect.php'] = array(
    'module' => 'mobile_detect',
    'weight' => 0,
  );
}

This adds the class to registry so it will be autoloaded if instantiated as $detect = new Mobile_Detect();.  You can edit the $library_path = ... line to suit your situation.
Answer: In your module symlink library's folder
ln -sf /var/www/include_php/the_folder /var/www/drupalSite/sites/all/mymodule/the_folder

Then in you mymodule.info add series of lines:
files[] = the_folder/first.inc
files[] = the_folder/second.inc

and so on. That way your module will know about them and Drupal will be able to autoload them.


Como escapar aspas no PostgreSql?

Question: Tenho uma função no postgresql function f_criaproduto(p_texto text), eu tenho que passar da seguinte forma f_criaproduto(' meu produto é 'televisao' da melhor qualidade').
Como eu posso escapar as aspas no texto?
Comment: Qual erro aparece? o problema é anular as aspas simples?
Answer: Pra escapar aspas, você pode duplicá-las. Exemplo:
f_criaproduto('meu produto é ''televisao'' da melhor qualidade')

Para evitar o problema de ter muitas aspas para dar escape, você pode utilizar a sintaxe de dollars:
$$meu produto é 'televisao' da melhor qualidade$$

Note que, nesse caso, a string é definida com um par de dolares ao invés de aspas. Caso a sua string também tenha dolares, você pode utilizar uma tag para isso, definindo ela entre os dolares. Por exemplo, para definir a string minha 'televisao' custa uma boa $!
$string$minha 'televisao' custa uma boa $!$string$

Ainda é possivel usar o caracter \ você precisa de duas coisas:

no arquivo de configuração, você deve abilitar standard_conforming_strings:
standard_conforming_strings = off
Iniciar a suas strings com E. Exemplo:
E'meu produto é \'televisao\' da melhor qualidade'

Mas essa ultima opção é antiga e deve ser evitada se possível
Answer: Basta duplicar as aspas simples para gerar um escape.
Exemplo:
f_criaproduto(' meu produto é ''televisao'' da melhor qualidade')

Comment: Valeu resolveu meu problema..
Answer: O delimitador $ pode ser utilizado para separar expressões. Exemplo:
DO $_$
BEGIN
    RETURN QUERY EXECUTE $$ || SELECT * FROM alunos || $$;
END;
$_$

Repare que a utilização do $ pode ser nomeada: $x$ ou $teste$. Não se esqueça que é necessário fechar da mesma forma que você começou

Como escapar aspas no PostgreSql?


Pie Chart Margins seems hard-coded in plotly 4.6 (CRAN version)

Question: I am using plotly R library to create pie charts and in the latest CRAN version 4.6 it seems there is some default hardcoded margins, padding added to it. I am not able to override the margins, padding through layout configuration also :
Example :
USPersonalExpenditure <- data.frame("Categorie"=rownames(USPersonalExpenditure), USPersonalExpenditure)
data <- USPersonalExpenditure[,c('Categorie', 'X1960')]
plot_ly(data, labels = ~Categorie, values = ~X1960, type = 'pie') %>%
layout(title = 'United States Personal Expenditures by Categories in 1960')

The above code is making pie chart without any padding or margins in plotly 4.5.6 but in 4.6 there is a default margin, padding added to it.
I tried to remove the margins but it doesn't seem to have any effect :
plot_ly(data, labels = ~Categorie, values = ~X1960, type = 'pie') %>%
layout(title = 'Tes Graph', margin = list(t = 0, b = 0, l = 0, r = 0, pad = 0))

Can you please help me if I am missing something, I want to remove the margins, padding from my pie chart? It was perfect with version 4.5.6
Comment: Can't reproduce, but I am using the newest version too - 4.7.0. Try that, you can install it with `devtools::install_github("ropensci/plotly") `
Comment: yeah its fixed, I in 4.7.0 Thanks :)
Answer: Its fixed in the dev version 4.7


Como contar caracteres de uma String ignorando os espaços em brancos?

Question: Estou tentando fazer um return na variável para contar quantas letras existem na palavra digitada e ignorar os espaços digitados, mas não sei como juntar o length com o trim, diz a seguinte mensagem:

Cannot invoke trim() on the primitive type int.

    public Integer contaCarateresSemEspaços(String texto) {
    return texto.length().trim();
}

Answer: length é um método da classe String, este método retorna um inteiro.
trim também é um método da classe String, porém você está invocando trim no retorno de length, que é um inteiro. Inteiros não possuem o método trim, por isso o erro.
Se por outro lado você estivesse invocando os métodos na seguinte ordem:
texto.trim().length()

Primeiro trim seria executado, retornando uma String, e então length seria executado sobre essa String retornada, o que é um código válido.
Porém trim só irá remover os espaços no começo e no final da sua String, se você quiser remover todos os espaços em branco, você terá que substitui-los com o método replaceAll:
texto.replaceAll("\\s","").length()

Comment: Só lembrando que `\s` também vai remover outros caracteres, como o TAB e quebras de linha (dependendo da string e do que vc precisa, pode fazer diferença ou não). Se quer remover **somente** os espaços, use `replaceAll(" ", "")`
Answer: Boa noite, se você quiser contar somente os caracteres a função Trim() não é indicada, ela remove somente os espaços do inicio e fim da string. Ao invés disso usse a função Replace().
Exemplo:
        string var = "meu valor estatico";
        Console.WriteLine(var.Replace(" ", "").Length);

Comment: ahh perfeito, eu esqueci de dizer que também preciso remover os espaços do inicio e do fim da string, como fazer ?
Comment: Se quiser remover somente do inicio e fim use a função Trim(), rimou kkkk.
Answer: Se você mudar a ordem do retorno do seu método, o erro deixará de acontecer, no entanto não fará o que você deseja.
O sensato seria você retornar o valor length após remover todos os espaços em branco:
return texto.replace(" ", "").length();

Dessa forma você retornará a quantidade de caracteres sem os espaços em branco.


Percent of records with a single MySQL Query

Question: Say I have a table users with columns user_id, name, rate. I want to get % positive rating. The formula for this is:
((all records with rating >= 3) / (total records)) * 100

This can be done by getting total records and all records with rating >= 3 by two queries. But I want to get this done using only one query. How can I do this?
Comment: Show your current query as well
Answer: select sum(rate >= 3) * 100 / count(*)
from users


Does Apache Directory support computer Entry?

Question: I want to create a complex Apache Directory Server with all components that i can, so i have a question if i can create a entity(entry) in ApacheDs of type computer like in Active Directory or Apache support only Users and Group?
Answer: If it does not contain a schema that meets your needs, you can add schema elements. So, if it doesn't have a "computer object" schema element already, you can just add that.
Here's the documentation: https://directory.apache.org/apacheds/basic-ug/2.3.1-adding-schema-elements.html
I do not know whether it contains built-in schema elements for managing computer objects, but it does support more than "Users and Groups".
Comment: There seems to be a pretty decent amount of information in the site I linked.
Comment: do you know where i can find info about all build-in schema elements?


Limit of Intersection of Three Closed Sets

Question: Let $A_n$, $B_n$ and $C$ be closed subsets in euclidean space such that
\begin{equation}
A_n \cap B_n \neq \varnothing, A_n \cap B_n \cap C\neq \varnothing
\end{equation}
for each $n=1,2,...$, and $\lim_{n\to \infty}A_n=A$, $\lim_{n\to \infty}B_n=B$. If $A\cap C\neq \varnothing$, $B\cap C\neq \varnothing$ and $A\cap B \neq \varnothing$, then can we have $A\cap B\cap C \neq \varnothing$?
Comment: What is your definition of $\lim A_n=A$? Do you want an example where $A \cap B \cap C $ is not empty or do you want to prove that this set cannot be empty?
Comment: Your assumptions $A_n\cap B_n\ne\varnothing$ and $A_n\cap B_n\cap C\ne\varnothing$ are redundant, as the latter implies the former.
Comment: @Kabo Murphy  Consider a correspondence $F(x)$, where $x\in E^n$ and $F(x) \in E^m$, the limit notation here is that when $x_n \to x$, $F(x_n) \to F(x)$, imagining $A_n$, $B_n$ as some $F_1(x_n)$ and $F_2(x_n)$. I want to prove this set is not empty...
Answer: If
$A_n=\{-1\}\cup\{-2\}\cup[n,\infty)$,
$B_n=\{-1\}\cup\{-3\}\cup[n,\infty)$,
$C=\{-2\}\cup\{-3\}\cup[0,\infty)$,
then:
$A_n,B_n,C$ are closed subsets of $\mathbb R$;
$A_n\cap B_n=\{-1\}\cup[n,\infty)\ne\varnothing$;
$A_n\cap B_n\cap C=[n,\infty)\ne\varnothing$;
$A=\lim_{n\to\infty}A_n=\{-1,-2\}$;
$B=\lim_{n\to\infty}B_n=\{-1,-3\}$;
$A\cap C=\{-2\}\ne\varnothing$;
$B\cap C=\{-3\}\ne\varnothing$;
$A\cap B=\{-1\}\ne\varnothing$;
$A\cap B\cap C=\varnothing$.
Does that answer your question?
Comment: It is very clear, thanks!


Regex: keep everything in  tags

Question: I have a large XML, looking like this:
<gender>M</gender>
<last-name>*</last-name>
<profession>2165dda2-dc59-41af-acb5-06d8914c4841</profession>
<first-name>*</first-name>
<mail-confirmation>1</mail-confirmation>
<fax-confirmation>1</fax-confirmation>

I only want to keep the  tags. I found a way to search IN the tag, like this:
<profession[^>]*>([^<]*?)</profession>

but how do I search everything outside of it? I tried to just flip it, like:
</profession[^>]*>([^<]*?)<profession>

or
</profession>([^<]*?)<profession[^>]*>

but that won't work.
Comment: What exactly are you trying to achieve? Just get the data _in_ those tags? Only the data outside? Just strip out these tags?
Comment: Exactly; I only want the data inside the profession tag. The rest has to go.
Comment: "How do I use a regex to parse XML/HTML" has been asked countless times here
Comment: are you trying to remove elements named "profession"?
Answer: Don't use regular expressions to parse XML. Use an XML parser:
#!/usr/bin/perl

use strict; use warnings;

use XML::LibXML::Reader;

my $reader = XML::LibXML::Reader->new(IO => \*DATA)
    or die "Cannot read from \\*DATA\n";

while ( $reader->read ) {
    print $reader->readInnerXml if $reader->localName eq 'profession';
}

$reader->finish;
print $reader->document->toString(1);

__DATA__
<person>
<gender>M</gender>
<last-name>*</last-name>
<profession>2165dda2-dc59-41af-acb5-06d8914c4841</profession>
<first-name>*</first-name>
<mail-confirmation>1</mail-confirmation>
<fax-confirmation>1</fax-confirmation>
</person>

Output:
C:\Temp> t
2165dda2-dc59-41af-acb5-06d8914c4841
See XML::LibXML::Reader.
Answer: Strictly you can't parse XML with a regex.
Quick and dirty solution with sed is to grep the lines with profession then replace "profession"
and "/profession" with ""  (markup is stripping the < > )
Comment: A quick and simple grep, that I didn't think of that!
Comment: You have to be careful if line endings can appear inside the tag
Answer: You might consider using XSL to select the values. For example, creating a comma-separated list of professions from your XML.
<xsl:stylesheet version='2.0' xmlns:xsl='http://www.w3.org/1999/XSL/Transform'>
    <xsl:output method="text" />
    <xsl:template match='/'>
        <xsl:for-each select="person">
          <xsl:value-of select="profession" /><xsl:text>,</xsl:text>
        </xsl:for-each>
    </xsl:template>
</xsl:stylesheet>

Notepad++ has an XML plugin that will run XSL against open files.
(Plugins > XML Tools > XSL Transformation)
Answer: What about
# Perl
$xml =~ s/^<profession>.*<\/profession>$/<profession><\/profession>/m;

Just make sure to use the multiline modifier.
Comment: I'm not really using perl. I'm stuck with notepad++'s regex capabilities. Or sed.
Comment: @skerit You do not have a programming question then, do you?
Comment: I have a question about a certain regular expression, it's the same thing as asking something about a specific SQL query.
Comment: Oops, I just realized you want the data OUTSIDE the profession tags gone. Well, Sinan Ünür's example is really good but Perl. Actually it's quite easy, like $xml =~ /(.*?)<\/profession>/sg but I'm not sure about the regex capabilities of Notepad++. Sorry for the misunderstanding.

Regex: keep everything in  tags


Can I proxy my no-ip domain using a .htaccess file on my hosted domain?

Question: I have a domain http://www.example.com which has a hosting package and website on it.
I also have a http://example.no-ip.org domain which contains some content I would like to appear under the same domain.
Can I setup a .htaccess file at http://www.example.com/proxy/ which proxies the files at http://www.example.no-ip.org/files/
Similarly, could I host an entire domain in the same way?, e.g. http://www.example2.com/ proxying http://example.no-ip.org/files2/
Alternatively, if someone were to say "That's stupid, use this free (or super-cheap) dynamic DNS host:" I would probably accept that answer.
Comment: Is there a reason why you can't simply put all the content on one host and eliminate the problem completely?
Comment: That host doesn't allow ssh access, and I like to update via git.
Answer: Basically, you'd need to setup a Reverse Proxy. In Apache, this is done using ProxyPass and ProxyPassReverse, but these directives can't be used in .htaccess.
There is a proxy option in mod_rewrite, so if mod_rewrite and mod_proxy is enabled, you could maybe use something like
<code>RewriteRule ^/(.*)$  http://example.no-ip.org/files2/$1 [P]
</code>
in your .htaccess, this would proxy example2.com/x.html to example.no-ip.org/files2/x.html. Redirects etc. sent from the backend will not work since you can't specify ProxyPassReverse. All in all not the way to go, so either putting everything in the same server or registering a DNS entry are both way better options.
Comment: DNS is out because getting a Static IP on my ISP is magnitudes costlier than getting a cheap hosting package... Thanks for the response though I'll try it out.
Comment: Or, if you really have to do it as you described, but you can't configure Apache to do it, you could use a "handmade" proxy in PHP or CGI.
Answer: Nope
But you can use apache's mod proxy to do something similar:
http://httpd.apache.org/docs/2.0/mod/mod_proxy.html#proxypass


Steps allowed while dribbling without putting the ball down?

Question: Looking for clarification, in basketball how many steps are allowed while dribbling without putting the ball down? So in other words drag dribbles, floats, push cross, etc. According to the rules you cannot travel while dribbling. Since the dribble is live, when does ball in hand go too far? There's nothing in the rules that's clear on it. Obviously not talking about palming or hand under ball situations.
Answer: The is no rule governing how many steps you can take per bounce when you are dribbling. You can dribble slowly and take 50 tiny quick steps between bounces if you want. Dribbling is dribbling.
Before you dribble or after the dribble, then the steps do count.
But then you arent dribbling anymore.
Comment: Interesting. Thanks. That's what I've always understood. So a good drag cross or push cross could be several steps as long as hand is on the side of the ball?
Comment: No, I get that, of course. I'm talking about ball is making contact with hand, is there a count? I've heard two, which would obviously match the max number allowed but I don't know if that's the case.
Comment: there is no count while dribbling. Either its a carry or no carry. the number of steps is irrelevant. So it all about hand under the ball , ball "stopped" or 2 hands.
Just concentrate on the whether the dribble handling is ok.  steps play no role here.


How to plot the log likelihood associated with each iteration of EM algorithm and the density of each component?

Question: I am new to R. I am working with mixture model and would like to get the plot the of the log likelihood associated with each iteration of the EM algorithm and the density of each component. I would like to do that manually.
Any help please?
Answer: I'm assuming you are able to calculate the log-likelihood and density in each iteration of your EM algorithm.
The general way to plot a function in R (of which a log-likelihood and density are) is to (a) create inputs for the function over the domain your interested in, (b)  generate outputs from your function and save in a data frame, then (c) plot the function using the data frame. Here's an example of plotting a mixture model density using the 2D mixture model given by:
library(mvtnorm)
# example 2D mixture density
mixture_model_density <- function(x){
  a <- 0.3
  a * dmvnorm(x = x, mean = c(3, 2)) + (1 - a) * dmvnorm(x = x, mean = c(2.5,5.5))
}

(a) Create inputs:
grid_space <- 0.2
x <- seq(from = 0, to = 10, by = grid_space)
y <- seq(from = 0, to = 10, by = grid_space)
grid_mat <- expand.grid(x=x,y=y)

(b) Generate outputs:
 grid_mat$mm_dens <- mixture_model_density(grid_mat)

(c) Plot results:
ggplot(grid_mat) + stat_contour(aes(x = x, y = y, z = mm_dens))

Comment: Thanks a lot. Not sure how to find log likelihood for each iteration but I will work and see. I just replicate one code (because I am new to R) and tried to get the plot. Thanks a gain.
Comment: @Silver what's the package/code you're using for the EM algorithm? Maybe you can update your question with some more details on that?
Comment: I replicated this code "http://stats.stackexchange.com/questions/55132/em-algorithm-manually-implemented/57928#57928" (in the answer). But, I was confused why the author of the code used "mysum*(pi1(p1*dnorm) so here the author repeat pi1 two times. And I could not produce the plot.
Comment: In the initialisation line `loglik[2]<-mysum(pi1*(log(pi1)+...`, the first `pi1` here is not `tau1` because `pi1` and `pi2` each start at `0.5` therefore they sum to `1` and `tau1` can be simplified. Look later in the code where `loglik[k+1]<-mysum(tau1*(log(pi1)+...`. As for the plot I need some more information. Maybe you can post a new question with a minimum working example with some example data and code of your first attempt?
Comment: Ok I will send my code as soon as I can. Thanks a lot for your replay.


How do you scale an image's position to the size of the window in HTML/JavaScript

Question: I am trying to create a JavaScript function that takes an image and scales the image's distance from the center of the window when the window is scaled. So if someone scales the width of the window to 70%, the image is now 70% as far away from the center of the window. The same applies vertically, so if the window's height is scaled to 50%, the vertical distance from the image's center to the center of the window is 50%.
The function has to use the center of the image to get its position in order to scale correctly.
I tried using this code for the function:
<code><!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <style>
</style>
</head>
<body>

<script>
var img = new Image();
img.src = "https://tier-list-cache.s3.us-east-2.amazonaws.com/movies)65.jpg
";

function continuousSpacing(image) {
    let center = {
    x: window.innerWidth / 2,
    y: window.innerHeight / 2
    };
    let distance = {
    x: image.offsetLeft - center.x,
    y: image.offsetTop - center.y
    };

    function scaleImage() {
    let currentCenter = {
        x: window.innerWidth / 2,
        y: window.innerHeight / 2
    };
    let scale = {
        x: currentCenter.x / center.x,
        y: currentCenter.y / center.y
    };
    let newDistance = {
        x: distance.x * scale.x,
        y: distance.y * scale.y
    };
    image.style.left = currentCenter.x + newDistance.x + 'px';
    image.style.top = currentCenter.y + newDistance.y + 'px';
    }

    scaleImage();

    window.addEventListener('resize', scaleImage);
}
</script>
</body>
</html>
</code>
Currently, the function instantly snaps the image to the center of the window when the window is scaled.
Comment: Maybe you could also provide HTML-Code so that we can also try it ourselves? Stackoverflow has a nice Code-Feature :D
Comment: I believe @Gykonik is talking about [Stack Snippets](https://meta.stackoverflow.com/q/358992/215552).
Comment: Yep, thanks! Always wondered how they're called :D
Comment: If you just want to center the image, the why you are not using the css??
Comment: Please complete the HTML page in the post. It doesn't append the image to the document or specify how the image is positioned before calling `scaleImage` (which translates the image position without scaling). For a test case I suggest using an inline block element ( with a border or colored background) rather  than a link to an off-site image that may or may not be permanent. In its current state the test page "doesn't work".
Answer: It looks like that you are not updating the center and distance variables inside the scaleImage() function when the window is resized.
It needs recalculation of values based on the new dimensions of the window.
<code>  function continuousSpacing(image) {
  let center = {
    x: window.innerWidth / 2,
    y: window.innerHeight / 2
  };
  let distance = {
    x: image.offsetLeft + image.offsetWidth / 2 - center.x,
    y: image.offsetTop + image.offsetHeight / 2 - center.y
  };

  function scaleImage() {
    let currentCenter = {
      x: window.innerWidth / 2,
      y: window.innerHeight / 2
    };
    let scale = {
      x: currentCenter.x / center.x,
      y: currentCenter.y / center.y
    };
    let newDistance = {
      x: distance.x * scale.x,
      y: distance.y * scale.y
    };
    image.style.left = currentCenter.x + newDistance.x - image.offsetWidth / 2 + 'px';
    image.style.top = currentCenter.y + newDistance.y - image.offsetHeight / 2 + 'px';

    // update center and distance for next resize event
    center = currentCenter;
    distance = {
      x: image.offsetLeft + image.offsetWidth / 2 - center.x,
      y: image.offsetTop + image.offsetHeight / 2 - center.y
    };
  }

  scaleImage();

  window.addEventListener('resize', scaleImage);
}
</code>
the center and distance variables are updated inside the scaleImage() function after the image is repositioned. This will ensures that the next time the scaleImage() function is called (on the next resize event), it will  update center and distance values to calculate the new image position.
Comment: This is still snapping the image to the center of the screen instantly.
Comment: I have added few changes, if that works.


Shrinkage effects in a hierarchical model

Question: I am working on the chimpanzees dataset from Richard McElreath's text, "Statistical Rethinking", edition 2.
I have built 2 simple models, one a fixed effects model and the other a hierarchical model.
library(modelr)
library(gridExtra)
library(rethinking)
data(chimpanzees)
d <- chimpanzees

rm(chimpanzees)
detach(package:rethinking, unload = T)
library(brms)

d$treatment <- 1 + d$prosoc_left + 2 * d$condition
d$treatment <- as.factor(d$treatment)
d$pulled_left <- as.factor(d$pulled_left)
d$actor <- as.factor(d$actor)
d$block <- as.factor(d$block)

The pulled_left fraction in the dataset as per the different actors is

As we can see, the actor #2 always pulls left.
The first fixed effects model
priors <- c(prior(normal(0, 0.5), class = b))

bm_11.4 <- brm(data = d, family = bernoulli,
          formula = pulled_left ~ 1 + actor + treatment,
          sample_prior = T, prior = priors,
          cores = 4, chains = 4)

Another, a hierarchical model
bm_11.5 <- brm(data = d, family = bernoulli,
          formula = pulled_left ~ 1 + (1|actor) + (1|treatment),
          sample_prior = T, #prior = priors,
          cores = 4, chains = 4,
          control = list(adapt_delta = 0.95))

When I take a look at the predictions based on the posterior draws using the following code

p1 <- d %>%
        data_grid(actor, treatment) %>%
        add_fitted_draws(bm_11.4) %>%
        median_qi(.value) %>%
        ggplot(aes(x = .value, y = treatment)) + geom_pointintervalh() + facet_wrap(actor~.) +
          ggtitle("Fixed Effects model")

p2 <- d %>%
        data_grid(actor, treatment) %>%
        add_fitted_draws(bm_11.5) %>%
        median_qi(.value) %>%
        ggplot(aes(x = .value, y = treatment)) + geom_pointintervalh() + facet_wrap(actor~.) +
          ggtitle("Random effects model")

grid.arrange(p1, p2, nrow = 1)

Now, I plot the predictions from both the models as shown below:

For the actor #2, the chimpanzee always pulls the left level and hence the probability of pulling the left level is closer to 1, in both the models. However, given the shrinkage I expected that the hierarchical model would be pulled away from 1 and not the fixed effects model. I would like to understand this behavior, please help.
Comment: Welcome to the site. You have commented out (`#`) the set priors for the hierarchical model, but not the fixed effects model. You have also set `sample_prior = TRUE`, even though this is incompatible with improper priors, [including the default ones](https://www.rdocumentation.org/packages/brms/versions/2.10.0/topics/brm). Could you elaborate on these choices?
Comment: 1. I am setting sample_prior to TRUE to understand the visualize the priors once the model fit is complete. 2. I have commented out the prior for the hierarchical model, as my understanding is that in brms, the group level priors are normal and there isn't flexibility to add a different prior (except for the standard deviation).
Answer: Not all group effects in a hierarchical model are shrunk equally.
One factor which determines how much a group estimate is shrunk towards the population estimate is how much data we have from that group. Broadly — more data, less shrinkage. (The choice of prior plays a role too but I don't look at the priors in my answer.)
The estimate of a group with many observations will be shrunk less than the estimate of a group with few observations: when we have a lot of data about a group we learn about that group directly from its data; there is less "information borrowing" from other groups.
Now let's look at the chimpanzees data. There are 72 observations per actor (chimpanzee), 18 under each of 4 treatments.
To demonstrate how shrinkage decreases as we collect more data about actor 2, I sub-sample the data starting with 1 data point per treatment all the way up to 18. (Throughout I keep all the data for the other six actors.)
Here is the posterior probability that actor 2 (the chimpanzee which always ended up pressing the level) presses the lever under the L/N treatment (food on the left, no partner).
Notice that when there are very few observations of actor 2, the random effect estimate is shrunk more than the fixed effect estimate, in agreement with the OP's expectation. Once we gather more observations, the difference between the two estimates disappears. Of course, the prior specification makes a difference too, but with more data, the likelihood becomes more informative about the parameter than the prior.

This is the R code I used to fit the fixed-effects and random-effects models with brms 2.20.4.
library("brms")
library("tidybayes")
library("tidyverse")

data(chimpanzees, package = "rethinking")

d <- chimpanzees %>%
  mutate(
    treatment = case_when(
      prosoc_left == 0 & condition == 0 ~ "R/N",
      prosoc_left == 1 & condition == 0 ~ "L/N",
      prosoc_left == 0 & condition == 1 ~ "R/P",
      prosoc_left == 1 & condition == 1 ~ "L/P"
    ),
    across(c(actor, block, treatment), as.character)
  ) %>%
  select(
    actor, block, treatment, pulled_left
  )

treatments <- unique(d$treatment)
treatments

prep_data <- function(k) {
  set.seed(1)
  bind_rows(
    d %>% filter(actor != "2"),
    d %>% filter(actor == "2") %>% group_by(treatment) %>% slice_sample(n = k)
  )
}

fit_fixef_model <- function(dat) {
  # Omit the intercept even though there are two categorical predictors.
  # `y ~ 0 + x1 + x2` works when there is a single categorical predictor
  # but not with two or more categorical predictors.
  model <- bf(
    pulled_left ~ x1 + x2,
    x1 ~ 0 + actor,
    x2 ~ 0 + treatment,
    nl = TRUE
  )
  # The prior also makes a difference; here I use the prior in Rethinking.
  prior <- c(
    prior(normal(0, 1.5), class = b, nlpar = x1),
    prior(normal(0, 0.5), class = b, nlpar = x2)
  )
  brm(
    model,
    prior = prior,
    family = bernoulli,
    data = dat
  )
}

fit_ranef_model <- function(dat) {
  model <- bf(
    pulled_left ~ x1 + x2,
    x1 ~ 0 + (1 | actor),
    x2 ~ 0 + treatment,
    nl = TRUE
  )
  # I use the same normal prior for the random effects even though
  # in practice we cannot compare a normal prior on "fixed" coefficients `b`
  # to a half-normal prior on the variance of "random" effects.
  prior <- c(
    prior(normal(0, 1.5), class = sd, nlpar = x1),
    prior(normal(0, 0.5), class = b, nlpar = x2)
  )
  brm(
    model,
    prior = prior,
    family = bernoulli,
    data = dat
  )
}

make_epred <- function(fit) {
  tibble(
    treatment = treatments,
    actor = "2"
  ) %>%
    add_epred_draws(fit) %>%
    median_qi(.epred)
}

# Vary k from 1 to 18
dat <- prep_data(k = 3)

fit.fe <- fit_fixef_model(dat)
fit.re <- fit_ranef_model(dat)

epred.fe <- make_epred(fit.fe)
epred.re <- make_epred(fit.re)


Outsource functionality from main.js

Question: I've started building an electron application from scratch. So far I can modify the mainWindow by inserting all the logic in main.js, but I'd like to keep the logic in separate files. Unfortunatly I fail to import them correctly, so help would be nice..
// main.js
const { app, BrowserWindow, Menu, dialog } = require('electron');
const fs = require('fs');

const { test } = require('./test'); // test module to be imported

let win;

function createWindow() {
  win = new BrowserWindow({
    icon: `file://${__dirname}/dist/text-editor/favicon.ico`,
    webPreferences: {
      nodeIntegration: true
    }
  });
  win.maximize();

  win.loadURL(`file://${__dirname}/dist/text-editor/index.html`);

  win.webContents.openDevTools();

  //menu bar
  const template = getTemplate();
  const menu = Menu.buildFromTemplate(template);
  Menu.setApplicationMenu(menu);

  win.on('closed', function () {
    win = null;
  });

  console.log(test.getName()); // throws error
}

// create window on electron init
app.on('ready', createWindow);

// quit when all windows are closed
app.on('window-all-closed', function () {
  if (process.platform !== 'darwin') {
    app.quit();
  }
});

app.on('activate', function () {
  if (win === null) {
    createWindow();
  }
});

function getTemplate() {...} // for example move this method to new file

// test.js
const getName = () => {
  return "Tom";
};

module.exports.getName = getName;

Running this code with ng build --prod && electron . throws:

Cannot read property 'getName' of undefined.

Thanks in advance ;)
Comment: How are you exporting the test.js file?
Comment: const test = require("./test")
Comment: try importing `const test  = require('./test')` and use `test.getName()`
Comment: Thanks to DenisTsoi & @namila007 ! Can sb. explain why using { test } did not work?
Comment: refer this answer https://stackoverflow.com/a/48064893/6771708 :)
Answer: Please import your js as const test = require('./test') and then use test.getName() in your main js file. when calling const {test} is trying to import the test object in the at relevant file . You can read more about that from this answer

Outsource functionality from main.js


Time complexity of betweenness centrality?

Question: What is the time complexity of computing betweenness centrality if we are given the shortest path predecessor matrix of a graph?
Predecessor matrix cells look like this:

if node i and node j are directly connected then value in the cell is 0;
if node i and node j are not connected then value in the cell is -1;
else  cell = predecessor(j) - this can be only one predecessor if there is a single shortest path or an array of predecessors if there are more than one shortest paths between i and j.

Thank you for your answer,
I am familiar with Brandes Algorithm. However Brandes Algorithm will compute the betweenness for all the nodes inside a network. I think that time spent for computing CB for one vertex is the same as the time for computing CB for all vertices as Brandes algorithm can't be adapted for such a case.
So, my idea was to store the predecessor matrix, and to be able to compute CB for a certain vertex (and not have to wait for all vertices CB computations).
I am aware I can't achieve smaller time complexity but I think that the difference in amount of time can be made by not computing CB for all 7000 vertices. Instead, by having this matrix I am able to compute CB for only one single vertex.
I think it is possible to compute CB in O(n^2*L) where L is the average shortest path when we have predecessor matrix.
What is your opinion about this concept?
Answer: As far as I can find out, the best known algorithm for computing betweenness centrality is the one described in this paper:

Ulrik Brandes (2001). A Faster Algorithm for Betweenness Centrality. Journal of Mathematical Sociology 25(2):163-177.

You'll see that this algorithm computes, as a first step, the number of shortest paths between every pair of nodes. It is natural to do so in a way that simultaneously computes the predecessor matrix too. So it appears that there's no benefit to pre-computing the predecessor matrix: you get it essentially for free anyway while executing Brandes' algorithm.
(Of course, this isn't a proof that it makes no difference, and maybe someone else knows better. You might want to ask on cstheory.stackexchange.com.)
Comment: On 18 Nov 2013, Nasre, Pontecorvi and Ramachandran suggested a faster algorithm. See http://arxiv.org/abs/1311.2147.


Combine Multiple Discrete Probability Density Functions

Question: I'm a bit stuck trying to figure out the combined probability from several discrete PDF's.
Lets say I have a bunch of different classes (Truck, Sports Car, Station Wagon, etc) and a bunch of different scenarios I've worked out from my dataset (Location, behaviour, size, heading, speed).
Probability that the vehicle at the position x,y (Golf course) is a:

Truck: 0.1
Sports Car: 0.7
Station Wagon: 0.2

Vehicle is white, probability that it is a:

Truck: 0.6
Sports Car: 0.1
Station Wagon: 0.3

Vehicle has 3 passengers, probability that it is a:

Truck: 0.2
Sports Car: 0.1
Station Wagon: 0.7

If I had info on an unknown vehicle (Has 3 passengers, painted white and is at the golf course) how would I create the combined probabilities that it is each class?
How would I do this for n discrete distributions.
Are these dependant or independent variables? What is the wording for what I'm trying to do so that I can look it up properly.
Sorry if this is a duplicate question, I don't know the correct jargon for this problem.
Answer: I ended up just taking the probabilities for each class and multiplied them together. After that I normalized back to 1. Probably not correct:
Prob that test-car (Has 3 passengers, painted white and is at the golf course) is class x:

Truck: (0.1 x 0.6 x 0.2)/Total = 0.012/(0.012+0.007+0.042)
Sports Car: (0.7 x 0.1 x 0.1)/Total = 0.007/(0.012+0.007+0.042)
Station Wagon: (0.2 x 0.3 x 0.7)/Total = 0.042/(0.012+0.007+0.042)

thus:
 - Truck: 0.20
 - Sports Car: 0.11
 - Station Wagon: 0.69
Hope someone comments with a better method than this
Comment: Multiplication is permitted only when the conditioning variables are *independent.*  In general, you cannot obtain an answer with just the information given in your question.


Extreme values of a function

Question: $x_1,x_2,...,x_m\in R^n$ are given. How to find $u\in R^n$ such that $\sum^{m}_{i=1} (d(u,x_i))^2$ is minimal.
I tried this way:
If
$J(u)=\sum ^{m}_{i=1} (d(u,x_i))^2$
$x_i=(x^{1}_{i},x^{2}_{i},...,x^{n}_{i}), i=1,2,...,m$
$u=(u_1,u_2,...,u_n)$
$(d(u,x_i))^2=\sum^{n}_{k=1} (u_k-x^{k}_{i})^2, i=1,..., m$
I tried to find partial derivatives of $J$ and I got that
$\dfrac{\partial J}{\partial u_i}=2(mu_i-(x^{i}_{1}+...+x^{i}_{m})), i=1,...,n$
If this is ok, I should determine when these partial derivatives are equal to $0$ and such values for $x_i$ would be the coordinates of vector $J'(u)$.
Then I should calculate matrix $J''(u)$ and check if it is positive definite.
Is this a good way to solution?
Are partial derivatives correct?
Comment: So does that mean the minimizer is the center of gravity?
Answer: This all seems fine. A few things you may want to consider as well (just out of interest, they shouldn't really impact your working) is how changing just one coordinate of $u$ affects $J(u)$ as well as if you can interpret in a simple way where $u$ is in relation to all the $x_i$

Extreme values of a function


only one of three sliding tabs is showing data

Question: I have three tabs, each of them should show specific data that was fetched from server. I have created only one fragment that is supposed to show different information based on which tab is open, however when app starts only one tab is showing data and after some time it crashes at the line when I am creating new NamesAdapter`
public class TabFragment extends Fragment {

    private static final String TAG = "TabFragment";
    int page;
    private ArrayList<Names> namesList;
    NamesAdapter adapter;
    ListView listView;

    public TabFragment() {
        // Required empty public constructor
    }

    public static TabFragment newInstance(int page) {
        TabFragment fragment = new TabFragment();

        return fragment;
    }

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

    }

    @Override
    public View onCreateView(LayoutInflater inflater, ViewGroup container,
                             Bundle savedInstanceState) {

        namesList = new ArrayList<>();
        listView = (ListView) getActivity().findViewById(R.id.listView);
        // Inflate the layout for this fragment
        new MyTask().execute();

        return inflater.inflate(R.layout.fragment_tab, container, false);

    }

    private class MyTask extends AsyncTask<String, String, Boolean> {

        @Override
        protected Boolean doInBackground(String... params) {

            String content = HttpManager.getData("http://union.ge/catalog/1455299520.json");

            Log.e("doInBackground", "was called");
            //return content;

            try {
                JSONArray json= new JSONArray(content);

                // get json objects one by one

                for (int i = 0; i < json.length(); i++){

                    Names names = new Names();
                    // this json object now has key-value pairs and
                    JSONObject jRealObject = json.getJSONObject(i).getJSONObject("name");
                    //Log.e("jrealobject", jRealObject.toString());

                    names.setEn(jRealObject.getString("en"));
                    names.setGe(jRealObject.getString("ge"));
                    names.setRu(jRealObject.getString("ru"));

                    //migebuli monacemebi shevinaxot chvens array-shi
                    namesList.add(names);

                }

                return true;

            } catch (JSONException e) {
                e.printStackTrace();
            }

            return false;

        }

// on postExecute method runs on the UI thread

            protected void onPostExecute(Boolean b) {
                super.onPostExecute(b);

                if(b){

                    adapter = new NamesAdapter(getContext(), R.layout.list_item, namesList, page);
                    Log.e(TAG, String.valueOf(page));

                    listView = (ListView) getActivity().findViewById(R.id.listView);
                    listView.setAdapter(adapter);

                } else {
                    Toast.makeText(getActivity(), "data was not parsed", Toast.LENGTH_SHORT).show();
                }
            }
    }

}

`
public class TabsAdapter extends FragmentPagerAdapter {

    final static int PAGE_COUNT = 3;
    private static final String TAG = "TabsAdapter";
    private String tabTitles[] = new String[] { "EN", "GE", "RU" };
    private Context context;

    public TabsAdapter(FragmentManager fm, Context context) {
        super(fm);
        this.context = context;
    }

    @Override
    public Fragment getItem(int position) {

        switch (position) {
            case 0:
                return TabFragment.newInstance(0);
            case 1:
                return TabFragment.newInstance(1);
            case 2:
                return TabFragment.newInstance(2);
            default:
                return null;
        }

    }

    @Override
    public int getCount() {
        return PAGE_COUNT;
    }

    public CharSequence getPageTitle(int position) {
        // Generate title based on item position
        return tabTitles[position];
    }

    @Override
    public void destroyItem(ViewGroup container, int position, Object object) {
        super.destroyItem(container, position, object);
        FragmentManager manager = ((Fragment)object).getFragmentManager();
        android.support.v4.app.FragmentTransaction trans = manager.beginTransaction();
        trans.remove((Fragment)object);
        trans.commit();
    }

}

and the adapter for Names class
public class NamesAdapter extends ArrayAdapter<Names> {

    private static final String TAG = "NamesAdapter";
    ArrayList<Names> items;
    Context context;
    int resource;
    int page;

    LayoutInflater vi;

    public NamesAdapter(Context context, int resource, ArrayList<Names> items, int pageNumber) {
        super(context, resource, items);

        this.context = context;
        this.resource = resource;
        this.items = items;
        this.page = pageNumber;

        vi = (LayoutInflater) context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);

    }

    public View getView(int position, View convertView, ViewGroup parent) {

        ViewHolder viewHolder;

        if (convertView == null){

           // convertView = vi.inflate(resource, null);
            convertView = LayoutInflater.from(context).inflate(R.layout.list_item, parent, false);
            viewHolder = new ViewHolder();

            viewHolder.textView = (TextView) convertView.findViewById(R.id.textView);

            convertView.setTag(viewHolder);

        } else {
            viewHolder = (ViewHolder) convertView.getTag();
        }

        switch (page) {
            case 0:
                viewHolder.textView.setText(items.get(position).getEn());
                break;
            case 1:
                viewHolder.textView.setText(items.get(position).getGe());
                break;
            case 2:
                viewHolder.textView.setText(items.get(position).getRu());
        }

        return convertView;
    }

    static class ViewHolder {
        public TextView textView;
    }

}

Comment: paste logcat with which error app crashed in adapter?
Comment: E/AndroidRuntime: FATAL EXCEPTION: main
java.lang.NullPointerException at android.widget.ArrayAdapter.init(ArrayAdapter.java:310) at android.widget.ArrayAdapter.(ArrayAdapter.java:153)  at ge.mobility.anako.parser.NamesAdapter.(NamesAdapter.java:33)at ge.mobility.anako.parser.TabFragment$MyTask.onPostExecute(TabFragment.java:111)at ge.mobility.anako.parser.TabFragment$MyTask.onPostExecute(TabFragment.java:62)
Comment: @AnaKoridze See my updated answer... Hope it helps you.
Answer: Try this:
Change in MyTask.class as follows:
In doInBackground(), replace

namesList.add(names);

with

publishProgress(names);

And override this method
@Override
protected void onProgressUpdate(String... values)
{
    super.onProgressUpdate(values);
    namesList.add(values[0])
}

The way you are creating an adapter is not correct. See this:
public class TabFragment extends Fragment {

    private static final String TAG = "TabFragment";
    private static final String PAGE_NO = "page_no";
    int page;
    private ArrayList<Names> namesList;
    NamesAdapter adapter;
    ListView listView;

    public TabFragment() {
        // Required empty public constructor
    }

    public static TabFragment newInstance(int page) {
        TabFragment fragment = new TabFragment();
        Bundle args = new Bundle();
        args.putInt(PAGE_NO, page);
        fragment .setArguments(args);
        return fragment;
    }

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

    }

    @Override
    public View onCreateView(LayoutInflater inflater, ViewGroup container,
                             Bundle savedInstanceState) {

        namesList = new ArrayList<>();
        listView = (ListView) getActivity().findViewById(R.id.listView);
        // Inflate the layout for this fragment

         page = getArguments().getInt(PAGE_NO);
         adapter = new NamesAdapter(getContext(), R.layout.list_item, namesList, page);
         Log.e(TAG, String.valueOf(page));

         listView = (ListView) getActivity().findViewById(R.id.listView);
         listView.setAdapter(adapter);

        return inflater.inflate(R.layout.fragment_tab, container, false);

    }

    @Override
    public void onViewCreated(View view, @Nullable Bundle savedInstanceState)
    {
        super.onViewCreated(view, savedInstanceState);

        // Start the task when view is created.
        new MyTask().execute();
    }

    private class MyTask extends AsyncTask<String, String, Boolean> {

        @Override
        protected Boolean doInBackground(String... params) {

            String content = HttpManager.getData("http://union.ge/catalog/1455299520.json");

            Log.e("doInBackground", "was called");
            //return content;

            try {
                JSONArray json= new JSONArray(content);

                // get json objects one by one

                for (int i = 0; i < json.length(); i++){

                    Names names = new Names();
                    // this json object now has key-value pairs and
                    JSONObject jRealObject = json.getJSONObject(i).getJSONObject("name");
                    //Log.e("jrealobject", jRealObject.toString());

                    names.setEn(jRealObject.getString("en"));
                    names.setGe(jRealObject.getString("ge"));
                    names.setRu(jRealObject.getString("ru"));

                    //migebuli monacemebi shevinaxot chvens array-shi
                    publishProgress(names);

                }

                return true;

            } catch (JSONException e) {
                e.printStackTrace();
            }

            return false;

        }

        @Override
        protected void onProgressUpdate(String... values)
        {
            super.onProgressUpdate(values);

            namesList.add(values[0]);
            adapter.notifyDataSetChanged();
        }

            // on postExecute method runs on the UI thread
            protected void onPostExecute(Boolean b) {
                super.onPostExecute(b);

                if(!b){
                   Toast.makeText(getActivity(), "data was not parsed", Toast.LENGTH_SHORT).show();
                }
            }
    }

}

Comment: it does the same thing, I mean, when I click first tab, sometimes it shows second tab elements, sometimes third tab elements, with no logic
Comment: Is the `NullPointerException` you was facing before, is it appearing now?
Comment: And I don't understand where and why use are using `page` variable.
Comment: I wan to use some parameter to check which tab is open to display relevant information on listView
Comment: Okay, `TabFragment ` is used in 3 different tabs. And same API is being used for each tab. That's  why same data is displayed. Change the API based on the page number.
Comment: what should I do more specifically?


Datas recorrentes a partir de uma inicial com PHP

Question: Estou tentando trazer as próximas datas para inserir contas de um pagamento que se repete por X vezes, mas estou tendo problemas quando isso envolve os dias posteriores a 28 de fevereiro.
$qtde_meses = 6;
$cur_date = '2023-01-30';
$date = strtotime($cur_date);
$day = date("d", $date );

$newdate = new DateTime($cur_date);

    for ($i=0; $i<$qtde_meses; $i++) {
        $tmpDate = clone $newdate;

    if ($day == 31) {
        $tmpDate->modify('last day of +' . $i . ' month');

    } else {
        $tmpDate->modify('+' . $i . ' month');
    }
        $datafinal[] = $tmpDate->format('Y-m-d');
    }
foreach($datafinal as $d) {
    echo $d;
}

O resultado do array acima, por exemplo, é:
2023-01-30
2023-03-02 <- Incorreto
2023-03-30
2023-04-30
2023-05-30
2023-06-30

Possuem alguma ideia para resolver isso? Testei em PHP 7.3 e 7.4.
Comment: Porque `2023-03-02` é a data incorreta e não `2023-03-30`?
Comment: Olá Augusto. Parece ser a data correta para o código acima, que somou 30 dias a partir da data inicial, mas para um boleto recorrente, deveria ser o último dia de fevereiro (28/02) ao invés de 02/03).
Comment: https://pt.stackoverflow.com/q/375462/112052
Comment: @hkotsubo já havia testado dessa forma e aqui ele piora a situação, pois ele soma 1 ao mês anterior do array e o resultado fica assim: 2023-01-30, 2022-03-02, 2023-04-02, etc..
Comment: $d = new DateTime();
$d->setDate(2022, 12, 30);
$quantidade_meses = 6;
for ($i = 0 ; $i < $quantidade_meses ; $i++) {
    $d->add(new DateInterval('P1M'));
    echo $d->format('d/m/Y');
}
Comment: Não é necessário essa lógica toda e nem clone (já que parece que não vai reusar) para operar a cada format, você poderia pegar, simplesmente usar o `last day` e então pegar os resultado com format adicionar ao array e só em seguida somar o +1 mês, isso no final da execução atual do loop
Comment: Oi @GuilhermeNascimento o código está um pouco sujo pela quantidade de tentativas que acabei fazendo, mas o problema não está sendo somar um mês e sim quando o resultado do array envolve o final de fevereiro, como explico no texto.. Preciso achar uma forma de fazer ele trazer o 28-fev e não o 02-mar..
Comment: É isso o que precisa https://ideone.com/O21zJm? (atualizado)
Comment: @GuilhermeNascimento Valeu, mas não funciona.. Note que nesse caso vc usou o dia 30 e imprimindo o último dia de cada mês sequente.. Se eu lanço um boleto no dia 30, ele precisa se repetir no dia 30 de cada mês e não no dia 31, como acontece no seu resultado.. Isso corrige o caso de fevereiro, mas gera erro nos demais meses..
Comment: Não é só criar uma `if($x->format('d') > 30)` pra checar o dia, se passar do dia 30 você ajusta apenas o valor da variavel? Veja isso https://ideone.com/6Jj5Np
Answer: Não é necessário essa lógica toda com:
$date = strtotime($cur_date);
$day = date("d", $date );

E nem clone, já que parece que não vai reusar.
Para operar a cada format, você pode simplesmente usar o last day of ... month, em seguida pegar o dia obtido, checar se passou do 30 com uma if (), se passou você só ajusta a variável do dia, e depois você pega só o ano e o mês e concatena com dia da variável que salvou anteriormente e em segui você usa o modify('last day of +1 month') para que no próximo uso seja o próximo mês.
Algo como:
<?php

$my_dates = [];
$qtde_meses = 6;

// Apenas ano e mês são suficientes no caso especifico
$cur_date = '2023-01';

// Dia limite que deseja
$dia_lancado = 30;

$date = new DateTime($cur_date);

// Define o último dia do mês atual
$date->modify('last day of this month');

for ($i = 0; $i < $qtde_meses; $i++) {
    // Pega o ultimo dia do mês atual
    $dia = $date->format('d');

    // Limita os dias ao 30 (conforme a variável)
    if ($dia > $dia_lancado) $dia = $dia_lancado;

    $my_dates[] = $date->format('Y-m') . '-' . $dia;

    // Vai para o último dia do próximo mês
    $date->modify('last day of +1 month');
}

print_r($my_dates);

Saída:
Array
(
    [0] => 2023-01-30
    [1] => 2023-02-28
    [2] => 2023-03-30
    [3] => 2023-04-30
    [4] => 2023-05-30
    [5] => 2023-06-30
)

Teste online no ideone: https://ideone.com/6Jj5Np
Comment: Muito bom Guilherme. Funcionou pro que eu precisava!


Como impedir que o conteúdo de um elemento seja copiado após Ctrl+A e Ctrl+C?

Question: Eu tenho uma página e quero permitir a cópia normalmente, mas gostaria que elementos específicos não fossem copiados após usar Ctrl+A e Ctrl+C, por exemplo formulários, menus de navegação e banners de propaganda.
Isso porque o conteudo geralmente é o que interessa se alguém quiser copiar, mas o navbar não é interessante.
Eu tentei isto:
form, nav {
    -webkit-user-select: none;
       -moz-user-select: none;
        -ms-user-select: none;
  -webkit-touch-callout: none;
            user-select: none;
}

Mas se eu pressionar Ctrl+A e Ctrl+C os formulários e o navbar são copiados juntos de qualquer maneira.
Existe alguma maneira de prevenir a copia de elementos específicos após o uso de Ctrl+A?
Comment: Leitura recomendada : [Como copiar para área de transferência em JavaScript?](http://pt.stackoverflow.com/q/89139/14213), [Maneira cross-browser de copiar texto para a Área de Transferência (Clipboard)](http://pt.stackoverflow.com/q/17030/14213), [Como pego o que está copiado na área de transferência](http://pt.stackoverflow.com/q/57752/14213)
Comment: @GuilhermeLautert só corrigindo um dos links é duplicata de http://pt.stackoverflow.com/q/50083/3635
Comment: Verdade, e a resposta é sua ainda '-', creio que você poderia capturar o conteúdo e realizar um replace no que não quer e sobrescrever a área de transferência.
Comment: Parece uma boa ideia, mas a manipulação do Clipboard é complexa e muitas vezes não funciona, ainda mais se tratando de HTML e copiar as propriedades, isso poderia até afetar a performance, mas obrigado pela ideia, vou ver o que consigo
Comment: Ola, para não selecionar uso sempre html o unico até ao momento que foi-me fiel até ao fim :).
 Tente mesmo em html nas tags que não pretende coloque: unselectable="on" onselectstart="return false"
Answer: Após alguns testes, encontrei soluções com :before {} e content: ""; do CSS, mas isto só funciona pra texto simples, então resolvi testar o evento oncopy do javascript.
Consegui assim, usando os eventos copy, beforecopy, a propriedade display: none e document.querySelectorAll pra pegar os elementos (não precisa colocar no onload).
Suporte

Firefox
Chrome
Internet Explorer 10+

Código de exemplo

(function (d) {
    if (d.addEventListener && d.querySelectorAll) {
        //Edite esta variável conforme a necessidade (elementos que deseja impedir que sejam copiados)
        var query = "nav, form";

        //Nota: no IE a tag script é copiada, então prefira deixar sempre ele
        query += ", script";

        var running = false;

        function finishEventCopy() {
            running = false;
        }

        //Oculta os elementos pra não serem copiados
        function preventEventCopy(e) {
            if (running) return;

            running = true;

            //Clona o body
            var cloned = document.body.cloneNode(true);

            //Cria elemento temporário que será selecionado pelo range
            var tmpNode = document.createElement("div");
            var els = cloned.querySelectorAll(query);

            for (i = 0, j = els.length; i < j; i++) {
                els[i].parentNode.removeChild(els[i]);
            }

            //Cria um elemento vazio pra ajustar o range de maneira mais fácil
            var empty = document.createElement("div");
            empty.innerHTML = " ";

            tmpNode.appendChild(cloned);
            tmpNode.appendChild(empty);

            document.body.appendChild(tmpNode);

            //define o range no DIV temporário
            var range = document.createRange();
            range.setStart(tmpNode.firstChild, 0);
            range.setEnd(empty, 1);

            //Seleciona o div temporário
            var sel = window.getSelection();
            sel.removeAllRanges();
            sel.addRange(range);

            //Executa o comando de cópia
            document.execCommand("copy");

            document.body.removeChild(tmpNode);

            empty = range = sel = tmpNode = null;

            setTimeout(finishEventCopy, 10);
        }

        function triggerByKeyboard(e) {
            if (e.ctrlKey && e.keyCode == 67) preventEventCopy(e);
        }

        d.addEventListener("beforecopy", preventEventCopy, false);
        d.addEventListener("copy", preventEventCopy, false);
        d.addEventListener("keydown", triggerByKeyboard, false);
    }
})(document);

Navegação

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut porttitor orci at purus elementum feugiat. Ut sed ipsum tempor, consequat tortor in, egestas metus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In viverra volutpat tellus ullamcorper condimentum. In pharetra mauris vel neque congue consectetur. Maecenas semper accumsan dictum. Nam augue lectus, consequat ut sapien id, sollicitudin auctor sapien. Curabitur libero sapien, egestas eu mauris ut, aliquet pharetra orci. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam sapien tortor, viverra eget leo vel, pellentesque iaculis velit. Praesent vehicula, erat sit amet fermentum scelerisque, justo lorem gravida orci, ut euismod nisi nibh a dolor. Mauris in pellentesque sapien. Sed lobortis magna quam, in sodales ante pulvinar et. Pellentesque et rhoncus orci, in blandit elit.

<form>
Formulário
</form>

Desabilitando copia da página toda
Talvez se tiver a necessidade de prevenir a copia completa pode usar algo como:
function blockCopy(e) {
    e.preventDefault();
    e.stopPropagation();
}

function blockPressCtrlC(e) {
    if (e.ctrlKey && e.keyCode == 67) {
        blockCopy(e);
    }
}

document.addEventListener("copy", blockCopy, false);
document.addEventListener("beforecopy", blockCopy, false);

document.addEventListener("keydown", blockPressCtrlC, false);

Mas note isso não é infalível, apenas dificulta a copia do conteúdo, mas é possível ainda sim burlar.

Uma resposta relacionada com isto que pode ser útil é:

Proteção de Informações ao inspecionar o código através do navegador
Comment: O simples fato do elemento esta com `display:none;` faz com que ele não seja copiado? oO
Comment: Função anonima no `addEventListener`? quero ver você cancelar isso depois :D
Comment: @GuilhermeLautert quer dizer remover o evento? Mas tem algum objetivo pra remove-lo?
Comment: Isso depende de suas necessidade, eu só tava brincando.


Node-horseman with promises

Question: I'm trying to use promises inside horseman's evalute function. A simple example:
var Horseman = require('node-horseman');
var horseman = new Horseman();

horseman
  .open('http://google.com')
  .evaluate(function() {
    var promise = new Promise(function(resolve,reject){
      setTimeout(function(){resolve("Hello")},2000);
    });
    return promise;
  })
  .then(function(mesg){
    console.log(mesg);
  })
  .close();

I'm expecting the resolved value "Hello" to be passed to the next then function as the mesg argument when the promise is fulfilled, but instead I'm getting the following error:
Unhandled rejection fn
evaluate

global code
evaluateJavaScript@[native code]
evaluate@phantomjs://platform/webpage.js:390:39

Am I missing something?
Answer: PhantomJS has a very limited communications channel into and out of the browser. It has to send everything over basically a text-based path so it has to serialize what it sends in, and deserialize what comes out.
You can send "in" a function to be executed inside the browser's JS context because it's just converting it to a string. But what comes out can't be a promise because you lose the whole object wrapped around it. You can only return scalar values and primitive objects. If you look at the example provided in the module's home page, a primitive object is returned:
.evaluate( function(selector){
  // Trimmed for clarity...
  return {
    height : $( selector ).height(),
    width : $( selector ).width()
  }
}, '.thumbimage')

Node-horseman does have a pattern for async operations, but instead of returning a promise you must call a done() function that you ask to be given as a param to your function:
.evaluate(function(ms, done){
    var start = Date.now();
    setTimeout(function() {
        done(null, Date.now() - start);
    }, ms);
}, 100)

The return value from evaluate() is already a promise. You don't need to return one of your own. You just need to do any async work you want, and then call done() with your result value.

Node-horseman with promises


Find the maximum and minimum values absolute value

Question: So I am asked to find the maximum and minimum  of the function:
$$f(x)=(2+|x|)\sqrt{2+(2-x)^2}$$ on the interval $[-1,2]$
So I took the derivative and got:
$$\frac{ 2((x-2)|x|+x(x^2-3x+3)) }{|x|\sqrt{(2+(2-x)^2}}$$
which is in fact correct. So I set it equal to zero and try to find the critical points.
The problem is though, how do I solve for $2((x-2)|x|+x(x^2-3x+3))=0$??? I certainly can't see a way...
Comment: $|x|=\begin{cases} x\quad \text{if}\quad x\geq 0\\-x\quad \text{if}\quad x<0\end{cases}$
Comment: Beware: Numerator and denominator are both zero for $x=0$
Comment: Yep! I got 0 as one of them. Another solution is 1 though. How would I solve for that? :/ .
Comment: Why is $f$ differentiable? I can't see why it should be at $0$.
Comment: Because it's also where the denominator is undefined. So the denominator is undefined when x=0 hence it is also a critical point.
Comment: @SubhashisChakraborty. While not a proof, it doesn't hurt to graph the function. It can be very helpful
Comment: It is not differentiable at $x=0$.
Comment: It's okay, I got it!
Comment: Yes but it is undefined at that point so it is still a critical point!
Answer: Note that $f$ is positive on the entire domain of consideration.
Note that $f$ is non increasing on $[-1,0]$ since it is the product of two
non increasing, non negative functions. Hence we need only consider the points
$\{-1,0\}$ in the context of $x \in [-1,0]$.
For $x\ge 0$, we have
$f(x) = (x+2)\sqrt{x^2-4x+6}$, and $f'(x) = 2{(x-1)^2 \over \sqrt{x^2-4x+6} }$.
In particular, $f'(x) \ge 0$ for $x \in [0,2]$. Hence we need only consider
the points $\{0,2\}$.
Since $f$ is non increasing on $[-1,0]$ and increasing on $[0,2]$, the
maximum value must occur at either $\{-1,2\}$, evaluating shows that the maximum occurs at $f(-1) = 3 \sqrt{11}$. Similarly, the minimum must occur
at $f(0) = 2 \sqrt{6}$.


App quit after throwing an error on console - win death . !

Question: HI All.
I m working on a app that is exactly like a radio when i start a channel and go to the home screen radio is running successfully but some time it throws an error and crash and quit the application i got the error on my console .What kind of this error is and how can we remove this error.
INFO/WindowManager(90): WIN DEATH: Window{448521c0 com.cls.radio.app.screen/com.cls.radio.app.screen.StationListScreen paused=false}
Any code or idea will be helpful.
Thanks
Comment: looks like android is killing your app by way of garbage cleaning..
Answer: That's when the app (alone or together with others running) is using too much resources.


Riemann sum of Lipschitz function

Question:
Let $f : [0,1] →\mathbb{R}$ be a Lipschitz function such that $|f(x) − f(y)| ≤ λ|x − y|$ for all $x,y ∈ [0,1]$. Let  ̇$P$ be a tagged
partition of $[0,1]$ such that $||  ̇P||< \frac{1}{m}$ for some $m ∈\mathbb{N}$. Show that $|
\int_0^1f(x)dx - S(f,P)| < \frac{λ}{m}$.

I know I have to use the upper and lower Riemann sums here and somehow relate it to the Lipschitz definition. I have no clue where to start for that, though.
Answer: Hint:
Let $0=x_0<x_1\dots<x_n=1$ be a partition of $[0,1]$ with $\|P\|<\frac{1}{M}$. Write
\begin{align}
\left|\int_0^1 f-S(f,P) \right|&=\left|\sum_{k=0}^{n-1}\int_{x_k}^{x_{k+1}} f-S(f,P) \right| \\
&\stackrel{MVT}{=}\left|\sum_{k=0}^{n-1}f(x_k^*)(x_{k+1}-x_k)-S(f,P) \right|
\end{align}

Riemann sum of Lipschitz function


Inverse AR/MA roots and near non-stationarity/invertibility

Question: I computed an algorithm to find out the best ARMA (p,q) model via minimisation of the AIC. It turned out ARMA(5,5) is the best one with AIC=-2693.12.
However, the inverse roots of the AR and MA characteristic polynomial are the following:

Many of them looks very close to the unitary circle. That makes me feel like I'm in presence of a near non-stationarity & invertibility series (isn't it?). However ACF and PACF show the model as great for capturing autocorrelation in the residuals.
If I use auto.arima() in $R$ to find the best p,q instead, it turns out the best model is a simple AR(1). The AIC worst off to -2687.08.

By looking on internet I figured out that auto.arima() looks yes at AIC (if you specify so) but also at "numerical stability" in returning the "optimal" orders p,q.
What fools me is:

What does it mean? What are the implication for a statistical analysis?

,and consequently:

Which order should I use? Should I trade some AIC "points" in exchange of much numerical stability proposed by auto.arima()?
Would the previous answer change in case the scope of my ARMA model is forecasting or testing?

Here the dput() of my dataset for replicability.
Comment: The models you are suggesting are preposterous to someone who has been bulding minimally sufficient arima models for 50+ years. Please review https://stats.stackexchange.com/questions/319732/i-have-correlogram-acf-and-pacf-below-for-a-temperature-time-series-can-i-say-i/319745#319745 for an extended discussion of the faults of auto.arima primarily due the lack of treatment of deterministic structure
Comment: Analysis of data can efficiently suggest a possible hybrid model. If you wish to post your data , i will try and elaborate.
Comment: @ IrishStat thank your for your comment! Cannot copy the `dput()` directly on the post as my post would exceed the character limits. However, you can access it at the end of the (now edited) post.
Comment: how many observations are there . What is the frequency of measurement  ? scale your data way down and post a  1 column csv
Comment: @IrishStat 1488 daily observations.
Answer: I took your 1488 daily values from this economic time series and found an adequate/sufficient model. There are a number of outliers ( only a few shown here ) and the best forecast is a simple constant. . The Actual/Fit and Forecast are here  . The residual ACF suggesting sufficiency is here  .
Very far from correct but directionally ok is https://people.duke.edu/~rnau/arimrule.htm detailing the way an ARIMA model is formed (IF and only if there are no deterministic structure ( such as pulses) in the data. The acf of the original series is here  . Your "problem" is that you think auto.arima is a model identification tool  ... not so much ! .. It can be under very rare circumstances such as if the following 6 characteristics are true for your data .1 ) no pulses in the the data  2) no step/level shifts in the data  .... 3) no seasonal pulses in the data .... 4) no deterministic time trends in the data ... 5) parameters are constant over time .....6 )error variance is constant over time . Unfortunately your data doesn't match the required profile .
Comment: How did you found out the best model is a simple costant? Did you look at AIC?