Views exposed filter auto submit loses focus

Question: I have setup a view with ajax enabled. I have also added an exposed filter with autosubmit on so that as the user types in the text field the view will automatically update. The problem is that when the form autosubmits, the text box loses focus.
I have read that I need to separate the exposed form into a separate block, however this option is not available as my view is a block itself.
Any help would be appreciated, as it is a real pain having to keep clicking in the text box to keep typing.
Comment: This is a known problem: http://drupal.org/node/1217948 and is not that easy to fix
Comment: Can you do a exposed_filter_alter to manually change values and add your own javascript?
Answer: You really only have one choice here.
One way or another you need to (re)move your form out of the views.
The reason why the exposed filter looses focus is because the entire view is being replaced on ajax. If your form is being render by the same view there's just no way you're going to be able to keep focus.
Solutions (All involve moving your form out of the view):

Use views "exposed filters in a block" Recommended option
Set up a custom block to render the expose filter form (Require writing code)

See views_plugin_exposed_form::render_exposed_form for an example on how to do this.

Below are some options you should avoid, but mentioning here as a last resort.

You can always just "hack" your views.tpl.php to move your expose form out of the main view div. I use "hack" here because this is not a nice option. This will place the form right above your view, but views JS code is only going to replace the view div. That being said the JS isn't going to just work. You'll need to fix that.
Then as a last resort you can always just use JS to move your form. This one is highly discourage. Because again the form is going to loose the JS behaviors after views updates the display
Comment: Just in case anyone else was wondering, I believe the solution recommended here by @ericduran to select Views' option to put "exposed form in block" is just available if the view is of a page (it's not available for a view as block).
Answer: Also a workaround for the core problem, but including this js in my theme solved this problem for me.


I have list in the array .,how do I show one of its element should come first based on condition in mapping?

Question: Here is the object.
 var arr = {
    firstValue:"xyz",
    content:[
      {
        "value": "abc",
        "checked": true
      },
      {
        "value": "xyz",
        "checked": false
      },
      {
        "value": "lmn",
        "checked": true
      }
    ]
    }

In this firstValue is xyz so while mapping xyz should come first like this:
 var arr = {
firstValue:"xyz",
content:[
  {
    "value": "xyz",
    "checked": true
  },
  {
    "value": "abc",
    "checked": false
  },
  {
    "value": "lmn",
    "checked": true
  }
]
}

How to achieve this by using javascript,
Thanks.
Comment: It's an array, I guess you sort it. How you determine the sort order is up to you. What have you tried?
Comment: Or find the item and splice it to the top.
Answer: const arr = {
  firstValue: "xyz",
  content: [
    {
      value: "abc",
      checked: true,
    },
    {
      value: "xyz",
      checked: false,
    },
    {
      value: "lmn",
      checked: true,
    },
  ],
};
const index = arr.content.findIndex((item) => item.value === arr.firstValue);
if (index !== -1 && index !== 0) {
  arr.content.unshift(...arr.content.splice(index, 1));
}

Answer: It looks like you just want to sort the matching object to the top.
You can either use Array#sort(), though this will mutate the original object, so if it's in state you'll want to clone before sorting.

var arr = { firstValue: 'xyz', content: [ { value: 'abc', checked: true, }, { value: 'xyz', checked: false, }, { value: 'lmn', checked: true, }, ], };

const sorted = [...arr.content].sort((a, b) =>
  (b.value === arr.firstValue) - (a.value === arr.firstValue));

console.log(sorted);

Otherwise you can find the index of the object using Array#findIndex() and then use Array#splice() and Array#unshift() to move it to the beginning of the array. Here declaring a utility function and returning a copy of the passed array to avoid mutation.

var arr = { firstValue: 'xyz', content: [ { value: 'abc', checked: true, }, { value: 'xyz', checked: false, }, { value: 'lmn', checked: true, }, ], };

const orderFirst = (a) => {
  const res = [...a];
  const i = a.findIndex((o) => o.value === arr.firstValue);

  if (i && i !== -1) {
    res.unshift(res.splice(i, 1));
  }

  return res;
};

const sorted = orderFirst(arr.content);

console.log(sorted);


preg_match_all(): Delimiter must not be alphanumeric or backslash

Question: class Check {

    private static $Data;
    private static $Format;

    public static function Email($Email) {
        self::$Data = (string) $Email;
        self::$Format = "/[a-z0-9_\.\-]+@[a-z0-9_\.\-]*[a-z0-9_\.\-]+\.[a-z]{2,4}$/";

        if (preg_match(self::$Data, self::$Format)):
            return true;
        else:
            return false;
        endif;
    }

}

Acima meu código onde uso, o pre_match está dando erro:
Delimiter must not be alphanumeric or backslash

Pesquisei sobre mas nao consegui resolver.
O que estou fazendo errado ?
Comment: quais os dados de entrada?
Comment: email email@example.com , no caso o $Data esta recebendo esse email.
Comment: não se pode te ajudar, mas para validação de email eu costumo usar filter_input($string, FILTER_VALIDATE_EMAIL)
Comment: obrigado, consegui com tua dica ;)
Answer: Você está invertendo os parâmetros, uma lida no manual evita estes problemas.

preg_match ( string $pattern , string $subject )

A ordem correta seria esta:
 preg_match(self::$Format,self::$Data)

Aproveitando, isso aqui não faz muito sentido:
if (preg_match(self::$Data, self::$Format)):
    return true;
else:
    return false;
endif;

Se é pra retornar true ou false já retorne o valor diretamente, não existe razão pra if.
class Check {

    private static $Data;
    private static $Format;

    public static function Email($Email) {
        self::$Data = (string) $Email;
        self::$Format = "/[a-z0-9_\.\-]+@[a-z0-9_\.\-]*[a-z0-9_\.\-]+\.[a-z]{2,4}$/";

        return preg_match(self::$Format,self::$Data);
    }
}

Vale dizer ainda que as variáveis neste caso poderiam ser locais (só do método, e não da classe).
Conforme o Rafael mencionou nos comentários, se preferir o PHP já tem uma alternativa pronta para isso, o que é recomendável no lugar do seu RegEx (que por sinal não respeita o padrão válido de emails, negando coisa que não deve).
filter_input($string, FILTER_VALIDATE_EMAIL)

Comment: @DiegoGo acostume sempre a ver o manual do PHP, que evita esses problemas e vc não fica sofrendo dependendo de dar sorte de alguém responder na hora que você estiver precisando. Como o PHP tem uma ordem muito estranha de parâmetros, até quem tem experiência precisa "tirar a teima" de vez em quando. Estamos aqui pra ajudar, mas quanto mais você conseguir resolver essas coisas por conta própria, menos tempo fica preso em problemas desse tipo. Mas quando precisar, pergunte, claro.
Comment: Bacco Obrigado pela aula, realmente tu esta certo, não me atentei a leitura básica. E o meu if que não havia necessidade, cara,obrigado mesmo. :D


Bar lines in lilypond

Question: I want to control on the locations of the bar lines (yellow marked).
I want to move them to any other places when ever I would like to do. its seems to me that bar lines pop up automaticaly...
For example, see part of my code:
\relative {
  \key g \minor
  \numericTimeSignature
  \time 4/4
  \fdo g'8 \ \bar "|"
  \fre d'4  \fre d4  \fsol g4, \fdo c4 \fdo c4 \fsol g4 \bar "|"
  \fsi b2.
}

Comment: Looking at your Lilypond code, it looks like what you want isn't 4/4 time (unlike your code), but 6/4 or 3/2 time.
Answer: The bar lines pop up wherever the time signature tells them to. If you want them in the correct places, use the right \time signature for your piece, and don't forget to use \partial if you have an upbeat.
Comment: The Lilypond code in the OP uses 4/4 time, though. How come the highlighted barlines aren't in the usual places for 4/4 time (pickup or no pickup)?
Answer: As Kilian says, you'll need to change the time signature whenever you do so.
However, you can suppress the display of the time signature by including \with { \omit TimeSignature } in your Staff. For example, the following snippet from the LilyPond Snippet Repository shows how this can be done:
\new Staff \with { \omit TimeSignature } {
  \relative c'' {
    \time 3/4
    c4 c c
    \time 2/4
    c4 c
    \time 3/4
    c4 c c
    \time 2/4
    c4 c
    \time 5/4
    c4 c c c c
    \time 2/4
    c4 c
  }
}

Comment: ok , relating to part of my code, how do I need to re-write it?
Comment: Can you edit your question to include your full original code? That way I can more clearly shows what needs to happen.
Comment: \relative {
  \key g \minor
  \numericTimeSignature
  \time 4/4
  \fdo g'8 \ \bar "|"
  \fre d'4  \fre d4  \fsol g4, \fdo c4 \fdo c4 \fsol g4 \bar "|"
  \fsi b2.
}
Comment: I can not reload the notes in comment mode but the image added at the begining of my question. Thank a lot.
Comment: please ignore to \fdo..\fre, those are woodwing functions...
Answer: As Kilian and Richard both say, you need to do two things:

Change the time signature to 6/4 (or 3/2)
Add a "partial" (because of the upbeat). This is an eighth note, so \partial 8

With these changes, Lilypond will automatically add bar lines in the correct places. You can add a "|" symbol to indicate that you think a bar line should go here (and Lilypond will check that it agrees).
Here's your example with these changes (omitting the fingering diagrams):
\version "2.22.1" % It should work with other versions too!
\language "english"

\relative {
  \key g \minor
  \numericTimeSignature
  \time 6/4 % changed!
  \partial 8 % added!
  g'8 |
  d'4 d4 g,4 c4 c4 g4 |
  b2.
}

Result:

Bar lines in lilypond


What is the "pdm" stat in the "rms" R package?

Question: I am building an ordinal logistic regression model (ORM). In order to fit my ORM model, I am using the 'orm' function of 'rms' package from R (http://cran.r-project.org/web/packages/rms/rms.pdf).
Now I am trying to assess the goodness of fit of my model. By reading the R documentation, I can see the following statement in the 'stat' property of the 'orm' object (p. 98):

(...) Nagelkerke $R^2$ index, the $g$-index, $gr$ (the $g$-index on the odds
  ratio scale), and pdm (the mean absolute difference between 0.5 and
  the predicted probability that $Y\geq q$  the marginal median) (...).

I don't have enough background to understand the short description of the pdm measure. But when I try to do more research on this measure, I am not able to find related material (e.g. I've been finding "prescription drug misuse"). In summary, my question is:
Would you know if the 'pdm' measure has some synonym which is more widely used? Or can you provide some references where I can study the pdm metric?
Comment: I've tried to look it up possibly it is related to D. R. Cox "Two further applications of a model for binary regression" Biometrika , Dec., 1958, Vol. 45, No. 3/4 (Dec., 1958), pp. 562-565 but I could not make it work.
Comment: @FrankHarrell you wrote that package not?
Answer: I don't know yet about the background of the statistic but below is an illustration how it is computed.
$$pdm = \frac{1}{n} \sum_{k=1}^n \left| \hat{P}(Y \geq median|X_k) - 0.5 \right| $$
It is an indication of how much the conditional predicted probability varies around the point of the marginal median.
<code>library(rms)

###
###  generate some data according to a logit model
###
set.seed(1)
n = 10^2
k = 5
x = runif(n,-2,2)  # predictor
noise = rnorm(n,0,1)   # noise
y = x + noise        # latent variable
bounds = seq(-2,2,1) # values to be predicted
z = as.numeric(sapply(y, FUN = function(yi) sum(bounds<yi)))  # ordinal variable

### compute ordinal regression
mod = orm(z ~ x, family =probit)

### marginal median of the data
median = mod$stats[3]
### predictions hat p(y>=median | x) for all x in the data
pr = coef(mod)[median] + coef(mod)[k+1]*x
prs = pnorm(pr)
### mean absolute deviation  ... 0.3308655
mean(abs(prs-0.5))
### value from the function  ... 0.3308655
mod$stats[14]
</code>
Comment: That's it.  The idea behind this probability is that if the model has not discrimination ability ($R^{2} = D_{xy} = \rho = 0$), it will estimate the same probability of $Y\geq y$ for every observation.  And depending on how one handles ties the probability of exceeding the sample median will be 0.5 for all $X$.  How far from 0.5 it actually predicts across $X$ is a measure of discrimination ability.  As an aside, I'm adding some new pseudo $R^2$ measures to the `rms` package as described [here](https://hbiostat.org/bib/r2.html).
Answer: This link: https://www.rdocumentation.org/packages/rms/versions/5.1-0/topics/validate.lrm
describes pdm as a "new" metric, which would imply that is has not been used before.
Comment: If this is new why would a question about it have come up in 2014?
Comment: I meant new in the sense of not previously reported in the literature, or used outside of the rms package. I myself am not sure, I am just going on based on what they are saying in the package it sounds like this is a new metric they came up with, as opposed to a standard metric that they gave a different name to.


Calculating the expectation in limiting case

Question: I am interested in calculating the expectation of the following function:
$\log(1+xa)$ with respect to the random variable $x$, with distribution $f(x),x \in [0,\frac{1}{a}]$ when $a\to \infty$.
thus,
$\lim_{a \to \infty}\mathbb E[\log(1+ax)] = \lim_{a \to \infty}\int_{0}^{\frac{1}{a}} \log(1+xa)f(x)dx$
I can take the function inside the integral as $g(x,a)$. Thus I need to find the area of the curve $g(x,a)$ when a is very small. As per my understanding, as $\frac{1}{a} \to 0$, the area will become the realization of the function $g(x,a)$ at $0$(The area will become infinitesimally small, and thus the realization of the function at a given point).
thus $\lim_{a \to \infty}\mathbb E[\log(1+ax)] = g(0,a) = f(0)$. But this is not correct as the answer is given as $\mathbb P(x\leq \frac{1}{a})$.
Where did I go wrong in my understanding? How to achieve the correct answer?
Thank You!
Edit: If it helps then the distribution of $f(x) \sim \exp(1)$
Comment: Thank you, I'll do one thing. I'm still not fully sure about where $P(X \leq \frac 1a)$ comes from, but I can understand and give you an explanation for $(i)$ and $(ii)$.
Comment: It definitely helps that the distribution of $f$ is specified as the exponential distribution. Note that if $f$ changes, the answer to this question is expected to change as well.
Comment: @SarveshRavichandranIyer I was under the impression that the way I am solving will give the answer $f(0)$ under all cases, but the final answer will obviously depend on what $f(.)$ is.
Comment: Actually, a question because I'm a little confused. So you are trying to calculate $\int_{0}^{\frac 1a} \log(1+xa) e^{-x}dx$ , if I'm not wrong? But then, you're taking the limit as $a$ is going to infinity, so how can the final answer depend upon $a$?
Comment: @SarveshRavichandranIyer. In that case as we know the distribution of $x$ as $\exp(1)$ I can simply the final answer as $1/a = 0$. Actually, this is a part of a larger calculation and there are other expectation terms also. I am stuck in this particular expectation.
Comment: Ok, so let me try to understand this carefully. Actually, you're concerned with the limit $\lim_{a \to \infty} \int_0^{\frac 1a} \log(1+ax)f(x)dx$, and it turns out that you guessed the answer should be $f(0)$, when in fact for $f(x) = e^{-x}$ you're getting $0$ which isn't equal to $f(0)$ so you're asking (i) what's going wrong here and (ii) what is the answer to $\lim_{a \to \infty} \int_0^{\frac 1a} \log(1+ax)f(x)dx$ for a large class of $f$, is that right? In any case, I can't see how the answer is $P(X \leq \frac 1a)$ which depends on $a$.
Comment: @SarveshRavichandranIyer. (i) and (ii) are correct. Also, it is known that $P(X \leq 1/a)$ is the answer. Let me explain why it does not depend on a: As X is exponential I can write $P(X \leq 1/a)$ as $\int_0^{1/a} e^{-x} = 1-e^{-1/a}$ and for large value of a this will simplify to $1-(1-\frac{1}{a}) = 1/a = 0$
Comment: Sure, take your time. I'll register my interest and will answer this question once it's fully clear to me.
Comment: @SarveshRavichandranIyer, please check the updated comment.
Answer: Question : Given $f: [0,\infty) \to [0,\infty)$, what can be the nature of the limit $$
\lim_{a \to \infty} \int_0^\frac 1a \log(1+ax)f(x)dx
$$Must it exist? What can its values be?

Note that the limit and integral cannot be exchanged : the dominated convergence theorem doesn't apply (and using a partial converse of DCT from Brezis, it's possible to prove that the limit and integral exchange actually lead to different answers in some cases) because the function $\log(1+ax)f(x)$ is unbounded in $a$. This is why the argument presented in the original post fails.
If this could have been done, then the answer would have been expected as $f(0)$, as shown in the original post.

The usual way to get rid of this issue is to do an integration-by-parts. Let $g$ be any anti-derivative of $f$(assume that $f$ is integrable on $[0,1]$). Note that we haven't specified $g$ at a point : so we know that the answer must not involve the value of $g$ at any point (but maybe a difference at two points). We have $$
\int_0^\frac 1a \log(1+ax)f(x)dx = \left[\log(1+ax)g(x)\right]^{\frac 1a}_{0} - \int_0^\frac 1a g(x) \frac{a}{1+ax} dx \\ = \ln(2)g\left(\frac 1a\right) - \int_0^{\frac 1a} \frac{ag(x)}{1+ax}dx
$$
where we can now focus on the RHS. Now, $g$ is continuous so the first term will go to $\log(2)g(0)$ as $a \to \infty$. The second term can be tackled using dominated convergence : which leads to the rigorous answer. For doing this, we write $$
\int_0^{\frac 1a} \frac{ag(x)}{1+ax}dx = \int_0^{1} \frac{g(x)}{\frac 1a+x}1_{[0,\frac 1a]}(x) dx
$$
The pointwise limit of this quantity is $0$. Note that it is dominated by $\frac{g(x)}{x}$ for all $a$. However, if $\frac{g(x)}{x}$ is integrable on $[0,1]$ then we must have $g(0)=0$ (which shows that we have lost arbitrariness of the anti-derivative). Thus, the following result holds :

If $f$ is an integrable function such that $\frac{g(x)}{x}$ is integrable on $[0,1]$ where $g$ is the integral of $f$ with $g(0)=0$, then $\lim_{a \to \infty} \int_0^{\frac 1a} \log(1+ax)f(x)dx = 0$.

This would capture many, many functions like constant $f$, or $f(x) = \frac 1{\sqrt x}$ and so on. It holds for $f(x) = e^{-x}$ as well : the antiderivative $g(x) = 1-e^{-x}$ is adjusted to give $g(0)=0$ , and $\frac{1-e^{-x}}{x}$ is integrable on $[0,1]$.
You can use the above to prove the following :

Suppose that $f$ is measurable and satisfies $\lim_{x \to 0} xf(x) = 0$. Then, $\lim_{a \to \infty} \int_0^{\frac 1a} \log(1+ax)f(x)dx = 0$.

However, here's a question : what happens if $f$ is on the edge of integrability, like say $f(x) = \frac 1x$, which is not integrable on $(0,1)$?
The answer for $f(x) = \frac 1x$ is itself quite interesting : the integral $$
\int_{0}^{\frac 1a} \frac{\log(1+ax)}{x}dx
$$
doesn't depend upon the value of $a$. To see this, perform the change of variable $Cu = x$ where $C$ is any positive constant, and get $$
\int_{0}^{\frac {1}{Ca}} \frac{\log(1+Cau)}{u/C}du/C = \int_{0}^{\frac {1}{b}} \frac{\log(1+bu)}{u}du
$$
where $b=Ca$. Since $C$ was arbitrary (positive), the answer is independent of $a$, and it will therefore equal the value for $a=1$, which is $\int_0^1 \frac{\log(1+x)}{x}dx$, some positive quantity. (Turns out, it's $\pi^2/12$). With this and a simple use of the comparison principle, one can prove the following :

Suppose that there exist constants $C_1,C_2$ such that $C_1<xf(x)<C_2$ for all $x \in [0,\delta)$ for some $\delta>0$. Then, $\lim_{a \to 0} \int_0^{\frac 1a} \log(1+ax)f(x)dx$ exists and is some positive constant.

What about $f$ going even slower? Once again, one can use a comparison principle (with the previous result) to see that if $xf(x)$ is unbounded in a neighborhood of $0$, then $\lim_{a \to 0} \int_0^{\frac 1a} \log(1+ax)f(x)dx$ is positive infinite.
Therefore, to summarize, the limit $\lim_{a \to \infty} \int_{0}^{\frac 1a} \log(1+ax)f(x)dx$

is $0$ if $\lim_{x \to 0} xf(x) = 0$.

is a positive finite quantity if $C_1<xf(x)<C_2$ for positive $C_1,C_2$ in a neighborhood of $0$.

is $+\infty$ if $xf(x)$ is unbounded in a neighborhood of $0$.
Answer: Define
$$\phi_a(x)=\mathbb{1}_{(0,1/a]}\log(1+ax), \qquad a>0$$
if $0<x<1/a$ then $1<1+ax<2$ and so, $0<\phi_a(x)\leq \log 2$ in $(0,1/a]$. Consequently
$|\phi_a|\leq\log2\mathbb{1}_{(0,1/a]}\leq\log2$
Notice $\lim_{a\rightarrow0}\phi_a(x)=0$ for all $x>0$.
The rest is dominated convergence:
$$\lim_{a\rightarrow\infty}E[\phi_a(X)]=E[\lim_{a\rightarrow\infty}\phi_a(X)]=0$$
or if dominated convergence is not within your grasp,
$$ E[\phi_a(X)]\leq\log2 P[X\leq1/a]\xrightarrow{a\rightarrow\infty}\log2 P[X=0]=0$$
since $X$ has a distribution that admits density.


What is the best way to work with large data sets in the Google App Engine Java Development server?

Question: I am developing a Java application using Google App Engine that depends on a largish dataset to be present. Without getting into specifics of my application, I'll just state that working with a small subset of the data is simply not practical. Unfortunately, at the time of this writing, the Google App Engine for Java development server stores the entire datastore in memory. According to Ikai Lan:

The development server datastore stub is an in memory Map that is persisted
  to disk.

I simply cannot import my entire dataset into the development datastore without running into memory problems. Once the application is pushed into Google's cloud and uses BigTable, there is no issue. But deployment to the cloud takes a long time making development cycles kind of painful. So developing this way is not practical.
I've noticed the Google App Engine for Python development server has an option to use SQLite as the backend datastore which I presume would solve my problem.

dev_appserver.py --use_sqlite

But the Java development server includes no such option (at least not documented). What is the best way to get a large dataset working with the Google App Engine Java development server?
Comment: You can try for hadoop, it will help you to handle large datasets.
Answer: There's no magic solution - the only datastore stub for the Java API, currently, is an in-memory one. Short of implementing your own disk-based stub, your only options are to find a way to work with a subset of data for testing, or do your development on appspot.
Comment: Thanks for taking the time to answer. Unfortunately, working with a subset of data is not always practical. Imagine for example a spell-check feature based on an incomplete dictionary. And developing on appspot means constantly deploying which is a slow multi-step process. That makes it kind of painful. Suppose I did want to implement my own disk based (maybe database driven) stub. How would I go about doing that? What hooks are available for me to plug in my implementation? If I get it working I would open source it on github.
Comment: @Asaph You could easily test a spell-check on a small dictionary - just select test cases that suit the data. If you can't pare your dataset down, you're going to have extreme trouble testing it. Building your own would require understanding the innards of the dev-appserver (source not released yet, but on its way), specifically the api_stub_map.
Comment: @Nick Johnson: Ok, perhaps spell-check is not a perfect example. It's not my actual use case; just something I contrived to try to illustrate my point. And I only pondered building my own disk-based stub because I thought you were suggesting it might be possible in your answer. Are there any plans to offer more datastore stub options in the Java dev server similar to what's available in the Python dev server?
Comment: @Asaph It's certainly possible for you to do it, but it would be a significant undertaking. I'm just suggesting that a better alternative is probably to find a way to work with a subset of data, since having work with a large dataset impacts testability as well.
Comment: @Nick Johnson: Please don't misunderstand me. I'm not using large data-sets in unit tests. I'm using `LocalServiceTestHelper` with `LocalDatastoreServiceTestConfig` as documented [here](http://code.google.com/appengine/docs/java/tools/localunittesting.html#Writing_Datastore_and_Memcache_Tests). My unit tests basically start with a blank datastore and insert a few rows of data in the `setUp` method before testing. So they run fast and use little memory. I'm already working with a subset of the data. But as my UI design emerges, it's becoming difficult for me to make informed UI decisions.
Comment: @Nick Johnson: +1. Thanks for your super helpful comments. I'm going to leave the question open for a while in case some enterprising programmer decides to tackle the disk-based implementation for the Java dev server. If Google decides to provide a solution, please update your answer. BTW: Great job on App Engine overall. We love it. Keep up the great work. :)
Answer: I've been using the mapper api to import data from the blobstore, as described by Ikai Lan in this blog entry - http://ikaisays.com/2010/08/11/using-the-app-engine-mapper-for-bulk-data-import/.
I've found it to be much faster and more stable than using the remote api bulkloader - especially when loading medium sized datasets (100k entities) into the local datastore.


Criação de JSON com numeral

Question: Gostaria de criar um JSON assim (essa é apenas uma parte), crio sempre o objeto com o valor, porém não consigo fazer dessa forma utilizando o 200 como objeto:
path.get.response = new Object()
path.get.response.200 = new Object()

  "responses": {
      "200": {
        "description": "Operação realizada com sucesso!",
        "content": {
          "application/json": {
            "schema": {

....
Comment: `path.get.response.["200"] = new Object()`
Comment: @Isac interessante, dessa forma você converte 200 para string né? Só que depois para acessar esse objeto é só dessa forma também né? `path.get.response.["200"]`
Comment: Sim, no entanto enganei me e coloquei um `.` a mais. É apenas `path.get.response["200"]`
Comment: não funcionou dessa forma amigos...
Answer: É realmente necessário nomear o objeto como 200? Acredito que objetos não podem iniciar com número, hífen ou carácter especial. Talvez o ideal seria você chama-lo de status200 ou S200, enfim depende da sua criatividade.

Um nome de propriedade de um objeto pode ser qualquer string
  JavaScript válida, ou qualquer coisa que possa ser convertida em uma
  string, incluindo uma string vazia. No entanto, qualquer nome e
  propriedade que não é um identificador JavaScript válido (por exemplo,
  um nome de propriedade que tem um espaço ou um hífen, ou que começa
  com um número) só pode ser acessado(a) usando-se a notação de
  colchetes [...]

Você pode conferir mais nesse artigo - Trabalhando com Objetos
Comment: De acordo com o exemplo do json o nome do objeto deveria ser 200, pois é a definição do json, mas se não tem como fazer isso com objeto e valor, terei que achar outra alternativa.
Comment: A API te retorna o objeto assim ou esse JSON está local?
Comment: Eu estou gerando esse json, em cima de padroes existentes, estou fazendo um gerador, onde eu digito as informações e o gerador gera o json, mas acho que vou gerar com outro nome no lugar do 200 e ao gerar o arquivo eu renomeio para 200.
Comment: Entendi, tem a maneira que o @Isaac mostrou também `path.get.response.["200"]`, pelo que entendi dessa forma você converte o 200 para string, seria uma forma de contornar esse problema

Criação de JSON com numeral


clicking on a image submits the form by default?

Question: I didn't realize this, and just want to confirm.
If I have a html form, and an input tag of type image like:
<code><input type="image" name="blah" src="..." />
</code>
Clicking on the image will submit the form?
My use case is, I want to create a custom button for a submit button.
Comment: Seriously, did you try to do what you're asking about before asking?  Do you even recognize that you're now not only looking at documentation before asking questions, you're plain not even trying to run your own code before asking questions?
Answer: Yes, input-images will submit the form naturally. See: http://w3schools.com/tags/att_input_type.asp

image: Defines an image as a submit button


Properties for interior and closure in metric space.

Question: I found the some following properties for general topology and prove these.
But, I want to verify that the proofs are really true.
Let $(X,d)$ be metric space. Let $A$ be any subset of $X$.
Define the interior of $A$ ($ int(A) $) and the closure of $A$ ($ cl(A) $).
$ int(A):=\{x\in X~|~\exists r>0 ~~s.t~~\Bbb B (x,r) \subset A  \}  $.
$ cl(A):=\{x\in X~|~\forall r>0 ~~~\Bbb B (x,r) \cap A\ne \phi  \}  $.
Properties;
(1) $ (int(A))^c =cl(A^c) $
(2) $ (cl(A))^c =int(A^c) $.
Proof)
(1) $ (int(A))^c $ = $ \{x\in X~|~\forall r>0~~\Bbb B (x,r) \nsubseteq  A  \} $=$\{x\in X~|~\forall r>0~~\Bbb B (x,r) \cap A^c\ne \phi  \} $=$cl(A^c) $
(2)  $ (cl(A))^c = \{x\in X~|~\exists r>0 ~~s.t~~\Bbb B (x,r) \cap  A=\phi    \} $
 = $\{x\in X~|~\exists r>0 ~~s.t~~\Bbb B (x,r) \subset A^c \} $=$int(A^c) $
ㅁ.
Answer: Those properties do hold, and your proof looks fine (though perhaps it could benefit aesthetically from some explanation between steps).
Another useful thing to talk about is the boundary of a set.  That is, we define
$\partial A=\{x\in X:\forall r>0, \mathbb B(x,r)\cap A \neq \emptyset \text{ and } \mathbb B(x,r)\cap A^c \neq \emptyset \}$.
See if you can find an alternate definition related to closures, interiors, and exteriors.


Get list of applicable shipping methods

Question: I would like to get the list of applicable shipping methods for the current cart but dependent on the selected shipping profile. I'm programming a custom controller that should returns the list of available shipping methods and rates but, obviously, given a cart and a customer's shipping profile.
Has commerce shipping module any service or class that could I use to get this information? Perhaps, is there any other way to get it?
I've been reading source code from:
Commerce_Shipping\src\Plugin\Commerce\CheckoutPane\ShippingInformation.php
That is the CheckoutPane plugin that renders shipment options in checkout page. buildPaneForm method renders this form, and those lines of code should give me the list of shipping methods:
 public function buildPaneForm(array $pane_form, FormStateInterface $form_state, array &$complete_form) {
    ...
    $shipments = $this->order->get('shipments')->referencedEntities();
    ...
    foreach ($shipments as $index => $shipment) {
    ... // all shipments
    }

But I have tried it in my custom controller and It always returns empty array. I think there should be a prior process that populate this list, but I can't find it. I have tried appling a state transition but It doesn't work.
Answer: After some in-depth research on commerce modules, I have found a way to get them. This is an excerpt of the source code with the main instructions (controller class that returns JSON):
public function getShippingMethods(Request $request, $id) {

    $commerce_order = Order::load($id);
    // ... some checks to ensure it is a commerce order

    $shipments = $commerce_order->get('shipments')->referencedEntities();

    $options = [];
    foreach ($shipments as $shipment) {
      assert($shipment instanceof ShipmentInterface);
      $rates = $this->shipmentManager->calculateRates($shipment);
      foreach($rates as $rate) {
        $options[] = [ "id" => $rate->getShippingMethodId(),
                       "method" => $rate->getService()->getLabel(),
                       "import" => $rate->getAmount()->getNumber(),
                       "description" => $rate->getDescription()];
      }
    }

    // Return JSON with all shipping methods
    return new JsonResponse( $options );
  }

Hope It could help someone.

Get list of applicable shipping methods


Method of successive approximation to solve integral equation.

Question: Use the method of successive approximations to solve the integral equation:
$$u(t)=1+\int_{0}^{t}(t-s)u(s)ds \space,u(t_0)=1$$
Solution: I used the Picard's iteration:
$$u_1=1+\int_{t_0}^{t}(1+\int_{0}^t ((t-s)u(s)ds))ds$$
$$=1+(t-t_0)+\int_{t_0}^{t}\int_{0}^t ((t-s)u(s)ds))ds$$
Is this the right approach to the problem ? should I have to go to $n$ iterations ?
Comment: The successive approximations method goes like this: you pick some $u_0$, then express $u_1$ in terms of $u_0$, using the equation. Then you express $u_2$ in terms of $u_1$ and so on. You should have $u_0$ on the right hand side in your example
Comment: $u(t_0)=1=u_0$.
Comment: I understand. But you also have the unknown function $u$ on the right hand side. How are you going to find $u_1$? You need to explicitly replace $u$ on the rhs by $u_0$ and on the lhs by $u_1$. Maybe I'm just confused about what you have written here
Answer: Picard's iteration would be
$u_0= 1$,
$u_1= 1+ \int_0^t (s- t)(1)ds= 1+ \left[s^2/2- st\right]_0^t= 1- t^2/2$,
$u_2= 1+ \int_0^t (s- t)(1- s^2/2) ds= 1+ \int_0^t t- s+ ts^2/2- s^3/2 ds= 1+ \left[ts- s^2/2+ ts^3/6- s^4/8\right]_0^t= 1+ t^2/2+ t^4/24$,
etc.
Comment: But Shouldn't I take the whole $u_t$ in the integral part of Picard's iteration to get the double integral. I might have some lacking in understanding.
Answer: $$u(t)=1+\int_{0}^{t}(t-s)u(s)ds \tag 1$$
First differentiation with respect to $t$ :
$$u'=(t-t)u(t)+\int_{0}^{t}u(s)ds=\int_{0}^{t}u(s)ds$$
Second differentiation with respect to $t$ :
$$u''=u(t)$$
$$u(t)=c_1 e^t+c_2 e^{-t}$$
The constants are not arbitrary since $u(t)$ has to satisfy Eq.$1$.
$$c_1 e^t+c_2 e^{-t}=1+\int_{0}^{t}(t-s)\left( c_1 e^s+c_2 e^{-s}\right)ds$$
$$c_1 e^t+c_2 e^{-t}=1+c_1 e^t+c_2 e^{-t}-c_1t+c_2t-c_1-c_2$$
After simplification :
$$(c_2-c_1)t-c_1-c_2=0$$
Valid any $t$ implies
$$c_1=c_2=\frac12$$
$$u(t)=\frac12(e^t+e^{-t})=\cosh(t)$$
The solution of equation $(1)$ is $u(t)=\cosh(t)$
Specifying the condition $u(t_0)=1$ is excessive. Either $t_0=0$ and the condition is consistent with the integral equation, or $t_0\neq 0$ which is contradictory and there is no solution.
This might be the cause of the difficulty to use the method of successive approximations : Without a-priori setting $t_0=0$ no solution can be found.


Show that concatenation of two processes is measurable

Question: Let

$(E_n,\mathcal E_n)$ be a measurable space;
$(\Omega_n,\mathcal A_n,\operatorname P_n)$ be a probability space;
$(\mathcal F^n_t)_{t\ge0}$ be a filtration on $(\Omega_n,\mathcal A_n)$;
$\tau_n$ be an $(\mathcal F^n_t)_{t\ge0}$-stopping time on $(\Omega_n,\mathcal A_n)$
$(X^n_t)_{t\ge0}$ be an $(E_n,\mathcal E_n)$-valued $(\mathcal F^n_t)_{t\ge0}$-adapted (maybe we even need to assume progressive) process on $(\Omega_n,\mathcal A_n,\operatorname P_n)$

for $n=1,2$ with $E_1\cap E_2=\emptyset$, \begin{align}E&:=E_1\cup E_2;\\\mathcal E&:=\sigma(\mathcal E_1\cup\mathcal E_2);\\\Omega&:=\Omega_1\times\Omega_2\end{align} and $$X_t:\Omega\to E\;,\;\;\;(\omega_1,\omega_2)\mapsto\begin{cases}X^1_t(\omega)&\text{, if }t<\tau_1(\omega_1);\\ X^2_{t-\tau_1(\omega_1)}(\omega_2)&\text{, otherwise}\end{cases}$$ and $$\mathcal F_t:=\sigma(\mathcal F^1_t\cup\mathcal F^2_t)$$ for $t\ge0$.

How can we show that $(X_t)_{t\ge0}$ is $(\mathcal F_t)_{t\ge0}$-adapted?

Let $t\ge0$. We've clearly got $$F:=\{t<\tau_1\}\cap\Omega_2\in\mathcal F_t$$ and it is enough to show that $\left.X_t\right|_F$ and $\left.X_t\right|_{F^c}$ are $(\left.\mathcal F_t\right|_F,\mathcal E)$ and $(\left.\mathcal F_t\right|_{F^c},\mathcal E)$-measurable, respectively.
$X^1_t$ is clearly $(\mathcal F^1_t,\mathcal E)$-measurable. So, the only tricky part is the measurability on $F^c$.
I know that if $(Y_t)_{t\ge0}$ is an $(\mathcal G_t)_{t\ge0}$-progressive process and $\zeta$ is an $(\mathcal G_t)_{t\ge0}$-stopping time, then $Y_\zeta$ is $\mathcal G_\zeta$-measurable.
Answer: Let $$\tilde\tau_1(\omega):=\tau_1(\omega_1)$$ and $$\tilde X^n(\omega):=X^n(\omega_n)\;\;\;\text{for }n=1,2$$ for $\omega\in\Omega$. Since $\tau_1$ is an $(\mathcal F^1_t)_{t\ge0}$-stopping time, $$\{\tilde\tau_1\le t\}=\{\tau_1\le t\}\times\Omega_2\in\mathcal F_t\tag1$$ and \begin{align}\{\tilde X^1_t\in B\}&=\{X^1_t\in B\}\times\Omega_2\in\mathcal F_t;\tag2\\\{\tilde X^2_t\in B\}&=\Omega_1\times\{X^2_t\in B\}\in\mathcal F_t\tag3\end{align} for all $B\in\mathcal E$ for all $t\ge0$. Assuming $(X^2_t)_{t\ge0}$ is $(\mathcal F^1_t)_{t\ge0}$-progressive, we see that $\tilde X^2_{t-\tilde\tau_1\wedge t}$ is $\mathcal F_t$-measurable (by the same simple proof yielding that $Y_{\zeta\wedge t}$ is $\mathcal G_t$-measurable whenver $(Y_t)_{t\ge0}$ is $(\mathcal G_t)_{t\ge0}$-adapted and $\zeta$ is an $(\mathcal G_t)_{t\ge0}$-stopping time) for all $t\ge0$. Thus, $$\left\{\left.X_t\right|_{F^c}\in B\right\}=\left\{\tilde X^2_{t-\tilde\tau_1}\in B\right\}\cap F^c=\left\{\tilde X^2_{t-\tilde\tau_1\wedge t}\in B\right\}\cap\left\{\tilde\tau_1\le t\right\}\in\mathcal F_t\tag4$$ for all $B\in\mathcal E$ and $t\ge0$.


Second-Price Sealed-Bid Auction

Question: Consider 2 individuals who are interested in one indivisible object. Each player $i$ has a valuation $v_i > 0$ for the object. Assume $v_1 \geq v_2$. In this scenario, each player submits a bid $b_i \geq 0$. The individual with the highest bid wins the auction and obtains the object at a price equal to the second highest bid, in case of a tie, the object goes to player 1. The payoff functions for each player are given by
$$u_1(b_1, b_2) =  \left\{
  \begin{array}{l l}
    v_1 - b_2 & \quad \text{if $b_1 \geq b_2$}\\
    0 & \quad \text{if $b_1 < b_2$}
  \end{array} \right. $$ and
$$u_2(b_1, b_2) =  \left\{
  \begin{array}{l l}
    v_2 - b_1 & \quad \text{if $b_2 > b_1$}\\
    0 & \quad \text{if $b_2 \leq b_1$}
  \end{array} \right. $$
Am I correct if I say that in this case, the best reply functions are given by:
$$\beta_1(b_2) =  \left\{
  \begin{array}{l l}
    \{b_1 | b_1 \geq b_2\} & \quad \text{if $b_2 < v_1$}\\
   \{b_1 | b_1 < b_2\}  & \quad \text{if $b_2 > v_1$} \\
[0, \infty]  & \quad \text{if $b_2 = v_1$}
  \end{array} \right. $$
and
$$\beta_2(b_1) =  \left\{
  \begin{array}{l l}
    \{b_2 | b_2 > b_1\} & \quad \text{if $b_1 < v_2$}\\
   \{b_2 | b_2 \leq b_1\}  & \quad \text{if $b_1 > v_2$} \\
[0, \infty]  & \quad \text{if $b_1 = v_2$}
  \end{array} \right. $$
Answer: These look right to me.  Why did you doubt them?
Answer: The second price auction has a dominant strategy equilibrium in which each player bids their true valuation. Given the other player bids $b_{-i}$, consider the 3 possible cases for player $i$:

$v_i > b_{-i}$. Player $i$ should bid any $b_i \in (b_{-i}, \infty)$ and get payoff $v_i - b_{-i}$. Otherwise he gets $0$.
$v_i = b_{-i}$. Player $i$ is indifferent between any bids.
$v_i < b_{-i}$. Player $i$ should bid any $b_i \in (-\infty, b_{-i})$ and get payoff $0$. Otherwise he gets negative payoff.
Comment: This ignores player 1 wins in the case of a tie
Comment: How does that change the strategy? In case 2, player $i$, $i = 1$ or $2$, is indifferent between winning and not winning the bid regardless of this tie-breaking rule. In case 1, one might nitpick and say $b_1 \in [b_2, \infty)$ but the payoff is the same. Player $1$ pays the auctioneer $b_2$ even if he ties the bid.
Comment: The question asked if his best responses were right, they were. The best responses posted here have small differences as you pointed.

Second-Price Sealed-Bid Auction


SQL server placement

Question: I have a small user site of about 6 users.  They have two servers a SBS 2008 box and a server 2008 terminal server.  I would like to know where the best place is to install SQL server.  I know that it is not considered best practice to install SQL on the SBS box but isn't it a better solution to installing SQL on a terminal server box?
Thanks
Answer: I have a setup in the exact same scenario you're in.
I chose to install the SQL server on the same server as the Terminal Server, for a few reasons:

The SBS server was running near to capacity for RAM and HDD space
The TS had lots of excess RAM and HDD space
I had < 10 users

Ultimately it doesn't really matter though, as Microsoft do support installing SQL Server on the SBS box, or on a seperate box, so it comes down to where you have the most free resources. If your SBS server has a lot of spare resources, then install it there, same vice versa.
Answer: I'd really consider adding a third server for this.   Sql Server does much better when you can give it dedicated hardware.  And when you add that server you're right on the cusp of where you might also want to consider putting things inside virtual machines for redundancy and simplified management.  It's amazingly cool the first time you avoid a prolonged outage after a hardware failure because you just spin up a copy of your vhd on one of the other machines until the part arrives.
If you can't do this, you'll want to look at load and place your database on the server with the lowest load.  Especially consider disk io here.  Depending on how you use terminal server, my preference would be that the sbs server is likely your better choice, but you'll need to measure to know for sure.
And thinking about it, my favorite two-server option involves removing terminal services and giving that box primarily to sql server.  Then install hyper-V or vmware on both boxes and have a terminal services instance running in each, to split the load to which ever server can handle it.  You can set some users to connect to one terminal server and send the rest of your users to the other server.
Comment: Yes hyper-v would be better.  Sadly I am not the tech in charge of the servers.  I am a developer supporting our product.  I am trying to convince the tech to move SQL from the terminal server to the SBS box as we are running into issues with it setup like that.

SQL server placement


Telethon and Telebot libraries on Python. Do they work together?

Question: I have recently started writing a script based on telethon. The only thing it does is - retriving videos, photos from channel/group, saving them to folder.
So then I have decided to actually make a bot with command to run the telethon script which is going to download specific amount of files and send it back to user.
Sounds simple, however I ran to bunch of issues when tried to use telebot library to write a bot which then is going to run the telethon script. Makes no sense to post here errors I got, as I have tried too many things and got too many different errors.
So my main question is Can Telebot run Telethon function as one is sync and the other async?
If it can - are there any templates/example of such usage?
If it cannot - is there an example/possibility to write the bot with commands only on Telethon?
If non of the above is possible, are there any advices you can give me how can I implement the idea i described at first.
Thanks everyone <3
Comment: Telethon is one of the worst libraries I've ever used, it seems complete, is seems well documented but it is heavily buggy and very complex to use in multi threading environments.
Answer: See pyrogram library
I think you can write your own code with Pyrogram.
Pyrogram supports asyncio and you can combine this library with celery for automation task and also this library is very easy to understand.
Comment: Thanks for reply, but I'm really looking forward to hear some comments on my post.


Ho to explore the interaction between dependent variable and covariate?

Question: I'm conducting LMM by adding three independent variables (A, B, C) and a covariate (E) as the fixed effect, and the random intercept for each subject as the random effect in the model, as shown here below:
model <- lmer(RT_log ~ A * B * C * E + (1|Subject)

The results showed a significant three-way interaction between A, B, and E, of which the covariate E is a continuous variable (0-100). As we know that if E has only two or three levels, we can further conduct two LMM analyses, by adding two dependent variables (A, B) as the fixed effect and the same random effect as here above, for the two levels of E, separately. I was wondering how should I do if the E is a continuous variable in my case?
Thanks in advance for your any suggestion!
Comment: Why do you speak of dependent variables when they are on the right-hand side of the equation, and what is the difference between those and the "covariate" E?
Comment: Sorry, it should be "independent variables", rather than "dependent variables" as I stated before. The difference between them and the "covariate" E is that independent variables are what we manipulated in the experiment, each have two levels; whereas "covariable" E is the subjective score reported by participants, which is a continuous number. As I'm interested in whether this score interact with other independent variables, it is therefore included in the fixed effects as well.
Comment: @Qian: Please do these corrections/clarifications as an edit to the post, not only in comments. Not everybody reads comments ...
Answer: The distinction between independent variable and covariate is unimportant for the way you specify the model. It also doesn't matter whether E is continuous or categorical. If E is continuous then it makes the interpretation less cumbersome, however it is important to note that when a variable is involved in an interaction, then the estimates are conditional on the other variable(s) it is interacted with being zero. Since E has a scale of 0 to 100 it may make more sense to centre this variable around zero (which will correspond to 50 on the original scale)
Comment: Thanks a lot for your useful suggestions Robert, all variables are mean-centered. I have a further question, as I found the three-way interaction between A, B, and E, in order to explore the-way interaction pattern, can I divide subjects into two groups according to the median value of E, and then explore the interaction pattern between A and B in the two groups, separately?
Comment: You're welcome. If you include `E` in an interaction, then there is no point in splitting the dataset by `E`. Also, spliting the dataset will result in a loss of statistical power.


Is this time series stationary? What would be your approach to forecasting it?

Question: I've been working on the time series prediction of a signal and came across a small misunderstanding. The signal is depicted below:

Apparently it looks like there are several stationary local areas but overall it does not look stationary to me. However, ADF confirms the stationarity with the p-value to be almost 0.

ADF Statistic: -6.554090 p-value: 0.000000 Critical Values: 1%: -3.431
  5%: -2.862 10%: -2.567

When I draw a graph of first differences, it visually looks stationary: First differences
ACF of the original signal does not look promising:
I am now confused in the approach to predict, should I use the signal itself or first differences to build an ARIMA model? What would be your approach?
Thanks
Answer: ADF test is not test for stationary.
ADF test is for if data is unit root or not.
Your data seems to have several change point, I recommend to separate dataset.
Then check adf, auto correlation, seasonality.. again.
If it has enough auto correlation, you can use ARIMA model.
Comment: I am not well versed in topic, but what is the idea behind separating time series?
Comment: For example, in observing motor temperature, stopping motor and accelerating motor have completely different physical phenomenon. machine learning is hard to describe those at the same time. So separate dataset by the situation in especially univariate data.
Comment: How one would then make a prediction when the new data comes? The graph above is a CPU voltage which changes based on a server workload (I should have perhaps mentioned this in the question). The server is running continuously (except the maintenance days, but those are not in this time series). Workload basically comes on demand (let's say, someone needs to run an application on server).  I understand changes perhaps can't be predicted due to a random nature, but I am looking to capture a trend (let's say, change was predicted with a delay, but then prediction follows the trend)


Docker: PermissionError: [Errno 13] Permission denied

Question: I got this when I was running docker-compose
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=iramiro-api.jar&target=&ulimits=null&version=1": dial unix /var/run/docker.sock: connect: permission denied
Running docker compose
[6099] Failed to execute script docker-compose
Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 677, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1252, in request
  File "http/client.py", line 1298, in _send_request
  File "http/client.py", line 1247, in endheaders
  File "http/client.py", line 1026, in _send_output
  File "http/client.py", line 966, in send
  File "docker/transport/unixconn.py", line 43, in connect
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests/adapters.py", line 449, in send
  File "urllib3/connectionpool.py", line 727, in urlopen
  File "urllib3/util/retry.py", line 403, in increment
  File "urllib3/packages/six.py", line 734, in reraise
  File "urllib3/connectionpool.py", line 677, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1252, in request
  File "http/client.py", line 1298, in _send_request
  File "http/client.py", line 1247, in endheaders
  File "http/client.py", line 1026, in _send_output
  File "http/client.py", line 966, in send
  File "docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', PermissionError(13, 'Permission denied'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker/api/client.py", line 205, in _retrieve_server_version
  File "docker/api/daemon.py", line 181, in version
  File "docker/utils/decorators.py", line 46, in inner
  File "docker/api/client.py", line 228, in _get
  File "requests/sessions.py", line 543, in get
  File "requests/sessions.py", line 530, in request
  File "requests/sessions.py", line 643, in send
  File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', PermissionError(13, 'Permission denied'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 123, in perform_command
  File "compose/cli/command.py", line 69, in project_from_options
  File "compose/cli/command.py", line 132, in get_project
  File "compose/cli/docker_client.py", line 43, in get_client
  File "compose/cli/docker_client.py", line 170, in docker_client
  File "docker/api/client.py", line 188, in __init__
  File "docker/api/client.py", line 213, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 255
Finished: FAILURE

Answer: You  just need to run this command, it saved my day
sudo chmod 777 /var/run/docker.sock

Comment: Anyone who can access the Docker socket can pretty trivially root the entire system; think carefully before you do this.  The socket does not need to be executable, and making anything both executable and world-writable is usually a considered a no-no (so consider mode 0666 for world-writable or 0755 for executable but never 0777).
Comment: This answer works for Ubuntu
Comment: Brilliant answer
Comment: @DavidMaze 666 works, 755 does not the trick to get it running. Ubuntu 22.04
Comment: To reiterate: making the socket world-accessible at all opens your machine up to local compromise.  Think carefully before making the permissions more lenient than 0660.


Is this series convergent or divergent $\sum_{n=1}^\infty{\dfrac{\arctan{n}}{n^{2}+1}} $?

Question: I have problem with convergence $$\sum_{n=1}^\infty{\dfrac{\arctan{n}}{n^{2}+1}} $$
I should use integral test for convergence.
Thanks for help.
Comment: ...$arctan(n)<2$
Comment: Sorry, I forgot to mention that limit comparison test can't be used. It's too easy...
Answer: In integral test for convergence $\sum_{n=n_0}^\infty f(n)$ is convergent or divergent if the improper integral $\int_{1}^\infty f(x)\,dx$ is convergent or divergent.  But, You have to check three conditions:
1) functon $f(x)=\dfrac{arctan{x}}{x^{2}+1}$
 is positive for $ x \ge 1$
2) continuous  for $ x \ge 1$
3) and $f'(x)=\dfrac{1-2x\arctan{x}}{(x^2+1)^2}$ so $f(x)$ is monotone decreasing  for $ x \ge 1$ .
Now, we may use this test.
$$\int_{1}^\infty {\dfrac{arctan{x}}{x^{2}+1}}\,dx$$
$$lim_{\epsilon\to\infty}\int_{1}^\epsilon{\dfrac{arctan{x}}{x^{2}+1}\,dx}$$
$\arctan{x}=t$
$\dfrac1{x^2+1}\,dx=\,dt$
$$lim_{\epsilon\to\infty}\int_{1}^\epsilon{t\,dt}= lim_{\epsilon\to\infty}[\dfrac1{2}t^2+C]_1^\epsilon=lim_{\epsilon\to\infty}\dfrac1{2}[(\arctan{\epsilon})^2+C -\arctan{1} -C]= \dfrac1{2}(\dfrac{\pi}{2})^2-\dfrac{\pi}{4}$$
This integral is convergent, so $\sum_{n=1}^\infty{\dfrac{\arctan{n}}{n^{2}+1}} $ is convergent too.
Answer: HINT:
$$\sum_{n=1}^\infty{\dfrac{\arctan{n}}{n^{2}+1}}<\sum_{n=1}^\infty{\dfrac{\frac12\pi}{n^{2}+1}}$$
Answer: HINT: Use limit comparison test with $1/n^2$.
Answer: You sum is smaller than
$$
\sum_{n\ge 1}\frac{2}{n^2+1}<2\sum_{n\ge 1}\frac{1}{n^2}\le \frac{\pi^2}{3}.
$$


Final model for time series forecasting

Question: I am struggling with understanding how the final performance on an independent test set is obtained in a time series forecasting scenario.
My understanding is that rolling CV is used (as seen as in the picture) to prevent future datapoints to leak into the model. The ensemble of scores can than be used to deduce performance.
This method results in many models (in this case, four). What is the final model then selected for further use? Is it the final model, trained on most of the data? Is the performance of this model then described by the scores of the model itself, or by the ensemble of itself and all preceding models?
Comment: Thank you @RichardHardy, I am aware of how SE works but I am afraid the issue remains unclear for me. Ignoring hyperparameter tuning, would you agree the following is how CV is used for time series? (1) train a model on all data. (2) run a rolling CV where you, at each iteration, train a model and validate is on the respective sets as in the image. (3) the performance of the model from step 1 is said to be the average of all performances in step 2.
Comment: No, this is not how CV is used for time series. Step 1 is not taken. Step 2 is taken. The performance of the model is characterized by the union of the errors or losses from the different test sets. Summary measures such as the mean (over all the test sets) squared error are often used. The procedure above is carried out for a variety of candidate models. The model with the best performance is selected. That model is then reestimated on the entire dataset. It can then be used out of sample.
Comment: As I understand it we describe the same process, albeit my step 1 is your step 3. Whether the final model is trained before or after the CV would not matter, it might as well run in parallel.
Comment: What you call step 1 should be done for only one (the best) model out of all candidate models. Therefore, we need to do the other steps before it, as these other steps are required for selecting the best model. Doing it as step 1 means a bunch of unnecessary calculations. But this is not a big deal, really, if you are doing cross validation, as that is already quite costly. What seemed a bigger deal to me was to say *the performance of the model from step 1 is said to be the average of all performances in step 2.* But when I think about it again, this might be a matter of formulation.
Comment: I agree. However I also believe CV is not only needed when selecting a model or doing hyperparameter tuning, but also for getting a better estimate (read: reduce bias and variance) of the model performance when only a single model is used. If you add this to your answer I am happy to accept it. Thank you for your help
Answer: This does not have to result in 4 models. A common restriction is that the model is the same in all training samples, only its parameters are reestimated in each of them. The model is evaluated on a joint test set which is comprised of the four individual test sets.
If model selection needs to be done, the above is carried out for each model under consideration. The model that yields the best performance on the joint test set gets selected. The model's performance on the joint test set serves as an estimate of its performance on unseen future data. (The latter holds regardless of whether any model selection is going on at all.) The model's parameters are then reestimated on the entire data set, and genuine out-of-sample forecasting begins.
Comment: Perhaps there is some confusion on my end. When I look at the figure, I see four training sets. For each of these, I would imagine a model is trained on this data and validated on the corresponding test set. In my mind this results in four models. You say "[...] the model is the same in all training samples [...]", so then I wonder how this would work as there would be only a single instance of training. I worry this is a semantics issue on my end.
Comment: @TimStack, across the training sets, the model is the same up to parameter estimates. You estimate the model on four different training sets and test it on the corresponding test sets each time. You still consider this as one model. You do the same for all the different model specifications you have. You then select the best specification (based on the average performance across the four test sets) and proceed with it to real out-of-sample forecasting. Does that help?
Comment: Thank you for your elaboration Richard. I think it's a partial semantics issue since I don't consider two models trained on different data subsets as the same model. Could you clarify what you mean by "model specification"?
Comment: @TimStack, that must be it. Sorry for not being clearer about that. If you consider a parametric model, it consists of its functional form + parameter estimates such as [$y=X\beta+\varepsilon$] + [$\hat\beta=(1,2,3)'$]. What I mean by *specification* is the functional form $y=X\beta+\varepsilon$. The specification/form stays the same across the training samples, but the parameter estimates vary: in the first training sample we can have $\hat\beta^i=(1,2,3)'$, in the second $\hat\beta^{ii}=(1.5,1.5,2.5)'$, and so on. I call this the same model, albeit with different parameter estimates. Better?
Comment: Apologies for the struggles with terminology; studying at a uni does not expose you to too many variations of certain terms. As I understand it, the hyperparameters (part of the functional form) stay the same across the four samples, while the parameters (i.e. parameter estimates) change. Then it could be entirely possible the model/specification with the least data is deemed the best performer and used for actual forecasting. Could this not pose a problem, considering a lot of the data will be left unused?
Comment: Additionally, is the performance of this final model then validated on an independent test set or are the scores from the CV iteration used instead?
Comment: @TimStack, I do not understand *the model/specification with the least data*, as every specification (in my sense of the word) is tried out on the same data as any other specification. Re your second comment, if you have an independent test set, you can validate on it. If not, you are left with what you have from CV. If there has been some model selection, the CV performance of the best model will be an optimistic estimate of the true out-of-sample performance, and that should be kept in mind.

Final model for time series forecasting


Can I prove $2^{-n}$ converges to $0$ without invoking the Axiom of Completeness or the Archimedean property of the reals?

Question: Is it possible to show that for any real number $x > 0$, there exists a natural number $n$ such that $2^{-n} < x$ without invoking the Axiom of Completeness or the Archimedean property of the reals?
Comment: Doing that proves the Archimedean property, so I'm not so sure what "invoke" should mean.
Comment: You need some property that distinguishes the rational numbers with the usual metric from the rational numbers with the $2$-adic metric; in the $2$-adic metric this sequence diverges.
Comment: Doesn't non-standard analysis work in a complete field that contains the real numbers in which this series does not converge to zero (and you guessed right, the field is not archimedian)?
Comment: @Dirk Sort of. The catch is that hyperreal analysis generally doesn't use the word "limit", instead obtaining the standard concept as "the standard part of the result of substituting an infinite number in some domain, provided this standard part doesn't depend on the infinite number we chose".
Comment: @Dirk: The hyperreals are (very badly) incomplete, when viewed from the external perspective. (they are, of course, complete when viewed internally)
Answer: You haven't specified what properties you are allowing to be used.
It's pretty easy to find ordered fields where such a sequence can't converge to zero. For example, take any ordering on the field of rational functions $\mathbb{R}(x)$ i.e. the field of all functions that can be written as the quotient of two polynomials. Without loss of generality, we can take $x$ to be a positive and infinite; that is, $x > a$ for all real numbers $a$.
In this field, we always have $0 < 1/x < 2^{-n}$ for every natural number $n$, so the sequence clearly cannot converge to zero.
There are even worse examples; in some ordered fields, the only convergent sequences are the ones that are eventually constant! (where "sequence" is limited only to sequences indexed by natural numbers)
(if you replace the field of rational functions with some other suitable example, then all of the above applies to real closed fields as well, which is basically just a completeness axiom suitable for working with polynomials)
Answer: If for all $\varepsilon>0$ there exists $N_{\varepsilon}\in\Bbb N$ such that $2^{-N_{\varepsilon}}<\varepsilon$, then for all $\alpha>0$ you have $2^{N_{1/\alpha}}>M$. Which proves the archimedean property. So I guess you cannot avoid it. Completeness can be avoided and, in fact, if you try to prove it as usual you won't ever need to use the fact that bounded sets have a least upper bound.
Answer: It depends on how you define convergence.
If you define $(x_n)_{n\in\mathbb{N}}$ to converge to $a$ if, for every positive rational $\varepsilon$, there exists $k$ such that, for $n>k$, $|x_n-a|<\varepsilon$, then you don't need the Archimedean property.
Indeed, it's easy to show that, for every natural $n$, $2^n>n$ and the argument can be quickly completed.
However, you need the Archimedean property for proving that, given a positive real $\varepsilon$ there exist a rational $q$ such that $0<q<\varepsilon$. This is necessary for the above definition to be equivalent to the one with "for every positive real $\varepsilon$".
Answer: The "Archimedean property" is already present in the natural numbers, and can be proven from the Peano axioms. It therefore makes no sense to avoid making use of it.
Let an $x\in{\mathbb R}_{>0}$ be given, and put $$n:=\left\lceil{1\over x}\right\rceil\geq{1\over x}\ .$$ The set $\{1,2,3,\ldots,2^n\}$ contains the $n+1$ numbers $1$, $2$, $4$, $8$, $\ldots$, $2^n$. It follows that $2^n>n$, hence
$$0<2^{-n}<{1\over n}\leq x\ .$$
Comment: "Is this invoking archimedean property?" I claim $\left\lceil \frac1x\right\rceil$ does not exist, since you've been so unlucky to choose $x$ to be the reciprocal of a very special real number which is bigger than any natural number.
Answer: For $\epsilon\in\mathbb{R}$ be a real number such that $\epsilon > 0$
let $N_{\epsilon}\in\mathbb{N}$ be the smallest integer such that
$N_{\epsilon} > ln(\frac{1}{\epsilon})/ln(2)$.
Then for all $n>N_{\epsilon}$, $2^{-n}<\epsilon$.
Comment: I think you need the Axiom of Completeness to ensure $\log_2\left(\frac{1}{\epsilon}\right)$ is well defined.
Comment: Edited to the more palatable term.


Probability a person chosen at random owns an automobile or a house, but not both

Question: A marketing survey indicates that $60\%$ of the population owns an automobile, $30\%$ owns a house, and $20\%$ owns both an automobile and a house.  Calculate the probability that a person chosen at random owns an automobile or a house, but not both.
(A) $0.4$
(B) $0.5$
(C) $0.6$
(D) $0.7$
(E) $0.9$
So the answer given for this question is $0.5$, but it doesn't make sense to me from the perspective of addition rule in probability. If we know that $\Pr(\text{Owning Automobile}) = 0.6$ and the $\Pr(\text{Owning a House}) = 0.3$ and the $\Pr(\text{Owning both an automobile and a house}) = 0.2$,
according to the formula probability of $\Pr(A)~\text{or}~\Pr(B) = \Pr(A) + \Pr(B) - \Pr(A~\text{and}~B)$, wouldn't the answer be $0.6+0.3-0.2=0.7$?
Is it because the ordering matters? so the last term $\Pr(A~\text{and}~B) = 2 \cdot 0.2$?
Comment: You computed the probability of "owning an automobile or a house." They asked for the probability of "owning an automobile or a house *but not both*."
Comment: Please type your question rather than posting an image.  Images cannot be searched.
Comment: @pino231, in fact "owning an automobile or a house" in this case means "owning an automobile, or a house, or both". This usage of the word "or" -- the so-called "inclusive or" -- is standard in mathematics. If we want the opposite (the exclusive or) it is almost always explicitly stated, as it is in your question.
Comment: @angryavian I still don't get it, I thought "owning an automobile or a house" is the same as "owning an automobile or a house but not both" ??
Answer: Let event $A$ be owning an automobile.  Let event $B$ be owning a house.
Consider the diagram below:

If we simply add the probabilities that a person owns an automobile and a person owns a house, we will have added the probability that a person owns both an automobile and a house twice.  Thus, to find the probability that a person owns an automobile or a house, we must subtract the probability that a person owns both an automobile and a house from the sum of the probabilities that a person owns an automobile and that a person owns a house.
$$\Pr(A \cup B) = \Pr(A) + \Pr(B) - \Pr(A \cap B)$$
Note that on the left-hand side of your equation, you should have written $\Pr(A \cup B)$ or $\Pr(A~\text{or}~B)$ rather than $\Pr(A)~\text{or}~\Pr(B)$.
Since we are given that $60\%$ of the population owns an automobile, $30\%$ of the populations owns a house, and $20\%$ owns both, $\Pr(A) = 0.60$, $\Pr(B) = 0.30$, and $\Pr(A \cap B) = 0.20$.  Hence, the probability that a person owns an automobile or a house is
$$\Pr(A \cup B) = \Pr(A) + \Pr(B) - \Pr(A \cap B) = 0.60 + 0.30 - 0.20 = 0.70$$
However, the question asks for the probability that a person owns an automobile or a house, but not both.  That means we must subtract the probability that a person owns an automobile and a house from the probability that the person owns an automobile or a house.
$$\Pr(A~\triangle~B) = \Pr(A \cup B) - \Pr(A \cap B) = 0.70 - 0.20 = 0.50$$
In terms of the Venn diagram, $A \cup B$ is the region enclosed by the two circles, while $A~\triangle~B = (A \cup B) - (A \cap B) = (A - B) \cup (B - A)$ is the region enclosed by the two circles except the region where the sets intersect.
Since $A - B = A - (A \cap B)$,
$$\Pr(A - B) = \Pr(A) - \Pr(A \cap B) = 0.60 - 0.20 = 0.40$$
Since $B - A = B - (A \cap B)$,
$$\Pr(B - A) = \Pr(B) - \Pr(A \cap B) = 0.30 - 0.20 = 0.10$$
Hence,
$$\Pr(A~\triangle~B) = \Pr(A - B) + \Pr(B - A) = 0.40 + 0.10 = 0.50$$
which agrees with the result obtained above.
Comment: ahh so the question is actually asking the probability that a person chosen at random owns an automobile or a house, OR not both
Comment: Not quite.  The word *but* should be read as *and*.  The question is asking the probability that a person chosen at random owns an automobile or a house and not both: $\Pr(A~\triangle~B) = \Pr(A \cup B) - \Pr(A \cap B)$.
Answer: 10 $\%$ of people don't have a house or automobile so you can leave them. There is 90 $\%$ of people left, also 20$ \%$ have both a house and a automobile. This 20% of people is included in the 60% and 30%, so you have to substract this percentage. What you get is 40% + 10% = 50%. So you were probably right by assuming that you have to count the last term $P(A \cap B)$ two times.
Comment: If we subtract $\Pr(A \cup B)$ two times, we will obtain a negative answer.  Instead, we must subtract $\Pr(A \cap B)$ twice.
Comment: Yes of course, that's what I meant. Will edit!
Answer: There's no ordering in this question. Using the addition rule is fine, but if you want to use it, note that:
$ \mathbb{P}(A) + \mathbb{P}(B) = \Big( \mathbb{P}(A \setminus B) + \mathbb{P}(A \cap B) \Big) + \Big( \mathbb{P}(B \setminus A) + \mathbb{P}(A \cap B) \Big)$.
This should probably help.


Trying to understand Caliburn Micro EventAggregator

Question: I understood how does EventAggregator work in Caliburn Micro but i don't understand this:
Let's take this as an example:
ViewModel A
private IEventAggregator _eventAggregator;
public ViewModelA(IEventAggregator eventAggregator)
{
   _eventAggregator = eventAggregator
   _eventAggregator.PublishOnUIThread("Hello");
}

ViewModel B
private IEventAggregator _eventAggregator;
public ViewModelA(IEventAggregator eventAggregator)
{
   _eventAggregator = eventAggregator
   _eventAggregator.Subscribe(this)
}
public void Handle(string message)
{
   MessageBox.Show(message + " From ViewModel B")
}

ViewModel C
private IEventAggregator _eventAggregator;
public ViewModelA(IEventAggregator eventAggregator)
{
   _eventAggregator = eventAggregator
   _eventAggregator.Subscribe(this)
}
public void Handle(string message)
{
   MessageBox.Show(message + " From ViewModel C")
}

So how can i select which viewmodel handle to use when i publish events?
Comment: You can create view model specific message types and have the interested parties listen/handle those types of messages.
Answer:
So how can i select which viewmodel handle to use when i publish events?

Several view models may subscribe to the same message. This is how event aggregation works. The publisher publishes a single message without any knowledge of the number of receivers.
If you want a particular view model to be able to subscribe to an event, it should implement the IHandle<T> intertface and call the Subscribe method on the event aggregator:
public class ViewModelA : IHandle<string>
{
    private readonly IEventAggregator _eventAggregator;

    public ViewModelA(IEventAggregator eventAggregator)
    {
        _eventAggregator = eventAggregator;
        _eventAggregator.Subscribe(this);
    }

    public void Handle(string message)
    {
        MessageBox.Show(message);
    }
}

All view models that does this will receive the message.
Answer: I managed to send messages to particular viewmodels via Message class:
public class Message<TTarget, TValue>
{
    public TValue Value { get; set; }

    public Message(TValue value)
    {
        Value = value;
    }
}

Then add to a class
IHandle<Message<ViewModel, string>>

Usage:
_eventAggregator.PublishOnUIThread(new Message<ViewModel, string>("Hello from ViewModel"));


Running some test automation before code checkin

Question: I am maintaining the source control for my organization and I want to run some automated tests on every checkin before it reaches the central store. Which means that when users submit their changes, I will build all the code and start the test suites.
What do I need to setup for this - assuming I have perforce as my source control?
This should be a very common problem but I probably not have the right vocabulary.
Comment: As stated your question is too broad or even not a question at all.
Comment: @malenkiy_scot Sorry, i forgot to add the question. edited now.
Answer: I assume you're using Perforce from your tags.  There are a couple of ways you could do this:

Using branches - have the developers check into a "test" branch and make Jenkins monitor that branch for changes to build your code and run tests.  You can then integrate the test code into a separate "main" branch for everyone else to pull from.
If you have a recent version of Perforce, you can use the shelving feature. Have the developers create shelved changelists and get Jenkins to unshelve the changes to build and run the tests.  If the tests pass, the changelist can be submitted to the central depot.

We use option 2 at my work, so I can give you more details if needed - just post a comment back here.
Comment: Sounds like you will have to do some custom tooling whatever you do - Git could be a good option, but you'll have to figure out how to get the changes from Git to Perforce once they're blessed.  You could also look at the new P4 Sandbox feature, which replicates a lot of Git's features in Perforce (I haven't used this so can't offer much advice)
Comment: Thanks gareth. We actually use perforce as the central repository for the company. But my team wants to go through an automation run before we checkin to perforce. I don't have control of the central server; but I will be controlling this intermediate setup. I am thinking about git, as I read that branches are cheaper in git. I have used jenkins to monitor branches, but if the branch is created dynamically - as a result of a checkin submission -; can i dynamically create a build in jenkins for this branch? any other suggestion? Thanks.


select oid from order_details where UserName= '"+uname+"' order by oid desc limit 1

Question: What is the syntax error in the query:
select oid from order_details where UserName= '"+uname+"' order by oid desc limit 1;
Here i am gettin error of syntax.... plz help!!! what is the error???
Full Code:
<%@page import="javax.validation.constraints.Null"%>
<%@page import="java.text.SimpleDateFormat"%>
<%@page import="java.sql.*"%>
<%@page contentType="text/html" pageEncoding="UTF-8"%>
<!DOCTYPE html>

        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
        <title>JSP Page</title>

<%
String languages;
int price;
int tot;
String[] lang=request.getParameterValues("dish");
String event=request.getParameter("eve");
String date=request.getParameter("date");
String place=request.getParameter("place");
String person=request.getParameter("PERSON");
String uname=(String)session.getAttribute("username");
String oid="0";
int row=32;
Class.forName("com.mysql.jdbc.Driver");
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/ready to serve", "root", "");
Statement st=con.createStatement();
st.executeUpdate("insert into order_details(event_date,PLACE,NO_OF_PERSON,Event,UserName)values('"+date+"','"+place+"','"+person+"','"+event+"','"+uname+"')");
String getoid= "select oid from order_details where UserName= '"+uname+"' order by oid desc limit 1";

ResultSet rs= st.executeQuery(getoid);
if(rs.next())
{
oid=rs.getString(1);
}
int noof=Integer.parseInt(request.getParameter("PERSON"));
tot=0;
ResultSet pric= st.executeQuery("select sum(a.price) from menu a,dorder b where a.dish=b.dish");
while(pric.next())
{
tot=pric.getInt(1)* noof;
}
for(int i=0;i<lang.length;i++)
{
    languages = lang[i];
    st.executeUpdate("insert into dorder values('"+languages+"','"+oid+"','"+uname+"')");
}
String totin="update order_details set tot='"+tot+"'where oid="+getoid;
st.executeUpdate(totin);
out.println("Data is successfully inserted into database.");
%>

Comment: 1)  code in JSP == BAD.  2) creation of SQL command in this manner is a likely candidate for SQL Injection.  3) Indent you code so that it is easy to read
Comment: data is inserting in database but value of 'tot' is not being updated in the table......//
Answer: You need some spaces in the update command
 String totin="update order_details set tot = '"+tot+"' where oid = "+getoid;

According to the below error, the Select also seems wrong
try
select oid from order_details where UserName = 'fred' order by oid desc limit 1

Comment: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'select oid from order_details where UserName= 'rahul0190' order by oid desc limi' at line 1
root cause

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'select oid from order_details where UserName= 'rahul0190' order by oid desc limi' at line 1
Comment: well Dude, that is a select statement, not an update.  Try adding a space before the `=`
Comment: But what that error means ...... data is inserting into database properly but its still generating error in select query.....
Comment: String totin="update order_details set tot='"+tot+"'where oid="+getoid;
st.executeUpdate(totin);          Its not updating the value of tot  column in database.......
Comment: The error you posted above is related to the select.  The update you have just posted has not been changed to reflect my answer I gave.
Comment: see my answer.  Again, you need some spaces.  If this does solve your problem then lets us know what the error is.
Comment: add a spacing here too `String totin="update order_details set tot='"+tot+"' where oid="+getoid;`


Vocal manipulation during live performance

Question: I'm looking for a way to transform vocals of my performers during a live performance. The performers will be using 5 microphones in total and will sing together as a choir most of the time. I want the vocals to be mechanical and play with the pitch for example.
Can I archive this with software like Ableton or do I have to look for other things to make this work?
I would like to be able to put different effects on each of the voices but also have the option to have one effect on all voices.
Comment: I do have one question for clarification though - do you want to be able to put different effects on each of the voices, or is it sufficient to be able to effect the mix of 5 voices together?
Comment: I've retracted my close vote and answered because the explanation of what you are trying to achieve makes it much less broad. I do want to add that you basically have to learn to be an engineer yourself to make things like this work well. If you are one of the singers, you may find that doing the singing and the mixing and effects is a lot to handle.
Comment: @topomoro Thank you for the quick reply. To be specific: yes, I would like to be able to put different effects on each of the voices but also have the option to have one effect on all voices.
Comment: @ToddWilcox I know the question is very broad - but any type of advice is welcome! I'm curious to see it. I'll look for a sound engineer, but it would be best if I could work on this myself. Especially during rehearsals for example.
Comment: @ToddWilcox Thanks again Todd for your fantastic reply! Luckily I'm the director of the performance so I have the time to play with it in the rehearsals, but I'm gonna ask around for someone to help me out in the first weeks. Cheers!
Answer: Basically, you need a mixer with mic preamps and aux sends and you need effects. These days for a live show a digital mixer is certainly the way to go.
There are two broad categories of what we could call "digital mixers". There are dedicated hardware digital mixers, and then there are DAWs and software mixers that you install on a computer that is connected to one or more audio interfaces.
With a hardware digital mixer, you generally get everything (mixing, signal routing, and effects) all in one box. Most of the time, what you get in a hardware mixer is what you got, so whatever effects it comes with is all it can do. The advantages of hardware digital mixers is that they are cost effective, reliable, low latency, and easier to set up and use than software based solutions. Often you can change the settings on hardware digital mixers from anywhere using a wifi connected tablet. The Behringer X25 line of digital mixers is an extremely popular example of these products.
With software, you first need a fast computer and a fast audio interface in order to apply the effects you want in real time without excessive latency. The audio interface provides all the inputs and outputs to the system, the computer provides the processing power, and the software uses the computer and interface to route the audio and apply effects. The advantages of a software based system is that you can completely customize each aspect of the system, and you can add third party effects, so pretty much any effect you can imagine, you can use (if you pay for it). The downsides are that these systems are more complicated, more expensive, and can be prone to more kinds of failures than dedicated mixer systems. In terms of software, Ableton Live is very popular for live sound processing (hence the name), and Apple Mainstage is also used by many big shows. Mainstage requires an Apple Mac computer to run, and can be controlled remotely with an iPad using free Apple Remote software. Ableton Live can be controlled remotely with an iPad, but you have to buy a customizable MIDI/OSC app like Lemur, set up the Lemur interface, and then manually map the Lemur controls to Live. This is a much more complicated setup but is also very flexible. Another good control option for Live is using Ableton's own Push 2 hardware interface, but that connects with a USB cable and is not wireless. Also, Push 2 is not really designed for mixing and effects, it's focused on triggering beats and loops and samples.
On the software side, there is one hybrid option, but it is likely the most expensive route. There are interfaces that have dedicated outboard DSP, the Universal Audio UAD system is probably the most famous one of these. The advantage of these systems is they give you very low latency, high reliability, and all the flexibility of choosing what effects you want to use, but the interfaces and effects are generally quite expensive.
Comment: @topomorto As per the requested functions, aux sends (really internal effects busses for the digital solutions) would be used for effects that apply to multiple channels, and inserts would be used for effects on individual channels.
Comment: Would aux *sends* necessarily be the preferred way to set this up? With pitch shift effects etc, I'm imagining that full wet is probably what will be wanted, so inserts on the channels might be neater..?
Comment: yes, true that sends would be neater for multiple channels.
Comment: @ToddWilcox This is everything! Thanks so much for taking the time to answer my question. I'm gonna dig into it!
Answer: As you've suggested and Todd has expanded on in his answer, doing this on a computer (assuming you already have something suitably powerful) is likely to be the most general, flexible, and repeatable way to do this - and probably the cheapest too, especially if you already have an audio interface with 5 or more inputs.
A less general solution, which might be too silly, but might be fun if the theatrical considerations allow, would be to get some kind of voice changer toy:

Various models are available for a relatively low price. I doubt many of them allow very fine control, and of course you'd be reliant on the performers to manipulate them - which could be a good or a bad thing.
You'd probably want to experiment with one to see how it worked when mic'd up before splashing out on 5!
Comment: @PandaCat if there's any kind of recording of the performance and it's possible to share it I'm sure we'd be interested!
Comment: I will! Performance will happen in September, I'll post a link when the documentation is online!
Comment: Hi! The recording is available online now for streaming online: https://open.spotify.com/album/3CZz2hxSTyrzqSEBVCnsT8. Enjoy and thanks a lot for helping me out last year! @toporeinstatemonica
Comment: @PandaCat it's taken me this long to dig out my spotify login but am finally listening now! thanks for the link.
Comment: That's definitely something I'm going to check out. Would be interesting to rebuild the toy into another object that transforms voices. Thank you!
Comment: @PandaCat if you're interested in building, you can get things like [this](https://www.ebay.co.uk/itm/Velleman-MK171-Voice-Changer-Kit/121650451261), already in unbuilt form (albeit with a built in mic - which may or may not be good enough for what you need). You could probably make something with a raspberry PI too, although then you're drifting back towards using a computer again!
Comment: Yes! This is great. I'm going to look into it. I'm so grateful for the awesome help I'm getting! Thank you!
Comment: We're releasing the soundtrack of the performance on vinyl next year, if you're interested listen to the first release here: https://operatingmanualforfloatingin.space/soundtrack. On this page you'll also find instructions executed within the performance piece.

Thank you all for helping me out!


Page template, single page display

Question: I've built a custom page.php template. Very simple, essentially:
<?php get_header(); ?>

<?php get_sidebar(); ?>

<?php the_post(); ?>

<?php the_title(); ?>

<?php get_footer(); ?>

I've created a few pages, and if I visit their url, I just get the same page title.
I read up in the documentation, and it says to use the_title() and such only when in "the loop".
So presumably, I'm just being shown the first page in the "array".
Is there any way I get get the contents of a single page based on the url?
Edit: In fact, should I even need to do this? Refering to example templates, it looks like I'm doing everything right?
Answer: In my sidebar I was using a custom query.
This was called before trying to access the main page content, without resetting.
When doing a custom query you must reset after you've finished your loop like so:
$originalPost = $post;
$sidePosts = get_posts($queryArgs);
foreach($sidePosts as $post) {
    setup_postdata($post);
    // echo it out like a normal post.
}
$post = $originalPost;

or if you are using query_posts() (which you shouldn't in a sidebar):
wp_reset_query();

Which will take your post back to it's previous value.
Answer: For a custom page template please use the following to get everything correct
<?php
/* Template name: My custom template */

get_header();

    if ( have_posts() ) while ( have_posts() ) : the_post();

the_title();
the_content()

endwhile;
get_sidebar();
get_footer();
?>

Comment: Hey, thanks for your answer. Turns out it was a custom query that was mucking up my posts object. Unfortunately I had to move get_sidebar() before "the loop" since that's how my css floats work (I know it's not ideal from an SEO point of view) -when I did this, because of the error in the sidebar code, it stopped working properly.

Page template, single page display


How can I make a grid fixed, not affected by scrolling?

Question: The structure of the code I am working on is looking like this:
<code><Grid>
  <ListView>
     <ListView.Resources>
     </ListView.Resources>

     <ListView.ItemTemplate>
        <DataTemplate>
          <Border>
             <StackPanel>
                <Grid>
                </Grid>

                <ListView>
                </ListView>

                <Grid>
                <!-- List of the data can be long. So I want this grid to be fixed, not affected by scrolling, but there could be multiple lists as well. So when I scroll to the end of one list, the other one begins, then the grid of other list should appear, and stay fixed -->
                </Grid>

             <StackPanel>
          </Border>
        </DataTemplate>
     </ListView.ItemTemplate>
  <ListView>
<Grid>
</code>
The red arrow tells where the grid is and how it should look like, currently the list is long with more than 100 items in it, and to get to this grid I have to scroll all the way down which is not good in practice...
Currently
Expected
Comment: I need a picture of what you're expecting.
Comment: I have posted the picture in description.
Answer: You can keep the header table in sync with the content table horizontally on scroll event. Use table-layout: fixed so that applied column width remains the same. One more thing adjust the header table by giving an extra td, at last, to get accurate width as same as content table after getting a scroll bar on content table


Limit of Riemann Sum equal to integral

Question: I don't understand why Limit of a Riemann sum  suddenly turns into a integral. It makes no sense both intuitively and geometrically to me. In case, differentiation it is easy understand though.
Comment: You don't understand why $$\lim_{n\to \infty} \sum_{k=1}^n\frac{k}{n} \cdot \frac{1}{n} = \frac{1}{2}\,\,?$$
Comment: Have you seen a drawing with smaller and smaller rectangles, like [this one](https://upload.wikimedia.org/wikipedia/commons/thumb/2/2a/Riemann_sum_convergence.png/300px-Riemann_sum_convergence.png)?
Comment: Incidentally $1/n\to 0$ but it doesn't "suddenly" turn into $0.$
Comment: Well, if differentiation is easy to understand, can you explain what makes the limit of the secant a tangent? You can't, because it is a definition: the limit of the Riemann sum is a definition of the (Riemann) integral, just as the limit of the secant's slope is a definition of the derivative.
Comment: I think you are asking the same question as this one http://math.stackexchange.com/q/1767809/72031
Answer: the Reimann sum is the definition of the integral.
I think what you are really saying is that you don't understand why the integral equals the anti-derivative.
And that is the "Fundamental Theorem of Calculus."
suppose $F(x) = \int_0^t f(t) dt$
That is, we have a curve $f(t)$ and we color everyting above the interval (0,x) on the $x$-axis to the bottom of the cuve.  That area equals $F(x).$  Now we want to make that area just a tiny bit larger.  So we make a very thin stripe just to the right of $x.$  How much has $F(x)$ changed?  The area has changed by the length of the stripe $\times$ the thickness of the stripe.  Lets call the thickness h.
$F(x+h) = F(x) + f(x)h$
Now what is the derivative of $F(x)$?
$\lim_\limits{h\to 0} \dfrac{F(x+h) - F(x)}{h} = \dfrac{f(x) h}{h} = f(x)$
Perhaps not rigorous, but it might help with the intuition.
For a more formal proof, I am sure your calculus book has one, and it should be easy enough to google it.
Comment: Thanks, but what is the proof of: suppose $F(x) = \int_0^t f(t) dt$?
Answer: There is no reason why the limit of a Riemann sum turns into an integral: the (Riemann) integral of a function over an interval $[a,b]$ is by definition the limit of a Riemann sum. However, if you are aware of the geometric definition of the integral, as the area under a curve, we can see why this definition makes sense.
Say we want to compute the area under the curve described by the equation $y=f(x)$ on some interval $[a,b]$. One way to do this is to cut up our interval into small sub-intervals, and add up the area of rectangles with these sub-intervals as their base, and whose heights are approximately that of the graph of $f$. More formally, we can define a subdivision of $[a,b]$ by specifying $N+1$ points $a=x_0<x_1<x_2<..<x_{N-1}<x_N=b$. Under the right conditions, the value of $f$ on each of the $[x_i,x_{i+1}]$ will be sufficiently close to some $f(a_i)$ with $a_i\in [x_i,x_{i+1}]$, so that the area under the graph of $f$ on $[x_i,x_{i+1}]$ will be close to $f(a_i)\cdot(x_{i+1}-x_i)$. Then we may sum up all these approximate areas to get a total approximate area under the graph of $f$ on $[a,b]$ , that is, let $$S_N=\sum_{i=1}^{N}f(a_{i-1})\cdot(x_{i}-x_{i-1})$$
$S_N$ is precisely a Riemann sum associated with the subdivision $\sigma=(x_0,..,x_N)$. We may take our subdivision and choice of $a_i$'s in some convenient way, for instance make the $x_i$'s evenly spaced, take $f(a_i)$ to be the sup or the inf of $f$ on $[x_i,x_{i+1}]$, or any other choice we find useful, but under the right conditions, we find that the value of $S_N$ converges as the subdivision gets finer (that is, $\sup\{x_{i+1}-x_i,0\leq i\leq N\}$ gets small enough), regardless of the particular subdivisions or choice of $a_i$'s. If this is indeed the case, we say $f$ is (Riemann) integrable on $[a,b]$, and define $$\int_a^bf(x)dx=\lim_{n\mapsto \infty}S_n$$
Where $(S_n)$ is any sequence of Riemann sums whose corresponding subdivisions get finer and finer. In other words, if $\sigma=(x_0,..,x_N)$, let $\delta(\sigma)=\sup\{x_{i+1}-x_i,0\leq i\leq N\}$, and let $(S_n)$ have corresponding subdivisions $(\sigma_n)$. Then we require that $\lim_{n\mapsto \infty}\delta(\sigma_n))=0$.
Notice, the very notation of the integral informally reflects this idea of discrete summation. the $\int$ sign is an elongated S for "sum", and the $dx$ stands for an infinitesimal variation in $x$, that is, $x_{i+1}-x_i$ if the subdivision is "infinitely fine"

Limit of Riemann Sum equal to integral


Fast dropout: How to compute the mean and variance of the approximating Gaussian?

Question: In the Fast Dropout article by Wang and Manning, they talk about approximating the input to a hidden layer by a Gaussian with the same mean and variance as the inputs to the layer (see page 5). But they never discuss how the mean or the variance would be computed. Do they select a minibatch and take statistics over the various elements of the minibatch each time we move to the next layer?
For example, suppose we have a minibatch $(\mathbf{x}_i, y_i)_{1 \leq i \leq n}$. Suppose we've just moved to layer $5$, and $f(\mathbf{x})$ represents the input to layer $5$ given input $\mathbf{x}$ to layer $0$. Then in order to find the appropriate Gaussian, we look at the mean and the variance of $\{f(\mathbf{x}_i)\}_{1 \leq i\leq n}$ in order to find the right Gaussian? (Sort of like we do in batch normalization?)
Answer: The paper describes on page 5:

... we may again approximate their inputs as Gaussians and characterize their outputs by the  output  means  and  variances

These means and variances are computed per sample: Every sample in the batch has its own mean and variance representing the distribution approximating the distribution of its values under various dropout masks. That's what is "fast" about this method; instead of direct sampling of dropout masks (which requires many samples, and thus dropout usually slows down the training), you pass the whole approximative distribution through the network, simultaneously evaluating how would the sample be perturbed by any of the dropout masks (or rather, how the whole distribution of the sample values would behave).
So, instead of outputting "values" of transformed samples $f(\mathbf{x})$, units output means and variances thereof: $\mathbb{E}[f(\mathbf{x})]$ and $\mathrm{var}[f(\mathbf{x})]$. You can use these in the next layer as the means and variances.
In the first layer, you consider the values of $\mathbf{x}$ to represent the individual means and variances are set to 0.
Here you can find my keras implementation of fast dropout.


Matrix by vector multiplication along third axis

Question: I am using numpy to perform some mathematical calculations in Python.
I have an A matrix of dimensions (N, N) and a vector b of dimension m.
I would like to multiply A by b so as to obtain a tensor of size (N, N, m).
For example, if it were:
A
1 2
3 4

b
1 2 3

I would like to make the product in order to obtain the tensor C = A*b:
C
1 2
3 4

2 4
6 8

3 6
9 12

Finally, I would like to get a (N,N) matrix with the maximum value of C along the third direction, that is
max(C)
3 6
9 12

is there a numpy function that allows this operation?
Update
if finally I want to add to each matrix (N, N, i) of C the i-th element of the vector b?
In the example:
A
1 2
3  4

b
1  2  3

C = A*b
1  2
3  4

2  4
6  8

3  6
9  12

D = C+b
2  3
4  5

4  6
8  10

6  9
12 15

Answer: Here's one way leveraging broadcasting:
(b[:,None,None] * A).max(0)

array([[ 3,  6],
       [ 9, 12]])

Update: you can add b to each inner matrix with:
b[:,None,None] * (1+A)

array([[[ 2,  3],
        [ 4,  5]],

       [[ 4,  6],
        [ 8, 10]],

       [[ 6,  9],
        [12, 15]]])

Setup:
A = np.array([[1,2], [3,4]])
b = np.array([1,2,3])