MySQL Query for deleting entries

Question: I have a Account-Database with a "last-login" row, I want to delete all accounts which are empty (NULL) at last-login row.
Does someone know a MySQL Query for this? (Database-Structure is like: User-Id, password, last-login)
Comment: I removed php-tag, because no php related question IMO.
Answer: This should do the the tricks:
DELETE FROM Account-Database WHERE last-login IS NULL;

or if the field is just an empty field (not null but '')
DELETE FROM Account-Database WHERE last-login= '';

See: http://dev.mysql.com/doc/refman/5.0/en/delete.html
Answer: delete from Account-Database where `last-login` is NULL;

It's not clear in your question whether Account-Database is the table name or the database name but you get the idea.
Comment: In SQL, `NULL=NULL` is `NULL`, not `false`.
Comment: correct sql will be more like `delete from Account-Database where last-login is NULL` ;-)
Answer: <?php
mysql_query("Delete from Account-Database where last-login = NULL");
?>

Answer: DELETE FROM `Account-Database` WHERE `last-login` IS NULL;

The syntax for a DELETE query is as follows:
DELETE FROM `tablename` WHERE conditions

MySQL Query for deleting entries


matrix normal form

Question: How can I prove that any $2\times2$ complex matrix is similar to a unique matrix of the form:$$\begin{bmatrix}0 & 1\\\theta_1 & \theta_2\end{bmatrix}$$
$\theta_i$ are complex.
I tried to use schur decomposition but it is not of this form and not unique.
thank you
Comment: Can you show that any $2\times 2$ matrix in JNF is similar to exactly one such?
Comment: JNF is not unique
Comment: Jordan normal form. The usual normal form of similarity classes. It _is_ unique up to rearrangement of the blocks. You can define an order to make it unique.
Answer: The characteristic polynomial of this matrix is
$$\det(\lambda I - A)  = \lambda^2 + \theta_2 \lambda + \theta_1$$
Since similarity transforms leave the characteristic polynomial alone, the matrix is unique. It remains to be shown that a $2\times 2$ matrix with this characteristic polynomial is indeed similar to $A$.
For this we use the JNF. A $2\times 2$ matrix can have one of two (ordered) JNFs with $\lambda_1 \ge \lambda_2 \in\mathbb C$ $\circledast$ it's eigenvalues; not necessarily distinct:
$$\begin{align*}
& M \sim \begin{bmatrix}\lambda_1&0\\0&\lambda_2\end{bmatrix} \\
\text{or } & M \sim \begin{bmatrix}\lambda_1&1\\0&\lambda_1\end{bmatrix} & \text{where }\lambda_1=\lambda_2
\end{align*}$$
In the former case, $\theta_1 = \lambda_1\lambda_2$ and $\theta_2 = \lambda_1+\lambda_2$, and in the latter case $\theta_1 = \lambda_1^2$ and $\theta_2=0$. I'll leave it to you to find the respective transform matrices.
$\circledast$: You can chose any ordering here. If you don't want that, you can expand the first case into two cases linked by the additional transform $\big(\begin{smallmatrix}0&1\\1&0\end{smallmatrix}\big)$.
Comment: thank you I will try form here
Comment: How are you proposing to well-order $\mathbb{C}$ to order your eigenvalues?
Comment: @Chappers This entirely irrelevant to the proof. Only the _existence_ of _some_ order is required. For example take the lexicograpic product of the absolute value ordering with the argument ordering (i.e. $z\le w \Leftrightarrow |z|<|w| \vee (|z|=|w| \wedge \mathrm{arg}(z) \le \mathrm{arg}(w))$.

matrix normal form


Custom ticks are not shown on reacharts x axis

Question: When I use the default ticks, it's shown. But when I use a custom tick function, it's not shown:
<ResponsiveContainer width="100%" height={300}>
    <LineChart
        width={500}
        height={300}
        data={data}
        margin={{
            top: 5,
            right: 30,
            left: 20,
            bottom: 5,
        }}
    >
        <CartesianGrid strokeDasharray="3 3" />
        <XAxis
            dataKey="name"
            tick={() =>
hi
}
        />
        <YAxis />
        <Tooltip />
        <Legend />
        <Line dataKey="pv" fill="#8884d8" />
        <Line dataKey="uv" fill="#82ca9d" />
    </LineChart>
</ResponsiveContainer>

If I remove the tick function, then labels are shown on the x axis. But when I want to show a simple hi message as test, nothing is shown:
This is the image without custom tick function:

And this is the image with custom tick function:

I have tried setting the interval={0} but that changes nothing. I followed this issue but none of the solutions worked.
What do I miss here?
The point is that those
hi
 messages are rendered and I can see them in the SVG tags in the dev tools. But they are not shown. I changed the style too, but could not make them show.
Comment:  elements are not valid svg elements and cannot be nexted inside svg elements. Are you trying to display a div inside an svg?
Comment: @Undo, what should I show there?
Comment: I was asking if the tick function injects elements inside an svg, in that case,  won't work, you have to use svg elements like ,  etc
Comment: @Undo, your note about nesting `div` in `svg` actually did the trick. I removed the `div` and used `` from the recharts library itself and it worked. Can you post it as an answer so that I can accept it?
Answer: HTML elements are not valid inside SVG elements. It appears the rechart  authors have made a component <Text /> to use as text which produces text in vector form which is valid svg and subsequently in the tick callback.


Optional matches in structural pattern matching

Question: I have a variable data incoming from a websocket and I'm using match/case to determine what to do with it. It's coming in as a tuple as below, but I want to match both of these the same.
data = ("foo", )
# or
data = ("foo", {"bar": 213})

Some cases I need the data, others I don't. In the case I don't need it (i.e. the case above where the first element is foo) I just want to essentially ignore the second variable if it's present.
This is my current way of doing this but am I going about it wrong? Or is there a more succinct form?
match data:
        case ["foo"] | ["foo", _]:
            ...
        case ["baz", x]:
            ...
        ...

Comment: `case ['foo', *_]` or do I misunderstand your question?
Comment: @SUTerliakov actually didn't think that was going to work in the case where there was only one element. I didn't even give it a try because so. Works perfectly! Post as an answer and I will mark it for you
Answer: As usual with star unpacking, variable (or underscore) with a star matches zero or more items. So the following will work:
data = ['foo']

match data:
    case ['foo', *_]:
        print('Matched')

It is similar to common iterable unpacking:
a, *b = [1]
assert a == 1
assert b == []

Note: it is different from your original code, because ['foo', {}, {}] will be matched too. There is no analogue of regex ? quantifier, so "no more than one" cannot be expressed this way.
Comment: Thanks for including that note in the answer. Again, don't know why I didn't think of this. But if I didn't then I'm sure others will too. (maybe)


What is the name of the property for two random variables having disjoint supports?

Question: Let $X$ and $Y$ be two real-valued random variables.  There is a specific name for the property when the support for $X$ is disjoint from the support for $Y$.  I can't remember it, and I can't seem to find it in search results.  What is the name of that property?

More rigorously, let $R_X \subset \mathbb{R}$ be the values that $X$ will take with nonzero probability
$$
R_X = \arg \inf_{A \subset \mathbb{R}} \{ P(X \in A) = 1, A \, \text{closed} \}
$$
Similarly, let $R_Y \subset \mathbb{R}$ be the values that $Y$ will take with nonzero probability.
$$
R_Y = \arg \inf_{B \subset \mathbb{R}} \{ P(Y \in B) = 1, B \, \text{closed} \}
$$
Say $R_X \cap R_Y = \emptyset$.  What is this property called?
Comment: Good call.  Fixed.
Comment: Your $R_X$ and $R_Y$ do not make any sense, because they are not well defined.  The infima of those probabilities are *always* zero, no matter what the random variables $X$ and $Y$ might be, and--by definition of infimum--both $R_X$ and $R_Y$ are *numbers,* not subsets of real numbers, and definitely not the supports of the variables!  Often, the support of a variable is taken to be the smallest closed set with probability $1.$ Getting the definition correct is crucial because it determines whether the answer you posted is correct or not.
Comment: Thank you for noticing that.  I made an edit yesterday to a definition based on infimum, and didn't fully consider it I suppose!  Fixed.
Comment: Could you please explain what you mean by "arg inf"?  I would guess you mean something like $$R_X=\bigcap_{A\subset \mathbb R \mid P(X\in A)=1}\bar A$$ where $\bar A$ is the closure of $A.$ If that's the case, then having disjoint supports *implies,* but is not equivalent to, mutual singularity of the distributions.
Comment: [arg inf](https://math.stackexchange.com/a/1436838/426612) is similar to arg min.  Its result is the minimizing argument rather than the minimized quantity.  Also I believe the my answer only claims implication, not equivalence.  If you provide feedback to make that clearer, that would be wonderful.
Comment: The initial assertion in your answer takes the form of "this property *is*...;" one usually interprets "is" as "is equivalent to."  I agree you attempt a proof only in one direction.
Answer: After more research, the nearest property I can find is singularity of measure.  I don't think it's correct to say that random variables are themselves singular, but we can say it of their distributions.  I'll leave this answer unaccepted, just in case someone has a better answer.

Specifically, say the random variables $X, Y$ have the respective supports $R_X, R_Y$ disjoint as defined in the question, with distributions (or laws) $P^X, P^Y$.
Then $P^X, P^Y$ are singular.  Denoted $P^X \perp P^Y$
Proof:
By definition of support for probability distributions, we have that $P^X(R_X) = 1$.
Since we have assumed $R_X \cap R_Y = \emptyset$, we know that $R_Y \subset R_X^C$.
Also, $P^X(R_X^C) = 1 - P^X(R_X) = 1 - 1 = 0$.
Similarly, $P^Y(R_Y) = 1$ and $P^Y(R_Y^C) = 0$.
But $P^X(R_X^C) = 0 \implies P^X(R_Y) = 0$ by monotonicity of measure.
So we have that there exists $A,B \subset \mathbb{R}$ such that $A \cup B = \mathbb{R}$ and $P^X(A) = 0$ and $P^Y(B) = 0$.  So $P^X, P^Y$ are singular. $\square$

Note that, as per @whuber's comments, this only proves implication.  The converse is not true. As a counterexample, consider two random variables, one distributed uniformly over the closed unit interval, and the other distributed by Dirac delta centered in the midpoint:
$$
X \sim \text{Uniform}[0,1]\\
Y \sim \delta(\frac{1}{2})
$$
$P^X$ and $P^Y$ are mutually singular since $P^X(\{ \frac{1}{2} \}) = 0$ and $P^Y(\mathbb{R} \setminus \{ \frac{1}{2} \} ) = 0$, but the support of the former $R_X = [0,1]$ contains the support of the latter $R_Y = \{ \frac{1}{2} \}$.
Comment: +1. As an example of two distributions that have singular measures but whose supports are *not* disjoint, consider a Bernoulli variable $X$ and a variable $Y$ with uniform distribution on any open interval $U$ containing $0.$ The support of $Y$ is the closure of $U$ which has nonempty intersection with the support of $X$ (which is $\{0,1\}$).
Comment: Thanks for the example making non-equivalence clear.  That's important.  In fact I think I'll call that out in the answer explicitly.


How to Evaluate and Visualize the Positivity Assumption for the ATT

Question: I am trying to formally evaluate and visualize the satisfaction of the positivity assumption when estimating the ATT using R and I am having a tricky time figuring out how to do so. As Greifer and Stuart point out, the positivity assumption can be relaxed a bit for estimating the ATT because we would only need counterfactuals for the treated units. In their paper, along with others I have seen, positivity can be visually assessed with density plots overlaying the density of treated and non-treated units for some X variable.
If we have several variables, what can be done to evaluate the distribution of treatment across a combination of covariates? Can the propensity score be used for these purposes? If so, is the solution as simple as:

Estimate propensity scores
Create a density plot of propensity scores for treated units
Overlay density plot of propensity scores for non-treated units
For ATT, check the degree to which the two density plots overlap
Answer: For estimating the ATT, overlap means that the control gorup "encloses" the treated group; that is, the support of the treated group is in the support of the control group. The relaxedness of this requirement is that the control group doesn't need to be entirely in the support of the treated group; there can be control units that are vastly different from any treated units; they simply will be down-weighted or unmatched.
There is no universal way to assess overlap. For individual variables (including the propensity score), you can ensure that the range of the variable in the treated gorup is within the range of the variable in the control group. Ideally this "range" is not just the largest and smallest point, which may be outliers, but rather the range of the part of the distribution that contains enough data to make useful inferences without interpolating. One way to do this is to create a histogram or kernel density plot of the variable in each treated group and see that they overlap. The distributions don't have to be identical; that comes after matching or weighting. But there shouldn't be significant parts of the treated distribution that are outside the support of the control distribution.
There are many tools you can use to make such a plot, but the R package cobalt (of which I am the author) make it very easy using the bal.plot() function. See the documentation page for examples.


How to use Angular Store in Cypress e2e tests

Question: I need to inject a value (dispatch an action) in my Angular Store in order to make my Cypress e2e tests to work. Indeed, I need my store to contain { isLoggedIn : true } to access any page in the app, otherwise user is redirected to the login page.
I found a similar question here Cypress - programmatically manipulating Angular/NGRX app, but the answer does not seem to work in my case.
I always get the error: "cy.its() waited for the specified property store to exist, but it never did."
None of the console.log are printed in the console of cypress when running the test.
Here is my code in app.component.ts
export interface IWindowCypress {
  store: Store;
  Cypress: unknown;
}

@Component({
  selector: 'my-root',
  templateUrl: './app.component.html',
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class AppComponent {
  constructor(protected store: Store) {
    const windowWithStore: IWindowCypress = window as unknown as IWindowCypress;
    console.log('AppComponent');
    if (windowWithStore.Cypress) {
      console.log('save store');
      windowWithStore.store = store;
    }
  }
}

Here is my cypress test
describe('ContactListComponent', () => {
  before(() => {
    cy.window().its('store').invoke('dispatch', { type: '[User Effect] Load user success', user: { isLoggedIn : true } });
  });

// some tests

So I wonder what I am missing here or what I am doing wrong. Can anyone help ? Thank you !
PS: If I run my app with npm run start I have "AppComponent" printed in the console
Comment: Hi @TCH, I am stuck with the same issue, did you fond a way to fix this?
Comment: Unfortunately not. I gave up testing actions with Cypress, only UI stuffs
Answer: This difference between your code and the reference question is
window.store = this.store;   // referring to component instance store property

I'm not sure why the injected store isn't working, but try
export class AppComponent {
  constructor(protected store: Store) {
    const windowWithStore: IWindowCypress = window as unknown as IWindowCypress;
    if (windowWithStore.Cypress) {
      windowWithStore.store = this.store;
    }
  }
}

You might also want the interface properties to be optional since they are not always present.
export interface IWindowCypress {
  store?: Store;
  Cypress?: unknown;
}


Community dedicated to music transcriptions

Question: Do you happen to know any e-community related to music transcription? Many times I have doubts about parts of my transcriptions and it would be nice to seek others' opinion.
Comment: Requesting off site resources is off topic.
Answer: Transcriptions are derivatives of original recordings and the vast majority of "interesting" original recordings not played from score sheets (which would then have a copyright situation separate from the recording) would likely be still copyrighted as well.
So most such "e-communities" would likely be barred by copyright legislation from operating in a reasonably open manner.
Comment: That's too bad because it would be really cool to have a website that crowd sourced transcription help and checks of ones work. How do sites like ultimate-guitar post tab versions of those songs? Wouldn't the tab have the same issues. Or could it be because they're so bad they aren't close enough to the recording ;)
Comment: @b3ko Ultimate Guitar has licensing agreements with music publishers but sometimes they are forced to remove certain songs at the request of the artists. Many sites have been shutdown though. In the early 2000s Powertab and MySongBook both had some legal issues and did not have such agreements. Many tabs migrated to Ultimate Guitar, which if I recall correctly, was operating out of Russia at the time so the legality was a bit more complicated.
Comment: @b3ko - you mean they're supposed to represent the song in their title..?


How would you train for Chopin's Étude op. 10 no. 2?

Question: This Étude is really painful for the right hand. Difficulty comes from medius, ring and little fingers playing chromatic scales while thumb and index fingers punctuate.
The thumb cannot be used as a pivot while playing it. After a single execution the muscles are painful, because we are not accustomed to producing legato with only fingers 3, 4, and 5.
Is there a hand position that can help? I have heard that Asians have a different hand morphology that can help them for that piece, is this true?
Link: score (Public Domain) on IMSLP.
Comment: Not posting as an answer because it's kind of ridiculous: I looks like you could cross your arms and play the hands reversed, if your arms aren't too short.  That might introduce many other problems, but it would move the part in question to fingers 1-3.
Comment: @Matthew Read: I am sure you would be successful in a concert :) Still, doing fourth intervals with ring and little fingers is maybe a bit difficult.
Answer: I don't know why anyone hasn't brought this up. Buy the Cortot edition of the Chopin etudes. Cortot was one of the great interpreters of Chopin and each etude is prefaced by a large number of preparatory exercises. Make sure to combine this with another edition, because the Cortot is notorious for having misprinted notes.
I don't have the score in front of me, otherwise I'd cite some examples from the text. It's definitely worth the investment though, if you have the money.
Answer: The only advice I can give is the same advice I'd give for any piece that's painful while playing; take it slow, and speed up as it stops hurting.  The stretches to get the 3rd to cross over are a little uncomfortable, so practicing those sections slowly and holding the stretches will help you gain the flexibility to make those transitions without tensing.
Answer: Ascending: shift slightly forward (toward the fall board) when crossing over from a white key to a black key.
Descending: shift slightly backward (away from the fall board) when crossing under from a black key to a white key.

An example executing the first notes.
1. The hand starts out here.

2. Now finger 3 must cross finger 4.
2A. The painful way.
Attempting to move the hand in a directly lateral motion requires the most extreme crossover of the 4th over the 3rd finger. The problem is that finger 3 is blocked by the near edge of the B to its right.

2B. The comfy way.
Shift the hand very slightly forward and toward black key, leaving finger 3 in place by allowing it to curl a bit. This gives the clearance needed and, with a tiny rotation of the hand, brings finger 3 easily to its destination.1

3. All the rest of the notes
This strategy will work with 3 over 4, 3 over 5, and 4 over 5 when ascending. When descending, reverse the process, crossing the relevant finger under its neighbor: so 5 under 4, 5 under 3, and 4 under 3.

1As a general rule, twisting motions should be avoided. In this case, however, it is a very small movement, and necessary.


AuthenticateAsServerAsync is failing

Question: Accepting connections from browsers AJAX requests and establishing secure connection:
<code>var task = _stream.AuthenticateAsServerAsync(serverCertificate);
if (!task.Wait(60000, ct)) throw new Exception("Timeout");
</code>
one from ~20 clients with new browsers are disconnecting at this stage or connection times out. What can be wrong with this?
Comment: Another person on S/O said timeouts don't seem to work with this fn. http://stackoverflow.com/questions/37897425/authenticateasserverasync-hangs
Comment: From my observations task wait method is working as expected.
Answer: Turns out that this problem occur because browsers establish additional connections and not establishing ssl/tls. So everything works as expected and have to keep in mind that on AuthenticateAsServerAsync can be long waitings and timeouts.
Comment: Are you saying that AuthenticateAsServerAsync doesn't need to return for an SslStream to work properly?  If so, do you have any documentation to back that up?  I'm running into the same issue, but I'd need to show some proof.
Comment: Browsers are pooling tcp connections for future use and in most cases they do not use it.

AuthenticateAsServerAsync is failing


Are these Bayesian Inference?

Question: I am trying to understand what is and what is not considered Bayesian inference. Let say I am to estimate a parameter or a vector of parameters say $\theta$ and I have data on some features of the distribution say the first two moments $\hat{m_1}$ and $\hat{m_2}$. I think it is OK to use Bayes theorem to derive a sort of a posterior as follows
$p(\theta|\hat{m_1},\hat{m_2})\propto p(\hat{m_1},\hat{m_2}|\theta)p(\theta)$
where $p(\hat{m_1},\hat{m_2}|\theta)$ is the joint distribution of $\hat{m_1}$ and $\hat{m_1}$ conditional on $\theta$. Which one of these is not Bayesian inference?

use the actual distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula
use the Asymptotic distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula
How about the following when I have a frequentist estimate of $\theta$ and its distribution?

$p(\theta|\hat{\theta})\propto p(\hat{\theta}|\theta)p(\theta)$
Addendum: Here is an example to clarify the question: Suppose I want to learn about the parameters of the distribution of income given my data and let's assume it follows a parametric distribution with parameter $\theta$ . Often the data on income distribution is available only in the form of some summary statistics (e.g. deciles or Lorenz ordinates). for simplicity let say only 2 summary statistics of $\hat{m_1}$ and $\hat{m_2}$ are available. It is possible to obtain either the actual or more often asymptotic distribution of these summary statistics as a function of unknown $\theta$. Let's denote this with $p(\hat{m_1},\hat{m_2}|\theta)$. (1) and (2) above asks whether this can be used like a likelihood and proceed with the Bayesian inference. I personally think this is  a legitimate Bayesian analysis and I have seen papers doing such things.
Now if the above is legitimate Bayesian inference and the objective of Bayesian analysis is to incorporate prior information then I can use the above logic and easily make any frequentist estimator Bayesian without MCMC or any high power methods. I can calculate $\hat{\theta}$ (the frequentist estimate of $\theta$) and the frequentist analysis often gives me the (asymptotic) distribution of $\hat{\theta}$ as a function of $\theta$ which I denote with $p(\hat{\theta}|\theta)$. I can combine this (often a normal distribution) with prior $p(\theta)$ and do a simple Bayesian analysis. This does not seem right but what is wrong with it or what is that I am missing?
Addendum2: The latter may not as simple as I thought since the variance of the asymptotic normal distribution could depend on $\theta$ in a complicated way.
Comment: I believe this thread answers your question https://stats.stackexchange.com/questions/167051/who-are-the-bayesians doesn't it? TL;DR it is not about Bayes theorem.
Comment: Thanks for the link but it doesn't exactly answer my question. That link is about who can be called Bayesian whereas my question is whether the examples I gave make sense (or are interesting) in conventional Bayesian analysis.
Comment: Ok, but then how you want to use $p(\hat{m_1},\hat{m_2})$ and what it is? It is never used in your first equation.
Comment: $p(\hat{m_1},\hat{m_2}|\theta)$ is the joint distribution of $\hat{m_1}$ and $\hat{m_1}$ conditional on $\theta$ assuming this is available in (1) or its asymptotic distribution in (2). My own feeling is that (1) can lead to a reasonable Bayesian inference, (2) is still reasonable but it may be called approximate Bayesian inference. (3) it is  not interesting Bayesian analysis,
Answer: Let me split the answer into two parts.
Moments in the likelihood considered as regular data
First, I'll make it more general, and instead of considering
$$
p(\theta|\hat{m_1},\hat{m_2})\propto p(\hat{m_1},\hat{m_2}|\theta)\,p(\theta)
$$
I'll consider the general case
$$
p(\theta|X)\propto p(X|\theta) \,p(\theta)
$$
where $X$ can be any data. If you can define the likelihood function in terms of the moments $\hat{m_1},\hat{m_2}$ as the data (same as data about $n$ i.i.d. Bernoulli trials can be simplified to considering it as a binomial random variable for the total number of successes), then it is the same. In many cases, however, it might be hard to come up with such a likelihood function.

use the actual distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula

This question is confusing. Are you talking about the likelihood $p(\hat{m_1},\hat{m_2}|\theta)$ or the unconditional distribution $p(\hat{m_1},\hat{m_2})$?
If it is the latter, you cannot put unconditional distribution in the place of the conditional distribution, they are different things. It is hard to discuss it as a general case because it is not clear what you mean here and how exactly it refers to the initial problem. Say that you are interested only in the mode of the posterior (maximum a posterior estimation), so you are just maximizing the posterior probability, $p(\hat{m_1},\hat{m_2})$ cannot be maximized with regard of $\theta$ because it is not a function of $\theta$.
If you meant conditional distribution then lucky you! Usually, when deciding on a statistical model, you need to somehow come up with what the likelihood function should be for the problem. If you know it, you don't have to make this step.

use the Asymptotic distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula

Same as above. Unusually we don't "know" the likelihood function, but make some assumptions.

How about the following when I have a frequentist estimate of $\theta$ and its distribution?

First, in the frequentist approach, you would not have the estimate for the distribution of the parameter, because the parameter is not considered a random variable. You would have the point estimate. But answering the question, deciding on priors based on in-data statistics is called empirical Bayesian approach, it is controversial, but sometimes used.
Can we use moments in the likelihood function?
In some cases, it might be hard to come up with a likelihood function, but instead, the distribution can be characterized in other terms. In such cases, we can use likelihood-free inference and things like approximate Bayesian computation (ABC) (see also other threads for approximate-bayesian-computation). Your problem with using only the moments rather than the data might fall into this category. In ABC, the likelihood is replaced with a distance function that serves as a proxy for it.
Finally, "what is Bayesian inference?" is a variant of the Who Are The Bayesians? question that already got some nice answers. As you can learn from the linked thread, Bayesian statistics is not about using the Bayes theorem, but is more general than this. That is how ABC falls into this category, though it does not exactly use the Bayes theorem, just takes from the general idea behind it.
Comment: Thanks for the attempt to answer but I haven't fully got my answer yet. I have added some examples\explanations to clarify my question.
Comment: @user41838 my answer addresses exactly what you mentioned in the edit. The answer is abstract because you asked an abstract question. If you have a specific problem in mind, ask a specific question.
Comment: I believe you have answered number (1) and (2) but not (3). I still don't know from your answer that (3) is a legitimate Bayesian analysis. You say "First, in the frequentist approach, ..... estimate for the distribution of the parameter, because the parameter is not considered a random variable." I did not claim we get distribution of $\theta$ from a frequentist analysis, I said we can get distribution of $\hat{\theta}$. I don't also see the relevance of "answering the question, deciding on priors .... is called empirical Bayesian approach, it is controversial, but sometimes used".
Comment: @user41838 (3) is answered, if you create the priors based on estimates from the data, it's empirical Bayesian approach.
Comment: In which part of (3) i.e. $p(\theta|\hat{\theta})\propto p(\hat{\theta}|\theta)p(\theta)$  there is a prior that depends on data? The frequentist distribution of $\hat{\theta}$ i.e. $p(\hat{\theta}|\theta)$ is used as likelihood and  prior $p(\theta)$ does not depend on data.
Comment: @user41838 it is very hard to discuss an abstract case like this. "$p(\hat\theta|\theta)$" doesn't make much sense by itself, you cannot have the same parameter, estimated from the same data, on both sides. You gave an example, but it is not much less abstract. If you have a specific example in mind, maybe ask another question about the example. I can imagine a model for data in form of summary statistics (latent variable model, error in variables, etc) but it is hard to discuss the abstract case.

Are these Bayesian Inference?


Problemas com URLs no Django

Question: Caros,
Estou com o seguinte erro meu projeto:
django.core.exceptions.ImproperlyConfigured: The included URLconf
'api.urls' does not appear to have any patterns in it. If you see
valid patterns in the file then the issue is probably caused by a
circular import.

Este projeto funcionava corretamente, porém tive que formatar minha máquina. Após baixar novamente os arquivos do meu git, começou a dar este problema.
Segue minha api.url:
from django.contrib import admin
from django.urls import include, path
from rest_framework import routers
from ChatBot.views import UserViewSet
from ChatBot.views import GroupViewSet
from django.conf.urls import url

router = routers.DefaultRouter()
router.register(r'users', UserViewSet)
router.register(r'groups', GroupViewSet)

# Wire up our API using automatic URL routing.
# Additionally, we include login URLs for the browsable API.

urlpatterns = [
    path('', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework')),
    path('admin/', admin.site.urls),
    url(r'^', include('ChatBot.urls')),
]

Minha ChatBot.urls:
from django.conf.urls import url
from . import views

urlpatterns = [
    url(r'^resultado/$', views.ClassRankingView.as_view(), name='resultado'),
    url(r'^learning/$', views.WordEmbeddingCalculation.as_view(), name='learning'),
    url(r'^weight/$', views.NovaViewTeste.as_view(), name='weight'),
]

O erro completo:
Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/urls/resolvers.py", line 581, in url_patterns
    iter(patterns)
TypeError: 'module' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/utils/autoreload.py", line 54, in wrapper
    fn(*args, **kwargs)
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
    self.check(display_num_errors=True)
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/core/management/base.py", line 390, in check
    include_deployment_checks=include_deployment_checks,
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/core/management/base.py", line 377, in _run_checks
    return checks.run_checks(**kwargs)
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/core/checks/registry.py", line 72, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
    all_namespaces = _load_all_namespaces(resolver)
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
    url_patterns = getattr(resolver, 'url_patterns', [])
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/utils/functional.py", line 80, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/willian/ChatBot/venv/lib/python3.5/site-packages/django/urls/resolvers.py", line 588, in url_patterns
    raise ImproperlyConfigured(msg.format(name=self.urlconf_name))
django.core.exceptions.ImproperlyConfigured: The included URLconf 'api.urls' does not appear to have any patterns in it. If you see valid patterns in the file then the issue is probably caused by a circular import.

Este mesmo código, funciona em outras máquinas, porém na minha não. Saberiam me dizer o que pode estar acontecendo?
Comment: Você cita `api.url` e `api.urls` em diferentes momentos. Não seria apenas um erro de digitação/nomenclatura?
Answer: Tente o seguinte:
from django.contrib import admin
from django.urls import include, path
from rest_framework import routers
from ChatBot.views import UserViewSet
from ChatBot.views import GroupViewSet
#from django.conf.urls import url

router = routers.DefaultRouter()
router.register(r'users', UserViewSet)
router.register(r'groups', GroupViewSet)

# Wire up our API using automatic URL routing.
# Additionally, we include login URLs for the browsable API.

urlpatterns = [
    path('', include('router.urls')),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework')),
    path('admin/', admin.site.urls),
    path('', include('ChatBot.urls', namespace='chatbot')),
]

ChatBot.urls:
from django.conf.urls import url
from . import views

app_name = 'chatbot'

urlpatterns = [
    path('resultado/', views.ClassRankingView.as_view(), name='resultado'),
    path('learning/', views.WordEmbeddingCalculation.as_view(), name='learning'),
    path('weight/', views.NovaViewTeste.as_view(), name='weight'),
]

Creio que usando path resolva o problema e o "namespace" poderá auxiliar ao referenciar as urls no template, por exemplo, você poderá referenciar a url "weight" utilizando {% url 'chatbot:weight' %}.
Comment: muito obrigado pela ajuda. Testei o que tu mandou, porém continua dando o erro. O que eu fiz para resolver, foi limpar minhas urls e tirar as que não estava utilizando, deixando apenas as que iram realmente ser utilizadas. E mesmo assim, algumas vezes ainda ocorre o erro. Devo ter configurado algo errado em minha máquina.

Problemas com URLs no Django


OAuth2 with Spring security causing redirect loop

Question: I have a Java Spring application. I tried to configure OAuth2. So, far I have got the redirect to the SS) page correctly. Afterwards, if I enter the correct username and pwd, it acknowledges my login and then I get a redirect loop.
application.yml
<code>authorization-grant-type: authorization_code
redirect-uri: http://localhost:1234/oauth2/callback
</code>
I have set the authorization uri and token uri correctly.
This is my OAuthSecurityConfiguration.java code
<code>@Configuration
public class OAuthSecurityConfiguration {

    @Bean
    public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
        http.anonymous().disable();
        http.authorizeRequests()
                .antMatchers("/my-gui/**").authenticated()
                .anyRequest().permitAll()
                .and()
                .oauth2Login();
        return http.build();
    }

    @Bean
    WebClient webClient(ClientRegistrationRepository clientRegistrationRepository, OAuth2AuthorizedClientRepository authorizedClientRepository) {
        ServletOAuth2AuthorizedClientExchangeFilterFunction oauth2 = new ServletOAuth2AuthorizedClientExchangeFilterFunction(clientRegistrationRepository, authorizedClientRepository);
        oauth2.setDefaultOAuth2AuthorizedClient(true);
        return WebClient.builder()
                .apply(oauth2.oauth2Configuration())
                .build();
    }
}
</code>
This is my OAuthController code
<code>@Controller
public class OAuthController {

    @Autowired
    private OAuth2AuthorizedClientService authorizedClientService;

    @GetMapping("/oauth2/callback")
    public String handleCallback(HttpServletRequest request, Model model) {
        // Get the OAuth2AuthenticationToken from the request
        OAuth2AuthenticationToken authentication = (OAuth2AuthenticationToken) SecurityContextHolder.getContext().getAuthentication();
        // Get the authorized client from the authentication
        OAuth2AuthorizedClient authorizedClient = authorizedClientService.loadAuthorizedClient(authentication.getAuthorizedClientRegistrationId(), authentication.getName());

        // Redirect to the protected page with the access token included as a query parameter
        return "redirect:/my-gui/index.html?access_token=" + authorizedClient.getAccessToken().getTokenValue();
    }
}
</code>
So, I try to open localhost:1234/my-gui/index.html. This redirects me to an SSO page which is th eexpected behaviour. Here, when I enter the correct username and pwd, I expect it to open localhost:1234/my-gui/index.html which is why I redirect in OAuthController as well. But it just throws me a redirect loop.
I think it is due to the redirect going to a page that needs authentication and causes the loop. Basically, what I want to achieve is have OAuth authentication on localhost:1234/my-gui/index.html and when someone enters the correct pwd, they should see the page.
Answer: I had a similar issue and the challenge was that I did not have direct access to register redirect URIs in our provider (Azure). This limited my tests so please consider the following only as a reference.
I ended up requesting to add three redirect URIs and only one solved my issue. It seems that Spring is a little strict in the redirect URI formats. I ended up using redirect URI that matches the default which is:
<code>{baseUrl}/login/oauth2/code/{registrationId}
</code>
Spring also has an option to customize this redirect URI (haven't tested but I would assume the format also matters):
https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-redirection-endpoint


leitura de arquivo com scanner

Question: adoraria saber se alguém pode me ajudar com esse código abaixo, na minha aplicação o usuário com frequência irá subir alguns arquivos e eles tem duas informações, numero e nome separados por  -> ;
porem no arquivo eu simulo alguns possíveis problemas como números   sem nome e meu código não se comporta bem contra esses possíveis problemas.
Outro detalhe é que meu SOUT esta duplicando e eu não sei porque.
Se alguém puder me ajudar, agradeço.
    public void adicionaArquivo(MultipartFile file) throws IOException {

    InputStream inputStream = file.getInputStream();
    List<ContatoDTO> contato = new ArrayList<>();

    //cria um scanner para ler o arquivo
    Scanner leitor = new Scanner(inputStream);
    //variavel que armazenara as linhas do arquivo
    String linhasDoArquivo = new String();

    //percorre todo o arquivo
    while (leitor.hasNext()) {

        try {
            //recebe cada linha do arquivo
            linhasDoArquivo = leitor.nextLine();

            //separa os campos entre as virgulas de cada linha
            String[] valoresEntreVirgulas = linhasDoArquivo.split(";");

            for (int i = 0; i < valoresEntreVirgulas.length; i++) {
                ContatoDTO c = new ContatoDTO();
                c.setTelefone(valoresEntreVirgulas[0]);
                c.setNome(valoresEntreVirgulas[1]);
                contato.add(c);
            }

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    for (int i = 0; i < contato.size(); i++) {
        System.out.println("telefone : " + contato.get(i).getTelefone());
        System.out.println("Nome : " + contato.get(i).getNome());
    }
    System.out.println("qt de linhas : " + contato.size());
}

meu arquivo
71999998811;nome1
71999998812;nome2
71999998813;nome3
71999998814;nome4
71999998815;nome5
71999998816;nome6
71999998817;nome7
71999998818;nome8
71999998819;nome9
71999998810;nome10

71999998810;nome11
71999998810
71999998810;
71999998810;nome14
71999998855;nome15

meu log de erro q ficou grande
https://gist.github.com/fariasrod/d9d41e36856845327aa14110af5e3982
Answer: O problema é que você tenta acessar um index que não necessariamente existe (nome). Dá pra fazer:
//separa os campos entre as virgulas de cada linha
String[] valoresEntreVirgulas = linhasDoArquivo.split(";");

ContatoDTO c = new ContatoDTO();
if(valoresEntreVirgulas.length > 0) {
    c.setTelefone(valoresEntreVirgulas[0]);
}
if(valoresEntreVirgulas.length > 1) {
    c.setNome(valoresEntreVirgulas[1]);
}
contato.add(c);

Ou usar regex:
public void adicionaArquivo(MultipartFile file) throws IOException {
    Pattern pattern = Pattern.compile("^([\\d]+)(?:;*)(.*)$");
    ....
    ....
    Matcher matcher = pattern.matcher(string);
    if (matcher.matches()) {
        ContatoDTO c = new ContatoDTO();
        if (matcher.groupCount() >= 1) {
          c.setTelefone(matcher.group(1));
        }

        if (matcher.groupCount() >= 2) {
          c.setNome(matcher.group(2));
        }
        contato.add(c);
    }

leitura de arquivo com scanner


Calculate heat flow through the sphere

Question: $T(x,y,z)=x$, $S=\{(x,y,z): x^2+y^2+z^2=1\}$, I also got a formula for heat flow $\int \int_S (-k \nabla T)\circ dS$ for some parameter $k$.
My attempt:
$x=sin\phi cos\theta$, $y=sin\phi sin\theta$, $z=cos\phi$ for $0\leq \theta \leq 2\pi$ and $0\leq \phi \leq \pi$,
$\nabla T=\hat{i}$, $S(\phi, \theta)=\hat{i}sin\phi cos\theta +\hat{j}sin\phi sin\theta+\hat{k}cos\theta$,
$dS=\frac{\partial S}{\partial \phi}d\phi \times \frac{\partial S}{\partial \theta}d\theta=(\hat{i}cos\phi cos\theta+\hat{j} cos\phi sin\theta - \hat{k}sin\phi)d\phi \times (-\hat{i}sin\phi sin\theta+\hat{j}sin\phi cos\theta)d\theta=$
$=(\hat{i}sin^2\phi cos\theta + \hat{j} sin^2 \phi sin\theta+\hat{k}sin\phi cos\phi)d\phi d\theta$.
$\nabla T \circ dS=sin^2\phi cos\theta d\phi d\theta$, $\int \int_S (-k \nabla T)\circ dS=k\int_0^{2\pi}\int_0^{\pi}sin^2\phi cos\theta d\phi d\theta=k\int_0^{2\pi}\frac{\pi}{2} cos\theta d\theta=0$
Is this correct? Thanks in advance.
Answer: Yes there are two ways to look at it without direct computation
$\textbf{Option 1}$: Surface integral
Using $n = (x,y,z)$ for the unit sphere we have that
$$\iint -k\nabla T \cdot n dS = -k \iint x dS$$
$x$ is an odd function and the sphere has $x$ symmetry i.e. $(-x)^2 + y^2 +z^2 = 1 \iff x^2+y^2+z^2=1$ so the integral is $0$ automatically.
$\textbf{Option 2}$: Divergence theorem
By Divergence theorem we have that
$$-k\iint \nabla T \cdot n dS = -k \iiint \Delta T dV = -k \iiint 0 \:dV = 0$$

Calculate heat flow through the sphere


How to specify logical types when writing Parquet files from PyArrow?

Question: I'm using PyArrow to write Parquet files from some Pandas dataframes in Python.
Is there a way that I can specify the logical types that are written to the parquet file?
For for example, writing an np.uint32 column in PyArrow results in an INT64 column in the parquet file, whereas writing the same using the fastparquet module results in an INT32 column with a logical type of UINT_32 (this is the behaviour I'm after from PyArrow).
E.g.:
import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq
import fastparquet as fp
import numpy as np

df = pd.DataFrame.from_records(data=[(1, 'foo'), (2, 'bar')], columns=['id', 'name'])
df['id'] = df['id'].astype(np.uint32)

# write parquet file using PyArrow
pq.write_table(pa.Table.from_pandas(df, preserve_index=False), 'pyarrow.parquet')

# write parquet file using fastparquet
fp.write('fastparquet.parquet', df)

# print schemas of both written files
print('PyArrow:', pq.ParquetFile('pyarrow.parquet').schema)
print('fastparquet:', pq.ParquetFile('fastparquet.parquet').schema)

this outputs:
PyArrow: <pyarrow._parquet.ParquetSchema object at 0x10ecf9048>
id: INT64
name: BYTE_ARRAY UTF8

fastparquet: <pyarrow._parquet.ParquetSchema object at 0x10f322848>
id: INT32 UINT_32
name: BYTE_ARRAY UTF8

I'm having similar issues with other column types, so really looking for a generic way to specify the logical types that are used when writing using PyArrow.
Comment: @danodonovan not that I know of, just happen to be doing other lower level things with pyarrow already, so was easier to do all the writing via that rather than Pandas.
Comment: Just out of interest, is there a benefit to writing files directly through pyarrow (ie rather than using `pd.to_parquet`) ?
Answer: PyArrow defaults to writing parquet version 1.0 files by default, and version 2.0 is needed to use the UINT_32 logical type.
The solution is to specify the version when writing the table, i.e.
pq.write_table(pa.Table.from_pandas(df, preserve_index=False), 'pyarrow.parquet', version='2.0')

This then results in the expected parquet schema being written.


Java: selective synchronization of method based on parameter value

Question: I'm writing a java app that send some commands to a set of radios. In some cases, if a certain command is sent to a radio, the app needs to wait for the answer before sending a similar command to the same radio. However the same command can be sent to another radio without causing trouble.
Since each radio is identified by its MAC address, my goal is to synchronize the execution of the command sender method writeRemoteRegister(MAC,value) based on the MAC value passed to it. Is it possible somehow?
Thanks in advance!
Comment: I would probably use a [Lock](http://docs.oracle.com/javase/tutorial/essential/concurrency/newlocks.html).
Comment: This is really hard to achieve, because of race conditions. While you're checking if your message is 'special' (i.e. requires synchronization), the next message for that radio could arrive, and be processed, before the check for specialness completes, thus throwing your scheme out. AFAIK: All the answers provided here suffer from this problem, and it may be intractable unless, in this specific case, you can guarantee this won't happen.
Comment: It is, but what have you coded so far?
Comment: Yes. But this isn't typically a use case for the Java Monitor, and on the surface appears somewhat convoluted. You would probably be better of using a state machine, rather than coercing the monitor to suit your purpose.
Comment: @EngineerDollery it is not clear from the question whether out-of-order sending of messages would be an issue. All the solutions mentioned serialize the messages successfully, which is all that was asked for AFAICT.
Comment: @wds Yeah, but the OP said that if a special command is received, all subsequent similar commands /must be/ serialized, which isn't possible with partial serialization because of the race condition on the check. A subsequent message could arrive during the check and complete its own check before the first message gained the lock, thus causing out of order delivery. As you say, it may not be a specific issue, so this is a cautionary note rather than an answer.
Answer: I would use a map of <MAC, Semaphores> using initial value 1.
The write would acquireand release respectively in the method.
Here's a link in case you are not familar with them
http://tutorials.jenkov.com/java-util-concurrent/semaphore.html
Answer: You could store java.util.concurrent.locks.Lock implementation in a HashMap keyed by the MAC. Make sure that you also synchronise access to the HashMap, otherwise you'll have a possible race condition when you first encounter the MAC address.
But the Object-Oriented solution is to put the method writeRemoteRegister in the MAC class and make it synchronized.
Comment: If you assume there's a Mac class, why not use the monitor lock (`synchronized(instance)`).
Comment: The mapped `Locks` worked well, thank you! As for the second solution, `MAC` is actually an instance of `BigInteger`, so I'd rather keep all comm methods gathered in a single class.
Comment: @wds good point. However you would probably still need a Map from MAC to MAC in that case, to make sure that you always use the same instance of the MAC object (you would parse a MAC first, creating a new instance, but you would need the existing instance to synchronize on)
Answer: Simple example:
public void x(String y) {
    if (y == null) { ... Something here }
    else if (y.isEmpty()) { ... something here }
    else if (y.equals("ABC1234)) {
       synchronized(y) {

       }
    } else {

    }
}

Comment: That won't work. I can pass two different string instances obeying the same equality.
Comment: Might be a bit to simple to avoid race conditions.. You need to be synchronized (or otherwise guarded) when you check conditions.
Comment: It's a possible solution @Mike, but I'm guessing it would be hard to accomplish without hard-coding every radio's `MAC`, wich is not a possibility.
Answer: If I understood correctly you can simple synchronized on MAC value. I am assuming you are getting string as MAC value. So you can lock on that string. To make it more safe you can use String .intern.
synchronized(String.intern(MAC.toString())){
   // do your stuff
}


method boolean java.lang.String.equals(java.lang.Object)' on a null object reference

Question:  if(response.isSuccessful())
               {
                   if(response.body().getSuccess().equalsIgnoreCase("1")||response.body().getSuccess().equalsIgnoreCase("2"))
                   {

                       String message = response.body().getMessage();
                       // Snackbar.make(parentView, message, Snackbar.LENGTH_SHORT).show();
                       Toast.makeText(Login.this,message,Toast.LENGTH_SHORT).show();
                       Intent i = new Intent(Login.this, DashBoard.class);
                       startActivity(i);
                       finish();
                       overridePendingTransition(R.anim.enter_from_right, R.anim.exit_out_right);
                   }
                   else if(response.body().getSuccess().equalsIgnoreCase("0"))
                   {
                       String message = response.body().getMessage();
                       // Snackbar.make(parentView, message, Snackbar.LENGTH_SHORT).show();
                       Toast.makeText(Login.this,message,Toast.LENGTH_SHORT).show();
                   }else {
                       Toast.makeText(Login.this, getString(R.string.unexpected_response), Toast.LENGTH_SHORT).show();
                   }

               }else {
                   Toast.makeText(Login.this, response.message(), Toast.LENGTH_LONG).show();
               }
           }

Comment: probably ... `response.body().getSuccess()` returns null.
Comment: Possible duplicate of [What is a NullPointerException, and how do I fix it?](https://stackoverflow.com/questions/218384/what-is-a-nullpointerexception-and-how-do-i-fix-it)
Comment: paste your logcat please
Comment: Would be useful to know of what class type 'response' is as well.  Throw us a crumb.
Answer: Probably response.body().getSuccess() returning null here .
change your logic as.
 if(response.isSuccessful() && response.body().getSuccess() != null)
               {
..

Edits
and as per Andy's answer
Toast.makeText(Login.this, response.message(), Toast.LENGTH_LONG).show();

also can cause exception in case of null.
Comment: Doesn't quite match error in title and the OP code is already guarded by isSuccessful.
Comment: @Andy ... ya but not with `response.body().getSuccess()` null value.
Answer: Change your equal condition like this, And exception will be handled automatically
 if (response.isSuccessful()) {
        if ("1".equalsIgnoreCase(response.body().getSuccess()) || "2".equalsIgnoreCase(response.body().getSuccess())) {
            //do your work
        } else if ("0".equalsIgnoreCase(response.body().getSuccess())) {
            //do your work
        }else {
            //do your work
         // Toast.makeText(Login.this, getString(R.string.unexpected_response), Toast.LENGTH_SHORT).show();
        }
    } else {
        //do your work
    }

Hope it helps you.
Answer: This code executes when "not successful" which probably means you are passing a null to the toaster:
}else {
  Toast.makeText(Login.this, response.message(), Toast.LENGTH_LONG).show();
}


Why is it necessary to learn scales?

Question: I am in my middle age, started to learn guitar only 6 months ago. I do not have any career ambition in music instead it's more like a hobby. So far I have managed to learn quite a lot  when it comes to guitar -chords, strumming patterns, scales, Intervals, Circle of Fifths, fret-board knowledge, octaves, unisons, Barre chords, CAGED system, basics of Key, finger-muscle building practice exercises etc. As you know 6 months is not enough for anyone to be an expert in this area. But I feel confident that I know a lot of music theory now but my playing skills are still at the beginner's level
Coming to the crux of this post, I have been doing research on the necessity to learn the scales. In fact I am reasonably comfortable with practicing scales now (all the 7 positions up and down the neck). Everyone seems to say master the scales as much as possible but as I try to practice them I see no obvious benefits in learning them. Mind you, as I said I have no plans of writing/creating my own music, instead my interest is only on learning a few songs which I can hopefully play at events in the future.
So my hypothetical question is this - If I were to learn let's say for example Nothing Else Matters by Metallica, how exactly is the knowledge of scales going to help me here? Before I started to learn the scales, I learnt to play a few other songs such as Romeo and Juliet by The Killers, Comfortably Numb and a few others. It's not at the expert level, nevertheless I would like to think it's not too bad for someone who touched the guitar barely 6 months ago. So far I was learning to play melody by memorising all the notes so I would like to know if scales help at all here?
This question I am asking in this post is only to help me get the idea of how exactly scales help me in this situation. Any thoughts? Thanks in advance.
Ps. By the way, a big thanks to all those gentle souls on YouTube, and forums such as this, for helping people like me learn music - a few years ago I would never even have imagined such a way to learn something was possible! May you be abundantly blessed!
Comment: Scales are by far the most important construct in music as both the melody and harmony is derived from them. Nothing Else Matters is played in E minor. As you listen to the song, play notes from this scale and you'll see that it fits with the melody and harmony. you'll also find that all the chords that this song uses are diatonic chords of E minor. note that the v chord is usually played as V chord, so B major instead of B minor. (harmonic minor) http://www.piano-keyboard-guide.com/key-of-e-minor.html
Comment: Thank you. I think I am getting the overall idea now.
Answer: Welcome! By and large, mainstream music contains songs that use notes from one key. Diatonic notes. By knowing the scales, one can cut out the other notes which probably wouldn't feature in a particular song. Those are called the chromatic notes.
By knowing major, three minors, maj. and min. pentatonics, maj. and min. blues scales, one can isolate the working notes for particular tunes.The scales themselves aren't a lot of use, as not many pieces just go up and down scales. But humans like to organise, so a set of notes which constitute say, F# major, we play up and down in order as the F# major scale. When it's done enough, that set of notes becomes implanted in fingers and brain, so when we play a piece in F# major, we know which notes are very likely to be used in it.
Comment: Thank you. That helped me get a sense of perspective. So basically what you are saying is that scales help us confine our fingers to a subset of relevant notes from the natural notes for a particular tune (scale) so it helps us to be in the right area of the fret-board and this happens subconsciously over time?
Comment: @Steve when you listen to a song it *usually* is just played using 7 notes (a scale) that includes both the melody (the singing) and harmony (the chords), so it doesn't use all 12 chromatic notes of an octave. that's why scales are so relevant.

Why is it necessary to learn scales?


Google Map Javascript API v3 - OVER_QUERY_LIMIT Error

Question: I am displaying multiple locations in the map by iterating a for loop. After doing that few times using Google Map API v3 javascript I am getting this javascript error
Comment: possible duplicate of [OVER_QUERY_LIMIT while using google maps](http://stackoverflow.com/questions/3529746/over-query-limit-while-using-google-maps)
Answer: This has been answered in the posting below:
OVER_QUERY_LIMIT while using google maps
To save you a trip: you are being throttled by the API.  Batch the locations and make a single request.


Identify columns with identical values and iterate over a column saving changes in the original table

Question: I have a data frame like this
    Chr Start   Ref Alt Revel_Score RefSeq  CHR BP  SNP P
21  9   133710834   A   C   0.429   NM_005157.6 9   133710834   1   0.571001
22  9   133710834   A   G   0.424   NM_005157.6 9   133710834   1   0.576001
23  9   133710834   A   T   0.432   NM_005157.6 9   133710834   1   0.568001
24  9   133710835   T   A   0.395   NM_005157.6 9   133710835   1   0.605001
25  9   133710835   T   C   0.386   NM_005157.6 9   133710835   1   0.614001
26  9   133710835   T   G   0.389   NM_005157.6 9   133710835   1   0.611001
27  9   133710836   G   A   0.429   NM_005157.6 9   133710836   1   0.571001
28  9   133710836   G   C   0.418   NM_005157.6 9   133710836   1   0.582001
29  9   133710836   G   T   0.418   NM_005157.6 9   133710836   1   0.582001
30  9   133710837   T   A   0.381   NM_005157.6 9   133710837   1   0.619001
31  9   133710837   T   G   0.278   NM_005157.6 9   133710837   1   0.722001
32  9   133710838   T   C   0.378   NM_005157.6 9   133710838   1   0.622001
33  9   133710838   T   G   0.327   NM_005157.6 9   133710838   1   0.673001
34  9   133710839   G   C   0.352   NM_005157.6 9   133710839   1   0.648001
35  9   133710839   G   T   0.352   NM_005157.6 9   133710839   1   0.648001

# The same in a dict to allow to work
IGV_table_limited.head(15).to_dict()

{'Chr': {21: '9',
  22: '9',
  23: '9',
  24: '9',
  25: '9',
  26: '9',
  27: '9',
  28: '9',
  29: '9',
  30: '9',
  31: '9',
  32: '9',
  33: '9',
  34: '9',
  35: '9'},
 'Start': {21: 133710834,
  22: 133710834,
  23: 133710834,
  24: 133710835,
  25: 133710835,
  26: 133710835,
  27: 133710836,
  28: 133710836,
  29: 133710836,
  30: 133710837,
  31: 133710837,
  32: 133710838,
  33: 133710838,
  34: 133710839,
  35: 133710839},
 'Ref': {21: 'A',
  22: 'A',
  23: 'A',
  24: 'T',
  25: 'T',
  26: 'T',
  27: 'G',
  28: 'G',
  29: 'G',
  30: 'T',
  31: 'T',
  32: 'T',
  33: 'T',
  34: 'G',
  35: 'G'},
 'Alt': {21: 'C',
  22: 'G',
  23: 'T',
  24: 'A',
  25: 'C',
  26: 'G',
  27: 'A',
  28: 'C',
  29: 'T',
  30: 'A',
  31: 'G',
  32: 'C',
  33: 'G',
  34: 'C',
  35: 'T'},
 'Revel_Score': {21: 0.429,
  22: 0.424,
  23: 0.432,
  24: 0.395,
  25: 0.386,
  26: 0.389,
  27: 0.429,
  28: 0.418,
  29: 0.418,
  30: 0.381,
  31: 0.278,
  32: 0.378,
  33: 0.327,
  34: 0.352,
  35: 0.352},
 'RefSeq': {21: 'NM_005157.6',
  22: 'NM_005157.6',
  23: 'NM_005157.6',
  24: 'NM_005157.6',
  25: 'NM_005157.6',
  26: 'NM_005157.6',
  27: 'NM_005157.6',
  28: 'NM_005157.6',
  29: 'NM_005157.6',
  30: 'NM_005157.6',
  31: 'NM_005157.6',
  32: 'NM_005157.6',
  33: 'NM_005157.6',
  34: 'NM_005157.6',
  35: 'NM_005157.6'},
 'CHR': {21: '9',
  22: '9',
  23: '9',
  24: '9',
  25: '9',
  26: '9',
  27: '9',
  28: '9',
  29: '9',
  30: '9',
  31: '9',
  32: '9',
  33: '9',
  34: '9',
  35: '9'},
 'BP': {21: 133710834,
  22: 133710834,
  23: 133710834,
  24: 133710835,
  25: 133710835,
  26: 133710835,
  27: 133710836,
  28: 133710836,
  29: 133710836,
  30: 133710837,
  31: 133710837,
  32: 133710838,
  33: 133710838,
  34: 133710839,
  35: 133710839},
 'SNP': {21: 1,
  22: 1,
  23: 1,
  24: 1,
  25: 1,
  26: 1,
  27: 1,
  28: 1,
  29: 1,
  30: 1,
  31: 1,
  32: 1,
  33: 1,
  34: 1,
  35: 1},
 'P': {21: 0.571001,
  22: 0.5760010000000001,
  23: 0.5680010000000001,
  24: 0.605001,
  25: 0.614001,
  26: 0.611001,
  27: 0.571001,
  28: 0.5820010000000001,
  29: 0.5820010000000001,
  30: 0.619001,
  31: 0.722001,
  32: 0.622001,
  33: 0.6730010000000001,
  34: 0.648001,
  35: 0.648001}}

I want to do the following thing
If the values in the columns Chr, Start, Ref and Revel_Score are the same, modify the values in column P by adding a random number between 0.01 to 0.1
So the condition occurred in row number 28, 29 and  34, 35 so do 0.08 + 0.582001, 0.06 + 0.582001, 0.05 + 0.648001 and 0.07 + 0.648001
   Chr Start   Ref Alt Revel_Score RefSeq  CHR BP  SNP P
21  9   133710834   A   C   0.429   NM_005157.6 9   133710834   1   0.571001
22  9   133710834   A   G   0.424   NM_005157.6 9   133710834   1   0.576001
23  9   133710834   A   T   0.432   NM_005157.6 9   133710834   1   0.568001
24  9   133710835   T   A   0.395   NM_005157.6 9   133710835   1   0.605001
25  9   133710835   T   C   0.386   NM_005157.6 9   133710835   1   0.614001
26  9   133710835   T   G   0.389   NM_005157.6 9   133710835   1   0.611001
27  9   133710836   G   A   0.429   NM_005157.6 9   133710836   1   0.571001
28  9   133710836   G   C   0.418   NM_005157.6 9   133710836   1    0.662001
29  9   133710836   G   T   0.418   NM_005157.6 9   133710836   1    0.642001
30  9   133710837   T   A   0.381   NM_005157.6 9   133710837   1   0.619001
31  9   133710837   T   G   0.278   NM_005157.6 9   133710837   1   0.722001
32  9   133710838   T   C   0.378   NM_005157.6 9   133710838   1   0.622001
33  9   133710838   T   G   0.327   NM_005157.6 9   133710838   1   0.673001
34  9   133710839   G   C   0.352   NM_005157.6 9   133710839   1    0.698001
35  9   133710839   G   T   0.352   NM_005157.6 9   133710839   1    0.718001

I know how to apply the condition

IGV_table_limited[IGV_table_limited.duplicated(subset=['Chr','Start', 'Ref','Revel_Score'  ], keep=False)]

And I know how to do the maths in that column but I don't know how to get the original table with the changes
Answer: Use numpy.random.normal with DataFrame.loc, for count Trues use sum:
m = IGV_table_limited.duplicated(subset=['Chr','Start', 'Ref','Revel_Score'  ], keep=False)

IGV_table_limited.loc[m, 'P'] += np.random.normal(0.01, 0.1, size=m.sum())

If need exactly step 0.01 use numpy.random.choice:
arr = np.arange(0.01, 0.1 + 0.01, 0.01)
print (arr)
[0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 ]

IGV_table_limited.loc[m, 'P'] += np.random.choice(arr, size=m.sum())

I test data with changed index 28 in P column and no nagative values:
d = {'Chr': {21: '9',
  22: '9',
  23: '9',
  24: '9',
  25: '9',
  26: '9',
  27: '9',
  28: '9',
  29: '9',
  30: '9',
  31: '9',
  32: '9',
  33: '9',
  34: '9',
  35: '9'},
 'Start': {21: 133710834,
  22: 133710834,
  23: 133710834,
  24: 133710835,
  25: 133710835,
  26: 133710835,
  27: 133710836,
  28: 133710836,
  29: 133710836,
  30: 133710837,
  31: 133710837,
  32: 133710838,
  33: 133710838,
  34: 133710839,
  35: 133710839},
 'Ref': {21: 'A',
  22: 'A',
  23: 'A',
  24: 'T',
  25: 'T',
  26: 'T',
  27: 'G',
  28: 'G',
  29: 'G',
  30: 'T',
  31: 'T',
  32: 'T',
  33: 'T',
  34: 'G',
  35: 'G'},
 'Alt': {21: 'C',
  22: 'G',
  23: 'T',
  24: 'A',
  25: 'C',
  26: 'G',
  27: 'A',
  28: 'C',
  29: 'T',
  30: 'A',
  31: 'G',
  32: 'C',
  33: 'G',
  34: 'C',
  35: 'T'},
 'Revel_Score': {21: 0.429,
  22: 0.424,
  23: 0.432,
  24: 0.395,
  25: 0.386,
  26: 0.389,
  27: 0.429,
  28: 0.418,
  29: 0.418,
  30: 0.381,
  31: 0.278,
  32: 0.378,
  33: 0.327,
  34: 0.352,
  35: 0.352},
 'RefSeq': {21: 'NM_005157.6',
  22: 'NM_005157.6',
  23: 'NM_005157.6',
  24: 'NM_005157.6',
  25: 'NM_005157.6',
  26: 'NM_005157.6',
  27: 'NM_005157.6',
  28: 'NM_005157.6',
  29: 'NM_005157.6',
  30: 'NM_005157.6',
  31: 'NM_005157.6',
  32: 'NM_005157.6',
  33: 'NM_005157.6',
  34: 'NM_005157.6',
  35: 'NM_005157.6'},
 'CHR': {21: '9',
  22: '9',
  23: '9',
  24: '9',
  25: '9',
  26: '9',
  27: '9',
  28: '9',
  29: '9',
  30: '9',
  31: '9',
  32: '9',
  33: '9',
  34: '9',
  35: '9'},
 'BP': {21: 133710834,
  22: 133710834,
  23: 133710834,
  24: 133710835,
  25: 133710835,
  26: 133710835,
  27: 133710836,
  28: 133710836,
  29: 133710836,
  30: 133710837,
  31: 133710837,
  32: 133710838,
  33: 133710838,
  34: 133710839,
  35: 133710839},
 'SNP': {21: 1,
  22: 1,
  23: 1,
  24: 1,
  25: 1,
  26: 1,
  27: 1,
  28: 1,
  29: 1,
  30: 1,
  31: 1,
  32: 1,
  33: 1,
  34: 1,
  35: 1},
 'P': {21: 0.571001,
  22: 0.5760010000000001,
  23: 0.5680010000000001,
  24: 0.605001,
  25: 0.614001,
  26: 0.611001,
  27: 0.571001,
  28: 0.3220009999999999,
  29: 0.5820010000000001,
  30: 0.619001,
  31: 0.722001,
  32: 0.622001,
  33: 0.6730010000000001,
  34: 0.648001,
  35: 0.648001}}

IGV_table_limited = pd.DataFrame(d)

m = IGV_table_limited.duplicated(subset=['Chr','Start', 'Ref','Revel_Score'],keep=False)

IGV_table_limited.loc[m, 'P'] += np.random.normal(0.01, 0.1, size=m.sum())
print (IGV_table_limited)
   Chr      Start Ref Alt  Revel_Score       RefSeq CHR         BP  SNP  \
21   9  133710834   A   C        0.429  NM_005157.6   9  133710834    1
22   9  133710834   A   G        0.424  NM_005157.6   9  133710834    1
23   9  133710834   A   T        0.432  NM_005157.6   9  133710834    1
24   9  133710835   T   A        0.395  NM_005157.6   9  133710835    1
25   9  133710835   T   C        0.386  NM_005157.6   9  133710835    1
26   9  133710835   T   G        0.389  NM_005157.6   9  133710835    1
27   9  133710836   G   A        0.429  NM_005157.6   9  133710836    1
28   9  133710836   G   C        0.418  NM_005157.6   9  133710836    1
29   9  133710836   G   T        0.418  NM_005157.6   9  133710836    1
30   9  133710837   T   A        0.381  NM_005157.6   9  133710837    1
31   9  133710837   T   G        0.278  NM_005157.6   9  133710837    1
32   9  133710838   T   C        0.378  NM_005157.6   9  133710838    1
33   9  133710838   T   G        0.327  NM_005157.6   9  133710838    1
34   9  133710839   G   C        0.352  NM_005157.6   9  133710839    1
35   9  133710839   G   T        0.352  NM_005157.6   9  133710839    1

           P
21  0.571001
22  0.576001
23  0.568001
24  0.605001
25  0.614001
26  0.611001
27  0.571001
28  0.412194
29  0.576668
30  0.619001
31  0.722001
32  0.622001
33  0.673001
34  0.786122
35  0.707951

IGV_table_limited = pd.DataFrame(d)

m = IGV_table_limited.duplicated(subset=['Chr','Start', 'Ref','Revel_Score'],keep=False)

arr = np.arange(0.01, 0.1 + 0.01, 0.01)
print (arr)
[0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 ]

IGV_table_limited.loc[m, 'P'] += np.random.choice(arr, size=m.sum())
print (IGV_table_limited)
   Chr      Start Ref Alt  Revel_Score       RefSeq CHR         BP  SNP  \
21   9  133710834   A   C        0.429  NM_005157.6   9  133710834    1
22   9  133710834   A   G        0.424  NM_005157.6   9  133710834    1
23   9  133710834   A   T        0.432  NM_005157.6   9  133710834    1
24   9  133710835   T   A        0.395  NM_005157.6   9  133710835    1
25   9  133710835   T   C        0.386  NM_005157.6   9  133710835    1
26   9  133710835   T   G        0.389  NM_005157.6   9  133710835    1
27   9  133710836   G   A        0.429  NM_005157.6   9  133710836    1
28   9  133710836   G   C        0.418  NM_005157.6   9  133710836    1
29   9  133710836   G   T        0.418  NM_005157.6   9  133710836    1
30   9  133710837   T   A        0.381  NM_005157.6   9  133710837    1
31   9  133710837   T   G        0.278  NM_005157.6   9  133710837    1
32   9  133710838   T   C        0.378  NM_005157.6   9  133710838    1
33   9  133710838   T   G        0.327  NM_005157.6   9  133710838    1
34   9  133710839   G   C        0.352  NM_005157.6   9  133710839    1
35   9  133710839   G   T        0.352  NM_005157.6   9  133710839    1

           P
21  0.571001
22  0.576001
23  0.568001
24  0.605001
25  0.614001
26  0.611001
27  0.571001
28  0.422001
29  0.652001
30  0.619001
31  0.722001
32  0.622001
33  0.673001
34  0.688001
35  0.688001

Comment: I will have a look at this again. But with np.random.choice worked ok.
Comment: There is a value in the P column that is (before your code) 0.3220009999999999 and after your suggestion is -0.12811638110033008.
I copy the two lines
''' 9 133730223 G C 0.678 NM_005157.6 9 133730223 1 0.3220009999999999
and
9 133730223 G C 0.678 NM_005157.6 9 133730223 1 -0.12811638110033008 '''
Comment: I cannot have negative values in that column. Why that happens??
Comment: @ManoloDominguezBecerra - Do you add values by `+=` ? Is possible original values are negative?
Comment: @ManoloDominguezBecerra - I tested data from comment and all working well, added to answer.


Repetition of a word in String with out using collections

Question: Can any one suggest me to write a logic for this without using collections.
I have a string s="This is the place called island and it is a nice place ";
Input String to find repetition word is: "is";
output should be : 4
Comment: If you are using Java I think Regex is the best way instead of all the string operations. Check my post below for regex.
Answer: You can follow the below logic to do it.

Split the String on whitespace.
Initialize an integer counter to 0.
Traverse through each of the elements of the resultant array and for each String element of the array, do the following:
a) If stringElement.contains("is"), increment the counter created in step 2.
b) If !stringElement.contains("is"), do nothing and move on to the next element.
Do this till you exhaust all the elements of the array.
Print the counter value.

Try to write the code for this on your own and get back here if you're stuck up anywhere.
Comment: The function should be `String.contains(String)` otherwise the "is" in "this" will not be recognized.
Comment: @SteveBenett - Thanks for the correction! I made the necessary edit in the answer :)
Answer: string s="This is the place called island and it is a nice place ";
s = s+" ";
System.out.println(s.split("is").length-1);

Comment: that would fail on string like "penis" ;)
Comment: so add a space before or after it
Comment: sometimes you can't modify the input
Answer: As simple as
int count = 0;
for (int startIndex = 0; startIndex >= 0; startIndex = str.indexOf("is", startIndex)) {
  count++;
}

Answer: Use the following method, it should work:
public static int countSubStrings(String subString, String mainString){
    return (mainString.length() - mainString.replace(subString, "").length()) / subString.length();
}

Answer: you can use this. Hope you are using Java.
 public static void main(String[] args) {
 String str = "This is the place called island and it is a nice place";
    Pattern pattern = Pattern.compile("is");
    Matcher  matcher = pattern.matcher(str);

    int count = 0;
    while (matcher.find())
        count++;

    System.out.println(count);    // prints 4
}

Answer: this method will count how many time s2 appears in s1
public static int CountStr(String s1,String s2){
        int cnt=0,temp=0;
        if(s1.indexOf(s2)>0){
            temp=s1.indexOf(s2);
            cnt++;
        }
        while(s1.indexOf(s2,temp+1)>0){
            cnt++;
            temp=s1.indexOf(s2,temp+1);
        }
        return cnt;
    }


I'm trying to use the p.adjust function and I get the error object cannot be coerced to type 'double'

Question: So I read in my text file first:
x <- read.csv('filename.txt')

Then I try and run the p adjustment:
p.adjust(x, method = "BH")

The problem is I get this error:

Error in p.adjust(x, method = "BH") :
    (list) object cannot be coerced to type 'double'

Can anyone please help in dealing with this? Not sure what it means or how to proceed.
Comment: The error is because x is a data.frame, whereas `p.adjust` expects a numeric vector. Try `str(x)` and from those results, pull the p-values from your tests for use in `p.adjust`. Something like `x$pvals`  where pvals is the name of the variable with p-values.
Comment: Can you please give me some more detail on how the code should look for that? I'm still very new to R. Thanks!
Comment: Paste the results of `str(x)` into your question.
Answer: Little late to the party but this might help other people that get this problem.
The user Imo is correct that you're entering it in as a data frame instead of a numeric vector. Imo suggests converting it to a vector before processing.
You can do it a little easier (and without making a new data set) by using "as.matrix" in the same line. For your example:
p.adjust(as.matrix(x), method = "BH")


Find the range for $f(x) = 2x^2 + 1 + \sqrt{3x+16}$

Question: $f(x) = 2x^2 + 1 + \sqrt{3x+16}$
$f:[0,\infty)\to\mathbb R$
I need to find the range for this function.
I always have some problems for this type of requirements.
$2x^2 + 1 + \sqrt{3x+16}$ = $y$
How can I find $x$ that depends on $y$? How to approach this problems?
Answer: Notice that $f(x)=2x^2+1+\sqrt{3x+16}$ is always increasing in the interval $[0,\infty)$ since the derivative is always positive. Or, you can argue that $x^2$ and $\sqrt{x}$ are both increasing functions in the interval $[0,\infty)$.
Hence we only need to find the lower bound $f(0)$.
\begin{align}
f(0)&=2(0)^2+1+\sqrt{3(0)+16} \\
&=1+\sqrt{16}\\
&=5
\end{align}
So the range of the function is $[5,\infty)$ or $\{f\mid f\ge 5, f\in \mathbb R \}$.
Comment: Increasing isn't enough, $3 - \frac 1 x$ is increasing and bounded above.
Answer: We have:

$f(0)=5.$
$f'>0$ on $[0, \infty)$, hence $f$ is strictly increasing.
$f(x) \to \infty$ as $x \to \infty.$

Together with the intermediate value theorem, 1.,2. and 3. imply that $f([0, \infty)=[5, \infty).$
Comment: I understand. I always saw problems for which $f:R->R$ in that case I must calculate 2 limits ( x -> -∞ and x -> ∞ ) ?
Answer: Note that $2x^2 \geq 0$ and (since $x \geq 0$) $\sqrt{3x+16} \geq 4$. Using these facts, together with some well known properties of $y=x^2$ and $y=\sqrt{x}$, you can easily conclude that (for $x \geq 0$):
$$ 2x^2 + 1 + \sqrt{3x+16} \geq 5$$
that is:
$$ Im(f)= [5,+\infty) $$
Answer: $f$ is continuous and the derivative is $4x+\frac{3}{2\sqrt{3x+16}} > 0$ therefore $f$ is strictly increasing so $f(x) \to \infty$ as $x \to \infty.$ Now try to find the range.


In Java why is length a method for Strings, while it is a variable for Arrays?

Question:
Possible Duplicate:
Why is String.length() a method?

If I want to know the length of an array I will have to type the following:
array.length

Therefore, it is a variable in which is written the length of the array called 'array'.
On Strings the same thing will look like this:
string.length()

Therefore, it's a method that returns the length of the 'string' variable.
But why did they choose to make those two different, while it basically does the same thing... Is this because of efficiency or why else?
Thank you in advance.
Comment: "Why is this so" questions are typically not appropriate for this forum. Try (programmers.se](http://programmers.stackexchange.com/)
Comment: The first may be regarded as a property of the object array, but the second is a method accessing the object string. If I'm wrong, I'm waiting for correction.
Comment: It's basically history/tradition.  Theoretically, arrays could use `length()` or String could use `length` (which is conceptually a read-only instance field), but they got started off differently.  There is some slight advantage to having array `length` being a pseudo-field in terms of code interpretation/generation, but the advantage, while present at the inception of Java, is largely gone with modern implementations.
Comment: @JBNizet -- `length` is a public field of arrays, yet no one can change it.  It's one odd special case -- a wart on the architecture.
Comment: I forgot about the final keyword for a moment, don't know why :-/
Comment: Add to that size() for collections. length, length(), size() ... arrggh!
Answer: good question.
arrays are fixed in size and therefore have a public final length field.
strings are implementing the CharSequence interface that have the length() function that needs to be implemented.
Comment: Yes, indeed. I guess I had a mental slip ~)
Comment: Length was never "calculated".  The char array sharing was done invisibly, under the covers, and had no functional implications at the API.
Comment: The CharSequence interface is a very late invention.
Comment: @Hot: true but thats why the string implementation just returns its internal field


Find range of composite trigonometric function

Question: Find the range of the function:
$$f(x)= \cos^2x-\cos x $$
Answer is: $ [-1/4 ,2]$
I've factorized $\cos x$ and thus got min and max values equal to $[0,2]$ using inequalities. I know it's $-1/4$ because of $\cos(\pi/3).$ Any help or tip is appreciated, thanks.
Answer: $$f(x)=\cos^2 x-\cos x=\left(\cos x -\frac{1}{2}\right)^2-\frac{1}{4}\ge -\frac{1}{4}$$
with equality if and only if $\cos x=\frac{1}{2}$, i.e. $x=\pm\frac{\pi}{3}+2\pi k$ for some $k\in\mathbb Z$.
Also remember $\cos x\in[-1,1]$ for all $x\in\mathbb R$ by the definition of $\cos x$,
so $\cos^2 x-\cos x\le 2$ with equality if and only if $\cos x=-1$, i.e. $x=\pi+2\pi k$ for some $k\in\mathbb Z$.
$f(x)$ is also continuous in $\mathbb R$, so its range is $\left[-\frac{1}{4},2\right]$.
Answer: because $\cos x \in [-1,1]$ for any $x \in \mathbb R $ let be
$ \displaystyle f:[-1,1] \rightarrow \mathbb R, f(t)=t^2-t$
this quadratic function have a minimum in $t_V=\frac{-b}{2a}=\frac{1}{2}$ and the minimum is $f(t_V)=f(\frac{1}{2})=\frac{-1}{4}$
the maximum is the bigger value between $f(-1)=2$ and $f(1)=0$
so the range of $f(x)$ is $[\frac{-1}{4},2]$
Answer: If you derivate to find the extrems you get $f'(x) = (1-2\cos(x))\sin(x)$, so $\cos(x_1)=1/2$ or $\sin(x)=0$ (which means $\cos(x_2)=1$ or $\cos(x_3)=-1$). (indexing just for reference)
Testing for the 3 extreams you get:
$$f(x_1) = - 1/4$$ $$f(x_2)=0$$ $$f(x_3)=2$$
so it span [-1/4;2] with max and min over the x_1 and x_3 values (which are 'more than one' each).
Answer: $f'(x) =-2cos(x)sin(x)+sin(x)$
$-2cos(x)sin(x)+sin(x) = 0$
$sin(x)(1-2cos(x))=0$
$sin(x) = 0$ or $cos(x)=\dfrac{1}{2}$
$x = 0 + k\pi$ or $cos(x_0)=\dfrac{1}{2}$
$f(0) = 0 ; f(\pi) = 2 ; f(x_0)= \dfrac{1}{4}-\dfrac{1}{2} = \dfrac {-1}{4}$


What does at sign mean when listing directory using ls -l?

Question: I am removing the content of a directory using git reset hard, but I checked that the directories is still there and showing as
drwx------@
what does @ means here?
Comment: [duplicate](http://superuser.com/questions/155458/what-does-the-mean-on-the-output-of-ls-on-os-x-terminal)
Answer: according to man ls it says:

-@      Display extended attribute keys and sizes in long (-l) output.

and try this:
ls -l@

Comment: used, it shows  com.apple.quarantine, what does this mean?
Comment: extended attributes are used in mac os file system for various reasons. a simple explanation can also be read in here http://en.wikipedia.org/wiki/Extended_file_attributes. otherwise you have to go through apple docs.
Comment: and a good explanation here as well http://gotofritz.net/blog/geekery/os-x-extended-attibutes/


Showing progressbar until multiple async tasks are executed

Question: So I have a main Activity, where I have a button (ImageView), and a progressbar.
When I click that button I want to execute a series of AsyncTasks that download some data from a remote DB. I want to start the indeterminate progressbar when button is clicked and hide it when the last AsyncTask is done. Can I do that?
So I have

MainActivity.class (that contains the button, progressbar and the
code to call the asyncTasks)
AsyncTask1.class (containing the code to get some data from DB)
AsyncTask2.class (containing the code to get some data from DB)
...
AsyncTaskn.class (containing the code to get some data from DB)

I cannot include all the asyncTask classes in the activity so I can refference the progressBar directly, So how can I access the progressbar from the last AsyncTask class in onPostExecute in order to hide it?
So my mainActivity looks like:
ImageView upd = (ImageView) findViewById(R.id.iv_update);
    upd.setOnClickListener( new View.OnClickListener() {
        @Override
        public void onClick(View v) {
                ProgressBar progressBar = (ProgressBar) findViewById(R.id.progressBar);
                progressBar.setIndeterminate(true);
                progressBar.setVisibility(View.VISIBLE);
                new AsyncTask1(getApplicationContext()).execute();
                new AsyncTask2(getApplicationContext()).execute();
                new AsyncTask3(getApplicationContext()).execute();
                new AsyncTask4(getApplicationContext()).execute();
                new AsyncTask5(getApplicationContext()).execute();
                new AsyncTask6(getApplicationContext()).execute();
                new AsyncTask7(getApplicationContext()).execute();
        }
    });

An example of a AsyncTask (they are all similar)
public class AsyncTask1 extends AsyncTask<String, Integer, Double> {

    private Context mContext;
    InputStream is = null ;
    String result = "";

    public AsyncTask1 (Context context){
        mContext = context;
    }

    public void insSQLite(Integer iduzer, String tip_uzer, String nume, String parola, String numecomplet,
                          String email, String telefon, String idgrup, String den_grup, String idcoord, String nume_coord, String nume_coordfull ) {
        SQLiteDatabase db = new myDbHelper(mContext).getWritableDatabase();
        String sql = "";
        sql = "select iduzer from uzeri where trim(tip_uzer)='"+tip_uzer+"' and trim(numecomplet)='"+numecomplet+"'";
        Cursor cursor = db.rawQuery(sql, null);
        if (cursor != null) {
            if (cursor.moveToFirst()) {
                    Log.e("... It exists ...", Integer.toString(cursor.getInt(0)));
                    sql = "update uzeri set iduzer=" + iduzer + ",parola='" + parola + "',email='" + email + "',telefon='" + telefon + "', idgrup=" + idgrup + ", den_grup='" + den_grup + "', idcoord=" + idcoord + ", nume_coord='" + nume_coord + "', nume_coordfull='" + nume_coordfull + "'  where iduzer=" + Integer.toString(cursor.getInt(0));
                    db.execSQL(sql);
            }
        }else{
                sql = "insert into uzeri (iduzer,tip_uzer,nume,parola,numecomplet,email,telefon, idgrup,den_grup,idcoord,nume_coord,nume_coordfull) values (" +
                        String.valueOf(iduzer) + ",'" + tip_uzer + "','" + nume + "','" + parola + "','" + numecomplet + "','" +
                        email + "','" + telefon + "'," + idgrup + ",'" + den_grup +"'," + idcoord + ",'" + nume_coord + "','" + nume_coordfull + "')";
                db.execSQL(sql);
        }
            db.close();
    }

    @Override
    protected Double doInBackground(String... params) {
        postData("0");
        return null;
    }

    protected void onPostExecute(Double v){
        if (!String.valueOf(result).trim().equals("null")) {
            try {
                JSONArray Jarray = new JSONArray(result);
                for (int i = 0; i < Jarray.length(); i++) {
                    JSONObject Jasonobject = null;
                    Jasonobject = Jarray.getJSONObject(i);
                    Integer iduzer = Jasonobject.getInt("iduser");
                    String tip_uzer = Jasonobject.getString("tip_uzer");
                    String nume = Jasonobject.getString("name");
                    String pass = Jasonobject.getString("pass");
                    String numecomplet = Jasonobject.getString("namecomplete");
                    String email = Jasonobject.getString("email");
                    String telefon = Jasonobject.getString("phone");
                    String idgrup = Jasonobject.getString("idgroup");
                    String den_grup = Jasonobject.getString("den_group");
                    String idcoord = Jasonobject.getString("idcoord");
                    String nume_coord = Jasonobject.getString("nume_coord");
                    String nume_coordfull = Jasonobject.getString("name_coordfull");
                    insSQLite(iduzer, tip_uzer, nume, pass, namecomplete, email, telefon, idgrup,den_grup,idcoord,nume_coord,nume_coordfull);
                }
            } catch (Exception e) {
                Log.e("log_tag", "Error parsing data " + e.toString());
            }
        }else{
            Log.e("ASYNC UZ:","There are no users on the server");
        }
    }

    protected void onProgressUpdate(Integer... progress){
//        pb.setProgress(progress[0]);
    }

    public void postData(String valueIWantToSend) {
        // Create a new HttpClient and Post Header
        HttpClient httpclient = new DefaultHttpClient();
        HttpPost httppost = new HttpPost("http://www.myserver.com/getusers.php");
        final GlobalClass globalVariable = (GlobalClass) mContext;
        String uzid = globalVariable.getUzer();

        try {
            // Add your data
            List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
            nameValuePairs.add(new BasicNameValuePair("myHttpData", valueIWantToSend));
            nameValuePairs.add(new BasicNameValuePair("uzeru", uzid));
            httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
            // Execute HTTP Post Request
            HttpResponse response = httpclient.execute(httppost);
            HttpEntity httpEntity = response.getEntity();
            is =  httpEntity.getContent();
            try {
                BufferedReader br = new BufferedReader(new InputStreamReader(is));
                StringBuilder sb = new StringBuilder();
                String line = "";
                while((line=br.readLine())!=null){ sb.append(line+"\n"); }
                is.close();
                result=sb.toString();

            } catch (Exception e) {
                Log.e("log_tag", "Error converting result "+e.toString());
            }

        } catch (ClientProtocolException e) {
        } catch (IOException e) {
        }
    }
}

Thank you
Comment: Do you really need 7 different tasks running at the some time? You may want to think about using a `Service`. Anyway, you can use an `interface` with a callback to your `Activity`. But, doing it this way, you won't know which will finish last. You could have a listener to continually check if each is still running but not sure that is the best way either.
Comment: hey put progress on all AsyncTask then it automatically show progress with all tasks
Comment: I am using web services for update. Each AsyncTask updates a certain table from the DB
Answer: I will share you a idea. You could set up a variable like this:
int taskCount = 0;

Then,in every Asynctask's onPostExecute method, you could add the taskCount and test if taskCount is get the 7 value:
taskCount++; //WARN: here you maybe need java synchronized.
if(taskCount == 7){
    //all task is done , just do your work.
    //hide the progressbar.
}

Wish this idea could give you some tips.
Comment: How do I refference the progressbar on the activity, from the AsyncTask in order to hide it? The variable taskCount should be a global one then...??!?!?
Comment: 1.Make the progressbar global. 2 yes.
Comment: I must rephrase my question because it is not going the way I intended. Please check the edit above...
Comment: make the asynctask class in your mainActivity and after you have done all the thing in `onPostExecute` method, you could check whether the `taskCount` is equal to `7`.
Comment: I understand what you are saying, but, I would hate to put ALL my asyncTask classes inside my MainActivity class, because I will have there at least 7 AsyncTasks, most likely there will be around 15, so adding 15 AsyncTasks inside one class it would be awfull. I want to avoid that, so that is why I asked this question. Is there any way to access the progressbar of mainActivity from external class (AsyncTask) without having to add all the asyncTasks into the main class?
Comment: Another reason to not put them inside MainActivity is that I also have a service that periodically performs updates if necessary and I would like to have the service call the same 15 asyncTasks. Of course I could keep the external classes and copy them into mainActivity too, but that does not look like a healthy approach...


Como fazer efeito de loading no terminal em apenas uma linha?

Question: Como consigo fazer um efeito de loading dando refresh em apenas uma linha do terminal ao executar um programa em c?
Exemplo:

carregando...

os pontos ficam incrementando.
Eu consigo fazer esse efeito em um laço while mas apenas limpando a tela  com o comando
system("clear");

Gostaria apenas de atualizar uma linha.
Answer: Você pode imprimir um CR (\r, carriage return, que retorna o cursor pro começo da linha), limpar a linha e depois contiunar imprimindo os seus pontos, como no exemplo abaixo:
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
#include <time.h>

void main() {
    printf("Hello world\n");
    for (int i = 0; i < 10; i++) {
        printf("Carregando %d", i);
        for (int j = 0; j < 40; j++) {
            printf(".");
            Sleep(20);
        }
        printf("\r");
        for (int j = 0; j < 60; j++) {
            printf(" "); // apaga a linha anterior
        }
        printf("\r");
    }
    printf("\nGoodbye\n");
}

Comment: Vlw sua implementação ajudou pra eu me basear. Obrigado
Answer: Se você está usando Linux (como parece ser o caso, dada a sua tentativa de chamar clear), uma solução* interessante é usar os comandos do X-Term para limpar a tela e mover o cursor para a linha 0 e coluna 0. Por exemplo:
#include <stdio.h>

void cls(void)
{
    printf("\033[2J");   // Limpa a tela
    printf("\033[0;0H"); // Devolve o cursor para a linha 0, coluna 0
}

int main(void)
{
    printf("Esta é uma linha de texto\n");
    cls();
    printf("Esta é uma nova linha de texto\n");
    return 0;
}

Se a sua ideia é só "animar" uma mensagem de carregamento, ao invés de limpar a tela é mais prático e rápido simplesmente mover o cursor novamente para a coluna inicial. Eis um exemplo que anima os pontinhos:
#include <stdio.h>
#include <signal.h>
#include <sys/time.h>

int dots = 1;
void reset();
unsigned int alarm();
void animation(int signo);

int main(void)
{
    printf("\n\n\n\n      Carregando");
    signal(SIGALRM, animation);
    alarm(1);

    while(1)
        getchar();

    return 0;
}

void reset(void)
{
    printf("\033[10D");         /* Move 10 colunas para a esquerda */
    printf("          ");       /* Imprime 10 espaços em branco */
    printf("\033[10D");         /* Move 10 colunas para a esquerda */
}

unsigned int alarm (unsigned int seconds)
{
    struct itimerval old, new;
    new.it_interval.tv_usec = 0;
    new.it_interval.tv_sec = 0;
    new.it_value.tv_usec = 0;
    new.it_value.tv_sec = (long int) seconds;
    if (setitimer (ITIMER_REAL, &new, &old) < 0)
        return 0;
    else
        return old.it_value.tv_sec;
}

void animation(int signo)
{
    signal(SIGALRM, animation);
    alarm(1);

    (void)(signo); /* apenas ignora o parâmetro */

    printf(".");
    dots++;
    if(dots > 10)
    {
        dots = 1;
        reset();
    }
}

Note que essa solução não funciona no Windows. Se quiser usar algo
  mais padronizado para emular o seu próprio terminal, vale avaliar o
  ncurses (que tem também um porte para Windows).

* A minha resposta foi criada com ajuda desta resposta do SOEN.
Comment: Em relação a quando o `i` é maior eu me preciptei ao dizer sobre a resposta do carlos, isso vale apenas para o conteúdo do link que mencionei.
Comment: Ok, entendi, obrigado pelo esclarecimento. :) Eu só perguntei porque se tem algo errado é importante ser corrigido para que o conteúdo da pergunta e das respostas também sirva pra ajudar outras pessoas além de você.
Comment: Vlw @Luiz Vieira, obrigado pela ajuda :)
Comment: Obrigado luiz mas o resultado não saiu como o esperado aqui.
Comment: Por nada. :) Mas então não está suficientemente claro na sua pergunta o que você realmente esperava. Honestamente eu não enxergo porque a resposta do @carlosfigueira não serve, já que não há praticamente diferenças significativas dela para a que você ofecereu depois. Não está claro o que é o "problema conforme o i vai aumentando".
Comment: Quando eu encontrei a solução eu não havia visto a resposta do @carlosfigueira, eu havia me baseado no link que mencionei na minha resposta, e como o conteudo do carlosfigueira é quase o mesmo do que está no link, eu dei os créditos para ele tbm dizendo que me basiei na dele, e mais uma coisa, e eu resolvi responder tambem com a minha solução, pois ela é para linux, e portanto um tanto diferente, pode ver algumas funções difentes que sem ela não funcionariam. Por isso resolvi responder
Answer: Agradeço as respostas mas não funcionou comigo, utilizo linux, tentei adaptálas, mas mesmo assim o resultado não saiu como pretendia.
Consegui resolve, adaptei desse código e o código do @carlosfigueira, analizando o código do link  vocês podem ver que existe um problema nele conforme i vai aumentando.
Então tive que entender o funcionamento de um parametro do printf, o \r e só assim consegui formular uma solução.
A solução é essa:
  #include <stdio.h>
  #include <time.h>

  void limpa_linha(void);

  int main (int argc, char **argv){
     int i, j;

     system ("clear");//limpa tela
     printf ("\n\nCarregando: \n\n");

     for (i = 0; i <= 100; i++){
        printf ("%d%%  ", i);
        fflush (stdout);//garante a escrita de dados imediatamente na tela
        //repare mod 10, eu limito a qtd de pontos q serao gerados
        for (j = 0; j < i%10; j++){
           printf(".");
        }
        fflush (stdout);//garante a escrita de dados imediatamente na tela
        usleep(500000);//função espera por tempo, parametro em microsegundos.
        limpa_linha();
     }
     printf ("\n\n Fim\n\n");
     return 0;
  }

  void limpa_linha(){
     int i;//indice do caracter na linha
     int max_caracter=50;//indica o maximo de caracter que a linha pode chegar a ter, para linhas com mt texto, coloque um nmr bem maior
     printf("\r");//retorna para o inicio da linha que pretende reutilizar, isso não limpa a linha, apenas posiciona cursor ao inicio da linha

     //Agora precisamos limpar a linha,
     //substitui todos os caracteres existentes por espaço em branco
     for(i=0;i<max_caracter;i++){
        printf(" ");//vai preenchendo a linha com espaços em branco
     }

     printf("\r");//volta ao inicio da linha , dessa vez ela está em branco.

  }