What does the "::" mean in "::tolower"?

Question: I've seen code like this:
std::string str = "wHatEver";
std::transform(str.begin(), str.end(), str.begin(), ::tolower);

And I have a question: what does mean :: before tolower?
and std::tolower not works, but ::tolower works OK
Comment: Technically, it's called the **[scope resolution operator](http://en.wikipedia.org/wiki/Scope_resolution_operator)**. That should give you a term to begin your search for more details.
Answer: As to why the :: is necessary: the standard defines two tolower's,
a function template in std::, and a simple function in both :: and
std::.  Depending on which headers have been included (and that includes
headers indirectly included from other headers, which you may not know
about), either one, the other, or both may be visible.  Using :: ensures
that the older one, from the C standard, is used.  (If the one in std::
is considered, the call will be ambiguous, since transform is a template
itself, and the compiler will not be able to deduce the template
arguments.)
While I'm at it, I might mention that using ::tolower like this is
undefined behavior, at least if plain char is signed.  The input to
::tolower is an int, and must be in the range 0...UCHAR_MAX, or EOF. If
plain char is signed, some of the characters may have negative
encodings, which results in undefined behavior.  In practice, most
implementations make this work.  For all characters except 0xFF (ÿ in
Latin 1).  If you're not concerned with portability, some compilers have
a switch to make char unsigned---use it.  Otherwise, write a small
functional object to handle it correctly, either:
struct ToLower
{
    char operator()( char ch ) const
    {
        return ::tolower( static_cast<unsigned char>(ch) );
    }
};

or (better, but significantly more work---only worth it if your using it
a lot), a functional object whose constructor takes a locale (defaulting
to the global locale) and contains a reference to an std::ctype, which
it uses for the tolower function.  (Of course, if you're really
internationalized, tolower probably doesn't have any meaning.  And
you'll be using UTF-8, which is a multi-byte encoding, and doesn't work
with any of the available possibilities.)
Answer: Means that it is explicitly using the tolower in the global namespace (which is presumably the stdc lib one).
Example:
void foo() {
    // This is your global foo
}

namespace bar {
    void foo() {
        // This is bar's foo
    }
}

using namespace bar;

void test() {
    foo();   // Ambiguous - which one is it?
    ::foo(); // This is the global foo()
}

Comment: "using namespace bar" means that `foo()` by itself implies `bar::foo()`. However, maybe you meant to use the `foo()` in the global scope? That can also be accessed via `foo()`. So `foo()` could be used for both. Now the compiler isn't sure which one you meant, so you have to specify.
Comment: thanks, but I can't understand, you wrote - using namespace bar; why compiler will not use foo from bar? why you need to write bar::foo after using namespace bar; Can you please explain?
Comment: @EboMike "`foo()` by itself implies `bar::foo()`" is too strong. Name lookup considers `bar::foo` when looking for a match for `foo` (and in this case finds ambiguity with `::foo`)
Answer: Use the version from the global namespace.  (Probably included <ctypes.h> and not <cctypes> if std:: doesn't work)
Comment: That's the only one valid reason (But I'd plan for a switch if the project has to be continued).  We see far too many learners still using it.
Comment: Thanks. I use Visual C++ 6. So :: says to use tolower from global namespace
Comment: Yes, the other potential overload of tolower are in `` (I said potential because I don't know how conformant VC++6 is on that point... why use such an old compiler?)
Comment: "why use such an old compiler?" Oh, don't ask)) It's because of very old project
Answer: :: is the global namespace.
#include <iostream>

void bar()
{
    std::cout << "::bar" << std::endl;
}

namespace foo
{
    void bar()
    {
        std::cout << "foo::bar" << std::endl;
    }
}

int main()
{
    bar();
    foo::bar();
    ::bar();
    using namespace foo;
    foo::bar();
    ::bar(); // bar() would be ambiguous now without ::
}

Comment: @Nawaz: Read the comments carefully before downvoting. He says that `::bar()` would be ambiguous without the `::`. In fact, [it is](http://www.ideone.com/Pmi5u).
Comment: the comment wasn't clear enough: bar() without :: would be ambiguous
Comment: @Cody: the comment is edited within 5 mins. Previously it was *"bar() would be ambiguous"*  which is obviously wrong!
Comment: Ironically, the original comment was *ambiguous*! (It wasn't wrong, per se. The current interpretation is perfectly logical, even given the original wording.)
Comment: @Cody: The code was NOT ambiguous, but the comment was saying *"bar() would be ambiguous"*. Is there any confusion now?
Comment: @Cody: But I didn't get the comment in which you said *"Ironically, ..."*. :D
Comment: @Nawaz: It was just a joke. I meant that jhasse's original comment in the source code was ambiguous. It was unclear *which* version of `bar()` he meant would be ambiguous. The first time I read it, I interpreted it correctly, but I could see how you assumed something else entirely.

What does the "::" mean in "::tolower"?


Training and Testing dataset

Question: I am fitting a glm model on my data set. But in my case I don't have any competing models to decide upon, i.e., I just have only one model and have to estimate the coefficients of the model. In such a case shouldn't I use all the data to train the model or should I still divide the data set into training and testing?

Also cross-posted to the data science site.
Comment: Please don't cross post. It's bad.
Comment: @StudentT I made a similar comment on the other post, exemplifying the problems with it. :)
Answer: You could have competing models if you wanted. You could try elastic net, ridge, or a linear model.
You probably still want to have an idea of how your model performs on unseen data. This is the point of the training and test sets. So you do want to split the data.
Answer: Training on a single GLM model is not an excuse for overfitting. You always should try to split unless you only really want to use the model on your training set.

Training and Testing dataset


How to choose random- and fixed-effects structure in linear mixed models?

Question: Consider the following data from a two-way within subjects design:
df <- "http://personality-project.org/r/datasets/R.appendix4.data"
df <- read.table(df,header=T)
head(df)

Observation Subject Task Valence Recall
1           1     Jim Free     Neg      8
2           2     Jim Free     Neu      9
3           3     Jim Free     Pos      5
4           4     Jim Cued     Neg      7
5           5     Jim Cued     Neu      9
6           6     Jim Cued     Pos     10

I would like to analyze this using mixed-linear models. Considering all possible fixed- and random-effects there are multiple possible models:
# different fixed effects with random-intercept
a0 <- lmer(Recall~1 + (1|Subject), REML=F,df)
a1 <- lmer(Recall~Task + (1|Subject), REML=F,df)
a2 <- lmer(Recall~Valence + (1|Subject), REML=F,df)
a3 <- lmer(Recall~Task+Valence + (1|Subject), REML=F,df)
a4 <- lmer(Recall~Task*Valence + (1|Subject), REML=F,df)

# different fixed effects with random-intercept-random-slope
b0 <- lmer(Recall~1 + (1|Subject) + (0+Task|Subject) + (0+Valence|Subject), REML=F,df)
b1 <- lmer(Recall~Task + (1|Subject) + (0+Task|Subject) + (0+Valence|Subject), REML=F,df)
b2 <- lmer(Recall~Valence + (1|Subject) + (0+Task|Subject) + (0+Valence|Subject), REML=F,df)
b3 <- lmer(Recall~Task+Valence + (1|Subject) + (0+Task|Subject) + (0+Valence|Subject), REML=F,df)
b4 <- lmer(Recall~Task*Valence + (1|Subject) + (0+Task|Subject) + (0+Valence|Subject), REML=F,df)

# different fixed effects with random-intercept-random-slope including variance-covariance matrix
c0 <- lmer(Recall~1 + (1 + Valence + Task|Subject), REML=F,df)
c1 <- lmer(Recall~Task + (1 + Valence + Task|Subject), REML=F,df)
c2 <- lmer(Recall~Valence + (1 + Valence + Task|Subject), REML=F,df)
c3 <- lmer(Recall~Task+Valence + (1 + Valence + Task|Subject), REML=F,df)
c4 <- lmer(Recall~Task*Valence + (1 + Valence + Task|Subject), REML=F,df)

What is the recommended way to select the best fitting model in this context? When using log-likelihood ratio tests what is the recommended procedure? Generating models upwards (from null model to most complex model) or downwards (from most complex model to null model)? Stepwise inclusion or exclusion? Or is it recommended to put all models in one log-likelihood ratio test and select the model with the lowest p-value? How to compare models that are not nested?
Is it recommended to first find the appropriate fixed-effects structure and then the appropriate random-effects structure or the other way round (I have found references for both options...)?
What is the recommended way of reporting results? Reporting the p-value from the log-likelihood ratio test comparing the full mixed-model (with the effect in question) to reduced model (without the effect in question). Or is it better to use log-likelihood ratio test to find the best fitting model and then use lmerTest to report p-values from the effects in the best fitting model?
Answer: I'm not sure there's really a canonical answer to this, but I'll give it a shot.

What is the recommended way to select the best fitting model in this context? When using log-likelihood ratio tests what is the recommended procedure? Generating models upwards (from null model to most complex model) or downwards (from most complex model to null model)? Stepwise inclusion or exclusion? Or is it recommended to put all models in one log-likelihood ratio test and select the model with the lowest p-value? How to compare models that are not nested?

It depends what your goals are.

In general you should be very, very careful about model selection (see e.g. this answer, or this post, or just Google "Harrell stepwise" ...).
If you are interested in having your p-values be meaningful (i.e. you are doing confirmatory hypothesis testing), you should not do model selection. However: it's not so clear to me whether model selection procedures are quite as bad if you are doing model selection on non-focal parts of the model, e.g. doing model selection on the random effects if your primary interest is inference on the fixed effects.
There's no such thing as "putting all the models in one likelihood ratio test" -- likelihood ratio testing is a pairwise procedure. If you wanted to do model selection (e.g.) on the random effects, I would probably recommend an "all at once" approach using information criteria as in this example -- that at least avoids some of the problems of stepwise approaches (but not of model selection more generally).
Barr et al. 2013 "Keep it maximal" Journal of Memory and Language (doi:10.1016/j.jml.2012.11.001) would recommend using the maximal model (only).
Shravan Vasishth disagrees, arguing that such models are going to be underpowered and hence problematic unless the data set is very large (and the signal-to-noise ratio is high)
Another reasonably defensible approach is to fit a large but reasonable model and then, if the fit is singular, remove terms until it isn't any more
With some caveats (enumerated in the GLMM FAQ), you can use information criteria to compare non-nested models with differing random effects (although Brian Ripley disagrees: see bottom of p. 6 here)

Is it recommended to first find the appropriate fixed-effects structure and then the appropriate random-effects structure or the other way round (I have found references for both options...)?

I don't think anyone knows. See previous answer about model selection more generally.  If you could define your goals sufficiently clearly (which few people do), the question might be answerable.  If you have references for both options, it would be useful to edit your question to include them ... (For what it's worth, this example (already quoted above) uses information criteria to select the random effects part, then eschews selection on the fixed-effect part of the model.

What is the recommended way of reporting results? Reporting the p-value from the log-likelihood ratio test comparing the full mixed-model (with the effect in question) to reduced model (without the effect in question). Or is it better to use log-likelihood ratio test to find the best fitting model and then use lmerTest to report p-values from the effects in the best fitting model?

This is (alas) another difficult question.  If you report the marginal effects as reported by lmerTest, you have to worry about marginality (e.g., whether the estimates of the main effects of A and B are meaningful when there is an A-by-B interaction in the model); this is a huge can of worms, but is somewhat mitigated if you use contrasts="sum" as recommend by afex::mixed().  Balanced designs help a little bit too. If you really want to paper over all these cracks, I think I would recommend afex::mixed, which gives you output similar to lmerTest, but tries to deal with these issues.
Answer: Update May 2017: As it turns out, a lof of what I have written here is kind of wrongish. Some updates are made throughout the post.

I agree a lot with what has been said by Ben Bolker already (thanks for the shout-out to afex::mixed()) but let me add a few more general and specific thoughts on this issue.
Focus on fixed versus random effects and how to report results
For the type of experimental research that is represented in the example data set from Jonathan Baron you use the important question is usually whether or not a manipulated factor has an overall effect. For example, do we find an overall main effect or interaction of Task? An important point is that in those data sets usually all factors are under complete experimental control and randomly assigned. Consequently, the focus of interest is usually on the fixed effects.
In contrast, the random effects components can be seen as "nuisance" parameters that capture systematic variance (i.e., inter-individual differences in the size of the effect) that are not necessarily important for the main question. From this point of view the suggestion of using the maximal random effects structure as advocated by Barr et al. follows somewhat naturally. It is easy to imagine that an experimental manipulation does not affect all individuals in the exact same way and we want to control for this. On the other hand, the number of factors or levels is usually not too large so that the danger of overfitting seems comparatively small.
Consequently, I would follow the suggestion of Barr et al. and specify a maximal random effects structure and report tests of the fixed effects as my main results. To test the fixed effects I would also suggest to use afex::mixed() as it reports tests of effects or factors (instead of test of parameters) and calculates those tests in a somewhat sensible way (e.g., uses the same random effects structure for all models in which a single effect is removed, uses sum-to-zero-contrasts, offers different methods to calculate p-values, ...).
What about the example data
The problem with the example data you gave is that for this dataset the maximal random effects structure leads to an oversaturated model as there is only one data point per cell of the design:
> with(df, table(Valence, Subject, Task))
, , Task = Cued

       Subject
Valence Faye Jason Jim Ron Victor
    Neg    1     1   1   1      1
    Neu    1     1   1   1      1
    Pos    1     1   1   1      1

, , Task = Free

       Subject
Valence Faye Jason Jim Ron Victor
    Neg    1     1   1   1      1
    Neu    1     1   1   1      1
    Pos    1     1   1   1      1

Consequently, lmer chokes on the maximal random effects structure:
> lmer(Recall~Task*Valence + (Valence*Task|Subject), df)
Error: number of observations (=30) <= number of random effects (=30) for term
(Valence * Task | Subject); the random-effects parameters and the residual variance
(or scale parameter) are probably unidentifiable

Unfortunately, there is to my knowledge no agreed upon way to deal with this problem. But let me sketch and discuss some:

A first solution could be to remove the highest random slope and test the effects for this model:
require(afex)
mixed(Recall~Task*Valence + (Valence+Task|Subject), df)
        Effect    F ndf  ddf F.scaling p.value
1         Task 6.56   1 4.00      1.00     .06
2      Valence 0.80   2 3.00      0.75     .53
3 Task:Valence 0.42   2 8.00      1.00     .67

However, this solution is a little ad-hoc and not overly motivated.
Update May 2017: This is the approach I am currently endorsing. See this blog post and the draft of the chapter I am co-authoring, section "Random Effects Structures for Traditional ANOVA Designs".
An alternative solution (and one that could be seen as advocated by Barr et al.'s discussion) could be to always remove the random slopes for the smallest effect. This has two problems though: (1) Which random effects structure do we use to find out what the smallest effect is and (2) R is reluctant to remove a lower-order effect such as a main effect if higher order effects such as an interaction of this effect is present (see here). As a consequence one would need to set up this random effects structure by hand and pass the so constructed model matrix to the lmer call.
A third solution could be to use an alternative parametrization of the random effects part, namely one that corresponds to the RM-ANOVA model for this data. Unfortunately (?), lmer doesn't allow "negative variances" so this parameterization doesn't exactly correspond to the RM-ANOVA for all data sets, see discussion here and elsewhere (e.g. here and here). The "lmer-ANOVA" for these data would be:
> mixed(Recall~Task*Valence + (1|Subject) + (1|Task:Subject) + (1|Valence:Subject), df)
        Effect    F ndf  ddf F.scaling p.value
1         Task 7.35   1 4.00      1.00     .05
2      Valence 1.46   2 8.00      1.00     .29
3 Task:Valence 0.29   2 8.00      1.00     .76

Given all this problems I simply wouldn't use lmer for fitting data sets for which there is only one data point per cell of the design unless a more agreed upon solution for the problem of the maximal random effects structure is available.

Instead, I would One also could still use the classical ANOVA. Using one of the wrappers to car::Anova() in afex the results would be:
> aov4(Recall~Task*Valence + (Valence*Task|Subject), df)
        Effect         df  MSE      F  ges   p
1      Valence 1.44, 5.75 4.67   1.46  .02 .29
2         Task       1, 4 4.08 7.35 +  .07 .05
3 Valence:Task 1.63, 6.52 2.96   0.29 .003 .71

Note that afex now also allows to return the model fitted with aov which can be passed to lsmeans for post-hoc tests (but for test of effects the ones reported by car::Anova are still more reasonable):
> require(lsmeans)
> m <- aov4(Recall~Task*Valence + (Valence*Task|Subject), df, return = "aov")
> lsmeans(m, ~Task+Valence)
 Task Valence lsmean       SE   df lower.CL upper.CL
 Cued Neg       11.8 1.852026 5.52  7.17157 16.42843
 Free Neg       10.2 1.852026 5.52  5.57157 14.82843
 Cued Neu       13.0 1.852026 5.52  8.37157 17.62843
 Free Neu       11.2 1.852026 5.52  6.57157 15.82843
 Cued Pos       13.6 1.852026 5.52  8.97157 18.22843
 Free Pos       11.0 1.852026 5.52  6.37157 15.62843

Confidence level used: 0.95

Comment: I did some edits, you might want to make sure that I did not mis-represent  you.
Comment: (+1) "Unfortunately, lmer doesn't allow negative correlations" -- shouldn't this be "doesn't allow negative variances"? Also, re Update: could you be more explicit about what exactly is "wrong-ish" in this answer?
Comment: (I read [the linked post](http://singmann.org/mixed-models-for-anova-designs-with-one-observation-per-unit-of-observation-and-cell-of-the-design/) and it seems that the main message there is that the approach listed here as #1 is more kosher than you used to think. Correct? It's still not clear if you now think it's preferable to #3 or #4).
Comment: @amoeba Yes you are correct. I was just too lazy to update my answer here accordingly.
Comment: @amoeba And you are also right re correlations. `lmer` does not allow negative variances but obviously negative correlations among the variance components.
Comment: Correct me if I am wrong, but one big difference between #1 and #3, i.e. between `(Valence+Task|Subject)` and `(1|Subject) + (1|Task:Subject) + (1|Valence:Subject)` is that #1 can have *a lot* more parameters if `Valence` and `Task` have many levels. E.g. is `Valence` has 3 levels, then `(Valence|Subject)` alone estimates 3 variance parameters plus 3 correlation parameters (?), whereas `(1|Subject) + (1|Valence:Subject)` only estimates 2 variance parameters. Is that correct? If so, does it influence the choice between #1 and #3/4 somehow?
Comment: `(Valence+Task|Subject)` will estimate `V+T-1` variance parameters and `(V+T-1)(V+T-2)/2` correlations (where `V` = number of levels for `Valence` and `T` = number of levels for `Task`). And I also think you are right that `(1|Subject) + (1|Task:Subject) + (1|Valence:Subject)` will only estimate 3 variance parameters and no correlation. So yes, there is a difference in the number of parameters. This could make it a lot more likely that #3 converges, but on the other hand #1 seems to be the more flexible model (my intuition is that the negative variance problem does not apply there).
Comment: OK, thanks. I looked into the current version of your Chapter and saw that you now write "To nevertheless analyze such designs with mixed models the most reasonable decision is to remove the highest-order random slope", which I guess means that you endorse option #1. By the way, I think it could be useful to mention there option #3 as the one [almost] corresponding to the classical RM-ANOVA.
Comment: Indeed, that is what we suggest. Make the changes here, hmm. If you want to feel free.


normalise/denormalise data in in neuralnet prediction modell

Question: I built a neuralnet model to do predictions from measured data. I have a conceptual problem in understandinig how to normalise/denormalise my data in the "prediction-mode".
Here is what I did:
I used a dataset which I normalised with the min/max method:
normalized = (x-min(x))/(max(x)-min(x))
The I splitted the dataset and used 80% of the data to train model and 20% to validate the model. This worked all fine.
Only working with this initial dataset makes it also easy to "denormalise" the data to their initial values using this formula:
denormalized = (normalized)*(max(x)-min(x))+min(x)
formulas are taken from here:
De normalize predicted value
Now when I use the model to do predictions in realtime, I have difficulties in understanding how normalise/denormalise my data. I have every day new measured date from whcih I want to do a prediction. Do I have to use the min/max values that I gathered from my inital dataset, or do calculate every day new min/max values for the daily dataset to feed into the neuralnet model?
I have tried both ways, and both results are not too good... When I use the min/max values from the training data, the results becomes a bit better (less noisy).
For the model output I have to use the min/max values from the training, as I do not have any other data. Or is there a conceptual error in my thinking?
Any help is welcome..
Thanks a lot in advance,
Arne
Answer: Ideally you are supposed to save the min/max you used to normalize the training data; so that you can re-use it for test data/live data. DO NOT re-normalize the entire train + new data again because that is leakage.
Use the min/max of train data to normalize the test data
Answer: My best guess would be the following:
Since the "new" instances have not been there during the original normalization phase. One possible action is to include the new instances as bottom rows to your original dataset, re-normalize and then re-train your model. Doing so, you will adjust the relative location of the new instances with respect to the original dataset.
If your model still does not perform well in terms of prediction power, you might be experiencing an "overfitting" problem, which is a common problem in predictive models. One indication of overfitting is that your model does well on a training set but fails to generalize to unseen test records. If that's the case, you might want to look at how "complex" your neural network is and how many data records do you have.
Comment: I agree, that including the new data and re-training the model would be the "clean" way to do it. But this would somehow contradict the idea of prediction, because I would have to validate the model every day. Overfitting could be a topic, I trained my model over the weekend with less neurons. I will have to check the results today. But can you (or anybody else) give advice what the right approach would be in general?
Best regards,
Arne


How to check if my regression model is good

Question: One way to find accuracy of the logistic regression model using 'glm' is to find AUC plot. How to check the same for regression model found with continuous response variable (family = 'gaussian')?
What methods are used to check how well does my regression model fit the data?
Comment: The "Gaussian" family with a linear link is just ordinary least squares (OLS) regression; methods to check such fits are probably discussed in a thousand questions on this site (I do not exaggerate).
Comment: You may want to have a look at the `r-squared` tag and the `goodness-of-fit` tag..
Comment: This thread is relevant: https://stats.stackexchange.com/q/414349/121522
Answer: I would suggest a brief search on "linear regression model diagnostics" as a start. But here are some that I would suggest you to check:
Make sure the assumptions are satisfactorily met

Use scatterplot or component plus residual plot to examine the linear relationship between the independent predictor(s) and the dependent variable.
Compose a plot with standardized residual versus predicted value and ensure there isn't extreme point with very high residual, and the spread of the residual is largely similar along the predicted value, as well as spreading largely equally above and below the mean of residual, zero.
You can also change the y-axis to residual$^2$. This plot helps identifying unequal variance.
Re-examine the study design to ensure the assumption of independence is reasonable.
Retrieve the variance inflation factor (VIF) or tolerance statistics to examine possible collinearity.

Examine potential influential point(s)

Check statistics such as Cook's D, DFits, or DF Beta to find out if a certain data point is drastically changing your regression results. You can find more here.

Examine the change in $R^2$ and Adjusted $R^2$ statistics

Being the ratio of regression sum of squares to total sum of squares, $R^2$ can tell you how many % of variability in your dependent variable are explained by the model.
Adjusted $R^2$ can be used to check if the extra sum of squares brought about my the additional predictor(s) is really worth the degrees of freedom they'll take.

Check necessary interaction

If there is a main independent predictor, before you make any interpretation of its independent effect, check if it is interacting with other independent variables. Interaction, if left unadjusted, can bias your estimate.

Apply your model to another data set and check its performance

You can also apply the regression formula to other separate data and see how well it predicts. Graph like scatter plot and statistics like % difference from the observed value can serve as a good start.
Comment: (+1): Very complete answer!  If you're using R, `plot.lm` can give you most of the diagnostic plots Penguin_Knight mentions.
Comment: @Zach what do you mean with `plot.lm` in R?
Answer: I like to cross-validate my regression models to see how well they generalize to new data.  My metric of choice is mean absolute error on the cross-validated data, but root mean squared error is more common and equally useful.
I don't find R2 to be a good metric of how well your model fits the training data, as almost any error metric calculated on the training data will be prone to over fitting.  If you must calculate R2 on the training set, I suggest using adjusted R2.
Answer: You can use $R^2$ to examine how well your model fits the training data. This will tell you what percentage of the variance in the data are explained by the model.
I suggest using RMSE (root mean square error) of your predictions on your test set when compared to the actual value. This is a standard method of reporting prediction error of a continuous variable.
Comment: @Macro But the question originally asked for a performance metric for an OLS Regression with gaussian errors. He is coming from logistic regression.
Comment: @Erik, thanks, I misread. Anyway, regarding the first part, I don't think $R^2$, in isolation, can be used to "check if my regression model is good", to use the OP's words. Your model could fail miserably to predict effectively on vast majority of the data while still having a high $R^2$. See [here](http://stats.stackexchange.com/questions/32781/how-can-i-use-the-value-of-r2-to-test-the-linearity-assumption-in-multiple-re/32785#32785) for an example - in example (1), there's almost no predictive power but $R^2$ is still high.
Comment: @Macro, I agree with your comments but was aiming for a simple explanation to point the OP in the right direction
Answer: I am used to check the functional form of my parameter estimator by plotting a non-parametric (e.g. a kernel regression) or semi-parametric estimation and comparing it to the parametric fitted curve. I think this is in the first step often faster (and perhaps more insightful) than including interaction terms or higher-orders terms.
The R package np provides many nice non-parametric and semi-parametric functions, and its Vignette is well written:
http://cran.r-project.org/web/packages/np/vignettes/np.pdf


Derivative of a 1D function involving matrix inversion

Question: How do I compute the derivative of
$$
f(x) = \frac{1}{2} \left(\mathbf{a} - x \mathbf{b}\right)^T (x \mathbf{A} + \mu \mathbf{I})^{-1} \left(\mathbf{a} - x \mathbf{b}\right),
$$
where $\mathbf{a}$, $\mathbf{b}$ are column vectors, $\mathbf{A}$ is a square symmetric matrix, $\mathbf{I}$ is the identity matrix, and $\mu > 0$? Of course, we assume that $x$ is in the interior of the domain where the inverse is well-defined. If possible, I'd like to find a formula which avoids the need to compute the Eigendecomposition of $\mathbf{A}$.
Answer: $
\def\o{{\tt1}}\def\p{\partial}
\def\L{\left}\def\R{\right}
\def\LR#1{\L(#1\R)}
\def\BR#1{\Big(#1\Big)}
\def\vecc#1{\operatorname{vec}\LR{#1}}
\def\diag#1{\operatorname{diag}\LR{#1}}
\def\Diag#1{\operatorname{Diag}\LR{#1}}
\def\trace#1{\operatorname{Tr}\LR{#1}}
\def\qiq{\quad\implies\quad}
\def\grad#1#2{\frac{d #1}{d #2}}
\def\c#1{\color{red}{#1}}
$For typing convenience, define the variables
$$\eqalign{
g &= {xb-a} &\qiq dg = b\;dx \\
H &= {xA + \mu I} &\qiq dH = A\;dx \\
K &= H^{-1} &\qiq dK = -K\,dH\,K \\
}$$
Use the above notation to write the objective function.
Then calculate its differential and gradient.
$$\eqalign{
f &= \frac{1}{2}\BR{g^TKg} \\
\\
df &= \frac{1}{2}\BR{dg^TKg\;+\;g^T\,dK\,g\;+\;g^TKdg} \\
 &= \frac{1}{2}\BR{b^TKg\;-\;g^TKAKg\;+\;g^TKb}\,dx \\
\\
\grad{f}{x} &= \frac{1}{2}\BR{b^TKg\;-\;g^TKAKg\;+\;g^TKb} \\
}$$
Comment: Since $\mathbf{A}$ is symmetric, so does $\mathbf{K}$. Introducing $\mathbf{z}=\mathbf{Kg}$, you end up with the nice gradient expression $\mathbf{z}^T \mathbf{b}-\frac{1}{2} \mathbf{z}^T \mathbf{A} \mathbf{z}$.


Conditional negation probability problem

Question: K so I have an exam in probability coming up and we are going to have a similar question to this. I think this is a simple problem, but there is a twist to it:
So let's say the chance of a person smoking today in the U.S is 40%, and the chance that a person in the U.S will die of lung disease is 26%.
If a person smokes, the chance that he will die of lung disease is 55%.
What is the probability that a person does not die of lung disease, if they did not smoke.
So the formula that applies here is conditional probability I guess:
P(A|B) = P(A and B) / P(B).
We know P(A and B) is 0.55 x .26 = 0.143 or 14.3%. (I worked this out by solving for P(A and B) knowing what P(A|B) is but correct me if I'm wrong). So the first question I would ask is what exactly is it that we get if we subtract this 14.3% from 100%? The resulting 86%, is this the P(not A and not B) or is it something else?
Either way the formula we need for the answer is I think:
P(not A | not B) = P(not A and not B) / P(not B) ?
which is basically the negated version of the first formula.
But I'm not sure how to solve this.
Comment: What is $A$ and what is $B$ in your example? I see no reason to multiply $0.55$ and $0.26$. I think you are confusing some things. Slow down with your calculations and double think each one. First of all, you need to define what $A$ and $B$ are. Then, we can talk further.
Answer: First, you made a mistake in calculating $P(A\cap B)$ in that you multiplied $P(A\mid B)$ by $P(A)$, not by $P(B)$. The $55\%$ are the probability $P(\text{die}\mid\text{smoke})$, so you need to multiply by $P(\text{smoke})$, not by $P(\text{die})$.
Indeed you want $P(\overline{\text{die}}\mid\overline{\text{smoke}})$, and this is
$$
P(\overline{\text{die}}\mid\overline{\text{smoke}})=\frac{P(\overline{\text{die}}\cap\overline{\text{smoke}})}{P(\overline{\text{smoke}})}
$$
You already have $P(\text{smoke})$, $P(\text{die})$ and $P(\text{die}\cap\text{smoke})$ so you can calculate
$$
P(\text{die}\cup\text{smoke})=P(\text{smoke})+P(\text{die})-P(\text{die}\cap\text{smoke})\;,
$$
and the numerator you need is the complement of that.
You also asked what you get if you subtract $P(A\cap B)$ from $1$. You get the complement, $P(\overline{A\cap B})$, which is $P(\overline A\cup\overline B)$. You might want to check out De Morgan's laws.
Answer: $A$ is the event that a person is smoke, and $B$ is the event that a person will die of lung disease.
$P(A \cap B)=P(B|A)P(A)=0.22$, but since $P(A)=0.4$ and $P(B)=0.26$ we can conclude the following probabilities:
$P(\bar{A})=0.6$, $P(\bar{B})=0.74$
$P(\bar{A} \cap B)=P(B)-P(A \cap B)=0.26-0.22=0.04$
$P(\bar{A} \cap \bar{B})=P(\bar{A})-P(\bar{A} \cap B)=0.6-0.04=0.56$
$P(\bar{B} | \bar{A})=\frac{P(\bar{B} \cap \bar{A})}{P(\bar{B})}=\frac{0.56}{0.6}=0.93$
I am willing to intoduce you to the strategy of solving such a question. We can built a table:
\begin{matrix}
P & A & \bar{A} & \\
B & 0.22 & 0.04 & 0.26 \\
\bar{B} &  & 0.56 & 0.74 \\
 & 0.4 & 0.6 & 1
\end{matrix}
as you can see, the positions in the table are the probabilities of the intersection of $A$ or $\bar{A}$ with $B$ or $\bar{B}$, the right column contains the probabilities $B$ and $\bar{B}$ and the last row contains the probabilities $A$ and $\bar{A}$.
I hope you noticed the sum of each row is in the last column and the sum of each column is in the last row. Moreover, the sum of the last row/column is $1$. So you can easily fill the table and use Bayes theorem, e.g. $P(A \cap B)=P(B|A)P(A)$, to calculate the conditional probabilities.
Hope this will help you...
Comment: Thanks this was also very helpful


generate table rows while foreach is in progress

Question: Basically the title says it all.
I generate about 400 <trs> for a report and it takes about 30 seconds for the page to show any content(basically its  a blank page that is loading) and what I want is to see some data while the foreach generates the TRs
Is there any way to accomplish this?
Thank you.
Comment: 400 rows shouldn't take that long, for sure! Query optimization before all maybe ? beside that, if you can do this (not all the data needed at once), why not use pagination ?
Comment: web php or cli php?
Comment: @CarlosZerga i dont know what y r talking about lmao
Comment: so i think its impossible, because when user download an html file, all php code declared in that file has been executed at backend side( all the 400 elements has been already printed before the user recieve the html file), and the progress bar is something for front-end side
Comment: Is the 400 rows the result of a query?  Maybe some cURL?  You will need to show your code so we can find ways to help you optimize
Answer: You can try using output buffering:
Basically ob_start() prevents output from being sent to the page automatically and instead is stored in an internal buffer which you can call the contents of by using ob_get_contents().
So after each row is generated you can use ob_get_contents() and send the results then clean the buffer.  Check out the docs for more details.
Comment: Well ,I just put the next things right before the closing } for the foreach:  ob_get_contents();
            ob_end_flush();
  }  Did I do it wrong or ? It works pretty good tbh


Image files aren't recognized when changing a html file to page.tpl.php

Question: I want to create a drupal theme base on a static web template. after googleing I've found out that I should set up a .info file and also put my header in the html.tpl.php and body in page.tpl.php and so on.
I change index.html file into page.tpl.php, but the image in the site aren't recognized by drupal(although the index.html worked correctly).
It seem to me that drupal should show the page exactly same as static version because I have't added any code into the page ( in order to change the static version into the dynamic one ).
if so, how can I find the problem.
Thanks
HTML:

=====>
.info file:
    scripts[] = js/modernizr.js
    scripts[] = js/jquery-1.8.2.min.js
    scripts[] = js/rgpars-custom.js

html.tpl.php file :
    <head profile="<?php print $grddl_profile; ?>">
      <?php print $head; ?>
      <title><?php print $head_title; ?></title>
      <?php print $styles; ?>
      <?php print $scripts; ?>

Comment: can you come your code here? `index.html` and `page.tpl.php`?
Comment: Thanks @zhilevan For your reply, the code is too long to come here but I think the problem stem from javascript and they are not work properly. I've edited the post and add some of my script.
Comment: I dont think the problem is about js,can u tell me where did you put`html.tpl.php` exactly ?
Comment: yes you are right I have checked it before and it seems that the problem is somewhere else. sorry @zhilevan I couldn't find my post in the my menu(even in the deleted post) so I thought that it has been deleted I duplicated the post in stackoverflow site. if you let me I delete this post. I completed the post in : http://stackoverflow.com/questions/20260438/after-changing-website-template-into-drupal-images-dont-load
Comment: I suggest edit your question here and follow it here because drupalstackexchange is only for drupal maybe another person in future will have problem like you and search here to find solution for her/his problem, I think the main problem is `bad pathing` in your job, bad path in `image src`
Comment: @MohammadYousefi, how are you generating image? if you are creating content from admin, then you need to set `full html` as text format of the content.
Answer: If your images (ie. your img directory) are in your theme directory, use something like <img src="<?php print base_path() . path_to_theme(); ?>/img/works/works-full8.jpg" /> to include them.
If they are in your sites/default/files directory, use .
If they are in your site root, use


Sequelize, findOrCreate + findAll unexpected behavior

Question: I'm trying to fetch data from a dog API and I want to add to my database only their temperaments. I tried using some loops and a split to isolate the data and then using
findOrCreate() to add only those who are not already in the DB, after that I use findAll()
to get that info from the DB to send it using expressJS.
The unexpected behavior comes when I go to the route that executes all this and the route
only gives about half of the temperaments (they are 124 and it displays arround 54), then when I refresh the page it shows all 124 of them. The DB gets populated with all the 124 in one go so the problem is with findAll()
This is the function that isolates the temperaments and append them to the DB:
module.exports = async () => {
  const info = await getAllDogs();
  info.forEach(async (element) => {
    const { temperament } = element;
    if (temperament) {
      const eachOne = temperament.split(", ");
      for (i in eachOne) {
        await Temperament.findOrCreate({
          where: { name: eachOne[i] },
        });
      }
    }
  });
};

And this is the code that gets executed when I hit my expressJS sv to get the info
exports.temperaments = async (req, res) => {
  try {
    await getTemperaments();  //this function is the above function
  } catch (error) {
    res.status(500).send("something gone wrong", error);
  }
  const temperamentsDB = await Temperament.findAll();
  res.json(temperamentsDB);
};

So as you can see the last function  executes the function that appends all the data to the DB and then sends it with findAll and res.json()
Answer: forEach is a synchronous method so it doesn't await a result of the async callback. You need to do for of in order to get wait for all results:
module.exports = async () => {
  const info = await getAllDogs();
  for (element of info) {
    const { temperament } = element;
    if (temperament) {
      const eachOne = temperament.split(", ");
      for (i in eachOne) {
        await Temperament.findOrCreate({
          where: { name: eachOne[i] },
        });
      }
    }
  }
};


Describing the causal effect of coffe and heat on esophageal tumor. Interaction, confounding, mediation?

Question: The public health world is discussing in these days the news that coffee is no longer considered carcinogenic but heat of the drinks is the real culprit.
I'm trying to visualizing this as a causal network, but I'm having some difficulties.
If we compare coffe drinkers to not drinkers, heat can or cannot be present only in the drinkers, while it will be always absent in not drinkers. is this a sort of interaction? or is a mediation since heat is provoked by coffe (so to speak).
If we compare coffe drinks with other drinks as likely to be hot or not (eg, statistically uncorrelated to heat), heat and coffe are uncorrelated and so a univariate relationship should exist between heat and tumor no relationship for the kind of drink.
If we compare coffe with eg. soda, a cold drink, then we have a proper mediation?
I'm clearly confused and would appreciate a more expert opionion.
Comment: Can you please add a reference to papers/studies showing this?
Answer: I think I would call this a confounding relationship, not an interaction or mediation. Heat is confounding the apparent causal effect between coffee and cancer. Conditional on heat, coffee has no causal effect on cancer. I would draw the causal diagram as follows:
Coffee <-> Heat -> Cancer

Where <-> indicates correlation (since you can have cold coffee and hot non-coffee beverages), and -> indicates unidirectional causality.
Comment: uhm, I don't agree. A confounder is a phenomenon that cause both exposition and consequence. In this case heat cannot cause coffe exposure. I think it is more correct to interpretate heat as a mediator which convey all relation between coffe and diseases.


Comparing with =IFERROR(IF(VALUE.. with 2 Phone Numbers columns

Question: I am messing with <code>=IFERROR(IF(VALUE(LEFT())</code> and can't solve it after spending a whole night (my bad). I want auto generated result like below picture.
Solution will be like this

If A2 and B2 have different number, Expected Result should be B2.
If B3 is empty or having no value, Expected Result should be be A1.
If A4 and B4 have same value, Expected Result should be any of them.
If B2 have Fax #, Expected Result should be A5.

I am currently using
<code>=IFERROR(IF(VALUE(LEFT(B2,1)),IF(A2=B2,A2,B2),A2),B2)
</code>
as an array formula, but can't get the required results. I mess with array and looking for another simple solution.
Answer: Try,
<code>=if(isnumber(--left(b2)), b2, a2)
</code>
If you only want the single left-most character/digit from a string, you don't need the 1.


Musical "ellipsis mark" (name & usage of this symbol)

Question: I've been seeing this mark for years, usually in historical (e.g. Baroque) manuscripts, and it's meaning is clear to me. It's a sort of W-shaped squiggle with a tail that (usually) comes at the end of a line, and marks the pitch of the next note. In manuscripts, it is sometimes shortened to more of a "check mark" shape. As a performer, it a nice convenience to have, especially on the last line of a page, and I've even occasionally penciled the mark into scores that I'm playing or singing from, for my own benefit.
I like to think of it as a sort of musical "ellipses mark", but it recently occurred to me that I've never officially "learned" anything about this mark (i.e. from reading) -- I don't even know it's name! I don't see it in modern printed music (unless it's an edition that's trying to preserve the look of the original manuscript as much as possible), and I don't think I've seen it mentioned in any theory textbook. Has this mark become obsolete (and if so, when)? If it's not obsolete, why is it so hard to find information on? (e.g. is it's use restricted to some small subset of musicians?).
Granted, I'm having a hard time knowing what to google for, but it seems surprising that there's so little information on this mark, given it's apparent one-time ubiquity. I've selected a handful of examples below:

In Palestrina:

In Stradella (upper bass voice):

In Handel's Messiah:

In a Telemann treatise
In Bach's Brandenburg Concerti
Or, pretty much anything by Bach
Comment: Great! I have long wanted a sign for this use to notate when I need a reminder for what's coming on next line. I didn't know this existed...!
Answer: This is a custos (from latin, plural custodes), see lilypond documentation. No match in English Wikipedia, however.
The German Wikipedia has it and adds the following translations (as its usage mostly dating from the 18th century):

Guidon (French)
Direct (English)
Mostra (Italian) and
Kustos, Weiser (German).

A German Encyclopedia of Church Composition (A. Weger, Lexikon der kirchlichen Tonkunst), from 1870 has this entry:
[]
Summarized text: ... was in earlier times set to the end of a score line, to indicate, where the first note on the next line appears. Nowadays only used in choral music.
Comment: Dolmetsch has it as well, with pretty much the same names as the German Wikipedia: http://www.dolmetsch.com/musicalsymbols.htm.
Comment: Yep, these are all custos.  I'll just add that they go back to medieval music, where they are very usual.  Here's an example, where every line (but the first, which is the end of the previous section) ends with a custos.  https://bibleartists.files.wordpress.com/2012/09/medieval-illuminated-music-manuscript.jpg?w=594


Isotope Uncaught Reference Error

Question: I'm trying to create sortable menu by using isotope.
Long story short:

On line 4 i get Uncaught ReferenceError: $ is not defined.
Here is HTML

All items

Web Design

Graphic Design

Photography

Any ideas how I can make this jQuery code work? Thanks in advance.
Comment: I get error in line 5, sorry
Comment: `scripts/jquery.js` should be an external google jQuery URL, so it's cached.
Answer: The problem is that jQuery is undefined.
On your line , you are stating that jQuery is in a folder called scripts at the same level as the page you are including it on.
This could be simply that you have forgotten to include it in the project, but it's more likely that you referenced it incorrectly. Depending on where your jquery.js file actually resides, you will need to change this reference.

You can use the relative link ../ to navigate up one folder.
You can use the root-relative link /scripts/jquery.js to always reference the root.
You can use the absolute link https://code.jquery.com/jquery-3.2.1.min.js to load a hosted version of jQuery from the CDN.

For further reading on relative and absolute URLs, I recommend checking out this Coffee Cup article.
Hope this helps! :)
Comment: It's `src`, not `scr`. You also need the protocol (`https://` or `http://`) ...or the protocol-agnostic `src="://code.jquery.com/jquery-3.2.1.min.js"`. You're **probably** simply looking for `src="http://code.jquery.com/jquery-3.2.1.min.js"`.
Comment:
 I also thought that it is the most obvious way but still nothing
Comment:  that how it looks, not working :(
Comment: You should absolutely use https. No reason at all not to. If you don't, browsers will give insecure content warnings or block it if your page is served over https. No reason to use http in 2017.
Comment: http was there, it jus did not get copied in comment section somehow, and *yes* you are right about this damn typo, sorry for that. Now I have no error indeed, thank you but grid system is not working anyway
Comment: That is true, and there's more information regarding that at https://doesmysiteneedhttps.com/. However, there are plenty of legacy systems around, and setting up HTTPS can be a little much for someone just starting out :)
Answer: The two .js files haven't loaded when your code is executed. Hence why $ is still undefined. If you move the

Isotope Uncaught Reference Error


dpm in submit handler not working?

Question: I have a very interesting problem.
I have 2 dev machines (both mac, running XAMPP).
In my project I have a submit handler for a custom entity form.
When I try to dpm() (or kpr, or dsm, ...) the contents of $form or $form_state, well, nothing really happens (on one machine!)
On the other machine, the dpm (for the exact same project, in the exact same submit handler, for the exact same variable) works perfectly.
I have copied php.ini settings files, checked the php memory limit, ...
Nothing seems different...
When I try to dpm($form['entities']); then it works on both instances.
This seems to give me the impression that it works when you narrow down the array...
But when I inspect the array ($form) it only contains approx. 40 items or so... (okay there are arrays in arrays and so on).
No errors on in the php log, apache log, ...
Any suggestions would be very welcome!
Many thanks in advance!
Comment: I had a similar case where I couldn't display an array and then it was ruining the whole dsm of the page. but when I removed the buggy dsm it was working fine for the others. I didn't find a solution but it seems some arrays are not working well at all.
Comment: Thanks for the comment. But in this case it does not make sense that on one machine it works just fine, and on the other it does not. By now i have also compared with staging environment, and on staging the dpm also works perfectly...
Comment: dump the DB which doesn't work and import it in a local environment so you can test it. Try to compare what is different between the nodes/content types.
Answer: I had a similar problem. In a submit handler, tried to run dsm($form_state). This caused all submit handlers to not run. If I run $form_state['complete form'] = NULL; before the dsm(), it works. I presume it was too big with the complete form in there.

dpm in submit handler not working?


Filtrar os dados da tabela por um input type="text" separado

Question: Tenho uma tabela que usa o DataTables, e para mim filtrar os dados dela, utilizo um input que fica fora do código da tabela, um input separado. Acontece que recentemente eu implementei o server-side processing no meu DataTables, para carregar os registros mais rapidamente, e essa função do filtro na tabela parou de funcionar, não sei exatamente o porque, fica abaixo meu código:

// Código que era para direcionar o filtro da tabela ao input #filterbox
$("#filterbox").keyup(function() {
dataTable.search(this.value).draw();
});

index.php

<input type="text" name="filterbox" id="filterbox">

            ID
            First name
            Last name
            Position
            Office

            ID
            First name
            Last name
            Position
            Office

 ssp.class.php
/**
 * Searching / Filtering
 *
 * Construct the WHERE clause for server-side processing SQL query.
 *
 * NOTE this does not match the built-in DataTables filtering which does it
 * word by word on any field. It's possible to do here performance on large
 * databases would be very poor
 *
 *  @param  array $request Data sent to server by DataTables
 *  @param  array $columns Column information array
 *  @param  array $bindings Array of values for PDO bindings, used in the
 *    sql_exec() function
 *  @return string SQL where clause
 */
static function filter ( $request, $columns, &$bindings )
{
    $globalSearch = array();
    $columnSearch = array();
    $dtColumns = self::pluck( $columns, 'dt' );

    if ( isset($request['search']) && $request['search']['value'] != '' ) {
        $str = $request['search']['value'];

        for ( $i=0, $ien=count($request['columns']) ; $i<$ien ; $i++ ) {
            $requestColumn = $request['columns'][$i];
            $columnIdx = array_search( $requestColumn['data'], $dtColumns );
            $column = $columns[ $columnIdx ];

            if ( $requestColumn['searchable'] == 'true' ) {
                if(!empty($column['db'])){
                    $binding = self::bind( $bindings, '%'.$str.'%', PDO::PARAM_STR );
                    $globalSearch[] = "`".$column['db']."` LIKE ".$binding;
                }
            }
        }
    }

    // Individual column filtering
    if ( isset( $request['columns'] ) ) {
        for ( $i=0, $ien=count($request['columns']) ; $i<$ien ; $i++ ) {
            $requestColumn = $request['columns'][$i];
            $columnIdx = array_search( $requestColumn['data'], $dtColumns );
            $column = $columns[ $columnIdx ];

            $str = $requestColumn['search']['value'];

            if ( $requestColumn['searchable'] == 'true' &&
             $str != '' ) {
                if(!empty($column['db'])){
                    $binding = self::bind( $bindings, '%'.$str.'%', PDO::PARAM_STR );
                    $columnSearch[] = "`".$column['db']."` LIKE ".$binding;
                }
            }
        }
    }

    // Combine the filters into a single string
    $where = '';

    if ( count( $globalSearch ) ) {
        $where = '('.implode(' OR ', $globalSearch).')';
    }

    if ( count( $columnSearch ) ) {
        $where = $where === '' ?
            implode(' AND ', $columnSearch) :
            $where .' AND '. implode(' AND ', $columnSearch);
    }

    if ( $where !== '' ) {
        $where = 'WHERE '.$where;
    }

    return $where;
}

Answer: Bom dia, tenta dessa forma pra ver se funciona:
$('#filterbox').on( 'keyup', function () {
    table
        .search( this.value )
        .draw();
} );


Gradle spotbugs plugin

Question: I am new to Gradle and trying to configure Spotbugs for my Spring Boot multi module project.
In my parent, build.gradle,
buildscript {
    dependencies {
        classpath "org.springframework.boot:spring-boot-gradle-plugin:${versionSpringBoot}"
    }
}

plugins {
  id 'com.github.spotbugs' version '1.6.8'
}

allprojects {
    apply plugin: 'eclipse'
    apply plugin: 'idea'
}

subprojects {
    apply plugin: 'java'
    apply plugin: 'io.spring.dependency-management'
    apply plugin: 'pmd'
    apply plugin: 'jacoco'

    dependencyManagement {
        imports {

        }
    }

    configurations{
    }

    sourceCompatibility = '15'
    targetCompatibility = '15'

    dependencies {
    }

    pmd {
        consoleOutput = true
        toolVersion = "${versionPmd}"
        sourceSets = [sourceSets.main]
        ruleSets = ["category/java/errorprone.xml", "category/java/bestpractices.xml"]
    }

    spotbugs {
        toolVersion = "${versionSpotBugs}"
        sourceSets = [sourceSets.main]
    }

    jacoco {
        toolVersion = "${versionJacoco}"
    }

    jacocoTestReport {
        reports {
            xml.enabled = true
        }
    }

    tasks.withType(com.github.spotbugs.SpotBugsTask) {
        reports {
            xml.enabled = false
            html.enabled = true
        }
    }
}

Spotbugs doesn't run on running

./gradlew check
Answer: The main issue with your build configuration is that you apply the SpotBugs plugin only to your root project. The following configuration solves that (leaving out configurations that are unrelated to the SpotBugs plugin for brevity):
plugins {
    // we don't need to *apply* the plugin to the root project, do we?
    id 'com.github.spotbugs' version '4.7.0' apply false
}

subprojects {
    apply plugin: 'java'
    // this is the most important part, applying the plugin to the subprojects,
    // too:
    apply plugin: 'com.github.spotbugs'

    spotbugs {
        toolVersion = '4.2.2'
    }

    tasks.withType(com.github.spotbugs.snom.SpotBugsTask) {
        reports {
            xml.enabled = false
            html.enabled = true
        }
    }
}

With this configuration, ./gradlew check also runs the SpotBugs tasks of subprojects (tested with Gradle 6.8.3).
Please note that I've also made a few other changes:

I'm using a recent version of the plugin as the one that you've used (1.6.8) is several years old and doesn't seem to work with recent versions of Gradle.
I've removed the sourceSets configuration which is not needed and doesn't work anyway.
I've replaced the fully qualified name of the task type with an up-to-date version.

I hope this helps. Please let me know if you're stuck with the old SpotBugs version for some reason; knowing the Gradle version that you use would help in that case.
Answer: The below works (some adjustments to make it work locally)
gradle - 6.5.1
buildscript {
    dependencies {
        classpath "org.springframework.boot:spring-boot-gradle-plugin:2.4.3"
    }
}

plugins {
  id 'com.github.spotbugs' version '4.7.0'
}

import com.github.spotbugs.snom.SpotBugsTask

allprojects {
    apply plugin: 'eclipse'
    apply plugin: 'idea'
    apply plugin: 'java'
    apply plugin: 'io.spring.dependency-management'

    repositories {
        mavenCentral()
    }

    dependencyManagement {
        imports {

        }
    }

    configurations{
    }

    sourceCompatibility = '15'
    targetCompatibility = '15'

    dependencies {
    }

    spotbugs {
        toolVersion = '4.2.1'
    }

    tasks.withType(SpotBugsTask) {
        reports {
            xml.enabled = false
            html.enabled = true
        }
    }
}

Comment: I have this in the parent module and all my code is the child modules. Is there any changes to be done in the child module? "./gradlew check" does not run anything.
Comment: what happens with `./gradlew build` ?

Gradle spotbugs plugin


Como fazer um método retornar uma string em Java

Question: Quero criar um programa que leia o nome e a idade de uma pessoa e compare as idades e por fim mostre o nome da pessoa mais velha. Meu raciocínio foi comparar qual é a maior e retornar o índice da posição do array. Mas o retorno é sempre 0. E também eu não sei como pegar o retorno e mostra o conteúdo do array na posição.
package Lista1;
import java.util.Scanner;

public class Pessoa {
    private static String nome;
    private static int idade;

    public static int comparaIdade(Pessoa[] pessoa){

        int maior=0;
        int ind = 0;

        for(int i = 0; i<3; i++){
            if(pessoa[i].idade>maior){
                maior=pessoa[i].idade;
                ind=i;
            }
        }
        return ind;

    }

    public static void main(String[] args){
        Pessoa[] pessoa = new Pessoa[3];
        Scanner input = new Scanner(System.in);

        for(int i = 0; i<3; i++){
            System.out.println("Digite o nome da pessoa: ");
            pessoa[i].nome = input.nextLine();
            System.out.println("Digite a idade da pessoa: ");
            pessoa[i].idade = input.nextInt();
            input.nextLine();
        }

        System.out.printf("Nome da pessoa mais velha: %d", comparaIdade(pessoa));

    }

}

Answer: Se deseja que retorne o nome, deve fazer isto e não retornar o índice como estava fazendo.
Tem vários outros erros aí. Está misturando a pessoa com o algoritmo, está criando membros estáticos onde devem ser de instância, não está instanciando um objeto antes de usá-lo.
import java.util.Scanner;

class Pessoa {
    public String nome;
    public int idade;
}
class Programa {
    public static String comparaIdade(Pessoa[] pessoa){
        int maior = pessoa[0].idade;
        int indice = 0;
        for (int i = 1; i < 3; i++) {
            System.out.println(pessoa[i].idade);
            if (pessoa[i].idade > maior) {
                maior = pessoa[i].idade;
                indice = i;
            }
        }
        return pessoa[indice].nome;
    }

    public static void main(String[] args){
        Pessoa[] pessoa = new Pessoa[3];
        Scanner input = new Scanner(System.in);
        for (int i = 0; i < 3; i++) {
            pessoa[i] = new Pessoa();
            System.out.println("Digite o nome da pessoa: ");
            pessoa[i].nome = input.nextLine();
            System.out.println("Digite a idade da pessoa: ");
            pessoa[i].idade = input.nextInt();
            input.nextLine();
        }
        System.out.printf("Nome da pessoa mais velha: %s", comparaIdade(pessoa));
    }
}

Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura.
Comment: Está retornando sempre a última posição do array.
Comment: É o que eu falei, o seu código tem outros erros, não é só o que eu mostrei na resposta. Você não está criando 3 nomes, está criando 1 e sobrepondo ele, por isso fica sempre o último.
Comment: Consertei os erros principais.


How can I combine three select queries, each excluding the results from the previous ones?

Question: Let's suppose I have three different select statements, like this
SELECT 'Final', ID FROM Stage WHERE stageCompleted = 1;
SELECT 'Active', ID FROM Stage WHERE StartDate < NOW();
SELECT 'Inactive', ID FROM Stage;

What I would like is some way of combining these such that it displays one table that

returns the string 'Final' and the ID of the row for all of the rows matched by the first query
returns the string 'Active' and the ID of the row for all of the rows matched by the second query but not by the first
and returns the string 'Inactive' and the ID of all the remaining rows.

I looked into all sorts of joins, but they don't seem to really fit the problem I'm trying to solve. How can I combine three select queries in SQL, where each one excludes the results from the previous ones?
Comment: Think about the problem a different way. You are wanting to return every row in the Stage table, but each row will be in only one "status".  You could get the same result set by UNIONing three different result sets, and use anti-join patterns to limit the rows returned by each, but that isn't necessary (in your case), and such a query would not be nearly as efficient as a single scan through the Stage table.
Answer: Use a case statement
SELECT
   CASE WHEN stageCompleted = 1 THEN 'Final'
        WHEN StartDate < NOW() THEN 'Active'
        ELSE 'Inactive'
   END,
   ID
FROM Stage

Comment: +1 This is the cleanest, most efficient approach, a single shot through the entire table.  There are other approaches to returning the result set, but they aren't likely to perform nearly as well as this will. (To be a little more precise, it's really a CASE "expression" which is used within the SQL statement.)


Analise de código em produção

Question: Olá,
Gostaria de saber se há alguma ferramenta open source para eu analisar o uso de memória/processamento alocado por meus métodos em produção.
Gostaria disso para saber quais métodos precisam de atenção e melhorias, e assim diminuir o uso de recursos por minhas aplicações.
Meus ambientes de produção são compostos do php-5.6 ou php-7 (depende da aplicação), nginx como proxy reverso, Apache2 por trás do NGINX (por limitações em algumas aplicações) e Linux (Ubuntu/CentOS) como S.O.
Comment: Você pode ver algumas informações com o [Xdebug](https://xdebug.org/) (livre) e o [blackbird](https://blackfire.io/) (pago) ambos fazem profiling.
Comment: Qualquer profiler vai deixar sua aplicação (talvez até bem) lenta, você executar só para detectar os gargalos depois desliga, faz as devidas modificações e liga novamente para ver se surtiu efeitou ou não.
Comment: Open-Source não sei. Existe o NewRelic e afins, que monitoram todo o servidor e a aplicação. É capaz de lista o tempo médio de resposta, quais foram os mais lentos e porque foram, qual função consumiu mais tempo para ser processado,até mesmo a query que demorou para ser processada e, lógico, também informa erros de execução... Ele pode normalmente é mantido ligado "para sempre". Porém é paga.
Comment: No caso do XDEBUG ele não vai deixar minha aplicação mais lenta?
Comment: Ah sim entendi, achei que poderia haver algo como o Sentry (que faz análise de erros) de maneira que pudesse ficar ligado em produção sem afetar performance... (mas achei muito mágico no caso do profiling) vou fazer uns testes.
Comment: No caso do Xdebug, você executa aquele trecho de código e ele vai medindo quanto tempo **cada operação** (chamadas de métodos/funções) levou para ser realizada e quantia de mémoria alocada. Depois ele gera um arquivo com essas informações é necessário uma outra ferramenta para analizar esse log, o [webgrind](https://github.com/jokkedk/webgrind) é um exemplo existem outras opções.
Answer: Mas no caso, você precisa mensurar o pico de acesso nesses Servidores Web, com rrdtool, cacti + mrtg, você vai ter os gráficos desses acessos.
Depois tem a velocidade do disco e quantidade de memoria ram e tamanho e quantidade de acesso ao banco de dados.
Quando você tiver esses gráficos você vai ter uma ideia do contexto geral que está ocorrendo no Servidor como um todo, tente partir desse ponto primeiro.
Claro isso foi uma sugestão, minha que cuidei por quase 10 anos de ambientes de hospedagem, espero que consiga ajudar, qualquer dúvida estou a disposição.
Comment: Olá Nilton, mas eu gostaria de id identificar gargalos no meu código exemplo (meu método setPontuacao aloca x de memória demora em média x tempo para ser executado) com base nisso posso melhorar meus códigos

Analise de código em produção


Is it possible to run a python file from the command line, but interact with it? (REPL)

Question: Suppose I have a very simple test.py, and here is the code:
def sum(a, b):
    return a + b

print(sum(5, 6))

In my cmd, when I do python test.py, the command lines returns 11.
What I would like to do is interact with it, so I want to type, sum(4,2) and the command line should return 6, but instead I get:
sum is not recognized as an internal or external command ...
Basically I want to have the REPL feature, like WingIDE, Sublime Text. Is this possible from the command line?
Comment: `from test import sum` but you have to make sure your system knows where to look for test.py (in case it's not in your working directory)
Comment: type `python` in command line and it will run python interactively. You can then define `sum` as you do and call it, or import it from a predefined script.
Comment: Yes but how do I tell it I want it to call functions from the `test.py` file?
Comment: Thanks @Julien it worked
Comment: You can also use ``python -i test.py``, to leave it in interactive mode after the script finishes.
Comment: @jasonharper thanks I like this suggestion better!
Answer: For this purpose, you'd better pass input arguments when you run your program in cmd.
I put the code with minor modifications as below,
def sum(a, b):
    return a+b

import sys
a = int(sys.argv[1])
b = int(sys.argv[2])
print(sum(a, b))

Then in cmd, you just need to run like this:
>> python test.py 4 2
Comment: This works, but if I have multiple functions I don't think it does?
Comment: @VeeshaDawg would you give an example ? You mean passing same input arguments to different function?
Comment: What if I had more functions then just `sum`?


Migration of Virtual DC to another ESXI host

Question: I have DC on ESXi host. I want to migrate it to another ESXi host on same LAN.
What is the best option?
[I already have vCenter and VEEAM)

Turn Off DC, and do migration from ESXi-1 to ESXi-2 using VEEAM
Backup & Replication?
Turn Off DC, and do migration from ESXi-1
to ESXi-2 using vCenter ?
Don't Turn off DC, and do Live
migration from ESXi-1 to ESXi-2 using any of above application?

Or any other better option to ENSURE 100% successful migration ?
Comment: Do you have shared-storage? What version of ESXi and vSphere are you using?
Comment: Not using any shared storage.
Esxi and Vcenter both are 5.x
Answer: Use the native tools, assuming you have a vCenter. You could perform a live-migration or cold-migration (change host and datastore), depending on your license level. However, vCenter may not be happy if the DC is down. Do you only have one domain controller?
Comment: @SyedJahanzaib sounds like you don't have server or client DNS settings configured properly. Also, there is no "Primary" or "Secondary" DC - they are multimaster peers.
Comment: I have 2 DC, but strange issue is that if my Primary DC is not onlnie, then the secondary DC also donot resolve any host, either local or internet. also when main DC reboot, it takes around 20-35 minutes to response to dns queries.
Comment: on Primary DC, primary dns is point to 2nd DC, and  secondary dns to loopback client.
on 2nd DC, primary dns is pointed to primary DC, secondary dns to loopbackup of 2nd dc.


How to Present Text from Another Source?

Question: I have sometimes answered questions by copying-and-pasting text from other (disclosed!) sources, then usually italicizing the text to show it's from elsewhere.
In this thread, text from elsewhere is shown in a colored box, which does a good job of showing it as imported. How is this type of presentation accomplished?
Answer: That is a Markdown blockquote. On the desktop site you can simply select the text you copied, then press the blockquote icon on the toolbar. On mobile or if the toolbar doesn't work you can do that by starting the line with a <code>></code> character. For longer texts all lines (including empty ones) should be marked like this.
Example:
<code>> First quote
>
> Second paragraph
</code>
Will produce:

First quote
Second paragraph

Generally if you encounter a feature of the posts you can always click the edit button to see the source code of the post where you can see how stuff, like blockquotes were done. There is also a documentation on all markdown features at https://expatriates.stackexchange.com/help/formatting


How to read the txt file line by line.(int, unknown length) using C?

Question: I want to merge sort the data from a text file.(line by line)
For my merge sort to work, I have to read in the data line by line, and fit them into an array in order to sort them. (We only know that there are at most 10000 integer per line)
So I've done my research and tried these approach:
1. Use fgets, and strtok/strtol.
Problem: I don't know the max length of the char array. Besides, declaring a huge array may cause buffer overflow.
Source: How many chars can be in a char array?
2. Use fscanf to input into integer array.
Problem: The Same. I don't know how many integers there are in a line. So I won't be ok with "%d" part.(don't know how many there should be)
3. Use fscanf to input in the form of char array, and use strtok/strtol.
Problem: The Same. Since I don't know the length, I can't do something like
char *data;
data = malloc(sizeof(char) * datacount);

since the "datacount" is unknown.
Is there any way out?
UPDATE
Sample Input:
-16342 2084 -10049 10117 2786
3335 3512 -10936 5343 -1612 -4845 -14514

Sample Output:
-16342 -10049 2084 2786 10117
-14514 -10936 -4845 -1612 3335 3512 5343

Comment: Consider using [getline](https://linux.die.net/man/3/getline).
Comment: Not clear what you need to do. Please add an input output example. If you only need to sort int from several lines you don't need to read line by line, just int by int using for example `strtol` and skipping spaces and line break
Comment: @李智修 The POSIX `getline` function can automatically allocate just enough memory needed for the line, if passed the correct arguments.
Comment: Use `fscanf` **and a loop**.
Comment: @PaulR I just looked at the documentation you provided.
But I saw that this method still need to specify the length of buffer in size_t. I may be wrong though.
Comment: @Ôrel Ok, I will update my post. What do you mean by using strtol int by int? But I need to at least input the char array so I can do further process.
Comment: 'For my merge sort to work, I have to read in the data line by line, and fit them into an array in order to sort them' - not with a merge sort, no.
Comment: @MartinJames What do you mean? Is there another way to sort it without withdrawing data from the txt file?
Comment: Why are you reading the whole list of lines into an array, if you try to prove merge sort?  Preciselly, merge sort is useful in cases where not all the data fit in the core memory.  I don't understand some of your requirements you find needed, but aren't.
Comment: @LuisColorado Sorry, but our professor asked us to also implement the pthread, 1 for merging, 2 for merge sort.
I didn't mentioned this since this is unrelated to the problem.
Answer: You said:

We only know that there are at most 10000 integer per line

So just go for it. A standard-compliant compiler (and environment) should provide the ability to define an array of up to 65,535 objects. 10,000 is far less so just define a static array:
int a[10001], n;

int main() {
    // Open file
    n = 0;
    while (fscanf(fp, " %d", &a[n]) == 1) {
        // Process a[n]
        n++;
    }
}

If you know about platform specification (like sizeof(int) == 4), you can assume integer length. For example, the maximum length of a 32-bit integer is 11 characters (at -2147483648) or so. You can then define a char array with calculated length.
Comment: But what about the rows part? How can we be sure whether the flow is moving onto the next line?
Comment: .... But why is it necessary to read all the integers before sorting... merge sort can begin as soon as the first integer is read... no need to read even one line of text.
Comment: @LuisColorado It's because we need to also implement the multi-threading by pthread into 3 thread(2 merge sort, 1 merge). It would be easier in this way IMO.
Comment: anyway, there's no need to read all the data into a huge array.... the thread chain must be configured as a pipeline that does one merge phase and creates a new thread when it detects the phase is not enough for a single merge of all data.  You need an array only to store one row of data... not more.
Answer: You can indeed use fscanf to read the individual integers. What you need besides that is to know not only about pointers and malloc but also about realloc.
You simply do something like
int temporary_int;
int *array = NULL;
size_t array_size = 0;

while (fscanf(your_file, "%d", &temporary_int) == 1)
{
    int *temporary_array = realloc(array, (array_size + 1) * sizeof(int));
    if (temporary_array != NULL)
    {
        array = temporary_array;
        array[array_size++] = temporary_int;
    }
}

After that loop, if array is not a null pointer then it will contain all the integers from the file, no matter how many there were. The size (number of elements) is in the array_size variable.

After seeing the update it's much easier to understand what is wanted.
In pseudo-code it's easy:
while(getline(line))
{
    array_of_ints = create_array_of_ints();

    for_each(token in line)
    {
        number = convert_to_integer(token);
        add_number_to_array(array_of_ints, number);
    }

    sort_array(array_of_ints);
    display_array(array_of_ints);
}

Actually implementing this is much harder, and depends somewhat on your environment (like if you have access to the POSIX getline function).
If you have e..g getline (or a similar function) then the outer loop in the pseudo-code is easy, and will look just about what it already does. Otherwise you basically have to read character by character into a buffer that you dynamically expand (using realloc) to fit the whole line.
That brings us to the contents of the outer loop: Splitting the input into a set of values. The basic solution you already have by the first code-snippet in this answer, where I reallocate the array as needed in the loop. to split the values then strtok is probably the simplest one to use. And converting to an integer can be done with strtol (if you want validation) of atoi if you don't care about validating your input.
Note that you don't really need to allocate the arrays dynamically. 10000 int values will, on current systems where sizeof(int) == 4 be "only" 40000 bytes. That's small enough to fit even on the stack of most non-embedded systems.
Comment: You should mention that doing a `realloc` on each and every iteration is not the most efficient solution (correct me if I'm wrong).
Comment: @MichaelWalz True, it's not. But it's easier to understand and much less code than allocating chunks of multiple "elements". I tried to keep it as simple as possible.
Comment: Init the array with an arbitrary size then double the size each time there is no room left is easy to implement, in a second time once you have the parsing algo done.
Comment: @Ôrel Thanks, I think I know what you mean now.
Comment: @Someprogrammerdude But how can I check whether the whole process is going line by line? Since we choose to input one integer at one time, we may lost the track of the rows part.
I tried to use another while() outside of your loop to check if it is EOF, but my attempts always seems weird. Like:

while(fscanf(file, "%d", &anothertmp) != EOF)
In this case, it may be produce redundant outcome.
Comment: @李智修 After seeing your update it's easier to understand what you want. Updated answer
Comment: Thanks, I know what to do now :).
Answer: The one approach you did not consider, is probably the most portable one: write your own "read one token from a FILE stream, and convert it to a long; but never cross a newline boundary" function:
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>

/* Reads one decimal integer (long) from 'input', saving it to 'to'.
   Returns: 0    if success,
            EOF  if end of input,
            '\n' if newline (end of line),
            '!'  if the number is too long, and
            '?'  if the input is not a number.
*/
int read_long(FILE *input, long *to)
{
    char    token[128], *end;
    size_t  n = 0;
    long    value;
    int     c;

    /* Consume leading whitespace, excluding newlines. */
    do {
        c = fgetc(input);
    } while (c == '\t' || c == '\v' || c == '\f' || c == ' ');

    /* End of input? */
    if (c == EOF)
        return EOF;

    /* Newline? */
    if (c == '\n' || c == '\r') {
        /* Do not consume the newline character! */
        ungetc(c, input);
        return '\n';
    }

    /* Accept a single '+' or '-'. */
    if (c == '+' || c == '-') {
        token[n++] = c;
        c = fgetc(input);
    }

    /* Accept a zero, followed by 'x' or 'X'. */
    if (c == '0') {
        token[n++] = c;
        c = fgetc(input);
        if (c == 'x' || c == 'X') {
            token[n++] = c;
            c = fgetc(input);
        }
    }

    /* Accept digits. */
    while (c >= '0' && c <= '9') {
        if (n < sizeof token - 1)
            token[n] = c;
        n++;
        c = fgetc(input);
    }

    /* Do not consume the separator. */
    if (c != EOF)
        ungetc(c, input);

    /* No token? */
    if (!n)
        return '?';

    /* Too long? */
    if (n >= sizeof token)
        return '!';

    /* Terminate token, making it a string. */
    token[n] = '\0';

    /* Parse token. */
    end = token;
    errno = 0;
    value = strtol(token, &end, 0);
    if (end != token + n || errno != 0)
        return '?';

    /* Save value. */
    if (to)
        *to = value;

    return 0;
}

To advance to the next line, you can use e.g.
/* Skips the rest of the current line,
   to the beginning of the next line.
   Returns: 0 if success (next line exists, although might be empty)
          EOF if end of input.
*/
int next_line(FILE *input)
{
    int  c;

    /* Skip the rest of the current line, if any. */
    do {
        c = fgetc(input);
    } while (c != EOF && c != '\n' && c != '\r');

    /* End of input? */
    if (c == EOF)
        return EOF;

    /* Universal newline support. */
    if (c == '\n') {
        c = fgetc(input);
        if (c == EOF)
            return EOF;
        else
        if (c == '\r') {
            c = fgetc(input);
            if (c == EOF)
                return EOF;
        }
    } else
    if (c == '\r') {
        c = fgetc(input);
        if (c == EOF)
            return EOF;
        else
        if (c == '\n') {
            c = fgetc(input);
            if (c == EOF)
                return EOF;
        }
    }

    ungetc(c, input);
    return 0;
}

To read the longs on each line, you can use a dynamically resized buffer, shared across lines:
int main(void)
{
    long   *field_val = NULL;
    size_t  field_num = 0;
    size_t  field_max = 0;

    int     result;

    do {
        /* Process the fields in one line. */
        field_num = 0;

        do {

            /* Make sure the array has enough room. */
            if (field_num >= field_max) {
                void *temp;

                /* Growth policy; this one is linear (not optimal). */
                field_max = field_num + 5000;

                temp = realloc(field_val, field_max * sizeof field_val[0]);
                if (!temp) {
                    fprintf(stderr, "Out of memory.\n");
                    return EXIT_FAILURE;
                }

                field_val = temp;
            }

            result = read_long(stdin, field_val + field_num);
            if (result == 0)
                field_num++;

        } while (result == 0);

        if (result == '!' || result == '?') {
            fprintf(stderr, "Invalid input!\n");
            return EXIT_FAILURE;
        }

        /*
         * You now have 'field_num' longs in 'field_val' array.
        */

        /* Proceed to the next line. */
    } while (!next_line(stdin));

    free(field_val);
    field_val = NULL;
    field_max = 0;

    return EXIT_SUCCESS;
}

While reading input character by character is not the most efficient way (it tends to be slightly slower than reading eg. line by line), it is compensated by its versatility.
For example, the above code works for any newline convention (CRLF or \r\n, LFCR or \n\r, CR \r, and LF \n) (but in Windows you'll want to specify the "b" flag for fopen() to stop it from making its own newline mangling).
The read-field-by-field approach is also easily extended to e.g. CSV format, including its peculiar quoting rules, and even embedded newlines.
Comment: @李智修: No worries! I personally would use `getline()` + `strtol()`, myself. I just wanted to point out this approach, because you did not mention it in the question, and it works well for e.g. proper CSV parsing.
Comment: Your approach is good, yet I prefer more simple way.....since I'm implementing on a normal Linux machine.
Thanks for your answering anyway :).


Extracting values from multidimensional array where keys are dynamically generated

Question: I have an array
$array1 = Array (
       [a1] => Array  (
               [a_name] => aaaaa
               [a_value] => aaa
             )

       [b1] => Array (
               [b_name] => bbbbb
               [b_value] => bbb
           )
       [c1] => Array (
               [c_name] => ccccc
               [c_value] => ccc
           )

     )

Now I want to extract the value of $array1[b1][b_name]. But the thing here is the keys (b1 and b_name) will be generated dynamically. The situation here is I have a multidimensional array and the keys of which i want the value. So how do i get the value.
eg.
$array1[b1][b_name]

should return
bbbbb

and
$array1[c1]

should return
array([c_name]=>ccccc
       [c_value]=>ccc
   )

and so on...
EDIT
Lets keep it this way,
The second array is
$array2 = Array (
         [b1] => Array (
               [b_name]=> zzzzz
             )
      )

Now Intersecting $array1 and $array2, I want the value of the $array1 ie. bbbbb
Answer: Try this
<?php
   //you will have $firstkey and $secondkey as index values of $array1
   if (isset($firstkey) && array_key_exists($firstkey, $array1)) {
       if (isset($secondkey) && array_key_exists($secondkey, $array1[$firstkey])) {
           print_r($array1[$firstkey][$secondkey]);
       }
       else {
           print_r($array1[$firstkey]);
           echo "$secondkey does not exist";
       }
   }
   else {
       echo "$firstkey does not exist";
   }
?>

Answer: You can use a foreach loop to get at both the keys and the values. I am not certain though that it will help you if you only want specific ones and you don't know how to identify them:
foreach ($array1 as $key => $value) {
    echo "Key: $key; Value: $value
\n";
}

// From your example of the array inside a1:
// the $key would be: a_name
// the $value would be: aaaaa


Como transferir arquivo usando socket em python?

Question: Estou fazendo um programa que consiste em passar um arquivo usando protocolo UDP. O usuário que rodar o arquivo client.py deverá ter arquivos na pasta raiz do programa para poder transferir para a pasta em que o arquivo server.py está. Os programas não chegam a disparar erros, apenas não acontece a transferência.
Código do server.py
import socket
from os import write
HOST = '127.0.0.1'  # Endereco IP do Servidor
PORT = 5000            # Porta que o Servidor esta
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
orig = (HOST, PORT)
udp.bind(orig)
#Até aqui é a conexão sendo realizada

#Ouvindo os nomes dos arquivos do client que o server poderá escolher.
while True:
    msg, cliente = udp.recvfrom(1024)
    msg = msg.decode()
    if(msg == 'stop'):
        break
    print(msg)
#input da opção escolhida pelo server
opt = str(input())
optB = bytes(opt, encoding='utf-8')
udp.sendto(optB, cliente)

#tentativa frustrada de receber o arquivo do client
while True:
    dados = udp.recv(1024)
    if not dados:
        break
    write(dados)

udp.close()

Código do cliente.py
import socket, os

HOST = '127.0.0.1'  # Endereco IP do Servidor
PORT = 5000            # Porta que o Servidor esta
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
dest = (HOST, PORT)
print('Conectado!')
#Até aqui é a conexão sendo realizada
k = 0
dirs = os.listdir(".")

#Pegando nome dos arquivos que está na pasta do client e mandando pra o server
for file in dirs:
    i = k
    i = str(i)

    msg = '[' + i + '] - ' + file
    msgB = bytes(msg, encoding='utf-8')
    udp.sendto(msgB, dest)
    k+=1

#Mandando mensagens pra o server
msgB = bytes ('Escolha uma das opções acima digitando um número:', encoding='utf-8')
udp.sendto(msgB, dest)
msgB = bytes ('stop', encoding='utf-8')
udp.sendto(msgB, dest)

#opção que o server escolheu
opt, server = udp.recvfrom(1024)
opt = int(opt.decode())

#tentativa frustrada de enviar o arquivo
arq = open(dirs[opt], 'rb')
for i in arq.readlines():
    udp.send(i,dest)

udp.close()

Answer: TL;DR Tente adicionar uma forma de handshake entre o cliente e servidor ou adicionar um delay entre os pacotes enviados
O problema se dá no modo em que os pacotes UDPs são tratados. No seu exemplo temos os seguintes requerimentos:

Perda de pacote = 0%
Saber quando terminar de enviar ou ler um arquivo de tamanho arbitrário
Saber lidar com buffers de rede cheios

Infelizmente, o protocolo UDP não possui nenhum dos seguintes, mas mesmo assim é possível tentar fazer uma transferência de arquivos usando o mesmo (dito que a perda de pacote seja aceitavel e voce saiba lidar com pacotes perdidos.)
Perda de pacote
No seu exemplo, creio que voce tu estás a utilizar o localhost ou uma rede local (o que melhora a situação dos pacotes perdidos.)
Tamanhos arbitrários
O problema da transferência se dá por causa da ausência de um handshake, em outros termos, a negociação sobre a troca de informações entres dois participantes. Uma simples solução para isso seria: enviar o número de pacotes consecutivos a ser enviado, adicionar um time-out no tempo de espera pelos pacotes entre outros.
ex:
"Se o cliente não mandar nenhum pacote em 5 segundos, terminamos"

"Se tivermos um pacote faltando, mandamos outro pedido ao cliente"

"Depois do cliente mandar X pacotes, terminamos"

O método que você usa para fazer o handshake não importa mas sim saber lidar com os melhores e piores casos (perda de pacote, perda de conexão, ordem de pacote errada, etc.)
Congestionamentos no buffer da rede (Bufferbloat)
O Bufferbloat ocorre quando os o buffer da rede se enchem por causa da baixa taxa de transferência do servidor (o servidor não consegue ler os pacotes tão rápido,) com isso, a rede não tem mais espaço para "guardar os pacotes" e os pacotes UDPs que não conseguem entrar no buffer são "perdidos."
Soluções para esse problema seriam: pedir para que os pacotes não recebidos sejam re-enviados, reduzir a velocidade de transferência do cliente, etc.
Conclusão
Manter a integridade na comunicação usando UDP nao é facil mas pode ser feito. Um exemplo seria TFPT (Trivial File Transfer Protocol)
Minha Solução
Advertência: Como tinha dito antes erros como "tamanho do arquivo enviado > tamanho do recvfrom" podem causar erros e reações inesperadas no programa
client.py
import socket
import os
import time

# 3 Inicializando client
print("Configurando cliente UDP")
HOST = '127.0.0.1'
PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
ADDRESS = (HOST, PORT)
sock.connect(ADDRESS)
sock.settimeout(10.0)

# 4 Enviando arquivos ao servidor
file_paths = os.listdir()
print("Enviando nomes dos arquivos...")
for file in file_paths:
    sock.sendall(file.encode())

sock.sendall("stop".encode())

# 7 Esperando resposta do servidor sobre qual arquivo enviar
print("Esperando resposta do servidor...")
msg = sock.recv(4)
index = int.from_bytes(msg, "little")

# 8 Abrindo arquivo para ser enviado

file = open(file_paths[index], "rb")
file_size = os.path.getsize(file_paths[index])  # Size in bits
pacote_em_kilobytes = 512
pacote_em_bytes = pacote_em_kilobytes * 8

# 9 Enviando o numero de pacotes ao servidor
numero_de_pacotes = (file_size // pacote_em_bytes) + 1
sock.sendall(numero_de_pacotes.to_bytes(4, "little"))

# 11 Enviando pacotes
delay = 0.004
tempo_estimado = numero_de_pacotes*(delay*1.2)

print(f"Enviando {numero_de_pacotes} pacotes ao servidor")
print(f"Tempo estimado: {round(tempo_estimado)} sec")

for i in range(numero_de_pacotes):
    packet = file.read(pacote_em_bytes)
    sock.sendall(packet)
    enviado = f"{int((i+1)*pacote_em_kilobytes)}/{int(pacote_em_kilobytes*numero_de_pacotes)}Kb"
    print('\r'+enviado, end='')
    time.sleep(delay)

# Limpando buffers e sockets
sock.close()
file.close()

server.py
import socket
import time

# 1 Server Setup
print("Configurando servidor UDP")
HOST = '127.0.0.1'
PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((HOST, PORT))
sock.settimeout(10.0)

# 2/5 Esperando receber lista de arquivos do cliente
print("Recebendo lista de arquivos\n")
files = []
while True:
    data, address = sock.recvfrom(512)
    if data.decode(errors="ignore") == "stop":
        break
    file_name = data.decode()
    print(f"[{len(files)}] {file_name}")
    files.append(file_name)

# 6 Enviando ao cliente qual arquivo baixar
file_choice = int(input("\nQual arquivos receber?"))
while not (0 <= file_choice < len(files)):
    print("Opcao invalida!")
    file_choice = int(input("Qual arquivos receber? "))

sock.sendto(file_choice.to_bytes(4, "little"), address)

# 10 Recebendo numero de pacotes
# Queremos saber em quantos pacotes o arquivo sera mandado
data = sock.recv(4)
numero_de_pacotes = int.from_bytes(data, "little")

# 12 Recebendo pacotes
sock.settimeout(5.0)
file = open(files[file_choice], "wb")
pacote_em_kilobytes = 512
pacote_em_bytes = pacote_em_kilobytes * 8

print(f"Recebendo {numero_de_pacotes} pacotes...")
start = time.time()
for i in range(numero_de_pacotes):
    data = sock.recv(pacote_em_bytes)
    file.write(data)
    porcentagem = f"Baixando... {round((100*(i+1))/numero_de_pacotes, 2)}%"
    # print(porcentagem)
    print('\r'+porcentagem, end='')

tempo_de_download = round(time.time()-start, 2)
print(f"\nO download foi completo em {tempo_de_download} sec")

# Limpando buffers e sockets
file.close()
sock.close()


RaphaelJS animation loop

Question: I need help with adding a loop to my raphaelJS animation. The idea is to hit the button id="1" to rotate the purple square 120degrees. If you hit button id="2", the purple square will rotate 120degrees to the left instead.
I've accomplished to add the function which they both rotates, though the problem is that you can only click the order of, button id="1", button ID="2", 1, 2, , 1, 2, etc. To clarify, you can't rotate the blue square twice the same direction, you'll have to alternate between button 1/2. Second, you can't start with rotate left function. To be able to use rotate left you'll first have to rotate it to the right.
So what I need help with:
Be able to use the function multiple times.
To summarize the function:
By hitting the button, the purple square transforms by rotating 120degrees to the right.The animation timer is set to happen over 2 seconds. Adding a bounce effect to it.
Here is the rotate right function. The rotate left function is exactly the same except the transform part.
And then I tried to solve this by adding ".repeat(Infinity)", but it had no effect.
//Turn right
    p.rect(10, 80, 50, 25)
    .attr({
        fill : "grey",
        "stroke-width" : 1
    }).click(function(){
    purpleRect.animate({
        transform : "r120",
    }, 2000, "bounce").repeat(Infinity);
});
};

The purple square itself:
window.onload = function(){

    var p = Raphael(0, 0, 240, 140);
    p.rect(0,0,240,70);

    var purpleRect = p.rect(10, 10, 50, 50)
    .attr({
        fill : "#aaaaff",
        "stroke-width" : 3
    });

Comment: Have you tried making it so that if you click button1 it rotates it by 120 degrees then if you click it again it rotates it by 240 degrees then 480 degrees etc..?
Comment: Could you put it on a fiddle ?
Comment: @slebetman Nopp, not sure how to do it.
Comment: @Ian I tried, but the figures does not appear in fiddle.
Comment: @Winterwind: Simple, instead of `transform : "r120"` do `transform : "r" + (angle += 120)`. You need the brace to evaluate the angle as a number instead of a string. The outer `+` with `"r"` should then convert it to a string.
Comment: @slebetman Nothing happened with your code. It looks very similar to what I was trying with : transform : "r" + (120), which worked, but then again, I can only rotate once.
Comment: I added a fiddle. Got it to work: http://jsfiddle.net/j747g5a0/5/
Answer: Somehow, you need to either store the rotation, or gather the rotation from the existing transform. Either way is possible, it may be a simpler mindset to just store the rotation in a data element. So the main bit would look like this...
Set up the initial rotation store...
SomeElement.data('rotation',0)

Then use this in your transform...
    p.rect(70, 80, 50, 25)
    .attr({
        fill : "grey",
        "stroke-width" : 1
    }).click(function a(){
        purpleRect.data('rotation', purpleRect.data('rotation') - 120);
        purpleRect.animate({
            transform : "r" + +purpleRect.data('rotation'),
    }, 2000, "bounce");
});

jsfiddle
Comment: Many thanks! It makes no sense to me why raphael JS doesn't have more tutorials? Ah well, you saved me a lot of trouble.

RaphaelJS animation loop