Has the polynomial distinct roots? How can I prove it?

Question: I want to prove that the polynomial
$$
f_p(x) = x^{2p+2} - cx^{2p} - dx^p - 1
$$
,where $c>0$ and $d>0$ are real numbers, has distinct roots. Also $p>0$ is an even integer. How can I prove that the polynomial $f_p(x)$ has distinct roots for any $c$,$d$ and $p$.
PS: There is a similar topic that How to prove that my polynomial has distinct roots?
Comment: It seems that for $p=2$, $c=1$, $d\approx 2.6107186132760393498186490083840586275$ a root of $36d^4 + 125d^3 - 133d^2 - 801d - 899$, the polynomial *does* have multiple roots at $\pm 0.81073923436477156272173034309719150582 i$ (i.e., at the roots of $(6d + 2)x^2 + (d + 9)$, which is the $\gcd(f,f')$ for such $p,c,d$
Comment: Hint: A polynomial has distinct roots if and only if it (the polynomial) and its first derivative do not have any roots in common. See if you can use it here.
Comment: Thank you for response. Of course I know this. But for this polynomial I didn't achieve. I would be grateful if you could give me the solution.
Comment: @HagenvonEitzen how did you find the $d$ value ? Have you any program or any other tool for calculation this? Further, how can I prove it theoretically?
Comment: @HagenvonEitzen Is there any multiple roots for higher $p$ values? For example $p=4,6,8, \ldots$.
Comment: @drxy I found my counterexample by just playing around with $\gcd(f,f')$, motivated by *mathguy*'s solution for real roots. I'm sure that playing around will also allow finding counterexamples (regarding complex roots) for higher $p$ values one by one - but the handling becomes impractical.
Comment: @HagenvonEitzen, how did you find the polynomial $36d^4 + 125d^3-133d^2-801d-899$? Is there any way to find a new polynomial for $p=4,6,8$.
Comment: @HagenvonEitzen, I couldn't find any multiple root for any exact $d$ values. Since $d$ is an approximate value, you can find multiple root. Have you got any solution for any exact $d$ value?
Answer: Here is a partial solution that shows there can be no multiple REAL roots. The proof doesn't work for complex numbers though (and I am not sure the result is even true in complex numbers).
$f' = (2p+2) x^{2p+1} - 2pcx^{2p-1} - pdx^{p-1}$. If $x$ is a multiple root of $f$, then both $f$ and $f'$ vanish at $x$. But $(2p+2) f - xf' = -2cx^{2p}-(p+2)dx^p-(2p+2) < 0$ for all $x \in \mathbb R$ because $p$ is even and $c$, $d$ and $p$ are all positive.
Comment: I don't know what you mean by "mathematical way" - what I gave you is a complete, "mathematical way" proof that the polynomial does not have multiple roots that are real numbers. For complex numbers, I said I don't even believe the result is true (so you can't prove it, if it's not true!) and in a comment to your post, Hagen von Eltzen demonstrated that my suspicion was correct: the result is FALSE in complex numbers. Please see Hagen's comment to your question.
Comment: Can you read what I said? In complex numbers the result is FALSE, how do you want to "prove" something that is not true? Hagen gave you a counter-example!
Comment: Thank you to answer. But I must prove it mathematical way. how can I prove it for complex numbers?
Comment: Mathematical way means that "I must prove it completely" both real and complex numbers as in this topic http://math.stackexchange.com/questions/1740673/how-to-prove-that-my-polynomial-has-distinct-roots?lq=1. Your solution includes real numbers. Besides, it's so important for me that the proof must include the complex numbers. Thank you so much.
Comment: Sorry my bad English. Thank you for all answers.  Have you got any idea for $p=4,6,8$ or higher values. How can I be sure that the polynomial has multiple roots for any $p$ value.
Answer: 'All' you have to do is computing the g.c.d. of $f_p(x)$ and $f'_p(x)$ via the Euclidean algorithm. A polynomial has only simple roots (in $\mathbf C$) if and only if this polynomial and its derivative are coprime.
Comment: This should be the best and most familiar way to go. Moreover, you should expect the two polynomials to be relatively prime. I.e you should expect the gcd to be a constant.
Comment: @Bernard Since I couldn't achieve it, I'm writing here. Thank you very much for valuable answer.
Comment: Not very helpful though, eh? HOW do you compute the gcd? Note that the problem is subtle; as shown by Hagen, the result is false in general, but as I have shown, it is true about real roots; so the proof definitely is not based on gcd alone.


Manual calculation of McKelvey & Zavoina Pseudo $R^2.$

Question: I would like to calculate McKelvey & Zavoina pseudo $R^2$ manually for poisson regression. I based my calculations on the formula found here i.e.
$$ R^2 =\frac{\sigma^2(\hat y) }{\sigma^2(\hat y)+ \frac{\pi^2}3}$$
Let's consider the following model:
set.seed(42)
y <- c(1, 2, 3, 2, 1, 4, 5, 6, 7, 1, 2, 3)
x <- rnorm(length(y))
z <- runif(length(y))

mod <- glm(y ~ x + z, family = poisson)

And my calculation of pseudo $R^2$ is then:
var(predict(mod)) / (var(predict(mod)) + pi^2 / 3)
0.02075299

However, if we calculate this $R^2$ using performance library, we obtain:
performance::r2_mckelvey(mod)
0.1853505

Do you know, what the source of this discrepancy is?
Answer: To see the code behind a function, you can type the function name without () at the end. Sometimes the top level functions only do methods dispatch so you have to find out which function is called by the top-level function. But in the end, these are the relevant functions in your case:
performance:::.r2_mckelvey <- function (model) {
    faminfo <- insight::model_info(model)
    n <- insight::n_obs(model, disaggregate = TRUE)
    if (faminfo$is_binomial || faminfo$is_ordinal || faminfo$is_multinomial) {
    dist.variance <- switch(faminfo$link_function, probit = 1,
        logit = pi^2/3, clogloglink = pi^2/6, NA)
    } else if (faminfo$is_count) {
        # this line is where the residual variance is calculated,
        # in your case `.get_poisson_variance` is called (see below)
        dist.variance <- switch(faminfo$link_function, log = .get_poisson_variance(model),
            sqrt = 0.25, 0)
    }
    y.hat <- stats::predict(model, type = "link")
    yhat_columns <- ncol(y.hat)
    if (!is.null(yhat_columns) && yhat_columns > 1)
        y.hat <- as.vector(y.hat[, 1])
    # the two following lines are where the main calculations happen
    # (compare this to your code)
    dist.residual <- sum((y.hat - mean(y.hat))^2)
    mckelvey <- dist.residual/(n * dist.variance + dist.residual)
    names(mckelvey) <- "McKelvey's R2"
    mckelvey
}

performance:::.get_poisson_variance <- function (model) {
    mu <- exp(stats::coef(.null_model(model)))
    if (is.na(mu)) {
        return(0)
    }
    cvsquared <- stats::family(model)$variance(mu)/mu^2
    log1p(cvsquared)
}

As you can see, the source of the discrepancy is that this formula uses the residual sum of squares and computes the variance in a different way (log-normal approximation; second function).
Edit 08-08-2023: statistical reasoning
Following McKelvey and Zavoina (1975):
$$R² = \frac{S_E^2}{\hat{S_T^2}}\tag{eq. 3.21}$$
$S_E^2$ being the residual sum of squares and $\hat{S_T^2}$ the total sum of squares defined as
$$\hat{S_T^2} = S_E^2 + \hat{S_R^2} = S_E^2 + n \sigma^2 \tag{eq. 3.20 and 3.18}$$
(not assuming unit variance), which is exactly in line with the above code (mckelvey <- dist.residual/(n * dist.variance + dist.residual)). Then, as explained in Nagakawa et al. (2017), for Poisson with log-link $\sigma^2$ can be estimated by the lognormal approximation, as:
$$\sigma^2 = ln(1 + \frac{1}{\lambda}) \tag{tbl. 1 and §5}$$
with $\lambda$ approximated from $exp(β0)$ from the intercept-only model, which translates to exp(stats::coef(.null_model(model))) (note that mu in the code is $\lambda$ in Nagakawa's equation). Since with the Poisson model family stats::family(model)$variance(mu) is equal to mu, log1p(stats::family(model)$variance(mu)/mu^2) is equivalent to log(1 + 1/mu).
Comment: It's good advice to look at the code itself to see what calculations are being performed, but I have to wonder about the statistical reasoning behind doing these calculations.
Comment: @Dave, I have edited the answer to show where the code comes from
Answer: At least according to the documentation for the R package performance, that formula for McKelvey and Zavoina pseudo r-squared is only for binomial logistic regression with the logit link. ( rdrr.io/cran/performance/man/r2_mckelvey.html ).
I don't have access to the original paper by McKelvey and Zavoina to see what formula they use for Poisson regression models.
Also note that the webpage you cited is titled "Psuedo r-squared for logistic regression".
Comment: This doesn't seem to quite answer the question. Probably worth explicitly clarifying that $\pi^2/3$ in the formula is the logistic distribution-specific variance, and so inappropriate for the Poisson here. The documentation you link says "*for poisson-models, the residual variance [used to calculate R2] is based on log-normal approximation, similar to the distribution-specific variance*", citing [Nakagawa, Johnson & Schielzeth (2017)](https://doi.org/10.1098/rsif.2017.0213). Although a bit unclear, it seems as though it generalises the MZ formula with this approximation.


How to handle mixed-models with low sample size as well as low frequency of categorical level of interest?

Question: Overview
I am working on analyzing data from an observational study, details of which is as follows: 18 subjects with type 1 diabetes (T1D) + 14 healthy controls were recruited for a four-week long naturalistic driving study. Throughout the study period, subjects and their vehicles were constantly monitored to collect information on their blood glucose (BG) levels (only applies to T1D subjects) and their driving behavior. My part of the analysis is to focus on how acute (in-vehicle) glycemic episodes (details below) affect driver behavior at stop-controlled intersections.

Response variable (0/1): Stopping response is coded as 1 when a driver makes an unsafe stopping maneuver (rolling stop or no stop) and 0 when a driver makes a safe stopping maneuver (full stop)
Control variables: lead vehicle status (three levels: none present, present without effect, and present with effect), crossing vehicle status (three levels: none present, present without effect, and present with effect), and intersection type (minor-road-only or all-way stop)
Explanatory variable of interest: Glycemic episode of the driver during the stop sign encounters. Glycemic episode is computed from the blood glucose readings and is coded as a factor variable with four levels: hypo (BG is below a certain threshold for an extended period of time), hyper (BG is above a certain threshold for an extended period of time), normal (BG is within bounds), and control (for healthy controls). Clinical literature has shown that hyper episodes do not have a drastic effect on T1D patients in terms of cognition and other motor functions that are required for safe driving, but hypo episodes do (but they are infrequent). So the primary interest in the analysis is to see if hypo episodes in particular make drivers more risky at stop intersections.

Response variable and control variables were extracted through manual video review (watching video clips of drivers navigating through stop intersections). Also, whether a lead vehicle (the vehicle the subject is following in the same lane when approaching the intersection) or a crossing vehicle (the vehicle that crosses subject driver's path: think of North-Bound/West-Bound type situation at a 4-way stop) had an effect on subject driver's stopping behavior was based on human judgement. Data on glycemic episodes were made available to me.
Modeling approach
Since each subject gave multiple responses at stop-controlled intersections, I decided to use the mixed-effects logistic regression model using lme4::glmer() function in R. Additionally, I considered fitting models to three different subsets of the data (say subset1, subset2, and subset3): (a) stopping responses from all subjects (T1D + Control), (b) stopping responses from just T1D subjects, and (c) stopping responses from just T1D subjects who had both hypo and normal episodes. This data partitioning was done for overall data (say data1) that included cases where a lead vehicle or a crossing had an impact on subject driver's behavior and for the data obtained after omitting such cases (say data2). So, I tried fitting models to 6 variants of the data: data1-subset1, data1-subset2, ..., data2-subset3. Furthermore, I tested for both the subject and the intersection (same intersection was visited multiple times, partially-crossed) random effects terms in the model. So, overall I tried the following 12 models:
mod1 <- unsafe_response ~ glycemic_episode + control_variables + (1|subj), data = data1-subset1
mod2 <- unsafe_response ~ glycemic_episode + control_variables + (1|subj) + (1|intersection), data = data1-subset1
mod3 <- unsafe_response ~ glycemic_episode + control_variables + (1|subj), data = data1-subset2
mod4 <- unsafe_response ~ glycemic_episode + control_variables + (1|subj) + (1|intersection), data = data1-subset2
mod5 <- unsafe_response ~ glycemic_episode + control_variables + (1|subj), data = data1-subset3
mod6 <- unsafe_response ~ glycemic_episode + control_variables + (1|subj) + (1|intersection), data = data1-subset3

I repeated the same six model structure by removing the control variables and using data2-subset* data variants, e.g., unsafe_response ~ glycemic episode + (1|subj), data = data2-subset1.
Here's the descriptive summary statistics for the model data frames:
data1-subset*:

data2-subset*: (Notice records where lead vehicle and/or crossing vehicles had an effect on subject driver's behavior are omitted and control variables are no longer used when modeling using this data)

Problem
Unfortunately, none of the models that I tested resulted in glycemic episodes being a significant predictor in explaining the stopping responses. In my opinion, this may be due to the low sample size (I cannot increase it because it includes the entire data collection and these are all the stop sign encounters that I found) and low frequency of hypo glycemic episodes (can't increase this frequency either). What other modeling approaches can I use for this data besides mixed-effects modeling? Say after doing all this, the effects of glycemic levels did not turn out to be significant. Will this be a defensible analysis for a paper? (I have drafted the paper, but am not sure if I will face problems publishing it because the effects of interest are non-significant)
Note: It's a novel study and I haven't seen any paper where researchers connected clinical data and naturalistic driving data to model driver behavior. So, can these non-significant finding be reported as a pilot test results?
Comment: Interesting(+1) ! Did you do a power analysis / sample size calculation at the beginning ?
Comment: Also, I came across your answer (https://stats.stackexchange.com/a/48634/274692) on sample size calculation, so I am going to try that now.
Comment: @RobertLong, actually I did not because I extracted all possible stop sign encounters from all subjects during the study period. But it would be interesting to calculate the sample size to see if the minimum sample size is more than what I have in the data set. Thanks for the suggestion!
Comment: OK fair enough, but you should read up on the problem of "post hoc power analysis"
Comment: @RobertLong, thanks for the pointer, I will take a look.
Answer: Statistically insignificant results are not a reason to not publish. Unfortunately many people make this mistake (including reviewers) and that is a big part of the reason for publication bias. Perhaps the effects sizes are interesting in themselves, but if not and if these hypoglycaemic episodes do not actually affect driving behaviour then that is a good thing for safety, is it not ? On the other hand if the study and analysis is severly under-powered then that is a serious problem.
So I think you can look at several things here.

Mixed effects models are a good approach to the repeated measures problem. From your explanation you need to include random intercepts for both participant and intersection.

Subsetting the data will cause a big reduction in statistical power.  I don't think you need to do this. Retain the variables that you used to split/subset on fixed effects.

Do you have access to the underlying glucose readings ? If include these rather than the categorized version, this will increase statistical power - and you can also allow for nonlinear effects.

A different approach to mixed effects model is a generalized estimating equations model (GEE). However I doubt there would be much difference in power and the above considerations still apply.
Comment: That question is about the minimum cluster size. See [here](https://stats.stackexchange.com/questions/388937/minimum-sample-size-per-cluster-in-a-random-effect-model)
Comment: Robert, is [this lme4 question](https://stats.stackexchange.com/questions/478324/multilevel-predicted-values-dont-shrink-compared-to-their-non-multilevel-peers) possibly of interest to you?
Comment: Yes, thanks, I'll take a look soon :)
Comment: I am also of the view that we should report on whatever we find in the study and not forcefully try to make some effects significant even if they are not! But the general consensus among the researchers around me is that studies with statistically insignificant findings are much harder to publish and I don't understand why!
Comment: Similar to sample size, is there a way to compute the minimum number of records required per level of a categorical variable of interest? So in this analysis, I am short on the hypo side because it's infrequent. Is there a way I can find the minimum number of stop sign encounters required under hypo episodes to perform the analysis with significant power?


coreML model converted from pytorch model giving the wrong prediction probabilities

Question: I have a pytorch binary classification model that I converted to coreML. I converted my model directly and indirectly through onnx using the following tutorials/documentation respectively
https://coremltools.readme.io/docs/pytorch-conversion, and
https://github.com/onnx/onnx-docker/blob/master/onnx-ecosystem/inference_demos/resnet50_modelzoo_onnxruntime_inference.ipynb .
The output prior to the softmax function and the probabilities are similar for both the original pytorch and the onnx model converted from PyTorch.  But the output for the coreML model converted from PyTorch via the tutorial documentation is completely incorrect. I had no errors compiling the coreML method from either method.
Checking the weights of the last layer for coreML and Pytorch seem to be the same. the output of the coreML model prior to softmax gives me
{'classLabel': '_xx', 'classLabelProbs': {'_xx': 29.15625, 'xx': -22.53125}}
while the output from the pytorch model give me  [-3.2185674 3.4477997]
The output of the conversion from onnx to coreML looks like...
58/69: Converting Node Type Add
59/69: Converting Node Type Relu
60/69: Converting Node Type Conv
61/69: Converting Node Type BatchNormalization
62/69: Converting Node Type Relu
63/69: Converting Node Type Conv
64/69: Converting Node Type BatchNormalization
65/69: Converting Node Type Add
66/69: Converting Node Type Relu
67/69: Converting Node Type GlobalAveragePool
68/69: Converting Node Type Flatten
69/69: Converting Node Type Gemm
Translation to CoreML spec completed. Now compiling the CoreML model.
Model Compilation done.

While the output of the pytorch model when I print looks like this for the final layer....
(layer4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
(fc): Linear(in_features=512, out_features=2, bias=True).

How do I go about resolving the quantitative errors produced from my coreML model that was converted from PyTorch?
Answer: It's probably an issue with your image preprocessing options: https://machinethink.net/blog/help-core-ml-gives-wrong-output/
Comment: I started my model with a pretrained resnet18 model and I trained it with my data. I used the same preprocessing on the image before passing it to the model for both my pytorch model and my converted coreml model. That should have taken care of what the article is talking about. Right?
Comment: Yes, if you used the same preprocessing options for the Core ML model, then that shouldn't be issue. But note that your output is ~10x larger than the PyTorch output, which usually means the scale of the input is already wrong. Try converting a model with just one layer to see if that works OK.
Answer: update:
using the coreml unified api I have added a scaled layer. My outputs are not giving any probabilities for my classifier.
![last couple layers of converted pytorch model][1]
[1]: https://i.stack.imgur.com/9bzd2.png
the last layer prints out a tensor instead of probabilities. So I added a softmax function via the network.builder
builder.add_softmax(name="softmax", input_name="305", output_name="307:labelProbabilityLayerName")
the previous last node had the output name equal to "307:labelProbabilityLayerName" and I changed it to "305", prior to me adding the softmax(). this way the previous last node's output is the input to my softmax. Also, now the output can be passed to my softmax can now connect to the original string class printing out the intended probabilites.
I am still getting an error saying...
"RuntimeError: Error compiling model: "Error reading protobuf spec. validator error: Layer 'softmax' consumes an input named '307' which is not present in this network."."
Which doesnt make sense because I defined my softmax to consume '305' and also updated that last layer which is an innerproduct layer to output 305.


Junit Mock Repeated Feign Client Calls but Only the Last Mock Returned Multiple Times

Question: I have a Spring Boot application which uses Feign Client to call a microservice to add users to the User table when a new department is created (new department will be inserted into the Department table).  The request looks like:
Request:
<code> {
  "department": "math",
  "usernameList": ["aaa", "bbb", "ccc"]
 }
</code>
The User model:
<code> public class User {
    private String username;
 }
</code>
The Feign client:
<code> import org.springframework.cloud.openfeign.FeignClient;
 @FeignClient(name = "user-client", url = "/.../user", configuration = UserConfiguration.class)

 public interface UserClient {

     @RequestMapping(method = RequestMethod.POST, value = "users")
     User createUser(User user);
 }
</code>
UserService:
<code>@Service
public class UserService {

private final UserClient userClient;

public UserResponse createUser(@Valid Request request);

     List<User> userList = request.getUsernameList()
           .stream()
           .map(username -> userClient.createUser(mapToUser(username))
           .collect(Collectors.toList());
 ......
}
</code>
The above code worked and I was able to add 3 users into the database.  The userList has 3 correct username. However, when I ran the junit test below, it seemed that only the last userResp ("ccc") was returned 3 times as mock response.  When I ran the junit test in debug mode, I saw that each time the thenReturn(userResp) had the correct userResp, but in the UserService, the userList ended up containing 3 "ccc", rather than a list of "aaa, bbb, ccc". I tried using the FOR loop in the UserService rather than the stream, the result was the same, so it wasn't because of the stream.  I also tried to remove the FOR loop in the Junit and just called the mock 3 times, same result.  I am not sure if this has something to do with the Feign client mocking or if I did something wrong in my test case.  Can someone please help?
My Junit:
<code> public class UserTest {

    @MockBean
    private UserClient userClient;

    @Test
    public void testAddUser() throws Exception {

        for (int i=1; i<=3; i++) {

             User userResp = new User();

              if (i==1) {
             userResp.setUsername("aaa");
            // mock response
       Mockito.when(userClient.createUser(ArgumentMatchers.any(User.class)))
         .thenReturn(userResp);
           }
           if (i==2) {
             userResp.setUsername("bbb");
            // mock response
       Mockito.when(userClient.createUser(ArgumentMatchers.any(User.class)))
         .thenReturn(userResp);
           }
        if (i==3) {
             userResp.setUsername("ccc");
            // mock response
       Mockito.when(userClient.createUser(ArgumentMatchers.any(User.class)))
         .thenReturn(userResp);
           }
       }

       // invoke the real url
       MvcResult result = mockMvc.perform(post("/users")
            .content(TestUtils.toJson(userRequest, false))
            .contentType(contentType))
            .andDo(print())
            .andExpect(status().isCreated())
            .andReturn();
    }
</code>
Comment: Its returning "ccc" in every case because you are mocking the same call every time Mockito.when(userClient.createUser(ArgumentMatchers.any(User.class)))
         .thenReturn(userResp); and since "ccc" is the last mock its returning "ccc"
Comment: when i==1, the userResp has "aaa", shouldn't it return "aaa" the first time?  Or Junit treated 3 calls as 1 call because it was the same mock call?
Comment: No, because when your actual service call is made the value of userResp is set to "ccc"
Comment: I see. So it's not like when the service calls createUser each time, the mock gets called.  All mocks were executed before the service call was invoked....
Comment: exactly, I have suggested a way that can help you with your case
Answer: To make the method return different values for the subsequent call you can use
<code>Mockito.when(userClient.createUser(ArgumentMatchers.any(User.class)))
            .thenReturn("aaa")
            .thenReturn("bbb")
            .thenReturn("ccc"); //any

// Or a bit shorter with varargs:

    Mockito.when(userClient.createUser(ArgumentMatchers.any(User.class)))
        .thenReturn("aaa", "bbb", "ccc"); //any
</code>
Comment: is this really possible to use? mockClient returns real object, compiler does not allow you to return a string instead, does it? When you use mock client, you are mocking the real response (string) and let the client to decode it to object.. But when you mock with `Mockito.when(..)` you return the clients return object, and bypass the decoding process. Is there a way to let mockClient return different results in subsequential call using `mockClient.ok(..)` method?


Problema com MySQL ao testar o POST de uma API node.js

Question: Estou aprendendo a construir uma API e comecei a integrar o banco de dados usando o MySQL com Workbench, porém hora que tento fazer um teste de POST pelo Postman recebo o seguinte erro:
C:\API\routes\produtos.js:16
conn.query(
^

TypeError: Cannot read properties of undefined (reading 'query')
at C:\API\routes\produtos.js:16:14
at C:\API\node_modules\mysql2\lib\pool.js:60:18
at PoolConnection. (C:\API\node_modules\mysql2\lib\connection.js:775:13)
at Object.onceWrapper (node:events:646:26)
at PoolConnection.emit (node:events:538:35)
at PoolConnection._notifyError (C:\API\node_modules\mysql2\lib\connection.js:236:12)
at ClientHandshake. (C:\API\node_modules\mysql2\lib\connection.js:125:14)
at ClientHandshake.emit (node:events:538:35)
at ClientHandshake.execute (C:\API\node_modules\mysql2\lib\commands\command.js:39:14)
at PoolConnection.handlePacket (C:\API\node_modules\mysql2\lib\connection.js:456:32)
[nodemon] app crashed - waiting for file changes before starting...

O que pode ser este erro? O que faço para resolver? Estou iniciando na programação.
Answer: O erro basicamente diz que você está tentando chamar o método query a partir de um valor undefined, ou seja, a variável conn está com o valor de undefined, seria tipo: undefined.query().
Verifica o código de como você está definindo um valor para conn que tu vai resolver o erro de TypeError: Cannot read properties of undefined (reading 'query').


Bounding the integral of $\ln(1+e^{-x})$ from $0$ to $n$ using given double inequality

Question: Let $f(x) := \ln(1+e^{-x})$ and
$$u_n := \sum_{k=1}^n \frac{(-1)^{k-1}}{k^2}.$$
Given that for $x>0$
$$ \tag{1} \sum_{k=1}^n \frac{(-1)^{k-1}}{k}e^{-kx} - \frac{e^{-(n+1)x}}{n+1} \le f(x) \le \sum_{k=1}^n \frac{(-1)^{k-1}}{k}e^{-kx} + \frac{e^{-(n+1)x}}{n+1} $$
it is asked to prove that
$$u_n - \dfrac{1}{(n+1)^2} \le \int_0^n f(x) dx \le u_n + \frac{1}{(n+1)^2}$$
I tried to integrate the first double-inequality  form $0$ to $n$ but I get extra terms of unknown sign. For the left side for instance I get
$$ \tag{2} \left( u_n - \dfrac{1}{(n+1)^2} \right) + \left(\dfrac{e^{-(n+1)n}}{(n+1)^2} -
\displaystyle\sum_{k=1}^n \dfrac{(-1)^{k-1}}{k^2} e^{-nk} \right) \le \displaystyle\int_0^n f(x) dx $$
I tried to prove that the second term is positive (sufficient condition) but in vain.
thanks for any advice.
Comment: is it to be proven for every $n\in \mathbb{N}$ or just definitively?
Comment: The inequality (1) seems to be not suitable, since the second term in (2) can be negative. Let $v_n := \left(\dfrac{e^{-(n+1)n}}{(n+1)^2} -
\displaystyle\sum_{k=1}^n \dfrac{(-1)^{k-1}}{k^2} e^{-nk} \right)$ and check that $v_n$ is negative for $n=2$.
Comment: for every nonzero natural $n$
Answer: For the right part: integrate from $0$ to $n$ the inequality you've been given:
$$
\begin{split} \int_0^n f(x)dx &\le \int_0^n \left( \displaystyle\sum_{k=1}^n \dfrac{(-1)^{k-1}}{k}e^{-kx} + \dfrac{e^{-(n+1)x}}{n+1} \right)dx \\
& = \sum_{k=1}^n \dfrac{(-1)^{k-1}}{k} \int_0^n e^{-kx}dx + \dfrac{1}{n+1} \int_0^n e^{-(n+1)x}dx \\
& = \sum_{k=1}^n \dfrac{(-1)^{k-1}}{k^2} \left[1-e^{-nk} \right] + \dfrac{1}{(n+1)^2}[1-e^{-(n+1)n}] \\
& \le \sum_{k=1}^n \dfrac{(-1)^{k-1}}{k^2} + \dfrac{1}{(n+1)^2} \\
& = u_n + \dfrac{1}{(n+1)^2}
\end{split}$$
I've not tried the left part but I suppose it should be done similarly
Comment: Since the sum is alternating, one can not just use that $1-e^{-nk} \leq 1$. However, note that $\int_0^n f(x) \, dx \leq \int_0^\infty f(x) \, dx = u_n + \frac{1}{(n+1)^2}$. To show the left-part, this argument does not apply.
Comment: You not use that fact but that $\sum_{k=1}^n \frac{(-1)^k}{k}e^{nk} <0$ for all $n \in \mathbb{n}$
Comment: Yes, I have meant $\leq$ in place of an equality-sign. Yes, since $\sum_{k=1}^n (-1)^k k^{-2} e^{-nk} < 0$ for all $n \in \mathbb{N}$, the right-hand side follows. However, the left-hand side inequality cannot be shown in this manner.
Comment: You mean $\sum_{k=1}^n \frac{(-1)^k}{k} e^{-nk}$? Because $\sum_{k=1}^n \frac{(-1)^k}{k} e^{nk}$ is alternating in sign.
Comment: I meant $\sum_{k=1}^n \frac{(-1)^k}{k^2} e^{-nk} <0 $. You can get it in the third line. I just noticed the error of my previous comment. bad typo.
Answer: It is possible to give a closed formula for the integral. We have
\begin{align}
\int_0^n \ln(1+e^{-x}) dx = \int_{-n}^0 \ln(1+e^x) dx = -\left. \mathrm{Li}_2(-e^{x}) \right|_{x=-n}^0 &= \mathrm{Li}_2(-e^{-n}) - \mathrm{Li}_2(-1) \\& = \mathrm{Li}_2(-e^{-n}) + \frac{\pi^2}{12},
\end{align}
since it is well-known that $$\mathrm{Li}_2(-1) = -\eta(2) = - \zeta(2)/2 =- \pi^2/12,$$ where $\eta$ denotes the Dirichlet eta function. Now, one can easily do some numerical checks: Looking at
$$E(n):=  \mathrm{Li}_2(-e^{-n}) + \frac{\pi^2}{12} - \Big( u_n - \frac{1}{(n+1)^2} \Big)$$
one get that $E(1) < 0$ and $E(3) < 0$. Thus, at least for $n=1$ and $n=3$ the inequality is wrong.

Conjecture: For $n>3$ the inequality $$\int_0^n \ln(1+e^{-x}) \, dx \geq u_n - \frac{1}{(n+1)^2}$$ seems to be true.

However, the Taylor-type inequality (1) is not suitable for the verification of this conjecture. If we define
$$a_n:= \sum_{k=1}^n \frac{(−1)^{k}}{k^2} e^{-nk},$$
then one has the alternating series estimate
$$|a_n - \mathrm{Li}_2(-e^{-n})| \leq \frac{e^{-n(n+1)}}{(n+1)^2}.$$
Additionally, one has
$$\mathrm{Li}_2(-e^{-n}) = - \int_0^{e^{-n}} \frac{\ln(1+t)}{t} dt.$$
Because of $|t - \ln(1+t)| \leq |t|/2$ for $|t| \leq 1/2$, we see that
$$-e^{-n} \leq \mathrm{Li}_2(-e^{-n}) \leq - \frac{e^{-n}}{2}$$
for all $n \in \mathbb{N}$. Thus, we may conclude that
$$ \frac{e^{-n(n+1)}}{(n+1)^2} - \sum_{k=1}^n \frac{(-1)^{k-1}}{k^2} e^{-nk} =\frac{e^{-n(n+1)}}{(n+1)^2} +a_n \leq 2 \frac{e^{-n(n+1)}}{(n+1)^2} -\frac{e^{-n}}{2} \leq - \frac{(1-e^{-2})}{2} e^{-n}.$$
This shows that the second term in (2) is always negative!

Let us prove that $\int_0^n \ln(1+e^{-x}) \, dx \geq u_n - (n+1)^{-2}$  for all $n>3$.

Starting with the formula
$$\int_0^n \ln(1+e^{-x}) \, dx = \mathrm{Li}_2(-e^{-n}) + \frac{\pi^2}{12} = \mathrm{Li}_2(-e^{-n}) + u_n + \sum_{k=n+1}^\infty \frac{(-1)^{k-1}}{k^2}$$
we have to show that
$$\mathrm{Li}_2(-e^{-n})+ \sum_{k=n+1}^\infty \frac{(-1)^{k-1}}{k^2} \geq - \frac{1}{(n+1)^2}$$
for $n>3$. If $n$ is even, then the sum is positive and $\mathrm{Li}_2(-e^{-n}) \geq - e^{-n}$, as we have already seen in the previous step. Since $e^{-n} \geq - (n+1)^2$, if $n>3$, the inequality follows. If $n$ is odd, then
$$\mathrm{Li}_2(-e^{-n})+ \sum_{k=n+1}^\infty \frac{(-1)^{k-1}}{k^2} = - \frac{1}{(n+1)^2} + \mathrm{Li}_2(-e^{-n}) + \sum_{k=n+2}^\infty \frac{(-1)^{k-1}}{k^2}$$
and hence we need to verify that
$$\mathrm{Li}_2(-e^{-n}) + \sum_{k=n+2}^\infty \frac{(-1)^{k-1}}{k^2} \geq 0.$$
Here we may use the estimate
$$\sum_{k=n+2}^\infty \frac{(-1)^{k-1}}{k^2} \ge \frac{1}{(n+2)^2} - \frac{1}{(n+3)^2}$$
and note that
$$\mathrm{Li}_2(-e^{-n}) + \frac{1}{(n+2)^2} - \frac{1}{(n+4)^2} \geq - e^{-n} + \frac{1}{(n+2)^2} - \frac{1}{(n+3)^2} \geq 0$$
holds for $n \geq 6$. One case here is missing, namely $n=5$. Here one need to use the more precise estimate
$$\sum_{k=n+2}^\infty \frac{(-1)^{k-1}}{k^2} \ge \frac{1}{(n+2)^2} - \frac{1}{(n+3)^2} + \frac{1}{(n+4)^2} - \frac{1}{(n+5)^2}.$$

All in all, we conclude that $E(1)<0$, $E(2)>0$, $E(3)<0$ and $E(n)>0$ for $n>3$. Thus, the left-hand side of the inequality is valid with two exception, namely if $n=1$ or $n=3$.
Comment: Yes, your argument is correct. The same argument was already used by Gabrielek in his answer. :-)
Comment: I have added a complete answer showing that the lower bound is valid as well if $n>3$.
Comment: Is my approach correct for proving the right inequality :

The RHS is

$$\left( u_n + \dfrac{1}{(n+1)^2} \right) + R_n $$

Where

$$R_n := \displaystyle\sum_{k=1}^n \dfrac{(-1)^k}{k^2}e^{-kn} - \dfrac{e^{-n(n+1)}}{(n+1)^2} $$

Let $\alpha_k := \dfrac{(-1)^k}{k^2}e^{-kn}$

We have

$$\alpha_k + \alpha_{k+1} = e^{-kn}(-1)^k\left( \dfrac{1}{k^2} - \dfrac{e^{-n}}{(k+1)^2}\right)$$

So that this expression is negative for odd $k$.

Hence, for even $n$, we have

RHS $\le u_n + \dfrac{1}{(n+1)^2} $

for odd $n$ one term of the sum remains but negative so the result still
Comment: great work sir, thanks a lot


How to allow access to certain computers?

Question: Is there any way I could provide access depending on the computer name . How do I specify this condition in the configuration ?
Also, is there any way to grant access depending on user id (environment variable) . I used to do this using VBA . However, since this is native microsoft product i had to simply sprecify environ . What about tomcat ?
I read this article  but did not quite get it . I presume I need to use Remote Address Filter but I need to have a example code .
Comment: Check this question: http://stackoverflow.com/questions/1839618/tomcat-valve-settings
Answer: First Point, Filtering requests based on the DNS hostname is "expensive". It forces tomcat to do a reverse DNS lookup. When the request comes up it does not comes with a DNS name, but just the IP address of client. So tomcat has to ask the system for name (or names) that corresponds to client's IP address.
If you want to accept request from specific host name, you might already know the IP address (or range of IP addresses) that correspond to the hostname. If so then you can use Remote Address Filter to set the required value (IP addresses) to it.
Second Point, Using Remote Address Filter gives a flexibility with two attributes, allow and deny.
Example:
NOTE: The IP addresses used here are just to show as example.
Suppose there is a hostname myhostname with IP address range 188.8.131.52 - 18.104.22.168.
You want to allow only this range and deny all others.
You would do a DNS lookup for this range. Say If you find it as 220.127.116.11.
Then you would configure your Valve as:
 <Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="164\.32\.90\.100, 134\.87\.13\.\d{1,3}" />

Now suppose that within the range, you want to deny a subrange 18.104.22.168 - 188.8.131.52.
Then you would configure your Valve as:
<Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="164\.32\.90\.100, 134\.87\.13\.\d{1,3}" deny="134\.87\.13\.(5[1-9]|60)" />

Configuring dynamic values for allow and deny is possible using regular expression.
Might these details help you to solve your issue.
Edited
One Final Point, If you allow / deny range of IP addresses, you must not deny the request from "localhost". So add 127\.0\.0\.1 to your allow range.


Find two closest points on two functions

Question: We have two functions:
$$\begin{align*}
y &= x^4-5x^3+2x^2-5 \\
y &= -11x-20
\end{align*}$$
My task is to find two closest points that can be found on these two functions.
Can somebody give any hints on how to solve these type of exercises?
Thank you!
Comment: Do you mean the closest points of the curves, or the minimum difference between the two functions?
Comment: The minimum difference between the two functions. :)
Answer: You want to minimize $x^4-5 x^3+2 x^2+11 x+15$.  Take the derivative, set that to $0$, and solve.  Unfortunately this is an irreducible cubic, so the solutions are not very
nice.  There are three real roots: approximately $-.6828742578$, $1.275428188$, $3.157446070$.  I'll leave it to you to find which of these gives the minimum.


Duvida com javascript console.log

Question: eu tenho um javascript que executa algumas ações em um site. Eu acompanho pelo console pra saber o que ta acontecendo, porem em varios momentos ele executa um comando que printa no console console.log("check");
Eu quero fazer com que esse "check" nao apareça no console, porque ele se repete o tempo todo e suja a tela. se eu simplesmente apagar console.log("check") ele para de funcionar? ou apenas para de printar no console?
Desde já obrigado.
Answer: Apenas para de printar no console. A função console.log serve apenas para printar algo no console, não faz nada além disso.
Você encontra mais informações sobre o console aqui no site mesmo: O que é console.log?

Duvida com javascript console.log


setTimeout fazendo perde valor da variável?

Question: Vou resumir o código pois o meu está muito grande, vamos lá.Eu tenho uma variável global do tipo boolean que recebe false, ao entrar em um if ela tem que receber no lugar de false o true, porém quando entra no setTimeout o booleano está retornando "undefined" em vez de retorna true. Fiz esse código para vocês entenderem mais ou menos como está ocorrendo, mas o primeiro if não pega por algum motivo.
import { Component } from '@angular/core';
import { NavController, NavParams } from 'ionic-angular';
import { Memoria3Page } from '../memoria3/memoria3';

@Component({
  selector: 'page-memoria2',
  templateUrl: 'memoria2.html',
})
export class Memoria2Page {
  verificaViradasCima: number = 0;
  cartaViradaCima1: boolean = false;
  cartaViradaCima2: boolean = false;

  cartaViradaBaixo1: boolean = false;
  cartaViradaBaixo2: boolean = false;

  constructor(public navCtrl: NavController, public navParams: NavParams) {
  }

  fase2(): void{
    this.navCtrl.setRoot(Memoria3Page);
  }
  ionViewDidLoad(){
    console.log("Memoria Fase 2");

    criaTabueliro1();
  }

  gameOver(): void{
    if(document.getElementById("virada0").style.zIndex == "998" && document.getElementById("virada1").style.zIndex == "998" &&
       document.getElementById("virada2").style.zIndex == "998" && document.getElementById("virada3").style.zIndex == "998" &&
       document.getElementById("virada4").style.zIndex == "998" && document.getElementById("virada5").style.zIndex == "998"){
        document.getElementById("nextNivelButton").style.visibility = "visible";
        document.getElementById("msgFinish").style.visibility = "visible";
    }
  }

  point(): void{
    document.getElementById("bloqueiaClick").style.zIndex = "995";
    this.verificaViradasCima =0;
    document.getElementById("imgAcerto").className = "pontuacao";
    setTimeout(function(){
      document.getElementById("imgAcerto").className = "imgAcerto";
    },1500);
    this.gameOver();
  }

  pararParaVerificar(): void{
    if(this.verificaViradasCima == 2){

      console.log(vetor[0] +" "+ vetor[1] +" "+ vetor[2] +" "+ vetor[3] +" "+ vetor[4] +" "+ vetor[5]);
      // Verifica a carta 1
      if(vetor[0] == vetor[1] && this.cartaViradaCima1 == true && this.cartaViradaCima2 == true){
        console.log("ENTROU IF");
        this.cartaViradaCima1 = false;
        this.cartaViradaCima2 = false;
        this.point();

      }

      else{
        console.log("Entrou else"+this.cartaViradaBaixo1+this.cartaViradaBaixo2+this.cartaViradaBaixo3+this.cartaViradaBaixo4+this.cartaViradaBaixo5+this.cartaViradaBaixo6);
        this.verificaViradasCima = 0;

        // Este é o if em que é recebido o this.cartaViradaBaixo1 == true e this.cartaViradaBaixo2 == true porém perde o valor ao entrar no setTimeout mais abaixo
        if(vetor[0] != vetor[1] && this.cartaViradaCima1 == true && this.cartaViradaCima2 == true){
          console.log("Entrou no else, e dps no if 1");
          this.cartaViradaCima1 = false;
          this.cartaViradaCima2 = false;
          this.cartaViradaBaixo1 = true;
          this.cartaViradaBaixo2 = true;
        }

         setTimeout(function(){
          if(vetor[0] != vetor[1] && this.cartaViradaBaixo1 == true && this.cartaViradaBaixo2 == true){
            console.log("Entrou no else setTimeOut, e dps no if 1");
            document.getElementById("bloqueiaClick").style.zIndex = "999";
            document.getElementById("virada0").style.zIndex = "996";
            document.getElementById("virada1").style.zIndex = "996";
            document.getElementById("virada0").style.visibility = "hidden";
            document.getElementById("virada1").style.visibility = "hidden";
            this.cartaViradaBaixo1 = false;
            this.cartaViradaBaixo2 = false;

          }

           document.getElementById("bloqueiaClick").style.zIndex = "995";
         },2000)
      }
    };
  };

Neste último setTimeout ao fazer ao tentar fazer a condição no IF não é feita com sucesso pois "this.cartaViradaBaixo1 == true && this.cartaViradaBaixo2 == true" não estão retornando true mas sim undefined.
OBS: Não posso utilizar jQuery;
Comment: Mas não tem nada que chame esse `if`. Esses `if`s não teriam que estar dentro da função?
Comment: No meu código original eles estão, ai foi só uma forma reduzida pra vocês tentarem entender mais ou menos como o problema está acontecendo.
Comment: Blz, mas assim não tem como reproduzir o problema. Olhando o código que vc colocou, não parece ter erro algum.
Comment: Tente colocar o código para rodar e veja se o booleano retorna true, vai ver que não por causa da setTimeout, em vez disso vai retorna undefined...
Comment: Eu editei a publicação vê se ficou melhor assim...
Answer: O seu problema é a referência ao this. Esse é um erro bastante recorrente em JavaScript. O this que você usa no callback do setTimeout não é necessariamente o mesmo this que você tem no contexto de chamada do setTimeout.
O callback do setTimeout normalmente é invocado com o valor this sendo window no browser.
Uma solução para o seu problema seria usar bind:
setTimeout(function () { /* implementação */ }.bind(this))

Answer: Além do bind citado pelo Thiago (que irá afetar o this) você pode simplesmente criar uma variavel para se referir ao this:
var $this = this;

setTimeout(function(){
  if(vetor[0] != vetor[1] && $this.cartaViradaBaixo1 == true && $this.cartaViradaBaixo2 == true){
    console.log("Entrou no else setTimeOut, e dps no if 1");
    document.getElementById("bloqueiaClick").style.zIndex = "999";
    document.getElementById("virada0").style.zIndex = "996";
    document.getElementById("virada1").style.zIndex = "996";
    document.getElementById("virada0").style.visibility = "hidden";
    document.getElementById("virada1").style.visibility = "hidden";
    $this.cartaViradaBaixo1 = false;
    $this.cartaViradaBaixo2 = false;

  }

   document.getElementById("bloqueiaClick").style.zIndex = "995";
 },2000)


How to downgrade to Passenger 2.2.15?

Question: I am using Passenger 3.0.5. For some reasons, I want to downgrade to Passenger 2.2.15. Is there any convenient way to do that?
Answer: Yes, uninstall passenger 3.0.5 using the gem uninstall command and then install version 2.2.15.
<code>gem uninstall passenger
gem install passenger -v 2.2.15
</code>
You can then proceed to install the apache or nginx modules or simply use the standalone version of passenger.
Comment: Glad I could help. Be sure to mark the question as answered :)

How to downgrade to Passenger 2.2.15?


Liberar Campo após data

Question: Ola
Preciso de um código javascript no qual terei dois campos um com a data e um  Descrição  que será desbloqueado de acordo data indicada e bloqueie o anterior
Exemplo

A Partir do dia 06/06/2018
o campo anterior é bloqueado e desbloqueado o campo seguinte
e assim sucessivamente

<!doctype html>

<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">

<title></title>

Desbloquear Campo Após Data

                      Datas
                      Prazo
                      Campo

                      Data 1

                          <input type="text" class="form-control" id="data1" mask="00/00/0000" value="05/06/2018" readonly>

                      <input class="form-control" type="text" id="descricao1" name="descricao1" placeholder="Descrição" >

                      Data 2

                          <input type="text" class="form-control" id="data2" mask="00/00/0000" value="06/06/2018" readonly>

                      <input class="form-control" type="text" id="descricao2" name="descricao2" placeholder="Descrição" readonly>

                      Data 3
                      <input type="text" class="form-control" name="data3" id="data3" value="07/06/2018" readonly>
                      <input class="form-control" type="text" id="descricao3" name="descricao3" placeholder="Descrição" readonly>

Comment: O que já sabe de JavaScript? O que tentou fazer? Por quê não deu certo? Apareceu alguma mensagem de erro? Qual? Se não, apenas gerou um resultado indesejado? Qual?
Comment: Qual a duvida, não entendi?
Comment: Preciso de um codigo js que desbloqueie o campo de acordo com a data, exemplo dia se for dia 05/06/2018 o campo estará desbloqueado, apos essa data ele bloqueará o campo. Editei o a pergunta, vê se ajuda.
Answer: Primeiro você vai precisar pegar o valor que está sendo mudado ou ser a data atual ai depende de você, quando o formulário estiver carregado (ponha isso no header  :
      $(document).ready(function(){
        $("#data1").change(function() {
         var data1 = document.getElementById("data1").value;
         // faça o seu IF aqui com a condição
         document.getElementById('descricao1').readOnly = false;
        });
       });

Liberar Campo após data


Como filtrar informações ao carregar um arquivo local?

Question: Existe alguma forma de ignorar uma determinada informação ao carregar um arquivo local no MySQL?
Exemplo: Tenho um arquivo de 2 colunas: "TELEFONE" e "VALIDADO". A coluna "VALIDADO" possui os valores 'SIM' e  'NAO'.

Como não preciso dos casos negativos queria ignorá-los para poupar espaço e diminuir o tempo de consulta na tabela.
Existe alguma maneira de carregar somente os telefones com VALIDADO = 'SIM'?
Comment: como você importa esse aquivo?
Comment: Importo via LOAD DATA LOCAL INFILE
Comment: Hi Anthony, sorry for the delay.

Yes, I did the test and managed to filter only the YES Status by doing the process you indicated.

Thanks a lot for the help!
Answer: Você pode importar todos os dados para uma tabela temporária e então copiar apenas as entradas desejadas para a tabela destino.
Exemplo assumindo uma tabela chamada contatos:
CREATE TEMPORARY TABLE contatos_temp LIKE contatos;
LOAD DATA LOCAL INFILE 'meu_arquivo' INTO TABLE contatos_temp;
INSERT INTO contatos (telefone, validado, dt_imp)
SELECT telefone, validado, dt_imp
  FROM contatos_temp
 WHERE validado = 'SIM';
-- Opcional se você pretende manter a sessão aberta
DROP TEMPORARY TABLE contatos_temp;


Active Directory: Track old password changes and expiry dates?

Question: Is there anyway to find out when the last 2 passwords changes were made for an account? I reset a account password but I want to know if it already changed hours or minutes before I reset.
OR maybe find out when it was supposed to expire?
Now that I reset it already I could only fetch the next expire date and the time I had changed it through powershell.
Answer: Unless you have your auditing policies set up to audit Success events in the User Account Management subcategory (or just Account Management events, if using legacy auditing) and you actually have Security Logs that has data stretching back far enough on your DC's, you won't be able to tell.
You'll want to search for events with Event ID 4723 (Password Changed) and 4724 (Password Reset).
Answer: Another thing to look at, before you change a password, is the PasswordLastSet attribute.
Get-ADUser $user -Properties PasswordLastSet | select PasswordLastSet

That will show you the last time the password was set, but won't provide any history.


Effect of feature scaling on accuracy

Question: I am working on image classification using Gaussian Mixture Models. I have around 34,000 features, belonging to three classes, all lying in a 23 dimensional space. I performed feature scaling on both the training and testing data using different methods, and I observed that accuracy actually reduces after performing scaling. I performed feature scaling because there was a difference of many orders  between many features. I am curious to know why this is happening, I thought that feature scaling would increase the accuracy, especially given the large differences in features.
Comment: 1) How many training samples do you have? 2) Are you doing testing by testing on data not seen during training? (In this case since you are doing GMM, you are just doing clustering.) 3) What is the accuracy B4 and AFTER your change?
Comment: I should have added this in the question itself, sorry about it. I am using the GMM to build a Bayes classifier. Well before the scaling, I was getting an accuracy of 70%, and after the scaling, the accuracy reduces to 45%.
Comment: @Raghuram Do you have native pictures? What happens if you enlarge the native picture? Accuracy should not be lost here.
Answer:
I thought that feature scaling would increase the accuracy, especially given the large differences in features.

Welcome to the real world buddy.
In general, it is quite true that you want features to be in the same "scale" so that you don't have some features "dominating" other features. This is especially so if your machine learning algorithm is inherently "geometrical" in nature. By "geometrical", I mean it treats the samples are points in a space, and relies on distances (usually Euclidean/L2 as is your case) between points in making its predictions, i.e., the spatial relationships of the points matter. GMM and SVM are algorithms of this nature.
However, feature scaling can screw things up, especially if some features are categorical/ordinal in nature, and you didn't properly preprocess them when you appended them to the rest of your features. Furthermore, depending on your feature scaling method, presence of outliers for a particular feature can also screw up the feature scaling for that  feature. For e.g., a "min/max" or "unit variance" scaling is going to be sensitive to outliers (e.g., if one of your feature encodes yearly income or cash balance and there are a few mi/billionaires in your dataset).
Also, when you experience a problem such as this, the cause may not be obvious. It does not mean you perform feature scaling, result goes bad, then feature scaling is at fault. It could be that your method was screwed up to begin with, and the result after feature scaling just happens to be more screwed up.
So what could be other cause(s) of your problem?

My guess for the most likely cause is that you have high-dimensional data and not
enough training samples. This is because your GMM is going to estimating covariance
matrices using data that is 34000 in dimension. Unless you have a
lot of data, chances are one or more of your covariance matrices
(one for each gaussian) are going to be near singular or singular.
This means the predictions from your GMM are nonsense to begin with because your gaussians "blew" up, and/or the EM algorithm just gave up after a predefined number of iterations.
Poor testing methodology. You did not have data divided into proper
training/validation/test sets, and you did not perform the testing properly. What "good" performance you have in
the beginning was not credible. This is actually very common, as the natural tendency is to test using the training data the model was fitted on and not on a validation or test set.

So what can you do?

Don't use a GMM for image categorization. Use a proper
supervised learning algorithm, especially if you have known image
categories as labels. In particular, to avoid the feature scaling
altogether, use random forest or its variants (e.g., extremely
randomized trees).
Get more training data. Unless you are classifying "simple" (i.e.,
"toy"/synthetic images) or you are classifying them into a few image
classes (e.g., <= 5. Note this is just a random small number I pulled out of the air.), you really to have a good deal of images per
class. A good starting point is to get at least a couple of hundreds
per class, or use a more sophisticated algorithm to exploit the
structure within your data to arrive at better performance.

Basically, my point is don't (just) treat machine learning field/algorithms as black boxes and a bunch of tricks which you memorize and try at random. Try to understanding the algorithm/math under the hood. That way, you'll be better able to diagnose the problem(s) you encounter.

EDIT (in response to request for clarification by @Zee)
For papers, the only one I can recall off the top of my head is A Practical Guide to Support Vector Classification by the authors of LibSVM. Examples therein show the importance of feature scaling for SVM on various datasets. E.g., consider the RBF/Gaussian kernel. This kernel uses the square L2 norm. If your features are of different scale, this will affect the value.
Also, how you represent your features matter. E.g., changing a variable that represents height from meters to cm or inches will affect algorithms such as PCA (because variance along direction for that feature has changed.) Note this is different from the "typical" scaling (e.g., min/max, Z-score etc.) in that this is a matter of representation. The person is still the same height regardless of the unit. Whereas typical feature scaling "transform" the data, which changes the "height" of the person. Prof. David Mackay, on the Amazon page of his book, Information Theory for Machine Learning, has a comment in this vein when asked why he did not include PCA in his book.
For ordinal and categorical variables, they are mentioned briefly in Bayesian Reasoning for Machine Learning, The Elements of Statistical Learning. They mention ways to encode them as features, for e.g., replacing a variable that can represent 3 categories with 3 binary variables, with one set to "1" to indicate the sample has that category. This is important for methods such as Linear Regression (or Linear Classifiers). Note this is about encoding categorical variables/features, not scaling per se, but they are part of the feature preprocessing set up, and hence useful to know. More can be found in Hal Duame III's book below.
The book A Course in Machine Learning by Hal Duame III. Search for "scaling". One of the earliest example in the book is how it affects KNN (which just uses L2 distance, which GMM, SVM etc. uses if you use the RBF/gaussian kernel). More details are given in the chapter 4, "Machine Learning in Practice". Unfortunately the images/plots are not shown in the PDF. This book has one of the nicest treatments on feature encoding and scaling, especially if you work on Natural Language Processing (NLP). E.g., see his explanation of applying the logarithm to features (i.e., log transform). That way, sums of logs become log of product of features, and "effects"/"contributions" of these features are tapered by the logarithm.
Note that all the aforementioned textbooks are freely downloadable from the above links.
Comment: @Zee I've included more details. Feature scaling is more of an "art" or "preprocessing step" in research and is mentioned sporadically in different papers, often investigated in the context for a particular applications. I don't recall offhand any theoretical research on it, although if you understand the math for the algorithms, it is not hard to see how the scale changes things. Results in literature are mostly empirical (i.e., test on datasets and report best scaling method) which makes the procedures largely dependent on the nature of the data. You'll see some in the books above.
Comment: The part on representation of features could be better explained if we simply say that the large variance of a feature is dominating other features of smaller variance. e.g. Suppose we want to classify people based on height (in metres) and weight (in kilograms). The height attribute has a low variability, ranging from 1.5m to 1.85m, whereas the weight attribute may vary from 50kg to 250kg. If the scale of the attributes are not taken into consideration, the distance measure may be dominated by differences in the weights of a person. Source: Introduction to Data Mining, Ch.5, Tan Pan-Ning
Comment: Well, I did test my data on a training set. I am using GMMs to do the image categorization because I was curious to see how it would work out. Thanks for the advice!!
Comment: This is a pretty good answer, however can you reference some papers? I am trying to find some scientific evidence of the effect of feature scaling.
Comment: @lightalchemist thanks for the refs. I am familiar with basic normalisation and standardisation but I wanted some more info. Especially examples for success uses. Thank you for your answer it is exactly what I was looking for.
Comment: Thanks for the detailed answer. Ragarding: "Furthermore, depending on your feature scaling method, presence of outliers for a particular feature can also screw up the feature scaling for that feature.", won't outliers influence performance especially for distance based methods even if the data is not normalized? I cannot see what normalization could worsen performance when there are outliers in the data.

Effect of feature scaling on accuracy


Failure adding filter to pipeline in gstreamer

Question: I'm trying to do the exercise at the end of the Basic GStreamer tutorial here: Basic Tutorial 2. The exercise is merely to add a filter to a video. The exercise says to simply add the "vertigoTV" effect, so I did that.
Here are the relevant parts to my attempt:
<code>    #include <gst/gst.h>

int main(int argc, char *argv[]) {
  GstElement *pipeline, *source, *filter, *convert, *sink;
  GstBus *bus;
  GstMessage *msg;
  GstStateChangeReturn ret;

  /* Initialize GStreamer */
  gst_init (&argc, &argv);

  /* Create the elements */
  source = gst_element_factory_make ("videotestsrc", "source");
  filter = gst_element_factory_make ("vertigotv", "filter");
  sink = gst_element_factory_make ("autovideosink", "sink");

  /* Create the empty pipeline */
  pipeline = gst_pipeline_new ("test-pipeline");

  if (!pipeline || !source || !filter || !sink) {
    g_printerr ("Not all elements could be created.\n");
    return -1;
  }

  /* Build the pipeline */
  gst_bin_add_many (GST_BIN (pipeline), source, filter, sink, NULL);
  if (gst_element_link_many (source, filter, sink, NULL) != TRUE) {
    g_printerr ("Could not link all elements.\n");
    gst_object_unref (pipeline);
    return -1;
  }

  /* Modify the source's properties */
  g_object_set (source, "pattern", 0, NULL);

  /* Start playing */
  ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (pipeline);
    return -1;
  }

  /* Wait until error or EOS */
  bus = gst_element_get_bus (pipeline);
  msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);

  /* Parse message */
  if (msg != NULL) {
    GError *err;
    gchar *debug_info;

    switch (GST_MESSAGE_TYPE (msg)) {
      case GST_MESSAGE_ERROR:
        gst_message_parse_error (msg, &err, &debug_info);
        g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
        g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
        g_clear_error (&err);
        g_free (debug_info);
        break;
      case GST_MESSAGE_EOS:
        g_print ("End-Of-Stream reached.\n");
        break;
      default:
        /* We should not reach here because we only asked for ERRORs and EOS */
        g_printerr ("Unexpected message received.\n");
        break;
    }
    gst_message_unref (msg);
  }

  /* Free resources */
  gst_object_unref (bus);
  gst_element_set_state (pipeline, GST_STATE_NULL);
  gst_object_unref (pipeline);
  return 0;
}
</code>
However, the return value is a GST_STATE_CHANGE_FAILURE. Any idea why?
Thanks so much!
Comment: Check if the filter element has been created properly. Add the code if(!filter)  g_printerr ("Filter failed"); and see what happens.
Comment: The element is created properly. I've edited the OP to show the entire code I'm using, including that check you mentioned.
Comment: Ran your code, works fine for me. What is the full error message you're getting?
Comment: The full error is simply "Unable to set the pipeline to the playing state", from the code where I set the pipeline to a playing state. That's frustrating that it works fine for you. I can only assume it is a system specific issue. I'm on Ubuntu 12.10.
Comment: Try it on the command line: type gst-launch-0.10 videotestsrc ! vertigotv ! autovideosink and see what happens.
Comment: Huh, seems like it was some sort of issue with the imaage format. I got it working  now by adding a "ffmpegcolorspace" after the vertigo filter. Thanks!
Answer: Add a videoconvert after the filter as well:
<code>gst-launch-1.0 videotestsrc ! vertigotv ! videoconvert ! autovideosink
</code>
Comment: I couldn't add a videoconvert element to my code... the element refuses to be created. Any idea why? Is "videoconvert" only for the command version of the program?
Comment: Turns out I needed to use a "ffmpegcolorspace" converter instead of the "videoconvert" converter. Now it works... thanks so much!
Comment: Yes in 0.10 it was called ffmpegcolorspace.
Answer: This will definitely work, I have tried and got it working.
Please try some other steps or add the exact error you got.
You may also want to post the complete code.
P.S - wanted to add this as comment but fall short of 2 rep
Comment: I probably should have just put the entire code. I've edited the OP. Thanks!
Answer: <code>#include <gst/gst.h>

#ifdef __APPLE__
#include <TargetConditionals.h>
#endif

int tutorial_main(int argc, char *argv[])
{
  GstElement *pipeline, *source, *sink, *filter, *converter;
  GstBus *bus;
  GstMessage *msg;
  GstStateChangeReturn ret;

  /* Initialize GStreamer */
  gst_init(&argc, &argv);

  /* Create the elements */
  source = gst_element_factory_make("videotestsrc", "source");
  filter = gst_element_factory_make("vertigotv", "filter");
  converter = gst_element_factory_make("videoconvert", "converter");
  sink = gst_element_factory_make("autovideosink", "sink");

  /* Create the empty pipeline */
  pipeline = gst_pipeline_new("test-pipeline");

  if (!pipeline || !source || !filter || !converter || !sink)
  {
    g_printerr("Not all elements could be created.\n");
    return -1;
  }

  /* Build the pipeline */
  gst_bin_add_many(GST_BIN(pipeline), source, filter, converter, sink, NULL);
  // if (gst_element_link(source, sink) != TRUE)
  // {
  //   g_printerr("Elements could not be linked.\n");
  //   gst_object_unref(pipeline);
  //   return -1;
  // }
  if (gst_element_link_many(source, filter, converter, sink, NULL) != TRUE)
  {
      g_printerr("Elements could not be linked.\n");
      gst_object_unref(pipeline);
      return -1;
  }
  /* Modify the source's properties */
  g_object_set(source, "pattern", 0, NULL);

  /* Start playing */
  ret = gst_element_set_state(pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE)
  {
    g_printerr("Unable to set the pipeline to the playing state.\n");
    gst_object_unref(pipeline);
    return -1;
  }

  /* Wait until error or EOS */
  bus = gst_element_get_bus(pipeline);
  msg =
      gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE,
                                 GST_MESSAGE_ERROR | GST_MESSAGE_EOS);

  /* Parse message */
  if (msg != NULL)
  {
    GError *err;
    gchar *debug_info;

    switch (GST_MESSAGE_TYPE(msg))
    {
    case GST_MESSAGE_ERROR:
      gst_message_parse_error(msg, &err, &debug_info);
      g_printerr("Error received from element %s: %s\n",
                 GST_OBJECT_NAME(msg->src), err->message);
      g_printerr("Debugging information: %s\n",
                 debug_info ? debug_info : "none");
      g_clear_error(&err);
      g_free(debug_info);
      break;
    case GST_MESSAGE_EOS:
      g_print("End-Of-Stream reached.\n");
      break;
    default:
      /* We should not reach here because we only asked for ERRORs and EOS */
      g_printerr("Unexpected message received.\n");
      break;
    }
    gst_message_unref(msg);
  }

  /* Free resources */
  gst_object_unref(bus);
  gst_element_set_state(pipeline, GST_STATE_NULL);
  gst_object_unref(pipeline);
  return 0;
}

int main(int argc, char *argv[])
{
#if defined(__APPLE__) && TARGET_OS_MAC && !TARGET_OS_IPHONE
  return gst_macos_main(tutorial_main, argc, argv, NULL);
#else
  return tutorial_main(argc, argv);
#endif
}
</code>
Comment: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Comment: This code generates same output as  this gst command does with  vertigotv filter .

gst-launch-1.0 -v videotestsrc ! vertigotv ! videoconvert ! autovideosink


Why does Spark fail with "Failed to get broadcast_0_piece0 of broadcast_0" in local mode?

Question: I'm running this snippet to sort an RDD of points, ordering the RDD and taking the K-nearest points from a given point:
def getKNN(sparkContext:SparkContext, k:Int, point2:Array[Double], pointsRDD:RDD[Array[Double]]): RDD[Array[Double]] = {
    val tuplePointDistanceRDD:RDD[(Double, Array[Double])] = pointsRDD.map(point =>
                           (DistanceUtils.euclidianDistance(point, point2), point))
    sparkContext.parallelize(tuplePointDistanceRDD.sortBy(_._1).map(_._2).take(k))

}
Using just one SparkContext in my application and passing it as a parameter to my function, I'm getting a org.apache.spark.SparkException: Failed to get broadcast_0_piece0 of broadcast_0 error at the moment I call sparkContext.parallelize(tuplePointDistanceRDD.sortBy(_._1).map(_._2).take(k)) to get the KNN points from point2.
I'm constructing sparkContext as this snippet bellow:
var sparkContext = new SparkContext("local", "<app_name>")

What would be the possible causes of facing this kind of error?
Basically this is the LOG of my standalone spark environment with the stack trace of this error:
15/12/24 11:55:29 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@localhost:55731]
15/12/24 11:55:29 INFO Utils: Successfully started service 'sparkDriver' on port 55731.
15/12/24 11:55:29 INFO SparkEnv: Registering MapOutputTracker
15/12/24 11:55:29 INFO SparkEnv: Registering BlockManagerMaster
15/12/24 11:55:29 INFO DiskBlockManager: Created local directory at /private/var/folders/0r/3b6d3b6j45774_9616myw4440000gn/T/blockmgr-70e73cfe-683b-4297-aa5d-de38f98d02f1
15/12/24 11:55:29 INFO MemoryStore: MemoryStore started with capacity 491.7 MB
15/12/24 11:55:29 INFO HttpFileServer: HTTP File server directory is /private/var/folders/0r/3b6d3b6j45774_9616myw4440000gn/T/spark-f7bc8b6f-7d4f-4c55-8dff-0fbc4f6c2532/httpd-fb502369-4c28-4585-a37e-f3645d1d55a3
15/12/24 11:55:29 INFO HttpServer: Starting HTTP Server
15/12/24 11:55:29 INFO Utils: Successfully started service 'HTTP file server' on port 55732.
15/12/24 11:55:29 INFO SparkEnv: Registering OutputCommitCoordinator
15/12/24 11:55:29 INFO Utils: Successfully started service 'SparkUI' on port 4040.
15/12/24 11:55:29 INFO SparkUI: Started SparkUI at http://localhost:4040
15/12/24 11:55:29 WARN MetricsSystem: Using default name DAGScheduler for source because spark.app.id is not set.
15/12/24 11:55:29 INFO Executor: Starting executor ID driver on host localhost
15/12/24 11:55:29 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 55733.
15/12/24 11:55:29 INFO NettyBlockTransferService: Server created on 55733
15/12/24 11:55:29 INFO BlockManagerMaster: Trying to register BlockManager
15/12/24 11:55:29 INFO BlockManagerMasterEndpoint: Registering block manager localhost:55733 with 491.7 MB RAM, BlockManagerId(driver, localhost, 55733)
15/12/24 11:55:29 INFO BlockManagerMaster: Registered BlockManager
15/12/24 11:55:30 INFO TorrentBroadcast: Started reading broadcast variable 0

org.apache.spark.SparkException: Failed to get broadcast_0_piece0 of broadcast_0
java.io.IOException: org.apache.spark.SparkException: Failed to get broadcast_0_piece0 of broadcast_0
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1178)
at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:165)
at org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:64)
at org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:64)
at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:88)
at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
at org.apache.spark.rdd.HadoopRDD.getJobConf(HadoopRDD.scala:144)
at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:200)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
at org.apache.spark.rdd.RDD.sortBy$default$3(RDD.scala:548)
at LOF$.getKNN(LOF.scala:14)
at LOF$.lof(LOF.scala:25)
at BehaviourActivityScoreJudgeTest$$anonfun$1.apply$mcV$sp(BehaviourActivityScoreJudgeTest.scala:14)
at BehaviourActivityScoreJudgeTest$$anonfun$1.apply(BehaviourActivityScoreJudgeTest.scala:11)
at BehaviourActivityScoreJudgeTest$$anonfun$1.apply(BehaviourActivityScoreJudgeTest.scala:11)
at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)
at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683)
at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)
at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)
at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)
at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)
at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)
at org.scalatest.Suite$class.run(Suite.scala:1424)
at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)
at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)
at BehaviourActivityScoreJudgeTest.org$scalatest$BeforeAndAfterAll$$super$run(BehaviourActivityScoreJudgeTest.scala:4)
at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:257)
at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:256)
at BehaviourActivityScoreJudgeTest.run(BehaviourActivityScoreJudgeTest.scala:4)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:55)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2563)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2557)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:2557)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1044)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1043)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:2722)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1043)
at org.scalatest.tools.Runner$.run(Runner.scala:883)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:137)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: org.apache.spark.SparkException: Failed to get broadcast_0_piece0 of broadcast_0
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1$$anonfun$2.apply(TorrentBroadcast.scala:138)
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1$$anonfun$2.apply(TorrentBroadcast.scala:138)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1.apply$mcVI$sp(TorrentBroadcast.scala:137)
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1.apply(TorrentBroadcast.scala:120)
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1.apply(TorrentBroadcast.scala:120)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.apache.spark.broadcast.TorrentBroadcast.org$apache$spark$broadcast$TorrentBroadcast$$readBlocks(TorrentBroadcast.scala:120)
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:175)
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1175)
... 94 more

15/12/24 11:55:30 INFO SparkUI: Stopped Spark web UI at http://localhost:4040
15/12/24 11:55:30 INFO DAGScheduler: Stopping DAGScheduler
15/12/24 11:55:30 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
15/12/24 11:55:30 INFO MemoryStore: MemoryStore cleared
15/12/24 11:55:30 INFO BlockManager: BlockManager stopped
15/12/24 11:55:30 INFO BlockManagerMaster: BlockManagerMaster stopped
15/12/24 11:55:30 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
15/12/24 11:55:30 INFO SparkContext: Successfully stopped SparkContext
15/12/24 11:55:30 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/12/24 11:55:30 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/12/24 11:55:30 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.

Comment: Do you have other tests in ScalaTest that work when launched in IDEA? Can you use `local[*]` instead?
Comment: Which version of Spark are you using?
Answer: Just discovered why I was getting this exception: for a reason my SparkContext object started/stopped several times between ScalaTest methods. So, fixing that behaviour lead me to get spark working in the right way I would expect.
Comment: how you fixed that Start/Stop behaviour of SparkContext
Comment: Please elaborate on how you fixed the issue.
Answer: I was getting this error as well.  I haven't really seen any concrete coding examples, so I will share my solution.  This cleared the error for me, but I have a sense that there may be more than 1 solution to this problem. But this would be worth a go as it keeps everything within the code.
It looks as though the SparkContext was shutting down, thus throwing the error. I think the issue is that the SparkContext is created in a class and then extended to other classes.  The extension causes it to shut down, which is a bit annoying.  Below is the implementation I used to get this error to clear.
Spark Initialisation Class:
import org.apache.spark.{SparkConf, SparkContext}

class Spark extends Serializable {
  def getContext: SparkContext = {
    @transient lazy val conf: SparkConf =
          new SparkConf()
          .setMaster("local")
          .setAppName("test")

    @transient lazy val sc: SparkContext = new SparkContext(conf)
    sc.setLogLevel("OFF")

   sc
  }
 }

Main Class:
object Test extends Spark{

  def main(args: Array[String]): Unit = {
  val sc = getContext
  val irisRDD: RDD[String] = sc.textFile("...")
...
}

Then just extend your other class with the the Spark Class and it should all work out.
I was getting the error running LogisticRegression Models, so I would assume this should fix it for you as well with other Machine Learning libraries as well.
Answer: For me helped this, because SparkContext was already created
val sc = SparkContext.getOrCreate()

Before i tried with this
val conf = new SparkConf().setAppName("Testing").setMaster("local").set("spark.driver.allowMultipleContexts", "true")
val sc = SparkContext(conf)

But it was broken when i ran
 spark.createDataFrame(rdd, schema)

Answer: Related to the above answers, I encountered this issue when I inadvertently serialized a datastax connector (i.e Cassandra connection driver) query to a spark slave. This then spun off its own SparkContext and within 4 seconds the entire application had crashed
Answer: I was also facing the same issue. after a lot of googling I found that I have made a singleton class for SparkContext initialization which is only valid for a single JVM instance, but in case of Spark this singleton class will be invoked from each worker node running on separate JVM instance and hence lead to multiple SparkContext object.
Comment: Simply, just removed the  SparkContext initialization from Singleton class and put it in driver code.


Is there a way to determine what unicode character sets would be associated with a given language code?

Question: I'm looking for a way to determine that character set associated with a given language code.  For example if I passed in "en" for english, it might return me the unicode characters for a-zA-Z.  An API on Apple's platforms would be ideal, but I'd settle for an explanation if such a thing is possible in Unicode that I could code up myself.  Maybe like character classes or something.
Comment: `a-zA-Z` are the same letters and same Unicode code points regardless of the "character set."  Are you asking how to find what's considered a "word" character?
Answer: There are sets of characters used in different languages in the CLDR database. Its format is XML-based LDML, but you might find alternate derived formats or APIs for it, and you might find ICU applicable.
The sets are specified in character elements, and you can find summary charts of existing content, though in a rather awkward format (very wide table).
Perhaps the best way to quickly check whether the CLDR data on characters is useful for your purposes is to look at the data for some locales. The root locale data contains (as part of a large table) the following information about the English locale:
<code>exemplarCharacters main: [a b c d e f g h i j k l m n o p q r s t u v w x y z]
exemplarCharacters auxiliary: [á à ă â å ä ã ā æ ç é è ĕ ê ë ē í ì ĭ î ï ī ñ ó ò ŏ ô ö ø ō œ ú ù ŭ û ü ū ÿ]
exemplarCharacters currencySymbol[a b c č d e f g h i j k l ł m n o º p q r s t u v w x y z]
exemplarCharacters punctuation: [\- ‐ – — , ; \: ! ? . … ' ' ' " " " ( ) \[ \] @ * / \& # † ‡ ′ ″ §]
</code>
I think this demonstrates that the sets are generally too broad. For example, the main set (of letters) for English does not contain even "ë" (think about Brontë), and the auxiliary set contains, in addition to letters commonly used in English, letters that only occur in truly foreign words, like "ō".
There is a rather vague description of what these sets are for. Different use cases would require different approaches. For example, it would be natural to use the union of these sets to decide whether a font is suitable for texts in a given language (i.e., it contains all of the characters, in acceptable shape). But this would in practice exclude fonts that are just fine but lack a glyph for a very rarely used characters. Similarly, if you use information to decide which character encodings can be used, you would end up with the conclusion that only Unicode encodings are acceptable for English.
To conclude, the CLDR data in characters is a useful compilation but should be used with caution and care.


How to show that $\sup(A + B) = \sup\{x + y : x \in A, y \in B\} = \sup A + \sup B$ for bounded subsets $A, B$ of $\mathbb{R}$

Question: Let $A$ and $B$ be two non-empty bounded subsets of $\mathbb{R}$. Write
$A + B = \{x + y : x \in A, y \in B\}$.  Show that $\sup(A + B) = \sup A + \sup B$.
This is a problem from an assignment from my analysis course.
the two definitions I  know of supremum is that it is the least upper bound and also for all $\epsilon >0 \sup(S)-\epsilon < a$ for some $a$ belonging to the set $S$.
I am unable to prove this using the above definitions (though the second one is actually a theorem that can be proven from the previous definition).  I have already tried writing the definition of supremum and saying that for all $\epsilon >0$, there exists $a+b$ belonging to $A+B$ such that $\sup(A+B)<a+b$
But I am unable to proceed further.
Can anyone please provide a right approach to solve this problem and what lemmas must be proven.
Comment: What have you attempted?
Comment: Can you at least see why $\sup A+\sup B$ is an upper bound of $A+B$? Given that, can you see how to proceed?
Comment: I have edited the question to show how much I have actually tried. To be honest I am beginner to analysis and have not done a lot of progress on this question.
Comment: @Crosby Yep I see it's an UB.
Comment: @NKB Okay, now can you show it is the *least* upper bound? What do you know about $\sup A$ and $\sup B$, by definition.
Comment: How do I do that ? Is it by assuming that suppose a supremum less than sup(A) + sup(A) exists?
Comment: @NKB Use the fact that if $M$ is an upper bound of $A$ then $\sup A\le M$ and if $N$ is an upper bound of $B$ then $\sup B\le N$. What can we say about $M+N$?
Comment: Welcome to MathSE.  Please read this [tutorial](https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference) on how to typeset mathematics on this site.  Supposing that a supremum less than $\sup(A) + \sup(B)$ exists is a good idea for arriving at a contradiction.
Comment: @Crosby M+N  will always be greater than sup(A) + sup(B) .
Comment: @NKB Yes, and we are done.
Answer: You wrote in the comments that you see why $supA+supB$ is an upper bound of the set $A+B$. Now you just have to prove it is the least upper bound. Alright, so fix $\epsilon>0$. Then you know there exists $a\in A$ such that $a>supA-\frac{\epsilon}{2}$ and there exists $b\in B$ such that $b>supB-\frac{\epsilon}{2}$. So then $a+b\in A+B$ and $a+b>supA-\frac{\epsilon}{2}+supB-\frac{\epsilon}{2}=supA+supB-\epsilon$. Hence anything smaller than $supA+supB$ is not an upper bound of $A+B$.


Referencing hashed files in Vue's public/index.html mount

Question: I'm trying to add OGP meta tags to my relatively simple Vue.js application. I'm 'hosting' it on GitHub pages, thus it can't be server side rendered, the index.html page served is always static.
I want to add a image meta tag like this:
<meta property="og:image" content="https://example.com/img/some_image.jpeg">

Normally, this would not be a problem, but Vue.js adds hashes to all filenames, static images included.
Thus that tag above will actually look like this:
<meta property="og:image" content="https://example.com/img/some_image.09c37fe6.jpeg">

I could go edit this hash in every single time I build, but that's dumb and wouldn't work if I decided to go with a automatic build system. So, does Vue.js include any ways of automatically figuring out the hash for any given file?
I assume that this might have something to do with the <link rel="icon" href="<%= BASE_URL %>favicon.ico"> tag inserted by vue-cli, but I can't figure out what exactly what provides this functionality, and if it can be used.
I understand that disabling hashes is an option, but I'm curious if there is a solution that lets me keep hashes.
Answer: Didn't expect to be answering my own question immediately, it seems I've figured it out. Sorry if stuff like this is supposed to be insanely obvious to normal web developers.
Simply insert
require('./path/image.ext')

Files generated still use hashes, and if you're using said files in your actual src folder, relative paths work too:

<meta property="og:image" content="<%= require('./img/spirit.jpg') %>">

<meta property="og:image" content="<%= require('../src/assets/spirit.jpg') %>">


How to assign a lamba expression event to another event

Question: I am dinamically defining a wpf window with some textboxes and checkboxes programmatically defined.
After having added them I want to check if all textboxes and checkboxes are filled.
To do that for the textboxes I have the lamda expression defined event:
var mw = new MyWindow();
mw.KeyDown += (senderEvent, args) =>
{
  cycle on all windows and when it's a TEXTBOX or a COMBOBOX analyze if it's empty or not.
  That is a lot of code which I wouldn't not replicate nor use an external routine
}

but how can I associate the same code to be fired also when combobox are closed??
The best would be to change the code above to (pseudocode):
mw.AllEvents += (senderEvent, args) =>{ ... }

or
mw.KeyDown && mw.ComboBoxClose += (senderEvent, args) =>{ ... }

but that doesn't make sense since a window has not comboboxclose event
either this or when defining all the comboboxes to be able to do something like:
newComboBox.ComboBoxClosed +=  (senderEvent, args) =>{ ...clone from mw.KeyDown }

Thank you in advance
Answer: The method signatures are different, so you can't do exactly that.  However, you can point all your lambdas toward a common method:
private void MyHandler()
{
    //bulk of the work
}

//....
mw.KeyDown += (s,e) => MyHandler();
mw.ComboBoxClose += (s,e) => MyHandler();

If you need the EventArgs you could pass it through and do some sort of switch on the type in MyHandler
Comment: A window is not a comboBox so it has no mw.ComboBoxClose Event!
Comment: Oh I see.  You'll have to manually pass up / re-raise your ComboBoxClose from `MyWindow`.  That's pretty messy
Answer: You can't, they're different types of delegate.
DropDownClosed is an EventHandler while KeyDown is an KeyEventHandler, so when you write your lambda, args is type EventArgs in one and KeyEventArgs in other, they're just infering it's type from the assignation.
If both of them where EventHandlers or KeyEventHandlers then you can save it to a variable and assign that variable.


Boss RC-300 seamless change to new phrase memory for song medleys

Question: So far I've just been using the Boss RC-300 looper for single songs.
But now I want to use it for medleys of songs.  Problem is when I switch phrase memories using either the Loop FX switch or an external control switch like the Boss FS-6, the phrase memory switches but the second phrase memory doesn't start when the playing loop ends.
I want some way to cue the next phrase memory loop end just like I can cue the next track on loop end in a given phrase memory.
Any way of doing this seamlessly or am I just going to have to tap dance and hope for the best?
Answer: One possibility would be to set the tracks to play in series with each song in a different track.


Pegar os 3 primeiros resultados numa consulta mysql_query

Question: Estou recuperando dados do banco que seria "O indicador do indicado", só que pega todos até recuperar todos. O que eu preciso fazer é pegar só os 3 primeiros resultados.
E se der eu indicar quantos resultados eu quero pegar, ou seja, setar isso na função ou em alguma variavel e quando pegar o valor X da variavel, então ele para de buscar pois não tem mais necessidade.
Atualmente fiz essa função, mas está me retornando todos os dados certos, mas está retornando mais do que 3 e eu quero somente 3 por enquanto. Preciso pegar os 3 primeiros somente.
<?php
$db = mysql_connect('localhost', 'root', 'vertrigo');
$bd = mysql_select_db('cotas', $db);

$array = array();

function indicadores($id){

    global $array;

    $query = mysql_query("SELECT * FROM indicadores WHERE id_indicado = '$id'");

    if(mysql_num_rows($query) > 0){

        $fetch = mysql_fetch_object($query);

        $id = $fetch->id_indicador;

        $array[] = $id;

        indicadores($id);

    }
}

indicadores(10);

echo var_dump($array);
?>

Answer: Se eu entendi bem, basta você alterar sua SQL query.
Tente isto:
$query = mysql_query("SELECT * FROM indicadores WHERE id_indicado = '$id' LIMIT 3 ORDER BY id_indicado DESC");

Comment: Esqueci deste detalhe, obrigado por comentar.
Comment: pra melhorar use **order by id_indicado desc** para pegar o ultimo registro inserido no seu **BD**
Comment: Você não precisa de recursividade para resolver isso, creio que essa query seja o suficiente.
Comment: Não deu certo. Ele me retorna um array `empty` creio que é porque ele re-executa a função até achar o último indicador.
Comment: tente **if(count($query)>0)**
Answer: Eu tomei a liberdade de fazer algumas mudanças nos seu código.
Observação, você não deveria usa mysql_, são funcões depreciadas e não vão funcionar mais em futuras versões do php. Use mysqli_.
Obs: Teste a query num phpmyadmin ou qualquer um que você tiver, para ver se está retornando dados, se a query está correta.
<?php
$db = mysqli_connect('localhost', 'root', 'vertrigo');
$bd = mysqli_select_db($db, 'cotas');

function indicadores($id){

    global $db;

    $array = array();

    $query = mysqli_query($db, "SELECT * FROM indicadores WHERE id_indicado = '$id' ORDER BY id_indicador DESC LIMIT 3");
    if(mysqli_num_rows($query) > 0){

        while($fetch = mysqli_fetch_object($query)){
            $id = $fetch->id_indicador;
            $array[] = $id;
        }

    }

    return $array;
}

$array = indicadores(10);

var_dump($array);

Comment: Obrigado pelas modificações. Eu na verdade não uso `mysql`, uso `CodeIgniter`, fiz isso só na correria aqui rsrsrs. A questão de fazer com o seu código é que ele só me retornará os registros onde `id_indicado` é igual a 10. Eu na verdade preciso fazer o seguinte: Renato indicou Jorge, Então ganha 50%. Jorge indicou Luiz, Então Jorge ganha 50% e Renato 40%. Luiz indicou Bruno, Então Luiz ganha 50%, Jorge Ganha 40%, Renato Ganha 30.. E assim por diante. Só preciso saber os indicadores da pessoa indicada e ir subindo, tipo uma escada.
Comment: Entendi errado a descrição inicial. Desculpa.
Answer: RESOLVIDO
Achei essa solução:
<?php
$db = mysql_connect('localhost', 'root', 'vertrigo');
$bd = mysql_select_db('cotas', $db);

$array = array();

function indicadores($id, $niveis){

    global $array;

    if($niveis > 0){

    $query = mysql_query("SELECT * FROM indicadores WHERE id_indicado = '$id'");

        if(mysql_num_rows($query) > 0){

            $fetch = mysql_fetch_object($query);

            $id = $fetch->id_indicador;

            $array[] = $id;

            indicadores($id, $niveis-1);

        }

    }
}

indicadores(10, 3);

echo var_dump($array);
?>


Python Pandas Multi Indexing by keyword

Question: if I have a multi-index data frame in pandas like this.
test.index.names
FrozenList(['target', 'h', 'd', 'y', 'obsrv'])

test.index.values
('A', '15', '60', '0', 97),
('B', '15', '60', '0', 98),

I can index the data frame associated with target = A, h = 15, d = 60, y = 0, obrv = 97 by using a tuple like this
test.loc[('A', '15', '60', '0', 97)]

But I want to index this data frame somehow like this.

test(target = 'A', h = 15)...

such that the order inside does not matter. Is that possible ?
Comment: Can you provide a sample of your dataframe?
Answer: Instead of loc you can use .xs() to get cross sections with the index intact. Not exactly the form you're looking for, but should serve your purposes I'm guessing:
test.xs(('A', '15', '60', '0', 97), level=('target', 'h', 'd', 'y', 'obsrv'))

You could then use the answer provided in DataFrame with MultiIndex to dict to convert to a dict.
I can give a more specific answer for your problem if you can provide a dataset.
Answer: For multiple index
test.loc[pd.IndexSlice['A', '15',:,:,:],:]

Python Pandas Multi Indexing by keyword