"Constant variance" violation

Question: If we apply linear regression on a data which has a BINARY(0,1) dependant variable, the very important assumption of "constant variance" of the dependant variable across independant variables is violated. Can anyone explain how ?
Comment: Linear regression with a binary dependent variable? That is asking for trouble in so many ways beyond the constancy of variance. Can you elaborate why you would want to do this instead of (e.g.) logistic regression?
Comment: What's the variance of a Bernoulli($\pi$) random variable?
Comment: @glen_b : its p(1-p) where p is the probability that your event is 1
Comment: @NickSabbe : I am trying to understand why do we NOT use linear for binary dependant. As per text books its not wise to use linear for some violations. So I was trying to understand individually what all assumptions are violated
Comment: So what happens to $p(1-p)$ when $p$ is near 0 or 1 as compared to when it's near the middle? *It changes*. The variance is not constant. Since the $p$ changes with the IVs, so does the variance.
Comment: @Glen_b : can you explain using an example / graphs.May be some random data .
Comment: Just think about it for 5 seconds. What is $p(1-p)$ when $p=0.5$? What is it when $p=0.01$? Are they the same?
Comment: So if linear is a bad idea, what kind of regression should one use (provided the dependent variable is continuous)?
Answer: You may try to visualize the idea by making a residual plot of the said regression:

The constant variance assumption is that the expected dependent variable conditioned on all independent variables is constant. If we, very roughly, slice up the fitted values into chunks, and calculate the variance of the residual within each, they should be roughly similar. From the above plot you can see that it's not always true for binary outcome (red fonts indicate the variance within each segment.)
Not always because sometimes the situation is not that bad, especially if the independent variable is not very predictive (aka, there is a good deal of overlapping in the groups outcome = 1 and outcome = 0.) However, the normality assumption will still be violated.
Comment: you just +1 your count of number of your fans. This is the precise answer I was looking for. So this exactly in  line with your previous answer to my old question. So we check for constant variance within subsamples of the data by considering the segments(roughly) as mentioned above in your red fonts. That is how we look for constant variance. Hope my understanding is correct
Comment: can you explain why do we have two distinct lines above. Is it because one line corresponds to the residuals calculated for 1's and the other is for 0's. And also as per your last comment. The height of the Red arrows (previous post) is a proxy for varaince. Right ?
Comment: @Mukul For me, this would serve just to understand the concept. In real life, I have never segregated the data and check the variance chunk by chunk (too many tests, and the cutting of the sample can also be controversial.) You can simply plot residual$^2$ against the predicted value to observe if there are any violation of constant variance.
Comment: @Mukul Yes, correct. One line is for the outcome = 0 and the other one is outcome = 1. This is, however, only true if you have one predictor. As you add more predictors, the picture can change to either smearing out (with continuous predictor) or having many straight lines (with categorical predictor.) As for the red arrows, yes you're correct. They show the range of the error, which is not variance. Please consider that as a simplified representation.

"Constant variance" violation


Docker Could not build image from Maven

Question: I need your help I got project to do and I don't know how to start it ,I hope do you are can help me .
and this repository: https://github.com/spring-guides/top-spring-boot-docker
First thing I built a Dockerfile According to README, I make Dockefile from repository to my host
FROM openjdk:8-jdk-alpine as build
WORKDIR /workspace/app

COPY mvnw .
COPY .mvn .mvn
COPY pom.xml .
COPY src src
RUN --mount=type=cache,target=/root/.m2 ./mvnw install -DskipTests

ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} target/application.jar
RUN java -Djarmode=layertools -jar target/application.jar extract --destination target/extracted

FROM openjdk:8-jdk-alpine
RUN addgroup -S demo && adduser -S demo -G demo
VOLUME /tmp
USER demo
ARG EXTRACTED=/workspace/app/target/extracted
WORKDIR application
COPY --from=build ${EXTRACTED}/dependencies/ ./
COPY --from=build ${EXTRACTED}/spring-boot-loader/ ./
COPY --from=build ${EXTRACTED}/snapshot-dependencies/ ./
COPY --from=build ${EXTRACTED}/application/ ./
ENTRYPOINT ["java","-noverify","-XX:TieredStopAtLevel=1","-Dspring.main.lazy-initialization=true","org.springframework.boot.loader.JarLauncher"]

After I build the Dockerfile
C:\Users\LS\Desktop\Job\Test>docker build -t myorg/myapp .

    [+] Building 6.4s (13/19)
     => [internal] load build definition from Dockerfile                                                               0.0s
     => => transferring dockerfile: 32B                                                                                0.0s
     => [internal] load .dockerignore                                                                                0.0s
     => => transferring context: 2B                                                                                0.0s
     => [internal] load metadata for docker.io/library/openjdk:8-jdk-alpine                                            3.6s
     => [internal] load build context                                                                                0.0s
     => => transferring context: 897B                                                                                0.0s
     => [stage-1 1/7] FROM docker.io/library/openjdk:8-jdk-alpine@sha256:94792824df2df33402f201713f932b58cb9de94a0cd5  0.0s
     => CACHED [stage-1 2/7] RUN addgroup -S demo && adduser -S demo -G demo                                           0.0s
     => CACHED [stage-1 3/7] WORKDIR application                                                                       0.0s
     => CACHED [build 2/9] WORKDIR /workspace/app                                                                      0.0s
     => CACHED [build 3/9] COPY mvnw .                                                                                0.0s
     => CACHED [build 4/9] COPY .mvn .mvn                                                                              0.0s
     => CACHED [build 5/9] COPY pom.xml .                                                                              0.0s
     => CACHED [build 6/9] COPY src src                                                                                0.0s
     => ERROR [build 7/9] RUN --mount=type=cache,target=/root/.m2 ./mvnw install -DskipTests                           2.7s
    ------
     > [build 7/9] RUN --mount=type=cache,target=/root/.m2 ./mvnw install -DskipTests:
    #13 1.665 [INFO] Scanning for projects...
    #13 1.898 [INFO]
    #13 1.898 [INFO] --------------------------< com.example:demo >--------------------------
    #13 1.898 [INFO] Building demo 0.0.1-SNAPSHOT
    #13 1.898 [INFO] --------------------------------[ jar ]---------------------------------
    #13 2.166 [INFO]
    #13 2.167 [INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ demo ---
    #13 2.454 [INFO] Using 'UTF-8' encoding to copy filtered resources.
    #13 2.454 [INFO] Using 'UTF-8' encoding to copy filtered properties files.
    #13 2.460 [INFO] Copying 1 resource
    #13 2.492 [INFO] Copying 0 resource
    #13 2.494 [INFO]
    #13 2.494 [INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ demo ---
    #13 2.608 [INFO] Changes detected - recompiling the module!
    #13 2.612 [INFO] Compiling 1 source file to /workspace/app/target/classes
    #13 2.683 [INFO] ------------------------------------------------------------------------
    #13 2.683 [INFO] BUILD FAILURE
    #13 2.683 [INFO] ------------------------------------------------------------------------
    #13 2.685 [INFO] Total time:  1.040 s
    #13 2.685 [INFO] Finished at: 2022-06-24T11:36:26Z
    #13 2.685 [INFO] ------------------------------------------------------------------------
    #13 2.687 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project demo: Fatal error compiling: invalid target release: 17 -> [Help 1]
    #13 2.687 [ERROR]
    #13 2.687 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    #13 2.687 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    #13 2.687 [ERROR]
    #13 2.688 [ERROR] For more information about the errors and possible solutions, please read the following articles:
    #13 2.688 [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
    ------
    executor failed running [/bin/sh -c ./mvnw install -DskipTests]: exit code: 1

Comment: he is using openjdk:8
Comment: You have JDK 8 in your Docker container, but your Maven build is configured to compile for Java 17. That's not going to work, the Java compiler in JDK 8 cannot compile code for Java 17. Either use JDK 17 in your Docker container, or modify your Maven project so that it works with JDK 8.
Comment: you could use jib to build it. Google provides a maven plugin for it
Comment: Maven fails with "Fatal error compiling: invalid target release: 17" - what version of the jdk are you using?
Comment: weird, the java project you referenced seems to use java 8, and your base image is java 8, but it complains about 17, did you change the java version of the project?
Answer: Please look more at line, you could provide a correct jdk version.
#13 2.687 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project demo: Fatal error compiling: invalid target release: 17 -> [Help 1]
Answer: The javac in the docker image is too Old to be able to compile Java 17 code
Either convert to java 8 or use a java17-based image.
Comment: Thank for your comment, I edited  the Dockerfile to JDK17 and  build it , after  than I tried to run the image and I get this message again .

[C:\Users\LS\Desktop\Job\Test>docker run -p 9090:9090 myorg/myapp
Error: Invalid or corrupt jarfile /app.jar]
Comment: It is most likely invalid or corrupt then.   That can have many reasons. I would look inside the container and investigate `/app.jar`.
Comment: Also this looks needlessly complicated for a relatively simple compile-and-package dockerfile.

Docker Could not build image from Maven


Conditional PDO bindParam Update

Question: So I'm trying to do a conditional update but I seem to be having problems with binding the data with the statement.
function:
function updateEditor($email, $first, $last, $id){
global $DBH;
$response = false;
$upemail = "";
$upfirst = "";
$uplast = "";

$stmt = "SELECT memEmail, memFirst, memLast FROM MEMBER WHERE memID = :id";
try{
    $STH = $DBH->prepare($stmt);
    $STH->bindParam(':id', $id);
    $STH->execute();

    $STH->setFetchMode(PDO::FETCH_ASSOC);
    $row = $STH->fetch();
    if($row['memEmail'] != $email){ $upemail = $email;}
    if($row['memFirst'] != $first){ $upfirst = $first;}
    if($row['memLast'] != $last){ $uplast = $last;}

}catch(PDOException $e) {
    echo $e->getMessage() . "first";

}
$stmt .= "UPDATE MEMBER SET ";
if(!empty($upemail)){
    $stmt .= "memEmail = :memEmail";
    if(!empty($upfirst) || !empty($uplast)){
        $stmt .= ", ";
    }
}
if(!empty($upfirst)){
    $stmt .= "memFirst = :memFirst";
    if(!empty($uplast)){
        $stmt .= ", ";
    }
}
if(!empty($uplast)){
    $stmt .= "memLast = :memLast";
}
if(empty($upemail) && empty($upfirst) && empty($uplast)){
    return false;
}else{
    $stmt .= " WHERE memID = :id";
}

try{
    $STH = $DBH->prepare($stmt);
    if(!empty($upemail)){$STH->bindParam(':memEmail', $upemail);}else{$STH->bindParam(':memEmail', $row['memEmail']);}
    if(!empty($upfirst)){$STH->bindParam(':memFirst', $upfirst);}else{$STH->bindParam(':memFirst', $row['memFirst']);}
    if(!empty($uplast)){$STH->bindParam(':memLast', $uplast);}else{$STH->bindParam(':memLast', $row['memLast']);}
    $STH->bindParam(':id', $id);
    $STH->execute();

    $STH->setFetchMode(PDO::FETCH_ASSOC);

    $response = true;
}catch(PDOException $e) {
    echo $e->getMessage() . "second";
    $response = $e->getMessage() . "second";

}
return $response;
}

I have tried putting the variables into the statement, using ?, and the code above so far. The error I keep getting is:

SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens
Comment: sorry that was testing. the error is coming from the second `try` statement
Comment: You're setting the param in both if and else statement. But in the query you include the param only if it's not empty.
Answer: Here:
$stmt .= "UPDATE MEMBER SET ";

you append the UPDATE to the previous $stmt string. You'll end up with:
$stmt = "SELECT memEmail, memFirst, memLast FROM MEMBER WHERE memID = :idUPDATE MEMBER SET "; // and the rest

reulting in one identifier more (:idUPDATE).
Remove the . to start a new query in this string.
$stmt = "UPDATE MEMBER SET ";

Note:
You are making this way too complicated. Skip the checks for empty values, just update all columns when you update a dataset, you don't gain anything by checking what has changed and what hasn't first.
Comment: I put it without the `.` still same error. and I changed the name of the first `stmt` and am still getting same error
Comment: Well, echo `$stmt` before you prepare it and compare it with your binds.
Answer: Besides @Gerald Schneider answer, you are setting the param in both cases (if/else)
if(!empty($upemail)){$STH->bindParam(':memEmail', $upemail);}else{$STH->bindParam(':memEmail', $row['memEmail']);}

But are defining the parameters only in one case
if(!empty($upemail)){
    $stmt .= "memEmail = :memEmail";
    if(!empty($upfirst) || !empty($uplast)){
        $stmt .= ", ";
    }
}
if(!empty($upfirst)){
    $stmt .= "memFirst = :memFirst";
    if(!empty($uplast)){
        $stmt .= ", ";
    }
}
if(!empty($uplast)){
    $stmt .= "memLast = :memLast";
}

There's no else condition

Conditional PDO bindParam Update


Drush command to run cron won't work in Crontab

Question: I am attempting to run two Drush commands from my crontab:
# m h  dom mon dow   command
*/2 * * * * drush -r /path -l https://www.domain.com/ cron
*/2 * * * * drush -r /path -l https://www.domain.com/ -u civicron civicrm-api job.execute

Both of these commands work when I run them manually on the server, however they do not appear to work through crontab, even though the log shows them running:
Apr 17 20:16:01 ip-000000000000 CRON[3736]: (ubuntu) CMD (drush -r /path -l https://www.domain.com/ -u civicron civicrm-api job.execute)
Apr 17 20:16:01 ip-000000000000 CRON[3737]: (ubuntu) CMD (drush -r /path -l https://www.domain.com/ cron)

Any suggestion as I cannot see why these would not be running?
Answer: I ran into a similar issue multiple times. I think in all cases cron was running a different version of php.  To fix I ended up having to define my php path in my drush call:
*/5 * * * * /usr/local/bin/php /home/myuser/bin/drush elysia-cron run --root=/home/mysite/public_html --uri=www.mysite.com

Also, another thing that helped me debug the issue is by adding a log path > path/to/file.log like this and then looking at the exact error:
*/5 * * * * /usr/local/bin/php /home/myuser/bin/drush elysia-cron run --root=/home/mysite/public_html --uri=www.mysite.com > /home/mysite/cron.log 2>&1

Comment: I have also been caught by this, all I needed was the path to Drush, I did not have to worry about PHP - I just used 'which drush' for the user the crontab was set up for and substituted this path in the script.  You can check it by creating a simple script that does a drush @your_alias status and redirect that to a log.  Add it to cron to run every 5 mins and you can check all is OK.  Just delete this task after you are happy


Incrementar ao chamar função javascript

Question: Ola.
Eu gostaria que a cada vez que eu chamar a função seja incremntado o valor de ( i ) e exibido no alert, não sei oque estou fazendo errado.
Meu codigo;
function incrementar()
{
var count = 0.
count++;
alert(count);
}

Obrigado
Answer: Você está redefinindo o contador a cada vez que chama a função, porque está abrindo var count dentro. Defina count antes, como no exemplo, pra que a mesma variável esteja disponível sempre que for chamada.

var count = 0;

function incrementar() {
    count++;
    // use console.log ao invés de alert para ver os resultados
    // no console do navegador, aperte F12 no Chrome para ver o console.
    console.log(count);
}

incrementar();
incrementar();
incrementar();
incrementar();
Comment: Legal, mas eu preciso que o valor incrementado seja global para eu poder utilizar em outras funções do javascript. Pensei em definir assim fora da função ( var totalIncremento; ) dai dentro da função fazer assim ( totalIncremento = count++; ). Daria certo isso
Comment: Nesse caso o valor incrementado já é global, dentro do escopo que você passou. Se você precisa de um valor que seja realmente global, pra ser acessado por outros escopos, outros scripts, pode usar `window.count = 0` antes e `window.count++` dentro da função.
Answer: A função sempre vai exibir o valor 1, pois o algoritmo está setando o valor da variável count para zero toda vez que a função é executada.
Você pode resolver o problema da seguinte maneira:

var count = 0;
function incrementar() {
  alert(++count);
}
Comment: Obrigado amigo.

Incrementar ao chamar função javascript


No instance for (Floating Integer) arising from a use of '**'?

Question: I have the following Haskell code:
two :: Integer -> Integer
two i = toInteger(2 ** i)

Why isn't it working?
Answer: (**) requires floating point input based on the function signature:
(**) :: Floating a => a -> a -> a

toInteger on the other hand requires input that is integral in nature:
toInteger :: Integral a => a -> Integer

Therefore, you cannot reconcile the two the way you use it. That said, since you seem to be expecting integer input anyway, you might consider using (^) instead, like so:
two :: Integer -> Integer
two i = 2 ^ i

As @leftaroundabout correctly points out in the comments, (^) will fail for negative values of i. This can be resolved by checking for value and handling in an alternate manner, something like this:
two :: Integer -> Integer
two i = if i > 0 then 2 ^ i else floor (2 ** fromIntegral i)

Comment: It should be noted that `^` only works for non-negative exponents. If round-off to zero is desired, like `floor (2 ** fromIntegral i)` would give for negative `i`, then this needs to be handled with an extra clause.
Comment: There's also `^^`, whose result can be rounded.
Comment: @leftaroundabout Thanks for this. Do you mind if I add it into my answer?
Answer: Use ^ instead:
two i = 2 ^ i

And then there is no need for to cast the result back to an Integral type.
Answer: The reason this...
two :: Integer -> Integer
two i = toInteger(2 ** i)

...doesn't work is because you've declared i to be an integer, and if we look at the type of (**)...
Prelude> :t (**)
(**) :: Floating a => a -> a -> a

... all it's arguments are of the same type, and that type has to be an instance of the Floating type-class. Integer is not an instance of Floating. This is what "No instance of (Floating Integer)" means.
The simplest solution is to use ^ as ErikR suggests. It raises a number to an integral power.
(^) :: (Integral b, Num a) => a -> b -> a

If you want to work through using ** to learn a bit more, keep reading.

So we need to convert your integer into a type which is an instance of Floating. You can do this with fromIntegral. If we do this:
two :: Integer -> Integer
two i = toInteger(2 ** fromIntegral(i))

...we still get a load of error messages complaining that various types are ambiguous. These aren't as clear as the first message, but the issue is the use of toInteger which becomes apparent if we look at it's type.
Prelude> :t toInteger
toInteger :: Integral a => a -> Integer

As we're passing the result of ** to toInteger, and that is a Floating, not an Integral, toInteger is the wrong function. round is a better choice.
two :: Integer -> Integer
two i = round(2 ** fromIntegral(i))

This now works.


why wont my script work when i test it with postman

Question: when i text my code in JavaScript with postman there was error
the index code is :
 const bodyParser = require('body-parser');

const patient = require("./routes/patient");
const express = require("express")
const mongoose = require("mongoose");

const app =express()
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use("/", patient);

mongoose.connect("mongodb://localhost/DataStore", {
  useNewUrlParser: true,
  useUnifiedTopology: true,

});
app.listen(3000, () => {
    console.log("working on 3000 port");
  });

app.get("/", (req, res) => {

});

the patient.js is :
    const experss = require("express");
const Route = experss.Router();
const Patient = require("../modules/patient");

Route.post("/addpatient", (req, res) => {
    const patient = new Patient({
        patientId: req.body.patientId,
        patientName: req.body.patientName,
        patientage: req.body.patientage,
        patientMumName: req.body.patientMumName,
        patientfatherid: req.body.patientfatherid,
        createdAt: req.body.createdAt,

    });

    Patient.create(Patient)
        .then((result) => {
        res.status(201).json({
        message: "done with adding",
        data: result,
         });
    })

       .catch((err) => {
      res.status(500).json({
        error: err,
         });
       });

});

Route.get("/getPatient/", (req, res) => {
  // Patient.find({})
  //   .then((data) => {
  //     if (data) {
  //       res.json(data);
  //     } else {
  //       res.json("nothig to show");
  //     }
  //   })
  //   .catch((err) => res.send({ error: "Nothing fonddddd" }));
  res.send("done");
});

Route.put("/updatePatient/:id", (req, res) => {

  Patient.findByIdAndUpdate(req.params.id, {
    $set: {

      patientName: req.body.patientName,
      patientage: req.body.patientage,
      patientMumName: req.body.patientMumName,
      patientfatherid: req.body.patientfatherid,

    },
  }).then(() => {
    Patient.findOne({ StuId: req.body.id }).then((data) => {
      res.send(data);
    });
  });
});

Route.delete("/DeletePatient/:id", (req, res) => {
  Patient.findByIdAndDelete(req.params.id).then(() => {
    res.send("Done with delete");
  });
});

module.exports = Route;

the error was:

        Error

        TypeError: Cannot read properties of undefined (reading 'then')
                at C:\Users\nsr\Desktop\clinic\routes\patient.js:21:9
                at Layer.handle [as handle_request] (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\layer.js:95:5)
                at next (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\route.js:137:13)
                at Route.dispatch (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\route.js:112:3)
                at Layer.handle [as handle_request] (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\layer.js:95:5)
                at C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\index.js:281:22
                at Function.process_params (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\index.js:341:12)
                at next (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\index.js:275:10)
                at Function.handle (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\index.js:174:3)
                at router (C:\Users\nsr\Desktop\clinic\node_modules\express\lib\router\index.js:47:12)

Comment: The error is in `patient.js` so please post the code of that file.
Comment: @GabrielePetrioli
i update the question
Comment: Re edit, so it says that the return value of `Patient.create(Patient)
` is `undefined` so we still don't have a [mcve]. (You should probably remove all the web server code and make a simple test script to debug this with).
Comment: sorry but I don't understand @GabrielePetrioli
Answer: I think the error is with  Patient.create(Patient)
It should be Patient.create(patient) (lowercase patient as param, which is the one you created a couple lines above)
Comment: thankyou very much that's right the problem was with the letter P
Answer: As Gabriele pointed out the error seems to be the capital cased Patient instead of patient. Alternatively, here's a another solution using async-await:
Route.post("/addpatient", async (req, res) => {
    const patient = new Patient({
        patientId: req.body.patientId,
        patientName: req.body.patientName,
        patientage: req.body.patientage,
        patientMumName: req.body.patientMumName,
        patientfatherid: req.body.patientfatherid,
        createdAt: req.body.createdAt,

    });
    try {
      const result = await Patient.create(patient);
      return res.status(201).json({
        message: "done with adding",
        data: result,
      });
    } catch(err){
       return res.status(500).json({
        error: err,
       });
    }

});


performing PCA first before logistic regression

Question: I am reading this:
https://towardsdatascience.com/pca-using-python-scikit-learn-e653f8989e60
This is performing PCA before performing a logistic regression (in Python). I am not fluent in Python (I am using Matlab).
My questions are regarding the mathematical side of the process being performed.
I have taken the pca of my data sets, and found that I have 95% of the variability in the first three principal components.
I do not understand (from the link above) what is being trained/what is being done to the original data to train the logistic regression.
Here are my specific questions:

Given that my first three principal components account for 95% of the variability of the data, what do I do with it? Do I transform the data? How do I use the loadings to train the logistic regression?
Can someone please give a mathematical supplement/lecture regarding this procedure?

Thanks
Answer: Intuitively you can think the three component as the projection of original data(input data matrix X) to lower dimensions. Or you can think like the original data is transformed to lower dimensions.

Once you got the components keep aside the original data. Just use the three components you got instead of your original data(input data matrix X) in training the model. I hope you have no doubt on training logistic regression.
This is the best resource so far I encountered for PCA.
Comment: Thank you for your answers. But this is the part I understand. PCA is 'rewriting' into lower dimensions as a linear combination of the original variables. My question is: what will happen to my data? Of course I wont be training my original data because otherwise, why perform PCA to begin with, right? So will be written in place of the old data? Do I transform them in terms of the loadings? And if so, how to?
Comment: Sorry, I didn't get you. Can you please be more clear?


Show that : $\frac{1}{\zeta(3)}<2C-1$

Question: Problem :
Show that :
$$\frac{1}{\zeta(3)}<2C-1$$
Where we can see the zeta function and the Catalan's constant .

It recall me the Faulhaber problem which relate the case $n=1,3$ with a square .
There are many representation for the Riemann's zeta function see https://functions.wolfram.com/ZetaFunctionsandPolylogarithms/Zeta/07/01/01/
Currently there is also nice representation for the Catalan's constant see https://en.wikipedia.org/wiki/Catalan%27s_constant
One hint :
For $x\geq 1$ we have :
$$\left(2C-1\right)\left(\sum_{n=1}^{100}\frac{x}{n^{3}}\right)^{-1}>\left(3-C\right)\frac{\ln\left(x\right)}{1+x^{2}}$$
But with all of this I cannot find a proof ...
Update :
Using positives function and Cauchy-Schwarz inequality we have :
$$\left(\int_{0}^{\infty}\left(\frac{x^{2}}{e^{x}-1}\frac{\left(\frac{x}{\cosh\left(x\right)}-\frac{1}{1.18293097883}x^{\frac{3}{2}}\cdot e^{-\frac{2}{3}x^{\frac{3}{2}}}\right)}{2}\right)^{\frac{1}{2}}dx\right)^{2}<\int_{0}^{\infty}\frac{\left(\frac{x}{\cosh\left(x\right)}-\frac{1}{1.18293097883}x^{\frac{3}{2}}\cdot e^{-\frac{2}{3}x^{\frac{3}{2}}}\right)}{2}dx\int_{0}^{\infty}\frac{x^{2}}{e^{x}-1}dx=1.000028\cdots$$
Now I really think we can use Cauchy-Schwarz for integral .
We have also empirically :
$$\frac{1}{\zeta(3)}<\frac{99486}{100000}\left(2\left(\int_{0}^{\infty}\frac{t}{\cosh\left(t+\frac{1}{t}\right)}dt\right)-1\right)\left(\int_{0}^{\infty}\frac{t}{e^{t}+1}dt\right)<2C-1$$
Update 2 :
Tricky Cauchy Schwarz inequality :
Let :
$$h\left(x\right)=\frac{2ax}{\cosh\left(ax\right)},g\left(x\right)=\frac{h\left(x\right)}{\int_{0}^{\infty}h\left(y\right)dy}$$
Then $\exists a ,1<a<\zeta(3)$ such that :
$$\frac{1}{a}\int_{0}^{\infty}\sqrt{\frac{x^{2}}{e^{x}+1}\cdot\frac{\left(h\left(x\right)-g\left(x\right)\right)}{2}}dx=1$$
Update 3 :
I finally found a reference see https://arxiv.org/pdf/2105.11771.pdf we have :
$$\int_{0}^{\frac{\pi}{2}}\int_{0}^{\frac{\pi}{2}}\frac{\ln\left(\cos\left(\frac{x}{2}\right)\right)-\ln\left(\cos\left(\frac{z}{2}\right)\right)}{\cos\left(x\right)-\cos\left(z\right)}dxdz=\pi G-\frac{7}{4}\zeta(3)$$
Then we can use $a,x\in[0,\pi/2)$ :
$$f\left(x\right)=\frac{\ln\left(\cos\left(\frac{x}{2}\right)\right)-\ln\left(\cos\left(\frac{a}{2}\right)\right)}{\cos\left(x\right)-\cos\left(a\right)},g\left(x\right)=\frac{e^{\frac{x}{4}}}{2}+\frac{e^{-\frac{x}{4}}}{2}-1+f\left(0\right),h\left(x\right)=f\left(x\right)-g\left(x\right)$$
Then we have :
$$f(x)\geq \frac{h''\left(0\right)}{2}x^{2}+\frac{h''''\left(0\right)}{4!}x^{4}+g\left(x\right)$$
Last update :
Why not invoke probability and Zeta function and probability and prime numbers https://arxiv.org/abs/2203.01832
How to show it without a calculator ? Is it an isloted problem and if so have you  a reference ?
Ps: For the story I found it accidentaly in setting the value 0.831 in Wolfram Alpha see https://www.wolframalpha.com/input?i=0.831
Comment: Using rapidly convergent alternating series for $\zeta(3)$ from [Apéry's constant - Wikipedia](https://en.wikipedia.org/wiki/Ap%C3%A9ry%27s_constant) and $C$ from [Catalan's constant - Wikipedia](https://en.wikipedia.org/wiki/Catalan%27s_constant),
it might not be too torturous to prove by exact calculation:
$$
\zeta(3)(2C-1)>
\frac52\left(\sum_{k=1}^6\frac{(-1)^{k-1}}{k^3f(k)}\right)
\left(1+\sum_{k=1}^5\frac{(-8)^k(3k+2)}{(2k+1)^3f(k)^3}\right) > 1,
$$
where $f(k) = \binom{2k}{k}.$ Rather you than me, though.
Comment: @ClaudeLeibovici On one side the value is showed to be irrational and on the other hand the other it irrationality remains unproven . Yes that's interesting .
Comment: Accident ? Interesting
Comment: I am looking for a good lower bound of $C$, which should give $\zeta(3)(2C-1)>1$.
Comment: I'm curious, where does the "one hint" come from? Is the upper limit of the summation being $100$ sharp?
Comment: @ChrisCulter If we integrate one of the side we get the Catalan's constant and the limit 100 is an example where I have done the calculation with Desmos .
Comment: @CalumGilhooley It seems promising and feel free to post it if you have some proof .Thanks for your comment .
Comment: All I did was use a pocket calculator to check that in both series, 6 terms suffice, so the second inequality is true, as well as the first. But all the tricks I could think of to simplify the hand computation lost too much accuracy. I wasn't obsessed enough to do it without simplifications, although I think it would be *possible* to do so.
Comment: For the update 3 nota bene : $x,a\in[0,\pi/2)$ then we have : $$f\left(x\right)=\frac{\ln\left(\cos\left(\frac{x}{2}\right)\right)-\ln\left(\cos\left(\frac{a}{2}\right)\right)}{\cos\left(x\right)-\cos a},f(x)\geq -\ln\left(\cos\left(\frac{ax}{1+3a}\right)\right)+f\left(0\right)$$
Comment: We have better result with : $a,x\in[0,\pi/2)$ $$f\left(x\right)=\frac{\left(\ln\left(\cos\left(\frac{x}{2}\right)\right)-\ln\left(\cos\left(\frac{a}{2}\right)\right)\right)}{\cos\left(x\right)-\cos\left(a\right)},f(x)\geq \cosh\left(\frac{x}{4}\right)-1+f\left(0\right)$$
Comment: @CalumGilhooley It seems too hard I give up for a moment .
Comment: I shall concentrate my effort on https://mathworld.wolfram.com/InverseTangentIntegral.html
Comment: @CalumGilhooley I have finished !
Comment: Cross posted https://mathoverflow.net/questions/446888/an-inequality-related-to-catalans-constant-and-zeta3
Comment: $$\int_{0}^{1}\left(\frac{\left(\arctan\left(x\right)\right)^{2}}{x^{2}}-\arctan\left(x\right)^{2}\right)dx=2C-\frac{\pi^{2}}{8}$$
Comment: $$\int_{0}^{1}\left(\frac{\left(x-\frac{x^{3}}{3}\right)^{2}}{x^{2}}-\left(x-\frac{x^{3}}{3}\right)^{2}+\frac{1}{3}x^{4}\left(1-x\right)+\frac{1}{2.5}x^{5}\left(1-x\right)^{2}+\frac{5}{20}x^{5}\left(1-x\right)^{3}\right)dx$$ and so on and $\pi<355/113$
Answer: I put here a failed attempt but if we find a refinement for Cauchy-Schwarz integral inequality perhaps it can be done :
let :
$$f\left(x\right)=\frac{x}{\cosh\left(x\right)}-\frac{x^{c^{\frac{1}{c^{d}}}}e^{-\frac{1}{c}x^{c}}}{\int_{0}^{\infty}t^{c^{\frac{1}{c^{d}}}}e^{-\frac{1}{c}t^{c}}dt},b=\frac{\left(\int_{0}^{\infty}f\left(ax\right)dx\right)}{\int_{0}^{\infty}f\left(x\right)dx},d=2,c=1.27,a=0.95$$
Then we have as numerical value :
$$1<1+\frac{\left(1-\left(\frac{\left(\int_{0}^{\infty}\left(\frac{\frac{x^{2}}{e^{x}-1}\left(f\left(ax\right)\right)}{2}\right)^{\frac{1}{2}}dx\right)^{2}}{b}\right)\right)}{6}<(\zeta(3))(2C-1)$$
In fact using Callebaut's inequality in a continuous form we have :
$\exists v\in[0.25,0.5)$ such that :
$$\int_{0}^{\infty}\left(\frac{x^{2}}{e^{x}-1}\right)^{-v+\frac{1}{2}}\left(\left(f\left(ax\right)\right)\right)^{v+\frac{1}{2}}dx\cdot\int_{0}^{\infty}\left(\frac{x^{2}}{e^{x}-1}\right)^{v+\frac{1}{2}}\left(f\left(ax\right)\right)^{-v+\frac{1}{2}}dx\cdot\frac{1}{2b}=1$$
Then we can separate the problem in two with $d,c,a$ as above and $v=0.48$ we have :
$$b^{\sqrt{\frac{77}{16}}}\int_{0}^{\infty}\left(\frac{x^{2}}{e^{x}-1}\right)^{-v+\frac{1}{2}}\left(\left(f\left(ax\right)\right)\right)^{v+\frac{1}{2}}dx>1$$$$\int_{0}^{\infty}\left(\frac{x^{2}}{e^{x}-1}\right)^{v+\frac{1}{2}}\left(f\left(ax\right)\right)^{-v+\frac{1}{2}}dx\cdot\frac{1}{2b^{\left(1+\sqrt{\frac{77}{16}}\right)}}>1$$

Some numerical result we have :
$$\left(3\int_{0}^{\left(2-\sqrt{3}\right)}\frac{x-\frac{x^{3}}{3}+\frac{2}{15}x^{5}+7x\left(-\frac{x^{3}}{3}+\frac{2}{15}x^{5}\right)^{2}}{x}dx-\frac{1}{4}\pi\ln\left(2-\sqrt{3}\right)-1\right)\zeta(3)>1$$
$$\zeta(3)>1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{1+\frac{1}{1+\frac{1}{1+\frac{1}{4+\frac{1}{1+\frac{1}{9+\frac{1}{9}}}}}}}}}}=177658/147795\tag{I}$$
Proof by hand of $I$
It's not hard to see as the number of fraction is even or odd then it's greater or less than the Apery's constant so we have :
$$\zeta(3)>1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{1+\frac{1}{1+\frac{1}{1+\frac{1}{4+\frac{1}{1+\frac{1}{9+\frac{1}{9}}}}}}}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{1+\frac{1}{1+\frac{1}{1+\frac{1}{4+\frac{1}{1+\frac{1}{\frac{82}{9}}}}}}}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{1+\frac{1}{1+\frac{1}{1+\frac{1}{4+\frac{1}{\frac{91}{82}}}}}}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{1+\frac{1}{1+\frac{1}{1+\frac{1}{\frac{446}{91}}}}}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{1+\frac{1}{1+\frac{1}{\frac{537}{446}}}}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{1+\frac{1}{\frac{983}{537}}}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{1+\frac{1}{18+\frac{1}{\frac{1520}{983}}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{1+\frac{1}{\frac{28343}{1520}}}}$$
Or :
$$1+\frac{1}{4+\frac{1}{\frac{29863}{28343}}}$$
And at the end :
$$1+\frac{1}{\frac{147795}{29863}}=177658/147795$$
Doing the same matter for the Catalan's constant we have :
$$C>\frac{1}{1+\frac{1}{10+\frac{1}{1+\frac{1}{8+\frac{1}{1+\frac{1}{88}}}}}}=\frac{9690}{10579}$$
So we have :
$$(2\frac{9690}{10579}-1) 177658/147795=\frac{1563568058}{1563523305}>1$$
Comment: Aren't [A013631 - OEIS](https://oeis.org/A013631) "Continued fraction for zeta(3)" and [A014538 - OEIS](https://oeis.org/A014538) "Continued fraction for Catalan's constant" themselves computed by unspecified numerical procedures? If so, then evaluating their convergents does not constitute an independent proof of the inequality.
Comment: We have $$\zeta(3)>\frac{1}{2}\int_{0}^{\infty}x^{2}\left(\sum_{n=1}^{k}e^{-nx}\right)dx$$
Comment: It seems that $b=19/20$.
Comment: @CalumGilhooley So let's it for the future .
Comment: Correction it seems we have $b=20/19$
Answer: Using, as @Calum Gilhooley suggested, the truncated series representation of the constants (these are generalized hypergeometric functions)
$$C_p=\sum_{k=1}^p\frac{4^{4 k-3} \left(580 k^2-184 k+15\right)}{k^3 (2 k-1)
   \binom{4 k}{2 k} \binom{6 k}{3 k} \binom{6 k}{4 k}}$$
$$\zeta_p(3)=\frac 1{64}\sum_{k=1}^p \frac{(-1)^k \left(205 k^2+250 k+77\right) (k!)^{10}}{((2
   k+1)!)^5}$$
Define
$$\Delta_p=\zeta_p(3)\,(2C_p-1)-1$$
$$\Delta_2=\frac{67988471}{29638224000000}=2.29395\times 10^{-6}$$
$$\Delta_=\frac{2311829612939580593}{81097164321132960000000}=2.85069\times 10^{-5}$$
Tedious (but doable) : using the "derivative" of the resulting  generalized hypergeometric functions, we can show that
$$\log(\Delta_{p+1}-\Delta_p)\sim 4.45 -5.22 \, p$$
Comment: Thankks you Claude ! I try to do better now...
Answer: Not a complete answer, but this approach relies on a less numerical (albeit there is still a lot of tedious algebra involved) method -- there may even be hope to compute a closed-form for $(2C-1)\zeta(3)$. The method of proof is similar in flavor to the famous $\frac{355}{113} > \pi$.
Let
\begin{align*}
I(p) &= \int_0^1 p(x)\frac{(-\log x)^{3/2}(2\frac{1-x}{1+x})^{1/2}}{x^2+1}dx \\
J(p) &= \int_0^1 p(x)\frac{(-\log x)(2\frac{1-x}{1+x})}{x^2+1}dx \\
K(p) &= \int_0^1 p(x)\frac{(-\log x)^2}{x^2+1}dx
\end{align*}
defined on $\mathbb{Q}^*=\{p \in \mathbb{Q}[x]: p(x) \ge 0\}$. By Cauchy-Schwarz, $I \le \sqrt{JK}$. One can then show, for any $k \in \mathbb{N}$, that
\begin{align*}
J(x^k) &= \alpha_1 + \alpha_2C + \alpha_3\pi^2 \\
K(x^k) &= \beta_1 + \beta_2\xi(3)+ \beta_3\pi^3
\end{align*}
for rationals $\alpha_1, \alpha_2, \alpha_3, \beta_1, \beta_2, \beta_3$. My idea to prove the inequality in this post is then to identify a $p \in \mathbb{Q}^*$ such that
\begin{align*}
J(p) = k_1(2C-1) \qquad \text{and} \qquad K(p) = k_2 \zeta(3)
\end{align*}
for which then $(2C-1)\zeta(3) \ge (\frac{I(p)}{k_1k_2})^2 > 1$. This is guaranteed to work, because $\sqrt{JK} - I$ can be made arbitrarily close to 0 by considering $p(x) = x^n q(x)$ and letting $n\rightarrow \infty$.
[The reason why this works, and also the motivation behind choosing the integrands of the functions, is the equality condition of Cauchy-Schwarz $\|fg\|_2^2 \le \|f\|_2\|g\|_2$ is $f \propto g$; in this context, $-\log(x) \propto 2\frac{1-x}{1+x}$. The latter is obviously not true, but $\lim_{x\rightarrow 1} \frac{-\log x}{2\frac{1-x}{1+x}} = 1$, and the discrepant portions are made arbitrarily small by multipling by $x^n$.]
The trouble is now finding this $p$, which will involve computer algebra. I've attempted finding lower-ordered $p$'s and found
\begin{align*}
p(x) = x\left(\frac{3456}{7}x^4 + \frac{11736}{35}x^3 + \frac{34928}{105}x^2+
\frac{11736}{35}x - \frac{2336}{15}\right)
\end{align*}
which gives $I(p) = \frac{16}{3}(2C-1)$ and $J(p) = \zeta(3)$; unfortunately, $p \notin \mathbb{Q}^*$ since $p$ can be negative. But, I'm putting this method out there for others to find higher-degree polynomials (and in the original $\frac{355}{113} > \pi$ integral proof, the authors had to consider a degree-18 polynomial).
Comment: Well this is interesting !
Comment: Have you tried something like ? $$\int_{0}^{1}\int_{0}^{1}\left(-\frac{1}{2}\frac{\ln\left(xz\right)}{1-xz}+\left(f\left(x\right)-f\left(z\right)\right)\right)dxdz=\zeta(3)$$ with Cauchy-Schwarz
Answer: Too long for a comment :
If we show with Callebaut inequality :
$$\frac{9}{8}\int_{0}^{\frac{\pi}{2}}\left(\left(\left(-\frac{\ln\left(\tan\left(x\right)\right)}{4}+\ln\left(\sec\left(x\right)+\tan\left(x\right)\right)-\frac{x}{\int_{0}^{\frac{\pi}{2}}tdt}\right)\cdot\frac{4}{7}\left(x\ln\left(\sec\left(x\right)+\tan\left(x\right)\right)\right)\right)^{\frac{1}{2}}\right)dx<\int_{0}^{\frac{\pi}{2}}\frac{4}{7}\left(x\ln\left(\sec\left(x\right)+\tan\left(x\right)\right)\right)dx\int_{0}^{\frac{\pi}{2}}\left(\ln\left(\sec\left(x\right)+\tan\left(x\right)\right)-\frac{x}{\int_{0}^{\frac{\pi}{2}}tdt}\right)dx$$
Then one can show that :
$$\frac{9}{8}\int_{0}^{\frac{\pi}{2}}\left(\left(\left(-\frac{\ln\left(\tan\left(x\right)\right)}{4}+\ln\left(\sec\left(x\right)+\tan\left(x\right)\right)-\frac{x}{\int_{0}^{\frac{\pi}{2}}tdt}\right)\cdot\frac{4}{7}\left(x\ln\left(\sec\left(x\right)+\tan\left(x\right)\right)\right)\right)^{\frac{1}{2}}\right)dx>1$$
It seems promising to be continued ...
I think I'm not the first who find it so if someone have a reference...Anyway I set it in community wiki .

Using Bernoulli's numbers , geometric series we have a rought approximation for $\zeta(3)$ we have :
$$\zeta(3)\simeq \int_{0}^{2}e^{-x^{2}}x^{3}\left(1+\frac{x^{2}}{2}+\frac{\frac{1}{6}x^{4}}{2}-\frac{1}{30\cdot24}x^{8}+\frac{1}{42\cdot720}x^{12}\right)dx+\int_{2}^{\infty}x^{5}\left(e^{-x^{2}}+e^{-4x^{2}}\right)dx$$
Proof :
We have as approximation :
$$\frac{1}{2}\int_{0}^{3}e^{-x}x\left(1+\frac{x}{2}+\frac{\frac{1}{6}x^{2}}{2}-\frac{1}{30\cdot24}x^{4}+\frac{1}{42\cdot720}x^{6}-\frac{1}{30\cdot8!}x^{8}\right)dx+\frac{1}{2}\int_{3}^{\infty}x^{2}\left(e^{-x}+e^{-2x}+e^{-3x}+e^{-4x}\right)dx<\zeta(3)$$
The proof can be done by hand .
The first integral is equal to : $$11/10+148087/(22400\exp\left(3\right))$$
The second integral is equal to :
$$1/2\left(85/(32e^{12})+101/(27e^{9})+25/(4e^{6})+17/e^{3}\right)$$
Now starting from my previous answer we have :
$$\left(3\int_{0}^{\left(2-\sqrt{3}\right)}\frac{x-\frac{x^{3}}{3}+\frac{2}{15}x^{5}+7x\left(-\frac{x^{3}}{3}+\frac{2}{15}x^{5}\right)^{2}}{x}dx-\frac{1}{4}\pi\ln\left(2-\sqrt{3}\right)-1\right)<C$$
The integral is equal to :
$$(151012882-87183909\sqrt{3})/7425$$
Now we need some approximation of roots,logarithm,$\pi$,$e$
To be continued.
Comment: I forgot a square and hopefully it changes nothing to the validity of the result .
Comment: We can use   https://en.m.wikipedia.org/wiki/Van_Wijngaarden_transformation and $$e^x=1/(1-x+x^2/2+...+(-1)^nx^n/n!+...)$$
Comment: We can use for $x$ small $$\frac{\frac{12}{5}^{x}+3^{x}}{2}\simeq e^x$$
Answer: §1. No solution but just an idea suggested by the inverse of $\zeta(.)$ and not mentioned before here: take the Euler product.
Euler's famous formula reads in our case
$$\frac{1}{\zeta(3)}= \prod_{i>=1}(1-\frac{1}{p_i^3})$$
here $p_i$ is the $i$-th prime number ($p_1=2$, $p_2=3$, ...).
We can write
$$\frac{1}{\zeta(3)}= \prod_{i=1}^{17}(1-\frac{1}{p_i^3})\times\prod_{i>=18}(1-\frac{1}{p_i^3})<\prod_{i=1}^{17}(1-\frac{1}{p_i^3}) $$
so it is sufficient to show that
$$\prod_{i=1}^{17}(1-\frac{1}{p_i^3}) < 2C-1$$
I have chosen the sub product from numerical considerations.
Now this might help if we had a product form of $C$ ...
Sorry, my wisdom ends here.
§2. Numerics
Truncated series play a role in some of the solutions provided so far
Approximating $C$ and $\zeta$ in the product
$$p = (2C-1)\zeta(3)$$
by its naive series, i.e.
$$p(m) :=\left(\sum _{n=1}^{m} \frac{1}{n^3} \right)\left(2 \sum _{n=1}^{m} \frac{(-1)^{n+1}}{(2 n-1)^2}-1\right)$$
where $m$ is a truncation index, we find the point $m_1$ where $p(m_1)=1$ is $m_1=61$ for odd $m$ and $m_1=158$ for even $m$.
Answer: Cutting Gordian knot by Mathematica:
N[1/Zeta[3]]

0.831907

N[2*Catalan - 1]

0.831931
Comment: Where is the proof ?
Comment: @EricSatie: See numerical methods to this end.
Comment: this is the best answer. Why the downvotes?

Show that : $\frac{1}{\zeta(3)}<2C-1$


Efficient way to get first value larger than x in a list?

Question: I have two sorted arrays, Haystack and Needles. I need to iterate over Needles, and each time find the first point in Haystack with a value larger than that Needle, in order to perform the next step.
For example:
double [] dHaystack = { 1.2, 2.6, 7.0, 9.3, 19.4 }
double [] dNeedles  = { 1.4, 6.4, 6.5, 7.0, 10.3 }

//  expected indices     0    1    1    2    3

So the index I should get is the first index equal to or lower than the needle value.
The obvious approach is just to iterate from the beginning of the haystack for each needle, or to iterate onward from the last found index (as Needles is also sorted).
But part of my brain is shouting "bisection!". Would a bisection actually be faster here, since the compiler will find it harder to optimise than a simple block read and iteration? Would it need an incredibly long Haystack to be worthwhile?
Comment: My brain is shouting "example", without which there is no point in thinking what to optimize. Other then then, searching from the last found seams good. I wouldn't try to out smart the compiler. Usually it is very good
Answer: You need to consider the scenario,
n*lg(m) < n+m,
Where n is the size of Needle and m is the size of Haystack.
Therefore, it all depends on the various combination of values of n and m.
Answer: Use std::upper_bound, which is O(log n) for random access iterators and provides exactly what you need in the shortest and simplest code.
Before you worry about minute performance, test your current code (and maybe test alternatives) instead of making assumptions.  In particular, note you can start searching (the first parameter to upper_bound) from the last found index on each iteration.
// Available in Boost, C++0x, and many other places.  Implementation copied
// here for the sake of the example.
template<class T, int N>
T* end(T (&a)[N]) {
  return a + N;
}

void example() {
  double haystack[] = {1.2, 2.6, 7.0, 9.3, 19.4};
  double needles[] = {1.4, 6.4, 6.5, 7.0, 10.3};
  double *begin = haystack;
  for (double *n = needles; n != end(needles); ++n) {
    double *found = std::upper_bound(begin, end(haystack), *n);
    if (found == end(haystack)) break;
    std::cout << *n << " at index " << (found - haystack) << '\n';
    begin = found;
  }
}

Answer: std::upper_bound will give you the iterator to first element strictly greater, or "end" of the collection if none of them apply
upper_bound takes iterators for begin and end, end being one past the end of the collection. If you are iterating through an increasing list of search values, you would of course not need to run through the entire collection but your "begin" can shift further to the right.
Of course with a haystack of just 5 elements it doesn't really matter what search algorithm you use, but if it got very large, using a linear search would be potentially very slow, particular if there were very few needles.
This is a situation where it does really matter what both sizes are. For example if your search space N is large but the number of items being searched (M) is small then O(M log N) really is a lot smaller. (eg M = 20, N = 16K then log N = 15 and M log N is 300) compared with O(M + N) which in this case is 16K. If M is approximately the same size as N then O(M log N) is effectively a lot worse than O(N).
Therefore depending on the sizes of your collections you can choose which algorithm to use.
Comment: The PO knows this. He just wants to know when bisection will be worthwhile
Comment: I have answered that. I was just editing my answer after giving the first statement.
Answer:
The obvious approach is just to iterate... onward from the last found index (as Needles is also sorted).

Yes.

But part of my brain is shouting "bisection!". Would a bisection actually be faster here, since the compiler will find it harder to optimise than a simple block read and iteration? Would it need an incredibly long Haystack to be worthwhile?

I don't think the compiler optimisation's an issue (it just removes unnecessary work) so much as the amount of actual inherent, necessary work.  If both sets are similar in size then I'd stick with the obvious approach.  If the haystack is massively larger than the needles set, then bisection or even interpolation might yield slightly better performance.  Unless this is crucial to your app you're unlikely to notice the differences, and if it is you should benchmark, particularly as you can presumably get a working implementation quickly using std::set and upper or lower bound (I can never remember which I'll need - don't use the often enough), maybe using the last position as a hint at starting location if your library supports that.


To refute : a function with one discontinuity point is integrable on $\left[0, 1\right]$

Question: If, let $f: \left[ 0, 1\right] \to \mathbb{R}$ continuous with only one discontinuity point is integrable on $\left[ 0, 1 \right]$.

I think this is false but I can't find a example that contradicts the statement.
Comment: Was there any mention as to whether the function is assumed to be bounded?
Comment: Wouldn't it be true? If f is a part of real space from [0, 1] than you couldn't have a value which was not in real space, which is irrational, rational, integers, etc. numbers. You are saying it is continuous from [0, 1], and you are integrating it on [0, 1], and a property of integrals is if it is continuous on an interval you can integrate it on that interval.
Answer: Define $f:[0\,,1]\to\Bbb R$ by $f(0)=0$ and $f(x)=1/x$ for $x\in (0\,,1]$. The integral diverges, because $\ln |x|\to-\infty$ as $x\to0$. For an example not involving the logarithm, replace $1/x$ by $1/x^2$; the integral diverges in this case because $1/x\to\pm\infty$ as $x\to0$.
Comment: Remember me why this function isn't integrable. By Darboux?
Comment: @hlapointe: I expanded my answer; I hope that helps.
Comment: Yes, thanks you.
Comment: If I have to prove it, should I use the Darboux integral?
Comment: @hlapointe: The integral is not definable (i.e. it diverges)  whatever definition of integration one uses: put simply, the area bounded by the hyperbola $xy=1$ and the $x$ and $y$ axes is not defined (it is infinite).
Comment: For most presentations of the Riemann and improper Riemann integrals one defines such an integral as $$\int_0^1 \frac1x \,dx= \lim_{t\to0+} \int_t^1 \frac1x \,dx$$ and the understanding is that the integral $\int_t^1$ on the right exists (since $f(x)=x^{-1}$ is continuous on $[t,1]$) but that the limit fails to exist.  That may seem overly pedantic but on first learning such a subject it is best to rely on definitions until your intuition develops. [Since this function is unbounded on $[0,1]$ there is no Riemann or Darboux integral possible, only this "improper" one.]
Comment: @B.S.Thomson: Your improper integral is not defined, because the limit does not exist. I don't see what point you are making here.
Comment: The point is that this is the actual definition.  The fact that the limit fails to exist is your proof that the improper Riemann integral does not exist. (This is implicit in what you did, but not completely transparent for the poster.)  If you wish to prove this function does not have a Lebesgue integral (among others) you would have to come up with a different proof.  If you wish to prove that the function is not Riemann integrable, you would claim that it is unbounded and hence not Riemann integrable.


Is order of evaluation gauranteed with short circuit evaluation?

Question: Is the left part of the && always evaluated before the right part?
I want to know because I'm wondering if I can change
if(i > 0)
    if(someFunc(arr[i-1], arr[i]))
        //do work

to
if(i > 0 && someFunc(arr[i-1], arr[i]))
or would this cause undefined behaviour if the right side gets evaluated first and arr[0-1]is referenced?
Comment: yes, guaranteed by C.
Answer: Yes, because of short-circuit behavior of logical && operator, in case of && second expression evaluates only  when first is true. Read following:

6.5.13 Logical AND operator
4 Unlike the bitwise binary & operator, the && operator guarantees left-to-right evaluation;
  there is a sequence point after the evaluation of the first operand. If the first operand
  compares equal to 0, the second operand is not evaluated.

From Is short-circuiting boolean operators mandated in C/C++? And evaluation order?
So  if i > 0 is false (for example if i = 0) then result of i > 0 expression will be false, and then second operand  someFunc(arr[i-1], arr[i] will not be called (evaluated).
Accordingly,  if(i > 0 && someFunc(arr[i-1], arr[i])) is safe to code, but be-careful i - 1 shouldn't be > max index value of arr[]. Really I will prefer this form of if compare to nested if blocks (flat is better then nested).
From @Maroun Maroun's answer "Is there any reason for asking if(1 || Foo())?" Additional information that might help you:

if(a && b) - if a is false, b won't be checked.
if(a && b) - if a is true, b will be checked, because if it's false, the expression will be false.
if(a || b) - if a is true, b won't be checked, because this is true anyway.
if(a || b) - if a is false, b will be checked, because if b is true then it'll be true.
Answer: Yes. Order of evaluation for the operators &&, ||, , and ? :(ternary operator) are guaranteed from left to right. A sequence point occurs between the left and right sub-expression of these operands.  All the side effects of the left sub-expression of these operators (&&, || and ,) are completed before any access to its right sub-expression.

would this cause undefined behavior if the right side gets evaluated first and arr[0-1] is referenced?

Yes. If that were the case, it would cause undefined behavior. (But it doesn't happen that way, so there won't be any undefined behavior.)


Absolute value inequality $3 > |x + 4| \geq 1$

Question: I've just started with absolute value equations and I have a real hard time understanding how to solve this. I got the following question, and I can't make heads or tails out of it.
Assume that $x, y$ are points on the real line. Explain what $|x − y|$ means geometrically. Use this to illustrate the inequalities (as subsets of the real line) below and write the inequality without absolute values.
$3 > |x + 4| \geq 1$
All help will be much appreciated, even as much as small pointers will be more than welcome...
Comment: Ok, If you have just started with absolute values, is this the first equation you were confronted with??
Comment: In this particular setup yes, how would one illustrate the inequalities is what I have a hard time to comprehend.
Comment: Apart from the other suggestions below, do you know how to graph the function$y=|x+4|$ in the $xy$ plane?
Answer: $|x-y|$ is the distance between $x$ and $y$ on a number line.
Answer: Building up on what Aleksandar already said. Your inequality can be written as:$$1\le|x-(-4)|<3$$
So geometrically, $x$ are the numbers on the real line such that they are at least a distance 1 from -4 and at most (but not equal to) a distance 3 from -4. So $$-7<x\le-5$$ or $$-3\le x<-1$$
A more systematic way of doing such questions is to consider the 2 possibilities i.e $1\le x+4<3$ and $1\le -(x+4)<3$ and simplying should give the same answer as above.


Where are the Hive Table/Database settings?

Question: i need to find the configuration files in hdfs for a Hive Table. I have found the Table data with "describe formatted ;" but where are the col_name or data_type informations stored?
Answer: This is stored in the metastore database, but I would recommend not to edit it directly.
You should use
<code>DESCRIBE TABLE mytable;
</code>
with possible options as advised
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Describe


JavaScript page refresh issue

Question: Is there any way to handle page refresh in JavaScript ? After browser close button is clicked my application should be logged out and after clicking on browser back button application should be logged out also.
Comment: perhaps the onbeforeunload event (not likely though), or sendBeacon on modern browsers could help
Comment: what if user's browser has javascript disabled?
Answer: Try this snippet by clicking run and reload or backwards in your browser.

window.onbeforeunload = function(e) {
  //call your function
  return "Attention Reload"
};

Have a look at this eventHandler
Answer: Try This Answer
This answer will help you out
Also this only shows how to access the back button functionality in cross browser, However in the scope of those functions,
You can destroy your token or whatever to logout or you can directly use window.location.href, description of the same is given in Window.location.href
Hope that helps, Also if this does not solve specify more details in question that how are you logging out,
Thanks & Regards
Shohil Sethia

JavaScript page refresh issue


Support vector regression on skewed/high kurtosis data

Question: I'm using support vector regression to model some fairly skewed data (with high kurtosis). I've tried modeling the data directly but I'm getting erroneous predictions I think mainly due to the distribution of the data, which is right skewed with very fat tails. I'm pretty sure a few outliers (which are legitimate data points) are affecting the SVR training, and perhaps also in the cross validation, where at the moment I'm optimizing the hyperparameters by minimizing mean-squared error.
I've tried to scale my data before applying SVR (e.g. using a sqrt function to reduce the outliers) as well as use a different hyperparameter minimization function (e.g. absolute error), which seems to give better results, but still not very good. I'm curious if anyone has encountered similar problems and how they approached it? Any suggestions and/or alternate methods most welcome.
Comment: if variables are positive than applying log (a special case of Box-Cox transformation) can be helpful
Comment: Thanks Yevgeny. The variables are unfortunately both positive and negative. Is there still a way to apply a power transform or Box-Cox transform?
Answer: You can use skewed or heavy-tailed Lambert W distributions to transform your data to something more well-behaved (disclaimer: I am the author of both papers and the LambertW R package). The advantage over the Box-Cox transformation is that they do not have any positivity restriction, the optimal parameters of the transformation can be estimated (MLE) from the data, and you can also forget the transformation and model your data as a Lambert W x F distribution directly.
The LambertW R package provides several estimators, transformations, methods, etc. I especially recommend a look at
   Gaussianize()
   IGMM()
   MLE_LambertW()

The skewed Lambert W x F distribution is a general framework to make a skewed version of any distribution F. Conversely you can then make your skewed data again symmetric; the distribution of this symmetrized data basically determines what kind of Lambert W x F you have; if the data is just a bit asymmetric, then you might have a skewed Lambert W x Gaussian; if your data is additionally heavy-tailed maybe you can try a skewed Lambert W x t.
Heavy-tailed Lambert W x F are a generalization of Tukey's h distribution, and they provide an inverse-transform to make data Gaussian (also from asymmetric). In the paper I demonstrate that even a Cauchy can be Gaussianized to a level that you - and also several Normality tests - can't distinguish it from a Normal sample.
Comment: Welcome to our site, Georg!
Answer: One way to deal with negative values is to shift variables to the positive range (say to greater or equal to 0.1), apply Box-Cox transform (or just log() for a quick test), and then standardize. The standardization can be important for SVR since SVR relies on quadratic penalty applied to all coefficients uniformly (so SVR is not scale invariant and can benefit from variable standardization). Make sure to check the resulting variable distributions - they should not be skewed much (ideally they should look Gaussian)
Another technique one could try is to apply "spatial sign" transformation to the input vectors x <- x / norm(x) as per "Spatial sign preprocessing: a simple way to impart moderate robustness to multivariate estimators". J. Chem. Inf. Model (2006) vol. 46 (3) pp. 1402–1409 I did not have much luck with this technique though but the mileage may vary.
Comment: Thanks Yevgeny. I'll take a look into it. I was thinking about using an inverse hyperbolic sine transformation as it was a technique that seemed to work for both positive and negative values. Unfortunately I am unsure how to estimate the theta parameter. Are you familiar with this at all? Thanks.
Comment: I've seen this (parameter free) formula: x <- log(x + sqrt(x^2 + 1))
Answer: One way to approach the solution would be building two models: one for the values which are in line with the distribution and other for the outliers. My suggestion in this regard would be to create a binary response variable (0,1) with 0 being the value if the datapoint is within the bounds of your distribution and 1 if it lies outside. So for the cases of the outliers which you want to keep in your data, you will be having 1 in your target variable and the rest as 0. Now run a logistic regression to predict the probabilities of the outliers and you can multiply the average value for the group of outliers with the individual probabilities to get the predictions. For the rest of the data, you can run your SVM to predict the values.
Because the values are outliers, they will have low probabilities associated with it and even if you take mean of the outliers which will be skewed, the expected value of the outliers will be pulled down by their attached low probabilities and there by making it a more reasonable prediction.
Had met with a similar scenario while predicting claims amount for an Insurance service provider. I had used the above-mentioned technique to increase the performance of my model drastically.
Another way could be taking log transformation  of your target variable which is possible if you have only positive value in your target variable. But make sure if you are taking a log transformation of your target variable, while predicting the variable you need to include the error component as well.
So, $\log(Y) = a + B'X + \epsilon$ is your model equation for e.g
then, $Y = \exp(a+B'X+\epsilon)$
You can take a look into the following link for log-transformation:
http://www.vims.edu/people/newman_mc/pubs/Newman1993.pdf


PHP Saving and outputting GD Lib image - relative and absolute paths

Question: I have a php script which generates an image using GD Lib, saves it to a predefined location. And then should output it.
My directory structure is like this:
www.example.com/projects/project-1/

Inside the project-1 directory I have these directories:
- /imgs/
- /js/
- /css/
- /php/

The script using GD Lib is in /php/ with another config.php script where the constants are defined. This is then included in the main script.
Say I have two constants:
define('SAVE_PATH', '/projects/project-1/imgs/'); //The image will not save - this does not work
define('OUTPUT_PATH', '/projects/project-1/imgs/'); //this works if there is an image in this location

I then save the image like so:
imagejpeg($im, SAVE_PATH.$name, 100);

I get the following error:
failed to open stream: No such file or directory in /public_html/projects/project-1/php/main.php

Is it possible to do this with just one constant that works for both saving and outputting?
I know I can't have an absolute save path like: http://www.example.com/projects/project-1/imgs/
And I know I can't have an absolute output path like: /public_html/projects/project-1/imgs/
So what is the most elegant solution to this problem?
Answer: Your issue is likely due to using absolute pathing in your SAVE_PATH.  An absolute path begins with /, and a relative path does not.  These will probably work:
define('SAVE_PATH', '../imgs/'); //path relative to the php script, assuming the php file isn't being included from another path
define('SAVE_PATH', '/public_html/projects/project-1/imgs/');

BUT
In efforts to make this more versatile, I would instead do the following.  Set 2 constants, one for the base application directory, and one for the path to the image folder.  The latter will be usable in conjunction with the former for the save path, and will be usable on its own as the output path:
//const instead of define is a little prettier, but this is a preference
const APPLICATION_PATH = '/projects/project-1';
const IMAGE_PATH = '/imgs/';

//Save the image - Absolute path to the file location
imagejpeg($im, APPLICATION_PATH.IMAGE_PATH.$name, 100);

//Echo the image url - Absolute path to the file url
echo "";

You now only need to edit IMAGE_PATH to make changes to where the images reside which will appropriately affect both the system directory and the web url.


How do you increase the default volume of MuseScore?

Question: I am using the MuseScore music notation program on Windows 7.
For audio playback in MuseScore, I am using the default General MIDI virtual instrument sample library in Windows 7 [the Microsoft GS Wavetable Synth from Windows Media Player].
The overall volume output is too low, so I have to turn my computer volume way up.
I'm not the only person with this issue as seen here
I found that you can up the volume in the Mixer, but I shouldn't have to max out every instrument and lower every other program's volume, so is there a way to up the output of MuseScore so that it's a decent level from the start?
*(extra)*Can I switch from using the Microsoft GS Wavetable Synth from Windows Media Player to some other General MIDI virtual instrument or orchestral library, such as a VST sample library of some kind? If so what do you recommend for each price range?
Comment: I'm using Windows 7 :)
Comment: When I did a Google search, I learned that many users on Windows have different suggestions for replacing the Microsoft General MIDI instrument with another freeware or inexpensive sound font or other kind of virtual instrument. With the info I've provided you should be able to search for a better alternative to get better-quality sounds.
Comment: It would be good to get that as a suggestion in answers, is it worth me rewording the question to make it more open than MuseScore?
Comment: Why not post your own separate question, for instance, "How can I replace the default General MIDI instrument in Windows with something better?" But you should not post it on this StackExchange site. That would be off-topic. Try the Audio-Video Production stack at http://avp.stackexchange.com/?as=1
Comment: I'm surprised I couldn't find this question already on there: http://avp.stackexchange.com/questions/8713/how-do-you-change-the-default-windows-midi-with-something-better
Answer: No, you don't want the mixer (F10). You want the master volume. This is found in the play panel (F11). This automatically saves your setting so that you don't have to adjust it every time. You can also find a master volume slider in the Synthesizer panel (Display > Synthesizer).
The other option is to adjust the program's relative volume in Window's volume mixer (not sure if there is a similar feature on Macs). This also seems to remember the last used settings so you're not resetting it every time you use MuseScore.


Using Multiple SUMIFS in VBA

Question: How would one go about changing this formula of sumifs into an VBA macro.
=SUMIFS('Staff Allocation'!N3:N6,'Staff Allocation'!B3:B6,Sheet2!A1,'Staff Allocation'!E3:E6,Sheet2!B1,'Staff Allocation'!F3:F6,Sheet2!C1)
 +SUMIFS(Modules!H2:H4,Modules!B2:B4,Sheet2!A1,Modules!G2:G4,Sheet2!B1,Modules!E2:E4,Sheet2!C1)

So it can be used in an if formula not =0 then do a specified action else move to the next process.
A1 is the first criteria value,
B1 is the second criteria value,
C1 is the third criteria value
Comment: Have you tried the macro recorder?
Answer: You use Application.WorksheetFunction to access worksheet functions.
According to the documentation for SumIfs you need to provide your arguments as Range objects.
So you need to take each range you want to evaluate and declare it as a range object.  In Excel, the Range object is a member of the Worksheet object so you need to specify which worksheet the range is on.   You can do this by specifying the name or index number from the Sheets Collection.
E.g. Sheet2!A1 can be obtained using SomeRange = Sheets("Sheet2").Range("A1")
Now, the Worksheet object is a member of the Workbook Object, so you can also specify what workbook you want - but if you read through the documentation links above, you'll see that the default for the Sheets collection is the ActiveWorkbook - so we'll rely on this to keep the example simple.
So to pass all your SumIfs arguments you could do something like below.
Granted this might not be the most elegant solution, I think it should be easy enough to understand how it works.
Sub Foo()
    Dim Arg1 as Range
    Dim Arg2 as Range
    Dim Arg3 as Range
    Dim Arg4 as Range
    Dim Arg5 as Range
    Dim Arg6 as Range
    Dim Arg7 as Range
    Dim Arg8 as Range
    Dim Arg9 as Range
    Dim Arg10 as Range
    Dim Arg11 as Range
    Dim Arg12 as Range
    Dim Arg13 as Range
    Dim Arg14 as Range

    Set Arg1=Sheets("Staff Allocation").Range("N3:N6")
    Set Arg2=Sheets("Staff Allocation").Range("B3:B6")
    Set Arg3=Sheets("Sheet2").Range("A1")
    Set Arg4=Sheets("Staff Allocation").Range("E3:E6")
    Set Arg5=Sheets("Sheet2").Range("B1")
    Set Arg6=Sheets("Staff Allocation").Range("F3:F6")
    Set Arg7=Sheets("Sheet2").Range("C1")
    Set Arg8=Sheets("Modules").Range("H2:H4")
    Set Arg9=Sheets("Modules").Range("B2:B4")
    Set Arg10=Sheets("Sheet2").Range("A1")
    Set Arg11=Sheets("Modules").Range("G2:G4")
    Set Arg12=Sheets("Sheet2").Range("B1")
    Set Arg13=Sheets("Modules").Range("E2:E4")
    Set Arg14=Sheets("Sheet2").Range("C1")

    If Application.WorksheetFunction.SumIfs(Arg1,Arg2,Arg3,Arg5,Arg6,Arg6,Arg7)
     + Application.WorksheetFunction.SumIfs(Arg8,Arg9,Arg10,Arg11,Arg12,Arg13,Arg14) > 0 Then
        'Do Something
    Else
        'Do Something Else
    End IF
End Sub

Using Multiple SUMIFS in VBA


Mapbox GL JS API Set Bounds

Question: I've been trying to get my map (JS below) to be able to auto zoom to the extent of one of the JSON files used for display (there will be a number of separate ones), but I can't seem to figure out how I would achieve that.
    var map = new mapboxgl.Map({
    container: 'map', // container id
    style: 'mapbox://styles/mapbox/streets-v8', //stylesheet location
    center: [-79.376828,43.635570], // starting position
        zoom: 12,
        pitch: 07, // pitch in degrees
        bearing: -16.1// bearing in degrees
     });

 map.on('style.load', function () {

    map.batch(function (batch) {

        ////////////////////////////
        //ADDSOURCE for 0
        map.addSource("0", {
            "type": "geojson",
            "data": {
                "type": "Feature",
                "properties": {},
                "geometry": {
                    "type": "LineString",
                    "coordinates": [[-79.378821,43.645557],[-79.378862,43.645594],[-79.378909,43.645616],[-79.418429,43.635570],[-79.413358,43.636944]]

                }
            }
        });

        //ADDLAYER for 0
        map.addLayer({
            "id": "0",
            "type": "line",
            "source": "0",
            "layout": {
                "line-join": "round",
                "line-cap": "round"
            },
            "paint": {
                "line-color": "#0324a7",
                "line-width": 3
            }
        });
        ////////////////////////////
        //ADDSOURCE for 1
        map.addSource("1", {
            "type": "geojson",
            "data": {
                "type": "Feature",
                "properties": {},
                "geometry": {
                    "type": "LineString",
                    "coordinates": [[-79.418429,43.635570],[-79.413358,43.636944],[-79.413197,43.636982],[-79.378821,43.645557],[-79.378862,43.645594]]

                }
            }
        });

        //ADDLAYER for 1
        map.addLayer({
            "id": "1",
            "type": "line",
            "source": "1",
            "layout": {
                "line-join": "round",
                "line-cap": "round"
            },
            "paint": {
                "line-color": "#0324a7",
                "line-width": 3
            }
        });

    });//BATCH

});
map.fitBounds(map.getBounds());

map.addControl(new mapboxgl.Navigation());
map.scrollZoom.disable();

I have tried using various combinations of map.fitBounds(map.getBounds()), but I just can't seem how to figure out how to have the map launch to the extents of the first JSON. Any help would be greatly appreciated.
Also, should I be doing the map a different way (other than using map.batch(function (batch)) to load a number of different JSON files?
Comment: see: http://stackoverflow.com/questions/35586360/mapbox-gl-js-getbounds-fitbounds
Answer: As far as i know (and could search) there is currently no way to derive bounds from a mapboxgl.GeoJSONSource object. You'll need to calculate those yourself. After that you can use mapboxgl's fitBounds method:
map.fitBounds([
    [-79.418429, 43.63557],
    [-79.378821, 43.645616]
]);

Mapbox GL JS API Set Bounds


Dell Precision T7600 OMSA Perc H710

Question: Due to recent hardware failure in my RAID 5, I need to re-think my monitoring strategy.
My inherited "servers" are a pair of Precision T7600 workstations running Auto DJ radio software that requires Ubuntu/Debian (Unsupported by Dell).
The pair each have a RAID 5 configured with 4 2TB disks. Previous problems with this setup forced us to upgrade the default 310 Perc, to a H710 Perc.
I've now setup my failed primary with a boot drive for the OS running the needed software, and my 5TB RAID drive mounted for data. This is working, I just need to monitor these drives.
I've installed the OMSA from here: http://linux.dell.com/repo/community/ubuntu/
The installation went smoothly, but I'm seeing this used mostly with Poweredge servers, not workstations.
Here are the result of some common commands with OMSA:
<code># /opt/dell/srvadmin/sbin/srvadmin-services.sh restart
DSM SA Connection Service is already stopped
Stopping Systems Management Data Engine:
Stopping dsm_sa_snmpd: Not started *
Stopping dsm_sa_eventmgrd: Not started *
Stopping dsm_sa_datamgrd: Not started *
Stopping Systems Management Device Drivers:
Stopping dcdbas: *
Stopping dell_rbu: Not started *
Starting Systems Management Device Drivers:
Starting dcdbas: *
Starting dell_rbu: *
Starting Systems Management Data Engine:
Failed to start because system is not supported
Starting DSM SA Connection Service:  *

# omreport chassis info
Error! Chassis info setting unavailable on this system.

# omreport storage controller
No controllers found
</code>
Is it possible to monitor the Perc controller with this configuration? My other option might be to install ESX to both these systems and run Ubuntu as a VM. I've read here: http://downloads.dell.com/Manuals/Common/dell-opnmang-sw-v7.4_User%27s%20Guide2_en-us.pdf how to install on ESX, but would my hardware be supported?
Answer: The H710 is a re-branded LSI, so you can use MegaCLI to get the status of your drives.

On a related note, RAID 5, with four 2TB disks, and I am assuming that these disks are SATA disks, is not a very good idea, because in the event of a disk failure, the rebuild time on those 2TB disks is going to be high enough that the chance of another disk failing during the rebuild is very high.
If you can, get two more drives and convert the array to RAID10
Comment: Thank you for the MegaCLI link, I have not seen this tool before. I know RAID 5 really isn't the best option, but it is my only option right now because of the vast music library. I can't put more disks in the RAID because there aren't enough available slots inside the case. I'll look into hardware that could help.
Comment: We converted our system to a RAID 6 with the same disks.
Comment: Following the instructions here: https://calomel.org/megacli_lsi_commands.html There is an awesome bash script called lsi.sh that gives me everything I need!! Thank you.
Comment: @BrandonMartin RAID6 with 4 disks gives you the same usable space as RAID10, but with degraded performance. RAID10 is a much more sensible solution if you have 4 disks. The advantage is that you can expand the array by one disk at a time I guess, which you can't usually do with RAID10 (you have to expand it 2 disks at a time).

Dell Precision T7600 OMSA Perc H710


How do I execute a command string in BASH?

Question: I would like to execute the following from a bash script
exiftool -r -v5 -P -ext mov -ext avi -ext mp4 -ext 3gp "-Directory<MediaCreateDate" -d "/media/data/Home Movies/testdestdir/%Y/%b/" "/media/data/Home Movies/testsrcdir/"

I have built this by concatenating strings into a $CMDEXEC when echoing the variable it looks fine but I cannot execute it from bash script.
I tried various combinations without
luck.
`$CMDEXEC` >> $LOGFILE

$($CMDEXEC) >> $LOGFILE

etc.
Comment: Don't put your commands into variables! See [BashFAQ/050](http://mywiki.wooledge.org/BashFAQ/050).
Comment: Great link @gniourf_gniourf I will improve my script when time allows.
Answer: Did you try eval "$CMDEXEC" >> "$LOGFILE" ?
Comment: I did and is working now. I realised that I left out the E from the variable as in $CMDEXE
Comment: @SebS Good to know. But as  gniourf_gniourf linked in his comment, it is considered a poor practice to put commands inside variables. So please avoid unless you have a good reason to do otherwise. As for `` not working, `eval` causes the shell interpreter to interpret again after variable expansion whereas `` does not. See http://stackoverflow.com/questions/11065077/eval-command-in-bash-and-its-typical-uses


How to test methods of an class that don't have parameters or returns in python using pytest

Question: Let's say that I have a class, this class have some attributes and some methods. This methods don't have any parameters or any return, they only work by changing the attributes of the class using the 'self.' property. How can I test these types of methods?
Let's create the imaginary class Foo to serve as example:
class Foo:
     example_value = -1

     def function_1(self,a,b):
        self.attr1 = a
        self.attr2 = b

        self.function_2()
        self.function_3()
        self.function_4(self.example_value)

    def function_2(self):
        self.attr1 = self.attr1 + 50
        self.attr2 = self.attr2 + 40

    def function_3(self):
        self.attr1 = self.attr1 * 2
        self.attr2 = self.attr3 * 3
        self.example_value = self.attr1 + self.attr3

    def function_4(self,value):

        if value == -1:
            raise Exception('snippet')

In order to test this class, what should I mock? Since two of the methods don't have any input parameters or returns, how can I track the value change of the class attributes? Should I mock only a method or the entire class? And how can I do it in a simple way with pytest? What could be changed in this code? The addition of a simple get_value() would simplify the test development for this class?
Comment: Don't mock anything. Its a tool of last resort anyway. Just check how the attributes have been changed.
Comment: This code is too abstract to make any reasonable suggestions. Maybe there's a reason no method takes arguments or returns values. Maybe there's a reason all communication takes place via the instance attributes. But without knowing what the class is supposed to model or what purpose the methods serve, we can't say anything meaningful.
Answer: You may just verify the attributes, after the method being called, something like this:
from unittest import TestCase

class TestFoo(TestCase):

    def test_f1(self):
        x = Foo()
        x.function_1(10, 20)
        self.assertEqual(x.attr1, 120)
        self.assertEqual(x.attr2, 30)
        self.assertEqual(x.attr3, 10)
        self.assertEqual(x.example_value, 130)

    def test_f4(self):
        try:
            Foo().function_4(1)
        except Exception:
            self.fail("function_4() raised Exception unexpectedly!")

    def test_f4_error(self):
        with self.assertRaises(Exception):
            Foo().function_4(-1)

Comment: I think i was overcomplicating the things because i'm not too experienced in unit testing. This will probably be enough to help and guide me, Thank you. :D


Can any handballs get carded when not in the box?

Question: If a handball by a defender while the ball is in the box, I can understand being carded.
I'm wondering if there is a handball so egregious when outside the box that it can get a card, yellow or red.
Comment: Any unsportsmanlike handball is to be cautioned (e.g. if they do it on purpose). And a goalie handling the ball outside of his box will see a card as well. Denying goal scoring opportunities is a straight red card. Is there a specific situation you want elaborated? Right now the full answer would be pretty long.
Answer: Of course they can, the most obvious example being if the handball is used to deny an obvious goalscoring opportunity.
Comment: Sure. Pass is made, tackle misses, defender sticks out their hand to knock it away. Or even more straightforward, goalkeeper outside the box touches with hands to stop the shot. Plenty of videos where it happens at pro level.
Comment: I think question is about other cases (the rules say that the yellow card can be given for especially cynical handball even if they don't prevent a goal). Anyway, do you have any example in which this happens outside the box?
Answer: Yes, they might get booked for it as well. For instance, if they interrupt in between, unnecessarily or as it has already been commented, even a goalkeeper can get booked if he is handling the ball outside of his box.


How to show an image while my files (most images) are loading?

Question: I'm making an app with a webview loading my html page. It has a lot of images, and i need to load it all when the app starts.
Is there any way to show an image, while my HTML file is loading all the images?
Answer: Sure - just add an UIImageView to your view, put it in the foreground, display your image in there, and when your web view finishes loading (in the webViewDidFinishLoad delegate), blend out the UIImageView.
Answer: //set an image in your imageView and remove it from hidden, set webview hidden
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0ul);
dispatch_async(queue, ^{
// load your html page code here
}
dispatch_sync(dispatch_get_main_queue(), ^{

//set your imageview to hidden and display your webview.
});
});


MongoDB Retrieving data that is missing in collection

Question: Let's suppose we keep track of attendance for multiple classes. If the student was not attending class on a date, they would not appear in the collection. For example we have class "A" and now we want to find out which student was missing on April 30th 2021 from the previous date:
db = {
  "Students": [
    {
      "date": "2021-04-29",
      "student": "Michael",
      "class": "A"
    },
    {
      "date": "2021-04-29",
      "student": "Sarah",
      "class": "A"
    },
    {
      "date": "2021-04-29",
      "student": "Thomas",
      "class": "A"
    },
    {
      "date": "2021-04-30",
      "student": "Michael",
      "class": "A"
    },
    {
      "date": "2021-04-30",
      "student": "Sarah",
      "class": "A"
    }
  ]
}

We would want to retrieve something like this:
[
  {
    "name": "Thomas",
    "class": "A"
  }
]

Is that possible?
Comment: If Thomas was also absent the next day, i.e 1 May (or 3 May?) should he be listed for that day event though he wasn't in the previous day?
Comment: No, this is strictly for comparing two dates. So if you would run something like:
`db.getCollection("Students").distinct('student',{date:'2021-04-29',class:'A'})` and `db.getCollection("Students").distinct('student',{date:'2021-04-30',class:'A'})`
you would get `['Michael', 'Ana', 'Thomas']` and  `['Michael', 'Ana']` as results respectively. I just wish I could find the difference between the two results ('Thomas'). I know this example is not perfect, it's just hypothetical, so sorry about that.
Answer: Possible, yes, but I'm not sure this is the most efficient way.
My idea is to use aggregation:

$match to select the days in question
$group with $push to make an array of students for each day
$sort to make sure the days are in the right order
$group to get both arrays into a single document
$project with $setDifference to subtract the second day's array from the first
$unwind to list each student separately

Note that this all fails if there are no entries for the class for one of the days.
db.Students.aggregate([
  {$match: {$or: [{"date": "2021-04-29"}, {"date": "2021-04-30"}]}},
  {$group: {
      _id: {date: "$date", class: "$class" },
      students: { $push: "$student" }
  }},
  {$sort: {"_id.date": 1}},
  {$group: {
      _id: "$_id.class",
      days: {$push: "$$ROOT"}
  }},
  {$project: {
      _id: 0,
      class: "$_id",
      name: {$setDifference: [
          {"$arrayElemAt": ["$days.students",0]},
          {"$arrayElemAt": ["$days.students",1]}
      ]}
  }},
  {$unwind: "$name"}
])

Playground


Edit content created by users with the same role

Question: Is there any module that would allow me to edit/delete nodes created by users with the same role I have?
Comment: Sorry, I still don't get it? You want the nodes to be edited/deleted by the same role as node author?
Comment: yeah.drupal permission dont allow me to set permision like this.
Answer: I would approach this by grouping users with organic groups module instead of by role. Roles are only for defining access level, not for making groups of users.
Comment: I think this would be the best solution. You can use collaborative groups.
Answer: I believe rules is capable of doing that. Something like:
On node edit:
If user has role:
  author role (any other role except authenticated user)
  OR
  admin role
let it edit. Else, redirect with message.
Of course, this is just the logic outline. You'd have to do the actual PHP code for this to work. ^^
Answer: First of all, the module that provides the content type (via an implementation of hook_node_info()) is the only authority over who may edit content of that content type. If the module chooses not to consult role based permissions, the node access table or other modules, there is not much you can do about it (except for hacking the module). A module that meets the requirements you described can therefore (in general) not exist.
If you restrict your question to those content types, that are provided by the node module (i.e. the ones that are created via the Durpal UI): every authenticated user has the role "authenticated user". A module that meets the requirements you described would therefore grant all authenticated users write access to all nodes created by an authenticated user. This can be done by granting the permission edit any $type content to the role authenticated user for every content type <code>$type</code>.
This has the effect that authenticated users can also edit content created by anonymous users. I do not think this is a problem, because an authenticated user can always become an anonymous user, simply by logging out.
What remains do be done is granting anonymous users the permission to edit content of other anonymous users (yes, they are a role as well), but not content created by authenticated users. This is only a problem when you allow anonymous user to create content.


How to iterate through ComboBox in Vaadin?

Question: I am working with Vaadin and I have some trouble iterating through choices in a ComboBox. I have my object looking like:
class MyObject{
    private String text;
    private Integer i;
    public MyObject(String text,Integer i){
        this.text = text;
        this.i = i;
    }
    public String toString(){
        return text;
    }
    //Getters and setters omitted
}

I add it to the box like this:
MyObject o1 = new MyObject("o1",23);
MyObject o2 = new MyObject("o2",44);
ComboBox box=new ComboBox();
box.addItem(o1);
box.addItem(o2);

This works great when I want to get the chosen data:
MyObject o3 = (MyObject)box.getValue();

But now I need to iterate through the choices in the ComboBox and I don't know how. I seem to need some kind of ID but I don't know how to use that. I tried the following with no success but it doesn't work(and is really ugly):
Collection IDs = box.getItemIds();
Iterator it = IDs.iterator();
while(it.hasNext()){
    Object id = it.next();
    Item item = IDs.getItem(id);
    //What to do now?
}

I'd like to keep my object simple and avoid using beans and complex containers. Vaadins examples are mostly for String and that doesn't help me so much. I'd really appreciate any help.
Answer: If you look at the javadoc for ComboBox, you'll see that the addItem method is actually defined on the AbstractSelect class, and it actually takes the itemId as the parameter. (This is in turn delegated to the Select's container, which in this default case is an IndexedContainer)
So, Collection IDs=box.getItemIds(); will return you the collection of MyObject - i.e. what you are actually after.


filter Hashes in Ruby

Question: q = {"It", "was", "the", "best", "of", "times", "it", "was", "the", "worst", "of", "times"}

write an expression to return
wasworsttimes

without using any character or string literals.
Can someone help me figure out how to do this?
Comment: This sounds like homework.  What have you tried?
Comment: That isn't really a hash. A hash is a map of key-value pairs. Ruby will evaluate it as a hash, taking every odd element as a key and every subsequent element as the value, but it's not clear if that's what you meant here.
Comment: yea its a valid Hash literal...
Comment: @bnaul: `{a,b,c,d}` is the same as `{a => b, c => d}` in ruby 1.8.
Comment: @bnaul, that is a valid Ruby hash literal.
Comment: Its a question from one of my sample exam papers the professor has put on... I'm actually stuck with this...
Comment: I tried every creative way i can... even tried with patterns ;)(of course i'm a noob)
Comment: @matthew-flaschen @sepp2k I know, I think my comment even says that...I just wanted to clarify since the rest of the question didn't indicate that he knew that.
Comment: Am I the only one who has no idea what the question is here? What relationship does the hash have to the string "answer"?
Answer: I'm pretty sure the hash should be {"it", "was",...} (lower case i at the beginning).
Then this literal will create the following hash {"it"=>"was", "the"=>"worst", "of"=>"times"}.
Note that the words you're supposed to extract are the value of the hash. So you can just use Hash's values method to get them and Array's join method to turn the array of values into a string.

filter Hashes in Ruby


Change drush language

Question: Suddenly my drush has switched language and displays error messages in my native language, which is unfortunately not convenient if one tries to search for solutions on these errors.
Do you know how to switch the language of drush errors back to English?
Answer: Adding the following string to drushrc.php worked for me with Drush 5.4:
$options['variables']['language_default']->language = 'en';

In my setup I placed drushrc.php into sites/default/drushrc.php, for other possible locations see documentation in examples/example.drushrc.php in Drush installation directory.
Comment: This works for me, whereas the solution by schnippy does not.
Answer: Create (or modify) your drushrc.php file and add an option for your default language:
$override = array(
  'language_default' => (object) array(
    'language' => 'en',
  ),
);

Comment: Unfortunately, putting that in ~/.drush/drushrc.php did not solve the problem. I still get localized errors.
Comment: did you try a 'drush cache-clear' command after updating the drushrc.php file?

Change drush language


Trying to match variables in a PHP array

Question: I'm stuck with a php array problem. I've to a webpage that takes values from a URL, and I need to cross reference those values against some values on the page and if they match output a 'yes'. It's an expression engine bodge job.
The URL is something like domain.com/page/C12&C14
The C12 and C14 represent different categories. I've taken the last bit of the url, removed the 'C' from the values and then exploded the 12&14 into an array.
I print_r the array on the page and it shows: Array ( [0] => 12 [1] => 14 )
So, the values are in the array. Lovely.
Now on the page I have an html list which looks like
10
12
14
15
I want to output a YES next to the variables that are current in the array so the ideal output would be:
10
12 - YES
14 - YES
15
I was trying this but it keeps just saying No next to all of them.
$currentnumber = 12;
foreach ($tharray as $element) {
if ($element == $currentnumber) { echo "Yes"; } else { echo "No"; }
}

I thought that should work, but it's not. I checked and the array and the variable are both stings. I did a strlen() on both to see if they are the same, but $currentnumber outputs '13' and the array variable outputs '2'.
Any ideas as to why it's saying 13?
Is the variable the wrong type of string - and if so how would I convert it?
Comment: what is var_dump($tharray) ??
Answer: You can try
$tharray = array(10,12,14,15);
$allowed = array(12,14);

echo "
";
foreach ( $tharray as $element ) {
    if (in_array($element, $allowed)) {
        echo $element, " -  Yes ", PHP_EOL;
    } else {
        echo $element , PHP_EOL ;
    }
}

If you don't mind a one line solution
array_walk($tharray, function($item)use($allowed) { echo in_array($item, $allowed) ?  $item. " -  Yes ". PHP_EOL : $item . PHP_EOL  ;});

Output
10
12 -  Yes
14 -  Yes
15

Comment: Thanks very much Baba - turns out it was an annoying php parsing problem with ExpressionEngine - not a problem with the php code. Thanks for the help


Playing audio in python at given timestamp

Question: I am trying to find a way in python to play a section of an audio file given a start and end time.
For example, say I have an audio file that is 1 min in duration. I want to play the section from 0:30 to 0:45 seconds.
I do not want to process or splice the file, only playback of the given section.
Any suggestions would be greatly appreciated!
Update:
I found a great solution using pydub:
https://github.com/jiaaro/pydub
from pydub import AudioSegment
from pydub.playback import play

audiofile = #path to audiofile
start_ms = #start of clip in milliseconds
end_ms = #end of clip in milliseconds

sound = AudioSegment.from_file(audiofile, format="wav")
splice = sound[start_ms:end_ms]
play(splice)

Answer: step one is to get your python to play entire audio file ... several libraries are available for this ... see if the library has a time specific api call ... you can always roll up your sleeves and implement this yourself after you read the audio file into a buffer or possibly stream the file and stop streaming at end of chosen time section
Another alternative is to leverage command line tools like ffmpeg which is the Swiss Army Knife of audio processing ... ffmpeg has command line input parms to do time specific start and stop ... also look at its sibling ffplay
Similar to ffplay/ffmpeg is another command line audio tool called sox
Comment: I looked into ffplay and found pydub, which is a great solution. Many thanks!
Comment: @jordansilva What is that solution their docs is not specific enough.
Comment: @MattQuaine did you look at the question at top ... there is an Update showing code to play an audio file from specific start + stop
Answer: Use PyMedia and Player. Look at the functions SeekTo() and SeekEndTime(). I think you will be able to find a right solution after playing around with these functions.
Answer: I always have trouble installing external libraries and if you are running your code on a server and you don't have sudo privileges then it becomes even more cumbersome. Don't even get me started on ffmpeg installation.
So, here's an alternative solution with scipy and native IPython that avoids the hassle of installing some other library.
from scipy.io import wavfile # to read and write audio files
import IPython #to play them in jupyter notebook without the hassle of some other library

def PlayAudioSegment(filepath, start, end, channel='none'):

    # get sample rate and audio data
    sample_rate, audio_data = wavfile.read(filepath) # where filepath = 'directory/audio.wav'

    #get length in minutes of audio file
    print('duration: ', audio_data.shape[0] / sample_rate / 60,'min')

    ## splice the audio with prefered start and end times
    spliced_audio = audio_data[start * sample_rate : end * sample_rate, :]

    ## choose left or right channel if preferred (0 or 1 for left and right, respectively; or leave as a string to keep as stereo)
    spliced_audio = spliced_audio[:,channel] if type(channel)==int else spliced_audio

    ## playback natively with IPython; shape needs to be (nChannel,nSamples)
    return IPython.display.Audio(spliced_audio.T, rate=sample_rate)

Use like this:
filepath = 'directory_with_file/audio.wav'
start = 30 # in seconds
end = 45 # in seconds
channel = 0 # left channel
PlayAudioSegment(filepath,start,end,channel)