How to evaluate $\int_{0}^{1} \frac{\ln x}{x+1} dx$

Question: I want to evaluate:
$$\int_{0}^{1} \frac{\ln x}{x+1} dx$$
If I was asked I would to evaluate:
$$\int_{0}^{1} \frac{\ln x}{x-1} dx$$
That would be easy because if I use the Taylor series for $\ln x$ centered at $1$ then things will cancel out and leave me with a easy integral.
So how about this integral, I'm guessed to use  the Taylor series centered around $-1$. But even with that thought in mind,  it does not take me anywhere because $\ln (-1)$ is not defined. Can someone help.
Comment: Is this a normal calculus class? If not I'd suggest looking up Polylogarithms, most specifically dilogarithms
Comment: I only know about basic techniques but thanks anyway.
Answer: Make the change of variables, $x = 1 - u$ to get
$$-\int_0^1{\log(1-u)\over u}$$
Now you can do the Taylor series easily
$$\sum_{k=1}^\infty \int_0^1{u^{k-1}\over k}\,du$$
This gives us
$$\sum_{k=1}^\infty {1\over k^2}={\pi^2\over 6}.$$
Comment: @AhmedS.Attaalla note that the powers of $u$ are $(u-0)^k$ so it **must** be centered at $0$.
Comment: I'm sorry but I'm forced to accept another answer. Your substitution idea gives  $2-u$ in the denominator.
Comment: About where did you center the Taylor series? $u=0$?
Comment: Can you help me with another thing, I'm trying to get a technique  down that will help me to solve integrals of these kind and possible others, I will edit in another question in the body of this question if you will help.
Comment: @AhmedS.Attaalla it is standard MSE procedure to put new questions in new topics.
Comment: Ok I will. (I need 15 characters or something).
Comment: @AhmedS.Attaalla : he didn't mean you cannot discuss of other ways for solving it in the comments...
Comment: Note you found the second integral not the first.
Answer: Consider:
$$I=\int_{0}^{1}\frac{\ln(x)}{1+x}dx= \sum_{n=1}^{\infty}\frac{(-1)^n}{n^2}$$
You can show this result by noticing since $0<x<1$ that $$\frac{1}{1+x}= \sum_{n=0}^{\infty}(-x)^n$$ and performing term by term integration. You will need integration by parts to do that.
Now you can also show that:
$$-I=\int_{0}^{1}\int_{0}^{1}\frac{1}{1+xy}dydx$$ by expanding the integrand into a geometric series and doing term by term integration with respect to $y$ and the with respect to $x$.
Now, in the  double integral, let $x=u+v,y=-u+v$
Then $\left|\frac{\partial(x,y)}{\partial(u,v)}\right|=\begin{vmatrix} 1 && 1\\ -1 && 1 \\ \end{vmatrix}=2$
The change of variables causes the double integral to become:
$$\iint_{T} \frac{2}{1+v^2-u^2}dvdu$$ where $T$ is the square with vertices $(0,0),(\frac{-1}{2},\frac{1}{2}),(0,1),(\frac{1}{2},\frac{1}{2})$
Now, we need to compute two double integrals (over two different subregions of the square) and sum them to get the final answer.
First let $\frac{-1}{2}\leq u \leq 0 , -u \leq v \leq 1+u$.
$$\int_{\frac{-1}{2}}^{0}\int_{-u}^{1+u} \frac{2}{1+v^2-u^2}dvdu=\int_{\frac{-1}{2}}^{0}\frac{2\tan^{-1}\left(\frac{1+u}{\sqrt{1-u^2}}\right)+2\tan^{-1}\left(\frac{u}{\sqrt{1-u^2}}\right)}{\sqrt{1-u^2}}du$$
For the first term on the right hand side, let : $z=2\tan^{-1}\left(\frac{1+u}{\sqrt{1-u^2}}\right),dz=\frac{1}{\sqrt{1-u^2}}du$. For the second term, if you draw a right triangle, notice $2\tan^{-1}\left(\frac{u}{\sqrt{1-u^2}}\right)=2\sin^{-1}\left(u\right)$ so let $z=\sin^{-1}\left(u\right),dz=\frac{1}{\sqrt{1-u^2}}du$ for the second term.When evaluating these two terms with these substitutions, the sum of these terms should be $\frac{\pi^2}{24}$.
On the other hand, let $0\leq u \leq \frac{1}{2} , u \leq v \leq 1-u$.
$$\int_{0}^{\frac{1}{2}}\int_{u}^{1-u} \frac{2}{1+v^2-u^2}dvdu=\int_{0}^{\frac{1}{2}}\frac{2\tan^{-1}\left(\frac{1-u}{\sqrt{1-u^2}}\right)-2\tan^{-1}\left(\frac{u}{\sqrt{1-u^2}}\right)}{\sqrt{1-u^2}}du$$
For the first term on the right hand side, let $z=2\tan^{-1}\left(\frac{1-u}{\sqrt{1-u^2}}\right),dz=\frac{-1}{\sqrt{1-u^2}}$ and for the second term, use $2\tan^{-1}\left(\frac{u}{\sqrt{1-u^2}}\right)=2\sin^{-1}\left(u\right)$ and let $z=\sin^{-1}\left(u\right),dz=\frac{1}{\sqrt{1-u^2}}du$ for this.When evaluating these two terms with these substitutions, the sum of these terms should be $\frac{\pi^2}{24}$.
So:
$$\iint_{T} \frac{2}{1+v^2-u^2}dvdu=\frac{2\pi^2}{24}=\frac{\pi^2}{12}$$
thus, $$I=\int_{0}^{1}\frac{\ln(x)}{1+x}dx=\frac{-\pi^2}{12}$$
Answer: We know that $$\int_0^1 x^a~dx=\frac{1}{a+1}$$
Then we can differentiate with respect to $a$:
$$\int_0^1 x^a \ln x ~dx=-\frac{1}{(a+1)^2}$$
Now we can use the geometric series:
$$\sum_{a=0}^\infty (-1)^a x^a=\frac{1}{1+x},~~~|x|<1$$
$$\sum_{a=0}^\infty (-1)^a  \int_0^1 x^a \ln x ~dx=- \sum_{a=0}^\infty (-1)^a  \frac{1}{(a+1)^2}$$
(Now we interchange integration and summation on the left hand side)
$$\int_0^1 \frac{\ln x}{1+x} ~dx=-\sum_{a=0}^\infty \frac{(-1)^a}{(a+1)^2}=\sum_{a=1}^\infty \frac{(-1)^a}{a^2}=-\frac{\pi^2}{12}$$
Comment: @AhmedS.Attaalla, I edited in the middle. No, I don't use integration by parts here
Comment: @AhmedS.Attaalla : he started from $\int_0^1 x^a dx$ and made the $\ln x$ appear by differentiating ***with respect to***  $a$
Comment: I understand everything until you get to the .actual integral, what did you do? Integration by parts? It would really help. Thanks
Comment: I understand it now thanks
Answer: Convert the integral to the easier one via
\begin{align}
\int_{0}^{1} \frac{\ln x}{x+1} dx &=
 \int_{0}^{1} \frac{\ln x}{1-x} dx - \int_{0}^{1} \frac{2x\ln x}{1-x^2} dx, \>\>\>\>\>t=x^2\\
&= \int_{0}^{1} \frac{\ln x}{1-x} dx -\frac12 \int_{0}^{1} \frac{\ln t}{1-t} dt \\
&= \frac12 \int_{0}^{1} \frac{\ln t}{1-t} dt=\frac12\cdot \frac{\pi^2}{6}=\frac{\pi^2}{12}
\end{align}


Express a quotient of polynomials as a continued fraction

Question: I have two polinomials P(x) and Q(x), with the degree of P larger than the degree of Q, both polynomials with known real coefficients. Is there a method to express P/Q as a continued fraction of x?
Comment: The same as if they were integers, just divide $P=QA_1+R_1$. Then $\frac{P}{Q}=A_1+\frac{R_1}{Q}=A_1+\frac{1}{Q/R_1}$ and keep dividing the new improper fraction $Q/R_1$.
Comment: It usually helps to give a specific example, even when you want a general result.
Comment: It may help to reverse the process. Start with a small continued fraction and evaluate it as the quotient of polynomials. Then you can see how to reverse that to get back the continued fraction. Compare with the Euclidean algorithm.
Answer: You have the division algorithm in polynomials, so:
$$p(x)=q(x)d_1(x)+r_1(x)\tag{1}$$ for some $d_1(x),r_1(x)$ with $\deg r_1(x)<\deg q(x).$
Then divide both sides of (1) by $q(x)$ and you get:
$$\frac{p(x)}{q(x)}=d_1(x)+\frac{r_1(x)}{q(x)}=d_1(x)+\dfrac1{\frac{q(x)}{r_1(x)}}$$
Now continue by applying the same process to $\frac{q(x)}{r_1(x)}.$ So we have:
$$q(x)=r_1(x)d_2(x)+r_2(x): \deg r_2<\deg r_1\\
\frac{q(x)}{r_1(x)} = d_2(x)+\dfrac1{\frac{r_1(x)}{r_2(x)}}$$
Keep doing this until the remainder $r_n(x)=0,$ and the process stops. It will eventually terminate because the degrees of the $r_i$ are strictly decreasing.
Then $$\frac{p(x)}{q(x)}=d_1(x)+\dfrac1{d_2(x)+\dfrac1{ \ddots+\frac1{d_n(x)}}}$$
This really is the same process as for rational numbers:
$$\begin{align}\frac{12}{5}&=2+\frac{2}{5}\\
&=2+\dfrac1{\frac{5}{2}}\\
&=2+\dfrac1{2+\frac12}
\end{align}$$
Comment: Great! It seems so obvious now... Thanks for the explanation.


Which method to use for predicting variable a (score based on a likert scale) with an variable b and c

Question: I have three variables. The energy behavior is a score based on a Likert scale. The other two variables age and education are nominal.
Now I want to show that the two nominal variables, age and education influence the energy-saving behavior.
What is a good suitable method in this situation? I am using SPSS.
Thank you.
Answer: Answering directly to the way you placed your question, if you think it's acceptable to assume that the Likert scale variable follows a Normal distribution and that both age and education are nominal, you can do a 2-way ANOVA.
However, assuming Normality on a Likert scale variable is far from being consensual. I'll assume that it is not a continuous variable, hence it may be treated as discrete or as ordinal.
Furthermore, it's common to have age as a continuous variable, when its range is large enough, or if you have the date of birth and the date of evaluation.
As for education it is common to use it as ordinal or a discrete scalar if you have the number of years that people went to school.
Hence, an ordinal regression may be the way to go if you assume that the Likert scale variable is ordinal.
Otherwise, if you assume that the Likert scale, age and education can be reasonably treated as scalars, a linear regression may be applicable.
Comment: I suggest that you add the above information into the question. Note that age and education are ordinal. In this case, the ordinal regression might be the more adequate approach.
Comment: Age: 15-25, 26-35, 36-45, Over 45 // Education: Less than high school, High school, Graduate, College diploma, Bachelor's degree, Master's degree, Doctorate degree, Others // The energy saving behavior score is the mean out of 5 questions (Likert Scale) - Thank you


$f(0)=0 \neq f'(0) \Rightarrow \frac{\delta(x)}{f(x)} = \frac{\delta(x)}{x f'(0)}$?

Question: In this reference (Eq. 15 and sentence below) the following property of Dirac's delta is used:
$$ k_n \frac{ \delta(x)}{\sin(k_n x)} \rightarrow k_n \frac{ \delta(x)}{x \, k_n} = \frac{ \delta(x)}{x} $$
How can one justify this relation?
This obscure step is necessary because the result on the right side must be independent of the constant $k_n$.
I know that $g(x) \delta(x) = g(0) \delta(x)$ for valid test functions $g$, but in this case we encounter $g(x) = \frac{1}{f(x)}$ with $f(x)=0$. I'm not familiar with this case.
For consistency, I'm using the standard definition as a distribution of the delta. I imagine in this particular case an extended definition would be needed in order to include this type of undefined test functions at the origin, in a way that is compatible with the standard one. Moreover, the distribution $D_{\frac{\delta(x)}{f(x)}}[h]$ has to be well-defined for test functions $h(0) = 0 \neq h'(0)$, because later in the manuscript (Eq. 20) the following is used: $\sin(k_n x) \frac{\delta(x)}{x} = k_n \delta(x)$
I believe there is already a mainstream way of handling this type of issues, but I can't find it. If it helps, my guess is that $\frac{\delta(x)}{f(x)} = \frac{\delta(x)}{x f'(0)}$ and $\frac{h(x) \delta(x)}{f(x)} = \frac{h'(0) \delta(x)}{f'(0)}$, where I used the lowest-order term in the Taylor expansion at the origin.
Comment: Isn't this just an application of the small angle approximation? Assuming small angles we get that $sin(k_nx) \approx k_nx$
Comment: If you haven't seen this approximation done before, this is the wiki for it : https://en.wikipedia.org/wiki/Small-angle_approximation
Comment: @Joey small-angle approximation is the same thing as the lowest-order Taylor approximation. This does not explain the result in the sense of distributions
Answer: In the sense of distributions, such a quantity is only defined on the dual of a subspace of a function space where $V = \{\varphi\in C_0^\infty(\Bbb{R})| \varphi(0)=0\}$. However, it does make sense to talk about the following equality in a more general function space $V$ - does there exist a distribution $g\in V^*$ such that for $u\in V$
$$u(x)g(x) = k\delta(x)$$
when $u(0) = 0 \neq u'(0)$ ? Well yes, the derivative of $\delta$ does the job perfectly
$$u(x)\delta'(x) \equiv - u'(0) \delta(x) = k\delta(x)$$
And so the question becomes, is the action of $\frac{\sin(k_n x)}{k_n}$ on $\delta'$ the same as the action of $x$ on $\delta'$ ? Absolutely because they both share values of their $0$th and $1$st derivatives:
$$\frac{\sin(k_n x)}{k_n}\delta'(x) \equiv -\delta(x) \equiv x\delta'(x)$$
In other words, taking $g = -\delta'$, we get the equality of distributions we are after
$$\frac{k_n \delta(x)}{\sin(k_n x)} = \frac{\delta(x)}{x}$$
however this is only defined on the subspace where $\varphi(0)=0$. This can always be regularized with the homogenized version of the function on the larger function space
$$\left\langle\frac{\delta(x)}{x},\varphi\right\rangle \equiv \left\langle \delta, \frac{\varphi(x) - \varphi(0)}{x}\right\rangle$$
which is not a proper linear functional technically. Further regularization with higher order derivatives vanishing would use higher order derivatives of the delta as intermediaries like we did here with the first derivative.
Comment: Thank you for your answer, but I don't follow how you get $$\frac{k_n \delta(x)}{\sin(k_n x)} = \frac{\delta(x)}{x}$$ by simply taking $g=-\delta'$? I know it works for $u(0)=0$, but not for functions of the indeterminate form $u(0)=1/0$


Lat/long gets reset after updating

Question: At the moment I'm having some struggles with the location module in Drupal, well I'm not really sure that it's the location module that is causing the problems.
The best I can explain it with a little code. There is a module which connects to a backoffice and gets venues with lat / longitudes.
I do the following:
<code>    dpm($node->field_address[LANGUAGE_NONE][0]);
    node_save($node);
    dpm($node->field_address[LANGUAGE_NONE][0]);
</code>
Before updating the node the values are in the object
<code>    array(6) {
      ["street"]=>
      string(16) "Street 1"
      ["city"]=>
      string(7) "Middle earth"
      ["postal_code"]=>
      string(7) "12345"
      ["latitude"]=>
      string(8) "52.000"
      ["longitude"]=>
      string(7) "4.000"
      ["location_settings"]=>
      array(1) {
        ["form"]=>
        array(1) {
          ["fields"]=>
          array(2) {
            ["latitude"]=>
            array(1) {
              ["default"]=>
              string(8) "52.000"
            }
            ["longitude"]=>
            array(1) {
              ["default"]=>
              string(7) "4.000"
            }
          }
        }
      }
    }
</code>
After saving the node the lat / long is reset to int(0).
The lat / long in the location settings is still in tact.
I've search a long time to debug this, i was thinking maybe there's a hook that resets the value, but I just can't find it. I'm fairly new to Drupal so any help will be appreciated :)
Thanks!
Answer: Found the solution.
The lat/long came from a different source and when the location module tried updating the lat/long I got NULL/NULL, because I was on the max amount of requests to google.
Solution was to set the 'source' column in the location table to 1, so the module thinks the value is overruled by the user. The lat/long doesn't get updated / reset to null anymore.

Lat/long gets reset after updating


Passing Class Name as Argument to Method

Question: I've built a REST Service with Spring Boot. In one request method I'm validating the incoming request vs. an Object that I have annotated for validation (@NotNull etc.) using Hibernate Validator. The code for the REST endpoint looks like this:
@GetMapping(value = "/parameter-dates")
    public ResponseEntity getParameterDates(ParameterDateRequest parameterDateRequest) {

        Set<ConstraintViolation<ParameterDateRequest>> inputErrors = validator.validate(parameterDateRequest);

        if (!inputErrors.isEmpty()) {

            objectValidationErrorMessages = new ArrayList<>();
            for (ConstraintViolation<ParameterDateRequest> constraintViolation : inputErrors) {
                objectValidationErrorMessage = new ObjectValidationErrorMessage();
                log.info("Error for user: " + loggedInUser.getUserEmail() +
                        " field: " + constraintViolation.getPropertyPath() + " with value: " + parameterDate.getParameterDateUnadjusted() +
                        " has error: " + constraintViolation.getMessage());
                objectValidationErrorMessage.setFieldWithError(constraintViolation.getPropertyPath().toString());
                objectValidationErrorMessage.setErrorMessage(constraintViolation.getMessage());
                objectValidationErrorMessages.add(objectValidationErrorMessage);
            }

            return (new ResponseEntity(objectValidationErrorMessages, headers, HttpStatus.BAD_REQUEST));

        }
//The rest of the code that is used when there is not validation errors

I would like to move the creation of the validation error messages to a metod of it's own like this:
public List<ObjectValidationErrorMessage> getErrorMessages(Class<?> clazz, Object model) {

        List<ObjectValidationErrorMessage> objectValidationErrorMessages = new ArrayList<>();
        Set<ConstraintViolation<?>> inputErrors = validator.validate(model);

        if (!inputErrors.isEmpty()) {
            for (ConstraintViolation<?> constraintViolation : inputErrors) {
                objectValidationErrorMessage = new ObjectValidationErrorMessage();
                objectValidationErrorMessage.setFieldWithError(constraintViolation.getPropertyPath().toString());
                objectValidationErrorMessage.setErrorMessage(constraintViolation.getMessage());
                objectValidationErrorMessages.add(objectValidationErrorMessage);
            }
        }
        return objectValidationErrorMessages;
    }

With the code as it is now I get the following error Unknow class: validateClass for Set<ConstraintViolation<validatedClass>>. How do I pass a Class name, in this case ParameterDateRequest as an argument to a method?
Update:
I manged to pass the Class as Class<?> clazz. I also realised that I need to pass the Object as an Object but I get this error:
validate (T, Class<?>...) in Validator cannot be applied
to (java.lang.Object)
reason: Incompatible equality constraint: ? and T

Answer: You can generify your method:
public <T> List<ObjectValidationErrorMessage> getErrorMessages(Class<T> clazz, T model) {

This will ensure that your Class<?> token and your Object have matching types.
Then pass the arguments as such:
validator.validate(model, clazz)

Note I could be misreading the error (I couldn't find the javadocs for the validator), so it's also possible that validator has a class-level generic (e.g. Validator<MyType>). If that's the case, then the T/Object you pass to validator#validate must match the type of the Validator, but the second parameter seems to accept any classtype (Class<?>). You also wouldn't need to generify the method as well, you would just have to match the type for the validator.


get a bash's cwd

Question: If I open up 2 terminals in Ubuntu, via ps u I can see 2 bash's pid.
My problem is from one terminal, how can I know the bash's cwd in the other terminal?
Comment: What are you really trying to accomplish here? Why do you need the PID from the other terminal.  There may be ways to solve your issue more directly.
Comment: @JeremyJStarcher, Sorry, but I can't even figure out one.
Comment: @Alcott: if *we* know why you need to know the current working directory of another shell, we may be able to suggest a better solution.
Answer: You can check
/proc/<pid>/cwd

Comment: But how should I know the abs path of `/proc//cwd`?
Comment: @Alcott That _is_ an absolute path, just replace `` with a valid process id. You can get the pid of the current shell from the `$$` variable. So if you have two pids, then use the one not equal to `$$`.
Answer: The pwdx command prints the current working directory of each given process ID.
Example from Solaris (believe Ubuntu also has the command):-
$ cd /tmp
$ pwdx $$
22281:  /tmp

get a bash's cwd


Low performance – Patch Match. Image Processing on GPU (CUDA)

Question: I have a performance problem: CPU and GPU performances are almost the same.
The Problem I Dealing with is PATCH MATCH. I Have 2 Matrices. I want to find where is the maximum similarity between the big matrix and the small one.
The Matrices has Binary values 0/1 (Black and White).
When I am checking a match between a small matrix to a big one with i5 CPU, it takes 30ms (using multithreading).
When I am checking a match between a small matrix to a big one in a Ge-force GT 730, it takes also 33ms.
I would expect that The GPU will work faster in at least 1 magnitude of order. I pretty disappointed from my current results.
I have two matrices:
1) Big - 300000 (300 rows, 1000 columns)
2) Small 50000 (50 rows, 1000 columns)
The comparing process is done by dividing the big matrix into 250 sub matrices and then comparing each one to the small matrix, then find highest similarity.
The Similarity criterion is the sum of corresponding black pixels on both matrices (the small and the sub-big) divided by the sum of black pixels on sub-big.
I did the last task using the following CUDA code:
 __global__ void matCompare_cuda (uint8_t  *D_SUB , uint8_t  *D_SMALL ,  float *D_RSLTS , unsigned int step, int numOfIndentations ,int SUB_size, int SMALL_size)
{
    int  i = 0 , j = 0 , success = 0, sumZero = 0;
    int tid = threadIdx.x + blockIdx.x * blockDim.x;

    int LoopIndex = ( tid * step );

    if (tid < numOfIndentations)
    {
        for ( j = 0 ; j < (SMALL_size) ; j++)
            {
                i = j + LoopIndex;
                if ( D_SUB[i] == 0 )
                    {
                        {
                        sumZero++;
                        if ( D_SMALL[j] == 0 )
                            success++;
                        }
                    }
            }
        if (  success > 0 && sumZero > 500)
            D_RSLTS[tid] = 100*((float)success / sumZero) ;

    }
}

The Kernal launch:
int numOfIndentations = 300-50  //[ (big.row) - (small.row)]

int numBlock = 16;
int threadNumber = numOfIndentations/numBlock;

matCompare_cuda<<< numBlock , threadNumber >>> ( D_SUB , D_SMALL , D_RSLTS , step, numOfIndentations, SUB_size, SMALL_size );

The Cpu Code:
 for (i=0; i < (pixelNum) ; i++)
{
    if (SUB[i]==0)
    {
        sumDots = sumDots +1;
        if (SMALL->Image[i]==0)
        {
            success = success + 1;
        }
    }
}

if (success>0)
    if (sumDots>500)
        RSLT=((float)success/sumDots)*100;

Do you see any improvement that can be done in the GPU code?
Answer: A few things.
Try to avoid the if's if possible. You can write here:
sumZero += (1 - D_SUB[i])
success += (1 - D_SUB[i]) * (1 - D_SMALL[j])

However I don't think you're going to see a huge difference here. I see two reasons.
One is that there's overhead in invoking cuda. The data needs to be copied to the graphic card and back. That eats some of the speedup you get. Not sure how much it is, but since the run-time is so short it could play a role. I hope you didn't time the compilation of the kernel and other one-time things (take them out by running the code in a loop and ignoring the first few iterations).
Second your big matrix is too small and your small matrix is too big. Because the small matrix is so big (1000 columns) I'm guessing it plays really well with the CPU cache lines. If the small matrix were smaller you would have to go to the next line more often which would increase the chances of breaking the cache line. The gpu uses rectangles for caching so it wouldn't be a problem. If the big matrix were to be bigger you would also increase the amount of computation required so the GPU would start to get ahead.
Comment: Thanks, I did the:
sumZero += (1 - D_SUB[i])
success += (1 - D_SUB[i]) * (1 - D_SMALL[j])
And it did improve by little.
What you saying is that there is no solution?
Comment: @YosefAlon 30 ms is already fast. If this is the entire program I don't think you should bother optimizing anything. If you need to run this part many times it would be good to give some context so that people can give you the right suggestions.
Comment: First, Thanks.
Which context is needed?


Upload and save an image using PrimeFaces and JSF

Question: Hello i'm new at JSF and primefacess and i want to upload an image and save it in a folder in my project
when Execute all the code pass properly but when i check the save directory i don't find the image that i saved.
//Java Code
    private UploadedFile file;
public UploadedFile getFile() {
    return file;
}

public void setFile(UploadedFile file) {
    this.file = file;
}

public void upload() {
    if(file != null) {
        try {
                FacesContext context = FacesContext.getCurrentInstance();
                ServletContext scontext = (ServletContext)context.getExternalContext().getContext();
                String rootpath = scontext.getRealPath("/");
                File fileImage=new File(rootpath+"upload\\temp\\text.png");
                InputStream inputStream=file.getInputstream();
                SaveImage(inputStream,fileImage);

                FacesMessage message = new FacesMessage(rootpath);
                FacesContext.getCurrentInstance().addMessage(null, message);
            }
        catch(IOException e) {
            e.printStackTrace();
            FacesMessage message = new FacesMessage("There was a probleme your file was not uploaded.",e.getMessage());
            FacesContext.getCurrentInstance().addMessage(null, message);
        }
    }
}

public void SaveImage(InputStream inputStream, File ImageFile) throws IOException {
    OutputStream outputStream=new FileOutputStream(ImageFile);
    IOUtils.copy(inputStream, outputStream);
}

//XHTML Code
<h:form enctype="multipart/form-data">

</h:form>

Comment: could you please check what is the value of rootpath?
Comment: Yes i check and i add the path to the folder i wand it worked for me
Comment: so it worked when you put the path to the file manually?
Comment: i didn't put all the path manually i only add rootpath+"upload/temp/file.png"
Comment: what OS are you testing on? linux?
Comment: no i'm using windows 10
Answer: It might be because of the OS you are running on and it does not recognize the path correctly when you are using \ :
File fileImage=new File(rootpath+"upload\\temp\\text.png");

Replace this line with:
File fileImage=new File(rootpath+"upload"+File.separator+"temp"+File.separator+"text.png");

This is a good practice when you are working with file paths across different platforms.
Answer: You can use this method :
You can get the inputstream using the FileUploadEvent like :
InputStream in = event.getFile().getInputstream() ;

public void upload(String fileName, String destination, InputStream in) {
    try {

        // write the inputStream to a FileOutputStream
        OutputStream out = new FileOutputStream(new File(destination + "/" + fileName));
        int read = 0;
        byte[] bytes = new byte[1024];

        while ((read = in.read(bytes)) != -1) {
            out.write(bytes, 0, read);
        }
        in.close();
        out.flush();
        out.close();
        System.err.println("New file created!");
    } catch (IOException e) {
        System.out.println(e.getMessage());
    }

}

Comment: can he use this from primefaces? file upload does not work this way for primefaces
Comment: Ye s you can use it and i will edit it to make it clear Thank you for the comment


Python: Getting weird symbols when I use get request?

Question: I am trying to scrape spreadsheets after logging into a asps.net website.
the process flow is: Login to the webpage, then click on the spreadsheet link and get the excel data
The login link is : https://www.canfax.ca/Main.aspx.
I figured out a way to login using requests and beautiful soup. But now when I try to download the spreadsheet, I am getting weird symbols.
<code>link = 'https://www.canfax.ca/Main.aspx'

payload = {
    'ctl00$MainContent$userNameTextBox':'username',
    'ctl00$MainContent$passwordTextBox': 'password',
    'ctl00$MainContent$signInButton': 'Sign In'

page = 1

with requests.Session() as s:
    s.headers['User-Agent'] = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36'
#     s.headers["Accept-Encoding"]:" deflate;q=0.9,en;q=0.8"
    s.headers["Accept-Language"]: "en-GB,en-US"
    s.headers["Content-Type"]:"application/x-www-form-urlencoded"
    s.headers["Referer"]: "https://www.canfax.ca/Main.aspx"
    s.headers["Sec-Fetch-Dest"]: "document"
    s.headers["sec-ch-ua"]: '" Not A;Brand";v="99", "Chromium";v="100", "Google Chrome";v="100"'
    s.headers["sec-ch-ua-platform"]: '"macOS"'
    s.headers["Sec-Fetch-Mode"]: "navigate"
    s.headers["Sec-Fetch-Site"]: 'same-origin'
    s.headers["Sec-Fetch-User"]: '?1'

    s.headers["Cookie"]: "AspxAutoDetectCookieSupport=1; ASP.NET_SessionId=iekqccbp4lzkjuiqujpweifk; __utmc=219132171; __utmz=219132171.1649363793.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=219132171.167833154.1649363793.1649432375.1649498873.5; __utmt=1; __utmb=219132184.108.40.2069498873"

    r = s.get(link)
    soup = BeautifulSoup(r.text,"lxml")
    payload['__VIEWSTATE'] = soup.select_one("#__VIEWSTATE")['value']
    payload['__VIEWSTATEGENERATOR'] = soup.select_one("#__VIEWSTATEGENERATOR")['value']
    payload['__EVENTVALIDATION'] = soup.select_one("#__EVENTVALIDATION")['value']

    res = s.post(link,data=payload)
    soup = BeautifulSoup(res.text)

    s.headers["Content-Type"]= "application/vnd.ms-excel; charset=utf-8" # this is to change the previous content type.
    url1 = 'https://www.canfax.ca/Report/SpreadsheetReport.aspx?catalogue=FullSpreadsheets&group=Weekly+Fed+Steer&report=Alberta'
    t=s.get(url1)
</code>
my output that i get is :
<code>�"$"#,##0�\$#,##0�0.0Calibri1���Calibri1���Calibri1���Calibri"$"#,##0;\-"$"#,##0"$"#,##0;[Red]\-"$"#,##0"$"#,##0.00;\-"$"#,##0.00#"$"#,##0.00;[Red]\-"$"#,##0.005*0_-"$"* #,##0_-;\-"$"* #,##0_-;_-"$"* "-"_-;_-@_-,)'_-* #,##0_-;\-* #,##0_-;_-* "-"_-;_-@_-=,8_-"$"* #,##0.00_-;\-"$"* #,##0.00_-;_-"$"* "-"??_-;_-@_-4+/_-* #,##0.00_-;\-* #,##0.00_-;_-* "-"??_-;_-@_-�"$"#,##0_);\("$"#,##0\)!�"$"#,##0_);[Red]\("$"#,##0\)"�"$"#,##0.00_);\("$"#,##0.00\)'�""$"#,##0.00_);[Red]\("$"#,##0.00\)7�2_("$"* #,##0_);_("$"* \(#,##0\);_("$"* "-"_);_(@_).�)_(* #,##0_);_(* \(#,##0\);_(* "-"_);_(@_)?�:_("$"* #,##0.00_);_("$"* \(#,##0.00\);_("$"* "-"??_);_(@_)6�1_(* #,##0.00_);_(* \(#,##0.00\);_(* "-"??_);_(@_)4�/_(* #,##0.0_);_(* \(#,##0.0\);_(* "-"??_);_(@_)0�+_(* #,##0_);_(* \(#,##0\);_(* "-"??_);_(@_)
�0.000�"Yes";"Yes";"No"�"True";"True";"False"�"On";"On";"Off"]�,[$� -2]\ #,##0.00_);[Red]\([$� -2]\ #,##0.00\)��� � ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� � � ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� ��� �� �  �� ���� �
�}�00_)����[$� -�##0.�          ��� ��� �� ��� ���� ��� �`� ��� �� ��� �� ��� ���� �    �� �� ��� �� ��� �a�� ��� �� � � � � � � �"� � H/ �� � � � �#� �#� �    #� �#� K�|L�A
00_)����[$� -???�##0.???�           ???�
00_)(65
??v�00_)�̙�[$� -�##0.�          �
�e�00_)���[$� -x:����00_)�������[�����##    �����
???�00_)����[???�## ???�
    ???�
60% - Accent6   23��������%�"Accent1A�Accent1O�������%�#Accent2A!�Accent2�PM�����%�$Accent3A%�Accent3��Y�����%�%Accent4A)�Accent4�d������%�&Accent5A-�Accent5K�������%�'Accent6A1�Accent6   ��F�����%�(Bad9�Bad��������%�)Calculation��Calculation�������}�%������  ���*
Check Cell�
Check Cell���������%�???��???��???� �???��+�� �Comma�,��(�  Comma [0]�-��&Currency�.��.�Currency [0]�/Explanatory TextG5�Explanatory Text��%�0� �F  �Followed Hyperlink���� 1Good;�Good������a�%�2  Heading 1G� Heading 1I}�%O����3 Heading 2G� Heading 2I}�%�?�����4   Heading 3G� Heading 3I}�%23�����5   Heading 49� Heading 4I}�%�6�4�  Hyperlink
���
7Inputu�Input��̙��??v�%������   ���8Linked CellK�Linked Cell��}�%�����9NeutralA�Neutral������e�%���"�Normal�    :Noteb
�Note��������������������   ������;Outputw�Output������???�%�???��???��???� �???��<��$�Percent�
=Title1�TitleI}�%�
>TotalM�Total�%O���O����?Warning Text?�Warning Text���%X�TableStyleMedium9PivotStyleLight16��8�������������������������������3f������ff���f����������������������������������̙����3f�3���������fff����3f3�f333�3�3f33�333\���`�F2Alberta Fed Steer Price����5��OAlberta Weekly Fed Steer PricesSource:  CanFaxWeek  Cdn $/cwt�
�1^B�
d����MbP?_*+��%�����&�?'�?(�?)�?M�\\CFXDC02\Lexmark MS817n����
dXXLetter����GIS4DINU"p��W�pSMTJ`{4104DF81-B4B1-427d-93EA-628FE2120C5A}InputBinUSEPRINTERRESDLLUniresDLLPaperSizeLETTERMediaTypeAUTOResolution600DPIColorModeMonoOrientationPORTRAITCollateONDuplexNONEOutputBinUSEPRINTERJobStapleAllDocumentsUSEPRINTERJobHolePunchUSEPRINTERV4DM�"dXX�?�?&U}�A}A}$   A:w�������  �
��������������������
@�
B�
B�
C�HDp�@Dt�@Dx�@D|�@D��@D��@D��@D��@D��@D��@D��@�NC�?E�u�@E���@E��@Fs�@F�G�@F�%�@AZ�@AQ�@A���@A�l�@A���@�C@E�?�@E ]@E�G�z�`@�$FA��@F�}�@F�O�@A�-�@A�b�   A�G�ztd@�
�@GA7�@F�P�@F�>�@A�@A[�@AA�@Am�@A�N�@�C@E3�@E{�@Eq=
ף�a@�6F��@FAy�@F��@A���@A�6�@A��@A���@A�q�@�*   C@E��@E��@EM�@F�D�@FAN�@    F=
ףp�c@   A���(\gd@   Afffff^c@~
        A�@
Ffffff�b@   H�
C@E���@E���@
E�G�zDa@�
G(g@F��@
F=
ףp�c@
Aq=
ףpd@
A��(\��b@~

    A��@

A��(\�c@~

A$�@�6C@E���@E�p�@E�b�@FAz�@GA��@F��@F�@A=
ףp�b@�  A�h�@A��@A���@�0C @E��@E�g�@G��@F���@GY�@F�U�@H~
F��@    J~

C"@E���@E��@G�@F��@F�u�@F��@A�3�@A�U�@A���@
A=
ףp�b@~
A��@�C$@E�m�@EO�@E���(\a@~
G`h@G�*F��@A��@A    �@A��@A�H�@A��@�6C&@Ec�@G:�@E��@F�7�@FA�@F\�@A�!�@A��(\�Zc@~
    A��@
A�G�z�b@~
A��@�HC(@E��@E�@E���@FAF�@F�e�@F���@A��@A��@F0b@A�b@
�C*@E�m�@E9�@E�����1b@�*F�q�@FA2�@F�@Fd@A�O�@A��@
HA�G�zLd@�C,@Ep[@E��@E��(\�*b@�F���@G�d@FA��@A��(\�zc@~
Ai�@    F������b@�
Au�@A���@�0C.@E�\�@E�@E^�@F���@F�X�@F��@A���(\_c@�AF�@A��@A���@
�6C0@E��@E=�@E�r�@FA��@F��@FA�@A���@Afffffd@�   A�I�@Ae�@
�0C1@E���@E��@EHb@F���@FA*�@F�S�@A�G�zTd@�A���@A���@F�c@
�$C2@E�X�@E�]�@E�b@FA��@F�G�zDc@~
F���@A��(\�zd@Ffffff&d@�    A���@F��@
�0C3@E���@E�R�@E�f�@F���@Gd@F���@Aq=
ף�c@�A�v�@F�Z@A���@
�0C4@E��@EV�@E���@F��@F���@F    �@A���(\Gb@�Ax�@A���@
A��(\�rd@�6C5@E��@EL�@G���@F��@F�D�@FA��@A��@A�G�zlb@~
    A���@
A=
ףpmd@�C6@E�@E��@Ffffff�b@�0F���@Fj�@F�e�@A�@A�}�@A1�@A�@
�$C7@E�g�@E��@F�~�@F���@F�G�z�c@�F���@A���@A=
ףp-b@�  A���@A�@
�<C8@E�@E���@F���@FAy�@F��@F�g�@A��@FW� A=
ףpMa@~

�@�$C9@E�s�@E�W�@F��@F��@Ffffff�a@F������c@�A�G�@A���@A��@A���@
�C:@E*�@GF�G�ztc@~
F��@Gq=
ףb@�Fs�@A���@A���@Ar�@
Aq=
ף�c@�C;@EZ�@G�G�b�@Fx�@F��(\��b@�F�a@A��@I� A"�@A�^�@
�B�XZR~Rh��hvftzLx�bbb�bnlbx`n� �!�"�#�$�%�&�'�(�)�*�+�,�-�.�/�0�1�2�3�4�5�6�7�8�9�� C<@E���@E0�@ Ffffff^d@� G�@F���@ F=
ףpb@~
 A�4�@ Aq=
b@�     A�$�@A���@
�*!C=@E��@E�9�@F��@F��@FO�@!Fq=
ףb@~
!A���@!FfffffFb@�!  A�@Ac�@
�@FA��@"F�G�ztb@~
"F���@"A��(\��b@�"A���@A.�@ "
Aq=
ף�b@�$#C?@Ew�@E�@G�^�@G�@#F=
ףpmb@�#F���@A��@#A�G�zTb@�# A?�@Ab�@
�$$C@@E��@E�"�@F$�@F��@$F��(\�Zb@�$F���@A��@AHb$    A�Q���`@~
$
A���@�0%C�@@E��@EA�@F��@F��@F���@F���@%A��(\��a@%I% A=
ףpa@~
%
A���@�$&CA@E��@E4�@F�>�@F��@&F�Q��ka@�&F��@A��@&Aq=
ף�a@~
&   A���@&
A�G�zLc@�'C�A@E�l�@E�F�@'F���(\d@�'F��@G�@F��@A��@'A=
ףp�a@~
'   A���@'
Aq=
ף�c@�*(CB@Eg�@E�]@Fpd@F���@F%�@(F��(\��`@�(A���@A�[�@A�@A�c@
�B)C�B@EF�@E?�@F���@G+�@GP�@F&�@A���@A���@A�I�@ )
A�Q���c@�**CC@E�4�@E���@F���@F���@F�/�@*F��(\��`@�*A���@F��@A�*�@A���@
�+C�C@E���@EO�@F���@+F���(\d@+F���(\W`@�+F���@A��@+A�Q��ka@�+   A�,�@A���@
�H,CD@E��@E�@G�d@G�L�@F:�@F���@Au�@At�@A�L�@A��@
�0-C�D@E��@E�3�@F)�@F���@FV�@F��@-A�Q��c@-Afffff�a@�-   A}�@A���@
�0.CE@E��@E��@G�d@G���@F��@F���@.A�Q���b@.A���(\�a@.    A�Q���`@.
A�Q��sc@�$/C�E@E�}�@E��@F���@FA��@/F=
ףp�`@/Fq=
ף�a@~
/A��@/Aq=
ף�a@�/  AH�@F0c@
�60CF@E9�@E'�@GA�@FA��@F�   �@F���@A��@0A�Q���a@0   Ffffff�`@0
H�$1C�F@E�G�@E�W�@F�e@F���@1F��(\�ja@~
1F&�@1Afffff�b@�1A��@F�`@A��@
�<2CG@E�@�@Eo�@G�e@G��@F��@F��@F�@A���2 H2
A�Q���c@�03C�G@E���@EY�@F�(�@F;�@F�Y�@FJ�@3A������b@�3A���@A���@    3
H�*4CH@E�
�@E���@F���@F���@F��@4Ffffff�b@~
4A�t�@4F������b@�4  A���@A�@
�$5C�H@E*�@E��@F��@G6�@5F���(\�b@�5F���@A��@5A�G�zDc@�5 F��@A��@
�6CI@EE�@E��@G��@6F���(\Wc@�6F�@F!�@A/�@AC�6    A=
ףpma@~
6
A�#�@�7C�I@EK�@E��@FA��@7F�Q��#d@~
7F�-�@7F�Q��d@~
7A���@7A�Q���c@�7   A�^�@AA*�@
�$8CJ@F�?�@E``@G���@FA�@8F������c@8G8A�Q��Sd@�8 A���@A.�@
~
9C�J@9H~
��v�xvp��bXb�Ln|�hv\fvxv�l>�@9:A
�7����D
��
����JanetJanet HovisMicrosoft Excel@LS��*�@��`L�K�Gf�����%  �Ji�8'��'����"System
�-����-'�����   �7'�����    �7'�����    �7����- �������-��-@    !���-����-�7����-����-  -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$      --� ����-   -$

    --� ����-   -$

--� ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   ����-   -$--�   -'��'��--�7-����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)*    *   ))--�   ����-   -$  )   *
*
)   )--�    ����-   -$
)
**)
)--�    ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  ����-   -$)**))--�  -'��'��--�7-����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;    ;   ::--�   ����-   -$  :   ;
;
:   :--�    ����-   -$
:
;;:
:--�    ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  ����-   -$:;;::--�  -'��'��--�7-����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KL    L   KK--�   ����-   -$  K   L
L
K   K--�    ����-   -$
K
LLK
K--�    ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  ����-   -$KLLKK--�  -'��'��--�7-����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]    ]   \\--�   ����-   -$  \   ]
]
\   \--�    ����-   -$
\
]]\
\--�    ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  ����-   -$\]]\\--�  -'��'��--�7-����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mn    n   mm--�   ����-   -$  m   n
n
m   m--�    ����-   -$
m
nnm
m--�    ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  ����-   -$mnnmm--�  -'��'��--�7-����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~     ~~--�   ����-   -$  ~

~   ~--�    ����-   -$
~
~
~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    ����-   -$~~~--�    -'��'��--�7-����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$��    �   ��--�   ����-   -$  �   �
�
�   �--�    ����-   -$
�
���
�--�    ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  -'��'��--�7-����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$��    �   ��--�   ����-   -$  �   �
�
�   �--�    ����-   -$
�
���
�--�    ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  -'��'��--�7-����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$��    �   ��--�   ����-   -$  �   �
�
�   �--�    ����-   -$
�
���
�--�    ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  -'��'��--�7-����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$��    �   ��--�   ����-   -$  �   �
�
�   �--�    ����-   -$
�
���
�--�    ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  -'��'��--�7-����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$��    �   ��--�   ����-   -$  �   �
�
�   �--�    ����-   -$
�
���
�--�    ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  ����-   -$�����--�  -'��'��-���� Arial-     ��� 2

1   2
+
2   2
<
3   2
M
4   2
^
5   2
o
6   2
�
7   2
�
8   2
�
9
2
�10
2
�11--   ������-7-@  !�--�7-����-
-$--�
����-
-$--�
����-
-$--�
����-
-$--�
����-
-$--�
����-
-$--�
����-
-$--�
����-
-$--�
����-
-$      --�
����-
-$

        --�
����-
-$

--�
����-
-$--�
����-
--�
����-
--�
����-
-$--�
����-
-$--�
����-
-$--�
-'��'��--�7-����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$��    �   ��--�
����-
-$� �
�
�   �   --�
����-
-$�
���
�
--�
����-
-$�����--�
����-
��--�
����-
--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
-'��'��--�7-����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$��    �   ��--�
����-
-$� �
�
�   �   --�
����-
-$�
���
�
--�
����-
-$�����--�
����-
��--�
����-
--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
-'��'��--�7-����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
����-
-$��    �   ��--�
����-
-$� �
�
�   �   --�
����-
-$�
���
�
--�
����-
-$�����--�
����-
��--�
����-
--�
����-
-$�����--�
����-
-$�����--�
����-
-$�����--�
-'��'��--�7-����-
-$55665--�
����-
-$55665--�
����-
-$55665--�
����-
-$55665--�
����-
-$55665--�
����-
-$55665--�
����-
-$55665--�
����-
-$55665--�
����-
-$55    6   65--�
����-
-$5 5
6
6   5   --�
����-
-$5
566
5
--�
����-
-$55665--�
����-
65--�
����-
--�
����-
-$55665--�
����-
-$55665--�
����-
-$55665--�
-'��'��--       ��� 2
JA      2
�B      2
�C      2
D   -����-
    ������@ !�@ !�-&&iAFf

(

    ���62'��'��-    --'�����    ����7����-  ���@    !��]����� Calibri--
Alberta Weekly Fed Steer Prices

���� Calibri-2
.Source:  CanFax2
2   Cdn $/cwt���� Calibri-
2Week
22012
22013
_2014   2
pK1���� Calibri-2
p�114.992
p�117.452
p130.56-    2
�K2-2
��113.912
��116.502
�133.64-    2
�K3-2
��114.162
��115.262
�138.51-    2
�K4-2
��113.662
��115.102
�142.52-    2
�K5-2
��112.082
��116.242
�139.78-    2
�K6-2
��110.792
��116.332
�138.14����--'��--���   �7����-�    �����������-��-@    !��-��-@    !��-��-@    !��-]]�-@   !�h]-+�]�-@ !�2+�-+�]�-@    !�2+�-+�]�-@    !�2+�-6]6-@ !�L6-nn�-@  !�hn-�-@    !��-o���-@  !�go�-o���-@    !�go�-o���-@    !�go�-o6�6-@    !�go6-8-@   !�-**8-@    !�*-;;8-@   !�;-LL8-@   !�L-8-@ !�-��8-@    !��-��8-@   !��-��8-@   !��-��8-@   !��-��8-@   !��--'��-���    ����7�7-'��-��� ����7�- �����-7-@   !�7-�-@ !��-'��#A���7(��
���CCAAlberta Fed Steer PriceWorksheets
 !"#$%&'()*����,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv����xyz{|}~����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������Root Entry���������F����Workbook�������������USummaryInformation(����+L�DocumentSummaryInformation8������������w
</code>
When I access the link : https://www.canfax.ca/Report/SpreadsheetReport.aspx?catalogue=FullSpreadsheets&group=Weekly+Fed+Steer&report=Alberta in the browser, an excel file gets downloaded:

For the spreadsheet link, These are the headers and payload values:

I tried a lot and I am new to scraping and python. Please help. Thank you
Comment: Excel files are not basic text you need to read the file correctly. Try `xlrd`
Comment: @ViaTech Could you give an example of how i could use it with the url?
Comment: @ViaTech I tried the xlrd method and i got an error: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n<!DO'
Answer: This worked : excel_data = pd.read_excel(t.content). now read_excel reads binary data as well


c/c++ Linux equivalent of "bool DllMain()" - but I need to return failure to dlopen()

Question: I am porting a DLL from Windows to Linux (OS X actually).
I used this StackOverflow article to do that change.
i.e. I've ported the Windows "bool DllMain()" to the Linux way:
<code>__attribute__((constructor)) void dllLoad();

__attribute__((destructor)) void dllUnload();
</code>
... but both are void return types. I need to be able to do the same as Windows and return FALSE if a condition isn't met in the constructor so that the dlopen() fails and the .so doesn't load.
How do I get the calling dlopen() to fail?
Comment: Try throwing an exception.  I don't know if this will work or not, but if dlopen is sane (and it probably is) it should catch it and indicate a failure.
Comment: @tjameson This isn't a constructor in the C++ sense. It's a special function called when the dynamic library is loaded.
Comment: you want dlopen to fail if some runtime criteria within the shared library isn't met?
Comment: What does this have to do with C? C doesn't have formal constructors.
Comment: @JamesMcLaughlin - Huh, didn't know that. That's actually pretty cool.
Comment: @SamMiller Yes. Based on hardware capabilities I need to abort loading so that the parent program doesn't use this .so
Comment: Are you just interested in the architecture as a capability ? If so you can hack your way through it.
Comment: dlopen() is not allowed to fail in dllLoad.
Answer: The answer is that it isn't possible.
As stated above you can't error out in the constructor - be it an exception or exit()
Answer: You need to approach this differently.
If you are dynamically loading a library, than you must also be using GetProcAddress() and dlsym() to actually do anything with it. dlsym() is your path forward here.
You clearly control the plugin's code, since otherwise you couldn't even add these APIs to it. So, all your 'dllmain' on either platform needs to do is set some global 'is valid' state information. You then simply call a known API like "ModuleIsValid()" whose entire job is to simply read that state info & return true/false. If it returns false, you close the library and report failure.


Null Hypothesis formulation from a "Fewer than" statement

Question: The aim of this exercise is to translate statements into Null and Alternative Hypotheses.
The example statement is: "Fewer than 5% of adults ride the bus to work in Los Angeles."
My Answer was : 0: ≤0.05;: >0.05.

Given that 0 always has a symbol of =, ≤, or ≥.
Given that  always has a symbol of >, <, ≠.

However, the correct answer is : 0: =0.05;: <0.05

Why is the 0 statement 0: =0.05 and not : >0.05 ?
Why is the  statement : <0.05 and not : >0.05 ?
Answer:
"Fewer than 5% of adults ride the bus to work in Los Angeles."

The null hypothesis to be verified is the complement: $H_0\geq 5\%$
To set up a test, we expect that the data contraddict this Hypothesis, i.e. sample mean is $4\%$. The test is performed to tell us if actually $4<5$ or the difference has to be attributed to the phenomena's variability.
The system is the following
$$\begin{cases}
H_0: p\geq 5\%\\
H_1: p<5\%
\end{cases}$$
This is a good answer and correct too.
For some reasons that should be clear to you, the previous Hypothesis System is equivalent to the following one
$$\begin{cases}
H_0: p= 5\%\\
H_1: p<5\%
\end{cases}$$


react js which col number value with function call. Failed prop type: invalid prop 'md' of type function supplied to Col, expected number

Question: I have a react js component which has Col element as defined
<Col md= {3}>

</Col>

When I used a number like 3 or 2, it works fine.
I added a function to check for an incoming props bool value and switch between 3 and 2 like the following
<Col md={ this.getColVal}>
</Col>

getColVal = () => {
  return (this.props.IsValue === true) ?  3 : 2;
}

Also tried in the constructor
const colVal = (this.props.IsValue === true) ?  3 : 2;

I am getting this error
Failed prop type: Invalid prop 'md' of type 'function' supplied to 'Col', expected number
Any help is appreciated. Thanks
Comment: Why don't you simply use ``?
Answer: You need to call the getColVal method as below:
<Col md={ this.getColVal()}>
</Col>

by using above approach you will get the integer return value.
Comment: He was getting a function, not undefined, otherwise, you're right.
Comment: awesome. Thanks.


How to take the sum of the value of keys?

Question: I have a collection like this:
{
    "2.2": {
        "BALL":{"white":9,"black":11,"red":4},
        "BAG":{"white":2,"black":11,"red":1},
        "COVER":{"white":3,"black":8,"red":1}
    },
    "2.3": {
        "BALL":{"white":1,"black":0,"red":7},
        "BAG":{"white":3,"black":0,"red":0},
        "COVER":{"white":9,"black":2,"red":28}
    }
}

I want to calculate the sum of colors (white, black, red) are there in total with respect to "2.2" and "2.3".
OUTPUT
The desired output should be like this:
{
    "2.2": {"white": 14, "black": 30, "red": 6},
    "2.3": {"white": 13, "black": 2, "red": 35}
}

I have tried using underscore .map function but i am not able to understand it properly. Anyone can help please. ?
PS: Need a solution using underscore.js only.
Comment: What do you mean by "with respect to "2.2" and "2.3"" ?  You want two sums?
Comment: An easy solution would be to do it in a nested iteration. Outer iteration over 2.2 and 2.3 and inner iteration calculates sum of whites...
Comment: Please check the edited question. I m sorry i didnt explain it in the good manner.
Answer: try :
var obj = {
    "2.2": {
        "BALL": {
            "white": 9, "black": 11, "red": 4
        },
        "BAG": {
            "white": 2, "black": 11, "red": 1
        },
        "COVER": {
            "white": 3, "black": 8, "red": 1
        }
    },
    "2.3": {
        "BALL": {
            "white": 1, "black": 0, "red": 7
        },
        "BAG": {
            "white": 3, "black": 0, "red": 0
        },
        "COVER": {
            "white": 9, "black": 2, "red": 28
        }
    }
}
var w = 0;
for(var i in obj){
    for(var j in obj[i]){
        w +=obj[i][j]['white'];
    }
}

UPDATE:
var obj = {
    "2.2": {
        "BALL": {
            "white": 9, "black": 11, "red": 4
        },
        "BAG": {
            "white": 2, "black": 11, "red": 1
        },
        "COVER": {
            "white": 3, "black": 8, "red": 1
        }
    },
    "2.3": {
        "BALL": {
            "white": 1, "black": 0, "red": 7
        },
        "BAG": {
            "white": 3, "black": 0, "red": 0
        },
        "COVER": {
            "white": 9, "black": 2, "red": 28
        }
    }
}
for (var i in obj) {
    var tmp = {
        white: 0,
        black: 0,
        red: 0
    };
    for (var j in obj[i]) {
        tmp['white'] += obj[i][j]['white'];
        tmp['black'] += obj[i][j]['black'];
        tmp['red'] += obj[i][j]['red'];
    }
    obj[i] = tmp
}
console.log(obj);

Comment: Your solution is summing all the white colors. Please check the desired output i have mentioned. Thanks
Comment: @ahmad hamza already updated to get the **OUTPUT** as you later added.
Comment: Thanks a lot. I have marked it as accepted answer though it is not using underscore.js methods.
Answer: Here is the underscore.js solution.
_.mapObject(collections, function(collection) {
  return _.reduce(collection, function(memo, col) {
    // credit: Bergi on http://stackoverflow.com/a/17350790/1465828
    // the following line reads:
    // for (var p in col) => loop all properties in col
    // memo[p] = ... => assign memo[p] as ...
    // (p in memo ? memo[p] : 0) => IF property p exists in memo, then memo[p], otherwise 0
    // + col[p] => add col's p
    for (var p in col) { memo[p] = (p in memo ? memo[p] : 0) + col[p]; }
    return memo;
  }, {});
});

Comment: I'm getting type error with `mapObject` function. Any idea why ?
Comment: Which version of `_.js` do you use please? I can find `mapObject` in http://underscorejs.org/
Comment: `map` is for Array, while `mapObject` is for Object.
Comment: The latest version is 1.8.3, I have looked into 1.7.0 source, and found that there is no `mapObject` yet. So, my solution won't work for v1.7.0
Comment: what is that `p in memo` ?


"No modal dialog is currently open" after setting up proxy Python Selenium

Question: I'm using a proxy in my webdriver, and it has a user/pass authentication dialogue that pops up. After a lot of searching I decided the easiest way to deal with it is to use an autoit script which looks like this:
<code>#Include <File.au3>

WinWaitActive("Authentication Required")
Send(proxyuser)
Send("{TAB}")
Send(proxyuser)
Send("{ENTER}")
</code>
I invoke the script like so:
<code>subprocess.run('proxyscript.exe')
</code>
The problem is, after I do this, anything running in my python selenium web driver script gives the error "No modal dialog is currently open". I'm not really sure what it means or how to fix it.
My webdriver settings look like this:
<code>        caps = DesiredCapabilities().FIREFOX
        caps["pageLoadStrategy"] = "eager"
        fp = webdriver.FirefoxProfile('firefoxadblocked')

        proxyString = ad_proxy_array[0] + ':' + ad_proxy_array[1]

        caps['proxy'] = {
                    "proxyType": "manual",
                    "httpProxy": proxyString,
                    "ftpProxy": proxyString,
                    "sslProxy": proxyString
                }
</code>
Comment: You should be able to handle the pop with webdriver itself without the use of autoit.
Comment: How so? I've searched about using proxies with authentication and just can't find a simple solution to it directly in selenium.
Comment: https://stackoverflow.com/questions/27322871/handling-authentication-required-alert-box-with-python-2-7-selenium-webdrive
Answer: Such tricks were possible in FF 53. I've moved to FF 60 recently, and it's over. Once you change something with Autoit (open tabs, change UserAgent, for example), Selenium gets broken.


$\text{sin}(\sqrt{x+4n^2\pi^2})\to 0$ as $n\to\infty$

Question:
Show that $\text{sin}\sqrt{x+4n^2\pi^2}\to 0$ as $n\to\infty$ for each $x\in\mathbb{R}$.

Intuitively, it makes sense to me since $x+4n^2\pi^2$ is approximately $4n^2\pi^2$ for large $n$, but I want to be sure I can show this formally. $\frac{x+4n^2\pi^2}{4n^2\pi^2}\to 1$ as $n\to\infty$, $\frac{x+4n^2\pi^2}{4n^2\pi^2}=\epsilon(n)$, where $\epsilon(n)\to 1$ as $n\to\infty$.
So $\underset{n\to\infty}{\text{lim}}\text{sin}(\sqrt{x+4n^2\pi^2})=\underset{n\to\infty}{\text{lim}}\text{sin}(\sqrt{4n^2\pi^2\epsilon(n)})=\text{sin}(\sqrt{\underset{n\to\infty}{\text{lim}}4n^2\pi^2\underset{n\to\infty}{\text{lim}}\epsilon(n)})=\text{sin}(\sqrt{\underset{n\to\infty}{\text{lim}}4n^2\pi^2})=\underset{n\to\infty}{\text{lim}}\text{sin}(\sqrt{4n^2\pi^2})=0$
Does this look correct? Thank you
Comment: $n$ is assumed to be an integer
Comment: @TomChalmer: You can write `\sin` for the sine and `\lim_{n \to \infty}` for the limit.
Answer: Your argument is not correct.
$$
 \lim_{n \to \infty} \sin \sqrt{4n^2\pi^2 \epsilon(n)} =
  \sin \sqrt{\lim_{n \to \infty} 4n^2\pi^2 \epsilon(n)}
$$
assumes that
$\lim_{x \to \infty} \sin x$ exists, and that is not the case.
But you can use the mean value theorem.
Since $|\sin'(x)| = |\cos (x)| \le 1$ we have for fixed $x \in \Bbb R$
and all sufficiently large $n \in \Bbb N$ (such that $ x+4n^2\pi^2 \ge 0$):
$$
|\sin(\sqrt{x+4n^2\pi^2})| = |\sin(\sqrt{x+4n^2\pi^2}) - \sin(\sqrt{4n^2\pi^2}) |\\
 \le \left | \sqrt{x+4n^2\pi^2} - \sqrt{4n^2\pi^2} \right |\\
 = \frac{|x|}{\sqrt{x+4n^2\pi^2} + \sqrt{4n^2\pi^2}}
 \to 0
$$
for $n \to \infty$.
Answer: $$\sin\sqrt{x+4n^2\pi^2}-\sin2n\pi=2\sin\frac{\sqrt{x+4n^2\pi^2}-2n\pi}{2}\cos\frac{\sqrt{x+4n^2\pi^2}+2n\pi}{2}=$$
$$=2\sin\frac{x}{2\left(\sqrt{x+4n^2\pi^2}+2n\pi\right)}\cos\frac{\sqrt{x+4n^2\pi^2}+2n\pi}{2}\rightarrow0$$


Add PayPal WPS payment payload information to Commerce admin e-mail

Question: I'm using Commerce with the Commerce E-mail module to customize the e-mail sent to admins whenever there is an order (a payment is received) at the store.  Payment is handled using PayPal WPS through the Commerce PayPal module.
On my site, buyers must input their e-mail addresses before moving on to PayPal, but these e-mail addresses (on my site) sometimes differ from the e-mail addresses registered on buyers' PayPal accounts.
For this reason, I want the admin e-mail (sent when an order is paid for) to include both the e-mail address that the buyer input on my Drupal site as well as their PayPal e-mail address.
Commerce E-mail provides a [commerce-email:order-items] token, but there is no token for the PayPal WPS information.  This information is stored within Drupal; in the Order Admin UI, if you click on the payment details of the transaction, under Payload there is a printout of an Array that includes [payer_email] => email@example.com.
Payload array:
Array (
    [13775355555-ipn] => Array
        (
            [mc_gross] => 17.00
            [invoice] => 555-1377531555
            [protection_eligibility] => Eligible
            [payer_email] => email@example.com

How can I add the buyer's PayPal registered e-mail address ([payer_email]) into the admin e-mail sent by Commerce?  It would be great if the solution would also allow use of other information in the payload array, such as the buyer's physical address.
Answer: You can extract those array and add them data object. After adding them to the token replace function you can use that to create custom tokens which will help you to get the tokens.
That custom tokens can now be used to get the buyer's email address
// Write a code for porting data to token_replace()
$data = new val();
$data->mc_gross = variable_get('mc_gross', '');
$data->invoice = variable_get('invoice', '');
$data->protection_eligibility = variable_get('protection_eligibility', '');
$data->invoice = variable_get('payer_email', '');
token_replace(['payload']', (array) $data);

--------write this in  module_name.tokens.inc file------------------------------------------
 function hook_token_info() {
      $info = array();

      // Add any new tokens.
      $info['tokens']['payload']['mc_gross'] = t('Gross');
      $info['tokens']['payload']['invoice'] = t('Invoice');
      $info['tokens']['payload']['protection_eligibility'] = t('Protection Eligibility');
      $info['tokens']['payload']['payer_email'] = t('Email');
      // Return them.
      return $info;
    }

    function hook_tokens($type, $tokens, array $data = array(), array $options = array()) {
     $replacements = array();

    if ($type == 'payload') {
    foreach ($tokens as $name => $original) {
          switch ($name) {
            case 'mc_gross':
              $replacements[$original] = $data['mc_gross'] ;
              break;
            case 'invoice':
              $replacements[$original] = $data['invoice'];
              break;
            case 'protection_eligibility':
              $replacements[$original] =  $data['protection_eligibility'];
              break;
            case 'payer_email':
              $replacements[$original] = $data['payer_email'];
              break;
          }
        }
      }

      return $replacements;
    }

Comment: Thanks for this, but where should I place the first part of the code (**// Write a code for porting data to token_replace()**)?


A major on all 6 strings of the guitar

Question: I know that the A major chord is commonly played by fretting strings 2, 3, and 4, then playing only strings 1 - 5, giving the notes A, E, A, C♯, and E, notes in the A major triad.
If I also add the 6th string, E, that note is also part of the triad. Isn't this still an A major chord? I understand that the 6th string E "roots" the chord at E, but the notes are still the same ones as the triad. Isn't this just a different voicing?
Comment: You are playing an A major chord as long as what you play consists of the A, C#, and E notes. If A, C#, and/or E appear multiple times in the chord, it is still A major. You can play this combination of notes all over the neck in different octaves and strings. If C# is your bass note then it is called the first inversion. If E is your bass note then it is called a second inversion.
Answer: It is a different voicing, but since the lowest note i.e. the bass note is different, it's also a different inversion, and it changes the chord's function a little bit. With the lowest E sounding your chord is an A/E, which is slightly different. It's still an A major, but it's a second inversion A major. (First inversion would have C# as the lowest note. And the regular one with A in the bass is a root position A major chord.)
A/E is commonly used in ending chord sequences like: A/E - E7 - A. When the A/E chord is sounding, I don't feel like being completely home yet, but when the bass moves to A, then I feel completely at home and at rest. Here's a question about the properties of different inversions and the second inversion specifically: Why are second inversion triads considered less consonant than first inversion triads?
Another important thing is that playing both the low A and E at the same time makes the sound a little muddy and unclear. When playing an A/E, you should mute the low A so that there aren't low sounding notes that are too close to each other. Because of the muddiness issue, play either the low A or the E, but not both at the same time. For higher pitches, having notes in a very close voicing is not a problem.
The low E can also be used in alternating bass that's used a lot in country, latin, polka, etc. Play A, A/E, A, A/E ... alternating the bass note between A and E, while keeping the rest of the chord notes the same.
Comment: Nice answer, alternating bass is a great way to incorporate the low string 5th on this and the C shape too.
Answer: It is simply a different voicing - but it doesn't really sound very good.
The E underneath might just be the 5th of the A chord, but it's just a bit overpowering that loud when it's below the low A.
You can usually get away with it ringing in sympathy a bit, if you're not fully damping it, but hitting it loud & proud doesn't really work.
Comment: Well put, I agree. Ralph, try an E shape barre chord at the 5th fret if you want a 6 note A chord.
Comment: @JohnBelzaguy - and that alternating (1>5) bass works there too, fretted>open.

A major on all 6 strings of the guitar


How to get a configuration array of objects using viper in golang?

Question: I have a config file that looks like:
{
    "auth": {
        "client_id": "foo",
        "client_secret": "bar",
        "account": "firstname.lastname@example.com",
        "secrets_type": "auto"
    },
    "jobs": [
    {
        "source_folder": "folder1",
        "create_albums": "off",
        "delete_after_upload": false,
        "include": [
            "*.jpg"
        ],
        "exclude": []
    },
    {
        "source_folder": "folder2",
        "create_albums": "folderName",
        "delete_after_upload": true,
        "include": [
            "*.jpg", "*.webp"
        ],
        "exclude": [
            "abc*"
        ]
    }
    ]
}

And I'm reading it using the viper package in golang.
I'm having problems to get the jobs settings:
    type FolderUpload struct {
        sourceFolder      string   `mapstructure:"source_folder"`
        createAlbums      string   `mapstructure:"create_albums"`
        deleteAfterUpload bool     `mapstructure:"delete_after_upload"`
        include           []string `mapstructure:"include"`
        exclude           []string `mapstructure:"exclude"`
    }

    // Read the "jobsSettings" section as a slice of maps
    jobsSettings := viper.Get("jobs").([]interface{})

    // Iterate over each job item
    for _, job := range jobsSettings {
        jobMap := job.(map[string]interface{})

        j := FolderUploadJob{
            sourceFolder:      jobMap["source_folder"].(string),
            createAlbums:      jobMap["create_albums"].(string),
            deleteAfterUpload: jobMap["delete_after_upload"].(bool),
            include:           jobMap["include"].([]string),
            exclude:           jobMap["exclude"].([]string),
        }
    }

I've tried with viper.UnmarshalKey() too, but I always receive a panic when trying to read the include and exclude.
Can anyone point me to the right way?
Answer: viper.UnmarshalKey works and please note that the fields of FolderUpload should be exported. See the demo below:
package main

import (
    "fmt"
    "strings"

    "github.com/spf13/viper"
)

func main() {
    viper.SetConfigType("json")

    jsonConfig := `
{
    "jobs": [
    {
        "source_folder": "folder1",
        "create_albums": "off",
        "delete_after_upload": false,
        "include": [
            "*.jpg"
        ],
        "exclude": []
    },
    {
        "source_folder": "folder2",
        "create_albums": "folderName",
        "delete_after_upload": true,
        "include": [
            "*.jpg", "*.webp"
        ],
        "exclude": [
            "abc*"
        ]
    }
    ]
}`

    type FolderUpload struct {
        SourceFolder      string   `mapstructure:"source_folder"`
        CreateAlbums      string   `mapstructure:"create_albums"`
        DeleteAfterUpload bool     `mapstructure:"delete_after_upload"`
        Include           []string `mapstructure:"include"`
        Exclude           []string `mapstructure:"exclude"`
    }

    if err := viper.ReadConfig(strings.NewReader(jsonConfig)); err != nil {
        panic(err)
    }
    var jobs []FolderUpload

    if err := viper.UnmarshalKey("jobs", &jobs); err != nil {
        panic(err)
    }

    fmt.Printf("%+v\n", jobs)
}

Comment: Thanks a lot @Zeke-lu I guess the key was to export the fields inside the struct, because I already tried with `UnmarshalKey` before without success... :clap:
Comment: Yes, the key was to export the fields. Such kind of problems happen too many times. See this answer that list some of them: https://stackoverflow.com/a/76320290/1369400.


Transmitting a large image incrementally via patches over a low bandwidth connection

Question: Scenario: Most smartphones have a high resolution camera that generates photos that may range up to 1-1.5 MB. Sending that over a low bandwidth connection like 2G would take a long time.
Is there an algorithm that can:

Create a highly compressed first-view (like a thumbnail, but the same size) and send that across, and
Create subsequent 'patches' of data that add to the first file?
The more 'patches' that are received, the higher the end-quality, until the original is completely reconstructed.

I'm looking for something that would work on jpeg images.
Comment: JPEG has already that built in. See progressive JPEG http://www.faqs.org/faqs/jpeg-faq/part1/section-11.html http://whatis.techtarget.com/definition/0,,sid9_gci212835,00.html
Comment: Hi there EugeneO - I had a look at progressive jpegs earlier, but thanks for the links! As far as I understand it, its still part of the same 'connection' / stream. However, I'm looking for implementing the same concept, but as 'patch' files that are sent discretely after the first. Webservers will probably reject a partial upload of an progressive jpeg which is 1.5 MB big, but the chances of a 100 kb initial file being uploaded successfully are much higher.
Comment: I'm also exploring breaking the jpeg into Luma, Cb, Cr channels like http://stackoverflow.com/questions/1685739/using-existing-tools-how-can-i-extract-into-separate-images-the-luma-cb-cr-ch and then piecing it back together.
Answer: You can use any progressive image compression for such requirements. JPEG 2000 has a good progressive encoding. You can even specify progressive transmission count. So, you can implicitly set patch size. If that's not enough for you, then you can use the other progressive wavelet image compression algorithms such as EZW, SPIHT etc.
Comment: Hi there. I've been reading up on SPIHT and that seems very close to what I need. But it seems like it will need a lot of adaptation for it to work with jpegs (their algorithm works with RAW files) and for it to output progressive quality 'patches' which I was thinking of. Do you have any other thoughts on how I can achieve my goal?
Comment: SPIHT and EZW like codecs are some research implementations. Most of research implementations have some serious limitations such as restricted to 2-power-of dimensions, only grayscale, only RAW images etc. So, unfortunately you need to cope their limitations on your own. Also, if you're familiar with wavelet algorithms, you can build your own progressive schema too. As you know, wavelet has several sub bands which stores some frequency ranges. And you can use a statistical model and a proper arithmetic coder to encode those sub bands. So, you can progressively transmit whole image.
Comment: Thanks Osman - could you help me understand what is 'progressive transmission count'? I wasnt able to find anything on the 'count' aspect.
Comment: I don't know exact term in here, so I've used "progressive transmission count". Actually, it's wavelet decomposition level and it approximately means number of patches in your term. Some image converters allow you to set it, while others simply use 5 as default.


What does the data sample (xi) means in z-score normalization?

Question: Is $\mu_i$ computed by averaging over the entire features or by averaging over each feature respectively? Here is a snapshot I found from a journal article? Can anyone help to answer the above two question?

Normalization operation is used to increase the classifier performance.  Frequent normalization methods are used after feature extraction.  However, in this study $z$-score normalization process is used.

$\begin{eqnarray}z=\frac{x_i - \mu_i}{\sigma_i}\end{eqnarray}$

Here, $x_i$ represents the data samples.  $\mu_i$ represents mean of the column and $\sigma_i$ represents the standard deviation of the same column.
Answer: Does your article state what $i$ is here?  I'm assuming $i$ index the features.  In that case, if you look at the last sentence, assuming each feature is its own column (as is standard) and indexed by $i$, $\mu_i$ is the mean for feature $i$ (i.e. averaging over all the data for that feature).
So for example if you had two features, say, income and housing price, you'd simply compute the mean and standard deviation separately for income an then for housing price.  Next you subtract each income data point from the mean income and divide by the standard deviation for the income.  You'd then do the same for housing price, only using the housing price data.
I think part of the confusion here is that $x_i$ is really a vector of your data points, and not a single value, in which case it would probably be best represented by double-subscripts like $x_{ij}$, where $i$ is the feature, and $j$ is an individual measurement of element of feature $i$.
Answer: Suppose 8,9,7,6,1,4,3,7,5 are your observation data and you want to normalise these values.
First you calculate the mean and standard deviation.
If you want to normalise the value '8' in your dataz then this is your x_i
Comment: This answer doesn't seem to address the OP's question about multiple features though.


Accessing a Windows 7 print share without a password

Question: In our network we have a Windows 7 print server. Users connect to this machine by typing <code>\\server_name</code> on their own workstations. The print server and the users` computers are members of Active Directory. In AD, only computers have accounts, users are using local accounts.
Is it possible to configure Windows 7 so that it doesn't ask for login and password when a user tries to access it from computer which is member of domain?
Comment: Why would you configure AD in this way?
Comment: Don't try to print via SMB.  Setup IPP/LPR or some other sharing protocol?
Comment: I second what @Shads0 says. Why the hell are users logging in with local accounts on domain-joined computers? That's ridiculous.
Comment: @Shads0 We don't use domain account because we don't need them. Most of network environment is based on Linux. AD is used only for remote Management of workstations south  as running startup scripts, software management etc...
Comment: @user101141 based on your question, it sure sounds like you need them to me.
Comment: @user101141 You don't need "the Active Directory" but you should implemet some "Directory Service" around according to your requeriments.
Answer: Since you have AD implemented, you're better off saving yourself a lot of other headaches in the future by giving the users accounts on the domain instead of locally and having them log in this way.  The benefits of this vastly outweigh any other type of pieced together solution that is not supported by Microsoft in the firstplace.
Answer: The user would have to logon with an AD/network account.  Alternatively, you could recreate each users' local user account on the print server (username and password MUST match).  In short, not possible.
Answer: I might be wrong but it is not possible.
User must be members of the domain to log or access computer of the domain.
With Windows 7, you might be able to store password with the credential manager but still the account need to be a active directory user account.
Answer: Creating a user entry in the directory, granting that user to access print server shares, then saving that user's cridentials to all clients should be a solution.
Edit from comments:
When local user tries to access that win7 share, o.s. will perform a login screen. That local user could type the credentials of the user, which you created and edited its permissions of that specific share, and save that information clicking "Remember bla bla".
By doing that, creating a user and cridential saving,

You can have some control over that share.
Local users won't be have to type that user-password info all the time.
Comment: What does this even mean? `"then saving that user's credentials to all clients"` That doesn't even make sense.
Comment: Maybe I let that open ended.

*Clients trying to access the share
*A login box appears
*User enters credentials
*User can save cridentials


Book recommendation for associative algebras

Question: Currently, I am reading David Radford's Hopf Algebra, and I would like to pick up some representation theory of associative algebras as well since my knowledge of them is pretty shallow at the moment.
Are there any books which gives a good account of associative algebras, and the representation theory of associative algebras?
Comment: Who on earth considers a reference request for representation theory of associate algebras to be too localized?
Answer: In addition to Mariano's list:

Etingof et al.: Introduction to representation theory (which is also available online).

Addendum: And one should also mention (although as a starter its speed is too high)

Benson: Representation and cohomology

Addendum2: One that I haven't read but seems to fit to the question:

Skowronski, Yamagata: Frobenius algebras
Comment: [Etingof et al.] has a much more all-over-the-place style which is quite great; the other three are more... programmatic :-)
Answer:
Pierce's Associative algebras is pretty great.
Auslander, Reiten and Smalø wrote the classic Representation theory of Artin algebras
Assem, Skowroński, Simpson: Elements of the Representation Theory of Associative Algebras.

The good first half of any of these and you are pretty much set.
Answer: In addition:
C.W.Curtis, I.Reiner, Representation Theory of Finite Groups and Associative Algebras, 2006.
Despite the title, it contains a lot of information on representations of algebras.


Linking directly to Wagtail modeladmin views

Question: I'm successfully using Wagtail's wagtail.contrib.modeladmin to make a regular Django model editable from the Wagtail admin. I'd like to hyperlink to the modeladmin "create" and "edit" views from my template. What is the URL "name" I can use to reference these views? There appears to be no urls module in wagtail.contrib.admin and no documentation on this.
Here's my directory app's models.py:
from django.db import models

class Organisation(models.Model):
    title = models.CharField(max_length=255)
    logo = models.ImageField(upload_to='logos', blank=True)
    ...

And my project's urls.py:
from django.conf.urls import include, url

urlpatterns = [
    url(r'^admin/', include(wagtailadmin_urls)),
    ...
    url(r'^directory/', include('directory.urls')),
]

Answer: The URL name will be of the following form, where [action] is one of index, create, edit, delete, inspect or choose_parent:
[app_label]_[model_name]_modeladmin_[action]

For example:

{% url 'directory_organisation_modeladmin_index' %}
{% url 'directory_organisation_modeladmin_create' %}
{% url 'directory_organisation_modeladmin_edit' instance_pk=37 %}
{% url 'directory_organisation_modeladmin_delete' instance_pk=37 %}
{% url 'directory_organisation_modeladmin_inspect' instance_pk=37 %}
{% url 'directory_organisation_modeladmin_choose_parent' %}

Since the URLConf provided doesn't contain a namespace argement to include(), no namespace is required.
Similar to Django's built-in Admin, Wagtail generates these URLs dynamically in the ModelAdmin.get_admin_urls_for_registration() with some helpers like AdminURLHelper.get_action_url_name().


What is $\gcd(0,a,b)$?

Question:
What is $\gcd(0,a,b)$ when  $a$ and $b$ are relatively prime?

$\gcd(a,b,c)$ is the greatest integer $d$ which divides $a,b$ and $c$ simultaneously. In that case what is $\gcd$ of any triplet containing exactly one zero? If we define $(0,a,b)=\min(a,b)$, then we can easily find a counterexample (As $(0,3,4)$). Can we say that $\gcd(0,a,b)$ is $1$ if $a$ and $b$ are relatively prime?
Comment: $1$ is the largest dividing $a$ and $b$ if they are relatively prime by definition. Does $1$ divide $0$ as well?
Comment: Why did you choose to use $\min$? Wouldn't $\gcd$ be more natural?
Comment: Simply Beautiful Art, yes it does.
Comment: Arthur, you are right. I should have thought a bit more before posting this...
Answer: An answer more general than the question:
If you use the standard definition of gcd in algebra, namely, $\gcd(a_1,a_2,\dots,a_n)$ is the nonnegative generator of the ideal $\langle a_1,a_2,\dots,a_n\rangle$, it is obvious that $\gcd(0,a,b)$ is the nonnegative generator of the ideal generated  by $a$ and $b$, i.e.
$$\gcd(0,a,b)=\gcd(a,b)=1.$$
Comment: BillBubuque: Yes, but the O.P. asked his question in the context of integers.
Comment: While researching to write an answer, I found a statement that gcd(a[1], a[2], ... a[n-1], a[n]) = gcd(gcd(a[1], a[2], ... a[n-1]), a[n]) if *none* of the *a*s are zero. But I don't see why it shouldn't suffice that *not all* of the *a*s are zero.
Comment: This only works for PIDs, but the result holds much more generally in any GCD domain (or whenever the gcd exists).
Comment: I agree with you. I can't see any side-effect of extending the definition.
Answer: $d\mid 0,a,b\iff d\mid a,b\ $ so $\ 0,a,b\,$ and $\,a,b\,$ have the same set $S$ of common divisors $\,d,\,$ so they have the same greatest common divisor $(= \max S)$
Comment: The same argument works if we replace $0$ by $\,ja+kb\,$ for any $\,j,k\in\Bbb Z\ $ (OP is case $\,j,k=0)\ \ $
Answer: The number $0$ is the greatest element in the partial ordering of the set of integers w.r.t. divisibility. Thus the gcd of $0,a,b$ equals the gcd of $a,b$.

What is $\gcd(0,a,b)$?


Does linear mixed effect models fit to analyse nested data?

Question: I am searching for correlations between symptoms severity and the parameters of neural activity in a disease model. As predictors, I have two (slightly collinear) measures of disease severity on each side of the body that may differ (sometimes substantially) on the right and left body sides. Also, in each brain hemisphere, I have several recordings (4-16) from the structure involved in pathology – and for each recording, we have several estimated parameters of activity. And I want to study how these parameters correlate with symptoms severity.
So, I am wondering what is the best way to analyze the data that would account for all these conditions.
Now I am thinking about using mixed linear model function for my task (nlme package):
fit <- lme(parameterA ~ measure1 + measure2 + random=~1|animal/side)
Here I wish to account for a subset of parameters derived from the given hemisphere. All the parameters in the hemisphere may be seen as repeated measures. But two subsets of recordings from the right and left hemisphere may also be dependent as they come from one animal.
Is my approach correct? Do I need to account for 'measure1 * measure2' instead of 'measure1 + measure2'? Do the variables for random effect estimates (animal and side) appear in the right order?
Some of the neural parameters I want to study also have collinearity – what is the best way to account for it? I am going to analyze them in separate models and then adjust the estimates for multiple comparisons. Am I wrong?
If you know a more appropriate way to analyze such data please let me know.
Comment: As I understand this, you have 2 different measures of disease severity and each is assessed on both sides of the body. Is that correct? If so, when you specify `side` in your formula do you mean the side of the body on which the severity measure was assessed, or the side (hemisphere) of the brain from which the activity recordings were made? More information about what you mean by the "parameters of activity" might help with an answer, too. That information might help find a way to cut down on the number of models and thus on the magnitude of any multiple-comparison corrections.
Answer: I understand that measure 1 and 2 are nested within side, and animal is randomly take from the population. Assuming that parameterA is a numeric response, it could be
fit <- lme(parameterA ~ side + side:measure1 + side:measure2, data= yourData, random=~1|animal)


Form rebuild without using buttons

Question: I'm trying to figure out how to rebuild a form without using a button click trigger. Instead, I'm looking to rebuild whenever a blur event occurs on a textfield. All non-button form types lack the #submit value. It doesn't seem possible to trigger a form rebuild with $form_state['rebuild'] = TRUE; unless the submit callback is triggered. Demo code below.
function test_form($form, &$form_state) {
  $form['form_wrapper'] = array(
    '#prefix' => '
',
    '#suffix' => '
',
  );

  if (isset($form_state['textfield_vale'])) {
    $textfield_vale = $form_state['textfield_vale'];
  }
  else {
    $textfield_vale = 'not set';
  }

  //will not trigger page rebuild
  $form['form_wrapper']['test_textfield'] = array(
    '#type' => 'textfield',
    '#value' => 'Value = '.$textfield_vale,
    '#ajax' => array(
      'callback' => 'test_result',
      'wrapper' => 'form_wrapper',
      'event' => 'blur',
     ),
  );

  //will trigger page rebuild, changing the textfield value
  $form['form_wrapper']['trigger_rebuild'] = array(
    '#type' => 'submit',
    '#value' => 'I don\'t want to press it',
    '#submit' => array('submit_rebuild'), //has #submit value unique to buttons
    '#ajax' => array(
      'callback' => 'test_result_with_button',
      'wrapper' => 'form_wrapper',
    ),
  );

  return $form;
}

function test_result($form, &$form_state) { //unable to set $form_state['rebuild'] = TRUE
  $form_state['textfield_vale'] = 'textfield trigger'; //TRUE;
  $form_state['rebuild'] = TRUE;
  return $form['form_wrapper'];
}

function submit_rebuild($form, &$form_state) {
  $form_state['textfield_vale'] = 'button trigger'; //TRUE;
  $form_state['rebuild'] = TRUE;
}

function test_result_with_button($form, $form_state) {
  return $form['form_wrapper'];
}

Is there a way to trigger a form rebuild without clicking a button (perhaps with an ajax command)? Alternatively, how can I change my code to allow an ajax callback to change $form_state values?
Comment: Have you tried https://drupal.stackexchange.com/a/126952/15055?
Comment: I think this GET thing only comes from the question's code and isn't actually necessary for the autosubmit to work. But I'm unsure about that.
Comment: If not a button what else should trigger the rebuild? Onchange?
Comment: Preferably a blur event, but onchange would work as well.
Comment: How is that blur being triggered?
Comment: The blur event would trigger whenever I deselect the textfield. The solution you linked might work but would probably require a lot of rebuilding due to avoiding drupal's default submit and validation functions.
Comment: Then you could try the other solution and blur and submit at once? https://drupal.stackexchange.com/a/127158/15055
Comment: Sorry that I wasn't clear. I'm trying to rebuild the form using a process that usually requires a submit button. Both proposed solutions assume $form['#method'] = 'get'; is used for the module. I'd like to avoid using a GET method, since it removes drupal form validation and would require a significant code rebuild.
Comment: Returning to this problem. AJAX is causing me some significant trouble. I might be able to leverage the following behavior: pressing the ENTER key within a textfield triggers the 'click' event of the form's first submit button. I'd still need to figure out how to initiate the same behavior using a blur event. Will revisit prior suggestions and see if I can adapt them to my current form.
Answer: I've always handled changing form values in the form build or form_alter function itself. I can't remember where I saw to do this, but I've never been able to alter the form in the ajax callback itself.
Have your form trigger an ajax callback. This should return the form or piece of the form in question, like your test_result_with_button() function.
When an ajax callback is triggered, the build method and alters are run again. Add logic in your build / form_alter to make updates to the form or form_state based on whichever action was triggered. Information about how the rebuild was triggered will be available in $form_state['triggering_element'].
Comment: I haven't been able to update the form after AJAX callback. I can change variable values but the form elements will not change unless $form_state['rebuild'] is called. I think I might be able to get what I want with careful javascript. And to clarify, I'm trying to change the attributes of elements (disable some buttons, change the text of a textfield). Replacing them with AJAX will cause problems.
Answer: I've found that jQuery can be used to click a Drupal submit button (source). I can use an event listener to trigger a submit event by interacting with a specific button that can be hidden anywhere on the form. Below is my solution based off of my original demo code.
<?php

function test_form($form, &$form_state) {
  drupal_add_js('
(function ($) {
  Drupal.behaviors.test_form = {
    attach: function() {
      $("#test_textfield").blur(function() {
        $("#trigger_rebuild").mousedown();
      });
    }
  }
}(jQuery))', 'inline');

  $form['form_wrapper'] = array(
    '#prefix' => '
',
    '#suffix' => '
',
  );

  if (isset($form_state['textfield_vale'])) {
    $textfield_vale = $form_state['textfield_vale'];
  }
  else {
    $textfield_vale = 'not set';
  }

  //will not trigger page rebuild
  $form['form_wrapper']['test_textfield'] = array(
    '#type' => 'textfield',
    '#value' => 'Value = '.$textfield_vale,
    '#id' => 'test_textfield',
  );

  //will trigger page rebuild, changing the textfield value
  $form['form_wrapper']['trigger_rebuild'] = array(
    '#type' => 'submit',
    '#value' => 'I don\'t want to press it',
    '#id' => 'trigger_rebuild',
    '#submit' => array('submit_rebuild'), //has #submit value unique to buttons
    '#ajax' => array(
      'callback' => 'test_result_with_button',
      'wrapper' => 'form_wrapper',
    ),
  );

  return $form;
}

function submit_rebuild($form, &$form_state) {
  $form_state['textfield_vale'] = 'button trigger'; //TRUE;
  $form_state['rebuild'] = TRUE;
}

function test_result_with_button($form, $form_state) {
  return $form['form_wrapper'];
}

Form rebuild without using buttons


gdbserver: execute shell commands of the target

Question: E.g. !ls would execute ls command in gdb itself, but how to do it on remote side?
It should be simple, but I can't figure it out. Per documentation something like target remote | ls or target remote | !ls ought to do the trick, but either it's wrong or I don't understand something: such command makes gdb to try to close current session, and start debugging ls binary.
I also found some monitor cmd mentioned, but monitor !ls just triggers Unknown monitor command message.
Comment: I see that the remote debugging protocol includes, as one of the remote file i/o requests, a request to run an arbitrary command in a remote shell (see https://sourceware.org/gdb/onlinedocs/gdb/system.html) but I don't know offhand how to tell gdb to make that request.
Comment: `target remote | command` is used when you want to run a `command` locally where that `command` is (typically) a custom-written proxy for a gdbserver running somewhere else. Do you want to have the remote gdbserver run `ls` as a target process to be debugged, or do you want to run `ls` and view its output without affecting the remote target?
Comment: @MarkPlotnick yes, I want just run `ls` and look at the output, without affecting the process that being debugged.
Comment: Also as I just understood, the output most probably would go to the shell where «gdbserver» is running, not to the gdb that is attached to. To me this is okay anyway.
Answer: A workaround is to implement a custom gdb command that does the following:

fork the remote process;
switch to the child inferior;
replace the child process with a shell that executes the user's provided command;
display the remote command's output;
go back to the parent inferior.

There are several limitations to take into account:

Symbols are not resolved on gdb call command: we need to call libc functions by address, which requires loading addresses beforehand;
Requires libc debug info to be installed, otherwise symbols can't be retrieved;
Huge slowdown can be experienced during automatic loading of child's remote library symbols: instead, we can preload addresses using the parent process, then disable automatic symbol loading;
Shell command output executed from gdb call command is not echoed in the gdb terminal: however, we can capture it in a remote temporary file, then read to memory and print.

Example gdb session:
# Given remote terminal running `gdbserver :2345 ./remote_executable`, we connect to that server.
target extended-remote 192.168.1.4:2345

# Load our custom gdb command `rcmd`.
source ./remote-cmd.py

# Run until a point where libc has been loaded on the remote process, e.g. start of main().
b main
r

# Don't need the main() breakpoint anymore.
del 1

# Run the remote command, e.g. `ls`.
rcmd ls

remote-cmd.py:
#!/usr/bin/env python3

import gdb
import re
import traceback
import uuid

class RemoteCmd(gdb.Command):
    def __init__(self):
        self.addresses = {}

        self.tmp_file = f'/tmp/{uuid.uuid4().hex}'
        gdb.write(f"Using tmp output file: {self.tmp_file}.\n")

        gdb.execute("set detach-on-fork off")
        gdb.execute("set follow-fork-mode parent")

        gdb.execute("set max-value-size unlimited")
        gdb.execute("set pagination off")
        gdb.execute("set print elements 0")
        gdb.execute("set print repeats 0")

        super(RemoteCmd, self).__init__("rcmd", gdb.COMMAND_USER)

    def preload(self):
        for symbol in [
            "close",
            "execl",
            "fork",
            "free",
            "lseek",
            "malloc",
            "open",
            "read",
        ]:
            self.load(symbol)

    def load(self, symbol):
        if symbol not in self.addresses:
            address_string = gdb.execute(f"info address {symbol}", to_string=True)
            match = re.match(
                f'Symbol "{symbol}" is at ([0-9a-fx]+) .*', address_string, re.IGNORECASE
            )
            if match and len(match.groups()) > 0:
                self.addresses[symbol] = match.groups()[0]
            else:
                raise RuntimeError(f'Could not retrieve address for symbol "{symbol}".')

        return self.addresses[symbol]

    def output(self):
        # From `fcntl-linux.h`
        O_RDONLY = 0
        gdb.execute(
            f'set $fd = (int){self.load("open")}("{self.tmp_file}", {O_RDONLY})'
        )

        # From `stdio.h`
        SEEK_SET = 0
        SEEK_END = 2
        gdb.execute(f'set $len = (int){self.load("lseek")}($fd, 0, {SEEK_END})')
        gdb.execute(f'call (int){self.load("lseek")}($fd, 0, {SEEK_SET})')
        if int(gdb.convenience_variable("len")) <= 0:
            gdb.write("No output was captured.")
            return

        gdb.execute(f'set $mem = (void*){self.load("malloc")}($len)')
        gdb.execute(f'call (int){self.load("read")}($fd, $mem, $len)')
        gdb.execute('printf "%s\\n", (char*) $mem')

        gdb.execute(f'call (int){self.load("close")}($fd)')
        gdb.execute(f'call (int){self.load("free")}($mem)')

    def invoke(self, arg, from_tty):
        try:
            self.preload()

            is_auto_solib_add = gdb.parameter("auto-solib-add")
            gdb.execute("set auto-solib-add off")

            parent_inferior = gdb.selected_inferior()
            gdb.execute(f'set $child_pid = (int){self.load("fork")}()')
            child_pid = gdb.convenience_variable("child_pid")
            child_inferior = list(
                filter(lambda x: x.pid == child_pid, gdb.inferiors())
            )[0]
            gdb.execute(f"inferior {child_inferior.num}")

            try:
                gdb.execute(
                    f'call (int){self.load("execl")}("/bin/sh", "sh", "-c", "exec {arg} >{self.tmp_file} 2>&1", (char*)0)'
                )
            except gdb.error as e:
                if (
                    "The program being debugged exited while in a function called from GDB"
                    in str(e)
                ):
                    pass
                else:
                    raise e
            finally:
                gdb.execute(f"inferior {parent_inferior.num}")
                gdb.execute(f"remove-inferiors {child_inferior.num}")

            self.output()
        except Exception as e:
            gdb.write("".join(traceback.TracebackException.from_exception(e).format()))
            raise e
        finally:
            gdb.execute(f'set auto-solib-add {"on" if is_auto_solib_add else "off"}')

RemoteCmd()

Comment: Wow, this is amazing! One nitpick: instead of setting `br main`, and later manually deleting it, you can do it all in one command with `tb main`. That's a temporary breakpoint, which will be deleted on the first hit.
Answer: ! ls > /tmp/ls.txt

All the command that you try to execute using ! cmd will works but you can't get the output because gdbserver is not able to redirect the stream correctly.
Just open a new remote connection to see the output of your gdb remote command redirect to a file like the example before.
Comment: I just tested, and this does not execute the command on the remote side. Steps I did: 1. in /tmp/ dir run `gdbserver :2345 ls` 2. in $HOME dir run `gdb`, and evaluate in it `target extended-remote localhost:2345`. 3. evaluate in gdb `! ls`. It shows content of home but not /tmp. 4. Evaluate in gdb `! ls > 1` This creates a file `1` in $HOME but not in /tmp, and the content of the `1` file is the list of files in $HOME.