Montar query para consulta em banco de acordo com checks marcados

Question: Tenho a seguinte situação, o usuário pode selecionar o nível do idioma, como:
Básico, Intermediário ou Avançado e disponho a ele em forma de checkbox.
Preciso montar uma query dinâmica de acordo com a ou as opções selecionadas pelo usuário, como exemplo, se o mesmo escolher as opções de nível Básico e Intermediário, como poderia montar essa query para efetuar a pesquisa? Podendo ser uma, duas ou todas as opções.
Um exemplo dessa como estava fazendo essa pesquisa, de forma errônea:
if ($NivelIngles != ""){
    if ( $lcWhere <> "" ) {
        $lcWhere.= " AND (`candidato`.NivelIngles = '$NivelIngles ')";
    } else {
        $lcWhere.= " WHERE (`candidato`.NivelIngles = '$NivelIngles ')";
    }
}

O resultado antes de montar minha pesquisa está assim:
Array ( [0] => Basico [1] => Intermediario )

Espero que tenha conseguido explicar minha questão
Answer: Caso a grafia do DB seja a mesma do form:
$checkboxes = Array( 'Básico', 'Avançado' );

$query = 'SELECT * FROM cursos';
$cola = ' WHERE '; // Deixar os espaços em branco "em volta".

foreach ($checkboxes as $nivel) {
   //$nivel = mysqli_real_escape_string( $nivel ) pra prevenir injection. Adapte ao caso.
   $query .= $cola.' `candidato`.`NivelIngles` = "'.$nivel.'" ';
   $cola = ' OR ';
}

echo $query;

Caso use grafia diferente dos ítens:
Segue uma opção sem loop, caso a escrita no DB esteja diferente do form
(Básico x Basico, por exemplo) :
$checkboxes = Array( 'Básico', 'Avançado' );

$query = 'SELECT * FROM cursos';
$cola = ' WHERE '; // Deixar os espaços em branco "em volta".

if ( in_array( 'Básico', $checkboxes ) ) {  // Escrever igual ao Form
   $query .= $cola.' `candidato`.`NivelIngles` = "Basico" '; // Escrever igual ao DB
   $cola = ' OR ';
}
if ( in_array( 'Intermediário', $checkboxes ) ) {  // Escrever igual ao Form
   $query .= $cola.' `candidato`.`NivelIngles` = "Intermediario" '; // Escrever igual ao DB
   $cola = ' OR ';
}
if ( in_array( 'Avançado', $checkboxes ) ) {  // Escrever igual ao Form
   $query .= $cola.' `candidato`.`NivelIngles` = "Avancado" '; // Escrever igual ao DB
   $cola = ' OR ';
}

echo $query;

Resultado:
SELECT * FROM cursos
    WHERE `candidato`.`NivelIngles` = "Basico"  OR  `candidato`.`NivelIngles` = "Avancado"

Quebra de linha adicionada para legibilidade

Lembre-se de digitar a cedilha e os acentos da mesma forma que está no form nos ifs e na query, senão as comparações falharão.

PS: As soluções aqui apresentadas servem para localizar mostrando todos os resultados, caso nenhum checkbox esteja selecionado.
Comment: Olá Bacco, muito obrigado pela excelente ajuda, mas estou com uma dificuldade para adaptar essa query ao modelo que tenho aqui, pois vou concatenando opções com WHERE e AND para depois gerar a consulta, será que pode me dar mais essa dica? O modelo passei na postagem inicial.
Comment: Basta adicionar as condições adicionais assim, após o código mencionado: `$query .= $cola.' curso="alemao" '; $cola = ' AND '` e repetir para todas as cláusulas adicionais. O segredo está na `$cola` que vai ser 'WHERE' ou 'AND' automaticamente.
Comment: Seria interessante um exemplo mais concreto de query, para que eu possa ver o que você precisa, pois quando se usa OR e and, é importante que estejam agrupados corretamente com `( )` pra dar o efeito desejado.
Comment: Como posso te passar? Pelo comentário o espaço é pequeno e a questão do AND e OR é o que mais pesa.
Comment: Aí já não sei se seria um caso de uma nova questão, ou editar e acrescentar embaixo da existente. Mas teria que ter um certo cuidado, pois se alterar demais a essência da pergunta original, prejudica as respostas já existentes... Se achar que é o caso de nova pergunta, seria bom ser bem específico para explicar os requisitos e o problema com detalhes, assim não corre o risco de determinada resposta não te atender 100%.
Answer: Uma opção também seria você fazer WHERE column IN, exemplo:
$checkboxes = array('Basico','Intermediario');

$whereClause = "'".implode("','", $checkboxes)."'";
$query = "SELECT * FROM cursos WHERE `candidato`.`NivelIngles` IN ($whereClause);";

Obs: Sua query está selecionando na tabela cursos e seu WHERE filtrando na tabela candidato, suponho que você esteja fazendo um JOIN das duas tabelas.
Exemplo SQLFiddle
Comment: Olá abfurlan, sim estou fazendo um JOIN.


Linux mail command not working like it used to

Question: I was using the following mail command in my crontab to which worked flawlessly until we re installed our mail server last Friday:
echo "Testing 123" | mail -s "ECS PostDate Processed" "firstname.lastname@example.com firstname.lastname@example.com" -- -r "firstname.lastname@example.com" -F "ServerCronJobs"

previously the "--" would rout the command line parameters to sendmail to send from a name and add a reply-to address.  Now it seems the -- is being ignored and all of the extra parameters are being treated as email addresses. so the email is going to:
from root <email@example.com>
to me, chrish, -r, noreply, -F, ServerCronJobs

What is missing now?
Comment: What's the new and old version of the OS? You probably simply aren't running the same 'mail' command. Check `man mail` for the options.
Answer: You probably got a different version of the "mail" command with the OS upgrade, and it doesn't have the same options.
Why not just use sendmail directly, instead? That's a standardized command that unix MTAs always emulate, and is designed well for scripted usage:
(
  echo "Subject: ECS PostDate Processed"
  echo "To: email@example.com, firstname.lastname@example.com"
  echo "Reply-To: email@example.com"
  echo "From: ServerCronJobs <firstname.lastname@example.com>"
  echo
  echo "Testing 123"
) | /usr/sbin/sendmail -oi -t

That can be done ( echo ; echo ; echo ) | sendmail style instead to fit on one line.
Or, if it is all just echo instead of a command somewhere in that pipe, replace with a 'here document':
/usr/sbin/sendmail -oi -t <<EOT
Subject: ECS PostDate Processed
To: email@example.com, firstname.lastname@example.com
Reply-To: email@example.com
From: ServerCronJobs <email@example.com>

Testing 123

EOT


How to scrape time tag using BeautifulSoup?

Question: I am trying to get the date of the tweet using the following code
div_class="css-901oao r-18jsvk2 r-37j5jr r-a023e6 r-16dba41 r-rjixqe r-bcqeeo r-bnwqim r-qvutc0
page = driver.page_source
        soup = BeautifulSoup(page, "html.parser")
        contents = soup.find_all(class_=div_class)

        for p in contents:
            print(p.time)

but None is printed
Comment: On stack overflow images should not be used for textual content, take a look [Why should I not upload images of code/data/errors?](https://meta.stackoverflow.com/a/285557) for why. Question could be improved by further details (HTML, url, expected result) to clarify and formatting.
Answer: Provided that the element is included in your soup - Classes look high dynamic so better change your selection strategy and use more static id, HTML structure, attriutes.
Following css selector selects all <time> that is an directly child of an :
for t in soup.select('a>time'):
    # tag / element
    print(t)

    # text
    print(t.text)

    # attribute value
    print(t.get('datetime))

Comment: Thank you @HedgeHog
I want to get the text from
div_class="css-901oao r-18jsvk2 r-37j5jr r-a023e6 r-16dba41 r-rjixqe r-bcqeeo r-bnwqim r-qvutc0

and the corresponding time from the 'a' class


HTTP service and local IP addresses

Question: If I am binding a http service to 192.168.1.2, will it make additional requests to my router  in order to determine what my local IP is, or it will compare the IP to the local network configuration and if they match going nowhere?
Also why i can't bind the service to 127.0.0.1:8888 ?
Comment: can you give some more details of what you're trying to bind to port 8888? We need to know the service (apache/tomcat/etc), the configuration your using, and the error you're getting.
Comment: I'm using node.js webserver. I'm trying to bind http server. When i'm using 192.168.1.2 as binding address requests are directed properly to webserver. But when i'm using 127.0.0.1 it's like they are going some wrong way and browser getting "webpage unavailable"
Comment: Guess i'll have to dive a bit deeper into the low level network mechanics. :D
Answer: If you host has already configured 192.168.1.2 as it's IP address, no additional requests need to be made to the router in order to bind and listen on a port with that IP.
However, many HTTP servers will do a reverse DNS lookup of the IP in order to attempt to understand what hostname is associated with this IP.  This is probably something you don't need to worry about.
Answer: I'd say that your first question is operating system dependent, but there's no modern OS that I can think of that would work that way. If you have that IP assigned to a NIC that's local to the machine, your traffic will not leave the NIC.
As for the second question, you need to let us know what HTTP server you're using and what OS you're on.
Comment: node.js on Ubuntu. Actually it's listening, but when i'm trying to connect it's not letting me to http server. I have DMZ turned ON on the router for 192.168.1.2
Answer: If the particular IP is configured locally on a network interface, the server will match the IP you give it to the interface the IP lives on. It will then only listen for traffic coming via that device. (It won't need to go 'check' on the network).
This can be a good security measure, for instance if you have a service which is only needed locally  (e.g. MySQL), don't bind it to your internet-facing interface.
Normally if you give a service an IP that doesn't exist locally it will return an error along the lines of "failed to bind to X". However there are some situations where you need to a service to bind to an IP that may not currently existing on the service. An example is in a highly-available setup, where IPs can move between servers.
In that case the sysctl setting net.ipv4.ip_nonlocal_bind will let you do so without an error.

HTTP service and local IP addresses


CSS/Javascript based drop down menu. Issue with onclick event

Question: I am having some issue getting a menu to .hide() after clicking on an element.

$(document).on("click", "div.dropdown", function() {
  $(this).find("ul").show(0, function() {
    $(this).find("li").click(function() {
      var a = $(this).html();
      $(this).parent().parent().find(".dropdown-title p").html(a);
      $(this).parent().hide(); //Onclick won't hide the UL
    });
    $(this).mouseleave(function() {
      $(this).hide();
    });
  });
  return false;
});
div.dropdown {
  margin: 0 20px 0 0;
  float: left;
  border-radius: 3px;
  background: rgba(0, 0, 0, .1);
  height: 50px;
  line-height: 50px;
  color: #999;
  text-transform: uppercase;
  position: relative;
}

div.dropdown div.dropdown-title {
  padding: 0 20px;
  display: inline-block;
  cursor: pointer;
}

div.dropdown div.dropdown-title p {
  margin: 0 20px 0 0;
  float: left;
  font-size: 12px;
}

div.dropdown div.dropdown-title span {
  float: left;
}

div.dropdown ul {
  position: absolute;
  top: 50px;
  left: 0;
  width: 200px;
  overflow-y: auto;
  overflow-x: hidden;
  border-radius: 0 0 3px 3px;
  z-index: 1001;
  border: 1px solid #ccc;
  list-style-type: none;
  padding: 0;
  margin: 0;
}

div.dropdown ul li {
  line-height: 32px;
  background: #fff;
  color: #999;
  font-size: 12px;
  white-space: nowrap;
  text-transform: uppercase;
  padding: 0 20px;
  cursor: pointer;
}

Sort By

Option 1

Option 2

Option 3

Option 4

Option 5

Option 6

Option 7

Option 8

I am attempting to get the dropdown menu (the ul element) to .hide() when you click on one of the li elements in the menu, as well as when you mouseleave from the ul. Right now, just the mouseleave portion works and not when you click on one of the li elements. I am unsure of what I am overlooking, so if you see any errors or issue please let me know.
Here's the link to my fiddle
Comment: When you click `div.dropdown` you show the `ul` which is within the `div.dropdown`. When you click on a ul item in the menu it run both the code `$(this).parent().first().hide();` AND `$(this).find("ul").show(0, function() {` as clicking on the `ul` will fire the `div.dropdown` click handler. Hope that helps.
Comment: [demo](https://jsfiddle.net/rtf7j2hk/2/) Press `F12` and view console output.
Answer: When clicking on the li the event propagating to the ul, so what really happens that the click on the li closing the list but the 'click' on the ul opens it again, you can stop the event propagation
$(document).on("click", "div.dropdown", function() {
    $(this).find("ul").show(0, function() {
        $(this).find("li").click(function(event) {
            event.stopPropagation();
            var a = $(this).html();
            $(this).parent().parent().find(".dropdown-title p").html(a);
            $(this).parent().hide(); //Onclick won't hide the UL
        });
        $(this).mouseleave(function() {
            $(this).hide();
        });
   });
   return false;
});

Comment: More times than not, I always forgot to check if the event is propagating. Thank you for the help and point this out.
Comment: Quick question. If I add an alert to the click on the li element, and then go back and click on another item, it fires the alert multiple times. Why is that? I thought the stopPropagation would take care of that? https://jsfiddle.net/rtf7j2hk/4/
Comment: It appears that if I unbind the click event on the li it will not only stop the propagation, but also stop the alert from firing multiple times. Any downside to this? https://jsfiddle.net/rtf7j2hk/5/


Change color of Angular Material component mat-list-option checkbox for checked background, frame and check mark

Question: In Angular application, using of Angular Material
I've changed color for <mat-checkbox>My Checkbox</mat-checkbox> checked background and frame like this:
::ng-deep .mat-checkbox .mat-checkbox-frame {
  border-color: rgb(130, 238, 148);
}

::ng-deep .mat-checkbox-checked .mat-checkbox-background {
  background-color: rgb(251, 255, 0) !important;
}

but I can't find the way to do the same with  checkbox checked background, frame and check mark:
<mat-selection-list>
  <mat-list-option
    *ngFor="let dt of data"
    [value]="dt.value"
  >
    {{ dt.myValue }}
  </mat-list-option>
</mat-selection-list>

Using of same ::ng-deep with .mat-list-option .mat-list-option-frame does not gives result.
Answer: You can alter selection-list checkboxes with a different set of rules. Here I have changed the accent color checkboxes:
::ng-deep {

  .mat-accent .mat-pseudo-checkbox-checked,
  .mat-accent .mat-pseudo-checkbox-indeterminate {
    background: green;
  }

  .mat-accent .mat-pseudo-checkbox.mat-pseudo-checkbox-checked,
  .mat-accent .mat-pseudo-checkbox.mat-pseudo-checkbox-indeterminate {
    border-color: blue;
  }

  .mat-accent .mat-pseudo-checkbox::after {
    color: red;
  }
}

https://stackblitz.com/edit/angular-ubma2v?file=src%2Fapp%2Flist-selection-example.scss
Comment: @iose936 I've updated the answer to include frame and check, hopefully that gets you closer to what you need.
Comment: Hello, thank you for your answer. I've already tried this code, but it changes only background. My goal is also change frame and check mark color (I've missed to note last one in my post) please check edit


How to generate uniq namespace from a geven string?

Question: I tried to assign automatically a XMLNS from PHP namespace but the namespace in PHP is so long and I don't want to send to client side a structure of my app... example:
Namespace in PHP = crux\hci\cli\msg_sys\presenters
And in HTML tag:
So how to convert crux_hci_cli__msg_sys_presenters to khgs12so7 for example?
NB: khgs12so7 must be the same even if the PHP class was invoked many times...
Answer: Use some kind of hash function like e.g. hash('algo', 'your namespace')
Use a hash algo which produces some short strings. You can see all available hash algos with hash_algos()
Comment: ok great thank, just one question please, there is many (algo) in hash_algos() list, do you know if ther is one that output s a short string, md5 or sha... have a so long output...?


Negative inverse trig functions

Question: I was practising my integration earlier and for one question I got $-\arccos(x - 2)$, while the book has $\arccos(2 - x)$.
The answers seems so close and I was wondering (a) am I right? But more importantly, (b) is there some relationship between $-\arccos(-x)$ and $\arccos x$. More generally, is there a relationship between inverse trig functions and their negatives and what is it?
Comment: $\arccos(-x) = \pi - \arccos(x)$
Comment: Where is the original question?
Comment: The original question was $\displaystyle \int\frac{1}{\sqrt{(3 - x)(x - 1)}} \; dx$
Answer: $\arccos(-x)=\pi-\arccos x$, so you are correct.
You also have $\arcsin(-x)=-\arcsin x$, and $\arctan(-x)=-\arctan x$.

We can use the identity $\arccos x=\frac{\pi}{2}-\arcsin x$ to derive the first identity from the second, or vice versa,  and to show that the answer to the original problem could also have been written as $\arcsin(x-2)+C$.
Answer: We have the identity $$\arccos(x)+\arccos(-x)=\pi\quad*$$
So $$-\arccos(x-2)=\arccos(2-x)-\pi$$
Since you've found the antiderivative, you have a function plus some constant of integration $C$. In this case, $C=-\pi$. So yes, you're right.
*$\quad x\in[-1,1]$
Answer: To see if $-\arccos{(x-2)}=\arccos{(2-x)}$, check the value at $x=2$.
The first expression gives $-\arccos{(2-2)}=-\arccos{(0)}=-\pi/2$
And the second gives $\arccos{(2-2)}=\arccos{(0)}=\pi/2$
They must be different.
Answer: Inorder to understand the relationship between $\text{trig}^{-1}(x)$ and $\text{trig}^{-1}(-x)$ where $\text{trig}(x)$ is any fundamental trigonometric function, one must first come to terms with the relationship between $\text{trig(x)}$ and $\text{trig}(-x)$ which is as follows:
$$\sin(-x) = -\sin(x)\\
\cos(-x) = \cos(x)\\
\tan(-x) = -\tan(x)$$
The above is a direct result of the trig functions being either even or odd functions.
For an even function (where graph is symmetric about the y-axis), $f(-x) = f(x)$ while for an odd function (where graph is symmetric about the origin): $f(-x) = -f(x)$
If you are to draw the inverse of the graphs of the above function (Just constrict to suitable domain and reflect about $y=x$ line)http://www.purplemath.com/modules/invrsfcn.htm
Then, hopefully the relationships will be obvious from the graphs.

Negative inverse trig functions


Error : statusCode 400 with bad request in Node js

Question: i have worked with node js with joi and hapi i have create a registration with post method.
when i call this by this url "http://localhost:9000/api/users" and parameter header content Content-Type: application/x-www-form-urlencoded. It show Error like:
{"statusCode":400,"error":"Bad Request","message":"name is required"}

Comment: Please share some code.
Answer: Judging by the error message, The Joi schema that you've defined requires a name field to be present.


Machine learning techniques for time series estimation - forecasting price

Question: Can anyone recommend any machine learning techniques for time series estimation?
I have a series of times $t_{1}...t_{n}$, each having a set of associated features $f_{1}...f_{m}$, and a value $x$.
I want to estimate the value of $x$ for the next time in the series given a set of corresponding features $f_{1}...f_{n}$.
Thanks
Edit:
Just to add clarification, this is for price forecasting, where x is a price  and the features will be extracted from news articles.
Answer: An ARMAX model might be a good place to start.
Comment: Quite correct ! Care should also be taken to test the Gaussian Assumptions and to introduce needed remedies, if necessary.
Comment: @S0rin: I would recommend starting with the `arima` function in R.  Use the `xreg` argument for your regressors.  You may also find the `auto.arima` function in the forecast package useful (this function also has the xreg argument).
Comment: @Zach thanks. Can anyone recommend any implementations, in python, R, etc?
Comment: @IrishStat (and Zach), would this model be suitable if the set of features is also a time series?
Comment: @Moominpappa: Yes.  ARMAX models are often used with independent variables that vary over time.
Answer: Recurrent neural networks:

no assumptions on the distributions of $f_i$,
distribution of $x_t$ can be modelled via an adequate loss function (sum of squares for Gaussian, sum of differences for Laplace, cross entropy, kulback leibler divergences, ...)
rather difficult to implement (need advanced techniques such as Hessian free optimization or Long short-term memory to work well).

PyBrain has a LSTM implementation.
Answer: Critically, what's your data frequency? Is this high or low frequency news? Also critically, you want to forecast returns, not prices.
Your question is sufficiently broad that every supervised learning/regression technique can be listed legitimately.
For example, your news could be high frequency news, meaning the response is basically an inhomogeneous time series and a discrete process. Whereas if it is monthly data it is much more Gaussian but is also much more efficiently priced and you have no sample size to test your model's ability to generalise. Data frequency, the market's liquidity, microstructure and other domain specific issues will completely change the statistical model chosen.


DTO List Mapping / Streaming

Question: It's maybe an easy problem but I won't get the best solution. My method like looks this:
public static MlpConfigDto toMlpConfigDto(MlpConfig mlpConfig) {
    return new MlpConfigDto().setActivationFunction(mlpConfig.getActivationFunction().getType().name())
            .setBatchSize(mlpConfig.getBatchSize()).setDescription(mlpConfig.getDescription())
            .setEpochNumber(mlpConfig.getEpochNumber()).setId(mlpConfig.getId())
            .setLastUpdated(mlpConfig.getLastUpdated())
            .setLayers(new List<LayerDto>(
                    mlpConfig.getLayers().stream().map(layer -> new ModelMapper().map(layer, LayerDto.class)))
                            .collect(Collectors.toList()));
}

The last part of  course does not work because there is no new List<LayerDto> in Java. With a Set this works like a charm but I need a List here. How to do it the easiest way for Lists then?
For Sets it would look like this:
.setLayers(new HashSet<LayerDto>(
    mlpConfig.getLayers().stream().map(layer -> new ModelMapper().map(layer, LayerDto.class)))
            .collect(Collectors.toSet()));
}

Comment: Hmm, does this compile? Looks like you're invoking `collect` on `Set`, or am I reading it wrong?
Comment: did you mean `new ArrayList` instead of the interface? and yes you don't need the wrapping when its already collected to the same type.
Answer: The following line returns a list:
mlpConfig.getLayers()
    .stream()
    .map(layer -> new ModelMapper().map(layer, LayerDto.class))
    .collect(Collectors.toList())

There is no need to wrap it into another collection.
Comment: @CptDayDreamer no problem, have a great day!

DTO List Mapping / Streaming


string manipulation in JavaScript: retrieve the matched text and then replace the matched text

Question: JavaScript text manipulation
I need to make little manipulation in the string. I need to retrieve the matched text and then replace the matched text. Something like this
Replace("@anytext@",@anytext@)
My string can have @anytext@ any where in string multiple times.
Answer: This is not jQuery, but regular JavaScript
var stringy = 'bob john';

stringy = stringy.replace(/bob/g, 'mary');

Comment: That handles the replacing it, not the retrieving it.
Comment: @T.J. IIRC the method returns the a string with the replaced substring. So stringy would contain a string in which the substring bob was replaced by the substring mary.
Comment: @ChadNC: Yes, that's right. My interpretation of the question was she(?) wanted to retrieve the substrings that matched, in addition to the string with the replacements having been made. But that could have been just my interpretation.
Comment: @T.J. After re-reading the question it's obvious that I didn't read it carefully enough and that your interpretation is correct.
Comment: Thanks @T.J. and @ChadNC for your answers.I was exactly looking for that @T.J. has suggested.
Answer: You can make the second argument to replace a function:
str = "testing one two three";
str = str.replace(/one/g, function(match) {

    return match.toUpperCase();
});

That replaces the "one" with "ONE". The first argument to the function is the matched result from the regex. The return value of the function is what to replace the match with.
If you have any capturing groups in your regex, they'll be additional arguments to the function:
str = "testing one two three";
str = str.replace(/(on)(e)/g, function(match, group0, group1) {

    return match.toUpperCase();
});

That does exactly what the first one does, but if you wanted to, you could see what was in the capturing groups. In that example, group0 would be "on" and group1 would be "e".


Get post result with cURL and PHP

Question: I have a little function which performs a form submission and it should return the result (html).
The form has only one text input:
<code><input type="text" value="" maxlength="5" name="CODE" size="10">
</code>
This is how I try to fill this variable with cURL:
<code>//open connection
$ch = curl_init();

$myValue = "Test123";
$formUrl = "http://www.mywebsite.com/post.php";

//set the url, number of POST vars, POST data
curl_setopt($ch, CURLOPT_URL, $formUrl);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 120);
curl_setopt($ch, CURLOPT_TIMEOUT, 120);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "CODE=".$myValue);

//execute post
$result = curl_exec($ch);
$info = curl_getinfo($ch);

//close connection
curl_close($ch);

echo $result;
</code>
My problem is that I don't get any results... Is there any problem with this code?
Thank you very much for the help!
EDIT:
I also tried with this kind of setup, with the same result:
<code>$formData = array('CHKCODE' => $tesseractOutput);

    curl_setopt($ch, CURLOPT_URL, $formUrl);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_AUTOREFERER, true);
    curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 120);
    curl_setopt($ch, CURLOPT_TIMEOUT, 120);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($formData));
</code>
Comment: Try to use following syntax `curl_setopt($ch, CURLOPT_POSTFIELDS, ["CODE" => $myValue]);`
Comment: I tried, sadly it's the same :(
Answer: You could try a bit different approach. A simple example:
<code>    $data = array("CODE" => "Test123");
    //$ch = curl_init("http://www.mywebsite.com/post.php");
    //EDIT
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    //curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "PUT");
    //EDIT
    curl_setopt($ch, CURLOPT_URL, "http://www.mywebsite.com/post.php");
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS,http_build_query($data));

    $response = curl_exec($ch);
    var_dump($response);
    curl_close($ch);
</code>
Comment: I get with the "customrequest" option this error: "Method PUT denied by policy"

Get post result with cURL and PHP


Does Google Cloud Firewall log allows and denies?

Question: I cannot seem to find a way to see allowed and denied traffic within my Google Cloud Platform logs.  Is it true the GCP Firewall does not log allowed/denied traffic?
Answer: From the GCP firewall documentation:

GCP firewall cannot log as an action. It can only accept or reject a connection. GCP does not collect statistics per rule at this time.
Answer: This is now possible by enabling firewall rules logging. The logging is enabled in individual basis (for each firewall rule one is interested).
Answer: If none of the rules match it goes to the lowest priority default rule which is denies all requests without logging them.
You can create a deny all rule with logging at a priority slightly higher than the default (and lower than all your other rules).
This will show all the requests that got denied. It may also create a lot of logs if you suffer a DOS attack or high traffic.
Something like this
<code>  deny_all_with_logging_rules = {
    deny-all-with-logging-rule = {
      description          = "Log all denied requests that bounce of the firewall"
      direction            = "INGRESS"
      action    = deny
      priority  = 9999
      source_ranges        = []
      rules                = []
      enable_logging = {
        include_metadata = true
      }
    },
</code>
See also this article on getting alerts
https://medium.com/google-cloud/notification-of-firewall-denies-c3476a0ea79b


Intellij plugin development "the selected directory is not a valid home for intellij platform plugin sdk"

Question: I'm following these guidelines on how to develop Intellij plugin.
So I

Open project settings.
Project SDK --> New
Choose the community edition path I just downloaded

Expected the directory is selected fine.
Actual: I get the error message:
"the selected directory is not a valid home for intellij platform plugin sdk"
See screenshot:

Am I doing something wrong?
Thanks
Answer: Intellij platform plugin sdk home should be your installed or builded IntelliJ, not sources. You can add that intellij-community folder as sources to your IntelliJ SDK later or not at all.
Answer: Goto Intellij installed directory

C:\ProgramFiles\JetBrains\IntelliJ IDEA Community Edition 2016.3.4
Select plugin folder
Create a directory (eg: scala)
Copy the scala lib folder
Restart the intellij
Answer: In my case, I had to download a newer IDEA version (Community Edition) and after the installation, everything worked fine.
This is the official and updated link for setting up your plugin development environment.
Answer: In my case, after trying many ways and nothing helped, uninstall IntelliJ IDE and reinstall it. This error has been resolved.
I think the error exists because I installed IntelliJ IDE before installing the SDK.
Note: I use IntelliJ IDE 2023.2.5 Community and downloaded JDK 21 for window.


Notation question: Adding half a beat to a bar?

Question: I've run into a quirk in a piece I'm writing, something I've never had to deal with—any ideas?:
The piece is in 4/4. One bar has an extra half-beat (with 18 sixteenth notes rather than 16).
4.5/4 = 9/8, so it seems like I can switch to 9/8 time there, right?
But what about tempo? I've indicated a metric tempo at the start of the piece—but in 4/4, each quarter note gets one beat, and in 9/8, each eighth note gets a beat. So at the same tempo, you're suddenly going twice as fast, right?
Do I solve this by putting "metric modulation" symbols before and after the 9/8 bar? E.g., "[quarter note] = [half note]", then "[half note] = [quarter note]"? Or do I have that turned around? (Or considering this bar's contents, would "[sixteenth note] = [eighth note]", etc., be clearer?)
Or should I simply enter a new metronome value—from "[quarter note] = 80" to "[quarter note] = 40", then back?
Hope this makes sense. Thanks!
Comment: In 9/8, there are **three dotted-quarter beats** in a bar. It's not clear in your question what you actually want to achieve with the 9/8 bar (is each eighth note the same throughout the piece, or do you want the 9/8 **bar** to take the same time as a 4/4 bar?)
Comment: I'm a little confused about how long you want the 9/8 bar to be. First, you write that "one bar has an extra half-beat," which makes it sound like you want the 9/8 bar to be only slightly longer than each 4/4 bar, with an eighth note in the 9/8 bar equal to an eighth note in each 4/4 bar. But then you write that you're considering various metric modulation and metronome marks, which makes it sound like you want the 9/8 bar to be over twice as long as each 4/4 bar, with an eighth note in the 9/8 bar equal to a *quarter* note in each 4/4 bar. Which one are you looking for?
Comment: "So at the same tempo, you're suddenly going twice as fast, right?" – If the length of a beat stayed the same when moving from 4/4 to 9/8, then it would sound like the musician is suddenly playing **half** as fast, not twice as fast. (But my understanding from reading the answers below is that that's moot, because the length of a _quarter note_, not the length of a beat, stays the same when moving from 4/4 to 9/8.)
Comment: If there are indeed 18 notes, could you play them as tuplets, retaining 4/4?
Comment: "So at the same tempo, you're suddenly going twice as fast, right?" There is a notational convention for specifying that the eighth note of the new meter has the same duration as the eighth note of the old meter, so no, not necessarily.
Comment: **Tubular Bells** is calling.    (yeah, you pedants I know they didn't split this fine)
Comment: I'd suggest perhaps   4/4 measures with a  1/8 measure tossed in here and there as desired.  You should add a  " 1/8 = 1/8" note  to avoid any confusion.
Comment: You can write: listesso tempo.
Comment: Do you want the eighth note to stay the same?  If yes, do like Stravinsky and go from 4/4 or 8/8 to 9/8 for one measure.
Comment: FWIW, here is what Grieg does in Solveig's song: You have a repeating pattern of three notes at the end of a section. Without further ado, the last note would not fit into the bar. Grieg, however, instructs *poco ritardando* and simultaneously makes the last three notes into a triplet, i.e., he tells you to slow down and speed up at the same time. I assume, he wanted the music to continue in a regular pattern, but squeeze in an extra note into a bar. And then for the next bar, you get a completely different beat and tempo instruction.
Answer: Here's some possibilities.
If the note grouping is as clear as in my example, maybe the metric modulation is unnecessary in A.
D is acceptable in today's music.  But you will find it a challenge to produce in many notation programs!
Is there actually an extra note?   Maybe E or F are what you really need.
Comment: @Darrel Hoffman  Of course, you can only vote for E or F if that's what the music DOES...    Maybe there really are 4½ in-tempo beats in that bar!
Comment: I could see someone interpreting C as triplets with the entire measure lasting a single beat, the same length as each beat in previous measures. Would that interpretation be incorrect?
Comment: I like [A] the best.
Comment: For me [C] appears to be the best option.  I'm reminded for some reason of one of the movements of Carmina Burana.
Comment: I'd vote for [E] or [F], personally, though it might depend on the situation.  Avoiding time signature changes when you just want to extend a measure a bit longer seems like the simplest solution.  If you want to be really precise, you could just include a textual note there indicating exactly how long to hold the fermata.
Comment: Agreed with @LaurencePayne. Personally if there really was an extra half beat I would go with B, C or D, whichever better captures the feel you are after. I don't think A (9/8) is correct because it implies a triple meter for the entire measure
Comment: I'd stongly lean toward B or C, depending upon whether the extra beat should be accented (use B) or unaccented (use C).  I think C would be most likely to result in a conductor's beat pattern that matching performer expectations.
Comment: @Nigel As it would have been easy to stay in 4/4 and write a triplet on the 4th beat, I don't think there's much danger of that misinterpretation.  But if you feel there was, you could clarify with an '8th=8th' metric modulation.
Comment: I think A works best but it's the kind of thing editors and writers aren't always consistent about and I've seen it lead to vastly different interpretations depending on who's reading it. The quarter = quarter makes it unambiguous, but I've seen bandleaders interpret this, more than once, as a typo. "I'm sure they meant for that to be dotted!". To be even clearer, (A) could be rewritten with the 4/4 parts in 8/8 (maybe that's too extreme) and the 9/8 bar could have the second quarter note written instead as two tied eighth notes, to indicate that it spans two beats.
Comment: @DarrenRinger: I would expect a conductor who encountered a 9/8 bar in a 4/4 peace to conduct it using three baton strokes that took 1.5 times as long as the baton strokes in 4/4 measures, when the music would probably fit much better with four or five baton strokes.
Answer: In addition to other answers.

But what about tempo? I've indicated a metric tempo at the start of the piece—but in 4/4, each quarter note gets one beat, and in 9/8, each eighth note gets a beat. So at the same tempo, you're suddenly going twice as fast, right?

I guess you are used to define tempo in "beats per minute", but in sheet music it is typically specified as number of notes of given value – e.g. quarter notes – per minute. Tempo specified this way does not change.
Also, if 9/8 is in fact a compound meter of 4/4 + 1/8, possibly the beat base doesn't even change, except for the time when it needs to shift by an eight note.
Answer: Well, that depends. Time Signatures imply an inherent beat structure. 4/4 will have 4 beats of 1/4th notes. 9/8 as three beats each made up of 3 1/8th notes or one dotted 1/4th. Depending on what your music actually does this might be exactly what you want, or it might be confusing. Other options might include 4/4+1/8 or even simply 18/16.
About the tempo: Tempo is not given in beats per second, but in a specific duration value per second. Common modern consensus is that this tempo remains the same even if time signature changes happen. This would have been different in baroque time, which is why there is some old practise of putting the tempo indication L'istesso tempo (the same tempo) to indicate that the tempo does not change.
From what you're saying you seem to imply you do not actually want this, but you want to have this 9/8 measure to be played in half the tempo of the 4/4 measures. This could simply be done by putting eighth = quarter as tempo indication.
But why not simply use one 9/4 with double note values instead? This way you would not induce any confusion by suddenly taking half the tempo for a single measure.
Comment: @MarkBeadles Fugue, you're right!
Comment: "Key Signatures imply an inherent beat structure" - I think you mean time signatures?
Answer: You're overthinking this.
"and in 9/8, each eighth note gets a beat"

I could say "yes, and those beats are twice as fast" and be done with it. You get precisely the effect you want.
But really, music theory does not have beats that are independent of note values. That only happens in DAWs / sequencers. Normally the beginning of a piece indicates how many quarter or eighth notes there are per minute, and that keeps holding, even with time changes. So: do not indicate anything. The player will figure "eighth note here is eighth note there" and play it correctly.


Ranquear database [#1 de 10]

Question: Preciso ranquear os registros do DB durante a paginação. A principio não pretendo criar um campo para o ranking. Supondo ter 10 registros
1) Ordenando a paginação por AZ, quero listar com o número correspondente ao rank - #N de 10
2) Quando exibir o conteúdo, preciso o número correspondente ao rank - #N de 10
• PAGINAÇÃO
#01 de 10, Lorem ipsum
#02 de 10, Lorem ipsum
...
#10 de 10, Lorem ipsum
SELECT *,
FIND_IN_SET( score,
    ( SELECT GROUP_CONCAT( score ORDER BY score DESC ) FROM table )
) AS rank
FROM table limit X,X

• CONTEÚDO
Lorem ipsum, Ranking #03 de 10
SELECT *,
FIND_IN_SET( score,
    ( SELECT GROUP_CONCAT( score ORDER BY score DESC ) FROM table )
) AS rank
FROM table WHERE ...

Peguei essa SQL de um exemplo e está funcionando, mas eu não domino MYSQL e não tenho conhecimento para este caso...
Gostaria de saber se há outra forma para montar esta QUERY... Aceito sugestões.
Comment: Para facilitar o entendimento seria preciso que você postasse a estrutura da tabela do banco de dados usado!
Comment: @Willams de Jesus, não precisa da estrutura do DB pois pretendo ordenar pela ordem crescente e não há uma chave que valha como pontuação. O exemplo que postei eu encontrei no `stackexchange` - funciona mas é baseado em pontos(`TBL-SCORE`)
Comment: O que você chama de rank? O número da linha no resultado?
Comment: Não estou entendendo bem. Então você não quer um campo como o *score* dos seus exemplos? E a tal numeração seria baseada em que? Na ordem alfabética dos resultados de cada página?
Comment: @bfavaretto sim, supondo que o ID 05 na paginação seja listado como #2 no rank AZ, gostaria também que quando acessar o conteúdo ele também faça o calculo para exibir `Posição 2 de 10`.
Answer: Não entendi se é isso que você quer, mas se é apenas a numeração das linhas da saída, pode fazer algo desta forma:
SELECT
    @linha := @linha + 1 AS contador,
    tabela_desejada.*
FROM
    (SELECT @linha := 35) AS nada,
    --                 ^ Usar o mesmo valor inicial do limit
    tabela_desejada
LIMIT 35,10;
--     ^ Usar o mesmo valor inicial da subquery

Notar que a subquery com o valor de exemplo 35 tem que usar o mesmo valor inicial do limit. Provavelmente, como a query será gerada dinamicamente, basta usar o mesmo parâmetro nos dois lugares.
Lembre-se também de que o limit do MySQL se inicia do zero.
  Para iniciar de 1 na saída, basta fazer ( @linha := @linha + 1 ) + 1 ...

Versão com totalizador:
SELECT
    CONCAT( "#", @linha := @linha + 1, " de ", total.cnt ) AS ranking,
    tabela_desejada.*
FROM
    (SELECT @linha := 35) AS nada,
    (SELECT COUNT(*) AS cnt FROM tabela_desejada) AS total,
    tabela_desejada
ORDER BY nome
LIMIT 35,10;

Neste caso, se você for usar WHERE, lembre de replicar a condição na query principal e na subquery:
...
    (SELECT COUNT(*) AS cnt FROM tabela_desejada WHERE id > 100) AS total,
    --                                                     ^ atenção
    tabela_desejada
WHERE id > 100
--       ^ atenção
ORDER BY nome
LIMIT 35,10;

Comment: @PapaCharlie caso você queira realmente complicar :), e fazer com que o ranking não se altere com o WHERE, o caminho é passar o contador para dentro da subquery. Assim, a contagem é feita na base toda, antes do resultado ser filtrado (com uma penalização no processamento, obviamente).
Comment: usei como critério a ordem AZ com o intuito de simplificar propositalmente, pois eu preciso compreender como funciona a QUERY, e posteriormente aplicar às necessidades usando algumas médias. Vou botar o seu exemplo em prática e depois dou um feedback. +1 pelas explicações dadas.
Answer: Tomando como base o seu exemplo, basta modificar os campos da subselect para ficar de acordo com o seus critérios: (obter o rank ordenado pelo nome de AaZ)
SELECT *,
FIND_IN_SET( id,
    ( SELECT GROUP_CONCAT( id ORDER BY nome ASC ) FROM `table` )
) AS rank
FROM `table` order by nome;

E para pegar um registro:
SELECT *,
FIND_IN_SET( id,
    ( SELECT GROUP_CONCAT( id ORDER BY nome ASC ) FROM `table` )
) AS rank,
( SELECT COUNT( id ) FROM `table` ) AS total
FROM `table` where id=5;

Ou seja, o FIND_IN_SET procura o id 5 na lista de ids (1,3,5,8,4,10,9,6,7,2) que foi ordenada pelo nome, retornando a posição 3...
Veja o funcionamento no sqlfiddle: http://sqlfiddle.com/#!2/9a84a9/4

Ranquear database [#1 de 10]


Bitmap recycle error with no bitmap recycle call

Question: I have an adapter that loads images into views. It runs fine except that when I test the ListView by scrolling really fast up and down the listview, I get a "Canvas trying to use a recycled bitmap". However I never call bitmap.recycle because I am using an imageloader. The error does not give me a location of where the error is happening. My guess is that the adapter automatically recycles bitmaps on its own since I do not make a manual call. This problem is not very consistent but I can recreate it if I really try. What is causing this problem or how can I locate it?
Here is the stacktrace.
<code>11-26 11:01:41.131    1124-1124/com.android.wallfly E/AndroidRuntime﹕ FATAL EXCEPTION: main
Process: com.android.wallfly, PID: 1124
java.lang.RuntimeException: Canvas: trying to use a recycled bitmap android.graphics.Bitmap@432bc210
        at android.graphics.Canvas.throwIfCannotDraw(Canvas.java:1084)
        at android.view.GLES20Canvas.drawBitmap(GLES20Canvas.java:844)
        at android.graphics.drawable.BitmapDrawable.draw(BitmapDrawable.java:496)
        at android.widget.ImageView.onDraw(ImageView.java:1058)
        at android.view.View.draw(View.java:15304)
        at android.view.View.getDisplayList(View.java:14198)
        at android.view.View.getDisplayList(View.java:14240)
        at android.view.View.draw(View.java:15018)
        at android.view.ViewGroup.drawChild(ViewGroup.java:3298)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3135)
        at android.view.View.getDisplayList(View.java:14193)
        at android.view.View.getDisplayList(View.java:14240)
        at android.view.View.draw(View.java:15018)
        at android.view.ViewGroup.drawChild(ViewGroup.java:3298)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3135)
        at android.view.View.draw(View.java:15307)
        at android.view.View.getDisplayList(View.java:14198)
        at android.view.View.getDisplayList(View.java:14240)
        at android.view.View.draw(View.java:15018)
        at android.view.ViewGroup.drawChild(ViewGroup.java:3298)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3135)
        at android.view.View.draw(View.java:15307)
        at android.widget.FrameLayout.draw(FrameLayout.java:472)
        at android.view.View.getDisplayList(View.java:14198)
        at android.view.View.getDisplayList(View.java:14240)
        at android.view.View.draw(View.java:15018)
        at android.view.ViewGroup.drawChild(ViewGroup.java:3298)
        at android.widget.ListView.drawChild(ListView.java:3393)
        at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3135)
        at android.widget.AbsListView.dispatchDraw(AbsListView.java:2918)
        at android.widget.ListView.dispatchDraw(ListView.java:3388)
        at android.view.View.draw(View.java:15307)
        at android.widget.AbsListView.draw(AbsListView.java:5184)
        at android.view.View.getDisplayList(View.java:14198)
        at android.view.View.getDisplayList(View.java:14240)
        at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:3272)
        at android.view.View.getDisplayList(View.java:14135)
        at android.view.View.getDisplayList(View.java:14240)
</code>
Here is the ImageLoader info
<code>        ImageLoaderConfiguration config = new ImageLoaderConfiguration.Builder(context)
            .threadPriority(Thread.NORM_PRIORITY - 2)
            .denyCacheImageMultipleSizesInMemory()
            .memoryCache(new WeakMemoryCache())
            .diskCacheFileNameGenerator(new Md5FileNameGenerator())
            .tasksProcessingOrder(QueueProcessingType.LIFO)
            .writeDebugLogs() // Remove for release app
            .build();

            options = new DisplayImageOptions.Builder()
                .cacheInMemory(true)
                .cacheOnDisc(true)
                .bitmapConfig(Bitmap.Config.RGB_565)
                .resetViewBeforeLoading(true)
                .imageScaleType(ImageScaleType.IN_SAMPLE_INT)
                .build();
</code>
Comment: I guess the "imageloader" in question is https://github.com/nostra13/Android-Universal-Image-Loader ? If not, please reference it.
Comment: @shkschneider yes this is the imageloader.
Answer: Your ImageLoader is doing some recycling obviously.
See Android trying to use a recycled bitmap, not in my code and Canvas: trying to use a recycled bitmap android.graphics.Bitmap in android for a better understanding of the problem.
How to fix it depends on your implementation (code) and your "imageloader".
Posting some code or the StackTrace might help.
Comment: I am guessing it has something to do with my imageloader as you say. I have posted the config and options. I have suspicion of "reset view before loading". However I am not sure.
Comment: `resetViewBeforeLoading()` seems to cause some errors. Try to play around with options (including removing `resetViewBeforeLoading()`). See https://github.com/nostra13/Android-Universal-Image-Loader/issues/362 sorry can't help more
Answer: Please remove

bitmap.recycle();

from your code. And it will solve your error.
Comment: Welcome to Stack Overflow. Explaining why removing `bitmap.recycle();` solves the problem could be helpfull. Thanks.


Installing PCNTL module for PHP without recompiling

Question: Is it possible to install the pcntl module without recompiling php? I'm trying to set this up on CentOS.
Answer: I haven't been able to get it working purely through a package manager, and yum search pcntl finds no matches, even after setting up some additional repositories.
I ended up finding one way that worked without reinstalling PHP entirely, though it does end up requiring a tiny bit of recompilation.  A post found here mentions a somewhat straightforward way of compiling and enabling just the pcntl extension ( with a little editing and reformatting of the post):
Get your current php version (check phpinfo())
Get the sources for that version here: http://www.php.net/releases/
Extract this file; you will got folder name like php5-x.x.x
Run following lines in terminal:

cd php5-x.x.x/ext/pcntl
phpize
./configure
make
cp modules/pcntl.so /usr/lib/php/modules/
echo "extension=pcntl.so" > /etc/php.d/pcntl.ini
restart apache

Note: if your /usr/lib/php/modules/ or /etc/php.d/ is not same
as this, you could try 'locate php/modules' or 'locate php.d'

Comment: This was *extremely* help to me and worked!  Thank you very much.
Comment: After adding the ondrej ppa and installing `php8.0-dev` error is gone
Comment: When running configure `configure: error: Cannot find php-config. Please use --with-php-config=PATH`
Answer: The pcntl module is in the php-cli package and can be installed with yum by using the following command:
sudo yum install php php-cli

Answer: try to find module with yum command
yum search pcntl
Answer: Check if this extension enabled.
php -m

or
php-fpm -m

If not exist, so you should it enable during build or build it as extension.
Answer: Download your existing PHP version repository from http://www.php.net/releases/ (This is no matter where you have to download & extract. You can either download to local machine. Just want to copy ext/pcntl to somewhere in the server)
Extract downloaded folder (Example: /home/downloads/PHP_5.x.x/)
cd /home/downloads/PHP_5.x.x/ext/pcntl
phpize
./configure
make
cp modules/pcntl.so /usr/local/lib/php/extensions/

[*This path is depend on your PHP installation/configurations. You can check the PHP extension path in php.ini or echo phpinfo(); ]
echo "extension=pcntl.so" > /usr/local/lib/php.ini

or
Add the line "extension=pcntl.so" to your php.ini
vim /usr/lib/php.ini

[*again depend on your PHP installation. You can find the php.ini path in locate php.ini or  echo phpinfo(); ]


mod_rewrite remove sub directory from certain URLs

Question: I have a situation where I'm trying to remove a subdirectory from certain URLS.
Here is what I'm trying to accomplish
http://www.example.com/shop/some_product should go to http://www.example.com/some_product
But
I need http://www.example.com/shop/customer/account to stay the same.
So essentially I need to rewrite anything shop/... except shop/customer/... shop/admin/...
Here is what I have tried
RewriteCond %{REQUEST_URI} !^/shop/customer
RewriteCond %{REQUEST_URI} !^/shop/admin
RewriteCond %{REQUEST_URI} !^/shop/sales
RewriteRule ^/shop/(.*) http://www.example.com/$1 [L]

Anyone have any advice here?
Comment: Looks fine, except that you are missing the initial `RewriteEngine on`.
Comment: And whether that leading slash `^/` in the `RewriteRule` is correct depends on _where_ you put that rule.
Answer: Thanks arkascha.
Turns out I had a couple problems. Here is what I ended up with:
in /shop/.htaccess
RewriteCond %{REQUEST_URI} !^/shop/(media|skin|js|customer|admin|sales|checkout)/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*) /$1 [L]

I needed to not rewrite URLs matching a file or directory. Also, it seems that having /shop/ was a requirement for the RewriteCond, but would not work in the RewriteRule.


Clustering using histograms

Question: I need to find clusters in a very large amount of data (>2M data points), and I was looking for ways to speed up the usual algorithms, i.e. k-Means, DBSCAN, ...
Is there any major issue, especially regarding statistical significance, to run the algorithms on a histogram, where sparse bins have been filtered out, instead of the real data?
What I mean is to create a histogram and throw away all bins with a count below a specific threshold, and run a clustering algorithm on the resulting data points.
Obviously, I will get some uncertainty and data loss, as well as the problem of choosing further parameters, such as the bin size. But the data set I have has a few very dense regions that I am interested in and the rest is very sparse.
Answer: I have been wondering the same thing.  My conclusion is that doing what you propose is valid for say establishing cluster centres but the challenge is that those change as you vary bin width.  An interesting exercise is to take your data and produce a series of histograms changing (say linearly) the bin width for each histogram.  As width diminishes the number of data points per bin decreases.  That means you get increasingly 'spiky' histograms possibly reaching a point where each bin has one data point.  The question you are faced with is 'what bin size produces the most correct spikes for the data'.  Downsampling is a more robust way to go.
Answer: For either method there have been many papers written on exact and approximate techniques to speed them up. Now 2M is not that much - a good index accelerated implementation should handle this in a few minutes.
For k-means there are methods that can even guarantee you approximate the result to a certain quality. For DBSCAN there are grid-based approaches that try to detect cells that cannot contain core points or where all points must be core points to speed up the process. That usually only works for Minkowski Norma and in low dimensional space, though.
But what about just downsampling your data? Few people know what to do afterwards anyway with the full data... So just pick 20k or 200k points and cluster these. Try using this result first before even bothering to cluster all your data.

Clustering using histograms


How to sample value from Symmetric Geometric Distribution

Question: How can I sample a value from a symmetric geometric distribution, as defined in this link.
There, the density of the symmetric geometric proposal distribution is given by
\begin{equation*}
f(\theta;p_g)\propto \frac{p_g (1 - p_g)^{|\theta|}}{2(1- p_g)},
\end{equation*}
where the symmetry centers at $\theta$. However, it seems that this distribution is not widely known under this name. It seems that this distribution is somehow related to the Laplace distribution, for which I know how to sample values, but I couldn't establish this relation myself.
As such, my question is actually twofold:

Is this distribution known under a more common name, and
Is there any other distribution I can relate it to, in order to generate values from this distribution more easily?
Answer: The linked document is unclear; it doesn't even indicate the random variable (which I
will call $X$). I assume it means $p_g$ & $\theta$ are parameters.  It doesn't define the values taken by $X$ - which I believe should appear in the exponent as $|x-\theta|$ not as $|\theta|$ - nor the values taken by $\theta$ (which I presume to be integer) and it refers to the distribution as discrete but calls it a density.
I believe a correct implementation of a "symmetric geometric" is as follows:
$$p_X(x;\theta,p_g)= \frac{p_g (1 - p_g)^{\mid x-\theta\mid}}{2- p_g},\: x\in \mathbb{Z};\, \theta\in \mathbb{Z}, 0<p_g<1$$
(note the change in the denominator also)
This distribution has the property that as you move out from the center, the ratio of probability to the next probability further into the tail remains constant (in geometric ratio). This is what makes it "geometric".
Here's an example, with $p_g=0.3$ and $\theta=10$:

As you see it's symmetric, geometric and centered at $\theta$.
If the distribution has (sub)exponential tails, that should work fine as a proposal.
What they attempted to define (but failed to) is slightly different:
$$p_X(x;\theta,p_g)= \frac{p_g (1 - p_g)^{|x-\theta|}}{2(1- p_g)},\: x\in \mathbb{Z};\, \theta\in \mathbb{Z}, 0<p_g<1$$
This is not in geometric ratio, because the central spike is twice the height it should be to be in the same ratio to the values either side as those values are to the ones further out again.
While I don't think it merits the names symmetric geometric, this distribution can be generated by generating a geometric (the version indexed from 0), attaching a random sign and shifting by $\theta$ (i.e. add $\theta$).
(The one I defined in the beginning is slightly more complicated to generate, but one way to do it is as above but if you generate a $-0$ you throw it out and generate again.)

A different one suitable for heavier tails would be a discrete equivalent of a table-mountain distribution, such as
$$p_X(x;\theta,p_g)= \frac{\min(1,\mid\! x-\theta\!\mid^{-s})}{2\zeta(s)+1},\: x\in \mathbb{Z};\, \theta\in \mathbb{Z},s>1$$
(specifically with $s=2$ for the usual table mountain proposal).
This is effectively a symmetric version of a zeta(s) distribution with a "flat" bit at $\theta$ (in that it has the same probability as the values either side of it). You could replace the value in the numerator at $\theta$  (i.e. $1$) with an arbitrary positive value (if you also fix the denominator), or expand the "flat" part (ditto) or use a different $s$ if needed.
Answer: I mostly agree with @Glen_b but I think the correct probability function is
$$f(\theta) = \begin{cases} p_g &\theta = 0\\ \frac{1}{2}p_g(1-p_g)^{|\theta|-1} &\theta \neq 0\end{cases}$$
for integer $\theta$. This seems to be the only way to get the correct variance, and it is the same as the given formula up to a scalar. As Glen_b said, a draw from this distirbution can be obtained by drawing from a geometric distirbution and multiplying the result by a uniform random choice of $+1$ or $-1$. You can work this out by hand and/or verify it by simulation.
I also agree that the intention is probably that the proposed value $\theta_{new}$ is meant to be $\theta + x$ where $x$ is a random draw from $f$. This is the only way I can see for the text in your link to make sense.
Unfortunately, we cannot know for sure because we don't have access to the source!
Comment: Yes, it is. I don't understand what they mean by that either, but I assume they mean that the symmetry is centered at the current value of $\theta$. I don't think their formula for the variance can be wrong because it *is* the correct formula for the variance you get if you generate a geometric random variate and multiply it by +1 or -1.
Comment: @Cesar if the random variable is generated in this way (which is the only way I can see to get the variance to agree with the formula given by SAS) then 0 gets a double probability because when you multiply it by -1, you still get 0.
Comment: Hi Flounderer; in your expression for $f$, $\theta$ is the random variate, I presume? [I ask because then how does the part where it says "where the symmetry centers at $\theta$" make sense? By contrast I'd assume the variance doesn't match because they computed it after they screwed up the distribution.]
Comment: Yeah, I figured  out what the discrepancy is. That way of generating doesn't actually correspond to a geometrically decreasing function -- the prob at the center is twice as high as it should be.
Comment: Many thanks! May I ask why a special case is needed for \theta = 0?
Answer: Discrete Laplace distribution is a very similar to the one you describe (check Inusah and Kozubowski, 2006, and Kotz, Kozubowski and Podgorski, 2012).
Discrete Laplace distribution has probability mass function:
$$ f(x) = \frac{1-p}{1+p} p^{|x-\mu|} $$
and cumulative distribution function
$$
F(x) = \left\{\begin{array}{ll}
\frac{p^{-|x-\mu|}}{1+p} & x < 0 \\
1 - \frac{p^{|x-\mu|+1}}{1+p} & x \ge 0
\end{array}\right.
$$
The name discrete Laplace comes from the fact that if $U \sim \mathrm{Geometric}(1-p)$ and $V \sim \mathrm{Geometric}(1-p)$, then $U-V \sim \mathrm{DiscreteLaplace}(p)$, where geometric distribution is related to discrete Laplace distribution in similar way as exponential distribution is related to Laplace distribution.
Sampling from it is straightforward: you draw $U$ and $V$ from geometric distribution parametrized by $q = 1-p$, and then take $U-V$.
In R it is implemented in DiscreteLaplace, disclap and extraDistr packages.
Below you can see your distribution (in black) and discrete Laplace distribution (in red) parametrized by different values of $p$ (for discrete Laplace by $1-p$). As you can see, they differ but the idea behind them is very similar.

Kotz, S., Kozubowski, T., & Podgorski, K. (2012). The Laplace distribution and generalizations: a revisit with applications to communications, economics, engineering, and finance. Springer Science & Business Media.
Inusah, S., & Kozubowski, T.J. (2006). A discrete analogue of the Laplace distribution. Journal of statistical planning and inference, 136(3), 1090-1102.


Problem with Facebook Feed: Warning: file_get_contents(): https:// wrapper is disabled

Question: I'm having a problem with my Facebook pull feed. It works fine on a development server, but on my live server the block doesn't appear on the page, and I get two php errors, and a single Facebook pull error in my recent logs.
The first php error, which I think is significant is:
Warning: file_get_contents(): https:// wrapper is disabled in the server configuration by allow_url_fopen=0 in _facebook_pull_feed() (line 280 of /home/phislub9/public_html/sites/all/modules/facebook_pull/facebook_pull.module).

The second php error is related to the first one:
Warning: file_get_contents(https://graph.facebook.com/oauth/access_token?client_id=1392108967676515&client_secret=563ff2dd27ddec75d9b79dc918fe93d4&grant_type=client_credentials): failed to open stream: no suitable wrapper could be found in _facebook_pull_accesstoken() (line 299 of /home/phislub9/public_html/sites/all/modules/facebook_pull/facebook_pull.module)

And the Facebook Pull error doesn't give any diagnostic information beyond that the module isn't working.
Answer: Put this in your settings.php:
ini_set('allow_url_fopen' , 1);

If that solves the issue you will want to hard code that into your php.ini.
Comment: One thing I should have asked to clarify: Are you talking about a phpinfo.php you created or the one you access through Drupal (mysite.com/admin/reports/status/php)?  That makes a difference if you are trying to go through the ini_set route. I'm not seeing a ton on Google about alternate ways to update php settings with ANHosting (usually googling how to increase memory limits is more productive since that is a more common issue and is changed the same way). You may want to get on the phone with the host and ask if the change is possible or categorically blocked by their system.
Comment: so, that doesn't seem to have fixed it, although I'm not getting a log error yet, so I'm not sure if the facebook pull has run yet. I just checked the phpinfo.php, and the local and master values for 'allow_url_fopen' are both set to off, so I don't think the override in the settings file is working. The site is on a shared server, so I don't have access to the php.ini file.
Comment: Which shared host? Is it one of the major ones?
Comment: It's a host called anhosting.com
Comment: Thanks, I just did this. it's as simple as dropping a php.ini in the public html folder. I'll try that and report back
Comment: Thanks, that took triggering one scare WSOD but I fixed it.
Comment: Glad to hear it!


how to send http response using stream

Question: I would like to have simple API in my http server so everytime I write to HttpResponse I use stream.
so I convert all object into stream, ie object->json->stream
Stream<List<int>> toStream(Object value) {
    var json = JSON.encode(value);
    var controller = new StreamController<List<int>>(onListen: () => UTF8.encode(json));
    return controller.stream;
}

and later
(response as HttpResponse).addStream(toStream({"a": 1, "B": 2})
.then(() => response.flush())
.catchError((e, stack) {
    _logger.error("Handling ${context.path} finished with an error: $e");
    _logger.debug(stack.toString());
})
.whenComplete(() => response.close());

but I get error
Uncaught Error: Bad state: StreamSink is bound to a stream
Stack Trace:
#0      _StreamSinkImpl.close (io_sink.dart:122)
#1      _HttpOutboundMessage.close (http_impl.dart:481)

Im not sure what I am doing wrong here. I saw examples where File's input stream was piped to response but I cannot make it work either.
any help appreciated!
Answer: There are a couple of things to note here:
First, the JSON and UTF8 encoders will produce a single value when you're calling encode(), so you'll be creating a Stream of just a single value. I wonder how much value you get from that.
Second, your Stream doesn't actually get any values. onListen is called when the first listener subscribes, but any value returned from onListen isn't used for anything. To create a Stream from a single value you probably want to use new Stream.fromIterable():
<List<int>> toStream(Object value) =>
    new Stream.fromIterable([UTF8.encode(JSON.encode(value))])

So that should cause a value to actually get into the Stream, and take care of your error. I think the source of you error is that the input stream is not closed, and creating it from an Iterable will close the Stream when the Iterable is exhausted.
pipe() should also now work, so you can try this:
toStream({"a": 1, "B": 2}).pipe(response)
  .then((_) {
    print("done");
  });

All said though, I don't see how this is better than:
response
  ..write(JSON.encode(value))
  ..close();

Comment: well of course, but having simple API that always return Stream is nicer if you want add files and other stuff that actually can produce data in chunks. If I work on stream I can transform it (ie. Gzip) in other part of server, am I wrong?

for example I have a use case where I send large JSON object like 70MB! crazy :)
Comment: And you right, I somehow missed pretty important thing to use controller.add(UTF8.encode(json)); controller.close(); as described here https://www.dartlang.org/articles/creating-streams/
Answer: Try this example POST Request for json application with Streaming response:
POST Request here :
request = http.Request('POST', url);
request.headers.addAll({'Content-Type': 'application/json'});
  request.body = jsonEncode({'code': 'aQDZl8cp'});

StreamingResponse here:
 var streamedResponse = await request.send();
streamedResponse.stream.listen((List<int> chunk) {
//Todo
}

Full code:
Future<void> main() async {
  var url = Uri.parse('http://127.0.0.1:5000/download_file/');

  var request = http.Request('POST', url);
  request.headers.addAll({'Content-Type': 'application/json'});
  request.body = jsonEncode({'code': 'aQDZl8cp'});

  var streamedResponse = await request.send();

  int totalBytes =
      int.tryParse(streamedResponse.headers['content-length'] ?? '') ?? 0;
  int downloadedBytes = 0;

 streamedResponse.stream.listen((List<int> chunk) {
    downloadedBytes += chunk.length;
    double progress = downloadedBytes / totalBytes * 100;
    print('Progression: ${progress.toStringAsFixed(2)}%');
  });
}

how to send http response using stream


Require.js and optimize through r.js: how to figure out optimal way to combine modules for multiple pages?

Question: Using Require.js and it works pretty solid.
However, I'm reading about optimization (http://requirejs.org/docs/optimization.html) using r.js.
To my understanding it bundles top-level defined modules into 1 file  (great because: less http-calls, minified ) .
This seems fine when I only have 1 pagetype defined, i.e: 1 set of required modules to load. However, in any normal site, you'd have multiple pagetypes each requiring different modules.
Now, I'm just wondering how clever r.js really is? Does it take all the multiple pages (with different 'requires')  into account when determining which modules to package together, so that I don't end up with X bundled javascriptf-iles for X pagetypes, even though there's substantial overlap in the modules included in each of these bundled scripts.
Any hints I need to give to the optimizer, is it handled automatically, etc?
Clarification much appreciated,.
Comment: Don't think it's that clever. Think the best you could do would be to optimise your common modules into common.js and shallow-exclude each (used) common module from your optimised page modules. Your pages would then import a single, optimised page module, and also import the optimised common module.
Answer: What I understand is that the optimizer finds dependencies for a file by looking for require() calls, and makes a list of what it finds.  (So it only finds dependencies passed to require() as literal strings.) Then the file is combined with its dependencies (recursively) into one file, and minimized.
You can control what goes into a module in the configuration.  Here's a sample:
({

baseUrl: "Website/Scripts",
dir: "Staging/Scripts",
modules: [
    { name: "main", include: ["Rotator"], exclude: ["jwPlayerDialog"] },
    { name: "jwPlayerDialog" }
],
paths: {
    "jquery": "empty:"
},
fileExclusionRegExp: /^\.|Microsoft|modernizr|unobtrusive|\.min\.|\.debug\./
})

If you turn off minimization, you can easily see what's being included in each file.  A module could be included in more than one file.  For example, the Rotator module above will be in main.js, and it will still be available as Rotator.js.


Use content field as data attribute in Views html

Question: I have a piece of content with an 'Asset Type' field value of Vidyard.  I am using the Views module to render this content on another page.  I have added those fields to the View, but is it possible to add them as a data-attribute to the HTML.  For example, the Views module outputs:

    etc.

How can I add my "Asset Type" field as a data-attribute to the list item.  Making my output:

Is there a way to do this without altering the template?
Answer: There are some limitations if you can't change the output templates for the view (and I'd assume if that's the case a views pre render is also out of scope). If you can accept them within the
 (instead of on the
) you can rewrite the field to use a token. Make sure the field that is just data is listed before the field it will be attached to, enabled rewriting on the field, and use the field tokens to attach the value as an attribute. You'll most likely want to suppress the field from output and do some data cleaning to make sure invalid characters don't end up in the token (like ").

You can attach classes based on the row value to the li by using the same token you would use on the rewrite. Drupal will ensure it's safe for output, but then your data is a class and not the attribute you are seeking.
You'll have more control over the output, and you'll be able to build more elegant solutions if you use hook_views_pre_render() to modify the output more directly.


Odds of a number being the smallest number not picked?

Question:
Have a game where there are $N$ players choosing some $n\in\mathbb N$,
  and the one who picked the lowest number no one else picked wins. Now,
  lets have a scenario where all players pick uniformly randomly a number from $1$ to $N$.
What are the odds that $n$ will be picked and win?
   (Talking about numbers, not players as each player is equally likely to win in this scenario)

Below are my observations so far.

Odds of no one winning are $$ \frac{\text{$f(N)$}}{N^N}$$
Where $f(N)=0, 2, 3, 40, 205\dots$ for $N=1, 2,3,4,5\dots$ players and is defined here:
 $A231797$
Now, how do we find $P(N,n)=?$ ; The odds of number $n$ winning among $N$ players who each pick some $n$ randomly from $1$ to $N$?
One trivial thing is that $P(N,n+1)\le P(N,n),\space N,n\in\mathbb N, n \le N$
 (Lower numbers win more often than larger)

We can write $P(N,n)=\frac{p(N,n)}{N^N}$, then I've found so far:
If $N<2$, then $p(N,1)=1$, else
$$p(N,1)= N(N-1)^{(N-1)}$$
If $N \lt 3$, then $p(N,2)= 0$, else
$$p(N,2)= p(N,1)\times \left( 1- \left(\frac{N-2}{N-1}\right)^{N-2} \right) $$
If $N \lt 3$, then $p(N,3)= 0$, else it equals $6, 36, 440, 6630, 117852\dots$  for $n=3,4,5,6,7\dots$
These so far are based on computed results:
$p(1,n)= 1$
$p(2,n) = 2, 0 $
$p(3,n) = 12, 6, 6 $
$p(4,n) = 108, 60, 36, 12 $
$p(5,n) = 1280, 740, 440, 260, 200 $
$p(6,n) = 18750, 11070, 6630, 3990, 2430, 1230 $
$p(7,n) = 326592, 195342, 117852, 71442, 43512, 26502, 17892 $
I've stopped at this point since I can't extract a formula for $p(N,3)$ already.

How would one mathematically solve this and find a closed expression
  for $P(N,n) ?$
Comment: See the recurrence given at https://oeis.org/A241580 (your $p(N, n)$ are clearly related to the numbers here).  I'm not sure if a non-recursive formula exists.
Answer: Let $X_n$ be the number of players choosing $n$. Then
$$ (X_1, X_2, \ldots, X_N) \sim \text{Multinomial}\left(N; \frac {1} {N}, \frac {1} {N}, \ldots, \frac {1} {N} \right)$$
When the winning number $n = 1$, the winning condition is equivalent to $X_1 = 1$ and the probability is given by
$$ \binom {N} {1} \left(\frac {1} {N}\right)^1 \left(1 - \frac {1} {N}\right)^{N-1} = \left(1 - \frac {1} {N}\right)^{N-1}$$
which is well-mentioned in above discussion. When the winning number $n > 1$, the winning condition becomes $X_n = 1$ and $X_1 \neq 1, X_2 \neq 1, \ldots , X_{n-1} \neq 1$. Thus, the probability is
$$ \begin{align}
&~ \Pr\left\{(X_n = 1) \cap \bigcap_{i=1}^{n-1} (X_i \neq 1)\right\} \\
=&~ \Pr\left\{(X_n = 1) \cap \left(\bigcup_{i=1}^{n-1} (X_i = 1)\right)^c\right\} \\
=&~ \Pr\{X_n = 1\} - \Pr\left\{(X_n = 1) \cap \bigcup_{i=1}^{n-1} (X_i = 1)\right\} \\
=&~ \left(1 - \frac {1} {N}\right)^{N-1} - \Pr\left\{\bigcup_{i=1}^{n-1} [(X_n = 1) \cap  (X_i = 1)]\right\} \\
=&~ \left(1 - \frac {1} {N}\right)^{N-1} - \sum_{j=1}^{n-1} (-1)^{j-1}
\binom {n-1} {j} \Pr\left\{(X_n = 1) \cap \bigcap_{i=1}^{j} (X_i = 1)\right\} \\
=&~ \left(1 - \frac {1} {N}\right)^{N-1} - \sum_{j=1}^{n-1} (-1)^{j-1}
\binom {n-1} {j} \frac {N!} {(1!)^{j+1}(N-j-1)! } \left(\frac {1} {N}\right)^{j+1} \left(1 - \frac {j + 1} {N} \right)^{N-j-1} \\
\end{align}$$
If you pull out the factor $\displaystyle \frac {1} {N^N}$, the expression becomes
$$ \frac {1} {N^N} \left[ N\left(N-1\right)^{N-1} - \sum_{j=1}^{n-1} (-1)^{j-1}
\binom {n-1} {j} \frac {N!} {(N-j-1)!} \left(N - j - 1 \right)^{N-j-1} \right]$$
Comment: Sorry last night I have a typo in last line, accidentally writing $\displaystyle \frac {N!} {(N-j-1)!}$ as $\displaystyle \binom {N} {j+1}$ which missed the $(j+1)!$ which equal to $2$ in the denominator, so that is why it does not match. Now it is corrected.
Comment: At first glance I went to check your last expression; taking out $1/N^N$ leaves us with the rest that makes $p(N,n)$, where for $n=2$, it is not in agreement with my $p(N,2)$ in the post; but becomes when we multiply the sum by $2$ before subtracting it from the left term; seems there could be a typo somewhere in there (or I missed something)? ($p(N,2)= 0,0,6,60,740,11070\dots$ according to my data)
Answer: For $n$ to win, one player has to pick it and all the others must pick something higher.  The probability for that to happen are $N$ (to pick the player who chooses $n$) $\frac 1N$ (that the player picks $n$) $\left(\frac {N-n}N\right)^{N-1}$ (that all the other players pick larger numbers).  The winning number is most likely to be $1$, with probability $\left(\frac {N-1}N\right)^{N-1}=\left(1-\frac {1}N\right)^{N-1}$ which tends to $\frac 1e$ as $N$ gets large.  In general $n$ will be the winning number with probability about $\frac 1{e^n}$ for $n \ll N$.  The chance of having a winning number at all, again for large $N$, is about $\frac 1{e-1}$
Comment: Are you saying $P(N,n)=\left(\frac {N-n}N\right)^{N-1}$ ? That seems to be correct only for $P(N,1)$, which I've already written in the post. You missed out a condition. For $n$ to win all others must pick higher numbers **or** all others must *be disqualified*. For example: $2,2,3$ is a win scenario for $3$, since **it is the lowest number not picked by anyone else.** I should've perhaps made the question more clear.
Comment: Yes, I missed that.  I'll have to think some more.


Views: show nodes that share taxonomy term with user

Question: I have a content type requests with the taxonomy field categories.  I have added categories to my user's profile as well.
I need to create a view that displays all the requests that have category terms the user has selected in their profile.  How do I go about this?
Answer:
Create a view showing requests.
Add a contextual filter for Field: Category.
When the filter value is NOT in the URL choose "Provide default value" of type PHP.
In the "PHP contextual filter code" write:

global $user;
$account = user_load($user->uid);
return $account->field_category['und'][0]['tid']; // this line may need changing based on your category field name


Gerar um array a partir de uma query no mongoDB (Mongoose)

Question: Bom dia! Preciso dos dados de uma coleção do mongo no front end.
Estou usando o Mongoose como ODM, e fiquei meio perdido na hora de enviar os dados para o front end.
Na query ele me retorna os documentos da coleção, e aí eu preciso deles em JSON (eu acredito) pra usar eles no front end pra gerar um gráfico.
Essa é a rota que eu estou usando
router.get('/home', isAuthenticated, function(req, res){

    atendimentos.find({}, async function(err, atendimentos) {
         if(!err){
              console.log(atendimentos);
         }
    })

    res.render('home', {user: req.user, atendimentos: atendimentos});
})

Vi que dava pra usar o lean(), mas ele retorna objetos puros do javascript pelo que eu entendi. Eu poderia usar ele e usar um JSON.stringify eu imagino, mas aí na rota não posso usar dois res, se não a aplicação crasha.
Gostaria de saber como eu poderia fazer para ter esses dados no front end, se alguém puder me dar uma luz, agradeço muito!
Answer: Para passar os dados e receber no front.
router.get('/home', isAuthenticated, function(req, res){
    atendimento.find({}, (err, result) => {
        if(!err){
            res.render('home', { result })
        }else{
            console.log(err)
        }
})

E uso ejs como engine de view ai para ter acesso aos dados é bem simples apenas
usando essa sintaxe <% = atendimento.name %> ou <%= user %>

Async/await

router.get('/home', isAuthenticated, async (req, res) =>{
    const result = await atendimento.find({})
    res.render('home', { result })
})

Comment: Eu preciso dos dados do atendimento no front, o modelo dele no mongo tem 13 campos se eu não me engano, mas eu só preciso de 4 (não sei se isso é relevante, mas eu sei filtrar isso, sem problemas). Acho que desse jeito que tu falou passando const data pode funcionar. Vou testar aqui, preciso dos dados para renderizar um gráfico. Obrigado pela ajuda, vou testar aqui e retorno!
Comment: oks, qualquer coisa só falar.
Comment: Tentei algo assim:  const data = {atendimentos} porque o user já tenho e funciona direitinho, aí passei os dados assim: res.render('home', {user: req.user, chartData: data}, mas aí no gráfico deu que chartData é undefined.
Comment: Mas ai é mesmo coisa que você fez antes, diferença que adicionou mais uma camada nos dados, acho que fica pior assim. Poderia me dar mais informações, de que forma você esta manipulando os dados no front?
Comment: Eu tenho essa view home aí, aí lá eu tenho um script que tem a parte do ChartJS que eu estou querendo usar pra renderizar o gráfico a partir do banco. Lá tem um campo data que é aonde vão os dados, aí tentei passar esse chartData pra lá e não rolou. To meio perdido.
Comment: Então, você saberia me dizer se ta usando alguma engine para a view?
Comment: ExpressJS, .ejs é minha view engine
Comment: Ótimo, o script ta importado na page certo? Se ele estiver charData é sua variavel la, da um `<%=JSON.stringify(chartData)%>` em qualquer lugar da pagina só para ver se os dados estão dentro. Usando esse render seu aqui `res.render('home', {user: req.user, chartData: data}`
Comment: Se for o script do chart, sim. Vou ver aqui, vamos testar. Qualquer coisa vamos para o chat porque aqui  está ficando longo.
Comment: Vamos [continuar esta discussão no chat](https://chat.stackexchange.com/rooms/81008/discussion-between-danibrum-and-ams).


Create a Groupedby HTML table using linq query

Question: Using the following:  Windows 7, Visual Studio 2013, VB, Bootstrap, Jquery
I am new to web design.
I am trying to create a list/table...anything...to show a list of Equipment that is Grouped by their Department.  I currently just have the Department listed.  There are 4 records in the table:
Two records are in Department 01 and Two are in Department 02
There are two equipment in each department.
The code below is pulling in all the records creating a multiple list of same department:
 - 01
 - 01
 - 02
 - 02

I would like to have:
 - 01
 - Equipment1
 - Equipment2
 - 02
 - Equipment1
 - Equipment2

The line:
mUserHistorys = mUserHistorys.OrderBy(Function(x) x.Department_Selected).ToList()

is where I am pulling in from data in memory which is loaded when the page is loaded.
How do I get unique Departments?  I've tried Distinct and GroupBy but I don't believe I'm placing them correctly.
Private Sub RenderPage()

Dim HTMLString As String = ""
Dim SortKey As String = "A"
Dim CompareValue As String = ""
Dim SubmitString = ""

mUserHistorys = mUserHistorys.OrderBy(Function(x) x.Department_Selected).ToList()
For Each p In mUserHistorys
    SubmitString = "Bookmark:" & p.Department_Selected & "-" & p.Equipment_Selected

    HTMLString &= "
 " & vbCrLf
    HTMLString &= "" & p.Department_Selected & "	" & vbCrLf
    HTMLString &= "
 " & vbCrLf

Next
Me.literalListHTML.Text = HTMLString
End Sub

Answer: @stuggyg
This what I ended up with that is working.  My only issue now is that the Equipment are not ascending.  But, thank you very much for your help!  It helped me come up with this code:
    Private Sub RenderPage()

    Dim HTMLString As String = ""
    Dim SortKey As String = "A"
    Dim CompareValue As String = ""
    Dim SubmitString = ""

    Dim Depts = From dept In mUserHistorys
                Select dept.User_ID, dept.Department_Selected, dept.Equipment_Selected
                Where User_ID = _user.User_ID
                Order By Equipment_Selected Ascending
                Group By DeptName = Department_Selected
                Into EqInDept = Group
                Order By DeptName

    HTMLString &= "
 " & vbCrLf
    For Each p In Depts
        HTMLString &= "" & p.DeptName & "" & vbCrLf

        For Each r In mUserHistorys
            SubmitString = "Bookmark:" & r.Department_Selected & "-" & r.Equipment_Selected
            If r.Department_Selected = p.DeptName Then
                HTMLString &= "<a id=""equiplist"" class=""list-group-item"" <button id=""iconButton"" onclick=""SubmitNextForm('" & SubmitString & "');""  type=""button"" class=""btn btn-normal""></button>" & r.Equipment_Selected & "" & vbCrLf
            End If
        Next
        HTMLString &= " " & vbCrLf
    Next
    HTMLString &= "
 " & vbCrLf

    Me.literalListHTML.Text = HTMLString
End Sub

Answer: var info = (from b in mUserHistorys
                        where b.Department_Selected != null
                        select b).ToList()
                        .GroupBy(x => x.Department_Selected)
                        .Select(y => y.First())
                        .OrderBy(z => z.Department_Selected);

I have had luck using this linq query for something similar. If it is already a List of objects you can start at the .GroupBy i.e
mUserHistorys.GroupBy(x => x.Department_Selected)
                        .Select(y => y.First())
                        .OrderBy(z => z.Department_Selected);

Comment: I've tried both and they have flags on the x, y and z.  They state: x is not declared. It may be inaccessible due to its protection level.
Comment: I have tested a sample code. Apologies for my VB code style I am unfamiliar with VB.NET compared to C#.

 [.NetFiddle Demo](https://dotnetfiddle.net/BD9TH9).


A sorted dictionary where it is easy to move to the next key

Question: I have an ordered set of key-value pairs, for example with the following entries:
1 "one"
2 "two"
4 "four"
50 "fifty"

I would like to have a quick lookup (so given an int key, I want to find the value for that key), but also ideally have a quick way of finding the next key in the dictionary from a current key - so that given the key 2, find that the next key is 4, and then 50.
I know that a Dictionary does the first one quickly, and something like a linked-list for the second part too (but it's difficult to 'jump in' to start at a specific key).
I've had a look here, and it seems like some of this might be possible with a sorted dictionary? I wondered if there is a good data structure in C# to do both of these things (lookup by key and moving to the next key)?
I don't need the number of items to be very large (maybe in the thousands), but if possible, I would like to do a large number of lookups and move forward between keys quickly (without checking wheter 5, 6, 7... are present in the dictionary).
Comment: You may want to look at http://stackoverflow.com/questions/4720674/how-do-i-get-previous-key-from-sorteddictionary
Comment: dictionary already has collection of keys. so you just iterate over the keys. `foreach (var key in dict.Keys)`
Comment: I would suggest if a singular data type like SortedDictionary or SortedList doesn't help, it may be optimal to store the data in two separate structures, and interchangably use either depending on the optimal strategy. This would increase the cost of keeping both up to date, though. @M.kazemAkhgary I believe he means optimizing for entering a function with just `int 2` rather than doing full collection loops.
Comment: you may want to try `.Skip(1)` to get the next item in the `IEnumerable`.
Comment: @Katana314 if `1,2,4,50` are the only items inside dictionary then you just iterate over them. there is no `3,5,6,...`. but if user wants to specify orders himself he can either have separate ordered collection of keys. and just iterate over that.
Comment: @M.kazemAkhgary If the numbers go up to 3,729, 4,012, 5,623, and higher, then he's going to be doing a lot of iterations over possibly-absent numbers to find the point of insertion he's interested in (eg, 3500 - and the three collected numbers just above it). Dictionaries are not really optimized for that sort of sequential iteration, so this would have a very high complexity.
Answer: What you are looking for is OrderedDictionary in System.Collections.Specialized
In this collection you can get Key, and you can take item at next index next to already to found one, but out of the box implementation from Microsoft won't work, since its missing all required by you methods like TryGetValue or IndexOf.
Have a look at those pages:
MSDN
Custom ordered dictionary
Answer: You could put the info in your Value:
public class MyValue
{
  string Value;
  int NextId;
  int PreviousId;
}

public Dictionary<int, MyValue>();

It's then trivial to get your previous or next id. It's then trivial to get your next or previous value.
Of course, the insert logic requires you to update the previous & next each time you add something.
Comment: it would be better to have `int?` for next and previous ids. because then you know where it ends. other wise `0` can be mistaken by another id.
Answer: SortedList is suitable for this.
SortedList<int, string> sortedList = new SortedList<int,string>();

sortedList.Add(1, "one");
sortedList.Add(2, "two");
sortedList.Add(4, "four");
sortedList.Add(50, "fifty");

int currentIndex = sortedList.Keys.IndexOf(2);

Console.WriteLine(sortedList.Keys[currentIndex+1]);
Console.WriteLine(sortedList.Values[currentIndex+1]);


Section 6.6. Tu's Introduction to manifolds, definition of partial derivative.

Question: I have an issue with understanding the notation of the following definition of partial derivative
$$
\left. \frac{\partial}{\partial x^i} \right|_{p} f := \frac{\partial f}{\partial x^i}(p) := \frac{\partial \left( f \circ  \phi^{-1} \right)}{\partial r^i}(\phi(p)) := \left. \frac{\partial}{\partial r^i} \right|_{\phi(p)} \left( f \circ  \phi^{-1} \right).
$$
What I cannot really figure is if the following is actually well posed
$$
\frac{\partial f}{\partial x^i}(p) := \frac{\partial \left( f \circ  \phi^{-1} \right)}{\partial r^i}(\phi(p))
$$
Because I don't really understand the meaning, to me the idea should be to transform the partial derivative of a function defined on a manifold to a partial derivative of a function defined in a open set of $\mathbb{R}^n$. Now if we look at the RHS of the last equation I wrote we have $f \circ \phi^{-1}$, which is a function from an open in $\mathbb{R}^n$ to $\mathbb{R}$, however the variable w.r.t. we differentiate is also a function from $\mathbb{R}^n \to \mathbb{R}$.
My question is, why is such definition well posed?
Comment: what you mean by "well posed"? The definition of the partial derivative is just the partial derivative transported to an euclidean space, so it is well-defined as a function. Moreover: this partial derivative holds the properties of a derivation, as you should verify
Comment: I tried to explain at my best what I meant with "well posed". I also highlighted the specific part of the definition that puzzles me and why.
Comment: By standard coordinate, I think he means the usual projection map to the ith coordinate. I am almost certain you do this without it. I like to offer a different view. If you recall from calculus, you often do implicit differentiation on surfaces $f(x,y,z) = 1$, it is usually assumed that $y = y(x)$, but we often also treat $y$ as a variable to compute $\partial_y f$
Answer: Let $g = f \circ \phi^{-1} : V\to \mathbb R$ which is defined on an open subset of $\mathbb R^n$. The meaning of
$$\frac{\partial g}{\partial x^i}$$
is clear (although in multivariable analysis usually $x_i$ is written instead of $x^i$). Tu writes it in the form
$$\frac{\partial g}{\partial r^i}$$
to avoid confusion with the local coordinates $x^i$ on $U$ introduced by $\phi : U \to V$. It is just the usual partial derivative of $g(r^1,\ldots,r^n)$ with respect to the $i$-th coordinate.
I think you are confused because Tu says

If $r^1,\ldots,r^n$ are the standard coordinates on $\mathbb R^n$, then $x^i = r^i \circ \phi$.

We may of course regard the $r^i$ and $x^i$ as coordinate functions living on $V$ and $U$, but this does not affect the meaning of $\frac{\partial g}{\partial r^i}$. You must not interpret this as a special case of an expression $\frac{\partial g}{\partial \psi}$ where $\psi : U \to  \mathbb R$ is some function - such a concept does not exist.
Comment: @Masacroso It makes of course sense with respect to a function $\beta : (a, b) \to U$, but it is not known to me how to generalize this to functions living on $U$. One could perhaps define $\frac{\partial g}{\partial \psi}(r) = \lim_{h\to 0}\frac{g(r+h) -g(r)}{\psi(r+h) - \psi(r)}$, but this would not be compatible with $\frac{\partial g}{\partial r^i}$.
Comment: well the derivative respect to a function is an old concept, it indeed exists, but im not sure if this concept is compatible with this context
Comment: Need to read through your answer. Thank you
Comment: @PaulFrost Could please help me to understand the following better? In the answer, $\partial g/ \partial r^i$ means the differentiation of $g$ with respect to $r^i$, where $r^i:\mathbb{R}^n \to \mathbb{R}$ is a function. Here, $g = g(r^1, \cdots, r^n)$ is a function of functions $r^i$. If we are given an explicit form of $g$, then we can differentiate it and get an expression, in general, in terms of $r^1, r^2, \cdots r^n$; which can be considered as a function with domain $\mathbb{R}^n$ and hence can be evaluated at $\phi(p)$, $p\in U$. This is understanding correct?


Can you run a multiple regression where variables use different Likert scales?

Question: I have 8 predictor variables and 1 criterion variable, all variables are measured by 9 different constructs, some are 0-4 Likert scales, some  are 1-5 Likert scales, can I use their mean scores and run a multiple regression?
Comment: Can you say more about your situation, @Shirley? Do you mean that you have 9 different Likert *items*, where some are numbered 0-4 & others 0-5? Or do you have multiple items (forming a *scale*) per construct, and you want to average over all the the items w/i a construct to get a single mean for each variable? Or something else? In formulating your question, it might help to read this blog post: [how to ask a statistics question](http://www.statisticalanalysisconsulting.com/how-to-ask-a-statistics-question/). Also, if your criterion is ordinal, you might consider ordinal logistic regression.
Comment: Thanks for prompt response and advice. Here is the situation:
Comment: I have 8 predictor variables using 8 different questionnaires. These 8 variables are stress level, self esteem, peer relationship, family relationship, neuroticism, boredom proneness, depression, social anxiety (So there are 8 different questionnaires). However, this 8 questionnaires have its each form of scales.
Comment: For example, Stress questionnaire is using 0-4 scale (0 = strongly disagree, 4 = strong agree); Self esteem is using 0-3 scale (0 = strongly disagree, 3 = strong agree); Peer relationship is using 1-4 scale (1 = strongly disagree, 4 = strong agree) and etc. My criterion variable is Online gaming addiction using 1-4 scale (1 = strongly disagree, 4 = strong agree)
Comment: So what I'm doing now is that I calculate mean score for these 8 predictor and 1 criterion variable (in other word, I have one single mean for each variable, so in total I have 8 predictor mean scores and 1 criterion mean score now).
Comment: My question is, since the questionnaires are using different form of likert scale, I wonder if this is alright to use these 8 predictor mean scores and 1 criterion mean score to run multiple regression, in order to get the prediction relationship.
Comment: Please do advise! Appreciate your help!
Comment: More information for you, I'm planning to run test of normality to test if this is normally distributed, then followed by correlation among all variables and multiple regression.
Answer: @Shirley, using the mean scores for each of the 8 questionnaires, even though they have different response anchors (i.e., 0-4 or 1-7 or 1-4), is definitely acceptable in multiple regression. In fact, it is very rare that you'll end up having measures/questionnaires that use the same set of response anchors.
Comment: Thanks for answering my question! As I read from some stats articles, some researchers center or standardize the data. So I'm just confused..
Comment: On top of the 8 predictors variables, I'm also measuring gender difference on the online gaming addiction (my criterion variable). So obviously gender has 2 levels (Males and Females). Since it is nominal data, I'm thinking to use independent t test to test out the gender difference on the criterion variable. Wondering for this particular test, do I still use the mean or centered mean?
Comment: One last question...Just to confirm, it is fine to use the mean even the response anchors involves 0, correct?
Comment: Hi Shirley, I think each of these questions could be separated as they touch on different regression related methodological issues. But just to give you an idea:
Comment: Centering data is useful if you plan on running a moderation or mediated moderation analysis. For simple multiple regression there is really no need to standardize and center your data (as far as my knowledge of regression, but you may want to check with others).
Comment: You don't necessarily have to run t-tests (although it is an option) just because your outcome variable is gender. Technically, you could run a regression analysis with a dichotomous outcome variable, its just that the interpretation doesn't sound to great (i.e., as stress increases so does male(ness)?, etc.) but the analysis will tell you if there are significant relationships between gender and your other variables.
Comment: Regarding your question if it is fine to use means even if one of the anchors involves 0 I would definitely look at how other researchers have used the scales and computed the means, etc. This is very important and should be mentioned in your methods section.
Comment: CORRECTION: Centering data is useful for testing interactions, quadratic effects, etc.
Comment: Thank you so much for explanation! Indeed it helps! Thanks again!