Why is Unix/Terminal faster than R?

Question: I'm new to Unix, however, I have recently realized that very simple Unix commands can do very simple things to large data set very very quickly.  My question is why are these Unix commands so fast relative to R?
Let's begin by assuming that the data is big, but not larger than the amount of RAM on your computer.
Computationally, I understand that Unix commands are likely faster than their R counterparts.  However, I can't imagine that this would explain the entire time difference.  After all basic R functions, like Unix commands, are written in low-level languages like C/C++.
I therefore suspect that the speed gains have to do with I/O.  While I only have a basic understanding of how computers work,  I do understand that to manipulate data it most first be read from disk (assuming the data is local). This is slow.  However, regardless of whether you use R functions or Unix commands to manipulate data both most obtain the data from disk.
Therefore I suspect that how data is read from disk, if that even makes sense, is what is driving the time difference.  Is that intuition correct?
Thanks!
UPDATE:  Sorry for being vague.  This was done on purpose, I was hoping to discuss this idea in general, rather than focus on a specific example.
Regardless, I'll generate an example of counting the number of rows
First I'll generate a big data set.
row = 1e7
col = 50
df<-matrix(rpois(row*col,1),row,col)
write.csv(df,"df.csv")

Doing it with Unix
time wc -l df.csv
real    0m12.261s
user    0m1.668s
sys     0m2.589s

Doing it with R
library(data.table)
system.time({ nrow(fread("df.csv")) })
...
user   system   elapsed
26.77    1.67     47.07

Notice that elapsed/real > user + system.  This suggests that the CPU is waiting on the disk.
I suspected the slow speed of R has to do with reading the data in.  It appears that I'm right:
system.time(fread("df.csv"))
user  system  elapsed
34.69   2.81    47.41

My question is how is the I/O different for Unix and R.  Why?
Comment: There are many functions in R and in unix that have varying rates of speed. Do you have at least one concrete example of identical functions in R and unix with timed tests?
Comment: This question is extremely broad...
Comment: @cr1msonB1ade and nrussell is this better
Comment: @Jacob H Yes it is, I'm voting to reopen. Also, you may find it useful to examine the [source code](http://www.gnu.org/software/cflow/manual/html_node/Source-of-wc-command.html) for `wc`.
Comment: Or perhaps this is the [source](http://code.metager.de/source/xref/gnu/coreutils/src/wc.c).
Answer: I'm not sure what operations you're talking about, but in general, more complex processing systems like R use more complex internal data structures to represent the data being manipulated, and constructing these data structures can be a big bottleneck, significantly slower than the simple lines, words, and characters that Unix commands like grep tend to operate on.
Another factor (depending on how your scripts are set up) is whether you're processing the data one thing at a time, in "streaming mode", or reading everything into memory.  Unix commands tend to be written to operate in pipelines, and to read a small piece of data (usually one line), process it, maybe write out a result, and move on to the next line.  If, on the other hand, you read the entire data set into memory before processing it, then even if you do have enough RAM, allocating and organizing all the necessary memory can be very expensive.
[updated in response to your additional information]
Aha.  So you were asking R to read the whole file into memory at once.  That accounts for much of the difference.  Let's talk about a few more things.
I/O.  We can think about three ways of reading characters from a file, especially if the style of processing we're doing affects the way that's most convenient to do the reading.

Unbuffered small, random reads.  We ask the operating system for 1 or a few characters at a time, and process them as we read them.
Unbuffered large, block-sized reads.  We ask the operating for big chunks of memory -- usually of a size like 1k or 8k -- and chew on each chunk in memory before asking for the next chunk.
Buffered reads.  Our programming language gives us a way of asking for as many characters as we want out of an intermediate buffer, and code that's built into the language ("library" code) automatically takes care of keeping that buffer full by reading large, block-sized chunks from the operating system.

Now, the important thing to know is that the operating system would much rather read big, block-sized chunks.  So #1 can be drastically slower than 2 and 3.  (I've seen factors of 10 or 100.)  But no well-written programs use #1, so we can pretty much forget about it.  As long as you're using 2 or 3, the I/O speed will be roughly the same.  (In extreme cases, if you know what you're doing, you can get a little efficiency increase by using 2 instead of 3, if you can.)
Now let's talk about the way each program processes the data.  wc has basically 5 steps:

Read characters one at a time.  (I can assure you it uses method 3.)
For each character read, add one to the character count.
If the character read was a newline, add one to the line count.
If the character read was or wasn't a word-separator character, update the word count.
At the very end, print out the counts of lines, words, and/or characters, as requested.

So as you can see it's all I/O and very simple, character-based processing.  (The only step that's at all complicated is 4.  As an exercise, I once wrote a version of wc that contrived not to do all of steps 2, 3, and 4 inside the read loop if the user didn't ask for all the counts.  My version did indeed run significantly faster if you invoked wc -c or wc -l.  But obviously the code was significantly more complicated.)
In the case of R, on the other hand, things are quite a bit more complicated.  First, you told it to read a CSV file.  So as it reads, it has to find the newlines separating lines and the commas separating columns.  That's roughly equivalent to the processing that wc has to do.  But then, for each number that it finds, it has to convert it into an internal number that it can work with efficiently.  For example, if somewhere in the CSV file occurs the sequence
...,12345,...

R is going to have to read those digits (as individual characters) and then do the equivalent of the math problem
 1 * 10000 + 2 * 1000 + 3 * 100 + 4 * 10 + 5 * 1

to get the value 12345.
But there's more.  You asked R to build a table.  A table is a specific, highly regular data structure which orders all the data into rigid rows and columns for efficient lookup.  To see how much work that can be, let's use a slightly far-fetched hypothetical real-world example.
Suppose you're a survey company and it's your job to ask people walking by on the street certain questions.  But suppose that the questions are complicated enough that you need all the people seated in a classroom at once.  (Suppose further that the people don't mind this inconvenience.)
But first you have to build that classroom.  You're not sure how many people are going to walk by, so you build an ordinary classroom, with room for 5 rows of 6 desks for 30 people, and you haul in the desks, and the people start filing in, and after 30 people file in you notice there's a 31st, so what do you do?  You could ask him to stand in the back, but you're kind of fixated on the rigid-rows-and-columns idea, so you ask the 31st person to wait, and you quickly call the builders and ask them to build a second 30-person classroom right next to the first, and now you can accept the 31st person and in fact 29 more for a total of 60, but then you notice a 61st person.
So you ask him to wait, and you call the builders back again, and you have them build two more classrooms, so now you've got a nice 2x2 grid of 30-person classrooms, but the people keep coming and soon enough the 121st person shows up and there's not enough room and you still haven't even started asking your survey questions yet.
So you call some fancier builders that know how to do steelwork and you have them build a big 5-story building next door with 50-person classrooms, 5 on each floor, for a total of 50 x 5 x 5 = 1,250 desks, and you have the first 120 people (who've been waiting patiently) file out of the old rooms into the new building, and now there's room for the 121st person and quite a few more behind him, and you hire some wreckers to demolish the old classrooms and recycle some of the materials, and the people keep coming and pretty soon there's 1,250 people in your new building waiting to be surveyed and the 1,251st has just showed up.
So you build a giant new skyscraper with 1,000 desks on each floor and 100 floors, and you demolish the old 5-story building, but the people keep coming, and how big did you say your big data set was?  1e7 x 50? So I don't think the 100-story building is going to be big enough, either.  (And when you're all done with all this, the only "survey question" you're going to ask is "How many rows are there?")
Contrived as it may seem, this is actually not too bad an analogy for what R is having to do internally to build the table to store that data set in.
Meanwhile, Bob's discount survey company, who can only tell you how many people he surveyed and how many were men and women and in which age brackets, is down there on the streetcorner, and the people are filing by, and Bob is jotting down tally marks on his clipboards, and the people, once surveyed, are walking away and going about their business, and Bob isn't wasting time and money building any classrooms at all.
I don't know anything about R, but see if there's a way to construct an empty 1e7 x 50 matrix up front, and read the CSV file into it.  You might find that significantly quicker.  R will still have to do some building, but at least it won't have any false starts.
Comment: Thanks!  In general I guess that the reason why the Unix code is fast is two fold.  First, the data structure is more complicated for R than Unix.  Second, everything is loaded into RAM in the complex data structure of R, the same is not true for Unix, namely, only small chunks of data are read in at a time.    As a side note, what is the data structure for the Unix command (sorry that last sentence was probably worded poorly.  As you might have guessed I'm not a computer scientist)
Comment: Another factor to consider: R will test whether any of the data is NA (= missing datum, datum Non Available).

Why is Unix/Terminal faster than R?


Normalize data by use of ratios based on a changing dataset in R

Question: I am trying to normalize a Y scale by converting all values to percentages.
Therefore, I need to divide every number in a column by the first number in that column. In Excel, this would be equivalent to locking a cell A1/$A1, B1/$A1, C1/$A1 then D1/$D1, E1/$D1...
The data needs to first meet four criteria (Time, Treatment, Concentration and Type) and the reference value changes at every new treatment. Each treatment has 4 concentrations (0, 0.1, 2 and 50). I would like for the values associated to each concentration to be divided by the reference value (when the concentration is equal to 0).
The tricky part is that this reference value changes every 4 rows.
I have tried doing this using ddply:
`MasterTable <- read.csv("~/Dropbox/Master-table.csv")`
  MasterTable <- ddply(MasterTable, .(Time, Type, Treatment), transform, pc=(Value/Value$Concentration==0))

But this is not working at all. Any help would be really appreciated!
My data file can be found here:
Master-table
Thank you!
Comment: Possible duplicate of [In R, how do I compute factors' percentage given on different variable?](https://stackoverflow.com/questions/24901061/in-r-how-do-i-compute-factors-percentage-given-on-different-variable)
Answer: dplyr is very efficient here:
library(dplyr)
result <- group_by(MasterTable, Time, Type, Treatment) %>%
  mutate(pc = Value / Value[1])

Comment: Thank you Remko! You figured it out, its exactly what I was looking for.
Comment: Hello Remko, the above solution you provided to me last year is no longer working... it always divides the number by the first value in the column. It's as if the grouping function is being ignored. Has there been a change to dplyr that would cause this? Any help you could provide me would be greatly appreciated.
Comment: I will take a look. No surprise that dplyr is not backward compatible.


Model performance graph is oscillating?

Question: I've seen a fair amount of information online regarding the loss graph of a deep learning model oscillating, but not so much about the performance graph. In general, when I "properly" train a neural network, the performance usually increase and then plateaus. However, I've noticed that in some cases the performance itself is very shaky. What is the cause of such behavior and what could be a fix?
One thing that comes to mind is that decreasing the learning rate may help because the variance seems high, but I'm not sure if this is based on any proof. Any tips are appreciated. Thanks.
Edit
Here's an example of what I'm talking about. I would expect that the macro-F1 score should also display similar patterns to micro-F1 despite being lower. However, it's oscillating quite a bit.
Comment: @SextusEmpiricus Sure, just edited it in.
Comment: @SextusEmpiricus By "performance graph" I meant a graph of the performance metrics being printed after each epoch.
Comment: I should probably have made my post a little clearer. Both of the lines are for the validation. The orange line is the micro-F1 measure and the blue is the macro-F1 measure. The two metrics differ simply in how they're being calculated, and so to me it's strange that the macro-F1 is showing such "shaky" behavior.
Comment: @SextusEmpiricus A loss graph can be viewed as a performance graph. I was simply referring to performance metrics here.
Comment: I think it is more clear now. To confirm: So your question is how you can have differences in the shaky behaviour of performance graphs for different performance measures (like in the displayed graph).
Comment: @SextusEmpiricus Yes, that is correct. I'm assuming that it's because of buggy training processes, but am unsure which prompted me to post a question.
Answer: With a classification problem one can compute multiple $F_1$ scores for the classes seperately. The micro and macro $F_1$ scores compute a single score for all classes together.
If a few categories have little representation in the validation sample then their individual $F_1$ scores will be noisy and this influences the grouped macro and micro $F_1$ scores. The differences in the weight these two give to different classes makes that the one can be more noisy than the other.

The macro-$F_1$ score computes a non-weighted average of $F_1$ scores for multiple categories.
The noise in the $F_1$ score of one or more small categories influences the macro-f1 score as well.

The micro-$F_1$ score sums all the false/true negative/positive cases before computing a single $F_1$ score.
The number of false/true negative/positive cases in the categories that are small will be small and so these have relatively less influence on the sum. The micro-$F_1$ score will be relatively less noisy in comparison to the macro-$F_1$ score which gives more weight to the smaller categories.
Answer: Begin digression:
micro-F1 is just accuracy. A short proof:

accuracy is the number of correct predictions divided by the total number of examples.
Recall is TP/(TP+FN) that is the accuracy of the positive class (TP+FN are the total number of positive examples).
Precision is TP/(TP+FP) which is in some way the accuracy when the classifier says "positive" (TP+FP are all examples classified by positive by the classifier).
F1 is the harmonic mean of precision and recall
micro recall is $\sum TP_i / (\sum TP_i + \sum FN_i)$ where $i$ are the classes.
$\sum TP_i $ are all examples correctly classified
$\sum FN_i$ are all the examples incorrectly classified
therefore micro-recall is correctly-classified divided by total number of examples, and therefore accuracy
micro-precision is $\sum TP_i / (\sum TP_i + \sum FP_i)$ .
$\sum FP_i$ is also all the examples incorrectly classified,
therefore micro-precision is also the accuracy
micro-F1 is the harmonic mean of the same value, accuracy, which is also accuracy.

end digression
Now answering the question. @SextusEmpiricus answer show that macro-F1 should be sensitive to small changes in the accuracy of the small classes.

If the set being used to measure the macro-F1 changes , than one will expect larger  changes in the macro-F1. But the problem is that the usual way generating a performance graph is to have a fixed validation set. So I need the OP to confirm that indeed the set being used to generate the measures plotted in the graph is fixed.

if the set is fixed, then the most likely explanation is that the validationn set is unbalanced  - there are a few classes with large number of examples and a few with few examples. The neural network is is optimizing a global measure, not accuracy but something similar, and in the process it finds that it is worth to be a little more bold with the large classes - if something looks like it is form a large class A, say it is from A. Since A has a lot of examples, this increases the global measure. But this may remove examples that were correctly classified in a small class B. Now B's accuracy (recall) is worse and since there are only a few examples of B, this reduction on its accuracy is large, and there is a large reduction on b's F1. That would explain some reduction on the macro-F1.

there is no problem with that and is not an indication of error. The NN is optimizing a global measure (like but not exactly accuracy) and in the process these small classes may suffer. But if what you want is a NN with high macro-F1 (and not necessarily high accuracy) you will need to change what the NN is optimizing for.
This is the limit of my knowledge - there has been a recent literature on imbalanced multi-class classification, and I would guess that some of it is about neural networks. Maybe this literature proposes different cost fuctions or different regularizations for the cost function so metics like macro-F1 are "optimized" (the metric is not optimized, the cost function plus the regularization are).

Model performance graph is oscillating?


What are the examples for stochastic processes in Electrical Engineering and Computer Science?

Question: I wanted to find out what kind of different usages of stochastic processes theory in EE & CS are out there. For example, I find these kinds of usages interesting:

using stochastic signal as carrier that is modulated by the information signal for communication
using stochastic process analysis for improving photographed pictures after long exposition

It would be good if every example would be in separate answer, so it would be possible to up-vote those that are 'the best'. I would classify good example as one that:

truly takes advantage of probability and statistics theory
has greater 'usefulness' then other examples
Answer: What about monitoring network congestion. An underlying stochastic process is assumed to be driving the congestion, and the process has reached an equilibrium state and is stationary. That is, process characteristics such as the mean level does not depend on time. You can then use this the stochastic process to give you an idea of when you check the network for possible congestion.
Some recent papers:

N. Dueld, C. Lund, and M. Thorup. Optimal combination of sampled network measurements. Proceedings of IMC '05, Internet Measurement Conference, 91-104, 2005.
B. M. Parker, S. G. Gilmour, and J. Schormans. Measurement of packet loss probability by optimal design of packet probing experiments. IET Communications, 3:979-991, 2009.
Answer: Google's Pagerank algorithm is a stochastic approximation (through Markov's stationary state) of the proportion of visits received asymptotically by website $x_i$.
Answer: As you have alluded to, communications engineering (and signal processing in general) is filled with stochastic processes. See, for example:

Chapter 9 and later of Communications Systems by Carlson and Crilly
Chapter 13 of A Course in Digtial Signal Processing by Porat

This is only two textbook examples. You will find many (many) more. It's fair to say that your mobile phone, the internet and etcetera all rely extremely heavily on our understanding of stochastic processes.


How to access the url or page that is currently loading on navigation with Next.js

Question: I want to add per page custom skeleton loaders on page navigation. To do that I need to know which page is currently being loaded.
I have tried this approach with useRouter:
//_app.tsx
useEffect(() => {
    const handleStart = () => setLoading(true);
    const handleComplete = () => setLoading(false);
    router.events.on('routeChangeStart', handleStart);
    router.events.on('routeChangeComplete', handleComplete);
    router.events.on('routeChangeError', handleComplete);

    console.log(router);
  }, [router]);

The routechange events are being triggered when starting to navigate, but the route console log does not print the page I'm going to until it's already loaded. I have noticed that the url in the browser is not changing before the page loading is completed.
I need a way to get the current loading page (the pathname or anything that can tell me what is currently loading). Is this even possible?
I am using Next.js, TypeScript and SSR.
Answer: You can access the path in the routeChangeStart handler function as it receives the URL of the route you're navigating to when fired in its first argument.
useEffect(() => {
    const handleStart = (url: string) => {
        console.log(url); // Will log the path to the page being navigated to
        setLoading(true);
    };
    const handleComplete = () => setLoading(false);
    router.events.on('routeChangeStart', handleStart);
    router.events.on('routeChangeComplete', handleComplete);
    router.events.on('routeChangeError', handleComplete);

    // Cleanup event listeners
    return () => {
        router.events.off('routeChangeStart', handleStart);
        router.events.off('routeChangeComplete', handleComplete);
        router.events.off('routeChangeError', handleComplete);
    }
}, [router]);

As a side note, it's good practice to cleanup any event listeners set beforehand in the useEffect's cleanup phase.


How add icon in QListView using QStringListModel?

Question: Is somehow possible to add icons to a ListView using a QStringListModel ?
This is what I'm doing.
QStringListModel* model;
QStringList List;
model->setStringList(List);
ui->listView->setModel(model);
...

model->setData(index, "Test");
model->setData(index,QIcon(":/icon.png"),Qt::DecorationRole);

unfortunately the icon does not appears on the list.
How can I add icons to the list?
Answer: QStringListModel does not support roles other than DisplayRole and EditRole.
Use QStandardItemModel instead in order to display icons via DecorationRole:
auto model = new QStandardItemModel(this);
ui->listView->setModel(model);
model->appendRow(new QStandardItem(QIcon(":/icon.png"), "Test"));


Difference between leg-spin & flipper

Question: What is the difference between Leg spinner's leg-spin & Flipper?
Answer: Leg Spin from a bowler can be varied as Googly, Top Spinner, slider and flipper.

To bowl a googly, you grip the ball the same as you would for a leg break but hold it so your palm faces away from you. As you bowl, you bring your elbow close to your head and cock your wrist so that back of your hand faces the batsman. You spin the ball by rotating the fingers. If done properly, the ball should turn from off to leg for right handers. If the batsman comes out of his crease and misses the googly, he can be stumped.
To bowl a topspinner, grip the ball as you would a leg break. Bowl it as you would a googly, but don't cock you wrist during the action. This should impart forward spin that will cause the ball to dip later and bounce higher.
To bowl a slider, grip the ball as you would a leg break, but hold it so that your palm faces yourself. Bowl normally. This should impart backwards spin and should feel like a horribly failed leg break. If bowled properly, however, the ball will dip earlier and keep lower to the ground.
To bowl a flipper, you must hold the ball as you would normally, but you must also have your thumb on the side. Instead of rotating your fingers and wrist to give it spin, you pinch it out between your index finger, middle finger, and thumb, kind of like trying to squeeze it. The flipper will come out faster than a leg break, won't spin, and will keep close to the ground.

For more clarification click here

Difference between leg-spin & flipper


Como somar um campo de uma determinada ID com vários registros?

Question:
A história é a seguinte: um político pode ser votado em várias zonas, por isso o motivo do mesmo ID de candidato conter vários registros dentro da tabela.
O ID_CANDIDATO sendo 120000000171, ele tendo 6 registros no banco de dados referente as 6 zonas diferentes que ele recebeu votos.
Como montar uma sql que mostrará 1 registro com todas as informações desse candidato e o total de votos recebidos por este?
ID        ID_CANDIDATO       TOT_VOTOS
21        120000000171       339
45        120000000171       405
73        120000000171       513
88        120000000171       380
92        120000000171       752
96        120000000171       439

Um exemplo visual do que estou buscando está nesta página. O último campo é o TOT_VOTOS.
Answer: Você pode utilizar o SUM. Para isso, será preciso agrupar os registros:
SELECT SUM(tot_votos) as total_votos FROM ... GROUP BY id_candidato

Para obter as informações do candidato na mesma pesquisa, você precisará de um JOIN com a tabela de candidatos:
SELECT SUM(tz.tot_votos) as total_votos, tc.*
    FROM tabela_zonas tz
    JOIN tabela_candidatos tc ON tc.id = tz.id_candidato
    WHERE ...
    GROUP BY tz.id_candidato

Comment: Todos os dados estão dentro de uma mesma tabela. Pode me ajudar a extrair o `SUM` e o resto das informações? Obrigado.
Comment: `SELECT *, SUM(TOTAL_VOTOS) as total_votos FROM VOTACAO_CANDIDATO_MUN_ZONA_2012_MS WHERE NOME_MUNICIPIO = 'CAMPO GRANDE' GROUP BY SQ_CANDIDATO`, obrigado velho !!!
Answer: Você pode usar SUM() para somar os votos e GROUP BY para juntar os registros pelo código do seu candidato.
SELECT SUM(TOT_VOTOS) FROM sua_tabela
WHERE id_candadito = $seu_id GROUP BY id;

Você pode recuperar informações como nome e afins adicionando JOIN. Mas a princípio o SQL acima dará o resultado que você deseja.
Mas caso não seja o resultado esperado ou precise modificar ou adicionar algo, por favor avise!


Suppose $A$ is an $n \times n$ idempotent matrix and let $I$ be an $n \times n$ identity matrix. Prove that the matrix $I −A$ is an idempotent matrix

Question: A square matrix $A$ is called idempotent if $A^2 = A$.
(a) Suppose $A$ is an $n × n$ idempotent matrix and let $I$ be the $n × n$ identity matrix. Prove that the matrix $I −A$ is an idempotent matrix.
(b) Assume that $A$ is an $n×n$ non zero idempotent matrix. Then determine all integers $k$ such that the matrix $I − kA$ is idempotent.
I need help. I didn't know what to do... All I know is that I need to show that $(I−A)^2=(I−A)$. I don't have any clue on how I should proceed. Thanks in advance
Answer: An idempotent matrix $A$ by definition satisfies
$A^2 = A; \tag 1$
from this we immediately have
$(I - A)^2 = I^2 - 2A + A^2 = I - 2A + A = I - A, \tag 2$
that is, $I - A$ is also idempotent.
Now if $I - kA$ is idempotent, then via (1) we find
$I - kA = (I - kA)^2 = I - 2kA + k^2A^2 = I - 2kA + k^2A, \tag 3$
whence
$(k^2 - k)A = 0; \tag 4$
since $A \ne 0$ this forces
$k^2 - k = 0 \Longrightarrow k = 0, 1; \tag 5$
thus the only candidates for $I - kA$ are
$I - kA = I, I - A. \tag 6$
.
Comment: thanx very much...thats crystal clear
Comment: @johnDoe:  my pleasure sir!


problems with Recursion - C++

Question: Given a natural number n (1 <= n <= 500000), please output the summation of all its proper divisors.
Definition: A proper divisor of a natural number is the divisor that is strictly less than the number.
e.g. number 20 has 5 proper divisors: 1, 2, 4, 5, 10, and the divisor summation is: 1 + 2 + 4 + 5 + 10 = 22.
<<--This is a challenge i am trying to do and i am using recursion
int find_sum(std::vector <int> nums,long int sum,int num_now,long int j)
{

if(j<nums[num_now])
{
    if(nums[num_now]%j==0)
    {
        sum=sum+j;
    }
    return find_sum(nums,sum,num_now,j+1);
}
else
{
 return sum;
}

}

sum is the sum of all divisors,nums is the vector i stored number in,num_now is current member in vector,int j is 1 i use it to search for dividers,sadly using this i cant use numbers like 500000 it give's me error,is there any better way to do it or have i made a mistake somewhere.
--Thank you for your time
Comment: Does it have to be made using recursion?
Comment: Any reason that you want to use recursion?
Comment: Before considering optimizations, did you try a brute force solution just to make it work?
Comment: Be aware of *Stack Overflows*, where your program runs out of memory due to the depth or recursion.  You can better control the iterations using a loop instead.
Comment: are you aware that there is a straightforward [formula](http://planetmath.org/formulaforsumofdivisors) to calculate this quantity? No need to use recursion...
Comment: I understand that i do not need to use recursion but i wish to use it for educational means
Answer: Here is a recursive way to solve your problem:
int find_sum(int x, int i)
{
  if(i == 0)
    return 0;
  if(x % i == 0)
    return i + find_sum(x, (i-1));
  return find_sum(x, (i-1));
}

You need to call find_sum(N, N-1); in order to find sum of dividers of N (i must be less than given N because of strict inequality).
In your case it would be find_sum(20, 19);
e.g. my function returns:

71086 for N = 50000
22 for N = 20
0 for N = 1
Answer: I don't see the reason why you need to use recursion for solving this problem. I would prefer a more staightforward way to solve it.
long CalculateSumOfDivisors(int number)
{
    long sum = 0;
    for(int i=1; i<number; i++)
    {
        // If the remainder of num/i is zero
        // then i divides num. So we add it to the
        // current sum.
        if(number%i==0)
        {
            sum+=i;
        }
    }
    return sum;
}

Furthermore, we could write a more optimal algorithm, if we note the following:
Let that we have a number n and d is the smallest divisor of n that is greater of 1. (Apparently if the number n is a prime number there is any such a divisor). Then the larget divisor of n is the number n/d.
Based on this we can formulate a more optimal algorithm.
long CalculateSumOfDivisors(int number)
{
    int smallestDivisor = FindSmallestDivisor(number);
    if(smallestDivisor==1) return 1;

    long sum = smallestDivisor;

    // Calculate the possible greatest divisor.
    int possibleGreatestDivisor = (int)floor(number/smallestDivisor);

    for(int i=smallestDivisor+1; i<=possibleGreatestDivisor; i++)
    {
        if(number%i==0)
        {
            sum+=i;
        }
    }
    return sum;
}

int FindSmallestDivisor(int number)
{
    int smallestDivisor = 1;
    for(int i=2; i<number; i++)
    {
        if(number%i==0)
        {
            smallestDivisor = i;
            break;
        }
    }
    return smallestDivisor;
}

Comment: You can also stop at `number/2`.   No value greater than 1/2 can be a divisor, right?
Comment: This is correct ! But if the smallest divisor is 3 then we can stop at `number/3`, which I think is a bit better.
Answer: I don't think you should use recursion.
Instead start by looping from 1..N-1
When you find a divisor adjust the end value for the loop. Example if 2 is a divisor then you know N/2 is also a divisor. And just as important you know there can be no further divisors in the range ]N/2:N[. Likewise if 3 is a divisor then you know N/3 is also a divisor and you know there are no more divisors in the range ]N/3:N[.
Following that concept you can reduced the number of loops significantly for most numbers.
Something like:
long find_sum(int num)
{
    long sum = 0;
    int max = num;
    int i = 1;
    while(i < max)
    {
        if(num % i == 0)
        {
            sum += i;        // Add i to the sum
            max = num / i;   // Decrement max for performance
            if (max != i && max != num)
            {
                sum += max;  // Add max when max isn't equal i
            }
        }
        i++;
    }
    return sum;
}

Example:
num = 10
sum = 0
i = 1 -> sum = 1, max = 10
i = 2 -> sum = 1+2+5, max = 5
i = 3 -> sum = 1+2+5, max = 5
i = 4 -> sum = 1+2+5, max = 5
i = 5 -> return 8 (1+2+5)

num = 64
sum = 0
i = 1 -> sum = 1, max = 64
i = 2 -> sum = 1+2+32, max = 32
i = 3 -> sum = 1+2+32, max = 32
i = 4 -> sum = 1+2+32+4+16, max = 16
i = 5 -> sum = 1+2+32+4+16, max = 16
i = 6 -> sum = 1+2+32+4+16, max = 16
i = 7 -> sum = 1+2+32+4+16, max = 16
i = 8 -> sum = 1+2+32+4+16+8, max = 8
i = 9 -> return (1+2+32+4+16+8)

The number of loops are kept down by changing max whenever a new divisor is found.
Answer: I tried writing code with main function asking user to give the it wants to get sum of. here is the code , hope it helps.
#include<iostream>

using namespace std;

int Sum(int min, int max, int &val, int &sum){
if(min >= max)
    return 0;

for ( ; min < max; min++){
    if ( val%min == 0){
        sum += min + val/min;
        return Sum(++min,val/min, val,sum);
    }
}
    return 0;
}

int main(){
    int s=1;
    int val;
    cout <<"Enter Val to sum:";
    cin >> val;
    Sum(2,val,val,s);
    cout <<"Sum is :"<<s<<endl;
    return 0;
}

Here Sum function is used recursively and passed parameters as shown in the code.
Hope it helps.

problems with Recursion - C++


mysqldump error #1557: corrupt event table?

Question: I just noticed the following error from my old mysql backup scripts:
% mysqldump -hsql -uXXXX -pXXXX --events --all-databases > /dev/null
mysqldump: Couldn't execute 'show events': Cannot proceed because system tables
     used by Event Scheduler were found damaged at server start (1577)

Poking around seems to indicate some sort of corruption with mysql.events (I can dump without errors if I skip the --events flag). I'm worried that something's seriously wrong with the database and can't fine any clear instance of someone reporting the same problem. Closest match seems to indicate that myisamchk should be used to rebuild the ISA index, but I'm not convinced it's the same problem.
I'm running mysql v14.14 on an ubuntu 10.04.4 LTS system. Other than this error, I'm not seeing any other symptoms/problems.
I was hoping someone could provide some advice before I start poking around and risk really corrupting the db.
Answer: Sounds like a broken mysql upgrade.
I would suggest to try :
mysql_upgrade -u root -h localhost -p --verbose --force

and then /etc/init.d/mysql restart
Finally, try again :)


Is there a way to show a page dynamically when hovering on it's link?

Question: Just wondering if there is a module or some way in general to do the following:
When hovering on a page link show the content of that page as a tooltip or some kind of small dropdown window.
Thanks.
Comment: There's an answer here: http://stackoverflow.com/questions/3162815/jquery-webpage-preview which covers the requirement in general terms. I don't know if anyone has built a Drupal module for it, though it should certainly be possible.
Answer: Beauty Tips is a drupal module and a jQuery plugin that will allow you to display content tooltip style via hover, click or a range of other events.
It provides an API for adding custom beautytips as well as some built-in features. Admins can enable rollover balloon-help style tooltips for text fields within forms. It also supplies hover tips for Drupal help links and advanced help links.
Most importantly, it allows developers to add their own custom beautytips popups to their site without having to delve into jQuery.
You can use this module and fetch the data which do you want to display in tool tip.


Time machine concept in SonarQube

Question: I am using SonarQube version 5.6 and it is having the feature to "COMPARE". But I am unable to find the Time Machine option in this version. In older version like 5.1, Time machine option is available. Is there any way to achieve the same in SonarQube version 5.6 or higher?
Comment: See this [discussion](https://groups.google.com/forum/#!topic/sonarqube/rpBplmbFFD0)
Answer: Thanks Jeroen Heier for the link. It is very useful and I understood the similar feature of "History" in Measures tab satisfies my requirement. It displays the analysis history in graphical format which is easy to understand.

Time machine concept in SonarQube


NestJs monorepo shared lib injection

Question: I have NestJS application with couple microservices stored in single repository (monorepo approach).
AccessControl module stores in libs, it should be shared across multiple microservices. It has AccessControlModule.ts file
@Global()
@Module({
  providers: [
    {
      provide: 'CONNECTION1',
      useFactory: (configService: ConfigService) => {
        return ClientProxyFactory.create(
          configService.getRMQConnection(),
        );
      },
      inject: [ConfigService],
    },
    ACGuard,
  ],
  exports: [ACGuard],
  imports: [ConfigModule],
})
export class AccessControlModule implements OnModuleDestroy {
  constructor(
    @Inject('CONNECTION1')
    protected readonly orgConnection: ClientProxy,
  ) {}

  onModuleDestroy(): any {
    this.orgConnection.close();
  }
}

This file responsible for module description, it creates connection for another microservice and provide it to ACGuard service. ACGuard.ts:
@Injectable()
export class ACGuard implements CanActivate {
  constructor(
    private readonly reflector: Reflector,
    private config: ConfigService,
    @Inject('CONNECTION1')
    private readonly userConnection: ClientProxy;
  ) {}

  public async canActivate(context: ExecutionContext): Promise<boolean> {
   // do some stuff
  }
}

This part by itself compiles well and logically works fine. Problem begins when I try to inject it into one of microservices. I do it as usual by adding AccessControlModule into import part of some module. For example KioskModule:
@Module({
  imports: [
    ...
    AccessControlModule
  ],
  providers: [
    ...
    KiosksResolver
  ]
})
export class KiosksModule {}

Since AccessControlModule marked as Global and exports ACGuard I expect it to be injectable into my providers.
@Resolver('Kiosk')
export class KiosksResolver {
  ...

  @UseGuards(ACGuard)
  @Query()
  kiosks() {
    // ...
  }

   ...
}

But this code falls on the compilation step with error:

[Nest] 9964   - 05/07/2020, 9:33:02 PM   [ExceptionHandler] Nest can't resolve dependencies of the ACGuard (Reflector, ConfigService, ?). Please make sure that the argument CONNECTION1 at index [2] is available in the KiosksModule context.

On the other hand, if i inject it in KiosksResolver's constructor, application builds successfully.
I will appreciate any help and ideas, thanks!
Answer: The way how i solved this issue was exporting CONNECTION1 provider in AccessControlModule.ts.
@Module({
  providers: [
    {
      provide: 'CONNECTION1',
      useFactory: (configService: ConfigService) => {
        return ClientProxyFactory.create(
          configService.getRMQConnection(),
        );
      },
      inject: [ConfigService],
    },
    ACGuard,
  ],
  exports: [ACGuard, 'CONNECTION1'],
  imports: [ConfigModule],
})
export class AccessControlModule ...

With this export KioskModule creates it's own ACGuard but provides here connection exported from AccessControlModule.
It's not clear for me why KioskModule doesn't get built instance of ACGuard exported from AccessControlModule but try build it once more.

NestJs monorepo shared lib injection


Why is the Wilcoxon test not significant for this data?

Question: The 2 data.frame count values look quite significantly different to me, so I'm wondering why the output of the Wilcoxon test is not "significant."
Here is my data:
df1 <- structure(list(time = c("Day 1", "Day 1"), count = c(59L, 1L)), row.names = c(NA, -2L), class = "data.frame")

df2 <- structure(list(time = c("Day 2", "Day 2", "Day 2", "Day 2", "Day 2", "Day 2", "Day 2", "Day 2"), count = c(1L, 207L, 5L, 3L, 3L, 11L, 2L, 1L)), row.names = c(NA, -8L), class = "data.frame")

The p-value calculated is: p-value = 0.8353
The code I ran for the test is:
wilcox.test(df1$count, df2$count, alternative = 'two.sided', exact = FALSE)

Comment: In the example data above there are only 2 points for df1, thus I don't believe there is a large enough sample to make a fair comparison to df2 thus a high p-value.
Comment: what is the test you ran with this data?
Comment: (i.e. - what was the line of code to do the Wilcox test?)
Comment: Since your provided data has 4 groups, you could not have used this data for a Wilcox test unless you eliminated two of the groups.
Comment: @dcarlson, there are only 2 groups?
Comment: Apologies, I shared the wrong code. I've edited the original post now to include the two data frames I used.
Comment: This doesn't appear to be a specific programming question that's appropriate for Stack Overflow. If you have general questions about interpreting results from statistical mdoels, then you should ask such questions over at [stats.se] instead. You are more likely to get better answers there.
Answer: Here is part of what ?wilcox.test will give you:

if both 'x' and 'y' are given and 'paired' is 'FALSE',
a Wilcoxon rank sum test (equivalent to the Mann-Whitney test: see
the Note) is carried out.  In this case, the null hypothesis is
that the distributions of 'x' and 'y' differ by a location shift
of 'mu' and the alternative is that they differ by some other
location shift (and the one-sided alternative '"greater"' is that
'x' is shifted to the right of 'y').

Note that df1 has only two entries, while df2 has only eight, and (importantly!) they overlap in range - specifically, the entire range of df1$count is contained in the range of df2$count.
There is simply not enough signal there to reject the null hypothesis that the distributions do not differ in location.

Put conversely, for sample sizes of 2 and 8, your distributions would need to be completely disjoint for the Wilcoxon test to reject the null:
> wilcox.test(1:2, 3:10)

        Wilcoxon rank sum exact test

data:  1:2 and 3:10
W = 0, p-value = 0.04444
alternative hypothesis: true location shift is not equal to 0

Of course, since this is a rank-based test, if they samples do not overlap, it does not matter how far apart they are:
> wilcox.test(1:2, 4:11)

        Wilcoxon rank sum exact test

data:  1:2 and 4:11
W = 0, p-value = 0.04444
alternative hypothesis: true location shift is not equal to 0

(Note how both the test statistic and the p value are identical to the previous case.)

You write that "The 2 data.frame count values look quite significantly different to me", but (1) differences in distributions is not what the Wilcoxon test looks at. As above, it looks at differences in location between two samples. If you want to check whether two samples are different "as such", you could try something based on tabulating the data, perhaps with a $\chi^2$ test. However, (2) your sample is also much too small for that.


80% of population is vaccinated and 25% of infected had received the vaccine, what is the efficacy of the vaccine?

Question: 80% of population is vaccinated and 25% of infected had received the vaccine before being infected, what is the efficacy of the vaccine?
Lets denote
$\Pr(V) = 0.8$ (V="vaccinated")
$\Pr(I)$ is not given (I = "infected")
$\Pr(V\mid I) = 0.25$
What is the efficacy of the vaccine?
I am dumbfounded as I need to have either $\Pr(I)$ or $1-Pr(I)$ to solve any probability question related to this problem. For example $\Pr(I \mid V) = \frac{Pr(V \mid I)Pr(I)}{Pr(V)}$ which can be simplified because we know both $Pr(V)$ and $Pr(V \mid I)$: $Pr(I \mid V)=\frac{0.25*Pr(I)}{0.8}$, but then we hit brick wall as Pr(I) is not given.
Comment: How do you quantitatively define efficiency? Also, surely you mean $P(V|H)=0.25$.
Comment: Are you sure you don't mean "efficacy" or "effectiveness"?  Can you provide a link to the use of term "efficiency" in this context?
Comment: Pr(V | H) =0.25, Pr(!V | H) = 0.75, Pr(V) = 0.8, Pr(!V) = 0.2. I don't know what is vaccine efficiency, but maybe it is something like Pr(H | !V) / Pr(H | V), which could be calculated wih Bayes formula.
Comment: I assume efficacy is correct term here. English is my second language. I'll submit this question to language exchange as well.
Answer: My response to the OP's concluding paragraph:
It is a common misconception that vaccine efficacy means the probability $P(I^c|V)$ of non-infection given that one is vaccinated.
Rather, vaccine efficacy measures the increase in protection conferred by the vaccine.
Your risk $P(I^c|V)$ of non-infection depends on the disease prevalence, and your exposure, safety measures, state of health, and vaccination status; vaccine efficacy (and vaccine effectiveness, for that matter) is not measuring this probability!
(Vaccine effectiveness is with reference to real-world data, whereas vaccine efficacy refers to clinical-trial results.) Note that none of this discussion, including what follows, takes immunity duration into consideration.

My Answer to main Question:
The technical definition of vaccine efficacy is given at the bottom of my Answer here. Note that this formula is based on an equal number of vaccinated and unvaccinated study subjects, which is not the case in your problem.
But we can adapt: based on your given setup/context,

there are $4$ times as many vaccinated as there are unvaccinated
subjects;
$25\%$ of the infected subjects had been vaccinated prior.

Thus, if the vaccine effectiveness is exactly $0\%$ (in other words, the vaccine has no effect), then we expect that among the infected, there will $4$ times as many vaccinated as there are unvaccinated subjects.
Since the actual multiple is $\frac{25}{75}=\frac13$ instead of $4,$ then the relative risk $R$ is $$\frac{\frac13}4=8.33\%,$$ so the vaccine effectiveness is $$1-R=91.67\%.$$
Comment: The answer cannot be 93.75% (15/16) since if you consider a population of 300 susceptible people with a vaccine coverage rate of 80% then 240 are vaccinated and 60 are non-vaccinated. If the efficacy is 15/16 then 240x(1 – 15/16)= 15 of the vaccinated fall sick. Add this to the 60 non-vaccinated who fall sick gives 15/(15+60)=0.2, or 20%, not the required 25%.
Comment: @DerekJennings Thanks for catching my original wrong answer $(93.75\%),$ which had arisen from a typo (I'd originally written $\frac14$ instead of the intended $\frac13$) in the final paragraph. The answer has been rectified. Reverse the downvote?
Comment: Downvote reversed, no problem!
Answer: Suppose we have a group of $n$ people all of whom are susceptible to the disease when not vaccinated, and let $e$ be the efficacy of the vaccine (expressed as $ 0 \le e \le 1 $ rather than as a percentage) and let $v$ ($0 \le v \le 1$) be the population coverage of the vaccine. Then the number of vaccinated people falling sick, $s$, will be given by
$$ s = nv \left( 1 – e \right) \quad (1)$$
and the number of non-vaccinated people falling sick, $\bar s,$ will be given by
$$ \bar s = n \left( 1 – v \right) . \quad(2)$$
In the question the vaccine coverage is $80\%$, and so $v=0.8$. If we denote by $p$ the fraction of people falling sick that are vaccinated then we have
$$ p = \frac{ s } {s+\bar s}.$$
We are also given that $p=0.25$ since $25\%$ of the people falling sick have been vaccinated. Using equations (1) and (2) we get
$$ p =\frac{ s } {s+\bar s} = \frac{nv \left( 1 – e \right)}{nv \left( 1 – e \right) + n \left( 1 – v \right)}.$$
And so
$$ p= \frac{v-ve}{1-ve} ,$$
whereby solving for $e$ we obtain
$$ e= \frac{v-p}{v\left( 1-p \right) }.$$
Plugging in $p=0.25$ and $v=0.8$,
$$e= \frac{0.8-0.25}{0.8 \left( 1 – 0.25 \right) } = \frac{11}{12} \text{ or about } 91.67\%.$$
We can verify this by considering a population of $300$ susceptible people. Since the vaccine coverage is $80\%$, $240$ are vaccinated and $60$ are non-vaccinated. As the vaccine effectiveness is $11/12$ then $ 240 \times \left( 1- 11/12 \right) = 20$ of the vaccinated fall sick, which we add to the $60$ who fall sick in the non-vaccinated portion. Hence the fraction of vaccinated among all the sick is $20/ \left(20+60 \right) = 0.25,$ as required. Note that I have replaced the word efficacy with effectiveness. This is because we normally talk about efficacy in a trial situation and effectiveness in a real world situation and since we are reverse-engineering to obtain $e$ the data is very likely to come from a real world situation.


How does the application know the controller class when i do an ActionLink? ASP mvc

Question: When i do @Html.ActionLink("About us", "About", "Home"), how does the application know that the controller file is in Controllers\HomeController.cs? I see controller folders separated such as 'Views\Home', 'Views\Booking' etc. However, how does it know to look for Controllers\HomeController.cs when the controller name specified was Home? My code is perfectly working due to visual studio's generated files. However, i would like to know what is the theory behind it. Please explain, thank you.
Here is my route config,
    public static void RegisterRoutes(RouteCollection routes)
    {
        routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

        routes.MapRoute(
            name: "Default",
            url: "{controller}/{action}/{id}",
            defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
        );
    }

here is my HomeController,
public class HomeController : Controller
{
    public ActionResult Index()
    {
        return View();
    }

    public ActionResult About()
    {
        return View();
    }
}

Comment: @user859385 you'd have to dig into the MVC sources. Afaik there is no fixed "Home" = "HomeController"-mapping. It is a "" = "Controller"-mapping. Most likely in the standard MVC routing classes. If you add another controller and e.g. name it "HappyController" with an Action "Laugh", you can access it with the URL /Happy/Laugh. The framework adds the "Controller" part to the class name on its own.
Comment: It doesn't. It'll just generate the link `/Home/About`, not caring whether a controller for that route exists, and if so, what file it comes from. It does take configured routes in account.
Comment: @CodeCaster i see, my `/home/About` is About.cshtml, does this mean that it loads About.cshtml first? And for some reason, the About.cshtml gives off the controller file name? I have updated the route config inside the question. Please give you expert advice. Thanks!
Comment: the connection is between the route and the controller, the view doesn't come into play, it is determined by the controller. In your /home/About-case, the controller is determined by the "home"-Part of your route. MVC has some fixed rules that for instance turn a route with controller "Home" into a route that uses a class HomeController (if that exists). You can see in your default route, that the first part of the URL makes up the controller name, the second part the action name, the last part is the parameter used when calling the action.
Comment: https://weblogs.asp.net/scottgu/asp-net-mvc-framework-part-2-url-routing might be helpful.
Comment: @DirkTrilsbeek Yes sir i understand you. However, could you please further explain on the part where you said, "MVC has some fixed rules that for instance turn a route with controller "Home" into a route that uses a class HomeController (if that exists).". Yes, HomeController exists. Can you please guide me to the part (or perhaps file/code?) where the rules set such that when "Home" is specified it routes to HomeController.cs? I hope you understand! Thank you!
Comment: Yes. Now i understand. So its actually not very complicated. Great! Thats cleared up! @user2326106 helped me with his answer too! Thank you!
Answer: Essentially, it's all .net mvc doing this. So when you feed it that ActionLink, it knows that "Home" is the Controller which houses a function named About.
When it's time to invoke the function to render something; the DefaultControllerFactory .net mvc uses will be start enough to figure out that the given controller is missing the word "Controller", so it will append it prior to invocation.
There's a really nice guide I've used in the past about how to override  the DefaultControllerFactory:
https://www.codeproject.com/Articles/599189/DefaultControllerFactory-in-ASP-NET-MVC
Which might give you a better sense of what's going on under the hood :)
Comment: Thank you! Your answer and @Dirk Trilsbeek comments helped me out! Great link to know whats under the hood (:


Porque tá retornando esse erro?

Question: Estou utilizando o plugin jQuery Datatable e comecei a fazer a ativação server-side
mas ao carregar ele informa um alert:

DataTables warning: table id=example - Invalid JSON response. For more information about this error, please see http://datatables.net/tn/1

Seguindo eu fiz o debug, e no último passo, em que é para mostrar o JSON com os dados, aparece: This request has no response data available.
Porque esse erro ainda está ocorrendo? PoQUE reSposta Invalida do JSON, o que está invalido????????
A página com a tabela html:

E o server_processing.php:
<?php
$table = 'tbl_medic';
$primaryKey = 'id_Medic';

$columns = array(
    array( 'db' => 'nome', 'dt' => 0 ),
    array( 'db' => 'cidade',  'dt' => 1 ),
    array( 'db' => 'cep',   'dt' => 2 ),
    array( 'db' => 'crm',     'dt' => 3 ),
    array( 'db' => 'email', 'dt' => 4 ),
    array( 'db' => 'pastaDocumentos',     'dt' => 5 )
);

$sql_details = array(
    'user' => '***', //Usuário do banco de dados
    'pass' => '***', //Senha do banco de dados
    'db'   => '***', //Banco de dados
    'host' => 'localhost'
);

require_once( 'ssp.class.php' );
echo json_encode(
    SSP::simple( $_GET, $sql_details, $table, $primaryKey, $columns )
);

Answer: Olá!!
Tente forçar o array para object da seguinte maneira:
$columns = (object) array(
    array( 'db' => 'nome', 'dt' => 0 ),
    array( 'db' => 'cidade',  'dt' => 1 ),
    array( 'db' => 'cep',   'dt' => 2 ),
    array( 'db' => 'crm',     'dt' => 3 ),
    array( 'db' => 'email', 'dt' => 4 ),
    array( 'db' => 'pastaDocumentos',     'dt' => 5 )
);

Porque tá retornando esse erro?


Query MYSQL não ordena corretamente

Question: Olá. Tenho a seguinte query:
    SELECT *
      from usuarios_cursos_matriculados
 LEFT JOIN cursos ON cursos.cedoc_doc_id_fk = usuarios_cursos_matriculados.cedoc_doc_id_fk
 LEFT JOIN cedoc_doc ON cedoc_doc.cedoc_doc_id = cursos.cedoc_doc_id_fk
 LEFT JOIN cursos_aulas ON cursos.cedoc_doc_id_fk = cursos_aulas.cedoc_doc_id_fk
 LEFT JOIN (SELECT * FROM ultimas_aulas order by data_realizada DESC) AS c ON c.id_curso = cedoc_doc.cedoc_doc_id
  GROUP BY cursos_aulas.cedoc_doc_id_fk DESC

Neste caso, a tabela deveria puxar os cursos nos quais o usuário está matriculado contidos na tabela usuarios_cursos_matriculados, a primeira aula do curso, contida na tabela cursos_aulas e a última aula visualizada, contida na tabela ultimas_aulas. A query está funcionando pela metade, ela puxa corretamente os cursos matriculados e a primeira aula de cada curso, porém, ela não ordena corretamente as últimas aulas visualizadas pelo usuário nos cursos selecionados. Ao invés dela ordenar de ordem decrescente de acordo com a coluna data_realizada, que contém a data na qual a data foi visualizada, ela ordena em ordem crescente. Quando eu removo o group by, ela ordena corretamente, porém, os registros ficam duplicados.
Como eu poderia corrigir o funcionamento dessa query?
Comment: Creio que aqui: `GROUP BY cursos_aulas.cedoc_doc_id_fk DESC` você pulou a cláusula ORDER BY`.
Comment: A consulta está correta? Só a ordem que está errada? Você quer somente a última aula ou todas ordenadas?
Answer: Você deve adicionar a clausura Order by no final da consulta segue como ficaria:
    SELECT *
      from usuarios_cursos_matriculados
 LEFT JOIN cursos ON cursos.cedoc_doc_id_fk = usuarios_cursos_matriculados.cedoc_doc_id_fk
 LEFT JOIN cedoc_doc ON cedoc_doc.cedoc_doc_id = cursos.cedoc_doc_id_fk
 LEFT JOIN cursos_aulas ON cursos.cedoc_doc_id_fk = cursos_aulas.cedoc_doc_id_fk
 LEFT JOIN (SELECT * FROM ultimas_aulas order by data_realizada DESC) AS c ON c.id_curso = cedoc_doc.cedoc_doc_id
  GROUP BY cursos_aulas.cedoc_doc_id_fk DESC
  ORDER BY c.data_realizada DESC

Comment: Dessa maneira ele até ordena pela data, porém, ainda puxa o primeiro registro de cada curso, ao invés do último.
Comment: VocÊ pode adicionar mais order by em sequencia, ex:
neste caso foi:
**ORDER BY c.data_realizada DESC**
voce pode ir adicionando com uma virgula, ficando assim:
**ORDER BY c.data_realizada, (outra coluna para ordernar), (outra se quiser), (Etc) DESC**

Query MYSQL não ordena corretamente


How am I able to test that a Backbone model's method bound to the event bus has fired when using Karma and Sinon?

Question: When testing that a backbone model's event has fired with a sinon spy, it erroneously errors: expected doSomething to be called once but was called 0 times, even though it seems to execute when a console log is put in the method's body. The testing function looks like:
it('Y U NO WORK', function() {
    const events = {};
    _.extend(events, Backbone.Events);
    const Model = Backbone.Model.extend({
        initialize: function() {
            this.listenTo(events, 'doSomething', this.doSomething);
        },
        doSomething: function() {},
    });
    const model = new Model();
    const spy = sinon.spy(model, 'doSomething');
    events.trigger('doSomething');
    sinon.assert.calledOnce(spy);
});

I know that to fix, you'd have to put the sinon spy on the Model's prototype like const spy = sinon.spy(Model.prototype, 'doSomething'); in the line before the new Model() call, however it seems to work without issue when put in the model instance, like below:
it('And this does work', function() {
    const Model = Backbone.Model.extend();
    const model = new Model();
    const spy = sinon.spy(model, 'set');
    model.set('foo', 'bar');
    sinon.assert.calledOnce(spy);
});

Curious why it needs to be put on the model's prototype in the first instance, but works on the model instance in the second?
Answer: Spy kind of replaces the original method with a custom one to know when it's invoked (It's holds reference to original for restoring later). So in the first case you set up an event listener before creating the spy. The event system is actually holding direct reference to the original method, not to the spy. Spy can do nothing about it, Spy wouldn't know when it is invoked.
You need to set up the spy first before setting up the event listener,  something like:
it('Y U NO WORK', function() {
    var spy;
    const events = {};
    _.extend(events, Backbone.Events);
    const Model = Backbone.Model.extend({
        initialize: function() {
            spy = sinon.spy(this, 'doSomething');
            this.listenTo(events, 'doSomething', this.doSomething);
            //now same as this.listenTo(events, 'doSomething', spy);
        },
        doSomething: function() {},
    });
    const model = new Model();
    events.trigger('doSomething');
    sinon.assert.calledOnce(spy);
});

Or avoid keeping direct references to the original method like:
this.listenTo(events, 'doSomething', function() {
  //by the time this is invoked, original has been replaced with spy
  this.doSomething();
});

It'd work because it's not holding reference to original method, the method invokation is dynamic


php multi-dimensional array help needed

Question: I've got this array (shortened for this question), and I need to extract the country_code ("AF" and "AL" in this demo) in order to insert the region info into a table based on the country.
How do I get the country code while iterating the array and is this the correct way to do this?
$countries = array("AF" => array("BDS" => "Badakhshan",
                                 "BDG" => "Badghis",
                                 "BGL" => "Baghlan",
                                 "BAL" => "Balkh",
                                 "BAM" => "Bamian",
                                 "DAY" => "Daykondi"),
                   "AL" => array("BR" => "Berat",
                                 "BU" => "Bulqizë",
                                 "DL" => "Delvinë",
                                 "DV" => "Devoll",
                                 "DI" => "Dibër",
                                 "DR" => "Durrës",
                                 "EL" => "Elbasan",
                                 "FR" => "Fier")
);

foreach ($countries as $country) {
  $country_code = $country[]; // How do I get the country code here?
  foreach ($country as $region_code => $region_name) {
    // insert region info into table
  } // foreach ($country as $region_code => $region_name)
} // foreach ($countries as $country)

Answer: foreach($countries as $code => $list) {
 foreach($list as $rcode => $name) {

 }
}

code and rcode will have the two region codes
i Mentioned in my comment, that that was the only way, but, i stand corrected
foreach($countries as $country)
{
        $code = array_keys($countries, $country);
        $code = $code[0];
}

might get you what you are looking for, super weird way to do it tho, and i dont suggest using it. The first method is better
Comment: Ignored my question "How do I get the country code while iterating the array"
Answer: Your array is setup with key => value pairs, meaning you have a value, and an identifier for that value.
$myArray = array( "Key" => "Value" );

Or, in the case of your code:
$myArray = array( "Country Code" => array( "Region" => "Codes" ) );

If you wish to get the key while looping, use the following syntax:
foreach ( $myArray as $key => $value ) {
  echo $key; // "Country Code"
  foreach ( $value as $region_key => $region_code ) {
    echo $region_key; // Region
  }
}

Now you're able to access the identifer with each iteration.
Comment: Perfect, just what I was looking for.
Comment: MB34, this is the same thing i posted. and the same thing the other person posted
Comment: the exact equivalent but using while loop http://stackoverflow.com/a/9164501/1165128
Comment: [quote]this is the same thing i posted.[/quote] It wasn't either. It showed how to get the country_code and yours did not originally. I stand by my accepting Jonathan's post as answer.
Comment: mine did show how, i said in my post "code and rcode will have the two region codes"
Answer: Well, you are using it already in the nested loop:
foreach ($countries as $country_code => $country) {
    foreach ($country as $region_code => $region_name) {
        // foobar
    }
}

The variable $country_code then holds the country code.
Comment: No I did not. Sorry, I thought it was obvious that the variable `$country_code` would hold the country code. I edited my answer to clarify that.
Comment: Ignored my question "How do I get the country code while iterating the array"
Answer: while($country = current($countries)){
    while($region = current($country)){
        echo "region:".$region."(".key($country).")"." country:".key($countries);
        next($country);
    }
    next($countries);
}

php multi-dimensional array help needed


OpenSSL 1.0.0 RSA parameters in C

Question: I realized that I cannot use the function RSA_get0_key in OpenSSL 1.0.0 to extract the values of n, e, d by reading the private key from a file and passing it as parameter to the aforementioned function.
It is not a programming issue, I mean, I know how to use the functions, but I don't know if there is an alternative to do this.
Indeed, the warning that stops me during the compiling operation is the following:
warning: implicit declaration of function 'RSA_get0_key'; did you mean 'RSA_check_key'? [-Wimplicit-function-declaration]
Do you know how to do that? I check the manual here (https://www.openssl.org/docs/man1.0.2/man3/) but it seems that there isn't a properly function to make this. Further, I need to be compliant to OpenSSL 1.0.0.
Code
#include <stdio.h>
#include <stdlib.h>
#include <openssl/rsa.h>
#include <openssl/obj_mac.h>
#include <openssl/rand.h>
#include <openssl/bn.h>
#include <openssl/sha.h>
#include <openssl/evp.h>
#include <openssl/err.h>
#include <openssl/pem.h>

int main()
{
    OpenSSL_add_all_algorithms();
    ERR_load_crypto_strings();

    RSA *privkey = RSA_new();
    FILE *privkeyfile = fopen("private.key", "rb");

    PEM_read_RSAPrivateKey(privkeyfile, &privkey, NULL, NULL);
    fclose(privkeyfile);

    BIGNUM *n, *e, *d = NULL;
    RSA_get0_key(privkey,&n,&e,&d);

    return 0;
}

Comment: Hmm, line 217 of [rsa.h](https://github.com/openssl/openssl/blob/56784203ec2e4c8d94fccb25b956e21331b800b1/include/openssl/rsa.h) says this function is deprecated.
Comment: FWIW, in OpenSSL < 1.1.0, almost all of the structures are open and not opaque, so you can dive right in and access just about any field you want.
Comment: Show us your code.
Comment: @RobertHarvey question updated.
Answer: The RSA_get0_key function was added in OpenSSL 1.1.0 as an abstraction to retrieve the n, e, and d values for an RSA key.  For earlier versions you need to access the fields directly.
n = privkey->n;
e = privkey->e;
d = privkey->d;

If you want your code to be able to handle 1.0.x and 1.1.x, you can check the value of OPENSSL_VERSION_NUMBER:
#if OPENSSL_VERSION_NUMBER >= 0x10100000L
    RSA_get0_key(privkey, &n, &e, &d);
#else
    n = privkey->n;
    e = privkey->e;
    d = privkey->d;
#endif

Comment: @RobertHarvey That's for migration to OpenSSL 3.0.0.  OP is using 1.0.x.
Comment: There's probably a better way to do it.  Read the [Migration Guide](https://www.openssl.org/docs/man3.0/man7/migration_guide.html); you can do a search on the page for `RSA_get0_key`.

OpenSSL 1.0.0 RSA parameters in C


loadbalancing with difference nginx location context and backend server context

Question: I used nginx and upstream module for load balancing with the following config
upstream lb {
  server 127.0.0.1:8080;
  server 127.0.0.1:8081;
 }
 server {
  listen 88;
  server_name localhost;

  location /cas/ {
    proxy_pass              http://lb;
    proxy_redirect          off;
    proxy_connect_timeout   2;
    proxy_set_header        Host            $host;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
  }
}

the problem is the "location /context/" have to match to the context of backend server so when i request localhost/context/index.html then nginx routes it to 127.0.0.1:8080/context/index.html or 127.0.0.1:8080/context/index.html.
Is it possible to have difference backend context and nginx location for example with "location /" nginx will routes the request to 127.0.0.1:8080/context/index.html or 127.0.0.1:8080/context/index.html
Thank you.
Answer: You can have multiple location statements each with it's own upstream
location /cas/ {
  proxy_pass              http://lb;
  }

location /web/ {
    proxy_pass              http://2b;
  }

location /mail/ {
    proxy_pass              http://3b;
  }

You can also use if statements that checks the $request_uri variable and uses that to decide what upstream to pass the request to.
if ($request_uri ~* "^/(.+?)/context")
{
  proxy_pass   http://domain.com$request_uri;
  break;
}

Comment: actually it's a bad idea to put proxy_pass inside if {}


How to find the date difference in SQL Server?

Question: I have a passed_date column in table , I wants to find the difference between this date and current date, And have to check whether this difference is >365days?  for example 'passed_date' column consists   09/26/2013 09:16:48.000 PM . How can I find the difference between the date ?
sqlcommand cmd = new sqlcommand("select passed_date from table");
cmd.executereader();
dr.read();
sring date1 = stdr[0].tostring();
string date2 = DateTime.Now.ToString();

Comment: Use TimeSpan in C# or DATADIFF in SQL Server
Comment: What did you try? You are probably not the first person trying to solve this problem, did you google?
Answer: This uses index:
Select passed_date from table where passed_date > Dateadd(day,-365,getdate())

Answer: This works:
select passed_date from table where DateDiff(day,passed_date,GetDate()) > 365

Comment: This will cause slow performance. @AmirrezaKeshavarz posted a better answer
Answer: try this ...
declare @startDate datetime
set @startdate = DATEADD(day, -365, GETDATE())
-- or set @startDate = DATEADD(year, -1, GETDATE())
select passed_date, datediff(day, passed_date, getdate()) from table where passed_date > @startDate

... if you want a solution in t-sql
Answer: Try this code :-
SqlCommand sqlcmd = new SqlCommand("select passed_date from table", con);
var date = sqlcmd.ExecuteScalar();
DateTime startDate = (DateTime)date;
DateTime endDate = DateTime.Now;
TimeSpan span = endDate.Subtract(startDate);

double days = span.TotalDays;
if (365 < days)
{
    ********your logic ************************
}


Presentation on Parabola

Question: All right, I'm presenting a class in front of some $20$ or so $12^\text{th}$ grade students in a couple of days on the topic of Parabola. I was wondering what would be the best way to teach it to them, and also if anyone had some really amazing problems as I have to give them $2$ insanely hard problems\ or just problems with a really great solution or analysis to be done in the class. Also if anyone had their hands on any great lecture notes from college I would be very grateful since we have been instructed to not restrict our self to any sort of level of difficulty.
Any ideas would be welcome.
Comment: What about https://www2.bc.edu/mark-reeder/1103quadparab.pdf?
Comment: @MichaelHoppe So from the little bit I have read, it sort of seems that the paper explores all the properties of the parabola to prove the statement given at the beginning by Archimedes. If so I think that is a great idea for the way I could structure the lesson. Thank you so much
Comment: I once uses to teach my students "constructing tangent and perpendicular to a parabola" as well "constructing focus of a given parabola" and so on. Most of them enjoyed it.
Comment: That seems like a good idea, but since the class is virtually based that might be a little bit difficult logistically
Answer: Too long for a comment ...  One special Archimedean Lemma I've used for a lesson: Let $f\colon x\mapsto ax^2$ for positive $a$ and let $P(p_1,p_2)$ a point with $p_2<0$.  The two tangents from $P$ to $f$ touch $f$ in $A_1$ and $A_2$, resp., midpoint of $A_1A_2$ is $M$.  Now show that the midpoint $S$ of $MP$ is a point of $f$.

Use this fact to cleverly construct the tangent in any point $A$ of $f$.
Answer: I never taught this subject, but if I were to I would start from the definition as section of a circular cone,
and their projective properties, invariants, and pass after to the metric properties. So I would start with
 - how is it projected by the cone on a plane normal / parallel to its axis;
 - shifting the sectioning plane, what the resulting parabolas and their projections have in common;
 - same,  keeping instead firm the sectioning plane and widening the angle of the cone;
 - consider the normals to the cone in the plane of the parabola, where do they converge in the projection
onto a plane normal to the cone axis, etc.
 - considering the cone double, and two symmetric parabolas on the two foils, where do their planes cross, what
is the meaning of that line;
etc.
Concerning an interesting exercise, I would propose that of reproducing Archimedes steps in computing the area of
a parabolic segment, centuries before calculus.

Presentation on Parabola


omreport missing after CentOS 5.4 yum update

Question: I just ran yum -y update and now I get
omreport: command not found

uname -a Linux smtp.alexx.net 2.6.18-164.6.1.el5 #1 SMP Tue Nov 3
  16:18:27 EST 2009 i686 i686 i386 GNU/Linux
Answer: found it
ln -s /opt/dell/srvadmin/bin/omreport /usr/local/sbin/omreport


Cisco Catalyst 3550 Switch - VLAN ACL question

Question: I can't seem to get Vlan ACLs to work properly. I want to be able to get to servers on the 10.100.x.x network as well as the 10.200.x.x network when I'm connected through the VPN. But, I want to prevent the 10.100.x.x network from seeing 10.200.x.x
Currently, I have 2 Vlans:
interface Vlan111
 description "vlan 111"
 ip address 10.100.1.1 255.255.255.0

interface Vlan222
 description "vlan 222"
 ip address 10.200.1.1 255.255.255.0
 ip access-group vlan222_acl in
 ip access-group vlan222_acl out

And I have 2 servers connected to each VLAN:
!--- the below port is connected to a windows machine with static ip address 10.100.1.10 and gateway 10.100.1.1
interface FastEthernet0/4
 description server 1
 switchport access vlan 111
 switchport mode access

!--- the below port is connected to a windows machine with static ip address 10.200.1.10 and gateway 10.200.1.1
interface FastEthernet0/5
 description server 2
 switchport access vlan 222
 switchport mode access

The problem is when I configure vlan222_acl.
Here is what I enter:
ip access-list extended vlan222_acl
 deny icmp 10.100.0.0 255.255.0.0 10.200.0.0 255.255.0.0

But here is what gets applied to the running config:
ip access-list extended vlan222_acl
 deny icmp 0.0.0.0 255.255.0.0 0.0.0.0 255.255.0.0
!--- the above line is not what I actually entered in the terminal, for some reason
!--- the source and dest ip addresses are getting replaced with zeros

Because of the above line, everyone now is unable to ping the server at 10.200.1.10. When I remove the line, ie
no deny icmp 0.0.0.0 255.255.0.0 0.0.0.0 255.255.0.0
then I can ping it again.
How can I restrict only 10.100.x.x from being able to ping 10.200.x.x?
Many thanks in advance for the help!
G
Answer: you need to reverse your netmasks...i.e.,
instead of 255.255.0.0 use 0.0.255.255
Example:
ip access-list standard external_traffic
 deny   172.16.0.0 0.15.255.255
 deny   192.168.0.0 0.0.255.255

Additionally, you do understand that you need a router between VLAN'd subnets, right? (not trying to sound snotty, I'm just not sure of your experience level)
edit:
Is there a reason you're using /24 subnets for address assignment, but /16's for your ACL?
My suggestion would be to toss out the ACL on 222, and put restrictions on the 111 subnet, since that's what you're actually wanting to restrict anyway.
ip access-list extended block-icmp
deny icmp 10.100.1.0 0.0.0.255 10.200.1.0 0.0.0.255
permit ip any any
!
interface vlan 111
 ip access-group block-icmp in

Comment: In all access-lists you need an allow rule at the end of the list if you want to access *anything*. By default a Cisco ACL is implicit deny.
@Greeblesnort: The Cisco 3550 is layer 3 capable and can route between VLANs.
Comment: Unfortunately, this doesn't work either: what ends up happening is everyone gets blocked when I apply the deny rule, then everyone gets access again once I apply the permit any any command :S
Comment: @HampusLi - I should have known that, as I've got a 3550 floating around somewhere in my datacenter, even though it's only doing L2.
Comment: Thanks very much for your reply.

I tried the following:

deny icmp 10.100.0.0 0.0.255.255 10.200.0.0 0.0.255.255

which seemed to effectively block the 10.100.1.10 host from getting to 10.200.1.10, but it also ended up blocking everyone else from getting to it (same as my initial problem).

I.E: I also tried pinging from a host on a different network (192.168.x.x), and before adding the above command, I was able to ping and get to 10.200.x.x, but after adding it, I'm no longer able to.
Answer: You have applied the ACL to VLAN 222 for both incoming and outgoing traffic and the ACLs are not stateful. This means that they won't allow return traffic by their own, you will have to add an implicit rule for that. So, in this case, you're blocking the return traffic.
One solution would be to add another rule for the return traffic on the same ACL but what I think would be best, is to have two different ACLs for incoming and outgoing traffic, if you really need them.
You can also try to use reflexive ACLs which is a good way to allow return traffic. Check here for more details: http://www.cisco.com/en/US/docs/ios/12_2/security/command/reference/srfreflx.html
Here are some additional tips that may help t-shooting.

show access-list NAME/NUMBER shows the number of hits for each rule show you can see which rules are matching and denying or allowing packets. This wouldn't help much in your case since you have the same ACL on both directions and you wouldn't know where it matched.
If you add log at the end of each rule (i.e. deny icmp 10.100.1.0 0.0.0.255 10.200.1.0 0.0.0.255 log) the switch will log each match which the you can revies with show logging This is CPU intensive though so try not to leave it on for long, just for t-shooting.


Power-like functional equation

Question: I would like to know what are the especifications of a functional equation that give us a power function as a solution.
For example, if $f:\Bbb R \to \Bbb R$ is continuous and monotonic, such that $$f(x)+f(y)=f(z)$$ iif $$f(\lambda x)+f(\lambda y)=f(\lambda z)$$ for all $\lambda>0 $, then $f(x)=ax^b$.
Does anyone know another functional equation that gives a power function as a solution?
Comment: How is $z$ related to $x$ and $y$ in your equation?
Comment: Continuous functions satisfying $f(xy)=f(x)f(y)$ are power functions.
Comment: @mihaild OP is talking about functions of the type $x^{b}$ and you are talking about $b^{x}$.
Comment: @Arnaldo Yes, the only continuous solutions are power functions.
Comment: @KaviRamaMurthy: there is no relation. They can be any real number.
Comment: @KaviRamaMurthy: on that case, the power function is the only one solution?
Answer: For all $\ x\ $ and $\ y\ne 0,\ $ the only continuous solutions of the equation $\ f(x)^2 = f(xy)f(x/y)\ $ is
 $\ f(x) = ax^b.$
An alternative equation similar to
 Cauchy's functional equation is
$\ f(1)f(x\ y) = f(x)f(y)\ $ for all $x,y$.
Comment: It is right, by it is a particular case of Cauchy functional $f(xy)=f(x)f(y)$. Isn't it?
Answer: $$f(x+y)=f(x)+f(y)$$ gives a linear function and $$f(x+y)+f(0)=f(x)+f(y)$$ an affine one.
Then using logarithmic transformations,
$$f(xy)f(1)=f(x)f(y)$$ is your answer.

By the same reasoning, the exponential $ab^x$ is the solution of
$$f(x+y)f(0)=f(x)f(y),$$ and the logarithm $a+\log_bx$ that of
$$f(xy)+f(0)=f(x)+f(y).$$
Comment: I don't get how you go from $f(x+y)+f(0)=f(x)+f(y)$ into $f(xy)f(1)=f(x)f(y)$. The first functional equation shouldn't be $f(x+y)=f(x)f(y)$?
Comment: @Arnaldo: which first ?

Power-like functional equation


Need help to complete the proof about change of probability

Question: Got a question about change of probability. $P$ and $Q$ are two probability measures on the same space $(\Omega,\Lambda)$,and let $f=\dfrac{dQ}{dP} $ denote the Randon-Nikodym derivative of $Q$ respect to $P$. Show that
$$E_Q[X\mid\mathcal{G}]=\frac{E_P[Xf\mid\mathcal{G}]}{E_P[f\mid\mathcal{G}]}$$
I had no idea how to reach this conclusion.My logic is listed below
$$E_Q[X]=\int X\,dQ$$
$$E_P[Xf]=\int X\frac{dQ}{dP} \, dP=\int X\,dQ$$
which tells $E_Q[X\mid\mathcal{G}]=E_P[Xf\mid\mathcal{G}]$ which is consistent with the what need to prove above. Where did I go wrong? PLz help!!
Answer: Given $X$, the conditional expectation $\mathbb E^\mathbb Q[X\mid \mathcal G]$ is the $\mathcal G$-measurable function such that
$$\int_A Xd\mathbb Q=\int_A E^\mathbb Q[X\mid \mathcal G]d\mathbb Q,\quad\forall A\in\mathcal G.$$
Hence, you must prove that
$$\int_A Xd\mathbb Q=\int_A \frac{E^\mathbb P[Xf\mid \mathcal G]}{E^\mathbb P[f\mid \mathcal G]}d\mathbb Q,\quad\forall A\in\mathcal G.$$
Hint: If you define $\tilde{\mathbb P}=\left.\mathbb P\right|_\mathcal G$, is easy to show that
$\mathbb E^\mathbb P[f\mid \mathcal G]=\dfrac{d\tilde{\mathbb P}}{d\tilde{\mathbb Q}}.$


Feeds import and related fields

Question: I have a content type with an entity reference field (I suppose a term reference field would work as well).  The target type is a taxonomy term.  The taxonomy term has a name and ID field (among others).  My feeds source that I would like to import has the taxonomy ids that I can import but I'd like to display the name not the id.  My problem is I can't get the feeds map correctly unless I have a list of names in my feeds source, not ids.
Thanks
Sharon
Answer: I've figured it out.  If I import the ID field as GUID when adding the taxomony terms, then I can select that as an option when I import into my content type :)

Feeds import and related fields


Running queries in SQL while big stored-procedure is executing

Question: I have a situation where I need to run a stored procedure (>5mins) and want my (silverlight) application to remain 'alive'
This I do with an async WCF Service for my UI thread to be responsive, at least until I request some data from the same DB again
Then, only after SP is finished the other request are executed
Can I somehow (in SQL I suppose) set the storedprocedure to be able to run alongside with other queries?
Thanks
edit:

DB is on SQL Server
starting SP from Linq2Sql model in <code>[OperationContract(AsyncPattern = true)]</code> service method
other data is handled with WCF Ria Services (DomainService from Linq2SQL model, same connection string)

edit 2:
@dan1111: With testing I saw that until the point when WCF service returns result to Silverlight, DomainContext 'freezes'
Initially:

I call WCF Async method with Thread.Sleep(15000)
Application continues to work good
When I go to a page where data is needed, I request DataContext.Load()
DataContext.IsLoading becomes true
I must wait the remaining of the 15seconds
Now the completed event is fired for the previous DataContext.Load

Please tell me your opinion
Comment: Which DBMS are you using? PostgreSQL? Oracle? DB2? Firebird?
Comment: Execute the stored procedure using another connection than your 'alive' queries.
Answer: A very simple solution is to open more than one connection to the database.  Each connection can do its own stuff.
However, there is another way to do it that actually sets up a procedure to run asynchronously:
http://rusanu.com/2009/08/05/asynchronous-procedure-execution/
Comment: @dvox, I can't tell from the information you have provided whether it is a code issue or a database issue.  The second connection needs to be used in a separate thread in your code; i.e. it needs to be running asynchronously.
Comment: what do you mean by 'separate thread'?
On client side (silverlight): call to the service starts in a separate thread
On server side: The connection is created inside the service method
Comment: @dvox, you will need two threads on the server side--one for each connection to the database.


Regex in file names

Question: I use free little program Metamorphose for changing file names.
Problem is I need to use regex to change names in order as shown below:
<code>Find: nice-tree-([\s\S]*?)
Replace: nice-tree-$1-abc
</code>
As you can see all files that start with nice-tree-ANYTHING should be replaced with -abc at the end of every file name.
I'm everything just not expert for regex usage...
Both of you were right. It works now. Thanks.
Comment: What's your question?
Comment: (.*) would match `ANYTHING` so that any `SOMETHING` would get matched at that point, or did I understand you wrong?
Comment: do you consider file extensions in the file name?
Comment: Get rid of the `?` after the `*`. You don't want minimal matching here. And just use `.` instead of `\s\S` (or `\d\D`, `\w\W`, ...).
Answer: Assuming you don't need to replace the full file name with the extension. The following regex would match all the file names that you are looking for.
/nice-tree-.+/

Regex in file names


Headers and Selenium Webdriver 2

Question: Is there any way to add a header in Selenium WebDriver test? (Like with Firefox Modify Headers plugin) I cannot use HtmlUnitDriver, because the browser has to be visible.
Comment: For those people using Python to author their Selenium tests, you might consider [Selenium Wire](https://github.com/wkeeling/selenium-wire) which gives you the ability to add headers sent by the browser, as well as inspect requests and responses.
Answer: WebDriver does not allow you to change or set the headers using any of the browser based drivers. You can find a lot of information about their decision about the headers and the response codes at the following URL.
http://code.google.com/p/selenium/issues/detail?id=141
We use Apache HTTP Client for this type of testing where we do not want to check the rendered page elements, but just the responses and header information.
You can also give browser mob proxy with your selenium tests as well as mentioned in the url above. I have used this for other purposes and it is awesome.
Answer: Here is a short example of how to use Seleniumwire with Python:
from seleniumwire import webdriver

def set_chrome_driver():
        options = webdriver.ChromeOptions()
        options.add_argument("--start-maximized")
        options.add_argument("--disable-infobars")
        options.add_argument("--no-proxy-server")
        driver = webdriver.Chrome(executable_path=r'C:\Automation_package\chromedriver.exe')
        driver.get('http://220.127.116.11:5000/path/api/')
        driver.header_overrides = {"iv-user": "Admin", "iv-groups": "SuperAdmin", "iv-roles": "Viewers",}
        driver.get('http://126.96.36.199:5000/path/api/')

Answer: You can achieve that by making use of a remote-contolled extension using the chrome.declarativeNetRequest api.
As it doesn't use any proxies or IO, the network speed won't reduce noticeably.
Selenium-Injector provides an implementation for that.
Install with:
pip install selenium_injector

Example script:
from selenium_injector.webdriver import Chrome

driver = Chrome()

# modify headers
driver.injector.declarativeNetRequest.update_headers({"test": "test_2", "sec-ch-ua-platform": "Android"})

driver.get("https://httpbin.org/headers")

input("press ENTER to exit")
driver.quit()

Disclaimer: I am the author of Selenium-Injector
Answer: There are alternative methods to do this like Browsermob-Proxy
Since Browsermob-proxy has its own limitations while we work on the selenium grid, the below is how I fixed the problem in my case. Hopefully, might be helpful for anyone with a similar setup.

Add the ModHeader extension to the chrome browser

How to download the Modheader? Link
ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));

// Set the Desired capabilities
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);

// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);

Go to the browser extensions and capture the Local Storage context ID of the ModHeader

Navigate to the URL of the ModHeader to set the Local Storage Context

.
// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");

Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2

Now add the headers to the request using Javascript

.
   ((Javascript)driver).executeScript(
         "localStorage.setItem('profiles', JSON.stringify([{  title: 'Selenium', hideComment: true, appendMode: '',
             headers: [
               {enabled: true, name: 'token-1', value: 'value-1', comment: ''},
               {enabled: true, name: 'token-2', value: 'value-2', comment: ''}
             ],
             respHeaders: [],
             filters: []
          }]));");

Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.

Now navigate to the required web-application.
driver.get("your-desired-website");

Headers and Selenium Webdriver 2


Lighttpd rewrite rule for Flight PHP microframework

Question: I'm using the Flight PHP microframework, and it includes its' own router, and needs the server to rewrite requests to the index.php file.
It features rewrite-rules for both Apache and Nginx but unlucky me, I'm using lighttpd in production, and I'm not really interested in changing that.
So, I hate to be that guy, who just says: Give me the solution.. But seriously, somebody help me, I have no idea how to rewrite Lighttpd requests.
The Apache config is like this:
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php [QSA,L]

And the Nginx config like this:
server {
    location / {
        try_files $uri $uri/ /index.php;
    }
}

My own Lighttpd config currently looks like this:
$HTTP["host"]  == "zyx.abc.dk"{
        server.document-root = "/var/www/app1/zyx"
        accesslog.filename         = "/var/log/lighttpd/zyx-log.log"
        url.rewrite-once = ( "(.*)" => "/index.php" )
}

But it doesn't work. How can I make the subdomain be rewritten like the above, to make Flight PHP work? I like the framework, so I would be quite sad for having to change it.
Thanks guys! :)
Answer:
your rewrite rule didn't do "QSA" - it deleted the query string
your rewrite did trigger for existing files too (not matching !-f and !-d)

The following should keep the query string and only trigger if the requested path is not  normal real file (but triggers for directories, which doesn't match the apache !-d - although I'd argue that !-d is probably a bug in the apache config).
url.rewrite-if-not-file = ( "(\?(.*))?" => "/index.php$1" )

You can also use an anchored regular expression:
url.rewrite-if-not-file = ( "^.*(\?(.*))?$" => "/index.php$1" )


some php to ASP.NET

Question: anyone kind enough to transcribe this php to ASP.NET ??
thank you kindly in advance.
jJ
<?php
if(isset($_POST['data']) && $_POST['data'] != '') {
 $output = $_POST['data'];
 $newfile = time() . ".xml";
 $file = fopen ($newfile, "w");
 fwrite($file, $output);
 fclose ($file);
 echo 'file created: ' . $newfile;
} else {
 echo "not saved";
}
?>

Comment: You are not asking a question, just asking for free work.
Comment: you will need to define a language, .Net is framework, c# or VB.net are the main languages
Answer: Untested code but enough to get you started
if(!string.IsNullOrEmpty(Response.Form["data"]))
{
    string output = Response.Form["data"];
    string newfile = DateTime.Now.ToString("hhMMss")+".xml";
    File.WriteAllText(newfile, output);
    Response.Write("file created: " + newfile);
}
else
{
    Response.Write("not saved");
}

This was how I learned PHP so I felt obligated to pay back :)
Edit: As everyone else has said, you should of course don't use this as-is but instead use it for learning.
Answer: As far as I can see, this is not directly translateable to one chunk of ASP.NET-code as you'd probably want.
You need an .aspx-file, and you need the code behind (.aspx.cs-file) to do it in the most basic way. Opening, writing and closing a text file could be done like this. The rest relies on specific ASP.NET-techniques like using a <asp:whatever id="data"> element and then retrieving this code behind.
You're asking for a fairly simple task, and you would probably be better off if you took some time to look at ASP.NET yourself.
Edit:
@Jesper Palm did more or less what I said was not possible in my first line. Weird use of the ASP.NET framework anyways, in my opinion. :-)

some php to ASP.NET


Is $\inf S=\lim \inf_{n \to \infty}s_n=1 ?$ Yes/No

Question: Is the following statement true/false ?

let $p_n$  denote  the $n-$th prime  number, when we  enumerate  the prime numbers in  the increasing order.For  exampe  ,$p_1=2 ,p_2 =3 , p_3=5 $ and  so on.Let  $S= \{s_n=p_{n+1}-p_n| n \in \mathbb{N} , n \geq 1  \}$.Then  $\lim \inf_{n \to \infty} s_n \ge 2$

My attempt : I think this statement  is false because $\lim \inf_{n \to \infty}s_n=1$
Here $p_1 =2 , p_2=3, p_3=5 ,p_4 =7 ,p_5=11 , p_6=13$
$\implies  S_1=p_2-p_1=3-2=1$
$S_2=p_3-p_2=5-3=2$
$S_3=p_4-p_3 =7-5=2$
$S_4=p_5-p_4=11-7=4$
.....
Therefore $S=\{1,2,2,4,2,....\}$
$\sup S =\infty$ and $\inf S=1$
I think $\inf S=\lim \inf_{n \to \infty}s_n=1$
Am i right ?
Answer: You are not right.
Formally for a sequence $(a_n)_n$ we have:$$\liminf_{n\to\infty}a_n=\lim_{n\to\infty} b_n\text{ where }b_n=\inf\{b_k\mid k\geq n\}$$
Here $(b_n)_n$ is an increasing sequence so that: $$\liminf_{n\to\infty} a_n\geq b_n\text{ for every }n$$
Consequently for every fixed positive integer $n$ the liminf does not depend on the values $a_1,\dots,a_n$.
You can also think of the liminf as the smallest value such that a subsequence $(a_{n_k})$ exists that converges to it provided that $(a_n)$ is a bounded sequence.


Problem with auto-generated url alias when attempting to make a third-level menu item

Question: Using [node:menu-link:parents]/[node:title] as the pattern for nodes in Url Aliases works fine for a menu that is only two levels, but if I attempt to assign a node to a third level under a menu, instead of getting top-level-node/2nd-level-node/third-level-node as the url alias, I get top-level-node-2nd-level-node/third-level-node as the alias. Is this the expected behavior? Is there a way to auto-generate top-level-node/2nd-level-node/third-level-node when creating a menu item at the third level?
Answer: Try:
<code>/[node:menu-link:parent:url:path]/[node:menu-link:title]</code>
Comment: That did the trick. Thanks so much!


How to wrap long lines for olcAccess?

Question: I tried to define the access permissions by adding a space at the beginning of a new line. But I get an syntax error, I do not understand:
<code># ldapmodify -Y EXTERNAL -Q -H ldapi:///
dn: olcDatabase={2}hdb,cn=config
add: olcAccess
olcAccess: to *
 by dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage
 by self write
 by anonymous auth
 by * read

modifying entry "olcDatabase={2}hdb,cn=config"
ldap_modify: Other (e.g., implementation specific) error (80)
        additional info: <olcAccess> handler exited with 1
</code>
If I remove the line break and put everything in one line, It works:
<code># ldapmodify -Y EXTERNAL -Q -H ldapi:///
dn: olcDatabase={2}hdb,cn=config
add: olcAccess
olcAccess: to * by dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by self write by anonymous auth by * read

modifying entry "olcDatabase={2}hdb,cn=config"
</code>
What is wrong with the first version?
The daemon logs:
<code>Feb 28 14:33:59 k3ls slapd[21199]: slapd: line 0: missing "=" in "*by" in to clause
Feb 28 14:33:59 k3ls slapd[21199]: <access clause> ::= access to <what> [ by <who> [ <access> ] [ <control> ] ]+
                                   <what> ::= * | dn[.<dnstyle>=<DN>] [filter=<filter>] [attrs=<attrspec>]
                                   <attrspec> ::= <attrname> [val[/<matchingRule>][.<attrstyle>]=<value>] | <attrlist>
                                   <attrlist> ::= <attr> [ , <attrlist> ]
                                   <attr> ::= <attrname> | @<objectClass> | !<objectClass> | entry | children
                                   <who> ::= [ * | anonymous | users | self | dn[.<dnstyle>]=<DN> ]
                                           [ realanonymous | realusers | realself | realdn[.<dnstyle>]=<DN> ]
                                           [dnattr=<attrname>]
                                           [realdnattr=<attrname>]
                                           [group[/<objectclass>[/<attrname>]][.<style>]=<group>]
                                           [peername[.<peernamestyle>]=<peer>] [sockname[.<style>]=<name>]
                                           [domain[.<domainstyle>]=<domain>] [sockurl[.<style>]=<url>]
                                           [dynacl/<name>[/<options>][.<dynstyle>][=<pattern>]]
                                           [ssf=<n>] [transport_ssf=<n>] [tls_ssf=<n>] [sasl_ssf=<n>]
                                   <style> ::= exact | regex | base(Object)
                                   <dnstyle> ::= base(Object) | one(level) | sub(tree) | children | exact | regex
                                   <attrstyle> ::= exact | regex | base(Object) | one(level) | sub(tree) | children
                                   <peernamestyle> ::= exact | regex | ip | ipv6 | path
                                   <domainstyle> ::= exact | regex | base(Object) | sub(tree)
                                   <access> ::= [[real]self]{<level>|<priv>}
                                   <level> ::= none|disclose|auth|compare|search|read|{write|add|delete}|manage
                                   <priv> ::= {=|+|-}{0|d|x|c|s|r|{w|a|z}|m}+
                                   <control> ::= [ stop | continue | break ]
                                   dynacl:
                                           <name>=ACI        <pattern>=<attrname>
</code>
I do not understand where the assignment is missing?
Answer: OpenLDAP deletes the first white space of a continuation line and joins all lines. So, e.g., the last three lines are seen by OpenLDAP as:
<code> by self writeby anonymous authby * read
</code>
which is syntactically invalid. Using 2 spaces of indentation will solve your problem.

How to wrap long lines for olcAccess?


O que significam esses argumentos no comando de "git-checkout"?

Question: Tenho o seguinte comando no Git para recuperar um arquivo:
git checkout (hash-commit)^  --(caminho-arquivo)

Nesse comando o que significa o ^ e --?
Answer: O ^ indica que você quer o commit um nível acima deste especificado. Ele é o mesmo que ^1, se você quisesse 2 níveis acima usaria ^2.
O -- indica que você vai passar um subcomando, um argumento do que fará com o comando principal.
Coloquei no GitHub para referência futura.
Comment: Muito bem explicado. Obrigado !