'Variável' não inicializada usada nessa função?

Question: Após a compilação do programa, recebe 2 warnings do mesmo tipo:

"warning: 'frase' is used uninitialized in this function"

e

"warning: 'palavra' is used uninitialized in this function"

#include<stdio.h>
#include<stdlib.h>
#include<string.h>

char *alocar_palavra(char *palavra);
char *alocar_frase(char *frase);

int main(){

  char *frase, *palavra;
  frase = alocar_frase(frase);
  palavra = alocar_palavra(palavra);
  printf("%s", frase);
  printf("%s", palavra);

  return 0;
}

char *alocar_palavra(char *palavra){

  palavra = (char*)malloc(20 * sizeof(char));
  scanf(" %s", palavra);

  return palavra;
}

char *alocar_frase(char *frase){

  char letra;
  int k = 0;
  frase = NULL;

  do{
      letra = getchar();
      frase = (char*)realloc(frase, (k+1)*sizeof(char));
      frase[k] = letra;
      k++;
  }while((letra != '$') && (letra != '\n'));

  return frase;
}

Comment: `realloc` so pode ser chamado depois de `malloc`, `calloc`, ou `realloc` como indica a [documentação](http://pt.cppreference.com/w/c/memory/realloc), caso contrário pode obter resultados indeterminados
Answer: Não tem porque passar as variáveis se logo depois irá descartar seus valores, quando está fazendo isso sem valor causa esse problema.
O código pode ser bem melhorado, perceba tudo o que eu fiz. Pra falar a verdade, essas variáveis do main() nem são necessárias.
#include<stdio.h>
#include<stdlib.h>

char *alocar_palavra() {
    char *palavra = malloc(20);
    scanf(" %s", palavra);
    return palavra;
}

char *alocar_frase() {
    char letra;
    int k = 0;
    char *frase = NULL;
    do {
        letra = getchar();
        frase = realloc(frase, k + 1);
        frase[k++] = letra;
    } while (letra != '$' && letra != '\n');
    return frase;
}

int main() {
    char *frase = alocar_frase();
    char *palavra = alocar_palavra();
    printf("%s", frase);
    printf("%s", palavra);
}

Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura.


Correlation between fitted and observed values in linear model

Question: This is about linear model. How do I show/calculate the Correlation between the fitted values and observed values?
Answer: It depends on the response variable. If the response variable is continuous, try the paired T-test; if the response variable is categorical, try the chi square test in the contingency table.
However, I don't understand your purpose to calculate the correlation between fitted values and observed values. The correlation must be strong if the model you applied is kind of appropriate for the data. If you simply want to check if the model fitting is good, there are goodness-of-fit tests, like R square and lack-of-fit sum square.
Comment: The correlation between observed and fitted values is a square root of $R^2$.  (The question does not appear to ask for recommendations of tests.)
Comment: Yes, the square of the correlation is $R^2$. Thanks for pointing out, whuber.


Sub-net Routing

Question: Applications hosted on servers located in the same subnet cannot reference domain names (VIP) hosted on the load balancer for the subnet.
Assumption: server A and server B are hosted in the same subnet.
Application hosted on Server A is connecting to the VIP (which references an application hosted in Server B).
Connection initiation will go from Server A, via the VIP, to Server B; however, the response comes back directly from Server B to Server A.
Server A will not accept the response as it had sent the original request to the VIP only.
Question: Is this normal occurrence in network set-up? The suggested solution was to create a special internal-to-subnet IP for Server A to call app on Server B: I do not like this solution as it can lead to proliferation of internal IPs that can be cumbersome to manage.
The other option is to sprout subnets....
Any thoughts or suggestions?
Comment: vip = virtual ip ? the published ip of the load balancer ?
Comment: Is server A gets a response from server B or does Server B initiate a new call as a respones ?
Comment: Where is your programming-related question in all that?
Answer: You're doing DSR (direct server return). For it to work properly, the load balancer needs to be aware of that, so that it only modifies the destination MAC address, but leaves both IP addresses in the packet intact. Also, the VIP needs to be configured on a loopback interface of all servers behind the load balancer, so that the incoming packets with VIP in the destination field are accepted. Whatever service lives there will also need to be configured to use VIP, unless it binds to 0.0.0.0. Once you have those pieces in place, everything will work, like this:

A sends a packet with its own IP and MAC in the source, VIP and LB's MAC in the dest.
LB receives the packet, rewrites dst MAC from its own to the MAC of one of the balanced servers and sends it over to B.
B receives the packet, sees valid dst IP (since it's VIP, and it has VIP on the loopback), kicks it down to the service to handle.
The service sends the answer, with VIP and server MAC now as the source, and A's IP and MAC as the dest.
The packet arrives at A, which recognizes it as a valid response to the original request.

BTW, your tags are totally wrong, since there's no routing involved. It's mostly Layer 2 tricks, with some configuration modification on Layer 3.
Reference: LBWiki

Sub-net Routing


How to trigger a function at regular intervals of time using hooks and when certain criteria is met I want to clear the time interval?

Question: I have a react component that performs a certain task at regular intervals of time after mounting. But I want to clear the interval once after a criterion is met. How can I achieve that?
My code
const [totalTime, setTotalTime] = React.useState(10000);

const foo = () => {
      console.log("Here");
  };

React.useEffect(() => {
    const secondInterval = setInterval(() => {
      if (totalTime > 0) setTotalTime(totalTime - 1000);
    }, 1000);
    return () => clearInterval(secondInterval);
  });

React.useEffect(() => {
    let originalInterval;
    if (totalTime > 0)
      originalInterval = setInterval(() => {
        foo();
        console.log(totalTime);
      }, 5000);
    return () => clearInterval(originalInterval);
  }, []);

When I watch the console even after 10000ms It is still logging Here and also totalTime is always being 10000ms. I am not able to figure out what exactly is happening.
Comment: Please don't remove code from questions when answers are present that require the code for comparison or reference.
Answer: You may need to pass the older state as an argument to the setTotalTime updater function. You also may need to pass (another) state variable as a dependency  to the useEffect hook so that the function is executed every time the state variable changes
React.useEffect(() => {
    const secondInterval = setInterval(() => {
      if (totalTime > 0) setTotalTime(totalTime => totalTime - 1000);
    }, 1000);
    return () => clearInterval(secondInterval);
  }, [...]);

Answer: Depends on your criteria, and what you call a criteria, but you could just use another state and then useEffect on that another state:
const [criteriaIsMet,setCriteriaIsMet] = useState(false);
useEffect(() => { if(criteriaIsMet) {clearInterval(...)} },[criteriaIsMet])

And then somewhere when you do your actual "criteria logic" you just go ahead and do setCriteriaIsMet(true)
And how would you know Id of interval in above useEffect - again you could just create a special state that will store that id, and you could set it in your original useEffect
As for why your current code is not clearing the interval is because when you use useEffect with empty array as second argument, and return function in first function argument - that will be execute on component unmounting
And these are just one of numerous options you have actually :)


Mathematical ideas that took long to define rigorously

Question: It often happens in mathematics that the answer to a problem is "known" long before anybody knows how to prove it.  (Some examples of contemporary interest are among the Millennium Prize problems: E.g. Yang-Mills existence is widely believed to be true based on ideas from physics, and the Riemann hypothesis is widely believed to be true because it would be an awful shame if it wasn't.  Another good example is Schramm–Loewner evolution, where again the answer was anticipated by ideas from physics.)
More rare are the instances where an abstract mathematical "idea" floats around for many years before even a rigorous definition or interpretation can be developed to describe the idea.  An example of this is umbral calculus, where a mysterious technique for proving properties of certain sequences existed for over a century before anybody understood why the technique worked, in a rigorous way.
I find these instances of mathematical ideas without rigorous interpretation fascinating, because they seem to often lead to the development of radically new branches of mathematics$^1$.  What are further examples of this type?
I am mainly interested in historical examples, but contemporary ones (i.e. ideas which have yet to be rigorously formulated) are also welcome.

Footnote: I have some specific examples in mind that I will share as an answer, if nobody else does.
Comment: The notion of a function. See [history of the function concept](https://en.wikipedia.org/wiki/History_of_the_function_concept)
Comment: A corollary of this question is prompted by the observation that the 19th and 20th century seemed to do a lot of"cleaning up" of older mathematics by putting it on a rigorous foundation. Is it possible that in years to come, people will look back at 21st century math and scoff at our impreciseness? Or are we "done" tidying the house?
Comment: Sets were simply known as random aggregations of possibly no entities long before Zermelo and Fraenkel defined their modern meaning. Still, the old definition is good enough for math basics.
Comment: Ideas from what we now call generalised functions or distributions were around for a while before being formalised
Comment: Shouldn't this be community wiki?
Comment: Zero?  (It's too short even to be a comment, much less an answer)
Comment: I am not educated on the topic, so I'll leave this as a comment, but I think the field with one element $\mathbb{F} _{1}$ is an example in the making. The possibility has been discussed for decades, and still nobody can say what the "correct" definition should be.
Comment: Calculus. Co-invented by Newton and Leibniz, but the idea of what happens in the limit was unsatisfactory to Betrand Russell, hence *Principia Mathematica*.
Comment: I might have guessed **fractal** would be one such example. Perhaps someone better versed in its history as an "idea" before the formalized definition (in terms of Hausdorff dimension) can comment in this direction.
Comment: It's not my area, so I'd rather someone more knowledgeable give it as an answer, but I know [motives](https://en.wikipedia.org/wiki/Motive_(algebraic_geometry)) are a modern example. Only very recently have pure motives seen a formal definition, and mixed motives are still yet to be defined!
Comment: Generalised functions (Schwartz). Integration (Riemann).
Comment: I was astonished to learn that the Fundamental Theorem of Arithmetic is barely over 200 years old.
Comment: @AlwaysLearning: We are done, because we have reached the notion of an **algorithm**, which we have described in terms of **syntactic** manipulation of **finite symbol strings**. Prior to that, these were only vaguely known, and so firm foundation of mathematics was essentially impossible. After that, there is nothing more, because **every formal system** that can ever be conceivable by humans will have to be an algorithmically describable one. All that is left is to find out what are the theorems proven by what formal systems. In that sense mathematics is now 100% precise (in theory).
Comment: Possibly related: https://mathoverflow.net/questions/56677/
Comment: I suppose [Fermat's last theorem](https://en.wikipedia.org/wiki/Fermat%27s_Last_Theorem) falls into the first category and doesn't answer your queston, right?
Comment: @AlessandroCodenotti That is a perfect example.  I suggest posting it as an answer.
Comment: @EricDuminil Correct.  Fermat's Last Theorem was a well-formulated conjecture, and so not within the scope of this question.
Comment: Almost everything in mainstream mathematics. I think it's the reverse question which is nearly devoid of examples, especially if we consider that important ideas were developed to satisfy some specific need(s).
Comment: Fermat's Last Theorem
It was first conjectured in 1637. It was solved by Andrew Wiles and published finally and completely in 1995.
Comment: Assuming something is true solely because of assumed consequences of the converse is a fallacy.
Comment: Related: https://math.stackexchange.com/a/4431348/169085
Answer: The notion of probability has been in use since the middle ages or maybe before. But it took quite a while to formalize the probability theory and giving it a rigorous basis in the midst of 20th century. According to wikipedia:

There have been at least two successful attempts to formalize probability, namely the Kolmogorov formulation and the Cox formulation. In Kolmogorov's formulation, sets are interpreted as events and probability itself as a measure on a class of sets. In Cox's theorem, probability is taken as a primitive (that is, not further analyzed) and the emphasis is on constructing a consistent assignment of probability values to propositions. In both cases, the laws of probability are the same, except for technical details.
There are other methods for quantifying uncertainty, such as the Dempster–Shafer theory or possibility theory, but those are essentially different and not compatible with the laws of probability as usually understood.
Comment: In fact, I would argue that that effort is not over, even as of today; see https://en.wikipedia.org/wiki/German_tank_problem and the XKCD take on a similar problem, https://xkcd.com/1132/
Comment: @DomQ I think the examples you mentioned are not so relevant as [this one](https://en.wikipedia.org/wiki/Bertrand_paradox_(probability)#Recent_developments), especially the part mentioning: _Nicholas Shackel affirms that after more than a century the paradox remains unresolved_
Comment: It is definitely not over, there remains today considerable controversy about the philosophy (and hence the formalism) of probability--start with Wikipedia "Interpretations of probability," and then Google to see the many papers and book chapters about the frequentist vs. Bayesian and belief based (e.g., Dempster-Shafer) models.
Answer: Euclidean geometry. You think calculus was missing rigorous understanding? Non-Euclidean geometry? How about plain old Euclidean geometry itself? You see, even though Euclid's Elements invented rigorous mathematics, even though it pioneered the axiomatic method, even though for thousands of years it was the gold standard of logical reasoning - it wasn't actually rigorous.
The Elements is structured to seem as though it openly states its first principles (the infamous parallel postulate being one of them), and as though it proves all its propositions from those first principles. For the most part, it accomplishes the goal. In notable places, though, the proofs make use of unstated assumptions. Some proofs are blatant non-proofs: to prove side-angle-side (SAS) congruence of triangles, Euclid tells us to just "apply" one triangle to the other, moving them so that their vertices end up coinciding. There's no axiom about moving a figure onto another! Other proofs have more insidious omissions. In the diagram, does there exist any point where the circles intersect? It's "visually obvious", and Euclid assumes they intersect while proving Proposition 1, but the assumption does not follow from the axioms.

In general, the Elements pays little attention to issues of whether things really intersect in the places you'd expect them to, or whether a point is really between two other points, or whether a point really lies on one side of a line or the other, etc. We all "know" these concepts, but to avoid the trap of, say, a fake proof that all triangles are isosceles, a rigorous approach to geometry must address these concepts too.
It was not until the work of Pasch, Hilbert, and others in the late 1800s and early 1900s for truly rigorous systems of synthetic geometry to be developed, with the axiomatic definition of "betweenness" being a key new fundamental idea. Only then, millennia since the journey began, were the elements of Euclidean geometry truly accounted for.
Comment: Maybe instead of *Euclidean geometry*, it is the very idea of *rigor* that this is about
Comment: @PyRulez: $\mathbb{R}^2$, once constructed, formalizes *one model of* a Euclidean plane. It doesn't formalize what a Euclidean plane *is*. What if you declare, "I say a Euclidean plane is whatever is isomorphic to $\mathbb{R}^2$"? Then you still have to say what structure the "isomorphisms" have to preserve. The idea that needs formalization is the structure - the lines/circles/angles/etc. system that Euclid describes (but with gaps). "$\mathbb{R}^2$ is far from being the end of the story.
Comment: @JiK, for a long time, it seemed like it *was* a mistake.  Euclid's other axioms have a "fundamental" feel to them, but the parallel postulate seems like it should be provable from the others, and many efforts were made to develop such a proof.
Comment: Did the Cartesian plane formalise it?
Comment: Calling the parallel postulate *infamous* sounds like it was a mistake to include it.
Comment: @PyRulez: You may be interested in a recent conversation I had in the Logic room starting [here](https://chat.stackexchange.com/transcript/44058?m=42256549#42256549). Note that different mathematicians have had different axiomatizations of geometry, some of which fall to the incompleteness theorems, while others that don't naturally can't prove some basic 'geometrical facts'.
Comment: @PyRulez :  Descartes certainly didn't formalize it (actually the Cartesian plane goes back to the ancients (Ptolemy the astronomer used it), but Descartes pioneered using it to study equations through their graphs), because he didn't formalize $\mathbb{R}$.  The difficulties in formalizing $\mathbb{R}$, which were resolved around the same time, are much better known, so this is still a good answer, regardless of which was finished first.
Answer: Natural transformations are a "natural" example of this.  Mathematicians knew for a long time that certain maps--e.g. the canonical isomorphism between a finite-dimensional vector space and its double dual, or the identifications among the varied definitions of homology groups--were more special than others.  The desire to have a rigorous definition of "natural" in this context led Eilenberg and Mac Lane to develop category theory.  As Mac Lane allegedly put it:

"I didn't invent categories to study functors; I invented them to study natural transformations."
Comment: The way I heard it was that "I didn't invent functors to study categories; I invented them to study natural transformations", which makes more sense, since Functors are less important than the other two concepts.
Comment: Fun fact I learned in grad algebra: Functors form a category, the morphisms of which are natural transformations.
Answer: Following from the continuity example, in which the $\epsilon$-$\delta$ formulation eventually became ubiquitous, I submit the notion of the infinitesimal. It took until Robinson in the 1950s and early 60s before we had "the right construction" of infinitesimals via ultrapowers, in a way that made infinitesimal manipulation fully rigorous as a way of dealing with the reals. They were a very useful tool for centuries before then, with (e.g.) Cauchy using them regularly, attempting to formalise them but not succeeding, and with Leibniz's calculus being defined entirely in terms of infinitesimals.
Of course, there are other systems which contain infinitesimals - for example, the field of formal Laurent series, in which the variable may be viewed as an infinitesimal - but e.g. the infinitesimal $x$ doesn't have a square root in this system, so it's not ideal as a place in which to do analysis.
Comment: Patrick, the fact is that Cauchy never gave an epsilon-delta definition of continuity.  Cauchy defined continuity in terms of *infinitesimals*. Some historians have been engaged in a [quest for departed *quantifiers*](http://matstud.org.ua/texts/2017/47_2/115-144.pdf) in Cauchy which yielded meager results, because the thrust of Cauchy's approach to the calculus was via infinitesimals.
Comment: No, Cauchy's work is an outstanding confirmation of this principle.  Of course, Leibniz did it two centuries earlier already.
Comment: What was Cauchy's formulation exactly?
Comment: @MikhailKatz The $\epsilon$-$\delta$ formulation is what I understand by those words.
Comment: In that case I'll remove Cauchy's name from this :)
Comment: On the contrary you should mention Cauchy as one of the pioneers of the infinitesimal method.  Many publications on this subject can be found at my page on the history and philosophy of infinitesimals.
Comment: The point of my answer is to highlight that infinitesimals are something that everyone worked with, but we didn't really have a good rigorous foundation for them for a long time. Does Cauchy's work contradict that?
Answer: Sets.  As late as the early twentieth century, Bertrand Russell showed that one leading theory of them was self-contradictory, because it led to Russell's Paradox: Does the set of all sets that do not contain themselves, contain itself?  The accepted solution was ZF set theory.
Another example that jumps to mind is counting up:  Peano arithmetic was axiomatized in the nineteenth century (and has been considerably revised since).  Or algorithms.
Which raises the point, I guess, that we're still looking for the best foundation for mathematics itself.
Comment: I'd in fact argue that we still do not have a satisfactory axiomatization of "collection". ZFC sets don't do well, since there are 'classes'. MK set theory doesn't work, for the same reason. I prefer a type theory with a universal type, and it is possible, but something still has to be given up to avoid Russell's paradox. And you are right about 'algorithms'; I also mentioned that in my comment on the question, before I came to your answer. =)
Comment: I think the point of Russell's paradox is that there _is_ no satisfactory axiomatization of "collection" because the idea is self-contradictory, unless you severely limit what a collection is allowed to be.
Answer: Continuity is an example of a concept that was unclear for some time and also defined differently from what we now consider its "correct" definition. See this source (Israel Kleiner, Excursions in the History of Mathematics, pp. 142-3) for example.

In the eighteenth century, Euler did define a notion of "continuity"
  to distinguish between functions as analytic expressions and the new
  types of functions which emerged from the vibrating-string debate.
  Thus a continuous function was one given by a single analytic
  expression, while functions given by several analytic expressions or
  freely drawn curves were considered discontinuous. For example, to
  Euler the function
$$ f(x) = \left\{\begin{array}{ll} x^2 & x > 0 \\ x & x \leq 0 \end{array}\right. $$
was discontinuous, while the function comprising the two branches of a
  hyperbola was considered continuous (!) since it was given by the
  single analytic expression $f(x) = 1/x$.
[...]
In his important Cours d'Analyse of 1821 Cauchy initiated a
  reappraisal and reorganization of the foundations of
  eighteenth-century calculus. In this work he defined continuity
  essentially as we understand it, although he used the then-prevailing
  language of infinitesimals rather than the now-accepted $\varepsilon - \delta$
  formulation given by Weierstrass in the 1850s. [...]

(Note that it's useful to make explicit on what domain you're saying something is continuous; here, the author of the book is actually slipping up, since $1/x$ is in fact continuous on its natural domain $\mathbb R \setminus \{0\}$. Thanks @jkabrg)
Comment: Related to this, the notion of a function itself ([see](https://en.wikipedia.org/wiki/History_of_the_function_concept) ). Basically since the 17th century, there was an intuitive understanding of what a function is, but it took at least until the 19th century before the appearance of any definition that was resilient enough not to immediately crumble under close scrutiny.
Comment: $1/x$ is continuous
Comment: @tomsmeding The domain of the function $1/x$ is *not* the whole real line
Comment: @jkabrg The function defined by $f(x) = 1/x$ is only continuous on the subset $(-\infty, 0) \cup (0, \infty)$, not on the whole real line. By "the two branches of a hyperbola", the author refers precisely to the two branches of the "standard" hyperbola $1/x$, which are the two continuous components, one on $(-\infty,0)$ and one on $(0,\infty)$.
Comment: @jkabrg Fair point. You may write the author of that book if you'd like to have the book corrected. :)
Comment: :) That's not to detract from the rest of your answer
Comment: Cauchy did *not* define continuity via epsilon-delta.  Those historians who claim he did are involved in a [vain quest for the ghosts of departed *quantifiers*](http://matstud.org.ua/texts/2017/47_2/115-144.pdf).
Comment: @Mikhail I am basically as far from a historian as one can get, so I'll just assume that you're right. If you have a better source/quote, please improve the answer. :)
Answer: At the risk of having it called an obvious example, I submit Euclid's parallel postulate. It was formulated in his Elements cca. 300 B.C., then rationalized (including challenges and proof attempts) for many centuries before Saccheri laid down the $\,3\,$ alternatives of one/none/multiple parallels to a line through a given point in the $\,18^{th}\,$ century, then it took another $100$ years for the non-Euclidean geometries to be formalized by Lobachevsky and Bolyai.
Comment: Wish the downvoter had left a comment why.
Comment: Your reading is absolutely as valid as my own. To my reading, it didn't feel like an example. I didn't downvote, but I hope perhaps it may provide some explanation why someone might.
Comment: Well, it's not exactly a mathematical idea that took time to define rigorously. It's a mathematical proposition that took time to prove (rather, took time to disprove). The question explicitly distinguishes between ideas and propositions and is asking only about the former.
Comment: @6005 We'll have to agree to disagree. The OP specifically asked about "*instances where an abstract mathematical "idea" floats around for many years before even a rigorous definition or interpretation can be developed to describe the idea*". The notion that geometry must be based on a set of axioms was a novel and "*abstract*" idea back then. The particular significance of the parallel postulate was noted early, but it "*floated around*" for more than two millennia before a "*rigorous definition or interpretation*" was developed. So, in my reading at least, the answer above is fully on-topic.
Answer: The delta "function" showed up in a paper by Fourier - "Théorie analytique de la chaleur" in 1822.
It wasn't until ~1945 that Schwartz formally defined the delta functional as a distribution.
Comment: Can you link to that paper by Schwartz?
Comment: @nbubis it seems it was a book, not a paper — _"Théorie des distributions." Hermann, 2 vols., 1950/1951_, according to Wikipedia
Answer: The notions of real numbers themselves - as rational cuts, as equivalent rational Cauchy sequences, or as elements of the unique model for the theory of complete ordered fields - would only appear in the 19th century, despite the centrality of calculus in mathematics and other sciences.
Comment: @AntonTykhyy: I don't see it. If the magnitudes are rational, then their ratios are rational as well; so I don't see how that definition is a construction of real numbers at all. Furthermore, the only similarity I see between that definition and Dedekind cuts is that both involve comparison of rational numbers. What am I missing?
Comment: Euclid (V, def. 5) already had the construction of real numbers as rational cuts, except he was calling them ratios of magnitudes.
Comment: Euclid didn't apply the notion of rationality/irrationality to geometric magnitudes (lengths, areas and volumes) but to their ratios. It was known since Pythagoras that some ratios are irrational, and Euclid loc. cit. provides the definition of what it means for two ratios to be equal, less than or greater than one another, regardless of whether these ratios are rational or not. A Dedekind cut's upper and lower halves are the sets of rational numbers the numerators and denominators of which are those integers for which Euclid's equimultiples "alike exceed" or "alike fall short of" one another.
Comment: I'd accept this as a definition of real numbers, equivalent to Riemann's cuts, if Euclid had remarked that, given (say) a fixed line segment $x$, any line segment $y$ is determined (up to congruence, so really we're talking about lengths as a kind of geometric magnitude) by the pairs of natural numbers $(m,n)$ such that $my$ is shorter than $nx$, or something like that.
Comment: (Preferably he would also have characterized which collections of pairs of natural numbers could arise from such a ratio of magnitudes, namely that there exists at least one pair $(m,n)$ in the collection, and for each such pair, every pair $(m',n')$ such that $mn' < m'n$ is in, and also so is some pair $(m',n')$ such that $m'n < mn'$.  Only then could one *define* a positive real number as an appropriate collection of pairs of natural numbers, equivalent to Riemann's definition of a real number as an appropriate collection of rational numbers.  But perhaps this is asking too much.)
Comment: @TobyBartels : (Only) Some of your "$my$ is shorter then $nx$" thought is captured in Euclid's algorithm (for computing gcds).  He argued in terms of using one quantity as a unit to partition another quantity, leaving a remainder quantity, used to partition the first quantity, et c.  It's not quite what you're looking for, but the ingredients are all on the countertop, waiting to be mixed in the specific way you're discussing.
Comment: I also neglected that there must exist at least one pair $(m,n)$ that is *not* in the collection (else we allow $y = 0$, which Euclid would not allow as a line segment; and if we did allow that, then we'd have to explicitly require $x \ne 0$, so I left *something* out in any case).  And I should have put @AntonTykhyy in my first comment!
Comment: @EricTowers :  Yes, the necessary preliminaries are there, but not (as far as I understand) any motivation to put them together in this way.  Euclid would have easily understood what Riemann was doing, but he would not have understood so easily why!
Answer: Differentiable manifolds are an example. A rigorous definition only appeared about 100 years ago, in the works of Hermann Weyl and Hassler Whitney, although they were studied long before that time. Gauss's Theorema Egregium can already be seen as a theorem about this kind of concept, although stated long before it was formaly defined.
Comment: Do you think it might be worth mentioning the strong Whitney embedding theorem explicitly, as this was an important milestone uniting two different notions of "manifold" floating around at the time.
Comment: @WetSavannaAnimalakaRodVance That's an interesting complement, yes.
Answer: Complex Numbers are an example of
" ..abstract mathematical "idea" floats around for many years before even a rigorous definition or interpretation can be developed to describe the idea ".
And it was  much an embarrassing idea at Cardano and Bombelli times (XVI century) that took lot of imagination (sic) and mental stress to be settled.
Answer: "Computation" (or effective calculability) is still an abstract mathematical idea that floats around awaiting a rigorous definition.
There are various candidates for defining what the term could mean -- e.g. the set of strings that can be generated by a certain type of grammar, or the set of strings that can be accepted by a certain type of machine, or the set of functions that can be defined given a certain set of function-construction rules. And there are rigorous proofs of the equivalences between many of those definitions.
But we are still left with an intuition about whether those definitions are adequate. That intuition is called Church's Thesis or the Church-Turing Thesis, but it remains (merely) a thesis. We might still come up with a broader definition of what constitutes a "computation" that cannot be subsumed under the existing candidates.
Comment: I would say that the results of Church and Turing and all the equivalences then have *solved* the problem of defining computation, or effective calculability. It certainly can be said that most mathematicians take Turing-computability to be the definition of what an algorithm or effective computation is. However, this is a philosophical point and I like your answer.
Comment: I like this example a great deal, but do you really think that this is purely mathematics? Is there not a certain experimental aspect to this: that we won't come up with a definition of computability that isn't equivalent to the Church-Turing-Gödel triad of notions: the "candidates" you refer to are already rigorous. My impression is that although many quantum computing physicists conjecture that quantum computing will not come up with anything that isn't in principle computable in the classical sense, the idea that it might is not ruled out of the question.
Answer: Weil's conception of a cohomology theory for varieties adequate enough to solve the Weil Conjectures (the "Riemann Hypothesis" for varieties over finite fields) are an example. The idea for a Weil Cohomology was formulated in 1949 by Weil, and then Grothendieck came along in the 60's with etale and $\ell$-adic cohomology, which fit Weil's criteria and allowed Deligne prove the conjectures in 1974.
25 years may not be the longest time for something mentioned here, but exploring this idea and trying to rigorously realize it definitely helped create a decent portion of 20th century math.
Answer: Structure-preserving function.
It seems that this concept doesn't have a general definition yet. Category theory define the rules for calculations with morphisms but doesn't provide a general and formal rule what a structure-preserving function is, when the objects of the category are sets with additional structures.
For the kind of structures appearing in universal algebra it's clear enough but, for example, from an algebraic perspective, what makes continuity to the natural concept in topology?

This may provide a clue?
There is a subcategory of the category of relations as objects and relations between relations as morphisms, consisting of all relations as objects and certain relations between relations, that can be expressed by two relations, as morphisms.
Given two relations $R\subseteq A\times B$ and $R'\subseteq A'\times B'$. Some relations $r\subseteq R\times R'$ can be characterized by two relations $\alpha\subseteq A\times A'$ and $\beta\subseteq B\times B'$ so that
$((a,b),(a',b'))\in r \iff \Big((a,a')\in\alpha\wedge (b,b')\in\beta\wedge (a,b)\in R\implies (a',b')\in R'\Big)$
and if $R''\subseteq A''\times B''$, $r'\subseteq R'\times R''$, where
$r'$ is characterized by
$\alpha'\subseteq A'\times A''$ and $\beta'\subseteq B'\times B''$, then the composition $r'\circ r$ is characterized by the relations
$\alpha'\circ\alpha\subseteq A\times A''$ and
$\beta'\circ\beta\subseteq B\times B''$?
(Where $\circ$ denote the composition of relations).
Suppose $A=B\times B$ and that $R\subseteq A\times B$ is the composition in a magma. Then the functions among the morphisms between two such objects defines magma morphisms $B\to B'$.
Suppose $B=\mathcal P(A)$ and that $R\subseteq A\times B$ is the relation
$(a,S)\in R\iff a\in\overline{S}$ for some topology on $A$. Then the functions among the morphisms between two such objects define continuous functions $A\to A'$.
Comment: We have a general definition of a structure-preserving *bijection*, that is an isomorphism, and thus of the *groupoid* of sets equipped with a given structure, and this goes back at least to Bourbaki I (first published slightly before Eilenberg & MacLane).  But that's not good enough for what you're talking about, I agree.
Answer: You might want to check out Imre Lakatos' "Proofs and Refutations", which depicts in a fictional dialog the evolution of the idea of a "polyhedron" over the centuries. His goal is to illuminate the dialectical process of definition and redefinition in mathematics, and perhaps in cognition generally.
Answer: Computation seems to fall into this category - for a long time, there had been an informal notion of something like "information processing". There had, of course, been the idea of a function for a long time. There were also prototypical algorithms, even as far back as Euclid. But the general idea of a well-defined process that implements a function based on small steps did not appear until Turing defined it in his 1936 paper.
Answer: About Fractals (I know a fractal when I see it). What is it mathematical definition of this concept?
Till nowadays the notion of fractal does not have yet a proper mathematical definition.
Basically, a fractal is figure or shape that have a self-similarity  property. the geometry of a fractal differ from  one shape to another.
To this end  I would like to quote a speaker at conference in Leipzig last year (2017) He was asked by an attender: Sir what is a fractals ?
His answer: I know a fractal when I see it
Patently once someone has shown you a fractal for your first time , next time you will certainly recognise a fractal just by looking at his shape no matter how different it could be from the previous one: this is a fact.
There are different kind of  well know fractals : Speinski-Gasket (whose the triangle below), Mandelbrot set (see the second figure), Julia set.....
https://georgemdallas.wordpress.com/2014/05/02/what-are-fractals-and-why-should-i-care/
Answer: The calculus of variations was actively applied from the 17th Century and put on a firm theoretical foundation with the introduction of Banach spaces in 1920.
Answer: The Egyptians and the Babylonians knew a lot of mathematical facts (e.g. the Pythagorean theorem, the quadratic formula, the volumes of prisms and pyramids, etc.) which were later proved by the Greeks.
Also, Pascal and Fermat used mathematical induction and the well ordering of the naturals (in the form of infinite descent) 250 years before Peano formalized the axioms defining the natural numbers.


Standard error for coefficient on dummy with no variation

Question: Say I have a dummy variable $X$ (e.g., gender) and that units are divided into groups. I want to compute the average value of $X$ in each group. To achieve this, we can estimate the following linear model via OLS:
$X_i = \sum_{g = 1}^G Group_{i, g} \, \beta_g + \epsilon $
with $Group_{i, g}$ a dummy variable equal to one if the $i$-th unit belongs to the $g$-th group, and $G$ the number of groups. One can show that $\beta_g$ corresponds to the mean of $X$ in the $g$-th group. So, the advantage of this approach consists of getting standard errors for the average values in each group.
Suppose that my $X$ has no variation in one of the groups, and assume that this happens for the first group without loss of generality. Is this a problem for my linear model? The point estimate should be fine, but what about the standard error? Can I interpret it conventionally, i.e., measuring the sampling uncertainty of my estimated coefficient? I am getting a hard time thinking about this, though I may be simply overthinking. How should I think about this sampling uncertainty, if there is no variation of $X$ in the first group?
Follows a toy example to make the point, where I code $X$ as x2 and $Groups$ as groups:
## Generate data.
set.seed(1986)

n <- 1000

groups <- factor(sample(c(1, 2, 3), n, replace = TRUE))

x1 <- rnorm(n)
x2 <- rbinom(n, size = 1, prob = ifelse(groups == 1, 0, 0.5))

dta <- data.frame(x1, x2, groups)

## Dummy variable is always 0 in the first group.
sd(x2[groups == 1])
#> [1] 0

## Regressing x2 on groups to get group means and standard errors.
model <- lm(x2 ~ 0 + groups, data = dta)
summary(model)
#>
#> Call:
#> lm(formula = x2 ~ 0 + groups, data = dta)
#>
#> Residuals:
#>     Min      1Q  Median      3Q     Max
#> -0.5000 -0.4414  0.0000  0.5000  0.5586
#>
#> Coefficients:
#>         Estimate Std. Error t value Pr(>|t|)
#> groups1  0.00000    0.02174    0.00        1
#> groups2  0.50000    0.02251   22.22   <2e-16 ***
#> groups3  0.44144    0.02213   19.95   <2e-16 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 0.4039 on 997 degrees of freedom
#> Multiple R-squared:  0.4721, Adjusted R-squared:  0.4705
#> F-statistic: 297.1 on 3 and 997 DF,  p-value: < 2.2e-16

Comment: Why did you include `x1`, `betas`, and `y` in this example if you are only asking about `x2` and `groups`? Just want to make sure I'm interpreting your question correctly and that those are indeed extraneous.
Comment: Oh, you are right. I just adapted an old example I had. I will edit the question.
Answer: There is a violation of the assumption of homoscedasticity required for the usual OLS standard errors to be valid. It is violated in two ways: for the groups other than 1, homoscedasticity is always violated for binary outcomes. For group 1, the residual variance is 0, which is different from the residual variances of the other two groups. If you use a heteroscedasticity-consistent standard error, the lack of variability in group 1 becomes apparent and invalidates any inferences:
> lmtest::coeftest(model, vcov = sandwich::vcovHC)

t test of coefficients:

        Estimate Std. Error t value  Pr(>|t|)
groups1 0.000000   0.000000     NaN       NaN
groups2 0.500000   0.027951  17.889 < 2.2e-16 ***
groups3 0.441441   0.027293  16.174 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The standard error is 0, so the t-statistic cannot be computed, which is what you would expect.
Comment: @PlasticMan Using a HC SE makes the inferences for the other two groups valid. It should be about the same as if you had simple not included group 1 in the analysis.
Comment: Is the inference invalidate only for the first group?


A Basic Permutations Problem the order matters or doesn't confusion

Question: I just decided to refresh my basic combinatorics problem-solving skills and picked up this fun book by Hollos "Combinatorics Problems and Solutions." I am having a severe brain freeze over the solution to this problem, my question is, why is this a permutation?
There are 10 jobs that need to be filled, 4 of which must be filled by men, 3 by women, and the remaining 3 by either. If there are 20 men and 6 women available, in how many ways can we fill the jobs?
The solution says:"The 20 men can fill the 4 jobs in $\frac{20!}{(20-4)!}$ ways, the 6 women can fill the 3 jobs in $\frac{6!}{(6-3)!}$ ways. There are 16 men and 3 women still available to fill the 3 remaining jobs, which can be done in $\frac{19!}{(19-3)!}$ ways. So the total number of ways to fill the 10 jobs is $$\frac{20!}{(20-4)!} \times \frac{6!}{(6-3)!} \times \frac{19!}{(19-3)!}$$
So my question is, but isn't the order irrelevant here, i.e. shouldn't we use combinations instead of of permutations because we'd want to avoid overcounting and divide, for example, 20! men by 16! and also 4! to avoid overcounting?
Comment: If the jobs are distinguishable, then it matters who receives which job.
Answer: The order is relevant if each job is different and fixed to a given class of allowed genders. The question itself is a bit ambiguous but, I read it as: "There are a total of 10 jobs, (label them 1,2,...,10). 4 of these jobs must be filled by men (i.e. jobs 1,2,3,4 must be filled by men), 3 of them must be filled by women (5,6,7 must be filled by women),..."


Asymptotics for the probability that two $n$-cycles generate $S_n$ (or $A_n$).

Question: Is there a known asymptotic formula for the probability that two $n$-cycles generate $S_n$ (or $A_n$ in the event that $n$ is odd)?
There seems to be a lot of published research on this question for two arbitrary permutations. This is more a reference request -- no proof necessary. (In fact, I am only interested in the case when $n = p$ is prime, if that is somehow easier or known.)
Comment: The case $n$ is a prime is particularly easy. In that case, except for a "thin" family of exceptional primes, the only way two $n$-cycles can fail to generate $A_n$ or $S_n$ is if they are a power of each other, and this probability is easy to calculate. (Once you pick the first cycle, there are exactly $n-1$ "bad" choices for the second one.)
Comment: Do you need a lot of precision? For example, would the statement that it tends to 1 be enough, or you'd like to know the rate of convergence?
Comment: The exceptional values are when $n$ is of the form $\frac{q^x-1}{q-1}$ for a prime power $q$, plus a finite set. See https://mathoverflow.net/questions/378454/proving-an-inequality-regarding-number-of-transitive-subgroups-of-the-symmetric/379119#379119
for a brief explanation.
Comment: @verret nice. You should make this last an answer and I will accept it.
Answer: The case $n=p$ is a prime is particularly easy.
A subgroup of $S_p$ containing a $p$-cycle is either a subgroup of $\mathrm{AGL}(1,p)$, or it is an almost simple $2$-transitive group. (This is due to Burnside.)
Now, these almost simple groups are classified. For most values of $p$, this is only $S_p$ and $A_p$. The exceptional values are $11$, $23$ and when $p$ is of the form $\frac{q^x−1}{q−1}$ for a prime power $q$. (This is due to Guralnick I think.)
So, if $p$ is not exceptional, then the only way for two $p$-cycles not to generate $A_p$ is to be inside a copy of $\mathrm{AGL}(1,p)$. But $\mathrm{AGL}(1,p)$ has a unique (normal) subgroup of order $p$, so the two $p$-cycles must be powers of each other. The probability of this occuring is small and easy to calculate. (Once you pick the first cycle, there are exactly $p−1$ choices for the second one.)
This leaves the exceptional primes, but it could also be handled, as it known what the almost simple group is in this case, and this gives a smallish upper bound on the probability.


Restricting users to specific domains in Postfix

Question: I have a user list and domain list. All users in user_list should only be able to send mail to domains listed in allow_urls.
user_list
/^user1@*/   OK
/^user2@*/   OK
/^user3@*/   OK
/.*@*/ REJECT 5.4.3. Sorry..

allow_urls
domain1.ru limited_class
domain2.ru limited_class

I have these lists defined in my main.cf, but it doesn't appear to be working. Any user can send to any domain.
smtpd_restriction_classes = limited_class
limited_class = check_sender_access regexp:/etc/postfix/border/user_list, reject
smtpd_sender_restrictions =  check_recipient_access hash:/etc/postfix/border/allow_urls

What am I doing wrong?
Answer: You did nothing wrong. The restrictions you configured is for incoming mail, where incoming means sent from "outside".
When you now say "every user can send..." then this is true. You can't restrict the destination address when sending to the outside world.
If this doesn't answer your question, then please rework on your question to express what you really mean. Or better: What do you want to achieve? You only say that it doesn't work without specifying what.
Answer: Right solution below
user_list
/^user1@*/   limited_class
/^user2@*/   limited_class
/^user3@*/   limited_class
/.*@*/ REJECT 5.4.3. Sorry..

allow_urls
domain1.ru OK
domain2.ru OK

main.cf
smtpd_restriction_classes = limited_class
limited_class =  check_recipient_access hash:/etc/postfix/border/allow_urls reject
smtpd_sender_restrictions = check_sender_access regexp:/etc/postfix/border/user_list


Different version of Wald test statistic formula

Question: I came across two formulas for the Wald test statistic in a maximum likelihood framework:
One has $(R\hat{\theta}-r)'(RI_n^{-1}R')^{-1}(R\hat{\theta}-r)$, where $I_n^{-1}$ is the inverse of the information matrix.
The other one has $(R\hat{\theta}-r)'(R\frac{\hat{V}}{n}R')^{-1}(R\hat{\theta}-r)$, where V is a consistent estimator of the variance - covariance matgrix
I am confused since the inverse of the information matrix should be equal to the variance covariance matrix? Where does the division by n come from?
Answer: Without a source giving full definitions of all quantities in your formulae, that is hard to answer. But let me give an answer in a simplified context of a Wald statistic, namely a t-statistic. A Wald statistic for testing a single restriction is just a squared t-statistic.
So, take the simple case of testing $\mu=0$ for a random sample $x_i\sim (\mu,\sigma^2)$. The t-statistic can be written as
$$
t=\frac{\bar x}{s.e.(\bar x)}
$$
where the standard error of the sample mean is defined as
$$
s.e.(\bar x)=\sqrt{\frac{s^2}{n}}
$$
with $s^2=\frac{1}{n-1}\sum_i(x_i-\bar x)^2$. Hence, you could also write
$$
t=\frac{\sqrt{n}\bar x}{s}
$$
That is, in one notation, the denominator (what corresponds to the inverse in the Wald statistic) might be defined as the standard error, in another as the estimator of (the square root of) the error variance, with then the normalization (the square root of sample size) in the numerator.


What is a good SSH server to use on Windows?

Question: In my spare time I remotely support my wife's office via VPN into a Windows Server. I am about to purchase a wireless broadband service which doesn't support VPN.
I don't want to open up the remote desktop ports directly, and I would like to set up an SSH tunnel into the network, and if necessary then VPN over the top of that.
What is the best windows SSH Server implementation to use on a Windows 2003 Server, or should I just be using sshwindows?
Comment: Shopping Questions and product recommendations are Off-Topic on any of the [se] sites. See [Q and A is hard, lets go Shopping](http://blog.stackoverflow.com/2010/11/qa-is-hard-lets-go-shopping) and the [FAQ] for more details.
Answer: We've been using Bitvise WinSSHD for a few years now and have been very happy with it.  It's free for "Non-Commercial/Personal" use, but even the for-pay version is not terribly expensive.  It's nice because it's designed specifically for Windows and supports just about any type of authentication you would throw at it (AD, Kerberos, self-contained user db, etc).  Installation and Upgrades are a breeze.  Initial config has a nice GUI and I believe most of the config is also scriptable as well.  Their site gives a more complete feature list.
*Edit: Looks like Bitvise decided to rename the product to Bitvise SSH Server
Comment: It is very impressive solution and pricing is reasonable: free for personal use and $100 USD per production server. The public key encryption supports public keys from OpenSSH. It is fast and light. The GUI exposes powerful tweakery and there is an automation API. Very nice product.

(And unlike cygwin/openssh or interix/openssh it works correctly with powershell.)
Comment: Just installed it for cross platform development. Now I work from Linux machine all the time. Was up and running in 5 minutes.
Comment: @BrianReiter Do you know if this software runs each client session in the user accounts security context? Or does it suffer from the same issue as your comment above on the accepted answer?
Comment: @Tyson If the user logging in is a real windows local/domain account, it uses that security context.  If it's a "virtual" user, it runs in whatever security context that virtual user has been configured against.  It's also configurable what type of login is used (i.e. Network or Interactive).  More info available in the docs.
Comment: Just installed on Win7 sp1 - works fine.
Comment: Thanks for sharing this! I'm trying it right now and I'm loving it! Looks exactly like what I was looking for. :-)
Answer: I've been using FreeSSHd on my home Windows box, and have not run into any limitations.  Highly recommended.
Comment: FreeSSHd starts your sessions in the security context of the service itself. That means that while you are authenticated by one of several mechanisms, your actual security context is whatever the serer is running as (eg local system). I think this is a serious weakness of the FreeSSHd design.
Comment: FreeSSHD has problems being installed and run on Windows 7, if it is done by elevating from Standard User to Administrator.
Comment: Tried this one and it works!!! Was testing some c# SSH local file upload code
Comment: Based on my experience, FreeSSH is not able to deal with `~/.ssh/authorized_keys` file.
Comment: It doesn't necessarily need the authorized_keys file. Its not optimal but for every user you would add their pub key. And the name of the pub key must match the user name EXACTLY. What hung me up was trying to drop `vagrant.pub` for a user named `vagrant`. The key ultimately needs to be named `vagrant` to match the user.
Answer: I have been evaluating KpyM SSH Server and it looks pretty good and is BSD-licensed with source. It is based on cryptlib fundementally and creates a Window Station for each session which is why it is one of the few open source SSH solutions that works correctly with PowerShell.
We have been using OpenSSH compiled for Interix/SFU/SUA for the last several years. The downside is that OpenSSH doesn't play all that nicely with PowerShell which is an annoyance and it requires the full POSIX subsystem from Services for Unix 3.5 or the Subsystem for Unix Applications (Win2k3 R2 and later).
Years ago we used to use Vandyke and it worked well.
Bitvise WinSSHD is very nice. Supports aes256 and aes128 out of the box. It is not open source but it is free (with AD integration crippled) for personal use and very reasonable $100 USD per server for commercial use. Can be configured to use powershell as the default shell and powershell works correctly. WinSSHD has very granular configuration per-account and per-group and per client IP and per client DNS. There are logon and logoff actions that can be configured per account or group. Supports OpenSSH public key files. Exposes an automation API. Write logs to the Windows event log and/or text file. Still has a small and light service process.
Answer: CopSSH is nice.  It wraps a Cygwin OpenSSH in an installer and makes it very quick and easy to get things configured.
Answer: I would use the one from cygwin.
This is not directly answering your question, but I think that SSL is as secure as SSH and you could also use stunnel or socat ( http://www.dest-unreach.org/socat/ ) to open a certifacte-authenticated ssl-encrypted port for remote desktop. Socat would authenticate using ssl client certificates and forward the traffice towards the rdp port. On your machine you would do the same in reverse. The man page has samples for this and socat is available for windows
Comment: I like OpenSSH but I find it frustrating that it doesn't play nice with nicely with PowerShell. Maybe that's PowerShell's fault but it is still very annoying. You have to do "powershell -noexit -command -" or it will hang loading powershell. And then you get a wierd shell that has no prompt. It seems like the SSH servers designed for Windows that create a Window Station for eahc session don't have this problem.
Comment: I've used Cygwin successfully on many different machines.
Comment: After trying most of the free suggestions in this thread (KpyM lacks documentation, FreeSSHd will not launch even when run as administrator, OpenSSH is hard to configure), Cygwin was the only one I got working successfully. It also helps that there are plenty of tutorials for it, [like this one](http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/)
Answer: One option is OpenSSH for Windows:

OpenSSH for Windows is a free package
  that installs a minimal OpenSSH server
  and client utilities in the Cygwin
  package without needing the full
  Cygwin installation.
The OpenSSH for Windows package
  provides full SSH/SCP/SFTP support.
  SSH terminal support provides a
  familiar Windows Command prompt, while
  retaining Unix/Cygwin-style paths for
  SCP and SFTP.
Comment: any security concerns with this one?  the sourceforge project hasn't been updated since 2004
Answer: I have always had good experiences with Van Dyke's software (they make IMO the best Windows SSH client). For Windows SSH server use, they have what they call VShell Server.
http://www.vandyke.com/products/vshell/index.html
Answer: Personally, I'd avoid the Cygwin variants. I've had problems with OpenSSH running as a service blocking windows updates. Fine for non-production servers, but not something you want to rely on for your remote access solution if you're trying to apply those very updates.
Answer: I have found FortressSSH to be a fast and reliable solution.  They have even been through Microsoft certification for Windows
http://pragmasys.com/FortressSSHServer.asp
Comment: At $700 + maintenance, it's expensive, but you can't argue with FIPS-140-2 certification. These guys seem to know what they're doing :-)


Generate a clickable pdf from html

Question: In Summary:
I want to dynamically generate pdfs from html. I want the resultant pdf to be searchable/clickable - NOT an image. I would like a way to do this that is compatible with vue3.
Background:
I have a Vue3 project that returns long and wordy lists of results from an API.
On the site, the results are in accordions and look great.
I want to let my users download a PDF of the results. I would like to make a minimal template for the pdf that shows an expanded (no accordions) version of the result list.
As far as I can tell there are two basic ways to do this:

Create a print style in css and let users "save" the result to file as a pdf
use one of a number of "html2pdf" style modules to generate the pdf

Neither approach seems to really do what I want:
routing the user to a print result leads to differing behaviour on mobile and desktop and I can't figure out how to tell the browser to just offer a pdf download.
all the various "htmltpdf" type modules I have found so far have great options and defaults but they output the pdf as an image...
I want my users to be able to select and copy the contents of the pdf - there are phone numbers and links embedded for example.
Is there some way to do this in vue; A pleasant and nice-looking download experience AND a clickable/searchable pdf?
Comment: Does this answer your question? [Links, Hyperlinks into a canvas using PDF.js](https://stackoverflow.com/questions/19953879/links-hyperlinks-into-a-canvas-using-pdf-js)
Comment: Not really. I don't want to render an existing pdf. I want to generate one from html. Ideally in a new tab with the browser's own pdfviewer. Or just download the pdf.
Comment: @KJ I'm experimenting with a "hidden" version of the content not in accordions to display during a print call. This seems to help standardize the look of the site. It means I am loading at least two full versions of the content though. Not sure if there's a better way to do this.
Answer: You can use puppeteer to create pdf using html.
Step 1: Install puppeteer
npm i puppeteer

Step 2: Write a function to that renders your html into the browser and saves the page as pdf
async getPdfFromHtml(html: string): Promise<any> {
  const browser = await puppeteer.launch({ headless: true });

  const page = await browser.newPage();
  await page.setContent(html);
  const pdfBuffer = await page.pdf({ printBackground: true, format: 'A4' });
  await page.close();
  await browser.close();

  return pdfBuffer;
}

Step 3: Call this function wherever you want:
const html = 'your-html-template-string';

// here you will get the pdf buffer
const pdfBuffer = await getPdfFromHtml(html);

Step 4: Now you can send your pdf buffer to frontend. The following is an example of how I do in express.js
const buffer = pdfBuffer.toString('base64').replace(/([^\0]{76})/g, '$1\n') + '\n\n';

res.header('Content-type', 'application/pdf');
res.send(buffer);

Step 5: Now on frontend, all you need is call this api using an achor tag and the pdf will be downloaded.
Optional: Another alternative is to upload this buffer to aws-s3 and return the url to FE, the pdf will be downloaded once you redirect to that url.
Comment: So basically bypass code and pass my html directly to the browsers built-in print renderer?
Comment: Am I right in thinking puppeteer needs to install a full chromium client server-side for this to work?
Comment: If you want a quick solution without the interaction of backend, then use `window.print()`(this will have same behaviour as pressing cmd+P in browser) OR you can implement this puppeteer solution on FE and pass your html into the function.

But ideally, if I have a download button, then I would call an api on click and that api would return the pdf file which I would be able to download on FE.

Generate a clickable pdf from html


Get IP address of client in JSP

Question: I need to get the IP address of the client in the JSP page. I have tried the following ways:
request.getRemoteAddr()
request.getHeader("X_FORWARDED_FOR")
request.getHeader("HTTP_CLIENT_IP")
request.getHeader("WL-Proxy-Client-IP")
request.getHeader("Proxy-Client-IP")
request.getHeader("REMOTE_ADDR")

However, none of those ways did return the desired IP address. How do I get the IP address of the client in the JSP page?
Comment: try request.getRemoteHost() method, if the client not use a gateway ip of client return by this method. else this method returns the ip of gateway.
Comment: If you use custom header, always name them `X-`.
Comment: What does `request.getRemoteAddr()` return?
Comment: Also see [this](http://stackoverflow.com/q/1423347/473637) question.
Comment: request.getRemoteAddr() returns 0:0:0:0:0:0:0:1 which is not the true value of IP of client.
Answer: To get IP address of the client, I've used the following method
<%   String ip = request.getHeader("X-Forwarded-For");
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("Proxy-Client-IP");
        }
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("WL-Proxy-Client-IP");
        }
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_CLIENT_IP");
        }
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_X_FORWARDED_FOR");
        }
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getRemoteAddr();
        }
        %>

Hope this helps,
please leave a feed back.
Answer: <%
out.print( request.getRemoteAddr() );
out.print( request.getRemoteHost() );
%>

You may not get the real client IP if a the client is behind a proxy, you will get the IP of the proxy and not the client. However, the proxy may include the requesting client IP in a special HTTP header.
<%
out.print( request.getHeader("x-forwarded-for") );
%>

Comment: But how to get The IP of client when no proxy is configured between client and requested server.
Comment: [link](http://stackoverflow.com/questions/1423347/problem-with-jsp-getremoteaddress) See this post.
Comment: output comes like 0:0:0:0:0:0:0:1 you have any idea why??
Answer: Is your application server behind a load balancer, a proxy or a web server? Just an example; F5 load balancer exposes the client IP address with the "rlnclientipaddr" header:
request.getHeader("rlnclientipaddr");

Answer: do you use reverse proxy like apache proxy?
http://httpd.apache.org/docs/2.2/mod/mod_proxy.html
When acting in a reverse-proxy mode (using the ProxyPass directive, for example), mod_proxy_http adds several request headers in order to pass information to the origin server. These headers are:
X-Forwarded-For
The IP address of the client.
X-Forwarded-Host
The original host requested by the client in the Host HTTP request header.
X-Forwarded-Server
The hostname of the proxy server.

Comment: No Proxy is used between server and client . All Values getting using X-Forwarded-For , X-Forwarded-Host , X-Forwarded-Server are giving null cause of proxy is not used.
Comment: if you are not using any of reverse proxy, just check use http servlet API - httpServletRequest.getRemoteHost() and getRemotePOrt()  http://docs.oracle.com/javaee/6/api/javax/servlet/ServletRequest.html
Comment: Nothing is present in httpServletRequest that will provide me the IP of Client. I Have Checked The Whole API.

Get IP address of client in JSP


What is B II, followed by a dashed line, and what does it mean?

Question:
In the guitar tab of Opus 35, No 22 in B minor of Fernando Sor that I get from classical clef, there is a notation BII as shown in the above picture.
What does it mean?
Comment: See also http://music.stackexchange.com/questions/14634/what-does-the-notation-2-3bii-mean?rq=1
Answer: The 'BII' is instructing you to bar your finger across the second fret like you would with a barre chord.
The 'B' stands for bar and the 'II' represents the second fret. You can even see in the sections where the 'BII' occurs the lowest notes on your fret board are on the 2nd fret. This is just a another instruction similar to that of a fingering that isn't "necessary", but helps you play the part.
Comment: That is correct.  I would also add, the dashed lines and the small vertical line at the end of the dashed section indicate how long to hold that barre on the second fret. The beginning of each of the corresponding measures call for two notes to be played simultaneously which is most easily accomplished by barring.


Wordpress - alterar título do post no header da página

Question: Neste link: http://wp.ubiqui.com.br/2017/05/24/central-de-marketplaces/ vocês verão que sobre a imagem está o título do post.
Gostaria de colocar um código no Wordpress para que, apenas quando acessar um post, ele troque para "Notícias".
Nas demais páginas, é para ficar do jeito que está.
Answer: Você pode usar o filtro "wp_title". No seu arquivo functions.php você pode inserir algo como:
<code>function trocar_titulo($title){
    if(is_single()){
        return "Notícias";
    }
    return $title;
}
add_filter( 'wp_title', 'trocar_titulo', 10, 2 );
</code>
Ref: https://developer.wordpress.org/reference/hooks/wp_title/


How to find One hour before and One hour after from reminder Time

Question: I need to display the time one hour before and one hour after my reminder time. Suppose my reminder time is 11 Am the reminder need to show from 10 am and ends at 12 pm. now it's showing the whole day.
reminder_date is my reminder time
var countDownDate = new Date($('#end_time').val()).getTime();
 var countStartDate = new Date($('#start_time').val()).getTime();
var now = new Date().getTime();
var dsr = <?=$todo['dsr']?>;
if(dsr.length> 0){
let dsr_str ='
';
$.each(dsr, function(index, item){
 let now = new moment(item['reminder_date']);
 var start_distance = countStartDate - now;
if(start_distance < 0){
var distance = countDownDate - now;
 dsr_str += '
'+item['company']+' | '+item['phone']+' | '+now.format("hh:mm A")+'
';});
 dsr_str +='
'
 $(document).Toasts('create', {
 title: 'DSR Reminder',
 class: 'bg-success',
  body: dsr_str
  })
  }
 if (distance < 0) {
 clearInterval(x);
 location.reload(true);
  }
}
}, 1000);

});

Answer: Just add/substract 3600000 (60 min * 60sec * 1000 ms) from countDownDate /countStartDate which are just numbers of milliseconds
var countDownDate = new Date($('#end_time').val()).getTime();
var countStartDate = new Date($('#start_time').val()).getTime();
countDownDate+=3600000;
countStartDate-=3600000;

Or use
start_distance < 3600000
which will check if current time is less than an hour before the start date
Comment: actually, I don't know how to put in my code can you help me Dobromir Velev
Comment: only this much need to add in code Dobromir Velev
Comment: Lines 3-4 in the code above will add an hour to countDownDate and substract an hour from countStartDate. The first 2 lines are just to help you see where to put it
Comment: sorry i didnt under stand can you put in my code  Dobromir Velev
Comment: No, I cannot put it in your code - you just need to copy the lines with 3600000  in them and paste them
Comment: only need to add this much  Dobromir Velev


Repetitive entries in SQL JOIN query

Question: I have three tables posts, comments, and users. posts table includes two types of posts, question and answer. Comments are made on questions and answers. My purpose is to get a single question with comments on it, answers on it and comments on that answers. Also I need username from the users table as author on each question, answer and comment I fetch. I am using Postgres 9.5, and making use of json_agg() function.
While example output I need should be something similar to the first following, I get repeated entries.
What is the thing I am missing here? Correct group by clauses may be. Or the subquery to gather the answers with their comments is not the way to do it. When I comment out the left join from posts on the comments table I get desired results without comments on questions. Also when I cancel the left join containing the subquery I get non-repetative results as I expect, and again that is not complete dataset I desire. These are things I have collected so far to solve my issue.
What I need:
[
  {
    "post_id": "10",
    "created_at": "2016-05-10T00:16:54.469Z",
    "post_type": "question",
    "post_title": "qwerty",
    "post_text": "asdasd asda sdasd",
    "post_author_id": 1,
    "author": "isikfsc",
    "parent_post_id": null,
    "is_accepted": null,
    "acceptor_id": null,
    "answers": [
      {
        "post_id": 17,
        "created_at": "2016-05-10T04:58:56.350229",
        "post_type": "answer",
        "post_title": null,
        "post_text": "222asda dasdad asdada",
        "post_author_id": 1,
        "author": "isikfsc",
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 5,
            "created_at": "2016-05-10T10:56:30.220128",
            "text": "qweqwe",
            "author_id": 1,
            "author": "isikfsc",
            "parent_post_id": 17
          },
          {
            "id": 8,
            "created_at": "2016-05-10T11:00:00.182991",
            "text": "sasasd",
            "author_id": 1,
            "author": "isikfsc",
            "parent_post_id": 17
          }
        ]
      },
      {
        "post_id": 14,
        "created_at": "2016-05-10T04:19:19.005556",
        "post_type": "answer",
        "post_title": null,
        "post_text": "asdasdasdasd",
        "post_author_id": 1,
        "author": "isikfsc",
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 2,
            "created_at": "2016-05-10T05:25:34.671008",
            "text": "qeqweqwe",
            "author_id": 1,
            "author": "isikfsc",
            "parent_post_id": 14
          }
        ]
      }
    ],
    "comments": [
        {
          "id": 1,
          "created_at": "2016-05-10T10:56:30.220128",
          "text": "qweqwe",
          "author_id": 1,
          "author": "isikfsc",
          "parent_post_id": 10
        },
        {
          "id": 4,
          "created_at": "2016-05-10T11:00:00.182991",
          "text": "sasasd",
          "author_id": 1,
          "author": "isikfsc",
          "parent_post_id": 10
        }
    ]
  }
]

My query is:
SELECT
    q.*,
    json_agg(ac.*) AS answers,
    json_agg(c.*) AS comments --comments on posts of post_id questions
FROM posts q

LEFT JOIN
    (
        SELECT
            a.*,
            json_agg(c.*) AS comments -- comments on posts of post_id answers
        FROM posts a
        LEFT JOIN comments c
        ON a.post_id = c.parent_post_id

        GROUP BY a.post_id
    ) ac
ON q.post_id = ac.parent_post_id

LEFT JOIN comments c
ON q.post_id = c.parent_post_id

WHERE q.post_id = 10
GROUP BY q.post_id

What I get:
[
  {
    "post_id": "10",
    "created_at": "2016-05-10T00:16:54.469Z",
    "post_type": "question",
    "post_title": "qwerty",
    "post_text": "asdasd asda sdasd",
    "post_author_id": 1,
    "parent_post_id": null,
    "is_accepted": null,
    "acceptor_id": null,
    "answers": [
      {
        "post_id": 17,
        "created_at": "2016-05-10T04:58:56.350229",
        "post_type": "answer",
        "post_title": null,
        "post_text": "222asda dasdad asdada",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 5,
            "created_at": "2016-05-10T10:56:30.220128",
            "text": "qweqwe",
            "author_id": 1,
            "parent_post_id": 17
          },
          {
            "id": 8,
            "created_at": "2016-05-10T11:00:00.182991",
            "text": "sasasd",
            "author_id": 1,
            "parent_post_id": 17
          }
        ]
      },
      {
        "post_id": 17,
        "created_at": "2016-05-10T04:58:56.350229",
        "post_type": "answer",
        "post_title": null,
        "post_text": "222asda dasdad asdada",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 5,
            "created_at": "2016-05-10T10:56:30.220128",
            "text": "qweqwe",
            "author_id": 1,
            "parent_post_id": 17
          },
          {
            "id": 8,
            "created_at": "2016-05-10T11:00:00.182991",
            "text": "sasasd",
            "author_id": 1,
            "parent_post_id": 17
          }
        ]
      },
      {
        "post_id": 17,
        "created_at": "2016-05-10T04:58:56.350229",
        "post_type": "answer",
        "post_title": null,
        "post_text": "222asda dasdad asdada",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 5,
            "created_at": "2016-05-10T10:56:30.220128",
            "text": "qweqwe",
            "author_id": 1,
            "parent_post_id": 17
          },
          {
            "id": 8,
            "created_at": "2016-05-10T11:00:00.182991",
            "text": "sasasd",
            "author_id": 1,
            "parent_post_id": 17
          }
        ]
      },
      {
        "post_id": 17,
        "created_at": "2016-05-10T04:58:56.350229",
        "post_type": "answer",
        "post_title": null,
        "post_text": "222asda dasdad asdada",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 5,
            "created_at": "2016-05-10T10:56:30.220128",
            "text": "qweqwe",
            "author_id": 1,
            "parent_post_id": 17
          },
          {
            "id": 8,
            "created_at": "2016-05-10T11:00:00.182991",
            "text": "sasasd",
            "author_id": 1,
            "parent_post_id": 17
          }
        ]
      },
      {
        "post_id": 14,
        "created_at": "2016-05-10T04:19:19.005556",
        "post_type": "answer",
        "post_title": null,
        "post_text": "asdasdasdasd",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 2,
            "created_at": "2016-05-10T05:25:34.671008",
            "text": "qeqweqwe",
            "author_id": 1,
            "parent_post_id": 14
          }
        ]
      },
      {
        "post_id": 14,
        "created_at": "2016-05-10T04:19:19.005556",
        "post_type": "answer",
        "post_title": null,
        "post_text": "asdasdasdasd",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 2,
            "created_at": "2016-05-10T05:25:34.671008",
            "text": "qeqweqwe",
            "author_id": 1,
            "parent_post_id": 14
          }
        ]
      },
      {
        "post_id": 14,
        "created_at": "2016-05-10T04:19:19.005556",
        "post_type": "answer",
        "post_title": null,
        "post_text": "asdasdasdasd",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 2,
            "created_at": "2016-05-10T05:25:34.671008",
            "text": "qeqweqwe",
            "author_id": 1,
            "parent_post_id": 14
          }
        ]
      },
      {
        "post_id": 14,
        "created_at": "2016-05-10T04:19:19.005556",
        "post_type": "answer",
        "post_title": null,
        "post_text": "asdasdasdasd",
        "post_author_id": 1,
        "parent_post_id": 10,
        "is_accepted": null,
        "acceptor_id": null,
        "comments": [
          {
            "id": 2,
            "created_at": "2016-05-10T05:25:34.671008",
            "text": "qeqweqwe",
            "author_id": 1,
            "parent_post_id": 14
          }
        ]
      }
    ],
    "comments": [
      {
        "id": 1,
        "created_at": "2016-05-10T05:25:28.200327",
        "text": "asadasdad",
        "author_id": 1,
        "parent_post_id": 10
      },
      {
        "id": 4,
        "created_at": "2016-05-10T10:25:23.381177",
        "text": "werwer",
        "author_id": 1,
        "parent_post_id": 10
      },
      {
        "id": 1,
        "created_at": "2016-05-10T05:25:28.200327",
        "text": "asadasdad",
        "author_id": 1,
        "parent_post_id": 10
      },
      {
        "id": 4,
        "created_at": "2016-05-10T10:25:23.381177",
        "text": "werwer",
        "author_id": 1,
        "parent_post_id": 10
      },
      {
        "id": 1,
        "created_at": "2016-05-10T05:25:28.200327",
        "text": "asadasdad",
        "author_id": 1,
        "parent_post_id": 10
      },
      {
        "id": 4,
        "created_at": "2016-05-10T10:25:23.381177",
        "text": "werwer",
        "author_id": 1,
        "parent_post_id": 10
      },
      {
        "id": 1,
        "created_at": "2016-05-10T05:25:28.200327",
        "text": "asadasdad",
        "author_id": 1,
        "parent_post_id": 10
      },
      {
        "id": 4,
        "created_at": "2016-05-10T10:25:23.381177",
        "text": "werwer",
        "author_id": 1,
        "parent_post_id": 10
      }
    ]
  }
]

Answer: Grouping happens once all of the parties have been joined, so aggregates will depend on the resulting cardinality. Joining posts with answers AND comments causes a full join between them, duplicating all values. They need to be separated and performed individually, one way you can do it is the following:
SELECT
    q.*,
    (SELECT json_agg(ac.*)
     FROM (
       SELECT a.*, json_agg(c.*) AS comments
       FROM posts a
       LEFT JOIN comments c ON (a.post_id = c.parent_post_id)
       WHERE a.parent_post_id = q.post_id
       GROUP BY a.post_id
       ) ac
    ) AS answers,
    json_agg(c.*) AS comments --comments on posts of post_id questions
FROM posts q
LEFT JOIN comments c ON (q.post_id = c.parent_post_id)
WHERE q.post_id = 10
GROUP BY q.post_id;

Alternatively:
SELECT q.*, qa.answers, qc.comments
FROM posts q
LEFT JOIN (
  SELECT ac.parent_post_id, json_agg(ac.*) AS answers
  FROM (
    SELECT ac.*, json_agg(c.*) AS comments
    FROM posts ac
    LEFT JOIN comments c ON (c.parent_post_id = ac.post_id)
    GROUP BY ac.post_id
    ) ac
  GROUP BY ac.parent_post_id
  ) qa ON (qa.parent_post_id = q.post_id)
LEFT JOIN (
  SELECT c.parent_post_id, json_agg(c.*) AS comments
  FROM comments c
  GROUP BY c.parent_post_id
  ) qc ON (qc.parent_post_id = q.post_id)
WHERE q.post_id = 10;

Comment: I get two error messages from these queries at my terminal. First one is for the former: `[error: syntax error at or near "."]`and the second one for the latter: `[error: syntax error at or near "FROM"]`. I could not find why? I keep checking to figure out.
Comment: By `LEFT JOIN comments c (q.post_id = c.parent_post_id)` did you mean `LEFT JOIN comments c ON q.post_id = c.parent_post_id`
Comment: @Işık Yes, fixed both bugs now
Comment: The first one is ok now, no errors. The second one at the last left join lacks a ON I think. When I add it says at Line 9 table q cannot be referenced from this part of the query.
Comment: It still says the same thing.
After all the first one also does not produce the result I need. It does not put the answers inside the answers, but it put the post itself with the comments on this post inside the answers key.
Comment: I have made the edit that give it final juice. Can you review? And the second alternative is still problem. But you can remove it if you do not think it has any significant performance or any other advantages to the first one. Alternatively we can fix it. Then I will accept the answer. What to do?
Comment: @Işık Apologies for having posted such buggy code (it is easiers when users provide sample data in their question). Thanks for your correction, that would have fixed it. I applied it to the second case (which had the same problem), and added fields to the main SELECT (which I had left blank!). Hopefully this time they'll both work, for you to compare which one performs better with your data.
Comment: My apologies for lack of sample data. The second one still not working and have the same q cannot be referenced here problem.
Comment: @Işık I rewrote the second query entirely, I rushed it and it was really bad! It may still have a bug, but now it makes sense.
Comment: Ok. It still have the same kind of error, this time for ac. But we can remove it and accept the answer. Or we can put a line on it telling that was for general approach.

Repetitive entries in SQL JOIN query


C# Search and Set Node Values

Question: With my other question: Showing Multiple XML Data Entries With the Same Node Value
I have one more question, in the part:
XmlNodeList allNodes = doc.SelectNodes("/Lessons/Lesson[Date='01/01/2010']");

Is there a way to put a string with a value in it (that I already have setup) in the part '01/01/2010'?
Comment: do you mean use a variable as opposed to a literal value?

var myDate = "'01/01/2010'";

and then

XmlNodeList allNodes = doc.SelectNodes("/Lessons/Lesson[Date=myDate]");
Answer: This simple change would do...
string yourstring = "01/01/2010";

XmlNodeList allNodes = doc.SelectNodes("/Lessons/Lesson[Date='" + yourstring + "']");

C# Search and Set Node Values


How to calculate ratings/rankings from Paired comparison / Pairwise comparison of large data-sets?

Question: As part of my graduate thesis (area: psychology) I have gathered preference data. The data includes approximately 50000 heads-up comparison between elementX and elementX. I have a total of 15 elements. The participant would be shown 36 comparisons in random order, of which he/she would then choose the preferred element. I need to be able to calculate a meaningful "interval" scale-value for each element in order to see if the actual preferences of each element can predict theoretically generated scale values using regression analysis.
The Data:
14 elements, forming a 14x14 matrix table
91 unique possible comparisons ((14*14-14)/2)
= 50000 comparison (NOT evenly distributed across all comparisons)
The problem:
Since showing the participant all 91 possible comparison would be too time consuming I've just presented each participant 36 comparisons. Moreover, each participant was not eligible for each element, the eligible elements was determined if the participant owned the element.
For example: ParticipantX owns element; 1, 5, 6, 8 and 10. And would therefore only be shown comparisons that includes those elements.
Furthermore, as owning some elements were considered more rare, I made a priority list for each element, so that element1 > element2 > .... > element14
For example: If the participant would be eligible for 50 possible comparisons (but restricted by the 36 comparison-cap) the comparisons shown to the participant would be based on the rarity order.
I'm afraid I destroyed the data by making this rarity order, since it resulted in some some elements being compared unequally to some elements; For example: of 1000 comparisons that element 1 was included in, 30% was with element3, 10% with element4 etc ...
My questions: What method would you suggest would be the best for calculating "scale" values for each element? Currently I've calculated the basic probability that a given element has been chosen over another element.
For example; element1 being chosen 95% of times to be preferred over any other element.
However, I would prefer a more refined method to calculate the element values.
Comment: Bradley-Terry model is supposed to be the way to do this, but I don't understand it.  Did you have any luck since you posted the question?
Answer: Hopefully this isn't too late, but your data is still valuable using the Bradley-Terry model. This paper is a nice reference MM Algorithms for generalized Bradley-Terry models. Equation 4 is the one you want.
Brief explanation as I guess you are not a mathematition, and I'm being nice as I'd really like a copy of your data to test my implementation of this algorithm!
Explanation of Bradley-Terry Model
Basically Bradley-Terry assigns a value to each object, which is the answer you want - e.g. if you are asking "Which of these cars looks sexier", the value would measure sexiness. Then it says if you have two cars with sexinesses $s_i$ and $s_j$, the probability of someone choosing car $i$ over car $j$ is:
$$P(o_i\succ o_j)=\frac{e^{s_i}}{e^{s_i}-e^{s_j}}$$
That actually only depends on $s_j-s_i$ as follows:

So if they have the same sexiness the chances are 50-50 (random guessing), and if one has a higher sexiness it is more likely to be chosen.
This model is mostly plucked out of the air for its plausibility and nice mathematical properties. I don't know how realistic it really is. It is widely use though.
Maximum Likelihood
BT then uses something called Maximum Likelihood to estimate the values of $s_i$. Put in words, it finds the sexinesses of all the cars that would make the choices that you measured ("Dave says car A is sexier than car B" etc.) as likely as possible, according to our choice model above. So if people choose car A over car B a lot, then giving $s_A$ a lower sexiness than $s_B$ would mean that their choices (A > B) are very unlikely. So those sexiness values are not a maximum likelihood estimate.
To actually find all the $s_i$'s you can use a generic optimisation method (e.g. fminsearch in matlab). Or there is an iterative algorithm that they claim is guaranteed to converge on a "unique maximum" (which I assume means global?).
That algorithm is equation 4 in the linked paper (plus normalisation).
Algorithm.
Basically you count the number of times each object is beaten by every other. Store that in $w_{ij}$. The total number of comparisons between any two objects is $N_{ij}=w_{ij}+w_{ji}$. Define new numbers $\gamma_i=e^{s_i}$. Then start with all $\gamma_i=0$ and iteratively update, looping repeatedly through the $i$'s.
$$\gamma_i\leftarrow W_{i}\left[\sum_{j\neq i}\frac{N_{ij}}{\gamma_{i}+\gamma_{j}}\right]^{-1}$$
You don't need to complete a whole loop before updating each $i$. By which I mean you don't need to store old values of $\gamma_i$. As soon as a new value is calculated you can use it in subsequent updates.
$W_i$ is the total number of wins by $i$: $\sum_{j}w_{ij}=W_{i}$
You also need to normalise $\gamma$ after each loop over it:
$$\gamma \leftarrow \frac{\gamma}{\mathrm{mean}(\gamma)}$$
Regularisation.
You probably won't need this since your data set is so large and dense. In some cases you need to regularise by adding "phantom" comparisons of every object with an extra virtual one. Apparently the standard method is to count $\lambda$ wins for the real objects vs virtual, and $\lambda$ losses. I've found $\lambda$=0.1 to work fine.
Please can you send me your data and expected results if you still have it? :-)
Comment: This does make me wonder if there's a simple extension to include $(\mu, \sigma)$ for each variable not just $\mu$.
Comment: Imagine a comparison between $A$ and $B$ as sampling an observed score for $a$ and $b$ from two different underlying distributions, then comparing the scores. Then you get the same sort of sigmoidal curve for $P(A > B)$. However, the implied distribution here is somewhat longer-tailed than a normal distribution. More in next comment as I ran out of space.
Comment: Imagine $x_1 \sim \mathcal{N}(\mu_1, 1)$ and $x_2 \sim \mathcal{N}(\mu_2, 1)$ and look at the asymptotics when $\mu_2 >>> \mu_1$. I get $P(x_1 > x_2) \approx \frac{1}{\sqrt{2\pi}}\frac{e^{\frac{-(\mu_2-\mu_1)^2}{2}}}{\mu_2-\mu_1}$. Compare the tail behavior  of BT, where $P(x_1 > x_2) \approx e^{-(\mu_2-\mu_1)}$. Note the linear exponent for BT and quadratic for Gaussian. This means that BT is modelling a heavier-tailed distribution. BT also assumes that all values have the same $\sigma$, which may not be a good approximation.
Comment: I am glossing over the normalization step and the log/exp here I realize.


Save .RData in a different directory

Question: I load my files (.RData) from a particular folder, and i created a subfolder to save some samples and subsets. So, i want to save these elements in the subfolder, and they don't have the same name structure because i have multiple datasets (for example it cannot be sub1, sub2 etc, i have to write try1, full_sample, sub_2021 and so on).
I tried the following :
subsets_samples <- file.path <-("/Volumes/WD_BLACK/Merge/SAMPLES_SUBSETS")
fname <- file.path(subsets_samples, ".RData")
save(mydata, file=fname)

But obviously there is a problem with the saving part. My goal is to have something like :
save(mydata, file = "newname")

With the .RData format from fname that is put automatically.
I saw some answers with loops and so on but i don't really understand the process i'm sorry.
Thanks !
Answer: The problem with file.path is that it will place a separator (e.g., /¸) between each of the elements. So you would have to use paste0 in addition for the actual file name:
# If I understand you correctly, you want the iteration, like try1, full_sample, sub_2021 and so on in your file name. define them somewhere in your loop/script
iteration <- "full_sample"
fname <- file.path("Volumes", "WD_BLACK", "Merge", "SAMPLES_SUBSETS", paste0(iteration, ".Rds"))

Additionally, I would suggest to use saveRDS instead of save, since it is the appropriate function if you want to save just one object.
saveRDS(mydata, file = fname)

Comment: Well, for instance, if my subset is named "sample2021" in my global environment, i want it to be saved as "sample_2021.RData" (or .Rds) in my  "SAMPLES_SUBSETS" subfolder. In addition, i have multiple subsets from several datasets, so another subset can be named "supply_y1" in my GE and so on. Here is the result with your code :" Error : impossible to open the compressed file 'Volumes/WD_BLACK/MSI/SAMPLES_SUBSETSfull_sample.Rds', probable cause : 'No such file or directory'
Comment: Sorry, my bad. I never actually come across `file.path` before. Should work now.
Comment: Well the path and file name look correct now. Do you have such a folder on your computer?
Comment: When i do saveRDS(sample2021, file=fname), i get this : 'Volumes/WD_BLACK/MSI/SAMPLES_SUBSETS/full_sample.Rds',  : 'No such file or directory'  ; I don't understand the "iteration" line in fact, i just want to save what is in my global environment in the folder "SAMPLES_SUBSETS", but not with the name "full_sample". What i meant is that when i looked on Stack for this problem before to poste my message, i only saw this topic with datasets having the same structure : "sub1, sub2" and so on, which make their loops impossible to work for me has i have different structure each time.
Comment: Like, one my sample is "full_sample", the other one is "p1", the other one "suppliers_21" and so on, there is no common structure in the name.
Comment: `'No such file or directory'` means just that. the folder doesn't exist. You are creating the file so that part does not apply. You can call the file anything you want. But I don't understand where the names come from. You can define the name of the file in your script by assigning it to an object. I called the object iteration since it seems you want to iterate over some list of things.
Comment: I'll look better during the afternoon, because honestly i feel so dumb lol, it still doesn't work. I'll give you a feedback !
Comment: I would love to help you but I honestly don't get what your problem is. That said, describing your problems in a way so random people on stackoverflow.com can understand them is a skill one must practice to learn like many other things. You'll get better at it and better at solving your own problems if you keep spending time here.

Save .RData in a different directory


How does c++ (c++0x) handle implicitly typed variables?

Question: I am trying to learn how to use implicitly typed variables in c++.
Should i be using 'auto' from C++0x? If so how?
Can some one provide me with a simple example or a good tutorial on this?
Thank you.
Comment: Look for discussion of the `var` keyword in C#; a lot of the arguments for and against will be the same.
Comment: To provide a link, http://stackoverflow.com/questions/41479/use-of-var-keyword-in-c has discussion of the `var` keyword in C#.
Comment: Duplicate: http://stackoverflow.com/questions/2924901/how-do-i-use-auto-in-c-c0x
Answer: Check out this explanation from Bjarne Stroustrup himself: auto -- deduction of a type from an initializer.
Answer: auto x = f();

The type of x will be whatever f() returns.
Comment: @LoudNPossiblyRight: I have no idea what you are trying to say. (Whatever interpretation I can come up with seems wrong.)
Comment: also x has to be cast back into the type f() returns :)


How to convert non-annual interest rate to annual interest rate?

Question: If a student makes monthly deposits of 1,200 into an account with a nominal annual interest rate of 4.5% compounded monthly, will he have enough after 5 years to purchase a $105,000 property in cash?
I already have the solution.
I just want to understand why did he use the following to find the annual effective interest rate
i = 4.5% / 12 = 3.75%
and the Number of compounding periods he used is 60
Why he didn't use the following standard formula:
Comment: I don't know about the formula, but I would calculate it as
$$
(1.045)^{1/12} \approx 1.0036748
$$
So that's $0.36748~\%$ per month. Slightly different from $0.375~\%$ (after typo correction)
Comment: What was the calculation in whole? I'm not sure where your problem is.
Answer: If the nominal annual interest rate is 4.5%, then 4.5%/12 represents the monthly rate. The annual effective rate (compounding monthly) as per your formula is
$$(1+0.045/12)^{12}-1.$$

Now, as per the problem you mention, the timings are unclear, but let me give you a general formula (you can adjust accordingly to deal with different timings). If you make deposits of $x$ at the beginning of every period (starting immediately) and the per-period interest rate is $r$, observe that at the end of $t$ periods, your balance will be
$$x(1+r)^t+x(1+r)^{t-1}+...+x(1+r)=x(1+r)\frac{(1+r)^t-1}{r}.$$
Your setup is the case $x=1200,t=5*12=60,r=0.045/12.$