Why do we use PCA to speed up learning algorithms when we could just reduce the number of features?

Question: In a machine learning course, I learned that one common use of PCA (Principal Component Analysis) is to speed up other machine learning algorithms. For example, imagine you are training a logistic regression model. If you have a training set $(x^{(i)},y^{(i)})$ for i from 1 to n and it turns out the dimension of your vector x is very large (let's say a dimensions), you can use PCA to get a smaller dimension (let's say k dimensions) feature vector z. Then you can train your logistic regression model on the training set $(z^{(i)},y^{(i)})$ for i from 1 to n. Training this model will be faster because your feature vector has less dimensions.
However, I don't understand why you can't just reduce the dimension of your feature vector to k dimensions by just choosing k of your features at random and eliminating the rest.
The z vectors are linear combinations of your a feature vectors. Since the z vectors are confined to a k-dimensional surface, you can write the a-k eliminated feature values as a linear function of the k remaining feature values, and thus all the z's can be formed by linear combinations of your k features. So shouldn't a model trained on an training set with eliminated features have the same power as a model trained on a training set whose dimension was reduced by PCA? Does it just depend on the type of model and whether it relies on some sort of linear combination?
Comment: What's Polymerase Chain Reaction got to do with it?   :-)   --- In all seriousness, you should always spell out a term before using an abbreviation.
Comment: drop columns will lead to lose more information comparing to using PCA
Comment: Very related: http://stats.stackexchange.com/questions/141864.
Comment: You can view the eigenvectors obtained by PCA as new features, so PCA does allow reducing features - by recombining the ones we have into ones capturing more of the variance than the ones we started with.
Answer: Let's say you initially have $p$ features but this is too many so you want to actually fit your model on $d < p$ features. You could choose $d$ of your features and drop the rest. If $X$ is our feature matrix, this corresponds to using $XD$ where $D \in \{0,1\}^{p \times d}$ picks out exactly the columns of $X$ that we want to include. But this ignores all information in the other columns, so why not consider a more general dimension reduction $XV$ where $V \in \mathbb R^{p \times d}$? This is exactly what PCA does: we find the matrix $V$ such that $XV$ contains as much of the information in $X$ as possible. Not all linear combinations are created equally. Unless our $X$ matrix is so low rank that a random set of $d$ columns can (with high probability) span the column space of all $p$ columns we will certainly not be able to do just as well as with all $p$ features. Some information will be lost, and so it behooves us to lose as little information as possible. With PCA, the "information" that we're trying to avoid losing is the variation in the data.
As for why we restrict ourselves to linear transformations of the predictors, the whole point in this use-case is computation time. If we could do fancy non-linear dimension reduction on $X$ we could probably just fit the model on all of $X$ too. So PCA sits perfectly at the intersection of fast-to-compute and effective.
Comment: +1. It still makes sense to ask though, why variation in X (that PCA tries to retain) should be relevant for predicting Y... This is a related thread: http://stats.stackexchange.com/questions/141864.
Answer: PCA reduces features while preserving the variance/information in the original data. This helps with enabling computation while not losing the data's resemblance of reality.
Answer: PCA solution
First, beware when using PCA for this purpose. As I wrote in response to a related question PCA does not necessarily lead to selection of features that are informative for the regression you intend to do (see also Jolliffe 1982).
OP proposed solution
Now consider the proposed alternative mechanism: reduce the dimension of your feature vector to k dimensions by just choosing k of your features at random and eliminating the rest. Now in the problem statement we were asked to suppose that dimension of your vector x is very large. Let's call this dimension $p$
There are $pCk$ ways to choose $k$ predictors from a group of $p$. To give an example if $p=1000$ and we choose $k=5$ predictors from the dataset there would be $\approx 8.25 \times 10^{12}$ different models we would have to fit. And that's supposing we knew that $k=5$, and not $k=6$ etc etc. Put simply, it's not a problem you'd want to brute force in a large $p$ setting.
Suggested solution
To cope with regressions where $p$ is large a number of penalised regression strategies have been proposed. In particular the LASSO method will do dimension reduction while constructing a regression model by zeroing out the contribution from predictors that do not contribute enough to the model. There is a very clever algroithm (LARS) to fit the model efficiently.


How do I able to see the source code of this website www.samy.pl?

Question: This website (www.samy.pl) uses some technique so that no one can inspect the code (Ctrl+Shift+I) or view source (Ctrl+U). It can detect if you have opened the console or inspector the code automatically changes.
How this is possible?
Comment: Possible duplicate of [How to hide html source & disable right click and text copy?](http://stackoverflow.com/questions/6597224/how-to-hide-html-source-disable-right-click-and-text-copy)
Comment: @Adam No, it's not the same as that site.
Comment: Press F12 to open your console. Use IE11 to fool their script.
Comment: what about if i goto More Tools>Developers Tools or (ctrl+shift+i) option on chrome this website redirect to another page so that no one can see the code of homepage. How to do it then?
Comment: The site somehow detects when you open the console, and the source changes.
Comment: have you tried it @yezzz
Comment: yep I tried. Works, just IE11 is old and the dev tools are horrible to work with. Might want to try Edge. BTW, in chrome: press F12, then in dev tools shift+ctrl+M to get the mobile view... it will show some elements of the site.
Comment: Nope, sorry that was samy.pl/code. IE11 works though
Comment: Edge works too. Developer console and everything
Comment: How can I do this to my website?
Comment: Duplicate of [Detect when "Inspect Element" is open](http://stackoverflow.com/questions/42193700/detect-when-inspect-element-is-open/42194142#42194142)
Comment: Ctrl-U
Works every single time ¯\_(ツ)_/¯
Are you using Firefox?
It dosent work on chrome.
Comment: I found [this on Reddit](https://www.reddit.com/r/webdev/comments/67ch5v/how_is_this_site_done/).
Answer: You could always save the website.
In chrome open the menu -> more tools -> Save page as..
You should be able to run the html file in chrome, and use Dev Tools without it blocking you.
Comment: Yeah opening the file to see the code actually works!
Comment: It would still be blocked because the code is what is being downloaded and is what is blocking it. But you WILL be able to open it in a text editor to view the code.
Answer: There is a console "clearing" API that is mentioned in this answer.
Here's what I think is happening:
First, I think Google Chrome DevTools emits an event when it is open/launched.
The page author listens for this 'launch' event with the following handler logic:

Run a chrome.devtools.inspectedWindow.eval DevTools inspection API
Set document.body.innerHTML to the winky-face div
Also, a setInterval that executes a console.clear() before the console.log() of "no source for you".

I also inspected the EventListeners tab of Chrome DevTools while reading the "no source for you"-page.
Unfortunately, the two event listeners you can view don't seem to do anything useful:

one returns false
the other wires together forms and their associated submit actions.

Hope this helped
Comment: So they also clear their script tags, and remove any event listeners and/or callbacks that mess with the console.
Answer: hahaha, this is a good one, you should not expect less from samy!
Now I see why people think it is the wrong answer (the source get replaced if you do otherwise)
1.Navigate to site with inspector open there you end up with "No source for you! You found easter egg #7."
2.and then check the source to find Easter egg #2:
view-source:https://samy.pl

you will see:
/*
No source for you! Easter egg #2
*// AFTER SOME LONG WHITE SPACES HERE  /.source.replace(/.{7}/g,function(w){document.write(String.fromCharCode(parseInt(w.replace(/ /g,'0').replace(/   /g,'1'),2)))});

This is the way he loads the site(by injecting js), by replacing two type of white spaces for 1 and 0 and parsing that to char code by some regex and then that to string of course :D
https://de.wikipedia.org/wiki/Whitespace_(Programmiersprache)
If you paste that part to a js console, you will get the tags that loads the page.

ps. to see the rendered source, just load the page normaly and hit F12 after disabling javascript.
Have fun!
Comment: This is no longer valid.
Comment: "

 ...
Answer: Create an index.html somewhere, containing an . Load it in your browser and then open devtools. You will see the source inside the iframe.
Comment: this doesn't really add to the answer does it? its just shows how you could see the websites source, but doesn't answer how to recreate this?
Answer: Update:
Go to view-source:http://samy.pl which is the link of what the view source right click brings you to. It only works on Chrome I think.
If you are using chrome, go to the menu at the top right. Then go to more tools, and click developer tools.
Answer: After a lot of tinkering and speculating together, my roommate succeeded in the end with quite a naïve approach.
Simply open the Chrome devtools and copy out the source before it gets replaced. Giving Chrome processes low priority and doing some heavy-lifting with your CPU (compilation or other) helps.
Chop, chop.
Answer: Ah! I have found an extremely easy way to do it.
1. Open Firefox (Chrome only shows you part of it).
2. In the URL bar, type in javascript:alert(document.documentElement.outerHTML);
3. Press Enter.
4. Voilà!
Answer: New Way to find the source code! But its complicated!
Open a new tab.
Open Devtools (Inspect Tab)
Click On Toggle Device toolbar (the mobile like icon)

Click on "No Throttling" to slow down the page loading time

Select "Low - Tier Mobile"

Open the page!
When you see the icons appear on the page, you can now see the source code! :)
Answer: sorry very late
but i think i have the answer
He used this node package
to which he has contributed and forked too
the package docs say that it wont work if the devtools are undocked
Answer: You can simply use the developer tools in almost all the browsers. Just Press F12 and the developer tools section will be there.
Thanks
Comment: That might also be blocked by the code on the website because if right click is blocked, and those keyboard commands are blocked, then F12 is probably blocked too.
Comment: F12 does work, it brings the dev tools, but the site detects this and loads other content. Sites have limited control over browsers. Why do you think they're limiting js xss.


Como dar um UPDATE trocando \\ por \?

Question: Eu estou tentando realizar um UPDATE para trocar o \\ por \ , como no SQL abaixo:
UPDATE `post` SET `descricao`= replace(descricao, '\\','\');

Me falaram que o MySQL não aceita expressão regular no UPDATE e que eu deveria usar o REPLACE, mas ele da erro de sintaxe. Enfim, como eu resolvo isso?
Answer: A barra invertida tem significado especial nas strings. Ela indica que o próximo caractere é especial e não deve ser tratado como um caractere normal. Ao mesmo tempo ele pega algum caractere especial e transforma em caractere normal. Quais são os caracteres especiais?
Obviamente que as aspas são especiais para encerrar a string, portanto se você quiser que aspas sejam parte do texto a única forma de fazer isso é colocando uma barra invertida antes para dizer que é um caractere normal e não está querendo encerrar o texto.
Também se você quiser que uma barra invertida seja um caractere normal precisa colocar uma barra invertida antes dela para que a primeira tenha o significado especial e a segunda seja o caractere normal.
Então quando coloca só uma barra invertida e depois vem aspas ela não está encerrando o texto (que é necessário) e causa o erro de sintaxe. Para funcionar precisa colocar a barra invertida duas vezes, assim só uma será considerada a barra como caractere e aí fecha aspas normalmente sem erro.
No outro não há erro de sintaxe mas há erro semântico do que deseja. Como tem duas barras só uma será considerada pelo mesmo motivo acima. Então se quer representar duas barras invertidas como caracteres normais precisa colocar 4 delas assim as duas primeiras formam uma barra real e as duas seguintes formam a segunda barra real. Assim:
UPDATE post SET descricao = replace(descricao, '\\\\','\\');

Coloquei no GitHub para referência futura.
Não estou dizendo que isso resolve satisfatoriamente o que deseja porque nem tem detalhes e não é o foco da pergunta.
Comment: Muito obrigado pela a sua explicação e ajuda, além de me esclarecer ela me ajudou muito!!!
Comment: @j você sabe que pode votar em tudo no site também, além de aceitar respostas dadas em perguntas suas?
Answer: tente isso.
o replace ele faz o seguinte, pega o primeiro parâmetro e troca pelo segundo parâmetro informado. ou seja onde tiver \ vai trocar por .
UPDATE post SET descricao = replace(descricao, '\','\');

Como dar um UPDATE trocando \\ por \?


Given a sequence of head to tail vectors forming a closed loop, how can I determine if they form a knot?

Question: Consider if we have some sequence of vectors placed head to tail which form a closed loop. How can one determine whether they form a loop?
We assume that it is given that the vectors close, that is their vector sum is zero, and that the piecewise curve they form is non-intersecting. If it helps, we can also assume they have the same length. An example is shown here:
https://i.stack.imgur.com/Rp26t.jpg
An example of how such configurations might be generated is through self-avoiding random walks on a lattice, where each vector represents a single "step".
EDIT:
Okay so clearly this problem is not nearly as trivial as I had first thought. Some more details:
I am considering loops that are generated by a sequence of edges on a lattice. For example, we could take a cubic lattice, although in my case the lattice is non-Bravais. By construction, the loop does not self-intersect, as would be the case for a self-avoiding random walk. My idea was simply to determine whether or not the loop is the unknot or not (I use the loops for other purposes but thought it might be an interesting research thread to pursue to study their knotted-ness).
Comment: I'm voting to close this question because this is a well known currently active research topic.  Brute force algorithms exist, however it remains unknown whether the problem can be solved in polynomial time.
Comment: How do you define a "knot" in this context?  Are you just asking if they form a knot different than the unknot (*which is still technically an example of a knot*)?  This is one of the classic problems that knot theory tries to answer, the final result being *it is hard to be sure*.
Comment: @JMoravitz thanks for pointing that out, of course in this context I do mean a knot other than the unknot.
Comment: I haven't heard of a result showing it is undecidable, but then again the last knot theory book I read I had only properly made it through the front half and it is probably 20 years old at this point (Adams, The Knot Book).  I could believe that there have been significant advancements in that time.  The last results I had read were the existence of various knot invariant polynomials that one could calculate, and if the polynomials are different between two knots then you can be sure the knots themselves are different.  If the polynomials are the same however that isn't enough to conclude.
Comment: Related: [Wikipedia - Unknotting Problem](https://en.wikipedia.org/wiki/Unknotting_problem).
Comment: @JMoravitz It's not expected to be polynomial time, and I don't think "brute force" really explains the known algorithms. One way is to find a homomorphism of the fundamental group of the knot complement to $SL_2(\mathbb{C})$ whose image is not just diagonal matrices, and one exists if and only if the knot is knotted. This amounts to solving a system of polynomial equations. Practically speaking, computing the Jones polynomial is sufficient, especially if the knot in question has at most 23 crossings, since the conjecture that it is an unknot detector has been verified at least to that point.
Answer: Basically what you are asking is for an overview of the theory of knots, which is rather too broad.
This wikipedia page contains a brief overview of what's known about the unknotting problem, which is the problem of algorithmically determining whether a given knot is an unknot. In short, it is algorithmically decidable, and the decision problem is known to be NP-complete.
For a more in depth look at knot theory, look at this other wikipedia page, or better pick up a good textbook such as Rolfsen's. You seem to know what the correct tags are, i.e. group-theory, knot-invariants, etc., and you'll find lots of information about those topics of you follow the links given..
Comment: While the unknotting decision problem in an arbitrary 3-manifold is $\text{NP}$-complete (Agol-Hass-Thurston 2006), in $S^3$ it's in $\text{NP}\cap\text{co-NP}$, a class that's unknown whether it intersects $\text{NP}$-hard. I think Lackenby has a quasi-polynomial-time algorithm for unknot detection in $S^3$.
Answer: Practically speaking, this is not quite so bad as others are making it out to be.  There is a conjecture with much computational evidence that the Jones polynomial is an unknot detector (it's true, at least, for knot diagrams with up to 23 crossings).  The Jones polynomial isn't so difficult to compute, and, assuming the conjecture, the given knot is the unknot if and only if the Jones polynomial is $1$.
For knots on a lattice, what you have to do is find a knot diagram.  If you choose a random projection direction, almost surely the projection will be in general position: no two vertices project to the same point, no vertex projects onto the interior of any segment, and the projections of any two segments are transverse, i.e., when they intersect, they are not collinear.  While extremely unlikely, if you find the projection is not in general position, you can try projecting in another random direction.
From here, for each pair of intersecting projected segments, you figure out which is 'over' and which is 'under'.  It's convenient to then store the result in oriented PD format (see the Knot Atlas wiki and maybe this page about representing knots on computers).  If you know how to read Mathematica, I once wrote a little bit about computing Jones polynomials in this answer: Software for computing virtual knot invariants
(I'm not just speaking theoretically that it's plausible to go through all this.  A colleague of mine did the very same calculations for studying whether certain orbits in a discrete vector field in a triangulation of $S^3$ might be knotted.)
If you want to be absolutely sure your knot is unknotted, you can use Dynnikov's algorithm, for example.  It's very slow, but it's not so hard to implement.
Comment: thanks! This is very useful information, and I use MMA all the time
Comment: @Kai By the way, you do not actually need to keep track of orientations, and it's fine just using `X` objects for crossings and `P` objects for paths.  Then you calculate `kb` for the Kauffman bracket, and if it is $(-A^3)^k$ for some $k\in\mathbb{Z}$, your knot is conjectured to be unknotted.  The orientations are used to pin down a normalization using the writhe of the diagram.
Comment: thanks, I am not sure if this project is going to be worth the time to dig into right now, but if I do I will definitely be coming back to this answer!


Comma inside MySQL query

Question: $sql_query_posts = "SELECT * FROM `posts`";

$sql_form_permission = mysql_query("SELECT * FROM `integrations` WHERE `rank_id`='$user_level' AND `mode`='form_per'");
    if ( mysql_num_rows($sql_form_permissions) > 0 ) {

Now the part I'm struggling with:
   $sql_query_posts .= " IN (";
    while ( $row_form_permissions = mysql_fetch_assoc($sql_form_permissions) ) {
    $sql_query_posts .= $row_form_permissions['form_id'] . ",";
    }
    $sql_query_posts .= ")";
    }

I want the output to be something like:
SELECT * FROM `posts` IN (4, 3, 2)

But the code above gives an output like:
SELECT * FROM `posts` IN (4, 3, 2,)

So I have to get rid of the comma at the end. How can I make sure the script disables the comma at the last row.
Answer: I would use implode for that:
$post_ids = array();
while ( $row_form_permissions = mysql_fetch_assoc($sql_form_permissions) ) {
    $post_ids[] = $row_form_permissions['form_id'];
}

$sql_query_posts .= " IN (" . implode(",", $post_ids) . ")";

Comment: I just saw that you posted your answer before MaX, so I'll comment on this one also. I agree implode() should be used rather than substring(), but the question was asked in such a way that it seemed like substring() was what was asked for. I upvoted your answer to reflect the importance of using implode() over substr() or trim().
Answer: Use $sql_query_posts = substr($sql_query_posts, 0, -1); like this:
$sql_query_posts .= " IN (";
while ( $row_form_permissions = mysql_fetch_assoc($sql_form_permissions) ) {
    $sql_query_posts .= $row_form_permissions['form_id'] . ",";
}
$sql_query_posts = substr($sql_query_posts, 0, -1);
$sql_query_posts .= ")";

Just to address the case when you might end up with no records fetched (an empty array), it would maybe be wiser to use implode(), like this (I always use it like that):
$ins = array();
while ( $row_form_permissions = mysql_fetch_assoc($sql_form_permissions) ) {
    $ins[] = $row_form_permissions['form_id'];
}
$sql_query_posts .= sprintf(' IN (%s)', implode(',', $ins));

Comment: What if no record was fetched?
Comment: I agree that at least a check should be made for empty result set and I updated the answer accordingly, using implode().
Answer: A prettier way, in my opinion, than the suggested is:
$forms = array();
while ( $row_form_permissions = mysql_fetch_assoc($sql_form_permissions) )
  $forms[] = $row_form_permissions['form_id'];

$sql_query_posts .= " IN (".implode(",",$forms).")";

Comment: I agree implode() should be used rather than substring(), but the question was asked in such a way that it seemed like substring() was what was asked for. I upvoted your answer to reflect the importance of using implode() over substr() or trim().
Answer: You could just use substr to get the comma off. Like this:
$sql_query_posts .= " IN (";
while ( $row_form_permissions = mysql_fetch_assoc($sql_form_permissions) ) {
$sql_query_posts .= $row_form_permissions['form_id'] . ",";
}
$sql_query_posts = substr($sql_query_posts, 0, -1); // <- this is the new line
$sql_query_posts .= ")";
}

Answer: As an alternate to substr you can also use trim to get rid of the comma.
$sql_query_posts = trim($sql_query_posts, ',') . ')';

Trim is slightly safer than substr because you can limit it to an exact character or set of characters.
Answer: I would use trim or (rtrim) function is such situations (http://php.net/manual/en/function.trim.php)
You give the string to it, and specify the extra character using its second parameter and it cleans it up for you.

Comma inside MySQL query


Prove that $\mathbb R$ is not a nontrivial disjoint union of open subsets.

Question: I want to show that if we write $\mathbb R$ as a disjoint union of more than one subset of $\mathbb R$, then at least one of the subsets is not open in $\mathbb R$.
I'm sure I can prove this, but not very cleanly, by contradiction: if $\mathbb R$ were a nontrivial disjoint union of open subsets, then we could decompose each subset into its maximally-sized intervals of the form $(-\infty, a)$ or $(b, +\infty)$ or $(a, b)$ with $a < b$; then there exist two neighboring intervals in the overall decomposition, and the neighboring intervals do not contain their common boundary point, which would contradict the disjoint union covering $\mathbb R$.
Is there a better/cleaner/more elegant proof?
Comment: You might want to take a look at this question: http://math.stackexchange.com/q/1480796/9464
Comment: Your proof is not actually correct: there might be infinitely many maximal intervals which are densely ordered, so there are no "neighboring" intervals.
Comment: See [Any open subset of R is a at most countable union of disjoint open intervals](http://math.stackexchange.com/questions/318299/any-open-subset-of-bbb-r-is-a-at-most-countable-union-of-disjoint-open-interv).
Comment: @user251257 I'm not sure your point. I only said that I can decompose open subsets into disjoint unions of open intervals. Did I say something blatantly wrong? :o
Comment: @EricWofsey that doesn't sound right... any open interval has a positive (or infinite) diameter, so then a set of disjoint intervals must be countable, and thus we can find the "neighboring" interval of any interval. What's wrong with this thought?
Comment: @feralin ah. Not exactly. My comment is wrong. Sorry. But see Eric's comment.
Comment: @Jack thanks for the link! That makes me immediately think of the following proof: since $\mathbb R$ is connected, if $\mathbb R = A \sqcup B$ then $A$ or $B$ is not open. This is much simpler than what I was thinking!
Comment: @feralin aren't you trying to prove that $\mathbb R$ is connected?
Comment: @user251257 no, I'm trying to prove that $\mathbb R$ is not a nontrivial disjoint union of open subsets. This ends up just being a simple consequence of the connectedness of $\mathbb R$, but the fact that $\mathbb R$ is connected is not my end result.
Comment: @feralin: The rationals are countable, but there are no "neighboring" rationals.
Answer: If this were possible, then it would be possible to write $\mathbb{R}$ as the union of two nonempty disjoint open subsets (since the union of any arbitrary collection of open sets is open).  This, however, is impossible.  Suppose $\mathbb{R} = V \cup W$, where $V$ and $W$ are nonempty open sets and $V \cap W = \emptyset$.
Now consider an element $x \in \overline{V} \setminus V$.  Can we have $x \in W$?
Comment: More generally, a topologically space $X$ is connected $\iff$ the only clopen sets are $\emptyset$ and $X$.  This is fairly straightforward; try proving it yourself first, but there's also discussion here: http://math.stackexchange.com/questions/361387/how-to-prove-that-a-topological-space-is-connected-iff-it-has-exactly-two-clopen
Comment: How can I show that $\bar V \setminus V$ is not empty? I think I would need to prove that in the reals, only the empty set and the reals are clopen...
Comment: Hint : Try to prove that $\mathbb{R}$ is connected (using a bounded interval, for instance) first and combine the ideas of the discussion (mainly  Kaj Hansen's comment). This will serve you further for all spaces connected by arcs
Answer: (1) All intervals in $\mathbb{R}$ are connected.
$$\mathbb{R} = \bigcup_{n \in \mathbb{Z}} (-n,n)$$
(2) Consider the diffeomorphism;
$$f: (-a,a) \to \mathbb{R}; \ x \mapsto \frac{ax}{\sqrt{a^2-x^2}}$$
Since all intervals of $\mathbb{R}$ are connected then $(-a,a)$ is connected. Moreover, since $f$ is continuous then $f(-a,a) = \mathbb{R}$ is connected.
Answer: The statement is equivalent to proving that $\mathbb{R}$ is not the union of two disjoint nonempty open sets. Indeed, if $\mathbb{R}$ is the disjoint union of nonempty open sets (more than one), then taking one of them and the union of the others, you obtain $\mathbb{R}$ as the disjoint union of two nonempty open sets.
Suppose $\mathbb{R}=A\cup B$, with $A$ and $B$ disjoint, open and nonempty. Since the complements of open sets are closed, $A$ and $B$ are also closed.
Let $a\in A$ and $b\in B$; without loss of generality, $a<b$.
Consider $S=\{x\in\mathbb{R}:a\ge x, [a,x]\subseteq A\}$. The set $S$ is upper bounded (by $b$) and not empty (because $a\in S$). Therefore $s=\sup S$ exists. Since $A$ is closed and $S\subseteq A$, we have that $s\in A$, because the supremum of a set belongs to its closure.
I claim that $s\in S$. Indeed, if $[a,s]\not\subseteq A$, then there is $t\in [a,s]\cap B$ and $a<t\le s$. Since $B$ is open, there is $\varepsilon>0$ such that $(t-\varepsilon,t+\varepsilon)\subseteq B$. Note that $t-\varepsilon>a$, because $a\in A$. On the other hand, since $s=\sup S$, there is $x\in S$ with $t-\varepsilon<x\le s$, a contradiction because $[a,x]\subseteq A$.
Since $A$ is open, there exists $\delta>0$ such that $(s-\delta,s+\delta)\subseteq A$. But then $[a,s+\delta/2]\subseteq A$, contradicting the fact that $s=\sup S$.

This is the proof that $\mathbb{R}$ is connected. A straightforward modification shows that every interval in $\mathbb{R}$ is connected.


How to define one-to-many connection between a same model through another model

Question: I want to define one-to-many relationship as follows;

User has one introducer
User has many newcomers(who is introduced by the user)
Use "Introduction" model instead of adding a column to users table.

My table and model definition is as follows;
DB Scheme:
create_table "introductions", force: true do |t|
  t.integer "introducer_id"
  t.integer "newcomer_id"
  t.datetime "created_at"
  t.datetime "updated_at"

User model:
class User < ActiveRecord::Base
  has_many :introductions, foreign_key: :introducer_id
  has_many :newcomers, through: :introductions, source: :newcomer
  belongs_to :introduction, foreign_key: :newcomer_id
  belongs_to :introducer
end

Introduction model:
class Introduction < ActiveRecord::Base
  belongs_to :introducer, class_name: 'User'
  belongs_to :newcomer, class_name: 'User'
end

This works fine:
user1.newcomers.push user2

but,
user2.introducer
# => nil

How can I define belongs_to relationship correctly?
Comment: `belongs_to :introducer` in your `User` model should be `belongs_to :introducer, class_name: 'User'`
Answer: At first, I think should have used has_one instead of belongs_to.
then, thanks to @Paven, my solution becomes,
User < ActiveRecord::Base
  has_many :introductions, foreign_key: :introducer_id
  has_many :newcomers, through: :introductions, source: :newcomer

  has_one :introduction, foreign_key: :newcomer_id
  has_one :introducer, through: :introduction, class_name: 'User'
end


How to add a css class to the nodes in the front page according to taxonomy of each node in drupal 7?

Question: in the front page, I have displayed multiple nodes (article) in the teaser view, each article has taxonomy.
I want to theme each node teaser according to the taxonomy it has, I think it can be done in the theme_preprocess_html(&$variables, $hook), for example
function theme_preprocess_html(&$variables, $hook) {
  if ($variables['is_front']) {

    // something here, but I don't know what to do
  }
}

the first step is how to find all the nodes in the frontpage, the second is to get the taxonomy of each node, the third is to add the specific css class to each node according to the taxonomy of each node
Does any have suggestions?
Comment: Have you considered using drupal.org/project/views, it allows you to add a style to each row based on a property of a node.
Answer: Supposing that "field_tags" is the taxonomy field for which you want to use the taxonomy term values, you could use the following code.
function theme_preprocess_html(&$variables, $hook) {
  if ($variables['is_front']) {
    $terms = array();

    foreach ($variables['page']['content']['system_main']['nodes'] as $nid => $array) {
      $items = field_get_items('node', $array['#node'], 'field_tags');
      foreach ($items as $tid) {
        if (!isset($terms[$tid]) && ($term = taxonomy_term_load($tid))) {
          $terms[$tid] = drupal_html_class($term->name);
        }
      }
    }

    if (!empty($terms)) {
      foreach ($terms as $class_name) {
        $variables['classes_array'][] = $class_name;
      }
    }
  }
}

Comment: It is shown in the page listing the fields used by a content type.
Comment: I am trying to understand what "my" taxonomy field is ("field_tags") in this context so I can change it, where can I see it? Or is there a way to avoid manually defining this in the code above? thanks
Comment: Thanks! When I add the code and change theme to mytheme as well as changing the taxonomy field it brakes the page and shows following error: `Warning: Illegal offset type in isset or empty in MYTHEME_preprocess_html() (line 67 of ...\sites\all\themes\MYTHEME\template.php).` which points to this line: `if (!isset($terms[$tid]) && ($term = taxonomy_term_load($tid))) {` do you have an idea why this error is shown?
Answer: an easier way could be with views - you can create / enable a view that will replace the homepage. there you can add the taxo as a hidden field and use it in the class setting of the row


Cannot Visit .Onion Website with CURL

Question: I'm using a class that Connects into Tor Proxy Port I can visit http://check.torproject.org and says that I'm using Tor Browser, But when I'm trying to visit a .onion Website I get blank page.
I used this class
Comment: You should definitely add some more details, including code. Please check here on how to ask questions: https://stackoverflow.com/help/how-to-ask
Answer: It fails to load .onion addresses because it uses the curl proxy type of CURLPROXY_SOCKS5 instead of CURLPROXY_SOCKS5_HOSTNAME which would use the Tor SOCKS proxy to also resolve DNS addresses.
You would need to change:
curl_setopt( $this->ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5 );

to:
curl_setopt( $this->ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME );

Beyond that, the example class doesn't give you the ability to customize your requests at all.
I have written a Tor cURL wrapper class called TorCurlWrapper (examples here and here) which is part of a larger Tor PHP library called TorUtils.  It wraps curl in an easy to use set of functions and ensures that requests can only be made through Tor and takes care of setting the proxy type properly and will fail if it is not supported (really old curl versions).
You can download it standalone or install using Composer by running composer require dapphp/torutils.

Cannot Visit .Onion Website with CURL


How to switch template for search results without using views and display modes?

Question: I want to show my Drupal 10 Solr search results in either grid view or list view.
I set this up using Views and Display Modes, and I have icons on the page which simply link to the URLs of the different display modes.
However I am using Facets with my search, and using the above approach requires setting up duplicate Facet Blocks for each display mode. This is inefficient and difficult to maintain.
Is there a more efficient way of re-rendering the same page but with a different set of templates for displaying the search results?
Answer: There is a solution for this limitation of Facets applying to only one display in one view, discussed here:
https://www.drupal.org/project/facets/issues/2826650
The workaround consists of altering the search query event and overriding the checkbox widget for the facet blocks.


Difficulty to find the Standard Error in a simple hypothesis testing

Question: I'm having some difficulty to solve a very simple hypothesis testing problem.
$$\begin{array}{c|c|c|}
 & \text{X} & \text{Y} \\ \hline
\text{1} & 1 & 1 \\ \hline
\text{2} & 3 & 4 \\ \hline
\text{3} & 5 & 4 \\ \hline
\text{4} & 6 & 4 \\ \hline
\text{5} & 9 & 9 \\ \hline
\text{mean} & 4,8 & 4,4 \\ \hline
\end{array}$$
I'm running a linear regression with this table in a software and verifying that:
beta = 0,8804
Standard Error = 0,2057
I want to know how to do this by hand.. I know that beta comes from:
$$
\beta = \frac{\sum xiyi}{\sum xi^2}
$$
But I have no idea how to find the Standard Error without the software. Anyone knows where the 0,2057 comes from?
Comment: I think you mean "linear regression" instead of "linear test". Is that correct?
Comment: @BradS. yes! already edited.
Answer: Let $X_k$ be the $k$th sample from data $X$, where $k\in1,2,3,4,5$, so that
$X_1=1,X_2=3,X_3=5,X_4=6,X_5=9$
Let $Y_k$ be the $k$th sample from data $Y$, where $k\in1,2,3,4,5$, so that
$Y_1=1,Y_2=4,Y_3=4,Y_4=4,Y_5=9$
The standard error is the square root of the variance of the least squares estimate of the gradient, which is $beta$ in your case, for a linear model.
It is calculated using the following formula:
$\large SE = \sqrt{\frac{\frac{1}{n-2}\sum_{k=1}^ne_k^2}{\sum_{k=1}^N(X_k-\bar{X})^2}}$
where $n$ is the number of samples (which is $5$ in our case), $e(k)$ is the residual error between $Y_k$ and $\hat{Y_k}$, where $\hat{Y_k}$ is the value we would have obtained from $X_k$ using the linear model, and $\bar{X}$ is the mean of $X$.
$e(k)=Y_k-\hat{Y_k}$
where $\hat{Y_k}$ is calculated as follows:-
$\hat{Y_k}=beta\times X_k+ alpha$
$beta$ is the estimated gradient, $0.8802$, $alpha$ is the estimated intercept $0.1739$, according to my free online regression calculator (at http://scistatcalc.blogspot.co.uk/2013/10/web-app-testing.html)
We thus obtain
$\hat{Y}=[1.0543,2.8151,4.5759,5.4563,8.0975]$
$\sum_{k=1}^5e_k^2=(1-1.0543)^2+(4-2.8151)^2+(4-4.5759)^2+(4-5.4563)^2+(9-8.0975)^2\\=4.6739$
and $\bar{X}=24/5=4.8$
$\sum_{k=1}^5(X_k-\bar{X})^2=(1-4.8)^2+(3-4.8)^2+(5-4.8)^2+(6-4.8)^2+(9-4.8)^2\\=36.8$
so that
$\large SE = \sqrt{\frac{(\frac{1}{5-2})4.6739}{36.8}}=0.20576$
Answer: If your observations are $(x_i, y_i)$ for $i = 1, 2, \ldots, n$, then the correct least squares point estimate for the slope $\beta$ should be $$\hat \beta = \frac{\sum x_i y_i - \frac{1}{n} \sum x_i \sum y_i}{\sum x_i^2 - \frac{1}{n}\left(\sum x_i\right)^2}.$$  From this, we can obtain the variance of $\hat \beta$:  $${\rm Var}[\hat\beta] = \frac{\sigma^2}{\sum (x_i - \bar x)^2},$$ where $\sigma^2$ is the variance of the IID normal errors.  Since this is an unknown parameter, we must estimate it:  $$\begin{align*} \hat \sigma &= \frac{1}{n-2} \sum_{i=1}^n (y_i - \hat y_i)^2 \\ &= \frac{1}{n-2} \left( \sum y_i^2 - \frac{1}{n} \Bigl( \sum y_i \Bigr)^2 - \hat\beta \Bigl( \sum x_i y_i - \frac{1}{n} \sum x_i \sum y_i \Bigr) \right).\end{align*}$$  Thus the estimated variance of $\hat \beta$ is $$\widehat{\rm Var}[\hat\beta] = \frac{\hat\sigma^2}{\sum (x_i - \bar x)^2}.$$  The square root gives the standard error, and for the hypothesis $$H_0 : \beta = 0, \quad H_a : \beta \ne 0,$$ the distribution of the test statistic $$T = \frac{\hat \beta - 0}{{\rm s.e.}(\hat\beta)}$$ is Student's $t$-distributed with $\nu = n-2$ degrees of freedom.


List all maximal and minimal elements of the partial order R = {(a,a), (b,b), (c,c), (a,c)}

Question: I know what the definitions of maximal and minimal elements are but I'm not sure how to apply them in this case. Any help would be great.
Answer: If you draw a picture of this partial order, you get this:
<code>            c
            |
            |     b
            |
            a
</code>
The only element with a strictly smaller element is $c$, so $c$ is the only non-minimal element; $a$ and $b$ are minimal, because there is no element strictly smaller than either of them.
Can you tell now what the maximal elements are?
Comment: I'm still a bit confused about why the picture looks like that.
Comment: Let me write $x\preceq y$ to mean that $\langle x,y\rangle\in R$; it helps the intuition. The ordered pairs $\langle a,a\rangle,\langle b,b\rangle$, and $\langle c,c\rangle$ don't really tell you anything; they're just $a\preceq a,b\preceq b$, and $c\preceq c$, which are required since the partial order must be reflexive. The only pair that actually tells you something non-trivial is $\langle a,c\rangle$, or $a\preceq c$; and since $a\ne c$, this actually tells you that $a\prec c$. In this partial order $a$ is smaller than $c$, and that's the only non-trivial ordering, just as in the diagram.
Comment: Right. That clears it up. Thank you. So then a and b are the minimal elements and c is the only maximal element?
Comment: Sorry, I meant that a and b are minimal and c and b are maximal. Is that right?
Comment: @zeqof: Yes, that's right: $b$ is both, $a$ is minimal, and $c$ is maximal.
Comment: @zeqof: You're welcome.


How to backup my Node.js setup when upgrading

Question: I want to upgrade Node.js on Ubuntu, but I want to make sure that if anything goes wrong I could revert back my source code, installed modules and node binary to the working setup. How to achieve that?
Answer: You can install any version of node.js just go to downloads. Go into Other Releases and download what version you want with respect to your OS and 32-bit or 64-bit.
Example:

node-vX.XX.X-OS-x64.tar.gz (64-bit)
node-vX.XX.X-OS-x86.tar.gz (32-bit)

Then, follow the instruction on the github.com. Building and installing Node.js
Another way is to use NVM, the Node Version Manager (it works in a very similar way to rvm for ruby). This allows to install and manage multiple versions of node. Here's a snippet (source):
Usage:
nvm install <version>       Download and install a <version>
nvm use <version>           Modify PATH to use <version>
nvm ls                      List versions (installed versions are blue)

You can also check this n. A Simple flavour of node binary management.

Update (as per comments):
If you have installed the nvm after node.js. You can check already installed by using nvm ls which lists already installed version(s) ready to use.

amolkulkarni@ubuntu:~$ nvm ls
  current: v0.10.18
Comment: I didn't see anything in the docs about this, but perhaps you can elaborate (thanks btw, i didn't know about NVM).  If you already have an existing node package say for v 0.8.17.  You then install NVM; would it be necessary to nvm install that same version again, or does NVM automagically detect this?
Comment: @thtsigma You can check it by using `nvm ls` command which lists all the installed version(s) available for use. So already installed version will appear in that. Happy coding ;)
Answer: You should use Node Version Manager https://github.com/creationix/nvm


Tokyo Tyrant ulog / update log management

Question: I'm testing Tokyo Tyrant in a master-master setup and have found the ulog grows out of control and locks up the disk.
At first I found the -ulim option useful and limited the logfile size, however it simply rolls over to a new log, leaving the old ones to clutter up the partition.
I suppose I'll write a shell script that will delete ulogs older than X, once I find out how far back Tokyo Tyrant needs in the update log in order to failover.
Does anyone have any experience with this Tokyo Tyrant?  Do you have a feel (acknowledging that every install is different based on what is being stored) for the optimal ulog size vs how far back a Tokyo Tyrant instance needs to look in the ulog to assume master status?
Thanks,
nathan
Answer: Just to follow up, below is from Mikio Hirabayashi's (TT developer) reply to a similarly worded e-mail:

If you can confirm only one slave,
  which is another part of dual master,
  accesses the master server, please
  query the delay time to the slave by
  the command "tcrmgr inform -st ..."
  and you can determine which file can
  be removed.

Running that command will allow you to see how far back a slave is behind a master.  Once you know that you can spend some time finding the right ulog turnover size and haw many ulog files back you can trash and feel safe.  Probably best to do it under a load that simulates a heavy day on your Tokyo Tyrant key/value databases etc..
I shamelessly ripped off a script from stackoverflow:

> #!/bin/bash
>
> # Deletes all but the newest 5 files to keep Tokyo Tyrant ulogs from
> killing the disk.
> logdir='/path/to/ttserver/ulog/'
> mydir=`ls -t $logdir` it=1
>
> for file in $mydir
>     do
>         if [ $it -gt 5 ]
>         then
>             echo file $it will be deleted: $logdir$file
>             #rm -rf $file
>         fi
>         it=$((it+1))
>     done

@kubanskamac's answer was correct in abstract, but Mikio gives the command to start optimization.
Answer: FYI, I've written a ulog management script that takes the replication delay into account:
http://conigliaro.org/2010/04/28/tokyo-tyrant-update-log-ulog-management/
Answer: Disclaimer: this is the first time I've even heard about Tokyo Tyrant. I just see some familiar patterns looking at the docs.
In the transactional systems (e.g. databases) I know, attention is paid to two types of unexpected events:

power failure - when you lose for some reason contents of RAM cache, but after restart you are still able to access disk files
disk failure - you are no longer able to access data from your disk files; files are lost or contain only garbage; you have to restore from backup

Each log passes usually through three stages of existence:

Active log that has been just
created with very fresh transactions; the data it contains have not yet been saved to data files; you absolutely need this to restore consistency of data files in case of power failure
After data are committed (written to database files) the log is now called committed log; data are on disk in two different formats/places - in this log and in database file; you do not need this log in case of power failure, but you need it in case of disk failure - after restoring database file (from previous week?), you can re-apply all transactions from consecutive commited+active logs in the same chronological sequence to arrive at fresh copy of data; if you have lost latest logs, you've lost recent transactions, but at least you still have pretty fresh and completely consistent database
After the database file is backed up (consistently copied to safe place), your log becomes archive log. You do not need it to protect against either failure.

I have no idea how to figure out which of your ulogs are committed by Tokyo Tyrant. But maybe this general outline will help.


Make flex-wrap ignore certain elements

Question: I have a 4-column flex-box layout with flex-wrap enabled:

When I resize the page and the edge of page touches the
 header, the blue column correctly wraps down:

What I now want to do is to add let's say 25 images next to each other inside a div that overflows on x, brings up a scrollbar, but doesn't wrap the blue column downwards unless the other content gets too small

So the scrollbar comes up correctly, but as soon as it does, the blue column tries to expand and wraps down. I want the image reel to always be smaller than the blue column, but even if i set it to 50% width, it still expands the blue column which then wraps.
Is there a way to only make the blue column wrap if the content is smaller than the
 tag and totally ignore the image reel?
I've made a codepen snippet for it and also added sample code below:

* {
  box-sizing: border-box;
}

body {
  margin: 0;
  padding: 0;
  color: white;
}

.green {
  background-color: green;
}

.red {
  background-color: red;
}

.blue {
  background-color: blue;
}

.black {
  background-color: black;
}

.display\:flex {
  display: flex;
}

.flex-direction\:column {
  flex-direction: column;
}

.flex-direction\:row {
  flex-direction: row;
}

.flex-wrap\:wrap {
  flex-wrap: wrap;
}

.flex-grow\:100 {
  flex-grow: 100;
}

.flex-grow\:2 {
  flex-grow: 2;
}

.flex-grow\:1 {
  flex-grow: 1;
}

.flex-grow\:0 {
  flex-grow: 0;
}

.flex-shrink\:1 {
  flex-shrink: 1;
}

.height\:100vh {
  height: 100vh;
}

.reel {
  display: flex;
  overflow-x: auto;
}

.reel>img {
  height: auto;
  max-width: 25%;
}

.reel>*+* {
  margin-left: 10px;
}

    Column1

    Column2

    <input>

    <input>

    Column3

    <input>

    <input>

      This is some long header text in Column4

Comment: I have seen this before, but why use `.class\:value` in your CSS rules? Added value? In the mean time looking into your problem...
Comment: Don't allow your images to shrink `.reel>img { flex-shrink: 0 }`, that'll show the scrollbar. Not sure when the blue box should wrap.
Comment: What grow factors do you want the columns to have? In the first mock-up, it looks like they should all shrink to their minimum width with the fourth column filling the rest of the space. So, do the columns need to have `flex-grow` 0, 1, 2, and 100? `flex-grow:100` looks particularly unusual.
Comment: i want the first 3 columns to be at their minimum width at all times unless the last column runs out of space (excluding the image reel which should never impact the width of the blue column) in which case the second and third columns should fill the space and the last column should wrap
Answer: 1) Settings .reel>img { flex-shrink: 0 } will show the scrollbar under your images
2) When you don't use flex-grow for relative growth compared to sibling elements, but just set .red,.green,.blue { flex-grow: 1 } and use flex-basis: xx% per color then the wrapping occurs.
Somehow the browser needs a width to work with. See /* ADDED CODE */ in the CSS...
UPDATE
The OP inquired:

Is there a way to keep the black, red and green columns from taking up more space than they need so that the blue column eats up all the available space?

This can be achieved by setting all but .blue to flex-grow: 0; flex-basis: auto (flexbox defaults) to start with and .blue flexbasis to a percentage > 0% and < 100% (more exact: less than 100% minus the total width of .black, .red and .green). Setting .blue { flex-basis: auto } will not work as it will immediately wrap because of its flex-grow: 1.
Updated the code to reflect the above (OVERRIDE 1) and added the final values the OP eventually used (OVERRIDE 2).
Simply try and see what works for you...

        * {
            box-sizing: border-box;
        }

        body {
            margin: 0;
            padding: 0;
            color: white;
        }

/* ADDED CODE (original answer) */
.black  { flex-basis:  5% } /* example values */
.red    { flex-basis: 15% }
.green  { flex-basis: 30% }
.blue   { flex-basis: 50% } /* [MUST] add up to 100% */

        .reel > img {
            flex-shrink: 0;
        }

/* OVERRIDE 1: ADDED CODE, as per OP remarks (UPDATE) */
.black  { flex-basis: auto }
.red    { flex-basis: auto }
.green  { flex-basis: auto }
.blue   { flex-basis: 25%  }

/* OVERRIDE 2: OP final choice */
.black  { flex-basis:  50px }
.red    { flex-basis: 100px }
.green  { flex-basis: 100px }
.blue   { flex-basis:  25%  }

/**/
        .green {
            background-color: green;
        }

        .red {
            background-color: red;
        }

        .blue {
            background-color: blue;
        }

        .black {
            background-color: black;
        }

        .display\:flex {
            display: flex;
        }

        .flex-direction\:column {
            flex-direction: column;
        }

        .flex-direction\:row {
            flex-direction: row;
        }

        .flex-wrap\:wrap {
            flex-wrap: wrap;
        }

        .flex-grow\:100 {
            flex-grow: 100;
        }

        .flex-grow\:2 {
            flex-grow: 2;
        }

        .flex-grow\:1 {
            flex-grow: 1;
        }

        .flex-grow\:0 {
            flex-grow: 0;
        }

        .flex-shrink\:1 {
            flex-shrink: 1;
        }

        .height\:100vh {
            height: 100vh;
        }

        .reel {
            display: flex;
            overflow-x: auto;
        }

        .reel > img {
            height: auto;
            max-width: 25%;
        }

        .reel > * + * {
            margin-left: 10px;
        }

        Column1

        Column2

        <input>

        <input>

        Column3

        <input>

        <input>

            This is some long header text in Column4

Comment: this is a good solution! Is there a way to keep the black, red and green columns from taking up more space than they need so that the blue column eats up all the available space? I'll go read up on flex-basis in the meantime and see what my options are.
Comment: Probably set all but `.blue`  to `flex-grow: 0; flex-basis: auto` (flexbox defaults) to start with and `.blue flexbasis` to a percentage you can work with (the higher the %, the sooner it wraps), but when `.blue` wraps the other colors won't grow to fill the gap (unless that is what you want, of course).
Comment: FYI, saw that the % on `.blue flex-basis` doesn't matter, as long it is not 0% or lower than 100% minus (computed?) widths of the others (to keep things simple). `.blue flex-basis: auto` will make it wrap immediately because of the `flex-grow: 1`
Comment: i gave it an upvote, this is very close to what i'm looking for, just need to experiment with your solution a bit and get it working 100%, then i'll accept it

i need the other colours to fill the remainder of the space when the blue wraps, but only when it wraps, otherwise blue needs to fill as much of the space as possible.

will play around with percentages and see if i can get it just right
Comment: flex-basis: 50px then 100px then 100px and then 25% on the blue column does exactly what i need, the other columns stay relatively fixed and the blue column only wraps once it gets smaller than 25% of its original size.

Make flex-wrap ignore certain elements


Is it possible to define a ring as a category?

Question: Is it possible to define a ring as a category? For example, a group can be defined as a category with just one object and all morphisms being isomorphisms.
Comment: This makes me think why we don't want also fields or vector spaces as categories as well?
Answer: A monoid is a category with exactly one object. A homomorphism of monoids is just a functor of the corresponding categories.
A ring is a linear category with exactly one object. A homomorphism of rings is just a linear functor between the corresponding linear categories.
More generally, if $R$ is a commutative ring, then an $R$-algebra is an $R$-linear category with exactly one object, and $R$-algebra homomorphisms correspond to $R$-linear functors.
This offers a compact definition of modules: A left module over $R$ is just a linear functor $R \to \mathsf{Ab}$. Actually this has lead to the following more general notion ( which has found applications in category theory and algebraic topology): If $C$ is any linear category, then a $C$-module is a linear functor $C \to \mathsf{Ab}$. Homomorphisms of $C$-modules are natural transformations. Thus, we optain a category $C$-modules.
Besides, it shows that the category of monoids or rings is actually a bicategory: If $f,g : M \to N$ are homomorphisms of monoids (or rings), then a $2$-morphism $f \to g$ is an element $n \in N$ such that $n f(m) = g(m) n$ for all $m \in M$.


How do I adjust maxheight in tippy.js?

Question: I dont know how to adjust heigh of tooltips from tippy.js
Coulnt find way to acces it there must be some easy way
Answer: As far as I can tell from source code tippy.js uses this style sheet:

.tippy-box[data-animation=fade][data-state=hidden] {
  opacity: 0
}

[data-tippy-root] {
  max-width: calc(100vw - 10px)
}

.tippy-box {
  position: relative;
  background-color: #333;
  color: #fff;
  border-radius: 4px;
  font-size: 14px;
  line-height: 1.4;
  white-space: normal;
  outline: 0;
  transition-property: transform, visibility, opacity
}

.tippy-box[data-placement^=top]>.tippy-arrow {
  bottom: 0
}

.tippy-box[data-placement^=top]>.tippy-arrow:before {
  bottom: -7px;
  left: 0;
  border-width: 8px 8px 0;
  border-top-color: initial;
  transform-origin: center top
}

.tippy-box[data-placement^=bottom]>.tippy-arrow {
  top: 0
}

.tippy-box[data-placement^=bottom]>.tippy-arrow:before {
  top: -7px;
  left: 0;
  border-width: 0 8px 8px;
  border-bottom-color: initial;
  transform-origin: center bottom
}

.tippy-box[data-placement^=left]>.tippy-arrow {
  right: 0
}

.tippy-box[data-placement^=left]>.tippy-arrow:before {
  border-width: 8px 0 8px 8px;
  border-left-color: initial;
  right: -7px;
  transform-origin: center left
}

.tippy-box[data-placement^=right]>.tippy-arrow {
  left: 0
}

.tippy-box[data-placement^=right]>.tippy-arrow:before {
  left: -7px;
  border-width: 8px 8px 8px 0;
  border-right-color: initial;
  transform-origin: center right
}

.tippy-box[data-inertia][data-state=visible] {
  transition-timing-function: cubic-bezier(.54, 1.5, .38, 1.11)
}

.tippy-arrow {
  width: 16px;
  height: 16px;
  color: #333
}

.tippy-arrow:before {
  content: "";
  position: absolute;
  border-color: transparent;
  border-style: solid
}

.tippy-content {
  position: relative;
  padding: 5px 9px;
  z-index: 1
}

So you can try mess with that, adding your overrides, but it would be far better just to use a div with max-height as the content you provide for the tooltip and tippy.js will adjust itself.
var html = "
I'm a Tippy tooltip!
link
"

var instance = tippy(elems, {
  content: html,
  allowHTML: true,
});

Comment: I alredy knew that but you made me reailse where i had the problem so big thanks,you saved me so bad!

How do I adjust maxheight in tippy.js?


Changing the colors inside of Netbeans programatically?

Question: So I came up today with an idea of having a java program that runs in the background on my PC and periodically checks the system time, and when I notice a change in the seasons ( Winter, Fall, Summer, Spring), I would change all the colors of the IDE (the keywords and comments and background) to fit with the colors of the season. Unfortunately, I can't find where this information is stored on file. Does anyone have any idea where it could be or if this is possible? Thanks!
Answer: Yes it is possible !!!

Run time options
Using configuration file
Using netbeans libraries

1. Run time options
If you like to have 'MetalLookAndFeel & fontsize 14, run below command
netbeans --laf javax.swing.plaf.metal.MetalLookAndFeel --fontsize 14

You can install list of themes from this link → Netbeans themes. You can launch a different theme programmatically based upon your choice.
2. Using configuration file
This can be achieved by updating the netbeans.conf file located in below location ${nb-install}/etc/netbeans.conf
for ex:

C:\Program Files\NetBeans 8.1\etc\netbeans.conf

Update the conf file with desired theme & font size programmatically.
for ex:

netbeans_default_options="-J-client -J-Xss2m -J-Xms32m
   -J-Dapple.laf.useScreenMenuBar=true -J-Dapple.awt.graphics.UseQuartz=true -J-Dsun.java2d.noddraw=true -J-Dsun.java2d.dpiaware=true -J-Dsun.zip.disableMemoryMapping=true"

You can update multiple look and feel options

Windows - com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Metal - javax.swing.plaf.metal.MetalLookAndFeel
GTK - com.sun.java.swing.plaf.gtk.GTKLookAndFeel
Aqua - apple.laf.AquaLookAndFeel

Additionally, you can configure other parameters as well.
All startup parameters listed here → Startup Parameters
3. Using netbeans libraries
You might be more interested in this.
Full netbeans API → netbeans Api
Corresponding Jar files → jar files
`org.netbeans.api.editor.settings.FontColorSettings` will be used to change the font settings which include keywords, syntax, background, foreground etc.

One small example for your reference
public void updateColors() {
    EditorUI editorUI = Utilities.getEditorUI(textComponent);
    if (editorUI == null) {
        return;
    }
    String mimeType = NbEditorUtilities.getMimeType(textComponent);
    FontColorSettings fontColorSettings = MimeLookup.getLookup(MimePath.get(mimeType)).lookup(FontColorSettings.class);
    Coloring lineColoring = Coloring.fromAttributeSet(fontColorSettings.getFontColors(FontColorNames.LINE_NUMBER_COLORING));
    Coloring defaultColoring = Coloring.fromAttributeSet(fontColorSettings.getFontColors(FontColorNames.DEFAULT_COLORING));

    if (lineColoring == null) {
        return;
    }

    // use the same color as GlyphGutter
    final Color backColor = lineColoring.getBackColor();
    // set to white by o.n.swing.plaf/src/org/netbeans/swing/plaf/aqua/AquaLFCustoms
    if (org.openide.util.Utilities.isMac()) {
        backgroundColor = backColor;
    } else {
        backgroundColor = UIManager.getColor("NbEditorGlyphGutter.background"); //NOI18N
    }
    if (null == backgroundColor) {
        if (backColor != null) {
            backgroundColor = backColor;
        } else {
            backgroundColor = defaultColoring.getBackColor();
        }
    }
}

Comment: Jar files download link → http://www.java2s.com/Code/Jar/o/Downloadorgnetbeansmoduleseditorsettingsjar.htm , I have updated my answer as well for future reference.
Comment: Thank you for this! I tried importing the package for the Netbeans API that you suggested but it clearly isn't there by default. I checked online for a download but couldn't find it. Any idea how I add it? Is it in Netbeans somewhere? Thanks!


wieghted mean on multiple columns for all rows

Question: I want to calculate a weighted mean of a huge dataset.
What I need is the following (for each row) and I have NAs,
so I need to somehow incorporate na.rm = TRUE.
I want the following to be calculated (for distance 1 to distance 10):
(distance1 * X1CityNumber + ... + distance10 * X10CityNumber) /
(X1CityNumber + ... + X10CityNumber)

I wrote the following code, but it is producing wrong numbers.
for (i in 1:378742) {
  rcffull$distance[i] <- weighted.mean(cbind(rcffull$distance1[i],
                                             rcffull$distance2[i],
                                             rcffull$distance3[i],
                                             rcffull$distance4[i],
                                             rcffull$distance5[i],
                                             rcffull$distance6[i],
                                             rcffull$distance7[i],
                                             rcffull$distance8[i],
                                             rcffull$distance9[i],
                                             rcffull$distance10[i]),
                                       cbind(rcffull$X1CityNumber[i],
                                             rcffull$X2CityNumber[i],
                                             rcffull$X3CityNumber[i],
                                             rcffull$X4CityNumber[i],
                                             rcffull$X5CityNumber[i],
                                             rcffull$X6CityNumber[i],
                                             rcffull$X7CityNumber[i],
                                             rcffull$X8CityNumber[i],
                                             rcffull$X9CityNumber[i],
                                             rcffull$X10CityNumber[i]),
                                       na.rm = TRUE)
  }

Any suggestions?

sample data with with fewer columns:
 distance1    Weights1    distance2        Weights2
1    5            3            8              2
2    NA           2            3              3
3    5            NA           4              4

#desired output:
    Mean distance
1      6.2 #= (5 * 3 + 8 * 2) / (3 + 2)
2      3.0 #= (3 * 3) / 3
3      3.0 #= (4 * 4) / 4

Answer:
NA happens in both weights and distances. When doing (d1 * w1 + d2 * w2) / (w1 + w2), NA should be eliminated from both nominator and denominator and normalization of weights needs account for this.

dat <- structure(list(distance1 = c(5L, NA, 5L), Weights1 = c(3L, 2L, NA),
distance2 = c(8L, 3L, 4L), Weights2 = c(2L, 3L, 4L)), .Names = c("distance1",
"Weights1", "distance2", "Weights2"), class = "data.frame", row.names = c("1",
"2", "3"))

A <- as.matrix(dat[c(1, 3)])  ## distance columns
B <- as.matrix(dat[c(2, 4)])  ## weight columns
B[is.na(A)] <- 0
rowSums(A * B, na.rm = TRUE) / rowSums(B, na.rm = TRUE)
#  1   2   3
#6.2 3.0 4.0

Remark 1:
If there is no NA in neither data and weights, just do
rowSums(A * B) / rowSums(B)

Remark 2:
Alternative way to deal with NA: set all NA in both data and weights to 0, then use rowSums without na.rm:
ind <- is.na(A) | is.na(B)
A[ind] <- 0
B[ind] <- 0
rowSums(A * B) / rowSums(B)

Remark 3:
NaN can happen due to 0 / 0, if there is no pair of non-NA datum and non-NA weight.
Remark 4:
weighted.mean can only remove NAs in data, not in weights. It is also undesired, as you want to do calculation for all rows. There is no "vectorized" solution with it; you have to do a slowish R-level loop.


Are MTLVertexAttributeDescriptors necessary? Why are they needed?

Question: I've been learning Metal for iOS / OSX, and I began by following a Ray Wenderlich tutorial.  This tutorial works fine but it makes no mention of MTLVertexAttributeDescriptors.
Now that I'm developing my own app, I'm getting weird glitches and I'm wondering if the fact that I don't use MTLVertexAttributeDescriptors may be related to the problem.
What difference do they make?  I've been able to make a variety of shaders with varying vertex structures and I never even knew about these things.
I know you use them to describe the layout of vertex components for use in a shader.  For example a shader might use this structure for vertices, and it would be set up in a vertex descriptor in the function below.
typedef struct
{
    float3 position [[attribute(T_VertexAttributePosition)]];
    float2 texCoord [[attribute(T_VertexAttributeTexcoord)]];
} Vertex;

 class func buildMetalVertexDescriptor() -> MTLVertexDescriptor {

    let mtlVertexDescriptor = MTLVertexDescriptor()

    mtlVertexDescriptor.attributes[T_VertexAttribute.position.rawValue].format = MTLVertexFormat.float3
    mtlVertexDescriptor.attributes[T_VertexAttribute.position.rawValue].offset = 0
    mtlVertexDescriptor.attributes[T_VertexAttribute.position.rawValue].bufferIndex = T_BufferIndex.meshPositions.rawValue

    mtlVertexDescriptor.attributes[T_VertexAttribute.texcoord.rawValue].format = MTLVertexFormat.float2
    mtlVertexDescriptor.attributes[T_VertexAttribute.texcoord.rawValue].offset = 0
    mtlVertexDescriptor.attributes[T_VertexAttribute.texcoord.rawValue].bufferIndex = T_BufferIndex.meshGenerics.rawValue

    mtlVertexDescriptor.layouts[T_BufferIndex.meshPositions.rawValue].stride = 12
    mtlVertexDescriptor.layouts[T_BufferIndex.meshPositions.rawValue].stepRate = 1
    mtlVertexDescriptor.layouts[T_BufferIndex.meshPositions.rawValue].stepFunction = MTLVertexStepFunction.perVertex

    mtlVertexDescriptor.layouts[T_BufferIndex.meshGenerics.rawValue].stride = 8
    mtlVertexDescriptor.layouts[T_BufferIndex.meshGenerics.rawValue].stepRate = 1
    mtlVertexDescriptor.layouts[T_BufferIndex.meshGenerics.rawValue].stepFunction = MTLVertexStepFunction.perVertex

    return mtlVertexDescriptor
}

But even without the MTLVertexDescriptor setup, the shader can already access the vertex buffer and the position / texCoord components of vertices in the array.  Just by setting the vertex buffer, the shader has access to all of the components.  So what good does the descriptor do?
Answer: There are, of course, multiple ways of doing things. The vertex descriptor is only used for one of them.
For example, a vertex function might be declared like this:
vertex MyVertexOut vertex_func(device const float3 *positions [[buffer(0)]],
                               device const float2 *texCoords [[buffer(1)]],
                               uint vid [[vertex_id]])
{
    // use positions[vid] and texCoords[vid] to fill in and return a MyVertexOut structure
}

This dictates that the vertex attributes be supplied in separate buffers, each of a specific layout.
You could also do:
struct MyVertexIn
{
    float3 position;
    float2 texCoord;
};
vertex MyVertexOut vertex_func(device const MyVertexIn *vertexes [[buffer(0)]],
                               uint vid [[vertex_id]])
{
    // use vertexes[vid].position and vertexes[vid].texCoord to fill in and return a MyVertexOut structure
}

This dictates that the vertex attributes be supplied in a single buffer of structs matching the layout of MyVertexIn.
Neither of the above require or make use of the vertex descriptor. It's completely irrelevant.
However, you can also do this:
struct MyVertexIn
{
    float3 position [[attribute(0)]];
    float2 texCoord [[attribute(1)]];
};
vertex MyVertexOut vertex_func(MyVertexIn vertex [[stage_in]])
{
    // use vertex.position and vertex.texCoord to fill in and return a MyVertexOut structure
}

Note the use of the attribute(n) and stage_in attributes. This does not dictate how the vertex attributes are supplied. Rather, the vertex descriptor describes a mapping from one or more buffers to the vertex attributes. The mapping can also perform conversions and expansions. For example, the shader code above specifies that the position field is a float3 but the buffers may contain (and be described as containing) half3 values (or various other types) and Metal will do the conversion automatically.
The same shader can be used with different vertex descriptors and, thus, different distribution of vertex attributes across buffers. That provides flexibility for different scenarios, some where the vertex attributes are separated out into different buffers (similar to the first example I gave) and others where they're interleaved in the same buffer (similar to the second example). Etc.
If you don't need that flexibility and the extra level of abstraction, then you don't need to deal with vertex descriptors. They're there for those who do need them.
Comment: Thanks - I was wondering what about the difference between the  [[stage_in]] and [[buffer]] syntax.  Unfortunately this also means that the lack of vertex descriptors probably aren't the reason for my glitching.
Comment: Just a note - sometimes you may need to use `packed_floatN` instead of `floatN` in yours structs depending on how you send up your data.


Real-world example on significance testing with large samples

Question: The issues that arise when one uses statistical hypothesis testing framework with large samples are dissussed on CV (see e.g. this thread). The main problem discussed there is that in the real world the null hypothesis like "there is no connection between variable X and variable Y" is almost always false (at least in domains like social studies when the research is not based on perfect controllable randomized experiment) and so it will be rejected if our sample is large enough.
I'm looking for references to empirical studies that support "null hypothesis is almost always false" statement.
I believe there is a study where some large survey with a hundreds of questions was used. For every question the authors tested the hypothesis that the answer to that question depends on the gender. They used t-test and choose the alternative hypothesis at random. As a result, they get about 45% of significant results. I read this paper some time ago but lost the reference and cannot find it now.
Are there any other studies like this? I'm looking for them to use as an illustration in statistical courses (when I teach others), to emphasis the distinctions between "statistical significance" and "practical significance".
Comment: You have to narrow down your domain. In physics you won't have an issue with large sample sizes which you describe.
Comment: @Aksakal, I'm speaking mostly about social studies with large datasets obtained outside of perfect randomized experiment framework. I added a clarification to the question.
Comment: In what sense do you expect to find *empirical* support? Failure to reject even at sample sizes of billions or trillions doesn't indicate that the null is true.
Answer: The thing that I find so teethgrating about this expression:

"null hypothesis is almost always false"

is that it underscores the sloppy way with which modern, frequentist hypothesis testing is done. If you adhere to that framework, then it is true, virtually all causal relations are in some, albeit miniscule and complex, way statistically true and can be found if one were to sample enough.
To me, it begs for a return to Fisherian testing. To remind people, Fisher never advocated a decision rule approach to significance testing, he merely said that a $p$-value should be compared to the statistical power of the analysis.
What this does is that it requires the investigator to a priori specify what they might consider a significant effect. By doing so, in the interpretation of results, it is made plainly apparent that the results are coming from an overpowered analysis. Results from overpowered analyses usually report very small effects, and the power to detect such an effect is usually very low. So while the $p$-value is very significant, the power is very small, and we call into question how "significant" these findings really are.
On the other hand, when you compare the power for the apriori effect size, overpowered analyses will have powers that are so large, and p-values that are so small they cannot be practically compared. This illustrates the intuitive discrepancy between what the researcher said they would find, and what they actually found.
For instance, suppose you have a trial to see the effect of blood pressure lowering meds. You realize, "ah! In power analyses, the investigator used a mean difference of 1.20 mmHg of blood pressure thinking that was a clinically significant effect, but in their analyses found a difference of 0.0300 mmHg with a 95% confidence interval 0.0299 - 0.0301 mmHg." And at that point you realize while these results are statistically significant, they really aren't clinically significant.
Comment: Thanks for your explanations. Actually, I understand the difference between "statistical significance" and "practical (e.g. clinical) significance". However, I'm looking for references to empirical researches that can be used as an illustration of that kind of problems.
Comment: @IlyaV.Schurov What do you mean by "empirical researches"?
Answer: I think you should into the American's Statistic association statement on p-values.
They do quote that big data researches should not draw any conclusions from p-values by itself.
Answer: Page 205 of Meehl (1990) briefly describes a study of 57,000 high-school seniors in which 92% of 990 different cross-tabulations (between 45 variables; 45 choose 2 is 990) were statistically significant. Most people who've heard of this study are probably familiar with it from Cohen (1994).
Standing, Sproule, and Khouzam (1991) examined a dataset of 135 variables from 2,058 Canadian grade-school students. 4,506 of 17,936 correlation coefficients (25%) had a two-tailed $p < .001$.
In this age of big data, a similar study to Meehl's and Standing et al.'s with a very large sample size and number of variables would be nice. But we do have Kramer, Guillory, and Hancock (2014), a study of some 690,000 Facebook users that found signifcant effects that were positively microscopic, such as a reduction of positive posts in users' News Feeds decreasing the percentage of positive words in the users' own posts by $-0.1\%$ [$t(310,044) = -5.63$, $P < 0.001$, Cohen's $d = 0.02$]. What's really rich is that Kramer et al. pooh-pooh another significant effect they didn't want to find on the justification that it was tiny: "Positivity and negativity were evaluated separately given evidence that they are not simply opposite ends of the same spectrum. Indeed, negative and positive word use scarcely correlated [$r = -0.04$, $t(620,587) = -38.01$, $P < 0.001$]." (p. 8,789).
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997–1003. doi:10.1037/0003-066X.49.12.997
Kramer, A. D., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24), 8788–8790. doi:10.1073/pnas.1320040111
Meehl, P. E. (1990). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66(1), 195–244. doi:10.2466/pr0.19188.8.131.52
Standing, L., Sproule, R., & Khouzam, N. (1991). Empirical statistics: IV. Illustrating Meehl's sixth law of soft psychology: Everything correlates with everything. Psychological Reports, 69(1), 123–126. doi:10.2466/PR0.69.5.123-126
Comment: Yeah! *Meehl's conjecture* is a keyword I was looking for! And actually the paper I described in the question is Waller, N.G., 2004. The fallacy of the null hypothesis in soft psychology. (I initially found a reference there from a paper Mindless statistics by Gerd Gigerenzer, 2004.) I will investigate other papers you mention as well. Thanks a lot, this is exactly what I was asking for!


How to do apoyando in Romance de Amor?

Question: I am trying to play Romance de Amor, sometimes attributed to Vincente Gomez.
I have heard someone recommend using apoyando (rest stroke) for playing the melody (on first string) - since the beginning is very easy, it's a good piece to practice apoyando.
My question is - which fingering would be best for that? I have only used apoyando with the thumb ('p' finger), so anything else seems equally alien to me.
Just to show what I am talking about, I tried the following:

pami - could be convenient, because it uses the "traditional assignment" of each finger to its string (i=3, m=2, a=3)
paai or paam - after doing the apoyando, the ring finger lands on the second string; might as well use it while it's there
pmmi - the same, but not using the ring finger

I have looked at various performances on youtube, and they invariably use the pami fingering, but, as far as I can tell, they don't use apoyando.
Answer: I usually play this piece with pami and that's the way I learned it. But recently when I'm trying to get a nice tone out the melody and trying to add apoyando, I've discovered that my fingers actually want to play it pmip or pamp. It seems to vary depending upon which of my two fingers m or a has a smoother fingernail (and thus, a sweeter tone). I'm the first to admit that this forces a weird jump for the thumb, which makes the chord changes trickier to accomplish (both hands changing positions at the same time). But it doesn't appear to adversely affect the sound. A little hesitation at the ends of the measures seems to help support the uneven 3/4 time (or 9/8 depending on whether you take the arpeggios as triplets or as a compound subdivision).
I'm surprised that I never considered your paai etc. idea to combine the apoyando with a sweep. This is something I may have to play around with. If it were combined with my thumb-heavy idea, the piece could be played with just two fingers. That may make it still easier to focus on the tone of the melody notes, while simultaneously making the timing more difficult. :/ If you're getting this fancy with the piece, then it may be interesting to try a triple sweep piii.
Answer: Never tried that with the Romance, but now I'm learning to play Tarrega's etude in Em (a tube link) which has a similar stroke pattern with apoyando on the first string. My teacher says I definitely should use pami for it. He admits it's a bit tricky at first, but says the pattern can be learned with a little practice. Don't see why this would not apply to the Romance as well.

How to do apoyando in Romance de Amor?


Calculate $\lim_{n\to\infty}\int_0^n\frac{\mathrm{d}x}{n+n^2\sin\frac{x}{n^2}}$

Question: Calculate $$\lim_{n\to\infty}\int_0^n\frac{\mathrm{d}x}{n+n^2\sin\frac{x}{n^2}}$$, well first of all the function isn't monotonic, so I cant use the Lebesgue therem to go with the limit under the integral. What I could do is to express $sinx=x+O(x^3)$ then $$\lim_{n\to\infty}\int_0^n\frac{\mathrm{d}x}{n+n^2\sin\frac{x}{n^2}}=\lim_{n\to\infty}\int_0^n\frac{\mathrm{d}x}{n+x}=\lim_{n\to\infty} \ln2n-\ln n=\ln2$$, but i am not sure if this is correct and also would like to know why this is correct, as of course I ate the $O(x^3)$ when approximating sin function.
Comment: Double limit always needs extra care. Although your approach is totally justifiable, it would be easier to first substitute $x=nu$ and the pass limit to the inside of the integral.
Comment: nice suggestion!
Answer: Hint
Since $u-{u^3\over 6}\le \sin u\le u$ for $0\le u<<1$ (small enough $u$) you can write $${1\over n+x}\le\frac{1}{n+n^2\sin\frac{x}{n^2}}\le \frac{1}{n+x-{x^3\over n^4}}\le{1\over n+x-{1\over n}}$$and integrate the sides.


How to parse YAML files in Jenkins pipeline using readYAML method from the Jenkins plugin Pipeline Utility Steps

Question: I am trying to parse YAML files in Jenkins pipeline using the readYaml method from the Jenkins plugin "Pipeline Utility Steps".
I read on the forums that the readYml method should be called in the node block of the pipeline.
Before tinkering with this readYml method my pipeline worked flawlessly.
But after adding readYml to my pipeline's node block I get the following error.

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 5: Expected to find someKey "someValue" @ line 5, column 14.
           node {
                ^

1 error

    at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
    at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
    at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
    at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
    at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
    at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
    at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
    at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
    at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
    at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
    at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
    at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
    at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:290)
    at hudson.model.ResourceController.execute(ResourceController.java:97)
    at hudson.model.Executor.run(Executor.java:421)

I won't bore you with the complete pipeline code since the issue is really after editing my node block.
The way I call the plugins readYml method is as follows.
pipeline {
    agent {
        node {
            label 'lsrv9557.linux.rabobank.nl'
            customWorkspace '/appl/jenkins/workdir'
            datas = readYaml file: "manifest.yml"
        }
   }

How do I get this to work properly and get rid of the error?
Thanks in advance.
Comment: You are using declarative pipeline and you are not able to make step calls like that in the `node` configuration.
Comment: Thank you for your answer.

How could I set this up so it does work?
Since I really do need a YAML parser.

There must be a parser fit for declaritive pipelines, right?
Comment: You need to follow the recommendation made in the comment and use this in the `steps` and not in the `node`. Check the documentation on declarative pipelines and their syntax for more info.
Answer: I have figured out what the issue was.
As the kind fellows above mentioned as well,calling the plugin won't work in a node block in a declaritive pipeline.
However, simply putting it in a step block also wasn't working.
The fix in the end was putting it in a script block within the step block.
        stage('Read YAML file') {
        steps {
            script{ datas = readYaml (file: 'manifest.yml') }
            echo datas.ear_file.deploy.toString()

        }
    }
}

note that the echo is just to verify for myself if the *.yml file was properly parsed.
Answer: Most convenient was for mine to work with base shell scripts after trying to template based on the javaISH strings.
$ cat logic/pipelines/obfuscate.sh
#!/bin/bash

echo "${REQUEST}" | json2yaml > "${PARAM_FILE}"

Then reading from this PARAM_FILE in scripts further on:
---
#!/bin/bash

# improvised dynamic extraction

SOME=`yq -r .scope.some $PARAM_FILE`
echo -n "${SOME}"


How to create a remote copy (not local clone) of a VSTS GIT repository

Question: I have a VSTS GIT repository, I want to create new remote repository which would be the copy of existing repository. How to do it ? Please note I am not looking for GIT clone to clone a local copy of remote repository.
Atul
Comment: You could clone the repo and then remove the git folder
Comment: So there is no in built command to do that ?
Answer: Some options:

Clone the repo with git clone --depth=1 (which only clones the most recent commit), then add a new remote and push.
If you're staying within your VSTS account, you can fork the repo.
Answer: I think there is no in built command, create a local clone remove the git folder, create empty repo & copy the folder & push.


KeyError while trying to read from Python dataframe

Question: Hi I am new to Python and I am trying to read from a csv file using the following code
dataFrame = pd.read_csv(path, header=None)

X = dataFrame.loc[:,1:93]
y = dataFrame.loc[:,94]
print(X)
print(y)

But I get the following error

KeyError: 'the label [94] is not in the [columns]'

But when I copy the contents of same csv file into another and run the code, it works.. Can anyone help me with this as I cannot keep on copying the csv files as there are huge number of files.
The csv file to be read in 'path' has been created using the following code
criterion = dataFrame[93].map(lambda x: x==some_value)
with open(temp_file, 'a') as f:
                dataFrame[criterion2].to_csv(f, sep='\t', encoding='utf-8',header=False)

Comment: why do you open `temp_file` for appending? Wouldn't that suggest there is already content in the file?
Comment: Yes actually there is a content in the temp file which is created using teh code  dataFrame[criterion1].to_csv(temp_file, sep='\t', encoding='utf-8',header=False)
Comment: ok, what do you mean by "copy the contents of the same csv file into another" ? I assume you do not mean programmably since then I see no reason it would be a pain, does it still work when copying the file through python?
Comment: I copied the contents manually to another csv file.... What i want programmably is to create some csv files and manipulate the contents from it... As of now I created csv files but I am not able to read from it programically.. So I copied contents from it to another csv manually and did the manipulations
Comment: Let me see if I understand this correctly: your python program creates a csv file, you open that file up in notepad / etc, copy the entire file and paste it into another (previously blank) file and then when you use the directly created file it fails but the one that was copied worked?  I do not see how that could happen.
Comment: Yes your interpretation is correct. I tried to copy the entire csv file to another by using python code also.. Then the new file created still does'nt work. So  I guess there is some  trailing null character or something in the file created which is interfering with its manipulation. When I copy it manually it is cleared of it thus it works. How can  I get rid of that?
Comment: I have a strong feeling that finding a way to only call `.to_csv` once without then opening the file for appending would fix the issue, however I can't help there unless you provide that section of your code.
Comment: Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/113700/discussion-between-athira-and-tadhg-mcdonald-jensen).
Comment: Seems to me this could be BOM related pandas issue. Try doing,    `encoding = 'utf-8-sig'`.
Comment: @NickilMaveli I tried doing 'utf-8-sig' but it doesnt work ...
Answer: My problem has been solved.. Actually my columns in csv were getting read as a single column with /t appended. So I used delimiter specification as follows
dataFrame = pd.read_csv(temp_file,sep=', ', delimiter='\t')
the code works now.


Marking some XIB/Storyboard strings as not localizable

Question: I am using Base Internationalization for XIB/Storyboard files and the "Export for Localization" method using XLIFF files for translators.
I have some labels, buttons, etc. that have text that should be translated, but I also have labels where we use some placeholder text (like a full-name) so you can see what the view would look like when populated with data, but those labels always have their text come from an outlet programmatically.
Is there some way to mark this label's .text property that is set in the XIB as non-localizable so that it doesn't end up in the XLIFF (or resulting .strings) files.
I know that I can remove the text -- I also thought about having a prefix (like @"!DNL!") to mean that the translator shouldn't localize, but I am hoping that there is just a standard way to do this.
Comment: It looks like this isn't possible.  I filed a radar (19232051) in case someone comes across this question in the future and wants to see if it got implemented.
Answer: I add a note "DNL" to the "Comment for Localizer" field in the identity tab. Then, I run this command to automatically remove all of those elements from the XLIFF:
xmlstarlet ed -d "//*[contains(text(), 'Note = \"DNL\"')]/.." en.xliff > out.xliff

Basically, it's using xmlstarlet (which can be downloaded via homebrew) to find all elements that contain the text Note = "DNL", and then deleting the parent of that element from the XLIFF.
Combined with using xcodebuild -exportLocalizations, you can make a pretty simple script for generating your XLIFFs:
xcodebuild -exportLocalizations -localizationPath build -project ProjectName.xcodeproj
xmlstarlet ed -d "//*[contains(text(), 'Note = \"DNL\"')]/.." build/en.xliff > build/out.xliff

Comment: NVM, it's in the `document` section of a object. Works great!
Comment: What do you mean with 'Comment for Localizer' field? Is that a field in XCode? Can't find it.
Answer: It turns out the localization export from Xcode ignores attributed strings in the storyboard.
So just set the type of text for every label/button you want to exclude to Attributed in the Attributes Inspector.

This will give you an attributed string rather than a plain string, which as far as I know has no implications, apart from the (empty) list of attributes that has to be kept in memory now.
Comment: This doesn't cover some cases like, for example, navigation item text, as there is no option to set the type to "Attributed" from the Attributes inspector.
Answer: UPDATE:
Check out ReMafoX, it's a Mac app that perfectly solves your problem. It can be easily installed and integrated within your project, watch this video for a detailed walkthrough.
To ignore specific Strings, simply add one of the customizable ignore flags from the "Interface Builder" pane in the "Comment for Localizer" field in your Storyboard/XIB files and the build-script you configured following the above linked video will exclude it on next build of your project (or press of the "Update" button in the config).

OLD ANSWER:
This is now possible using the BartyCrouch command line utility which I recently wrote to solve this problem (see installation instructions in my answer on that thread).
BartyCrouch runs ibtool for you and does additional processing on top of its resulting .strings file. It will exclude views from translation if you include #bc-ignore! into your value or comment within your base internationalized Storyboard/XIB file.
Please check out out the related section within the README on GitHub for detailed information.
Comment: @Borzh If you liked BartyCrouch, you might want to also checkout ReMafoX, its successor with a GUI and many other improvements over BartyCrouch: https://remafox.app
Comment: BartyCrouch rules!


Ways to show that words with exponent sum zero for each generator are elements of the commutator subgroup

Question: Say I have a free group on the generators $X = \{ x_1, x_2, ... , x_k \}$ with $k \geq 2$. I read (in an article) that if I have a word $w$ written in the generators and their inverses, and the exponents of all $x_i$ for $i \in \{ 1, ..., k \}$ in this word sum up to zero, then the word is an element of the commutator subgroup of $F(X)$, usually referred to as $F'$ or $[F,F]$.
I presume it is a rather trivial task to show that commutators and their products have this property, since it applies to commutators by definition and cancellation of generators and their inverses will also uphold this property for any cancellations in products of commutators.
However, I struggle to find an elegant way to prove the other direction, i. e.: All words $w$ in $F(X)$ with the property as mentioned above are themselves elements of the commutator subgroup $F'$.
I assume I can show this by looking at a word with the above mentioned property, say $w=x u x^{-1} v$, where $x$ is a generator of the free group in question (or its inverse respectively) and $u, v$ are words written in its generators or their inverses. I get
$$
x u x^{-1} v = xux^{-1}u^{-1}uv = [x, u] uv.
$$
Apparently $uv$ again has the above mentioned property of exponent sum zero with regards to the used generators, and the word for which we have to show to be an element of the commutator subgroup $F'$ has been shortened by $2$ compared to our original word $w$. So my assumption is that repetition of this process should result in a product of commutators.
My question being:

Is there a more elegant proof, which is not algorithmic?
Furthermore: Any hint where to find more information on such a proof and this property of elements of the commutator subgroup would be appreciated.
Answer: Your proof is completely correct. There is, however, also a more conceptual (to my eye) way to view this characterization of the commutator subgroup:
Let $F_n$ be the free group on $n$ generators $x_1,...,x_n$. Then the quotient $F_n/F_n'$ is the free abelian group $\mathbb{Z}^n$ on $n$ generators with the quotient map $\pi: F_n\to \mathbb{Z}^n$ sending each $x_i$ to $e_i$ (the standard basis vector).
It is then clear that for any word $w$, the $i$-th entry of the vector $\pi(w)$ is just the sum of the exponents of $x_i$ in $w$.
Hence, a word $w$ has all exponent-sums equal to zero if and only if it lies in the kernel of $\pi$, but that kernel is just the commutator subgroup by definition.


Can't get lines around table borders/cells

Question: I have several web pages containing tables, for which I'd like to have line-borders around the tables and the cells.   In fact, some of these pages existed for several years already, and rendered acceptably in IE6, IE7.
We switched about 6 months ago to a completely different set of style sheets to change our site look and feel.  We also switched to "modern" browsers such as IE8 (and because I couldn't stop Vista) to IE9.
Now the borders don't render at all.
I spent a day fighting with this about a month ago, and failed to fix it.  It seemed that I could reduce the page down to just the barest table and IE8 would still not render the border. I think I decided IE8 was just buggy, but I'm not an HTML expert so it is more likely that I'm buggy.
(I'm just getting back to this; I'll go see if I can find that reduced page).
Here is one such page:
http://www.semdesigns.com/products/DMS/DMSComparison.html

The tables should be obvious; you can tell them by their absence of lines :-{
The URI validates using the W3C service as HTML 4.01 Transitional.
Any suggestions?
EDIT: Posters all pointed out that the new style sheets set the default borders to 0. Oops!
Answer: From what I can see you have border:0; in the styling of the tables.
This will hide the borders on tables.
Comment: If that's it, I will be really thankful that somebody called me stupid, because its such an easy fix :-}  I'll check that out at work tomorrow.  Thanks much for seeing something I somehow didn't see.
Answer: I see no borders in Firefox either.

SDstyle.css, line 16. You do CSS reset which sets border: 0; for a big list of tags, including table, td, tr, th ...
You declare your table like
, but at the same time you have no such style defined in your CSS files.
Comment: The class="box" is the vestiges of an attempt to take control of the table styling.  If I define a border size in the definition of box, presumably that would fix the problem?
Comment: @Ira For tables that are defined in this way -- yes (I have seen only 1 out of 3 checked tables had `class="box"`), but you _may_ need to add `!important` in such definitions (e.g. `table.box {border: 1px solid #4C4B4B !important;}`).
Comment: didn't seem to need "!important"; what on earth does that do (and why is the other stuff not important)?
Answer: Your new stylesheets appear to use reset values, i.e. it sets lots of different elements to border:0 (including the table).  You'll need to specifically set the border on the table and cells.
To set the border around just the table simply use table.box {border:1px solid #000}.  If however you set the border on all sides of the table and cells you will have a border twice as thick as you need.
I therefore use the following trick to set the top and right border of the outside table, and the bottom and left of the table cells.  This gives the appearence of an even border across the whole content of the table.
table.box {
    border-top: 1px solid #000;
    border-right: 1px solid #000;
}

table.box td {
    border-bottom: 1px solid #000;
    border-left: 1px solid #000;
}

I've noticed however you have some empty cells in your box table, so you'll need to remove these to prevent the double borders.
Comment: Three good answers; hard to choose the best.  You had some additional nice advice.