Relation between linear regression prediction accuracy and correlation

Question: This question is more of theoretical. I am not sure if this is the right place, but still giving it a try.
I have two variables — direct cost and indirect cost. When sales persons go for a sales pitch to a customer they know about direct cost that they are going to incur for this service, but they don't know much about indirect cost (they will come to know about it in latter stages). An estimate of indirect cost at this stage will be valuable for sales persons.
I am trying to predict indirect cost as a function of direct cost. I am doing this via a simple linear regression. I plotted scatter plot between direct cost and indirect cost and see a good linear relationship between them. I also see that direct cost and indirect cost are highly corelated to each other with correlation coefficient as 0.98, so I expected a very good prediction accuracy. But surprisingly, my prediction accuracy is not so good. I have around 200,000 points in my training data and average prediction error on training data is 17 %. Though adjusted R-Square value is 0.97. I am using lm() function from R.
My question is that in case of simple linear regression, in general, should we expect better prediction accuracy if dependent and independent variables are highly correlated or is it my misconception? If we expect good accuracy, am I missing something here. Please note that I have also tried centering these variables around mean.
Comment: This is a good question...for CrossValidated, which is the SE site for statistical analysis. I've flagged your question to be closed and migrated there.
Comment: I'm curious to see how a model with 0.97 adjusted R-squared gives you such a poor predictive capability. What happens if you try to predict the training dataset? Do you use the `predict` function, or you try to manually use the coefficients? Do you have an intercept in your model? If you don't want to post your data would be good to simulate a pair of highly correlated variables and perform a similar analysis and check if you find the same problem if you follow exactly the same process.
Comment: @AntoniosK, sorry,  there was a typing mistake from my side! The accuracy that I talk about is on training data itself. I just edited it in my question. I did not use 'predict()' function, lm object provides error/residuals on training data. I am using residuals for my analysis. The suggestion of analysis on simulated variables looks good, I will give it a try.
Comment: how are you defining prediction accuracy? typically a measure of accuracy is used for qualitative dependent variables (i.e., for classification problems), not when your dependent var. is continuous.
Comment: Is it possible to post the model output and explain how you found that 17%? @Chris has a point here and I guess we had similar thoughts. Also, I think you can see the predictive capability of your model if you predict (training and test datasets) and then you plot predicted values against the true values.
Comment: @Chris, both of my variables are continuous. I calculate error as residual divided by actual value. So if actual indirect cost is 100 and predicted indirect cost is 110 (residual = 10), then my prediction error will be 10 %.
Comment: Can you give the code  you use to calculate this measure of error? My guess is that there is something wrong with how you are coding it.
Comment: Is there any chance that you simply interpret your average prediction error of 17% as the prediction accuracy? Also, have a look here : http://docs.statwing.com/interpreting-residual-plots-to-improve-your-regression/, at section "Understanding Accuracy with Observed vs Predicted".
Comment: @AntoniosK, No. I understand that 17 % is average prediction error and not the accuracy. But with given level of correlation between variables, I expected my error to be small, may be around 5-10 %.

Chris, I calculated average error in excel. So dont have a formula as such. But logic I used is that caclulate percentage error for each data point(Residual*100/Actual Value) and then take average of percentage error across all data points.
Comment: I think your calculations are correct. That 17% average error (and not accuracy as you mention in your post) is reasonable as there's a chance that some extreme error cases increase the error average. Try to see how the distribution of errors looks like. Also, the predicted vs. actual values plot will help you spot any prediction areas where you underestimate/overestimate a lot. What are the cases with the biggest errors? Are they maybe extreme values in your dataset? Try to find the nature of the problem and you can improve your model, or -at least- understand and report the limitations.
Comment: @AntoniosK, Thanks for the suggestion. Looks like a good direction to move on.
Answer: In a linear regression you are trying to find the best fit of a set of output $Y=(Y_1,\dots,Y_n)$ as a linear function of input $X=(X_1,\dots,X_n)$, i.e. you want the $a$ and $b$ that gives the smallest residuals:
$$(a,b)= arg \min \sum_i (Y_i-aX_i-b)^2$$
if it is what you did and you formally get (if you are still working with the same $n$ observations)
$$a=arg\min Var(Y-aX),  \;\; b=\bar{Y}-a\bar{X},\;\; and \;\; R^2=\frac{var(aX)}{var(Y)}=corr(X,Y)$$
(where for any two vector $U,V$ $\bar{U}$ is the empirical mean $Var(U)$ and $corr(U,V)$ the correlation).
You do not say how you compute your errors but if you are computing $var(Y-ax)$ then it is equal to $(1-R^2)*var(Y)$ so if you normalise the errors by $var(Y)$ then you have
$$Error=\frac{var(Y-ax)}{var(Y)}=1-R^2$$
which in your case should be 0.03 (because $R^2$ is 0.97). But what you did (I guess :) ) is that you computed the root mean error divided by the standard deviation of the output wich is exactly $\sqrt{Error}$ (according to my definition).
Your results are perfectly correct and conform to the theory because
$$ \sqrt{Error}=\sqrt{0.03}=0.1732051$$
nothing surprising !
CQFD
Comment: I recalculated errors as per your equations, and my result align with these euqations. As you say, it does not look so surprising now :-).
Comment: I just received enough privilege to do upvote :-)
Comment: Sorry, just catching up to this thread now...Can someone explain how the average value of residual/Y is the same as Error according to the equation listed above? Error according to the equation above is a function of variance, so how would this equate to mean(residual/Y)?
Comment: Hi Chris I'm not too sure I get your point. I don't compute the average value of residual/Y ? maybe one of my formulation is unclear and should be rephrased ?
Comment: The way that the error was calculated according to the user who posed was the average value of the residual/y. They came up with a prediction error of 17% with this method, which I thought you were saying was a correct way to calculate error? I was claiming that mean(residual/predicted) is not a good way to calculate model fit, and I thought you were saying it was appropriate? Maybe I'm missing something here.
Answer: I think I know what's going on. Intuitiviely, your method of calculating prediction error seems OK - You take the residual value (i.e error) and see how big it is relative to your data (i.e., relative to either the predicted value or the value of your dependent variable) by dividing by that value. However, there's at lease one issue with calculating error in this way that I can think of: When you divide by the value of the dependent variable, you are expecting the divisor not be close to zero - when it is, your residual/(actual value) can become huge, simply because the actual value may be a fraction (i.e., close to zero...e.g., .005). I ran the following simulations and confirmed my suspicion:
# we'll run about 150 linear models. The x value will be closely related to the y value (linear relationship), meaning that if x is close to 0, y will be close to 0. The mean of the independent variable (x) is stored in the following vector:
seq(-50, 50, .7) -> mean_vals
# dataframe to save our regression results in:
data.frame(r_sqrd = NA, resid_over_value = NA, mean_vals = NA) -> frame

for(i in 1:length(mean_vals)){
    x <- rnorm(n = 1000, mean = mean_vals[i], sd = 10)
    x + rnorm(n = 1000, mean = 0, sd = 5) -> y
    lm(formula = y ~ x) -> k
    # This - or some close relative of this - is how you are calculating error
    k$residuals/y -> z
mean(z) -> frame[i,"resid_over_value"]
# THe Rsquared value is a much more typical measure of fit/prediction error for a linear model:
summary(k)$r.squared -> frame[i, "r_sqrd"]
    # Put the mean of the x values in the frame so we can see how your method of prediction error changes as a function of mean
    mean_vals[i] -> frame[i, "mean_val"]
}

As you can see in the plot below, your method of error does not increase as a function of Rsquared (which is an issue in itself, though I'm not 100% sure on why this is happening)...The issue that I focused in on had to do with the outliers in the below graph.
plot(frame$r_sqrd, frame$resid_over_value, xlab = "Rsquared", ylab = "Error according to your method")

...Those points generally occur when the mean of the predictor variable is close to zero, as can be seen in the below graph:
plot(frame$mean_val, frame$resid_over_value, xlab = "Mean of independent variable", ylab = "Error according to your method")

To conclude, I suggest you use a more standard version of model fit/prediction error such as the mean squared error (MSE) or the rquared.
Comment: That's great and very useful info for @user3697157 to investigate more. However, he specified he's working with costs, so I don't expect negative values, but definitely some close to zero. This agrees with my assumption that maybe some/few cases generate big errors (in the way he defined them) and increase the mean prediction error. Maybe those few cases are the ones with costs close to zero.
Comment: @Chris, Thanks! While carrying initial analysis, I noticed that percentage error are higher when dependent values are close to zero. This is why, I fit regression model only on the data for which dependend values are more than 10000. 'regData1 <- subset(regData, Direct.Cost> 10000)'

Based on your answer, I analyzed error again. I can see that still percentage errors are higher when dependent value is small (between 10,000 to 100,000) and it is within acceptable range for higher dependent values. I need to do rethink on my approach. My dependent value range from zero to millions.
Comment: @AntoniosK, Yes. Few of my cases (with smaller dependent values) are generating unusually high errors. I am rethinking on my approach. Any suggestions are welcome!


How do I start a new phrase whilst modulating?

Question: Hi guys,
I have recently been composing and have run into an issue. I have finished a section with a perfect cadence (bar 52), and I would like to change to the subdominant key for the next phrase (bar 53). However, as it stands, bar 53 (Bb7) sounds like a continuation of the previous phrase, and it only feels like a new section when we land on the Eflat M chord in the second bar of the phrase. what is best practice here??
Is it best to modulate to E flat before the cadence at 52? Or best to start the new phrase in B flat and modulate mid-phrase?
thanks Ed
Comment: I've voted to close as "opinion based," but I have a larger concern: why is the question posed with only two options for how to fix the problem? Composition is hardly an either-or proposition. Is there some reason no other solution(s) is (are) viable?
Comment: I'm going to suggest that the answer will have more to do with phrase lengths—how many bars are in the phrase. There isn't enough context here to tell, but if you want m.53 to "feel like a new phrase," it will work best if 52 was the final bar of a phrase (probably 4th bar).
Comment: Either has been made to work by great composers; since it's your baby, you can decide which works better for you. Try out, as composers often do, different ideas, and choose the one which you think is best.
Comment: This question has a very easy answer. You start a new phrase after modulations with pivot chords. This is not an opinion this is a well-established music theory. IDK why this has been closed it has a very easy and widely accepted answer.
Comment: EdB123, there are votes to re-open this question. Can you give us a bit more context, to establish where these measures fall in the phrase length? I believe there are objective answers to this question, but my answer is still tied to where the phrase ends.
Comment: Yes- how is best to do this? Shall I upload more images? If so, where?
Comment: and bar 52 is the end of a phrase. but because i start the new phrase with V7, it doesnt sound like the new phrase has started until the following tonic chord in the new key (E flat M)
Comment: @NeilMeyer- this is not so much a new phrase 'after' modulation.  the perfect cadence here is in B flat M at bar 52. (bar 52 is chord I, and the preceding bar is V). No modulation at this point. Because I want to move to E flat M- I start the new phrase (bar 53) with V7 in E flat- moving to E flat. However, as it stands, the V7 chord sounds like it is part of the previous phrase, despite the fact the phrase before has an even number of bars.
Answer: Yes, you could have brought in a Bb7 a bit earlier and had a perfect cadence into Eb at bar 52.   But what you have done is fine - a perfect cadence in B♭ then a B♭7 chord leading to E♭ at bar 54.   Composer's choice!


How can i transmit the sender as an eventarg

Question: Instance X of a class register to the change event of instances Y of the same class.
I want to update X if Y is changed, but i dont want to use the static keyword all over the class. Is there a way to transmit the recipient of the event in the eventargs?
Here is some example code with an NUnit tests to illustrate where my problem lies exactly. I compiled and ran it. Two tests just verify the programming. The failing test illustrates my problem.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NUnit.Framework;

namespace Eventtest
{
    public class DependencyChain
    {
        public static int demonstrationcount = 0;

        private String hiddenstring = "";
        public String visiblestring
        {
            get { return hiddenstring; }
            set
            {
                hiddenstring = value;
                NotifyOnStringChanged();
            }
        }

        private void NotifyOnStringChanged()
        {
            if (changed != null)
            {
                changed(this, EventArgs.Empty);
            }
        }

        public EventHandler changed;

        private EventHandler Observer = new EventHandler((o, e) => {
            DependencyChain sender = (o as DependencyChain);
            demonstrationcount++;
            //THE FOLLOWING DOES NOT WORK SINCE "this" IS NOT STATIC
            //DependencyChain recipient = this;
            //recipient.visiblestring = sender.visiblestring;
        });

        public DependencyChain(string initialstring)
        {
            this.visiblestring = initialstring;
        }

        public DependencyChain(DependencyChain other)
        {
            this.visiblestring = other.visiblestring;
            other.changed += Observer;
        }

        public override string ToString()
        {
            return visiblestring;
        }

    }

    [TestFixture]
    class Eventtest
    {
        [SetUp]
        public void ResetStaticCounter()
        {
            DependencyChain.demonstrationcount = 0;
        }

        [Test]//PASS
        public void ShouldInitialiseAndCopyValues()
        {
            DependencyChain Y = new DependencyChain("initial");
            DependencyChain X = new DependencyChain(Y);
            Assert.AreEqual(X.ToString(), Y.ToString());
        }

        [Test]//PASS
        public void ShouldCallObserverOnChange()
        {
            DependencyChain Y = new DependencyChain("initial");
            DependencyChain X = new DependencyChain(Y);
            Assert.AreEqual(0, DependencyChain.demonstrationcount);
            Y.visiblestring = "changed";
            Assert.AreEqual(1, DependencyChain.demonstrationcount);
        }

        [Test]//FAIL
        public void ShouldChangeStringOnChange()
        {
            DependencyChain Y = new DependencyChain("initial");
            DependencyChain X = new DependencyChain(Y);
            Y.visiblestring = "changed";
            Assert.AreEqual(X.ToString(), Y.ToString());
        }
    }
}

Answer: I think you only have to move the initialization of Observer to a constructor of DependencyChain, so you can capture this.
Comment: That worked. Uncommented the code, moved to a method, called method in constructor. Thanks
Comment: OK, good. You could also write the evt handler as a 'normal' method instead of a lambda.


Question regarding the existence of an invariant, non-degenerate complex bilinear form.

Question: I have a question regarding the proof of proposition 6.1, p.345 of this article, i'm reading and trying to understand right now. In said proof, the author states that

self-duality of the hypergeometric group $H(a;b)$ (i.e. the parameter sets $a,b$ are invariant under $z \mapsto z^{-1}$) implies the existence of an non-degenerate bilinearform $F$ on $V=\mathbb{C}^n$, which is invariant under $H(a;b)$.

Firstly, i understand that "complex bilinear form" here just means a bilinear form on a complex vector space. That is, it is linear in each argument, rather than linear in one argument and conjugate-linear in the other. Which leads me to the following question:
My question:
Well, i want to clarify the statement above.
In theorem 4.3 of the same article, it is shown that if the parameters are invariant under $z \mapsto \overline{z}^{-1}$ there exists a non-degenerate, invariant hermitian form on $V=\mathbb{C}^n$.
Does thrm. 4.3 also hold true for a complex bilinear form rather than a hermitian one, when only assuming the invariance $z \mapsto z^{-1}$ instead of the invariance with the additional complex conjugation? So in particular i want to know, if thrm. 4.3 was used in the proof of 6.1 and in case it was, how exactly. If this is not the case, then why does this bilinear form exist?
I would be very grateful for any answers or comments on this matter. Thank you.
Comment: Have you looked into asking one of the authors? [This](https://www.math.ru.nl/~heckman/) one looks possibly still alive and friendly.
Answer: In the language of representation theory, a (finite dim) group representation is self-dual if and only if it has an invariant bilinear form. This is because an invariant non-degenerate bilinear form on a representation $V$ is the same as an injective map of representations $V\to V^*.$
In the language of the paper, a hypergeometric group is specified by matrices $h_0,h_1,h_\infty.$ Since we're working with matrices, the dual transformations are transposes. Applying the inverse transpose map $g\mapsto (g^{-1})^T$ gives another hypergeometric group by $(h_0^{-1})^T,(h_1^{-1})^T,(h_\infty^{-1})^T.$ In the case you're interested in, these two hypergeometric groups have the same parameter sets. So by Theorem 3.5, they are simultaneously conjugate: there exists a matrix $\alpha$ such that $\alpha h_i\alpha^{-1}=(h_i^{-1})^T$ for $i=0,1,\infty.$ The nondegenerate bilinear form given by $(u,v)=u^T\alpha v$ is invariant under each $h_i$:
$$(h_iu,h_iv)=u^Th_i^T\alpha h_iv=u^Th_i^T(h_i^{-1})^T\alpha v=u^T\alpha v=(u,v).$$


How to show that an estimator is consistent?

Question: Is it enough to show that MSE = 0 as $n\rightarrow\infty$? I also read in my notes something about plim. How do I find plim and use it to show that the estimator is consistent?
Answer: EDIT:  Fixed minor mistakes.
Here's one way to do it:
An estimator of $\theta$ (let's call it $T_n$) is consistent if it converges in probability to $\theta$.  Using your notation
$\mathrm{plim}_{n\rightarrow\infty}T_n = \theta $.
Convergence in probability, mathematically, means
$\lim\limits_{n\rightarrow\infty} P(|T_n - \theta|\geq \epsilon)= 0$ for all $\epsilon>0$.
The easiest way to show convergence in probability/consistency is to invoke Chebyshev's Inequality, which states:
$P((T_n - \theta)^2\geq \epsilon^2)\leq \frac{E(T_n - \theta)^2}{\epsilon^2}$.
Thus,
$P(|T_n - \theta|\geq \epsilon)=P((T_n - \theta)^2\geq \epsilon^2)\leq \frac{E(T_n - \theta)^2}{\epsilon^2}$.
And so you need to show that $E(T_n - \theta)^2$ goes to 0 as $n\rightarrow\infty$.
EDIT 2:  The above requires that the estimator is at least asymptotically unbiased.  As G. Jay Kerns points out, consider the estimator $T_n = \bar{X}_n+3$ (for estimating the mean $\mu$).  $T_n$ is biased both for finite $n$ and asymptotically, and $\mathrm{Var}(T_n)=\mathrm{Var}(\bar{X}_n)\rightarrow 0$ as $n\rightarrow \infty$.  However, $T_n$ is not a consistent estimator of $\mu$.
EDIT 3:  See cardinal's points in the comments below.
Comment: @MikeWierzbicki: I think we need to be very careful, in particular with what we mean by *asymptotically unbiased*. There are at least two different concepts that often receive this name and it's important to distinguish them. Note that it is *not true* in general that a consistent estimator is asymptotically unbiased in the sense that $\mathbb E T_n \to \theta$ even when the mean $\theta_n = \mathbb E T_n$ exists for all $n$. Many people call the convergence $\mathbb E T_n \to \theta$ *unbiasedness in the limit* or *approximate unbiasedness*...(cont.)
Comment: @G.JayKerns Unbiasedness is unnecessary for this.  Consider $S_n = \sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X_n})^2}$.  $S_n$ is a biased estimator of the standard deviation yet you can use the above argument to show that it's consistent.
Comment: Looks good (+1); and I'll delete my earlier comments.
Comment: Obviously, in order for a consistent estimator to be biased in the limit, the convergence in $L_2$ must fail since $\mathbb E(T_n - \theta)^2 = \mathrm{Var}(T_n) + (\theta_n - \theta)^2$ where $\theta_n = \mathbb E T_n$.
Comment: @cardinal, I think there is a typo in your comment since I'm pretty sure $E(T_n - \theta)^2 \rightarrow \theta$ is not enough ;)
Comment: @Macro: Wow. That one had escaped me even after rereading that a few times. This particular typo (and its complement) is actually quite common in older statistical texts (and a few newer ones); undoubtedly it is usually a transcription error due to a combination of poor handwriting and/or the transcriptionist not being fully versed on the material. I wish I could claim that excuse in this case!
Comment: @G.JayKerns Your comments were a necessary addition.  We must always make sure we're aware of the assumptions we're working under.
Comment: Just one more comment, and I'll go away.  The numerator of the Chebychev fraction really should be $E(T_{n} - \theta)^{2}$, which does indeed match the variance when $E T_{n} = \theta$.  Writing it that way, though, makes it clear that Beatrice was right with the first sentence of her question, because, of course, that's the definition of MSE.
Comment: Good point.  I just realized I completely ignored her first sentence.
Comment: @G.JayKerns: Please consider letting your comments remain in the future. I find them enlightening and sometimes they can be helpful for me (and others?) when reading the comment stream, even if the comments don't appear immediately relevant to an answer that has been edited to reflect the comments. :)
Comment: @cardinal Great comments - I was sloppy in one of my comments.  A consistent estimator need not be asymptotically unbiased -- I realize one of my comments from before may say otherwise, I think I shall delete it.  To summarize my response succinctly: if you can show that $T_n$ is asymptotically unbiased and $E(T_n - \theta)^2\rightarrow 0$ then $T_n$ is consistent.  However this is not the only way in which consistent estimators arise, nor is it the only way to prove consistency.  Is that where we stand in the discussion as of now?
Comment: @MikeWierzbicki: Thanks for the response. Note that you don't even need the "$T_n$ is asymptotically unbiased" part of your previous comment since $\mathbb E(T_n - \theta)^2 \to \theta$ is already enough. :) And, yes, this is certainly not the only way for consistency to arise, though it's one of the more common, as you previously noted. One easy way to convince yourself of the fact that $L_2$ convergence is not the only way is that the concept of convergence in probability would not be necessary otherwise! :)
Comment: Since $(E[T_n] - \theta)^2 = \mbox{var}(T_n) + (E[T_n-\theta])^2,$ doesn't the sufficient condition for an estimator to be consistent require both $\mathrm{lim}_{n\rightarrow\infty} \mbox{var}(T_n) = 0$ and $\mathrm{lim}_{n\rightarrow\infty} E[T_n] = \theta ?$


PHP Image don't upload

Question: I am dont' uploading image file to my path. I added enctype='multipart/form-data' in html code. That's php code;
$membeUrl = "inc/img/user/".$_SESSION['memberName'];

if(!is_dir($membeUrl)){mkdir($membeUrl);}
$profilePhoto = $membeUrl."/".$_FILES['profilePhoto']['name'];

if($_FILES['profilePhoto']['error'] > 0){
    alert("Profil photo error:".$_FILES['profilePhoto']['error']);
}

move_uploaded_file($_FILES['profilePhoto']['tmp_name'],$profilePhoto);

$w = " WHERE firstname.lastname@example.com' AND age='18'";

if((!preg_match("/[\-]{2,}|[;]|[']|[\\\*]/",$profilePhoto))){
    mysql_query("UPDATE member SET pp='".$profilePhoto."'".$w);
}

I don't see and understand. How we do solve?
Thank you for your insterest.
Good works..
Comment: @Sudhir How do you know? Maybe it has a php function definition in the source...
Comment: what's the output of
$_FILES['profilePhoto']['tmp_name'];
$profilePhoto
Comment: "alert(...)" is a javascript function not PHP
Comment: @hallaji well, its a guess, as the question does not mention anything about if "alert(..)" is a custom function done in PHP... !
Comment: what part exactly doesn't work? `move_uploaded_files()` or `mysql_query()`? What you get displayed with `var_dump($_FILES);`? Does the folder in which you move image exist and is writable? Are you using win or linux (unix-like)?
Comment: I have identified "alert" to view errors.

$_FILES['profilePhoto']['tmp_name'] is empty..
$profilePhoto is inc/img/member/fthmad9/DSC_0013.jpg

I am using xampp on the windows 8.

var_dump;
array(5) { ["name"]=> string(12) "DSC_0013.jpg" ["type"]=> string(0) "" ["tmp_name"]=> string(0) "" ["error"]=> int(1) ["size"]=> int(0) }
Answer: From var_dump() info you specified it is obvious file you are uploading exceeds php.ini upload_max_filesize directive (check the Error=1 code explanation in manual)
Either upload a smaller file or change php.ini value.
Also check that post_max_size ini directive is equal or larger than upload_max_filesize, since even if you put larger size for upload, post value is the absolute limit.
To know more about ini directives read manual: http://www.php.net/manual/en/ini.core.php#ini.upload-max-filesize

PHP Image don't upload


iOS Constraints of image view inside table view cell gets misplaced on scroll

Question: The constraints for the image view inside Table view cell gets distorted on scroll.
The constraints are fine on initial load and view. Once the table view cell goes out of the view and we scroll back to get it, the image view gets displaced.
This is the screenshot of single TableViewCell. The globe icon is misplaced from its original spot (left of name and designation) and overlaps on the text below.

Screenshot of prototype cell from storyboard

The black lines show the constraints added on the storyboard.
Using XCode 8.0 with Swift 3.
I am doing something wrong with the constraints ?
Comment: I find it very hard to understand your sketches, maybe take a screenshot from ur storyboard?
Comment: @YuchenZhong I have updated the question to include screenshot from storyboard.
Answer: It seems that you are missing a height constraint on the image view. Should be able to fix problem if you add that.
Also, maybe related as well, can you set your image view's content mode to aspect fit? Sometime like this:
ObjC
<code>[self.imageView setContentMode:UIViewContentModeScaleAspectFit];
</code>
Swift
<code>self.imageView.contentMode = .scaleAspectFit
</code>
Comment: Why I suggest you add a height is because, .. you don't have that. For images views, it would better to have one. Otherwise, if you think about it, the compiler cannot really tell how tall it is. After all, the height of the table is dynamic sometimes. So if you want to have a rule of thumb, always add a height and width constraints for image subviews inside a table view cell. And don't forget the set the `contentMode`.
Comment: It didn't work. I set the constraint height=40 for imageView. Also set the contentMode to aspect fit.
It asked for changing vertical hugging priority for "details" to 252. I did. Updated the frames.
The problem still persists.!
Comment: I am not much knowledgeable with Auto Layouts. It didn't work initially. I changed added constraints to a new view controller and then deleted the constraints, added to previous view controller and then clean, then run. It got resolved. FYI, I do have height attribute + I have ">=" for the title as well.

Can you explain a little more about your answer why it was the case ?
Comment: I cannot really understand what you did with "changed added constraints to a new view controller and then deleted the constraints, added to previous view controller and then clean ... ". I was suggesting you to add the height constraint because it seems that you were missing it from the screenshot that you posted. Adding that should help, if it doesn't, hmm, I don't know ... Yay, constraints are a bit tricky to get them all right at the very beginning.
Comment: I think it is important to start with a small number of constraints, and gradually add more. And also it is important to understand every constraint when you add them, otherwise, it will just be a mess later on and a nightmare to debug.
Comment: Well, yes, I added the height constraint. It still didn't solve. So I tried cleaning and starting afresh with new view controller, prototype cell. That's what I mentioned above.
Anyway, it seemed to work. Thanks. Constraints are bit tricky, but I am starting to get a grasp of it.
Comment: Can you let me know why you mentioned that adding "height" would solve it ? Can you explain of how you arrived at this so that I can get to debug next time ?


reverse proxy doesn't work with .svc url?

Question: here's my attempt to do a reverse proxy using url rewrite in IIS from mysubdomaintarget.mytargetdomain.com to mysubdomainreal.myrealdomain.com
<configuration>
    <system.webServer>
        <tracing>
            <traceFailedRequests>
                <add path="*">
                    <traceAreas>
                        <add provider="ASPNET" areas="Infrastructure,Module,Page,AppServices" verbosity="Verbose" />
                        <add provider="WWW Server" areas="Authentication,Security,Filter,StaticFile,CGI,Compression,Cache,RequestNotifications,Module,FastCGI,WebSocket,Rewrite,RequestRouting" verbosity="Verbose" />
                    </traceAreas>
                    <failureDefinitions statusCodes="400-999" />
                </add>
            </traceFailedRequests>
        </tracing>
        <rewrite>
            <rules>
                <rule name="ReverseProxyInboundRule1" stopProcessing="true">
                    <match url="(.*)" />
                    <action type="Rewrite" url="https://mysubdomainreal.myrealdomain.com/{R:1}" />
                    <serverVariables>
                        <set name="HTTP_X_ORIGINAL_ACCEPT_ENCODING" value="{HTTP_ACCEPT_ENCODING}" />
                        <set name="HTTP_ACCEPT_ENCODING" value="" />
                    </serverVariables>
                </rule>
            </rules>
            <outboundRules>
                <rule name="ReverseProxyOutboundRule1" preCondition="ResponseIsHtml1">
                    <match filterByTags="A, Form, Img" pattern="^http(s)?://mysubdomainreal.myrealdomain.com/(.*)" />
                    <action type="Rewrite" value="http{R:1}://mysubdomainreal.myrealdomain.com/{R:2}" />
                </rule>
                <rule name="RestoreAcceptEncoding" preCondition="NeedsRestoringAcceptEncoding">
                    <match serverVariable="HTTP_ACCEPT_ENCODING" pattern="^(.*)" />
                    <action type="Rewrite" value="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" />
                </rule>
                <preConditions>
                    <preCondition name="ResponseIsHtml1">
                        <add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" />
                    </preCondition>
                    <preCondition name="NeedsRestoringAcceptEncoding">
                        <add input="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" pattern=".+" />
                    </preCondition>
                </preConditions>
            </outboundRules>
        </rewrite>
    </system.webServer>
</configuration>

If I call website using mysubdomaintarget.mytargetdomain.com/myApp it works (render the page/etc). But if I call mysubdomaintarget.mytargetdomain.com/myApp.svc I got an 404.
Why? Where am I wrong on this configuration?
EDIT
Here's the failedRequest tag's attribute:
<failedRequest url="http://mysubdomaintarget.mytargetdomain.com/myApp.svc"
               siteId="6"
               appPoolId="services"
               processId="2192"
               verb="GET"
               remoteUserName=""
               userName=""
               tokenUserName="NT AUTHORITY\IUSR"
               authenticationType="anonymous"
               activityId="{8000002B-0000-9900-B63F-84710C7967BB}"
               failureReason="STATUS_CODE"
               statusCode="404"
               triggerStatusCode="404"
               timeTaken="141"
               xmlns:freb="http://schemas.microsoft.com/win/2006/06/iis/freb"
               >

EDIT2
xml opened by browser:
enter image description here
Comment: https://docs.microsoft.com/en-us/iis/extensions/url-rewrite-module/using-failed-request-tracing-to-trace-rewrite-rules
Comment: @LexLi enabled, and edited the answer! That's the first tag I got, but its not so "speaking". Which tag do I need to investigate more?
Comment: EDIT: added the whole web.config, so we can investigate further!
Comment: The article shows so clear how to read FRT log, so I wonder how much of it you have read.
Comment: @LexLi not really. I've read the log, there's nothing "wrong" on it.
Answer: This StackOverflow post from way back in 2013 suggests you may need to add the multipleSiteBindingsEnabled="true" attribute to the <serviceHostingEnvironment> tag of your WCF (.svc) service's web.config. Otherwise, WCF will be rejecting your HTTP requests because it does not believe that it is meant to be handling requests for that hostname and path.
Perhaps give that a try?
Comment: Already tried. Nothing changed...
Comment: Would you by chance be able to supply a *full* failed request tracing XML? The line you've provided gives no insights as to where the request lifecycle is failing.
Comment: looks at the image attached. better now?


How to represent beams broken with rests in LilyPond

Question: I'm trying to write the following in LilyPond notation:

As shown in the circled parts of the image, there are two different ways of representing a beam that spans a sixteenth note – a sixteenth rest – a sixteenth note.
In one of them, the beam is broken at the rest, in the other it is not.
I am representing this as:
<code>c16[ r16 c16]
</code>
This gives me the lower of the two patterns circled in the image.
But, how can I get the upper pattern with LilyPond notation?
Comment: Related: [Beams broken/unbroken with rests - meaning and usage](https://music.stackexchange.com/questions/118733/beams-broken-unbroken-with-rests-meaning-and-usage)
Answer: You can typeset this using
<code>\new RhythmicStaff
{
  \time 3/16
  16[ \once\set stemRightBeamCount = 1
      \once\set stemLeftBeamCount = 1
       r16 16]
  \time 9/16
  16[ r16 16] 16 16 r16 16 r r
}
</code>
and I don't see that using this at this place makes actual sense.
Comment: The `\once` commands don't do anything here.


Como concatenar variável PHP com coluna MySQL?"

Question: Tenho a seguinte consulta:
function retornaClientesDevedores ($mesReferencia, $dias) {

 require_once "PhpUtil.php";
 $PhpUtil = new PhpUtil();

 $string = "SELECT idClientes, nome FROM clientes WHERE idClientes IN (
     SELECT idClientes FROM planosclientes WHERE idPlanosClientes IN (
     SELECT distinct(idPlanoClientes) FROM pagamentos WHERE mesReferencia NOT IN ('".$mesReferencia."')
      ) and ".$PhpUtil->diferencaDatas(date('Y-m-d'), date('Y-m')."-diaVencimento'".)->days." > ".$dias."
      UNION
     SELECT idPlanosClientes FROM planosclientes WHERE idPlanosClientes NOT IN                                               (SELECT distinct(idPlanoClientes) FROM pagamentos)
                                             ) ";
    }

Minha dificuldade é que tenho uma função que vem de uma classe. Essa função pede duas datas. Uma atual e outra que preciso pegar a data atual sem o dia, portanto date('Y-m') e concatenar com um campo diaVencimento da tabela para montar a data. Como fazer essa concatenação no PHP?
Comment: Oi, Carlos, "Ajuda com consulta" é um título que não significa nada... para atrair a atenção de programadores interessados em te ajudar, faça um título que descreva exatamente o problema.
Comment: Obrigado @brasofilo. Vou ficar mais atento!
Comment: O que é `\"-diaVencimento\"` ?? Eu desconheço esse tipo de atribuição a métodos, principalmente, concatenado com uma função `date()`.  Explique...
Comment: diaVencimento é um campo da tabela. Preciso pegar esse valor e concatenar com o resultado de date ('Y-m')
Comment: um titulo adequado seria "como concatenar variável php com coluna mysql?"
Comment: Obrigado Daniel!
Answer: Para concatenar dois ou mais campos no MySQL utilize a função CONCAT da seguinte forma:
$sql = "SELECT CONCAT('".date('Y-m')."', '-', diaVencimento) AS data FROM pagamentos";

ou
$sql = "SELECT CONCAT('".date('Y-m-')."', diaVencimento) AS data FROM pagamentos";

Aplique conforme sua necessidade.
Espero que ajude, abraços
Answer: Pensando bem me parece que nem vou precisar disso.
Um Cliente tem um plano e se compromete a pagar todo dia '15' por exemplo. Então, no dia '7'$ por exemplo quero tirar a lista do mês.
Como só entra os que ainda não pagaram e que tem uma carência de 5 dias após a data, logo tria que fazer (diaHoje - DiaVencimento) o que dará (7-15 = -8) que é menor que os 5 dias de carência.
Na verdade é o seguinte
A função que estou usando é:
  function diferencaDatas ($dataInicio, $dataFim) {
$dataInicio = new DateTime($dataInicio);
$dataFim  = new DateTime($dataFim);
$diferencas = $dataInicio->diff($dataFim);

return $diferencas;
  }

Chamada:
$dataHoje = date('Y-m-d');
$dataVencimento = date('Y-m')."-dataVencimento ";

$diasDiferenca = diferencaDatas ($dataHoje , $dataVencimento ) ->days;

Saber quantos dias de diferença são.
Mas como a listagem será efetuada sempre dentro do mesmo mês, tanto  $diaHoje quanto o $diaVencimento ocorrerão no mesmo mês, logo não precisarei da função bastando fazer a simples diferença entre eles.
Estou certo em meu raciocínio?
Pois se eu vou tirar a listagem por exemplo no dia 25, o mês é 09. Logo, vou pegar a data da listagem (25) e tirar o dia do vencimento (caso ainda não tenha sido paga) se der até 5, esta no prazo, caso contrário, esta em débito.
classe corrigida.
<?php
 class Relatorios {

private $conexao;

public function __construct ($_conexao) {
    $this->conexao = $_conexao;
}
/*
     //COMENTÁRIO DA FUNÇÃO ABAIXO
//SELECIONAR OS ID'S E OS NOMES DOS CLIENTES
$string = "SELECT idClientes, nome FROM clientes WHERE idClientes IN (
                   //SELECIONAR ID'S DOS CLIENTES QUE TEM PLANOS E QUE A DIA DE VENCIMENTO É MENOR QUE A O DIA ATUAL
SELECT idClientes FROM planosclientes WHERE idPlanosClientes IN (
   //MAS NÃO PAGARAM A PARCELA DO MES CORRENTE
SELECT distinct(idPlanoClientes) FROM pagamentos WHERE mesReferencia NOT IN ('".$mesReferencia."')
                  ) and DATEDIFF (curdate(), CONCAT('".date('Y-m')."','-', diaVencimento)) > ".$dias."
  //UNIÃO COM
UNION
    //SELECIONAR ID'S DOS CLIENTES QUE TEM PLANOS E QUE NÃO PAGARAM NENHUMA PARCELA
SELECT idPlanosClientes FROM planosclientes WHERE idPlanosClientes NOT IN                                               (SELECT distinct(idPlanoClientes) FROM pagamentos)
  ) ";
*/

 function retornaClientesDevedores ($mesReferencia, $dias) {

$retorno = array();

$string = "SELECT DISTINCT(idClientes), nome FROM clientes WHERE idClientes IN (
              SELECT idClientes FROM planosclientes WHERE idPlanosClientes IN (
SELECT distinct(idPlanoClientes) FROM pagamentos WHERE mesReferencia NOT IN ('".$mesReferencia."')
) and DATEDIFF (curdate(), CONCAT('".date('Y-m')."','-', diaVencimento)) > ".$dias."
UNION
SELECT idPlanosClientes FROM planosclientes WHERE idPlanosClientes NOT IN                                               (SELECT distinct(idPlanoClientes) FROM pagamentos)
  ) ORDER BY nome";

$dados =  $this->conexao->query($string);

if($dados->num_rows == 0) {
$retorno = null;
}
else {
$contador = 0;
while ( list ($idClientes, $nomeClientes) = $dados->fetch_row())   {
$retorno[$contador][0] = $idClientes;
$retorno[$contador][1] = $nomeClientes;
$contador++;
}
}

      return $retorno;
  }
}
?>


Is a running back allowed to toss the football out of bounds to stop the clock?

Question: This question applies to all levels of American football from pee wee football all the way up to the NFL.
If a running back is past the line of scrimmage and is running towards the sideline with the intent of getting out of bounds to stop the clock but his path is blocked by defensive players, can the running back toss the football out of bounds to stop the clock?
I'm not talking about him passing the football over the heads of the defensive players into the sideline, but him tossing it like a softball towards the sideline and angling it a few yards back from his position on the field so its not a forward pass.
Will the ball be placed at the point where the ball entered the sideline and will this stop the clock?
Answer: This is illegal and penalized after the 2 minute warning:

NFL Rules 4.7.1 ILLEGAL ACTS

A team is not permitted to conserve time after the two-minute warning of either half by
    committing any of these acts:[...]
    (d) throwing a backward pass out of bounds;

Outside the two minute warning it is ruled as a ball out of bounds. The ball will be spotted at the point where it crosses the sideline and the gameclock will stop until the ref signaled that the ball has returned to the field of play.

NFL rules 8.7.2 BACKWARD PASS OUT OF BOUNDS

If a backward pass goes out of bounds between the goal lines, the ball is dead (7-6-2-a), and it is next put in play at the inbounds spot. Rule 11 governs if a backward pass is declared dead behind the goal line.

NFL rules 4.4 STOPPING THE GAME CLOCK

The game clock operator shall stop the game clock (timeout) upon a signal by any official or upon the operator's own positive knowledge:[...]
    (c) when the ball is out of bounds;

NFL rules 4.3.2 SCRIMMAGE DOWN

(f) If a fumble or backward pass by any player goes out of bounds, the game clock starts on the Referee's signal that a ball has been returned to the field of play.

So there is a minimal time gain possible, considering the time it takes for the ref to get the ball when the player is tackled in bounds, at the risk of losing a few yards and possibly the opponents stopping the ball before the sideline and avancing with it (even if it touched the ground in bounds, it still is a life ball)
Answer: The above answer is incomplete if not outright incorrect.  It's not just that it won't work--doing so would actually make the situation worse.

NFL Rules 4.7.1 ILLEGAL ACTS

A team is not permitted to conserve time after the two-minute warning of either half by committing any of these acts:
    ...
    (d) throwing a backward pass out of bounds
    ...
    Penalty: For Illegally Conserving Time: Loss of five yards unless a larger distance penalty is applicable.
    When actions referred to above are committed by the offensive team while time is in, officials will run off 10 seconds from the game clock before permitting the ball to be put in play on the ready-for-play signal unless another rule prescribes otherwise. The game clock will start on the ready-for-play signal. If the offensive team has timeouts remaining, it will have the option of using a timeout in lieu of a 10-second runoff, in which case the game clock will start on the snap after the timeout.

So not only does such an act risk recovery by the defense, it will not successfully conserve time and will in fact make the situation worse as (a) it will take a significant chunk of time off the clock with the; (b) the clock will start on ready-for-play rather than on the snap; and (c) there is a five-yard penalty on top of all that.
Comment: I know I was told I should do this as an edit to the existing answer rather than as a new answer, but I did so and it was rejected.  The existing answer is insufficient, so I see no other option but to post a new answer to complete it.
Comment: There is only one existing answer and it specifically calls out the illegality after the two-minute warning. Your proposed edit not only add nothing new along those lines, but deleted pertinent useful information against the wishes of the original poster; this is considered vandalism and the rejection notice makes that clear.
Comment: How is making something better "vandalism"?  The original response mentioned a penalty in passing, but said nothing about what that penalty was; furthermore, it included a bunch of information that I removed because it was irrelevant since the rules it cited about timing don't apply in this situation: they're superseded by the rules on timing that specifically apply to intentional fumbles out of bounds within the last two minutes.
Comment: It was clear that the second section of the answer refers to times prior to the warning, and therefore not superseded by rules applying after the warning; removing this section was inappropriate. It is not necessary to describe the entirety of every penalty, giving the rule that makes it illegal was sufficient to justify the answer.


JProfiler: How to load agent library?

Question: I'm trying to run jpenable on a standard java application in our remote server but I'm getting an error saying that:
ERROR: The agent could not be loaded: Failed to load agent library
Server OS:
Linux debian-server 2.6.32-5-amd64 #1 SMP Sat Mar 31 04:00:05 UTC 2012 x86_64 GNU/Linux
System variables:
LD_LIBRARY_PATH="/home/dev/jprofiler7/bin/linux-x64"
PATH="/home/dev/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/java/default/bin:/home/dev/jprofiler7/bin/linux-x64"

Java options:
JAVA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.net.preferIPv4Stack=true "
JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.trustStore=$HOME/keystorage/cacerts.jks -Djavax.net.ssl.trustStorePassword=$CACERTS_PASSWORD"

Any help would be appreciated.
Comment: The `LD_LIBRARY_PATH` should only contain the path not the file. Try it with `LD_LIBRARY_PATH=/home/dev/jprofiler7/bin/linux-x64`. Please also post the command line you are using to start Java.
Comment: Updated LD_LIBRARY_PATH and added Java options
Comment: Are you specifying these variables for jpenable? jpenable does not need any such variables. For Java 1.5+, you never need to modify LD_LIBRARY_PATH. Is JAVA_OPTS an environment variable that is used by your server start script? I think you're mixing two strategies here, jpenable is for profiling an already running JVM, adding an -agentpath VM parameter in the start script loads the profiling agent and jpenable is not required.
Comment: Yes, I'm using a start script for the app. I separated the option for agentpath so I can easily enable/disable. I tried running using both methods: (1)Disabling the agentpath option and running jpenable which gave me this error and (2)Enabling the agentpath option and starting the app which makes my app crash.

I'm removing the agentpath for clarity.
Answer: Got it to work. Problem was that the server machine uses 32-bit Java even if it's running 64-bit OS. I also had to download ia32-libs as I had missing dependencies as well.
There was also no need to modify/add LD_LIBRARY_PATH and PATH.

JProfiler: How to load agent library?


Choosing directories to output by timestamp named directories

Question: I have a bunch of directories that are named according to some timestamp. For example
/d_20141123_0435
/d_20141123_0436
/d_20141123_0437  <--- start
/d_20141123_0438
/d_20141123_0439
/d_20141123_0440
...
...
...
/d_20141124_0438
/d_20141124_0439
/d_20141124_0440  <-- end

I was wondering if there is a bash command that would output only certain dates I want. For example if I only wanted the output of directories between 20141123-0437 to 20141124-0440
ls (logic) >> output.text
Answer: The ls command does not have such a functionality but you can use a simple script. Output of ls should no be parsed but if you really want to use it, replace * by $(ls).
If the boundaries should not be included:

after=d_20141123_0437
before=d_20141124_0440
for a in * ; do
    if test "$a" \> "$after" && test "$a" \< "$before" ; then
        echo "$a"
    fi
done >> output.text

If they should be included:
after=d_20141123_0437
before=d_20141124_0440
for a in * ; do
    if { test "$a" \> "$after" && test "$a" \< "$before" ; } ||
            test "$a" = "$after" || test "$a" = "$before" ; then
        echo "$a"
    fi
done >> output.text

Both scripts can be on a single line if you replace newlines after commands by ;.
Comment: This is probably the most portable script which would work on old shells also +1
Answer: Assuming these directories in a file like shown you can use this awk:
awk -F_ -v s1='20141123-0437' -v s2='20141124-0440' 'BEGIN{
   split(s1, a, "-");
   split(s2, b, "-");
   start = a[1]a[2];
   end = b[1]b[2];
}
int($2$3) >= start && int($2$3) <= end' file

/d_20141123_0437  <--- start
/d_20141123_0438
/d_20141123_0439
/d_20141123_0440
/d_20141124_0438
/d_20141124_0439
/d_20141124_0440  <-- end

Comment: To show the problem directly on your script and the original data. Let's set the end value to the fourth directory from the end of the list: `s2=20141123-0440`. The resulting list wrongly ends at the second directory from the end. ------- To resolve the problem you can just take all the `+` operators away as I wrote in my first comment. (I wrongly wrote `$1$2` instead of `$2$3` there.)
Comment: @anubhava i was the one who downvoted, i was doing something else though and forgot to leave a comment. I can't actually remember what the downvote was for and looking at the script now it looks fine. If you edit the answer i'll remove the vote :)
Comment: The downvote is not from me. I did not thoroughly check your script before but now I see there is a bug there. Anyway I think it is a good idea to use `awk` for this problem. ------ The operator `+` you are using is performing a simple arithmetic addition but the operands do not represent the same thing (they are not of the same magnitude). ------ One possibility to correct the problem is to adjust the magnitude: `10000*$2+$3`. The other, shorter possibility is to use a string concatenation instead of arithmetic addition: `$1$2`.
Comment: @pabouk: Reason why `+` is also working (and output is also shown) because date is written as `yyyymmdd_hhmm` so sum is basically e.g. `20141124 + 0440` and being compared to `20141124-0440`, there replacing `-` by `+` so it becomes `20141124 + 0440`.
Comment: You are comparing numbers created the same way (`$2+$3` and `x[1]+x[2]`). I think that is clear and that is not the problem I was trying to point out. The problem is how you create the numbers. You are adding different things together - date and time. That is wrong. ------ Practical example: Let's compare the fourth and third line from the end of the list your way: `awk -F_ '{print $2,$3,$2+$3}' <<<$'d_20141123_0440\nd_20141124_0438'`. You see that the order is the opposite than it should be.
Comment: Thanks so much @pabouk for your valuable comments. I have updated awk script based on that.
Comment: @Jidder: No issues at all, I have edited the answer.
Answer: Sure, brace expansion is your friend:
ls d_20141123_0{437..440} >> output.text

brace expansion works by providing a sequence of numbers between {start..end}. So you can form any filename match you need by replacing the number sequence you are interested in with a brace expansion. In your case d_20141123_0{start..end}
Comment: Is not this something different than Liondancer asked for? Brace expansion generates a list of strings regardless of existence of the files/directories. ------ You can use something like `[d]_20141123_0{start..end}` but in `bash` you must enable `shopt -s nullglob`. Unfortunately This would not be very portable.
Comment: I do not think we know each other. --- You skipped the main point which is in the first two sentences of my comment. Portability was just an additional minor note almost hidden at the end :)
Comment: My friend, as with other tasks, there is more than one way to skin this cat. Chose whichever one suits your liking. The question is tagged **bash**, this is what **brace expansion** in **bash** was intended to do. Porability from **bash** to **bash** does not present a problem.
Comment: If needling other people about issues not asked for in the question is what makes you happy, feel free. Otherwise, please try an limit your comments and critiques to the current question. Your answer will get chosen just as often.


Find all solutions for $7x^2 \equiv 3 \mod5$, if any.

Question: First of all in $\mod5$, the congruence could be reduced to $2x^2 \equiv 3 \mod5$.
Since $2$ is a primitive root for $\mod5$, we have $2i\equiv3 \mod4$.
But $\gcd(2,4)=2$ and $2$ does not divide $3$, hence there are no solutions.
Is this correct? Where would I factor in the $2$ from the original congruence? I suspect there is indeed a solution but my calculation does not take the $2$ from $2x^2 \equiv 3 \mod5$ into consideration. Any help will be greatly appreciated.
Comment: Note that there are only $5$ numbers modulo $5$, so finding the solutions directly is always easy.
Comment: There are two solutions, $x\equiv \pm 2$.
Comment: Shouldn't it be $2i \equiv 3 \pmod{5}$?
Comment: No, because when equating the exponents, we have to solve in mod(phi(n))=mod(phi(5))=mod(4).
Answer: An easier way than you're attempting: 3 is the multiplicative inverse of 2 modulo 5, so to solve $2x^2 \equiv 3\; (\textrm{mod}\; 5)$, we can multiply through by 3 and look for solutions to $x^2 \equiv 9 \equiv 4\; (\textrm{mod}\; 5)$, whereby we see $x = \pm2$ are the only solutions.
Answer: Or, $\ 2x^2\equiv -2\,\Rightarrow\, x^2\equiv -1\equiv 4\,\Rightarrow\,x\equiv \ldots$
Answer: If you rewrite the original equation you have
$$2(y)\equiv 3 \mod 5 \implies y \equiv 4$$
The questions then becomes
$$x^2 \equiv 4 \mod 5$$ which is true for $x\equiv \pm2$


Ideal of cusp forms for $\Gamma_0(4)$ is principal

Question: Let $\Gamma_0(4)$ be a congruence subgroup of $SL(2,\mathbb{Z})$ defined as
$$\Gamma_0(4)=\{M=\begin{pmatrix}
a &b\\
c& d
\end{pmatrix}\in SL(2,\mathbb{Z})\mid c=0\bmod 4\}.$$
Dedekind eta-function is defined as
$$\eta(z)=q^{1/24}\prod_{n\geq1}(1-q^n).$$
How to prove that the ideal of cusp forms for $\Gamma_0(4)$ is principal and generated by
$$f(z)=\eta(2z)^{12}?$$
Answer: The form $f(z)$ generates the $\Gamma_0(4)$ cuspforms (as an ideal over the
graded ring of all modular forms for $\Gamma_0(4)$) for much the same
reason that $\Delta(z) = \eta(z)^{24}$ generates the cuspforms for
the full modular group $\Gamma(1) = {\rm SL}_2({\bf Z})$: this $f$ is
a cuspform with no zeros in the finite plane, and it vanishes at each cusp
only to the multiplicity needed to make it a cuspform.  Hence if $\varphi$
is any other cuspform, say of weight $w$, then $\varphi_1 := \varphi/f$
is a modular form of weight $w-6$, and we have written $\varphi$ as
$f \varphi_1$ for some modular form $\varphi_1$ as desired.
The fact that there are no zeros in the finite plane is clear from
the product formula for $\eta(z)$.  The vanishing order at the cusps
is easier to see if we change variables to $w := 2z$, because
then $f(z) = \eta(w)^{12}$ is a cuspform for the action of $\Gamma(2)$
on $w$, and the three cusps of $\Gamma(2)$ all have the same width
(namely $2$) because they're transitively permuted by $\Gamma(1)$,
so $\eta(w)^{12}$ vanishes to order $1$ at each cusp.
Similar examples for other groups are $\eta(3z)^8$ for $\Gamma_0(9)$
(equivalently $\eta(z)^8$ for $\Gamma(3)$), and $(\eta(z)\eta(pz))^w$
for $\Gamma_0(p)$ with $p=2,3,5,11$ (i.e. the primes with $p+1\mid12$) and
$(p+1)w=24$.  The last of these also works for $p=7$ and $p=23$
if we allow modular forms with quadratic character.
Comment: Noam, thank you! There is a little misprint in your answer: it seems that it should be $\phi_1=\phi/f$.
Comment: You're welcome, and thank you for the correction; I'll fix it next
(though retaining $\varphi$ in place of $\phi$ [**\varphi** vs. **\phi**
in TeX).
Comment: Dear Noam, can you explain please why the coefficients of the expansion of $f(z)$ are multiplicative? I know that this fact is related to the notion of Hecke operators for the group $\Gamma_0(4)$ but I haven't found the suitable reference. Actually as I understand I need to prove that $f(z)$ (which is normalized) is the eigenform of Hecke operators for $\Gamma_0(4)$. Thank you.
Comment: In Zagier's "1-2-3" the case of congruence subgroups of $SL_2(\mathbb{Z})$ is not explained.
Comment: The Hecke operators act on the space of cusp forms, and since that space
is one-dimensional $f$ is automatically an eigenform.  Sorry I don't have
a reference to hand (maybe somebody else here can suggest one) but you might
be able to figure  out for yourself how to generalize the definition
of $T_m$ that you learned from Zagier to congruence groups $\Gamma_0(n)$
with $\gcd(m,n)=1$  (and how this yields multiplicativity at least for
coefficients of $q^k$ with $\gcd(k,n)=1$).


Pandas DataFrame skip rows

Question: I am working on a weather webscraping project, and have scraped a site using selenium and exported it to excel using pandas. However, i can't find out how to make dates only appear in every fourth row, so that dates would fit in with the time. Excel Sheet: https://i.stack.imgur.com/27w0f.jpg. Full code:
from selenium import webdriver
from selenium.webdriver.common.by import By
import pandas as pd
from datetime import datetime

url="https://pent.no/60.19401,11.09936"

dates = "forecast-day-view-date-bar__date"
times = "forecast-hour-view-hour-label"
temps = "forecast-hour-view-weather-widget__temperature"
winder = "forecast-hour-view-weather-widget__wind-speed"
rainfalls = "forecast-hour-view-weather-widget__precipitation"

driver = webdriver.Chrome()
driver.get(url)

date = driver.find_elements_by_class_name(dates)

time = driver.find_elements_by_class_name(times)

temp = driver.find_elements_by_class_name(temps)
temp2 = temp[::2]
temp3 = temp[1::2]

wind = driver.find_elements_by_class_name(winder)
wind2 = wind[::2]
wind3 = wind[1::2]

rainfall = driver.find_elements_by_class_name(rainfalls)
rainfall2 = rainfall[::2]
rainfall3 = rainfall[1::2]

a = []
b = []
c = []
d = []
e = []
f = []
g = []
h = []

for dates in date:
    print(dates.text)
    a.append(dates.text)
    df1 = pd.DataFrame(a, columns= ["Date"])

#
for times in time:
    print(times.text)
    b.append(times.text)
    df2 = pd.DataFrame(b, columns= ["Time"])
#
for tempyr in temp2:
    print(tempyr.text)
    c.append(tempyr.text)
    df3 = pd.DataFrame(c, columns= ["Temp Yr"])

for tempstorm in temp3:
    print(tempstorm.text)
    d.append(tempstorm.text)
    df4 = pd.DataFrame(d, columns= ["Temp Storm"])
#
for windyr in wind2:
    print(windyr.text)
    e.append(windyr.text)
    df5 = pd.DataFrame(e, columns= ["Wind Yr"])

for windstorm in wind3:
    print(windstorm.text)
    f.append(windstorm.text)
    df6 = pd.DataFrame(f, columns= ["Wind Storm"])
#
for rainfallyr in rainfall2:
    print(rainfallyr.text)
    g.append(rainfallyr.text)
    df7 = pd.DataFrame(g, columns= ["Rainfall Yr"])

for rainfallstorm in rainfall3:
    print(rainfallstorm.text)
    h.append(rainfallstorm.text)
    df8 = pd.DataFrame(h, columns= ["Rainfall Storm"])
#
tabell = [df1, df2, df3, df4, df5, df6, df7, df8]
result = pd.concat(tabell, axis=1)

result.to_excel("weather" + str(int(datetime.now().day)) + ".xlsx")

driver.quit()

Answer: Try to create a variable which ensures its the 4th multiple iteration with in loop.
Check the below snippet.
row = 0
for dates in date:
    print(dates.text)
    a.append(dates.text) if row % 4 == 0 else a.append("")
    row = row + 1
df1 = pd.DataFrame(a, columns= ["Date"])

** Quick Check :**
a = []
row = 0

for i in range(10):
    a.append(i) if row % 4 == 0 else a.append("")
    row += 1

df = pd.DataFrame(a, columns= ["Date"])

print(df)

    Date
0   0
1
2
3
4   4
5
6
7
8   8
9

One more suggestion is to have a dataframe creation out of the loop not with in the loop.
Comment: I only get every fourth day with your tips. To fix this, i assume the date results has to be: 0, 1, 2 every fourth row instead. You forgot to include row += 1 in the original code.
Comment: It prints every fourth date when i include row +=1, which is kind useless. I am struggling to find a new code now that has the same results as my previous comment.
Comment: I finally solved the problem. The easy solution is:
`for dates in date:
    print(dates.text)
    a.append(dates.text)
    a.append("")
    a.append("")
    a.append("")` Thanks for the help :)
Comment: Yeah.. We should have solved simply with ```for dates in date:   a.extend([dates.text,"","",""])```

Pandas DataFrame skip rows


theming "field collection" in node pages drupal 7

Question: Drupal 7 "field collection" - If I want to theme a node page which has a "field collection", how do I go about separating the fields for customization? It seems thus far that only print render($content['field_collection_name']); outputs the result. The problem is that I can't figure out how to isolate the fields for theming.
I've created a node--content_type.tpl.php for this purpose but now stuck dead in the water. Would really appreciate someone saving the day for me.
Solved as follows:
Some of the links from below are really very helpful so do look, though, I've done something like this -
Enter dsm(print_r($variables));on node to be rendered in my case node--content_type.tpl.php with field collection to find out the arrays and field values needed, then hide of course.
The styling part then would look something like this

class="class-for-label-here">My Desired Label - <?php print render($content['my_desired_field_here']['#items'][0]['value']); ?>

Answer: There's a fairly good conversation about this same thing going on here:
http://drupal.org/node/1155752
Main answer is theming with template field--your-field-collection-name.tpl.php
After that you just need to figure out how to retrieve the contents from the arrays and/or objects!
Answer: You may create create a custom field formatter for your field and use this formatter in the set-up of your content type.
Field formatter gives total control over the html output before it is rendered into the node template.
You can use this great tutorial to create a custom field formatter : http://www.computerminds.co.uk/articles/creating-new-field-formatters-drupal-7
Comment: more elegant, will definitely give this a try
Answer: For a more normal, template based theming experience try the patch in http://drupal.org/node/1187990. Hopefully this will go into Field Collection 2.x.


Periodic Solution to ODE.

Question: We have the differential equation $x'(t)=ax+b(t)$ such that $b(T+t)=b(t)$ $\forall t \in R$ i.e. $b$ is periodic with $t$.
I need to show that if a is non-zero, then there is one and only one periodic solution $x(t)$ with period $T$ i.e $x(t)=x(t+T)$ $\forall t \in R$.
I have partially solved the equation with initial condition $x(0)=c$ first.
Comment: You might want to use Picard, but you can do it without it too, if you use the integrating factor method.
Comment: One possible approach: Suppose $x_1(t)$ and $x_2(t)$ are two period solutions of the ODE with period T, verify $\int_0^T |x_1(t)-x_2(t)|^2 dt = 0$.
Answer: Suppose $x_1(t)$ and $x_2(t)$ are two periodic solutions for the ODE
$$\frac{d}{dt}x(t) = a(t)x(t) + b(t)$$
with period T and $a(t)$ never change sign. Let $y(t) = x_1(t) - x_2(t)$, we have:
$$y'(t) = a(t)y(t)  \,\,\text{ and }\,\,   y(t) = y(t+T)$$
Multiply both side by $y(t)$ and integrate, we get:
$$\int_{0}^{T} a(t)y(t)^2 dt = \int_{0}^{T} y(t)y'(t) dt = \frac12 \int_{0}^{T} \frac{d}{dt} y(t)^2 dt = \frac12 \left[ y(t)^2 \right]_0^T = 0$$
Since $a(t)$ never change sign, this is only possible when $y(t) \equiv 0$ over $[0,T]$.
Since $y(t)$ is periodic, we get $x_1(t) = x_2(t)$ for all $t \ge 0$.
The condition that $a(t)$ is non-zero is important. For example, when $a(t) \equiv 0$ and $x_1(t)$ is a period solution, so does any $x_1(t) + const$.
EDIT
Since I'm doing this, let us prove this again using the integrating factor.
Let $x(t)$ be any period solution of the ODE with period $T$. Let
$$\phi(t) = e^{-\int_0^t a(s)ds} \implies \frac{d}{dt}\phi(t) = -a(t) \phi(t)$$
We have:
$$\begin{align}
&\frac{d}{dt} [x(t)\phi(t)] = x'(t)\phi(t) - a(t)x(t)\phi(t) = b(t)\phi(t)\\
\implies & x(T)\phi(T) - x(0)\phi(0) =  \int_{0}^{T} \frac{d}{dt}[ x(t)\phi(t) ]dt = \int_{0}^{T} b(t)\phi(t)dt\\
\implies & (\phi(T) - 1) x(0) = \int_{0}^{T} b(t)\phi(t)dt\tag{*}
\end{align}$$
If $a(t)$ never changes sign, $\int_{0}^T a(t)dt \ne 0$ and hence $\phi(T) \ne 1$,
then $(*)$ uniquely fixes the initial value $x(0)$. By the fundamental theorem of ODE, there is only one solution for the ODE with this initial value $x(0)$. So the period solution $x(t)$ is unique.
Comment: Thank you:), but how does y′(t)=a(t)y(t)?
Comment: @seanyAdams your ODE is linear in $x$! $$y'(t) = x_1'(t) - x_2'(t) = (a(t) x_1(t) + b(t) ) - (a(t)x_2(t) + b(t)) = a(t) (x_1(t) - x_2(t)) = a(t) y(t)$$
Comment: Ah yeas of course!:)
Comment: You argument shows uniqueness but says nothing about existence.
Comment: @Artem, will, start from any $x_0$, $x(t) = \phi(t)^{-1}\left(x_0\phi(0) + \int_0^t b(s)\phi(s)ds\right)$ provides a solution to the ODE with initial condition $x(0) = x_0$. If you choose $x_0$ to be the one that satisfies $(*)$, then you get a periodic solution.
Comment: @achille I know this, I just noted (browsing randomly through older problems) that the arguments you provided do not answer the question fully.
Answer: Solve by integrating factor method. There is a unique solution as this is linear regardless of any of constraints mentioned. $a$ being nonzero does not seem relevant.
$x'-ax=b$.  multiply with $m=e^{\int -a(t) dt}$. Rewrite as $(mx)'=bm$. Then $mx=\int bm +K$ and
$x=(\int bm +K)/m$. If $a$ and $b$ have the same period or $a$ is constant and $b$ periodic then $x$ will be periodic.
Comment: Apparently $a$ is a constant, so it is fine.

Periodic Solution to ODE.


How to (get or print) the value of my button on tkinter

Question: How can I get the value/text also print of my button A. B. C.? What should the right code to get that value.. thanks a lot!
Here is my sample code.. I'm starting to study the tkinter for my personal project and I want to create my own Multiple Choice Quiz. I already search the net but cant find the answer to my question :( Your help really appreciated!
import tkinter as tk

root = tk.Tk()

root.title("MultipleChoice")
root.geometry("500x500")

title = tk.Label(text="Multiple Choice Quiz")
title.place(x=250, y=10, anchor="center")
title.pack()

def ff():
    tk.Label(root, text= "").pack()

A = tk.Button(root, text="A", command = ff)
A.pack()
B = tk.Button(root, text="B", command = ff)
B.pack()
C = tk.Button(root, text="C", command = ff)
C.pack()

root.mainloop()

Comment: If you want to get the text of button A, you can use `A.cget('text')` or `A['text']`.
Answer: Maybe something like this is what you're looking for. This prints the corresponding buttons text into the console window:
import tkinter as tk

root = tk.Tk()

root.title("MultipleChoice")
root.geometry("500x500")

title = tk.Label(text="Multiple Choice Quiz")
title.place(x=250, y=10, anchor="center")
title.pack()

def ff(input):
    print(input)

A = tk.Button(root, text="A", command = lambda: ff(A['text']))
A.pack()
B = tk.Button(root, text="B", command = lambda: ff(B['text']))
B.pack()
C = tk.Button(root, text="C", command = lambda: ff(C['text']))
C.pack()

root.mainloop()


How to optionally pass in values to a closure in Swift?

Question: I have the following function which reads some heart rate data and accepts a callback in the form of a closure, as such:
func readHeartRate(callback: ((samples: [HKQuantitySample]?, average: Double?, error: NSError!) -> Void)!){
   // ...
   // ERROR - call the callback with no samples or average
   callback(samples: nil, average: nil, error: error)
})

Right now, I'm explicitly passing back nil for samples and average to the callback.
However, since samples and error are optional, and since I declared the names in the closure explicitly, I thought this would be possible, instead:
callback(error: error)

However, when I do this xcode complains and throws the error:
Missing argument for parameter 'samples' in call

Is there a way to call the callback with only the values necessary to avoid cluttering my code?
Answer: From The Swift Programming Language:

Closure expression syntax can use constant parameters, variable parameters, and inout parameters. Default values cannot be provided. Variadic parameters can be used if you name the variadic parameter and place it last in the parameter list.

So only way to be able to call closure with variable number of specified parameters is by using variadic params, which - I assume - is not what you want.
Answer: For a regular method, you can provide a default value for a parameter:
func myCallback(#error: NSError!, samples: [HKQuantitySample]? = nil, average: Double? = nil) {
}

myCallback(error: nil)

These parameters with default values should be at the end of the argument list.
This doesn't work in your case, since you're not defining a method; you're just specifying the method signature that callers need to pass in for the callback. You can say "pass in a method that takes an Int", for example, but there's no way currently to say "pass in a method that takes an Int and uses a default of 0 if I don't pass anything in".
Remember, optional values just mean the existence of a value is optional and the thing can be nil.
Comment: "These parameters with default values should be at the end of the argument list." - this holds only for unnamed params, i.e. if you have named params, any one of them can have default value.
Comment: It's more for consistency to keep argument order the same. "Place parameters with default values at the end of a function's parameter list. This ensures that all calls to the function use the same order for their non-default arguments, and makes it clear that the same function is being called in each case." – [The Swift Programming Language](https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/Functions.html#//apple_ref/doc/uid/TP40014097-CH10-ID166)
Comment: You are right, my bad. I took it for imperative instead of recommendation, but 'should' expresses the meaning properly.
Answer: I have had occasions where I decided to wrap a closure in a function so that I could provide a default parameter. It works like this:
let myClosure: String -> () = { string in
    println(string)
}

func myFunc(string: String = "default") {
    myClosure(string)
}

myFunc()       // prints "default"

I know that your goal is to "declutter" your code, so whether or not writing wrapper functions is what you want, I don't know, but it works.


Proxy server setting from Group policy

Question: I have added a computer based group policy to add a proxy server to a group of PCs. I used the user config in GP editor, but when I log in and run gpresults it displays "filtering: not applied" and the proxy server details are not applied.
Microsoft (R) Windows (R) XP Operating System Group Policy Result tool v2.0
Copyright (C) Microsoft Corp. 1981-2001
Created On 16/08/2013 at 14:23:58
RSOP results for xxxxxxxxxxx : Logging Mode
OS Type:                     Microsoft Windows XP Professional
OS Configuration:            Member Workstation
OS Version:                  5.1.2600
Domain Name:                 EUROPE
Domain Type:                 Windows 2000
Site Name:                   xxxx
Roaming Profile:
Local Profile:               C:\Documents and Settings\xxxx
Connected over a slow link?: No
COMPUTER SETTINGS
CN=xx,OU=AutoProxyLon,OU=xx,OU=PC,DC=wineur,DC=xxx,DC=com
Last time Group Policy was applied: 16/08/2013 at 13:39:26
Group Policy was applied from:      xx
Group Policy slow link threshold:   500 kbps

Applied Group Policy Objects
-----------------------------
    Domain Password Policy
    pc-services
    no au
    Default Domain Policy

The following GPOs were not applied because they were filtered out
-------------------------------------------------------------------
    LONPROXY
        Filtering:  Not Applied (Empty)

    ocr
        Filtering:  Not Applied (Empty)

    Local Group Policy
        Filtering:  Not Applied (Empty)

The computer is a part of the following security groups:
--------------------------------------------------------
    BUILTIN\Administrators
    Everyone
    BUILTIN\Users
    NT AUTHORITY\NETWORK
    NT AUTHORITY\Authenticated Users
    TECHD31$
    Domain Computers

USER SETTINGS
CN=Admin xxxx,CN=Users,DC=wineur,DC=xxx,DC=xxx
Last time Group Policy was applied: 16/08/2013 at 13:39:26
Group Policy was applied from:      xxxxx
Group Policy slow link threshold:   500 kbps

Applied Group Policy Objects
-----------------------------
    Default Domain Policy

The following GPOs were not applied because they were filtered out
-------------------------------------------------------------------
    Local Group Policy
        Filtering:  Not Applied (Empty)

    Domain Password Policy
        Filtering:  Not Applied (Empty)

The user is a part of the following security groups:
----------------------------------------------------

Comment: Can you include the gpresults and gp modeling wizard results?
Answer: It says Empty because the settings you have added are not in the Computer Settings section. As you can add settings into users and computer settings it's important you add the settings you require into the correct place.
I would check which section you are applying the settings to and move them to Computer Settings.
Comment: This is correct. The GPO either needs to be applied to an OU that contains the users you care about, or you need to configure loopback processing to force User settings to apply to the PCs in the OU.
Comment: Ok then it looks like you cannot add a proxy server based on computer settings.
Comment: I don't think you can, they are not settings for the computer, you can add the settings for users but then prevent users from changing them, please see this TechNet Wiki page for a great step by step... http://social.technet.microsoft.com/wiki/contents/articles/5156.how-to-force-proxy-settings-via-group-policy.aspx
Comment: Please re-read what I said about loopback processing. You can apply User settings to Computers via LB processing.
Comment: Thanks mfinni Ive tried to implement it by loopback but it still is not working I must be doing something wrong..
Comment: My domain level is Windows Server 2003 I wonder would that be the reason I cannot do LB Processing. Anyone have an idea on how to add a .pac file using GP or even at this stage .reg.
Comment: Troubleshoot why it's not working via loopback processing. Run the RSOP now that you've made these changes
Comment: RSOP does not have anything in there about this setting. Under scope I presume I need to leave authenticated users selected and not domain computers.
Comment: Looking at this document for inspiration - http://technet.microsoft.com/en-us/magazine/2007.02.troubleshooting.aspx
Comment: Ended up deleting this and doing a user based policy, which worked straight away but is not exactly what I was looking for.. didn't have the time to troubleshoot much thanks for the help.

Proxy server setting from Group policy


Add condition group in Views filter handler

Question: I have a Views filter handler, which does
$this->query->add_where(
  $this->options['group'],
  $this->real_field,
  vms_get_current_mlid(),
  $this->operator = ($this->operator == '=') ? 'IN' : 'NOT IN'
);

Which initially looks fine, but fails with "NOT IN", when the field value is NULL, as NOT IN (100) does not return TRUE for a NULL value, which is what I would like.
When the operator is 'NOT IN', I need to add a "OR IS NULL" operator as well, but still keep it within the same group.
If the filters are

Node is "page" AND NID_REFERENCE NOT IN (100, 101, 102)

I would like that to become

Node is "page" AND (NID_REFERENCE NOT IN (100, 101, 102) OR NID_REFERENCE IS NOT NULL)

How do I do that?
Answer: The answer turns out to be that  add_where() also accepts a DatabaseCondition argument, meaning I could do
$this->query->add_where(
  $group,
  db_or()
    ->condition($table, $vms_target, 'NOT IN')
    ->condition($table, $vms_target, 'IS NULL')
)

Comment: Doh! Glad you got it working!
Answer: I couldn't quite wrap my head around those queries in your last comment, but instead I tried exploring how Views does between operations. Check out db_or() which is used by views_handler_filter_numeric::op_between() - I think that will do it.
Comment: I tested this, and the problem is that the filter is moved to a _new_ group, which means the View will no longer respect the filter group settings. Somehow the existing group needs a _sub_ group, for this to work.
Comment: @Letharion I'm not sure what you mean by _the filter group settings_? Isn't the only group setting the operator (ie _AND_ or _OR_)? I think I'm missing something :)
Comment: My bad. If the View has, set in the UI, two groups of filters, like `(node.type = 'page' AND nid_ref NOT IN (100, 101)) AND (nid_ref IN (102))`, your suggestion turns this into `(node.type = 'page') AND (nid_ref IN (102)) AND/OR (nid_ref NOT IN (100, 101))` as opposed to `(node.type = 'page' AND (nid_ref NOT IN (100, 101) OR nid_ref IS NOT NULL)) AND (nid_ref IN (102))` because you add a $new_group, instead of manipulating the existing one.
Comment: Hehe, yeah, the queries got kinda long :P Suffice it to say that the parenthesis moved around in a way that broke the query. :)


Something more than multi-label classification?

Question: My training data has millions of rows with diagnosis and medicines prescribed for them by multiple doctors.
An example row:-
<code>Doctor name:  John ;
Diagnosis:  diabetes, hypertension;
Medications:  ABC, XYZ, KLP, FG-2 ;
</code>
Note: We are not aware which medicine is used to treat which diagnosis.
I need to be able to predict the medicines the doctor will prescribe given the diagnosis and doctor name.
Which machine learning modelling can be used here? I feel it's a bit more complex that multi-label classification.
Answer: There are multiple algorithms you can try (nearest neighbors, approximating probability distribution and using it for ML estimator), but in this case, it looks that the best way is to look the most frequent medication that a doctor uses for a given diagnosis and take that medication.
It's true there are the cases in which the doctor has not seen the disease yet but it's solvable (through other doctors maybe).
A better method can be used if you assume more about the model, but it's great for starters.


Prove that the set of integers is a Borel Set

Question: I understand the general definition of Borel Set. I also managed to prove the interval (a, b) is a Borel set. However, I'm having trouble proving mathematically that the set of integers is also a Borel Set. Gut feel says that I can use the fact that a singleton {a} is a Borel set by writing it as a countable infinite intersection of open intervals. Any help will be appreciated in systematically proving that a set of integers is actually a Borel Set.
Comment: Hint: Can you prove that $\{a\}, a \in \mathbb R$ is also a Borel Set?Hence Conclude.
Comment: " a singleton {a} is a Borel set by writing it as a infinite countable unions of open intervals " hopefully this is not really what you meant.
Comment: @AndrésE.Caicedo edited. My bad,
Answer: A singleton set is a Borel set since each integer $a \in \mathbb{Z}$ can be written as $\displaystyle \{a\} = \bigcap_{n=1}^\infty (a-\frac{1}{n}, a + \frac{1}{n})$, then the given set $E \subset \mathbb{Z}$ is clearly a Borel set, since the collection of Borel sets is a $\sigma$-algebra.
Comment: Got it. Many thanks. Just got a little confused by the set E in your answer :)
Comment: $E$ is just any set of integers, :)


Como criar um elemento em SVG dinamiacmente?

Question: Quero utilizar um número de uma variável e criar vários objetos SVG iguais. Apenas mudar a sua posição e nome em JavaScript.
Como no laço for exemplificado no pseudo-código:
for(i=0; i<variavel; i++){
    <circle title=circle cx="250" cy="250" r="5" stroke="black" stroke-width="2" fill="green" />
}

Answer: Aqui fica uma sugestão baseada nesta pergunta.
No fundo crio o elemento novo dentro de uma função, assim o ciclo for fica mais limpo.

var variavel = 10;
var svg = document.querySelector('svg');

for (var i = 0; i < variavel; i++) {
    criarCirculo(i);
}

function criarCirculo(tamanho) {
    var shape = document.createElementNS("http://www.w3.org/2000/svg", "circle");
    shape.setAttributeNS(null, "cx", numeroAleatório(tamanho));
    shape.setAttributeNS(null, "cy", numeroAleatório(tamanho));
    shape.setAttributeNS(null, "r", 20);
    shape.setAttributeNS(null, "fill", "green");
    svg.appendChild(shape);
}

function numeroAleatório(max) {
    return Math.random() * max + Math.random() * max * 100;
}
<svg width="400px" height="400" version="1.1" xmlns="http://www.w3.org/2000/svg"></svg>


number of points of tangency of the zero divergence vector field and the equator of the sphere.

Question: Let $V$ be vector field on the sphere $S^2$ and $\operatorname{div} V=0$. What is the minimum number tangency points of this vector field and the equator of the sphere?
Comment: @John I even don't know how to begin to solve this problem. $div V=0$ is a local condition, but the statement is global. I think that there should be some trick.
Comment: Hint: divergence theorem.
Comment: @AnthonyCarapetis I know divergence theorem only for $\mathbb R^3$. $$\int \int \int \operatorname{div} F dV= \int \int F*n dS.$$ where $S$ is boundary of $V$. Am I correct that in the case of $S^2$ this theorem says that $$\int \int \operatorname{div} F dV= \int F*n dS.$$ where $V$ is $1$ dimensional submanifold  and $S$ is its boundary? Then the problem is easy, let $V={(x,y,z)|x>=0}$, $S=S^1$. $int F*n dS=0$ then F*n has at least $2$ zeroes.
Comment: @user152187: that's correct (except V should be a 2-dimensional region rather than a 1-dimensional submanifold)
Answer: Following up on comments: at the non-tangency points, the field flows either out of Northern hemisphere or into it. Since the divergence is zero, both of these must occur. This implies the existence of at least two points of tangency to the equator, since they must separate the equator arcs  where the flow is inward from the arcs where it is outward.
However, this is not the end of solution. If you are going to answer the question "What is the minimum number..?" with "2", you  better have a proof that 2 can be attained.  Fortunately, a constructive proof is easy. Consider the velocity of the Terminator: it is described by the flow that goes west at every point, rotating around the globe in 24 hours. Turn this picture sideways, so that the stationary point are on the equator. (Equivalently, turn the equator so that it becomes the union of $0 ^\circ$ and $180^\circ$ meridians.)


How can I make default selectd value in input text box?

Question: I use typeahead feature in my project:
<input type="text"
ng-model="customPopupSelected"
placeholder="Custom popup template"
uib-typeahead="state.id as state.desc for state in states | filter:{name:$viewValue}"
class="form-control">

Here is Plunker.
How can I make by default selected sample_data with id=4?
Comment: Sorry, but I don't fully understand the question and the code doesn't match the Plunker. Are you trying to initialize the model value to the object from the "sample_data" array matching id: "4"?
Answer: You can initialize the model with :
angular.forEach($scope.sample_data, function(item, index) {
    if (item.id == 4) {
        $scope.monkey = $scope.sample_data[index]
    }
}

Fiddle


Example of an invertible ideal which contains a zero divisor as one of its generators

Question: Let $R$ be a ring and let $I$ be an (integral) invertible ideal of $R$. Let $J = \{a_1,\ldots,a_r\}$ be a minimal ideal generating set of $I$ in the sense that any proper subset of $J$ will not generate $I$.

I was wondering if it was possible that one of the $a_i$ was a zero divisor of $R$.

Clearly, any invertible ideal does indeed contain at least one regular element and hence any principal invertible ideal won't provide such an example.
Thanks a lot in advance!
Comment: How do you define invertible ideals inside a ring that is not a domain? I know them as $R$-submodules of the quotient field $K$ of $R$, but that makes no sense. Do you simply replace $K$ by the total ring of fractions?
Comment: @JeskoHüttenhain Yes, I think this is the common definition as used in Eisenbud.
Answer: Yes, it is quite possible.
 Let $k$ be  a field,   $R=k\times k$ and $I=R$.
Then  $a_1=(1,0), a_2=(0,1)$ form a minimal set of generators for $I=R=\langle a_1, a_2\rangle$ but both $a_1,a_2$ are zero divisors of $R$.
Nota Bene
Contrary to what you claim this minimal set of generators  contains no  regular element.
Comment: What if I change the minimality condition imposed on the ideal generating set to "there is no other ideal generating set with strictly less generators"? Then your counter example doesn't work anymore since $(1,1)$ is a generator of your ideal.
Comment: take the ideal generated only by $a_1$ it is still invertible and the generator is a zero divisor


Profiling Rcpp code on OS X

Question: I am interested in profiling some Rcpp code under OS X (Mountain Lion 10.8.2), but the profiler crashes when being run.
Toy example, using inline, just designed to take enough time for a profiler to notice.
library(Rcpp)
library(inline)

src.cpp <- "
  RNGScope scope;
  int n = as<int>(n_);
  double x = 0.0;
  for ( int i = 0; i < n; i++ )
    x += (unif_rand()-.5);
  return wrap(x);"

src.c <- "
  int i, n = INTEGER(n_)[0];
  double x = 0.0;
  GetRNGstate();
  for ( i = 0; i < n; i++ )
    x += (unif_rand()-.5);
  PutRNGstate();
  return ScalarReal(x);"

f.cpp <- cxxfunction(signature(n_="integer"), src.cpp, plugin="Rcpp")
f.c <- cfunction(signature(n_="integer"), src.c)

If I use either the GUI Instruments (in Xcode, version 4.5 (4523)) or the command line sample, both crash: Instruments crashes straight away, while sample completes processing samples before crashing:
# (in R)
set.seed(1)
f.cpp(200000000L)

# (in a separate terminal window)
~ » sample R  # this invokes the profiler
Sampling process 81337 for 10 seconds with 1 millisecond of run time between samples
Sampling completed, processing symbols...
[1]    81654 segmentation fault  sample 81337

If I do the same process but with the C version (i.e., f.c(200000000L)) both Instruments and sample work fine, and produce output like
Call graph:
1832 Thread_6890779   DispatchQueue_1: com.apple.main-thread  (serial)
  1832 start  (in R) + 52  [0x100000e74]
    1832 main  (in R) + 27  [0x100000eeb]
      1832 run_Rmainloop  (in libR.dylib) + 80  [0x1000e4020]
        1832 R_ReplConsole  (in libR.dylib) + 161  [0x1000e3b11]
          1832 Rf_ReplIteration  (in libR.dylib) + 514  [0x1000e3822]
            1832 Rf_eval  (in libR.dylib) + 1010  [0x1000aa402]
              1832 Rf_applyClosure  (in libR.dylib) + 849  [0x1000af5d1]
                1832 Rf_eval  (in libR.dylib) + 1672  [0x1000aa698]
                  1832 do_dotcall  (in libR.dylib) + 16315  [0x10007af3b]
                    1382 file1412f6e212474  (in file1412f6e212474.so) + 53  [0x1007fded5]  file1412f6e212474.cpp:16
                    + 862 unif_rand  (in libR.dylib) + 1127,1099,...  [0x10000b057,0x10000b03b,...]
                    + 520 fixup  (in libR.dylib) + 39,67,...  [0x10000aab7,0x10000aad3,...]
                    356 file1412f6e212474  (in file1412f6e212474.so) + 70,61,...  [0x1007fdee6,0x1007fdedd,...]  file1412f6e212474.cpp:16
                    56 unif_rand  (in libR.dylib) + 1133  [0x10000b05d]
                    38 DYLD-STUB$$unif_rand  (in file1412f6e212474.so) + 0  [0x1007fdf1c]

I would really appreciate some advice into if there is anything I'm doing wrong, if there is some other preferred way, or if this is just not possible.  Given that one of the main uses of Rcpp seems to be in speeding up R code, I'm surprised not to find more information on this, but perhaps I'm looking in the wrong place.
This is on OS X 10.8.2 with R 2.15.1 (x86_64-apple-darwin9.8.0), Rcpp 0.9.15, and g++ --version reports "i686-apple-darwin11-llvm-g++-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)".
A solution
Thanks to Dirk's answer below, and his talk here http://dirk.eddelbuettel.com/papers/ismNov2009introHPCwithR.pdf, I have at least a partial solution using Google perftools.  First, install from here http://code.google.com/p/gperftools/, and add -lprofiler to PKG_LIBS when compiling the C++ code.  Then either
(a) Run R as CPUPROFILE=samples.log R, run all code and quit (or use Rscript)
(b) Use two small utility functions to turn on/off profiling:
RcppExport SEXP start_profiler(SEXP str) {
  ProfilerStart(as<const char*>(str));
  return R_NilValue;
}

RcppExport SEXP stop_profiler() {
  ProfilerStop();
  return R_NilValue;
}

Then, within R you can do
.Call("start_profiler", "samples.log")
# code that calls C++ code to be profiled
.Call("stop_profiler")

either way, the file samples.log will contain profiling information.  This can be looked at with
pprof --text /Library/Frameworks/R.framework/Resources/bin/exec/x86_64/R samples.log

which produces output like
Using local file /Library/Frameworks/R.framework/Resources/bin/exec/x86_64/R.
Using local file samples.log.
Removing __sigtramp from all stack traces.
Total: 112 samples
  64  57.1%  57.1%       64  57.1% _unif_rand
  30  26.8%  83.9%       30  26.8% _process_system_Renviron
  14  12.5%  96.4%      101  90.2% _for_profile
   3   2.7%  99.1%        3   2.7% Rcpp::internal::expr_eval_methods
   1   0.9% 100.0%        1   0.9% _Rf_PrintValueRec
   0   0.0% 100.0%        1   0.9% 0x0000000102bbc1ff
   0   0.0% 100.0%       15  13.4% 0x00007fff5fbfe06f
   0   0.0% 100.0%        1   0.9% _Rf_InitFunctionHashing
   0   0.0% 100.0%        1   0.9% _Rf_PrintValueEnv
   0   0.0% 100.0%      112 100.0% _Rf_ReplIteration

which would probably be more informative on a real example.
Comment: +1 -- perftools, the gcc profiler or valgrind can all help. The fact that R is interactive does throw a spanner in here as it adds more code layers.
Comment: I ran into this myself when I upgraded my XCode, but I thought it might just be me. The thing is that Shark worked perfectly for profiling Rcpp before, but it is now defunct. I guess we can call this an Apple bug.
Answer: I'm confused, your example is incomplete:

you don't show the (trivial) invocation of cfunction() and cxxfunction()
you don't show how you invoke the profiler
you aren't profiling the C or C++ code (!!)

Can you maybe edit the question and make it clearer?
Also, when I run this, the two example do give identical speed results as they are essentially identical. [ Rcpp would let you do this in call with sugars random number functions. ]
R> library(Rcpp)
R> library(inline)
R>
R> src.cpp <- "
+   RNGScope scope;
+   int n = as<int>(n_);
+   double x = 0.0;
+   for ( int i = 0; i < n; i++ )
+     x += (unif_rand()-.5);
+   return wrap(x);"
R>
R> src.c <- "
+   int i, n = INTEGER(n_)[0];
+   double x = 0.0;
+   GetRNGstate();
+   for ( i = 0; i < n; i++ )
+     x += (unif_rand()-.5);
+   PutRNGstate();
+   return Rf_ScalarReal(x);"
R>
R> fc   <- cfunction(signature(n_="int"), body=src.c)
R> fcpp <- cxxfunction(signature(n_="int"), body=src.c, plugin="Rcpp")
R>
R> library(rbenchmark)
R>
R> print(benchmark(fc(10000L), fcpp(10000L)))
         test replications elapsed relative user.self sys.self user.child sys.child
1   fc(10000)          100   0.013        1     0.012        0          0         0
2 fcpp(10000)          100   0.013        1     0.012        0          0         0
R>

Comment: Hi Dirk, and thanks for helping!  I've edited the example above for clarity, I hope.  The profiler is invoked with the 'sample' command, and I've included output from the C version to show that it is C that is being profiled.  I've used this general approach successfully with R+C code before, but not with C++.  The examples should give identical results, and aren't representative of the problem I'm actually trying to profile.  Thanks again, Rich
Comment: I see now -- I know nothing about the OS X aspect of this. For straight up `gcc` et al, I put some notes about profiling into the 'Intro to HPC with R' slides on my website / from previous talks. You will need to see with r-sig-mac whether `sample` works on the R toolchain.

Profiling Rcpp code on OS X


How to specify partially crossed random effect in lme?

Question: I am new to R and would like your help with lme formula for partially crossed random effect in a random-intercept, random-slope model. In the longitudinal data I have, each subject (barring some dropouts) was tested at 5 different occasions. The standardized tests were administered by 3 different examiners, with 2 of them present at all occasions, and the 3rd one administering tests only on the last two occasions. The subjects were randomly assigned to the examiners.
Mock data:
data <- data.frame(subject=rep(A,5), time=1:5, examiner=c(2,1,2,2,3),
                   covariate=c(1,1.3,0.8,1,0.6), score=c(46,56,60,68,70))

I tested the following models:
>model1 <- lme(score~time*covariate, random=~time|subject, method="REML",
               na.action=na.omit, data=dat)
>model2 <- lme(score~time*covariate, random=list(examiner=~1,subject=~time),
               method="REML", na.action=na.omit, data=dat)

>anova(model1, model2) #gives p<0.05 with better model fit for model2.

I would like to know if model2 is the correct way to specify the partially
crossed random effect in the data I described.
Answer: This is not a direct answer for lme's syntax.
I would argue that while in theory a specific examiner is part of the greater examiner population and it does make sense to have it as a random effect, you have only 2 (and occasionally 3) replicates. It will most probably be more sensible to use it as fixed effect (possibly as an interaction).
Moreover I would not be too fast to jump on ANOVA for model selection. Assuming you do not want to consider issues of cross-validation etc., maybe an information criterion like AIC will be equally easy to apply and probably slightly more coherent.
I think your first model is quite reasonable as it stands. Maybe try a (0+time|subject)+(1|subject) random structure so that you will fit independently the slope and the intercept of the model...
I scavenged this thread from R-sig-mixed (I had also seen your question there about a fortnight ago) and these two older ones from R-help (thread1, thread2) that might be helpful as they contain some expert opinions on the matter.
Comment: Glad they were of some help. Furthermore: No, be careful; `model complexity` is definitely not similar. If anything it is an open question (in technical, theoretical as well purely philosophical terms) how many degrees of freedom one random effect should account for. Eg. Should you threat in a $N$ = 10000 sample, a random effects with 3000 levels as conveying the same information as one with 25 levels? Intrinsically I would say "no", though ANOVA would treat them as exactly the same. (I am not saying that my "no" is the correct answer, I am just commenting.)
Comment: Yes, the examiner is just a part of greater examiner population & something less important than other fixed effects. That said, is model 'complexity' similar if it is a random effect?
ANOVA for model comparison does appear in multiple sources. e.g. http://stat.ethz.ch/R-manual/R-devel/library/nlme/html/anova.lme.html
If there is reason to believe intercept & slope are correlated, e.g. subjects with higher score at baseline improve faster, would one still try fitting slope & intercept independently to check effects of a covariate?
Very helpful links. Thanks a lot for your reply!


React-Select, how to include the styles css file?

Question: I am trying to use the react-select component (https://github.com/JedWatson/react-select) and confused with this sentence while reading the documentation, see below:
<code>Be sure to include styles at some point, probably during your bootstrapping
import 'react-select/dist/react-select.css';
</code>
When I import this line in the React JSX file where I am using the Select component I am getting the following error:
<code> ...node_modules/react-select/dist/react-select.css:8\n.Select
    {\n^\nParseError: Unexpected token\n>",
    "traces": {
    "Application Trace": [
    {
      "id": 39,
    "trace": "app/views/layouts/application.html.erb:7:in

 `_app_views_layouts_application_html_erb___377503645297909977_70313107001300'"
}
],
"Framework Trace": [
{
"id": 0,
"trace": "browserify-rails (3.3.0) lib/browserify-rails/browserify_processor.rb:220:in `run_browserify'"
},
{
"id": 1,
"trace": "browserify-rails (3.3.0) lib/browserify-rails/browserify_processor.rb:51:in `call'"
},
{
....
}
</code>
Can someone provide some hints on this issue?
Answer: It looks like I found an answer towards this question.
Basically what I did is I manually copy the css file from node_modules/react-select/dist/react-select.css into /assets/javascript/stylesheets folder in my rails project.
For more infomation please look into: https://stackoverflow.com/a/10291177/5761338