Interpreting what this means in a paper - significantly different at the .05 level?

Question: I am having a hard time interpreting what something means in a paper I'm trying to get through. If you care, this is the paper: Gender Differences in the Effect of Education on the Slope of Experience-EarningsProfiles: National Longitudinal Surveyof Youth,1979-1988. By KEVINC. DUNCAN*
In one of the tables of the regressions, it explains that a little "a" (next to the coefficient) means the coefficient is significantly different at the .05 level from the comparable coefficient for white males.
What exactly does this mean?
Here is my understanding: If there are 2 correlations with sample sizes n's, they are each changed into z values. Under the null hypothesis that the population correlations are equal, the following has a standard normal distribution (one equation using both z's). If the z is great than or equal to 1.96 or less than or equal to -1.96, the two correlations are significantly different at the .05 level of significance. If the z is not significant at the .05 level, sampling error is a possible explanation for the difference. If the two are significantly different, the null hypothesis is rejected! There must be fairy large sample sizes used. Coefficients with p value of .05 or less would be judged to be statistically different.
Is this correct? Can you explain it a little more if I'm wrong.
Here is a link to the paper? http://www.jstor.org/stable/3487620?seq=3#page_thumbnails_tab_contents
Also which coefficients would you consider significant in Table 2?
Answer: Footnote 8 and 9 of the paper contained on page 470-471 of the paper contain information as to how the "a" is calculated.
The best way to think about it is as follows: Imagine the author has split the data 4 ways: Black Males, Black Females, White Males, White Females. The author then runs the same regression (See footnote 8 for details) on these 4 different groups and obtain 4 different estimates of the parameters on the independent variables.
Take Education in Table 3: The author runs 4 regressions and finds that a 1 additional year of schooling increases hourly wages by: 6.7% for Black Females, 7.1% for Black Males, 2.2% for White Females and 2.2% for White Males.
The author wants to test whether the parameter estimates stated above are significantly different for the 3 different groups compared to the estimates for White Males. In other words he want to test if the percentage increase in hourly wages is significantly different for the 3 groups that are not white males (the reference category).
The author uses the  Kniesner, et. al (1978) test ( see footnote 9 for the test equation). In brief this tests whether the difference between 2 parameter estimates (say the impact of education on wages for black females vs white males) is different from 0. The output of the test is a t-statistic which is compared to the 95% critical value from a t-distribution. If the t-statistic exceeds the critical value, the null hypothesis that the two parameter estimates are the same is rejected.
Significance is another thing entirely and is denoted using a t-test for the single parameter estimate different from 0. These are denoted with the stars $*$. In the economics literature, a p-value $\le$ 0.05 is usually deemed as "significant".
Comment: I have taken out the paragraph as I saw you included a better explained answer. vvv123 please refer to [link](http://stats.stackexchange.com/questions/31/what-is-the-meaning-of-p-values-and-t-values-in-statistical-tests) for an excellent explanation of the p-value.


CentOS 7 / Apache / mod_wsgi - UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 2727: ordinal not in range(128)

Question: I've deployed a Python web application (uses Pyramid) in Apache (httpd) using mod_wsgi.
After starting the Apache service the error below (from Apache log) occurs when we make a http request to the web application...
[Mon Feb 06 16:38:55.980119 2017] [wsgi:error] [pid 16031] [remote 172.16.13.1:204] mod_wsgi (pid=16031): Target WSGI script '/usr/local/lb/ve32/src/LBGenerator/lbgenerator.wsgi' cannot be loaded as Python module.
[Mon Feb 06 16:38:55.980205 2017] [wsgi:error] [pid 16031] [remote 172.16.13.1:204] mod_wsgi (pid=16031): Exception occurred processing WSGI script '/usr/local/lb/ve32/src/LBGenerator/lbgenerator.wsgi'.
Traceback (most recent call last):
  File "/usr/local/lb/ve32/src/LBGenerator/lbgenerator.wsgi", line 1, in <module>
    from pyramid.paster import get_app
  File "/usr/local/lb/ve32/lib/python3.2/site-packages/pyramid-1.5.1-py3.2.egg/pyramid/paster.py", line 10, in <module>
    from pyramid.scripting import prepare
  File "/usr/local/lb/ve32/lib/python3.2/site-packages/pyramid-1.5.1-py3.2.egg/pyramid/scripting.py", line 1, in <module>
    from pyramid.config import global_registries
  File "/usr/local/lb/ve32/lib/python3.2/site-packages/pyramid-1.5.1-py3.2.egg/pyramid/config/__init__.py", line 30, in <module>
    from pyramid.exceptions import (
  File "/usr/local/lb/ve32/lib/python3.2/site-packages/pyramid-1.5.1-py3.2.egg/pyramid/exceptions.py", line 1, in <module>
    from pyramid.httpexceptions import (
  File "/usr/local/lb/ve32/lib/python3.2/site-packages/pyramid-1.5.1-py3.2.egg/pyramid/httpexceptions.py", line 138, in <module>
    from pyramid.response import Response
  File "/usr/local/lb/ve32/lib/python3.2/site-packages/pyramid-1.5.1-py3.2.egg/pyramid/response.py", line 23, in <module>
    init_mimetypes(mimetypes)
  File "/usr/local/lb/ve32/lib/python3.2/site-packages/pyramid-1.5.1-py3.2.egg/pyramid/response.py", line 16, in init_mimetypes
    mimetypes.init()
  File "/usr/local/lb/py32/lib/python3.2/mimetypes.py", line 351, in init
    db.read(file)
  File "/usr/local/lb/py32/lib/python3.2/mimetypes.py", line 203, in read
    self.readfp(fp, strict)
  File "/usr/local/lb/py32/lib/python3.2/mimetypes.py", line 214, in readfp
    line = fp.readline()
  File "/usr/local/lb/py32/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 2727: ordinal not in range(128)

That web python application works perfectly in all these distros: Ubuntu 14.04, Debian 8, RedHat 6, CentOS 6, Suse 12 and openSUSE 13, but not in CentOS 7... =[
We are using "Python 3.2.2," "mod_wsgi 4.3.2" and "Apache 2.4.6" in a "CentOS 7". The web application was built on top of Pyramid Web Framework (http://docs.pylonsproject.org/projects/pyramid/en/latest/).
This is a fairly common error! So far I have not found a way to get more information about this error.
We have tried everything to solve this problem and we simply do not know why it occurs... =[
Please help me!
Comment: This seems to be a fairly common error. Please explain specifically what you have tried.
Comment: @MichaelHampton Sorry for that! See if it's okay now! Thank you!
Answer: The problem occurs because the /etc/mime.types file of CentOS 7 has the character "³" (line "application/vnd.geocube+xml                     g3 g³") which is invalid for ascii (not extended).

To figure this out I had to make some modifications to my Python 3.2.2 and reload my application.

The solution is remove the "³" character or any other invalid (ascii) character!

@MichaelHampton

Thanks!
Comment: Pretty sure this is happening because you don't have mod_wsgi set up with the correct encoding, so it's using ascii instead of utf-8. Your workaround may work OK, but you'll wind up with problems [like mine](https://stackoverflow.com/questions/64564924/how-to-set-preferred-encoding-in-wsgi-to-utf-8#64564924), I think.


formalize in predicate logic: All students join some course that they don't like.

Question: I came up with two formulas and have no Idea whether their correct.
1. ∀x(S(x)∧¬C(x))
2. ∀x∃y(S(x)∧C(y)→DoesNotLike(x,y))
And then I have one a bit more complicated, that I can think of multiple ways how to formalize but have no way of knowing if correctly:
Every student attends a class taught by a teacher that they dont like.
P.s. this is not my homework, just preparation for an exam.
Answer: You need to specify carefully what you're taking the predicates $\ S\ $ and $\ C\ $ to mean.
Presumably, $\ S\left(x\right)\ $ means "$\ x\ $ is a student".  I think your first formulation would then be synonymous with the sentence given in the title if $\ C\left(x\right)\ $ were taken to mean "$\ x\ $ likes all courses he or she has ever joined, or will ever join", and the conjunction were to be replaced with implication:$^{\mathbf \dagger}$
$$
\forall x\left(S\left(x\right)\rightarrow \neg C\left(x\right)\right)
$$
However, I can't think of any interpretation of $\ C\left(y\right)\ $ that would make your second formulation similarly synonymous with that sentence.
It looks like $\ C\left(y\right)\ $ in the second formulation was intended to mean "$\ y\ $ is a course", but if that were so, there wouldn't appear to be anything in that formulation to specify that the course $\ y\ $ is one that student $\ x\ $ has joined or will join at some time.  This could be partly rectified by making $\ C\ $ a two-place predicate for which $\ C\left(x,y\right)\ $ means "$\ y\ $ is a course which $\ x\ $ has joined or will join at some time", but even that's not enough to make the second formulation synonymous with the target sentence.
A second problem with the second formulation is the placement of the predicate $\ C\ $ before, rather than after, the implication.$^{\mathbf *}$  As written, the sentence
$$
\forall x\exists y\left( S\left(x\right) \wedge C\left(x,y\right)\rightarrow\mbox{DoesNotLike}\left(x,y\right)\right)
$$
merely says that for every student there is some course that the student wouldn't like if he or she were to join it.  It does not assert that every student will, in fact, join such a course at some time.  This can be rectified by placing the predicate $\ C\ $ after the $\mbox{implication:}^{\mathbf *}$
$$
\forall x\left( S\left(x\right) \rightarrow\exists y\left( C\left(x,y\right)\wedge\mbox{DoesNotLike}\left(x,y\right)\right)\right)\ \ .
$$
As now formulated the predicate $\ C\ $ represents an existential statement which could be further spelt out by putting $\ C\left(x,y\right) = \exists t\,J\left(x,y,t\right)\ $, where $\ J\left(x,y,t\right)\ $ is taken to mean "$\ y\ $ is a course which $\ x\ $ joined, or will join, at time $\ t\ $. The last formulation above would then become:
$$
\forall x \left( S\left(x\right) \rightarrow\exists y\exists t\left( J\left(x,y,t\right)\wedge\mbox{DoesNotLike}\left(x,y\right)\right)\right)\ \ .
$$
* Edit: Thanks to Daniel Schepler, in his comment below, for pointing out an error in my original attempt to rectify this deficiency.
$\mathbf\dagger$ Further edit: I forgot that the first formulation suffered from the same problem as my first attempted rectification of the second.
Comment: Should you also change $S(x) \wedge \cdots$ to $S(x) \rightarrow \cdots$ so the statement doesn't require that *everything* in the domain of discourse (including also each course - unless you're using a multi-sorted logic) is a student?
Comment: Yes, of course you're right.  I have now amended the answer.


Number of directed graphs on a $3\times 3$ grid where every vertex has both indegree and outdegree $\ge 1$

Question:
Consider nine vertices arranged in a square $3\times 3$ grid. Compute the number of directed graphs that can be drawn on the grid by connecting every vertex to its immediate horizontal and vertical neighbors, if every vertex must have both indegree and outdegree $\ge 1$ (at least one edge entering and at least one edge leaving). If feasible, extend this to general $n\times n$ grids.

I created this problem while idly musing in a linear algebra class recently. I note that the corners each have two edges, so one must be directed in and the other out (either clockwise or counterclockwise). But then I also noticed that the problem can be reduced slightly by reducing the vertex and pair of edges at each corner to a single edge from middle of side to middle of side, upon which a $45^{\circ}$ rotation changes the problem to a $2\times 2$ grid with a single vertex connected to all four outer vertices in the center. I don't know if this actually helps the solution behave any more nicely, but it's something I noticed that I think could possibly help, so I felt I should mention it.
I determined what I believe is an upper bound of $224$ possible arrangements because if the edges connected to the corner vertices all "flow" in the same direction, the four edges connected to the middle vertex can have any of $14$ possible arrangements ($2^4-2$ because they can't all point in or all out), and then we can multiply that $14$ by $16=2^4$ for the number of possible arrangements of the edges connected to the corners of the grid.
So my question is effectively "is there a better way of solving this question than laboriously drawing out all possible directed graphs and seeing whether they work?", not just because a non-brute force solution would be more interesting and almost certainly more enlightening than just brute-forcing it but also because such a technique is completely infeasible for the general question I pose at the end of the stated problem. Also, I'm sorry I don't have a drawing for this- I don't know how to do pictures in LaTeX. If anyone could create a helpful picture to accompany the problem, I'd be deeply grateful.

Edit: Here's an image of one admissible configuration. I drew it at virtual-graph-paper.com, which makes it very easy to draw this sort of thing. It will be saved there for $90$ days, so you can go and edit it to show further configurations if you like.
Comment: For reference, a brute force solution finds $78$ graphs that meet the criteria on a $3\times 3$ grid and $66644$ graphs on a $4\times 4$ grid.
Comment: @ThomasAndrews: "the corners each have two edges, so one must be directed in and the other out" seems to imply that it can't.
Comment: @DanielMathias: Unfortunately, those numbers don't appear together in an OEIS sequence.
Comment: Since an edge is always between a black square and a white square, we can think of this as two subgraphs of the non-directed adjacency, where each edge has valency at least $1.$ (Unless your directed graph can't have an edge from $a$ to $b$ and an edge from $b$ to $a.$)
Comment: I'm only allowing one directed edge between vertices, yes.
Comment: @joriki Thank you for the edit! Perfect example.
Answer: Here's a fun way to do this, though unfortunately it won't be feasible to apply it to larger grids without a computer.
If we use your simplification, there are only $5$ vertices, and thus only $5$ constraints to be considered in applying inclusion–exclusion.
There are $8$ edges in the simplified graph, so there are $2^8=256$ configurations without constraints.
If the constraint on a vertex is violated, that vertex has either all ingoing or all outgoing edges. For a vertex with $k$ edges, that leaves $2\cdot2^{8-k}=2^{9-k}$ configurations. There are $4$ vertices with $3$ edges and one with $4$, so we have to subtract $4\cdot2^{9-3}+2^{9-4}=288$ configurations.
If the constraints on two vertices are violated, if those vertices are neighbours, one must have all incoming and the other all outgoing edges, whereas if they aren't neighbours, they can independently have all incoming or all outgoing edges. There are $2$ pairs that aren't neighbours, and they leave $2$ edges unspecified, so they each contribute $2\cdot2\cdot2^2=16$ configurations. There are $4$ pairs of neighbours that leave $3$ edges unspecified, which also contribute $2\cdot2^3=16$ configurations; and there are $4$ pairs of neighbours that leave $2$ edges unspecified, which contribute $2\cdot2^2=8$ configurations. So in total we have to add back in $2\cdot16+4\cdot16+4\cdot8=128$ configurations.
Now it gets easier, because sets of vertices that contain an odd cycle can't all have their constraints violated. There are $6$ subsets of three vertices that don't form a cycle; they all consist of neighbours. $4$ of them fix all but one edge, so they each contribute $2\cdot2=4$ configurations, and $2$ of them fix all edges, so they each contribute $2$ configurations. Thus, we have to subtract $4\cdot4+2\cdot2=20$.
There's only one set of $4$ vertices that doesn't contain an odd cycle. It fixes all edges and thus contributes only $2$ configurations; so we have to add $2$ back in.
The entire set of five vertices contains an odd cycle, so we're done.
The total is
$$
256-288+128-20+2=78\;.
$$
This is considerably below your upper bound – which makes sense, since most of the configurations of the outer four edges constrain the inner edges.
Come to think of it, it's actually not too difficult to count using the configurations of the outer edges, so let's do that, too.
There are $2$ configurations of the outer edges that never change direction, $2\cdot\binom42=12$ configurations that change direction twice and $2$ configurations that change directions $4$ times.
As you wrote, the configurations that never change direction allow for $14$ configurations of the inner edges.
The configurations that change direction twice fix the two inner edges incident at the vertices with the changes in direction. Those inner edges satisfy the constraint at the central vertex, so the remaining $2$ edges can be freely chosen, which makes $2^2=4$ configurations of the inner edges.
Four changes in direction completely fix the inner edges, so that only leaves $1$ configuration.
The total is
$$
2\cdot14+12\cdot4+2\cdot1=78\;,
$$
in agreement with the inclusion–exclusion calculation.


How can I implement user timezone & user's last login in this format?

Question: Im trying to create a block and I'm  having trouble. I'm trying to get the user's last login & the user's timezone. How can I implement those functions in these hooks. All I know is the fact that I need to use the functions of hook_info & hook_view.
function hook_block_info() {

  $blocks['syndicate'] = array(
    'info' => t('Syndicate'),
    'cache' => DRUPAL_NO_CACHE,
  );

  $blocks['recent'] = array(
    'info' => t('Recent content'),
  );

  return $blocks;
}

function hook_block_view_alter(&$data, $block) {

  if (is_array($data['content']) && asset($data['content']['#contextual_links'])) {
    unset($data['content']['#contextual_links']);
  }

  if (is_array($data['content']) && $block->module == 'some module') {
    $data['contentr']['#theme_wrappers'][] = 'mymodule_special_block';
  }
}

Answer: You can use the global $user object to get user details, or load user details from a uid like this: $user = user_load($uid); From the $user->login property you can get last login timestamp.
Thanks


Denomination problem

Question: Suppose that the only denominations of the available bills are 11 and 19 dollars. In a
convenience store, what amounts can be paid? Justify your solution in full
detail.
Comment: closely related: http://math.stackexchange.com/questions/198690/coin-related-brain-teaser-frobenius-coin-problem
Answer: "Full detail" is context-dependent. It is not clear what it means unless one knows what has been done in your course, and what tools are accessible.
From general theory, it can be shown that all numbers $\ge (11-1)(19-1)$ are representable, and that $(11-1)(19-1) -1=179$ is not.
You can do "full detail" by brute force, perhaps computer aided. Once you find out by computation that the $11$ numbers from $180$ to $190$ are representable, then you know that all numbers above that are representable, by adding a suitable number of $11$ dollar bills to the representation of some number between $180$ and $190$.  For any number greater than $190$ is congruent modulo $11$ to some number between $180$ and $190$.
More simply put, once you get $11$ in a row, you know everything is all right from then on.
For numbers between $0$ and $179$, there is useful bu complicated general theory. However, careful book keeping will do the job. It is not much fun. Of course, $0$ is representable, though the convenience store owner may not be pleased. Then $1$ to $10$ are not, $11$ is, $12$ to $18$ are not, and so on. As our numbers get larger, the probability they are representable increases, and one has to be very careful. There will be $90$ representable numbers up to $180$. Quite a lot of work

Denomination problem


How to get the Template Chooser/Document Browser view on iOS?

Question: What is the view used to create the Template Chooser view in the iWork apps on iPad?

How can I create a view like that on iPad, and what would be a best way to implement it on iPhone considering screen size constraints.
Answer: There isn't any built-in view that will provide that functionality. Which means there will be quite some work to duplicate that functionality.
What you can try is:
1) Create a new UIViewController with a nib.
2) Add the top bar and an UIScrollView with opaque = NO and alpha = 0.
3) If you have a fixed number of "templates" they can be added directly in the nib. You should be able to use UIImageView
4) Otherwise you can add the "templates" dynamically in e.g. viewDidLoad. The only part which can be a little bit tricky is calculating the frame. The following pseudo code should get you started.
int MARGIN = 20;
float templateWidth = self.scrollView.bounds.size.width / 3;
float templateHeight = 300;

for (int i = 0; i < [templates count]; i++) {
  int row = i / 3;
  int col = i % 3;

  float x = MARGIN + col * templateWidth;
  float y = MARGIN + row * templateHeight;
  CGRect templateFrame = CGRectMake(x, y, width - 2 * MARGIN, height - 2 * MARGIN);

  // initialize `UIImageView` or similar

  templateView.frame = templateFrame;
  [self.scrollView addSubView:templateView];
}

self.scrollView.contentSize = CGSizeMake(self.scrollView.bounds.width, /* max bottom of templates */);
}

After you have got the layout perfected the rest should be quite straightforward since you only need to respond to taps on your template images. Take a look at UITapGestureRecognizer for one way of doing that.
Regarding the iPhone. I would probably do it in the way you select a document in the iWorks apps. Only one template is displayed on screen at the time and you swipe left/right to choose. But it all depends on the context. Perhaps a table view is better suited for the iPhone.
Good luck!
Answer: With iOS 6, use UICollectionView!


How to send an email on behalf of user

Question: I have two zimbra mail servers, on server-1 I have one domain with accounts. I want to be allowed to send email from server2, on behalf of email@example.com that I dont have on server2, but this domain *@domain.com I have on server1. For now I can send such email on behalf of user that I want, but I can receive only in spam folder. What record should I add and where, so this can be done without going to spam?
Answer: There are many reasons why email could end up in someone's spam folder. In your case, however, the first thing I would look into is correctly setting up the SPF records
From Wikipedia:

SPF allows the receiving mail server to check during mail delivery that a mail claiming to come from a specific domain is submitted by an IP address authorized by that domain's administrators.[3] The list of authorized sending hosts and IP addresses for a domain is published in the DNS records for that domain.

In other words, you will need to create a DNS record for your domain that explicitly authorizes server2 to send email on behalf of that domain. More information specifically for Zimbra can be found on the Zimbra Wiki
Comment: Thanks very much, this is answer of my question. After I "whitelisted" IP address in SPF on my domain name record, now emails not going to spam.

How to send an email on behalf of user


Algorithm for finding relations in an algebra

Question: I have a sub-algebra of the algebra of polynomials in $n$ variables, given by a set of $k \le n$ generators. Is there an algorithm to find all the possible relations these generators satisfy?
In other words my subalgebra will be isomorphic to $k[x_1,\dots,x_k]/I$ and I want an algorithm to find generators for $I$.
Comment: I would assume that the downvote is a knee-jerk reaction from someone who sees a problem statement and no attempts or thoughts on your part. It's common enough here, but I still think it's bad form to do it without also leaving an explanation.
Comment: Is there a specific reason I've been downvoted? Can someone explain how I can improve my question? :)
Answer: There is Buchberger's algorithm for finding a Gröbner basis.


Why does Val(5d886) or Val(5e886) cause an OverflowException?

Question: Example code:
<code>Public Class Form1
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        MessageBox.Show(Val(TextBox1.Text), "Val of text input")
    End Sub
End Class
</code>
Anyhow out of simple curiosity I am wondering why it does this?
From some other posts & personal experience I know VAL(expression) isn't recommended & is just a legacy function from VB6 days.  I tried every other alphabet letter & none caused this issue, I also tried a couple variations & it appears any number after d or e add that many zeros (seems like its multiplying). I understand that the value may be overflowing the data-type VAL placed it in. I see e is the default variable for the sub-procedure, so that's probably why, but I can't figure out any logic to d.
Please make it noted I am very new to VB.NET, just got down with a college course & haven't built any actual apps. Furthermore our teacher seemingly didn't teach us error-handling which I am very interested in, as my current work situation has some rather old apps that have issues. Would also like any recommended error-handling articles/readings
Comment: Post valid code, Val(5d886) cannot compile.
Answer: The Val() function "returns the numbers contained in a string as a numeric value of appropriate type" (cit. from http://msdn.microsoft.com/en-us/library/k7beh1x9(v=vs.71).aspx).
If you set a string value like 5exxx it means 5 * 10^xxx; but pay attention: the function returns a double and double goes approximately from ±5.0 x 10^-324 to ±1.7 x 10^308; so, for example, the string 5e307 works but 5e308 doesn't, returning overflow.
If you use some other letters "function stops reading the string at the first character it cannot recognize as part of a number", so it doesn't return overflow.
I hope it was helpful. Have a nice new year!
Comment: I kind of understood that was happening (the multiplication). That article you linked I had read, it says "stops reading the string at the first character it cannot recognize as part of a number". However I still do not understand why d & e do this & no other characters do? Furthermore how do you know it returns it as a double vs say single, integer, etc?
Comment: The strings xxxEyyy and xxxDyyy are two ways to write a number in scientific notation. Val function returns a double result for it recognizing a correct number format. Agreeding with help file, the function always returns a double except for the case that there is a char in input; in that case the fucntion returns an integer.
Comment: You nailed my question. A quick google search shows a website addressing this: http://www.powerbasic.com/support/help/pbcc/val_function.htm. Would love to see a Microsoft article at least explaining the e & d. Doubt it would address it in context of Val (how it would interpret). Feel free to also address my last sentence in the original question if you could
Comment: Maybe you can find useful the MSDN page on VB.NET errors: http://msdn.microsoft.com/en-us/library/ms973849.aspx; or this quick tutorial: http://www.codeproject.com/Articles/13658/VB-NET-Error-Handling; however there are a lot of tutorials online about it. If you think my post is useful you may check it as an answer. Thank you!
Comment: Here is a s function that will extract just the numbers (consider it home-made Val2): http://www.access-programmers.co.uk/forums/showthread.php?p=1315649#post1315649


Physics related antiderivatives problem

Question: I was wondering if I could get a little help with a calculus related word problem:
A car braked with a constant deceleration of 5 meters per second squared for 60 meters before stopping. How fast was the car traveling when the brakes were applied? I can't use the definite integral, only the antiderivative.
Thanks for any help!
Comment: If you use conservation of energy, you don't need either.
Answer: By definition the acceleration is $a = \dot v$, where $v$ is the velocity and $\dot{\phantom{v}}$ denotes derivative with respect to time. If $a$ is constant we can take the antiderivative to get
$$v(t) = at + v_0$$
where $v_0$ is the velocity at time $t=0$.
Now $v = \dot s$ where $s$ is the distance travelled. Using the above expression for $v$ we can again take the antiderivative and get
$$s(t) = \frac12at^2 + v_0t + s_0$$
where $s_0$ is the distance travelled at $t=0$.
Assume that the driver applies the brakes at time $t_0$ and that the car stops at time $t_1$. Then we have
$$\Delta s = s(t_1) - s(t_0) = (\frac12at_1^2 + v_0t_1 + s_0) - (\frac12at_0^2 + v_0t_0 + s_0) = \frac12a(t_1^2-t_0^2) + v_0(t_1-t_0) \tag{1}$$
where $\Delta s$ is the distance travelled during braking, and also
$$0 = v(t_1) = at_1 + v_0 \tag{2}$$
In the exercise $\Delta s = 60\,\text{m}$ and $a = -5 \text{m/s}^2$ are given and we are asked for $v(t_0)$.
Using (2) we get $t_1 = -v_0/a$, which inserted into (1) gives
$$\Delta s = \frac12a((-v_0/a)^2-t_0^2) + v_0(-v_0/a-t_0) = -\frac12v_0^2/a - v_0t_0 - \frac12at_0^2 = -\frac12(v_0+at_0)^2/a = -\frac12v(t_0)^2/a$$
Thus,
$$v(t_0) = \pm\sqrt{2a\Delta s}$$
Now you can just insert the given values of $a$ and $\Delta s$.
I have in these calculations deliberately not taken $t_0=0$.
Comment: Good job, excellent answer!
Comment: Thanks for the answer
Comment: @Alis a little bit too long.
Comment: @Salahamam_Fatima. Really too long or longer than necessary?
Answer: You just need the formula
$$V_1^2-V_2^2=2\gamma (X_1-X_2) $$
with $V_i =$ speed at position $X_i $ and $\gamma $ the constant acceleration.
thus
$$V_1^2-0=2.5.60$$
$$V_1=\sqrt {600} m/s$$
The formula comes from the integration of

$$\frac {d (V^2/2)}{dt}=V\frac {dV}{dt}=\gamma V=\gamma \frac {dX}{dt} $$
Comment: @MCCCS hasn't he made a mistake then? he should have 600 instead of 300, right? Or am I missing someting really obvious?
Comment: I think that it would be helpful to derive the formula
Comment: What is gamma? I haven't read of this formula before.
Comment: And do you think that you could solve this with calculus instead
Comment: @Alis See [this](https://math.stackexchange.com/a/2154578/357924) for more info about the formula.
Comment: @MCCCS that kind of just restates the formula without much help to the person who made this question
Comment: @CarlosManere I'm going to post an answer solving it by a different way.
Comment: I suppose that $\gamma$ is used to denote acceleration. However, I've never seen that symbol being used to denote such.
Comment: Thanks, but should there or should there not be a '2' in the equation, and if so, why is it gone from the next lines?
Comment: @Alis Oh yes i see now. Thx , i edited it.
Answer: Let $travel_m = as^2+bs+c$ (s = time, travel_m = travelled distance)
$\frac{\text{d}^2y}{\text{d}s^2} as^2+bs+c$ (The second derivative of distance gives you the acceleration.)
$=2a=-5$
$a=-2.5$
$travel_m = -2.5s^2+bs+c$
At first, the travel distance was $0$, $c=0$.
$travel_m = -2.5s^2+bs$
$\frac{\text{d}y}{\text{d}s}-2.5s^2+bs$ (The first derivative of distance gives you the speed.)
$=-5s + b$, but at $s=0$, when the car braked, the speed was b, and they are asking for b.
When it stopped the speed was zero. To see when it stopped:
$-5s + b=0$ -> $\frac{b}{5}=s$
$-2.5s^2+bs = 60$
Substitute: $-\frac{5}{2}\times\frac{b^2}{25}+\frac{b^2}{5} = 60$ ->
$b^2=600$
$\boxed{b=10\sqrt6}$

Physics related antiderivatives problem


Upper bound on number of triples

Question: We choose arbitrarily 100 lattice points in the coordinate plane. Let M be the number
of triples (A, B, C) of distinct points chosen from these 100 points such that A and B
have the same y-coordinate, and B and C have the same x-coordinate. Prove that the
greatest possible value of M is 8100.
Comment: Your question is phrased as an isolated problem, without any further information or context. This does not match many users' quality standards, so it may attract downvotes, or be put on hold. To prevent that, please [edit] the question. This will help you recognise and resolve the issues. Concretely: please provide context, and include your work and thoughts on the problem. These changes can help in formulating more appropriate answers.
Comment: What have you tried so far?
Comment: 8100 looks like a square with dots on every lattice point.
Answer: This is a short proof. Let $C$ be one of the maximal configuration (that maximses $M$). $C$ can always be modified to a square structure such that $M$ is non-decreasing, hence $M=9*9*100=8100$.

EXplicit Construction:

Consider re-construction of $C$, as in our intuition: ($C$ is assumed to be a structure on lattice point only.)

consider a $10 \times 10$ square, translate C such that it is mostly covered in the square.

if there is an empty row/column in this square, move one row/column (that have maximal number of points outside the square) outside the square to that row/column without disturbing the orignal configuration.
(meaning that if points in that row/column have different coordinates as other points, then its new coordinates are again different.)

Denote the new configuration $C'$.
If there is a row with ($n$ elements) in the square with a 'hole'and also a point $X$ outside the square in the same row. Move $X$ to the hole, then $M$ is not changed.
Now reorder each row in decreasing number of elements in each row. which is in a row echelon form, ensure that the top left corner is in the top left corner of the square. (M not changed since 'row/column operation' don't change $M$.
Suppose $C'$ is not a square now, let $Y$ be the last element in the bottom row, place $Y$ to the last of the second last row (if these two rows have different number element. If not, place it to the third, etc), then $M$ is non-decreasing.
This operation stops only if number of element in each row are equal. Hence $C'$ becomes a rectangle.
Since we can proceed from any $C$ to a rectangular ($a \times b$) configuration, hence we can maximise $M$ as follows, $M=(a-1)(b-1)*100$ & $a*b=100$
Maximised if it is a square, hence we have $M=9*9*100=8100$.

Upper bound on number of triples


How to insert STRUCT values in Google Big Query?

Question: We have standart analytics table in GoogleBigQuery:
We need to insert new data, with:
event_date,
event_timestamp,
event_name,
event_params.key.value,
event_params.value.string_value,
device.advertising_id

And I can't understand, how i can do it, with "event_params.key.value, event_params.value.string_value, device.advertising_id".
Can you help me make a request?
Somethink like this
INSERT
INTO `table`.`analytics_183424923.events_intraday_20191120`
(
  `event_date`,
  `event_timestamp`,
  `event_name`,
  `device.advertising_id`
)
VALUES
(
  '20191120',
  1574212435206765,
  'custom_event',
  'deviceId'
);

Here our table in JSON.
[
  {
    "event_date": "20191120",
    "event_timestamp": "1574236440549659",
    "event_name": "SHOW_INTERSTITIAL",
    "event_params": [
      {
        "key": "has",
        "value": {
          "string_value": "0",
          "int_value": null,
          "float_value": null,
          "double_value": null
        }
      },
      {
        "key": "firebase_event_origin",
        "value": {
          "string_value": "app",
          "int_value": null,
          "float_value": null,
          "double_value": null
        }
      }
    ],
    "event_previous_timestamp": "1574236295986010",
    "event_value_in_usd": null,
    "event_bundle_sequence_id": "19",
    "event_server_timestamp_offset": "6519649",
    "user_id": null,
    "user_pseudo_id": "1d28ad03ef14b6991cbbcf5c69c5db3f",
    "user_properties": [
      {
        "key": "first_open_time",
        "value": {
          "string_value": null,
          "int_value": "1538305200000",
          "float_value": null,
          "double_value": null,
          "set_timestamp_micros": "1538302277504649"
        }
      }
    ],

    "device": {
      "category": "mobile",
      "advertising_id": "8ae303e4-b94b-4599-929c-c8ae3c418ed7",
    }
]

Answer: INSERT
INTO `table`.`analytics_183424923.events_intraday_20191120`
(
  `event_date`,
  `event_timestamp`,
  `event_name`,
  `device`
)
VALUES
(
  '20191120',
  1574212435206765,
  'custom_event',
  struct<category string, advertising_id string>('mobile', 'deviceId')
);

A simpler form would work in your case, but sometimes your have to specify the full field list of a struct as above.
Simpler one:
...
VALUES
(
  '20191120',
  1574212435206765,
  'custom_event',
  ('mobile', 'deviceId')
);


,  in Tkinter respond badly and how to keep the whole frame size

Question: I'm beginner.
I'm making a program when the mouse cursor is entering the area, the buttons are shown and when the cursor is leaving the area, the buttons are invisible.
But, especially Leave sometimes responds badly.
Are there any alternatives to realize this program?
Also, I want to keep the whole frame size even when the buttons are shown.
How can I keep the Frame size?
import tkinter as Tk
Class Frame(Tk.Frame):

    def __init__(self, master=None):
        Tk.Frame.__init__(self, master)

        self.f = Tk.Frame(self)
        self.f.pack()

        self.text_frame = Tk.Frame(self.f)
        self.text_frame.pack()
        self.text_frame.bind('<Enter>', self.on_enter)

        self.text_la = Tk.Label(self.text_frame, text='Hello!')
        self.text_la.pack()

        self.f_button = Tk.Frame(self.f)
        self.f_button.bind("<Leave>", self.on_leave)
        self.button = Tk.Button(self.f_button, text='Button1')
        self.button.pack(padx=5, pady=5, side=Tk.LEFT)
        self.button2 = Tk.Button(self.f_button, text='Button2')
        self.button2.pack(padx=5, pady=5,side=Tk.LEFT)

        self.listbox = Tk.Listbox(self)
        self.listbox.pack()

    def on_enter(self, event):
        self.text_frame.pack_forget()
        self.f_button.pack()

    def on_leave(self, event):
        self.f_button.pack_forget()
        self.text_frame.pack()

if __name__ == '__main__':

f = Frame()
f.pack()
f.mainloop()

Comment: Your code seems to work for me 100% of the time. Are you aware that `text_frame` doesn't expand to fill the whole width of the window? Maybe when you think it isn't responding it's simply because your cursor isn't over the frame.
Comment: Please try to fix your indentation in the provided code above using **`{}`** button, and with the equation of: `an indentation = 4 spaces` in mind.
Comment: What does "respond badly" mean?
Comment: _"I want to keep the whole frame size even when the buttons are shown. How can I keep the Frame size?"_ I think you should be able to do that unsetting the flag for [`pack_propagate`](http://effbot.org/tkinterbook/pack.htm#Tkinter.Pack.pack_propagate-method). As in `f.pack_propagate(False)` for example.
Comment: Respond badly means sometimes it doesn't respond.
Seems like the listbox below the Frame causes it.
Only when the mouse cursor leave though the listbox, Leave sometimes doesn't respond.
Comment: @spider_hand Please make sure the code you've provided runs without error(s).
Answer: Why you don't use the .forget()? Also, it's not if name == 'main':
So you can write something like:
    def on_enter(self, event):
        self.f_button.pack()

    def on_leave(self, event):
        self.f_button.forget()

if __name__ == '__main__':
    f = Frame()
    f.pack()
    f.mainloop()

Comment: @spider_hand: `forget()` and `pack_forget()` are identical.
Comment: What's  the difference between forget() and pack_forget()?
Comment: in fact forget() is a shortcut for pack_forget(). So it's better as you use it because sometime later you will use the grid_forget() and I think there is no shortcut there. But I also changed the 2 functions.
Comment: I want to make the text invisible when the button is shown up.
This way is just to show or hide the button part I think.
Answer: First of all "<Enter>" and "<Leave>" events feel inconsistent because they're bound(binded) to sub-frames, you can try replacing:
self.text_frame.bind('<Enter>', self.on_enter)
...
self.f_button.bind("<Leave>", self.on_leave)

with:
self.f.bind('<Enter>', self.on_enter)
...
self.f.bind("<Leave>", self.on_leave)

As self.f is the parent to both those frames the events should feel more consistent.

You can use pack_propagate method to disable that parent's resizing based on its children widgets' size demands. I found it easy to have it disabled right before self.f needs to resize from its largest yet required size to a smaller size, which is just before hiding the buttons for me. Since the buttons are hidden in on_leave that's the point which you can unset pack_propagate's flag:
def on_leave(self, event):
    self.f.pack_propagate(False)
    self.f_button.pack_forget()
    self.text_frame.pack()

Working code with suggested modifications:
import tkinter as Tk

class Frame(Tk.Frame):

    def __init__(self, master=None):
        Tk.Frame.__init__(self, master)

        self.f = Tk.Frame(self)
        self.f.pack()

        self.text_frame = Tk.Frame(self.f)
        self.text_frame.pack()
        self.f.bind('<Enter>', self.on_enter)

        self.text_la = Tk.Label(self.text_frame, text='Hello!')
        self.text_la.default_fg = self.text_la['fg']
        self.text_la.pack()

        self.f_button = Tk.Frame(self.f)
        self.f.bind("<Leave>", self.on_leave)
        self.button = Tk.Button(self.f_button, text='Button1')
        self.button.pack(padx=5, pady=5, side=Tk.LEFT)
        self.button2 = Tk.Button(self.f_button, text='Button2')
        self.button2.pack(padx=5, pady=5,side=Tk.LEFT)

        self.listbox = Tk.Listbox(self)
        self.listbox.pack()

    def on_enter(self, event):
        self.text_frame.pack_forget()
        self.f_button.pack()

    def on_leave(self, event):
        self.f.pack_propagate(False)
        self.f_button.pack_forget()
        self.text_frame.pack()

if __name__ == '__main__':

    f = Frame()
    f.pack()
    f.mainloop()

Alternatively, you can match self.text_la's text color to that of its background color, which would essentially hide its text using:
self.text_la['fg'] = self.text_la['bg']

Also in order to restore it back to its initial make sure to first put its text color in an attribute like:
self.text_la.initial_fg = self.text_la['fg']

and then assign it back to its text color to restore initial later:
self.text_la['fg'] = self.text_la.initial_fg

Working code with suggested modifications:
import tkinter as Tk
class Frame(Tk.Frame):

    def __init__(self, master=None):
        Tk.Frame.__init__(self, master)

        self.f = Tk.Frame(self)
        self.f.pack()

        self.text_frame = Tk.Frame(self.f)
        self.text_frame.pack()
        self.f.bind('<Enter>', self.on_enter)

        self.text_la = Tk.Label(self.text_frame, text='Hello!')
        self.text_la.initial_fg = self.text_la['fg']
        self.text_la.pack()

        self.f_button = Tk.Frame(self.f)
        self.f.bind("<Leave>", self.on_leave)
        self.button = Tk.Button(self.f_button, text='Button1')
        self.button.pack(padx=5, pady=5, side=Tk.LEFT)
        self.button2 = Tk.Button(self.f_button, text='Button2')
        self.button2.pack(padx=5, pady=5,side=Tk.LEFT)

        self.listbox = Tk.Listbox(self)
        self.listbox.pack()

    def on_enter(self, event):
        self.text_la['fg'] = self.text_la['bg']
        self.f_button.pack()

    def on_leave(self, event):
        self.text_la['fg'] = self.text_la.initial_fg
        self.text_frame.pack()

if __name__ == '__main__':

    f = Frame()
    f.pack()
    f.mainloop()

Comment: Alternatively, you can remove the label's text by replacing it with `""`, again though you'd need to save its initial text to restore later; just like you did with the color trick.
Comment: Thank you for suggestions! your first suggestion is almost perfect.
But the first way is changing the size of the Frame when i entered the area first time
How can I improve?
Comment: @spider_hand You could move `self.f.pack_propagate(False)` from `on_leave` to `on_enter`.


In Max. Likelihood the expected score is zero for the true values. Is it also true for any other values?

Question: The usual proof of the Expected score in ML expected score being zero goes 'similar' to this:
$f(z;\theta)$ is the density function, for data $z$, and parameter vector $\theta$, so $\int f(z;\theta)dz=1$ for any $\theta$. This implies that, under the usual regularity conditions that allow us to interchange derivations and integration operators, we have:
$\int \frac{\partial f(z;\theta)}{\partial \theta}dz=0 \Leftrightarrow \int \frac{f(z;\theta)}{f(z;\theta)} \frac{\partial f(z;\theta)}{\partial \theta}dz=0\Leftrightarrow \int f(z;\theta) \frac{\partial log f(z;\theta)}{\partial \theta}dz=0$
From what I could gather this is the usual proof. However, in no steps do we make an assumption that $\theta$ must be the true value. In my reasoning, as long as the parameter vector was in the domain allowed for the function $f$ to be a density, then we would always have the expected score equal to zero. Or am I missing something?
Comment: You might find it illuminating to see what your analysis says when $f(z;\theta)$ describes the family of Normal distributions of unit variance and unknown mean $\theta$.
Comment: @whuber, If the thetas are all the same, even if not the true one, the equations still seem true...
Answer: Verbal formulation: "the expected value of the derivative of the log-likelihood of the sample evaluated (the derivative) at the true value, equals zero".
Innocent question-crucial remark: what is the "expected value" of a function? (informal) A: it is an integral through which we weigh a function using as weight the true density of the random variables that appear in the function.
Assume we have a sample of size $n$ and we have specified the joint density $g(\mathbf z;\xi$), which may, or may not, be the true joint density which is $f(\mathbf z;\theta_0)$.
We then examine the (multiple) integral
$$E\left(\frac {\partial \log g(\mathbf z;\xi)}{\partial \xi}\Big |\xi=\xi_0\right) = \int f(\mathbf z;\theta_0) \cdot \left(\frac {\partial g(\mathbf z;\xi)/\partial \xi}{g(\mathbf z;\xi)}\Big |\xi=\xi_0\right)\text {d}\mathbf z$$
Note carefully which components of the integrand are evaluated at $\xi_0$ and which are not: only the derivative of the specified log-likelihood is, while the true density is not "evaluated" -by construction, it contains the true parameter $\theta_0$, irrespective of what we have specified for our sample.
Now, If $g(\cdot ;\xi) = f(\cdot ;\theta)$ then the above becomes
$$= \int f(\mathbf z;\theta_0) \frac {\partial f(\mathbf z;\theta_0)/\partial\theta}{f(\mathbf z;\theta_0)}\text {d}\mathbf z = \frac {\partial}{\partial \theta} \int f(\mathbf z;\theta_0) \text {d}\mathbf z = \frac {\partial}{\partial \theta} \big(1\big) = 0$$
The integral equals unity (i.e. it is constant) so its derivative w.r.t to $\theta$ will be equal to zero.
So, first conclusion: misspecification with respect to the distribution family destroys the result (*). But assume that you have specified the distribution family correctly, and let's evaluate the expression at some point $\theta_1$:
$$E\left(\frac {\partial \log f(\mathbf z;\theta)}{\partial \theta}\Big |\theta=\theta_1\right) = \int f(\mathbf z;\theta_0) \cdot \frac {\partial f(\mathbf z;\theta_1)/\partial\theta}{f(\mathbf z;\theta_1)}\text {d}\mathbf z$$
Here, the ratio $f(\mathbf z;\theta_0) /f(\mathbf z;\theta_1)$ no longer cancels, and so we cannot interchange the order of integration and differentiation to obtain the result as before (because then we would subject to differentiation the ratio that doesn't cancel also). Therefore, even though $f(\mathbf z;\theta_1)$ is also a density that integrates to unity, we cannot "leave it alone" inside the integral and exploit this property.
So, only at the true value does the result hold, because, the expected value is taken with respect to the true and unknown density, irrespective of our sample specifications.
--
(*) It also destroys the "Information Matrix Equality", see this short exposition of mine, https://alecospapadopoulos.wordpress.com/2014/05/13/information-matrix-equality/
Comment: You're welcome. _Always_ with respect to the true value/density. This is a theoretical result.
Comment: So the Expectancy is taken with respect to the true value. It makes sense. Thanks Alecos. :)


Rails Unicorn - Delay between starting request and reaching controller

Question: I am using Unicorn as my app server for my Rails app, and am trying to figure out why there sometimes is sometimes a non-trivial (> 5 seconds) delay between the start of a request, and when it reaches my controller.
This is what my production.log prints out:
Started GET "/search/articles.json?q=mashable.com" for 18.104.22.168 at 2015-07-23 14:59:19 -0400**
  Parameters: {"q"=>"mashable.com"}

Searching articles for keyword: mashable.com, format: json, Time: 2015-07-23 14:59:26 -0400

Notice how there is a 7 second delay in between STARTED GET: and "Searching articles for keyword", which is the first thing the controller method does.
articles.json is routed to my controller method "articles" which simply does this for now:
def articles
        format = params[:format]
        keyword = params["q"]

        Rails.logger.info "Searching articles for keyword: #{keyword}, format: #{format}, Time: #{Time.now.to_s}"

end

This is my routes.rb
MyApp::Application.routes.draw do
    match '/search/articles' => 'search#articles'
    #more routes here, but articles is the first route
end

What could possibly cause this delay? Is it because an Unicorn worker is busy? Is it because an Unicorn worker is taking up too much memory which leads the system to be slow?
Note: I don't believe the delay is in making any database connections but I could be wrong. The code doesn't need to make a database call, and the max connections for my database is 1000, and there are usually at most 1-2 connections.
Comment: could be a myriad of reasons - you'll need to monitor your system resources as well as unicorn. If nothing is in place, adding a newrelic agent might be the quickest/easiest option
Comment: Is this happening with Puma, Passenger etc as well?
Comment: What does the last line of the GET request say, ie the one that sais `Completed 200 OK in 100ms (Views: 50.0ms | ActiveRecord: 50.0ms)`
Comment: Have you attempted to use a different app server to see if it's just unicorn?
Comment: In what environment is this app running, with as many particulars as you can share?  Is the Rails environment production or development?  Is it in the cloud?  Whose and what are the specs of the virtual server?  In other words, what is the environment?  Also, what have you done to try to monitor or resolve the problem?  When the delay occurs, what has happened immediately before that?  Have you pushed new code so that the server is preprocessing assets, for example?  Thank you...
Comment: Network delay? Could it be that your clients need 10 seconds to finish sending request to the server?
Answer: Three thoughts:

You'll probably be better served using Puma instead of Unicorn
It could be that your system is running out of memory, or it could have plenty of memory available: install New Relic to troubleshoot where the bottleneck is
It could also be that you have more Unicorn instances than the number of connections your DB allows, in which case the instance is having to wait for others to disconnect before it can connect. This would likely manifest itself with irregular 5-second delays rather than happening every time.
Comment: But there are no database connections being made in between "STARTED GET" and the first log statement being printed. No meaningful work occurs.
Comment: Actually ActiveRecord (and thus Rails) manages this automatically, and only spools up a DB connection upon the first SQL query, which is exactly when you seem to be experiencing the problem.  More on that here: https://devcenter.heroku.com/articles/concurrency-and-database-connections#connection-pool.  Because you're using Unicorn, you have many instances running, and if this # is larger than the # of connections the DB will allow, then a disconnected process will have to wait for others to disconnect once it has submitted its first SQL query.
Answer: Actually, it might be caused by an before_filter callback, you should check it
Answer: I think it can be because of lack of memory and thus frequent garbage collection, which freeze whole system.
Comment: Unicorn uses just 15% of total memory, so I doubt that's it
Answer: If it's a production problem it could be caused by slow clients sending requests. New Relic and Monit are good options. You could consider sending signals to Unicorn workers to restart them to better understand the problem.
You could also try adding preload_app true in your Unicorn config to speed up the startup time of worker processes.
Comment: How can slow clients cause this problem?
Comment: Some good information here: http://www.smashcompany.com/technology/if-unix-is-good-for-unicorn-why-cant-unicorn-handle-slow-connections


Why do polynomial regressions have larger variance at the end?

Question: In reading the book "An Introduction to Statistical Learning with Applications in R", I came across this graph:

It shows that the point-wise variance is larger at the ends of the regression curve. Why is that? I thought that the variance may be larger because there seem to be fewer data points near the end, but the variance is calculated on the coefficients so the number of data points used in the estimate has no impact. Then I thought that the variance is larger because the X values are larger, but the graph on the left side shows slight increases in variance on both sides (even when X is small).
In general, I have read that polynomials have notorious end behaviours - what causes this?
Thanks.
Comment: If you don't get an satyisfaying answer try asking in stats-stackexchange.com
Answer: It's essentially due to Runge's Phenomenon for interpolating polynomials. Even without uncertainty, the behavior of higher order polynomials at their endpoints is very sensitive to the parameter values.
Here is a link to a technical explanation that I will regurgitate here. It relates Runge's Phenomenon to regression. Below is a intuitive, but admittedly non-rigorous explanation.
A good way to think about it is that, in order to be a good fit to the sample data (i.e., not veer completely out of the range of the data), you need to balance out the contributions from the higher order terms using lower order terms. An imbalance in this over the range of the function will result in a bad fit, so least squares requires ever more stringent "tuning" as you add more polynomial terms. Thus, small errors result in the endpoints "wagging" a lot more than the center of the function. This is especially bad if you don't have a lot of data at the edges of your interval.
This is not totally rigorous, but hopefully it shows intuitively why this would be so. The details would come from actually calculating the CI bounds, but I don't think that is as informative for the intuition.
Comment: I don't think you are answering the OP's question. First of all, regression is **not** interpolation. In interpolation, we try to find a function (usually a polynomial) that approximates another **function**. In regression, however, there may be data points with identical abscissa (age) but different ordinates (wages) and hence the data set is not presented as a sample of some function.
Comment: Secondly, the OP asks why the deviation of the data points from the interpolating curve are larger near the end of the curve, but in fact this is data dependent. E.g. if the data points are concentrated at (20,50) and (80,50) and there are a few data points with age between 30 and 70 but large variance in wages, the regression curve will likely pass through somewhere near (20,50) and (80,50) and the phenomenon mentioned by the OP may not occur.
Comment: @user1551 I disagree with your second comment.   Op is referring to the CI bands in the figures. Second, I know this is not interpolation...however the underlying reason that higher order polynomial are erratic is related to this phenomenon.
Comment: I also think you haven't answered why the variance increases at the edges.
Comment: @Nameless this phenomenon is not unique to higher order polynomials. It even happens in linear regression too.  The underlying math gets messy beyond the linear case. What additional type of information are you looking for?
Comment: @Nameless I think I've found an alternative explanation, but take me some time to put together. The formulas for CI bands are not easy to derive beyond that for linear cases, but I can show how higher order polynomials can be seen as higher order linear equations, where the predictors are collinear to some degree. This will result in highly variable parameter estimates and resulting fitted polynomials.
Comment: @Nameless all of this is also due to the fact that polynomials are unbounded in both directions, unlike hyperbolic and exponential functions.
Comment: @user1551 I was able to find a link to a textbook that I used for regression. It explicitly shows how Runge's Phenomenon for interpolation is related to regression (in a technical sense). I've added it to my post because I didn't feel like simply reproducing material (quite a bit of it is needed to really answer the question rigorously).


predict() leaving out values instead of giving NA

Question: I'm using predict on my random forest and nnet model and there will be a few values missing from the final predictions.
predict(psm_list_item$pmod, test_pac, type = 'prob')$yes

i=which(is.na(psm_pred))
return(data.frame(pred=psm_pred[-i], truth=psm_truth[-i]))

This does not work because the length of psm_pred is 6744 but the labels/truth is length 6748. I don't know if there's a way to match up the predicted probabilities with the truth this way, because it could be any 4 in the truth that could correspond to the missing predictions. Has anyone dealt with this?
Answer: I think nnet supports the all the standard na.action arguments. So, you can set na.action = na.exclude in the nnet call. Then predict will insert NAs into the predicted values at the appropriate places and your code should work.
Alternatively, you could try something like
i = which(is.na(df))
return(data.frame(pred=psm_pred,truth=psm_truth[-i]))

(assuming I've read your code correctly) which would exclude the true values from rows with missing data, and leave only the predictions and truth from the complete cases.


How to pass dynamic data to child component in angular 4?

Question: I have a parent component and i have to pass data to child once i will receive response in parent component.

Challenges
1) I have to wait to pass data till i will receive Response in Parent Component.
2) I want to execute particular function in Child Component once I will receive data from child.

Parent.component.ts
JSON.parse(this.responseData, (key, value) => {
 if (key === 'patient-list') {
      this.mrn = this.totalPatientList[0]['value']['patientmrn'];
}
});

Parent.component.html
<router-outlet></router-outlet>

Child.component.html
@Input() patientdata:any;

ngAfterViewChecked(){
console.log('Data Received -----', this.patientdata);
}

I hope I am able to explain my problem to you all.
Comment: instead  i can call child component in Parent
Answer: There is no child/parent communication in your example as it requires a component to be directly in the parent component.html.
You need to rely on guard resolver to be able to load 'Child' upon data reception.
Answer:
Use Services to pass data between multiple components.
To pass data after processing return an Observer instead of data, and the other component will be notified once the data is returned properly.
Answer: you are using routing, maybe the best way is use a singleton service for the communication of the two components.
Answer: All of the other answers are sensible, and will work cleanly.
An alternative would be to use @ngrx to manage state & enable a uni-directional flow of data to your components.
Answer: I was having problem to pass dynamic data from parent to child, searched all over but did not get any help. Finally i tried this one and it superbly worked for me. Try this one, it might help you.
child component:
export class ChildComponent {
  @Input() title: string;

  //rest code goes here...
}

parent component:
export class ParentComponent {
        headerTitle: string = '';

        //rest code ...
//on click changing the value
        onHeaderChange() {
            this.headerTitle = "MyHeader";
        }
    }

in parent component html file place code like below with single quote
<child-component [title]='headerTitle'></child-component>


Sphinx on OSX - Building on OSX

Question: Could anyone give me a bit of insite to why sphinx might not be installing on my Snow Leopard OSX Machine?
I currently have up and running (from installs not MAMP); Apache, MySQL, PHP.
When running ./configure --with-mysql=/usr/local/mysql --without-unixodbc (suggested to get this working) I get:
enter code here

checking build environment

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... config/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no

checking for compiler programs

checking whether to compile debug version... no
checking for gcc... gcc
checking for C compiler default output file name... rm: a.out.dSYM: is a directory
a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for ranlib... ranlib

checking for header files

checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... rm: conftest.dSYM: is a directory
rm: conftest.dSYM: is a directory
yes
checking for sys/wait.h that is POSIX.1 compatible... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking netdb.h usability... yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/file.h usability... yes
checking sys/file.h presence... yes
checking for sys/file.h... yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking expat.h usability... yes
checking expat.h presence... yes
checking for expat.h... yes
checking iconv.h usability... yes
checking iconv.h presence... yes
checking for iconv.h... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking sql.h usability... yes
checking sql.h presence... yes
checking for sql.h... yes

checking for types

checking for stdbool.h that conforms to C99... yes
checking for _Bool... yes
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking for off_t... yes
checking for size_t... yes
checking whether time.h and sys/time.h may both be included... yes

checking for library functions

checking for pid_t... yes
checking vfork.h usability... no
checking vfork.h presence... no
checking for vfork.h... no
checking for fork... yes
checking for vfork... yes
checking for working fork... yes
checking for working vfork... (cached) yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible realloc... yes
checking sys/select.h usability... yes
checking sys/select.h presence... yes
checking for sys/select.h... yes
checking for sys/socket.h... (cached) yes
checking types of arguments for select... int,fd_set *,struct timeval *
rm: conftest.dSYM: is a directory
checking return type of signal handlers... void
checking whether lstat dereferences a symlink specified with a trailing slash... no
checking whether stat accepts an empty string... no
checking for vprintf... yes
checking for _doprnt... no
checking for library containing setsockopt... none required
checking for library containing gethostbyname... none required
checking for library containing XML_Parse... -lexpat
checking for library containing iconv... -liconv
checking for library containing inflate... -lz
checking for library containing logf... none required
checking for library containing SQLConnect... -lodbc
checking for dup2... yes
checking for gethostbyname... yes
checking for gettimeofday... yes
checking for memmove... yes
checking for memset... yes
checking for select... yes
checking for socket... yes
checking for strcasecmp... yes
checking for strchr... yes
checking for strerror... yes
checking for strncasecmp... yes
checking for strstr... yes
checking for strtol... yes
checking for logf... yes
checking for pread... yes
checking for clock_gettime in -lrt... no
checking for clock_gettime... no
checking for LOCK_EX in sys/file.h... rm: conftest.dSYM: is a directory
yes
checking for F_SETLKW in fcntl.h... rm: conftest.dSYM: is a directory
yes

configuring Sphinx

checking for CFLAGS needed for pthreads... none
checking for LIBS needed for pthreads... -lpthread
checking for pthreads... found
checking whether to compile with MySQL support... yes
checking for mysql_config... /usr/local/mysql/bin/mysql_config
checking for mysql_real_connect... yes
checking MySQL include files... -I/usr/local/mysql/include  -g -Os -arch i386 -fno-    common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -    DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL
checking MySQL libraries... -L/usr/local/mysql/lib -lmysqlclient -lz -lm     -lmygcc
checking whether to compile with PostgreSQL support... no
checking whether to use 64-bit document/word IDs... no
checking whether to compile with libstemmer support... no
checking for libexpat... found
checking for libiconv... found
checking for iconv() arg types... char **
checking for UnixODBC... disabled
checking for unaligned RAM access... yes

generating configuration files

configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating libstemmer_c/Makefile
config.status: creating sphinx.conf.dist
config.status: creating sphinx-min.conf.dist
config.status: creating config/config.h
config.status: executing depfiles commands

configuration done

You can now run 'make' to build Sphinx binaries,
and then run 'make install' to install them.

However when I run make I get this:
Making all in src
if test -d ../.svn; then svn info .. --xml | perl svnxrev.pl; fi;
make  all-am
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinx.o -MD -MP -MF .deps/sphinx.Tpo -c -o sphinx.o sphinx.cpp
sphinx.cpp: In member function 'virtual void CSphIndex_VLN::DebugDumpDocids(FILE*)':
sphinx.cpp:14647: warning: format '%d' expects type 'int', but argument 3 has type 'long     unsigned int'
mv -f .deps/sphinx.Tpo .deps/sphinx.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT     sphinxexcerpt.o -MD -MP -MF .deps/sphinxexcerpt.Tpo -c -o sphinxexcerpt.o sphinxexcerpt.cpp
mv -f .deps/sphinxexcerpt.Tpo .deps/sphinxexcerpt.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxquery.o -MD -MP -MF .deps/sphinxquery.Tpo -c -o sphinxquery.o sphinxquery.cpp
mv -f .deps/sphinxquery.Tpo .deps/sphinxquery.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxsoundex.o -MD -MP -MF .deps/sphinxsoundex.Tpo -c -o sphinxsoundex.o sphinxsoundex.cpp
mv -f .deps/sphinxsoundex.Tpo .deps/sphinxsoundex.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxmetaphone.o -MD -MP -MF .deps/sphinxmetaphone.Tpo -c -o sphinxmetaphone.o sphinxmetaphone.cpp
mv -f .deps/sphinxmetaphone.Tpo .deps/sphinxmetaphone.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxstemen.o -MD -MP -MF .deps/sphinxstemen.Tpo -c -o sphinxstemen.o sphinxstemen.cpp
mv -f .deps/sphinxstemen.Tpo .deps/sphinxstemen.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxstemru.o -MD -MP -MF .deps/sphinxstemru.Tpo -c -o sphinxstemru.o sphinxstemru.cpp
mv -f .deps/sphinxstemru.Tpo .deps/sphinxstemru.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxstemcz.o -MD -MP -MF .deps/sphinxstemcz.Tpo -c -o sphinxstemcz.o sphinxstemcz.cpp
mv -f .deps/sphinxstemcz.Tpo .deps/sphinxstemcz.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxutils.o -MD -MP -MF .deps/sphinxutils.Tpo -c -o sphinxutils.o sphinxutils.cpp
mv -f .deps/sphinxutils.Tpo .deps/sphinxutils.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxstd.o -MD -MP -MF .deps/sphinxstd.Tpo -c -o sphinxstd.o sphinxstd.cpp
mv -f .deps/sphinxstd.Tpo .deps/sphinxstd.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxsort.o -MD -MP -MF .deps/sphinxsort.Tpo -c -o sphinxsort.o sphinxsort.cpp
mv -f .deps/sphinxsort.Tpo .deps/sphinxsort.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxexpr.o -MD -MP -MF .deps/sphinxexpr.Tpo -c -o sphinxexpr.o sphinxexpr.cpp
mv -f .deps/sphinxexpr.Tpo .deps/sphinxexpr.Po
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinxfilter.o -MD -MP -MF .deps/sphinxfilter.Tpo -c -o sphinxfilter.o sphinxfilter.cpp
mv -f .deps/sphinxfilter.Tpo .deps/sphinxfilter.Po
rm -f libsphinx.a
ar cru libsphinx.a sphinx.o sphinxexcerpt.o sphinxquery.o sphinxsoundex.o sphinxmetaphone.o sphinxstemen.o sphinxstemru.o sphinxstemcz.o sphinxutils.o md5.o sphinxstd.o sphinxsort.o sphinxexpr.o sphinxfilter.o
ranlib libsphinx.a
g++ -DHAVE_CONFIG_H -I. -I../config  -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/local/mysql/include  -g -Os -arch i386 -fno-common   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT indexer.o -MD -MP -MF .deps/indexer.Tpo -c -o indexer.o indexer.cpp
mv -f .deps/indexer.Tpo .deps/indexer.Po
g++  -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG   -o indexer indexer.o libsphinx.a  -L/usr/local/mysql/lib -lmysqlclient -lz -lm     -lmygcc  -lodbc -lz -liconv -lexpat  -L/usr/local/lib  -lpthread
ld: warning: directory '/usr/local/lib' following -L not found
ld: warning: in indexer.o, file was built for i386 which is not the architecture being     linked (x86_64)
ld: warning: in libsphinx.a, file was built for unsupported file format which is not the     architecture being linked (x86_64)
ld: warning: in /usr/local/mysql/lib/libmysqlclient.dylib, file was built for i386 which is not the architecture being linked (x86_64)
ld: warning: in /usr/local/mysql/lib/libmygcc.a, file was built for unsupported file format which is not the architecture being linked (x86_64)
Undefined symbols:
  "_main", referenced from:
      start in crt1.10.6.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[2]: *** [indexer] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

Answer: setting LDFLAGS to
'-arch i386'

fixed the problem for me on Mac OSX 10.6.4

Sphinx on OSX - Building on OSX


Bijection between closed uncountable subset of $\Bbb R$ and $\Bbb R$.

Question: This is probably a really stupid question but I'm hoping it's true: can we find a bijection between any uncountable closed subset of $\Bbb R$ and $\Bbb R$ (real number line)?
Comment: What you mean by "closed?" Is this a closed subset of $\mathbb{R}$ using the usual topology?
Answer: This is a nice result due to Cantor that is good to understand well. Let me expand André Nicolas's answer.
We need a modicum of set theory, specifically, a working understanding of $\omega_1$, the first uncountable ordinal.
1.
First, we need the Cantor-Baire stationary principle: Suppose that to each countable ordinal $\alpha$ we assign a closed set $E_\alpha\subseteq\mathbb R$ in such a way that $E_\alpha\supseteq E_\beta$ whenever $\alpha<\beta$. There must then exist a countable ordinal $\alpha_0$ such that for all $\alpha>\alpha_0$, $E_\alpha=E_{\alpha_0}$.
The idea of the proof of this result is actually simple (for details, Kechris's book on Classical descriptive set theory is an excellent reference): If $E_\alpha\supsetneq E_{\alpha+1}$, there is an open interval $I_\alpha$ with rational endpoints that meets $E_\alpha$ but not $E_{\alpha+1}$. Using that the sets are decreasing, we can in fact assign different intervals to different ordinals. This gives us what we want, since there are only countably many open intervals with rational endpoints.
This principle is applied in a variety of places in analysis. For example, in the theory of the Denjoy integral (though modern presentations use the Henstock–Kurzweil approach). The other typical application is to develop the basic results connected with the Cantor-Bendixson derivative, which is what we need now.
2.
The (Cantor-Bendixson) derivative $E'$ of a set $E\subseteq\mathbb R$ is the set of limit points of $E$. Since $E$ and its closure have the same limit points, we may as well assume that $E=E_0$ is closed. We can now form a decreasing sequence of closed sets $E_0\supseteq E_1\supseteq\dots\supseteq E_\alpha\supseteq\dots$ indexed with the countable ordinals, by letting $E_{\alpha+1}=E_\alpha'$, and taking intersections of the sets obtained so far at limit stages.
By the Cantor-Baire stationary principle, we must reach a stage $\alpha$ such that $E_\alpha'=E_\alpha$. This means that either $E_\alpha$ is a (non-empty) perfect set, or else $E_\alpha$ is empty.
Now, for any $E$, if $x\in E\setminus E'$, then $x$ is an isolated point of $E$, so $E\setminus E'$ is necessarily countable. This means that any closed set can be written as the union of a countable set (namely, the union of the isolated points of the $E_\beta$ for $\beta<\alpha$) and a set that is either empty or perfect (namely, $E_\alpha$).
The outcome of this argument is that if a closed set is uncountable, then it necessarily contains a perfect subset.
3.
The conclusion we seek is now reached by arguing that any perfect set contains a copy of the Cantor set $C$. Since $|C|=2^{\aleph_0}=|\mathbb R|$, this gives us the result that any uncountable closed set has the same size as $\mathbb R$.
To prove this, we use a recursive construction: Suppose $A$ is perfect, so it has at least two points $p_0$ and $p_1$. Let $r=|p_1-p_0|$ be the distance between $p_0$ and $p_1$ We can find small intervals $I_0$ and $I_1$ with disjoint closures that contain $p_0$ and $p_1$, respectively. Since $A$ is perfect, $A\cap I_0$ must contain at least two other points $p_{00}$ and $p_{01}$, both at distance at most $r/2$ from $p_0$, and similarly, $A\cap I_1$ must contain distinct points $p_{10}$ and $p{11}$ at distance at most $r/2$ of $p_1$. We can find intervals with pairwise disjoint closure, $I_{00},I_{01}\subset I_0$ containing $p_{00},p_{01}$, respectively, and similarly for $I_1$. Continuing this way, we produce points $p_s$ in $A$ for each finite sequence $s$ of $0$s and $1$s in such a way that for each infinite sequence $x=(x(0),x(1),\dots)$, the sequence $p_{x(0)},p_{x(0)x(1)},p_{x(0)x(1)x(2)},\dots$ converges. Call $p_x$ its limit. One easily checks that the points $p_x$ are all distinct, and all belong to $A$.
4.
This gives the result. Let me mention an extension. A set is Borel if it can be obtained by starting with the open intervals, and closing under the operations of countable union and complementation. A set is analytic if it is the continuous image of a Borel set.
First, one can easily check that, as long as we start with a Polish space in place of $\mathbb R$, the sketch above goes through, so any closed subset of a Polish space is either countable or of the same size as $\mathbb R$. (Recall that $X$ is Polish iff it is a completely metrizable separable space.)
Second, one can check, though this takes some work, that any analytic set is the injective image of a closed subset of the Polish space $\mathbb N^{\mathbb N}$. This means that the result holds not just for closed sets, but in fact for analytic sets: Any such set is either countable, or of the same size as $\mathbb R$.
As André pointed out, one of the early suggested approaches to the continuum problem was to argue by some sort of induction on the topological complexity of the sets of reals that they are countable or of size $|\mathbb R|$. The first step is to see this for open and closed sets, and this was done by Cantor. Early results in descriptive set theory extended this effort to include all analytic sets, by the method just indicated. Further extensions (going well beyond any sets that ever show up in analysis) had to wait until the late part of the twentieth century, and require large cardinals. Of course, now we know that "arbitrary" sets of reals do not need to have this perfect set property, but the descriptive set theoretic work I'm indicating shows that any sets without the property are in essence pathological.
Comment: No, but I am assuming a significant large cardinal structure to be present in the universe.
Comment: Would you consider $\mathbf{\Pi}_1^1$ sets to be pathological?
Answer: This is true. It is a consequence of theorems of 1) Cantor/Bendixson and 2) Cantor.
1.) Every closed subset of $\mathbb{R}$ is the union of an at most countably infinite set and a perfect set.
2.) A non-empty perfect subset of $\mathbb{R}$ has cardinality $c$.
Remark: This result was part of Cantor's search for a proof of the Continuum hypothesis. Related questions are still an active area of research.
Answer: Here is a proof that does not use any transfinite induction at all!

Take any uncountable closed subset $S$ of $\mathbb{R}$.
Let $S_n = S \cap [n,n+1]$ for each integer $n$.
Then $S_n$ is uncountable for some integer $n$, so we can assume that $S \subseteq [a,b]$ for some reals $a,b$.
Let $p = \sup \{ x : x \in [a,b] \land \text{$S \cap [a,x)$ is countable} \}$, which exists because $S \cap [a,a)$ is countable.
Then $S \cap [a,p) = \bigcup_{n=1}^\infty S \cap [a,p-\frac1n)$ is countable.
Similarly let $q = \inf \{ x : x \in [a,b] \land \text{$S \cap (x,b]$ is countable} \}$.
Then $S \cap (q,b]$ is likewise countable.
Clearly $p < q$, otherwise $S$ would be countable.
Let $r = \frac12(p+q)$. Then $S \cap [a,r]$ and $S \cap [r,b]$ are both uncountable, by definition of $p$ and $q$.
Thus $S \cap [p,r]$ and $S \cap [r,q]$ are uncountable closed subsets of $[p,q]$, and $r-p = q-r < \frac12(b-a)$.
Therefore we can iterate this process starting from $[0,1]$. Let $T$ be the binary tree with root $[0,1]$ where each node $[a,b]$ has children $[p,r]$ and $[r,q]$ with length at most half their parent's, such that $S \cap [p,r]$ and $S \cap [r,q]$ are uncountable.
For every downward path $P$ in $T$:
  $P$ is a sequence of nested closed intervals with length decreasing to zero.
  Let $c$ be the unique point in their intersection.
  If $c \notin S$:
    Let $u = \inf \{ x : x \in [0,c] \land S \cap [x,c] = \varnothing \}$.
    Let $v = \sup \{ x : x \in [c,1] \land S \cap [c,x] = \varnothing \}$.
    Then $c \in (u,v)$ and $S \cap (u,v) = \varnothing$.
    But eventually the interval along $P$ will have length less than $\min(c-u,v-c)$.
    Thus some interval $I$ in $P$ is contained in $(u,v)$, contradicting $S \cap P$ being uncountable.
  Therefore $c \in S$.
  Also $c$ is different for any downward path in $T$ that is different from $P$.
Therefore $S$ has at least as many elements as there are countably infinite binary sequences.
Thus the reals inject into $S$, and hence there is a bijection from $S$ to the reals.

As pointed out by Andrés E. Caicedo in the comments, my initial comment was false, because this proof uses countable choice twice, via the theorem that a countable union of countable sets is also countable.
Comment: I think it is a nice argument, by the way.
Comment: Note that I purposely made the intervals shrink to zero length so that I do not have to invoke even countable choice.
Comment: Arnie Miller showed in "A Dedekind Finite Borel Set" (Arch. Math. Logic 50, No. 1-2, 1-17 (2011); or [on arXiv](https://arxiv.org/abs/0806.1957)), that you don't need choice to get perfect subsets of uncountable Borel sets, provided that they can be written as a countable union of $G_\delta$ sets. Once you have this perfect subset, using Cantor–Bernstein you can construct the bijection.
Comment: @AsafKaragila: Ah I see thanks for your comment! But my proof does not use the replacement schema, so what generalization can be proven if we don't have replacement, or if we have only replacement on $ω$?
Comment: Yeah, I don't know. I haven't looked at the proof in details for a while. So I don't really remember.
Comment: You appear to be using countable choice, to conclude that one of the $S_n $ is uncountable.
Comment: And again to conclude that $S\cap [a,p) $ is countable.
Comment: @AndrésE.Caicedo: Hmm let's see. If $S_n$ is countable for each integer $n$, then I think you can make an enumeration through them all, exactly once each, because they are disjoint. First identify the indices for which it is finite. If there are finitely many of them, then we can enumerate them first and then go on to the rest. If not, then we can separately enumerate those (by going alternately forward and backward) and enumerate the rest (by an onion procedure). If there still is a problem, then I'll have to write it out to check; right now I am just using intuition.
Comment: Any two separate enumerations of disjoint sets can be combined easily into a single enumeration of their union. This finishes the above argument (if it is correct). But your second objection is more severe. I'll have to think about it.
Comment: Whether the sets are disjoint does not really matter. It is consistent with $\mathsf{ZF}$ that $\mathbb R $ is a countable union of countable sets.
Comment: @AndrésE.Caicedo: Since you doubt it, I will check my intuitive argument, but if you can please identify the place where I used countable choice. I will deal with your second objection later. As for the possibility of the reals being a countable union of countable sets, that does not imply that we cannot prove that $[n,n+1]$ is uncountable for some integer $n$...
Comment: Yes, each $[n,n+1] $ is uncountable. This seems to be irrelevant to the issue at hand.
Comment: @AndrésE.Caicedo: Sorry I don't understand your last comment; I was saying that independence of reals being a countable union of countable sets does not seemt to imply that my argument for uncountability of some $S_n$ is invalid. Do you agree with that? As for the second objection, I will have to alter my argument a little. What we have is that $[a,x)$ is countable for each $x < p$. That implies without choice that $[t,x)$ is countable for each $t \ge a$, because you can just enumerate and discard what is not in, and this is unique. Also, I can rewrite the union as a countable disjoint union.
Comment: Look: Fix bijections $f_I $ between $\mathbb R $ and each open unit interval $I $. This can be done explicitly and uniformly without any choice. Suppose $A_n $ is countable for each $n $ and their union is $\mathbb  R $. Define $S $ so that its intersection with $[n,n+1]$ is precisely the image of $A_n $ under $f_{(n,n+1)} $.
Comment: @AndrésE.Caicedo: Okay I know what the problem is now... In my intuitive argument I had been assuming you have a **sequence** of enumerations of $S_n$. If this is the case, then my argument should work (which is why I couldn't see anything wrong). However, if all we know is that there exists an enumeration of $S_n$ for each integer $n$, then we can't get that uniform sequence of enumerations. In any case I've no objection at all to dependent choice, so it was just a side remark that turned out false. I'll edit my post to make it clear. Thanks!


Find a generating function involving a tiling problem

Question: Let $h_n$ be the number of ways to tile a $1 \times n$ rectangle with $1 \times 1$ tiles that are red or blue and $1 \times 2$ tiles that are green, yellow, or white. Find a closed formula for $$H(x)=\sum_{n\geq0}h_nx^n.$$
I am unsure how to start this problem. Is there a way to solve this without using recurrence relations?
Answer: A recurrence relation is a good place to start.  By considering the possibilities for the first tile, we obtain
$$h_n=2h_{n-1}+3h_{n-2}$$
for $n \ge 2$.
The boundary conditions are $h_0=1$ and $h_1=2$.
Now the recurrence relation and boundary conditions yield
$$
H(x)-1-2x = 2 x (H(x)-1) + 3x^2 H(x),
$$
from which we find that
$$
H(x) = \frac{1}{1-2x-3x^2}.
$$
Partial fraction decomposition then yields
$$H(x)=\frac{1/4}{1+x}+\frac{3/4}{1-3x},$$
which immediately implies the explicit formula
$$h_n=\frac{1}{4}(-1)^n+\frac{3}{4}\cdot 3^n=\frac{(-1)^n+3^{n+1}}{4}.$$
Comment: The OP was looking for a solution without recurrence relations. I think this kind of problem might be possible with exponential generating functions, but I'm not an expert.
Answer: In general, if you want the generating function for the number of sequences of objects with a total "weight" of $n$, where each entry is taken independently from a set of $a_1$ objects of weight one, and $a_2$ objects of weight two, etc, then the answer is
$$
\frac1{1-(a_1x+a_2x^2+\dots)}=1+\underbrace{(a_1x+a_2x^2+\dots)}_\text{sequences of one object}+\underbrace{(a_1x+a_2x^2+\dots)^2}_\text{sequences of two objects}+\dots
$$
In this case, you are counting sequences of tiles, where the weight of a tile is its area, and you want the generating function for tilings of weight $n$. There are two tiles with weight $1$ (two squares), and three tiles of weight $2$, so the answer is $1/(1-(2x+3x^2))$.


Choice Modeling and MaxDiff

Question: I am new to the market research industry and have been exposed to a procedure called MaxDiff. I have seen this also called Best-Worst scaling. I am looking for good literature overview for this method and how it fits with respect to other survey design question choices. I am especially looking for a technical analysis of the method itself rather than an application. Thanks!
Answer: This is the only thing that I found on your subject:
TenBerge, J.M.F. (1988) Generalized approaches to the Maxbet problem and the Maxdiff
problem, with applications to canonical correlations, Psychometrika, 53, 487-494.
I hope this helps a little.

Choice Modeling and MaxDiff


Making a bot respond with the prefix when i mention it

Question: I want to be able to type "@Cool Boi prefix" and the bot will respond with what the current prefix is, for some reason it is not working, and it also shows no error message, my current code is:
@bot.event
async def on_message(message):
    if message.content.startswith('@Cool Boi prefix')
        await bot.say(' The current prefix is')

Answer: The actual text of the mention is slightly different from what Discord displays. One way to check if you've been mentioned is to use bot.user.mentioned_in
@bot.event
async def on_message(message):
    if bot.user.mentioned_in(message) and 'prefix' in message.content:
        await bot.send_message(message.channel, f'My Prefix is {bot.command_prefix}')

Or you could build the string using bot.user.mention
@bot.event
async def on_message(message):
    if message.content.startswith(f'{bot.user.mention} prefix'):
        await bot.send_message(message.channel, f'My Prefix is {bot.command_prefix}')