How to override a twig template for only a specific page?

Question: The title of this user view page contains the name of the user. I want modify it and, thanks to twig debug, I see the template used is page-title.html.twig:
{% if title %}

{{ title }}

{% endif %}

If I override it, how can I make twig use my overrided template only in the user view page?
Comment: So, how you are going to collect that string? Are you going to provide a input field so user can fill that field for this. Or you are going to set manually. Because, if this is the dynamic, Two things required, First from where this data(string) is coming and Second where it will be shown. So where it will be shown we can show that will be not big task, but how data will be input and how to fetch that is the thing.
Comment: I think there is more than one way to achieve this, it depends on what content you want to display, whether that content is dynamic or static. If static, there is a simple way, even you do not need to touch code. If dynamic, then it depends. So what you want to change?
Comment: I want add a string before the user name in the view user page (/user). Instead "CodeNext", have "Explorer CodeNext" by example. This string is dynamic, it depend some user parameters.
Comment: I just thought push the value of the prefix string from the theme hook, like this (https://www.drupal.org/docs/8/theming/twig/create-custom-twig-templates-from-custom-module). Then, from the template, Ill add the prefix string to the "user" name string. By that way, the problem is the template I want use is generic and affect all title of all page, not only the user view page
Answer: As you mentioned your main issue is whatever data you are fetching it is overriding the all page-title on the site which you do not want. So to solve that follow following steps,
For common understanding I am referring bartik theme. I am showing with example to my local site so it will be very practical to understand.
In bartik.theme file,
function bartik_preprocess_page_title(&$variables) {
  $variables['staticString'] = 'Explorer ';
  $variables['user_profile_view'] = (\Drupal::routeMatch()->getRouteName() == 'entity.user.canonical'); }

in page-title.html.twig file,
{% if title %}
  {% if user_profile_view %}

{{ staticString }}{{ title }}

  {% else %}

{{ title }}

  {% endif %}
{% endif %}

Output image,
On user page,

On other page(It will not interfere),

So, this way you can Override the page-title of user-profile page and can insert your own defined variable. So, this answer is with static variable, now same way, you have to just replace that variable with your dynamic variable, so it will get change based on your code.
Note: In solving this question, I have asked another question here: How to get the part of URI and pass it to as a argument?.
Comment: @user2137454 I got your 1st question, and answer is, Yes, every variable can be override, using the appropriate _preprocess hook, and preprocess hooks are made for that purpose only in Drupal, so before presenting data on site, if we want to change we can apply change. Could you please re-frame your second question?
Comment: @user2137454 That is very easy, which preprocess you have to use that will be provided in the comment section of that twig file in last lines. For example, If you open, `page.html.twig` in comment section in last you can see one line written as `@see template_preprocess_page()`. This way every template provides which _preprocess is corrosponding to them. For page-title it is written `@see template_preprocess_page_title()`, I do not know why in my bartik theme's page-title.html.twig, that line is not there..:), But it should be...
Comment: Work perfectly ! I have two questions , every twig variables are overridable by that way ? and, in "_preprocess_page_title", page_title is the name of the template I want override ? If no, how can I know I must use the string 'page_title' in "preprocess_page_title' ?
Comment: sorry for my english writing x) In the future, if I want override another template variable, how can I know what is the {HOOK} (the string) I must use in the function hook_preprocess_{HOOK} name function ?


Numbers theory, combinatorics

Question: How many numbers up to $1000$ are there such that the sum of digits is divisible by $7$ and the number itself is divisible by $3$?
Comment: HINT: a number itself is divisible by 3 if and only if the sum of its digits is divisible by 3
Comment: Then the digit sum has to divisible by 21.
Comment: Then the digit sum must be 21 assuming positive integers.
Comment: hint2: max digit sum  is 27
Comment: And also [here](https://math.stackexchange.com/q/2402018)
Answer: Possible sums of digits that are divisible by 3: 3,6,9,12,15,21,24,27
Possible sums that are also divisible by 7: 21
One possible such number is 678. Six numbers can be formed using the digits 6, 7, and 8:
678, 687, 768, 786, 867, 876
But there is only one way to write those digits in increasing order: 678
So we need only to find the numbers whose digits (1): sum to 21 and (2): are in increasing order.
The other numbers will just be those numbers with the digits rearranged.
\begin{array}{|c|c|}
\hline
\text{number} & \text{number of distinct rearrangements} \\
\hline
399 & 3 \\
489 & 6 \\
579 & 6 \\
669 & 3 \\
588 & 3 \\
678 & 6 \\
777 & 1 \\
\hline
\text{TOTAL} & 28\\
\hline
\end{array}

Numbers theory, combinatorics


Finding probability of all success for in an order statistic

Question: () = 5^4; 0 ≤  ≤ 1
A group of 3 friends order small cups of soda, from the soda dispenser. If
the 3 small cups are considered a random sample from the dispenser fills, find the probability
distribution for the minimum fill and answer, according to the customer satisfaction benchmark.
If customer satisfaction is benchmarked as satisfied by a fill of 2/3 or more for their chosen cup size, what is the probability, to 3 decimal places, that all three friends are satisfied
with their small cup? (You must answer in terms of the minimum order statistic 1′.)
Using order statistic, I can find the f(Ymin) with the equation: n*[1-F(y)]^(n-1)*f(y), where F(y) is the CDF and f(y) is the given pdf from above (1st line).
I can find the probability of f(Y'1) or f(Ymin) by taking the integral of the pdf. But to find all three friends are satisfied, would I then cube the probability that I found since all three friends success are independent?
Comment: Welcome to the site! Please take the tour to learn how to ask good questions here. Your explanation of what you have done so far is welcome, but it is partial and a bit sketchy. _Explicitly,_ what is the CDF of $Y?$ What answer do you get for the CDF of the minimum $V$ of three? How do you use the CDF of $Y$ to get the probability $P(Y \ge 2/3)$ that the a given _one_ of the friends is satisfied? Is that related to $P(V \ge 2/3)?$
Comment: I would assume in the equation for finding joint pdf: fy1',y2'(u,v) we would substitute u for y in fy1' and v for y in fy2'. Here fy1' = 15(y^15+2y^9+y^4) and fy2'=30y^9-30y^15 and CDF of y2' Fy2'(y) = 3y^10-2y^15.

So we can the double integrate with respect to dv and du. We do the integral of:
6*(1-Fy2'(v))*fy1'(u)*fY2'(v) dv du
Next, I am struggling with finding what the limits of the integration would be?
Comment: You need to be more careful using JaX for math notation, your equations are difficult to decipher. Perhaps you are confusing even yourself. // Carefully, state the support of each distribution used. All random variables $(Y$s and $V)$ have the unit interval as support.
Answer: The distribution of $Y$ is a member of the beta
family of distributions: $Y \sim \mathsf{Beta}(5, 1),$ with density $f_Y(y) = 5y^4,$ for $0 \le y \le 1,$ (otherwise $0);$
mean $\mu = \int_0^1 x(5y^4)\,dy = 5/6;$  and CDF
$F_Y(y) = \int_0^1 5t^4\, dt = y^5$ for $0 \le y \le 1$ $(0$ for $y\le 0; 1$ for $y \ge 1.$
Here is a simulation of $100\,000$ realizations of $Y$ which match the PDF and CDF shown above (plotted in red below):
set.seed(2022)
y = rbeta(10^5, 5, 1)
par(mfrow=c(1,2))
hist(y, prob=T, xlim=0:1, col="skyblue2")
 curve(5*x^4, add=T, col="red", lwd=2, lty="dotted")
plot(ecdf(y),lwd=3, col="blue")
 curve(x^5, add=T, col="red", lwd=3, lty="dotted")
par(mfrow=c(1,1))
summary(y)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
 0.1084  0.7579  0.8717  0.8341  0.9448  1.0000

Here is a similar simulation of $V = \min(Y_1,Y_2,Y_3),$ where the $Y_i$ are independent
and distributed as above. With $100\,000$ simulated values of $V,$ sample summaries should agree
with population summaries to about two decimal
places.
set.seed(314)
v = replicate(10^4, min(rbeta(3,5,1)))
par(mfrow=c(1,2))
 hist(v, prob=T, xlim=0:1, col="skyblue2")
 plot(ecdf(v),lwd=3, col="blue")
par(mfrow=c(1,1))
summary(v)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
 0.1548  0.6200  0.7318  0.7112  0.8204  0.9899
mean(v > 2/3)
[1] 0.6547

I will let you find
an analytic formula for the CDF of $V.$
It may be helpful to notice that
$$P(V > v) = P(Y_1 > v, Y_2 > v, Y_3 > v),$$
for $0 \le v \le 1.$


Find a recurrence relation for the number of bit strings of length n that contain consecutive symbols that are the same

Question: Here is my attempt:

First there is $2 \cdot 2^{n-1}$ ways if string ends with $00$ or $11$.
Second there are ways when string end with $10$ or $01$. So it will give us $a(n-1)$ ways to solve remaining part.

So the final answer would be $a(n) = 2^{n-1} + a(n-1)$. Am I right? If it possible could you explain this in your way, because I could not completely understand what I wrote. Thanks)
Answer: If a string of length $(n-1)$ has consecutive bits that are equal, then this will hold true for any string of length $n$ which is formed by appending a bit onto a valid string with length $n-1$. This is where the $2a(n-1)$ comes from (2 since we get to choose the last digit). Now to make sure we count all possible strings, we need to ensure that we count the strings who's first $(n-1)$ bits do not fulfill the condition, but where the string fulfills the condition. The only such strings are those with first $n-1$ digits $$1010… \text{ and } 0101…$$ which end in a repeat so there are two such strings. So I think the solution should be $$a(n) = 2a(n-1) +2$$.
Comment: As a follow up, you can check your answer is correct, by solving for a closed form (note that $a(2) = 2$) so any solution which gives you the closed form $a(n) = 2^n - 2$ which is the number of strings minus the number of strings with no repeats should be correct.
Comment: If there is string with length 4 then by brutal counting number of such strings will be 12. By your formula answer is 14
Comment: I don't see how this is possible, I don't think there are more strings than $1010$ and $0101$ of length 4 which don't have consecutive bits (this is because it must start with either 0 or 1 and the remaining bits are forced), and there are $2^4 = 16$ total strings of length 4.
Comment: Maybe missing some variants but let's try count manually. So strings with length 4 are:0000,0010,0001,1001,1000,1100,1111,0100,0111,1101,1110,0011,1011
Comment: Oh turns out I missed some combinations. You are right!


pop up table using CSS

Question: i m trying to make a pop up table which happens next to the biggest div that i have(right side) when the user clicks on the white button. But i m kinda out of ideas about how to do it. Is there any way to do it with bootstrap? If not still any tipps would be greatly appreciated !
https://jsfiddle.net/e97ut32r/

    <button></button>

#a {
    border: 2px solid #a1a1a1;
    padding: 10px 40px;
    background: #dddddd;
    width: 300px;
    height: 300px;
    border-radius: 25px;
}
#b {
    border: 2px solid #a1a1a1;
    padding: 20px 100px;
    background-color: white;
    width: 10%;
    height: 50%;
    border-radius: 25px;
    float: left;
    margin: 5px;
}
button{
    margin-left: 1cm;
    margin-top: 20%;
}

Answer: You can use modals, basically is an overall div where you can put any kind of content.
http://getbootstrap.com/javascript/#modals
Comment: but the problem is how can i make it exactly right next to my biggest div with size and width which i want ? sorry a totall newbie here..

pop up table using CSS


Drupal 7 errors after upgrading from 7.12 to 7.14

Question: I just updated my drupal version from 7.12 to 7.14 and now I get these errors when trying to update/edit a content type:
Warning: array_merge() [function.array-merge]: Argument #1 is not an array in _form_set_class() (line 4134 of /var/www/vhosts/xxx/public_html/includes/form.inc).

Warning: implode() [function.implode]: Invalid arguments passed in theme_radios() (line 2809 of /var/www/vhosts/xxx/public_html/includes/form.inc).

An illegal choice has been detected. Please contact the site administrator.

Please note that this is happening when I edit the node not the actual content type definition. But only for this one particular content type. All the other work fine still.
If it's at all helpful here are the fields for that content type:
Title - title - Node module element

Photo - field_photo_image - Image - Image

Vote - field_photo_vote - Fivestar Rating - Stars (rated while viewing)

Domain access - domain - Domain Access settings.

Description - field_photo_description - Long text - Text area (multiple rows)

Author - field_photo_author Text - Text field

Admin selected - field_photo_winner - Boolean - Single on/off checkbox

Any help would be much appreciated.
C
Answer: I ran into the same issue and found the solution here: http://drupal.org/node/1427430
Do you happen to also have the opengraph_meta module installed?
If so just edit this file:
sites/[your-site]/modules/opengraph_meta/opengraph_meta.module

search for this line:
'#attributes' => array('class' => 'opengraph-thumbs-wrapper clearfix'),

and change it to look like this one:
'#attributes' => array('class' => array('opengraph-thumbs-wrapper','clearfix')),

Answer: I think that something has gone wrong with the update version. Are you sure you've followed step by step the upgrade guide ?
http://api.drupal.org/api/drupal/UPGRADE.txt/7
Comment: I didn't out the site into maintainance mode.. everything else I did correctly. What do I do now to fix this.... The databse has been updated. Do I need to roll back to the previous version and import a db backup? then try this again?
Comment: Honestly I'm not very sure., but I do not see another solution.. let's wait a while, maybe someone comes up with another idea.
Comment: Hi @Voislav Sauca I did a rebuild then imported the site again with backup and migrate.. problem was still there. I then deleted the Content Type and recreated it.. problem was still there. I then started to remove one field at a time and discovered once I removed the `image` field it worked and errors were gone. I tried the `file` field instead and the errors came back. Do you know why this is happening?


Naturality of bijection given by Yoneda Lemma

Question: I'm reading through Wikipedia's proof of the Yoneda Lemma (https://en.wikipedia.org/wiki/Yoneda_lemma), and am having trouble understanding what naturality means in this context.
The articles proves that $Nat(h^A, F) \cong F(A)$. It then goes on to state that if both both sides are seen as functors from $Set^C \times C$ to $Set$, then the bijection provided by the lemma is natural. Other proofs say something like "The bijection is natural in both $F$ and $A$".
My problem is that I don't see what a morphism from $Set^C \times C$ should be mapped to by either of the functors. The functor on the right hand side would need to take a morphism from $f:A\rightarrow B$ from $C$ and a natural transformation $\sigma:F \rightarrow B$  and map them to a function from $F(A)$ to $G(B)$. I don't really see a canonical way of doing so, can somebody help me?
Answer: The details are reasonably grim but if you stare at them hard enough you'll see that the morphisms are the only things they could possibly be.
The actions on objects of the two functors $\mathsf{Set}^{\mathcal{C}} \times \mathcal{C} \to \mathsf{Set}$, respectively, are
$$(F,A) \mapsto \mathsf{Nat}(y^A, F) \quad \text{and} \quad (F,A) \mapsto F(A)$$
The actions of the two functors on morphisms are given as follows. Fix $\alpha : F \to G$ and $f : A \to B$.
The first functor sends $(\alpha, f)$ to the function
$$\theta_{\alpha,f} : \mathsf{Nat}(y^A, F) \to \mathsf{Nat}(y^B, G)$$
defined as follows. Given $\eta : y^A \to F$ and $C \in \mathsf{ob}(\mathcal{C})$, we have $\eta_C : \mathsf{Set}(A,C) \to F(C)$. Define
$$\theta_{\alpha,f}(\eta)_C : \mathsf{Set}(B,C) \to G(C)$$
by
$$\theta_{\alpha,f}(\eta)_C(B \xrightarrow{g} C) = \alpha_C(\eta_C(g \circ f))$$
This yields a natural transformation $\theta_{\alpha,f}(\eta) : y^B \to G$, and hence a function $\theta_{\alpha,f} : \mathsf{Nat}(y^A,F) \to \mathsf{Nat}(y^B, G)$.
The second functor sends $(\alpha, f)$ to the function $G(f) \circ \alpha_B : F(A) \to G(B)$:
$$F(A) \xrightarrow{\alpha_A} G(A) \xrightarrow{G(f)} G(B)$$
Equivalently, by naturality of $\alpha$, it sends $(\alpha, f)$ to the function $\alpha_A \circ F(f)$.
Answer: A morphism $\left(F, A\right) \to \left(G, B\right)$ in $\operatorname{Set}^C \times C$ has the form $\left(\alpha, p\right)$, where $\alpha : F \to G$ is a morphism in $\operatorname{Set}^C$ (that is, a natural transformation $F \Rightarrow G$) and where $p : A \to B$ is a morphism in $C$.
For every morphism $p : A \to B$ in $C$, there is a natural homomorphism $h^p : h^B \to h^A$ of functors from $C$ to $\operatorname{Set}$. It is defined as the natural homomorphism whose $D$-th component (for $D \in C$) is the map $h^B\left(D\right) \to h^A\left(D\right)$ which sends every morphism $q : B \to D$ to the morphism $q \circ p : A \to D$. If you don't already know this construction, do check that it actually defines a natural homomorphism; this is an instructive exercise.
In order to turn $\operatorname{Set}^C \times C \to \operatorname{Set}, \ \left(F, A\right) \mapsto \operatorname{Nat}\left(h^A, F\right)$ into a functor, we need to explain how a morphism $\left(\alpha, p\right) : \left(F, A\right) \to \left(G, B\right)$ in $\operatorname{Set}^C \times C$ gives rise to a map from $\operatorname{Nat}\left(h^A, F\right)$ to $\operatorname{Nat}\left(h^B, G\right)$. Namely, the latter map sends every natural homomorphism $\zeta \in \operatorname{Nat}\left(h^A, F\right)$ to the composition $h^B \overset{h^p}{\to} h^A \overset{\zeta}{\to} F \overset{\alpha}{\to} G$, where $h^p : h^B \to h^A$ is defined as above. It is easy to see that this actually gives a functor (i.e., composition and identities are preserved).
In order to turn $\operatorname{Set}^C \times C \to \operatorname{Set}, \ \left(F, A\right) \mapsto F\left(A\right)$ into a functor, we need to explain how a morphism $\left(\alpha, p\right) : \left(F, A\right) \to \left(G, B\right)$ in $\operatorname{Set}^C \times C$ gives rise to a map from $F\left(A\right)$ to $G\left(B\right)$. Namely, the latter map can be defined either as the composition $F\left(A\right) \overset{F\left(p\right)}{\to} F\left(B\right) \overset{\alpha_B}{\to} G\left(B\right)$, or as the composition $F\left(A\right) \overset{\alpha_A}{\to} G\left(A\right) \overset{G\left(p\right)}{\to} G\left(B\right)$. Both definitions yield the same map, because $\alpha$ is natural. Again, it is easy to check that this actually gives a functor.
Answer: We can actually exhibit the isomorphism $\phi $ in
$\tag1Nat(h^A, F) \cong F(A)$.
For any natural transformation $\nu :h^{A}\to F$, define $\phi :Nat(h^A, F)\to F(A)$ by $\phi (\nu)=\nu _{A}(1_{A})$.
Note that $\nu _{A}$ is simply the $A$ component of $\nu $. i.e. $\nu _{A}$ is a morphism, in fact a $set$ map: $\hom(A,A)\to FA$ so that $\phi $ sends $\nu $ to the element $\nu _{A}(1_{A})\in FA$. It turns out that with this definition, $\phi $ is bijective, hence an isomporphism of sets.
And now we may observe that for each $A\in C$ and $F\in Set^{C}$ we get such a $\phi $ which we should then write $\phi _{A,F}$ for clarity.
To say that $\phi $ is natural in $A$ and $F$ is to say that $\phi $ is a natural transformation of the functors $$\tag2\mathcal F:Set^{C}\times C\to Set$$ defined on objects by
$\mathcal F(F,A)=Nat(h^A, F)$
and on arrows by
$\mathcal F(\tau :F\to G,f:A\to B)=\psi :Nat(h^A, F)\to Nat(h^B, G)$,
where $\psi $ is defined by $(\psi (\nu ))_{c}=\tau _{c}\circ \nu _{c}\circ f^*$
and
$$\tag3\mathcal G=eval:Set^{C}\times C\to Set$$
$\mathcal G$ is clearly a functor and you can check that $\mathcal F$ is also.
The claim is then that $\phi _{A,F}$ are components of the natural isomorphism $\phi:\mathcal F\cong \mathcal G$.
The proof is somewhat lengthy but routine. Awodey has a clear demostration of this in his book Category Theory (Oxford Logic Guides) 2nd Edition.
Answer: Since others have given detailed answers, at appropriate math.se length, I won't try to do that. But as Clive says, the details can seem reasonably grim at first sight. So I'll just add that if you want a more slow-motion version, I have a bash at making the full-dress Yoneda Lemma, including the naturality claims, as clear as I can in these draft Notes on Basic Category Theory, esp Ch. 11.
Comment: Thank you! That link to your notes no longer works. But I see that it is now a book, Category Theory: A Gentle Introduction, https://www.logicmatters.net/categories/


Determining correlation between continuous and binary (yes or no) data?

Question: I am working with data from a survey with a sample size of 25, although not every question has 25 responses. We are looking at data from farmers. We are assuming that the data is normal, since we are treating our sample as the entire population. I have been using the pearson correlation coefficient for this. I have been treating values of R > 0.9 as significant, but when I am comparing continuous variables with binary variables (for example, land owned vs. do you use land for conservation purposes?), R is almost always < 0.9, even if they ought to correlate.
How do I find correlation between continuous and binary (yes or no) data? Should I just lower my threshold for significance? I suppose I could also split the data into two different groups based on whether they answer yes or no and see if the continuous values are significant between the two groups using Welch's t test, but this would be incredibly tedious.
Here is an example of what some of the data looks like:

Owned Acres
Farmers using land for conservation

5
1

367
1

0
0

5
0

10
1

300
1

11
0

100
1

100
1

72
1

1.25
1

0
0

0
0

52
1

5
1

75
1

26
1

10
0

76
1

10
1

150
1

0
0

6.25
0

0
0

0
0

I have not taken any stats courses in college yet, so I apologize if this question seems silly.
Comment: Welcome to Cross Validated! 1) Why do you treat your data as normal? 2) Why do you think you have the entire population when there are more than $25$ farmers in the world? 3) What does it mean to have a significant correlation? That term in statistics typically refers to a p-value from a hypothesis test, which you are not doing. 4) What's wrong with the usual Pearson correlation? 5) Missing data (and a small sample size) might be big issues for you, and if your work is important (more than a class assignment), you might consider hiring a consulting statistician to address those and any others.
Comment: @Dave Thank you for the warm welcome. 1)&2) We are treating it as normal because it is our entire population. We are only doing descriptive statistics for our sample. We are not doing inferential statistics. Therefore, I want to see if, in our sample, different things are correlated. My PI said that we are effectively doing A/B testing. 3) Interesting. I thought that when R < 0.9, the variables are not correlated (I learned this in my intro psych course), although I have also read that R values differ depending on the discipline. Basically I am wondering, when does R  mean something?
Comment: @Dave 4) Even if I had something really correlated, such as do you own land? and How much land do you own?, the R value is nowhere near being > 0.9 (I ran this analysis, and R = 0.3). Therefore, even when things are correlated, such as whether someone owns land and how much land someone owns, R < 0.9, just because I'm comparing binary data to continuous data. 5) The missing data would be cases where someone did not answer a question or cases where things did not apply to people (although, in this case, it can be fixed by entering 0 when it does not apply, such as how many cows they own).
Comment: The answer posted by @Michael Cooney is extremely good!  I would only add that a Welch's t-test seems like the right test for you here - asking if there is a difference in the mean of Owned Acres for the two conservation groups.  It is not that tedious, actually, and can be done easily in R, Python, or even Excel without much effort.
Comment: @KirkD_CO the problem is that I would not just be comparing land owned with whether they use it for conservation, I would be comparing it to about 20 more continuous variables. Additionally, I have around 30 more binary variables. I guess I'm wondering how I could make it quicker? Maybe I can't, and I might just have to do it manually. Thank you for your help!!
Comment: I see.  Yes, manually that would be tedious.  Do you know how to use R?  It wouldn't be difficult to write an R script to generate the results you want.
Comment: @KirkD_CO. I have R downloaded, but I don't really know how to use it. How would I write a script like that? Are there any forum posts that talk about that? I'm sorry if these questions are silly, I'm just new to stat stuff and have not learned R yet.
Comment: Since you don't know how to use R, learning it for this one problem may be a lot of extra effort.   You could do the same thing in Python if you know how to use Python.  If you're new to Python, too, then I would be glad to help you with either R or  Python, but we should take the conversation off line and out of the comments.  In my profile I have a Twitter link and LinkedIn - are you on either of those? If so, connect with me on either of those and we can continue the details of that conversation there. Your main question is answered here already, so we can work on implementation directly.
Answer: Pearson's correlation coefficient is measuring how well your data points fall on a straight line.  If they fall almost perfectly on a straight line, you'll either have a correlation close to 1 or -1.  Note: this isn't the slope of the line.  You could have an extremely steep line and a nearly flat line that both have the same correlation.  If you data isn't in a line, but a curve, you'll likely have correlation close to 0.  If your data has a lot of variability, you also will have lower correlation.  The pictures here help.

Now, if you only have 2 values, like your data, then having a curve is not possible.  You'll only get a straight line.  So if your correlation is lower, that means your binary data has a lot more variability than your other data.  In practice, this is pretty common for binary measures.  Often when you divide your population up into only 2 groups, each of those groups typically have a lot of variability (e.g. heights between men and women).
Here's a plot of your variables.

Farmers who use land for conservation have a large range and variability in acres owned.  If you draw the "best" line between these two groups, there will still be a lot of distance between that line and all the individual points.  The correlation here is 0.4621.
This is still a statistically significant value (p = 0.02), and judging by the plot, the effect size is fairly large, so the difference in groups is meaningful.  It just doesn't land as perfectly on a straight line as other sets of data you have.
To see very high correlation, you'd need the conservation acres to be clustered all around the same value, like the non-conservation acres are.  I replaced your data with a fabricated dataset where the conservation acres are all in the 300s.  With this dataset, the correlation is 0.9922 with this scatterplot.

What is "high" correlation varies a lot with what industry you are in.  If you are in physics, 0.9 correlation might be low.  If you are in social sciences, 0.3 correlation might be high.
Best regards,
Michael


Best-Fit without point interpolation

Question: I have two sets of data. One is nominal form. The other is actual form. The problem is that when I wish to calculate the form error alone. It's a big problem when the two sets of data isn't "on top of each other". That gives errors that also include positional error.
Both curves are read from a series of data. The nominal shape (black) is made up from many different size radius that are tangent to each other. Its the leading edge of an airfoil profile.
I have tried various methods of "Best-Fit" I've found both here and on where ever google took me. But the problem is that they all smooth my "actual" data. So it get modified and is not keeping it's actual form.
Is there any function in scipy or any other python lib that "simply" can fit my two curves together without altering the actual shape?
I wish for the green curve with red dots to lie as much as possible on top of the black.

Might it be possible to calculate the center of gravity of both curves and then move the actual curve in x and y depending on the value difference from the center point? It might not be the ultimate solution, but it would get closer?
Comment: how the nominal form is defined? by a set of points, or a parametric formula (it looks like an ellipse)?
Comment: Can't you move the black curve such that it best fits the red dots?
Comment: Thats a good idea. But I still think the nominal curve will change shape instead if I fit that to the actual points. I only wish to have them overlapping each other as good as possible. Even a couple of microns of change in the form of the data is unacceptable.
Comment: [scipy.spatial.procrustes](https://docs.scipy.org/doc/scipy-0.16.0/reference/generated/scipy.spatial.procrustes.html) seems relevant.
Comment: Thanks Stelios, I will check at procrusters. The nominals are also created from a list. It consists of different size radius that are tanget to each other. Ive clarified this in the original question.
Answer: Here is a solution assuming that the nominal form can be described as a conic, i.a as solution of the equation ax^2 + by^2 + cxy + dx + ey = 1. Then, a least square fit can be applied to find the coefficients (a, b, c, d, e).
<code>import numpy as np
import matplotlib.pylab as plt

# Generate example data
t = np.linspace(-2, 2.5, 25)
e, theta = 0.5, 0.3  # ratio minor axis/major & orientation angle major axis
c, s = np.cos(theta), np.sin(theta)
x = c*np.cos(t) - s*e*np.sin(t)
y = s*np.cos(t) + c*e*np.sin(t)

# add noise:
xy = 4*np.vstack((x, y))
xy += .08 *np.random.randn(*xy.shape) + np.random.randn(2, 1)

# Least square fit by a generic conic equation
# a*x^2 + b*y^2 + c*x*y + d*x + e*y = 1
x, y = xy
x = x - x.mean()
y = y - y.mean()

M = np.vstack([x**2, y**2, x*y, x, y]).T
b = np.ones_like(x)
# solve M*w = b
w, res, rank, s = np.linalg.lstsq(M, b, rcond=None)
a, b, c, d, e = w

# Get x, y coordinates for the fitted ellipse:
# using polar coordinates
# x = r*cos(theta), y = r*sin(theta)
# for a given theta, the radius is obtained with the 2nd order eq.:
# (a*ct^2 + b*st^2 + c*cs*st)*r^2 + (d*ct + e*st)*r - 1 = 0
# with ct = cos(theta) and st = sin(theta)

theta = np.linspace(-np.pi, np.pi, 97)
ct, st = np.cos(theta), np.sin(theta)

A = a*ct**2 + b*st**2 + c*ct*st
B = d*ct + e*st
D = B**2 + 4*A
radius = (-B + np.sqrt(D))/2/A

# Graph
plt.plot(radius*ct, radius*st, '-k', label='fitted ellipse');
plt.plot(x, y, 'or', label='measured points');
plt.axis('equal'); plt.legend();
plt.xlabel('x'); plt.ylabel('y');
</code>

Best-Fit without point interpolation


MySQL not a valid win32 application

Question: I'm running mysqldump.exe with needed parameters from cmd and this all works fine.
Then, all of the sudden, it started to give me a popup "Not a valid win32 application" and in the console it's saying 'Access is denied...'
What's causing this?
Thanks in advance.
Comment: sounds like the executable became corrupted.
Comment: Jeremy has a good comment. try downloading the .exe again. ALso, If the program is designed for a 64-bit version of Windows and you're running it in a 32-bit version of Windows, it will not work and generate this error. Hope this helps
Comment: executable became 0kb ... i copied new file, thanks guys!
Comment: @eddyuk where did you copy the new file?
Comment: @M.OğuzÖzcan In MySQL installation folder (in today versions C:\Program Files\MySQL\MySQL Server 5.7\bin). I had it installed on another computer and took the file from there because I didn't want to reinstall.
Answer: CAUSE
This issue may occur if the path of the executable file for the service
contains spaces.
When Windows starts a service, it parses the path of the service from left to right. If both of the following conditions are true:

The path of a service's executable file contains spaces.
There is a file or folder on your computer's hard disk that has the same name as a file or folder in the path to the service's executable file.

Windows may locate and try to run the file or folder before it locates
and runs the executable file for the service.
For example, if the path of the executable file for a service is
C:\Program Files\MyProgram\MyService.exe, and if a folder that is named
C:\Program also exists on your hard disk, Windows locates the C:\Program
folder on your hard disk before the
C:\Program Files\MyProgram\My
Service.exe

file, and then tries to run it.
RESOLUTION
WARNING: If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system.
Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.
To resolve this issue, add quotation marks around the following value in
the Windows registry, where  is the name of the service:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\<ServiceName>\ImagePath

The ImagePath value contains the other serviceable file for the
service. To add quotation marks around the ImagePath value for a service:

Click Start, and then click Run.
In the Open box, type regedit, and then click OK.
Locate, and then click the following registry entry, where
<ServiceName> is the name of the service:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\<ServiceName>.
For example, if the service is named MyService, locate the
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MyService key.
In the right pane, double-click ImagePath.
In the Value data box, add quotation marks around the complete path of the executable file for the service.

For example, if the path of the MyService service is
 C:\Program
Files\MyProgram\MyService.exe

, change the entry in the Value data box
for the MyService service to the following
"C:\Program Files\MyProgram\MyService.exe"
6. Click OK, and then quit Registry Editor./
So, create a new folder and installed services.
BAM! It worked. I have a C:\Web Dev\ folder I was using for testing and
the space between web and dev was the problem.

MySQL not a valid win32 application


Can open URL HTML

Question: Can I check if I can open a URL like OBLevel://id=18291234 in JavaScript before I link to it?
Thank you!
Comment: Is your question ? How to check if the given URL OBLevel://id=18291234 is a valid URL using JavaScript
Comment: @HatSoft I think he means open it like how `itms://` opens iTunes. This would require some ActiveX to examine the system preferences.
Answer: Not with Javascript.  You'd need to know what sort of applications are available on the computer (other than the browser), which isn't something you can know using Javascript.
Answer: Have you tried a GET with jQuery? If there's a answer, then you know you can use it. Otherwise if it's some obscure protocol or application not supported by jQuery, there's no way to do it.
Experiment!
Comment: This is not an answer, this should be a comment.

Can open URL HTML


Visual technique for fielding at point or cover

Question: When fielding at point or cover, should I focus my eyes on the batsman and try to pick the ball up when it enters my visual field as it's been hit; or should I look at the bowler as he runs up and keep my eyes on the ball while it is traveling down the pitch towards the batsman, turning my head keep track of the ball?
So far I have tried both and it seems to be easier to focus on the batsman, however if the ball is hit hard then I don't pick it up until it's almost past me.
Answer: Personally I always watch the batsman whenever I'm fielding in line with or in front of the bat.  Trying to watch the ball places you at a disadvantage as you can take more cues about the shot from the way the batsman is setting themselves up.  If you try and watch the ball from the bowlers hand you can only react to whatever happens, if you watch the batsman you can anticipate the shot and give yourself a chance to set yourself for the sort of shot they look to be playing.


Is there any common way to do pre-trend analysis for generalized Difference in Difference?

Question: The generalized difference-in-differences (DiD) estimator is DiD with multiple time periods and multiple groups. A very common example is when the impact of laws on firms happens at different times, in part because the laws are passed at different times in different countries. A good example can be seen from Dasgupta, 2019, where he examined the impact of anti-collusion laws on firms' asset growth in the global context.
I am trying to find a way to graph the aggregate trends or do pre-trend analysis (satisfying the parallel trend assumption). However, I cannot find any reasonable way to do so. The way Dasgupta did in his paper is unable to be replicated from my side because I cannot create the control group's relative time to the event (window [-2;+5] around the implementation years). I am wondering if there is any other way to deal with parallel trend testing in generalized DiD?
Comment: If you're doing a test, then you don't need to define this window for the control group; the values are consistently 0. The coefficients on the lead indicators should be bounded around zero.
Comment: The coefficient plot is a popular way to support common trends before some exposure of interest. Simply report the coefficients on some of the lead indicators.
Comment: Yes. But you don't have to do it that way. Some even recommend limiting the data to the pre-period and running the test that way.
Comment: @ThomasBilach , thank you for your help, as we mentioned in [this post](https://stats.stackexchange.com/questions/526787/how-to-plot-the-graph-or-perform-a-formal-test-for-testing-paralell-trend-for-ge). It seems to be a dead-end from my side in plotting the Figure like what Dasgupta did, so I am looking for an alternative test to verify the parallel assumption of DID setting. The *coefficient plotting* seesm to be for supporting the stability trends but not satisfy the parallel assumption from my point of view. Please correct me if I fall into any fallacy.
Comment: As you mentioned in that post "*It's one way to demonstrate the stability of the trends in the pre-period. It doesn't prove it, it's just evidence to support it.*:"
Comment: Thank you @ThomasBilach, so, do you mean "*Simply report the coefficients on some of the lead and **lag** indicators*"
Comment: Yay, I see it from your conclusion in that post "* Similar to the cited research, this is a form of pre-trends testing.*" . Thank you, @ThomasBilach
Comment: @ThomasBilach wow, is there any document supporting that, it is a huge help, or you can  write an answer here with this idea, which can help a lot of students. Because from [this post](https://stats.stackexchange.com/questions/526787/how-to-plot-the-graph-or-perform-a-formal-test-for-testing-paralell-trend-for-ge), all cited examples documented the leads and lags.
Comment: Because if we only do the test for leads indicators, how come we justify it is stability, I am wondering, maybe I missed something important
Answer: Plotting the aggregate trends across groups is one way to proceed, but when the adoption years vary so widely across $i$ then this approach is a bit messy.
Assessing coefficient leads in one approach. Consistent with a Granger-type causality test, leading values of the policy variable should not predict the current outcome. Here is one specification:
$$
y_{it} = \alpha_i + \lambda_t + \sum_{\tau = 1}^{q}\theta_{+\tau} d_{i,t+\tau} + \delta D_{it} + u_{it},
$$
where the model includes unit fixed effects, time fixed effects, a series of lead indicators $d_{it}$, and the contemporaneous policy variable $D_{it}$. The leads should be standardized in a way that $d_{i,t+1}$ is equal to 1 if a treated jurisdiction is 1 year before adoption, 0 otherwise. Similarly, $d_{i,t+2}$ is equal to 1 if a treated jurisdictions is 2 years before adoption, 0 otherwise. The equation generalizes to any number of $q$ leads. The choice of how many leads to include is for you to decide. The estimates of the $\theta_{\tau}$'s should be indistinguishable from 0, which some evaluators investigate using a joint null test. The goal is to assess the "collective significance" of the lead coefficients. Note this could fail in practice for many reasons. Firms may change their behavior in response to impending regulations. Law enforcement agencies may design interventions in response to emerging crime patterns. And the list goes on.
In my opinion, it's more common in practice to observe evaluators estimate leads and lags of the policy (treatment) variable. This approach offers a more complete picture of how effects evolve in the pre- and post-adoption periods. In other words, we can assess anticipatory effects and phase-in (phase-out) effects in a single regression equation such as the one specified below:
$$
y_{it} = \alpha_i + \lambda_t + \sum_{\tau = 1}^{q}\theta_{+\tau} d_{i,t+\tau} + \sum_{\tau = 0}^{m}\delta_{-\tau}d_{i,t-\tau} + u_{it}.
$$
The sums on the right-hand side of the equation allow for $q$ leads (i.e., $\theta_{+1}, \theta_{+2}, \theta_{+3},...,\theta_{+q}$) and $m$ lags (i.e., $\delta_{-1}, \delta_{-2}, \delta_{-3},..., \delta_{-m}$) of the policy variable. In applied work, evaluators typically plot the estimates of the $\theta_{\tau}$'s and the $\delta_{\tau}$'s over time. You'd hope your estimates on the adoption leads bound zero. If they do, it's evidence in support of common trends in the pre-policy epoch.
I more often see the latter equation estimated in practice. It affords you the opportunity to inspect the adoption leads and assess whether treatment effects vary with time since exposure. The decision regarding how many leads and/or lags to include outside of the immediate adoption period is often arbitrary, though it will depend, to some degree, on the number of pre- and post-policy adoption periods.
In short, I would choose the number of adoption leads (lags) and then estimate a model. Don't modify your results to tell a certain story. If you choose to estimate both equations, then report the estimates of the adoption leads from both equations! Miller and Chillar 2021 synthesize results from both approaches in a fairly digestible way.


What is the syntax in C# .NET for the following PHP snippet? [DataTables]

Question: There are the conditions I'm trying to archive:
If the selected site id (simply called site) is not submitted as part of the request, an empty data array should be shown on the client-side.
When the selected site information is submitted it should be used as a WHERE condition so the child table only displays data that matches that site.
below is the PHP Where condition (I need it in C# .NET):
if ( ! isset($_POST['site']) || ! is_numeric($_POST['site']) ) {
    echo json_encode( [ "data" => [] ] );
}
else {
    Editor::inst( $db, 'users' )
        ->field(
            Field::inst( 'users.first_name' ),
            Field::inst( 'users.last_name' ),
            Field::inst( 'users.phone' )
        )
        ->leftJoin( 'sites', 'sites.id', '=', 'users.site' )
        ->where( 'site', $_POST['site'] )
        ->process($_POST)
        ->json();
}

What I have (C#):
DtResponse response = new Editor(WebApiApplication.Db, "users")
    .Model<JoinModelUsers>("users")
    .Model<JoinModelSites>("sites")
    .Field(new Field("users.site")
        .Options("sites", "id", "name")
        .Validator(Validation.DbValues(new ValidationOpts { Empty = false }))
    )
    .LeftJoin("sites", "sites.id", "=", "users.site")
    .Process(formData)
    .Data();

return Json(response);

THANK YOU!
Comment: What is the `Editor` type? Is this a part of some ORM?
Comment: @Dennis Yes, Datatable Editor
Answer: I found the solution, I hope it helps someone else out there one day:
public IHttpActionResult Users()
        {
            var request = HttpContext.Current.Request;
            var settings = Properties.Settings.Default;

            using (var db = new Database(settings.DbType, settings.DbConnection))
            {
                var site = "-1";
                if (request.Form.Keys.Contains("site"))
                {
                    site = request.Form["site";
                }

                var response = new Editor(db, "users")
                    .Model<JoinModelUsers>("users")
                    .Model<JoinModelSites>("sites")
                    .Field(new Field("users.site")
                        .Options(new Options()
                            .Table("sites")
                            .Value("id")
                            .Label("name")
                        )
                        .Validator(Validation.DbValues(new ValidationOpts { Empty = false }))
                    )
                    .LeftJoin("sites", "sites.id", "=", "users.site")
                    .Where("site", site)
                    .Process(Request)
                    .Data();

                return Json(response);
            }
        }


Reopen class in canJs

Question: As a class gets bigger and bigger it becomes harder to maintain. So I would love to divide some of my classes in multiple files. Defining the class in a file, something like:
var animal = can.Construct({
  someClassMethod: function() {...}
}, {
  somePrototypemethod: function() {...}
});

then later in a separate file I would love to reopen this class, and add new class methods/properties, and prototype methods/properties to it.
Is this possible in canJS? If no, what should be a possible workaround?
Answer: You can always add properties and methods to the prototype and the constructor. That's how JavaScript works.
So as long as you still have access to animal you can just do
animal.anotherClassMethod = function() {}
animal.prototype.anotherPrototypeMethod = function()

in another file. However, having class definitions get too big is usually a problem of them doing too much. If they get too big to be maintainable in one file (which would mean 2000+ lines of code)  there is definitely a better way to refactor besides just splitting them up. A downside of splitting it into separate files is definitely that it might be hard to know what functionality is where.

Reopen class in canJs


Is there any rule about the length of a phrase or the value of a bar?

Question: I'm new studying music, and I have a question about the value of a bar, I'm not sure but seems that I can put any number of notes, considering the prestablished meter, and practically make it of any length.
I would like to know if is any kind of rule about the length of a phrase.
Thanks.
!
Answer: The time signature indicates the total lengths of all the notes in a bar.  So if the time signature is 4/4, then you could have:

One semibreve/full note
Two minims/half notes
Four crotchets/quarter notes
Eight quavers/eighth notes
...and so on

Or you could have any mixture of different length notes that adds up to the same total.
Comment: Each bar of your image adds up correctly to three quarter notes, corresponding to the 3/4 time signature. The brackets over the notes marked with "3" are *triplets,* and mean that the three notes are played in the time of two ordinary notes. If you hadn't learned that yet, then you would be confused about having "too many notes in a bar."
Comment: "is any kind of rule about the length of a phrase" - No, a phrase can be any length, but especially in popular music phrases are often 4 bars long. Your image seems to be just one 11-bar phrase, though.
Comment: And with non equivalent values?
Answer: If the music has a time signature (and the majority of western music that is performed today does have one) then the "number of notes" in each bar should match the time signature, as in Simon B's example.
There are a small number of exceptions to that "rule" - for example, a piece of music can start "in the middle of a bar", and the first bar of the score (often called a "pickup bar") will have fewer notes than the time signature indicates, because the rests and the start of the bar are not written. The same can apply to the last bar of the score, and repeated sections (shown by signs which look similar to bar lines) can also start and end in the middle of bars.
Before about 1700, a lot of music (except for dance music, which by its nature usually has a regular rhythm) did not have a time signature, and the bar lines were mainly to indicate the start of the musical phrases, and provide a reference points if the piece was for several performers.
Some contemporary music also does not use time signatures, but if you are just beginning to study music with an teacher, most likely all the music that you study in the first stages of the course will have time signatures.
Comment: Ok, but I think have read some sheets that have non symmetrical values, like 4-4 meter and 12 notes grouped in 3-8.
12-8 isn't a 4-4 equivalent, maybe 8/4, but 12/4 doesn't fit.
Maybe it's syncopation.
Answer: You are correct. Any number of notes may be placed within a measure, so long as the sum of those notes equals that of the pre-established meter.
Phrases, on the other hand, may be any length. With that being said, classical music tends to be divided by phrases consisting of four or eight measures each. Phrases in popular music also tend to be four bars in length. While this is more a matter of customary tradition, there is not a necessary rule for the length of phrases, unlike for that of measures.
Comment: Phrases are usually based on the harmonic progression.
Comment: It should perhaps be pointed out that the musical example here is from Giovanni da Firenze, mid- 14th century, and the phrase length is based on the cantus firmus length, not harmony.  Phrase lengths based on harmonic progressions don't show up until much later.
Answer:
but I think have read some sheets that have non symmetrical values, like 4-4 meter and 12 notes grouped in 3-8. 12-8 isn't a 4-4 equivalent, maybe 8/4, but 12/4 doesn't fit. Maybe it's syncopation.

answer to your comment above:
12-8 is an 4-4 equivalent, it's quite identical!
you can consider the 3/8 as a triplets. (same for 12-4: this will fit to 4-2 where  3  quarters will be notated as triplets.
Answer: 12-8 is compound, split into 4 dotted crotchet beats, or 4 x 3 quavers, whereas 4-4 is 4 normal crotchets and splits into two quavers. It's just the difference between wanting a simple time signature, where the beats all split into two, or a compound, where the beat (most often) splits into 3. If you have 12-8, you can have 4 lots of 3 quavers in a bar, 12, whereas 4-4 splits into 8 quavers, 4 lots of 2 quavers (if grouped according to the beat!).


Filter a value from a JSON object using Scala

Question: I need to filter a value for this name/value pair ("seq":15) in my JSON object using Scala.
Following is the link for my JSON object.
http://www.jsoneditoronline.org/?id=23188a59d8a10673bcfb632bc6337173
Following is the function I have used :
<code>  def wsResponseToValidJsonString(resp: String): String = {
println("This is response :" + resp) // prints Response message in JSON format

//get latest seq number
if (resp.contains("seq")) {
  println("seq contains..") //prints the JSON response
  val regSeq = new Regex(""".*?changes.*?"seq":(.*?),""")
  lastSeq = ((regSeq findAllIn resp)).toString()
  println(lastSeq) //prints non-empty iterator
}
return resp.split( """for\(;;\);""").last.stripPrefix("[").stripSuffix("]").trim
</code>
}
Comment: [SO] isn't a code writing service.  Please show us what you have tried and where you are stuck. See [mcve]
Comment: I have tried to filter the value using Regex. Even though couldn't filter it.
Comment: Again, show us what you've tried (this means code) and read that [mcve] link. We can't just guess what your problem is.
Comment: It is printing "Non empty iterator. But the regex working fine when I check it in online Regex tester
Comment: `findAllIn` returns a `MatchIterator` because there can be multiple matches. Converting it with `toString` gives thestring "Non empty iterator" since that's what it is - `toString` does not attempt to show its contents. Instead, you want to get the list of matches, presumably, so try `((regSeq findAllIn resp),toList).toString` (or similar, depending on what you need to do with the results). And you need to declare `lastSeq` somewhere.
Comment: Thanks! I have declared `lastSeq` at the top.  I just need to get only the value of  "seq" from the Regex for further calculations. But `toList()` is returning the whole array again.
Comment: I'm not sure what you're asking but forAllIn will return all matches, and there are multiple matches- since there are two "changes" in your JSON, both of which are followed by "seq" (the first has the other "change" between it and the "seq" but the regex doesnt' stop that. But trying to parse JSON with a regex is never going to work out well. Use one of the numerous JSON reading packages, and read the key/value there in a more structured way.
Comment: I'll check. Thank You
Answer: Problem was with the Regex. I have used JavaScript regex before. Following is giving the required value.
<code>def wsResponseToValidJsonString (resp: String): String = {
println ("This is response :" + resp) // prints Response message in JSON format
//get latest seq number
if (resp.contains ("seq") ) {
println ("seq contains..")
val regSeq = new Regex (""""seq":\d""")
lastSeq = (regSeq findAllIn resp).mkString;
println ("Last Sequence :" + lastSeq)//print required value}
return resp.split ("""for\(;;\);""").last.stripPrefix ("[").stripSuffix ("]").trim
</code>
}
Comment: Please use better formatting and tabbing for clarity.


Can both model parameters in linear regression be called regression coefficients?

Question: I was going through a review and I found that in the equation
$$
f(x_{i}) = w_{0} + w_{1}x_{i}
$$
both $w_{1}$ and $w_{0}$ are being called regression coefficients.
I have found references on the internet and I see that there are examples supporting the same and against it in text such as:
Against

Link here

"The slope b of a line obtained using linear least squares fitting is called the regression coefficient."

Link here

" In linear regression, coefficients are the values that multiply the predictor values. Suppose you have the following regression equation: $y = 3X + 5$ In this equation, +3 is the coefficient, X is the predictor, and +5 is the constant."
References that call them regression coefficients

Link of a paper here

The regression coefficients ($a$ and $b$) are estimated by minimizing the sum of squares of deviations of $y_i$ from the regression line. For a point $x_i$, the corresponding  given by the regression equation will be: $$y_{i} = a + bx_{i}$$

As per my understanding it should be that $a$ is constant and $b$ is the coefficient here. Can someone please shed some light on this one.
Answer: Both are valid and used.
Yes, $a$ or $w_0$ are often called constant. However, one would also count constants among the coefficients.
The constant can be viewed as the "slope" of the constant predictor $x_0=1$, so that $w_0x_0=w_0$. In this case, even the constant would be "a value that multiplies the predictor values".
Comment: I suppose you could. And if you use the matrix inversion steps to calculate the OLS coefficients yourself based on data, you might just include that 1-column in your X matrix. But when using standard software, nobody bothers to include that in the data.
Comment: Thank you for your answer, the assumption with $x_{0}=1$ makes a lot of sense, but should we not express that we have this value in our data, just to give mathematical validation to the same
Comment: Thanks makes a lot of sense


How to average cyclic quantities?

Question: Looking on Internet, I mostly found this definition:
Given quantities on a cyclic domain D, first rescale the domain to [0;2$\pi$[, then, find $z_n$ the point on the unit circle corresponding to the $n$th value, and compute the average by:
$$z_m = \sum_{n=1}^N z_n$$
The average angle is then $\theta_m = \arg z_m$ and to obtain the average value you scale back to your original domain D.
I must say, I have problems with this definition. For simplicity, I will use oriented angles in degree for my examples (i.e. D = [0;360[). With this formula, having angles -90, 90 and 40 will give 40 as mean angle, when I would expect 13.33 as an answer (i.e. (90-90+40)/3).
For my own problems, I usually use:
$$v_m = \mathop{\rm arg\,min}_{v\in D} \sum_{n=1}^{N} d(v_n,v)^2$$
Where $d(x,y)$ is the distance in the cyclic domain, and $\{v_1, v_2, \ldots, v_n\}$ is the set of cyclic data I want to average of.
It has the advantage to work the same way whatever the domain (replace D by a non-cyclic domain and $d$ with the usual euclidean distance, and you find the usual definition of an average). However, it is expensive to compute and I don't know any exact method to do it in general.
So my question is: what is the appropriate way to deal with average of cyclic data? And do you have good pointers that explain the problem and its solutions?
Comment: I know about this page. However, there is no justification for it. Also, I left a comment in the discussion of this page in the hope of understanding. But for now, I still disagree with this method to calculate the average.
Comment: http://en.wikipedia.org/wiki/Circular_mean
Comment: The choice of distance metric depends crucially on the application.  Bearing data, for instance, might be derived from estimates of X and Y with normally distributed errors, and this leads naturally to the circular mean.  For other cases this might not be a good choice.
Answer: Like all averages, the answer depends upon the choice of metric. For a given metric $M$, the average of some angles $a_j \in [-\pi,\pi]$ for $j \in [1,N]$ is that angle $\bar{a}_M$ which minimizes the sum of squared distances $d^2_M(\bar{a}_M,a_j)$. For a weighted mean, one simply includes in the sum the weights $w_j$ (such that $\sum_j w_j = 1$). That is,
$$\bar{a}_M = \mathop{\rm arg\,min}_{x} \sum_{j=1}^{N}\, w_j\, d^2_M(x,a_j)$$
Two common choices of metric are the Frobenius and the Riemann metrics. For the Frobenius metric, a direct formula exists that corresponds to the usual notion of average bearing in circular statistics. See "Means and Averaging in the Group of Rotations", Maher Moakher, SIAM Journal on Matrix Analysis and Applications, Volume 24, Issue 1, 2002, for details.
http://lcvmwww.epfl.ch/new/publications/data/articles/63/simaxpaper.pdf
Comment: Thank you, it is exactly the kind of reference I was looking for!
Comment: This answer seems to be the same as this answer here, excluding some example GNU Octave code: https://stackoverflow.com/questions/491738/how-do-you-calculate-the-average-of-a-set-of-circular-data/5542503#5542503
Answer: The problem with expecting the mean of 90°, −90°, and 40° to be (90°−90°+40°)/3 = 13.33° is that you would then expect the mean of 10° and 350° to be (10°+350°)/2 = 180°, and not 0° which is the more reasonable answer. It only gets worse when you have more than two angles (What is the mean of 340°, 350°, 360°, 10°, and 20°? What about 340°, 350°, 0°, 10°, and 20°?). Essentially, what you're doing there is equivalent to setting $z_n = e^{i\theta_n}$ and computing $$\bar z = (z_1 z_2 \cdots z_N)^{1/N},$$ and the problem is of course that it's not obvious a priori which of the $N$ possible roots of that equation is the right one, if any.
The "circular mean" definition is not so bad. In fact, it corresponds to the point which minimizes the sum of its squared distances to the points corresponding to the data, $$\bar z = \underset{\lvert z \rvert = 1}{\arg\min} \sum_{n=1}^N \lvert z - z_n \rvert^2.$$ So this is almost the same as the formula you like to use; you only have to define the "distance" between angles as the distance between the corresponding points on the unit circle. That is, $d(\theta, \phi) = \sqrt{2 - 2\cos(\theta-\phi)} = 2 \sin(\lvert\theta-\phi\rvert/2)$. This metric is close to $\lvert\theta-\phi\rvert$ when $\theta$ and $\phi$ are close, and has the advantage of being really easy to find the solution to.
Comment: Of the $N$ possible roots, one will be the global minimum of his distance functions, so checking all of them is sufficient.  (See the reference Rob Johnson gave).
Answer: The variables you mention are points belonging to the manifold, which is the circle. Therefore, they cannot be treated as if they belonged to Euclidean space.
I recommend the material that I have prepared on this subject and today I am sharing it on YouTube:
Circular means - Introduction to directional statistics.
There are two main types of circular mean: extrinsic and intrinsic.
Extrinsic mean is simply the mean calculated as the centroid of the points in the plane projected onto the circle.
$$
\bar{\vec{x}}=\frac{1}{N}\sum_{j=1}^N \vec{x}_j=\frac{1}{N}\sum_{j=1}^N [x_j,y_j]=\frac{1}{N}\sum_{j=1}^N [\cos{\phi_j},\sin{\phi_j}]
$$
$$
\hat{\bar{x}}=\frac{\bar{\vec{x}}}{|\bar{x}|}
$$
$$
\DeclareMathOperator{\atantwo}{atan2}
\bar{\phi}_{ex}=\atantwo(\hat{\bar{x}})
$$
It is NOT a mean calculated using the natural metric along the circle itself.
Intrinsic mean, on the other hand, does have this property. This mean can be obtained by minimizing the Fréchet function.
$$
\DeclareMathOperator*{\argmin}{argmin}
\bar{\phi}_{in}=\argmin_{\phi_0\in C} \sum_{j=1}^N (\phi_j-\phi_0)^2
$$
For discrete data, you can also analytically determine the $N$ points suspected of being the mean and then compare them using the Fréchet function.
$$
\bar{\phi}_k=\arg \sqrt[N]{\prod_{j=1}^N e^{i\phi_j} }=\bar{\phi}_0+k\frac{2\pi}{N}
$$
Where the N-th root is a N-valued function with outputs indexed with $ k\in\{1,\dots,N\} $. They are distributed evenly on the circle. And $ \bar{\phi}_0 $ is a usual mean calculated in an arbitrary range of angle values of length of $2\pi$.
If somebody dislikes the complex numbers
$$
\bar{\phi}_k=\frac{1}{N} \left(\sum_{j=1}^N \phi_j+k2\pi\right)
$$
The result is, of course, the same.
Then you have to compare the points suspected of being the mean using the Fréchet function.
$$
\DeclareMathOperator*{\argmin}{argmin}
\bar{\phi}_{in}=\argmin_{k\in\{1,\dots,N\}} \sum_{j=1}^N (\phi_j-\bar{\phi}_k)^2
$$
Where the search for minimum runs over $N$ discreet indices.
Answer: The angle is supposed to be the independent variable, not the dependent variable.  If your function is cyclic (using degrees), z(-90)=z(270) so it doesn't matter which you use.  Then the average value of the function is $$z_m=\sum_{n=1}^Nz(\theta_n)$$  You are right that if you average the angles, it matters which lap of the circle you use (-90 vs 270) because the difference of 360 gets divided by N.
Comment: This is what I am reading on Internet, but I have issues with this definition (as described in my post) and there is never a good explanation on why we should use this also for cyclic data. Could you expand?
Comment: The definition of a cyclic function is that f(x+T)=f(x) for some period T.  The recommendation you saw was to rescale the units of x so that the period is 2pi radians or 360 degrees.  I don't think that is important, but was trying to stay with it.  As the function is cyclic, it doesn't matter which cycle you take the data from.  So if you want the average value over a cycle it would be (if we take T to be 360) $$\frac{1}{360}\int_a^{a+360}f(x)dx$$  You can then approximate this by a sum just by taking equally spaced steps like $$\frac{1}{n}\sum_{i=0}^{n-1}f(\frac{360i}{n})$$
Comment: It looks like I am working on a different problem than you.  Did you check the Wikipedia page that Rasmus suggested?  Is that what you are after?
Answer: For angles, one can adapt the iterative way of computing means to angles, that is:
given angles v[1] .. v[n]
m[1] = v[1]
m[i] = remainder( m[i-1] + remainder( v[i]-m[i-1], C)/i, C)   (i=2..n)
where remainder(x,y) gives the signed remainder on dividing x by y and C is the measure of a circle.
I suspect this gives your vm, but haven't been able to prove it.
Comment: A quick check indicates it might not work. Applying this formula to the same set of values, simply taking the elements in various order leads to different results.

How to average cyclic quantities?


How do you calculate ROI for a new product or service offering?

Question: Ok so you have an idea to launch a new product to help you reduce your service desk call. Maybe it is a Employee Password Reset System, a Knowledge Base, or even a tech support chat bot.
What are items due you use for calculating your ROI and proving to the bean counter your idea is worth the investment?
Answer: The basic math is how many man hours per month will this system/feature save the company.  Multiple that times the average salary of the people who's time will be saved.  That's the money saved per month, move that out per year and per three years.
Now figure out the cost to the company to create the system/feature.  This is done by figuring out the number of hours it'll take to create the system/feature up front, then how many hours it'll take to maintain and upgrade the system every month or year.  Take the number of hours times the average salary for the person doing the work.
Subtract the money spent creating the system/feature from the amount saved using the feature and you have your ROI.  Be sure that the amounts that you use are based over the same time period.
Using an employee password reset system an as example.  We'll say the helpdesk gets 20 calls a week to reset a password.  It takes the help desk employee 10 minutes to verify the user and reset the password, plus the 10 minutes the user takes to find the help desks number, call and wait on hold.  We'll say the help desk guy makes $15 per hour, and the user makes $10 per hour.
So by giving the user a way to go to another users computer and reset there password (working under this assumption we're talking about Windows passwords here) is costing the company $66 per week in user salary and $50 per week in help desk salary.  So that's $6032 per year in cost.
Now, assume it'll take 30 hours to build and test the system, and basically nothing to maintain (since AD isn't changed all that often).  The Developer makes $35 per hour, so that's $1050 to make the system plus we'll throw in $100 in salary to maintain annually.  So over 1 year that $1150.
Since three year ROI is standard, over three years our costs are $1350 and our losses by not having the system in place is $18096.  So over three years the company will save $16746 by building the system.
Answer: We use a very simple algorithm, with the following inputs:

Time old method took
Time new method takes
Number of times operation is performed per day
(Old Time - New Time) * Frequency = Time Saved

Then it's just Time Saved * Hourly Rate = Money Saved
That gives the daily amount of $ saved. It could be a very very small number, don't distress if it is, because a single day is a very small time measurement in the business world.
Then once you know how much money it's saving your per day, it's not hard to extrapolate that to a yearly savings, and then subtract from that the implementation and support costs.
Edit: Actually, just refer to Mr Denny's response. He has vocalised this method in a much clearer way.


read public http csv data into Apache Beam

Question: I'm trying to use apache_beam.dataframe.io.read_csv function to read an online source with no success. Everything works if the file is hosted on google storage 'gs://bucket/source.csv' but fails on getting the file from 'https://github.com/../source.csv' like sources..
from apache_beam.dataframe.io import read_csv

url  = 'https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100.csv'

with beam.Pipeline() as pipeline:
    original_collection = pipeline | read_csv(path=url)
    original_collection = original_collection[:5]
    original_collection | beam.Map(print)

Giving me
ValueError: Unable to get filesystem from specified path, please use the correct path or ensure the required dependency is installed, e.g., pip install apache-beam[gcp]. Path specified: https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100.csv

Could anybody give me a hint?
Answer: Beam can only read files from filesystems (like gcs, hdfs, etc.) not arbitrary URLs (which are difficult to parallelize reads from). Local files work as well on the direct runner.
Alternatively, you could do something like
def parse_csv(contents):
  [use pandas, the csv module, etc. to parse the contents string into rows]

with beam.Pipeline() as pipeline:
    urls = pipeline | beam.Create(['https://github.com/datablist/sample-csv-files/...'])
    contents = urls | beam.Map(lambda url: urllib.request.urlopen(url).read())
    rows = contents | beam.FlatMap(parse_csv)

Probably easier to just save the file to a proper filesystem and read that...
Answer: I think it's not possible to load an external file on Beam.
You can think about another process or service than Beam that copies your external files to Cloud Storage bucket (for example with gsutil cp).
Then in your Dataflow job, you could read files from GCS without issues.


How to clean data to produce a smooth histogram

Question: Is there any way to 'clean' a set of data to produce a smooth histogram, ie. without overrepresentative bins? Looking for solution in python3.
At the moment I have a histogram with overcounted bins:
histogram
So far I've tried KDE but I don't know how to get Quantiles of it - how to divide it into k equal areas.

Thanks for your response.
Indeed, the goal is to cluster my one-dimensional dataset in such a way, to get the best visual separation when displaying it on a colour map. Mean - at which values should I border my bins.
I build a colour palette on historical data (shown on histogram) and use this style to the current data. So far I used Quantiles, but I wondering, how it would affect my colouring if I get rid of these quantitative peaks (assuming the number of classes is stable).
For example, 1s is divided into separate class only because it is very numerous.
Comment: Unlike forum-style discussion sites, Cross Validated threads do not consist of
a sequence of posts forming a conversation between the original poster and the
answerers. We use a Q&A style where the question is posted once at the top, and
is updated (using the "edit" button) where necessary for clarification. Sometimes
it makes sense to ask a new, follow-up question instead of editing the old one.
Please don't use the "answer" space to post clarifications to your own questions
- have a look at the [tour] to understand how our site works.
Comment: Showing the same value in different bins makes no sense. For a variable on $[0,1]$ why not just show bins with break points$0(0.1)1$?
Answer: The histogram arises, I guess, because what you are showing is the result of dividing integers that are mostly or at least often small, so some values are much more common than others. I don't think it is helpful, or even good terminology, to talk about bins being "over-representative" or even "over-represented".
Kernel density estimation is challenging here because you have both exact zeros and exact ones, it seems, and it's an uphill challenge to respect those facts without artefacts.
I think you have four main options open to you.

Accept the histogram as what it is. Here you should know, as we don't, who will be looking at the graph and what they might find interesting or helpful or that they might want to know about. For example, I am guessing at a very large number of ratios each based on very little information. How the histogram arises is what I would want to know as a report reader and I wouldn't want you to mess with your data in pursuit of a simpler graph.

A histogram with larger bin width, say of the order of 0.05.

A square root scale for bin counts which will dampen the spikes.

A quantile plot or (empirical cumulative) distribution function plot showing cumulative probability on one axis and value on the other axis, which won't remove the spikes but will make them less disconcerting.

These are general ideas that should be easily manageable in any decent software.
Your last sentence seems to hint that you want to bin these data for some other purpose, which is a different question. On the face of it, implying that a smooth representation of the distribution makes sense rather contradicts the idea that binning into disjoint intervals also is a good thing to do.
Comment: If you found this answer helpful, then please consider
[upvoting](https://stats.stackexchange.com/help/why-vote)
and/or
[accepting](https://stats.stackexchange.com/help/accepted-answer)
it.


Reference about the surgery of Ricci flow

Question: I roughly read the Topping's LECTURES ON THE RICCI FLOW. There does not seem to be an introduction on surgery. Seemly, it is enough to deal singularity by blow up. Then, in order to know surgery, I read the Perelman's Ricci flow with surgery on three-manifold. But it is not easy to read.  Are there any books that introduce the surgery, and likely to Topping's book (I feel Topping's book is friendly to read) ?
Answer: A group of five mathematicians (Bessières, Besson, Boileau, Maillot and Porti) have written a book aiming to give a complete proof of the Geometrisation Conjecture from pre-Perelman's results.
As such, they present a version of Ricci flow with surgery. I hope the book will appear suitable to you (I haven't read it in detail, but the introduction and first chapters seem remarkably clear to me).


What Block Sizes and Stripe Sizes will cause performance issues using RAID5?

Question: I'm trying to understand Block Size and Stripe Size.  I realize this could apply to more configurations than just RAID5, but i am interested in RAID5 specifically, because I'd like to understand the relationship between Block Size and Stripe Size, and the sizes that will cause performance degradation and why they cause performance issues...
What I understand so far, is that Stripe size would be the amount of space on each "column" (drive?) on the disk that belongs to a Stripe (basically a line of the same amount of space going across the drive
So if the Stripe Size is Equal to the Block size, lets say both are 64kb, what happens when I run a write operation coming from SQL server, which would do all 64Kb at once?
Will parity be calculated after each 64kb write?
The Stripe is typically larger than the Block.  If it's much much larger, will this have an effect on performance?
Here is an example:  I have HP Array - RAID5 configured.  The Block size on my logical drive is 512 bytes, the Windows unit allocation size is 4KB, and the Stripe size is 256KB I'm in a high write environment, and the Logical Drive Acceleration is Controller Cache, enabled and is set to 100%/0% (read/write)
What should I expect in terms of performance?  What does this have to do with the relationships between 512 byte block size, and 256 KB strip size?
Answer: See, here you err....

lets say both are 64kb, what happens when I run a write operation that's only a few bytes on
  a section of the drive that's full?(perhaps a large sql server
  database)

This is not possible.
Besides the fact that NTFS actually has blocks of 4kb or more - and it is STRONLY adviced to set that to 64kb for SQL Server.
SQL Server manages 8kb pages, and always reads /Writes 8 pages as extent - 64kb.
http://msdn.microsoft.com/en-us/library/ms190969(v=sql.105).aspx
As a result, for SQL Server, there is no thing such as writing a few bytes. It will write out 64kb.
As such, for SQ LServer, it is recommended to use a 64kb NTFS node size (so the extends do not cause split IO) and obviously a multiply of 64kb Raid strize size (as enterprise edition of SQL Server loves reading ahead exteds).
For other elements things are similar - not SQL Server... it depends on the intelligence of the programmer and the patterns of access of specific software.
Comment: Ok, it's becoming clearer, I feel like I need to edit the question, but i don't know what to change...  and when you say Node size, did you mean Block size?  is that the same thing?
Comment: Yes. Block size. SQL Server should be 64kb anyway - a high usage SQL Server has dedicated discs for SQL files (except boot and maybe the system databases which are low use) and the large block size will not have any negative consequences. Go with 256-512 as stripe size.


View is not working the intended way

Question: I created a view and set it as in the following screenshots.

[

If (under the Format section) I select Show:Content or any other setting, I can see the view being displayed on page and on the preview like I want it to show; if I select Show:Fields, the view doesn't show on page, but it does show on the preview.
To clarify the it shows on page part:

The theme is the right one
The region is defined on that particular theme, and I can display content on that region besides the one I really want to show, which also dismisses block settings preventing it from showing

Why is this happening? What can I do to make the view show on page when selecting the Show:Fields option?
Since I am a Drupal beginner, I don't know if the issue is merely caused from a wrong value I set for one or more settings.
Comment: could you export your view and past here, or if you're not sure how to do that, screenshot the view UI
Comment: can you expand the "Advanced" section in the right column and replace screenshot?  chances are- you have a contextual filter applied to the view and would only be able to see the preview results if you pass an argument.
Comment: Did you assign it under the right theme? Does the block have any settings on it preventing it from displaying? Is the region being printed in the theme? Also, what does `I select something like Show:Content` mean?
Comment: So, about the last question Kevin, in one of the screenshots you can see (under the Format field) the Show option, which is now set to Show Fields. If I select any other option, the content shows on page. That also answers some of the other questions : Right theme? Yes. Printed in the theme?When other options are selected. Something preventing block from displaying? No.
Comment: So the question went to [closed] .. Can anybody explain me what I need to do so that it becomes clear what the issue/question is? I really need a answer on this .. @Kevin
Answer: Sometimes a clear cache solves the issue.  Don't know if yours is the same issue but it happened to me and i cleared cache and it resolved the issue.
Comment: Tried your solution and it didn't work. Thank you for your time though.

View is not working the intended way


Detecting structural breaks in a dataset.

Question: I have some trouble detecting structural breaks in a dataset.
The dataset has multiplied time-series, and it follows a trend. I want to find a structural break when the time-series breaks the trend. The structural breaks occurs when the value randomly goes from i.e. 2000 and drops to zero, or close to zero. Others can drop down to 200, and then continue the trend. And in some cases the value can randomly go up and down, in this case the model should not interpret it as structural break.
I have tried to figure this out with studentized residual, the thought was that when someone broke the trend and churned it would give a value close to -2. But as you can see from my code, the time-series which I want to recognize as a structural break (y) only has a studentized residual value of -0.43 at the periode where it drops to zero.
I also tried with bfast without luck.
Here are some example data:
# Trending
x <- c(12:1)
#Structural break
y <- c(20,19,0,0,0,0,0,0,0,0,0,0)
#Trending
z <- c(0,0,20,19,18,17,17,17,17,17,17,17)
#Random drop and trending trending
w <- c(20:16,10:4)

The output would be:
c(x=FALSE, y=TRUE, z=FALSE, w=FALSE)

This is the code I used to find structural break:
library(bfast)
#Time-series
x.ts <- ts(x, frequency = 12)
m1 <- lm(x~1, data=x.ts)
#Studentized plot and values
plot(rstudent(m1))
rstudent(m1)
#bfast for detecting structural breaks
bfast01(x.ts, x~1, test=c("BIC", "OLS-MOSUM", "supLM"), aggregate=any, bandwidth = 0.15)

Comment: Can you clarify your question? What is it? Eg, I don't see a "?".
Comment: In R there is a `strucchange` package, you can try it, read its documentation!
Comment: @gung My quistion is: how do I find a suitable method to predict a structural break like variable y in the example data.
Comment: @kjetilbhalvorsen I've read and tried the strucchange package without luck.
Answer: In principle, the methods in the strucchange package (upon which bfast builds) can be used to detect the patterns you have created here. However, the sample size with 12 observations is very challenging for asymptotic methods as those in strucchange. As long as you only want to detect shifts in a piecewise constant mean (patterns y and z), you are better off with using permutation tests as those implemented in maxstat_test() from the coin package. If you really want to start out from a linear regression model (patterns x and w) you probably need more observations (or very small errors) to have a fair chance of detecting anything.
For illustration let's set up a data.frame called d which contains a time variable (which is more convenient for the coin package) and a ts series s (which is more convenient for the strucchange package). The latter is also used for visualizing the patterns
d <- data.frame(y, x, z, w, time = 1:12)
s <- ts(cbind(y, x, z, w))
plot(s, type = "b")

For the two patterns that are essentially shifts in a piecewise (almost) constant mean, we can use maxstat_test() from coin relatively easily. The p-values are found by approximation (i.e., simulating a finite number of permutations):
library("coin")
set.seed(1)
maxstat_test(y ~ time, data = d, dist = "approx")
##  Approximative Generalized Maximally Selected Statistics
##
## data:  y by time
## maxT = 3.3153, p-value = 0.034
## alternative hypothesis: two.sided
## sample estimates:
##   "best" cutpoint: <= 2
maxstat_test(z ~ time, data = d, dist = "approx")
##  Approximative Generalized Maximally Selected Statistics
##
## data:  z by time
## maxT = 3.2837, p-value = 0.0299
## alternative hypothesis: two.sided
## sample estimates:
##   "best" cutpoint: <= 2

In both cases the "true" breakpoint is found with a significant p-value (at 5% level). One could also tweak the Fstats() function from strucchange to reveal the same breakpoint; but the asymptotic inference that Fstats() is based on is surely less appropriate than the permutation inference. For a more detailed discussion of the differences between these tests see: Achim Zeileis, Torsten Hothorn (2013). "A Toolbox of Permutation Tests for Structural Change." Statistical Papers, 54(4), 931–954. doi:10.1007/s00362-013-0503-4
For the pattern w where a piecewise linear regression is needed, one could use Fstats() and breakpoints() but estimating two intercepts and two slopes from just 12 observations is really pushing it, especially if the breakpoint between the two segments also should be determined. In this artificial setup without any noise, it can be done but I wouldn't recommend it in practical situations with more noise.
library("strucchange")
coef(breakpoints(w ~ time(s), data = s, h = 4))
##        (Intercept) time(s)
## 1 - 5           21      -1
## 6 - 12          16      -1

Thus, the true segments/parameters are recovered here. But admittedly with a minimal segment size of h = 4 and no noise there is not much to estimate.
Note also that the ~ time part in the formula means different things for maxstat_test() and breakpoints(). In the latter case, it means that a (piecewise) linear time trend is used. In the former case, it means that a split in a (piecewise) constant mean is searched in the ordering by the time variable. (In the latter case, the ordering is implicit through using a ts time series data.)
Comment: I cannot replicate that error. For me the code from the question combined with my answer still runs smoothly.
Comment: Error in model.frame.default(formula = ~time, data = list(x = c(-0.00801579648658075, : object is not a matrix


Question about dissonance handling and counterpoint

Question: I'm self-learning 18th-century music theory because I like the music from that era. I'm obscure about the concepts of counterpoint and harmony. I'm aware that rules are not absolute within the 18th-century music. I heard the music adorned with augmented seconds. So I have a few questions.

In 18-century harmony, there are two distinctions: structural chords and prolonging chords. In this case, it appears that such dissonant leap usage is permitted as a prolonging chord.

In the case mentioned earlier, the prolonging chord dominant inversion is singular, but, it seems possible to prolong the chord even further than this. So, I'm unsure about the condition for allowing melodic dissonance in the strict composition. Is melodic dissonance only permissible when it's singular, and is the concept of prolonging chords broader?

I'm also struggling to grasp the concept of 'seventh.' It seems there are situations where the concept of 'seventh' is necessary. When looking at this phrase, explaining F# as a suspension resolved to E doesn't seem to fit:

In such cases, there are several cases where it's not inappropriate to refer to a note as a suspension without invoking the notion of 'seventh':

I've also learned that seventh should resolve after preparation like any NCTs, so I'm not sure how to distinguish between them.

I have these questions, leading me to the following inquiry.

So, it may sound peculiar, but when listening to fugues, it seems that both non-chord tones and chord tones can coexist within a single chord. In other words, it appears that there are instances where the melodic dissonance and chordal dissonance do not align. Here are a few observations I can make in this case:

The chord of first beat of measure 109 is a structural chord.
If the answer of 1. is restricted to only 'singular,' then either B# or E of measure 108 must be a prolonging chord. In other words, either B# or E is contrapuntal phenomenon. (in a broader perspective, it is currently within the dominant domain)
To explain the chordal dissonance between Fx and E, either G or E on beat 2 of measure 108 is a chord tone.

then, possible explanations could be E is a suspension and Fx is not contrapuntal, or Fx is a neighbor tone and E is not contrapuntal. However, if E is indeed not contrapuntal, it raises questions regarding the conditions for B# to be contrapuntal in the context of melodic dissonance, or whether melodic dissonance and chordal dissonance should be considered separately.

I have these questions, and therefore, I'm unsure how to perceive dissonance.
Comment: "doesn't seem to fit": why not?  It is a suspension, so we can only help you understand where your analysis has gone wrong if we know what your analysis is.  One thing that might help is to keep in mind that Roman numeral analysis was developed after Bach and does not describe how early to mid 17th century composers thought about their compositions.  That is, it is a theory that describes 17th-century music, but it is not 17th-century music theory.
Comment: Also "it seems that both non-chord tones and chord tones can coexist within a single chord": of course.  Chord tones define the chord; if they were absent then there would be no chord, or the chord would be different.  Consider: if you analyze a measure as C major but there are no Cs, Es, or Gs, then how do you justify analyzing it as C major?
Comment: Oops, my first comment should say "18th century" -- I've never managed to keep it straight.  But also "there are several cases where it's not inappropriate to refer to a note as a suspension without invoking the notion of 'seventh'": this seems to imply that suspensions necessarily involve sevenths, at least normally, but that's not at all the case.  They can also involve the inversion of the seventh, which is the second.  One of the most common is the "4-3" suspension (though this also typically has a second between the fourth and fifth of the chord).  Sequences of 5-6 or 6-5 are also common.
Comment: If that's a suspension, then from my perspective, it seems reasonable to view it as "decorated resolve" at measure 105, beat 2. / I've checked some unexplained points in the Bach chorale.
Comment: So, I started to separate the terms "non-chord tone" and "contrapuntal." If we think completely contrapuntally, whenever there's dissonance between two voices, one of them should be contrapuntal. In such discussions, explicit cases like sevenths or six-four chords, or examples such as viio6, where there's a tritone with another voice, suggest that one of them is contrapuntal. However, when I delve deeper into this, it becomes a bit tricky to analyze.
Comment: Thomas Benjamin 'The Craft of Tonal Counterpoint' suggests that Fx of measure 109, beat 1 is a neighbouring tone.
Comment: Sorry, your question is just too broad. But one question before I even attempt to untangle it: Why would you use the word "appoggiatura" for the first example? Either way, that example is contrapunctually wrong -- the F is a sensitive (leading) note and it should resolve to Eb, regardless of that being a full tone rather than a semitone. If it doesn't, you need a good explanation for that. The only one you have here is that it's parallel 6-3ths (faux-bourdon). This is just to show that it gets tricky quite fast.
Answer: The first example: There is nothing special about the example you show, certainly not in the soprano. The leap by a tritonus is very common and just standard 3rd-7th-1st progression in minor tonality by the rule of the octave in the soprano. In the bass, it's even more standard 1st-2nd-3rd.
The only unusual thing is the fact that the F is not resolved to Eb, which would be the proper thing -- while it is by a full tone, the role of the F as a leading note should not be ignored. This can be explained by fauxbourdon (parallel 6th chords), but it is slightly non-standard, in particular if the Eb isn't in the same position in another voice.
Now a bit more general: You can't really separate the "harmonic" and "ornamental" function. It helps when you start, but I have a feeling from your first call of the B natural as "ornamental" that you already confuse things a bit. So I'll offer another way of thinking: Each note has "a" function, and the note can be where it is if you have a way of explaining it by "a" function, regarless of what function it is. The confusing thing could be that both "appoggiatura" and "leading note" have a similar function -- they create a tension, and we have a good feeling of how that tension shall resolve. (Note that this means that if you don't resolve it the expected way, it's fine, as long as that lack of resolution plays a function, maybe to surprise the listener -- see how it complicates?)
Second example: That's a nice and a bit complex one. If it were a four-fold cadence into C#, I would say it's simply G# with figures 7 5 (3) -- 6 4 -- 5 4 -- 5 3, which is a common variant. But here it seems modified as (8) 7 5 -- (8) 6 4 -- (9) 6 4 -- (3)=(10) 7 5 and who knows where it goes. (I actually love this idea of 8 6 4 -- 9 6 4 and I shall implement it at some time as it's so cool!)
Thinking about it in terms of "7th chords" (in the modern way of chord naming) is so limiting that I would recommend avoiding that and just thinking about it as a very nice cadential progression over the penultimate!
Third example: Offers two explanations. You seem to explain the Bb as a 7th over the C that resolved to a 3rd of the F. This is probably more plausible given there is already a C major chord before that Bb, but you could also explain it as a suspension of the C in the 16th, so as ornamentation. (See how it gets complicated?) The rest is funnily simple: F 7, F# 7b as a sharpening, leaping to Bb major -- that's quite courageous, but why not :)
Fourth example: First, I would avoid using marks such as viio7/V, that's so limiting compared to the contrapunctual richness of the music! But if you insisted, then I would say it's F##7 (yeah, diminished etc., but that's not that important) -- G#7 -- c# progression, basically just subdominant -- dominant -- tonic, with the subdominant being enriched with a 7th instead of the usual 6th, the rest is just ornamentation. Bar 110 is then again a 7 5 -- 6 4 cadence. If you asked what role does bar 108 play, then it's just a dominant that's not resolved to a tonic but rather prolonged by sidestepping to the subdominant, basically making one very long penultimate step. (The function of this sidestep is to enforce the dominant nature of the G# bass note.)
If you wanted to be precise, then I'd say that structurally it is[108] G# 9 7 - 6 4 | [109] F## 7b 5(b) -- 6 -- 6 5(b) | G# 7 3#, but maybe some of the numbers in bar 109 can be taken as ornamentations. (I would be reluctant to call the E as an ornament though, as it's doubled by the end of bar 108, suggesting that one of them is a prepared note, really belonging to the first chord of bar 109, where it then probably should be considered as structural. But who knows.)
Conclusion: I'm not sure I helped, but I hope I at least managed to show the complexity and maybe a (non-unique) way of thinking about the music. TBH, if you are new to the topic, you didn't choose some simple music to start with.
Comment: I intended to watch it like you, so I intentionally brought those examples, as I find musical interest in those parts. It seems like you know how music works.
Comment: I didn't mean to say that appoggiaturas and leading tones have upward functions at that moment. What I meant is viewing harmony in a broader context. For example, when considering the penultimate dominant section of a c# minor fugue, from the perspective of the soprano line in a broader context, B# would be structural, and the rest would be ornamental phenomena. Therefore, I think that, for "musical reasons", the E in measure 109 is structural, and beat 2 of measure 108 is an anticipation.
Comment: In the case of that ornamental dominant seventh, in some instances I know, as you said, consecutive three parallel tenth motion allowed the ascent of F. However, according to Aldwell, it seems that such dissonant leaps and parallel motions are also permitted, I don't feel particularly convincing.
Comment: So, what I meant to say is that melodic dissonance seems to be justified within the larger context of harmony, similar to what I mentioned earlier. The question was whether you know the exact conditions for this justification.
Comment: Any dissonance (or consonance for that matter) is justified as long as it has a function. At least that's how we seem to understand e.g. JSB's music nowaydays. As for consonance -- dissonance: in the dorian tocatta by him, there are moments where you so much expect a 7th that the 8th Bach placed there functions as a dissonance. (And now excuse me, but I don't have the time to get into what seems like an endless discussion -- that's not even the point of a question-and-answer site like this one; it's not a discussion forum.)
Answer: For Roman numeral analysis, the second half of that measure is V7/V.  The F double sharp is definitely not a non-chord tone.  Bach would have thought of it as a 7-6 suspension between the soprano and bass.
From the comments:

If that's a suspension, then from my perspective, it seems reasonable to view it as "decorated resolve" at measure 105, beat 2.

I'm not familiar with the term "decorated resolve," but it is certainly a resolution, and it certainly is decorated, so I suppose so.

I started to separate the terms "non-chord tone" and "contrapuntal." If we think completely contrapuntally, whenever there's dissonance between two voices, one of them should be contrapuntal.

If we think completely contrapuntally, how can anything be not contrapuntal?  What would it mean for a note or a voice to be not contrapuntal?

Thomas Benjamin 'The Craft of Tonal Counterpoint' suggests that Fx of measure 109, beat 1 is a neighbouring tone.

Well yes.  But it's not a non-chord neighboring tone (as is the D♯ in the "decorated resolution" discussed above) because the harmony changes.


How do I scroll to the last id with jQuery

Question: I want when I click load more button content will automatic scroll to the last id. Hmmm
Example
$(document).ready(function(){
    $("#loadmorebutton").click(function (){
        $('#loadmorebutton').html('');
    $.ajax({
        url: "loadmore.php?lastid=" + $(".postitem:last").attr("id"),
        success: function(html){
            if(html){
                $("#postswrapper").append(html);
                $('#loadmorebutton').html('Load More');
            }else{
                $('#loadmorebutton').replaceWith('No more posts to show.');
            }
        }
    });
   });
});

Live Demo. On the demo I want, after click load more window will scroll to Post no 51 by example.
Answer: scrollTop() might be the function you might be requiring.
var height = 100;
$("#postswrapper").scrollTop(height)
Answer: On Button click, find out the window height var wh = $(window).height();  and apply it to window scrollTop $(window).scrollTop(wh) This will make whatever is on button of the page be first thing on top.