Finding non-consecutive increase over last 6 rows - alternatives to for loop in R?

Question: I'm looking for instances in my dataset where 4 of the last 6 samples show progressively increasing concentrations. I've worked out the logic of a series of if statements but I'm having trouble applying it to my data. I was planning on using cbind to attach the list to my dataset but the list ends up with 28 values and my data only has 24 rows.
I can't figure out what's happening with the for loop and have read that it's not a great way to do things anyways so I'm looking for alternatives.
EDIT: I've added some photos of two specific examples where suggested answers fail. The trouble seems to be that the "simple" solutions look for increases between consecutive points only. I'm looking for four points over each set of six that increase.
Row 16 and Row 20 plots
Here is some of the data:
SAMPLE_DATE <- c("2013-08-02", "2014-06-13", "2015-09-03", "2016-06-12", "2016-09-27", "2017-05-30", "2017-05-30", "2017-09-14", "2017-09-14", "2017-12-02", "2018-03-29", "2018-06-05", "2018-10-19", "2019-02-27", "2019-06-04", "2019-08-28", "2019-10-22", "2020-02-04", "2020-06-06", "2020-08-26", "2020-10-23", "2021-02-01", "2021-06-15", "2021-08-03")
REPORT_RESULT_VALUE <- c(0.1470, 0.0623, 1.4600, 0.1810, 0.0509, 0.0801, 0.0801, 0.0999, 0.0980, 0.0820, 0.0698, 0.0884, 0.1060, 0.1010, 0.0984, 0.1050, 0.1100, 0.0980, 0.1000, 0.1090, 0.1050, 0.0662, 0.0944, 0.1220)
GWSubsetData <- data.frame(SAMPLE_DATE, REPORT_RESULT_VALUE)

And here is what I've attempted:
Groundwater_ST1 <- vector("list")

    for (i in seq_along(GWSubsetData$REPORT_RESULT_VALUE)) {

        if (i >= 6) {

            a <- i-5
            b <- i-4
            c <- i-3
            d <- i-2
            e <- i-1

            #If i > 3 of first 5 samples
            if (sum(GWSubsetData$REPORT_RESULT_VALUE[[i]] > GWSubsetData$REPORT_RESULT_VALUE[a:e]) >= 3) {
                #If i>E and E > 2 of first 4 samples
                if ((GWSubsetData$REPORT_RESULT_VALUE[[i]] > GWSubsetData$REPORT_RESULT_VALUE[[e]]) &
                    (sum(GWSubsetData$REPORT_RESULT_VALUE[[e]] > GWSubsetData$REPORT_RESULT_VALUE[a:d]) > 2)) {
                    #if E>D and D > 1 of first 3 samples
                    if ((GWSubsetData$REPORT_RESULT_VALUE[[e]] > GWSubsetData$REPORT_RESULT_VALUE[[d]]) &
                        (sum(GWSubsetData$REPORT_RESULT_VALUE[[d]] > GWSubsetData$REPORT_RESULT_VALUE[a:c]) >= 1)) {
                        Groundwater_ST1[i] = TRUE
                    #If E>C and C > 1 of first 2 sampels
                    }else if ((GWSubsetData$REPORT_RESULT_VALUE[[e]] > GWSubsetData$REPORT_RESULT_VALUE[[c]]) &
                              (sum(GWSubsetData$REPORT_RESULT_VALUE[[c]] > GWSubsetData$REPORT_RESULT_VALUE[a:b]) >= 1)) {
                        Groundwater_ST1[i] = TRUE
                    #If E>B and B>A
                    }else if ((GWSubsetData$REPORT_RESULT_VALUE[[e]] > GWSubsetData$REPORT_RESULT_VALUE[[b]]) &
                              (GWSubsetData$REPORT_RESULT_VALUE[[b]] > GWSubsetData$REPORT_RESULT_VALUE[[a]])) {
                        Groundwater_ST1[i] = TRUE
                    }else{
                        Groundwater_ST1[i] = FALSE
                    }
                #If i>D and D > 2 of first 3 samples
                }else if ((GWSubsetData$REPORT_RESULT_VALUE[[i]] > GWSubsetData$REPORT_RESULT_VALUE[[d]]) &
                          (sum(GWSubsetData$REPORT_RESULT_VALUE[[d]] > GWSubsetData$REPORT_RESULT_VALUE[a:c]) >= 2)) {
                    #If D>C and C > 1 of first 2 samples
                    if ((GWSubsetData$REPORT_RESULT_VALUE[[d]] > GWSubsetData$REPORT_RESULT_VALUE[[c]]) &
                        (sum(GWSubsetData$REPORT_RESULT_VALUE[[c]] > GWSubsetData$REPORT_RESULT_VALUE[a:b]) >= 1)) {
                        Groundwater_ST1[i] = TRUE
                    #If D>B and B>A
                    }else if ((GWSubsetData$REPORT_RESULT_VALUE[[d]] > GWSubsetData$REPORT_RESULT_VALUE[[b]]) &
                               (GWSubsetData$REPORT_RESULT_VALUE[[b]] > GWSubsetData$REPORT_RESULT_VALUE[[a]])) {
                        Groundwater_ST1[i] = TRUE
                    }else{
                        Groundwater_ST1[i] = FALSE
                    }
                #If i > c > b > a
                }else if ((GWSubsetData$REPORT_RESULT_VALUE[[i]] > GWSubsetData$REPORT_RESULT_VALUE[[c]]) &
                          (GWSubsetData$REPORT_RESULT_VALUE[[c]] > GWSubsetData$REPORT_RESULT_VALUE[b]) &
                          (GWSubsetData$REPORT_RESULT_VALUE[[b]] > GWSubsetData$REPORT_RESULT_VALUE[[a]])) {
                    Groundwater_ST1[i] = TRUE
                }else{
                    Groundwater_ST1[i] = FALSE
                }

            }else{
                Groundwater_ST1[i] = FALSE                }
        }else{
            Groundwater_ST1[i] = FALSE
        }
    }

Comment: One potential wrinkle here is that your sample data has two dates (2017-05-30 and 2017-09-14) which appear twice each with different values, making it ambiguous (aside from the original row order) which order they should be evaluated upon. You'd potentially see (or miss) trends based on that ordering. It might be simplest to resolve that by taking the average per day?
Answer: Edit:
I acknowledge that this approach doesn't lead to your expected result, but I wanted to post it in case it spurs you/others to come up with alternative approaches to brute forcing (if there are any...). Here, I'm defining "progressive increase" based on the direction of the slope of a linear relationship between date and val:
dat$date2 <- as.numeric(as.Date(dat$date, "%Y-%m-%d"))
w <- 6 # window

## With a loop
dat$slope <- NA
for(i in w:nrow(dat)){
  dat[i,"slope"] <- coef(lm(val ~ date2, data = dat[(i-w):i,]))[2]
}

## Without a loop
dat$slope2 <- NA
dat[w:nrow(dat), "slope2"] <- sapply(w:nrow(dat), function(i) coef(lm(val ~ date2, data = dat[(i-w):i,]))[2])

Previous answer:
I'm assuming that if the change between a value and its previous value is greater than 0, there was an increase and if this happens 4 or more times within six consecutive values, your condition Groundwater_ST1 == TRUE:
library(dplyr)
library(RcppRoll)

dat %>%
  mutate(change = dat$val - lag(dat$val)) %>%
  mutate(incr = change > 0) %>%
  mutate(roll_sum = roll_sum(incr, 6, align = "right", fill = NA)) %>%
  mutate(Groundwater_ST1 = roll_sum >= 4)

result:
         date    val  change  incr roll_sum Groundwater_ST1
1  2013-08-02 0.1470      NA    NA       NA              NA
2  2014-06-13 0.0623 -0.0847 FALSE       NA              NA
3  2015-09-03 1.4600  1.3977  TRUE       NA              NA
4  2016-06-12 0.1810 -1.2790 FALSE       NA              NA
5  2016-09-27 0.0509 -0.1301 FALSE       NA              NA
6  2017-05-30 0.0801  0.0292  TRUE       NA              NA
7  2017-05-30 0.0801  0.0000 FALSE        2           FALSE
8  2017-09-14 0.0999  0.0198  TRUE        3           FALSE
9  2017-09-14 0.0980 -0.0019 FALSE        2           FALSE
10 2017-12-02 0.0820 -0.0160 FALSE        2           FALSE
11 2018-03-29 0.0698 -0.0122 FALSE        2           FALSE
12 2018-06-05 0.0884  0.0186  TRUE        2           FALSE
13 2018-10-19 0.1060  0.0176  TRUE        3           FALSE
14 2019-02-27 0.1010 -0.0050 FALSE        2           FALSE
15 2019-06-04 0.0984 -0.0026 FALSE        2           FALSE
16 2019-08-28 0.1050  0.0066  TRUE        3           FALSE
17 2019-10-22 0.1100  0.0050  TRUE        4            TRUE
18 2020-02-04 0.0980 -0.0120 FALSE        3           FALSE
19 2020-06-06 0.1000  0.0020  TRUE        3           FALSE
20 2020-08-26 0.1090  0.0090  TRUE        4            TRUE
21 2020-10-23 0.1050 -0.0040 FALSE        4            TRUE
22 2021-02-01 0.0662 -0.0388 FALSE        3           FALSE
23 2021-06-15 0.0944  0.0282  TRUE        3           FALSE
24 2021-08-03 0.1220  0.0276  TRUE        4            TRUE

data:
dat <- data.frame(
  date =c("2013-08-02", "2014-06-13", "2015-09-03", "2016-06-12", "2016-09-27", "2017-05-30", "2017-05-30", "2017-09-14", "2017-09-14", "2017-12-02", "2018-03-29", "2018-06-05", "2018-10-19", "2019-02-27", "2019-06-04", "2019-08-28", "2019-10-22", "2020-02-04", "2020-06-06", "2020-08-26", "2020-10-23", "2021-02-01", "2021-06-15", "2021-08-03"),
  val = c(0.1470, 0.0623, 1.4600, 0.1810, 0.0509, 0.0801, 0.0801, 0.0999, 0.0980, 0.0820, 0.0698, 0.0884, 0.1060, 0.1010, 0.0984, 0.1050, 0.1100, 0.0980, 0.1000, 0.1090, 0.1050, 0.0662, 0.0944, 0.1220))

Comment: Not quite. The trouble is that the increases need to be progressive but not necessarily consecutive. For example, Row 16 should be TRUE because rows 11, 12, 14, and 16 progressively increase. Row 20 should be false because even though there are technically four increases, there isn't any grouping of 4 points in that set of 6 that progressively increases.
Comment: Could you define a "progressive increase" based on the direction of the slope of a linear model fit with date and val?
Answer: EDIT: Tricky! Here's a brute force tidyverse answer. Unless the data is many millions of rows long, this should be very performant, since its vectorized. Additional gains could be had by porting into data.table or collapse to get better performance for large numbers of groups.
My approach was to identify every possible ascending subset of four points within a window of 6. There are 15 of these patterns:
library(tidyverse)
subsets_to_try <- combinat::combn(1:6, 4) %>%
  t() %>%
  as.data.frame() %>%
  mutate(combination = row_number(),
         combo_pattern = paste0(V1,V2,V3,V4, sep=""), .before = 1)

#   combination combo_pattern V1 V2 V3 V4
#1            1          1234  1  2  3  4
#2            2          1235  1  2  3  5
#3            3          1236  1  2  3  6
#4            4          1245  1  2  4  5
#5            5          1246  1  2  4  6
#6            6          1256  1  2  5  6
#7            7          1345  1  3  4  5
#8            8          1346  1  3  4  6
#9            9          1356  1  3  5  6
#10          10          1456  1  4  5  6
#11          11          2345  2  3  4  5
#12          12          2346  2  3  4  6
#13          13          2356  2  3  5  6
#14          14          2456  2  4  5  6
#15          15          3456  3  4  5  6

For use below, we can save a version that is reshaped into long form. This produces a table that is 60 rows long (15 combinations x 4 positions).
subsets_long <- subsets_to_try %>%
  pivot_longer(-c(combination, combo_pattern),
               names_to = "trend_num", names_prefix = "V",
               names_transform = as.integer,
               values_to = "pos_in_window")

Now for the brute forcing. We can make 60 copies of each row, corresponding to each of the four positions an observation might have within the 15 sequences. We can then join the data to subsets_long so that each observation will now be situated at a particular position in a particular sequence.
We can rearrange the data so that each observation is in order within its sequence. Grouping within each possible sequence end date and combination, we can filter for just the situations where there is a 4-element increasing trend.
incr_sequences <- GW %>%
  uncount(nrow(subsets_to_try), .id = "combination") %>%
  left_join(subsets_long) %>%
  mutate(SEQ_END = DATE_NUM - pos_in_window + 6) %>%
  arrange(SEQ_END, combination, pos_in_window) %>%
  group_by(SEQ_END, combination) %>%
  filter(sum(val > lag(val, default = -Inf)) == 4) %>%
  ungroup()

That's basically it. We might look at the identified sequences like this as a visual confirmation (I've filtered out the extreme point to clarify).
In case it's of any use downstream, I've preserved the observation that there are actually two ways to get a four-element ascending sequence ending row 16 or row 17, depending on whether they include 14 or 15. If you just need to know which windows have any working trend at all, you could look at incr_sequences %>% distinct(SEQ_END) to see it's just row 16 and 17 that work.
ggplot(incr_sequences, aes(DATE_NUM, val)) +
  geom_point(data = GW %>% filter(val < 0.5), color = "gray70") +
  geom_point() +
  facet_wrap(~interaction(SEQ_END, combo_pattern))

Original answer:
I think we could do this by counting the number of cumulative increases, then looking to see how the increases have changed over a window of 6 values.
library(dplyr)
GWSubsetData %>%
  mutate(increases = cumsum(REPORT_RESULT_VALUE > lag(REPORT_RESULT_VALUE, default = 0)),
         n_incr_last_6 = increases - lag(increases, 6, default = 0),
         flag = n_incr_last_6 >= 4)

   SAMPLE_DATE REPORT_RESULT_VALUE increases n_incr_last_6  flag
1   2013-08-02              0.1470         1             1 FALSE
2   2014-06-13              0.0623         1             1 FALSE
3   2015-09-03              1.4600         2             2 FALSE
4   2016-06-12              0.1810         2             2 FALSE
5   2016-09-27              0.0509         2             2 FALSE
6   2017-05-30              0.0801         3             3 FALSE
7   2017-05-30              0.0801         3             2 FALSE
8   2017-09-14              0.0999         4             3 FALSE
9   2017-09-14              0.0980         4             2 FALSE
10  2017-12-02              0.0820         4             2 FALSE
11  2018-03-29              0.0698         4             2 FALSE
12  2018-06-05              0.0884         5             2 FALSE
13  2018-10-19              0.1060         6             3 FALSE
14  2019-02-27              0.1010         6             2 FALSE
15  2019-06-04              0.0984         6             2 FALSE
16  2019-08-28              0.1050         7             3 FALSE
17  2019-10-22              0.1100         8             4  TRUE
18  2020-02-04              0.0980         8             3 FALSE
19  2020-06-06              0.1000         9             3 FALSE
20  2020-08-26              0.1090        10             4  TRUE
21  2020-10-23              0.1050        10             4  TRUE
22  2021-02-01              0.0662        10             3 FALSE
23  2021-06-15              0.0944        11             3 FALSE
24  2021-08-03              0.1220        12             4  TRUE

Base R equivalent:
GWSubsetData$incr = cumsum(c(1, diff(GWSubsetData$REPORT_RESULT_VALUE) > 0))
GWSubsetData$flag = (GWSubsetData$incr - lag(GWSubsetData$incr, 6, default = 0)) >= 4

Comment: This is still unclear to me, could you please add your desired output to the question?
Comment: This works in some cases but the trouble is that the increases need to be progressive but not consecutive. For example, Row 16 should be TRUE because rows 11, 12, 14, and 16 progressively increase. Row 20 should be false because even though there are technically four increases, there isn't any grouping of 4 points in that set of 6 that progressively increases.
Comment: Could you add more explanation about what a "progressive" increase means in this context?
Comment: I added an image of the two instances that I described. I need to find a way to look for 4 out of 6 samples that increase relative to one another (which is what your solution seems to be missing).
Comment: Interesting question! I've added a brute force answer, but I'd be very curious if there's a smarter way to do this w/o loops.
Comment: Very cool! I never would have would have come up with this. Thanks for the help!


Finding the $i$th power of $i$

Question: I have been trying to find the $i$th power of $i$ as a mental exercise, I have tried two approaches
For the first, using properties of exponents
$$i^i=i^{\sqrt{-1}}=i^{-1^{\frac{1}{2}}} \implies i^i=i^{-\frac{1}{2}}=\frac{1}{i^2}=-1$$
However this seems a bit, for a lack of a better word "bodgy".

The next method I used was rewriting into exponential form
$$z=i^i + 0 \implies \arg(z)=\tan^-{1}\left(\frac{1^i}{0}\right)=\frac{\pi}{2}$$
$$ \therefore i^i=e^{\frac{\pi}{2}i}$$
I am quite sure that none of these answers are correct, what are the flaws in my method and how should I go about actually computing this ?
Comment: You might like this [youtube video](https://www.youtube.com/watch?v=9tlHQOKMHGA) on the subject
Comment: [See also this post](https://math.stackexchange.com/q/838908/81360) (not quite the same question but obviously related)
Comment: For both of your approaches, I do not understand the $\implies$ step. It would be helpful if you tried to explain your thought process there
Comment: You've conflated $a^{b^c}$ with $a^{bc}$, which doesn't even work with $a=b=2,\,c=3$.
Comment: @Filthyscrub Thanks for clarifying. For the first step, as J.G. notes,
$$
i^{(-1)^{1/2}} \neq [i^{-1}]^{1/2}.
$$
For the second, it's not clear where $1^i$ came from
Comment: @BenGrossmann For the first step I applied the exponent property $(x^a)^b=x^{ab}$ and for the next I used the property that $tan(z)=\frac{y}{x}$ and since the coefficient of the imaginary part is $1$ and 1 raised to any power is 1 I got the argument as $\frac{\pi}{2}$
Answer: The typical approach to the problem is this:
$$
i = e^{\pi i /2} \implies
i^i = [e^{\pi i/2}]^i = e^{\pi i^2 /2} = e^{-\pi/2}.
$$
However, as I note in my post here, this actually leads to several (infinitely many) possible values for $i^i$. Interestingly, each of these possibilities is a positive real number.
Answer: Write
$$i^i = \left(e^{\log i}\right)^i = e^{i \log i}$$
and use the fact that
$$ \log w = \mathrm{ln}|w| + i \,\mathrm{arg}\,w$$
Note that $\mathrm{arg}\,w$ is a multivalued function.

Finding the $i$th power of $i$


Common neighbors and preferential attachment score matrixes using igraph for python

Question: Is there an efficient way to calculate the matrix score for common neighbors(CC) and preferential attachment(PA) in python? I'm using igraph to calculate score matrixes for other methods such as jaccard's coefficient (Graph.similarity_jaccard()), dice (Graph.similarity_dice) and adamic/adar (Graph.similarity_inverse_log_weighted()), but I haven't found any function to compute score matrixes for CC and PA.
Currently I'm doing:
#Preferential attachment score between nodes i and j in a graph g
len(g.neighbors(i))*len(g.neighbors(j))

#Common neighbors score between nodes i and j in a graph g
len(g.neighbors(i) and g.neighbors(j))

but I have to do this for all edges(i,j) in the network which in my case is really large.
If anyone knows any mathematical matrix operation that generates such score matrixes I'm looking for, I would appreciate as well.
Answer: There is no direct function for this in igraph. However, note that len(g.neighbors(i)) is simply the degree of vertex i, so you can call g.degree(), treat it as a 1D matrix using NumPy, then multiply it with its own transpose s.t. a column vector is multiplied by a row vector from the right. This gives you the preferential attachment score matrix:
from numpy import matrix
d = matrix(g.degree())
pref_score_matrix = d.T*d

The common neighbors score is trickier using matrix notation, and I wouldn't operate with matrices anyway if your graph is really large (even with only 2000 vertices, you would have a matrix with 4 million elements). I would simply cache set representations of g.neighbors(i) for all possible values in a list, and then use that to calculate the common neighbor scores as sets can be intersected efficiently:
neisets = [set(g.neighbors(i)) for i in xrange(g.vcount())]
for v1, v2 in g.get_edgelist():
    common_neis = neisets[v1].intersection(neisets[v2])
    print "%d --> %d: %d" % (v1, v2, len(common_neis))

If you really want to stick to matrices, you can construct an n by n matrix consisting of zeros only using the zeros function from NumPy and then loop over the vertices once. Get all the neighbors of vertex v and note that any pair of neighbors of v have one neighbor in common: v itself:
from itertools import combinations
from numpy import zeros

n = g.vcount()
common_neis = zeros(n, n)
for v in xrange(g.vcount()):
    neis = g.neighbors(v)
    for u, w in combinations(neis, 2):
        # v is a common neighbor of u and w
        common_neis[u, w] += 1
        common_neis[w, u] += 1

Comment: Thanks, that worked for me. Taking advantage of the subject, is there an efficient way to calculate how many paths of lenght "l" there are between two nodes in a graph? I know that Mˆl (where M is the adjacency matrix) will give me that answer, but I just need to know that value for some nodes, so there's no need to operate over the whole matrix. Thanks in advance.
Comment: Not particularly efficient, but if you have a large graph and really need it only for a few pairs, then simply finding all the paths between two nodes should work: http://stackoverflow.com/questions/3971876/all-possible-paths-from-one-node-to-another-in-a-directed-tree-igraph


How to calculate area of quarter of circle ring in first quadrant?

Question: G is quarter of circle ring in frist quadrant: $G=\left\{ \begin{pmatrix} x \\ y \end{pmatrix} x\in \mathbb R ^2 : x\geq 0, y\geq 0, 9\leq x^2+y^2\leq 49\right\} $
Calculate its area: $\int_{G}^{} \! 1 \, d(x,y) $.
Can someone help me to determine the boundaries of the integral?
Answer: The easiest way to approach this would be to use polar coordinates.  We have $3 \leq r \leq 7$ and $0 \leq \theta \leq \pi/2$.  The integrand changes$^\dagger$ from $1 \ dx \ dy$ into $r \ dr \ d \theta$, so:
$$\int_G 1 \ dx \ dy = \int_0^{\pi/2} \int_3^7 r \ dr \ d\theta$$

$^\dagger$To elaborate, if changing to polar coordinates is unfamiliar, the substitution is $(x, y) = f(r, \theta) = \langle r \cos(\theta), r\sin(\theta) \rangle$.  If we proceed according to this, the differential $dx \ dy$ transforms as:
$$dx \ dy = \det \left( \left[
  \begin{array}{ c c }
     \frac{\partial f_1}{\partial r} & \frac{\partial f_1}{\partial \theta} \\
     \frac{\partial f_2}{\partial r} & \frac{\partial f_2}{\partial \theta}
  \end{array} \right] \right) dr \ d \theta = \det \left( \left[
  \begin{array}{ c c }
     \cos(\theta) & -r \sin(\theta) \\
     \sin(\theta) & r\cos(\theta)
  \end{array} \right] \right) dr \ d \theta \\ = \ r(\cos^2(\theta) + \sin^2(\theta)) \ dr \ d \theta \ = \fbox{$ \ r \ dr \ d \theta \ $}
$$
Comment: Clearly, there is an easier way : calculating $(49\pi-9\pi)/4$.
Comment: LOL, true @Levent.  I suppose I should've written "the easiest way to approach this *with integration*.
Answer: Kaj Hansen addresses the simpler option. If you insist on using Cartesian coordinates, then
$$\int_G1\ \mathrm{d}x\ \mathrm{d}y=\int_0^7\int_{\sqrt{9-x^2}}^\sqrt{49-x^2}1\ \mathrm{d}y\ \mathrm{d}x.$$
Here you do not have to know about the Jacobian determinant. However, you should be pretty good at manual integration involving roots.
Limits on $x$
Since we are in the first quadrant and the larger circle's $x$-intercept at $x=7$, then
$$0\le x\le7.$$
Limits on $y$
Solve the equations $$x^2+y^2=9, \ \ \ \ x^2+y^2=49$$
for $y$, taking into account that $y\ge0$.
Also note that $G$ is called a quarter-annulus (more familiar than quarter-circle ring).
Comment: Glad someone added the Cartesian route (+1)!  Pedagogically, it's good to see as many methods as possible.
Comment: @KajHansen Totally agree. Then one can view the change of coordinates as a natural development, and thus proceed with the method you described. Which is, objectively speaking, the way to go.


Playing Guitar Tabs on Piano

Question: I was wondering if anyone could possibly explain how I could somehow translate a guitar tab into notes on piano. Is there some sort of algorithm I can apply?
 ex:
<code>E|--------------------------------------------------------------9/12--|

B|---------------------7h9--12--12--12--10--10\9--9--9\7--7--7--------|

G|--------------6h8/9-------------------------------------------------|

D|--6h7--6h7/9--------------------------------------------------------|

A|--------------------------------------------------------------------|

E|--------------------------------------------------------------------|
</code>
Comment: This approach embraces all of the worst aspects on TAB, while providing none of the benefits! I would learn standard music notation, or just play by ear.
Comment: If you are asking for an algorithm to do this, you might try different SE sites, for which your question would be better suited.
Answer: It's actually quite simple: the strings (EADGBE) each correspond to a certain (white) key of the piano keyboard. For example, the E is the white key at the end of a group of two black keys (see the blue circle below). Which E key (i.e. which octave) can usually be heard, but on most pianos the second E from the bottom corresponds to the lowest E string of a guitar.
Then, the number in the tab is just the number of half-tones (white and black keys) you need to go up. E.g. the first note in your tab is a 6 on the D string. If you go up 6 keys from D, you end up on the black G#/Ab key. See the red marks below.

Playing Guitar Tabs on Piano


Three digit number divided by 5, 6 and 9

Question: A three digit number x when divided by 5, 6 and 9 gives a remainder of 1. Which is the sum of all x-es with that property?
Answer: So, $x$ will be of the form $1+[5,6,9]t=90t+1$ where $t$ is an integer
We need $$100\le90t+1\le999$$
Answer: Are you familiar with the Chinese Remainder Theorem?[1]
Well, I won't use it.
$X = 5M + 1 = 6N + 1 = 9K+1$
$5M = 6N =9K=X-1 = Y$ so
Note: This means $5|Y, 6|Y, 9|Y$ and so the least common multiple of $5,6,9= 90|Y$.
So $X = 90m + 1$.
If $X$ has $3$ digits then $100 \le 90 m + 1 < 1000$
So $99 \le 90m < 999$
$11 \le 10m < 111$ so $2 \le m \le 11$.
and the numbers are
$181,271,361,..., 721, 811, 901,991$.
If we add them together we have:
$181 + .... + 901 + 991 = $
$180 + ....  + 900 + 990 + (1+1+1.....+1)=$
$90(2+3+.... + 11) + 10 =$
$90(\frac {11*12}2-1) + 10=$
$5860$
......
[1] The CRT will let you know that there is precisely one unique number between $0$ and the least common multiple of $5,6,9=90$ with remainder $1$ for $5,6,9$ and therefore that number is $1$, and therefore all possible numbers with remainder $1$ for $5,6,9$ will be in the form $90m + 1$.


Is it possible to find a $k$-algebra endomorphism which fixes an arbitrary element?

Question: Let $R$ be a commutative $k$-algebra, $k$ is a field of characteristic zero.
Take an arbitrary $w \in R-k$.

Is it possible to define/find a $k$-algebra endomorphism of $R$, call it $f$, such that $f(w)=w$ (and $f \neq 1$)?

I guess this question is too general, so examples of algebras $R$ with this property are ok as a partial answer (perhaps this is still too general?).
Remark:
A non-answer: $R=\mathbb{C}[x,y]$, $w=x^2+y^2$. It is clear that the exchange involution $\alpha: (x,y) \mapsto (y,x)$ fixes $w$. But in my question it is not known if $w$ is symmetric with respect to some involution on $R$, if $R$ has an involution (in my question $w$ is arbitrary, and we wish to find an endomorphism of $R$ which fixes $w$).
Actually, I wish to find an answer to the (probably) more difficult following  question:

Is it possible to define/find a $k$-algebra endomorphism of $R$, call it $f$, such that $f(w)=w$ and $f(u) \neq u$, where $u \in R-k$ is another arbitrary element.

Thank you very much!
Comment: Do you want $f$ to not be the identity?
Comment: Yes. I will add this to my question. Thanks.
Comment: What does $R-\{k\}$ mean when $k$ is the field?
Comment: Actually, I wish to find such $f$ with a more restricted property, namely $f$ that satisfies the following two conditions: (1) $f(w)=w$. (2) $f(u) \neq u$, where $w,u \in R-k$ are two arbitrary elements.
Comment: @Chickenmancer, you are right. My mistake. I meant $R-k$.
Answer: Certainly not in general.  For instance, if $w$ generates $R$ as a $k$-algebra, then if $f(w)=w$ that automatically means $f$ is the identity.  Slightly less trivially, it is entirely possible that $R$ has no endomorphisms besides the identity, even if $w$ does not generate $R$.  For instance, the ring $\mathbb{R}$ has no endomorphisms besides the identity so you could take $R=\mathbb{R}$ and $k$ to be any subfield of $\mathbb{R}$.  More generally, I don't know how to make this precise, but I would suspect that a "random finitely generated $k$-algebra" will typically have no endomorphisms besides the identity.
Here's an example of a nontrivial finitely generated $k$-algebra for which it is true.  Let $V$ be any $k$-vector space of (finite) dimension $>1$ and let $R=k\oplus V$ with multiplication defined by $(a,x)\cdot(b,y)=(ab,ay+bx)$.  Note that any linear map $g:V\to V$ gives an endomorphism $f:R\to R$ defined by $f(a,x)=(a,g(x))$.  So in particular, given $w=(a,x)\in R$, you can take any linear map $g:V\to V$ that fixes $x$ but is not the identity (this is possible since $\dim V>1$), and this will give an endomorphism $f:R\to R$ which fixes $w$ and is not the identity.
Another case where it is true is if $R$ is an infinite-degree Galois extension field of $k$.  Then for any $w\in R$, $k(w)$ is finite-dimensional and hence a proper subfield of $R$, and so $R$ has many automorphisms over $k(w)$.
As for your final question, the answer is trivially no since if $f(w)=w$ and $u$ is in the subalgebra generated by $w$, then $f(u)=u$ as well.
Comment: I do not mind that $f$ will be an automorphism. ($\mathbb{C}[x,y]$ has infinitely many automorphisms: the affines and triangulars).
Comment: I don't know what your comment has to do with what I wrote...
Comment: a "random finitely generated $k$-algebra" will typically have no endomorphisms besides the identity.
Comment: (In an example I had in mind, $u \in R-k[w]$). It's nice that you have noticed the special case $u \in k[w]$, which should be excluded.


html dropdown text color change on selection

Question: I have a html dropdown with different color text
<select class="form-control">
  <option value="">-- Select --</option>
  <option style="color: Black" value="Black">Black</option>
  <option style="color: Blue" value="Blue">Blue</option>
  <option style="color: Red" value="Red">Red</option>
  <option style="color: Orange" value="Orange">Orange</option>
  <option style="color: Brown" value="Brown">Brown</option>
  <option style="color: Green" value="Green">Green</option>
  <option style="color: Purple" value="Purple">Purple</option>
  <option style="color: Cyan" value="Cyan">Cyan</option>
  <option style="color: Magenta" value="Magenta">Magenta</option>
</select>

Now You can all options are showing different colors but when I select any option selected value color has no color.. its default text color.
I want to keep same color
JSFiddle :https://jsfiddle.net/mahajan344/ztxws0n7/
Answer: This is because your HTML is stating only to style the option elements, not the overall select. Some browsers might automatically interpret this for you, but without it you cannot get this to work with just styling.
It can be overcome by some simple javascript which applies the selected option's style attribute on change of the select.
Below example uses jQuery:
$(function(){

$("select").change(function(){
    var $selectedOption = $(this).find("option:selected");
    $(this).removeAttr("style").attr("style", $selectedOption.attr("style"));
});

});

https://jsfiddle.net/ztxws0n7/2/
Comment: I find it strange how similar the *other* answer is to yours, they seem to have the same 'lack of indent' within the function. They also remove the style before setting the new one, which I don't believe is a very common technique at all... at least they changed some variable names :/
Comment: @musefan Haha I hadn't spotted that. Clearly this has become a new coding convention.
Answer: Try It Once you can find selected option style and remove previous styles in script then its works fine

$(function(){

$("#selectbox").change(function(){
    var $yourchoice = $('#selectbox').find("option:selected");
    $(this).removeAttr("style").attr("style", $yourchoice.attr("style"));
});

});

<select class="form-control" id="selectbox">
  <option value="">-- Select --</option>
  <option style="color: Black" value="Black">Black</option>
  <option style="color: Blue" value="Blue">Blue</option>
  <option style="color: Red" value="Red">Red</option>
  <option style="color: Orange" value="Orange">Orange</option>
  <option style="color: Brown" value="Brown">Brown</option>
  <option style="color: Green" value="Green">Green</option>
  <option style="color: Purple" value="Purple">Purple</option>
  <option style="color: Cyan" value="Cyan">Cyan</option>
  <option style="color: Magenta" value="Magenta">Magenta</option>
</select>
Comment: I'm just going to say this as bluntly as possible... did you copy the accepted answer?


How to place a div at top center position of a page using CSS?

Question: I am not able to place a div at the top center of a page. I am using CSS for this but nothing works here.

#loader {
  position: relative;
  margin: 0 auto;
  text-align: left;
  clear: left;
  height: auto;
  z-index: 0;
}

I had already given 2 hours on this but still not able to figure it out.
Answer: First remove display:none; from your div with id=loader & add text-align:center; to it
#loader{
    position:relative;
    margin:0 auto;
    clear:left;
    height:auto;
    z-index: 0;
    text-align:center;/* Add This*/
}

Working demo: http://jsfiddle.net/surendraVsingh/FCEhD/
Answer: You need a specific width for the div, otherwise it will use all the available with, and the margin has no effect. Example:
#loader{
  margin: 0 auto;
  width: 200px;
}

Answer: You have to specify width property if you want to use margin: 0 auto;.
If you don't know width just add display: table property and it will works.
#out {
   display: table;
   margin: 0 auto;
}

Answer: In the above example everyone is assuming he wants 'full width'.
This works within an absolute full width / height parent container but constraints the width automatically to only the content width
Parent Container
display: block;
position: absolute;
clear: both;
width: 100%;
height: 100%;
border: solid thin red;

Child Container
display: block;
position: absolute;
top:0;
transform: translateX(50%);
padding-left:.5em;
padding-right:.5em;
margin-left:auto;
margin-right:auto;
right:50%;

Answer: #loader{
   margin: 0px auto;
   width: 100px;
}

Answer:

press ctrl+space am not sure about the syntax
and in css margin and padding has to set 0
Answer: #loader
{
    position: absolute;
    width: xxpx;
    height: xxpx;
    margin:0px auto ;

}

or u can use
margin-left:xx;  and `margin-top:0px;`

Answer: #loader {
    display: none;
    clear: both;
    margin: 0px auto;
    width: [a number] px;
}

of course you should also note the browser cache. clear cache of your browser and test again


Maximum number of edges in bipartite $C_4$-free graph

Question:
Show that a $C_4$-free bipartite graph between two vertex parts of
sizes of $a$ and $b$ has at most $ab^{1/2}+b$ edges.

Let's introduce some basic notations: $\binom{X}{2}$ means the family of 2-subsets of $X$ and $v\sim A$ means that $v$ is adjacent to any vertex in $A$.
Proof: Suppose that $G=G[X,Y]$ is a bipartite graph with vertex parts $X$ and $Y$ such that $|X|=a,\ |Y|=b$ and with $m$ edges.

Assume that $a\geq b$ and consider $S=\{(A,v)\in \binom{X}{2}\times Y: v\sim A\}.$ One can obtain the following upper bound for the size of $S$: $$|S|=\sum \limits_{A\in \binom{X}{2}} \sum_{\substack{v\in Y\\v\sim A}} 1 =\sum \limits_{A\in \binom{X}{2}} |\{v\in Y: v\sim A\}|\leq \sum \limits_{A\in \binom{X}{2}}1=\binom{a}{2}.$$ In the same way one can obtain lower bound for the size of $S$:
$$|S|=\sum \limits_{v\in Y} \sum_{\substack{A\in \binom{X}2{} \\v\sim A}} 1=\sum \limits_{v\in Y} |\{A\in \binom{X}{2}: v\sim A\}|\geq \sum \limits_{v\in Y}|\{A\subset N(v): |A|=2 \}|=\sum \limits_{v\in Y}\binom{\deg (v)}{2}.$$
Combining bounds, we obtain: $$\sum \limits_{v\in Y} \binom{\deg(v)}{2}\leq \binom{a}{2} \quad \Leftrightarrow \quad \sum \limits_{v\in Y}\deg(v)^2-\sum\limits_{v\in Y}\deg(v)\leq a^2-a.$$ Since $\sum \limits_{v\in Y}\deg(v)=m$ and $\sum \limits_{v\in Y}\deg(v)^2\geq \frac{m^2}{b}$ by Cauchy-Schwarz inequality, we obtain: $$\frac{m^2}{b}-m\leq a^2-a \Leftrightarrow m^2-bm+(ab-a^2b)\leq 0 \Rightarrow$$ $$m\leq \frac{b+\sqrt{b^2-4ab+4ba^2}}{2}\leq \frac{b+\sqrt{b^2+4ba^2}}{2}\leq \frac{b}{2}+\frac{\sqrt{5}}{2}a\sqrt{b}.$$

If $b\geq a$, then performing analogous reasoning one can show that $m\leq \frac{a}{2}+\frac{\sqrt{5}}{2}b\sqrt{a}.$

Combining both cases we obtain that $e(G)\leq \min \left(\frac{b}{2}+\frac{\sqrt{5}}{2}a\sqrt{b}, \frac{a}{2}+\frac{\sqrt{5}}{2}b\sqrt{a} \right).$
And you see that what I've obtained is very close to the answer of the problem but I do not see what should I change in order in my reasoning.
If someone can show it, then it would be great!
Comment: Wouldn't that be less than the minimum, not the maximum? If $e(G) \leq m$ and $e(G) \leq n$, then $e(G) \leq \min\{m, n\}$. (I mean, the maximum isn't *wrong*; it's just not as tight.)
Comment: @BrianTung, of course! Edited! Thanks!
Answer: Technically, OP has correctly solved the problem.
I basically repeated what the OP has done.

We can use similar reasoning used to show a bound of the number of edges for general $C_4$-free graphs using double counting "cherries" ($K_{1, 2}$).
Proof. We count cherries with the middle point on the right ($b$ nodes). We have the number of cherries =
\begin{align}
\sum_{v \in Y} \binom{d_v}{2} \geq b \binom{\frac{1}{b} \sum_{v \in Y} d_v}{2} = b \binom{m/b}{2} = m(m-b)/2b,
\end{align}
where we use Jenson's for the first inequality.
On the other hand, we have the number $\leq \binom{a}{2}$. Otherwise, by Pigeonhole we will have a $K_{2,2} = C_4$.
Therefore, we have $m(m-b)/2b \leq a(a-1)/2$, which is
$$m^2 - bm -ab(a-1) \leq 0.$$
The axis of symmetry is $m = b/2$.
Since $ab^{1/2} +b > b/2$, when $m > ab^{1/2} + b$,
$$m^2 - bm -ab(a-1) > (ab^{1/2} + b)^2 - b * (ab^{1/2} + b) - ab(a-1) > 0,$$
completing the proof.
Comment: now that makes sense. It was really easy and I was so close to solve but ..) I'll accept your answer! Thank you! +1
Comment: Why do you think that I did not used $C_4$-freeness? I've used it here $|\{v\in Y: v\sim A\}|\leq 1$.
Comment: I see. Sorry  for that!
Comment: I think actually I just repeated what you have done
Comment: I did not read your answer in detail but what is the main difference between our approaches? I also counted the number of "cherries", i.e. $K_{1,2}$ copies in $G$.
Comment: I think the only difference is the last step. We do need to exactly solve the quadratic equation but we can just show that when $m$ exceeds the bound the inequality does not hold.
Comment: your last step of the proof is a bit confusing. What are you doing there?
Comment: I am showing that when $m > ab^{1/2} +b$, then $m^2 - bm - ab(a-1) > 0$ and thus the ineq does not hold
Comment: Sorry that I did not read your trial carefully... I could have made it much clearer and more concise.


Comparação com banco de dados Postgresql utilizando C#

Question: Criei uma aplicação de Login, fiz a conexão com o banco de dados, consigo obter o usuário e senha cadastrado no meu banco. Porém o que eu quero, é, se o usuário tentar mais de três vezes entrar e tiver algo errado, bloquear os textbox e button.
Teria que fazer a comparação com o usuário cadastrado no banco.
Segue o código até onde eu fiz.
<code>    {
        bool blnFound = false;
        NpgsqlConnection conn = new NpgsqlConnection("Server=localhost;Port=5432;User Id=postgres;Password=****;Database=HHH");
        conn.Open(); //abrir conexão
        NpgsqlCommand cmd = new NpgsqlCommand("select * from login where nome = '" + txtUserName.Text + "' and senha = '" + txtSenha.Text + "'",conn);
        NpgsqlDataReader dr = cmd.ExecuteReader();

        if(dr.Read())
        {
            blnFound = true;
            Principal pc = new Principal();
            pc.Show();
            this.Hide();
        }
        if(blnFound==false)
            MessageBox.Show("Nome ou senha estão incorrentos!","Erro",MessageBoxButtons.OK,MessageBoxIcon.Exclamation);
        dr.Close();
        conn.Close();
    }
</code>
Quando o usuário, errar sua senha ou seu usuario, os textbox ficaria desabilitado.
Mais ou menos assim: txtUserName.Text.Enabled = false
Comment: Tem claro, basta você editar a pergunta, postar tudo o que já fez e dizer onde você está enroscado. Talvez tenha um jeito até melhor do que está fazendo mas precisamos de mais detalhes.
Answer: Você pode adicionar uma variável no código, caso não queira adicionar uma coluna no banco para verificar quantas vezes foi tentado o login.
<code>int tentativas = 0;

{
    NpgsqlConnection conn = new NpgsqlConnection("Server=localhost;Port=5432;User Id=postgres;Password=****;Database=HHH");
    conn.Open();
    NpgsqlCommand cmd = new NpgsqlCommand("select * from login where nome = '" + txtUserName.Text + "' and senha = '" + txtSenha.Text + "'",conn);
    NpgsqlDataReader dr = cmd.ExecuteReader();

    if(dr.Read())
    {
        Principal pc = new Principal();
        pc.Show();
        this.Hide();
    }
    else
    {
        MessageBox.Show("Nome ou senha estão incorretos!", "Erro", MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
        tentativas++;
    }

    if (tentativas > 2)
    {
        MessageBox.Show("Login bloqueado")
        button1.Enabled = False;
    }

    dr.Close();
    conn.Close();
}
</code>
Answer: Procure fazer uma função para esta rotina.
Crie um laço para testar o usuário e a senha na quantidade de vezes que deseja (ou definiu para o sistema), assim você conseguirá fazer a validação, caso a validação ultrapasse a quantidade definida, pode travar o sistema, apresentar uma tela do sistema com erro, ou uma mensagem, e etc.


Matrix pointwise operation using a function

Question: How do I change the value of each element in a Matrix that involves a function call. Something like
matrix[i,j] = funct(matrix[i,j]*2)-1;
Answer: You can use MapInplace, for example:
matrix.MapInplace(Math.Sin);
matrix.MapInplace(x => Math.Sin(x-2));

Or as in your example
matrix.MapInplace(x => funct(x*2)-1);


O que é o problema das queries N+1?

Question: Sempre que trabalhamos com algum ORM, é comum cair no problema das queries N+1. É algo sobre performance, chamado até de antipattern.
Mas o que é realmente esse problema, por que ele acontece, quais suas principais causas e como, na teoria, resolvê-los?
Já ouvi também que para resolver, é só praticar o eager loading. Mas até que ponto ele é benéfico e capaz de resolver esse problema?
Comment: Você faz uma consulta para pegar uma lista. Então para cada elemento dessa lista, você faz 1 consulta. Ou seja, N+1 consultas. Quando me liberar de umas revisões de merge eu vou atrás de responder, caso não o tenham feito
Comment: O Gavin King, criador do Hibernate disse uma vez no blog que quando ele é contratado p/ fazer consultoria sobre ORM, etc muitas vezes tem que dizer "Neste caso não usamos ORM, use uma query". ORM é ótimo mão não serve p/ resolver 100% dos casos. Era uma vez um relatório que deveria mostrar apenas o total de consultas no mês feitas por uma rede de clínicas. O ORM tinha que carregas as clínias, os médicos, os pacientes, as consultas... acabava tentando carregar todos os registros de todas as tabelas e depois de 20 min dava timeout. Uma query fazia a mesma coisa em uma fração de segundo
Answer:
Mas o que é realmente esse problema

A melhor forma de explicar este problema é com um exemplo.
Imagine que você tem uma tabela Pessoa e uma tabela Endereco. Cada pessoa tem vários endereços, consolidando uma relação de um para muitos (1-N).
E agora você deseja pegar os endereços de várias pessoas. Normalmente, vemos a seguinte consulta utilizando o ORM de sua preferência (vou usar a notação JPQL do JPA):
public List<Pessoa> consultarPessoas() {
    String jpql = "select * from Pessoa";
    return em.createQuery(jpql).getResultList();
}

E, em seguida, você itera por cada Pessoa para pegar os seus endereços:
List<Pessoa> pessoas = consultarPessoas():
for (Pessoa pessoa : pessoas) {
    List<Endereco> enderecos = pessoa.getEnderecos();
}

Imaginando um LAZY entre pessoa e endereços, teremos o seguinte SQL para cada pessoa ao chamar o método pessoa.getEnderecos():
SELECT * from Endereco where pessoa_id = :id;

, por que ele acontece,

O problema ocorre porque para pegar os endereços das pessoas você pega primeiro a pessoa e depois busca os endereços de cada. Imaginando que a consulta anterior nos retornou 5 pessoas, a quantidade de SQLs gerados será algo assim:
 SELECT * from pessoa
 SELECT * from endereco where pessoa_id = 1;
 SELECT * from endereco where pessoa_id = 2;
 SELECT * from endereco where pessoa_id = 3;
 SELECT * from endereco where pessoa_id = 4;
 SELECT * from endereco where pessoa_id = 5;

Ou seja, 1 select de pessoa com N select para endereços, o famoso N + 1 .

quais suas principais causas

Normalmente ela é causada pelo uso inadequado dos ORMs. É preciso entender o que o ORM faz por trás dos bastidores. Embora eles estejam aí para facilitar nossa vida, eles precisam ser usados com sabedoria. Por serem muito permissivos de forma geral, resultados inesperados podem ser causados no mal uso da ferramenta.

e como, na teoria, resolvê-los?

No exemplo que dei anteriormente, seu objetivo era pegar os endereços de várias pessoas. Se forem os endereços de todas as pessoas do banco de dados, você precisa apenas fazer:
 SELECT * from Endereco

Mas se quiser aplicar um filtro para trazer aquelas 5 pessoas, isto pode ser feito evitando aquelas várias consultas com um JPQL diferente, partindo da tabela Endereco também:
 SELECT * from Endereco where pessoa_id IN (1,2,3,4,5);

Já ouvi também que para resolver, é só praticar o eager loading. Mas até que ponto ele é benéfico e capaz de resolver esse problema?

O EAGER loading é uma alternativa, pois o SQL gerado seria algo assim:
select p.id, p.nome, end.id, end.rua, end.pessoa_id from pessoa p JOIN endereco end ON end.pessoa_id = p.id

Contudo, o EAGER é um problema se adicionado entre o relacionamento de Pessoa e Endereço no seu ORM, pois toda vez que você buscar uma pessoa, os endereços também virão juntos. Acredite, você não quer isto como comportamento padrão do seu sistema. Os principais problemas de performance que vi em aplicações que envolviam o uso de algum ORM eram causados por isto.
Alguns ORMs tem a alternativa de usar o FETCH opcionalmente em uma consulta, assim você pode "ligar" o EAGER quando quiser. No JPQL, ficaria assim:
SELECT * from pessoa JOIN FETCH pessoa.enderecos

Resultando no mesmo SQL que citei anteriormente.
Porém, existe uma séria limitação com FETCH e EAGER se você tentar aplicar algum tipo de paginação na consulta. Usando EAGER ou FETCH isto não é possível de ser aplicado no próprio SQL gerado, e para ter uma consulta com este mesmo efeito (trazer pessoas e endereços na mesma consulta) você precisará apelar uma consulta nativa e/ou usar funções nativas (como o DENSE_RANK) do banco de dados.
Comment: Excelente explicação.
Answer: O problema não é exclusivo de ORMs, embora muitos o acham porque é um problema comum, mas não inerente a eles. E talvez não acreditem na capacidade das pessoas fazerem errado manualmente :).
No ORM aparece mais porque uma implementação ingênua forcará o problema ocorrer sempre.
Alias não é um problema só do ORM em si, mas de modelar objetos com dados relacionados. Ou usa um banco de dados com um modelo não relacional, que tem lá seus problemas, ou adota-se o modelo relacional na aplicação.
Ainda assim dá pra fazer criar algum problema quando mistura os dois modelos.
O problema é comum quando tem um objeto e outros N relacionados, daí o nome N + 1 que é o "pai" desses N. O problema fica claro quando a consulta pega o dado principal, vamos dizer que seja uma nota fiscal, e depois vai pegar as linhas de itens que compõe a nota. Ficar indo buscar dados individualmente no banco de dados pode se tornar um custo muito alto, ainda mais na arquitetura mal pensada que muitas pessoas fazem (alguns por necessidade).
Geralmente pelo menos o 1 + 1 é necessário, que é um falha da comunicação dos modelos relacionais (não do modelo, mas sim da forma com eles se comunicam nas implementações atuais, o que considero um erro, e em vez de consertarem isso, criaram outro modelo pior, é a história da nossa área, resolvem um problema com outro problema, mas nada que outro problema não possa resolver este também). Tem como evitar um pouco isso mas com baixa eficiência.
Alias, é por isso mesmo que alguns pessoas gostam de usar um DB NoSQL como proxy do relacional. Novamente, aumenta-se a complexidade da solução porque o ferramenta tem problemas fáceis de resolver, mas ninguém o faz.
Mas nesse caso se o tamanho é grande não será um grande problema.
O problema do eager loading é que pode trazer informações que sequer vai usar. Mas depende muito do problema, tem casos que isso é raro acontecer, tem outros que mesmo que aconteça não faz nem cócegas, e em muitos casos o fato de vir mais do que precisa gera um overhead tão pequeno que uma simples consulta extra já será pior, ou seja um 1 + 2 já pode ser pior. Imagine ler uma única nota fiscal e ele trazer todas as linhas de todas notas fiscais para evitar o N + 1, desperdício total.
Esse é o problema de soluções automatizadas ou de programadores que não entendem o que estão fazendo e adotam soluções de forma automática. A solução real é entender o que ocorrerá naquele caso e decidir o que é mais interessante. Mesmo manualmente é complicado atender todos os casos, depende da consulta. O ORM pode ter um mecanismo que tenta "adivinhar" qual é a melhor estratégia.
Em muitos casos traz-se muita repetição de informação pela maneira como se convencionou trabalhar com dados tabulares, geralmente usa-se o JOIN.
Na maioria das vezes trazer tudo de uma vez costuma ser mais interessante que trazer um a um.
Por falta de uma solução melhor seria algo como:
SELECT * FROM Nf
SELECT * FROM NfItem

Se tiver 1000 notas e em média exatamente 10 itens por nota, haverá 11 mil linhas ao todo, com 2 consultas, uma grande e outra enorme.
Em oposição da forma N+1:
SELECT * FROM Nf
SELECT * FROM NfItem WHERE NfNumero == Nf.Numero
SELECT * FROM NfItem WHERE NfNumero == Nf.Numero
SELECT * FROM NfItem WHERE NfNumero == Nf.Numero
SELECT * FROM NfItem WHERE NfNumero == Nf.Numero
.
.
.
Tantos quantos forem a quantidade de notas fiscais existentes.

Coloquei no GitHub para referência futura.
Aqui também terá 11 mil linhas, mas com 1000 consultas pequenas e 1 grande.
O código é bem abstrato, só para ilustrar.
Experimente fritar um filete de batata de cada vez e um monte de filete de uma vez. O primeiro método termina rápido individualmente, mas o todo fica trágico. O segundo método demora mais, mas quando terminar está tudo pronto. Só é um problema se descobrir que tinha vendido só 3 filetes, e fritou o pacote inteiro.
Comment: Conheço um sujeito que vende filetes de batata fritados individualmente, em pacotes de 50. Ele circula por aí num Fiat 147.
Comment: Só funciona. E olhe lá. A maioria das batatas já chega fria na hora de comer.
Comment: @bfavaretto e está certo? Ou só funciona?
Comment: Eu já vi frameworks resolvendo assim: `SELECT * FROM table`, `SELECT * FROM related_table WHERE table_id IN(?, ?, ?, ?)`

O que é o problema das queries N+1?


How prove this polynomial $p(x)$ is deg greater than $n-1$

Question: Question:

Let $P(x)$ be a polynomial satisfying $$P(k)=\cos{\dfrac{2k\pi}{n}},k=1,2,\cdots,n$$
Show that
$$\deg{P(x)}\ge n-1$$

I want to consider
$$P(m+1)-P(m)=\cos{\dfrac{2(m+1)\pi}{n}}-\cos{\dfrac{2m\pi}{n}}=2\sin{\dfrac{\pi}{n}}\sin{\dfrac{2m+1\pi}{n}}$$
Maybe this can use   mean value theorem for divided differences  Or can we also prove
$$P^{(n-1)}(x)\neq 0?$$ But I can't figure out how. Thank you.
Comment: Maybe [this](http://en.wikipedia.org/wiki/Lagrange_polynomial) will help.
Answer: Let $\Delta$ be the finite difference operator, $\Delta f(x)=f(x+1)-f(x)$.
Then, as you've shown,
$$\Delta P(m) = -2 \sin \frac{\pi}{n}\sin \frac{(2m+1)\pi}{n}$$
whenever $1\leq m \leq n-1$.
Similarly, we have
$$
\Delta^2P(m)=-2\sin\frac{\pi}{n}\left(\sin\frac{(2m+3)\pi}{n}-\sin\frac{(2m+1)\pi}{n}\right)=-2\sin\frac{\pi}{n}\left(2\sin\frac{\pi}{n}\cos\frac{(2m+2)\pi}{n}\right)
$$
for $1 \leq m \leq n-2$;
$$
\Delta^3P(m)=-4\sin^2\frac{\pi}{n}\left(\cos\frac{(2m+4)\pi}{n}-\cos\frac{(2m+2)\pi}{n}\right)=8\sin^3\frac{\pi}{n}\sin\frac{(2m+3)\pi}{n}
$$
for $1 \leq m \leq n-3$; and so on.
Continuing in this manner, it is convenient to define
$$
\operatorname{trig}_k m=\begin{cases}
\cos m && k\text{ even}\\
\sin m && k\text{ odd}
\end{cases}$$
Then it follows by induction that
$$
\Delta^k(m)=(-1)^{\lceil k/2 \rceil}2^k\sin^k\frac{\pi}{n}\operatorname{trig}_k \frac{(2m+k)\pi}{n}
$$
for $1 \leq m \leq n-k$.
In particular,
\begin{eqnarray}
\Delta^{n-1}(1)&=&(-1)^{\lceil (n-1)/2 \rceil}2^{n-1}\sin^{n-1}\frac{\pi}{n}\operatorname{trig}_{n-1}\frac{(n+1)\pi}{n}\\
&=&(-1)^{\lceil (n+1)/2 \rceil}2^{n-1}\sin^{n-1}\frac{\pi}{n}\operatorname{trig}_{n-1}\frac{\pi}{n}
\end{eqnarray}
can easily be seen to be nonzero for $n>1$: $\sin \frac{\pi}{n}$ only vanishes when $n=1$, $\cos \frac{\pi}{n}$ only vanishes when $n=2$, and when $n=2$ the formula involves $\operatorname{trig}_1 \frac{\pi}{2}=\sin \frac{\pi}{2}$.
If $n=1$, the problem statement is trivially true. But if $n>1$, we have just shown that the $(n-1)^{\rm st}$ finite difference of $P$ doesn't vanish everywhere. It follows that, if $P$ is a polynomial, it must have degree at least $n-1$.
Comment: Thank you,It's nice,and maybe this problem can use derivative prove this
Comment: @chinamath: Absolutely! If you combine this calculation with the [mean value theorem for divided differences](http://en.wikipedia.org/wiki/Mean_value_theorem_(divided_differences)), you actually get a proof that $P$ has a non-vanishing derivative of the appropriate order somewhere in $[0, 2\pi]$.
Comment: ,can you post it? Thank you
Comment: @chinamath: There's not a whole lot more to say beyond the computation in the answer. The mean value theorem for divided differences says precisely that the iterated difference I computed is equal to $\frac{P^{(n-1)}(x)}{(n-1)!}$ for some $x$ between the two farthest-apart points (that is, $2\pi/n$ and $2\pi$).


Campos com mesmo nome em diferentes tabelas

Question: Campos com o mesmo nome em tabelas diferentes podem causar conflito na hora de normalizar ou fazer algum inner join?
Exemplos
Tabela Fornecedor

Campo: nome
Campo: sobrenome
Campo: email

Tabela Produto

Campo: nome
Campo: data_de_entrega
Campo: Quantidade

Esse é um exemplo bem simples com apenas 2 tabelas, mas se eu possuo em torno de 50 tabelas ou mais, dentro de um determinado projeto, qual a solução eficaz? Já li sobre alguns tópicos dizendo para colocar nome_da_tabela_nome_do_campo, isso é uma forma de escapar?
De acordo com style guides do SQL.

Tabelas
Utilize um nome coletivo ou menos idealmente, plurais. Por
  exemplo, (em ordem de preferência) pessoal e empregados.
Não utilize
  prefixos com tbl ou qualquer outro prefixo descritivo ou notação
  húngara.
Nunca dê a uma tabela o mesmo nome de uma das suas colunas e
  vice versa.
> Evite quando possível concatenar dois nomes de tabelas
  para criar o nome de uma tabela de relacionamento.
No lugar de
  mecanicos_de_carro, prefira servicos.

Colunas
Sempre utilize nomes no singular.
Quando possível, evite usar apenas id como identificador primário da
  tabela.
Não adicione uma
  coluna como o mesmo nome da tabela e vice versa.
Sempre utilize caixa
  baixa, exceto onde capitalização fizer sentido (como em nomes
  próprios).

Então a parte que diz:

Evite quando possível concatenar dois nomes de tabelas para criar o
  nome de uma tabela de relacionamento.

Valeria a mesma coisa para os campos?
Comment: Sim pode causar conflito, mas é facilmente solucionável, basta você prefixar o nome do campo com o nome da tabela. Não vejo como isso pode ter qualquer impacto no processo de normalização.
Comment: tipo, seguindo o exemplo que foi dado: a `tabela Fornecedor` ficaria: `fornecedor_nome, fornecedor_sobrenome, fornecedor_email`, e a `tabela Produto`: `produto_nome, produto_data_de_entrega, produto_quantidade` desse jeito?
Comment: Não, eu quis dizer nome_da_tabela.nome_do_campo, toda a vez que houver alguma tipo de conflito com os nomes dos campos das tabelas envolvidas no comando. Não é na definição da tabela.
Answer: Não pode causar conflito, se fizer tudo certo, que é o que todo mundo faz, certo? Na hora de usar os nomes em uma query deve ter uma qualificação completa, ou seja, o nome da coluna e o nome da tabela (geralmente este nome é usado com um alias, então produto provavelmente será usado como p), então seu nome será p.nome, que será diferente de f.nome. Se não gostar da abreviação use o sobrenome completo (produto.nome e fornecedor.nome), isto é suficiente para desambiguar os nomes e evitar conflitos.
Colocar o nome da tabela no nome da coluna, de forma geral, é algo redundante e desnecessário. Existe apenas alguns casos em que o nome é descritivo. Por exemplo se você tem uma chave estrangeira em uma coluna, então você não vai usar só o nome dela, usará um qualificado, então se a chave estrangeira é o id de um fornecedor, o nome desta colina será fornecedor_id. Aí tem gente que acha que neste caso o id na tabela fornecedor também será ser completamente qualificado, mesmo sendo redundante, para que todas as colunas que representam a mesma coisa sejam iguais, mas há muita gente que discorda disto.
Dê nomes simples e semânticos para as colunas, evite redundâncias e complicações, por isso não coloque informação desnecessária, por isso os nomes que usou são adequados. Alguns dirão que data_da_ entrega já tem redundância, entrega é uma data, não precisa dizer, mas de fato há quem não concorde com isso.
A pergunta cita um tal de style guide do SQL, parecem boas recomendações gerais (não regras pra seguir cegamente), mas tem alguns pontos ruins. Aquilo é o trabalho de alguém que deve ser respeitado, mas não é alguém que tem autoridade sobre o assunto, foi algo que achou na internet, não quer dizer que tudo ali esteja correto ou que seja unanimidade, de fato olhando todos os elementos eu discordo de vários e alguns parecem bem errados, independente do meu gosto.

Nunca dê a uma tabela o mesmo nome de uma das suas colunas e vice-versa.

Isso é bom, mas não é uma verdade absoluta.

Sempre utilize nomes no singular

Geralmente sim, mas tem casos que isso não deve ser feito.

Sempre utilize caixa baixa, exceto onde capitalização fizer sentido (como em nomes próprios)

Bobagem. É só gosto da pessoa.

Evite quando possível concatenar dois nomes de tabelas para criar o nome de uma tabela de relacionamento.

Evitar sim, nunca fazer não.

Valeria a mesma coisa para os campos?

De forma geral, sim, só não tome como regra fixa, conforme citado acima.
O que você chama de campo na verdade se chama coluna, campo é uma informalidade que as pessoas entendem.
Coloquei no GitHub para referência futura.
Comment: A pergunta que farei agora foge um pouco do escopo da pergunta principal citada anteriormente, mas em questões a tabelas " [...] mas se eu possuo em torno de 50 tabelas ou mais [...]", o exemplo  `Tabela Curso` e `Tabela Evento` e ambos possuem uma outra tabela chamada `Categoria`, mas como não podemos colocar o nome de duas tabelas iguais, então deveríamos chamar a  segunda Tabela de `Categoria` como`Categoria_Curso` / `Categoria_Evento` (quebrando ela em duas), ou unificar essa unica tabela como `Categoria` e fazer um relacionamento entre as 3 tabelas?
Comment: Então faça uma nova pergunta, mas pense bem se ela está coerente, do jeito que fez não faz muito sentido. De qualquer forma, eu já disse que isso não é uma verdade absoluta, depende do que está fazendo.
Comment: Chamar campo à coluna é algo que sei que não é correcto mas que invariavelmente me esqueço.
Answer: Não, porque ao nome da coluna pode ser acrescentado o nome da tabela para a distinguir.
Ao nomear uma coluna a preocupação deve ser identificá-la/descrever seu conteúdo da melhor forma possível.
Nomes como fornecedor_nome e produto_nome não são necessários e devem ser evitados.
Quando for necessário identificar/distiguir uma coluna use fornecedor.nome e produto.nome
A excepção poderá ser em campos que representam chaves estrangeiras, quando não houver melhor forma de os distinguir.


Minimum area of an art gallery?

Question: I've adapted this problem from a TedEd riddle that I solved yesterday. It asks, what is the minimum number of rooms required such that every room except one room is connected to exactly three other rooms, except for one "special" room that is only connected to one other room. You can think of this like you're designing an art gallery to meet these conditions, and your "special" room contains a very rare painting which has to sit behind a vault door of which you only have one. I won't state the answer in this thread so as to not spoil the puzzle.
Anyway, I thought of an interesting extension of this puzzle. Suppose your building company is rather cheap, and can only build walls in integer multiples of one meter and walls can only intersect at right angles, i.e, they can only build on "gridlines". Note that non rectangular rooms are allowed, as long as any vertex has an interior angle of $90$ or $270$ degrees. Furthermore, a door takes up an entire one meter segment, and for ease of navigation, 2 rooms may not be touching without a door between them. My question is, what is the minimum area, in multiples of $1\text{m}^2$ boxes, of such a gallery design? How might I work this out?
Answer: Spoiler alert
If you're reading and you want to solve the TED problem, then quickly look away. I have hidden the diagrams & some key results to help reduce spoilers but it's too burdensome to hide all the text line by line.
Iteration 1
With problems like this, the best way to work is generally to do the following:

Come up with bounds & prove that your bounds apply
Share your results
Challenge others to improve on your bounds

The result is either that the minimum is found, or a series of better and better bounds are found - if enough interested people join in the quest. Having said all that, here is a very simple start. I'm not offering much, mathematically, but hopefully will encourage others to make further contributions.
So, without further ado, here is an image of a solution to the problem (but not necessarily the minimal such solution):

Let $A$ be the minimum area of a gallery satisfying your constraints. From the above image, we know that the following upper bound applies to $A$:

 $$A \leq 18$$

 We also know, from the original puzzle, that there must be $6$ rooms, and each room must necessarily take up at least $1m^2$. So we know that:

 $$6 \leq A$$

Therefore, we have bounds:

 $$6 \leq A \leq 18$$

How might I work this out?

Well, now that you have bounds, the next steps are:

Improve the bounds. In particular, I think there is a lot of low-hanging fruit on the lower bound .
Write a computer program to brute force through the search space and see if it comes up with anything better than the above- for this, maybe just look for a single integer improvement of the upper bound. The bound is currently $18$, so just write a program that looks for a solutions using $17$ squares.

I hope others can join in the puzzle. I love these collaborative things.
Iteration 2
Assuming the "not touching" rule doesn't count single point intersections, the following is allowable as a solution:

Therefore we have a new upper bound of $10$. So:

 $$6 \leq A \leq 10$$
Comment: And yes, the door criterion doesn't count for corner-touching rooms.
Comment: I did a little checking and it turns out $6$ isn't possible either. So now $7 \leq A \leq 10$. Getting closer!
Comment: This is so cool! The best I could come up with was $12$ and I was fairly sure it was the lowest possible but I suppose I was wrong. Nice work, thanks for the answer!
Answer: Here's a proof of a lower bound of $10 \text{ m}^2$ if we want to replicate the $6$-room solution. Here's the graph:

Let's modify the gallery as follows. First, forget about the room adjacent only to one other room for now. Now four rooms have $3$ adjacencies, and one room only has $2$. Second, take the room with $2$ adjacencies, and break down one of the walls separating it from an adjacent room. Now we have only $4$ rooms left, and each one is adjacent to all the others.
As a graph, this graph has a unique planar embedding: three vertices in a triangle, and one vertex in the middle of the triangle. If we try to turn this planar embedding into an art gallery, then the room corresponding to the middle vertex has to be entirely surrounded by the other three rooms.
Even if the middle room is only $1\text{ m}^2$, it takes $8\text{ m}^2$ to completely surround it; making the middle room bigger would only hurt. That gives us $9\text{ m}^2$ of area already. We must have had the same area before we tore down a wall; if we put that wall back, the gallery still has to have $9\text{ m}^2$ of area.
Finally, this is all ignoring the room with $1$ adjacency, which adds another $1\text{ m}^2$ of area, for a total of $10\text{ m}^2$.

More generally, the argument above shows that if we want to go below $10\text{ m}^2$, then the graph of the art gallery with the vault room deleted must be outerplanar: there can't be a room entirely surrounded by other rooms.
However, an outerplanar graph can't have the degree sequence $3,3,3,\dots, 3,3,2$ (with any number of $3$'s, and only one $2$). The argument is a bit technical, but essentially, if all the vertices lie on a cycle, and we take any chord of the cycle, we are forced to put a degree-$2$ vertices somewhere on each side of the chord.
In other words, it's impossible to have an art gallery with the property in the question, if every non-vault room has to border the outside wall of the building.
Comment: This is a beautiful solution and explanation. The step where you knocked down one of the walls to make the graph completely connected was really clever. Thank you.
Comment: @ColmBhandal I did not solve it! At most I did half the work of solving it. You did the other half :)
Comment: Bravo! You've solved it! Beautiful proof

Minimum area of an art gallery?


A finite topological space is T1 if and only the topology is discrete

Question:
Prove that a fnite topological space is T1 if and only the topology is
  discrete.

In first part od proof I want to use an equivalent definition of T1: a space is T1 if and only if every singleton is closed.
Proof of my exercise:
$\Rightarrow$:
Suppose $X$ is $T1$. I have to  show that every subset of $X$ is open. Let's take $U \subseteq X$. Let's define $H=X \setminus U$. I can write $H=\bigcup_{x \in H}\{x\}$. By defintion of $T1$ $\quad$ $\forall x \in H: \{ x \} $is closed in $X$. $X$ is finite, it follows that $H$ is the union of a finite number of closed sets of $X$. Sp $H$ is closed in $X$. $U$ is complement of $H$, so is open. We can choose arbitrary $U$, so every subset of of $X$ is open.
$\Leftarrow$:
Suppose $X$ is topological space with discrete topology. I have to show that for every pair of points $x \neq y $ there is an open set containing $x$ but not $y$. For every $x$ we have an open set $U_x$. Complement of $U_x$ is open and doesn't contain $x$. I don't know what do now.
Answer: If part: For $x\ne y$, since the topology is discrete, then $\{x\}$ and $\{y\}$ are open sets, and these sets are disjoint.


How do soccer broadcasts make money?

Question: Being in the US, and a sports fan, it is impossible not to be aware of the huge amounts of money changing hands for sports teams be it the NFL, NBA, or MLB. I've always assumed these contracts are worth so much money because these events draw viewers which in turn drive up advertising fees.
Now seeing the nature of soccer being two non-stop halves, this makes for a telecast with few commercials. My question to you, is how do broadcast stations make money with so few commercials during any given game?
Comment: Adding to the above, there is a huge following for European soccer in Asia (specifically south and SE Asia). The channels here are subscription-plus-ads, so you can imagine how lucrative they are.
Answer: I can only speak for Germany but here the Bundesliga (national soccer league) games are only available in paytv for live broadcasts so they collect a fee directly and are then commercial free. The only matches available in free tv are the openers of the season halves (so the season opener and the first match after winter break) and the relegation matches. (see wikipedia). Some free tv stations have shows presenting summaries of the games. Those usually are 3-10 minutes per game so there is space for commercials.
For international games the rights usually are bought by public tv (ard,zdf) which aren't allowed to show commercials after 8pm anyways but usually have bigger funds to buy the broadcasting rights than commercial tv stations.
So to sum in Germany soccer live broadcasts are financed by fees for either pay tv or public tv.
I would think most other countries have a similar system but I'm not really sure.
Answer: Sports are so lucrative for the reasons you mention (huge ad revenue). But you fail to mention the reason the ad revenues are so high. People don't Tivo sports. Sure some people do, but for the most part people want to watch sports live. This is why ad revenue is so high.
Football (Soccer) has struggled to get a TV footing in the states, and has only now just begun to do so. There are two main strategies for monetizing football games here:

Put your match on cable. This is the primary strategy, the ESPN networks, Fox Soccer channel and other cable TV channels do not rely as heavily on advertising as the "over the air" networks (ABC/CBS/FOX/NBC) do. This is because they get a significant chunk of their money directly from cable companies. Most channels that go heavy on soccer programming will be in a sports tier package that costs the subscriber more money because it costs the cable company more money.
Get some sponsors. When matches began airing in the US commercial free on non-cable TV (my first recollection of this was the 94 WC in the states, but that's probably because it's the first one I watched) they got companies to sponsor an entire half, or part of a half. They showed their logo next to the score and probably did some voice promotions.
Air your tournament both OTA and on cable. If you're a big company like Disney (ABC + ESPN family of networks), you might have both OTA providers and cable providers. That means you can air some games on your OTA networks and others on pay cable. This is what Disney does with the World Cup, many games are on ESPN and some marquee matches are on ABC. This provides access, but also protection for your OTA network, all of their time slots are not consumed by ad-less TV.

Most of the time you're going to see both of these strategies used. Even the cable networks use integrated advertisements during the half. Part of the reason you don't see more games on live OTA TV in the US is because it's currently hard to monetize and only appeals to a small subset of the American audience, this subset is willing to at least pay for EPSN and probably even pay for a higher sports tier cable plan so they monetize this way.
This all combines to result in football being a far less lucrative sport for the networks than the easier to monetize baseball, basketball, ice hockey and American football. You don't see multi-billion dollar soccer contracts yet, and until they figure out a better way to monetize it in the US they won't.
Comment: A bit irrelevant to the OP but since you touched the subject I feel like mentioning the fact that while NBA or NFL might feature the biggest international stars of the sport, the MLS is nowhere close to the same standard. This is partly due to the rather little viewership in the U.S. but also the fact that football goes way back in Europe. I can't see MLS being more than a place for fading stars to cash in before they retire (at least not for another 5-10 years).
Comment: @posdef Yes, the lack of MLS star power is a problem. But US networks that specialize in soccer mostly air the European stuff, as well as the major international competitions. Things like WC qualifiers and the WC itself also air. None of things things lack star power, although those stars are far less recognized than major baseball, football and basketball stars here.
Comment: This didn't explicitly answer my question, but I think it better explains the spirit of what I was asking.
Comment: @Jerrod basically what I'm trying to do here is that. You're asking how they make money, I'm explaining the revenue streams available. Basically for the cable channels, offering soccer is their way of getting more subscribers. For the networks they get sponsors for halves, but probably don't even break even there. It's a lot about gaining fans and driving people to subscribe to the cable channels.
Answer: Best leagues in Europe are almost always on pay channels, which end up cashing in serious sums of money from fans, especially from pubs and cafés that show the game to their guests.
In some countries you could see ads added on the dead spots on the pitch (e.g. behind the goals) and even overlaid on the fans sometimes. It's pretty annoying to have invasive ads like that but public channels don't have many other options that to rake in as much money from ads as they can.

How do soccer broadcasts make money?


How should I analyze repeated-measures individual differences experiments?

Question: I do quasi-experimental individual differences psychology research. I examine how people who differ in a cognitive ability (that I measure) perform on another task that always at least involves within-subject manipulations (and sometimes between-subject)–DVs are usually response time and/or accuracy. For this question I'd like to focus on response times (let's assume they are normally distributed). I then infer from the ability-task relations what it means theoretically for the cognitive ability. The nature of this work is correlational and involves repeated measures where each subject completes many task trials(mostly I'm not interested in changes over time, just the overall difference).
Researchers in my field often create categorical variables from the cognitive ability scores and compare the upper and lower quartiles of the distribution with a repeated-measures ANOVA. Because the cognitive ability is measured continuously, I am looking for an analytic strategy that treats the cognitive ability in this way. I've recently been investigating mixed-models, thinking that I can treat the people as a random effect grouping variable and the cognitive ability score as a random effect nested within people.  I would like to examine interactions between this nested random effect (cognitive ability) and the fixed effects of the experimental treatments by doing model comparisons.
Does this seem like a reasonable analytic strategy? Am I thinking about this right? What are some other ways (the simpler, the better) that I can take advantage of repeated measures -remove experimental subject variance- while also maintaining the cognitive ability measure as a continuous measure? Any citations or examples in R are appreciated.
In a typical experiment, I would expect there to be anywhere from 1-3 categorical IVs with 2-4 levels(measured by multiple trials), and 1 continuous IV(cognitive ability). The exact nature of the categorical variables changes from study to study. The designs are fully crossed.
Comment: @Matt If you have two levels on a within subject variable you can use the difference as dv (no need to go for repeated measures models). The same logic applies if you have two two-level within subject variables. The interaction is the difference of the differences (avoiding repeated measure models). However, if one of your within-variables has more than two levels, this approach does not work anymore, but you have to go multilevel. I recommend you read the special issue of the Journal of Memory and Language referred to in chl's answer: J. Mem. Language, 2008 59(4): Emerging Data Analysis
Comment: I think it would be great if you could provide a few example of the exact nature of the independent variables (i.e., how many factors and how many levels). I think you can get pretty far with traditional GLM depending on the exact nature of your designs.
Comment: It seems that you're on the right track but you might want to look at the interaction between cognitive ability and your other predictors.
Comment: @John that's exactly what I want to do. Are you suggesting that I'm not able to do that with mixed-models?
Comment: not at all... I'm suggesting that what you want to find is an interaction between cognitive ability and your other predictors.  All you need to do is add them to the model.
Comment: @Henrik Does traditional GLM do well with repeated measures (i.e. can it remove the subject variance)and continuous IVs? If so, can you provide me with examples or references?
Comment: @Matt, it's been a year since you asked this; have you come to any conclusions about what methods to use? Also, could you name a couple of papers that exemplify "individual differences" research?
Answer: There were already some useful comments, that are probably waiting for some updates in the question, so I will just drop some general online references:

Practical Data Analysis for the Language Sciences with R, Baayen (2008)
Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models, Jaeger (J. Mem. Language 2008 59(4))

Examples using R may be found on Doug Bates' lme4 - Mixed-effects models project.


Comparing binary and continuous predictors for diagnostic test

Question: There is a "traditional" biomarker (binary predictor) used in the diagnosis of a disease (binary outcome) that has a high cost to perform for the clinical labs. I'm studying alternatives, which are all analyte measurements (continuous predictors).
How can I compare the two? (as in, can we replace the binary predictor with one, multiple or a combination of the continuous predictors) I know I can evaluate the binary's performance by directly comparing to outcome (precision, recall, etc.) and the continuous by AUC but I can't think of any common measures between them.
Do I fit them both in logistic regression and compare coefs/odds ratios? Do I need to find a cutoff (youden?) for the continuous variable and treat it as a binary?
I'm probably overthinking this, but I appreciate the help.
Comment: [Don't dichotomize the continuous variable](https://stats.stackexchange.com/q/68834/28500).
Comment: @EdM right, I understand that, but how can I compare it to the binary predictor? Is there any performance measure in common I can use?
Answer: Instead of dichotomizing the continuous predictor(s), which is generally a bad idea, take advantage of the probability structure provided by the binary predictor. Then evaluate your continuous predictor(s) in a continuous probability model; don't jump to reduce that continuous probability to a yes/no decision. Any comparison between predictors should be done on the same cases.
You can calculate a Brier Score, which for a binary outcome can be taken as the mean-square difference between predicted outcome probabilities and observed outcomes. Unless the expensive "traditional" binary predictor is used as the definition of the disease, then presence/absence of the biomarker only provides a probability of presence/absence of the disease. That lets you calculate the Brier score for the binary-predictor model. The continuous probability model will provide continuous probability estimates case by case for a Brier score.
You can use the Akaike Information Criterion (AIC) to compare models built on the same data set but with the different biomarker choices. That's typically available in the summary of a logistic regression model. Some question whether it's appropriate for comparisons of non-nested models, but the consensus seems to be that it's OK. See this page and its many links.
You say that the biomarkers are "used in the diagnosis of a disease," which implies that there are other clinical variables used in diagnosis. A better approach might be to build logistic regression models that include both the biomarker(s) and the clinical variables, and see how much information the biomarker(s) add. Frank Harrell's post on added value explains how to do that, with an extensively worked-through example of a model for significant coronary artery disease. AIC comparisons could be done here, too.
Note that there are many pitfalls in developing biomarkers, described in Harrell's post on "How to Do Bad Biomarker Research". His Regression Modeling Strategies contains much useful advice on regression modeling. In particular for your interests, see the parts of Chapter 2 on allowing for non-linearity of outcome associations with continuous predictors, and Chapter 4 on how to build models without overfitting.
Comment: Thank you for the very complete answer! Just to answer you, there are other clinical variables but they're in patient records which I cannot access (unlike the biomarkers which are in my lab's database). I'll try using age+gender as the base model and see how it goes from there
Comment: @DanielNunes it should be possible to get access to the patient records, with permission from your institutional review board for human-subjects research, provided that protections are in place for privacy and confidentiality. You should work with clinicians having experience in the field to see what they consider important. Your study will be much more useful if you incorporate more clinical data and expertise.


apk how to only support 1920x1080 Resolution?

Question: I created a apk, but I just think it only supports 1920x1080 resolution, tag(supports-screens) should be how to write? Thank you everyone!
Comment: That's going to limit your target market quite a bit.
Comment: Refer to the [Distributing to Specific Screens guide](http://developer.android.com/guide/practices/screens-distribution.html) and have a look at `requiresSmallestWidthDp` in particular.
Answer: Please the below tag in AndroidManifest.xml for your <application/>
<supports-screens android:largeScreens="true"
    android:normalScreens="false" android:smallScreens="false"
    android:resizeable="true" android:anyDensity="false" />

Comment: only for 1280x720 resolution?
Comment: Yes. Then your application supports only largescreens as `android:largeScreens="true"`
Comment: The problem is, i want to provide multiple APK upload to Google play, support 800x480(854x480,960x540), 1024x600, 1280x720(1280x768,1280x800), 1920x1080(1920x1200) four resolution, if it is the same tag filtering error
Comment: I can't understand what is the filtering error. Change the specific attributes to `true` for particular resolution.
Comment: I mean, if 1280x720's APK and 1920x1080's APK are the tag ), when the user of the mobile phone is 1280x720, then Google play will download 1920x1080's APK, if 1920x1080's is the last upload, download the 1280x720's APK instead, right?
Comment: If this is your problem, simply place the resources in different folders - `drawable-xhdpi` for `xlarge` screens, `drawable-hdpi` for `large` screens.
Comment: Not feasible, resources are stored in the assets directory, and processed by NDK method

apk how to only support 1920x1080 Resolution？


is postgresql nullable foreign key with cascade delete feature possible

Question: Is it possible to declare table column that is used as foreign key, but can be null?
I have a table messages that contains user_uuid as foreign key reference to user_profiles table.
SELECT * FROM messages;

id   user_uuid                              message
--   ---------                              -------
 1   f52acab5-6115-4a09-ad81-eac662292968   Hello user john smith
 2   null                                   Hello all

I want to allow nulls for broadcast messages, but if user_uuid is specified, i want to make sure it is validated against user_profile table, and also to perform cascade deletion of user messages in case when user gets deleted.
Is there a way to do so by just database means?
p.s. I know the solution of making 0000000-0000-0000-0000-0000000000 fake record in a table and referencing it. But is there more civilized way?
Answer: You are already doing it correctly. If the foreign key column is NULL, referential integrity is not enforced for that row.
Answer: This is possible. Just check following threads:

PostgreSQL: NULL value in foreign key column

Can foreign key references contain NULL values in PostgreSQL?

The cascade deletion should also work, because at the end the column is a valid reference.


What are Darboux coordinates?

Question: What are Darboux coordinates?
Are they different from coordinates on $\Bbb R^n$ or some smooth manifold?
I am familiar with Riemannian manifolds, but Darboux coordinates came up in some materials.
Comment: Morally speaking, Darboux coordinates are a "normal form" for a geometric structure that has no local invariants or "situations in which one can apply Moser's trick."
Answer: Given a symplectic structure $\omega$ on a $(2 m)$-manifold $M$---that is, a closed $2$-form such that $\omega^m \neq 0$---Darboux's Theorem guarantees that for every point $p \in M$ there are local coordinates $(x^i, y^i)$ in which the coordinate representation of $\omega$ is
$$ \hat{\omega} = \sum_{i = 1}^m dx^i \wedge dy^i.$$
We call such coordinates Darboux coordinates and sometimes say that they are adapted to the symplectic form. The fact that Darboux coordinates always exist means that for any two symplectic manifolds $(M, \omega)$ and $(M', \omega')$ and points $p \in M$ and $p' \in M'$, there is a local diffeomorphism between neighborhoods of $p$ and $p'$ that pulls back $\omega'$ to $\omega$. In other words, the existence guarantees that all symplectic structures are locally equivalent.
This situation contrasts sharply with the behavior of Riemannian metrics. Given a Riemannian $n$-manifold $(M, g)$ and a point $p \in M$, in general there are not coordinates $(x^i)$ for which the coordinate representation of $g$ is $\hat g = \sum_{i = 1}^n (dx^i)^2$. In fact, the existence of such coordinates is equivalent to the vanishing of the Riemannian curvature in some neighborhood of $p$, and hence to the existence of a local isometry around $p$ of $(M, g)$ and the flat metric. Thus, a consequence of Darboux's Theorem is that for symplectic structures there is no analogue of Riemannian curvature---or any local invariant, for that matter.
Another (related) version of Darboux coordinates is instead adapted to a contact structure on a manifold, or more generally, a $1$-form $\theta$ for which $d\theta$ has constant rank.
Answer: A smooth manifold $M$ equipped with a closed, non-degenerate two-form $\omega$ is called a symplectic manifold. It follows almost immediately that a symplectic manifold is even dimensional.

Darboux's Theorem: Let $(M, \omega)$ be a symplectic manifold. For any $p \in M$, there is a coordinate chart $(U, (x^1, \dots, x^n, y^1, \dots, y^n))$ with $p \in U$ such that $\omega$ takes the form $$dx^1\wedge dy^1 + \dots + dx^n\wedge dy^n.$$

The coordinates in the above theorem are often called Darboux coordinates.

What are Darboux coordinates?


Error: any valid prefix is expected rather than "172.31.1.102/255.255.255.8"

Question: The command ifup from ifupdown fails when using some mask for static configuration on Ubuntu 20.04 LTS.
Example of working configuration
<code>$ cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto enp1s0
iface enp1s0 inet static
    address         172.31.1.102
    netmask         255.255.255.0
</code>
Example of non-working configuration
<code>$ cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto enp1s0
iface enp1s0 inet static
    address         172.31.1.102
    netmask         255.255.255.8
</code>
The error
<code>$ sudo ifup enp1s0
Error: any valid prefix is expected rather than "172.31.1.102/255.255.255.8".
ifup: failed to bring up enp1s0
</code>
Is it not possible to separate a network with non-continuous range of IP addresses? For example, I don't want 172.31.1.102 to be able to reach 172.31.1.118, but 172.31.1.230 should be reachable.
Comment: 255.255.255.8 is not a valid [netmask](https://en.wikipedia.org/wiki/Subnetwork)
Comment: "Computers that belong to a subnet are addressed with an identical most-significant bit-group in their IP addresses." I guess it is because of this sentence.
It is not possible to separate network with non continuous range of IPs?
For example, I don't want 172.31.1.102 to be able to reach 172.31.1.118, but 172.31.1.230 should be reachable.
Comment: That was not in your initial post, you said that it's not working, I told you why. Add more details and a question in your initial post, see [How do I ask a good question ?](https://serverfault.com/help/how-to-ask)
Comment: There is no need. You answered my question. I thought I could just do what I want with a netmask because I didn't know subnetwork had to be continuous range of IPs.
Answer: Stop.  255.255.255.8 is just plain dangerous, and the OS is warning you.
Your mask would use only addresses that end in multiples of 16!
You need to use the "high" bits in your mask, not the low ones, in order to make contiguous blocks.  Below are some valid examples.
<code>A Class C is a /24 mask, or 24 bits, and looks 255.255.255.0 or
11111111.11111111.11111111.00000000
</code>
/25 bits 255.255.255.128 or
11111111.11111111.11111111.10000000
/26 bits 255.255.255.192 or
11111111.11111111.11111111.11000000
/27 bits 255.255.255.224  or
11111111.11111111.11111111.11100000 << you probably want this one
(and so on)
/28 bits 255.255.255.240 (16 IPs, 14 usable);
/29 bits 255.255.255.248 (8 IPs, 6 usable);
/30 bits 255.255.255.252 (4 IPs, 2 usable);
/31 bits 255.255.255.254 (never really used, since there's no addresses free.)
/32 bits 255.255.255.255 (normally only used to describe a host, versus a network.)
Comment: I am working on a complex network and after listing all IPs (some are set on the hardware by the vendor and cannot be changed), 255.255.255.8 would have done exactly what I wanted in a simple way. I was not aware of the "high" bits constrain, which, in other words, means that subnetworks must be continuous.
Comment: While it is technically possible to do what you're asking (the first Ethernet cards didn't care what you did to them, as it was jumper based) but this helps you avoid other problems. You can still use a proper /27 or .224 mask to get your "stub nets" to work.  The simple workaround is to have your .230 device also listen to an IP inside your "stub net" range, as well as the .230 address. The .230 device keeps the wider, or over-arching, mask, while each virtual NIC will only listen inside the /27 mask.  You may want to take a quick read on subnetting and adding multiple IPs to the same NIC.
Comment: Ran out of space to reply - Thanks for marking mine as the best answer! Hope this helps you get past your issue!


Adjust points to closest point on a coastline in R

Question: I'm trying to find the closest point on a coastline to a given set of lat-long points in R. I have an sf object representing the coastline and a data frame with the lat-long points.
Here's an example of the data:
library(sf)
library(rnaturalearth)

# Load world coastline data
coastline <- ne_coastline(scale = "small", returnclass = "sf")

# Lat-long points
points <- data.frame(
  longitude = c(-71.5374, -72.1234, -70.9876),
  latitude = c(-33.865, -34.567, -33.456)
)
points_sf <- st_as_sf(points, coords = c("longitude", "latitude"), crs = 4326)

I want to find the closest point on the coastline for each point in the points_sf object, and create a new dataset of those coastal points that I can re-join to my old dataset and plot.
I have tried various methods, including using the distGeo function from the geosphere package and the st_nearest_feature function from the sf package. However, I'm encountering issues with incorrect dimensions or unexpected results.
Here's what I tried so far but keep encountering problems with the distances line.
library(sf)
library(geosphere)

# Calculate distances between each point and the coastline
distances <- distGeo(st_coordinates(points_sf), st_coordinates(coastline))

# Find the index of the nearest coastline point for each point
nearest_indices <- apply(distances, 1, which.min)

# Get the closest points on the coastline
closest_points <- st_coordinates(coastline[nearest_indices, ])

# Create an sf object with the closest points
closest_points_sf <- st_as_sf(data.frame(geometry = st_point(closest_points)),
                              crs = st_crs(coastline))

Any help you can offer would be amazing! I have spent a long time on this.
Tahnk you!
Comment: I guess this is a duplicate of: https://stackoverflow.com/questions/51292952/snap-a-point-to-the-closest-point-on-a-line-segment-using-sf , meaning that the solution provided there probably answers your question (although it's still somewhat cumbersome).
Comment: Hmmm. I'm not able to make that solution work in this case. Not sure if it's the package or the shapefile I'm using but the first loop wont even run.
Comment: EDIT: I was able to run but it completely butchers the location of my points. I am trying to snap to coastline in Argentina and the linestring method puts it in Antarctica!
Comment: Well, both starting with "A", that's a first step ;-) See answer please.
Answer: one approach, using the names of your example data:

place points at the desired resolution along the coastline (using st_geod_segmentize as we're working with spherical coordinates):

library(lwgeom)
library(units)

points_along_coastline <-
  st_geod_segmentize(coastline,
                     max_seg_length = set_units(10, km) ## adapt as fit
                     ) |>
  st_cast('POINT')

retrieve indices of nearest points for your offshore points:

indices_nearest <- st_nearest_feature(points_sf, points_along_coastline)

control:
library(ggplot2)

ggplot() +
  geom_sf(data = coastline) +
  geom_sf(data = points_along_coastline, alpha = .2, size = 1) +
  geom_sf(data = points_sf, col = 'blue') +
  geom_sf(data = points_along_coastline[indices_nearest], col = 'red') +
  coord_sf(xlim = c(-73, -70), ylim = c(-35, -33))

Answer: The answer by I_O building on segmentized coastline is interesting and ingenious; for completeness sake I propose an alternative approach building on sf::st_nearest_points(). It will be somewhat more precise (given that it does not use sampling) and avoids dependency on {lwgeom}, making do with {sf} alone.
What this piece of code does is:

unions all the world's coastlines to a single object (otherwise the calculation would be done by feature = many times)
calculates the connection of origins and coast as lines
casts the said lines to points (origin + destination)
subsets the object to get odd numbered points (i.e. filter out the blue origins)

The outcome is otherwise hard to distinguish from the answer by I_O.
points_on_coastline <- st_union(coastline) %>% # one coastline to rule them all!
  st_nearest_points(points_sf) %>% # lines from points to coast
  st_cast("POINT") %>% # convert lines to points / origin, end
  .[seq(1, 2* nrow(points_sf), 2)] # just the coastal parts, thank you!

# plot borrowed from earlier answer by I_O
library(ggplot2)

ggplot() +
  geom_sf(data = coastline) +
  geom_sf(data = points_sf, col = 'blue') +
  geom_sf(data = points_on_coastline, col = "red") +
  coord_sf(xlim = c(-73, -70), ylim = c(-35, -33))

Comment: I have little idea... a quick check of the source code https://github.com/r-spatial/sf/blob/main/R/nearest.R#L46 says that it calls a S2 function - and that would get us into a rabbit hole of C++ code that quickly gets above my paygrade.
Comment: Neat. Still baffles me that/how `st_nearest_points` finds orthogonals to the coastline - or does it discretise the coastline under the hood?
Comment: This solution works great!! super elegant. The only thing I wish it did was to retain the other columns from points_sf, which are metadata associated with the point -- essentially so I can plot the data as if they were the original ones. Any ideas?
Comment: I believe that the number and order of points is not changed, so you should be able to pass the metadata as a vector, perhaps after turning sfc to sf object...


Stochastic integrals and stopping times

Question: Suppose $\{H(s,\omega): s\ge 0 , \omega \in \Omega \}$ is progressively measurable and $\{ B(t): t \ge 0\}$ a linear Brownian motion. Show that for any stopping time T with: $\mathbb{E} \left[ \int_{0}^{T} H(s)^2 ds \right] < \infty$ we have:
(a)
$\mathbb{E} \left[ \int_{0}^{T} H(s) dB(s) \right]=0$
(b)
$\mathbb{E} \left[ \left( \int_{0}^{T} H(s) dB(s) \right)^2 \right] = \mathbb{E} \left[ \int_{0}^{T} H(s)^2 ds \right]$.
I have tried to use several lemmas such as approximations by progressively measurable processes. I don't really understand why T should be a stopping time. If I apply the optional stopping theorem for (a) would it work?
Also we could define the process $H^T(s, \omega)$ such that $\int_{0}^{T} H(s) dB(s)= \int_{0}^{\infty} H^T(s)dB(s)$ and try to work with that.
I am a bit confused of what should work in this case. Any help would be appreciated!
Answer: Let $M(t) = \int_0^{t \wedge T} H(s) dB(s)$. Then $M(t)$ is a stopped local martingale and hence is also a local martingale.
Further, $\langle M \rangle_t = \int_0^{t \wedge T} H(s)^2 ds$ and so your assumption gives $\mathbb{E}[\langle M \rangle_\infty] < \infty$. It is a standard result that this implies that $M$ is an $L^2$-bounded martingale and $\mathbb{E}[M(\infty)^2] = \mathbb{E}[M(0)^2] + \mathbb{E}[\langle M \rangle_\infty]$. Hence
$$\mathbb{E}[\int_0^T H(s)dB(s)] = \mathbb{E}[M(\infty)] = \mathbb{E}[M(0)] = 0$$
and similarly
$$\mathbb{E}[\bigg(\int_0^T H(s)dB(s)\bigg)^2] = \mathbb{E}[\langle M \rangle_\infty] = \mathbb{E}[\int_0^T H(s)^2 ds]$$
as desired.
Comment: The quadratic variation of $M$ at time $t$.
Comment: What does $_t$ represent?
Comment: Unfortunately I haven't covered too much in this regard, is there any way we could solve this with the stopped process $H^T$ that I previously defined?
Comment: I'm not sure what you mean by a solution "using $H^T$". Notice that $M(t) = \int_0^t H^T(s) dB(s)$.
Comment: Yes, thank you!

Stochastic integrals and stopping times