How to give edit permission only to specific fields in node edit in?

Question: I have content type with some cck fields:
<code>- task time
- task status
- task executor
- task category
</code>
and of course title/body and other drupal node default fields.
each node of this content type will be assigned to a user selected from site users in task executor field.
I want the when the executor logins view all fields of assigned to task(node) but only can change (edit) the value of task status field. node edit doesn't do that.
I try to use 'Content Permissions' of cck module but it is not what I really need. it doesn't have control on title and body field and also the user can only see the fields he has edit permission on them in node edit page.
What can I do?
I hope I could explain my question clearly.
Thank you.
Answer: You can use the Field Permissions module. It allows you to set permission for each field for every user type. It works simply fine.

The Field Permissions module allows site administrators to set field-level permissions to edit, view and create fields on any entity.
Features

Enable field permissions on any entity, not just nodes.
Role-based field permissions allowing different viewing patterned based on what access the user has.
Author-level permissions allow viewing and editing of fields based on who the entity owner is.
Permissions for each field are not enabled by default. Instead, administrators can enable these permissions explicitly for the fields where this feature is needed.
Comment: will it control title field too?
Comment: execume me, but I couldn't understand how to work with this module.
Comment: @hd. Enable this module and visit the user permission page. You will get all fields listed over there for several permission to give like view, edit, create for every user type... You can set permission for each field dependent upon your requirement...


What is a dynasty in sports?

Question: Now that the Blackhawks have won the cup twice in four years can they be considered a dynasty?
More importantly what is the definition of a professional team sport dynasty?
I vaguely recall, from football, that a team must win the championship three times in five years to be called a dynasty, but I'm unsure if that was correct. It probably depends on the sport.
Comment: I think, as you say, this will vary from sport to sport, and will not be applicable to many sports (for instance, I can't recall having heard it in cricket). As such I would consider rephrasing the question to ask something like "in what sports are dynasties recognised?", since the question "what is a dynasty in sports?" is likely to be unanswerable as it stands. It's a fair question for those sports where it does apply, though.
Comment: @Spinner Right, and it may be a cultural term as I have heard it used most in reference (but not limited) to North American sports. "In what sports are dynasties recognized?" seems really subjective in your context...any sport may have recognized dynasties if a person chooses to define legitimate occurrences as such.
Comment: @Spinner http://en.wikipedia.org/wiki/Dynasty_(sports)#Cricket
Comment: @edmastermind29 Thanks - I'm not disputing that there are teams in cricket history that fit this definition of a dynasty (indeed, the West Indian team of the 1980s sprang to my mind), just observing that I don't remember having heard the concept discussed much, if at all, in the realm of cricket.
Comment: @edmastermind29 That's a fair point, and in view of that I'm inclined to suggest that both the original question and my suggested altered question are too phrased too subjectively. The whole subject of "dynasties" in sport may be too subjective to answer questions on them definitively.
Comment: @Spinner I believe defining the term "dynasty" is straight-forward...but recognizing occurrences as such can be opinionated due to the subjectivity it presents (hence, the question). The attempt to focus on the term alone with several recognized examples is different than the use of the term itself in any given sport.
Answer: A dynasty in sports is a team that excels among their division, league, and/or sport over an extended period of time (multiple seasons for example). Dynasties are usually realized after the fact...or during the fact after an already established period of dominance.
The NHL recognizes several dynasties, the most recent being the Edmonton Oilers, who won five championships in seven seasons during 1984-1990.
The closest comparison, as recognized by the NHL, to the 2010-2013 Chicago Blackhawks would be the 1949-1955 Detroit Red Wings, who won four championships in six seasons (and two in the first four seasons, like the 2010-2013 Blackhawks), and the 1961-1964 Toronto Maple Leafs, who won three consecutive championships.
Other recognized dynasties in North American sports:

MLB - New York Yankees (1996-2003):  seven division titles, six AL pennants, and four World Series championships (1996, 1998, 1999, 2000) in eight seasons
NBA - Chicago Bulls (1991-1998): six championships in eight seasons
NFL - New England Patriots (2001-2012): three Super Bowls in four seasons (2001, 2003, 2004), five conference championships in eleven seasons (2001, 2003, 2004, 2007, 2011), and ten division titles in twelve seasons.
Comment: Given the changes to the league in a salary cap world, I think the definition of a dynasty by the NHL's standards may merit reconsideration.
Comment: The question asked is, "what is the definition of a **professional team sport** dynasty?" I answered the question in regards to a **professional team sport**. Feel free to add details about dynasties in regards to the NHL.
Comment: Oh, I'm not saying your answer was wrong by any means. I was just making a comment on the changes brought on by the cap in the NHL :)
Comment: Okay. Didn't think you were. :) I just wanted to be clear that I wasn't answering specifically to the NHL. I wasn't aware of the intricacies of the NHL and thought you provided good info in regards to that.
Comment: And the fact that the NHL doesn't recognize the Red Wings of the late 90s (back to back plus 3 in 6 years) seems to indicate Chicago is still quite a way from being a dynasty.
Answer: There are dynasties in almost every kind of team sports, not only NHL or the other North American sports... there are all over the world that have teams with dynasties.
For the rest of the world... especially in Europe, maybe Manchester United is the best example... 13 league titles since English First Division rebrands itself into English Premier League and never be finished lower that third in the dominance era. Also the other trophies, too.

What is a dynasty in sports?


Repeating many ANOVA tests with a large number of factors

Question: I have a large number of factors that categorize my numerical observations in different ways and no preconception which if any of these factors may be significantly explanatory, but I have a p-value threshold that I'm obliged to observe. I want to perform an analysis of variance for each factor and then select those factors that have significant p-values according to my threshold.
Because I'm running so many ANOVA tests, I'm worried about the possibility of a false discovery. Is this concern legitimate? And if so, what techniques are there for adjusting my p-value threshold to take account of the fact that I'm performing so many tests?
Comment: Different methods include - Benferroni correction (a conservative measure), FDR or permutation based p-value threshold calculation ...
Comment: @RamSharma And the Sidak adjustment, the Holm stepwise adjustment, the and the Holm-Sidak stepwise adjustment, the last being the most powerful of the methods listed between your and my FWER methods. :)
Answer: You also seem a little unclear (possibly) on which tests you are conducting. You would generally conduct a single omnibus ANOVA to answer the question Do any of the samples in each of the levels of my factor variable have sample means that are drawn from a population with a different mean than at least one other factor? If you reject H$_{0}$ for ANOVA, you would then proceed to multiple pair-wise comparisons, and indeed take a false discovery rate approach (or family-wise error rate approach, if that's your druthers) to correcting the Type-I error probabilities of multiple t tests.
The different adjustment methods in order of least to most powerful (assuming you have made $m$ pair-wise comparisons):
Family-wise Error Rate Methods
Bonferroni: Compare $p$-values to  $\frac{\alpha}{m}$.
Šidák adjustment: Compare your $p$-values to $1-\left(1-\frac{\alpha}{2}\right)^{\frac{1}{m}}$.
Holm's step-wise procedure:

order your $p$-values from smallest to largest.
Compare the smallest $p$ value to $\frac{\frac{\alpha}{2}}{m}$.
Compare the second smallest $p$ value to $\frac{\frac{\alpha}{2}}{m-1}$.
Compare the $i^{\text{th}}$ $p$-value to $\frac{\frac{\alpha}{2}}{m-i+1}$.
For all tests following the first test for which you fail to reject H$_{0}$ you will also fail to reject H$_{0}$.

Holm-Šidák step-wise procedure:

order your $p$-values from smallest to largest.
Compare the smallest $p$ value to $1-\left(1-\frac{\alpha}{2}\right)^{\frac{1}{m}}$.
Compare the second smallest $p$ value to $1-\left(1-\frac{\alpha}{2}\right)^{\frac{1}{m-1}}$.
Compare the $i^{\text{th}}$ $p$-value to $1-\left(1-\frac{\alpha}{2}\right)^{\frac{1}{m-i+1}}$.
For all tests following the first test for which you fail to reject H$_{0}$ you will also fail to reject H$_{0}$.

False Discovery Rate
Benjamini & Hochberg's step-wise procedure: (appropriate when tests are independent, or when any dependency between tests is positive)

order your $p$-values from largest to smallest.
Compare the smallest $p$ value to $\frac{\alpha}{2}$.
Compare the second smallest $p$ value to $\frac{m-1}{m}\times\frac{\alpha}{2}$.
Compare the $i^{\text{th}}$ $p$-value to $\frac{m-i+1}{m}\times\frac{\alpha}{2}$.
For all tests following the first test for which you reject H$_{0}$ we will also reject the null hypothesis.

Benjamini & Yekutieli's step-wise procedure: (appropriate when tests are dependent)

order your $p$-values from largest to smallest.
Compare the smallest $p$ value to $\frac{1}{C}\times\frac{\alpha}{2}$.
Compare the second smallest $p$ value to $\frac{m-1}{mC}\times\frac{\alpha}{2}$.
Compare the $i^{\text{th}}$ $p$-value to $\frac{m-i+1}{mC}\times\frac{\alpha}{2}$.
For all tests following the first test for which you reject H$_{0}$ we will also reject the null hypothesis.

Where the constant $C = \sum_{i=1}^{m}{\frac{1}{i}}$
Comment: Multiple comparisons adjustments have general applicability. See, for example, Shaffer, J. P. (1995). Multiple hypothesis testing. *Annual Review of Psychology*, 46:561–584.
Comment: @Alexis Thanks for the nice answer. I was wondering if the same approach can be used for a bunch of p-values coming, not from single ANOVAs, but from any other test (i'm thinking of 30 shapiro tests that i conducted and i was questioning the 5% level)? Thanks
Comment: Thanks. By factor, I mean a categorical variable with 3 or more levels.
Comment: Yes, that's right. You still perform *t tests* not additional ANOVAs subsequent to rejecting the null hypothesis of a oneway ANOVA.
Comment: OK, you are describing how to get to the pvalues of pair-wise combinations of the levels of a single factor. But I've got bunch of factors, and I'm trying to identify which one's are significant at the omnibus level.
Comment: No, I am not. Here's the procedure step by step: (1) ANOVA (1a) Not reject? Done! *No* pairwise tests for difference are rejected at your level of $\alpha$. (1b) Reject: proceed to step 2; (2) Conduct pairwise tests. If you have $k$ factors, then you may perform up to $\frac{k\left(k-1\right)}{2}$ of them; let's call $m$ the number of pairwise tests. You perform $m$ unpaired *t* tests for differences in means in one factor level versus another; these *t* tests use *pooled* variance equal to the "within variance" from your $F$ statistic. (3) Adjust your $p$ values (e.g. using FDR).
Answer: If I understand your question correctly, you have a multiple comparison issue. As a quick example of this issue, if you have a critical $\alpha$ value of 0.05 and you were to perform 100 hypothesis tests then we would expect 5 of the tests to come up significant even if all null hypothesizes  are actually true.
There are several ways to correct for this. The simplest (and most conservative) being the Bonferronni procedure where you divide the critical $\alpha$ value by the number of tests you are performing.


How to identify a Bayesian SEM parameter in R package blavaan

Question: I have fit a Bayesian SEM using the R package blavaan.
  Number of observations                            58

  Number of missing patterns                         1

  Statistic                                 MargLogLik         PPP
  Value                                       -384.987       0.432

Parameter Estimates:

Regressions:
                   Estimate  Post.SD  HPD.025  HPD.975   Std.lv  Std.all     PSRF     neff     Prior
  Nfixers ~
    temp              1.828    0.312    1.201    2.446    1.828    0.458    1.018   163.000 dnorm(0,1e-2)
    Ncat             -1.717    0.207   -2.129   -1.311   -1.717   -0.654    1.002   827.000 dnorm(0,1e-2)
  nonNfixers ~
    Ncat              3.173    0.219     2.74    3.608    3.173    0.887    1.007   246.000 dnorm(0,1e-2)
  Nfix ~
    temp              0.739    0.318    0.116    1.393    0.739    0.267    1.072   146.000 dnorm(0,1e-2)
    Nfixers           0.298    0.084    0.137    0.468    0.298    0.429    1.009  1971.000 dnorm(0,1e-2)
  Nup ~
    nonNfixers        0.243    0.057     0.13    0.354    0.243    0.477    1.004  1289.000 dnorm(0,1e-2)
    temp              0.912    0.264    0.367    1.408    0.912    0.329    1.012   124.000 dnorm(0,1e-2)
  GPP ~
    Nfixers           0.403    0.149    0.122    0.703    0.403    0.421    1.082   174.000 dnorm(0,1e-2)
    nonNfixers        0.692    0.105    0.491    0.905    0.692    0.985    1.049   203.000 dnorm(0,1e-2)
    temp              1.139    0.401    0.388     1.93    1.139    0.299    1.266    45.000 dnorm(0,1e-2)

Covariances:
                   Estimate  Post.SD  HPD.025  HPD.975   Std.lv  Std.all     PSRF     neff     Prior
 .nonNfixers ~~
   .Nup               0.096    0.069   -0.036    0.238    0.096    0.185    1.001   250.000    dbeta(1,1)
   .GPP              -0.191    0.088   -0.358   -0.021   -0.191   -0.335    1.018   187.000    dbeta(1,1)
 .Nfixers ~~
   .nonNfixers       -0.127    0.069   -0.272    0.007   -0.127   -0.227    1.014   413.000    dbeta(1,1)
 .Nfix ~~
   .Nup              -0.006    0.059   -0.124     0.11   -0.006   -0.014    1.003   613.000    dbeta(1,1)
   .GPP               0.075    0.060   -0.044    0.194    0.075    0.150    1.012   489.000    dbeta(1,1)
 .Nup ~~
   .GPP               0.112    0.064   -0.013    0.243    0.112    0.227    1.014   460.000    dbeta(1,1)

Intercepts:
                   Estimate  Post.SD  HPD.025  HPD.975   Std.lv  Std.all     PSRF     neff     Prior
   .Nfixers          -0.687    0.886   -2.442    1.102   -0.687   -0.557    1.019   141.000 dnorm(0,1e-3)
   .nonNfixers       -2.609    0.386   -3.371   -1.863   -2.609   -1.554    1.005   221.000 dnorm(0,1e-3)
   .Nfix             -0.171    0.788   -1.838    1.331   -0.171   -0.200    1.073   157.000 dnorm(0,1e-3)
   .Nup               3.432    0.696    2.089    4.846    3.432    4.006    1.012   123.000 dnorm(0,1e-3)
   .GPP               3.645    0.817    2.218    5.408    3.645    3.093    1.272    71.000 dnorm(0,1e-3)

Variances:
                   Estimate  Post.SD  HPD.025  HPD.975   Std.lv  Std.all     PSRF     neff     Prior
   .Nfixers           0.527    0.102    0.345     0.73    0.527    0.346    1.000  6178.000  dgamma(1,.5)
   .nonNfixers        0.600    0.112    0.396     0.82    0.600    0.213    1.001  2530.000  dgamma(1,.5)
   .Nfix              0.467    0.092    0.302    0.647    0.467    0.636    1.000 10073.000  dgamma(1,.5)
   .Nup               0.447    0.087    0.294    0.625    0.447    0.609    1.000  3288.000  dgamma(1,.5)
   .GPP               0.541    0.109     0.35    0.762    0.541    0.389    1.005  1075.000  dgamma(1,.5)

I have been using trace plots to evaluate parameters and model performance, here is the example of a plot for one parameter:

However, I'm trying to figure out which model parameter is represented here, in this case lambda[3,14,1]? Does anyone know how to find/extract this information in blavaan? The closest resource I could find was this site: https://faculty.missouri.edu/~merklee/blavaan/prior.html
and while it gives you an idea of what the greek letters represent it doesn't mention what the numbers in the brackets correspond to.
Answer: EDIT: In the development version of blavaan, the axis labels should now match the blavaan parameter names, so that the below is unnecessary. (This feature will eventually make it to CRAN in blavaan 0.3-3.)
If you use numbers in the plot command, the numbers should correspond to the ordering of parameters in coef(fit). For example, plot(fit, 1:4) should give you trace plots of the first four parameters found in coef(fit). To find specific parameters (like lambda[3,14,1]), it currently depends on whether you are using JAGS or Stan as your target (eventually, this will be more uniform). For JAGS:
pt <- parTable(fit)
pt[pt$pxnames=="lambda[3,14,1]" & !is.na(pt$pxnames), 2:4]

For Stan:
pt <- parTable(fit)
pt$pxnames <- with(pt, paste0(mat, "[", row, ",", col, ",", group, "]"))
pt[pt$pxnames=="lambda[3,14,1]", 2:4]


How to detect CLOSE_WAIT on socket in C on Linux?

Question: I'm writing TCP client and I want to reconnect immediately when connection will be broken (i.e. server is restarted). I want to restart connection as soon as possible, even if there is no data to send over this socket. I see in netstat that connection is in CLOSE_WAIT state, but there is no error from select, getsockopt or getpeername.
I know that I can try to write something to socket, but this connection is only for reading.
Answer: From man recv:

Return Value
These calls return the number of bytes received, or -1 if an error occurred. The return value will be 0 when the peer has performed an orderly shutdown.
Comment: `CLOSE_WAIT` means a `FIN` was received from the peer. That will cause `select()` to report the socket is readable, and then `recv()` will return 0. This is the "orderly shutdown" that the man page is referring to.


HATEOAS Link using UriBuilder

Question: I have a REST service defined as below:
@Path("/v1/employee/")
EmployeeService {

@Path("/designation/")
public void listDesignation() {

}
}

My goal is to generate the relative URL "v1/employee/designation/".
If I use UriBuilder.fromResource(EmployeeService.class),
it gives me "/v1/employee/"
And if i use UriBuilder.fromMethod(EmployeeService.class, "listDesignation"), it gives me "/designation"
Does anyone know how I can I get relative URL "v1/employee/designation" without doing any hard coding.
Thanks in advance.
Answer: You can use your UriBuilder#fromResource(Class), then chain a call to path(Class, String) (class, method name) or path(String) (actual path) or path(Method) (actual method if you have it).

HATEOAS Link using UriBuilder


coloring cube, additional constraint, three colors

Question: I have to paint nodes of cube such that opposing nodes has the same color.
We consider identical cubes such that rotatating.
My result is $15$
Is it correct ?
Ok, I 'll add my way to get a result.
$G$ is group of rotating.
$$G = \{id, o_1,o_2,o_3,r_1,..,r_1, ..., r_6, s_1,...,s_6,k_1,....,k_8\}\\
|G|=24$$
Const points with respect to $id$: $3^4 $
Const points with respect to $o_i$: $3*3^2 $
Const points with respect to $r_i$: $6 * 3$
Const points with respect to $s_i$: $6 * 3^3$
Const points with respect to $k_i$: $8*3*3$
Finally:
$$\frac{1}{24} (3^4 + 3 * 3^2 + 6 * 3 + 6*3^3 +8* 3 * 3) = 15$$
Comment: Could you add to this the method you used to get 15?
Comment: I edited my post
Answer: You used Polya counting theory and arrived at $15$, but are not convinced. So let's count the colorings of the four space diagonals again, this time using "brute force".
There are $3$ colorings using just one color.
If we restrict to just two colors we can do it in a $(3,1)$-way or in a $(2,2)$-way. For the $(3,1)$-way we can select the primary color in three ways and the secondary color in two ways. After the colors have been chosen there is just one way to color the diagonals, so that we obtain $6$ colorings of this sort. For the $(2,2)$-way we can select the two colors to be used in three ways. After the colors have been chosen there is just one way to color the diagonals, so that we obtain $3$ colorings of this sort.
If we use all three colors one of them, which can be chosen in $3$ ways, is used twice, and the other two colors are used once. Paint any two diagonals with these two colors and the remaining two with the chosen color – it always looks the same. It follows that we obtain $3$ colorings of this sort.
So it comes to $15$ colorings, as you have found out using the group theoretical approach.
Answer: This can also be done using the Polya Enumeration Theorem. With this in mind we need the cycle index $Z(G)$ of the group $G$ of rotations of the cube acting on the four diagonals.
There is the identity, which fixes the diagonals and contributes $a_1^4.$ There are $90$ degree and $270$ degree rotations about an axis passing through the midpoints of opposite faces, giving $3\times 2\times a_4.$ The $180$ degree rotations about these axes contribute $3\times a_2^2.$ Rotations about one of the four diagonals contribute $4\times 2 \times a_1 a_3.$ Finally there are flips about an axis passing through the midpoints of opposite edges, giving $6\times a_1^2 a_2.$
This gives the cycle index
$$Z(G) = \frac{1}{24}
(a_1^4 + 6 a_4 + 3 a_2^2 + 8 a_1 a_3 + 6 a_1^2 a_2).$$
Substituting three colors into this cycle index we obtain
$$1/24\, \left( A+B+C \right) ^{4}+1/4\, \left( A+B+C \right) ^{2} \left( {A}^{2}+
{B}^{2}+{C}^{2} \right)\\ +1/3\, \left( A+B+C \right)  \left( {A}^{3}+{B}^{3}+{C}^
{3} \right) +1/8\, \left( {A}^{2}+{B}^{2}+{C}^{2} \right) ^{2}\\+1/4\,{A}^{4}+1/4
\,{B}^{4}+1/4\,{C}^{4}$$
which expands into
$${A}^{4}+{A}^{3}B+{A}^{3}C+{A}^{2}{B}^{2}+{A}^{2}BC+{A}^{2}{C}^{2}+A{B}^{3}+A{B}^
{2}C+AB{C}^{2}\\+A{C}^{3}+{B}^{4}+{B}^{3}C+{B}^{2}{C}^{2}+B{C}^{3}+{C}^{4}.$$
We see that there are indeed $15$ colorings.
More generally for the case of at most $N$ colors we obtain
$$\frac{1}{24}(N^4 + 6N + 11N^2 + 6N^3) = {N+3\choose 4}$$
which gives the sequence $\{a_N\}$
$$ 1, 5, 15, 35, 70, 126, 210, 330, 495, 715, \ldots.$$
This is because the sequence of colorings $\{b_M\}$ using exactly $M$ colors, which is finite (there are only four diagonals available) is
$$1,3,3,1,0,0,0,0,\ldots$$
and
$${N\choose 1} +3{N\choose 2}+3{N\choose 3}+{N\choose 4}
= {N+3\choose 4}.$$
The four initial values of $\{b_M\}$ can be computed by inspection or using the inclusion-exclusion formula
$$b_M =  \sum_{N=1}^M {M\choose N} (-1)^{M-N} a_N =
\sum_{N=1}^M {M\choose N} (-1)^{M-N} {N+3\choose 4}.$$
Addendum. The cycle index can  also be computed by enumerating the
rotations of the cube and letting them act on the diagonals. The cycle
index  is then  obtained by  adding the  factorizations  into disjoint
cycles of the permutations of the diagonals that are obtained. This is
what  the  following  Maple  code   does, which  serves to  verify the
preceding results. It  yields the  following cycle index:
$$1/24\,{a_{{1}}}^{4}+1/4\,{a_{{1}}}^{2}a_{{2}}+1/3\,a_{{1}}a_{{3}}+
1/8\,{a_{{2}}}^{2}+1/4\,a_{{4}}$$
which is of course the same as above. The sequence of colorings is
$$1, 5, 15, 35, 70, 126, 210, 330, 495, 715, \ldots$$
which is also the same as above.

with(group);

pet_autom2cycles :=
proc(src, aut)
        local numa, numsubs;
        local marks, pos, cycs, cpos, clen;

        numsubs := [seq(src[k]=k, k=1..nops(src))];
        numa := subs(numsubs, aut);

        marks := [seq(true, pos=1..nops(aut))];

        cycs := []; pos := 1;

        while pos <= nops(aut) do
              if marks[pos] then
                 clen := 0; cpos := pos;

                 while marks[cpos] do
                       marks[cpos] := false;
                       cpos := numa[cpos];
                       clen := clen+1;
                 od;

                 cycs := [op(cycs), clen];
              fi;

              pos := pos+1;
        od;

        return mul(a[cycs[k]], k=1..nops(cycs));
end;

pet_cycleind_cubediag :=
proc()
local pg, els, idx, el, listel, diags, pdiags;

    pg := permgroup(8, {[[1,2,4,3], [5,6,8,7]],
                        [[1,6,7], [2,8,3]],
                        [[5,6], [4,3], [2,7], [1,8]]});

    diags := [{1,8}, {2,7}, {3,6}, {5,4}];

    idx := 0; els := elements(pg);

    for el in els do
        listel := convert(el, 'permlist', 8);
        pdiags :=
        map(d -> map(ent -> listel[ent], d), diags);

        idx := idx + pet_autom2cycles(diags, pdiags);
    od;

    idx/nops(els);
end;

v :=
proc(n)
    option remember;
    local sl, q;

    sl := [seq(a[q]=n, q=1..4)];
    subs(sl, pet_cycleind_cubediag());
end;


How to get the path to the requirements file in a Python repo using the GitHub API?

Question: I am trying to loop through a series of Python repositories and trying to get a link to the requirements.txt file in each of them (when present). However, it's not working. It seems I can't even search for a file by name, but only search the code for keywords and restrict the results to results inside files with a given name. I tried to make the search work by sending an empty querystring but it didn't work:
https://api.github.com/search/code?q=+repo:<repo name>+filename:requirements.txt

Sometimes I don't get results, sometimes I get them but they point to old, outdated code in old commits. I want the current contents from the requirements file so that I can use the safety library to see if there are any safety warnings for that repo. Any tips are welcome.
Answer: I would just use the contents endpoint on API V3. So in your case, the request would be -
https://api.github.com/<repo name>/contents/requirements.txt
eg. To get the file CHANGELOG.md on react.js project the API request would be.
https://api.github.com/repos/facebook/react/contents/CHANGELOG.md


Lilypond Scheme Substitution function not using its arguments properly?

Question: I would like to create a Lilypond music function that allows me to express this MWE:
{ \override Score.SpacingSpanner.base-shortest-duration = #(ly:make-moment 1/30) a b c }

as
\spn #1 #30 { a b c }

I tried with with this:
\version "2.22.1"

 spn =
#(define-music-function
  (beg end notes)
  (number? number? ly:music?)
  #{

 { \override Score.SpacingSpanner.base-shortest-duration = #(ly:make-moment #(cons beg end)) #notes }

  #})

\spn #1 #30 { a b c }

\relative c''
    { \override Score.SpacingSpanner.base-shortest-duration = #(ly:make-moment 1/30) a b c }

which does not produces same output as the MWE.
I get programming error: Unknown type predicate.
I suspect I am expressing the ly:make-moment argument improperly.
What am I doing wrong?
Answer: "unknown type predicate" would be an error with LilyPond's internals when typechecking the arguments of ly:make-moment because the complaint should be that the argument of ly:make-moment is not a rational number.  If this persists into the latest version, report a problem to the bug reporting list.
Of course your code also is faulty: you need to write (/ beg end) where you write (cons beg end) because the 1/30 you replace is not a LilyPond fraction 1/30 (which would be represented by (1 . 30) indeed) but a Scheme exact rational number 1/30.  Fixing that will help with your problem.
But you should get a proper error message rather than LilyPond not being able to tell you what it is complaining about.


Rails 3.0.1 set default file on a file_field

Question: I have two rails applications which are connected through an API. In the first rails app I have many users with avatar images. Now I want to send them through a file_field (in form_for) to the second rails app.
The problem is that I don't want manualy select the image-path for each user. Better would be if the image_path would be automaticaly inserted when the page is loaded. At the moment there is for each user a page where I select his avatar image and click on send.
The location of the images is on the first rails server. So this means that I want to send avatar images from one rails server to another with a file_field.
Is that somehow possible?
Answer: If you are trying to set a default image when opening a record for editing you have to do the following
In your edit view:
<code><%= form_for(@photo, :html => { :multipart => true }) do |f| %>
  <%= f.text_field :name %>
  <%= f.file_field :image %>
  <%= f.submit "Change" %>
<% end %>
</code>
In your controller:
<code>@photo = Photo.find(params[:id])
</code>
When you form gets displayed it will automatically assign the image in the database to the file_field.
Although the form might show "No File Selected", When you update your form your original image will still be there.


Check IP who is visiting my site on nginx

Question: I don't really want to know about this since I would like to keep it really private and give my visitor their privacy as much as possible (Not that my blog is popular though).
I just installed Ubuntu with nginx from Digital Ocean with the Ghost preinstalled, but previously I was with Wordpress. Right now (At this moment while I'm writing) I keep seeing this log
POST /bidRequest?exchange=smaato 500 2ms - 19b
POST /bidRequest?exchange=smaato 500 1ms - 19b
POST /bidRequest?exchange=smaato 500 1ms - 19b
POST /bidRequest?exchange=smaato 500 2ms - 19b
POST /bidRequest?exchange=smaato 500 1ms - 19b
POST /bidRequest?exchange=smaato 500 1ms - 19b
POST /bidRequest?exchange=smaato 500 2ms - 19b
POST /bidRequest?exchange=smaato 500 3ms - 19b
POST /wp-admin/admin-ajax.php 500 2ms - 19b
POST /bidRequest?exchange=smaato 500 2ms - 19b
POST /bidRequest?exchange=smaato 500 3ms - 19b
GET /winBid?erid=EzLM7nyV0n&eid=2&cpm=1.45449&bid=1628&w=1384697998 301 2ms
POST /bidRequest?exchange=smaato 500 2ms - 19b
POST /bidRequest?exchange=smaato 500 1ms - 19b
POST /bidRequest?exchange=smaato 500 3ms - 19b
POST /bidRequest?exchange=smaato 500 2ms - 19b

It is coming every second as you can see and they never stop on this and I'm thinking is that they are trying wether to hack my site while I was on Wordpress or something is going on from these requests. I would like to find out about these attacks and block their IP if this is something bad going on. How would I find out or should I install some kind of module/plugin in my box in order to prevent this kind of behaviour?
Comment: It looks like your IP was previously that of a server hosting some sort of API, and someone's still checking against it. Doesn't look like any sort of hacking attempt.
Comment: @ceejayoz I would suspect that too but I never actually host anything for people to check against? -- Or you are trying to say that the IP that digitalocean gave me was previously used for that?
Comment: Yes, I'd bet that the IP was previously used by someone else's service. If they're still showing up days from now, someone probably hard-coded the IP in their script.
Comment: Thanks @ceejayoz I will have to monitor this for a couple more days...
Comment: It's highly unlikely this is anything to worry about.
Answer: Log usually can be turned on and set the path on nginx.conf /etc/nginx/nginx.conf
If you vim or use your editor and edit that file and change or check this line
access_log  /var/log/nginx/access.log  main;

Answer: NGINX is capable of logging IP and other information that you might find interesting, but it seems that the log you're showing here isn't configured to include that information.  If you adjust your nginx logs you should be able to enable it.  If you need help, post your nginx logging configuration here.  Mine looks like this:
    log_format main
            '$remote_addr - $remote_user [$time_local] '
            '"$request" $status $bytes_sent '
            '"$http_referer" "$http_user_agent" '
            '"$gzip_ratio"';

And logs like this in the logs:

188.8.131.52 - - [17/Nov/2013:10:19:19 -0600] "GET / HTTP/1.1" 200 15411 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5)
  AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57
  Safari/537.36"

Once you find the IP(s) in question, you should be able to block them in a number of ways - the most effective is probably as part of the firewall (iptables) or, if a temporary block until reboot is acceptable, you could use a REJECT routing entry which basically makes your computer refuse to talk to that host at all.  If you think that's a little heavy handed, you could make rules to block in nginx in various ways probably (rate limiting comes to mind).
Answer: if your webserver is behind a load balancer, such as amazon ELB, $remote_addr will not give you the client IP.  instead you every entry will be IP of the upstream load balancing device.
to get around this, replace $remote_addr with $http_x_forwarded_for:

log_format  main
           '$http_x_forwarded_for - $remote_user [$time_local] "$request" '
           '$status $body_bytes_sent "$http_referer" '
            '"$http_user_agent" ';


How can add combined index to azure cosmo DB

Question: I've read this document over here https://learn.microsoft.com/en-us/azure/cosmos-db/indexing-policies and I coudn't find any way to add combined index on azure cosmo db at index polices section.
Or even I've tried to add the index from robo 3T and mongo compass and also it didn't work.
it seems that azure cosmo db don't allow to pass its "Indexing Policy"
Answer: Cosmos DB doesn't support this feature. You'll need to manage this combined index yourself. For instance:
<code>{
    "id":"value1-value2",
    "prop1":"value1",
    "prop2":"value2"
}
</code>


How to express the Riemann hypothesis in terms of the Gamma function?

Question:
The Riemann hypothesis (RH) states that all non-trivial zeros of the zeta function have real part $\frac{1}{2}$.
The zeta function is intimately connected with the Gamma function via the functional equation.

The second fact suggests that there is an equivalent form of RH which is expressed solely in terms of the Gamma function.
Question: What is the most natural form to translate RH as directly as possible (without mentioning the zeta function) into a hypothesis on the behaviour of the Gamma function?
Comment: Based on the philosophy that the Gamma function is the term in the Euler product "at infinity" I would expect that this is not possible, since a local term shouldn't give us enough information about the whole thing, but I would be pleasantly surprised if I were wrong.
Comment: I meant $(1-2^{-s})^{-1}$ above (editing comments seems to cause formulas to get mangled, especially annoying since it won't let you re-edit to fix this).
Comment: Let me play devil's advocate to the OP by pushing the argument further until its dubiousness becomes clear.  "(2) The zeta function is intimately connected with $\pi^s$ via the functional equation.  The second fact suggests that there is an equivalent form of RH which is expressed solely in terms of the powers of $\pi$."  Seems shakier, right?  For more convincing, try replacing $\pi$ with $2$...
Comment: ...so, my point was that concentrating on the gamma function is like concentrating on one of the $(1-p^{-s})^{-1}$ terms. Actually, if you wanted, you could redefine the zeta function as a sum over the odd positive integers which would have the effect of shifting the $(1-2^{-s})^{-1}$ term explicitly into the functional equation alongside the gamma function.
Comment: I agree with Qiaochu Yuan here. Trying to describe the Riemann hypothesis in terms of the gamma function alone is like trying to describe it with the function (1−2−s)−1(1-2^{-s})^{-1} alone, since it appears in the Euler product. All the gamma function does, afaict, is cancel out the trivial zeros at the negative even integers.
Comment: Pete - I think that's taking it a bit far. I guess that where the OP is coming from is supposing that the gamma and zeta function play more or less equal roles in the functional equation, and might not be convinced by suggesting looking at the $\pi^s$ term instead. Actually, using the product expansion, the functional equation involves a product of infinitely many terms, one $(1-p^{-s})^{-1}$ for each prime p and one $\Gamma(s/2)$ for the "prime at infinity" (aka, the Euclidean valuation). The zeta function encodes all but one of these terms, and the gamma function supplies the missing term.
Comment: As I see that refering to the functional equation is misleading I suggest to ignore (2) as an unfortunate choice of motivation. The question was: "What is the most natural form to translate RH into a hypothesis on the behaviour of the Gamma function?" Riemann defined $\zeta(s)=\Gamma(1-s)I(s)$ where $I(s)$ is a certain integral divided by a constant (H. M. Edwards, Riemann zeta function, 1.4(3)). Thus a way to put the question is: How to restate the RH for $\zeta(s)/I(s)$; however this might be futile as George points out.
Comment: Or Can there be an integral representation of the $\zeta$ function? Because for example there are certain integrals, whose values equal $\frac{\pi^{2}}{6} = \zeta(2)$, may be we should and can think of this question in that manner.
Answer: Since
$$\zeta(z)=\frac{\Gamma (1-z) \left(2^{-z} \left(\psi \left(z-1,1\right)+\psi \left(z-1,\frac{1}{2}\right)\right)-\psi(z-1,1)\right)}{\ln(2)}$$
where $\psi(x,z)$ is the generalized polygamma following Espinosa's generalization, whatever we say about Zeta function we can also say about the right hand part of this identity. It consists only of Gamma function, its (fractional) derivatives and integrals.
Comment: But it seems that Espinosa and Moll define their generalized polygamma function in terms of the Hurwitz Zeta function. Is there some equivalent characterization of their polygamma function that uses only Gamma?
Comment: The formula by Espinosa is obtained from fractional integration of polygamma by Adamchik (Adamchik, V. (1998). Polygamma functions of negative order. Jour. Comp. Appl. Math., 100, 191–199.) whose modified (by introducing proper integration constants for balancing) definition is used by Espinosa.
Comment: @Anixx This is interesting. How did you derive it?
Answer: The Wikipedia article gives a Mellin transform
$$\Gamma(s)\zeta(s) =\int_0^\infty\frac{x^{s-1}}{e^x-1} dx.$$
The Dirichlet series over the Möbius function gives the reciprocal
$$ \frac{1}{\zeta(s)} = \sum_{n=1}^{\infty} \frac{\mu(n)}{n^s} .$$
Thus we may write
$$\Gamma(s) = \sum_{n=1}^{\infty} \frac{\mu(n)}{n^s} \int_0^\infty\frac{x^{s-1}}{e^x-1} dx .$$
This holds true for every complex number s with real part greater than $1$. Now let's try to enlarge the domain of validity of this representation. Riemann showed (see the book of H. M. Edwards, Riemann Zeta Function, for the details) that modifying the contour gives a formula valid for all complex s.
$$ 2\sin(\pi s)\Gamma(s)\zeta(s) = i \oint_C \frac{(-x)^{s-1}}{e^x-1}dx .$$
This leads to
$$ \sin(\pi s) \Gamma(s) = \frac{i}{2} \sum_{n=1}^{\infty} \frac{\mu(n)}{n^s}
\oint_C \frac{(-x)^{s-1}}{e^x-1} dx . \qquad (*)  $$
However, this formula is again only valid for s with real part greater than $1$ because of the use of the Dirichlet series. Wikipedia remarks:

"The Riemann hypothesis is equivalent to the claim that [this
  representation of the reciprocal of the zeta function] is valid when
  the real part of $s$ is greater than $\frac{1}{2}$."

Thus a possible answer to my question is:

The representation $\,*\,$ is valid for all $s$ with real part greater than $\frac{1}{2}$ if and only if the RH holds.

Perhaps someone can elaborate further to give this relation a more geometric meaning? Where are the non-trivial zeros of the zeta function to be spotted in this setup?
Comment: Indeed, this is like claiming that $x=\tfrac12y$ is the result of getting rid of the $2$ in the equation $2x=y$ :)
Comment: this is sort of cheating.  You certainly haven't gotten rid of the zeta function; you've just moved it to the other side.
Comment: The remarks are delightful, but do they aid to answer the question which was: "What is the most natural form to translate RH as directly as possible (without mentioning the zeta function) into a hypothesis on the behaviour of the Gamma function?" Qiaochu Yuan's remark "I would expect that this is not possible" is the remark most agreed upon; however it does not help me to see the reason, as "the philosophy that the Gamma function is the term in the Euler product "at infinity"" is above my head and perhaps better suited for MO than for SE.
Answer: I don't think it's possible.
There is a paper that expresses very good approximations of zeta using truncated euler products and gamma functions.
The author in this paper, uses finite euler products, gamma functions and forces functional equation on the approximation, subtracts the principal component part from the approximation, and some other tricks. The approximation is very efficient to compute the zeros of Zeta. However, the principal part on the imaginary line is an infinite sum of of reciprocal functions with coefficients involving gamma values, and a finite number of primes. The regularization is based on the observation that the euler product should have a natural boundary at sigma=0, but the analytical continuation of zeta has none on the imaginary axis. Hence the principal part of the euler product must vanish somehow, and the author simply removed it cleverly.
However, I don't think you can express the principal part as finite sum of gamma functions, but as a contour in integral yes of course.
But why even bother, when you already have a hankel contour integral and mellin integral representation for the zeta function in terms of the exponential.


Cross-entropy cost function in neural network

Question: I'm looking at the cross-entropy cost function found in this tutorial:
$$C = -\frac{1}{n} \sum_x [y \ln a+(1−y)\ln(1−a)]$$
What exactly are we summing over? It is, of course, over $x$, but $y$ and $a$ don't change with $x$.  All of the $x$'s are inputs into the one $a$.
$a$ is even defined in the paragraph above the equation as a function of the sum of all $w$'s and $x$'s.
Also, $n$ is defined as the number of inputs into this particular neuron, correct? It is worded as "the total number of items of training data".

Edit:
Am I correct in thinking that
$$C= -\frac{1}{n} \sum_x [y \ln a+(1−y)\ln(1−a)]$$
would be the cost function for the entire network, whereas
$$C = [y \ln a+(1−y)\ln(1−a)]$$
would be the cost for the individual neuron? Shouldn't the sum be over each output neuron?
Answer: Here's how I would express the cross-entropy loss: $$\mathcal{L}(X, Y) = -\frac{1}{n} \sum_{i=1}^n y^{(i)} \ln a(x^{(i)}) + \left(1 - y^{(i)}\right) \ln \left(1 - a(x^{(i)})\right) $$
Here, $X = \left\{x^{(1)},\dots,x^{(n)}\right\}$ is the set of input examples in the training dataset, and $Y=\left\{y^{(1)},\dots,y^{(n)} \right\}$ is the corresponding set of labels for those input examples. The $a(x)$ represents the output of the neural network given input $x$.
Each of the $y^{(i)}$ is either 0 or 1, and the output activation $a(x)$ is typically restricted to the open interval (0, 1) by using a logistic sigmoid. For example, for a one-layer network (which is equivalent to logistic regression), the activation would be given by $$a(x) = \frac{1}{1 + e^{-Wx-b}}$$ where $W$ is a weight matrix and $b$ is a bias vector. For multiple layers, you can expand the activation function to something like $$a(x) = \frac{1}{1 + e^{-Wz(x)-b}} \\ z(x) = \frac{1}{1 + e^{-Vx-c}}$$ where $V$ and $c$ are the weight matrix and bias for the first layer, and $z(x)$ is the activation of the hidden layer in the network.
I've used the (i) superscript to denote examples because I found it to be quite effective in Andrew Ng's machine learning course; sometimes people express examples as columns or rows in a matrix, but the idea remains the same.
Comment: Thanks! So this would give us a single number for our error for the whole network, over all of our samples. For back propagation I need to find the partial derivative of this function wrt the weight matrix in the final layer. How would I do that?
Comment: Doing backprop is a whole separate can of worms! The page you linked to has a description of computing derivatives etc. and there are many questions about backprop on stackoverflow and this site. Try looking around a bit and then posting a separate question specifically about backprop.
Comment: This might be useful for you in understanding backprop it goes through the back prop with a four layer neural network with a cross entropy loss in gory detail :) https://cookedsashimi.wordpress.com/2017/05/06/an-example-of-backpropagation-in-a-four-layer-neural-network-using-cross-entropy-loss/?frame-nonce=f08dbcf84b
Answer:
What exactly are we summing over?

The tutorial is actually pretty explicit:

... $n$ is the total number of items of training data, the sum is over all training inputs...

The original single neuron cost function given in the tutorial (Eqn. 57) also has an $x$ subscript under the $\Sigma$ which is supposed to hint at this. For the single neuron case there's nothing else to sum over besides training examples, since we already summed over all the input weights when computing $a$:
$$
a = \sum_{j} w_jx_j.
$$

Later on in the same tutorial, Nielsen gives an expression for the cost function for a multi-layer, multi-neuron network (Eqn. 63):
$$
C = -\frac{1}{n}\sum_{x}\sum_{j}[ y_j \ln a^{L}_{j} + (1 - y_j) \ln (1 - a^{L}_{j})].
$$
In this case the sum runs over both training examples ($x$'s) and individual neurons in the output layer ($j$'s).
Comment: He also said in his tutorial that "y  can sometimes take values intermediate between 0 and 1" but the function he gave is all on y and there was no activation input. How could we implement intermediate values in the st function ?
Comment: Thanks for the insight, one question: the latter you defined is not the categorical cross entropy, right?
Comment: In Nielsen's tutorial, which shows a single-layer perceptron, a = \sigma( \sum_{j} w_j x_j) because you have sigmoid activation function for your output layer, not a = \sum_{j} w_j x_j


Evaluation of the integral $\int \sqrt{t^4-t^2 + 1}\,dt$

Question: My friend took his Calculus $2/3$ test yesterday.
One of the questions he had trouble with was this integral:
$$\int \sqrt{t^4-t^2 + 1}dt$$
My attempt
It seems rather clear that the only approach was trigonometric substitution.
First, completing the square:
$$\int \sqrt{t^4-t^2 + 1}dt = \int\sqrt{t^4-t^2 + \frac{1}{4} + \frac{3}{4}}dt = \int \sqrt{\left(t^2 - \frac{1}{2}\right) + \frac{3}{4}}dt$$
Next, I let $$\sec \theta = \frac{\sqrt{\left(t^2 - \frac{1}{2}\right) + \frac{3}{4}}}{\frac{\sqrt{3}}{2}}$$
$$\frac{\sqrt{3}}{2}\sec \theta = \sqrt{\left(t^2 - \frac{1}{2}\right) + \frac{3}{4}}$$
$$\tan \theta = \frac{t^2 - \frac{1}{2}}{\frac{\sqrt{3}}{2}} = \frac{2t^2 - 1}{\sqrt{3}}$$
$$\sec^2 \theta d\theta = \frac{4}{\sqrt{3}}tdt,\ dt = \frac{\sqrt{3}\sec^2 \theta}{4t} d\theta$$
$$t = \sqrt{\frac{1}{2}\left(\sqrt{3}\tan \theta + 1\right)}$$

Substituting this all in:
$$\int \left(\frac{\sqrt{3}}{2} \sec \theta\right) \frac{\sqrt{3}\sec^2 \theta}{4\sqrt{\frac{1}{2}\left(\sqrt{3}\tan \theta + 1\right)}} d\theta$$
How would I approach this from here?
I'm thinking of using u-substitution but I'm sure that it would bring me back to where I started, meaning I would have to use trig. substitution again.
Comment: You forgot to raise your quantity $$\left(t^2-\frac{1}{2}\right)$$ to a power of two.
Comment: This looked pretty unmanageable, and WolframAlpha gives an answer involving elliptic integral functions, so presumably there is no elementary closed form. Perhaps the problem was (intended to be) $\int \sqrt{t^4 - 2 t^2 + 1} \,dt$ or something similar?
Comment: @Travis If that is the case, then maybe there is no closed form. It seemed to me that there would be.
Comment: I know that you can use euler substitution when you got $\sqrt(at^2+bt+c)$, maybe there is some way you can use euler substitution on this?

Also I think it is going to be $(t^2-\frac{1}{2})^2$ instead of $(t^2-\frac{1}{2})$
Answer: The indefinite integral cannot be expressed in terms of elementary functions. Evaluating it requires knowledge of elliptic integrals. However, it is interesting to note that its definite counterpart, when evaluated over the entire real line, and subtracted from its quadratic or parabolic asymptote, yields
$$\int_{-\infty}^\infty\bigg[\sqrt{t^4-t^2+1}-\bigg(t^2-\dfrac12\bigg)\bigg]dt~=~\dfrac23K\bigg(\dfrac34\bigg)+\dfrac43E\bigg(\dfrac34\bigg),$$
which is nothing else than the volume of the oloid, whose decimal expansion can be found on OEIS


Fiverr style textarea with css and JS

Question: i wan't to make a textarea that looks like fiverr textarea to create a gig, the text "I wil" is fixed and disabled and i can write just next to it.

i managed to create something similar with css

but i'm stuck on the following issue when the text is written on many lines the first fixed text isn't moved to top with the other lines that the user wrote

i know javascript is the solution but i don't know exactly how , any ideas please ? i'm using angularJS if there is anything specific to angular it's also welcomed. here is my code :

        <textarea class="text-combien" style="height:100px;padding-right:15px;" ng-model="question.text" placeholder=""> </textarea>

Pour Combien,

?

Comment: you could make the border of the actual input trasparent with a .before with the "i will" part, and wrap it in a div with border
Comment: something like this: http://jsbin.com/tolifa/1/watch?html,css,output ?
Comment: @balexandre thanks for the reply, the jsbin you wrote is what i have now but take a look at the third picture. that's what happens when a user write more than two or three lines
Comment: @zerzer yes, but you can either hide the `.before-title` when there's more lines, or make the `textarea` auto-sizable that would grow automatically upon each new line... plenty of jquery plugins for that :)
Comment: @vsync yes i reposted this question because i didn't get an answer for this one
Comment: @zerzer - that is spamming...you can just edit your question or think of a better title to make it more "attractive"
Answer: You can use the text-indent CSS property on the textarea, and then you just need to position a span to go where you've indented the text. Check out this jsFiddle. You will need to tweak the positioning/sizing based on what font you want.

<textarea placeholder="do something I'm really good at"></textarea>
I will

Comment: yes this is what i have now but look at the picture 3 that's my real problem , when i write too many lines i need the text to move up
Answer: The answer I was looking for is here :
https://jsfiddle.net/ttwbxwon/66/
with the following javascript function binded to onscroll event of the textarea :
function heigthChange() {
  var e = document.getElementById("text-combien");
  var height =  e.scrollTop;
  document.getElementById("before-text").style.top = (0 - height) + "px" ;
}

Fiverr style textarea with css and JS


How can I seperately display a view's exposed filter and its content in page.tpl?

Question: Using <?php dpm($node) ?> I found the exposed filter, $...->view->exposed_widgets, however I'm not sure how to display it. I tried <?php echo $node->view->exposed_widgets; ?> but that returns nothing.
<?php dsm($form); ?> and <?php dpm($form); ?> both give

Notice: Undefined variable: form in include() . . . page.tpl.php

Are those deprecated? I see them mentioned sometimes but they've never worked for me.
The only place I see the form in <?php dpm($page); ?> is ['content']['system_main']['main']['#markup'], and that can't be separated, obviously.
Answer: As per my understanding, why you want to doing via coding. Create views exposed filter as blocks and place in the sidebar and by default the content will load in the right side of the content area. Now you will get filters and content separately...


Spirtes' example of d-separation not leading to independence in a directed cyclic graph with non-linear structural equations

Question: In Spirtes (1995) there is an example (Fig. 4 on page 495, reproduced below) of a directed cyclic graph with non-linear structural equations in which $d$-separation of $X$ and $Y$ given $\{Z, W\}$ does not lead to conditional independence of $X$ and $Y$ given $\{Z, W\}$. I have a problem understanding the first part: why do we say that $X$ and $Y$ are $d$-separated given $\{Z, W\}?$ Both $Z$ and $W$ are colliders, and we include them both in the conditioning set.
Comment: Good question, given that $X$ and $Y$ are $d$-separated given the empty set $\varnothing.$ I can think of one possible explanation, but I have to see if it's correct.
Answer: Here is my explanation. I believe the author is right. It comes down to this: for a double arrow relationship $W\longleftrightarrow Z,$ neither $W$ nor $Z$ is considered a descendant of the other (unless you have other edges relating them). That is, $W$ is not a descendant of $Z,$ nor is $Z$ a descendant of $W.$ So let us consider your graph, but only one direction at a time:

Here, conditioning on the set $\{W,Z\}$ does open up the collider at $Z$. However, the path from $X$ to $Y$ is still blocked by the chain at $W,$ since $W$ is in the conditioning set. Similarly, if we consider the other "half" of the graph,

the same conditioning set opens the collider at $W$ but closes the chain at $Z.$
In either setting, causal information cannot flow from $X$ to $Y,$ hence $\{W,Z\}$ $d$-separates $X$ and $Y.$
References: Causality: Models, Reasoning, and Inference, 2nd Ed., by Judea Pearl, pp. 17-18. Note that in the example of Fig. 1.3(a), Pearl has to resort to the path $Z_3\to Z_2\to Z_1$ to show that $Z_1$ is a descendant of $Z_3;$ he does not use what would be the obvious $Z_1\longleftrightarrow Z_3$ relationship.
Comment: 1/2 It is hard for me to lend credence to your answer or to call it flawed, since Spirtes does not, to my read, really land on the issue of time in DCGs.(My comment expresses a dissatisfaction with Spirtes.)  In the quoted DCG it is unclear whether $W$ is measured at a single time, or whether it is (as in the case of signed digraph representations of causal feedback elsewhere) intended to represent the variable $W$ at *all* times during which the system exists.
Comment: 2/2 Your answer's analysis is trying to have it both ways in that "one direction at a time" treats $W$ and $Z$ as single fixed points in time, however, the original DCG implies $W$ and $Z$ are both "ancestors" which implies these nodes represent variables at more than one point in time. Contrast the OP's DCG with a DAG of $X_0 \to W_1 \to Z_2$ and $Y_0 \to Z_1 \to W_2$ (Possibly with $X_0 \to W_2$ and $Y_0 \to Z_2$ also) where the subscripts explicitly indicate time (or at least temporal *order*).
Comment: 1/2 Spirtes ignores decades of theory of causal inference and graphical representation of complex causal systems, e.g., Levins, R. (1974). [The Qualitative Analysis of Partially Specified Systems](http://www.academia.edu/download/50256413/j.1749-6632.1974.tb20562.x20161111-9773-8soz.pdf). Annals of the New York Academy of Sciences, 231, 123–138; Sugihara, G., & May, R. M. (1990). [Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series](https://rmgsc.cr.usgs.gov/outgoing/threshold_articles/Sugihara_May1990.pdf). Nature, 344(6268), 734–741.
Comment: 2/2 While Spirtes references some influential papers from the 40s and 50s (a period when cybernetics and feedback in electronic circuits were being tackled as practical problems), he leaves untouched work, especially in eco*system* sciences which made both theoretical and applied advances in the interim. That said, the formal graphical conventions of Levins, and also those of Sugihara are different than Spirtes' DCGs (in how time fits into the graph).
Comment: Alexis: Sorry, I haven't got a foggy clue what you're really saying. Could you please be more specific and use less jargon?
Comment: Which "jargon" do you specifically object to? My gist is that Spirtes in 1995 was *not* really up on the state of the art in different forms of causal analysis of systems of causal feedback.
Comment: Perhaps. Are you saying the original question is flawed? Or that my answer is flawed?
Comment: In Pearl's usage, the "spousal" relationship of $W$ and $Z$ is treated as if neither is an ancestor of the other. How would you answer the question? Or do you view it as an ill-posed question?
Comment: I think it is a good question. I also think my first two comments sum my dissatisfaction. Counterfactual formal causal reasoning *a la* Pearl is amazingly useful ***for narrow causal questions having specific forms,*** but it breaks, and breaks *hard* on systems thinking (like the bidirectional arrow between $W$ and $Z$ implies). Spirtes' ignorance of methods of formal causal analysis of complex causal systems (i.e. systems where variables are directly or indirectly caused by all other variables) grates. Harumph.
Comment: Finally, if neither $W$ nor $Z$ are ancestor of the other, then what the hell do the pointy ends of the graph edge joining them mean‽


Form with managed_file value as link in email

Question: Like the title says I want to make managed_file value to be a link on form submissions in the email.
Webform has this option. When a user uploads file, on the email form submission there's a link with the path of the uploaded file. I want to achieve this with custom module.
This is my code:
<?php

function mymodule_page($form, &$form_state) {
    $form['description'] = array(
        '#type' => 'markup',
        '#markup' => t('Add description'),
    );
    $form['file'] = array(
        '#type' => 'managed_file',
        '#title' => t('Upload image'),
        '#upload_location' => 'public://files',

    );
    $form['submit'] = array(
        '#type' => 'submit',
        '#value' => t('Submit'),
        '#submit' => array('mymodule_page_submit'),
    );

    return $form;
}

function mymodule_page_submit($form, &$form_state) {
    $file = file_load($form_state['values']['file']);
    $file->status = FILE_STATUS_PERMANENT;
    file_save($file);

    //dpm($file);

    $description = $form_state['values']['description'];

    $params = array('file' => $file->uri, 'description' => $description );
    drupal_mail('mymodule', 'mymodule_news_mail', 'myemail', language_default(), $params);
    drupal_set_message(t('Message sent!'));
}

/*
** Implements hook_mail().
*/
function mymodule_mail($key, &$message, $params) {
    switch($key) {
        case 'mymodule_news_mail':
            $message['subject'] = t('Hello');
            $message['body'][] = t('This is a test');
            $message['body'][] = t('Image upload: @file', array('@file' => $params['file']));
            $message['body'][] = t('Description: @description', array('@description' => $params['description']));
        break;
    }
}

As you can see it from my code I'm saving the file properly into the server. After that with dpm($file->uri) I can see the string public://files/file_name.
How can I display the link from the path of the uploaded file on the email, so will be able to click on the absolute path?
Answer: Thanks to Jaypan from drupal.org, I've got the answer:
<?php
file_create_url($file->uri);
?>

Simply with adding this code and make another variable that will use the code, I've got the right result.


Oz Mozart factorial function

Question: I have this factorial function that is currently working, but the result I'm having is not the one I need.
The code is:
declare
fun {Fact N}
if N==1 then [N]
  else
    Out={Fact N-1}in
     N*Out.1|Out end
end
{Browse {Fact 4}}

The result is : [24,6,2,1] but I need that the result shows: [1,2,6,24] and I can't see where the error is.
Answer: You want to decrement N, having no other argument than N.
But there is a problem:

a list [1 2 6 24] can actually be written as '|'(1 '|'(2 '|'(6 '|'(24 nil)))) in oz
so at the first or last call, your function has to return 24|nil...
However, your function can not know if one call is the first call or the last, as you don't have any parameter.
your function is not tail recursive (if you use a book to learn oz, you'll discover why, and why it's bad)

Here is the best function I can think of:
declare
fun {Fact N}
   fun{Aux N Nmax FactNminus1}
      if N>Nmax then nil
      else (FactNminus1*N)|{Aux N+1 Nmax FactNminus1*N}
      end
   end
in
   {Aux 1 N 1}
end
{Browse {Fact 4}}

N is increment until Nmax
FactNminus1 contain the {Fact N-1} so you don't have to compute it each time.
Comment: Thanks!! your explanation was very useful to see that I was not creating a tail recursive function, and I'm taking a look to one book to learn more about oz.

Oz Mozart factorial function


Invalidate node cache so hook_node_view() runs on each node view

Question: My module uses hook_node_view() to add some contents to the node view.
function example_node_view() {
  if ($view_mode == 'full' && $node->bundle() == 'example')

  // Insert file listing content part
    $contents = Class::getContents();
    $build['example'] = $contents;
}

Drupal 8 caches the node view by default and only calls hook_node_view() once after each cache rebuild. For this module to function, hook_node_view() should run every time the node is requested. So, I am looking for ways to invalidate node:nid cache tag.
NB:
- I am not looking for ways to disable the caching for a development.
public function getContents(){
  return [
    "#type" = 'table',
    "#header' = 'header',
     .....
  ];

Solved by adding the CacheDependency to hook_node_view() as indicated by @4k4
$renderer = \Drupal::service('renderer');
$renderer->addCacheableDependency($build, $node->id());

Comment: Have you tried going to Configuration > Development > Performance? (I'm getting this off the top of my head for D7, it might be a little different for D8)
Comment: Where is `$content` coming from? Is this from the database or external? How often does this change?
Comment: @chris-happy, thanks, but I am not trying to clear chache, I want to invalidate the cache-tag for the node requested.
Comment: @4k4 $contents is created trough some Classes and has it's own caching.
Comment: @Bundio, I'm not sure, what exactly is in $contents. I've based my answer on files, but this works for all database content.
Comment: The code you've added solves the problem by disabling all caching of the node. The node id has no cache data and then the cache max-age is set to 0. This is probably not what you've intended to do. You need to use as second parameter the entity you add to the build array. The node is already set, so you don't need to add it again.
Comment: If you add other nodes, then you would need to add the cache dependencies from these nodes, so that the cache item from the node will contain multiple node tags. Use `$node`, not `$node-id()`.
Comment: I think you're asking the wrong question. You can easily disable caching with a max-age = 0. But most likely, you don't want to disable caching. You want it to get invalidated when there is actually a change. The question you should be asking IMHO is how to get the correct cacheability metadata (cache tags + contexts) for the data you display to get the best possible cache hit rate while still having it invalidated when necessary. For that, we need to know what you're actually displaying, that code snippet doesn't help. You coul ask that in a new question
Comment: @4k4 & berdir, thanks! In my module hook_node_view() MUST run every time node/nid is called, like in D7. $contents is not a entity. It is a list of files. By design it chosen that this file-list is un-managed and can change without Drupal knowing it. The only way to know, is to run the class that will find-out if there is any changes in the list. It will use it's own caching for efficiency. That process is only triggered by hook_node_view(). So while I understand the process is selectively invalidating the cache-tag by setting cache-context, this solution seems to be the right one.
Comment: Module is [Filebrowser](https://drupal.org/project/filebrowser)
Answer: For new content added to $build you have also add the cache metadata.
Example if you add data from a file entity do this:
$renderer = \Drupal::service('renderer');
$renderer->addCacheableDependency($build, $file);

This adds the cache tag for the file and whenever this file is changed, the cache item of the node which contains information from the file gets invalidated.
If  the added $content is already a render array with cache metadata drupal will merge this. You can check, which cache tags are missing in $content and add them in the class that generates this data.
Reference: https://www.drupal.org/docs/8/api/render-api/cacheability-of-render-arrays
If what the comment in the code suggests you have a list of files, you can add the tag 'file_list'. All entities invalidate the tag [entity-type]_list when creating, changing or deleting an entity.
Edit:
If you use the standard tools, that is building single entity displays with the view() method and list of entities with Views, then you don't need to take care of cache dependencies, because the render arrays these tools produce contain all necessary cache metadata.
Comment: You tag the node with all entities it contains. For example if you have a node with a author and a file field, then you have three tags, from the node itself, the author and the file(s). If any of them are changed, then the cached node is invalidated and will be rebuilt in the next request for the node. As long as all of the content of the node remains unchanged, there is no need to run the code again, because this would produce the same result as the cached version.
Comment: I have included a sample function how $contents is created. Indeed it returns a render array. If I understand well I can add a tag to this array and associate it with the node view owning this custom content. But I don't understands how that works because getContents() will only run when hook_view() runs and that will only run when the node cache is invalidated.


Getting PID from within a Start-Job

Question: I am having trouble getting the PID from a Start-Process within a Start-Job in Powershell (latest version).  The Start-Process cmdlet runs and returns the PID as expected when run outside of the Start-Job Scriptblock. When the statement is added to the Start-Job Scriptblock, no PID is returned.  Can someone point this newbie in the right direction?
$myJob = Start-Job -Name newJob -ScriptBlock {
$procID = (Start-Process myprogram.exe -PassThru -ArgumentList "myArgs" -WorkingDirectory "myWorkingDir").Id
}

Comment: Well, you assign the process id value to `$procID` and never do anything with it. Remove `$procID =` and the job result should be the process ID of `myprogram.exe`
Comment: I should add that there may be multiple instances of myprogram.exe running; therefore, Get-Process -Name myprogram.exe  | select Id will not work in this case.
Answer: Perhaps you think that assigning to a variable in a script block being executed in a background job makes that variable visible to the caller, which is not the case.
Background jobs run in an entirely separate session, and, in order to communicate data to the caller, the success output stream must be used, whose content the caller must collect via the Receive-Job cmdlet:
$myJob = Start-Job -Name newJob -ScriptBlock {
  # Implicitly output the PID of the asynchronously launched process,
  # by using -PassThru and returning the output object's .Id property value.
  # Note: By default, Start-Process runs the process
  #       *asynchronously*, in a *new window*.
  (
    Start-Process myprogram.exe -PassThru -ArgumentList "myArgs" -WorkingDirectory "myWorkingDir"
  ).Id
}

# Use Receive-Job to collect the background job's output, once available,
# then remove the job automatically.
# Note: The process launched *asynchronously* from the background job
#       with Start-Process lives on.
$procId = Receive-Job -Wait -AutoRemoveJob

Taking a step back: Start-Process is itself asynchronous, so there's no need to use a background job: just start myprogram.exe directly in the caller's context:
# Launch myprogram.exe asynchronously, in a new window.
# $proc receives a value once the program has *launched* and
# your script continues right after.
$proc = Start-Process myprogram.exe -PassThru -ArgumentList "myArgs" -WorkingDirectory "myWorkingDir"

# $proc.Id retrieves the PID (process ID)
# $proc.HasExited tells you whether the process has exited.
# $proc.WaitForExit() allows you to wait synchronously for the process to exit.
#   * If you intend to wait *right away*, i.e run in a new window, but
#     in a blocking fashion, you can omit -PassThru and pass -Wait instead.

However, if myprogram.exe is a console (terminal) application that you want to run asynchronously and whose output you want to capture, do use Start-Job, but don't use Start-Process to launch myprogram.exe: invoke it directly from the background job:
$myJob = Start-Job -Name newJob -ScriptBlock {
  Set-Location "myWorkingDir"
  # Launch console app myprogram.exe *synchronously*, *invisibly*,
  # and relay its (stdout) output as the job's output.
  myprogram.exe
}

# ... do other things

# Wait for myprogram.exe to finish, at which point the
# job exits, relaying its (stdout) output.
$myJob | Receive-Job -Wait -AutoRemoveJob

Getting PID from within a Start-Job


Strategies for transposing in real time

Question: I recently went to a ukulele meet-up to play some music and meet some people and I bought the song book that they use. It consists of mostly pop/rock songs and old folk tunes however, they have all been transposed to keys that are easy to play on the uke (C, F, G, D and, A). I am interested in now playing the songs along with the original records (just strumming the chords at this point) so will need to transpose them. As an exercise I'd like to do it in real time. So far I have tried:

just moving every chord up or down the appropriate interval which is easy for some jumps (up or down a step for example)
analyzing the song with Roman numeral notation and then using that to play in the new key. This is easier if the jump is hard to calculate in my head quickly but is hard for some chords like V/III.

Are there other methods of doing this. I know whatever method I choose will have its pros and cons and they will all take practice but just want to know if there are other options and what the pros and cons are?
Comment: @b3ko - This is different from Roman numeral notation in that you don't need to analyze complete chord progressions--you just need to analyze individual notes. It's like translating the first notes of Beethoven's Symphony No. 5 in C Minor to 5-5-5-(b)3, then figuring out what those notes are in F minor.
Comment: A book on music theory or harmony theory would outline a set of basic approaches to transcription.  Just moving everything up or down is, imo, the easiest to comprehend and apply on the fly.  Especially for guitar.  If you can play without looking at the hand you can read in A and play in D with no effort.  The 2 approaches you have tried are among the standard set.  Another is identifying the degree of each note in the key and finding the same in the new key.
Comment: @ggcg "Another is identifying the degree of each note in the key and finding the same in the new key". How is this different than Roman notation?
Answer: A capo is one way.
Another is to know what the 'family' of chords are in each key. As in F and G in key C are like A♭, B♭ in key E♭, or E and F♯ are in key B. Takes a bit of learning, but when you consider that A>D>E is the same pattern as A♭>D♭>E♭, and C>F>G is the same as C♯>F♯>G♯, there really isn't the whole dozen to go through!
Another is the Blue Peter method - draw two concentric circles, one larger than the other. Mark as on a clockface with twelve note names, consecutively - C, C♯, D, E♭, and so on round each. Join in the centre. Inside disc is written chord, rotate outside one till it matches up with new key. Read each and every one off, inside to outside!
Comment: I'm curious: why do you call this the "Blue Peter method"?
Comment: @Lynn - in Britain, there's a tv programme called 'Blue Peter', part of which features making things for kids from string, tape, cardboard, etc, for next to nothing. Not associated with rolling one's own, or particularly a flag on a ship.
Answer: Another method I have used when transposing a piece to a different key and humming that is to shift the first note(s) up or down the appropriate interval, then use relative pitch (and the sheet music if available) to determine the rest of the notes by preserving the original intervals between them and the previous notes. For example, I can do this with humming the first notes of "O Christmas Tree" in F major by going C-up perfect 4th-same note-same note-up major 2nd-up major 2nd-same note-same note.
This is vulnerable to off-by-one errors (e.g. humming a minor 6th instead of a major 6th), possibly subpar sight-reading skills (e.g. forgetting to check for a key signature), and possibly a poor memory of the original tune (humming filler to get to the portion of the piece I'm familiar with may throw me off).
Answer: Transposing in 2 steps

You can transpose any note in the key of X to the key of Y using
  two simple steps:

transpose the pitch difference in key X
get the enharmonic in key Y

Example
In your example, you have the notes C, F, G, D and, A in the key of C.
Transposing to B
Let's say you want to transpose your music to the key of B. (1) The difference in pitch is -1 since B is a half-step below C. This means we flatten the notes first. (2) This also means we translate the degree of any note by -1 since B is one degree below C.

Cb, Fb, Gb, Db and Ab (transposed pitches in key of C)
B, E, F#, C# and G# (enharmonic degrees in key of B)

Strategies for transposing in real time


Ratio of CDFs $F(x)/x$ property

Question: In my research project is useful to classify cumulative distributions functions of random variables with support in $[a,b]$ with $0\le a<b\le+\infty$ depending on whether the ratio, $$\dfrac{F(x)}{x}$$ is increasing (or decreasing or constant) everywhere in [a,b].
Of course several random variables do not fit in this classification. Also if $0<a$ then the ratio can never be  everywhere constant nor decreasing.  My question is:

Does the property of $F(x)/x$ being increasing has a known name?
As we can interpret this ratio as a ratio of the CDF of our original random variable and the CDF of a Uniform $[0,b]$ if $b<\infty$, is there any known property/name regarding the ratio of two CDFs being increasing?
If yes to 1 or 2, any references?
Comment: I wonder whether you have accurately described the problem, because when $0\lt a$ certainly the ratio can decrease. Take, for instance, $F(x)=1-\exp(-(x-1))$ for $1\le x\lt\infty$: $F(x)/x$ decreases for $x\gt 2.14619$ (approximately). This contradiction with one of your assertions suggests there may be some typographical errors that need to be fixed or at least some clarification of what you mean by "increasing" and "decreasing" would be in order.
Comment: @whuber At zero one should use limits and also we could also allow for $+\infty$ as a value. For example, if $X$ is uniform $[0,b]$ then the ratio $F(x)/x$ is constant; if $X$ has CDF $F(x)=\sqrt{x}$ so $a=0$ and $b=1$ then the ratio is $F(x)/x=\dfrac{1}{\sqrt{x}}$ which is decreasing in $(0,1]$ and if use the extended-real numbers (allowing $F(0)/0=+\infty$) then it is decreasing in $[0,1]$.
Comment: That helps; thank you. It suggests that perhaps you might prefer to analyze $Y=\log(X)$ instead of $X$. If $Y$ has a PDF $g$ and CDF $G$, your decreasing criterion amounts to $G(y)\gt g(y)$ for all $y\lt \log(b)$. I doubt there has ever been interest in this condition holding for *all* $y$, but certainly it is of interest in studying the behavior of the left tail and is relevant to the behavior of extreme (minimal) values.
Comment: @whuber: I (wrongly) assumed it was clear that it was decreasing **everywhere** in the support. Your example, fails to be decreasing near 1.
Comment: Thanks.  I am still puzzled, because the distinction between $0\le a$ and $0\lt a$ seems to be immaterial. You cannot have $F(0)\ne 0$ because $F(0)/0$ would be undefined, whence necessarily $0 \lt a$. That would seem to eliminate your "classification" altogether because it reduces to the single class of everywhere increasing!
Answer: First, a few comments:

As shown in the comment by whuber, the assertion that if $a<0$, then $F(x)/x$ cannot decrease on $[a,b]$ is false. I have no problem with the assertion that $F(x)/x$ cannot be constant.
The CDF of a uniform random variable on [0,b] (with $b<\infty$) is given by $F(x)=\frac xb$. Thus, if $b$ is finite, then $b\cdot \frac{F(x)}x$ is the ratio of the CDFs you mention. Note, however, that $\frac{F(x)}x$ will be increasing/decreasing if and only if $b\cdot \frac{F(x)}x$ is increasing/decreasing. If $b=\infty$,
then there is no such thing as a uniform random variable on $[0,b)$,
hence your intuitive interpretation of the ratio can only make sense when $F$ is supported on a finite interval.

As for some sort of classification,
if you assume the variables you're considering have a density (i.e., $F$ is differentiable),
then coming up with a nice algebraic test to see if $F(x)/x$ increases/decreases shouldn't be too difficult:
Suppose that the density $f(x)=F'(x)$ exists.
According to the quotient rule for differentiation,
one has
$$\left(\frac{F(x)}x\right)'=\frac{F'(x)x-F(x)}{x^2}=\frac{f(x)x-F(x)}{x^2}$$
Given that a differentiable function $g$ is increasing on $[a,b]$ if and only if $g'(x)\geq0$ for all $x\in[a,b]$ and decreasing if and only if $g'(x)\leq0$ for all $x\in [a,b]$,
then we obtain

On $[a,b]$, $F(x)/x$ is

increasing if and only if $f(x)x\geq F(x)$ for every $x\in[a,b]$;
decreasing if and only if $f(x)x\leq F(x)$ for every $x\in[a,b]$; and
constant if and only if $f(x)x=F(x)$ for every $x\in[a,b]$.

Maybe this does not offer an entirely satisfying classification,
but it seems like a good place to start.
Comment: If $a>0$ then $F(a)/a=0$ and $F(b)/b=1/b>0$ so clearly this ratio can not be everywhere constant. It also can not be everywhere decreasing.
Answer: Reading the book "Stochastic Orders" I found the answer. Two random variables $X$ and $Y$ are ranked by the reverse hazard rate,  $X\stackrel{rhr}{\le} Y$, if and only if:
$$ \dfrac{F_X(x)}{F_Y(x)}\text{ is non-decreasing in } [\min(a_X,a_Y),+\infty)$$
where $a_X$ and $a_Y$ are the lower bound of the supports of the random variables.
This is equivalent to $\dfrac{f_Y}{F_Y}\ge \dfrac{f_X}{F_X}$ (which are the respective reverse hazard rates).
The classification above is similar (but not entirely identical) to ranking the the uniform $[0,b]$ and the original random variable according to the reverse hazard rate order.
Comment: It is unclear how this is related to your $F(x)/x$ condition in any more than a rough, qualitative way. Could you be more explicit about that?
Comment: @whuber If Y is uniform $[0,b]$ and $F=F_X$ is the CDF of X then $F(x)/x$ is proportional to $F_X(x)/F_Y(x)$. In the case $a=0$: $F(x)/x$ increasing means $X$ is dominated by Y in the reverse hazard rate; $F(x)/x$ decreasing means $Y$ is dominated by $X$ in the reversed hazard rate.

Ratio of CDFs $F(x)/x$ property


How to build a Dockerfile with a Testcontainers?

Question: I don't know Docker that much and maybe I'm doing something wrong? But everything seems to follow the instructions: https://java.testcontainers.org/supported_docker_environment/continuous_integration/dind_patterns/
When building locally, everything goes fine. But when I try to build a docker image, an error pops up:
2023-10-21T06:35:22.415+0000 [DEBUG] [TestEventLogger]
2023-10-21T06:35:22.416+0000 [DEBUG] [TestEventLogger]       .   ____          _            __ _ _
2023-10-21T06:35:22.416+0000 [DEBUG] [TestEventLogger]      /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
2023-10-21T06:35:22.416+0000 [DEBUG] [TestEventLogger]     ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
2023-10-21T06:35:22.417+0000 [DEBUG] [TestEventLogger]      \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
2023-10-21T06:35:22.417+0000 [DEBUG] [TestEventLogger]       '  |____| .__|_| |_|_| |_\__, | / / / /
2023-10-21T06:35:22.418+0000 [DEBUG] [TestEventLogger]      =========|_|==============|___/=/_/_/_/
2023-10-21T06:35:22.419+0000 [DEBUG] [TestEventLogger]      :: Spring Boot ::                (v3.1.4)
2023-10-21T06:35:22.420+0000 [DEBUG] [TestEventLogger]
2023-10-21T06:35:22.691+0000 [DEBUG] [TestEventLogger]     2023-10-21T06:35:22.690Z  INFO 231 --- [ers-lifecycle-0] .t.d.DockerMachineClientProviderStrategy : docker-machine executable was not found on PATH ([/opt/java/openjdk/bin, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin])
2023-10-21T06:35:22.691+0000 [DEBUG] [TestEventLogger]     2023-10-21T06:35:22.691Z ERROR 231 --- [ers-lifecycle-0] o.t.d.DockerClientProviderStrategy       : Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
2023-10-21T06:35:22.692+0000 [DEBUG] [TestEventLogger]          UnixSocketClientProviderStrategy: failed with exception InvalidConfigurationException (Could not find unix domain socket). Root cause NoSuchFileException (/var/run/docker.sock)
2023-10-21T06:35:22.692+0000 [DEBUG] [TestEventLogger]          DockerDesktopClientProviderStrategy: failed with exception NullPointerException (Cannot invoke "java.nio.file.Path.toString()" because the return value of "org.testcontainers.dockerclient.DockerDesktopClientProviderStrategy.getSocketPath()" is null)As no valid configuration was found, execution cannot continue.
2023-10-21T06:35:22.692+0000 [DEBUG] [TestEventLogger]     See https://java.testcontainers.org/on_failure.html for more details.
2023-10-21T06:35:22.777+0000 [DEBUG] [TestEventLogger]     2023-10-21T06:35:22.773Z ERROR 231 --- [    Test worker] o.s.boot.SpringApplication               : Application run failed
2023-10-21T06:35:22.777+0000 [DEBUG] [TestEventLogger]
2023-10-21T06:35:22.777+0000 [DEBUG] [TestEventLogger]     java.util.concurrent.CompletionException: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration

My Dockerfile:
ARG BUILD_HOME=/inquiry-service-example
ARG DEFAULT_APP_NAME=inquiry-service-0.0.1-SNAPSHOT-boot

FROM gradle:jdk17 as build-image

ARG BUILD_HOME
ARG DEFAULT_APP_NAME
ENV APP_HOME=$BUILD_HOME
WORKDIR $APP_HOME

COPY build.gradle $APP_HOME
COPY settings.gradle $APP_HOME
COPY src $APP_HOME/src
COPY config $APP_HOME/config

ENV ESTCONTAINERS_HOST_OVERRIDE=host.docker.internal
STOPSIGNAL SIGKILL
VOLUME /var/run/docker.sock:/var/run/docker.sock

RUN gradle build --debug

FROM openjdk:17

ARG BUILD_HOME
ARG DEFAULT_APP_NAME
ENV APP_HOME=$BUILD_HOME
ENV APP_NAME=$DEFAULT_APP_NAME

COPY --from=build-image $APP_HOME/build/libs/$APP_NAME.jar app.jar

EXPOSE 8080 8443

ENTRYPOINT java -jar app.jar

My test class:
package com.iprody.inquiryservice;

import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.lifecycle.Startables;

@SpringBootTest(
        webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
        properties = {

        }
)
public abstract class AbstractIntegrationTest {

    private static final int POSTGRES_SQL_PORT = 5432;
    private static final PostgreSQLContainer<?> POSTGRES_SQL_CONTAINER;

    static {
        final String root = "root";
        POSTGRES_SQL_CONTAINER = new PostgreSQLContainer<>("postgres:alpine")
                .withExposedPorts(POSTGRES_SQL_PORT)
                .withPassword(root)
                .withUsername(root)
                .withReuse(true);
    }

    @DynamicPropertySource
    static void postgresProperties(DynamicPropertyRegistry registry) {
        Startables.deepStart(POSTGRES_SQL_CONTAINER).join();

        registry.add("spring.datasource.url", POSTGRES_SQL_CONTAINER::getJdbcUrl);
        registry.add("spring.datasource.username", POSTGRES_SQL_CONTAINER::getUsername);
        registry.add("spring.datasource.password", POSTGRES_SQL_CONTAINER::getPassword);
    }
}

I thought there was something wrong with Windows, but it's the same error on Linux.
Answer: I found your post because I have the similar problem with "Could not find a valid Docker environment". I have exactly the same info as you with docker-machine executable was not found etc. In your dockerfile, there's a typo. Instead of ESTCONTAINERS_HOST_OVERRIDE you should have TESTCONTAINERS_HOST_OVERRIDE. I have correct variable name but it still does not work.


Is there a way to change android phone's mtu size?

Question: My phone is having problem with sending data to the network. I think I have to change its MTU size to make it work.
Answer: To change MTU settings you need a to root your phone. Search in foruns how to do it.
After that your need to install a terminal emulator and check what is your network interface
Run the command:
netcfg

and look at the IP address what is your desired device (wlan0, tiwlan0, etc)
then use this command
ifconfig $DEVICE mtu $MTU_VALUE

and substitute $DEVICE with device name and the desired MTU value in $MTU_VALUE
You can also see the value at this file:
cat /sys/class/net/$DEVICE/mtu

Comment: `error: SIOCSIFMTU (Operation not permitted)` <-- How to fix?
Comment: @L'Esperanza you need to root your device, otherwise it won't work. check the first sentence.
Comment: This is temporary. If you restart the device which you configured its MTU setting, the setting changes to default (e.g. 1500 for me). There would be a way to automatically run the ifconfig command every time a specific device restarts or to set something like Cron jobs. Or, is there any way to make the MTU values fixed?
Comment: @PauloFidalgo Change "To change MTU settings you need a to root your phone. Search in foruns how to do it. After that your need to install a terminal emulator and check what is your network interface Run the command:" to "To change MTU settings you need to root your phone. Search in foruns how to do it. After that your need to install a terminal emulator and check what is your network interface Run the command:"


Meaning of tcpdump P direction

Question: The situation is a VM running under kvm/libvirt/qemu (10.202.7.128) pinging another hardware-based system (10.202.7.4) via the hypervisor's bridge.  Everything works OK but 'tcpdump -neli any icmp and host 10.202.7.128' shows three entries per request:
14:42:39.921785 P 52:54:00:14:71:1b
ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 2485, seq 1, length 64

The second entry is identical except the P is replaced with "Out", the third entry is identical to the first.  This is followed by three replies from the same MAC address for 10.202.7.4 with the only difference being the P, Out, P sequence.
Why are there three entries per ping and what does the P in the direction mean?  I've searched the web in vain unless I searched for the wrong thing.
Update:
Based on Zoredache's answer I clanged the interface specification to the three involved: vnet5 (the guest VM's NIC on the hypervisor), br0 (the hypervisor's bridge) and eno1 (the physical NIC identified as the bridge port for the bridge) and got one request/reply for each ping when testing each interface so I'm (wildly) guessing that's the reason for the three entries using "any".  I'm still baffled about the P and sequence of P, Out, P.
Any insight?
Further update (and I want to thank Anton for his research, much appreciated.)
Ran test again with four tcpdump specifications, all '-neli <interface> icmp and host 10.202.7.128' with interfaces vnet5, br0, eno1 and "any".  Noted timestamps to determine what "any" entry matched the individual interface entries.
"Any" results (vnet5/10.202.7.128 is 52:54:00:a2:bd:48, 10.202.7.4 is b0:00:b4:02:93:6e)
00:06:57.775050   P 52:54:00:a2:bd:48 ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775074 Out 52:54:00:a2:bd:48 ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775050   P 52:54:00:a2:bd:48 ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775940   P b0:00:b4:02:93:6e ethertype IPv4 (0x0800), length 100: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64
00:06:57.775956 Out b0:00:b4:02:93:6e ethertype IPv4 (0x0800), length 100: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64
00:06:57.775940   P b0:00:b4:02:93:6e ethertype IPv4 (0x0800), length 100: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

vnet5 results
00:06:57.775050 52:54:00:a2:bd:48 > b0:00:b4:02:93:6e, ethertype IPv4 (0x0800), length 98: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775956 b0:00:b4:02:93:6e > 52:54:00:a2:bd:48, ethertype IPv4 (0x0800), length 98: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

br0 results
00:06:57.775050 52:54:00:a2:bd:48 > b0:00:b4:02:93:6e, ethertype IPv4 (0x0800), length 98: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775940 b0:00:b4:02:93:6e > 52:54:00:a2:bd:48, ethertype IPv4 (0x0800), length 98: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

eno1 results
00:06:57.775074 52:54:00:a2:bd:48 > b0:00:b4:02:93:6e, ethertype IPv4 (0x0800), length 98: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775940 b0:00:b4:02:93:6e > 52:54:00:a2:bd:48, ethertype IPv4 (0x0800), length 98: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

The 775050 timestamp corresponds to vnet5 and br0, the expected first "hop" (vnet5 to br0).  The 775074 corresponds to eno1 which is the expected second "hop" (br0 to eno1).  I have no explanation for the repeat of the second 775050 entry because it is still a request.
The 775940 timestamp corresponds to eno1 and br0, the expected first "hop" of the reply (eno1 to br0).  The 775956 timestamp corresponds to vnet5, the expected second "hop" of the reply (br0 to vnet5).  Again, I have no explanation for the repeat of the second 775940 entry because it is still a reply.
Final note, both "Out" entries refer to br0 transmitting to an interface (eno1 on outbound, vnet5 on inbound).
If anyone has an explanation for the repeat of the two entries please respond.
Comment: Are you running `tcpdump` on the hypervisor or inside the guest? Also, is the second guest running on the same hypervisor? Are the two guests and/or the hypervisor in the same subnet?
Comment: tcpdump is being run on the hypervisor while the guest is pinging, the source, destination and hypervisor are on the same subnet.
Answer: I've looked in the source code of the tcpdump.
Meanings of the packet type field:

In - packet addressed to host
Out - outgoing packet
B - broadcast
M - multicast
P - packet addressed to other host

Update
Duplicated packets in the tcpdump output caused by -i any option.
When a ethernet frame passed through linux bridge and you use -i any option of the tcpdump, you will see this frame several times. First time you see the frame on the input interface (vnet5), second time on the bridge interface itself (br0), and third time on the output interface (eno1). Seeing transit frames on the bridge interface is the side effect of implementation of ethernet bridging in the linux kernel.
Also, when a bridge interface have several ports (slave interfaces), in some cases you see copies of frames on every port, when an ethernet frame is flooded. It happens for broadcast, multicast and unknown unicast frames.
Answer: Most of your duplication is caused by the -i any option you have passed.
When you do that you tcpdump will capture on every interface, and in the case of a router, or a bridge or something where a packet is being forwarded between interfaces it will return a copy of the packet for every interface that it hits.
I am not very familiar with KVM, but I believe if you have 2 VMs connected by a bridge interface you would have on virtual interface per VM, and the bridge interface.
Anyway, if you don't want duplication, restrict tcpdump to a single interface, by providing the name of a specific interface to the -i option. Though for some problems you may actually want all interfaces, and in that case you just have to understand that you are going to see duplication.

Meaning of tcpdump P direction


Convert Map> to Map> using streams

Question: I have structure map of maps like:
Map<Center, Map<Product, Value>> given

and I want to get
Map<Product, Map> result

I've used Java streams
Map<Product, Map> result = given.entrySet().stream()
        .flatMap(entry -> entry.getValue()
                 .entrySet().stream()
                 .map(e -> Triple(entry.getKey(), e.getKey(), e.getValue())))
        .collect(Collectors.groupingBy(Triple::getProduct,
                    Collectors.toMap(Triple::getCenter, Triple::getValue)));

where Triple is simple value class. My questions is if it is possible to do it functional without using additional classes like Triple or e.g. Table from guava?
Comment: Welcome to Stack Overflow!  Please take the [tour] and visit our [help] to learn what kinds of questions are appropriate for this site.  If you can [edit] your question to fit the requirements of this site, please do so.
Comment: why exactly do you want to avoid mapping to an intermediate object i.e. `Triple`? seems just fine as is.
Comment: Because I'm creating this class only to use in two lines. I think is no bad, but maybe it is possible better :) I can use `Tuple` for vavr.io too.
Answer: Some things are easier done without streams:
Map<Product, Map> result = new HashMap<>();
given.forEach((c, pv) -> pv.forEach((p, v) ->
        result.computeIfAbsent(p, k -> new HashMap<>()).put(c, v)));

Answer: Unfortunately, if you want to proceed with your stream approach it's unavoidable to create some type of intermediate object i.e. Triple, or AbstractMap.SimpleEntry or any other type applicable.
You're essentially looking for something like C#'s anonymous types i.e. you could just map to
new { k1 = entry.getKey(), k2 = e.getKey(), k3 = e.getValue()) }

and then immediately access those in the groupingBy and toMap phase.
Java has something similar but not quite i.e. you could do:
Map<Product, Map> result =
           given.entrySet()
                .stream()
                .flatMap(entry -> entry.getValue()
                        .entrySet().stream()
                        .map(e -> new Object() {
                            Center c = entry.getKey();
                            Product p = e.getKey();
                            Value v = e.getValue();
                        }))
                .collect(Collectors.groupingBy(o -> o.p, Collectors.toMap(o -> o.c, o -> o.v)));

credit goes to @shmosel.
The only benefit here being you don't need to predefine a custom class.
Comment: It's not true though. The anonymous type is inferred and you can access its fields.
Comment: @shmosel something new learnt today :). my first time actually using this syntax in java `new Object() { .... }` so all credit goes to you. thanks.
Comment: Why do you say there's no type inference?
Comment: @shmosel probably the wrong wording there. edited to be more specific.
Comment: @shmosel not that i know of, so would be cool to be shown how you can group by `k2` and access `k1` and `k3` in the `toMap`

Convert Map> to Map> using streams