Gmail API decoding messages in Javascript

Question: I am having serious problems decoding the message body of the emails I get using the Gmail API. I want to grab the message content and put the content in a div. I am using a base64 decoder, which I know won't decode emails encoded differently, but I am not sure how to check an email to decide which decoder to use -- emails that say they are utf-8 encoded are successfully decoded by the base64 decoder, but not be a utf-8 decoder.
I've been researching email decoding for several days now, and I've learned that I am a little out of my league here. I haven't done much work with coding around email before. Here is the code I am using to get the emails:
gapi.client.load('gmail', 'v1', function() {
var request = gapi.client.gmail.users.messages.list({
  labelIds: ['INBOX']
});
request.execute(function(resp) {
  document.getElementById('email-announcement').innerHTML = 'Hello! I am reading your inbox emails.

------
';
  var content = document.getElementById("message-list");
  if (resp.messages == null) {
    content.innerHTML = "Your inbox is empty.";
  } else {
    var encodings = 0;
    content.innerHTML = "";
    angular.forEach(resp.messages, function(message) {
      var email = gapi.client.gmail.users.messages.get({
      'id': message.id
      });
      email.execute(function(stuff) {
        if (stuff.payload == null) {
          console.log("Payload null: " + message.id);
        }
        var header = "";
        var sender = "";
        angular.forEach(stuff.payload.headers, function(item) {
          if (item.name == "Subject") {
            header = item.value;
          }
          if (item.name == "From") {
            sender = item.value;
          }
        })
        try {
          var contents = "";
          if (stuff.payload.parts == null) {
            contents = base64.decode(stuff.payload.body.data);
          } else {
            contents = base64.decode(stuff.payload.parts[0].body.data);
          }
          content.innerHTML += 'Subject: ' + header + '
';
          content.innerHTML += 'From: ' + sender + '
';
          content.innerHTML += contents + "

";
        } catch (err) {
          console.log("Encoding error: " + encodings++);
        }
      })
    })
  }
 });
});

I was performing some checks and debugging, so there's leftover console.log's and some other things that are only there for testing. Still, you can see here what I am trying to do.
What is the best way to decode the emails I pull from the Gmail API? Should I try to put the emails into


Variance/standard deviation versus interquartile range (IQR)

Question: Variance and interquartile range (IQR) are both measures of variability.
But IQR is robust to outliers, whereas variance can be hugely affected by a single observation.
Since variance (or standard deviation) is a more complicated measure to understand, what should I tell my students is the advantage that variance has over IQR?
Comment: One candidate for advantages of variance is that every data point is used.
Comment: Variance isn't of much direct use for visualizing spread (it's in squared units, for starters -- the standard deviation is more interpretable, since it's in the original units -- it's a particular kind of generalized average distance from the mean), but variance is very important when you want to work with sums or averages (it has a very nice property that relates variances of sums to sums of variances plus sums of covariances, so standard deviation inherits a slightly more complex version of that. IQR doesn't share that property at all; nor mean deviation or any number of other measures)
Comment: I don't think thinking about advantages will help here; they serve mosstly different purposes.
Answer: The main use of variance is in inferential statistics.
So, variance and standard deviation are integral to understanding z-scores, t-scores and F-tests.
This means that when your data are normally distributed, the standard deviation is going to have specific properties and interpretations. When your data are not normal (skewed, multi-modal, fat-tailed,...), the standard deviation cannot be used for classicial inference like confidence intervals, prediction intervals, t-tests, etc., and cannot be interpreted as a distance from the mean.
You can say things like "any observation that's 1.96 standard deviations away from the mean is in the 97.5th percentile." if your data are normally distributed.
Answer: There are several advantages to using the standard deviation over the interquartile range:
1.) Efficiency: the interquartile range uses only two data points, while the standard deviation considers the entire distribution. If you are estimating population characteristics from a sample, one is going to be a more confident measure than the other*.
2.) Meaning: if you data is normally distributed, the mean and standard deviation tell you all of the characteristics of the distribution. The interquartile range doesn't really tell you anything about the distribution other than the interquartile range.
3.) If you are willing to sacrifice some accuracy for robustness, there are better measures like the mean absolute deviation and median absolute deviation, which are both decent robust estimators of variation for fat-tailed distributions.
4.) Better yet, if you distribution isn't normal you should find out what kind of distribution it is closest to and model that using the recommended robust estimators.
*It's important here to point out the difference between accuracy and robustness. Standard deviation is never "inaccurate" per ce, if you have outliers than the sample standard deviation really is very high. "Outliers" usually means either data that you're not certain is legitimate in some sense or data that was generated from a non-normal population.
TL;DR don't tell you're students that they are comparable measures, tell them that they measure different things and sometimes we care about one and sometimes we care about the other. Tell them to think about what they are using the information for and that will tell them what measures they should care about.
Answer: A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the values are spread out over a wider range1. The standard deviation and mean are often used for symmetric distributions, and for normally distributed variables about 70% of observations will be within one standard deviation of the mean and about 95% will be within two standard deviations(68–95–99.7 rule). For non-normally distributed variables it follows the three-sigma rule.
As shown below we can find that the boxplot is weak in describing symmetric observations.

Generated by this snippet of R code(borrowed from this answer):
set.seed(1)
normal <- rnorm(10000)
a_vector <- c(-3, -2.65, rep((-2:2)*.674, 5), 2.65, 3)
boxplot(normal, a_vector)

We can see that the IQR is the same for the two populations 1 and 2 but we can see the difference of the two by their means and standard deviations.
mean(normal); var(normal); mean(a_vector); var(a_vector)

-0.00653703946166382
1.02486558733286
-3.0626842058625e-17
1.95567142857143

We can see from the above case that what median and IQR cannot reflect can be obviously conveyed by the mean and variance.
References:
1. https://en.wikipedia.org/wiki/Standard_deviation
Comment: This post is flawed. First, the standard deviation does not represent a typical deviation of observations from the mean. That would be the mean absolute deviation, $\frac{1}{n}\sum\big\vert x_i-\bar{x}\big\vert$. Second, what you're saying about 70% of the points being within one standard deviation and 95% of the points being within two standard deviations of the mean applies to normal distributions but can fail miserably for other distributions. Chebyshev's inequality bounds how many points can be $k$ standard deviations from the mean, and it is weaker than the 68-95-99.7 rule for normality.
Comment: Finally, the IQR is doing exactly what it advertises itself as doing. Your plot on the right has less variability, but that's because of the lower density in the tails. One (evidently weak) way to judge kurtosis differences is to take the ratio of the variance and the IQR. If you have a lot of variance for an IQR, high tail density could explain that.
Comment: Why do you say that it applies to non-normal distributions?
Comment: @Dave Sorry for the mistakes I made, and thank you for pointing out the error. I have updated the answer and will update it again after learning the kurtosis differences and Chebyshev's inequality.
Comment: All generalisations are dangerous (including this one). For example, distributions that are, or are close to, Poisson and exponential are always skewed, often highly, but for those mean and SD remain natural and widely used descriptors. (The SD is redundant if those forms are exact.)


Parameter estimate for linear regression with regularization

Question: For given cost function $S(\beta) = (Y - X \beta)^T(Y - X \beta) + \lambda \beta^T \beta$, where $\lambda$ is regularization parameter, the $\beta$ that minimizes the given cost function is $\beta = [X^T X + \lambda]^{-1} X^T Y$.
is it right?
Answer: This is known as ridge regression in statistics (which is a handy search term). You need an $I$ after the $\lambda$ in your last equation.
Is this work for some subject?
Comment: I am trying to understand the regularization parameter for linear regression. And not sure how to take the derivative $\frac{\partial }{\partial \beta} \lambda \beta^T \beta$. Is it correct that answer should be $\beta = [X^T X + \lambda I]^{-1} X^T Y$
Comment: Well, that's for formula for $\hat{\beta}$. If you're unfamiliar with derivates with respect to a vector, you can compute the relevant derivative term by term. Note that $\lambda\beta^T\beta$ is scalar. Take the derivative of that with respect to $\beta_i$, and collect the terms up into a vector of the same dimensions as $\beta$, to obtain $2\lambda\beta$.


System of coupled differential equations

Question: I have the following system of three linear differential equations that I need to solve numerically using finite diferences:
$$\begin{aligned}
\frac{dn_0}{dt} &= -n_0(t)W_{01}(t) + n_1(t)K_{10}\\
\frac{dn_1}{dt} &= -n_1(t)W_{12}(t) - n_1(t)K_{10} + n_2(t)K_{21} + n_0(t)W_{01}(t)\\
\frac{dn_2}{dt} &= n_1(t)W_{12}(t) - n_2(t)K_{21}
\end{aligned}$$
where the boundary conditions are
$$ n_0(0)=1 $$
$$ n_0(N)=0 $$
$$ n_1(0)=0 $$
$$ n_1(N)=1 $$
$$ n_2(0)=0 $$
$$ n_2(N)=0 $$
The three equations are restricted by $n_0 + n_1 + n_2 =1$, I tried to replace and write the equations, in an attempt to
$$\frac{dn_0}{dt} + \frac{dn_1}{dt} +\frac{dn_2}{dt} = 0$$
Since I can not use the finite difference method for coupled equations,thinking that as a solution the values of the derivatives should be equal to zero, so I could just rewrite one of the equations by isolating the variables, but I'm not sure about that. Does anyone know a way or some technique that I can decouple the 3 equations?
Comment: Since the equations are linear, have you considered trying to solve them analytically?
Comment: Why can't you use finite difference for that system? It looks like $W's$ are variables as well, what are they?
Comment: $W=A{e^{-(t-t0)^2}}$  I tried solving with finite differences by solving each equation at a time, but failed to eliminate the coupled terms as you did
Answer: The system is a linear time-variant system. We can write it in matrix form as
$$\dfrac{d}{dt}\begin{bmatrix}n_0 \\ n_1\\ n_2 \end{bmatrix}=
\begin{bmatrix}
-W_{01}(t) & K_{10} & 0\\
W_{01}(t) & -K_{10}-W_{12}(t)&K_{12}\\
0 & W_{12}(t) & -K_{21}\\
\end{bmatrix}\begin{bmatrix}n_0 \\ n_1\\ n_2 \end{bmatrix}.$$
Then we have $n_0=1-n_1-n_2$. This invariant allows us to remove the line of the equation. Because if we know $n_1$ and $n_2$ we can simply calculate $n_0$. If we substitute this invariant into the second and third line we obtain.
$$\dfrac{d}{dt}\begin{bmatrix} n_1\\ n_2 \end{bmatrix}=
\begin{bmatrix}
-K_{10}-W_{12}(t)-W_{01}(t) &K_{12}-W_{01}(t)\\
W_{12}(t) & -K_{21}\\
\end{bmatrix}\begin{bmatrix} n_1\\ n_2 \end{bmatrix}+\begin{bmatrix}W_{01}(t)\\0 \end{bmatrix}.$$
Can you supply more information on $W_{12}(t)$ and the other terms? In general, it will be problematic to help you to decouple both equations. But you should be able to solve this coupled system without any problems with numerical solvers for differential equations.
Comment: $W12=A{e^{-(t-t0)^2}}$ and $W10=B{e^{-(t-t0)^2}}$. k10, k12 and A and B are constants.

I did not quite understand what you did to remove n0. What exactly did you do?
Comment: And what is the goal that you want to achieve? Just obtain numerical solutions? Or do you want to carry out a stability analysis of the system?
Comment: Yeah, I want to check how n0, n1 and n2 behave in the system over time. I would need the numerical solution to do other types of simulation.

I tried solving with finite differences by solving each equation at a time, but failed to eliminate the coupled terms as you did
Comment: You should be able to solve this differential equation by using Euler forward. As long as your step size is small enough and the simulation time is not too long you should get reasonable results. Why do you want to use finite difference?
Comment: is the method that I am most familiar with. But I think it will be more difficult with him. Thank you anyway


How many $x$ for which $ \tan x=x-x^3$?

Question: What are the number of solutions of
$$
\tan x=x-x^3
$$
where $x$ belongs to $(-1,1)$?
Comment: Notice that $x=0$ is a solution. Notice further that the function $x \mapsto \tan{x} -(x-x^3)$ is monotonically increasing on $(-1,1)$ (why?). Conclude that...
Answer: There are an infinite number of solutions over the natural domain of tan.
Thus we must focus on real solutions over $(-1,1)$. Here is a generalised approach to such problems.
One such and obvious solution is $x = 0$, but any others must be deduced using the bisection method. The intermediate value theorem states that if $f$ is a continuous function over $[a,b]$, there exists a $c$ in $(a,b)$ such that $f(c) = z$ for some $z$ in $(f(a),f(b))$ (if $f(a) < f(b)$, or $(f(b),f(a))$ if $f(a) > f(b)$. Thus we can say there is a solution, $c$ such that $f(c) = 0$ for some $c$ in $(a,b)$ where $f(a) < 0$ and $f(b) > 0$ or vice versa.
For instance let us consider if there is (at least one) a solution in the interval $[0.01,1] $.
Let $f(x) = \tan (x) - x - x^3$ and compute
$f(0.01)$ and $f(1)$ say.
If there is a sign difference between these, by IVT, there is a least one real solution in that interval. You can split the interval further to see if there are more roots in that interval (or more accurately, define the bounds for roots). Also helpful tip: for every root in the interval $(0,1)$ there must be a root in $(-1,0)$ given $f$ is odd.
Of course noting the graph of $f$ say in Wolfram Alpha (after you do your calculations) can help to verify your solutions.
N.B. Non existent sign change does not necessarily imply there are NO zeros over the chosen interval. This just means we cannot guarantee there will be a zero (or in fact no zeros) over the interval unless we sufficiently prove $f(x) < 0$ or $f(x) > 0$ for all $x$ in $(a,b)$. In some cases, real roots only become obvious when we take an appropriate subset of the interval.

How many $x$ for which $ \tan x=x-x^3$?


Method for estimating (thermodynamic) scaling fit parameters

Question: Let's say I have some measurements, label them $N$, which depend on the variables $x$ and $y$.
Further, I hypothesize that there is a scaling relationship between $N$, $x$, and $y$ of the form $x^{\alpha}N = f(x^{\beta}(y-\gamma))$, where $\alpha$, $\beta$, and $\gamma$ are unknown parameters and $f$ is some unknown function.
If I have many values of $N$, recorded for many values of $x$ and $y$, is there a way to estimate the values of the parameters $\alpha$, $\beta$, and $\gamma$? I don't really care what the function $f$ is.
I'm sorry if this is unclear, or simplistic; I'm embarrassed to admit I know very little statistics.
It would be extremely helpful if you know of an existing computational library (e.g. an R package) that would be helpful here, too.
Thanks!
Answer: First of all, I'd start by restating your hypothesized relationship in the more "conventional" format, i.e., with the dependent variable $N$ appearing on one side of the equation, and all of the independent variables and unknown parameters on the other side, like so: $$N = g(x,y;\alpha,\beta,\gamma) = \frac{f(x^{\beta} (y-\gamma))}{x^{\alpha}}$$
Restated in this fashion, you have a classic nonlinear regression problem (i.e., the mathematical model is "nonlinear" in the parameters $\alpha$, $\beta$, and $\gamma$, because some of these parameters appear as exponents within the model equation--not to mention the fact that the function $f$ itself may also be nonlinear as well).
There is a package in R called nls which is designed to solve problems of this type.  If you are actually working on some kind of experimental physics problem (which I surmise based on the appearance of the word "thermodynamics" in the question title), please be aware that the international academic physics community also has independently developed its own computation package, minuit, precisely for addressing nonlinear parameter estimation problems such as this.  I believe that MATLAB has tools for this class of problem as well, e.g., lsqnonlin or LFMsolve.
For a good and very approachable introduction to non-linear regression for the physical sciences, I recommend chapter 8 of "Data Reduction and Error Analysis for the Physical Sciences", by Philip R. Bevington and D. Keith Robinson.  However, the material is built extensively on ideas first introduced in previous chapters, so if you want to really "get it," you're probably going to have to read a sizeable preceding chunk of the book.
Comment: This is very helpful, thank you @stachyra. Do these methods always require that you make some assumption about the form of the function $f$, for instance that it's a polynomial? Is it possible to get estimates of the parameters without having to determine $f$? For instance, rather than minimizing the distance between the data points and some prediction, could one minimize the distance between the data themselves?
Comment: First things first: if you found a particular answer helpful, it's customary to upvote it.  If you found an answer definitive, you may, at your discretion, award it "best answer" status, though that's really up to your own discretion.
Comment: Sorry, argghh, comment editor cut me off for going over 5 minute timer mark. To continue my prior response, as far as I know, the purpose of regression methods is always to compare a model or prediction to a measurement, so my bias is to say no, I don't see how you could set up the problem to minimize the distance between the data and themselves. And since you're trying to make a specific numerical prediction, yes, you do need to make a concrete assumption about the form of $f$.
Comment: That's what I thought. Thank you for the great recommendations, and the help on non-linear regression.


Manage Display page not displaying anymore

Question: I set (or at least tried to) my user display page (admin/config/people/accounts/display) to use DS 1col on friday and part way through the request my computer crashed out (the server is running on the local machine too) and now I can't seem to access that page if I let it run forever I get:

Fatal error: Maximum execution time of 120 seconds exceeded in /Applications/MAMP/htdocs/modules/field_ui/field_ui.admin.inc on line 145

I've tried to:

Clear caches
Run cron
Disable and re-enable DS

Although I'm continuing to play with DS, I'm assuming whatever got corrupted is related to the field_ui module because of the error and it only seems to be affecting the manage display for the user account page; no other user account pages or manage display pages are affected.
Does anyone know if there is a simple way to reset that page to the default?
Answer: after playing around in the database for awhile and trying things (I backed it up first) I found the problem in the field_group table by deleting all the default user groups I can now see and edit the manage display page for the account


FactoryGirl RSpec No method error when creating factory object where factory sets a property of that object

Question: Anytime I create a Factory object, I get this error where the factory sets a property on that object -- in this case 'name'. Why???
businesses.rb
FactoryGirl.define do
  factory :business do

    factory :business_main do
      name "Dave's Toys"
    end
  end
end

businesses table
        Column        |            Type             |                        Modifiers
----------------------+-----------------------------+---------------------------------------------------------
 id                   | integer                     | not null default nextval('businesses_id_seq'::regclass)
 name                 | character varying(255)      |
 address_line1        | character varying(255)      |
 address_line2        | character varying(255)      |
 city                 | character varying(255)      |
 state                | character varying(255)      |
 zip                  | character varying(255)      |

business_spec.rb
...
business  = Factory(:business_main)
...

 # Failure/Error: business  = Factory(:business_main)
 # NoMethodError:       undefined method `name=' for #<Business:0x007fc991684590>

-
Using rails (3.1.3)
Using factory_girl (2.3.2)
Using factory_girl_rails (1.4.0)
Using rspec-core (2.7.1)
Using rspec-expectations (2.7.0)
Using rspec-mocks (2.7.0)
Using rspec (2.7.0)
Using rspec-rails (2.7.0)

Comment: Ack, I hadn't run my migrations in the test database!
Comment: it's pretty strange. how do you run your specs?
Comment: What are you trying to accomplish by nesting `business` and `business_main` factories like that?
Comment: @99miles, yeah, it was my first guess
Answer: Run rake db:test:clone and then all test DB columns will be mapped properly to Active Record methods.


When are there no critical points?

Question: Is there ever a time when there are no critical points of a function?
For example, I am trying to find the critical points and the extrema of $\displaystyle f(x)= \frac{x}{x-3}$ in $[4,7]$
I am not finding any critical points in this, is this correct?
Comment: the Weierstrass function
Comment: @Gahawar: Wouldn't that function have a critical point everywhere? Since there exists no derivative?
Comment: In the calculus textbook I used, critical points were any point $x = c$ such that $f'(c) = 0$ or $f'(c)$ does not exist. By this definition, the Weierstrass function has critical points at every real number.
Comment: @JimmyK4542 The definition that I am familiar with is the former condition.
Comment: The terminology isn't standardized. Some authors only include domain points where the derivative vanishes, some also include domain points where the derivative doesn't exist, and some include boundary points. (This is all in the context of real-valued functions of a real variable.) I learned it as the "intermediate" definition: a critical point is a domain point where the derivative vanishes or fails to exist. Then local extrema occur at critical points or boundary points.
Answer: Plenty. For example $f(x)=x$ has no critical points. Neither does $f(x)=e^x$.
And your function has no critical points, according to many definitions. Some definitions would include endpoints among the critical points. In that case, if we consider the function as having domain $[4,7]$, you have $4$ and $7$.
The function is continuous on the interval $[4,7]$. So it does attain a maximum and a minimum in the interval. Since the derivative is defined everywhere in the interval, and nowhere $0$, the endpoints $4$ and $7$ are your only candidates for maximumhood/minimumhood.
Answer: Note that $f(x) = \dfrac{x}{x-3} = 1 + \dfrac{3}{x-3}$. So, $f'(x) = -\dfrac{3}{(x-3)^2} < 0$ for all $x \neq 3$,
Hence, there are no critical points on $(4,7)$.

When are there no critical points?


How to create a random race track for SpriteKit?

Question: I want to create a racing game with a racetrack that gets generated randomly.
Here is a sketch of how I want it to be:
Racetrack sketch
I want the racetrack to be a physics body. While the racetrack moves to the left side of the screen it changes it gets generated on the right screen but with random directions(curves) like displayed in the sketch.
If the car at some point touches the edge of the road the game ends.
The only thing I need help with right now is how to generate the road as described above.
Answer: If you make the racetrack a physics body, the car will be a physics body too I suppose, moving in a physics body. It sounds a bit counter-intuitive to how Spritekit is designed. You could consider making the racetrack sides (upper curve and lower curve) edges. Then have the car move there in the middle, if it touches you should get a collision.
I did something similar recently (did a Dune!-like ball jumping game on a curvy landscape) and approached so that you randomise a set of y-coordinates in your given range, e.g. something like
<code>    let yPointRandomizer = GKGaussianDistribution(lowestValue: -500, highestValue: -200)
    var highGroundPositions = [CGPoint(x: 0, y: yPointRandomizer.nextInt())]
    for i in 1...50 {
        highGroundPositions.append(CGPoint(x: (i * 400) + 200, y: yPointRandomizer.nextInt()))
    }
</code>
What this gives you is are the coordinates that the curve goes to. Then to draw the shape and get an edge:
<code>let splineShapeNode = SKShapeNode(splinePoints: &highGroundPositions,
                                      count: highGroundPositions.count)
splineShapeNode.physicsBody = SKPhysicsBody(edgeChainFrom: splineShapeNode.path!)
</code>
Note that the code above creates the first at x=0 and moves to the right, so adjust per your needs. Hope this help you to get started.
Comment: In which function do I have to put those two parts of the code? @GOR
Comment: Well, the first part is the y-coordinates for the landscape and the second part is the Node and its physics body. Really depends on your project structure. My suggestion would be to take a look here https://developer.apple.com/documentation/spritekit and try out something even simpler first perhaps to learn the framework. Even something like just creating ball shapes and physics for them and running just to get comfortable with how SpriteKit works.
Comment: Do you know how to move the move the SKShapeNode to the left?


Recursive Type Families

Question: I am experimenting with an mtl-style class that allows me to lift Pipe composition over an outer monad.  To do so, I must define which two variables of the type are the domain and codomain of Pipe composition.
I tried using an associated type family approach, but to no avail:
{-# LANGUAGE TypeFamilies #-}

import Control.Monad.Trans.Free
import Control.Monad.Trans.State
import Control.Pipe hiding (Pipe)

data Pipe a b m r = Pipe { unPipe :: FreeT (PipeF a b) m r }

class MonadPipe m where
    type C a b (m :: * -> *) :: * -> *
    idT :: C a a m r
    (<-<) :: C b c m r -> C a b m r -> C a c m r

instance (Monad m) => MonadPipe (Pipe i o m) where
    type C a b (Pipe i o m) = Pipe a b m
    idT = Pipe idP
    (Pipe p1) <-< (Pipe p2) = Pipe (p1 <+< p2)

instance (MonadPipe m) => MonadPipe (StateT s m) where
    type C a b (StateT s m) = StateT s (C a b m)
    idT = StateT $ \s -> idT
    (StateT f1) <-< (StateT f2) = StateT $ \s -> f1 s <-< f2 s

However, the above code does not type-check.  GHC gives the following errors:
family.hs:23:15:
    Could not deduce (C a a m ~ C a0 a0 m0)
    from the context (MonadPipe m)
      bound by the instance declaration at family.hs:21:14-52
    NB: `C' is a type function, and may not be injective
    Expected type: C a a (StateT s m) r
      Actual type: StateT s (C a0 a0 m0) r
    In the expression: StateT $ \ s -> idT
    In an equation for `idT': idT = StateT $ \ s -> idT
    In the instance declaration for `MonadPipe (StateT s m)'

family.hs:24:10:
    Could not deduce (C b c m ~ C b0 c0 m1)
    from the context (MonadPipe m)
      bound by the instance declaration at family.hs:21:14-52
    NB: `C' is a type function, and may not be injective
    Expected type: C b c (StateT s m) r
      Actual type: StateT s (C b0 c0 m1) r
    In the pattern: StateT f1
    In an equation for `<-<':
        (StateT f1) <-< (StateT f2) = StateT $ \ s -> f1 s <-< f2 s
    In the instance declaration for `MonadPipe (StateT s m)'

<<Two other errors for 'C a b m' and 'C a c m'>>

It's hard for me to understand why the types won't unify, especially for the idT definition, since I'd expect the inner idT to be universally quantified over a so it would match the outer one.
So my question is whether this is implementable with type families, and if not possible with type families, how could it be implemented?
Comment: The last part of my answer using data families comes quite close to your desired code.  How does that solution miss what you need?
Answer: Type inference is, by default, a guessing game. Haskell's surface syntax makes it rather awkward to be explicit about which types should instantiate a forall, even if you know what you want. This is a legacy from the good old days of Damas-Milner completeness, when ideas interesting enough to require explicit typing were simply forbidden.
Let's imagine we're allowed to make type application explicit in patterns and expressions using an Agda-style f {a = x} notation, selectively accessing the type parameter corresponding to a in the type signature of f. Your
idT = StateT $ \ s -> idT

is supposed to mean
idT {a = a}{m = m} = StateT $ \ s -> idT {a = a}{m = m}

so that the left has type C a a (StateT s m) r and the right has type StateT s (C a a m) r, which are equal by definition of the type family and joy radiates over the world. But that's not the meaning of what you wrote. The "variable-rule" for invoking polymorphic things requires that each forall is instantiated with a fresh existential type variable which is then solved by unification. So what your code means is
idT {a = a}{m = m} = StateT $ \ s -> idT {a = a'}{m = m'}
  -- for a suitably chosen a', m'

The available constraint, after computing the type family, is
C a a m ~ C a' a' m'

but that doesn't simplify, nor should it, because there is no reason to assume C is injective. And the maddening thing is that the machine cares more than you about the possibility of finding a most general solution. You have a suitable solution in mind already, but the problem is to achieve communication when the default assumption is guesswork.
There are a number of strategies which might help you out of this jam. One is to use data families instead. Pro: injectivity no problem. Con: structor. (Warning, untested speculation below.)
class MonadPipe m where
  data C a b (m :: * -> *) r
  idT :: C a a m r
  (<-<) :: C b c m r -> C a b m r -> C a c m r

instance (MonadPipe m) => MonadPipe (StateT s m) where
  data C a b (StateT s m) r = StateTPipe (StateT s (C a b m) r)
  idT = StateTPipe . StateT $ \ s -> idT
  StateTPipe (StateT f) <-< StateTPipe (StateT g) =
    StateTPipe . StateT $ \ s - f s <-< g s

Another con is that the resulting data family is not automatically monadic, nor is it so very easy to unwrap it or make it monadic in a uniform way.
I'm thinking of trying out a pattern for these things where you keep your type family, but define a newtype wrapper for it
newtype WrapC a b m r = WrapC {unwrapC :: C a b m r}

then use WrapC in the types of the operations to keep the typechecker on the right track. I don't know if that's a good strategy, but I plan to find out, one of these days.
A more direct strategy is to use proxies, phantom types, and scoped type variables (although this example shouldn't need them). (Again, speculation warning.)
data Proxy (a :: *) = Poxy
data ProxyF (a :: * -> *) = PoxyF

class MonadPipe m where
  data C a b (m :: * -> *) r
  idT :: (Proxy a, ProxyF m) -> C a a m r
  ...

instance (MonadPipe m) => MonadPipe (StateT s m) where
  data C a b (StateT s m) r = StateTPipe (StateT s (C a b m) r)
  idT pp = StateTPipe . StateT $ \ s -> idT pp

That's just a crummy way of making the type applications explicit. Note that some people use a itself instead of Proxy a and pass undefined as the argument, thus failing to mark the proxy as such in the type and relying on not accidentally evaluating it. Recent progress with PolyKinds may at least mean we can have just one kind-polymorphic phantom proxy type. Crucially, the Proxy type constructors are injective, so my code really is saying "same parameters here as there".
But there are times when I wish that I could drop to the System FC level in source code, or otherwise express a clear manual override for type inference. Such things are quite standard in the dependently typed community, where nobody imagines that the machine can figure everything out without a nudge here and there, or that hidden arguments carry no information worth inspecting. It's quite common that hidden arguments to a function can be suppressed at usage sites but need to be made explicit within the definition. The present situation in Haskell is based on a cultural assumption that "type inference is enough" which has been off the rails for a generation but still somehow persists.
Comment: Kind of you to say so. I am also one of the least anonymous referees in the business...
Comment: It only ever takes a paragraph or two to recognize one of your answers. Love your style.
Comment: Thanks for explaining the type error.  I now have a greater appreciation for the machine as an assistant and not an oracle.  I tried the associated data type, but it requires explicit wrapping/unwrapping to use and I couldn't find a way to automate that for the user.  Initial prototypes that I had in mind incorporated an isomorphism between the monad transformer and the associated data type so that this could be done automatically, but I could never get it to work.
Answer: EDITED Three times: see bottom for data family version. And changed GADT version to drop m.
Let me guess: leftovers?
Let me walk myself through the type error first, the a SOLUTION.
The class defines :
type C a0 b0 m  where a0 and b0 are fresh.
idT :: C a a m r, where a and r are fresh.

The idT in the (Pipe i o m0) instance is okay by what I think is the logic:
LHS is idT :: C a0 a0 (Pipe i o m0) r0 which becomes Pipe a0 a0 m0 r0
RHS is Pipe idP :: Pipe a1 a1 m1 r1 starts fresh
And then these unify
because Pipe is a data constructor.

The idT in the MonadPipe m0 => (StateT s0 m0) instance:
LHS is idT :: C a0 a0 (StateT s0 m0) which becomes StateT s0 (C a0 a0 m0)
RHS is StateT (\s -> idT) :: StateT s1 m1 r1
Some unification seems to happen...
RHS is StateT (\s -> idT) :: StateT s1 (C a0 a0 m0) r1
  where expression idT :: MonadPipe m1 => (C a2 a2 m2) r2 starts fresh
        context of idT :: (C a0 a0 m0) (a1, s1)
And then (C a0 a0 m0) does not unify with (C a1 a2 m2)
because C is a type constructor.

Your previous newtype way to make the Category instances probably works here if the type family becomes a data family.
EDIT: You alter the order of parameters and newtype StateT to solve it:
{-# LANGUAGE TypeFamilies, MultiParamTypeClasses #-}

import Control.Monad.Trans.Free
import Control.Monad.Trans.State
import Control.Pipe hiding (Pipe)

data Pipe m a b r = Pipe { unPipe :: FreeT (PipeF a b) m r }

newtype StatePipe s mp a b r = SP (StateT s (mp a b) r)

class MonadPipe mp where
    idT :: mp a a r
    (<-<) :: mp b c r -> mp a b r -> mp a c r

instance (Monad m) => MonadPipe (Pipe m) where
    idT = Pipe idP
    (Pipe p1) <-< (Pipe p2) = Pipe (p1 <+< p2)

instance (MonadPipe mp) => MonadPipe (StatePipe s mp) where
     idT = SP . StateT $ \s -> idT
     (SP (StateT f1)) <-< (SP (StateT f2)) = SP . StateT $ \s -> f1 s <-< f2 s

Though MonadTrans might now be sad.  Another approach keeps the argument order by using a GADT to, perhaps, more cleanly express what you are trying to build:
{-# LANGUAGE MultiParamTypeClasses, GADTs, FlexibleInstances #-}
import Control.Monad.Trans.Free
import Control.Monad.Trans.State
import Control.Pipe hiding (Pipe)

data Pipe s a b m r where
  FPipe :: { unPipe :: FreeT (PipeF a b) m r } -> Pipe () a b m r
  LPipe :: StateT s1 (Pipe s2 a b m) r -> Pipe (s1,s2) a b m r

class MonadPipe s where
    idT :: Monad m => Pipe s a a m r
    (<-<) :: Monad m => Pipe s b c m r -> Pipe s a b m r -> Pipe s a c m r

instance MonadPipe () where
    idT  = FPipe idP
    (FPipe p1) <-< (FPipe p2) = FPipe (p1 <+< p2)

instance MonadPipe s2 => MonadPipe (s1,s2) where
    idT  = LPipe (StateT $ \s -> idT)
    (LPipe (StateT f1)) <-< (LPipe (StateT f2)) =
       LPipe (StateT $ \s1 -> (f1 s1 <-< f2 s1))

And I can translate this into a perhaps even nicer data family?
{-# LANGUAGE TypeFamilies #-}
import Control.Monad.Trans.Free
import Control.Monad.Trans.State
import Control.Pipe hiding (Pipe)

data family GPipe s :: * -> * -> (* -> *) -> * -> *
newtype instance GPipe () a b m r = Pipe { unPipe :: FreeT (PipeF a b) m r }
newtype instance GPipe (s1,s2) a b m r = LPipe ( StateT s1 (GPipe s2 a b m) r )

class MonadPipe s where
  idT :: Monad m => GPipe s a a m r
  (<-<) :: Monad m => GPipe s b c m r -> GPipe s a b m r -> GPipe s a c m r

instance MonadPipe () where
  idT = Pipe idP
  (Pipe p1) <-< (Pipe p2) = Pipe (p1 <+< p2)

instance MonadPipe s2 => MonadPipe (s1,s2) where
  idT = LPipe (StateT (\s -> idT))
  (LPipe (StateT f1)) <-< (LPipe (StateT f2)) = LPipe (StateT (\s -> f1 s <-< f2 s))

Comment: Hmmm...it may be clearer to change GPipe () to GPipe Void and declare (data GStateT s1 s2) and change GPipe (s1,s2) to GPipe (GStateT s1 s2).  Then it is more obvious that GPipe is open to adding other monad stacks.
Comment: The first solution I've already tried out: https://github.com/Gabriel439/Haskell-Pipes-Library/blob/pipetrans/Control/Pipe/Trans/State.hs.  It works perfectly, BUT, then I have to reimplement the entire `transformers` package so there is no code reuse.  The data family approach looks like it should work.  It seems pretty elegant, and I like the fact that it makes the layering explicit.  I will have a chance to test it thoroughly this afternoon.  I think your comment has a mistake, though, because then you would have a different number of parameters for each type.
Comment: My point was that () for Pipe and (,) for LPipe/StateT was arbitrary.  I should make a new empty type for each GPipe instance, where each new empty type can have whatever number of parameters might be needed.
Comment: Oh, I understand what you mean now.  Right now, I think I'm relenting on something similar to your first solution using the following generalized newtype to rearrange the type variables: `C t p a b r = C { unC :: t (p a b) r }`.  This does involve some newtype wrapping and unwrapping, but I think I can abstract even that away with some cleverness.  Let me spend more time on it first.
Comment: I accepted this because the first answer you gave ended up being the closest to what actually solved my problem.  Also, to answer your question about leftovers, I'll just leave [this](http://hpaste.org/73647) here.  :)

Recursive Type Families


Issue parsing JSON object response to C# object

Question: I have the following code:
[WebMethod]
public bool AddUser(long warnId, double longitude, double latitude)
{
    try
    {
        string jsonFeature = string.Empty;
        jsonFeature += "[{'geometry': {'x': " + longitude + ",'y': " + latitude +
                       ",'spatialReference': {'wkid': 4326}},'attributes': {'UID': '";
        jsonFeature += warnId + "','Latitude': '" + latitude + "','Longitude': '" + longitude + "'}}]";

        string reqURL = System.Configuration.ConfigurationManager.AppSettings["WARNURL"] + "addFeatures";

        using (System.Net.WebClient client = new System.Net.WebClient())
        {
            client.Headers["Content-type"] = "application/x-www-form-urlencoded";
            client.Encoding = System.Text.Encoding.UTF8;
            var collection = new System.Collections.Specialized.NameValueCollection();
            collection.Add("f", "json");
            collection.Add("features", jsonFeature);
            byte[] response = client.UploadValues(reqURL, "POST", collection);
            MemoryStream stream = new MemoryStream(response);
            StreamReader reader = new StreamReader(stream);
            string aRespStr = reader.ReadToEnd();

            JavaScriptSerializer jss = new JavaScriptSerializer();
            AddResult addResult = jss.Deserialize<AddResult>(aRespStr);

            return aRespStr.Contains("true") && aRespStr.Contains("success");
        }
    }
    catch(Exception e)
    {
        string message = e.Message;
        return false;
    }
}

when I run it the string aRespStr is:

"{\"addResults\":[{\"objectId\":28,\"globalId\":\"{740490C6-77EE-4AC0-9561-5EBAACE3A0A7}
  \",\"success\":true}]}"

I created the following classes to hold the object once I deserialize it:
    public class Error
{
    public int code { get; set; }
    public string description { get; set; }
}

public class AddResult
{
    public int objectId { get; set; }
    public object globalId { get; set; }
    public bool success { get; set; }
    public Error error { get; set; }
}

public class RootObject
{
    public List<AddResult> addResults { get; set; }
}

but when I run the code the addResult object contains the default object values not the json object values.
The api for this particular response is here:
http://help.arcgis.com/en/arcgisserver/10.0/apis/rest/fsadd.html
any help on getting this to work is greatly appreciated
Comment: The string matches your RootObject class but you are deserializing as AddResult in your code sample.
Comment: thanks bmavity, when I changed it to deserialize to the RootObject it worked great
Answer: Try this
RootObject addResults=jss.Deserialize<RootObject>(aRespStr);

or similary you can try this
List<AddResult> addResults = jss.Deserialize<List<AddResult>>(aRespStr);

as you are returning list of AddResult in your json Response.
And Change the content type to application/json
Answer: change
client.Headers["Content-type"] = "application/x-www-form-urlencoded";

to
client.Headers["Content-type"] = "Application/json";


Ruby precedence of methods in objects extended with multiple modules

Question: Given the following:
class User; attr_accessor :roles; end

module RegisteredUser
  def default_context
    Submission
  end
end

module Admin
  def default_context
    Review
  end
end

current_user = User.new
current_user.roles = ["registered_user", "admin"]
current_user.roles.each do |role|
  role_module = role.gsub(/ /, '_').camelize
  if module_exists?(role_module)
    current_user.extend role_module.constantize
  end
end
context = self.extend current_user.default_context

Is there a way to set the priority of User#default_context? That is, can I say that the Admin#default_context always takes priority over RegisteredUser#default_context regardless of the order in which current_user is extended?
Comment: You might be able to do something with `extended` or `method_added`, but you'd need to have some sort of priority rules mechanism either in the modules or somewhere... erm... else.
Comment: I can't find either of those methods in ruby doc.
Comment: Moved to pseudo-answer for formatting.
Answer: method_added is in Module.
I actually meant included, not extended, but both are also in Module.
The mechanism would revolve around doing something like this:
module Foo
  def self.included(base)
    base.extend(FooMethods)
  end

  module FooMethods
    def bar
      # Whatever
    end
  end
end

Inside Foo.included you can determine, based on arbitrary criteria, whether or not the methods in question should be added to base (the entity including the module).
In your case, you could check to see if a "higher priority" module was already included, or see if the module is the "higher priority" module. Based on that you'd decide whether or not to add the methods.
Comment: I get "NoMethodError: private method `method_added' called for Foo:Module" when I try Foo.method_added.
Comment: @ReedG.Law That's not how you'd use it, nor am I convinced it's what you'd want to use--I think it's an "after" hook, which doesn't help. `included` would be more appropriate.
Answer: Since an Admin is also a RegisteredUser, I would do
module Admin
  include RegisteredUser
  ...
end

and then only
current_user.extend Admin

I am not sure if this is the correct way. If Admin and RegisteredUser were classes, it would make sense making Admin inherit from RegisteredUser. In case of modules, dunno.
Comment: This method seems to work. The only drawback I can see is if there were many modules and complex combinations (i.e. a user who is an admin but not an editor, but Admin includes Editor).
Answer: You cannot; in Ruby the order of module inclusion is the order in which modules are searched (after the current class, before parent classes). The only way to change the 'priority' is to include the modules in the order you want, or to move them to a parent class.
Although not pure Ruby, you can use the Remix library by banisterfiend to change module ordering (or unmixin a module, or...other things).
Comment: I'd have to play, but it sure seems like some hooks into `included`/`extended` could be used to do this, no?
Comment: Remix looks good, but it's hard to jump to that kind of solution without further evaluation. Although that kind of functionality could be essential to using the DCI design pattern.


CSV writer - writing to csv as the script runs but does not save

Question: My code appears to be working and saving down data to the csv but after the script ends, there is no data. As the script is running, my csv file stores the data and if I interrupt it, I can see some data.
import requests
from bs4 import BeautifulSoup
import csv

def main():

    page = 1
    page_has_data = True
    while page_has_data:

        r = requests.get("http://scrapethissite.com/pages/forms/", params=dict(per_page=100, page_num=page))
        soup = BeautifulSoup(r.text, features="lxml")

        print("----- Scraping page number", page)
        page += 1

        teams = soup.findAll("tr", "team")
        if len(teams) == 0:
            page_has_data = False

        with open("hockey_stats.csv", "w", newline="") as results:
            writer = csv.writer(results)

            for team in teams:
                team_name = team.find("td", "name").get_text(strip=True)
                team_year = team.find("td", "year").get_text(strip=True)
                team_win = team.find("td", "wins").get_text(strip=True)
                team_loss = team.find("td", "losses").get_text(strip=True)
                team_pct_win = team.find("td", "pct text-success")
                team_goalsfor = team.find("td", "gf").get_text(strip=True)
                team_goalsagainst = team.find("td", "ga").get_text(strip=True)
                team_plus_minus = team.find("td", "diff text-success")
                writer.writerow([team_name, team_year,team_win, team_loss, team_pct_win, team_goalsfor, team_goalsagainst, team_plus_minus])

            results.close()

Answer: I think you have two issues with your snippet.  You are opening the file in write mode.  Instead open it it append mode("a").  When you are iterating over, with new requests, you end up overwriting the data you last wrote to the file (if the file is opened in write mode - "w").  The last request will not have any data, and you overwrite your file with empty data.
Secondly, you don't need 'results.close()'.  The with statement will take care of closing the file.
Answer: Try this: with open("hockey_stats.csv", "a")
The last entry is wiping off the previous data as it is in "w" mode and the last entry to the file is empty. Appending works in this case.
Answer: You have two ways to fix this. First, open file with 'a' mode. Or, open your file out of the while loop. In your code, whenever loop starts again, it starts with an empty file


Can I use APCu in Lithium of the PHP Framework with no configuration, or should I change the code?

Question: When I setup a lithium project (below process) and start PHP web server, I get an information of Apc disable (Apc ✗) on the framework's welcome page.
But I run sed -ir "s/'apc'/'apcu'/" libraries/lithium/storage/cache/adapter/Apc.php, then I get Apc enabled (Apc ✓).
This code change is correctly? or not?
Environment

Fedora 26 (x86_64)
PHP 7.1.7 (cli) (built: Jul  6 2017 12:10:54) ( NTS )
php-pecl-apcu 5.1.8
Composer version 1.4.2 2017-05-17 08:17:52

Detail:
$ rpm -qa | grep php
php-pdo-7.1.7-1.fc26.x86_64
php-pecl-apcu-devel-5.1.8-2.fc26.x86_64
php-7.1.7-1.fc26.x86_64
php-mysqlnd-7.1.7-1.fc26.x86_64
php-json-7.1.7-1.fc26.x86_64
php-devel-7.1.7-1.fc26.x86_64
php-cli-7.1.7-1.fc26.x86_64
php-pecl-apcu-5.1.8-2.fc26.x86_64
php-common-7.1.7-1.fc26.x86_64

How to setup of the Lithium, and access the framework's welcome page:
$ composer.phar create-project --prefer-dist unionofrad/framework app
Installing unionofrad/framework (v1.1.1)
  - Installing unionofrad/framework (v1.1.1): Loading from cache
Created project in app
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 2 installs, 0 updates, 0 removals
  - Installing composer/installers (v1.3.0): Loading from cache
  - Installing unionofrad/lithium (v1.1.0): Loading from cache
Writing lock file
Generating autoload files
$ cd app/
$ php -S 0.0.0.0:8080 -t app/webroot index.php

Setting diff
.orig file is the backup.

/etc/php.ini:
$ diff /etc/php.ini.orig /etc/php.ini
460c460,461
< error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
---
> ;error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
> error_reporting = E_ALL

/etc/php.d/40-apcu.ini:
$ diff /etc/php.d/40-apcu.ini.orig /etc/php.d/40-apcu.ini
9a10
> apc.enable_cli=1

Cases
with a -c /etc/php.ini parameter
Run with the parameter, but it no effect,:
$ php -c /etc/php.ini -S 0.0.0.0:8080 -t app/webroot index.php

shows Apc disable (Apc ✗).
Comment: Perhaps you need to start the PHP built-in web server with a `-c /etc/php.ini` parameter?
Comment: @blubear Thanks, but it couldn't make no sense... I tried to run with the case of 'with a `-c /etc/php.ini` parameter'. (See updated question's Cases section)
Answer: Install php-pecl-apcu-bc package by dnf makes adapting to Apc ✓. But php-pecl-apcu-bc is "APCu Backwards Compatibility Module"... oh.
Before installed:
$ php -m
[PHP Modules]
apcu
bz2
calendar
Core
ctype
curl
date
exif
fileinfo
filter
ftp
gettext
hash
iconv
json
libxml
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
readline
Reflection
session
sockets
SPL
sqlite3
standard
tokenizer
zlib

[Zend Modules]

After installed:
$ php -m
[PHP Modules]
apc
apcu
bz2
calendar
Core
ctype
curl
date
exif
fileinfo
filter
ftp
gettext
hash
iconv
json
libxml
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
readline
Reflection
session
sockets
SPL
sqlite3
standard
tokenizer
zlib

[Zend Modules]


Any reason not to enable DoS Defense in my router?

Question: I recently found a DoS Defense setting in my DrayTek Vigor 2830 router, which is disabled as default. I'm running a very small server on this network and I take it very serious to have the server up and running 24/7.
I'm a bit unsure if the DoS Defense could cause me any kind of problems. I haven't experienced any DoS attacks yet, but I would like to avoid possible attacks. Is there any reason not to enable the DoS Defense setting?
Comment: Rather than ask **us** if you should/should not enable this "DoS Defense" feature, why not ask your router vendor *what it actually does* when you check the box, then decide if those rules make sense in your environment?
Comment: (After digging up the manual from their website I can say the list of things it checks for and deals with is a relatively sane one -- Unlikely to break anything legit, so no real harm in turning it on. Just don't expect it to protect you from everything - [there are some attacks it can't mitigate](http://serverfault.com/a/459752/32986))
Comment: As this is mostly a risk analysis question, you might consider asking to migrate this to [security.se].
Answer: It means the router has to maintain additional state and do additional work on each packet. And how can it really help in the case of a DoS? All it can do is drop a packet that you have already received. Since you've already received it, it has already done the damage by consuming your inbound Internet bandwidth.
Comment: @SpacemanSpiff: Two reasons this is not true: 1) A typical ADSL link can't carry enough traffic to take out a service anyway. Typical DoS attacks over such links work by consuming your bandwidth. 2) The device can't reliably tell attack traffic from legitimate traffic. So stopping the DoS attack is just a DoS attack on yourself since you're dropping the legitimate traffic too. (At most, this will preserve your outbound bandwidth for other services because you're not replying to the attack traffic. But with no useful inbound, protecting your outbound usually doesn't help much.)
Comment: @SpacemanSpiff: If they can overwhelm his bandwidth, they can take him down even with it on. He's dropping the traffic *after* it has consumed his bandwidth. Having a defended edge inside the slowest link does you little to no good. Most likely, his weakest link is the router CPU and his inbound bandwidth.
Comment: Pretty much... Generally, if you get dossed on any line a dreytek would be holding up, you're going down.
Comment: What you've told him to do is turn off the following, just so you know: SYN flooding, UDP flooding, ICMP flooding, Port Scan Detections, IP Spoofing, Tear Drop Attacks. Just because this vendor leaves it off by default, doesn't mean everyone does. Juniper NetScreen and SRX Branch routers come out this enabled, as does the ASA5505.
Comment: Yes, but you've turned all basic edge defense, now even an idiot with a Linux ping command can take him down.
Comment: I'm pretty sure this is to keep the service running, not to stop the bandwidth drain. I really can't agree with this answer at all.
Comment: @SpacemanSpiff: I'm not giving him advice on everything, just what to do with this device in this application. Sure, branch and core routers try to protect narrower downstream links. But he doesn't have any narrower downstream links.
Comment: Wouldn't that mean...  If I got DOSed while having the DOS protection, and if I find out who's DOSing me, I can DOS back at it with my unused outbound?
Comment: @TeaCrab No. That's not how any of this works at all. It's not like a gunfight.
Answer: One reason to not enable the DoS Defense setting is that trying to protect systems from DOSed will spike the CPU of the router/firewall causing a DoS itself.
Answer: An old thread I know, but I've just had to turn off the DoS defences on my Draytek 2850 home router to prevent some connection problems (almost everyone's in-bound bandwidth dropped to 0).  Oddly enough, when all the kids are using their iPhones, PCs and chatting on Skype, etc. it triggers the DoS defences!
My guess is that there's so much traffic going in both directions that the router thinks it's under attack from the outside and shuts down.  Turning off the UDP flood defence didn't do a complete fix so I turned off the SYN and ICMP defences too. (If you had to turn off both SYN and ICMP flood protection then I think the router was doing a very good job unless you are running a server or servers on your network) - SYN and ICMP requests are sent to servers during connection initiation, then the client devices receive a SYN-ACK back from the server.
Hey presto - no more connection issues.  Of course, I'll turn the defences back on and better-tune the values (measured in packets/second), but I've been trying to nail this problem for ages and it was quite a shock to find out the real cause.
I hope this helps someone else.
Comment: We had a very similar problem on a 2930 shortly after we started allowing mobile devices on the network. I upped the threshold rates for SYN, UDP and ICMP defense significantly and it stopped the problem.
Comment: I can confirm the same on an ASUS Wireless Router RT-N10. Enabling DoS protection will degrade the wireless connection.
Answer: Yes, absolutely, turn it on.
If this is implemented correctly your firewall's engine should inspect each packet. Once it's determined to drop this traffic as part of a DoS attack, it should install a rule into hardware and silently drop the traffic instead of processing it again and again. Where it will still fall on it's face is a distributed attack, but I suggest you turn this on.
What kinds of services is that server hosting?
Comment: If the traffic was hurting your link, it'll still be hurting your link. If it wasn't, you're now likely to be dropping at least some legitimate traffic, making the DoS attack worse. On a SoHo router, this is bad advice. It's off by default for a reason.
Comment: "If implemented correctly" isn't assured; especially on consumer grade hardware.  The Netgear router I was using at home several years ago had a major bug in it's DOS filter.  It was possible to send a single packet with malformed data that would cause DOS filter to crash and take the router down with it.
Comment: The whole point of a DoS is to make the DoS traffic indistinguishable from legitimate traffic so the victim has to choose between dropping legitimate traffic and responding to the DoS traffic. For example, if you're serving HTTP on port 80, one typical DoS attack you'll see is a multi-source SYN flood on port 80. How can you tell the flood SYNs from legitimate client SYNs?
Comment: No it's not, he can always turn it off or adjust thresholds. I've seen bottom-end devices defend networks with just this basic stuff while misconfigured enterprise class firewalls fall on their face.
Comment: It is running a lot of different stuff: IIS, MSSQL Databases, MySQL databases, Apache, Minecraft and all kinds of random stuff I'd need a server for :)
Comment: I'm pretty sure screens and DoS rules do not drop OTHER types of traffic.
Answer: Yes. It can turn "Denial of Service protection" into just "Denial of Service" from the outside Internet, even without any attacks.
In my case with ASUS RT-AC58U it made home HTTP(S) and FTP(S) servers to respond very slowly to half the requests even with a single outside client, or even not respond at all and timeout after several succesful connections. E.g. I open a page and half of the images are missing, and a click to the next page hangs for a minute.
I tried to setup various cover-up measures for years, like a file-caching reverse proxy (nginx) on my VPS, caching DNS server, even just storing home router's dynamic IP in /etc/hosts. VPS itself worked good, but connectivity to home back-end never improved until i disabled the "DoS Protection" in home router.
A daily backup using duplicity from my VPS to home PC via FTP(S), which makes a new connection for each file, might skip some diff files, take anywhere from half an hour to 10+ hours or not finish at all that day. Now it consistently takes 1.5 minutes for 4-5 folders via FTPS with "DoS Protection" disabled.
Although my other backup scripts with lftp, which use a single FTPS connection for a bunch of files at once, worked acceptably enough even with "DoS Protection" enabled.
Answer: Yes because it saves network from malicious attacks.
False Positives: In some cases, router DoS defence algorithms mistakenly classify genuine traffic as an attack and stop it. This may cause services to be interrupted or prevent authorised users from using your network. To reduce false positives, it's crucial to carefully configure and monitor the DoS defence settings.
Performance Impact: Enabling DoS defence on your router may use up system resources. The overall performance of your network may be impacted, depending on the specific implementation and the level of protection set. Increased protection may result in slower throughput or higher latency. Make sure the performance effect is appropriate for the demands of your network by evaluating it.
Advanced Attacks: Although DoS defense measures are successful against simple and medium-level attacks, complex attacks may still find a way around the barriers. Remember that determined attackers may use more sophisticated strategies, necessitating additional security measures beyond what your router can offer.
Answer: If the DoS attack doesn't kill your pc first, the heat generated from DoS protection will kill your router. If your that concerned about security then don't use the internet.
It is better to protect every individual device on your network with a properly set firewall and av, when not using the net turn off your wifi use it like you would your tap water.


Is "system_settings_form" voodoo?

Question: A friend just showed me system_settings_form in action in a configuration file for a form.
The code in question looks like this (simplified).
function my_module_generateAdminForm($form){
  $form = array();
  $form['super_secret_password'] = array(
    '#type' => 'textfield',
    '#title' => t('Headline'),
    '#default_value' => variable_get('super_secret_password','password'),
  );
  system_settings_form($form);
}

system_settings_form adds in submit buttons and handles the form submission, saving the variables. It also adds a "Reset to default" button - that resets all the variable to the default.
I basically guessed what it does because there's not a whole lot of documentation out there and my friend doesn't know exactly what it does either. My question is how does it know the variables to save/reset? Aren't I just passing an array?
Also is there any other magic in system_settings_form that I should know about?
Answer: See system_settings_form and system_settings_form_submit for implementation details. In Drupal 7 Reset to Default button is removed from System Settings form .
function system_settings_form($form) {
  $form['actions']['#type'] = 'actions';
  $form['actions']['submit'] = array(
    '#type' => 'submit',
    '#value' => t('Save configuration'),
  );

  if (!empty($_POST) && form_get_errors()) {
    drupal_set_message(t('The settings have not been saved because of the errors.'), 'error');
  }
  $form['#submit'][] = 'system_settings_form_submit';
  // By default, render the form using theme_system_settings_form().
  if (!isset($form['#theme'])) {
    $form['#theme'] = 'system_settings_form';
  }
  return $form;
}

System_settings_form :
Add's save configuration Submit button and submit callback to system_settings_form_submit function.
system_settings_form_submit:
function system_settings_form_submit($form, &$form_state) {
  // Exclude unnecessary elements.
  form_state_values_clean($form_state);

  foreach ($form_state['values'] as $key => $value) {
    if (is_array($value) && isset($form_state['values']['array_filter'])) {
      $value = array_keys(array_filter($value));
    }
    variable_set($key, $value);
  }

  drupal_set_message(t('The configuration options have been saved.'));
}

Iterates through all form elements and sets variables for each form element with key as form element key and value as user submitted form  element value read using $form_state['values']
So, it's just a form with submit handler that sets variables for all form values using variable_set function.
Comment: Good explanation. If you are confused by "each form element with key as form element key", what he means is that in your example above, $form['super_secret_password'] will cause Drupal to call variable_set('super_secret_password', ...), because system_settings_form_submit takes the name of the variable from the array key ('super_secret_password') of your form.
Comment: This is helpful. Why can't api.drupal.org just explain what it does in plain english. Instead it just gives the code...  So in summary not voodoo, just using my field keys.
Comment: I think it's worth noting that you should not use this to store large variables as it will stay in memory for every page request no matter there was a direct variable_get for it or not. They all are cached once in memory. Also, regular submit and validate handlers will work without a problem. Feel free to add them.

Is "system_settings_form" voodoo?


Showing exterior product is skew-commutative

Question: I am learning about the exterior product and wish to show the skew commutative property. Here is my work:

I am confused on how to handle the notation of $(\omega^k \wedge \omega^l)$ as at the end I want to be able to write $(\omega^l \wedge \omega^k)$ How is this handled?
Answer: Hint: Write $\omega^k$ in standard basis $dx^1, dx^2,\ldots, dx^n$ and note that $dx^i\wedge dx^j=-dx^j\wedge dx^i$  for $i\neq j$ according to your note. now compute $w^k\wedge \omega^\ell$.
Try yourself, and then look at Loring TU book for direct proof.
Comment: Hi CFG, what is $dx^1$ and such? I've not seen this notation.
Comment: the coordinate chart components. for example in 2d we have $(x,y)$ and $dx$ and $dy$.
Comment: Hm, well, what comes to my mind is the standard dual basis for the $k$-form. I have been reading from: http://www.math.toronto.edu/jkamnitz/courses/mat247/tensorproducts1.pdf Is this the same thing?
Comment: yes. these are dual basis of $\frac{\partial}{\partial x^i}$.
Comment: Okay, well, $\omega^k$ is a k-form on $\mathbb{R}^{n}$. There is a basis for the $\omega^k$ $k$-form  consisting of $\alpha_1(\xi_1, \cdots , \xi_n)$ so that $\alpha_1(\xi_1, \cdots) = \xi_1$ and so forth. Meaning each member of $(V^{\star})^{\otimes k}$ is written as $\alpha_1 \otimes \alpha_2 \otimes \cdots \otimes \alpha_n$. Now I can do my permutation work as I did in the OP?
Comment: what is the $\alpha_1$? a $1$-form or $k$-form or $n$-form?
Comment: I'm...not sure. I was just writing the alphas to denote the dual basis of the dual vector space which the k-form lives in. I guess this is not correct.


XPATH : Count Preceding Div element

Question: When the context points to division/division/*topic , I wanted to count all topic tags under body/division and also add result to count of previous body/division/division tag.
Output = All topics under body/division + count of body/division/division
<?xml version="1.0" encoding="UTF-8"?>

       <division>
          <Atopic><longtitle>TitleDiv1 Atopic</longtitle></Atopic>
          <Btopic><longtitle>TitleDiv1 Btopic</longtitle></Btopic>
          <Ctopic><longtitle>TitleDiv1 Ctopic</longtitle></Ctopic>
          <division>
             <Atopic><longtitle>TitleDiv1.1 Atopic</longtitle></Atopic>
          </division>
          <division>
                <Atopic><longtitle>TitleDiv1.2 Atopic</longtitle></Atopic>
          </division>
       </division>
       <division>
            <Atopic><longtitle>TitleDiv2 Atopic</longtitle></Atopic>
            <Dtopic><longtitle>TitleDiv2 Dtopic</longtitle></Dtopic>
          <division>
             <Atopic><longtitle>TitleDiv2.1 Atopic</longtitle></Atopic>
          </division>
           <division>
             <Ctopic><longtitle>TitleDiv2.2 Ctopic</longtitle></Ctopic>
          </division>
       </division>

So for example

If my context is in Div 1/2 , the logic should count all *topics under body/division1. So it should return 3 (Atopic,Btopic,Ctopic). Here there is no preceding body/division/division element. So o/p = 3+0 = 3
                                                                                2.Similarly from Div1/3 it should count the *topics under body/division tag i.e             3 and also count preceding body/div/div (i.e 3 + 1 = 4) .
When context is Div2/1 the counter should only consider the *topics under body/division2.
So if I use the logic from Div2/1 Atopic , it should return "2" since Div2 tag (body/division) has 2 topics and no preceding body/div/div . Output = 2 . Here the preceding body/division/division should be considered for Div2 tag and Div1's body/division/division should NOT be accounted. In short the body/division/division should be reset when new body/division is encountered.

pls advice.
Comment: How is this different from your previous question: http://stackoverflow.com/questions/39531286/xpath-get-preceding-childrens
Comment: P.S. Your example would be much clearer if (a) the divisions had a unique id attribute and (b) the topics had unique names and (c) it were properly indented. It's also not clear what exactly `.... DIV 1 ......` etc. are supposed to be; if they are elements, why not make them elementa?
Comment: hello Michael , the difference is I also need to add division/division to *topic i.e       "count(ancestor::division[last()]/*[contains(name(), 'topic')])".                                                                                your sol was perfectly fine when considering the *topics under body/division. But I also need division/division to be added to the count(ancestor::division[last()]/*[contains(name(), 'topic')]) result so that when context is 1/3 it returns (3 topics under body/div and 1 div/div = 4)
Comment: Div1/2 and 1/3 are child under Div1. Similarly Div2/1 is child of Div2. Just to show its division
Comment: I am afraid it's not any clearer now.
Comment: I have rephrased my issues , can you pls suggest
Comment: Ill put in a simpler way , pls help in framing logic so that body/div/div is counted. So for ex. from Div1/2 the count =0 . from 1/3 count=1. And from 2/1 count = 0 . Here for 2/1 it should only consider fresh body/div/div and not prev elements for Div1.
Comment: I made 4 suggestions how to make your question clearer; you did not take any of them. I also suggest you learn the meaning of `preceding` and `preceding-sibling` in XPath: https://www.w3.org/TR/xpath/#axes
Answer: I am only guessing here, but it seems to me you want:
<xsl:value-of select="count(ancestor::division[last()]/*[contains(name(), 'topic')])  + count(../preceding-sibling::division)"/>

Untested, because I don't know how to test this against your code.
Comment: Sorry Michael , count(preceding-sibling::division) return 0 everywhere.           The logic count(ancestor::division[last()]/*[contains(name(), 'topic')]) works fine for requirement.                                                                          But all i wanted to have is to count previous div/div tag too. I tried using count(preceding::division[parent::division]) but it adds the count of div/div elements of DIV1 to the result of DIV2. pls help , i am bit lost in this
Comment: I have made changes to the XML , the sample XML does not have any attribute.
Comment: Thanks a lot Michael ! it worked . Thanks for your patience

XPATH : Count Preceding Div element


Using formatting on a string, which has a '%' sign already

Question: I have a string, which stores my SQL code, and it looks like this:
'SELECT * FROM myTable WHERE myField LIKE '%2015' AND myOtherField = %d'

First question: How can I use formatting on myOtherField only? The code below does not seem to ignore the first % sign:
MyQuery.SQL.Add(Format(myString, myNumber));

Second question: How can I use formatting, when I need the first '%' sign for my SQL, but I want to format the date, too? Sadly, the following code does not work either.
'SELECT * FROM myTable WHERE myField LIKE '%%d' AND myOtherField = %d'

Comment: Stop doing this and use parameters. You are safe here, but you are a short step away from SQL injection
Comment: @DavidHeffernan Not to mention that parameterized queries are simply more performant, often significantly so.
Comment: We definitely need a "don't do this!" flag on stack overflow. This is a legitimate but wrong question - don't use string composition for possibly tainted values in SQL (I know %d is mostly safe, but the pattern is still problematic.)
Comment: As for `Format`, the answer to your question can be found in the documentation: http://docwiki.embarcadero.com/Libraries/Seattle/en/System.SysUtils.Format
Answer: MyQuery.Sql.BeginUpdate;
MyQuery.Sql.Add('SELECT * FROM myTable');
MyQuery.Sql.Add('WHERE myField LIKE :MyField AND');
MyQuery.Sql.Add('myOtherField = :MyOtherField');
MyQuery.Sql.EndUpdate;

MyQuery.Parameters[0].Value := '%' + '2015';
MyQuery.Parameters[1].Value := intval;

I suggest this should be the boilerplate solution for all your queries; it requires very little extra effort to do and is more readable than fiddling with quotes and escape characters.
That is not to say the accepted answer is wrong, given the question.
Comment: Alternatively, too, you can index your parameters by name.  This is useful if you change the query later and the order of the parameters changes.  ie: `MyQuery.Parameters.ParamByName('MyField').Value := '%2015'`
Comment: J... is correct - `ParamByName` is particularly common when the sql string is separate from the code that is setting the parameter values.  There is a small performance hit but it is safer.
Comment: Indeed, it can depend on the use case.  I would code for clarity and maintanability first until performance becomes an issue, generally.
Comment: @J... I am not sure I 100% agree with the addition of `BeginUpdate` `EndUpdate` in your edit.  Does that not obfuscate the point a little?
Comment: Why not just do a direct assignment to MyQuery.Sql.Text instead of the .Add()s?
Comment: @MartynA The `Add` method passes by reference - it saves copying and rebuilding intermediate strings, memory allocation, etc.
Comment: @J...I suppose, but I've lost track of the number of times I've seen qs where the OP has constructed a Sql statement which contained syntatic or semantic error that weren't obvious because of Add()ing to the Sql in installments, as it were, whereas if they'd constructed a single string in the first place they could probably have spotted the error in the IDE evaluator themselves.
Comment: @MartynA You can always add a watch on the `.Text` property... I find it cleaner than a sprawling concatenation, personally.
Comment: My (unedited) answer set Sql.Text in one go.  The point that I was trying to make was that parameters are easy to use.  The `BeginUpdate` .. `Endupdate` with `Add()` looks more complex to my eye.
Comment: @HughJones Yes, but you're also suggesting to use this as boilerplate and, if you're going to do that, then you might as well make everything right, no?
Answer: Add an extra % before the % sign you want to keep
Like this:
  Caption := Format('SELECT * FROM myTable WHERE myField LIKE ''%%2015'' AND myOtherField = %d', [4711]);

And same solution to question 2:
  Caption := Format('SELECT * FROM myTable WHERE myField LIKE ''%%%d'' AND myOtherField = %d', [4711, 4712]);

Comment: @Martyn Compare the string here, and that in the code, they are different
Comment: @JensBorrisholt I will, but I have to wait one more minute. :)
Comment: @DavidHeffernan: Yes, I know, I was just meaning that the OP had obviously already tried %% (but presumably had tripped over not escaping the quotes).
Comment: In my second code I was describing, that there is a '%' already, and the code is what I have tried to use, but did not work. This is the answer I was looking for, adding an extra % solves the problem. Thanks!
Comment: @ReyesJ.  I agree you should accept this answer since it directly and correctly answers the question you asked.  As others have noted, however, please take away from this that the above, although it now *works*, is still the *wrong* way to build an SQL query.  Hugh's answer demonstrates the use of parameters to build a query - this is how you should actually solve your problem.


Report Viewer Doesn't Show report created with Sql Server Reporting Services

Question: I created some reports in Sql Server Reporting Services, and deployed them in my local report server.  When I type URL :http://MyreportServer/ReportServer in my browser I can see my deployed reports, and when clicks on reports links they will show correctly.
In my web page I have a report viewer and set report server and report location for it to one of the deployed reports.  When I run the project it does not show any thing and the report viewer seems to be disabled.
How I can solve this problem?
thanks
Comment: Did you use report builder wizard?
Comment: No.I just deploy reports using Deploy item in reporting service project.
Answer: Finally I found stupid problem.
I Have a p tag that not closed in my page.and It cause this problem.


Google maps in jsFiddle

Question: I'm trying to implement Google maps in jsFiddle to ask another question, but I can't see the map:
http://jsfiddle.net/hashie5/aknYP/
I've added the gmap script in the resources
How can I show the map in jsFiddle?
function initialize() {
var myLatlng = new google.maps.LatLng(50.965049,5.484231);
var myOptions = {
  zoom: 14,
  center: myLatlng,
  mapTypeId: google.maps.MapTypeId.ROADMAP
}

var map = new google.maps.Map(document.getElementById("map_canvas"), myOptions);

var contentString =
    '
' +
    'Galaconcert
' +
    'Jaarbeurslaan 2-6
' +
    '3690 Genk' +
    '
'
;

var infowindow = new google.maps.InfoWindow();

var marker = new google.maps.Marker({
    position: myLatlng,
    map: map,
    title: 'Galaconcert'
});
google.maps.event.addListener(marker, 'click', function() {
    infowindow.setContent(contentString);
    infowindow.open(map, marker);
});
}

Comment: You are never calling `initialize()`
Answer: As explained here under "Add Resources" in a red box:

Warning: jsFiddle is recognizing the type of the resource by the
  extension. If you want to use a dynamic resource please add a dummy
  GET variable i.e. http://example.com/download/js/dynamically.js?somevar=somevalue&dummy=.js. This will trick jsFiddle to recognize it as JavaScript resource.

So to make jsFiddle recognize the Google Maps API (which indeed does not have .js extension) you need to add this as the resource: http://maps.googleapis.com/maps/api/js?sensor=false&dummy=.js (This is in addition to calling initialize of course)
Updated fiddle.
Comment: @rbp turns out the OP here manually deleted his fiddles, it's not jsFiddle fault. See [this comment](http://stackoverflow.com/questions/8171371/how-long-does-jsfiddle-host-your-code-for/8191278?noredirect=1#comment23144599_8191278) by jsFiddle creator.
Comment: True, it was obvious jsfiddle won't store all the scripts forever however my answer contains no code change just change in an included resource.
Answer: There you go http://jsfiddle.net/aknYP/4/
You were never calling initialize() nor loading the JS Google Maps API
Comment: +1 , it seems a bug in jsfiddle that `http://maps.google.com/maps/api/js?sensor=false` don't work when added on **Manage Resources** sidepane but works when setting it in a

Google maps in jsFiddle


Find the equation of the plane that passes through two lines.

Question: Find the equation of the plane that passes through

                                               x = 5 + 3t
                                               y = 1
                                               z = -t

and

                                              x = 4 + 3t
                                              y = 2 + 3t
                                              z = 1 - t

Can someone explain why these lines dont intersect, and therefore dont produce a plane equation...
the point of intersection of both lines is when x=5+3t = 4+3t
Comment: Yup. I think that`s it.
Comment: What do you mean with "and therefore don't produce a plane equation"?
Comment: The solution to this problem says that since the two lines don`t intersect there is no plane that passes through both of them.
Comment: $5+3t=4+3t$ is equivalent to $5=4$...I do not understand
Comment: I don`t get it either.
Comment: Ok, there exists no intersection point because there exists no $t$ s.t.  $5+3t=4+3t$, $1=2+3t$ and $-t=1-t$: is this ok?
Answer: Well, looking at the first equation, y = 1 (constant).  So if the lines intersect it will be at y = 1.  So solve the y in the second equation for 1.
$y = 2 + 3t \implies 1 = 2 + 3t \implies -1 = 3t \implies \frac{-1}{3} = t$
If we plug in that t value for the z equations, we see they are unequal.
The first equation:
$z = -t \implies z = \frac{1}{3}$
The second equation:
$z = 1 - t \implies z = 1\frac{1}{3} $
Since, since the lines can never share the same y value and the same z value at the same time, they do not intersect.
That said, since you tagged this problem [linear algebra] and [vector spaces] the work should probably be done with matrices.
A little post script, after seeing a comment that there is no such plane that intersects both lines.  Sorry to answer here I lack the points to comment.  That answer is ether poorly worded or wrong.  There are an infinite number of planes that pass through both lines. Take any point from each line and any other point not colinear with the first two, and you have defined a plane that intersects with the lines at the points you chose.
The answer should be phrased, there exists no plane in which both lines lie.


How to initialize a named volume shared across several containers with docker-compose

Question: I'm trying to build my own wordpress-nginx-php_fpm stack with docker-compose but I face a problem with named-volume and its initialization.
Here is my docker-compose.yml:
<code>version: '2'
services:

  db: #https://hub.docker.com/_/mysql/
    image: mysql
    restart: always
    volumes:
      - "wp-db:/var/lib/mysql:rw"
      - env_file:
      - "./conf/db/mysql.env"
    networks:
      - back

  nginx: #https://hub.docker.com/_/nginx/
    image: nginx
    restart: always
    volumes:
      - "wp-files:/usr/share/nginx/html"
      - "./conf/nginx:/nginx:ro"
      - "./conf/tools:/tools:ro"
    networks:
      - front
      - back
    ports:
      - "8080:80"
    environment:
      - "PHP_FPM_HOST=php-wp:9000"
      - "PHP_FPM_ROOT_DIR=/var/www/html"
    command: "bash /tools/wait-for-it.sh php-wp:9000 -t 30 -- bash /tools/detemplatize-it.sh /nginx/nginx.template:/nginx.conf -- nginx -c /nginx.conf"

  php-wp: #https://hub.docker.com/_/wordpress/
    image: "wordpress:fpm"
    restart: always
    volumes:
      - "wp-files:/var/www/html"
      - "./conf/tools:/tools:ro"
    env_file:
      - "conf/wp/wordpress.env"
    networks:
      - back
    command: "bash /tools/wait-for-it.sh db:3306 -t 30 -- php-fpm -F"

networks:
  front: {}
  back: {}

volumes:
  wp-files: {}
  wp-db: {}
</code>
As you can see, I have 2 named volumes. No problem with "wp-db" because it's used only with the "db" service.
My problem is with the "wp-files" volume, mounted in 2 services(=containers)

Which service will copy its data to the named volume first ?
Does the second container will overwrite the data put by the first one ?
How to "initialize" the named volume somewhere and just use it (after) the 2 containers where its mounted ? I heard about a "nocopy" flag.
Am I obliged to use other stuff (like data container) instead of named volume ?

Thank you.
Note: (everything is on the same physical host)
Answer: Here is what I could find as an answer:

Which service will copy its data to the named volume first ?

The container that starts first (thanks to volumes-from, depends-on, ...)

Does the second container will overwrite the data put by the first one?

No, once a named volume has been "initialized" (means no empty anymore), it will overwrite every mount point where it 's attached to.

How to "initialize" the named volume somewhere and just use it (after)
  the 2 containers where its mounted ? I heard about a "nocopy" flag.

There is indeed a "nocopy" flag, like in the "docker run" documentation, BUT doesn't seem to work with other flag ("ro" or "rw").

Am I obliged to use other stuff (like data container) instead of named
  volume ?

Therefore, no.
Comment: You can use 'nocopy', with other flags like "ro" and "rw", but you have to separate them with "," instead of ":".  For example, 'tesstvol:/test:ro,nocopy'


How to specify a PHP version when using "php" command

Question: I have a server which have both PHP 5.2 and 5.3 on it, default version is 5.2 and whenever i uses  command "php" it takes the 5.2 version. Is there any way i can specify the version while running command? php -v  only display the version.
For application i use .htaccess  to set version, like AddHandler application/x-httpd-php53 .php
Is there any similar way while using command "php"
Answer: /usr/bin/php5.3 but using your path to the correct php version. You can use which php to find the path.
If you don't want to type the full path, you can setup an alias in your .bashrc


CAN I raise custom SCOM Alerts in BizTalk

Question: I need to check that all three of the required files in a set have been FTP'd before doing further processing on them. If all three files for the previous day have not arrived by 08:00 I need to raise and alert to a support person to contact the supplier. (Event 1)
If all three files are there, I then check the totals equal the transactions and if not I discard all three files and raise an alert to a support person to contact the supplier (Event 2)
I have the BizTalk (2010) - SCOM Management Pack loaded and need to know if I can create a custom rule that will raise and event for SCOM to detect, upon which I need SCOM to send a scenario-specific email to a recipient (e.g. Event 1 - All Three Required Files not Available! // or // Event 2 - Sum of transactions cannot be reconciled with Totals File)
We have looked at creating a SQL table for the message event log and defining custom processes for each connection point but it is clumsy, and I was advised that the above manner could be achieved, letting SCOM handle the Alert sending based on Events being raised by the process in BizTalk even though they are more BUSINESS events than SERVICE events.
Please advise - we have been going round in circles with this for two or three weeks now.
Comment: Hi Andrew, I had a similar problem at a client site a few months ago and was trying to achieve an alert through SCOM if a single message had not been received over FTP within a certain time. Unfortunately, I couldn't find any reliable documentation on achieving this with SCOM, so we went down the SQL table route which I am happy to document if you are interested. It certainly wasn't 'clumsy'.
Answer: SCOM doesn't have anything out of the box like you are describing.
I do think you are looking at custom work here as Nick has suggested.
Another option is BizTalk360 does offer this feature.  It is called "Process Monitoring" but would require a software license.
Answer: One option would be to use Business Activity Monitoring (BAM) and raise alerts in there.


remove unnecessary words from string

Question: I want to remove some words from string to make my image alt tag more relevant.
The problem is, if I have some phase like this:
barack obama at a restaurant in venice

I want the exit to be:
barack obama restaurant venice

but what I had is:
barack obamaa restaurantvenice

Code:-
$a = array( ' at ', ' in ', ' arriving ', ' a ', ' leaving ', ' walks ', ' walking ', ' her ' );
$str = 'barack obama at a restaurant in venice';
echo str_replace($a,"",$str);

if I use words without space in the array the result is much horrible:
brck obm resturnt venice

Any ideas?
Comment: `echo str_replace($a," ",$str); add one space inside `""``
Comment: You understand alt text is commonly used by screen readers for visually impaired people? Why would you want to remove words from it?
Comment: isn't it better for image SEO? @iainn
Answer: You have to add one space inside ""like below:-
echo str_replace($a," ",$str);

Check here:-
<?php

$a = array( ' at ', ' in ', ' arriving ', ' a ', ' leaving ', ' walks ', ' walking ', ' her ' );
$str = 'barack obama at a restaurant in venice';
echo str_replace($a," ",$str);

Output:-https://eval.in/636890

remove unnecessary words from string


order of constructor component calls

Question: What order are these parts of a constructor called in?
<code>myClass::myClass(otherClass c) // part 1
 : baseClass(thing)            // part 2
{                              // part 3
    thing = c;
}
</code>
Is it 1, 3, 2?
Comment: Have you tried testing it yourself? Add `println`s to each constructor and see what order they get called in :)
Comment: @KshitijMehta I always thought `println` was a Java thing
Comment: Can't believe I messed that up! I'm programming in Java as I type this, and I just typed println subconsciously! Totally meant `cout`
Answer: 1,2,3.
First arguments are evaluated. Then initialization. Then the body of the function.

Order of arguments is unspecified.
Order of ctor init list is order of members in the class body.
Comment: @SirYakalot The base class constructor finishes before the derived class's is called.
Comment: so you're saying my code would pass an un-initialised `Thing` object to the base class?
Comment: is the body of the function called before the initialisation of the parent class? I mean, do constructors overlap at all in an inheritance hierarchy.
Answer: Part 1 doesn't result in any evaluation; it describes how to call the
function, and names the arguments (which of course must be evaluated to
call the function.
Initialization occurs before the constructor body is entered.  First
all virtual base classes are initialized, in the order resulting from a
depth first left to right search of the inheritance hierarchy, then all
non-virtual base classes are initialized, in the left to right order in
which they appear in the class definition, and finally the members are
initialized, in the order they are declared in the class definition.
Note that the order in which the initializers appear in the constructor
has no effect on the order.
Finally, the constructor body (your part 3) is executed.

order of constructor component calls


error: unknown mnemonic in armv8 when compiling with gcc

Question: I am trying to compile a project with multiple c files and an assembly file written in ARMv8, which I have not done before, so I am having some trouble understanding a few error messages I am getting.
I consistently receive the "unknown mnemonic" error throughout, for almost every single line. For example, for the following snippet of code,
100    ## Save oSum to the stack.
101    pushq %rdx
102
103    ## unsigned long ulCarry
104    subq $8, %rsp
105
106    ## unsigned long ulSum
107    subq $8, %rsp
108
109    ## long lIndex
110    subq $8, %rsp
111
112    ## long lSumLength
113    subq $8, %rsp

I receive the following messages,
file.s:101: Error: unknown mnemonic `pushq' -- `pushq %rdx'
file.s:104: Error: unknown mnemonic `subq' -- `subq $8,%rsp'
file.s:107: Error: unknown mnemonic `subq' -- `subq $8,%rsp'
file.s:110: Error: unknown mnemonic `subq' -- `subq $8,%rsp'
file.s:113: Error: unknown mnemonic `subq' -- `subq $8,%rsp'

What I am I doing wrong here? How can I fix this?
Your help is much appreciated.
Comment: I'm no assembler expert (especially wrt ARM) but they don't look like ARM instructions, to me. They look more like x86-x64 codes.
Comment: You will not be able to assemble this code for armv8 as it is written for amd64.  You'll have to rewrite it from scratch or find a way to compile the project without using this file.
Comment: That hand-written asm is for x86-64 (aka AMD64), not AArch64.  And BTW, it's ridiculously inefficient.  One `subq $32, %rsp` would have the same effect in one instruction, reserving space for multiple local vars.  If the rest of the asm is written that inefficiently, you might be better off with a pure C fallback implementation compiled with an optimizing compiler, if the project has one.
Answer: Problem is actually really simple, what you were typing was 64-bit x86 Assembly, not ARM. The code you've written does have an equivalent in ARM Assembly, but the register names and instruction names will be different. I believe armv8 is a 32-bit architecture (there's a 64-bit version, commonly called AArch64 or arm64.) I'm not familiar with that unfortunately. Try this:
push {r4,r5,r6,lr}  //ARM calling convention requires an even number of regs to be pushed
sub sp,sp,32

Comment: ARMv8 introduces 64-bit mode, as well as adding some new instructions for 32-bit mode.
 When people say "armv8" in a context like this, talking about an architecture, they usually mean it as a synonym for AArch64, even though it doesn't necessarily imply that.  If so, the question is mis-tagged as [arm] instead of [arm64].  Doesn't really change the key point of the answer, though, that x86-64 AT&T syntax isn't any kind of ARM.
Comment: Maybe the person asking was used to using a cross-assembler and got confused?


Show that there are no solutions to $x^2 + y^2 = 3z^2$ in $\Bbb{Z}$

Question: I'm attempting to work through some of the questions in Whitelaw's "Introduction to Abstract Algebra" but am having some difficulty.
The question is as follows

Show that $\forall n \in \Bbb{Z},$ $n^2 \equiv 0\  \text{or}\  1 \mod{3}.$ Show further that $$3\rvert(x^2 + y^2) \implies 3\rvert x\ \text{and}\ 3\rvert y.$$
  Deduce that there is no solution in $\Bbb{Z}$ to the equation
  $$x^2 + y^2 = 3z^2$$
  except $x = y = z = 0.$

For the first part, I simply let $n = 3k,$ $3k+1$ or $3k + 2$ with $k \in \Bbb{Z}$. Hence,
\begin{align}
(3k)^2 &\equiv 0 \mod{3}\\
(3k+1)^2 = 9k^2 + 6k + 1 &\equiv 1\mod{3}\\
(3k+2)^2 = 9k^2 + 12k + 3 + 1 &\equiv 1\mod{3}.
\end{align}
For the second part, since $n^2 \equiv 0$ or $1 \mod{3}$, and that the only case in which $n^2 \equiv 0 \mod{3}$ was when $3\rvert n$, it must then be the case that both $x^2$ and $y^2$ are divisible by $3$ for $3$ to divide their sum (else we get that $x^2 + y^2 \equiv 1$ or $2 \mod{3}$). We also know that if $3\rvert n$ then $3\rvert n^2$. Hence, $3\rvert x$ and $3\rvert y$.
I am, however, unsure how to approach the third part.
Thanks for any help!
Comment: Comment on the proof on the second part: If $3|n$ then $3|n^2$, but you want the converse of that statement. The only way that $3|(x^2+y^2)$ is if $x^2$ and $y^2$ are both $\equiv 0 \mod 3$. Now, use (from what you computed) that $n^2\equiv 0 \mod 3$ if and only if $n\equiv 0 \mod 3$. The idea was correct, but reorganizing might make it a touch clearer.
Comment: Your very last sentence doesn't follow.  If 3|n then 3|n^2, yes, but the converse doesn't follow.(9|36 but 9 doesn't divide 6)  Here we need to state that 3 is prime and if 3|n^2 then 3|n.  And that should give you a hint to the second part.  Everything up to that last sentence was perfect.
Comment: x^2 + y^2 = 0 mod 3 only if both x and y are divisible by 3 as x^2 and y^2 can only be = 0,1 (not -1) mod 3.  So if x^2 + y^2=3z^2 then 3 divides x and y.  So 9 divides x^2 +y^2 so 3 divides z^2 so 9 divides z^2.  so 27 divides x^2 and y^2 and so on ad infinitim.
Comment: I see, thanks for your help. I will amend this in my notes!
Comment: @fleablood, I forgot to add that $3$ is prime, which is a more elegant way of using $3|n^2|$ to establish that $3|n$.
Answer: Let's suppose that we have a solution $(a,b,c)$ where $c\neq 0$ is the smallest positive integer such that the equality holds.
$$a^2 + b^2= 3c^2 \Rightarrow 3|(a^2 + b^2)$$
But we know that this means $3|a$ and $3|b$. However, on squaring we get that $9$ divides the LHS, and so it must also divide the RHS:
$$a'^2 + b'^2 = \frac{c^2}{3}$$
Where $3a' = a, 3b' = b$. But $3$ is prime, so if it divides the product it must divide $c$. Factor that out as $3c'$ and divide:
$$a'^2 + b'^2 = 3c'^2$$
But this is another solution triple with $c' < c$ (and also $c' \neq 0$), and we chose the smallest possible $c$, so we have a contradiction.


Classes que implementam interfaces são consideradas subclasses?

Question: Tenho uma interface Veiculo, uma classe Peugeot que implementa Veiculo. Esse Peugeot é considerado subclasse?
Answer: Conceitos
Vamos conceituar as coisas. Embora nem todo mundo concorde com essas definições, vou colocar um entendimento que muitos têm.
Subclasse
É a capacidade de reusar código de algo existente.
É algo interno para a classe.
É ela mesma "copiando" seletivamente o que existe na classe base. Ela pega os campos e os métodos que existem na classe e os toma como se fossem seus.
Normalmente se dá por herança. Isto é, uma classe deriva de outra classe.
Ao contrário do que muita gente pensa, até mesmo as partes privadas de uma classe são herdadas pela classe que a herda, apenas a visibilidade fica limitada.
Subtipo
É a capacidade de um tipo se passar por outro, de alguma forma o substituindo em determinadas situações como se fosse o próprio tipo esperado.
É algo externo de quem consome o tipo.
Um método espera um objeto de um tipo, mas ele pode receber um objeto de outro tipo que atenda os requisitos do tipo originalmente aceito.
Normalmente se dá pelo polimorfismo. Pode ser de uma classe concreta, uma classe abstrata, uma interface ou um trait (em Java 8 é possível algo parecido), entre outras possibilidades em algumas linguagens.
Note que por ser algo de uso externo tudo que se refere ao subtipo deve ser obrigatoriamente público.
As classes costumam ter o subtipo estrutural e o subtipo comportamental. A interface, pura,  é apenas um contrato, é sobre tipagem e não sobre implementação, não tem nem estado, nem comportamento.
Java só trabalha com subtipo nominal.
Herança
É comum quando tratamos de herança de estarmos falando das duas coisas, apesar de serem conceitos separados. Tem linguagem que faz subclasse sem subtipo (não Java). Algumas possuem conceitos totalmente distintos, mas não vou entrar em detalhes, não é o foco da pergunta.
Por isso alguns dizem que Java não permite múltipla herança e outros dizem que sim. Depende da definição. Java permite simples subclasse e múltiplos subtipos.
A implementação de interface é uma subclasse?
Em parte a pergunta, não muito clara mas respondível, já está respondida. Interfaces tem mais a ver com subtipo e não com subclasse. Mas em Java 8 é possível ter interfaces que se comportam como subclasses. Mas apenas quando se usa os métodos default, mas vamos deixar isso de lado para facilitar.
Então:
public interface MeioTransporte { ... }
public interface Veiculo extends MeioTransporte { ... } //subtipo
public class Carro implements Veiculo { ... } //sutipo
public class Peugeot extends Carro { ... } //subclasse e subtipo

Coloquei no GitHub para referência futura.
Os exemplos não são bons, mas segui a pergunta. Veiculo dificilmente seria uma interface.
Classes que implementam interfaces estão apenas fazendo subtipo, portanto as interfaces são supertipos de classes ou de outras interfaces.
Conclusão
Vou reforçar que é difícil fazer subclasse e subtipo certo. A maioria dos programadores não tem a menor noção de como fazer certo, muitos sequer sabem o que significa OOP. Mesmo programadores experientes têm dificuldade de fazer certo em vários casos. Então muito cuidado para manipular essa "arma".
Comment: Perfeito!!!!!!!
Comment: Vai me ajudar em uma apresentação que farei hj.