Correct methodology using $g$-computation to estimate Average Treatment Effect on the Treated ($ATT$)?

Question: I have a question about the $g$-computation methodology for estimating the Average Treatment Effect on the Treated ($ATT$) in the following article.
The authors recommend estimating the $ATT$ by first performing a regression model for $Y$ on the binary treatment $A$ and covariates $X$, then finding $E(Y^1|A=1)$ from the observed values of $Y|A=1$ and estimating the counterfactual $E(Y^0|A=1)$ by imputing the predicted value of $Y$ from the regression formula by substituting $A=0$ into the regression formula, but only for the observed $A=1$ records. So far so good.
For the next step, Wang et al. stack the records with the $E(Y^1|A=1)$ and $E(Y^0|A=1)$ estimates into a single dataset coupled with their $A$ values of 1 and 0, respectively. They then regress $Y$ on $A$, where $ATT$ is estimated from the regression coefficient on $A$.
Rather than regress $Y^*$ on $A$ why not simply compute the average of the difference $E(Y^1|A=1) - E(Y^0|A=1)$ for all records where $A=1$?
When I run a simulation the two methods return different results for the ATT point estimates - see R code below on a simulated dataset.
# Simulation data from "Targeted Maximum Likelihood Estimation for Causal Inference in    #
# Observational Studies", Schuler & Rose, 2016.                                           #
x1 <- rbinom(n=10000, size=1, prob=0.55)
x2 <- rbinom(n=10000, size=1, prob=0.3)

# Binary treatment variable
A <- rbinom(n=10000, size=1, prob=exp(-.5 + .75*x1 + x2)/(1 + exp(-.5 + .75*x1 + x2)))

# Outcome variable
Y <- rnorm(n=10000, mean=24 - 3*A + 3*x1 - 4*x2 + 7*x1*x2 + 5*A*x1 - 10*A*x2 + 15*A*x1*x2, sd=4.5)

# Create data frame
df <- data.frame(cbind(Y, A, x1, x2))

### Run linear regression
lm_test <- lm(Y ~ . + x1*x2 + A*x1 + A*x2 + A*x1*x2, data=df)

### Calculate ATT using g-computation. ###
# ATT = Expected difference between observed and counterfactual Y values at record level. #
df_trmt <- df[df$A==1,]
df_a0 <- df_trmt; df_a0$A = 0;
y_a0 = predict(lm_test, newdata=df_a0)
y_a1 = df_trmt$Y

df_preds <- data.frame(cbind(df_trmt, y_a0, y_a1))

df_preds$att <- y_a1 - y_a0
mean(df_preds$att)
# [1] -0.284822 <==== ATT estimate
sqrt(var(df_preds$att)/length(df_preds$att))
# [1] 0.08707805

# Alternate g-computation calculation of ATT (based on Wang et al. (2017) article). #
df_a0_2 = data.frame(cbind(Y=y_a0, A=df_a0$A))
df_a1_2 = data.frame(cbind(Y=y_a1, A=df_trmt$A))
df_preds_2 <- data.frame(rbind(df_a0_2, df_preds_2))
lm_test_2 <- lm(Y ~ A, data=df2)
summary(lm_test_2)
# Call:
#   lm(formula = Y ~ A, data = df2)
#
# Residuals:
#   Min      1Q  Median      3Q     Max
# -32.536  -4.337   0.592   5.399  24.712
#
# Coefficients:
#   Estimate Std. Error t value Pr(>|t|)
# (Intercept)  25.1946     0.1265 199.207  < 2e-16 ***
#   A             0.5096     0.1723   2.957  0.00311 ** <==== ATT estimate
#   ---
#   Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#
# Residual standard error: 8.591 on 9998 degrees of freedom
# Multiple R-squared:  0.0008738,   Adjusted R-squared:  0.0007739
# F-statistic: 8.744 on 1 and 9998 DF,  p-value: 0.003113

Answer: I think you have a bug in your code. I don't remember what I changed, but your code doesn't even run in a fresh R session. Logic demands that the results should be equal, and that is what I get:
set.seed(666)

# Simulation data from "Targeted Maximum Likelihood Estimation for Causal Inference in    #
# Observational Studies", Schuler & Rose, 2016.                                           #
x1 <- rbinom(n=10000, size=1, prob=0.55)
x2 <- rbinom(n=10000, size=1, prob=0.3)

# Binary treatment variable
A <- rbinom(n=10000, size=1, prob=exp(-.5 + .75*x1 + x2)/(1 + exp(-.5 + .75*x1 + x2)))

# Outcome variable
Y <- rnorm(n=10000, mean=24 - 3*A + 3*x1 - 4*x2 + 7*x1*x2 + 5*A*x1 - 10*A*x2 + 15*A*x1*x2, sd=4.5)

# Create data frame
df <- data.frame(cbind(Y, A, x1, x2))

### Run linear regression
lm_test <- lm(Y ~ . + x1*x2 + A*x1 + A*x2 + A*x1*x2, data=df)

### Calculate ATT using g-computation. ###
# ATT = Expected difference between observed and counterfactual Y values at record level. #
df_trmt <- df[df$A==1,]
df_a0 <- df_trmt; df_a0$A = 0;
y_a0 = predict(lm_test, newdata=df_a0)
y_a1 = df_trmt$Y

df_preds <- data.frame(cbind(df_trmt, y_a0, y_a1))

df_preds$att <- y_a1 - y_a0

# Alternate g-computation calculation of ATT (based on Wang et al. (2017) article). #
df_a0_2 = data.frame(cbind(Y=y_a0, A=0))
df_a1_2 = data.frame(cbind(Y=y_a1, A=1))
df_preds_2 <- data.frame(rbind(df_a0_2, df_a1_2))
lm_test_2 <- lm(Y ~ A, data=df_preds_2)

summary(lm_test_2)
# Call:
#   lm(formula = Y ~ A, data = df_preds_2)
#
# Residuals:
#   Min      1Q  Median      3Q     Max
# -33.951  -2.485   1.001   4.143  24.921
#
# Coefficients:
#   Estimate Std. Error t value Pr(>|t|)
# (Intercept)  26.0230     0.1060 245.597   <2e-16 ***
#   A            -0.3081     0.1498  -2.056   0.0398 *
#   ---
#   Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#
# Residual standard error: 7.911 on 11146 degrees of freedom
# Multiple R-squared:  0.0003792,   Adjusted R-squared:  0.0002896
# F-statistic: 4.229 on 1 and 11146 DF,  p-value: 0.03977
mean(y_a1 - y_a0)
# [1] -0.3081427

The standard errors don't agree, but of course neither standard error is correct, so this isn't much of a concern to me.
Comment: That worked, thank you! Not sure where I messed up in my code. Yea the standard errors aren't correct - in the paper the authors used bootstrapping to estimate the SE.


Chain rule used to simplify partial derivatives

Question: This is an integral in my lecture and they said that the chain rule is used here to produce the second equation.
I understand the chain rule with single variables however I am baffled here and how the chain rule applies to multivariable equations since I don't think we can 'cancel out' anything due to partial derivatives being different to derivatives.
Answer: So the normal chain rule is
$$ \frac{d}{dx} f(g(x)) = f'(g(x))g'(x)$$
In words: the rate of change of $f(g(x))$ in terms of $x$ is equal to the rate of $f$ in terms of $g(x)$ times the rate of change of $g$ in terms of $x$.
Now we extend this idea to multiple variables:
If
$$ \phi(g(x), y, z) $$
then
$$ \frac{\partial}{\partial x} \phi(g(x), y, z) = \frac{\partial}{\partial g(x)} \phi(g(x), y, z) \cdot g'(x)$$
In words: the rate of change of $\phi$ in terms of $x$ is equal to the rate of change of $\phi$ in terms of $g(x)$ (assuming $y$ and $z$ remain unchanged) times the rate of change of $g$ in terms of $x$.
Now in your example we want to differentiate  $\phi(x(s), y(s), z(s))$ in terms of $s$. The same principle applies, only $x$, $y$ and $z$ all depend on $s$ so
$$ \frac{d}{ds} \phi(x(s), y(s), z(s))
= \frac{\partial\phi}{\partial x}\cdot x'(s) +
\frac{\partial\phi}{\partial y}\cdot y'(s) +
\frac{\partial\phi}{\partial z}\cdot z'(s)$$
Answer: The chain rule for multivariate maps states that the Jacobian of a map that is the two composition of two maps is the (matrix) product of the Jacobians of two maps.
This is what is applied in the special case you mention in your question.
Answer:
I don't think we can 'cancel out' anything

Well 'cancel out' is not exactly what is going on.  Leibniz notation is not actually a ratio; its an analogy.
However, this analogy does make the chain rule easier to remember.   So in the same manner, you may present the multivariate chain rule as a dot product of vectors.
$$\begin{align}\dfrac{\mathrm d \varphi}{\mathrm d s} &= \dfrac{\partial \varphi}{\partial\langle x,y,z\rangle}\cdot\dfrac{\mathrm d\langle x,y,z\rangle}{\mathrm d s}\\[2ex]&=\left\langle\dfrac{\partial\varphi}{\partial x},\dfrac{\partial\varphi}{\partial y},\dfrac{\partial\varphi}{\partial z}\right\rangle\cdot\left\langle\dfrac{\mathrm d x}{\mathrm d s}, \dfrac{\mathrm d y}{\mathrm d s},\dfrac{\mathrm d z}{\mathrm d s}\right\rangle\\[2ex]&=\dfrac{\partial\varphi}{\partial x}\dfrac{\mathrm d x}{\mathrm d s}+\dfrac{\partial\varphi}{\partial y}\dfrac{\mathrm d y}{\mathrm d s}+\dfrac{\partial\varphi}{\partial z}\dfrac{\mathrm d z}{\mathrm d s}\end{align}$$

due to partial derivatives being different to derivatives.

The partial differentiation of a monovariate function equals the differentiation, with respect to the argument.$$\dfrac{\partial f(s)}{\partial s}=\dfrac{\mathrm d f(s)}{\mathrm d s}$$


Prove/Disprove that $B \subset f(f^{-1}(B))$

Question: $f:X \rightarrow Y$ where $A \subset X$ and $B \subset Y$
Prove (if it is correct) that $$B \subset f(f^{-1}(B))$$
This is my approach
Proof
Choose an arbitrary element $y \in B$, by the definition of inverse image, we know that $x \in f^{-1}(B)$ such that $f(x)=y$. As $y \in B$ and $x \in f^{-1}(B)$, we can conclude that $y \in f(f^{-1}(B))$.
Is this correct?
Comment: Your mistake: $y\in B$ does not ensure the existence of an $x$ with $f(x)=y$. This because $f$ is not necessarily surjective so that it might happen that $y$ is not in the range of $f$.
Answer: This is false. Let $A = X = \{0\}$, $Y = \{0, 1\}$ and $B = \{1\}$. Let $f : X \to Y$ be the identity function. Then $f^{-1}(B) = \emptyset$, so $f(f^{-1}(B)) = \emptyset$ but $B$ is not empty.

The error in your reasoning is the statement "...by the definition of inverse image, we know that $x \in f^{-1}(B)$ such that $f(x) = y$". This is not true because the definition of inverse image states that:
$$
f^{-1}(B) = \{x \in X \mid f(x) \in B\}
$$
This definition asserts that $f$ maps every element in $f^{-1}(B)$ to $B$, but not necessarily to every element in $B$.
As such, if you assume that $f$ is surjective, then your statement holds - you can safely conclude that such an $x$ which $f(x) = y$ exists.
Comment: That's more of an illustration of the idea to why the statement is false than a concrete counterexample, but your idea in the illustration is correct.
Comment: Thank you for your clean explanation. Is this counterexample I made, valid? https://imgur.com/2SZs7IH
Answer: The statement you provide is in general wrong. It is the other inclusion that always holds.


Unit test a plugin

Question: I want to add a small unit test to a migrate plugin for a migration I am building. I looked for some examples but did not find much for unit tests of a plugin.
I want to run some assertions when the plugin is instantiated and check its values in a unit test. Is PluginTestBase the class to use? Can I not simply run a test without bootstrapping Drupal?
class EndpointTest extends PluginTestBase {
  // not sure what to do here
}

Answer: The test class PluginTestBase is a kernel test (which bootstraps Drupal, has a db, and limited schemas/configs installed). This can be valuable as they are faster than full functional tests, and require less service mocking than unit tests do for plugins that rely heavily on (for instance) the database or entity API.
That being said, you can write traditional unit tests for plugins depending on how they are written. Here's a few examples I am aware of:

Message module
Message Notify
Message Digest

Unit test a plugin


Approaching a Cohomology Computation

Question: If $U \subseteq \mathbb{R}^2$ is the complement of $d > 0$ points in the plane, and $H^k$ denotes the $k$-th cohomology group, how should I verify that $H^k (U)$ equals $\mathbb{R}$, $\mathbb{R}^d$, or $0$ if $k$ is equal to 0, equal to 1, or $> 1$, respectively? Will the computation play out analogous to a homology computation? As a knee-jerk reaction, I was thinking induction might be of service here, but I am not sure.
Any constructive feedback would be appreciated!
Comment: There's a version of Mayer–Vietoris for cohomology, and both homology and cohomology are invariant under homotopy equivalence.
Answer: Your intuition is correct, induction on $d$ is a straightforward way to go
The Mayer-Vietoris sequence for cohomology is
$\dots\rightarrow H^{i}(X)\rightarrow H^i(U)\oplus H^i(V)\rightarrow H^i (U\cap V)\rightarrow H^{i+1}(X)\rightarrow\dots$
By homotopy invariance, the base case follows from knowing the cohomology of the circle (also a Mayer-Vietoris computation).
Since $H^*$ is invariant under homotopy we can arrange that the deleted points are $(1,0),\dots,(d,0)$.  Then apply the cohomology Mayer-Vietoris sequence to the subspaces $U=\{ (x,y)\ |\ x<d \}$ (containing the first $d-1$ deleted points) and $V=\{ (x,y)\ |\ x>d-1 \}$.  We see that $H^i(U\cap V)=0$ for $i>0$ (since $U\cap V$ is contractible) and hence  $H^i(X)\cong H^i(U)\oplus H^i(V)$for all $i>1$ (where $X$ is $\mathbb{R}$ with $d$ points deleted).  This gives the result for $i>1$.
The rest of the sequence looks like
$0\rightarrow H^0(X)\rightarrow H^0(U)\oplus H^0(V)\rightarrow H^0 (U\cap V)\rightarrow H^1(X)\rightarrow H^1(U)\oplus H^1(V)\rightarrow 0$, so in particular there is a surjection $h:H^1(X)\rightarrow H^1(U)\oplus H^1(V)$.  If we can show that the map $H^0(U)\oplus H^0(V)\rightarrow H^0 (U\cap V)$ is also a surjection (I'm not sure how to do this part right now...), we will know $h$ is an isomorphism, giving the result for $i=1$.  We already know the result for $i=0$ since $H^0(X)=\mathbb{R}$ for any connected space.

Approaching a Cohomology Computation


A variation of the Lady Tea Tasting problem

Question: I am quite stuck on this question. My intuition would have been to say that we would prefer to pour milk first on heads, but I am not sure.
The setting is: the Lady guesses "milk first" in 2/3 of the cases and we have a coin that lands heads 2/3 of the time. Then for each of the 8 cups, you flip the coin and pour milk first or tea first depending on the outcome of the coin flip, would you prefer to pour milk first on heads or pour tea first on heads.
Thank you for your help.
Comment: BTW, although the original experiment was hypergeometric, this one is not: the Binomial distribution is the relevant one.
Comment: To do what? To guess what the Lady guesses, perhaps? Unclear. Like the actor asks, "What's my motive?"
Comment: I agree with @Carl.  I have considered offering an answer, but in framing it out I realized the experimental protocols are both (a) different from the original and (b) not completely or clearly set out (for instance, does the lady know you will be pouring randomly?), thereby making it unclear what the experiment is. I also saw that what constitutes significant evidence of the lady's prowess depends on the experiment, so I don't even know for sure what a "significant" result would be. Even the null hypothesis is unclear: does it make sense to test that the lady guesses 2/3s of the time?
Comment: Let me elaborate on that last point.  The lady in the original experiment knew the protocols and had an interest in performing well. This is effectively a two-person game in which we can find the optimal response for her, assuming she can't truly tell which liquid was poured first (and knows that). It is *not* optimal to guess milk two-thirds of the time: it is best, when she knows milk will be poured first with a chance of 2/3, to *always* guess milk. She can expect to get $8\times 2/3\approx 5$ correct answers and has a 4% chance of getting all 8 of them right.
Comment: Oh I see, thank you so much for your help! :)
Comment: *"she  states that she will choose at most six cups to be milk-first and at most six cups to be tea-first"* I don't follow this statement. Is this a stopping rule? After six cups of either type she stops tasting, or is it that after six cups she continues but will be wrong?
Comment: Which two mistakes does she make? If she already says that she is only selecting up to 6 cups then the error can be just due to that.
Comment: In the sense that she thinks any more than that is "far too unlikely to happen" and then she actually makes two mistakes, by misclassifying.
Answer: Note, the question has been merged and the text has changed. This answer is the answer to the following version: A modified Lady Tasting Tea problem

Let's say that a taster would completely randomly order the tea cups using a normal distribution and call cups that draw a positive figure milk first and a negative figure tea first. Except when there would be 6 of such category in which case we select only the outer 2 of the ordering. This would be the process of a taster that is completely unaware of the taste.
If the taster would not have the six cups rule then the probability of 2 mistakes would be binomial distributed. For each cup the taster independently selects correct or wrong with fifty-fifty probability.
But with the six limit there are cases when there are seven or eight cups of one kind. and the taster wrongly orders them automatically because of the six rule. You might think that this alters the probabilities in a similar fashion to the principle behind the geometric distribution where the probability is 1/70 to have everything correct instead of the binomial distribution all correct ${1}/{2^8}$.
Interestingly, it does't matter. The probability of two mistakes is still a binomial distribution.
We can have a similar experimental process where we have the taster first select the groups and then afterwards the experimenter randomly decides which cup is tea first or milk first (If the null hypothesis is correct, then it doesn't matter what happens first). Because this experimental setup (deciding whether a cup is milk/tea first) happens with independent coin flips the probability of two mistakes will be according to a binomial distribution.

I see now that you are considering the conditional situation:
"Then by chance only one of the cups is tea- first and seven of the cups are milk-first"
To compute this we have to count the ways that the taster can order the cups. But now, for this conditional case, the process of an unaware taster plays a role.
Above, the unaware taster uses a continuous random distribution to order the cups and select the two groups. But the number of cases where the groups with two/six distribution occur can differ depending on what the taster does when they encounter one/seven or zero/eight in the random draws from the normal distribution. Are they just selecting the two extremes, or are they repeating the random draws?
The differences in this process will change the probability for the taster to select a two/six combination. A different probability for the taster to select a two/six combination changes the probability for the taster to make two mistakes (the taster must select a two/six combination in order to be able to have only 2 mistakes if the reality is one/seven).
But let's ignore this for the moment and consider that the taster will flip coins for each cup and if there are 2 or less of one group, then they will randomly select from the other group to fill up. In this case the probability that the taster will have 2 tea-first cups and 6 milk-first cups is the probability that the coin flips ends with 0, 1 or 2 tea first. This probability is 37/256. Then the probability that the right tea-first cup is among those 2 selected tea-first cups is 1/4 and so the probability of only two mistakes is 37/1024.
Comment: Thank you for your help!


What makes african blues so distinctive?

Question: When you listen to the music by Ali Farka Toure, Rokia Traore, Afel Bocoum and other african blues artists you can recognize it in a second.
What is it what makes the african blues so distinctive?
Is it tuning, scales, rhytm, finger technique ... ?
Well I'm pretty sure it's the combination of all these, but I would be glad if someone could shed some light into my little african mystery with more then this common sense.
Please provide some examples of these tunings or scales if you want to make me really happy.
Thanks a lot for your input!
PS: I've been quite close few times just by playing a typical blues pentatonic scale in the 'right' mood :D
but I'm sure there's a much more to explore...
Answer: Listening to some Ali Farka Toure in order to have something fresh in my head to comment.
First, I'd say that the 12-bar progression we're all familiar with is absent, replaced with repeating measures that are more a part of funk, although the blues history videos seem to tie to John Lee Hooker. You don't get the familiar shuffle, either, and I'm actually not hearing drums.  The two tracks I've pulled up (From In the Heart of the Moon, if you're curious) sound more like I-IV vamps.
The instrumentation I'm hearing sounds like the melody instrument is a thumb piano and not a guitar, but I could be wrong. If you want to see as well as hear some of this kind of music being made, Bela Fleck made a movie about going to Africa and playing with African musicians, called Throw Down Your Heart, that's on Netflix Instant.
And there's an attitude difference here, too. I think there's a gunslinger vibe connected to blues guitar that I'm just not hearing here. It comes off more jamband than blues, in that respect.
Comment: Blues as we know it stems from slavery and the work songs produced by those people from Africa who were enslaved.   Although there is a heavy (Mostly Rhythmic)African influence in the early work songs and subsequent blues; its very different from the kind of music free African people were creating at that time.  More recent African music has progressed from this with only a minor western influence.
Answer: I claim that it is the rhythms. One book that is aimed at drummers will give lots of examples that I think will help demonstrate the different beats is West African Rhythms for Drumset. One drummer (who does just heavy metal) claims the rhythms are too complicated for him. Another drummer (who does just jazz) claims that they're not all that hard, but are quite different from what he normally plays. Your mileage may vary.


Missing Schema on custom field only during PHPUnit Test

Question: I have a custom field setup that I created similar to this tutorial. I exported all my fields on my node to a test module and enable it during my PHPUnit test.
I run my test with the following command:
php ./core/scripts/run-tests.sh --verbose --php /usr/local/bin/php --url http://nginx --class "Drupal\my_module\Tests\ExpenseTest"

And I get the following error:
The test did not complete due to a fatal error.
Exception Uncaught e ConfigSchemaCheck   95 Drupal\Core\Config\Development\Conf
Drupal\Core\Config\Schema\SchemaIncompleteException: Schema errors for
core.entity_form_display.paragraph.my_field.default with the
following errors:
core.entity_form_display.paragraph.my_field.default:content.field_city_state.settings.size
missing schema,
core.entity_form_display.paragraph.my_field.default:content.field_city_state.settings.autocomplete_route_name
missing schema,
core.entity_form_display.paragraph.my_field.default:content.field_city_state.settings.placeholder
missing schema in
Drupal\Core\Config\Development\ConfigSchemaChecker->onConfigSave() (line
95 of

This is part of my core.entity_form_display.paragraph.my_field.default.yml file:
field_city_state:
weight: 2
settings:
  size: '60'
  autocomplete_route_name: my_module.city_autocomplete
  placeholder: ''
third_party_settings: {  }
type: city_state_default
region: content

If I remove the settings it doesn't give me the error (Although I am not testing that field yet).
This is my module's Field Type Class:
<?php

namespace Drupal\my_module\Plugin\Field\FieldType;

use Drupal\Core\Field\FieldItemBase;
use Drupal\Core\Field\FieldStorageDefinitionInterface;
use Drupal\Core\TypedData\DataDefinition;
use Drupal\Core\Field\FieldItemInterface;

/**
 * Plugin implementation of the 'city_state' field type.
 *
 * @FieldType(
 *   id = "city_state",
 *   label = @Translation("City/State"),
 *   description = @Translation("An entity field containing city/state"),
 *   category = @Translation("Address"),
 *   default_widget = "city_state_default",
 *   default_formatter = "city_state_default"
 * )
 */
class CityStateItem extends FieldItemBase implements FieldItemInterface {

  /**
   * {@inheritdoc}
   */
  public static function schema(FieldStorageDefinitionInterface $field_definition) {
    return [
      'columns' => [
        'state_code' => [
          'type' => 'varchar',
          'length' => 2,
        ],
        'city' => [
          'type' => 'varchar',
          'length' => 255,
        ],
      ],
    ];
  }

  /**
   * {@inheritdoc}
   */
  public static function propertyDefinitions(FieldStorageDefinitionInterface $field_definition) {
    $properties['state_code'] = DataDefinition::create('string')->setLabel(t('State Code'));
    $properties['city'] = DataDefinition::create('string')->setLabel(t('City'));

    return $properties;
  }

  /**
   * {@inheritdoc}
   */
  public function isEmpty() {
    $value = $this->state_code;
    return $value === NULL || $value === '';
  }

  /**
   * Get city string.
   *
   * @return string
   *   Returns city name.
   */
  public function getCity() {
    return $this->city;
  }

  /**
   * Get state code string.
   *
   * @return string
   *   Returns the state code.
   */
  public function getStateCode() {
    return $this->state_code;
  }

}

This is my module's schema file:
city_state:
  type: mapping
  label: 'City/State settings'
  mapping:
    state_code:
      type: string
      label: 'State Code'
    city:
      type: string
      label: 'City'

Don't understand why I am getting this error and only during Testing. (No issues when running config import on dev and staging).
Comment: After removing the settings items and just have `settings: { }` I was able to still save to the custom field.
Answer: You only get that error during testing because the core testing base classes add a config schema checker during testing.
You have 2 options, the first is the easiest, but is not recommended, and that is to disable the strict config checking in your test by setting this variable to false in your test class:
  /**
   * Set to TRUE to strict check all configuration saved.
   *
   * @see \Drupal\Core\Config\Development\ConfigSchemaChecker
   *
   * @var bool
   */
  protected $strictConfigSchema = FALSE;

This approach is not preferred because you then have invalid config schema data in your module. It would be far better to fix the schema.
Instead of city_state: in your schema file, something more specific like:
field.field_settings.city_state:

or since this appears to be display related, it might be
field.formatter.settings.city_state:


Why this formula for 5 year trend?

Question: I've opened a spreadsheet that has a 5-year trend formula that I don't understand.
The sample data for $2012$ to $2016$ are: $190, 175, 160, 165, 200$.
The formula in the spreadsheet is:
$trend = 0.1*(-2*190+-1*175+0*160+1*165+2*200)/average(190, 175, 160, 165, 200) = 0.6\%$.
My questions are:

Is this formula standard practice?
Why are the weights ${-2,-1,0,1,2}$ used?
Why is the quantity multiplied by $0.1$?
Comment: I assume you can't ask the creator of the spreadsheet?
Comment: Note that the method in question outputs 9.5% for an exact 10% per annum growth. For smaller percentages, it seems to be a feasible approximation
Answer: The formula is the linear regression of the series (whatever the variable is), which I will call $y$, on time ($x$), giving units-of-$y$ per year. This is then divided by average $y$ and the result given as a percentage to get a percentage annual growth relative to the mean $y$ (the estimate of the middle-year value of $y$).
Note that the slope of a regression line is $$\hat{\beta}=\frac{\sum_i (y_i-\bar{y}) (x_i-\bar{x})}{\sum_i (x_i-\bar{x})^2}\,.$$
Now for the $x$'s being five consecutive years $(x_i-\bar{x})$ will always be $-2,-1,0,1,2$ and $\sum_i (x_i-\bar{x})^2=10$.
At the same time
\begin{eqnarray}
\sum_i (y_i-\bar{y}) (x_i-\bar{x})&=&\sum_i y_i (x_i-\bar{x}) - \bar{y}\sum_i (x_i-\bar{x})\\
&=&\sum_i y_i (x_i-\bar{x})\,,
\end{eqnarray}
but again the $x_i-\bar{x}$'s are just $-2, ..., 2$ so that's $-2 \, y_1 - 1 \, y_2 + 0 \, y_3 + 1 \, y_4 + 2 \, y_5$.
(Note that $\sum_i (x_i-\bar{x})$ is always $0$, which is why the second term disappeared.)
Consequently the estimated yearly increase using a least squares straight line fit is $\hat{\beta}= (-2 \, y_1 - 1 \, y_2 + 0 \, y_3 + 1 \, y_4 + 2 \, y_5)/10$. This will be in units-of-$y$ per year.
Now to get this to be a yearly relative (or percentage) increase we need to divide by a $y$-value. The mean of the $y$'s, $\bar{y}$ (which is also the estimated middle value in the trend line) is what they've divided by. I don't know why that particular choice was made, rather than some other possible choice --  but for some things I guess this would be a reasonable thing to do.
I don't know if this is standard, but I bet you could find a formula like this (estimated annual linear increase divided by the mean) in some business stats books or a similar source.
Comment: So we use a regression for the *linear* growth and try to estimate an *exponential* growth with it? I suppose, it would be more adequate to make a linear regression of the $\ln y_i$ and then use $\exp \tilde\beta$
Comment: @icc97 a product of two things is zero only if at least one of the two terms in the product is zero. In this case it's easy to figure out which of the two must be 0 ($\bar{y}$ certainly isn't in the example, so that's a big clue which term to focus on), and then to see that it will always be so. I can edit a brief comment in though.
Comment: I was just trying to explain it for others, I wasn't initially sure why the $\bar{y}$ term drops out. Once seen it's obvious enough. Might also help slightly if you put the final equation of that line on a separate line
Comment: Yes, understood; I was simply explaining how people should be able to see it quickly
Comment: The only step that is a bit harder to follow is that `y_bar * sum(x - x_bar) = 0` because `sum(x - x_bar) = 0`

Why this formula for 5 year trend?


Utilizando case em campo null

Question: Tenho a seguinte estrutura tb_boletim:
|Cod|boletim|dt_enviado|
| 1 |  N    | NULL     |
| 2 |  S    |24/08/2018|
| 3 |  S    | NULL     |
| 4 |  S    |23/08/2018|
| 5 |  S    |23/08/2018|
| 6 |  S    | NULL     |
| 7 |  N    | NULL     |
| 8 |  N    | NULL     |

Quero fazer SOMENTE um sql que me traga de duas formas, dependendo parâmetro passado para o campo "boletim".
Consigo fazer um case no where tipo assim?
select * from tb_boletim where
boletim = 'S' and
case
 when boletim = 'S' then dt_enviado is not null
 when boletim = 'N' then dt_enviado null
end

Resultado esperado:
|Cod|boletim|dt_enviado|
| 2 |  S    |24/08/2018|
| 4 |  S    |23/08/2018|
| 5 |  S    |23/08/2018|

ou
|Cod|boletim|dt_enviado|
| 1 |  N    | NULL     |
| 7 |  N    | NULL     |
| 8 |  N    | NULL     |

Answer: Sim, consegue.
O parâmetro recebido (@parametro_boletim) controlará se a validação será feita com ele igual a S ou a N):
SELECT *
FROM tb_boletim
WHERE boletim = @parametro_boletim
  ((boletim = 'S' and dt_enviado is not null) OR
   (boletim = 'N' and dt_enviado null))

Answer: Não sei se percebi completamente a questão…
Será isto que pretende?

select
   Cod
  ,boletim
  ,case when boletim='S' then dt_enviado else NULL END dt_enviado
FROM tb_boletim

/*WHERE BOLETIM='S'*/
/*WHERE BOLETIM='N'*/

Pode acrescentar uma clausula WHERE para filtrar ou não de acordo com o campo BOLETIM

Utilizando case em campo null


Dropdown menu with descriptions only in sub-menu elements in Drupal 7?

Question: I'm trying to implement a dropdown menu in my custom theme. At the moment, this means I have a region for the primary and another region for the secondary menu.
According to the design I would like implement, the description of a menu element should be visible - but only for sub-menu elements, like in this menu:

My problem is the following:
I can't find a way to handle different levels of menus accordingly, i.e. to add the markup of the descriptions only where necessary and the different classes on ULs.
How should I implement this kind of menu?
[I've installed the Menu Block module for some help with this. It adds several new hooks that might be useful, but we will see...]
Thanks for checking out this question!
Comment: So, the text "Small Example" would be the child menu item of "Nav Item 2" (and the example text would be the description for the menu item "small example") and would be exposed when "Nav Item 2" is hovered on? What would happen to a tertiary level menu item? And these are both part of the site's Main Menu? How is the secondary menu relevant in this situation? What have you tried so far to accomplish this? Most likely you will need to implement a customization of theme_menu_link() in order to add in the description for the secondary-level menu items.
Comment: @sheena_d: yes, your assumptions in your first question are right.

The behaviour of tertiary level of menus isn't defined yet (anyway if I know how to handle depth in this situtation, I will be able to figure out).

The secondary menu is relevant just because it has to work in the same way, the styling will be different only a little.

I've tried working with theme_menu_link() and theme_menu_tree(), didn't figured out how to check for level depth, but just saw a comment on drupal.org which might solve this.
Answer: Within your theme's template.php, you can override the theme_menu_link() function.
Target the $variables['element']['#original_link']['depth'] variable to check for the depth of the current menu item.
The variable $variables['element']['#original_link']['localized_options']['attributes']['title'] holds the "description" value entered on the menu item's edit page.
Comment: Detailed information about available variables is not included in the API documentation by design, because every active module or theme on the site can manipulate the variables, there's no way to accurately document that information. You can see all of the available variables by installing the Devel module (http://drupal.org/project/devel) and adding kpr($variables); to your theme function in template.php When you refresh a page that includes your menu, you will see the available variables printed out with Krumo.
Comment: Yeah, that's it, thanks! Could you point me to some documentation about $variables['element']? theme_menu_link()'s documentation lacks information (see the issue I've opened: http://drupal.org/node/1431570)


Is there a way access to all files of Windows 2003 regardless of permissions?

Question: Is there a way to access the c$ share of a server from another computer and gain complete access regardless of the file system privileges?  Perhaps the 'system' account can do this, can I access this account?  Also, for my purposes, I need to keep this system live, so I can't just mount the file system on Linux or anything like that (other than mounting the c$ share)
The why:
Right now, My tape backup drive is not working.  Until this is fixed, there is a windows file sharing server that I want to manually backup to an array.  This server can't have the array directly attached to it.  I can either access the computer with the array shared from the file server, or the other way around.
However, when I try to just copy the files from the file sharing host I get a can't copy file error that causes the copy to abort.  The reason, I think, is that administrator does not have privileges to access the shares right now.  Even though the administrator account can grant this access, I would rather not mess with the privileges.
Comment: Neither adding admin to the backup group or robocopy has worked.  Right now I am trying to use the windows backup utility, and will see how it goes.
Answer: You can grant a user account Backup Privileges, which allows it access to all files regardless of other permissions on those files.
Typically you add the user to the Backup Operators group on the server.
http://technet.microsoft.com/en-us/library/cc740217.aspx
Comment: Going to try this one first because it seems like the most simple for something I only plan on doing once or twice, will keep Robocopy in mind though.
Answer: The RoboCopy utility has a /B option that will allow access from an account with administrative privileges that would otherwise fail.   It also allows the copy to continue if only a single file fails or is locked.
Answer: The Administrator account should be able to access the hidden station shares (like c$). Do you have authentication/active directory problems?
Comment: I can access that share, just not all of the folders under that share, because some of those folders are set not to inherit permissions.
Comment: Aaah you wanna access the users folders. Sorry, I under stand. User Moo above me have the correct answer.
Answer: If you are an administrator, once you have accessed the c$ share you can take ownership of those folders and then change the permissions to allow yourself access.
Comment: Rather not change the permissions of possible, see the last line of 'The Why'
Comment: But you HAVE to change permissions to correct the misconfiguration, to gain access for file backups.  Otherwise, the backup software is locked out of those files.
Comment: I don't think that is true, the netbackup client is able to do it so there must be some sort of low level way to achieve this? Although, maybe there is no practical way.  I certainly do appreciate the help though, and am not *trying* to be difficult :-)
Answer: It is possible that permissions can be changed so remove Administrator read/write access to files/folders, also preventing changing the permissions by the admin.  However, the Administrator can still take ownership of the files, which enables access to change permissions.  Then, add access to Administrators.


How to conditionally send emails Drupal 8 Webform

Question: I am trying to figure out how to do the following; based on a submitted value change the email behavior:

If (field 'send' value == no) then don't send the email
If (field 'member' value == yes) then change the email message

If I can find out how to do #1 above I could use that to achieve #2, in fact I suspect they have the same answer. I am looking for a hook but perhaps that is just me being stuck in D7 style of doing things.
Answer: The latest dev release of Webform 8.x-5.x supports conditional emails and handlers.
Comment: can we send the mail for different user based on the field values ? If the values is empty , it should send to admin email ID, if not then it will be send to another user. Is it possible ?
Comment: I can't keep up with the rapid pace at which you are improving this module!
Comment: @logeshvaran the question is very interesting but I think you need to ask a question yourself in a separate post
Comment: The short answer is webform email handlers support email routing based on selected options (ie select menu, radios, & checkboxes)


Hot Swap Ultra320 SCSI LTO-Tape Drive

Question: Can an Ultra320 SCSI LTO tape drive (or any SCSI device) be hot swapped without needing to power down the system?  The current LTO tape drive is the only device on the SCSI host.
If the above can be done, then what problem(s) could I run into when I scan the SCSI bus after doing a hot swap?
Answer: If Linux, I try to rmmod or modprobe -r the relevant SCSI HBA modules to remove the device from the OS. In the case of a drive on an MPT Fusion controller I use the following:
rmmod mptspi
rmmod mptscsih
rmmod mptctl
rmmod mptbase

dmesg will report: "mptctl: Deregistered /dev/mptctl @ (major,minor=10,220)"
At that point, the device can be unplugged. To reinitialize or add a new device, I just modprobe mptspi or modprobe mptscsih and go from there...
For Adaptec, you'd rmmod the aic7xxx driver, replace the drive, then modprobe the driver.
Answer: In my experience this can be done. The risk is a bus-reset on the SCSI bus, which can definitely affect any other devices on the bus. Since this is an LTO tape drive, chances are pretty good only other tape drives are on it so they're probably OK. The biggest thing is to ensure no system resources are attempting to access the drive when you do the work (turn off your backup software, mostly).
Not all operating-systems behave well though, so it can be hit or miss.
Comment: This is the only device on the SCSI host and Linux is the OS.
Comment: @U85 That's even less likely to cause problems.
Comment: Is there any risk of hardware damage doing this?  Of course I would turn off the tape drive prior to removal (aka unplug it from power).
Comment: @u85 No, not really. The voltages on SCSI busses aren't enough to damage things. Termination is the one area that might *possibly* cause problems, but ensuring the SCSI bus is quiescent when you make the swap is how to make sure nothing will notice.
Answer: I've never tried this, but according this link you can use this command:
echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete

where where h is the HBA number, c is the channel on the HBA, t is the SCSI target ID, and l is the LUN. This numbers you can find in the dmesg listing.
In this link is it descripted for block device, like disk, but I think, that it is universal solution for SCSI devices...

Hot Swap Ultra320 SCSI LTO-Tape Drive


Schoof algorithm with division polynomials

Question: Please help me understand one aspect of Schoof's algorithm. Suppose there is a curve $y ^ 2 = x ^ 3 + 2x + 1$, $P = 19$ The array of primes, according to the algorithm, is: $\{2, 3, 5\}$ The problem arises when calculating $k = 19 \pmod 5$. Since $| k | < 5/2$, then we take $-1$ from the values $-1$ and $4$. For calculations, I use division polynomials. At the stage of calculating lambda, I get formulas. But some of the division polynomials have indices $-2$ and $-3$. As far as I know, division polynomials are numbered starting at $-1$. What to do with polynomials with indices $-2$ and $-3$?
I calculate lambda using formulas:
After that, I execute the body of the loop:

$\gcd(x^p - x, x^3 + 2x + 1) = 1$, where $p = 19$. And it means that $t1 = 1 \pmod 2$
$k = 19 \pmod 3 = 1$. $|k| < 3/2$. $\gcd(x^p - x, F3) = x+11 \neq 1$. Since result is not equal $1$, we can use TonelliShanks algorittm to find $w$. $w = 1$. $\gcd(4x^3+10x^2+2x+13, F3) = 6x + 9 \neq 1$.
Since $\gcd(4x^3+10x^2+2x+13, F3) \neq 1$, I define the sign of $w$: $\gcd(6x^3 + 15^2 + 14x - 2, F3) = 2x + 3 \neq 1$. It means that $w$ is positive. $t2 = 2 * w \pmod 3$. I get $t2 = 2 \pmod 3$
$k = 19 \pmod 5 = 4 = -1$. |$k| < 5/2$. That's why I take $k = -1$. But I can't find $x, y$ using division polynomials with $k < 0$. If I take $k = 4$, I get for $1\leq tau \leq 3$ that $\gcd(*\text{polynom, created with $k = 4$}*, F5) = 1$. And I get answer $0 \pmod 5$. But I must get $2 \pmod 5$. Futher I need to define sign: $t3 = 2 \pmod 5$ or $t3 = -2 \pmod 5$
As a result I must get:
$t = 1 \pmod 2$,
$t = 2 \pmod 3$,
$t = -2 \pmod 5$

But I get:
$t = 1 \pmod 2$,
$t = 2 \pmod 3$,
$t = 0 \pmod 5$
It is wrong.
I get F3 and F5  using formulas:
Comment: Someone is going to ask you to use tex/mathjax, so it might as well be me. It is much easier to read, and you can type any formulas you want.
Comment: Good job on the mathJax-ing! Some quick tips: use `\leq` and `\geq` for $\leq$ and $\geq$, `\gcd(a, b)` to get $\gcd(a, b)$, and `a\pmod{x}` to get $a\pmod{x}$ (which look much clearer/neater than $GCD(a, b)$ and $0(mod5)$, etc.).
Comment: Right now this question is missing context and it's likely to be closed and downvoted. Please see [how to ask a good question](https://math.meta.stackexchange.com/questions/9959/how-to-ask-a-good-question) on meta for information and advice on improving this post.
Comment: I made some edits to your latex, but wasn't sure about the =>. Are they meant to be $\geq$ or $\Rightarrow$? (Use `\geq` or `\Rightarrow` for these.)
Answer: The problem is solved. It is necessary to take the |k| for negative values of k and adjust the formulas a little.


Concatenate cells but check values before - Excel VBA

Question: So I want to concatenate the values from the D column to the ones in the J column, so far so good, but I want to check if the value from column D is already in the J column, I used InStr and it works until it meets a cell that already has the value that doesn't need to be concatenated
Script:
    Sub Concatenate()
    Dim counter As Integer
    Dim myValue As Variant
    myValue = InputBox("For how many rows do you want to concatenate D w/ Js")
    d = 4 ' Concatenate from this column
    j = 10 'Concatenate to this column
    ' counter -> Counter to iterate through all the cells in both columns
    For counter = 2 To myValue
      If Cells(counter, d).Value <> "" Then
        If InStr(1, Cells(counter, j).Value, Cells(counter, d).Value) = 0 Then
          Cells(counter, j).Value = Cells(counter, j).Value & "," & Cells(counter, d).Value
        Else
          ' Had to leave it blank
        End If
      End If

    Next counter

End Sub

Example
How it looks                         How it should look
|    D    |   J     |                 |    D     |      J      |
|    a    |   1     |result of code =>|    a     |     1,a     |
|    b    |   2,b   |result of code =>|    b     |     2,b     | <= (nothing modified)
|    c    |   3     |result of code =>|    c     |     3,c     |
|   abc   |   42    |result of code =>|   abc    |     42,abc  |

When meeting the row with the values |    b    |    2,b    | the program stops and the rest remain unmodified
Comment: Uhhh...what you want to do when the value from `column D` is in `J Column`? Your code looks good. Right now, your code, if D is in J, then it does nothing. You want to do something different? Add it after your `Else`
Comment: It just has to skip to the next cell, but I couldn't figure it out, and by means of basic programming increasing the counter doesn't seem like a good ideea
Comment: You are using a `For...Next`, so your counter will be increased automatically in every loop. Your code, when D is in J, does nothing, but goes to end of loop, increases counter and repeat the loop again. The only thing weird I see is that line `Next counter` should be `Next contor`. That's all I see. Your code does not work or what?
Comment: So silly, I added counter instead of contor so everyone would understand it, and no, it does not work when everything is written correctly
Comment: How many rows are there?
Comment: Well, 5000, but it doesn't have to run on all of them only on the N rows(input given by user when asked), because only the first 100 could have values in them, or the first 30(it varies)
Comment: Okay, then using the `Integer` datatype shouldn't be a problem. It can hold values untill 32767. Anything larger and your `Integer` will fail. Therefore its almos always better to use `Long`.
Comment: *... it does not work when everything is written correctly...* What do you mean? What you want to do? I still don't see what you want to do when D is in J
Comment: Let's say it has to execute for 15 rows, first 5 have D in J, next 5 don't have D in J, and the last 5 have D in J
This must execute for every row and see if it has the value or not, if it has the value->skip to next row and check again, doesn't have the value-> add the value to J
Comment: Your code does what it _seems_ to me you want. Maybe you want do something different ... Add some examples of a before and after scenarios
Comment: Edited now.....
Answer: theres a typo
in the for you have the variable contor
For contor = 2 To myValue

but then you are incrementing then variable counter
Next counter

change the counter to contor and it should work
good luck
Answer: Ok, after looking at the ouput you have posted:
How it looks                         How it should look
|    D    |   J     |                 |    D     |      J      |
|    a    |   1     |result of code =>|    a     |     1,a     |
|    b    |   2,b   |result of code =>|    b     |     2,b     | <= (nothing modified)
|    c    |   3     |result of code =>|    c     |     3,c     |
|   abc   |   42    |result of code =>|   abc    |     42,abc  |

This code should work. Image of data before applying macro:

And after applying macro, I get:

My code:
Sub Concatenate()

Dim Counter As Long
Dim myValue As Variant
myValue = InputBox("For how many rows do you want to concatenate D w/ Js")

For Counter = 2 To myValue Step 1
    If Right(Range("J" & Counter).Value, Len(Range("D" & Counter).Value)) <> Range("D" & Counter).Value Then Range("J" & Counter).Value = Range("J" & Counter).Value & "," & Range("D" & Counter).Value
Next Counter

End Sub

Answer: Final code, working:
Sub Concatenate()
Dim counter As Integer
Dim myValue As Variant
myValue = InputBox("For how many rows do you want to concatenate D w/ Js")
d = 4 ' Concatenate from this column
j = 10 'Concatenate to this column
' counter -> Counter to iterate through all the cells in both columns
For counter = 2 To myValue
  If Cells(counter, d).Value <> "" Then
    If InStr(1, Cells(counter, j).Value, Cells(counter, d).Value) = 0 Then
      Cells(counter, j).Value = Cells(counter, j).Value & "," & Cells(counter, d).Value
    Else
      ' Blank
    End If
  End If
Next counter

End Sub


Add a second RulesAction to a custom module

Question: I created a custom module containing one Rules Action that works perfectly. I tried adding a second by creating a second php file in the MODULE/src/Plugin/RulesAction folder, with the names and class etc all changed from the original action.
I've tried refreshing the cache, and uninstalling/installing the module but the new action is not showing up. Is there another step I need to take, other than just putting a new file in the MODULE/src/Plugin/RulesAction folder?
Answer: I'm an idiot, it's right there under a different subheading. I expected them to appear together, but I set a different value on this line:
 *   category = @Translation("This is the heading the action appears under."),


Properties of C# structs get unusable names in COM and VB6?

Question: My C# code has a struct which I am exporting to COM. The properties of this struct are coming through with strange names that aren't even valid syntax in VB6, so they cannot be accessed.
Is there some way to get these to export with normal, usable names? Am I missing an attribute or something?

The format  of the name in COM / VB6 is:
<original_name>k__BackingField

where only the original_name part was in my C# code.
I can only see these crazy property names in the VB6 Object Browser, Intellisense won't show it.
Here's the (slightly sanitized) code which is being built:
[Guid("....")]
[ComVisible(true)]
public struct MyStruct
{
    public string StringA { get; set; }
    public string StringB { get; set; }

    public MyStruct(string a, string b)
    {
        StringA = a;
        StringB = b;
    }

    ... // some other methods, no fields or properties
}

and for good measure here is the IDL that gets generated:
typedef [uuid(....), version(1.0), custom(xxxx, MyNamespace.MyStruct)]
struct tagMyStruct {
    LPSTR <StringA>k__BackingField;
    LPSTR <StringB>k__BackingField;
} MyStruct;

as shown by OleView. I can see it contains the same k__BackingField as noted above. So it seems these names are coming from the C# typelib export process.
Comment: You've got the [answer](https://stackoverflow.com/a/57827805/11683). You are using auto-implemented properties. Use fields directly instead.
Comment: @GSerg so unlike classes the properties of a struct don't get exported? Is that correct?
Comment: COM `struct` cannot contain methods or properties. [The grammar](https://learn.microsoft.com/en-us/windows/win32/midl/struct) only allows for fields.
Comment: @GSerg but a C# class similarly exported doesn't have this problem, that's what I meant. The COM field names match the C# class names fine in that scenario.
Comment: Because classes (actually, interfaces), [are allowed](https://learn.microsoft.com/en-us/windows/win32/midl/dispinterface) to have properties.
Comment: Auto properties in C# are syntax sugar, the CLR doesn't support anything like it.  So you got to see how the C# compiler did it.  The semantics of a struct in COM is the same as in the C language, just fields with no support for properties or methods.  So all the type library exporter can do is expose those auto-generated fields.  This is dangerous of course, you normally use auto properties so that some day you can easily switch to a more involved getter or setter.  They are never going to get called.
Comment: And much worse, a significant implementation detail of struct is that the client code has to figure out at what offset it should read a field value.  Related to the "packing" rules that the compiler used, compare to StructLayoutAttribute.Pack.  VB6 does that properly, it uses IRecordInfo to find out.  This is not fast, at all.  You really do favor a class.  Only an out-of-process server might consider it, the marshaling is faster than the round-trips for each field, but they are very hard to write correctly in C# so not much of a consideration.
Answer: public string StringA { get; set; } is only a short form of the following
private string _stringA_BackingField;
public string StringA
{
  get { return _stringA_BackingField; }
  set { _stringA_BackingField = value; }
}

And since COM structs only contain fields and no properties you will see the backing field. Your example should work if you change the code to the following:
[Guid("....")]
[ComVisible(true)]
public struct MyStruct
{
    public string StringA;
    public string StringB;
}

By definition the struct type is a value type that is typically used to encapsulate small groups of related variables.
If you want to provide properties or methods you need to use a class. You find more information about structs here:

https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/struct
https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/index
Comment: But I guess the bottom line is that TLBEXP is creating COM structs with unusable names. The fact that its an automatic backing field doesn't excuse that. TLBEXP could be much smarter here and do something more useful.


Running (and monitoring) nginx with supervisord

Question: I will start with my use case, since I very well may not be using the correct tools for this job. Please let me know if I am going about this all the wrong way
use case: I have a CentOS server hosting multiple web apps. I want to be able to trust that my web server and application server will be running. My stack looks like

web server: nginx
application server: uWSGI
web framework: flask / python

I want to use supervisord to monitor nginx and uWSGI. In my /etc/supervisor.conf, I have
[program:nginxgo]
command = /usr/sbin/nginx
autostart=true
autorestart=unexpected
exitcodes=0
stdout_logfile=/home/webdev/nginxgo.log
stderr_logfile=/home/webdev/nginxgoerr.log

[program:uwsgi_emperor_go]
command = uwsgi --emperor /etc/uwsgi/emperor.ini
autostart=true
autorestart=unexpected
stopsignal=INT
stdout_logfile=/home/webdev/emp.log
stderr_logfile=/home/webdev/emperr.log
directory=/home/webdev/
user=webdev

I got the uWSGI process to start. When I enter [root@mymachine]# /usr/local/bin/supervisord -n -c /etc/supervisord.conf
the output is
2014-11-26 14:07:56,917 CRIT Supervisor running as root (no user in config file)
2014-11-26 14:07:56,951 CRIT Server 'inet_http_server' running without any HTTP authentication checking
2014-11-26 14:07:56,952 INFO supervisord started with pid 31068
2014-11-26 14:07:57,957 INFO spawned: 'nginxgo' with pid 31071
2014-11-26 14:07:57,970 INFO spawned: 'uwsgi_emperor_go' with pid 31072
2014-11-26 14:07:59,095 INFO success: nginxgo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:07:59,095 INFO success: uwsgi_emperor_go entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:08:00,601 INFO exited: nginxgo (exit status 1; not expected)
2014-11-26 14:08:01,607 INFO spawned: 'nginxgo' with pid 31079
2014-11-26 14:08:02,684 INFO success: nginxgo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:08:04,189 INFO exited: nginxgo (exit status 1; not expected)
2014-11-26 14:08:05,194 INFO spawned: 'nginxgo' with pid 31080
2014-11-26 14:08:06,264 INFO success: nginxgo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:08:07,771 INFO exited: nginxgo (exit status 1; not expected)
2014-11-26 14:08:08,775 INFO spawned: 'nginxgo' with pid 31081
2014-11-26 14:08:09,808 INFO success: nginxgo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:08:11,314 INFO exited: nginxgo (exit status 1; not expected)
2014-11-26 14:08:12,319 INFO spawned: 'nginxgo' with pid 31082
2014-11-26 14:08:13,381 INFO success: nginxgo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:08:14,886 INFO exited: nginxgo (exit status 1; not expected)
^C2014-11-26 14:08:15,601 INFO spawned: 'nginxgo' with pid 31083
2014-11-26 14:08:15,603 WARN received SIGINT indicating exit request
2014-11-26 14:08:15,611 INFO waiting for nginxgo, uwsgi_emperor_go to die
2014-11-26 14:08:16,738 INFO success: nginxgo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:08:18,242 INFO exited: nginxgo (exit status 1; not expected)
2014-11-26 14:08:19,244 INFO waiting for uwsgi_emperor_go to die
2014-11-26 14:08:21,607 INFO stopped: uwsgi_emperor_go (exit status 0)

See how it says
2014-11-26 14:07:59,095 INFO success: nginxgo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-11-26 14:07:59,095 INFO success: uwsgi_emperor_go entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

but then it starts just cycling the nginxgo process. I kill the supervisord instance with CTRL-C, and I see in htop that the nginx master process and worker process are both active.
All I want is to start nginx and my uWSGI emperor on server startup/restart or failure of either program
Answer: supervisord can only handle processes in foreground. The default for nginx is running in background as daemon.
To ensure that your nginx is running with supervisord you have to set 'daemon off' in your nginx.conf (see also nginx docu at http://nginx.org/en/docs/ngx_core_module.html#daemon).
Comment: In the nginx config, enter `daemon off` then it works with supervisor
Comment: Does the `stop` action work when you use this technique ?
`start` worked for me after I added `sudo` in the supervisord `command` attribute. I tried running `supervisorctl stop nginx` and it complained with `OSError`.


Qt ignores const specifier

Question: I've created simple snippet to show strange behavior I noticed. That's it:
#include <QCoreApplication>
#include <QLineEdit>

class MyObject : public QWidget
{
public:
    explicit MyObject(QWidget *parent = 0) : QWidget(parent) {
        editor = new QLineEdit(this);
    }

    void setValue(const QString &s) const {
        editor->setText(s);
        editor->setReadOnly(true);
        editor->setMaxLength(s.length());
    }

private:
    QLineEdit *editor;
};

int main(int argc, char **argv)
{
    QCoreApplication app(argc, argv);
    return app.exec();
}

MyObject::setValue function has const specifier, but this code is compiled well. Note that setText, setReadOnly and setMaxLength functions aren't const:
void setText(const QString &);
void setReadOnly(bool);
void setMaxLength(int);

I just want to know what causes such behavior?
I use Qt 4.7.4 with MingGW 4.6.2.
Answer: (This is not Qt related. This is a general C++ question.)
The compiler is correct, because you are not modifying editor. What you are modifying is *editor; you are only modifying the object it points to. The const specifier will only disallow changing members directly contained in the object. The object editor points to is not a member and thus can be modified:
void setValue(const QString &s) const {
    editor->setText(s); // OK
    editor = new QLineEdit; // Error: 'editor' is changed.
}

Comment: Ouch! How can I missed that? :)
Answer: By the way there are two ways of seeing the constness of a method:

Bitwise constness or physical constness if the member function doesn't modify any of the object's data member (excluding static ones)
Logical constness doesn't allow any change even for the objects pointed.

Compilers only detect the first type of constness, as you experienced :)
This is discussed in one of the items in Effective C++ written by Scott Meyers

Qt ignores const specifier


Slow message retrieval from SQS

Question: Given a Java AWS Lambda with the following code:
private static final String QUEUE_URL = "https://sqs.us-east-1.amazonaws.com/<ACCT_NUMBER>/<QUEUE_NAME>";
private static final AmazonSQS client = AmazonSQSClientBuilder.standard().build();
private static final int MAX_SQS_MESSAGES = 10;

And:
private List<Message> getMessages() {
    return client.receiveMessage(new ReceiveMessageRequest().withQueueUrl(QUEUE_URL)
            .withMaxNumberOfMessages(MAX_SQS_MESSAGES).withWaitTimeSeconds(1)).getMessages();
}

I experience rather "long" SQS retrieval times (considering the specified 1 second base for long polling), as sample evidence from logs:

Got 3 SQS msgs: 1985ms
Got 8 SQS msgs: 1887ms
Got 9 SQS msgs: 2438ms
Got 5 SQS msgs: 1748ms

Are those time between normal operation or, could I be doing something wrong or improve something?
Maven dependency:
  <dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-sqs</artifactId>
    <version>1.11.488</version>
  </dependency>

Answer: Those are indeed very long delays, and something is wrong. With a non-empty queue, you should be able to get typical reads in the 5-500ms range (lower with more messages available). Even if your queue is empty, the request times should be topping out around a maximum of 1s based on your usage of withWaitTimeSeconds in the request.
There are a number of steps you can take to  narrow down the problem:

Make sure the queue and the lambda are in the same region - I mention this first as I've seen so many latency issues caused by cross-region calls in AWS.
Make sure you have accurate request metrics. I don't see how you are measuring the metrics timing in your code, but I do see how you're constructing your client.

Create an implementation of RequestHandler2 that implements afterError and afterResponse, and examines the details of request.getAWSRequestMetrics()
Add that request handler to the client via clientBuilder.withRequestHandlers(RequestHandler2... handlers)
This will give you accurate details of how the request is spending its time, and perhaps reveal some obvious problems, and may also point to a problem outside of the call to SQS.

Make sure you are reusing your client (and not creating a fresh one each time) - consider logging each time your client is created. Under the hood of the client there's a lot of setup, and if it's using a fresh client every time, there might be a lot of time wasted there.
Comment: Regarding #3: Client is static final and lambda is run every 2 mins, so I guess it's getting reused, but I'll set logging on that. And I'm measuring just a wrapper above the getMessages, not the client instantiation per-se
Comment: Logging shows that client IS being reused, so that option is being ruled out. Deleted the "long polling" associated to the 1 sec wait and overall behaviour is much better. Perhaps is related to our use case where the queue has just about 5-10 msgs per loop. I'll keep it working and monitoring this way and see how it behaves on a longer time frame

Slow message retrieval from SQS


How to find increasing interval of function $|x-1|+x^2$

Question: I  recently came across a function-
$$f(x)=|x-1|+x^2$$
The question was to comment on the monotonicity of the function-

What I tried-
This was a 2 part question, first part was to comment on continuity and derivability of the function, I found that function was continuous everywhere but was not derivable on $x=1$.

The only proof I know for checking monotonicity is by considering the derivative of function. But here I am confused that how will we check when the function in not derivable at certain points.
I also tried calculating $f'(x)$ by-
$$f'(x)=\lim_{h \to 0} \frac{f(x+h)-f(x)}{h}$$ but couldn't get anywhere. Hints will be appreciated.
Comment: I would define the function piecewise  with cases $x\ge 1,x<1$
Comment: Ah that would make it so easy, I feel so dumb, thank you so much.
Comment: If in doubt draw a diagram ...
Answer: When considering monotonicity, you don't so much care where the derivative is $0$ so much as where the derivative is positive or negative. Finding the points where the derivative is $0$ is simply a way of figuring out where a possible sign change might occur. Similarly, if the derivative is undefined at an isolated point, you can just treat it as a possible point where the derivative might change sign.
That is, in your case, calculating $f'$, we get
$$f'(x) = \frac{|x - 1|}{x - 1} + 2x = \begin{cases} 2x - 1 & \text{if }x < -1 \\ 2x + 1 &\text{if }x > 1,\end{cases}$$
which is never $0$, so just test the sign of $f'$ on either side of $1$.
Answer: If $x\ge1$ we have:
$$f\left(x\right)=x-1+x^{2}$$
since $x\ge1$, you can easily see its increasing. Now if $x<1$ we have:
$$f\left(x\right)=-\left(x-1\right)+x^{2}=x^{2}-x+1$$
This case is less intuitive, especially for smaller values of x (since $x^{2}\le x$ for $x\in\left[0,1\right]$). so we derive the function:
$$f'\left(x\right)=2x-1$$
which is positive for every $x\ge\frac{1}{2}$.
Considering both cases, we conclude that f is increasing $\forall x\ge\frac{1}{2}$


disable date field (only update with datepicker)

Question: I am using drupal 6, im my form I put the date field as
<code>$form['myform']['from_date'] = array(
    '#type' => 'textfield',
    '#title' => t('From date'),
    '#size' => 8,
    '#default_value' => (arg(5) == null)? '0':arg(5),
    '#attributes' => array('onclick' => 'GetdatePicker(this);'),
);
</code>
i want that user can not edit this field without using date popup(populated from date picker).
any suggestion.
Answer: Add 'readonly' => 'readonly' attribute to your field,
ie:
<code>$form['myform']['from_date'] = array(
'#type' => 'textfield',
'#title' => t('From date'),
'#size' => 8,
'#default_value' => (arg(5) == null)? '0':arg(5),
'#attributes' => array('onclick' => 'GetdatePicker(this);', 'readonly' => 'readonly'),
);
</code>


Proving $f_n$ is weakly convergent to $f$ if and only if following holds

Question: Weakly convergent definition (from Wikipedia): A sequence of points $(x_n)$ in a Hilbert space $H$ is said to converge weakly to a point $x$ in $H$ if $\langle x_n, x \rangle \rightarrow \langle x,y \rangle$ for all $y$ in $H.$
Let $X=C[a,b]$ and $\{f_n\}_{n=1}^{\infty}\in X$. Then $f_n$ is weakly convergent to $f$ if and only if there exists a constant $M$ such that
\begin{equation*}
\|f_n\|\leq M
\end{equation*}
for all $n$ and
\begin{equation*}
\lim_{n\rightarrow\infty}f_n(x)=f(x)
\end{equation*}
for any $x\in[a,b]$.
I don't really see how to approach this problem.
Comment: See https://math.stackexchange.com/a/873535/529121
Comment: How is $X$ a Hilbert space? Also, you meant $(x_n|x)\to(y|x).$
Answer: By weak-convergence, do you mean the following?
Let $X$ be a Banach space and $X^{\ast}$ its dual space (i.e., the
Banch space of all bounded linear functionals on $X$). Let $(x_{\alpha})$
be a net in $X$, $x\in X$. We say that the net $(x_{\alpha})$ converges
to $x$ iff for each $f\in X^{\ast}$, $\langle f,x_{\alpha}\rangle\rightarrow\langle f,x\rangle$.
In general, the weak-topology $\sigma(X,X^{\ast})$ on $X$ is not
first countable, so it is inappropriate to describe it using sequences.
(One should use nets or "generalized sequences"). However, we do have: If a sequence
$(x_{n})$ converges weakly, it is bounded. For, recall that $X\hookrightarrow X^{\ast\ast}$,
$x\mapsto\hat{x}$, where $\langle\hat{x},f\rangle=f(x)$, $f\in X^{\ast}$
is an isometric embedding. Now, let $(x_{n})$ be a sequence in $X$
and suppose that $x_{n}\rightarrow x$ weakly for some $x\in X$.
For each $f\in X^{\ast}$, $\{\langle\hat{x}_{n},f\rangle\mid n\in\mathbb{N}\}$
is bounded because $(\langle\hat{x}_{n},f\rangle)_{n}$ is a convergent
sequence of complex numbers. By uniform boundedness principle, $\sup_{n}||\hat{x}_{n}||<\infty$.
Note that $||x_{n}||=||\hat{x}_{n}||$...
For your question. $X=C([a,b])$ is a Banach space with respect to
the supremum norm (in fact, it has far more structures: It is a unital
commutative $C^{\ast}$-algebra). It is well-known that $X^{\ast}$
can be identified with the space of all complex-valued, regular Borel
measures on $[a,b]$, with total variation norm. (Riesz representation
theorem). Let $(f_{n})$ be a sequence in $X$ and $f\in X$. Suppose
that $f_{n}\rightarrow f$ weakly. By the previous discussion, $\sup_{n}||f_{n}||<\infty$.
For each $x\in[a,b]$, let $\delta_{x}$ be the Dirac measure at $x$,
i.e., $\delta_{x}(A)=\begin{cases}
1, & \mbox{ if }x\in A\\
0, & \mbox{ if }x\notin A
\end{cases}$, $A\in\mathcal{B}([a,b])$. Note that $\delta_{x}\in X^{\ast}$.
Then $\langle\delta_{x},f_{n}\rangle\rightarrow\langle\delta_{x},f\rangle$.
Recall that $\langle\delta_{x},f\rangle=\int fd\delta_{x}=f(x)$. Hence,
$f_{n}(x)\rightarrow f(x)$.
Conversely, suppose that $\sup_{n}||f_{n}||=M<\infty$, and for each
$x\in X$, $f_{n}(x)\rightarrow f(x)$. Let $\mu\in X^{\ast}$. By
Riesz representation theorem, $\mu$ is a complex measure and the duality
$\langle\mu,f\rangle$ is actually integral: $\langle\mu,f\rangle=\int f\,d\mu$.
Note that $|f_{n}|\leq M$ for all $n$ and the constant function
$M$ is $\mu$-integrable. By Lebesgue dominated convergence theorem, we have $\int f_{n}\,d\mu\rightarrow\int f\,d\mu$.
That is, $\langle\mu,f_{n}\rangle\rightarrow\langle\mu,f\rangle$.
In another word, $f_{n}\rightarrow f$ weakly.


redirect external url only as query string

Question: I'm attempting to redirect all external links thru a php file with the url of the external site set as a variable.
Example. If my site (mysite.com) has a link to google.com/results/1234 I'd like to automatically rewrite the url to something linke mysite.com/external.php?p=google.com/results/1234.
I'd like an htaccess solution, as I am unable to change files associated with external urls without affecting other urls on the site.
I DO Not want to redirect any link with mysite.com.
If someone could point me in the right direction, I would really appreciate it.
Comment: Once outgoing external links are generated, you have no control over them. The source of them, the code that generates them inside your site, is what might be modified, but it is missing in your question.
Comment: That really makes alot of sense. Thank you faa
Answer: Not feasible via .htacess file. .htaccess can ONLY rewrite incoming URLs with domain names that resolve to your web server. It has no control over outgoing URLs as these requests are directly going to outgoing web servers (in your example, google.com).
What you probably need is a scripting solution that redirects the user as you desire by hooking in to onclick events of all the links.
EDIT: Here's a quick proof of concept using jQuery. This should get you started.

 <title>jQuery global redirector</title>

 google url would redirect

 mysite.com url won't redirect

You should probably put the script inside another file (say redirect.js) and then include this script (near the end of ) selectively in your pages that need such redirection. And don't forget to import jQuery as well!
Comment: Thank you Ravi for this solution! I didn't think to use jquery. Do you have any opinion on how this technique will affect seo? I'm wondering if I still have a chance for leaks thru the hard coded links when bots visit the site.
Comment: This would neither push the ranking up nor down. It's neutral from SEO perspective since the `` text remains exactly the same - the links in your articles etc. remain the same. In fact, your link  tags are not even touched (like adding an onclick="" handler to them) and that's the beauty of jQuery.


How to display a field of a referenced entity in views contextual block?

Question: I have read several questions on topic, e.g. 1 2 3, but couldn't manage to get it work.
My setup:
In a Drupal 7 environment, I have a node type called organizer, which contains two fields: node title (default) and a link (to the organizer's website).
A second node type called event contains several fields one of which is an entity reference to organizer.
My site's design only has the node title and node body of an event being displayed in main content. All other fields are displayed in a sidebar using a views block with filter on node type (= event) and a contextual filter on node id. Among these fields is also the field organizer and it's displaying the organizer's name (title field) alright.
My goal:
Now I want to have the organizer's name link to the organizer's website. But the link is actually a field inside the organizer content type.
My approach:
I've added a relationship to Entity Reference: organizer. Then I added the link field that is used in the organizer content type to the list of fields using the relationship. I rearrange the fields to have this link field being loaded before the entity reference field. Then I set up the latter field to being rewritten as link using the plain url from the link field.
[The link field is currently being displayed for debug purposes, later I want to exclude it from display, of course.]
My result:
The link field is always empty. Hence, the organizer's name is never linked to the respective website.
Questions I stumbled upon:
When I add a relationship, I have two options like
Entity Reference: Referenced Entity
A bridge to the Content entity that is referenced via field_organizer
 Entity Reference: Referencing entity
A bridge to the Content entity that is referencing Content via organizer

My understanding is that I have to user the first one, because I want to load the organizer that is referenced in the event. [The second one I would be using, if I have a organizer and want to get all events that are referencing this organizer, right?] However, I tried both options to no avail.
Is the ordering of fields relevant? I've tried both having the link field before and after the entity reference field, it stays empty in both cases.
Since each event could have more than only one organizer, the entity reference field is set to allow unlimited values. Could that cause a problem?
Could anyone point me into the right direction of what I am missing?

UPDATE: It seems to break the system if you have both relationship types set up at the same time or maybe the ordering plays a role, too. However, when the first relationship in list is the one I need it kind of works.
It is working absolutely fine for a single organizer, the name is displayed and linked to the organizer's website. But it will not work for any additional organizers. The block lists all organizers (when having more than one) but each of them is linked to the first organizer's website.
Thinking about how that works, that is reasonable, since the relationship only adds one related object. When displaying a field (e.g.) from that relationship it will be the same for each delta of entity reference field.
Actually, in that case I should be able to define how the entity reference field is displayed. It already has an option to link to the entity itself but you can't define any fields from the entity to be shown (e.g. instead of title).
Comment: did you "use" the relation on the link field ?
Comment: Yes, indeed. When adding the link field I chose the respective relationship from the available dropdown.
Comment: and is it still there ? if you were changing the relationships it could have vanished :)
Comment: Yes, it is still there. I might have found the issue, though, and will update question in a couple of minutes.
Answer: The solution is a two step solution, but first you need to decided if you are going from Parent to Child or Child to Parent.
Background info
If you are going from Parent to Child, you setup a relationship using the "referenced" variable.
If you are going from Child to Parent, you setup a relationship using the "referencing" variable.
To make life simple for myself, when I specify the relationship, I also using the Administrative Option on the Reference Window and enter a more understandable name for the relationship.  In you case, since I would make the Admin Name Event reading Organizer or something like that.

Setup a relationship for Event reading Organizer in the Relationship
field under the Advanced tab.
Once you have related your Event content type to the Organizer
content type, you should now be able to see the Organizer fields
appear in your list of available fields.  Select the fields from the
Organizer content type that you want to appear in your view and hit
Apply. Here is the CRITICAL part, once you have defined a
Relationship, a new drop down menu (Relationship) will appear on the
Configure field windows.  For any fields that is in another table,
you MUST choose the appropriate relationship for the field. In your
case, you would choose the Event reading Organizer relationship.
This relationship instructs view to go Organizer table and bring
back that field.  If you do not specify the relationship in the
Configure field windows, this will not work.

Attached is a copy of the Configure field with a relationship.  Note: In your example, you would want the referenced variable. But in my example, I was working with the referencing variable. Please do not let that confuse you.  Just image the word in the relationship is referenced.

If you have any questions, please post them here and I will respond.
Comment: Thanks for sharing this. However, I already used this as you can see in the question's comments. And when "using" the relationship the problem is that it only works for the first relationship. That's why I had to come up with a completely different approach as described in my own answer (which btw. is more than 2 years old).
Answer: I found a complete different approach which doesn't require relationships at all.
First, I installed the Display Suite module. It allows you to create new view modes for your content types. So, I created a new view mode called entity.
Next, I enabled this view mode in the desired content type (-> manage display) and only view the link field.
Further, I edited node.tpl.php (you can use also a more specific theme file like node--organizer.tpl.php) to hide the node title when the view mode is entity using an if construction:
<?php if ($page && $title && $view_mode != 'entity'): ?>
  <h1<?php print $title_attributes; ?>><?php print $title ?>

<?php endif; ?>

Last, I edited the view and set the display option for the organizer field from Label to Rendered entity using the view mode entity.
Et voilà, the organizers are now displayed linking to the organizers' respective websites.
Note: This works for me because each link in a organizer node has its title set to the node title.


Allow all users with a certain role to set Groups Audience to any Organic Group

Question: I am using Drupal 7.
What I am trying to achieve is the following:
I have several users with a particular role say "A". And I have several content types set as "Group content"
When a user with role "A" creates content I would like the user to be able to set the Groups Audience field to any of the existing Organic Groups.
Is it possible to achieve without adding those users as Administrators to all existing Organic Groups?
Answer: I was able to achieve the above requirement using OG Role Override module.
From Global Permissions for role "A" we would set:
"Act as role 'member' in OG Node Group Main Page groups"
Hope this helps!


Can I setup static routes through ILO card?

Question: I am setting up a network and I have an HP Blade 380 G7 server with its own ILO module.  I am going to be giving users access to this box and have segmented the ILO card off.  I am wondering 2 things.
1.) I am an administrator on box can I use the ILO network port to gain access to that network segment?
2.) can I access the ILO itself?
to question 2 this is merely an access question I know I can add user IDs to the ILO with admin to the box but can I access the ILO directly even though its on a different network segment?
Any assistance you can provide on this would be helpful.
Comment: What are you asking? This is pretty unclear.
Answer: The DL380's not a blade, it's a rack-mount server - or do you mean a BL460 which is the blade equivalent? Now onto your questions.
The iLO module is completely separate from the server itself, you could be an administrator or  root but there's no normal/easy way to get data from the host OS out over the iLO port. I'm sure some hacker with intimiate knowledge of iLO firmware could do something along those lines as a challenge but it's not what it's there for and there's no realistic way to do it.
Your second question is easier and the answer is simply yes, if there's a route to it. So for instance if both the iLO and one of the regular NICs were configured correctly and on the same VLAN then you should be able to communicate easily, the same is true across correctly configured routed networks too. The iLO's ethernet port and IP stack are absolutely regular in their implementation so can be treated as another server or client in that respect. In fact I've been known to access iLO from a host OS just like this as a quick way to mount an ISO file before now.


Proving some coset stuff (should be easy)

Question: Let $H$ be the set of all permutations in $S_7$ with fixed points at 1, and $T$ be the set of all permutations in $S_7$ that send 1 to 2. Let $\sigma = (1234567)$. Prove $T = \sigma H$.
I've shown $\sigma H \subseteq T$, and am trying to show containment the other way. How do I show that any permutation that sends 1 to 2 can be written as $(1234567)\mu$ where $\mu \in H$?
Comment: Hint. Look at $(1234567)(12)$.
Answer: If you've shown that $\sigma H\subseteq T$, all you need to do now is to note that $T$ and $H$, have the same number of elements ($6!$).  And $H$ and $\sigma H$ also have the same number of elements.
Comment: How do we know that $H$ and $\sigma H$ have the same number of elements?
Comment: ah $H$ is a subgroup of $S_7$, so a bijection exists. never mind!
Comment: @mkl846 actually, this has nothing to do with the fact that $H$ is a subgroup. $H$ and $\sigma H$ have the same number of elements because $h\in H\mapsto \sigma h\in \sigma H$ defines a bijection.


How to Pivot row to columns for this kind of scenerio?

Question: I have a data source that returns data in the following format:
id | subId |  code |  name  |        col1       |    col2        |    col3
1  |     1 |   abc |    xyz |     "Whatever"    | "WhateverA"    | "WhateverB"
                            |     "Whatever2"   | "Whatever2A"   | "Whatever2B"

I need to make col1 row values as headers, while col2 and col3 values as subcolumns under col1 for respective rows as:
id | subId |  code |  name  |        "Whatever"          |        "Whatever2"         |
1  |     1 |   abc |    xyz |   col2     |    col3       |   col2      |    col3      |
                            |"WhateverA" | "WhateverB"   |"Whatever2A" | "Whatever2B" |

Answer: There's nothing that I know of, out of the box, that would achieve this.Here's a high level overview of a possible solution:

Fetch and cache the dataset on the server
Bind the "main\top level" data to a GridView
Using jQuery handle tr\td click events captured inside the GridView
Make an AJAX call to the server -> fetch the sub data from the cached dataset -> return it -> append the fetched result to the GridView

Also, instead of making an AJAX call every single time a td is clicked you can fetch all the data at once and store it in a variable or localStorage.I've done it this way before and there are many resources on the web with step-by-step articles how to achieve step 1 - 4.Hope it gives you an idea
Comment: Thanks for the  quick response Denis. Do you think this can be resolved using SQL pivot?
Comment: Yeah you can use pivot or some sort of temp table solution in SQL to construct the data the way you want to and then simply use the GridView to display all your data and apply CSS amd javascript to style the subheaders differently(so that they stand out)


VmWare / HP EVA: How many LUNs with one DiskGroup

Question: I know that similar questions have been asked already, but I feel that they were on a rather theoretical level and are not quite applicable to my case.
We're operating 3 ESXi 6.0 hosts, all backed by the same HP EVA P6300 via FC. As proposed by the EVA P6300 Best Practice Guide (https://h41368.www4.hp.com/h41111/rfg_formprocessor/SWD_Installed_Base/pl/pl/pdf/TWP_4AA3-2641ENW.PDF), we've put all out 48 disks (600GB, 10k RPM) into the same Disk Group.
I like to use vRaid6 because I personally feel a bit uneasy when using a Raid0 over 6x Raid5s with 8 Disks each (which is essentially what EVA does with vRaid5).
Overall we have about 40 VMs, most of them (~35) with very little Disk I/Os, and a few (~5) with high Disk I/O spikes every couple of minutes (SQL servers / file servers).
For handling reasons, it would be easiest to just create one big LUN/Datastore. Given the fact that all LUNs we create would be backed by the same physical disks anyway, are there any performance implications when only having one LUN? Is there a point in having more than one LUN because of locking or I/O queues?
The HP Best Practice Guide states that one should "Follow operating system and application requirements for LUN count", but I couldn't find any information by VmWare regarding a recommended LUN count.
Thanks in advance for any insight!
Answer: It would be great to know your current IOPS level, what does esxtop or VeeamOne report say?
Also, how much space do you need for the VMs? How much of that for the 5 active ones?
Calculating storage for your task requires additional variables to be precise.
With 48 Drives in a RAID 5 you can get 27.4 TB @ ~1600 - ~2800 IOPS (calulated for mixed Random I/O workload). I'm not 100% sure this will be sufficient performance for roughly 40 VMs. 5 intensive ones may eat a lot of the array's performance easily and make other 35 suffer.
What I would suggest is to do a RAID 10 LU for the 5 I/O-intensive VMs which will give them around 3.6 TB @ 1400 IOPS and guaranteed low latency.
Remaining drives go into ~20 TB RAID6 and still give you ~1200-2100 IOPS which is not that far from the original 1600-2800 you had when doing a 48-drive RAID6.
I don't want to go further into theoretical storage so I'll stay tuned for additional inputs about your environment.
The idea above illustrates the approach I recommend to take with I/O-intensive VMs.
Answer: Although we're phasing out our P6xxx boxes I've had a lot of experience with them in this exact situation - firstly the 'one big disk group' is the right way forward, secondly please avoid RAID 5/50 (we can bore you with the numbers behind it but essentially it's quite a dangerous thing to use the larger the disks are), use R6/60 or R1/10 ok.

are there any performance implications when only having one LUN?

Yes but very minor, the vSphere Performance Guidelines document for v6 state that ideally you should have more smaller LUNs than one big one but it's not that big a deal. Time was when one VM writing would lock that LUN but that's been gone for a long time and while there are one or two very small issues (the queues issue you mention is one) with very large LUNs you're only talking a couple of percent difference in performance. Personally I like 2TB LUNs but feel free to have larger ones if you like.
Comment: Totally correct answer. Just to add, usually, LUN separation may only be required if your backup routine includes storage snapshots. In most cases LUN size has no impact on the performance and stability of storage subsystem.


Why does $\int_0^1 \int_0^1 \text E \left(W^2(s) W^2(t) \right) \,ds\,dt = 2 \int_0^1 \int_0^t \text E \left(W^2(s) W^2(t) \right) \,ds \,dt$

Question: Let $W(t)$ denote the standard Wiener Process, i.e. $W(t) \sim \text N(0, t)$ for $t \geq 0$.
I have shown that
$$\text E \left(W^2(s) W^2(t) \right) = st + 2\left[\min(s,\, t)\right]^2$$
and I used it to show that
\begin{align}
\int_0^1 \int_0^1 \text E \left(W^2(s) W^2(t) \right) \,ds \,dt &= \int_0^1 \left[ \int_0^t \text E \left(W^2(s) W^2(t) \right) \,ds +\int_t^1 \text E \left(W^2(s) W^2(t) \right) \,ds  \right] \,dt \\
&= \int_0^1 \left[ \int_0^t \left( st + 2s^2 \right) \,ds +\int_t^1 \left( st+2t^2 \right) \,ds  \right] \,dt \\
&= \frac{7}{12}
\end{align}
It turns out that
\begin{align}
2 \int_0^1 \int_0^t \text E \left(W^2(s) W^2(t) \right) \,ds \,dt &= 2 \int_0^1 \int_0^t \left( st + 2s^2 \right) \,ds \,dt \\
&= \frac{7}{12}
\end{align}
Question:

I am hoping someone could provide some insight as to why $\int_0^1 \int_0^1 \text E \left(W^2(s) W^2(t) \right) \,ds \,dt = 2 \int_0^1 \int_0^t \text E \left(W^2(s) W^2(t) \right) \,ds \,dt$
Comment: You split the integral into two parts depending on $s\lt t$ or $t\lt s$.  Because of symmetry, they are equal, so your result follows.
Comment: The integrals turn out that way.  Exactly what are you looking for?
Comment: I am looking for the reason why the integrals turn out that way.
Answer: The integrand is symmetric in $s$ and $t$.
The second integral only integrates over the region where $s<t$
Comment: I do, I think it's pretty standard language
Comment: If $f(s,\,t) = f(t,\, s)$ then we say the function is symmetric in $s$ and $t$?
Comment: Since we are integrating over the unit square, we could integrate over the triangular region given by $s<t$ and then double it. Is that the proper reasoning?
Comment: yes - that was my reasoning.


How to block CloudFront or other CDN to access my website

Question: Some guys create a CloudFront CDN to proxy my site, so people can access my website using url like df03039lsls933.cloudfront.net instead of www.example.com
How to block this using nginx?
From CloudFront doc, it said..
Host: CloudFront sets the value to the domain name of the origin that is
      associated with the requested object.

So I cannot block by using the host header, what would be the other approach?
(CloudFront is just an example, I want to block all CDNs with similar approach)
Comment: It's not clear what it is precisely you are trying to prevent. Can you clarify precisely what problem you are trying to solve?
Comment: So someone set up DNS to resolve `df03039lsls933.cloudfront.net` to your server IP? Or proxying your content through his server?
Comment: I don't want user to view my content via df03039lsls933.cloudfront.net for example.
Comment: Are you taking any steps to prevent [hotlinking](http://serverfault.com/questions/tagged/hotlinking+nginx)?
Comment: @alexeyten, someone set up a AWS cloudfront distribution to proxy and cache my site
Comment: @TomBrossman, the reverse, someone setup a CDN to cache and proxy my site, so user view my site without using my url.
Answer: CloudFront requests come from the documented IP ranges as well as with a User-Agent string that includes Amazon CloudFront. You can block either, but with AWS's IP ranges expanding fairly frequently I'd go with the User-Agent block.
Comment: There's not going to be a generic solution.
Comment: Unfortunately this only work with CloudFront, I am seeing if any generic solution
Answer: I would expect the CDN to add headers which can be used to identify them.  These would typically indicate the real IP address of the user as well as some other information.  They should also add or append an X-Forwarded-For header, which should also contain the IP address they are forwarding for.
You could use the presence of the header to trigger a redirect to your domain.  I would not use the X-Forwarded-For header for this redirect as this may be added by proxies on network boundaries as well as CDNs.  You will likely have to identify the CDN's header on a case by case basis.
You should also contact the CDN is someone has configured your domain without your permission.
Comment: This will also block a valid forwarding proxy like Squid. I just don't want user to view my content if they are not viewing from www.example.com
Comment: @Yoga The CDN should use their own header in addition to the X-Forwarded-For header.  Base the header off their header, not the X-Forwarded-For header.
Answer: I believe you can use javascript code to check what is in the location. This check will be done in the visitor browser (so not server-side!). This approach is quite similar to protection against opening a webpage in a iframe.
If such a situation is detected, then you can redirect user to the valid URL (or just don't show anything).
Comment: This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
Comment: @dawud: hope EDIT made it a bit more like an answer.. :)


Why use ${1/\sigma^2}$ as a prior for $\sigma^2$?

Question: In a lot of cases, the prior for $\sigma^2$ is chosen so that it is proportional to ${1/\sigma^2}$. I have a few queries re this:

What is the intuition for this choosing this prior?
What is the information conveyed by this prior? Does it mean that a higher value for $\sigma^2$ is less likely?
I know that this is an improper prior, but is it non-informative? Sorry, I am not entirely sure how non-informative priors are different from improper priors.
Answer: First of all, $p(\sigma^2) \propto 1/\sigma^2$ is the Jeffreys prior (http://en.wikipedia.org/wiki/Jeffreys_prior) for a scale parameter. It also coincides with the reference prior under certain conditions.
1) and 2):
Intuitively, it can be understood as the only prior expressing correclty that $\sigma$ is a scale parameter:  let $X$ distributed according to $1/\sigma .f(x/\sigma)$ then $Y=c.X$ has the same distribution than $X$ but with a different scale, $p(\sigma^2) \propto 1/\sigma^2$ also fits this property.
3):
Jeffreys priors are generally improper. However, to my knowledge improperness is not related to informativeness (but simply to integrability considerations).


Welch's t test vs Mann-Whitney for very unequal groups

Question: I have two vastly unequal groups (n1=18 and n2=400), and want to test mean differences between the groups for some 700 parameters (more specifically, RNA expression levels for 700 genes).
While I understand that Welch's t works well for non-normally distributed samples if both groups are large enough, this is not the case here. Testing for normality 700*2 times (or examining QQ plots) would take a lot and would almost certainly show that some variables are normally distributed and some are not.
In this case, would you go for

Welch t-tests for everything (which would probably be breaking assumptions),
Mann-Whitney tests for everything (which have a lower power and might lead to not getting too many significant results), or
choosing the appropriate test for each variable, according to normality testing? This would probably not be a good idea, since I have to adjust the p-values using the FDR method, and I don't think that is correct if the p-values are generated by completely different tests
Comment: " I understand that Welch's t works well for non-normally distributed samples if both groups are large enough" --- 'works well' in that it attains close to its required significance level if the sample sizes are large enough (where large enough depends, among other things on the distribution you're sampling from). But getting near to the desired significance level is not the only thing you need for a hypothesis test to perform in a desirable manner.
Answer: Your idea:

Mann-Whitney tests for everything (which have a lower power and might lead to not getting too many significant results)

is potentially misleading you in 2 ways.
First, as jbowman says on another page:

the worst that the Mann-Whitney can ever perform relative to the t-test is about 0.864 in terms of asymptotic relative efficiency, i.e., it would require 1/0.864x as much data to give the same power (asymptotically.)... The point of course being that you can't shoot yourself in the foot by using the Mann-Whitney test instead of the t-test, but the converse is not true.

Second, at the usual "significance" standard of p < 0.05 and 700 RNA species to examine, you will find about 35 "significant" differences even if there aren't any true differences. You will need to deal with the associated multiple comparisons problem, typically done via false discovery rate control with this type of study.
It's not a good idea to decide what type of test to perform based on a preliminary test of your results. This page has extensive discussion related to your type of situation.
Comment: thanks for the explanations, i think i got it


How to convert an integer into a String value with specified length using Groovy script

Question: Does anyone know how to convert an integer into a String value with specified number of digits in using Groovy script code? For example, I want to convert the integer values 1, 2, 3, 4 to the 4-digit strings as "0001", "0002", "0003" and "0004".
Answer: Just use Java's String.format:
def vals = [ 1, 2, 3, 4 ]

def strs = vals.collect {
  String.format( "%04d", it )
}

strs.each { println it }

prints:
0001
0002
0003
0004

Other options can be found here
Comment: How to add localization settings for decimal punctuation?
Answer: Use sprintf, which is added to the Object class so it is always available:
assert sprintf("%04d", 1) == "0001"

See the JDK documentation for the format string for more information.
Answer: You can use String.format() like described in JN1525-Strings
values = [1, 2, 3, 4]
formatted = values.collect {
    String.format('%04d', it)
}
assert formatted == ['0001', '0002', '0003', '0004']


Getting a binary string when testing an Artisan command

Question: I'm getting trouble testing an artisan command.
When I run an assertEquals against a getDisplay() output,
my test fails becouse the expected string is different to the actual binary string.
Here there are my test and the PHPUnit output: http://paste.laravel.com/15rN
What's wrong?
Comment: Show your binary string.
Comment: @sectus sorry, I guessed it wasn't relevant: 0x1b5b33326d64756d6d7956616c75651b5b306d
Answer: It looks like it's serializing the string because you have 'dummyValue'.PHP_EOL in there. Try taking out the concatenated PHP_EOL and try again.
Also on a side note, make sure you are using the latest version of PHPUnit.
Comment: It was an *ansicon* issue, PHP_EOL seems not to be a problem, thanks anyway :)
Answer: I was using ansicon to get colorful outputs on windows and that caused the returning binary string.
I had to uninstall it by typing ansicon -u in terminal.