Cox regression model - testing the proportionality assumption for an interaction term?

Question: I'm interested in testing sex differences in the outcome Y (e.g., mortality), and if there are sex differences in the effect of BMI on mortality.
I planned to use a Cox regression model
primary<-coxph(Surv(time,status) ~ sex, data=data)
second<-coxph(Surv(time,status) ~ sex*bmi, data=data)
And my first question is 1) if I can test proportionality assumption for an interaction.
I use cox.zph function for the test
cox.zph(second)
and I'm wondering if it makes sense to test an assumption for an interaction term, and if this is the right way to do it.
Also, when I tested the assumption of proportionality for the first model (no interaction term), it was violated for sex. I read that the nonproportionality could be handled by including the time*covariate interaction term, so I did that...
coxph(Surv(start, stop, status.time) ~ sex+sex:stop, data=longdata)
And I think I can report that sex differences decreased over time (the interaction was significant), but I'm wondering what this means for my research question about the bmi*sex interaction ('second' model)...
2) should I include a three-way interaction term for that?
coxph(Surv(start, stop, status.time) ~ sex*bmi+sex:stop+sex:bmi:stop, data=longdata)
Thanks in advance for your help!!
Answer: It's OK to use the cox.zph() function (and inspection of the plots it can produce) to examine non-proportionality of hazard for an interaction term. You perhaps shouldn't worry too much about the non-proportionality in the first model using only sex as a predictor, as Cox models typically are improved by including as many predictors reasonably related to outcome as possible (without going so far as to overfit the model). For example, it's possible that omitting bmi or the bmi:sex interaction helps make sex appear to have a non-proportional hazard when used as a single predictor.
If you do find non-proportional hazards, an interaction of time with a covariate can be a way to deal with that problem. The particular way you propose to code it, however, doesn't work.
The stop time represents the actual last observation time (death or loss to follow-up), which isn't really a predictor: you didn't know it at the beginning of the study. It's a (potentially censored) outcome of the study. So using stop time in the regression is essentially using the outcome to help predict the outcome, which is circular reasoning. See this answer for a bit more detail and a link to a vignette that shows how to use a correct time-transform function to accomplish what you wish.
If you aren't interested in the association of outcome with sex per se but rather in the interaction of bmi with sex, you might be able to accomplish your goal more simply by using a predictor bmi*strata(sex). Analyzing by sex strata could also simplify any modeling of bmi with splines or other transformations that might be needed to meet the linearity assumption of Cox regression.
One final thought: although not necessary for statistical analysis, it might help to start with bmi values centered around some typical value like the median. Otherwise the coefficient value reported for sex in a bmi*sex interaction term will be the hazard ratio for sex at a bmi value of 0, which of course can't ever happen. Starting by centering such continuous predictors included in interaction terms can make it easier to explain to others what the hazard ratios mean in practice.
Comment: Thank you so much this is great help! I tested the assumption in the model with the interaction included and none of the main or interaction effects are significant there, but the global model is. Should I still say that the assumption is violated (and go down the time interaction road...?) or could this be because of the large sample size I have?
Comment: @llbia look carefully at the curves from `cox.zph()` to get an idea about what might be going on. Based on your initial finding that PH is violated by `sex` as a single predictor, I'd suggest starting with stratification by `sex`. Also, not modeling `bmi` properly to meet the linearity assumption of the regression could lead to loss of PH; see [this question](https://stats.stackexchange.com/q/379416/28500) for a worked-out example. With a large data set use regression splines for `bmi` to help with any linearity problem, provided  by `rcs()` terms from the R `rms` package.
Comment: The first issue is that I actually haven't been able to plot things cz I keep getting an error "Error in qr.default(t(const)) : NA/NaN/Inf in foreign function call (arg 1)" whenever I'm trying to use plot() or ggsurvplot()... Do you have any thoughts on this?
Comment: Second, thanks a lot for the suggestion. When I test a model, coxph(Surv(time,status)~ strata(sex) + rcs(bmi) +//there are other covariates//, data=data), there is no significant effect with the 4 bmi terms. Would that suggest linearity of the variable? When I compare the models Harrell's way as the answer here suggests, https://stackoverflow.com/questions/47937065/how-can-i-interpret-the-p-value-of-nonlinear-term-under-rms-anova, I see that the model is significantly different (p = .01).
Comment: @llbia questions about plotting problems and error messages are best asked on R help or other sites (with a reproducible example provided); they are off topic here. A significant difference between nested models with and without `rcs(bmi)` means that `bmi` is important overall as a predictor. The spline fits the nonlinear shape of the relation of `bmi` to outcome; with a large data set it makes sense to include the spline to improve linearity of the model as a whole even if the higher-order spline terms for `bmi` aren't individually "significant."


Why should we store log files and bin-log files on different path or disks in mysql

Question: I have replication setup mysql databases....the log file location the bin-log file all are at one path that is default my data directory of mysql.
I have read that for better performance one should store them separately.
Can anyone provide me how this improves the performance. Is there is documentation available for the same. The reason why one should do so?
Answer: Mainly because then, reads and writes can be made almost in parallel. Stored separately meaning on different disks.
Linux and H/W optimizations for MySQL is a nice presentation of ways to improve MySQL performance - it presents benchmarks and conclusions of when to use SSD disks and when to use SCSI disks, what kind of processors are better for what tasks.
Very good presentation, a must read for any DBA!!
Answer: It's mostly for cases where your database write traffic is so high that a single disk volume can't keep up while writing for both data files and log files. Disks have a finite amount of throughput, and you could have a very busy database server.
But it's not likely that separating data files from binlogs will give better performance for queries, because MySQL writes to the binlog at commit time, not at query time. If your disks were too slow to keep up with the traffic, you'd see COMMIT become a bottleneck.
The system I currently support stores binlogs in the same directory as the datadir. The datadir is on a RAID10 volume over 12 physical drives. This has plenty of throughput to support our workload. But if we had about double our write traffic, this RAID array wouldn't be able to keep up.
You don't need to do every tip that someone says gives better performance, because any given tip might make no difference to your application's workload. You need to measure many metrics of performance and resource use, and come up with the right tuning or configuration to help the bottlenecks under your workload.
There is no magic configuration that makes everything have high performance.
Answer: It also can be really embarrassing to have your log files fill the file system and bring the database to a halt.
Answer: One consideration is that using a separate disk for binlogging introduces another SPOF since if MySQL cannot write the binlog it will croak the same as if it couldn't write to the data files. Otherwise, adding another disk just better separates the two tasks so that binlog writes and data file writes don't have to contend for resources. With SSDs this is much less of an issue unless you have some crazy heavy write load and are already bound by SSD performance.


Lightbox2 theming

Question: Can anyone advice me on how to theme the lightbox popup?
I have a design where the close and image navigation bottons have to be repositioned from their default location.
Please help!?
Answer: The easiest way is to create a new CSS file (or add lines to your existing file) and overwrite the rules that are applied to the lightbox. Unfortunately we cannot help you any further if you don't give more information. Can you give an online test case and preferably an image of what you are trying to achieve?

Lightbox2 theming


Probability Proof about A and B

Question: I have to formally prove that:
$$P(A) = P(A\wedge \neg B) + P(A\wedge B)$$
so I did like this:
$$P(A\wedge \neg B) + P(A\wedge B)$$
$$=P(A\wedge \neg B) + P(A)\cdot P(B)$$
$$=P(A)\cdot P(\neg B) + P(A)\cdot P(B)$$
$$=P(A)\cdot (1-P(B)) + P(A)\cdot P(B)$$
$$=P(A)-(P(A)\cdot P(B)) + (P(A)\cdot P(B))$$
$$=P(A)-0$$
$$=P(A)$$
Is this a valid formal proof? Did I commit any errors?
Edit:
$$P(A\wedge \neg B) + P(A\wedge B)$$
$$= P((A\wedge \neg B) \lor (A\wedge B))$$
$$= P(A\wedge (B\lor \neg B))$$
$$= P(A)$$
I'm not sure where to go from here? Does $(B\lor \neg B) = 0?$
Comment: Unless you are assured that $A,B$ are independent you can't assert that $P(A\cap B)=P(A)P(B)$.
Comment: $P(A\land B)=P(A)P(B)$ is only in very specific cases, not generally true. The theorem is generally true.
Comment: @lulu The question did not state if they were independent or not.
Comment: Right.  So you can't argue that way.
Comment: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
Comment: @lulu What would I do then? Since this is my first proof writing class.
Comment: @LuisVera is that always true however?
Comment: @Link Yes, it is.
Comment: Most axiom systems for probability use something like $X,Y$ mutually exclusive $\implies P(X)+P(Y)=P(X\cup Y)$.  Start there.
Comment: @LuisVera This is probably stupid, but is U the same thing as ^?
Comment: @Link $\cup \equiv \lor$ and $\cap \equiv \land$. Use the hints they are giving you!
Comment: Alright! Thank you! I'll post an edit once done.
Comment: @LuisVera I tried it like this, I'm not sure if it's right..
Comment: @Link $B \lor \neg B$ is your whole space, what do you get if you intersect your whole space with $A$?
Comment: @LuisVera you get A?
Comment: @Link Correct!!
Answer: When talking about probability, you usually start with a probability space $(\Omega, \mathcal{F}, P)$, where:
1. $\Omega$ is an arbitrary set, called the sample space. It is the set of outcomes, for example the sample space of a dice roll is $\Omega =\{1, 2, 3, 4, 5, 6\}.$
2. $\mathcal{F}$ is a collection of subsets of $\Omega$, called events. This collection also satisfies certain properties or axioms that (a) involve unions and complements: if $A,B$ are events, then $A\cup B, A^c, B^c$ are also events (the union also holds for countable collections of events) and (b) an important subset that always belong to $\mathcal{F}$ (namely $\Omega$). It is what is called a $\sigma$-algebra. In the dice roll case, you take the power set $\mathcal{P}(\Omega)$ as $\mathcal{F}$, and an example of an event $A \in \mathcal{F}$, is that the dice roll is an even number, so $A= \{2\} \cup \{ 4\} \cup \{6\} =\{2,4,6\} \subset \Omega$, so $A \in \mathcal{F}.$
Using these axioms and some properties of the set operations you can see that in every probability space: (a)$\, \emptyset \in \mathcal{F}$ (the empty set), and (b) if $A,B \in \mathcal{F}$, then $A \cap B \in \mathcal{F}.$
3. P is a function that sets a value between $0$ and $1$ to each event $ A \in \mathcal{F}$, so $P: \mathcal{F} \to [0,1]$. It is called a probability measure, and also satisfy certain axioms: (a) $P(\emptyset)=0$, (b) $P(\Omega)=1$ and (c) $P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)$ if the sets $A_k$ are mutually disjoint, this is $A_1 \cap A_2 = \emptyset$, for example. (This is the hint they gave you, it is also satisfied if you take a countable collection of events). For the event $A$ in 2., for example, $P(A)=P(\{2,4,6\})= P(\{2\}) + P(\{4 \}) + P(\{6\})$, since $A$ is a union of mutually disjoint sets, because the outcome of a dice roll can't be $2$ and $4$ at the same time, etc...
As you can see, when talking about probability, your basic objects of study are sets ($\Omega, A, B$) and functions ($P$), so it is important to learn some properties about them and their operations. The most important operations between sets are the union $A \cup B$, intersection $A \cap B$, difference $A - B$, and complement $A^c = \Omega - A$ (what you write as $\neg B$), and their properties (for example, associative, commutative, distributive, De Morgan Laws, etc...) so you can be able to manipulate them.
On the comments and your edit, for example, the last question $B \lor \neg B = 0$?, made me think that maybe you don't know some of these important facts, but since you showed interest and work I decided to make this long post so you have some basic definitions. I leave to you the task of researching more about them.
The complete statement and proof of your theorem should be as follows:
Theorem: Let ($\Omega, \mathcal{F}, P$) be a probability space, and $A,B \in \mathcal{F}$. Then
$$P(A) = P(A \cap B^c) + P(A \cap B).$$
Proof: Since $A,B \in \mathcal{F}$, by 2., $B^c \in \mathcal{F}$ and also by 2., $A \cap B \in \mathcal{F}$ and $ A \cap B^c \in \mathcal{F}$ (This is to ensure that these are in fact events, so we can talk about their probabilities).
$Afirmation \, 1: (A \cap B) \cap (A \cap B^c)= \emptyset.$ (that is, this events are mutually disjoint)
$Proof:$ Since $\cap$ is associative and commutative then
$$(A \cap B) \cap (A \cap B^c) = A \cap B \cap A \cap B^c = (A \cap A) \cap (B \cap B^c) = A \cap ((\Omega - B) \cap B) = A \cap \emptyset = \emptyset,$$
hence $(A \cap B)$ and  $(A \cap B^c)$ are mutually disjoint.
Now that we now this fact we may continue with the main proof.
Using 3.(c) for these mutually disjoint events we get
$$P(A \cap B) + P(A \cap B^c) = P((A \cap B) \cup (A \cap B^c)),$$
but by the distributive property of the intersection over the union we have
$$P((A \cap B) \cup (A \cap B^c))= P(A \cap (B \cup B^c))= P(A \cap (B \cup (\Omega - B))= P(A \cap \Omega)$$
Since $A \in \mathcal{F}$ is a subset of $\Omega$ by definition, we have
$$P(A \cap \Omega) = P(A)$$
Hence $P(A \cap B) + (A \cap B^c) = P(A) \, \blacksquare$

Probability Proof about A and B


ROC curve and thresholds: why does it never have the ideal point at the top left for observations close to certainty?

Question: I am using ROC curves for multi-label classification. I have a classifier that produces a score for each label, say a Logistic Regression that produces a probability. I understand that an ROC curve is parameterized by a discrimination threshold and assigns to a class the observations where a class with the highest probability is above the threshold.
If so, imagine these predictions for 5 observations with labels A or B:
<code>Observation #  Label  Prob(A)  Prob(B)
            1    A     0.9        0.1
            2    A    0.51       0.49
            3    B    0.51       0.49
            4    A    0.49       0.51
            5    B    0.49       0.51
</code>
The first observation is a freebie. With a discrimination threshold of 0.9, we assign that observation correctly and no observation incorrectly. So True Positives are 1 and all others are zero (True Negatives, False Positives, False Negatives). The True Positive Rate is 1 and the False Positive Rate is 0, which is the ideal point at the top left in an ROC curve. We never see that point in an ROC curve, so I suspect my reasoning is wrong, or my concept of True/False Positive Rates is wrong.
Another possibility is to assign only observations with a probability above a threshold to the most likely class, and all others to the negative class. But that approach lumps together an observation that we are sure is in the negative class and one that we're not sure is in the positive class. A consequence is that it is not invariant under re-labeling (positive to negative and vice-versa).
How exactly does an ROC curve use the discrimination threshold?
Comment: I'm confused with your example, what is label 1 (index 1)? Is A the positive class? If so with threshold 0.9 you will missclassify observations 2 and 4...?
Comment: Yes it's right, we do that. So in your example we assign 1 to A and 2-5 to B. So 2 and 4 are wrongly assigned to B. That's not a perfect classification, and that's why the AUC is < 1.
Comment: @Calimo I edited the question. The positive class can be A or B, since with more labels I will draw an ROC curve for each label. With a threshold of 0.9, my understanding is that we only classify observation 1, and the other observations are not assigned. Am I wrong?
Comment: From your comment, I understand that I am wrong and that at a given threshold, we assign observations with a probability above it to their predicted classes (observation 1 to class A), and all other observations to the negative class (observations 2-5 to class B). Is that right?
Answer: You seem to have a few misunderstandings about ROC curves.

I am using ROC curves for multi-label classification.

ROC curve are tools to assess the discrimination ability of binary classifiers. Some extensions exist for different types of problems such as multi class or multi label classification, but they are not ROC curves strictly speaking.

an ROC curve is parameterized by a discrimination threshold

A ROC curve is parameterized over all possible discrimination thresholds between $-\infty$ and $+\infty$.

With a discrimination threshold of 0.9, we assign that observation correctly and no observation incorrectly.

With a threshold of 0.9, we indeed (correctly) assign observation 1 to the positive predicted class.
All other observations are assigned to the negative predicted class. Because observations 2-5 are < 0.9, we assign them to the negative predicted class. As a result, observations 2 and 4, which should be positive, are misclassified as negatives, and decrease the True Positive Rate (sensitivity) and the AUC.
Because ROC curve is designed for binary classification problems, there is no such thing as "unassigned". If things are not positive, they are negative. If this assumption is not appropriate for your problem, then you're not having a binary classification problem, and ROC curves may be the wrong tool for you.

The True Positive Rate is 1 and the False Positive Rate is 0, which is the ideal point at the top left in an ROC curve. We never see that point in an ROC curve

This is wrong, this point is seen as soon as you have a perfect classifier. This might be hard to achieve in your field or for your problem, but it definitely exists.

How exactly does an ROC curve use the discrimination threshold?

I refer you to this CV question: What does AUC stand for and what is it?, which should answer this part of your question.
Comment: "at thresholds between 0.52 and 0.9, all observations are assigned to the negative class, right": except the first observation which is going to be positive.
Comment: If you call B positive, at threshold 0.52 observation 1 (0.9) is above the threshold, therefore predicted to be positive, IE in class B.
There are no labels in ROC analysis, only binary classes which you can call the way you want, A or B, positive or negative, patient or control, true or false. Don't let that confuse you. Maybe it's easier if you stop thinking of A and B and simply focus on positive and negative.
Comment: So in my example, if we relabel class A as the negative class, at thresholds between 0.52 and 0.9, all observations are assigned to the negative class, right? This implies that the ROC is not invariant to class re-labeling and that ROC is not the right tool. What do you suggest for gauging algorithms that predict multiple classes?
Comment: I don't understand your first sentence: "When B is the positive class, at threshold 0.52 it is predicted to be negative". What is "it" referring to, what is predicted to be negative?
Comment: When B is the positive class, at threshold 0.52 observation 1 is predicted to be in class A, and so assigned to the negative class. The other four observations are below the threshold and also assigned to the negative class, or A. Is that correct?
Comment: Ah, I understand now. Yes, it makes sense. In my example, if B is the positive class, the probability of positive is 0.1 (not 0.9 of being in the negative class) and well below the threshold.


dihedral group operation

Question: Let $D_{2n}$ be the dihedral group of order $2n$ with $r^n=1, s^2=1$. Then since $rs=sr^{-1}$, $r^as^b=s^br^{(-1)^ba}.$ Is this correct?
Comment: Maybe adding details on how did you get to the formula (just guessed based on few specific values of $a$ and $b$? Or do you have your own complete proof? etc.), and within what context you came to that, could help in reopening the question. I think it could be worthwhile.
Answer: For $b$ even, your formula holds for every $a$. For $b$ odd, your formula reads:
$$r^as=sr^{-a} \tag 1$$
Induction on $a$: for $a=1$, $(1)$ holds; let $(1)$ hold for $a$; then:
\begin{alignat}{1}
r^{a+1}s &= r^ars \\
&= r^a(ss^{-1})rs \\
&= (r^as)s^{-1}rs \\
&= (sr^{-a})s^{-1}rs \\
&= (sr^{-a})s^{-1}rs^{-1} \\
&= sr^{-a}s^{-1}(rs^{-1}) \\
&= sr^{-a}s^{-1}(s^{-1}r^{-1}) \\
&= sr^{-a}(s^{-1}s^{-1})r^{-1} \\
&= sr^{-a}s^{-2}r^{-1} \\
&= sr^{-a}r^{-1} \\
&=sr^{-(a+1)}
\end{alignat}
so that $(1)$ holds for every $a$.
In conclusion, yes, to me your formula looks OK.

dihedral group operation


Hiding an image field based on theme in Drupal 7

Question: I want to hide an image field(field_image) of a node when a different theme is selected.
How can I do it?
I'm not able to find any sample code in internet but I know it can be done both in tpl.php and template.php files.
Comment: By which way do you change theme on your site?
Comment: Using https://drupal.org/project/mobile_theme. I want to hide images in mobile devices to save bandwidth.
Answer: It can be done in node.tpl.php
hide($content['field_image']);

or in hook_preprocess_node
hide($variables['content']['field_image']);

It works with views as well if you are using content row plugin.
For fields you have to override views-view-fields.tpl.php and remove image field from $fields variable.
unset($fields['field_image']);

Also take look at this modules:
https://drupal.org/project/cs_adaptive_image
https://drupal.org/project/adaptive_image
Comment: Thanks can you pl. tell me how to hide a views field similarly?
Comment: I've updated the answer.
Comment: Can you pl. also answer: http://drupal.stackexchange.com/questions/65526/change-image-styles-based-on-selected-theme ? I want to change image style to small image for mobile devices. Those adaptive_image modules I don't want because I simply want to show desktop images in 1/3 of their size on mobile devices. These modules will only resize image if it is greater than screen size. In my case it's all logos of small size which should change to even smaller size in mobile.
Answer: You should be able to get the current theme with this code (inside your template for example):
<?php global $theme; ?>

Then you can do
<?php if($theme=="my_awesome_theme") { render($item); } ?>


How to edit breadcrumb in drupal 7?

Question: How to edit breadcrumb info without install any drupal module?
Example: blog
Home >> blog >> admin's blog
I want to remove admin's blog
I cant find node--blog.tpl.php file.
I only can find node.tpl.php file.

Many Thank.
Comment: can you explain more detailed? on what info did you mean?
Comment: @mixerowsky, I had edited my question. Kindly take a look. Many thank.
Comment: @Topsitemakers Many Thanks. Do you have any idea where to find this file: node--blog.tpl.php add
Comment: No need to search for that file; check out [this answer](http://drupal.stackexchange.com/a/43841/10041) specifically. Code from that answer should be pasted in your theme's `template.php` file.
Comment: @Topsitemakers I found template.php at sites/all/themes/responsive_bartik. I add this code: function responsive_bartik_links($links, $attributes = array()) {
    unset($links['blog_usernames_blog']);
    return theme_links($links, $attributes);
}
Not working.
Comment: If you need node--blog.tpl.php you simply have to create it by duplicating your node.tpl.php file and then rename it.
Answer: Check the menu_get_active_trail() and menu_set_active_trail() functions. They should be of use for what you want (retreiving and changing your breadcrumb).
Try the following in your template.php in your theme :
/**
 * Implementing theme_preprocess_html
 */
function YOURTHEME_preprocess_html(&$vars) {
    $crumbs = menu_get_active_trail();
    $count = count($crumbs);
    // dpm($crumbs);
    unset($crumbs[$count - 1]);
    menu_set_active_trail($crumbs);
}

See how it goes.
Comment: Sorry I dun get what you mean...
Answer: You can use this code in page.tpl.php replace with breadcrumb code.
$breadcrumb_array = drupal_get_breadcrumb();

print $breadcrumb_array[0]
drupal_get_breadcrumb() gives the array.
You can use array as you want.
Please let me know code worked for you
Comment: Not really a good solution.
Example:
Home >> book >> may
Home >> blog >> admin' blog
Home >> car >> color >> brand

Dun really work for my case. I only wan to remove admin's blog
Comment: You can use if condition if($breadcrumb_array[$i] != " admin' blog" )
{
print $breadcrumb_array[$i];
}
Answer: you can override breadcrumbs theme in your template.php file
$variables : An array containing the breadcrumb links
/**
 * Implementing theme_breadcrumb
 */

function theme_breadcrumb($variables) {
  $breadcrumb = $variables ['breadcrumb'];

  if (!empty($breadcrumb)) {
    // edit $breadcrumbs here
    $output = '
' . implode(' » ', $breadcrumb) . '
';
    return $output;
  }
}

https://api.drupal.org/api/drupal/includes%21theme.inc/function/theme_breadcrumb/7

How to edit breadcrumb in drupal 7?


Opscode Chef Ohai plugin - How to get a custom plugin to run automatically?

Question: The Ohai docs are incomplete. Here's what I've been able to do so far:

I've created a custom plugin that adds one piece of node data called "my_custom_data"
it works when I load it manually in IRB
I've used the Ohai cookbook to get it loaded on the servers that need it

However, Ohai doesn't load it, neither during Chef runs nor if I run Ohai manually.
The docs, here, are of little use in answering this question. http://docs.opscode.com/ohai.html
Comment: The question could be improved by adding details about what you've tried, and how it failed.  I personally use the ohai cookbook to distribute (and load) custom plugins, and it works just fine.  Still, I've added an answer below that offers a few tips - hopefully one helps!
Comment: Why the downvote? How can I improve this question?
Answer: Your question implies that you're already using the Ohai cookbook to distribute the plugin, and that that much is working - ie, your custom plugin ends up installed to /etc/chef/ohai_plugins (in the default configuration).  If that's the case, it's almost certainly trying to load the plugin - and the plugin is failing.
Ohai doesn't worry about failing plugins - it simply moves on to the next one.
When running by hand, make sure you're telling ohai to look in the additional plugin directory.  Here's how that fails (for a custom plugin called 'aws'):
[zts@ip-172-31-39-167 ~]$ ohai aws
[2013-07-25T22:14:53+00:00] INFO: [inet6] no default interface, picking the first ipaddress
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/ohai-6.16.0/lib/ohai/system.rb:247:in `attributes_print': I cannot find an attribute named aws! (ArgumentError)
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/ohai-6.16.0/lib/ohai/application.rb:101:in `block in run_application'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/ohai-6.16.0/lib/ohai/application.rb:100:in `each'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/ohai-6.16.0/lib/ohai/application.rb:100:in `run_application'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/ohai-6.16.0/lib/ohai/application.rb:75:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/ohai-6.16.0/bin/ohai:51:in `<top (required)>'
    from /usr/bin/ohai:23:in `load'
    from /usr/bin/ohai:23:in `'

[zts@ip-172-31-39-167 ~]$ ohai aws -d /etc/chef/ohai_plugins
[2013-07-25T22:15:05+00:00] INFO: [inet6] no default interface, picking the first ipaddress
{
  "region": "eu-west-1",
  "rds": {
<--snip-->

Finally, if you're seeing no output (but no errors), enable debug logging and search through the output to find the part associated with your plugin:
[zts@ip-172-31-39-167 ~]$ ohai aws -d /etc/chef/ohai_plugins -l debug
[2013-07-25T22:17:24+00:00] DEBUG: Loading plugin os
[2013-07-25T22:17:24+00:00] DEBUG: Loading plugin kernel
[2013-07-25T22:17:24+00:00] DEBUG: Loading plugin ruby
[2013-07-25T22:17:24+00:00] DEBUG: Loading plugin languages
<-- huge amounts of logs removed -->
[2013-07-25T22:18:21+00:00] DEBUG: Loading plugin aws
<-- way more logs here -->

Answer: You need to make sure that you have the following line in /etc/chef/client.rb.  If you are using knife bootstrap, you will need to specify a template.
Ohai::Config[:plugin_path] << "/etc/chef/ohai_plugins"


Locating the source of Forbidden message

Question: When I login I see these "Forbidden" messages. How to track the source of this message?
Also I've enabled these settings:
<code>error_reporting(E_ALL);
ini_set('display_errors', TRUE);
ini_set('display_startup_errors', TRUE);
</code>
Comment: Did you check Drupal and webserver logs? What does they say?
Comment: @FiNeX I could find only single message: " Session opened for ws-admin." in Drupal recent log messages and nothing in the web-server error log
Comment: Did you already check the file system permissions? Are you using modules to restrict node/entity access?
Comment: I've don'e `chmod -R 777` but same problem. Let me check restricting nodes.
Comment: As it's not a warning, it may be someone who put that in there with drupal_set_message().
Comment: Try to search `Forbidden` in your code base. You may be able to find where it from....
Answer: I started disabling modules one by one and rest_auth was actually causing it though it was  not being used. Simply disabling solved it. And yes it does not contain any word "forbidden".


How to loop if response is not an option?

Question: So I've been working on this code and got stuck and was wondering if you guys could help me out.
# 31/05/17 (dd/mm/yy)
# encryption and decryption
#--------------------------------------------
print("Hello, in this program you will be able to encrypt your own message,
and decrypt others")
#--------------------------------------------
ans = int(input("""
What would you like to do:
1. Encrypt your own message
2. Decrypt a message
Type the number corresponding to the action that you want to preform, then
press ENTER
"""))
#--------------------------------------------
 if ans == 1:
  print("hi")

#--------------------------------------------
 else ans == 2:
  code = input("""
:""")
#--------------------------------------------
break

#--------------------------------------------

How would I make it where if they don't enter a 1 or a 2, so it will just re-ask them?
Comment: Have you tried  to write a while loop yet?
Comment: i dont know what while loop to use
Comment: It'd be useful to see what you tried, though
Answer: Create a while loop that will only break with a valid selection.
print("Please enter 1 or 2: ", end='')
while True:                        // always true
    choice = int(input()):
    if choice in (1, 2):
         break
    else:
        print("invalid entry, try again: ", end='')

if choice == 1:
    do somehting...
elif choice == 2:
    do something else...

Comment: thanks for the help
Comment: what does the end=' ' do?
Comment: The print statement usually defaults to end='\n', meaning at the end of a print statement it will start a new line by default in the terminal. By using end='' it means the print statement will not start a new line and instead ends with an empty string, so the input() statement be on the same line as the print statement, making it look nicer.
Answer: Here is a simple change that will get you most of the way there:
print("Hello, in this program you will be able to encrypt your own message, and decrypt others")
#--------------------------------------------
ans = int(input("""
What would you like to do:
1. Encrypt your own message
2. Decrypt a message
Type the number corresponding to the action that you want to preform, then
press ENTER
"""))
#--------------------------------------------

while True:
  #--------------------------------------------
  if ans == 1:
    print("hi")
    break
  #--------------------------------------------
  elif ans == 2:
    code = input("""
    :""")
    break
  #--------------------------------------------
  else:
    ans = int(input("""
    What would you like to do:
    1. Encrypt your own message
    2. Decrypt a message
    Type the number corresponding to the
    action that you want to preform, then press ENTER"""))

Comment: Won't this have NameError?
Comment: yea, I skipped the first couple of lines, since they didn't change... I'll add them in.
Comment: Duplication of the `int(input(` part seems unncessary
Answer: Try the bellow code
print("Hello, in this program you will be able to encrypt your own message,and decrypt others")
#--------------------------------------------
while True:
    try:ans = int(input("""
    What would you like to do:
    1. Encrypt your own message
    2. Decrypt a message
    Type the number corresponding to the action that you want to preform, then
    press ENTER
    """))
    except:continue
    #--------------------------------------------
    if ans == 1:
      print("hi")
      break
    #--------------------------------------------
    elif ans == 2:
      code = input("""
    :""")
      break

try except will help you re-ask when you enter non numbers too.


measurable process stopped at random time

Question: Let $\big( X_t , t\geq 0 \big)$ be a measurable process, that is,
$$\big( t, \omega \big) \in \mathbb{R}_+\times\Omega\longmapsto X_t(\omega)\in\mathbb{R} \quad\text{is $\mathscr{B}\big( \mathbb{R}_+ \big)\otimes\mathscr{F} \big/\mathscr{B}\big(\mathbb{R} \big)$-measurable. }   $$
$T : \Omega\to \big[ 0, +\infty \big]$ be a random time.
Show that the collection $\mathscr{G}$ of all sets of the form $\big( X_T\in A \big)$ and $\big( X_T\in A \big)\cup \big( T = \infty \big)$, $A\in\mathscr{B}(\mathbb{R} )$ is a $\sigma$-algebra on $\Omega$.
Problem 1.17. Karatzas et Shreve.
My confusion is how to understand this question given that  $X_T$ is undefined on $\big( T = +\infty \big)$.

EDIT:  there would be no problem if we make a change as follows:
Show that the collection $\mathscr{G}$ of all sets of the form $\big( X_T\in A , T < +\infty \big)$ and $\big( X_T\in A , T < +\infty\big)\cup \big( T = \infty \big)$, $A\in\mathscr{B}(\mathbb{R} )$ is a $\sigma$-algebra on $\Omega$.
Comment: $\big( X_T\in A , T < +\infty \big)=\big( X_T\in A \big)$, so there is no undefined question in this problem.
Answer: One way of defining $X_T$ on the whole of $\Omega$ is to let
$$
X_T(\omega):=
\begin{cases}
\lim\limits_{n\to\infty} X_{T(\omega)\wedge n}(\omega)\quad &\text{if the limits exists and is finite},\\
0 &\text{otherwise}.
\end{cases}
$$
With this definition we have a random variable $X_T$ defined on $\Omega$ with $X_T(\omega)=X_{T(\omega)}(\omega)$ for $\omega\in\{T<\infty\}$.


Simulate Moderated Regression with Cholesky Decomposition

Question: I want to simulate a moderated regression where the slopes are standardized (i.e., can be interpreted like correlations), and I am wondering how to do this with Cholesky decomposition.
My initial approach was to generate three random variables that represented the outcome, "y," and two predictors, "x1" and "x2."
# Create initial dataset of random variables for predictors and dependent variable
raw.variables <- matrix( cbind(rnorm(1000), rnorm(1000), rnorm(1000)), ncol=3, dimnames=list(NULL, c("y","x1","x2")))

Next, I multiplied x1 and x2 to create the interaction term, and I added this to the raw variable matrix.
# Create interaction term and attach it to raw variable matrix
interaction <- apply(raw.variables[,c('x1','x2')], 1, prod)
raw.variables <- cbind( raw.variables, interaction )

Next, I defined a correlation matrix that represented the standardized slopes and the relationships between all the predictors that I desired.
# Define desired regression parameters
b1 <- .3
b2 <- 0
b3 <- .5

# Define correlations between predictors
cor_x1_x2 <- 0 # No correlation between x1 and x2
cor_x1_interaction <- 0 # No correlation between x1 and interaction
cor_x2_interaction <- 0 # No correlation between x2 and interaction

# Create correlation matrix
correlation.matrix <- matrix( c( 1, b1, b2, b3,
                                 b1, 1, cor_x1_x2, cor_x1_interaction,
                                 b2, cor_x1_x2, 1, cor_x2_interaction,
                                 b3, cor_x1_interaction, cor_x2_interaction, 1 ),
                              nrow=4 )
print( correlation.matrix )

#     [,1] [,2] [,3] [,4]
#[1,]  1.0  0.3    0  0.5
#[2,]  0.3  1.0    0  0.0
#[3,]  0.0  0.0    1  0.0
#[4,]  0.5  0.0    0  1.0

As a last step, I applied the Cholesky decomposition to the correlation matrix and multiplied it together with the raw variable matrix.
# Multiply matrices to generate correlated data
model.data <- data.frame( t( t( chol( correlation.matrix ) ) %*% t( raw.variables ) ) )
names( model.data ) <- c("y","x1","x2","interaction")

Finally, run the regression and ask for standardized regression coefficients.
# Test code with standardized regression
library( QuantPsyc ) # Need function lm.beta()
lm.beta( lm( y ~ x1 + x2 + eval( x1 * x2 ), data=model.data ) ) # You have to manually multiply x1 and x2 together (i.e., instead of using x1*x2) or lm.beta doesn't estimate the interaction slope correctly

#           x1            x2 eval(x1 * x2)
# 0.2967814978 -0.0121392943  0.0005289024

In the standardized slope output, you can see that the main effects for the predictors have the correct slopes (plus some noise), but the interaction term is way off. This is because the interaction term in model.data is not the product of x1 and x2 from model.data:
# Compare generated interaction variable to the product of x1 and x2
sum(apply( model.data[,c("x1","x2")], 1, prod ) - model.data$interaction)==0

# [1] FALSE

The reason it isn't working is that the Cholesky decomposition is creating a multivariate dataset that meets the criteria of my correlation matrix, without "knowing" that the fourth variable must be kept as the product of the middle two variables. I assume I am just totally missing something about the relationship between products and their component variables. Is there any way to simulate data for a moderated regression that would yield an interaction term with a specific effect size using Cholesky decomposition?
For the record, this question is specifically about how to simulate moderated regression with Cholesky decomposition, because you could of course also do it like so:
# (Less scalable) Alternative
y <- with(data.frame(raw.variables), b1*x1 + b2*x2 + b3*x1*x2 + rnorm(1000, 0, sd=sqrt(1-sum( b1**2, b2**2, b3**2 ))))
lm.beta( lm( y ~ x1 + x2 + eval( x1 * x2 ), data=data.frame(raw.variables[,c("x1","x2")]) ) )

#           x1            x2 eval(x1 * x2)
#   0.28163137    0.02982228    0.49765757

Comment: When you do the Cholesky stuff, you are applying a linear transformation. Computing $x_3 = x_1x_2$ is not linear in $x_1$ and $x_2$. You can write $(y, \mathbf{x}) = A \mathbf{w}$, that is, write $(y, \mathbf{x})$ as a linear transformation $A$ of some vector $\mathbf{w}$ where the components of $\mathbf{w}$ are standard normal random variables and non-linear transformations of those standard normal random variables, but this approach is a mess.
Answer: This derivation may help explain what went wrong.
Let's say you have a random vector $\mathbf{z} \sim \mathcal{N}\left( \mathbf{0}, I \right) $ (i.e. mean zero, uncorrelated, variance 1).
From this you'd like to apply a linear transformation to $z_1$ and $z_2$ to get $x_1$ and $x_2$ such that:
$$ \mathrm{Cov}\left( \begin{bmatrix}x_1 \\ x_2 \end{bmatrix} \right) = \begin{bmatrix} 1 & \rho \\ \rho & 1\end{bmatrix} $$
One way to do this is to multiply $\begin{bmatrix}z_1\\z_2 \end{bmatrix}$ by the Cholesky decomposition of the above covariance matrix. For $|\rho| < 1$ the Cholesky decomposition is given by $\begin{bmatrix}1& 0\\\rho&\sqrt{1 - \rho^2}  \end{bmatrix}$
Hence multiplying $\mathbf{z}$ by this matrix gives:
$$\begin{align*}
x_1 &= z_1\\ \\
x_2 &= \rho z_1 + \left(\sqrt{1 - \rho^2}\right)z_2\\ \\
\end{align*}$$
If you want $x_3$ to be the interaction of $x_1$ and $x_2$ then:
$$\begin{align*}
x_3 &= x_1x_2 \\
&= z_1\left(\rho z_1 + \left(\sqrt{1 - \rho^2}\right)z_2 \right) \\
&= \rho z_1^2 + \left( \sqrt{1 - \rho^2}\right)z_1z_2
\end{align*}
$$
Observe that not only do you have a $z_1z_2$ term, you have a $z_1^2$ term!
Now let's use $z_3$ to make the error term have variance $\sigma^2$, hence error term $\epsilon = \sigma z_3$.
Hence $ y = \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$ can be written with the variables $\mathbf{z}$ as:
$$ y = \left( \beta_1 + \beta_2 \rho \right) z_1 + \beta_2 \left( \sqrt{1 - \rho^2} \right)z_2 + \beta_3 \rho z_1^2 + \left(\beta_3  \sqrt{1 - \rho^2}\right)z_1z_2  + \sigma z_3 $$
Hence if you constructed a random vector
$\mathbf{w} = \begin{bmatrix} z_1 \\ z_2 \\ z_1^2 \\ z_1z_2 \\ z_3 \end{bmatrix} $ then:
$$ \begin{bmatrix}y \\x_1\\x_2\\x_3 \end{bmatrix} = \begin{bmatrix} \beta_1 + \beta_2 \rho & \beta_2 \sqrt{1 - \rho^2}  &  \beta_3 \rho &\beta_3  \sqrt{1 - \rho^2} & \sigma \\ 1 & 0 & 0 & 0 & 0\\
\rho & \sqrt{ 1 - \rho^2} & 0 & 0 & 0\\
0 & 0 & \rho & \sqrt{ 1 - \rho^2} & 0 &
 \end{bmatrix} \mathbf{w}$$
Anyway, the non-linearities of an interaction term severely complicates matters.
Also observe that $\mathrm{Cov} \left( x_1 , x_1x_2\right)$ isn't some free parameter you get to specify (i.e. you can't set cov_x1_interaction like that.)
$$
\begin{align*}\mathrm{Cov} \left( x_1 , x_1x_2\right) &= \mathrm{E}\left[x_1^2x_2 \right] \\
&= \mathrm{E}\left[\rho z_1^3 + \sqrt{1 - \rho^2}z_1^2z_2 \right]\\
&= 0
\end{align*}
$$
(If all the variables weren't mean zero, this would be different.)
If I were doing this, I would:

Generate random variables $z$
Do the standard Cholesky stuff to generate $x_1$ and $x_2$ using $z_1$ and $z_2$.
Compute $x_3 = x_1x_2$
Compute $y = \beta_1 x_1 + \beta_2 x_2 + + \beta_3 x_3 + \epsilon$

This is basically your last, "of course you could always do it like so." What's wrong with that?
Comment: There isn't REALLY anything wrong with the solution I had given except that I wanted to scale up the simulation to many iterations with different values. I figured that using a cholesky decomposition would require fewer lines of code and be easier for other people to scrutinize than setting the error variance for each combination of values. Your version gets me closer to this goal, for sure!

I know comments shouldn't be used just to say, "thanks," but I really appreciate how clearly you explained the problem and provided a straightforward solution. This is great! Thank you so much!


Error while trying to build stasm minimal

Question: I do not have experience on C++. And I have to use stasm for face detection. I'm traying to build the minimal example. On page 4 of this tutorial is possible to know what is necessary to make it work. But I'm these two errors:
g++ -Wno-deprecated -o teste minimal.cpp `pkg-config opencv --cflags --libs` -I/home/caaarlos/workspace/StasmDesbravando/stasm

In file included from minimal.cpp:46:0:
/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/pinstart.cpp: In function 'void stasm::CopyPoint(stasm::Shape&, const Shape&, int, int)':
/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/pinstart.cpp:138:13: error: redefinition of 'void stasm::CopyPoint(stasm::Shape&, const Shape&, int, int)'
static void CopyPoint(     // copy a point from oldshape to shape
         ^
In file included from minimal.cpp:37:0:
/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/convshape.cpp:9:13: error: 'void stasm::CopyPoint(stasm::Shape&, const Shape&, int, int)' previously defined here
static void CopyPoint(     // copy a point from oldshape to shape

This is my code:
// minimal.cpp: Display the landmarks of a face in an image.
//              This demonstrates stasm_search_single.

   #include <stdio.h>
   #include <stdlib.h>
   #include "opencv/highgui.h"
   #include <opencv/cv.h>

   //#include          "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/stasm_lib.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/asm.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/atface.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/basedesc.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/classicdesc.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/convshape.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/err.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/eyedet.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/eyedist.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/faceroi.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/hat.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/hatdesc.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/landmarks.h"
   //#include        "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/landtab_muct77.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/misc.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/pinstart.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/print.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/shape17.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/shapehacks.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/shapemod.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/startshape.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/stasm.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/stasm_landmarks.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/stasm_lib.h"
   //#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/stasm_lib_ext.h"

  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/asm.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/classicdesc.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/convshape.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/err.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/eyedet.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/eyedist.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/faceroi.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/hat.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/hatdesc.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/landmarks.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/misc.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/pinstart.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/print.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/shape17.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/shapehacks.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/shapemod.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/startshape.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/stasm.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/stasm_lib.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/MOD_1/facedet.h"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/MOD_1/initasm.h"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/MOD_1/initasm.cpp"
  #include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/MOD_1/facedet.cpp"

  using namespace cv;
  using namespace std;

  int main()
  {
    static const char* const path = "../data/testface.jpg";

cv::Mat_<unsigned char> img(cv::imread(path, CV_LOAD_IMAGE_GRAYSCALE));

if (!img.data)
{
    printf("Cannot load %s\n", path);
    exit(1);
}

int foundface;
float landmarks[2 * stasm_NLANDMARKS]; // x,y coords (note the 2)

if (!stasm_search_single(&foundface, landmarks,
                         (const char*)img.data, img.cols, img.rows, path, "../data"))
{
    printf("Error in stasm_search_single: %s\n", stasm_lasterr());
    exit(1);
}

if (!foundface)
     printf("No face found in %s\n", path);
else
{
    // draw the landmarks on the image as white dots (image is monochrome)
    stasm_force_points_into_image(landmarks, img.cols, img.rows);
    for (int i = 0; i < stasm_NLANDMARKS; i++)
        img(cvRound(landmarks[i*2+1]), cvRound(landmarks[i*2])) = 255;
}

cv::imwrite("minimal.bmp", img);
cv::imshow("stasm minimal", img);
cv::waitKey();
return 0;
}

I'm compiling my code in thi way:
g++ -Wno-deprecated -o teste minimal.cpp `pkg-config opencv --cflags --libs` -I/home/caaarlos/workspace/StasmDesbravando/stasm

What am I doing wrong? Can someone help me?
Thanks.
Comment: `#include "/home/caaarlos/Documentos/TCC/stasm4.1.0/stasm/asm.cpp"` What? You need to learn how to use a build system like CMake, this is terrible! (No offense, but this is really unusable on any system but yours.)
Comment: Can you show me a way to learn it? Thanks for your reply... Like I said, I did not know c++, but I have to use stasm.
Comment: No problem, I'm glad for your help!!
Comment: There are many build systems out there, and many good tutorials for each of them.  CMake is probably the most popular right now , so go and use your favorite search engine to find a good tutorial and follow it. Don't worry, this is not hard.
Comment: Ok, I'll try it. But, if I use CMake my compilation will work? Or the CMake use is just to make my code reusable?
Comment: No, the CMake-advice was not supposed to answer your question, but only to help you write better code. That's why I posted it as a comment and not as an answer. :) I do not know stasm and I do not know what you did wrong, but providing an [MCVE](http://stackoverflow.com/help/mcve) would certainly help the folks here to provide you a nice and clear answer.
Comment: Ok, I'll change my question and study CMake.
Answer: I've found a solution for this problem. If somebody has a problem like mine, you should enter on this site and use the CMakeList.txt and CMake to build Stasm. If you get any error you can try the intructions provides o the readme.


Regression in a one-to-many dataset

Question: I'm rather new to R and am working with an unfamiliar dataset format containing patient discharge records. The records are distributed in a typical SQL format with matching key pairs. I'm attempting to look for predictors of length of stay as a result of surgery and have been joining the dataset together in R. When using a left_join for example on the discharge record table to diagnosis table, patients often have multiple diagnosis codes to a single discharge so such a procedure duplicates that discharge row for the number of codes.
For example:
Discharge Table:

  Discharge_ID    Length of Stay
  1               2
  2               13
  3               5

Diagnosis Table:

  Diagnosis_ID    Discharge_ID   Code
  1               1              001
  2               1              052
  3               3              356

left_join(Discharge Table, Diagnosis Table):

  Discharge_ID    Length of Stay   Code
  1               2                001
  1               2                052
  2               13               NA
  3               5                356

Given that the duplication increases the data points for length of stay of records with codes, does this impact regression analysis and if so, is there a way to correct for it?
Thanks in advance!
Comment: What you seem to be saying is that each patient can have more than one diagnosis and that you want to predict length of stay. We can ignore the stuff about SQL and joins - it doesn't seem relevant here and is off topic in any case.

I am sure questions like this have been asked here before, but I could not find them. Maybe someone else can.
Answer: I would recommend re-shaping the data so that for each patient you have a single "length of stay" - this is your dependent variable, and you should have one per row.
Given that you can have multiple diagnosis codes per "length of stay", or even no diagnosis code at all, the best approach might be to dummy code your "diagnosis code" into many binary columns. e.g.
Discharge_ID    Length_of_stay    Diagnosis_001    Diagnosis_052   Diagnosis_356
           1                 2                1                1               0
           2                13                0                0               0
           3                 5                0                0               1

Then try predicting "length_of_stay" using the rest of the table, minus "discharge_id".
This kind of dummy coding and data manipulation is easy to do in R - have a look at dplyr package for data manipulation. The caret package has some useful dummy coding functions.
Comment: Thanks so much, I think using the spread function here should be able to distribute it like that. I'll give it a go
Comment: Yes, I think the spread function should work as you want. If this has helped with your question, can you mark it as answered.

Regression in a one-to-many dataset


how to replace nan value using the value of which from other rows with common column value

Question: Using column B as the reference how can I replace NaN value
>>> a
A   B
1   1
Nan 3
1   1
Nan 1
Nan 2
5   3
1   1
2   2

I want result like this.
>> result
A   B
1   1
5   3
1   1
1   1
2   2
5   3
1   1
2   2

I tried merging on the column b but couldn't figure out
b=a.groupby('B').reset_index()
dfM = pd.merge(a,b,on='B', how ='left')
Comment: https://pandas.pydata.org/docs/reference/api/pandas.Series.fillna.html
Comment: @Riley Even tho I look at this doc I don't know which method to use it.. could you help me?
Comment: `a["A"].fillna(a["B"])`
Comment: @Riley I don't think it's the right way first Nan should be replaced to "5" by using 6th row
Comment: Ah ok, I see what you're doing now.  I assumed your first 5 was a typo.  So 1 in B, always means 1 in A.  And 3 in B, always means 5 in A etc?
Comment: yes! that correct! I feel like it'll be easy but can't figure out for 2 days...haha
Answer: We need a map from values in column B to the values in A.
mapping = a.dropna().drop_duplicates().set_index("B")["A"]

It looks like this
B
1    1.0
3    5.0
2    2.0
Name: A, dtype: float64

Filling null values becomes irrelevant at this point.  We can just map B to get column A
a["B"].map(mapping)

This gives you
0    1.0
1    5.0
2    1.0
3    1.0
4    2.0
5    5.0
6    1.0
7    2.0
Name: B, dtype: float64

Cast to int and use it to overwrite column A in your original dataframe if you need to.
Comment: AWESOME!!!!!! Thank you so much...!!Have never no experience using mapping...!!


What is the smallest value of $x$ such that the matrix $A$ is not invertible?

Question:
$$A = \begin{pmatrix}
-1 & -3 & x \\
-2 & -7 & -1 \\
x & -6 & 1
\end{pmatrix}$$

So far I tried taking the determinant by expanding down the last column but i still cant get the right answer.
x * det $ \begin{pmatrix}
-2 & -7 \\x & -6
\end{pmatrix} $ + 1 * det  $ \begin{pmatrix}
-1 & -3 \\
x & -6
\end{pmatrix} $ +1*det $ \begin{pmatrix}
-1 & -3 \\
-2 & -7
\end{pmatrix} $
I end  up with $7x^2+15x+7$$=$$0$
Comment: That is the correct determinant. What is the smallest value of $x$ that solves that equation?
Comment: It's not nicely factorable, so you are going to have some square root in your answer.
Comment: Great, now use the quadratic formula to find where that polynomial equals 0.
Answer: You're on the right way continue by finding the roots
$x_1=-\frac{15}{14} - \frac{\sqrt{29}}{14}$
$x_2=-\frac{15}{14} + \frac{\sqrt{29}}{14}$
And taking the $\min\{x_1,x_2\} = -\frac{15}{14} - \frac{\sqrt{29}}{14}$
That's it that is your answer


Should a InitialDirContext object be closed explicitly everytime?

Question: There is an application that talks to LDAP on a need basis for several operations like user info fetch, list of users, list of groups, email ids etc. Every time a request has to be made an InitialDirContext object is created and used as follows.
<code>
Properties ldapProperties = new Properties();
ldapProperties.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
ldapProperties.put(Context.PROVIDER_URL, "ldaps://serv:636");
ldapProperties.put(Context.SECURITY_AUTHENTICATION, "simple");
ldapProperties.put(Context.SECURITY_PRINCIPAL, "user");
ldapProperties.put(Context.SECURITY_CREDENTIALS, "password");
ldapProperties.put("java.naming.ldap.attributes.binary", "tokenGroups");
InitialDirContext ctx = new InitialDirContext(ldapProperties);
ctx.search(....);
</code>
Should this "ctx" object be closed using the close() method?
If its not closed and there are multiple  new InitialDirContext() creations will the old ones be automatically closed?
If it was just an internal object it will be garbage collected but what about this connection object?
Answer: No reason to close the context object each time.
You should be aware that it would be possible that the server (or some network device like a Load balancer proxy) would close the connection and then you may need to reinitialize the connection.
We have some Example JNDI solutions you may find helpful.
-jim
Comment: Jim, thank you. I went through the examples. In my current code, the context object is not closed. And a new one is initialized whenever there is a need to communicate to LDAPS.

One thing I noticed in the examples is that the "keystore" is explicitly being specified. Is this a must? I have the certificate imported into JRE and dont see the SSLHandshake anymore so I did not specify the keystore.

@ What parameter on the server side should I be looking for to see if its the server that is closing the connection? (I see an intermittent connection reset error when communicating to LDAPS)


Sendmail adds X-Authentication-Warning for trusted users?

Question: So, I've digged through tons of articles and answers to configure sendmail on Ubuntu 12.04 correctly but still my emails are being flagged with X-Authentication-Warning header by sendmail:

X-Authentication-Warning: advisor.travel: www-data set sender to
  firstname.lastname@example.com using -f

by simple console call:

echo "Test email being by sendmail" | sudo -u www-data
  /usr/sbin/sendmail -f firstname.lastname@example.com

Please mind that I am not trying to configure a spam-proof setup here as everything regarding SPF, DKIM and other stuff are already in place :)
Sendmail info:
# sendmail -d0.1
Version 8.14.4
 Compiled with: DNSMAP LDAPMAP LDAP_REFERRALS LOG MAP_REGEX MATCHGECOS
        MILTER MIME7TO8 MIME8TO7 NAMED_BIND NETINET NETINET6 NETUNIX
        NEWDB NIS NISPLUS PIPELINING SASLv2 SCANF SOCKETMAP STARTTLS
        TCPWRAPPERS USERDB USE_LDAP_INIT XDEBUG

============ SYSTEM IDENTITY (after readcf) ============
      (short domain name) $w = advisor
  (canonical domain name) $j = advisor.travel
         (subdomain name) $m = travel
              (node name) $k = advisor.travel
========================================================

Configuration:
/etc/mail  # cat trusted-users
www-data

/etc/mail  # cat sendmail.mc | grep use_ct_file
FEATURE(`use_ct_file')dnl

/etc/mail  # cat sendmail.cf | grep -A 5 trusted
Ft/etc/mail/trusted-users %[^\#]
Troot
Tdaemon
Tuucp

No luck, still having X-Authentication-Warning header :/
I've read in several sources that trusted users file should be enabled in submit.mc but no idea how :/
Answer: Add FEATURE(use_ct_file) to submit.mc and generate new submit.cf.

Explanation:
Sendmail-8.12+ uses submit.cf to control "send an email" sendmail execution.
It has been introduced to avoid installing sendmail as set root uid (security risk).
Sendmail: Secure Install
Comment: Sendmail command you executed to send email is controlled by submit.cf [in sendmail-8.12+] see https://www.sendmail.com/sm/open_source/security/secure-install/  . It seems that sendmail documentation has not been properly and throughly "synchronized" with many consequences of the above change.
Comment: It worked! But where can I find the details about WHY should I do it? :/


Is it possible to display more than one node in a page?

Question: I am trying to display two nodes in Drupalgap.
//page_callback
function tung_intro_page() {
  var display_page = '' + '

';
  display_page += '

' + '';
  return display_page;
}

function tung_intro_pageshow() {
//  drupalgap_alert('My pageshow event has been called!');
 try{
        var nid = 13;
        node_load(nid, {
        success:function(node){
         var title = node.title;
         var content = node.content;
         //var whole = html(title) + html(content);
         $('#' + 'my_node_' + nid).html(content).trigger('create');
          }
        });
        nid = 3;
        node_load(nid, {
        success:function(node){
         var title = node.title;
         var content = node.content;
         $('#' + 'my_node_' + nid).html(content).trigger('create');
          }
        });
  }
  catch (error) { console.log('tung_intro_pageshow - ' + error); }
}

This only shows the content of the last node. How can I show both the nodes?
Comment: What if you place the second call to `node_load()` inside the first call's `success` handler? Essentially load the first node, wait for it and inject it, then retrieve the second node, wait for it and inject it. Not necessarily ideal, but probably a quick work around. Otherwise, look into tthis issue too: https://www.drupal.org/node/2601764
Comment: Thank you very much. It works when I put my second node_load() into the end of my 1st success handler.
Comment: However, an interesting new problem is, I have other hook menu doing the same thing and node_load() the same node. It won't display correctly when one has been called. Is it true that node_load() can only load a node once?(other module cannot load the same node again)
Answer: From our comment conversation...
Try this in your pageshow handler:
function tung_intro_pageshow() {

  // Load the fist node and inject it.
  var nid = 13;
  node_load(nid, {
    success:function(node) {

      var title = node.title;
      var content = node.content;
      $('#' + 'my_node_' + nid).html(content).trigger('create');

      // Load the second node node and inject it.
      nid = 3;
      node_load(nid, {
          success:function(node){

            var title = node.title;
            var content = node.content;
            $('#' + 'my_node_' + nid).html(content).trigger('create');

          }
      });

    }
  });
}

Not the most elegant solution, but a workaround nonetheless. A more ideal solution is dependent on the outcome of this: https://www.drupal.org/node/2601764


Prove that a set is open on $\mathbb{R}^{n+1}$

Question: Let $f$ be a continuous function defined on an open subset ${\cal S}$ of $\mathbb{R}^n$.  Prove that the set $\{ (x_1, x_2, \ldots , x_n, y) : x \in {\cal S}, y>f(x) \}$ is an open subset on $\mathbb{R}^{n+1}$.
It would be useful to abbreviate the set $(x_1, x_2, \ldots , x_n, y) = ({\bf x}, {\bf y})$.
Comment: And what have you tried?
Comment: I tried to write the set {(x,y)} as the inverse of S, because then f-1(S) would be open.
Comment: @SeanRoberson any suggestions please?
Comment: Your set is not $f^{-1}(S)$, so you give up that idea. Try showing that if $({\mathbf x}. {\mathbf y})$ is in the set, then so is any $({\mathbf x'}. {\mathbf y'})$ sufficiently close to it.
Answer: You might try what Paul wrote. Another option is to show that the complement is closed (either with or without the limit point characterization).
Yet another approach, which is more in line with your own attempt is to define a function $\mu: \mathcal S \times \mathbb{R} \longrightarrow \mathbb{R}$ by $\mu(\boldsymbol{x}, y) := f(\boldsymbol x)-y$. This function is continuous, and $(0, \infty) \subset \mathbb R$ is open. You may then express your given set as
$$
\{(\boldsymbol x, y) \mid f(\boldsymbol x)-y > 0\} = \mu^{-1}((0,\infty))\, .
$$
which is open, by the definition of continuity of $\mu$.
Comment: The function $\mu$ should be defined on $\mathcal{S} \times \mathbb{R}$ and I think that $\mu^{-1}\big( ]0,+\infty[ \big)$ is:

$$ \lbrace (x,y) \in S \times \mathbb{R}, \; y - f(x) \color{red}{>} 0 \rbrace $$
Comment: Absolutely! Thanks. Fixed the typos.


Does CSS3 box-sizing work in Facebook app: padding or sizing issue, creating unwanted scrollbars?

Question: Our Facebook app displays unwanted scrollbars, even though we defined the width as 500px. This app appears within the context of Facebook fan pages.
We originally set the width at 520px with side padding of 10px on the left and right. We used the CSS3 box-sizing property (and its browser variants) and set it to border-box.
Then we weren't sure if Facebook somehow stripped out the box-sizing property, so we set the width to 500px. Still the scrollbars won't go away.
Here's an example: http://www.facebook.com/coboak?sk=app_198939416792674
1) How to eliminate the scrollbars? (Yes, the app is configured with Auto-Resize.)
2) Does Facebook support the box-sizing property (and its browser variants)?
Thanks!
Answer: 1) remove width: 520px; from #main_box
2) box-sizing will affect only browsers which support it, no matter if it's on facebook
Comment: Thanks, MatTheCat. This is what we thought, too, re box-sizing. If we set #main_box to to use "box-sizing: border-box," shouldn't this work (which is what we did at one point)?
Comment: I don't understand, #main_box has no margin or padding so box-sizing won't change anything. It's just larger than #page_box.


Does $\int _{\Omega} \left | f_{n} \right |^p \rightarrow \infty $ implies $\left | f_{n}(x) \right | \rightarrow \infty $ pointwise?

Question: Let $f_{n}$ be sequence of Lebesgue integrable functions. $\Omega$ is open, bounded subset of $\mathbb{R}^n$.
We know that if $f_n$ converges to some limit in $L^p$, then we have a subsequence $ f_{n_{k}} $ which is bounded pointwise by a $L^p$ function. I don't know how it changes when the limit is infinity. Do we have a subsequence $ f_{n_{k}} $ that tends to infinity pointwise on some positive-measure subset of $\Omega$ (or entire $\Omega$)?
Thank you.
Comment: To be sure: you have an integral on $|f_n|^p$ and not on $|f_n|(x)$
Comment: Yes. $p$ can be 1.
Answer: No. Consider $\Omega=(0, 1)$ and $f_n(x)=f(x)=|x|^\frac{-1}{p}$.
Re to comment. If you want the sequence to be in $L^p(0, 1)$, redefine $f_n$ as follows:
$$
f_n(x)=\mathbb{1}_{\left[\frac1n, 1\right]}(x)\cdot f(x),$$
where
$$\mathbb{1}_A(x)=\begin{cases} 1, & x\in A \\ 0, & x\notin A\end{cases}$$
Comment: But each $f_n$ is not integrable in your example.
Comment: @Chan: See edit.
Answer: Take $f_n$ to be $0$ on $[0,1]$, except on $[1-\frac{1}{n},1-\frac{1}{n+1}]$ where $f_n(x) = n(n+1)n^\frac{1}{p}$. So $$I_n^p = \int_0^1 (f_n(t))^p dt = n\,.$$
$I_n^p \to \infty$, while $f_n$ is $0$ (at least) on $[0,1-\frac{1}{n}[$.


Monotonicity of special case of Kullback-Leibler divergence

Question: I have two discrete distributions $\tau$ and $\rho$ with the same support $\Omega$. I'm considering a weighted mixture of these distributions described by the following function:
$$
f(w) = (1-w) \cdot \tau + w \cdot \rho, ~~ \text{where} ~~ w \in [0,1]
$$
I'm particulary interested in very special case of Kullback-Leibler divergence / relative entropy:
$$
KL(f(w), \rho) = \sum\limits_{i \in \Omega}
((1-w) \cdot \tau_i + w \cdot \rho_i) \cdot
\ln (\frac{(1-w) \cdot \tau_i + w \cdot \rho_i}{\rho_i})
$$
Generally, $KL(f(0), \rho) \geq 0$ and decreases towards $KL(f(1), \rho) = 0$ as $w$ goes from $0$ to $1$.
I would like to formally assure that this very specific case of Kullback-Leibler divergence is MONOTONICALLY decreasing, because I wonder if there's a possibility of such phenomenon that I would call "cannot escape from the valley", which is depicted below:
Comment: It is continuous and I think it is monotonically decreasing as w goes from 0 to 1, but haven't proved it. I think, but with less confidence, it's a convex function of w over that range. Anyhow, unless you're really worried about execution speed, just throw it in any descent pre-canned nonlinear equation solver which allows the 0 <= w <= 1 bound constraints, or alternatively, nonlinear optimizer with bound constraints using $(KL-C)^2$ as objective function. If KL(0) >= C >= 0, and is strictly monotonic, there should be unique solution. Bisection would also work. Maybe try Math board for proof.
Comment: None of these methods require monotonicity. If you don't have it, there could be a possibility of multiple solutions to KL(w) = C, but you should still find at least one of them. Note that in this comment and previous comment, I used abbreviation of KL(w) meaning what you wrote as $KL(f(w),\rho)$
Comment: Examine the second derivative of $f$ with respect to $w$.
Comment: @gung I hope this reworked form of my question is sufficient, especially that with some hits by whuber I solved my problem. So, if you find it as still irrelevant, I would delete it completly.
Answer: According William A. Huber comment, which solves the problem and proves that this function is monotonically decreasing:
The second derivative is always non-negative.
$$
h_i''(w) =
\frac{(\rho_i - \tau_i)^2}{\rho_i} \cdot
\frac{1}{(1 - w) \cdot \frac{\tau_i}{\rho_i} + w}
$$
$$
(1 - w) \cdot \frac{\tau_i}{\rho_i} + w =
(1 - \frac{\tau_i}{\rho_i}) \cdot w + \frac{\tau_i}{\rho_i}
~~\text{this is linear function of $w$}
$$
This linear function from denominator takes values from
$[\frac{\tau_i}{\rho_i}, 1]$ or $[1, \frac{\tau_i}{\rho_i}]$, thus
the whole second derivative is non-negative, hence convex and I won't get such "valley".
As William cleverly pointed out, that's no the end, because I still must prove that this function is decreasing. I can accomplish that by using $h_i$ convexity and Jensen's inequality:
$$\forall_{t \in [0,1]}~
h(t \cdot w_1 + (1-w) \cdot w_2) \leq t \cdot h(w_1) + (1-t) \cdot h(w_2)
$$
Let's fix $w_2 = 1$ for which $h(w_2) = h(1) = 0$.
Now, $w_3 = t \cdot w_1 + (1-w)$ denotes some point to the right of
$w_1$ (between $w_1$ and $1$):
$$\forall_{t \in [0,1]}~
h(t \cdot w_1 + (1-w) \cdot 1) \leq t \cdot h(w_1)
$$
So, we have $w_1 \leq w_3$ and thankfully $h(w_1) \geq h(w_3)$, thus
this function is monotonically decreasing.
Comment: You're halfway there: you still can get a valley with a positive second derivative (consider a parabolic function near its vertex, for instance).  But since the second derivative doesn't vanish, the first derivative cannot turn around.  You still need to establish that the first derivative is everywhere negative.


Read Cookies using Javascript

Question: I want to create a cookie using php,an also i need to read it using javascript.ie,
by using setcookie('mycookie','hello',time()+34000) in php i am creating a cookie in my system.Now,i need to read it using javascript.Is it possible?If anybody knows please give me some idea.Please....help me.I am using javscript as follows
`

Comment: possible duplicate of [Shortest function for reading a cookie in JavaScript](http://stackoverflow.com/questions/5639346/shortest-function-for-reading-a-cookie-in-javascript)
Answer: Here's a nice post that demonstrates this scenario. In order to make a cookie accessible through scripting languages such as javascript it must have the HttpOnly property set to false. This is the last parameter of the setcookie function.
Comment: Thank you for your help to move forward..Sorry for late aknoledgement.Thanks again

Read Cookies using Javascript


Prove that Sequence $\phi _n(x)=n\phi(nx)$ is delta sequence

Question: Let $\phi : \Bbb R \to [0, \infty)$ be such that $\int_{-\infty}^{\infty} \phi (x) dx=1 $. Define $\phi _n(x)=n\phi(nx)$. Show that $\{\phi _n(x)\}_{n\in \Bbb N}$ is a delta sequence.
Attempt: To show this I need to show
$$\lim_{n\to\infty}(\phi_n,\psi)=(\delta,\psi)=\psi(0), \quad \forall \psi\in \mathcal{D}(\mathbb{R}).$$
$(\phi_n,\psi)=\int_{-\infty}^{\infty}n\phi(nx)\psi(x)dx.$
After Substitution $y=nx$, $\int_{-\infty}^{\infty}n\phi(nx)\psi(x)dx=\int_{-\infty}^{\infty}\phi(y)\psi\left(\frac{y}{{n}}\right)dy.$
After that I dont know, what have to do.
Answer: Hint: let $f_n\colon y\mapsto \phi\left(y\right)\psi\left(y/n\right)$.

What is the pointwise limit of $\left(f_n\right)_{n\geqslant 1}$?
Can you bound $\left\lvert f_n(y)\right\rvert$ by a quantity independent of $n$ and integrable in $y$?
Answer: Hint
For all $y$ $\phi(y) \psi \left( \frac{y}{n}\right) \rightarrow \phi(y) \psi(0)$ and use dominated convergence theorem.
Comment: $\psi \in \mathcal{D}(\Bbb R)$ so $|\phi(y) \psi \left( \frac {y}{n} \right) |\leq \|\psi\|_\infty |\phi(y)| \in L^1 $


Which method has better performance

Question: I want to update some term id on some node (let suppose 1000 node),My changes is convert all news node with category(term_id) = 3040 to category(term_id)=5).
I know I can do it with several methods( use views bulk operation, write batch running etc, .... ) but my question is about performance of below approaches

Approach based on old method ( node_save)
$query = new EntityFieldQuery();
$query->entityCondition('entity_type', 'node')
        ->entityCondition('bundle', 'news')
        ->propertyCondition('status', 1)
        ->fieldCondition('field_category', 'tid', array('3040'));
$result = $query->execute();
$nids= array_keys($result['node']);
//ym($nids);

$nodes= node_load_multiple($nids);
foreach ($nodes as $node){
    foreach($node->field_category['und'] as $i=>$item){
       if($item['tid']==3040){
            $node->field_category['und'][$i]['tid']=5;
            node_save($node);
        }
    }

}

Approach based on entity wrapper method
    $query = new EntityFieldQuery();
    $query->entityCondition('entity_type', 'node')
            ->entityCondition('bundle', 'news')
            ->propertyCondition('status', 1)
            ->fieldCondition('field_category', 'tid', array('3040'));
    $result = $query->execute();
foreach($result['node'] as $nid => $stub_node) {
  // Load the full node and wrap it with entity_metadata_wrapper().
  $node = node_load($nid);
  $wrapped_node = entity_metadata_wrapper("node", $node);
  if ($wrapped_node->field_category->value()->tid == 3040) {
    $wrapped_node->field_category->tid->set(5);
    $wrapped_node->save();
  }

in first approach we have one node_load and one node_save but in second approach we have one node_load one creating entity metawrapper creation and one entity wrapper save.
I want to know which method has better performance?
let suppose we have the same situation for both 2 method, I just want somebody tell me ( take a benchmark on one node and  in bulk ( multi node) mode and tell me the performance of both that which one of this method is faster in the same situation.
please provide your answer with facts or benchmark testing result
Comment: You need to run the benchmarks yourself, it's very unlikely you'll convince someone to do that work for you. It would be pointless anyway, an answer could only make sense if a benchmark was run on the exact environment the site in question is running on. Then it comes down to what modules are installed, how everything is configured, and what resources are available. On shared hosting it would be even more impossible to predict, and probably even more pointless to try
Comment: Rolled back to original question - please don't substantially change questions once they've been answered. Thanks
Comment: @Clive I have a mistake in question, load node multiple and one by one and it make my question to suitable,So I change my question to manner that the situation on loading be same and with this manner others can has better answers,What is your idea Master Clive, Do u think which method is faster?
Answer: This is essentially impossible to answer.  You have one part of your question, wrong

Your first method uses a single node_load_multiple() with multiple node_save
Your second answer uses multiple node_load() with multiple Entity API calls

My gut tells me the first will be quicker, as a single node_load_multiple() is typically more performant than multiple node_load() calls, given the same set of nodes.  You also aren't creating the the metadata wrapper objects.
However, you need to do the benchmarking on your system.  There may be a difference, but it may be small enough to not make a real imapact.  Some things that make cross-system benchmarks unreliable

entitycache can have a tremendous impact on a system, especially if you use a non-SQL backend for it
the number of fields on your node can effect how long the wrapper creation process takes, and also how long the node_save() takes
APC and/or OpCache usage and configuration
PHP version
MySQL configuration (or whatever database backend you are using), including some of the Drupal-specific tricks that can improve performance
The number of hook_node_update() instances that fire for each save
Comment: Normally, a substantial edit to a question like this after answers isn't really allowed.  I will let another moderator make the call here since one of the answers is mine.  However, my answer stands.  There are too many other factors at play.  You need to benchmark this on your system to see how much of a difference this makes.
Comment: ok, I change my question that both use one method and strategy.
Comment: I update my question and make it easier to answer about it.
Comment: I am sorry about make some change in my question, it because  I found there is not the same situation, tnx for your attention, but I ask, if we suppose we are in a common shared host with default and equal situation which method is fast.
Comment: As far as I understand, EntityMetadataWrapper::save() somewhere eventually just calls `node_save()` anyways, but the additional memory and CPU overhead of using the entity wrapper is probably not significant for most use cases.  If you are hitting timeout or memory limits, it would be best to use Batch API.
Answer: Given that performance is an issue you may alter the code of your first approach (1) like this :
$field_name = 'field_category';
$tid = '3040';
$nids = db_query("SELECT nid FROM {node} n, field_data_{$field_name} c WHERE n.nid = c.entity_id AND c.bundle = 'blog_post' AND c.{$field_name}_tid = :tid", array(':tid' => $tid))->fetchCol();
$nodes = node_load_multiple($nids);
foreach ($nodes as $node) {
  if ($node->{$field_name}[LANGUAGE_NONE][0]['tid'] == 3040) {
    $node->{$field_name}[LANGUAGE_NONE][0]['tid'] = 5;
    field_attach_update('node', $node); // Instead of node_load
  }
}

What I did was to (1) change the use of EntityFieldQuery() with db_query() by the time it is much faster. Moreover (2) I changed the node_save with the field_attach_update since there is no need to save the whole node (ref).
From drupal.org: Static queries vs Dynamic queries
I hope that helps!
Comment: Two notes.  In general queries against the node table should be an EFQ or `db_select('node')->addTag('node_access')` to prevent any access bypass problems.  However, there are cases where `db_query` is OK (eg, admin only touchup scripts).  And, the node hooks don't fire and the node access tables don't get updated with `field_attach_update`.  Again, this may or may not be a problem, depending on the situation.  If you have grants that use the term data, then it is a problem and would require a manual permission rebuild.
Comment: tnx for your attention, but I just need compare between 2 method( I know I can do them with better solution)
Comment: Thus would be the fastest, and may only take a few seconds at worst only. However, as MPD mentioned node access and grants, you will need to rebuild permissions. Furthermore, taxonomy module implements node_update hook to update its index, which field_update_* isn't doing. You'll have to manually rebuild taxonomy index at least.
Comment: I thought that the question was about performance on how to update a node's field, possibly through a script. I didn't knew that `node access`, grants, rebuild permissions and indexes also had to be updated,I guess it depends on the use case like MPD said. Frankly, I didn't even knew that `field_attach_update` don't updates those things.Still I learned something new, thanks!
Comment: @zhilevan This code (if you don't know already) is a reliable execution time measurement: `$mtime = microtime();
  $mtime = explode(" ", $mtime);
  $mtime = $mtime[1] + $mtime[0];
  $starttime = $mtime;  // Code goes here $mtime = microtime();
  $mtime = explode(" ", $mtime);
  $mtime = $mtime[1] + $mtime[0];
  $endtime = $mtime;
  $totaltime = ($endtime - $starttime);` Perhaps will help!
Comment: I know it ,tnx for your mention,;)
Comment: @mchar Your code can be simplified like that `$starttime = microtime(TRUE); /* Your code here */ $totaltime = microtime(TRUE) - $starttime;` Secondly, there are already timer functions in Drupal: `timer_start("mytimer"); /* Your code here */ $totaltime = timer_read("mytimer");`
Answer: After some research and some benchmark  I get below results
In equal and regular situation using Entity Meta Wrapper for change the node fileds has better performance.

Loading the entity object just for passing into EMW. Unless you
  already have the pre-populated entity, it's more efficient to just
  pass the entity id to entity_metadata_wrapper(); EMW will lazy load
  the field and property data that it needs, hoorah! So much magic.

(if we have $node it is better use it but if we don't have it Don't use node_load to load it before pass it to entity_metadata_wrapper and just pass nid to it has better performance rather first load node then pass it to EMW
// Unnecessary node load... $node = node_load($nid);
$wrapper = entity_metadata_wrapper('node', $node); // More efficient
(if the object isn't handed to you)...
$wrapper = entity_metadata_wrapper('node', $nid)

Saving the entity with node_save(), taxonomy_term_save(), etc. instead of the EMW save() method. Fun fact: both EMW and standard entity controllers include save() methods, so you can call e.g. $node->save() on properly instantiated entity objects and wrappers.
// If you don't already have the node object, don't bother loading it just to save...
$wrapper->field_foobar = 'new value';
$node = $wrapper->value();
node_save($node); // Easier and prettier...
$wrapper->field_foobar = 'new value';
$wrapper->save();

Reference: Death to Field Arrays

Which method has better performance


Stand alone single thread optimized server vs vm running on VMWare

Question: I apologize for the length of my post.
We recently converted our business system to a product that is licensed by the number of Cores.
We purchased the 2 core license and are currently running it on VMWare. It is quite expensive to move to the next license level.
We are having some pretty significant performance issues.
We have eliminated RAM as a bottleneck, as well as network, database and disk access.
The times we have issues are not always characterized by high CPU usages.
Sometimes the cpu usage is less than 20%, but the CPU Ready is high.
I am pondering the possibility of running it directly on a separate physical server.
Since I am limited to two cores I would select the fastest "Single Thread Performance" cpu I could get, likely 4 cores.
Then I would set the business system up to use 2 cores and allow the OS to consume the other 2.
My logic goes something like this.
Right now the VM is getting two threads or vCPU.
This is really only one pCPU.
Even if I moved it to the same box without hyperthreading it would get 2 full cores instead of 2 vCPU.
But if I pick a chip that is much faster on single thread, I could gain significantly more.
I would also likely have a small gain since I would not be virtualizing it any more.
Our current 6 core cpu gets the following scores according to Passmark.
CPU Mark: 8570
Single Thread Performance: 1403
The 4 core processor I want gets these scores.
CPU Mark: 10231
Single Thread Performance: 2287
Does it make sense to build a server specifically to gain the best Single Thread performance for an application like this?
Are there any cons I need to worry about?
Would a newer OS make a big difference?
The current host server has the following specs:
  Proliant DL380 G7
  2 Physical CPU's : Intel(R) Xeon(R) X5675 @ 3.07 GHz
  6 Cores Each for 24 logical processors (vCPU) total
  Hyperthreading is enabled.
  144 GB of RAM
Storage is on a newer SAN and I am sure that is not playing a part in our performance issues.
The business system VM is configured with 2 vCPU's and 24 GB of RAM. The OS is Windows Server 2008 R2.
The business system software is optimized to run on 4 - 6 GB of RAM in the app pool in IIS.
I have never seen the VM go above 16 GB in use.
We are also running 3 other VM's on the host.
One is really doing very little right now and will likely be turned off in the very near future. It has 2 vCPU and 8 GB of RAM.
The 2nd is running our MS Sql database. That one has 8 vCPU and 64 GB of RAM.
We had a DBA look at it and he does not feel that it is the cause of our issues, at least not from the database performance side.
The 3rd VM is running our public website. It averages less than 25 percent on CPU. It has 3 vCPU and 16 GB of RAM.
Once again, any help or wisdom you can provide would be awesome and much appreciated.
Thanks!
Answer: You are correct in your assumptions. For your particular case stacking Mhz's instead of vCores is a right way to go. Take a look at this processor: (features 4x 3,5-3,7 Ghz physical cores + HT).
However, I cannot see the real issue from what you have described unless you can provide your actual "CPU Ready" value.
Comment: Thank you for the answer.  At least it confirms my sanity a little bit.  Honestly I am not even sure I am interpreting the CPU Ready correctly.  I see no other way to get the increase I need with the money I have to spend.  Changing the license would cost a lot more than I will spend on the server I am planning to build.
Answer: Couple of things;
1) Try to limit yourself to as many vCPUs that your VM actually needs. This has to do with the resource scheduler in VMware ESXi, if you have a VM with 8 vCPUs, it has to find a timeslot on a physical CPU that has 8 cores free at the given time to run the clock cycle of the VM. So if you have 6 cores per CPU, the scheduler has to find a timeslot on both pCPUs, which basically puts everything else on hold. VMs does not run continiusly as physical machines do, they are "frozen" in between clock cycles, and if the scheduler has problems keeping up your VMs will spend more and more time "frozen". If you aren't running a huge database on the SQL VM, try lowering it's vCPUs to 6 and see if that doesn't improve things. I usually run 1 vCPU per default, 2 for larger VMs, and up to 4 for some larger databases. I've never seen a need to add more.
2) Forget about HyperThreading in VMware, it gives you a little more performance, around 10% according to some tests by VMware, so leave it on, but don't count on having 2x times the performance when specing hardware.
3) CPUs with faster clock rates give you faster single thread performance, but fewer cores gives you less ability to run more VMs. There's a reason VMware shows you an aggregate of all the speed of all the cores on the host, that's basically how ESXi sees CPUs, as an aggregate pool of compute power.
Comment: Thanks for the answer.  I mostly understand the vCPU thing.  That is why I am looking at moving it off of VMWare to a box that is specially built for Thread speed.  I've noticed that it seems the newer the processor, the less it is optimized for Thread speed, because they expect everything to just use multiple cores.  I can't do that because of the license.  The cpu I am looking at is this one: http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-1630%20v4.html
Comment: Ok, if it's due to licensing I'd go down that route as well!


Calculating statistical power for a Cox proportional hazards regression with a continuous predictor variable

Question: I have 322 samples with survival data (71 had events) and wish to calculate the power to detect an association with a linear predictor (no covariates) at $\alpha$=0.05 and hazard ratio=1.561. Been struggling to find any help I can understand as a non-mathematician.
The results of the model are:

coef=-0.07427
exp(coef)=0.92843
se(coef)=0.11206
Z=-0.663
P=0.508

This information has been requested by a co-author as a reply to a reviewer.
Comment: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Comment: I need to calculate the statistical power to detect an association between survival data and a continuous predictor variable using the alpha/HR/sample size above
Comment: If you already have the data, there is little point to estimating the "power" now. Either you found a statistically significant difference and had sufficient power, or you didn't. See [this page](https://stats.stackexchange.com/q/12118/28500), for example. Please edit the question to show the results of your model (regression coefficient and standard error) and to clarify why you "need to calculated the statistical power" at this point.
Comment: I agree, its bad study design but it has been requested for a paper and I need to do it
Comment: Requested by a co-author or by a reviewer? Please edit the question to include that information and to show the actual model results, as the best way to proceed depends on both of those.
Comment: I've edited the question with the info requested
Comment: the HR has been changed compared to the model for other reasons
Comment: @ChristopherWills you do not need to comply with unsound requests. That's how statistical sections of applied literature has become the garbage pile that it is.
Answer: A power calculation requires estimates both of the magnitude of the value you hypothesize and the error in that value. Such calculations are important in study design, for example determining how many individuals need to be in the study sample for a treatment, based on the effect magnitude you expect.
Once you have the data and have performed the test, a "post-hoc power (PHP) test" is pretty meaningless. In that case, you are basing calculations on an observed statistic and its standard error. All a post-hoc power analysis tells is how lucky you would have been if you had happened to get a "significant" result based on random sampling from a population having those values.
As Russ Lenth explains in Technical Report 378 from the Department of Statistics and Actuarial Science at The University of Iowa, "Post Hoc Power: Tables and Commentary":

PHP is simply a function of the P value of the test, and thus adds no new information.

He nevertheless provides tables that translate p-values to PHP. Here are some examples for a z-test like that used for individual coefficients in a Cox model (infinite degrees of freedom in a two-tailed t-test), based on a significance criterion of p < 0.05:

Observed p-value
0.01
0.05
0.1
0.25
0.5
0.75

PHP
0.7310
0.5000
0.3765
0.2100
0.1035
0.0617

So if the p-value was 0.5, then the PHP is 0.1035. That doesn't seem to represent the situation for your original value of 1.56 for a hazard ratio (a Cox regression coefficient of 0.445), but if you have the corresponding p-value then you can interpolate from this table or follow the calculations explained in that document.
Unfortunately, reviewers sometimes nevertheless ask for PHP values. Don't forget that editors, not reviewers, are responsible for decisions about publishing. The editors need to maintain the journal's reputation, and reputable journals are likely to have statistical consultants to help resolve disagreements between authors and reviewers. So one way to deal with this is to answer the reviewer's request politely with the PHP in the response to the reviewer's critique, but to explain to the editor why you are not going to include PHP values in the publication itself. You might use the above document as your justification.
Comment: @ChristopherWills not precisely "incorrect," but only helpful if you use the information for designing a new study. The usual PHP assumes that the HR you found is the true HR. You could use the PHP to estimate the sample size needed to have adequate power to detect that HR and report that sample size as a guide to future research.
Comment: @ChristopherWills "to replicate a finding at a specific HR" is different from a usual PHP analysis. If there is some effect well established in the literature that you weren't able to replicate, you need to determine why. For example, if the "true" coefficient value is 0.445 (HR 1.56) with a standard error of 0.112, that's a z-score of almost 4 and there certainly was power to detect that. In that case, see if the population wasn't the same as in other studies, or important outcome-associated predictors weren't included, or the model was incorrectly specified, or prior studies were incorrect.
Comment: This is an astute answer and should be selected. The parameters of the power calculation that the OP supplies are curiously specific enough to show the data are collected. If you've already "looked" at the data, there's no point at all in running a power calculation.
Comment: so for clarification, if i was to do a PHP analysis to prove that we never had enough samples/events in the first place to replicate a finding at a specific HR that would be the incorrect thing to do?
Comment: Thank you for the help EdM
Answer: @EdM's answer should be selected for this particular scenario.
In general, the way to perform a power calculation for a Cox proportional hazards model is to simulate data according to an exponential survival model parametrized by the median survival or the event-rate parameter. The simulation may continue to account for several other design dependent considerations such as:

Rate of accrual to study
Rate of censoring/drop out
The trigger for the primary analysis (accrual of events, number of years from start, etc.)

All of which are endemic considerations for truly prospective study designs. Once a simulation strategy is established and the parameters are supplied, the simulated data are fit using a Cox model and the corresponding inference at the pre-specified significance level is used to calculate the fraction of "significant" results for a given sample size. There are elegant mathematical approximations to identify the sample size which are best left to professional software. For analytic formulations like OLS, the power and sample size calculation are often equivalent formulations. However, for Cox model and other simulation-based results, there is no straightforward approach other than grid-search.


ASP.NET/IIS <---> Solr/Tomcat setup on EC2 - "best practice" configuration

Question: I am an ec2 newbie and deploying an asp.net application that interfaces with Apache solr (on Tomcat). In my dev environment I am running ASP.NET and Solr/Tomcat on my windows box.

I am also thinking of using a single c3.2xlarge instance to run all of them together on the same instance (mirroring my dev setup). Do you see any issue?
Alternately, is it advisable to move Solr to a separate instance? and running ASP.NET/IIS on a windows instance. Is there a best practice for such a setup? i.e. how much memory/storage and what kind of ec2 instance to use?

I am confused on the trade-offs if any, and also the cost implications. Could anyone share their insights? Thanks in advance for helping a newbie out.
Answer: You can get a pre-configured Solr instance from Amazon marketplace.
See https://aws.amazon.com/marketplace/pp/B008ASKV8U
I have used this on Linux along with IIS on Windows successfully.