What is the optimal sampling split for stratified sampling?

Question: Suppose we have a population split into two strata of interest and we want to estimate the mean of some quantity of interest pertaining to the population.  Denote the quantities of interest for the two strata as $X_1,...,X_{N_X}$ and $Y_1,...,Y_{N_Y}$ so that the population mean is:
$$\bar{Q}_\text{pop} = \frac{\sum X_i + \sum Y_i}{N_X+N_Y}.$$
Assume that we know the population sizes $N_X$ and $N_Y$, so we know the proportion of people in each strata in the population.  Suppose we decide to undertake stratified sampling and we have resources to sample $n = n_X+n_Y$ people from the total population.  How do we decide the optimal split between the two strata?  Is there any simple "rule of thumb" for this sampling choice?
Answer: In this analysis we will assume that we are making an inference about the population by forming the Welch-T confidence interval for the population mean, which is the standard interval estimator.  We will first set out the form of that interval estimator for the present problem and we will then show how we can "optimise" the sample allocation between the two groups to minimise the width of the confidence interval.  Under a simplifying assumption we discuss below, this leads to a fairly simple "rule of thumb" which we will discuss.

Interval estimator for the population mean under stratified sampling (with two groups): We have known population sizes $N_X$ and $N_Y$ for the respective strata, and we can denote the corresponding strata proportions by:
$$\lambda_X = \frac{N_X}{N_X + N_Y}
\quad \quad \quad \quad \quad
\lambda_Y = \frac{N_Y}{N_X + N_Y}.$$
The inferential object of interest is the mean quantity over the whole population, which can be written as:
$$\bar{Q}_\text{pop} \equiv \frac{\sum_{i=1}^{N_X} X_{i} + \sum_{i=1}^{N_Y} Y_{i}}{N_X + N_Y}
= \lambda_X \bar{X}_P + \lambda_Y \bar{Y}_P.$$
Inference about this quantity can be undertaken by forming the Welch-t confidence interval, adjusted to account for finite population.  To do this, suppose we sample $n_X+n_Y = n$ values from the respective strata, and our observed data has sample means $\bar{x}$ and $\bar{y}$ and sample variances $s_X^2$ and $s_Y^2$.  The level $1-\alpha$ confidence interval for the population quantity is:
$$\text{CI}(1-\alpha) = \Bigg[ (\lambda_X \bar{x} + \lambda_Y \bar{y})
\pm t_{\alpha/2, DF} \sqrt{\Big( \frac{1}{n_X} - \frac{1}{N_X} \Big) \cdot \lambda_X^2 \cdot s_X^2 + \Big( \frac{1}{n_Y} - \frac{1}{N_Y} \Big) \cdot \lambda_Y^2 \cdot s_Y^2} \Bigg],$$
where we use the Welch-Satterwaite degrees-of-freedom:
$$DF = \frac{(n_X-1)(n_Y-1)}{(n_X-1) w_{XY}^2 + (n_Y-1) (1-w_{XY})^2}
\quad \quad \quad
w_{XY} = \frac{\Big( \frac{1}{n_Y} - \frac{1}{N_Y} \Big) \cdot \lambda_Y^2 \cdot s_Y^2}{\Big( \frac{1}{n_X} - \frac{1}{N_X} \Big) \cdot \lambda_X^2 \cdot s_X^2 + \Big( \frac{1}{n_Y} - \frac{1}{N_Y} \Big) \cdot \lambda_Y^2 \cdot s_Y^2}.$$
This gives us the form of the confidence interval for the population mean, which is implicitly a function of the sample sizes $n_X$ and $n_Y$.  With a bit of algebra and calculus it can be shown that the function $DF(w_{XY})$ is a quasi-concave function with bounds $\min(n_X-1, n_Y-1) \leqslant DF(w_{XY}) \leqslant n_X + n_Y - 2$.  In the special case where we have infinite populations (i.e., we are making inferences about a mean parameter rather than the mean of a finite population) we have $1/N_X = 1/N_Y = 0$, which simplifies the above form of the functions.

Optimising the sample sizes: Accuracy of the inference can be measured using the half-width of the confidence interval:
$$\text{Half-length} \equiv t_{\alpha/2, DF(n_X,n_Y)} \sqrt{W(n_X,n_Y)},$$
where:
$$W(n_X,n_Y) \equiv \Big( \frac{1}{n_X} - \frac{1}{N_X} \Big) \cdot \lambda_X^2 \cdot s_X^2 + \Big( \frac{1}{n_Y} - \frac{1}{N_Y} \Big) \cdot \lambda_Y^2 \cdot s_Y^2.$$
It is a relatively simple computational problem to minimise the half-length function over all integer values $n_X$ and $n_Y$ satisfying the constraint $n_X + n_Y = n$, and this yields the post-hoc optimal sample sizes for stratified sampling.  The optimisation depends on the sample variances for the two samples, so a priori optimisation must occur under assumed variance values, which will generally differ from the observed values.
Exact optimisation of the above function is not particularly illuminating in an intuitive sense, but we can get a simple and illuminating "rule of thumb" if we approximate the optimisation by treating the degrees-of-freedom as if it were fixed (instead of a function of $n_X$ and $n_Y$).  Treating $DF$ as a fixed quantity means that $t_{\alpha/2, DF}$ is a fixed quantity, which means that minimisation of the half-length is equivalent to minimisation of $W(n_X,n_Y)$.  The Lagrangian for this constrained minimisation problem is:
$$\mathcal{L}(n_X,n_Y| \kappa) = \Big( \frac{1}{n_X} - \frac{1}{N_X} \Big) \cdot \lambda_X^2 \cdot s_X^2 + \Big( \frac{1}{n_Y} - \frac{1}{N_Y} \Big) \cdot \lambda_Y^2 \cdot s_Y^2 - \kappa (n - n_X - n_Y).$$
The Lagrangian is convex in $n_X, n_Y$, so the solution occurs at the critical point defined by:
$$\frac{\lambda_X^2 \cdot s_X^2}{\hat{n}_X^2} = \frac{\lambda_Y^2 \cdot s_Y^2}{\hat{n}_Y^2} = \kappa,$$
which gives the explicit solutions:
$$\hat{n}_X = \frac{\lambda_X \cdot s_X}{\lambda_X \cdot s_X + \lambda_Y \cdot s_Y} \cdot n
\quad \quad \quad \quad \quad
\hat{n}_Y = \frac{\lambda_Y \cdot s_Y}{\lambda_X \cdot s_X + \lambda_Y \cdot s_Y} \cdot n.$$
This gives the "real" solution to the optimisation, and the integer solution should be nearby (i.e., one of the two integer pairs at either side of the real solution).  The equations for the optimal sample size can be written more succinctly in terms of proportionality as:
$$\boxed{\quad \\ \quad \hat{n}_X \propto \lambda_X \cdot s_X
\quad \quad \quad \quad \quad
\hat{n}_Y \propto \lambda_Y \cdot s_Y \quad \\}$$
This gives us a simple "rule of thumb" which says that we should sample from both strata in a manner that is proportionate to their weight in the estimator times their sample standard deviation.  Consequently, strata having greater weight in the estimator (e.g., due to larger population size) should be given proportionately more weight in the sampling, and strata having greater sampling variability should have greater weight in the sampling.  Of course, this is a slight approximation, since exact optimisation also adjusts to take account of the variation in the degrees-of-freedom in the estimator under the chosen sampling scheme.  Nevertheless, the latter should generally be a small effect, so the rule of thumb gives quite a good approximation to optimal stratified sampling.
As an intuitive example of application of this principle, suppose we have a large population composed of equal numbers of males and females, and we want to know the average weight of the people in the population.  In such a case it is a reasonable a priori hypothesis that males will have larger variance in weight (since they are heavier on average).  We might decide that a reasonable a priori supposition is that males will have weight standard deviation that is 20% higher than females, in which case our "rule of thumb" would tell us to sample 6:5 males vs females in our stratified sampling.


Spring reactive Webclient timeout

Question: I was trying to test the default timeout of Spring reactive Webclient .
For that purpose I created a rest endpoint that takes 10 hours to return a response.
I created a rest client using spring reactive Webclient. But I see that the spring Reactive Webclient keeps waiting for 10 hours.
Doesn't spring reactive Webclient has any default timeout?
Comment: what is the default timeout of Spring reactive Webclient (asuming we don't give any timeout option as in our code)
Answer: If you are using Reactor Netty as HTTP client library which is default using Spring WebFlux there is no default response timeout specified. If you would like to configure timeout settings there are multiple options as described in the Projectreactor reference documentation chapter 6.14.

Spring reactive Webclient timeout


Correct way to compare two (very) different regression models?

Question: I'm working with some piecewise linear regression models, and I'd like to compare their predictions with those produced by multiple (weighted) linear regression models. Both models describe the same physical system, but have very different parameterizations of the independent variable. The two different parameterizations are such that the averages of the independent variables are much different--that is, call $x_1$ the independent variables under the first parameterization, and $x_2$ the ind. vars. under the second parameterization. Generally, I have $\mathbb{E}x_1 \gg \mathbb{E}x_2$. This (in turn) means that the model coefficients may be very different.
Additionally, it is sometimes the case that the piecewise linear regression may have a segment with slope = 0 and intercept = 0, which would seem to cause a problem for a statistic like CVRMSE.
The best way I can think to compare these two models is to just use a training and a test set, but then, I'm not sure what statistic I should compute to (algorithmically) say "this one is better". Is there a better way to discriminate between these two models in an a priori way?
Comment: The dependent variable, however, stays the same? How many independent variables do you have in each model? I think you can use AIC (Akaike Information Criterion) for comparing your models, as long as $Y$ stays the same.
Comment: Ahh great. The Y values are indeed the same. If you put this in an answer I can accept it :)
Comment: Is the likelihood the same on both cases? If so, you can use the test likelihood.
Answer: Since the two models are not nested (that is, the independent variables of one model are not a subset of independent variables of the other model), one cannot use a maximum likelihood test. However, you can consider the AIC (Akaike Information Criterion) or one of its variants. If you have likelihoods of your models, let's call them $\mathcal{L}_1$ and $\mathcal{L}_2$, you can easily calculate the AIC's of your models with
$AIC_{1} = -2 \log(\mathcal{L}_1) + 2\cdot K_1$
where $K_1$ is the number of estimable parameters in the first model. Now, a single AIC value is not informative, or, rather, it is only informative relative to alternative models. Therefore often, when you have several models, one calculates the differences of the AICs of models relative to the AIC of the model with the smallest AIC:
$\Delta_i = AIC_i - AIC_{min} $
Now, you will not have a statistical test to compare these values. This is the information-theoretic approach, which is a different thing from the Neyman-Pearson hypothesis testing framework, and the two should not be mixed (Anderson 2001). However, there are some rules of thumb as to what is the magnitute of a delta that one considers "significant" (but in the common meaning of the word, and not as in "statistically significant"). In "Model selection and multimodel inference", Burnham and Anderson present the following table:
<code>Delta_i     Level of empirical support of model i
0-2          Substantial
4-7          Considerably less
> 10         Essentially none
</code>
That is, if the difference of AIC of your two models is 4-7, you can assume that one of the model is "considerably" better supported by evidence than the other one. In fact, the authors state that

It seems best not to associate the words significant or rejected with results under an information-theoretic paradigm. Questions concerning the strength of evidence for the models in the set are best addressed using the evidence ratio as well as an analysis of residuals, adjusted $R^2$ and other model diagnostics or descriptive statistics.

The variants of AIC include $AIC_c$ (or c-AIC) which is suitable for small sample sizes, and QAIC (for overdispersed count data).
There are alternatives, of course, which allow you actually to do hypothesis testing. See for example this question.
Comment: It seems to me that the larger delta-i is the larger the empirical support is for one model over the other. E.g. >10 is "substantial support" rather than "Essentially None" (and the reverse for <4).
Am I misunderstanding your wording or is the scale just listed backwards?
Answer: You might be interesting in taking a look at the Vuong's test and the subsequent literature.


How to enable horizontal scroll of UICollection View that is within a UITableViewCell?

Question: I have a UICollectionView placed inside a UITableViewCell. The collection view has its scroll direction set to horizontal. However, when I swipe left or right on it to try and scroll, it doesn't work. Instead the table view cell is just pressed. It seems like the table view is eating up the gesture and not allowing the collection view to register it.
How can I make it so the collection view can scroll left and right with horizontal swipes, while still allowing the parent table view to scroll vertically?
I went through this SO question and its accepted answer's linked blog post, but it did not seem to address the issue I am having. On a whim, I also tried throwing sendSubviewToBack(self.contentView) in my table view cell's awakeFromNib() based on this question, but it didn't seem to do anything.
Comment: We need more information to try to help. Are you designing this with Prototypes in Storyboard? Or doing everything via code?
Comment: I designed it with Storyboard. Let me know if/what code you want to see? It's literally just a UITableView which the cell is designed with a UICollectionView inside of it.
Answer: Difficult to know without seeing your project...
Here, though, is about as basic of a setup as possible.

Table view controller
cell prototype and class with a "row label" and a collection view
collection view prototype and class

Source for Storyboard:
<?xml version="1.0" encoding="UTF-8"?>
<document type="com.apple.InterfaceBuilder3.CocoaTouch.Storyboard.XIB" version="3.0" toolsVersion="19529" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES" useTraitCollections="YES" useSafeAreas="YES" colorMatched="YES" initialViewController="Ets-ss-TQd">
    <device id="retina3_5" orientation="portrait" appearance="light"/>
    <dependencies>
        <plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="19519"/>
        <capability name="System colors in document resources" minToolsVersion="11.0"/>
        <capability name="collection view cell content view" minToolsVersion="11.0"/>
        <capability name="documents saved in the Xcode 8 format" minToolsVersion="8.0"/>
    </dependencies>
    <scenes>

        <scene sceneID="6jQ-dU-swf">
            <objects>
                <tableViewController id="Ets-ss-TQd" customClass="TableColTableViewController" customModule="MoreScratch" customModuleProvider="target" sceneMemberID="viewController">
                    <tableView key="view" clipsSubviews="YES" contentMode="scaleToFill" alwaysBounceVertical="YES" dataMode="prototypes" style="plain" separatorStyle="default" rowHeight="-1" estimatedRowHeight="-1" sectionHeaderHeight="-1" estimatedSectionHeaderHeight="-1" sectionFooterHeight="-1" estimatedSectionFooterHeight="-1" id="zYt-i6-X2h">
                        <rect key="frame" x="0.0" y="0.0" width="320" height="480"/>
                        <autoresizingMask key="autoresizingMask" widthSizable="YES" heightSizable="YES"/>
                        <color key="backgroundColor" systemColor="systemBackgroundColor"/>
                        <prototypes>
                            <tableViewCell clipsSubviews="YES" contentMode="scaleToFill" preservesSuperviewLayoutMargins="YES" selectionStyle="default" indentationWidth="10" reuseIdentifier="someTableCell" rowHeight="125" id="B1r-57-a4y" customClass="SomeTableCell" customModule="MoreScratch" customModuleProvider="target">
                                <rect key="frame" x="0.0" y="44.5" width="320" height="125"/>
                                <autoresizingMask key="autoresizingMask"/>
                                <tableViewCellContentView key="contentView" opaque="NO" clipsSubviews="YES" multipleTouchEnabled="YES" contentMode="center" preservesSuperviewLayoutMargins="YES" insetsLayoutMarginsFromSafeArea="NO" tableViewCell="B1r-57-a4y" id="Zyt-9U-dTe">
                                    <rect key="frame" x="0.0" y="0.0" width="320" height="125"/>
                                    <autoresizingMask key="autoresizingMask"/>
                                    <subviews>
                                        <collectionView clipsSubviews="YES" multipleTouchEnabled="YES" contentMode="scaleToFill" dataMode="prototypes" translatesAutoresizingMaskIntoConstraints="NO" id="d9m-rN-Eyd">
                                            <rect key="frame" x="16" y="54" width="288" height="60"/>
                                            <color key="backgroundColor" red="1" green="0.83234566450000003" blue="0.47320586440000001" alpha="1" colorSpace="custom" customColorSpace="sRGB"/>
                                            <constraints>
                                                <constraint firstAttribute="height" constant="60" id="E26-bE-U5T"/>
                                            </constraints>
                                            <collectionViewFlowLayout key="collectionViewLayout" scrollDirection="horizontal" minimumLineSpacing="10" minimumInteritemSpacing="4" id="mrH-RV-vqe">
                                                <size key="itemSize" width="92" height="50"/>
                                                <size key="headerReferenceSize" width="0.0" height="0.0"/>
                                                <size key="footerReferenceSize" width="0.0" height="0.0"/>
                                                <inset key="sectionInset" minX="0.0" minY="0.0" maxX="0.0" maxY="0.0"/>
                                            </collectionViewFlowLayout>
                                            <cells>
                                                <collectionViewCell opaque="NO" clipsSubviews="YES" multipleTouchEnabled="YES" contentMode="center" reuseIdentifier="someCollectionCell" id="JEj-s8-EVA" customClass="SomeCollectionCell" customModule="MoreScratch" customModuleProvider="target">
                                                    <rect key="frame" x="0.0" y="5" width="92" height="50"/>
                                                    <autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMaxY="YES"/>
                                                    <collectionViewCellContentView key="contentView" opaque="NO" clipsSubviews="YES" multipleTouchEnabled="YES" contentMode="center" insetsLayoutMarginsFromSafeArea="NO" id="mgI-0a-PJH">
                                                        <rect key="frame" x="0.0" y="0.0" width="92" height="50"/>
                                                        <autoresizingMask key="autoresizingMask"/>
                                                        <subviews>

                                                                <rect key="frame" x="8" y="8" width="76" height="34"/>
                                                                <color key="backgroundColor" red="0.92143100499999997" green="0.92145264149999995" blue="0.92144101860000005" alpha="1" colorSpace="custom" customColorSpace="sRGB"/>
                                                                <fontDescription key="fontDescription" type="system" pointSize="17"/>
                                                                <nil key="textColor"/>
                                                                <nil key="highlightedColor"/>

                                                        </subviews>
                                                        <constraints>
                                                            <constraint firstItem="wJ1-UE-IvG" firstAttribute="leading" secondItem="mgI-0a-PJH" secondAttribute="leadingMargin" id="7v0-S7-Kcb"/>
                                                            <constraint firstItem="wJ1-UE-IvG" firstAttribute="top" secondItem="mgI-0a-PJH" secondAttribute="topMargin" id="NUW-9U-OkK"/>
                                                            <constraint firstAttribute="bottomMargin" secondItem="wJ1-UE-IvG" secondAttribute="bottom" id="YXn-Qi-b3X"/>
                                                            <constraint firstAttribute="trailingMargin" secondItem="wJ1-UE-IvG" secondAttribute="trailing" id="ZNu-px-4Va"/>
                                                        </constraints>
                                                    </collectionViewCellContentView>
                                                    <connections>
                                                        <outlet property="label" destination="wJ1-UE-IvG" id="0fz-E6-bfF"/>
                                                    </connections>
                                                </collectionViewCell>
                                            </cells>
                                            <connections>
                                                <outlet property="dataSource" destination="B1r-57-a4y" id="hHd-RQ-xJb"/>
                                                <outlet property="delegate" destination="B1r-57-a4y" id="3kW-TN-RKS"/>
                                            </connections>
                                        </collectionView>

                                            <rect key="frame" x="16" y="11" width="288" height="35"/>
                                            <fontDescription key="fontDescription" type="system" pointSize="17"/>
                                            <nil key="textColor"/>
                                            <nil key="highlightedColor"/>

                                    </subviews>
                                    <constraints>
                                        <constraint firstAttribute="trailingMargin" secondItem="d9m-rN-Eyd" secondAttribute="trailing" id="3Fd-ue-yEB"/>
                                        <constraint firstAttribute="bottomMargin" secondItem="d9m-rN-Eyd" secondAttribute="bottom" priority="999" id="5EL-HQ-hli"/>
                                        <constraint firstAttribute="trailingMargin" secondItem="GpB-tj-mDa" secondAttribute="trailing" id="UiZ-V8-tk0"/>
                                        <constraint firstItem="d9m-rN-Eyd" firstAttribute="leading" secondItem="Zyt-9U-dTe" secondAttribute="leadingMargin" id="axO-Nr-EUG"/>
                                        <constraint firstItem="GpB-tj-mDa" firstAttribute="leading" secondItem="Zyt-9U-dTe" secondAttribute="leadingMargin" id="hM3-MG-T66"/>
                                        <constraint firstItem="d9m-rN-Eyd" firstAttribute="top" secondItem="GpB-tj-mDa" secondAttribute="bottom" constant="8" id="muE-Zg-mgg"/>
                                        <constraint firstItem="GpB-tj-mDa" firstAttribute="top" secondItem="Zyt-9U-dTe" secondAttribute="topMargin" id="nea-19-Qvm"/>
                                    </constraints>
                                </tableViewCellContentView>
                                <connections>
                                    <outlet property="collectionView" destination="d9m-rN-Eyd" id="UOs-i9-xDd"/>
                                    <outlet property="rowTitleLabel" destination="GpB-tj-mDa" id="LpD-4r-JgM"/>
                                </connections>
                            </tableViewCell>
                        </prototypes>
                        <connections>
                            <outlet property="dataSource" destination="Ets-ss-TQd" id="RC7-c1-RLO"/>
                            <outlet property="delegate" destination="Ets-ss-TQd" id="sLA-mZ-FSY"/>
                        </connections>
                    </tableView>
                </tableViewController>
                <placeholder placeholderIdentifier="IBFirstResponder" id="435-nV-YGQ" userLabel="First Responder" customClass="UIResponder" sceneMemberID="firstResponder"/>
            </objects>
            <point key="canvasLocation" x="125.625" y="225"/>
        </scene>
    </scenes>
    <resources>
        <systemColor name="systemBackgroundColor">
            <color white="1" alpha="1" colorSpace="custom" customColorSpace="genericGamma22GrayColorSpace"/>
        </systemColor>
    </resources>
</document>

Classes:
import UIKit

struct SomeDataStruct {
    var color: UIColor = .white
    var num: Int = 0
}

class TableColTableViewController: UITableViewController {

    var myData: [SomeDataStruct] = []

    override func viewDidLoad() {
        super.viewDidLoad()

        let colors: [UIColor] = [
            .systemRed, .systemGreen, .systemBlue, .cyan, .magenta, .yellow,
            .red, .green, .blue, .orange, .brown, .purple,
        ]
        let nums: [Int] = [
            12, 15, 8, 21, 17, 14,
            16, 10, 5, 13, 20, 19,
        ]
        for (c, n) in zip(colors, nums) {
            let d = SomeDataStruct(color: c, num: n)
            myData.append(d)
        }

    }
    override func numberOfSections(in tableView: UITableView) -> Int {
        return 1
    }
    override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
        return myData.count
    }
    override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
        let cell = tableView.dequeueReusableCell(withIdentifier: "someTableCell", for: indexPath) as! SomeTableCell
        cell.rowTitleLabel.text = "Row \(indexPath.row)"
        cell.thisData = myData[indexPath.row]
        return cell
    }
}

class SomeTableCell: UITableViewCell {
    @IBOutlet var rowTitleLabel: UILabel!
    @IBOutlet var collectionView: UICollectionView!
    var thisData: SomeDataStruct = SomeDataStruct() {
        didSet {
            collectionView.reloadData()
        }
    }
}
extension SomeTableCell: UICollectionViewDataSource, UICollectionViewDelegate {
    func numberOfSections(in collectionView: UICollectionView) -> Int {
        return 1
    }
    func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
        return thisData.num
    }
    func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
        let cell = collectionView.dequeueReusableCell(withReuseIdentifier: "someCollectionCell", for: indexPath) as! SomeCollectionCell
        cell.label.text = "Cell \(indexPath.item)"
        cell.contentView.backgroundColor = thisData.color
        return cell
    }
}
class SomeCollectionCell: UICollectionViewCell {
    @IBOutlet var label: UILabel!
}

Result:
Comment: Thanks I'll check it out and let you know what I find.
Comment: So I guess I found my issues thanks to your answer. I had `userActionEnabled` set to `false` for my collection view lol. I did that originally because I wasn't able to click on the parent collection view cell otherwise. Now I have to figure out how to enable that, because my child collection view is taking all the touches now...


Spamassasin not checking SPF - Perl error?

Question: Getting lots of email spoofing spam from our own domains to my Webmin installation using Spamassasin, the error im receiving when debugging spam is as follows:

May 19 09:33:45.959 [15233] dbg: spf: cannot load Mail::SPF::Query module: Can't locate Mail/SPF/Query.pm in @INC (@INC contains: /usr/share/perl5/vendor_perl /usr/local/lib/perl5 /
  spf: one of Mail::SPF or Mail::SPF::Query is required for SPF checks, SPF checks disabled

This is allowing a lot of mail spoofing to come through.
My system is CentOS release 6.9 (Final)
How can i go ahead and install this module so spamassasin can make use of it?
Answer: It appear you don't have SPF installed. You should be able to install it using the command:
sudo yum install perl-Mail-SPF -y

Answer: I had similar issue with a clean SpamAssassin 4.0.0 installation on a fresh Debian 12.
The error i got (deep in manual spamassassin call debug output) was:
dbg: spf: cannot load Mail::SPF: module: Can't locate Mail/SPF.pm in @INC (you may need to install the Mail::SPF module)
The following solved the problem:
apt install libmail-spf-perl
Not sure why such a fundamental part was missing in a clean install of SpamAssassin. The first visible result in spamassassin was that any SPF checks were missing in the X-Spam-Status header (part tests= - no SPF_ value was there). There was no other "visible" indication of why the SPF checks were not evaluated at all.


Formulating a problem as linear programming optimization problem

Question:
Attempt:
Let $x_{INi}$ be the number of people who travel from Ithaca to Newark in class $i$ where $i=1,2,3$ where each number represents class Y,B, and M, respectively, by simplicity. Similarly, write $x_{NBi}$ and $x_{IBi}$. Our goal is to maximize
$$ z = 300x_{IN1}+220x_{IN2}+100x_{IN3} + 160 x_{NB1} + 130 x_{NB2} + 80 x_{NB3} + 360x_{IB1} + 280 x_{IB2} + 140 x_{IB3} $$
Now, let us focus on the constraints. First of all,
$$ \sum_{i=1}^3 (x_{INi}+x_{NBi}+x_{IBi}) \leq 30 $$
moreover, we must have
$$ 4x_{IN1}+8 x_{IN2} + 22 x_{IN3} \leq 34 $$
$$ 8 x_{NB1} + 13 x_{NB2} + 20 x_{NB3} \leq 41 $$
$$ 3 x_{IB1} + 10 x_{IB2} + 18 x_{IB3} \leq 31 $$
and finally, all the $x's$ must be positive.
Is this a correct formulation?
Answer: The following constraints must be respected.

The plane cannot seat more than 30 passengers.

What you know is that:

At first, the plane contains passengers that travel from Ithaca to Boston and Ithaca to Newark
After landing in Newark, the place will board passengers from Newark to Boston while seating passengers from Ithaca to Boston.

Therefore, $X_{IN} + X_{IB} \le 30$ and $X_{IB} + X_{NB} \le 30$ are necessary constraints. Here, $X_{IN} = X_{IN}^Y + X_{IN}^M + X_{IN}^B$ (and similarly for $X_{IB}$, $X_{NB}$).

Number of available tickets cannot exceed forecasted demand.

As an example, let's take the constraints related to $X_{IN}$.
$$
0 \le X_{IN}^Y \le 4, \qquad 0 \le X_{IN}^B \le 8, \qquad 0 \le X_{IN}^M \le 22.
$$
As pointed out in the comments, it is better to explicitly write down each constraint separately. Indeed, $X_{IN} \le 4 + 8 + 22$ would not contain any information about the upper bound of each fare class.

Maximize revenue.

Similarly to what you already proposed, this gets translated into maximizing
$$
Z := 300X_{IN}^Y + 220X_{IN}^B + 100X_{IN}^M + (\text{amounts related to other variables})
$$

Your approach seems to mix 0/1 programming with linear programming, particularly in the last set of equations. How are they meant to be interpreted?

Problem encoding in MiniZinc
<code>var 0..4: a;
var 0..8: b;
var 0..22: c;
var 0..8: d;
var 0..13: e;
var 0..20: f;
var 0..3: g;
var 0..10: h;
var 0..18: i;

var int: revenue = 300*a + 220*b + 100*c + 160*d + 130*e + 80*f + 360*g + 280*h + 140*i;

constraint a + b + c + d + e + f <= 30;
constraint a + b + c + g + h + i <= 30;

solve maximize revenue;
</code>
Answer: $$ \sum_{i=1}^3 (x_{INi}+x_{NBi}+x_{IBi}) \leq 30 $$
This constraint is incorrect. Consider that 30 passengers could enter at Ithaca and leave the airplane at Newark. Now your plane flies empty between Newark and Boston because this constraint disallows the empty seats to be filled.
Your other set of constraints is wrong as well. Right now you allow $5$ $IN1$ customers to book (since $4\cdot 5 \leq 34$), while the airplane can only accommodate $4$.
You really have to think more carefully, and be less inclined to try to "combine" multiple constraints into a single equation.
Comment: To fix the first constraint, would it be better if we put $\sum x_{INi} \leq 30$, $\sum x_{NBi} \leq 30$ and $x_{IBi} \leq 30$ ?
Comment: @JimmySabater That's still not correct. The passengers from $IB$ are always on the airplane, only $IN$ and $NB$ can board/deboard halfway through. This exercise is intentionally being tricky, you have to think carefully.
Comment: I think I have the fix now: $\sum (IN_i + NB_i) \leq 30 $ and $\sum IB_i \leq 30 $
Comment: @JimmySabater No, that's still not correct, hint: IN + IB and NB + IB, as those passengers are on the plane simultaneously. But I'm not here to verify your every attempt. I'm not responding to future comments, sorry.


Como listar um Array num checkbox, quebrando por um dos itens do Array?

Question: Tenho o Array abaixo:
this.ca = this.categorias = [
    {cate: 'oquefazer' , tipo: 'ACESSIBILIDADE:', nome: 'Cadeirante'        , icone: 'fa-wheelchair'},
    {cate: 'oquefazer' , tipo: 'ACESSIBILIDADE:', nome: 'Def.Visual'        , icone: 'fa-blind'},
    {cate: 'oquefazer' , tipo: 'ACESSIBILIDADE:', nome: 'Def.Auditivo'      , icone: 'fa-deaf'},
    {cate: 'oquefazer' , tipo: 'ACEITA CARTÕES:', nome: 'C.Cred. Amex'      , icone: 'fa-cc-amex'},
    {cate: 'oquefazer' , tipo: 'ACEITA CARTÕES:', nome: 'C.Cred. Visa'      , icone: 'fa-cc-visa'},
    {cate: 'oquefazer' , tipo: 'ACEITA CARTÕES:', nome: 'C.Cred. Diners'    , icone: 'fa-cc-diners'},
    {cate: 'oquefazer' , tipo: 'ACEITA CARTÕES:', nome: 'C.Cred. Mastercard', icone: 'fa-cc-mastercard'}
];

Gostaria de listar esse Array num checkbox da seguinte maneira:
ACESSIBILIDADE:
      [] Cadeirante   [] Def. Visual   [] Def.Auditivo

ACEITA CARTÕES:
      [] C.Cred.Amex  []C.Cred Visa    [] C.Cred. Diners  []C.Cred.Mastercard

Estou montando a instrução abaixo sem sucesso.
Alguém pode ajudar?

            <span *ngif {{item.tipo}} =
                {{item.tipo}}
                <input id={{item.nome}} type="checkbox" value={{item.nome}} />
            {{item.nome}}

Answer: Não sei qual seu conhecimento em Angular, mas, a montagem das diretivas *ngIf estão erradas, também não entendi o pq de utilizar tags li, mas vou deixar um exemplo de como pode ser feito, pelo layout que demonstrou na pergunta terá que percorrer o array 2 vezes para não permitir que repita  os tipos:

  ACESSIBILIDADE:

      <input type="checkbox" id="{{item.nome}}" value="{{item.nome}}" />
      {{ item.nome }}

  ACEITA CARTÕES:

      <input type="checkbox" id="{{item.nome}}" value="{{item.nome}}" />
      {{ item.nome }}

Pode ver um exemplo funcional com seu código aqui.
Comment: Entendi. Era isso que precisava. Percorrer os diferentes tipos do array. No caso, tenho mais alguns tipos, então posso fazer conforme seu exemplo. Pergunta: tem como fazer um nfFor dentro de outro, assim primeiro percorrro por tipo e depois por nome.
Comment: É, eu só percorri o array 2 vezes por causa da maneira que quer demonstrar na tela, dependendo da exibição dá pra percorrer apenas uma. Dá sim tbm, dá pra ter um ***ngFor** dentro de outro sim, só tem que ter cuidado que complica um pouco o controle do código, mas, dá pra usar sim.
Answer: pela diretiva *ngFor dá entender que está usando Angular 2+.
Sugestão para deixar mais autônomo o template (*.html) do controlador (*.ts):
com base no seu modelo de lista categoria, temos a interface:
interface CategoryValue {
  category: string;
  type: string;
  name: string;
  icon: string;
}

nesse momento a lista de categoria é do tipo CategoryValue[]

bem, a ideia principal é transformar essa lista em um objeto, no qual a chave é o CategoryValue.category e o valor o próprio CategoryValue[], exemplo do tipo:
type Categories = { [k: string]: CategoryValue[] };

uma opção para criar esse objeto é usar o reduce , exemplo:
example.component.ts
import { Component } from '@angular/core';

interface CategoryValue {
  category: string;
  type: string;
  name: string;
  icon: string;
}

type Categories = { [k: string]: CategoryValue[] };

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.scss'],
})
export class AppComponent {
  title = 'Angular';

 //poderia vir de um serviço, ex CategoriesService.getCategoryList
  categoryList: CategoryValue[] = [
    {
      category: 'oquefazer',
      type: 'ACESSIBILIDADE',
      name: 'Cadeirante',
      icon: 'fa-wheelchair',
    },
    {
      category: 'oquefazer',
      type: 'ACESSIBILIDADE',
      name: 'Def.Visual',
      icon: 'fa-blind',
    },
    {
      category: 'oquefazer',
      type: 'ACESSIBILIDADE',
      name: 'Def.Auditivo',
      icon: 'fa-deaf',
    },
    {
      category: 'oquefazer',
      type: 'ACEITA CARTÕES',
      name: 'C.Cred. Amex',
      icon: 'fa-cc-amex',
    },
    {
      category: 'oquefazer',
      type: 'ACEITA CARTÕES',
      name: 'C.Cred. Visa',
      icon: 'fa-cc-visa',
    },
    {
      category: 'oquefazer',
      type: 'ACEITA CARTÕES',
      name: 'C.Cred. Diners',
      icon: 'fa-cc-diners',
    },
    {
      category: 'oquefazer',
      type: 'ACEITA CARTÕES',
      name: 'C.Cred. Mastercard',
      icon: 'fa-cc-mastercard',
    },
  ];

  categories: Categories;

  constructor() {
    this.categories = this.categoryList.reduce((accumulator, currentValue) => {
      if (!Array.isArray(accumulator[currentValue.type])) {
        accumulator[currentValue.type] = [];
      }

      accumulator[currentValue.type].push(currentValue);

      return accumulator;
    }, {} as Categories);
  }
}

Com isso você consegue fazer algo assim no template, com a mágia do pipe keyvalue:
example.component.html

        {{categoryKeyAndValue.key}}:

              <input type="checkbox" [name]="categoryValue.category" [id]="indexKey + '_' + indexValue" [value]="categoryValue.name">
              {{categoryValue.name}}

resultado: veja o resultado online aqui
Comment: Bela resposta, não conhecia esse pipe, muito legal.


Linear relationship seems to depends from one value: would bootstraping help?

Question: Statistics beginner here so please don't shoot if the terminology isn't 100% neat.
In a sample of 45 individuals, I am looking at a relationship between two parameters (Annual_SOM_rate and Share_temp_past). When I calculate the p-value of the slope (linear regression), the result is significant:

However, the significance disappears when I omit the value in the top right corner (green circle). Although a little bit "alone", this value makes total sense from a practical point of view (we cannot discard it as an error). If I extend this argument, would removing other value also impact the relationship?
My question: which statistical operation would help me conclude if the relationship that I observe here is truly significant or not?
Would boostraping help me ?
Comment: You may be interested in the "fragility index", which measures how many values must change to reverse significance.
Comment: @Ben thanks for your message. I have looked into your idea but this index seems to be for clinical trials where the outcome is binary (e.g. survival/death). In my case, I do not see how this could be applicable?
Comment: The fragility index quantifies the sensitivity of an estimator to changes in e.g. outcome values and has been generalized beyond clinical trials and binary outcomes. See e.g. https://www.pnas.org/doi/abs/10.1073/pnas.2105254118 (my paper)
Answer: First off:

If I extend this argument, would removing other value also impact the relationship?

Almost certainly yes. The regression line slope depends on all observations, so removing any point will change the estimate, and therefore also the $p$ value.
(It is possible, but unlikely, that there are data points with no influence on the coefficient estimate. Removing such a point will not change the parameter estimate, but will change the $p$ value slightly, since this depends on the parameter estimate and the number of observations.)
Removing a data point may or may not move your $p$ value above or below the magic threshold of $p < .05$.

which statistical operation would help me conclude if the relationship that I observe here is truly significant or not?

Statistical significance depends on the observed data (and a correctly specified model). There is no "true" significance. Either the null hypothesis holds (e.g., there is no relationship), or it does not (e.g., there is a relationship) - and many statisticians will say that the null hypothesis never holds in the first place. All we can say given data (and a model) is whether the data are consistent with a null hypothesis or not.
Thus, if you have no reason to suspect a data error, you should take your $p$ value as it is. It would be good to be cautious about any conclusions you draw and note explicitly that significance depends on a single data point. Also, note that the difference between the slopes you estimate with and without this one data point will probably in itself not be significant (Gelman & Stern, 2006).

Would boostraping help me ?

Bootstrapping is just another way of looking at given data. It will give slightly different results. You may get $p<.05$ with the full dataset with a bootstrap - or not, and the same if you omit this observation. (And as above, a bootstrapped and a parametric parameter estimate may not be significantly different.) If all you are interested in is significance, then I would not worry overly about differences between a parametric and a bootstrap analysis.
Edge cases like these are one main reason to be careful not to over-interpret statistical significance. If you want more precision in your estimates, there is no way around collecting (a lot) more data.


Sobre SelectOneRadio

Question: Olá, pessoal. estou criando um programa que manipula cadastro de usuários.
Cada usuário cadastrado possui campos como nome, rg, cpf...... e um campo que se chama status, que mostra a situação do cadastro deste cliente. O status pode esta "ativo" ou "inativo".
Neste programa, estou com dificuldades na tela de pesquisa para visualizar os usuários cadastrados. Nesta tela existe um SelectOneRadio que possui três opções: "ativo", "inativo" e "todos".
Na tela de pesquisa, será realizado um filtro duplo. Por exemplo, se no campo de pesquisa for preenchido "bruno" e for selecionado a opção "ativo" no SelectOneRadio, o sistema terá que procurar todos os brunos com status "ativo".
Mas estou com dificuldades em criar uma função que identifique a opção que o usuário marcou no SelectOneRadio e realize o filtro duplo da pesquisa. A pesquisa pelo nome eu consegui implementar, mas estou com dificuldades para realizar a filtragem pelo nome e pelo status simultaneamente. Estou postando os códigos que influenciam a tela de pesquisa. são eles "PesquisaUsuarios.xhtml", "PesquisaUsuarioBean.java", "UsuariosRep.java" e "UsuarioFilter.java"
PesquisaUsuario.xhtml
    Pesquisar Usuarios
<ui:define name="corpo">
    <h:form id="frmPesquisa">

Pesquisa de usuários

            <h:outputText value="#{usuario.id}" />

            <h:outputText value="#{usuario.nome}" />

            <h:outputText value="#{usuario.rg}" />

            <h:outputText value="#{usuario.cpf}" />

            <h:outputText value="#{usuario.email}" />

            <h:outputText value="#{usuario.telefone}" />

            <h:outputText value="#{usuario.endereco}" />

            <h:outputText value="#{usuario.estado}" />

            <h:outputText value="#{usuario.cep}" />

            <h:outputText value="#{usuario.status}" />

                    <f:param name="usuario" value="#{usuario.id}" />

                <f:setPropertyActionListener value="#{usuario}" target="#{pesquisaUsuarioBean.usuarioSelecionado}"/>

</h:form>
</ui:define>

PesquisaUsuarioBean.java
@Named@ViewScopedpublic class PesquisaUsuarioBean implements Serializable{

private static final long serialVersionUID = 1L;

@Inject
private UsuariosRep usuarios;

private UsuarioFilter filtro;

private List<Usuario> usuariosFiltrados;

private Usuario usuarioSelecionado;

@Inject
private CadastroUsuarioService cadastroUsuarioService;

public PesquisaUsuarioBean() {
    filtro = new UsuarioFilter();
}

public boolean isInativo() {
    boolean inativo = false;
    for (int x = 0;x < this.usuariosFiltrados.size(); x++)
    {
        String ContasInativas = this.usuariosFiltrados.get(x).getStatus();
        if ( ContasInativas.equals("Inativo"))
        {
            inativo = true;
        }
    }
    return inativo;
}

public void excluir() {
    FacesContext context = FacesContext.getCurrentInstance();
    if (this.usuarioSelecionado.getStatus().equals("Ativo"))
    {
        this.usuarioSelecionado.setStatus("Inativo");
    }
    else
    {
        this.usuarioSelecionado.setStatus("Ativo");
    }
    try {
        this.cadastroUsuarioService.salvar(this.usuarioSelecionado);
        context.addMessage(null, new FacesMessage("O status do cadastro do usuário " + usuarioSelecionado.getNome() + " com o CPF : " + usuarioSelecionado.getCpf() + " foi alterado com sucesso."));
    } catch (NegocioException e) {
        FacesMessage mensagem = new FacesMessage(e.getMessage());
        mensagem.setSeverity(FacesMessage.SEVERITY_ERROR);
        context.addMessage(null, mensagem);
    }
}

public void pesquisar() {
    usuariosFiltrados = usuarios.filtrados(filtro);
}

public List<Usuario> getUsuariosFiltrados() {
    return usuariosFiltrados;
}

public UsuariosRep getUsuarios() {
    return usuarios;
}

public UsuarioFilter getFiltro() {
    return filtro;
}

public Usuario getUsuarioSelecionado() {
    return usuarioSelecionado;
}

public void setUsuarioSelecionado(Usuario usuarioSelecionado) {
    this.usuarioSelecionado = usuarioSelecionado;
}
}

UsuariosRep.java
public class UsuariosRep implements Serializable {

private static final long serialVersionUID = 1L;
@Inject
private EntityManager manager;

public Usuario guardar(Usuario usuario) {
    EntityTransaction trx = manager.getTransaction();

    trx.begin();

    usuario = manager.merge(usuario);

    trx.commit();

    return usuario;
}

public Usuario porNome(String nome)
{
    return manager.find(Usuario.class, nome);
}

public Usuario porId(Long id)
{
    return manager.find(Usuario.class, id);
}

public List<Usuario> listaDeUsu()
{
    return manager.createQuery("from Usuario", Usuario.class).getResultList();
}

public List<Usuario> raizes()
{
    return  manager.createQuery("from Usuario",Usuario.class).getResultList();
}

@SuppressWarnings("unchecked")
public List<Usuario> filtrados(UsuarioFilter filtro) {

    Session session = manager.unwrap(Session.class);

    Criteria criteria = session.createCriteria(Usuario.class);

    if (filtro.getNome() != null) {
        criteria.add(Restrictions.eq("nome", filtro.getNome()));
    }

    // orderBy do SQL
    return criteria.addOrder(Order.asc("id")).list();
}

public void remover(Usuario usuario) {
    this.manager.remove(usuario);
    EntityTransaction trx = manager.getTransaction();
    trx.begin();
    manager.flush();
    trx.commit();
}
   }

UsuarioFilter.java
public class UsuarioFilter implements Serializable {
private static final long serialVersionUID = 1L;

private String nome;

public String getNome() {
    return nome;
}

public void setNome(String nome) {
    this.nome = nome;
}

}

Para encurtar o código, apaguei algumas linhas iniciais que tratam com import, package entre outros.
Desde já agradeço por qualquer opinião ou sugestão.
Answer: Coloca na sua View de Pesquisa
...

    <f:selectItem itemLabel="Todos" itemValue="null" />
    <f:selectItem itemLabel="Ativo" itemValue="ativo" />
    <f:selectItem itemLabel="Inativo" itemValue="inativo" />

....

E no Filtro
...
private String status;

public String getStatus() {
    return status;
}

public void setStatus(String status) {
    this.status = status;
}
...

Na criteria
....
if (filtro.getStatus() != null) {
    criteria.add(Restrictions.eq("status", filtro.getStatus()));
}
....

Comment: Certo. muito obrigado, Jose. Mas quando eu marco a opção "todos" no SelectOneRadio, como eu poderia tratar? Neste caso a pesquisa simplesmente ignorará o campo status e pesquisará somente pelo nome.
Comment: Faz Assim: ``

Sobre SelectOneRadio


Comparar TextBox com campo em uma tabela SQLServer

Question: Estou travado nesse if. Quero comparar meu textbox com um campo em uma tabela no sql server, porem não sei como continuar.
segue o código:
<code>using (SqlConnection connection = new SqlConnection(connectionString))
{

string queryString = "SELECT id, nome, cpf FROM " + GetType().Name + "s";

SqlCommand comand = new SqlCommand(queryString, connection);

comand.Connection.Open();

comand.ExecuteNonQuery();

****GOSTARIA de verificar se já existe esse usuário cadastrado no banco de dados****

      if (reader.Read())
            {
                if (Convert.ToInt32(reader[0]) == txtCodigo.Text)//essa condição é verdadeira, deveria executar este if, porem, cai direto no else
                {
                    AtualizarCadastro();
                }
                else
                {
                    Gravar();

                }
            }
}
</code>
Obrigado desde já!!!
Answer: Tente desta maneira:
<code> SqlDataReader reader = comand.ExecuteReader(); //Executa o leitor
 if (reader.read())
 {
     if (txtCodigo.Text == reader[0].ToString()) //[0] é a primeira coluna que o select irá retornar, no seu caso o "id".
     {
      MessageBox.Show("Este usuário já foi cadastrado no banco de dados!"); //Só para testar.
     }
 }
</code>
Comment: Obrigado pela ajuda! Retornou este erro ao incluir a alteração sugerida:

"An unhandled exception of type 'System.InvalidOperationException' occurred in System.Data.dll

Additional information: Tentativa inválida de leitura quando não existem dados."

Estou pesquisando sobre.
Comment: Está retornando este erro somente quando o if é false ou em qualquer caso?
Comment: Em qualquer caso.
Comment: Tente adicionar um `if (reader.read())` em volta da condição. Editei a resposta, para que não haja dúvidas.
Comment: Ele até leu e entrou no if, porem estou testando com o codigo "id" igual ao da minha tabela do banco de dados, e depurando no Visual Studio, está caindo no else... Isso que eu não estou entendendo, parece que não está armazenando a informação que vem do DB no Reader...vou editar a pergunta com o if//else pra vc ver, de
Comment: Teste os valores que estão retornando. Exemplo: `MessageBox.Show(id +
"\n"  + reader[0].ToString());`
Comment: Pode me dizer qual era o problema e como consertou?
Comment: O reader[0], estava certo, lendo o primeiro registro... só que não era isso que eu queria, o certo era ler o registro selecionado em um datagridview, minha lógica que estava falha... Armazenei o ID selecionado no datagridview em uma propriedade dentro de uma classe no meu projeto, ai resolveu o problema.


Mapping k-means cluster centers and origins (measuring k-means accuracy)

Question: Say I generate a dataset $X$: the first $i$ samples follow $x_i\sim N(\mu_1,\Sigma_1^2)$, the next j samples follow $N(\mu_2,\Sigma_2^2)$ and the last $l$ samples from $N(\mu_3,\Sigma_3^2)$. Naturally, I denote the clusters $(1,2,3)$ by the order of creation.
When using kmeans(X,3)$clusters I get the predicted assignment of the samples, but here lies the problem: kmeans might enumerate the clusters differently, for example set $N(\mu_2,\Sigma_2^2)$ as cluster#3 and vice versa. In this situation, I cannot compare the 'original' cluster assignment (say $(1,1,2,2,2,3,3,3,3,3)$) with the predicted one (say $(1,1,3,3,3,2,2,2,2,2)$). this will result with only 20% accuracy, even though the actual accuracy is 100%.
Any ideas of how to map the centroids or measure the accuracy of the kmeans performance?
Answer: I'd suggest you start looking at the clusteval package for R, which offers various ways of evaluating clusterings.
For instance, cluster_similarity() measures the similarity between two clusterings by looping over all pairs of points and assessing whether they are assigned to the same cluster under both clusterings. Of course, this is agnostic to labelings of clusters. And one of the two clusterings could be the "gold standard" labels you may already have in supervised-learning.
Answer: You can use the Hungarian matching algorithm to find the best match of clusters.
The usual approach (see literature, and Wikipedia, on cluster evaluation, rather than reinventing the wheel) is to work with pairs of points, and the property if they are both in the same cluster, or different clusters. This reduces it to a two case problem (same, or different).
In your scenario, Gaussian Mixture Modeling will usually work much better...


How to prove inverse property of $P = \{hk \mid h \in H, k \in K\}$ is a subgroup of G?

Question: Let $G$ be a group with subgroups $H$ and $K$. If for every $g \in G$ and $k \in K$ we know that $g^{-1}kg \in K$, prove that $P$ is a subgroup of $G$, where $P = \{hk \mid h \in H, k \in K\}$.
I already proved the closure property but when I try to prove the inverse property, I end up with $ha^{-1} = hk^{-1}h^{-1}$!
Answer: So $K\triangleleft G$.  Now $(hk)^{-1}=k^{-1}h^{-1}=h^{-1}hk^{-1}h^{-1}=h^{-1}k'\in P$.


Add and/or option to Drupal Views' Contextual Filters?

Question: How can I add the "and/or" option to Drupal Views' Contextual Filters panel? I added it to Filter Criteria, but I can't seem to get it for Contextual Filters.
Answer: You can achieve your requirement by installing Views Contextual Filters OR module. By default contextual filters are in AND condition.

The module provides a views plugin which modify query to support OR
  conditions for contextual filters.


what is alternative/opposite to piecemeal statistical approach?

Question: I have only basic knowledge about bio-statistics. For not normally distributed data, I used Kruskal-Wallis test to investigate the statistical significance between different variables. I performed an exercise with three different intensities (different weights w1, w2, w3), then with one weight at three different speeds (s1, s2, s3). The readings were observed from 3 different points(p1, p2, p3).
I opted for statistical significance among p1, p2, p3 at (w1 and w2 and w3) and at (s1 and s2 and s3). then i opted statistical significance among w1, w2, w3 for p1 and p2 and p3. then i opted statistical significance among s1, s2, s3 for p1 and p2 and p3. So, there are 36 independent Kruskal-Wallis tests.
After that, Mann-Whitney test was performed for pairs (post hoc analysis).
I got comment on the test that, "the piecemeal statistical approach, consisting of a very large number of comparisons made between dependent variables during different conditions, without corrections for multiple testing, renders it probable that "significant" results may well be due to chance."
Can someone please suggest where I am wrong, and how and where to adjust the p value?
Answer: So far, what i found about this particular problem, it should be done by repeated measures of ANOVA, with Bonferroni method for posthoc.
The reason is the data is not independent. The p value is needed to be adjusted for multiple tests.


IIS URL Rewrite HTTP to HTTPS with Port

Question: My website has two bindings: 1000 and 1443 (port 80/443 are in use by another website on the same IIS instance). Port 1000 is HTTP, port 1443 is HTTPS. What I want to do is redirect any incoming request using "htt p://server:1000" to "https://server:1443". I'm playing around with IIS 7 rewrite module 2.0 but I'm banging my head against the wall. Any insight is appreciated!
BTW the rewrite configuration below works great with a site that has an HTTP binding on port 80 and HTTPS binding on port 443, but it doesn't work with my ports.
P.S. My URLs intentionally have spaces because the 'spam prevention mechanism' kicked in. For some reason google login doesn't work anymore so I had to create an OpenID account (No Script could be the culprit). I'm not sure how to get XML to display nicely so I added spaces after the opening brackets.
< ?xml version="1.0" encoding="utf-8"?>
< configuration>
  < system.webServer>
    < rewrite>
      < rules>
        < rule name="HTTP to HTTPS redirect" stopProcessing="true">
          < match url="(.*)" />
          < conditions trackAllCaptures="true">
                        < add input="{HTTPS}" pattern="off" />
          < /conditions>
          < action type="Redirect" redirectType="Found" url="htt ps: // {HTTP_HOST}/{R:1}" />
        < /rule>
      < /rules>
    < /rewrite>
  < /system.webServer>
< /configuration>

Comment: Curious why one would not `redirectType="Permanent"` in this rule?
Answer: Even though the question was answered awhile ago, this is the rule that I used.
<rule name="Redirect to HTTP on different port" enabled="true" stopProcessing="true">
      <match url="(.*)"/>
      <conditions>
        <add input="{HTTPS}" pattern="ON"/>
      </conditions>

      <action type="Redirect" url="http://{SERVER_NAME}:1000{HTTP_URL}" redirectType="Found" appendQueryString="false"/>
    </rule>

Comment: Your answer is for https-to-http, but the question is http-to-https.
Answer:             <rule name="HTTP to HTTPS on different SSL Port" enabled="true" stopProcessing="true">
                <match url="(.*)" ignoreCase="true" />
                <conditions logicalGrouping="MatchAll" trackAllCaptures="true">
                    <add input="{HTTPS}" pattern="off" />
                    <add input="{HTTP_HOST}" pattern="([^/:]*?):[^/]*?" />
                </conditions>
                <action type="Redirect" url="https://{C:1}:1443/{R:0}" appendQueryString="false" />
            </rule>

This solved my problem.
Comment: it should probably be: ([^/:]+)(:[^/]*)?  since port might not always be there.
Comment: For any future googlers: it looks like this answer comes from [this thread](http://forums.iis.net/t/1174819.aspx?Need+to+redirect+HTTP+to+HTTPS+on+non+standard+ports) on the IIS forum. A complete description of the regex is given there.
Answer: Try changing this:
<action type="Redirect" redirectType="Found" url="https://{HTTP_HOST}/{R:1}"/>

to this:
<action type="Redirect" redirectType="Found" url="https://{HTTP_HOST}:1443/{R:1}"/>

Comment: That doesn't seem to work. I wish the URL rewrite module had an easy way to see what it is changing the URL to.
Comment: So it looks like part of the problem is that {HTTP_HOST} includes the port e.g. if I type http:// server:1000 then {HTTP_HOST} contains 'server:1000'. This doesn't seem right especially since there is a separate {SERVER_PORT} variable.
Comment: The matching and condition part work. When I hard code the value of the redirect URL it works, but I need the redirect URL to be dynamic. If a user types in http:// 10.0.0.100:1000 (server IP) it needs to redirect to https:// 10.0.0.100:1443. Same if they type in http:// server.domain.com:1000 it needs to redirect to https:// server.domain.com:1443.
Comment: I just checked it on my server and it is in fact redirecting to domain.com:1443 as expected....  Here is what I added to the Redirect URL:  https://{HTTP_HOST}:1443/{R:1}
Comment: The main problem is the variable {HTTP_HOST} which already contains the PORT information.

IIS URL Rewrite HTTP to HTTPS with Port


What's the proper mathematical notation to say Range of Y in Domain of X

Question: How do I correctly express in mathematical notation with no words the bold part below
For F(x) F = 2x when x ranges from 0 to 2, y ranges from 0 to 4
or
For for log functions when x ranges from 0 to 1, y ranges from -inf to 0
Comment: The *image* of a set $A$ under a map or function $f$ is usually denoted by $f(A)$.  For example, if $f(x) = 2x$, then $f([0,2]) = [0,4]$, and if $g(x) = \log(x)$, then $g((0,1]) = (-\infty,0]$.  The *range* of a function is the image of the domain.
Answer: For the first one you write, $$F([0,2])=[0,4]$$
For the second one your write, $$ log((0,1))=(-\infty, 0)$$
Answer: You can simply say:
$$
\begin{aligned}
F(x) = 2x \qquad 0 \leq x \leq 2
\\ \therefore 0 \leq F(x) \leq 4
\end{aligned}
$$
Answer: Notice that, given a function $f:X\to Y$, the expressions $f(x),x\in X$ and $f(A),A\subset X$ are both meaningful expressions (in your case, the argument of your functions can be a real number or a set of real numbers). For example, the range of $f$ is defined as $f(X)$.
Thus, your first statement could be written $F\left([0,2]\right)=[0,4]$, your second, $g\left(]0,1]\right)=]-\infty,0]$.
Comment: Are you sure that *range of $f$* is $f(X)$?
Comment: @Sigur: The set $Y$ is the *codomain* of $f$. Although some people refer to the codomain as the *range*, I think it is generally more understood that the range, a subset of the codomain, is the image of the domain under the function, or $f(X)$. See the section on *image and pre-image* in the [Wikipedia article on functions](https://en.wikipedia.org/wiki/Function_(mathematics)).
Comment: It is written there: *The range of a function is the set of the images of all elements in the domain. However, range is sometimes used as a synonym of codomain, generally in old textbooks.* Since I like and usually study old books, so... :)
Comment: Possibly we should avoid the word *range* altogether; with *domain*, *codomain* and *image*, it's redundant whichever convention we use...


Passing an ID from startActivityForResult to onActivityResult

Question: I have a ListView with a ContextMenu entry, that starts the users photo gallery with startActivityForResult so he can pick an image to associate it with the ListItem. Picking the image works pretty well, and i get the image uri from the Intent passed to onActivityResult.
The problem is, i don't get the ListItem's id in onActivityResult.
I tried to attach it to the intent that i pass to startActivityForResult via putExtra, but the intent that i receive in onActivityResult is obviously not the same anymore.
Is there some way to pass this information to onActivityResult? Here's my code:
<code>private void pickPhoto(long itemId) {
    Intent i = new Intent(Intent.ACTION_PICK, MediaStore.Images.Media.EXTERNAL_CONTENT_URI);
    i.putExtra(ITEM_ID, itemId);
    startActivityForResult(i, PICK_PHOTO_REQUEST);
}

protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    switch (requestCode) {
    case PICK_PHOTO_REQUEST:

        long itemId = data.getLongExtra(ITEM_ID, 0);
        long imageId = ContentUris.parseId(data.getData());

        // That here says itemId is 0:
        Log.d(TAG, "adding image " + imageId + " to item " + itemId);

        setPhoto(itemId, imageId);
        break;

    default:
        super.onActivityResult(requestCode, resultCode, data);
    }
}
</code>
Thanxalot!
Comment: Did you check what is the itemId that is being set to the Intent extra?
Comment: Yes, now I did. It is not 0. So everything seems to be correct here.
Comment: Did you try using intent.hasExtra(ITEM_ID);
Comment: Yes, it's false. It seems that this is really just a newly created intent to return the chosen images uri, not the one that started the activity.
Answer: Ok, I stored the id in a field variable now and that works. Though I'm not so happy about this solution, because it would fail if the user starts two requests in parallel. More sort of a hack.


Why do people with perfect pitch perceive tunes not in 440 Hz out of tune?

Question: I do not have perfect pitch, but I heard that people who have it might have difficulty when a piece of music is not tuned to the usual 440 Hz pitch standard; it sounds out of tune for them.
But this makes no sense to me. As long everything is in the correct relation to each other why should it sound out of tune? It certainly sounds different, and someone with absolute pitch must easily tell the difference, but why might it sound "wrong"?
This is also very confusing with the notion that perfect pitch is a trait people just have, but the standard pitch 440 Hz is in essence just an arbitrary definition. So why should this ability be dependent on some arbitrary chosen standard pitch?
Let's make an analogy, most people can see and differentiate colours. Now suppose people define, very accurately on a physical level, what a standard red, green and blue are (like the 440 Hz standard tone). And from then on every monitor or TV uses just these standard R,G,B colours to combine other colors. Apart from contrast and brightness, the colours on every monitor or TV should then look the same. But then if someone manufactures a monitor (i.e. tunes not to 440 Hz) the colours would look somehow shifted, but does this imply that the colors look "wrong", or does people then have difficulty using this monitor? I guess not... they just might note that it looks different, but that's all.
Comment: As stated, your question merely embodies a tautology. Not being cursed with perfect pitch I am inclined to dismiss the premiss of your question as basess: such people might experience the total pitch of the performace as out of tune, but I see no reason why it should be perceived as out of tune within itself. Unclear what you're actually asking here.
Comment: To a person with perfect pitch, the phrase "out of tune" is ambiguous.  It can either mean "out of tune with itself - so it sounds bad" or it can mean "played at a pitch that isn't close to A=440 (plus or minus a whole number of semitones) - so it sounds musically nice but vaguely irritating".  Exactly how irritating the latter type of "out of tune" is depends very much on the listener.
Comment: Perfect pitch is *defined* by this - it's the ability to accurately judge the frequency of a tone (or sound). Why would you be surprised that people who can judge absolute frequency accurately would notice that the frequency is different than it's supposed to be? That's the whole point of perfect pitch :) Your TV analogy is not very good, since colours change all the time depending on lighting, so our eyes and brains are very capable at maintaining "correct" colours (the good old "yellow/blue?! dress" is an extreme example, as are many colour/contrast based optical illusions).
Comment: I know someone who has perfect pitch in the sense that they hear the musical notes (i.e. not Hertz directly), but unconsciously switches back and forth between A=440Hz and A=415Hz depending on what they used more, lately. There is nothing special about 440Hz. (should this be an answer?)
Comment: Your analogy actually works perfectly. The point is that many people _would_ notice that the new monitor was slightly off. In the real world, monitors are all slightly different and people who care about colour rendition buy a device to calibrate their monitors to produce specific absolute colours, just like people tune musical instruments to produce specific absolute pitches. If your monitor is miscalibrated, colours will look slightly wrong and, for example, colours on the screen won't match colours on a print. If you're a publisher or photographer, that monitor would indeed be hard to use.
Comment: Best explanation I've ever heard: https://youtu.be/AxEmaBqjUmY?t=270
Answer: As someone with absolute pitch and trained in A440 12-tone equal temperament (i.e. the usual) with plenty of vocal music as a backup, I perceive notes that are several cents out of tune as "off" because I am not used to them. I have difficulty listening to music containing those out-of-tune notes unless the music is atonal. (Granted, I don't like listening to atonal music for the most part, probably because of my absolute pitch, but I did find out that I didn't think of quarter-tones as unpleasant in an atonal piece of music I listened to once.)
My experience with this:

as long everything is in the correct relation to each other why should it sound out of tune?

If I find that the out-of-tune notes are in tune with each other, I figure that the music or instrument is out-of-tune and may be able to accept the music more. (I'm interested in hearing what B quarter sharp major sounds like, for example, but not in the context of a C major piece.) If I find that only some notes are out of tune, I figure that the performance is off-key or the instrument was not tuned properly.
I'll have to note that I will not detect music as out-of-tune if it's not several cents off (unless the in-tune note is played at the same time--in that case, everyone can hear the beats that get produced). This probably does make listening to vocal music easier, as people probably go for Pythagorean tuning instead of 12-tone equal temperament within a specific key because Pythagorean tuning uses perfect ratios (e.g. 3:2 for perfect fifths) and 12-tone equal temperament doesn't unless they're octaves.
As for why blatantly non-A440-12-tone-equal-temperament pieces sound wrong to me, I believe this is because of my musical training. A432 sounds slightly off to me, but I suspect that if I were trained in A432, it would be A440 that sounds off instead.
To use your colour analogy, the monitor with the not-perfectly-RGB pixels is just like the out-of-tune musical instrument--the colours look different, the web-safe colours don't display properly, and this time, everyone notices. The extreme example is the dead pixel where one of them is black instead of RGB--people start throwing out their monitors or getting them repaired at this point. (You are right--"people define, very accurately on a physical level, what a standard red, green and blue are"!)
(Caveat: I developed absolute pitch starting in Grade 8 (age 13)--I started with middle C as a reference, needing to use classical music for key references, and writing sorely thin transcriptions by ear. I've read before that absolute pitch developed this way may not be true absolute pitch, and needs constant maintenance or else it will be forgotten. On the other hand, I can instantly recognize soloing on the dominant of minor keys (an unorthodox mode), can recognize modes increasingly well, and can figure out when polytonality (2 or more keys being used at the same time) occurs...)
Comment: Just out of interest, have you ever listened to, say, baroque music played on A415 instruments? Does it sound "off" to you? Would it make a difference if you'd previously heard the same piece played with A440?
Comment: Aww, no one ever called me 'special' before :-)
Comment: The difference between A=440 and A=415 is 1.0127 semitones.  Your tuning would have to be super-accurate and your sense of pitch to be amazingly sensitive for this to be distinguishable from 1 semitone.  I've never encountered anyone whose "perfect pitch" can detect a difference of 0.0127 semitones.  To me, anything played at A=415 just sounds exactly "one semitone too low", and I can't imagine that anyone else with perfect pitch would hear anything different.
Comment: I don't have perfect pitch.  But a friend of mine who does, who often plays at A415, says that after a couple of days she hears 415 as being A, not G#.  I don't have perfect pitch, but I can sing an A440 pretty accurately if I think about it, and the same thing happens to me.  But I suspect that everyone's experience is different.
Comment: I can't even tell if the current note is higher or lower than the previous one.
Comment: @Bob, I find A415 to actually sound like Gb, so yes, it sounds very "off" to me. It does not make a difference whether I've ever heard the piece before: I generally have to swallow the entire piece being a semitone flat from modern convention in that case.
Comment: @Strawberry, if that truly is the case, you might be a special, serious kind of tone-deaf. Can you tell which drum beats are higher than others in a drum cadence? (Note that I mention drum cadences because they are entirely atonal, so even completely tone-deaf people can appreciate them.)
Comment: @Bob: Sorry, I meant Ab, not Gb. Guess I can't proofread that well before 7:00 AM...
Comment: I had this when I started playing Bb trumpet after years of piano. It took me a while to get "used" to how the world sounds in Bb, and at the time I blamed my (not quite perfect but pretty good) perfect pitch.
Comment: @David Wallace, I do hear A415 as one semitone too low (I'll need to hear a piece in A415 to determine whether I think it's even more off than that--I haven't listened to A415 music lately). About the only time A415 doesn't slightly stress me out when listening to older music is when I am not given the name of the piece.
Answer: I don't think I agree with what you have been told.
Perfect pitch is the ability to memorize, and recall, pitches.  Now just because you can do it doesn't mean that you always actually do it.
If someone with perfect pitch hears a note and it is slightly below A440 what does that mean?  Is the note an out-of-tune A, an in-tune A for a baroque orchestra or a note that the composer has indicated needs to be slightly flat in a modern piece?  In one context it is "wrong" and in the other contexts it is not "wrong".
And yes perhaps 440hz is perhaps arbitrary but if you can memorize and recall pitches then 435hz, 430hz etc. would be no different would it?
Comment: This answer is technically correct, but if you spend most of your life around pitches tuned to this standard, and you have perfect recall of those pitches, then yes surely a sub-semitone offset is going to be noticeable, and potentially irritating. A relatively overly-yellow streetlight kind of winds me up, if I'm honest.
Comment: That's a bit misleading.  "Perfect pitch" specifically means an innate ability to identify tones the way all of us can identify colors.  Many musicians who don't have perfect pitch have trained themselves to recognize a few pitches based on a lot of repetition.
Comment: Thanks for your answer. Maybe someone with this ability might share his/her experience. What is said in the question, that people with perfect pitch hear non-440hz tunes out of tune I heard (from musicians without it) and read a few times, so it seems to be a widespread believe.
Answer: The 440Hz reference point is a convention, but it's very consistently used.  This means that if you can sense it, you will likely form your entire understanding of harmony and music with this included (e.g. people with perfect pitch often report different keys having a distinct "feel") - and why wouldn't you?  Humans learn by finding patterns, and so if you perceive all music to be using a fixed set of pitches, that's a pattern that your brain has no reason not to assimilate.
As an analogy: I can see colour, so my understanding of driving includes the colour of traffic lights.  If I drove into a different county and their traffic lights were slightly different then I would find it frustrating that "yellow" now meant "go ahead" and I would spend my entire time on edge.
I would expect this effect to be stronger for performers attempting to play/sing along to the music, because their muscle memory will be very strongly linked to their perception of the pitch.
As an analogy for the muscle memory: piano keys are a standard width.  It's an arbitrary width, but it's so consistent that a pianist could play on an unfamiliar piano with their eyes closed, using muscle memory to find the intervals.  If I sat down at a piano and the keys were 90% the width, I would find it unsettling and difficult to play.  Same with playing a violin that's too small, or a clarinet where the keys are in the wrong place.
Comment: I have come across quite a few keyboards with different key widths, sometimes toy ones admittedly, but some harpsichords and small keyboard instruments.  It was not as difficult to adapt as I expected.
Comment: I've come across them once or twice, and I found it weird.  With a bit of exposure I could get used to it, but until then I was very aware of the different key size.  If I were feeling judgemental, I might say it were the "wrong" width. :)
Comment: actually, some cultures (e.g. japan) don't distinguish as strongly between green and blue as the english speaking world. this is supposedly because they use the same word for both in everyday speech. in those cultures, green traffic lights are sometimes shown as blue and that's *not* considered strange. source: https://en.wikipedia.org/wiki/Blue%E2%80%93green_distinction_in_language#Japanese
Comment: Also an analogy with colours :) So its an interplay between perception and muscle memory, just came to my mind: could you be a good musician with bad hearing but having expectionally good muscle memory O_o
Comment: Reminded me of the commercial with blue traffic light for surprise factor.
Answer: An addendum, rather than replacement for the two existing answers...
I don't have perfect pitch, but I have good relative pitch.
I have two friends with 'perfect' pitch.
The thing is, 'perfect' isn't actually perfect. It's good, but it's not an absolute. I can recall evenings in the pub listening to old 45s which were either playing at the wrong speed or had been tweaked up a bit at mastering to tighten them up. My two friends could amuse themselves by shouting the chords at each other... a semitone out from each other. One was perceiving the closest pitch to be A, the other B♭.
They were both 'right' for a given value of 'right'.
On the other hand, both of them are perfectly willing to accept a vocal or instrumental passage which to me clearly is suffering from tuning issues against the rest of the track.
10 cents out & I find it painful, neither of them are bothered in the slightest.
Comment: Almost all of my friends are musicians, yes ;) It used to be quite common to simply speed up the tape a bit when cutting a record on vinyl. It gives the track a little more punch & definition.... & of course, makes it play slightly sharp. There's always the added complication that people would just tune to whoever's guitar seemed to be closest in tune with itself & work from there, unless there was a piano in the studio.
Comment: Are your friends musicians? And what you mean by "tweaked up a bit [...] to tighten them up", sorry I am no english native speaker. Did that mean the pitch of the song was changed (and similar by playing it fast the pitch shifted), do you mean that?
Comment: @Tetsujin - and there was always the chance that the studio piano wasn't at concert...
Comment: @StefanH in my usage, "sharp" != "out of tune" in this context. If I tune my guitar a quarter-tone sharp and then everyone in the band tunes to that, the band is not out of tune, but it's playing a little sharp. On the other hand, if I go to sing a note and what comes out is a quarter tone higher than what I'd planned on, that would be out of tune. (even if it happens to wind up at a pitch of 440 Hz, and even if it was notated as an A)
Comment: "play [slightly] sharp" means "slightly out of tune"?
Comment: @Tim - Indeed. It used to be predominantly guesswork;) @ StefanH yes, the whole song would be faster & therefore higher pitch than it was recorded at.
Comment: This is one of the reason I enjoy vinyl DJ sets.  Not because of the audio quality, but because someone is manually beat matching which means the pitch is going to be off from what is "normal" .
Answer: Some people with perfect pitch may be irritated by music not at A=440 Hz. They're going to be irritated a lot when hearing modern recorded commercial music, and some not-so-modern stuff too!
Others can recognise a variance from A=440 Hz, but they aren't obsessed with it. Like looking at a photo or movie with a slight colour cast. At first sight it's noted, but it doesn't stop them enjoying the content.
As already stated, many people confuse perfect pitch with well-developed relative pitch. This is a skill that every musician requires. It can be learned and developed.
Answer: Not everyone perceives things the same.  In particular, not everyone perceives the relative pitches of a song in a vacuum, only compared to each other.
The human ear is a fascinating device.  Utterly fascinating.  I found out recently that the most favored theories to date is that the cochlea is actually an active amplifier of sound.  It has cells that appear to not just resonate at certain frequencies, but to actually actively provide energy into the fluid of the inner ear to make the sounds louder.  We do active EQ on the frequencies we hear with a startling Q factor.  Beautiful device.
If one's ear has been tuned for A440, those external effects like the effects of the active cells in our ears matter.  This gives them a reason to perceive a A443 as "off."  Some people tune their ear in this way, other's don't.  Neither is right nor wrong.
You give the example of a TV where the colors are all off, but off in a consistent way.  You may think that nobody should care, but a graphic artist who deals with white balance as part of their daily life may pick up the color issues immediately.  Why?  They're paid to be able to look at a piece of work and, in their heads, predict what it will look like in many different lighting conditions.  Their job causes them to care.
I can give an example in my life where perfect pitch lead me to consider an alternate pitch as "wrong," though in my case it was a half step, not a few cents.  I was in highschool choir and we were singing a song that was pushing the limits of the high ends of our voices.  To reduce strain, our teacher chose to teach the song a half step down.  Right before our performance, she tried to raise it back up to the correct pitch.  Unfortunately it didn't work so well.  There was one point in the song where there was a large interval up to a high note.  Every time we tried the song, we dropped back down half a step right there.
What had happened was the strongest singers in the choir all had perfect pitch, and instead of memorizing the interval for that jump, we had basically memorized the muscle positions for our own vocal chords which nailed that note.  Our own vocal chords became an anchor for the old pitch.  In the end, we performed the song a half step down because the leads simply couldn't make the transition to the correct pitch fast enough.
Comment: Your story: I'd have thought that the better singers would use 'relative pitch' to reach notes in a well-rehearsed song. Surprised they landed a semitone low - maybe they couldn't reach anyway. I often changed keys around when running choirs, but don't think I had the curse of any (or many) absolute pitchers. Always remember being at a school where the music teacher was explaining how to reach really high notes for the choir. Didn't have the heart to suggest dropping the key a tone...
Comment: @Tim it was fascinating, it really was.  It was quite the interval (although I forget the exact number), and right up near the vocal limits of a few of us, especially myself!  And while I don't know how common perfect pitch is, it just so happened that the lead soprano, alto, tenor, and bass all had it.  I think we could have learned to nail it if we'd given ourself a few more days to shake free of the pitch and learn the interval instead, but it was too late!
Answer: I kinda have perfect pitch. Your question is difficult to answer, because there are variations. Like, if a door shrieks, I can't immediately say 'That was a G with a slight glissando to a D at the end'.
However, if I hear a piece of music, or 'musical sounds' in general, I just hear "well that's a G" without having to think about it.
I don't have any problems with relative pitch though. I feel something is off, but certainly not overwhelming. I can for example thruly enjoy Jimi hendrix jamming along with an out of tune guitar, because musically it makes sense. If I were to write it down though, it would take me a while to adapt to it and transcribing would certainly not be effortless.
I do however go crazy when I hear 'synthetic' notes, like singing with autotune or slighty out of tune square waves (think ring modulators and the like). Too perfect really upsets my musical hearing.
Answer: A singer friend who has perfect pitch told me years ago that a typical problem is that when he is given written music to sing along some instrument and the instrument is tuned off 440, he has to transpose the music to be able to sing it, so the effort is a lot greater for him.
Comment: This is what I find. I like to be told "It's printed in E. We're singing it in F," although I can work that out. But *listening* to music in the wrong key is a lot easier.
Answer: People with perfect pitch are able to perceive absolute pitches much the way most people perceive differences in pitch. So non-standard absolute pitches can affect them the same way non-standard differences in pitch affect you.
Answer: <code>Perfect pitch</code> should really be called <code>learned pitch</code> because that's all it is: learning which pitch associates with a given symbol. I would equate it to recognizing colors, most people can name the general color they see (green) but painters would be able to name the more exact color (emerald). Similarly in music, most people can tell when a note is high or low and musicians that have spent time associating those notes with symbols are able to name the symbol.
All this is just to say that 440 is what musicians have spent time learning so anything outside that would cause difficulty. If you spend all your life listening to western pop and try to listen to a completely different genre, it can be difficult to feel the groove of the music.
Comment: Re: "perfect" and "learned" pitch, the standard term used today among music theory circles is "absolute pitch" (or AP for short).
Answer: So many contrasting perceptions here!
One morning I woke with a tune in my head: the D-E-C-C-A radio ad jingle from 30 years before.  Checked with piano, I was on-pitch.  Decided to learn to sing that C of 'Decca' at will - and after about a year reached about 90 percent reliability.
It's similar now, 20 years on, following refreshed practice.  Not true absolute pitch perhaps, but as with other people here, I dislike music played shy of 440.  Intonation sounds wrong from phrase to phrase, though the brain does settle in after a few minutes.  And then the timbre of ancient instruments is really enjoyed.
Similarly, when I was involved in broadcast, a turntable off-speed was immediately apparent unless exactly a semitone high or low.  Handed a 78 player recently, I first sang a record I recognised from one I owned aged ten and had not since heard.  My voice exactly on-pitch, turntable clearly slow!
But as for real singing?  I wish! Single notes are accurate, but strung together they lurch here and there by a semitone.   Never mind, anomalies aside, I do love music.  Wish it were central to every school curriculum from the outset - never mind 440.
Comment: So, the "decca" tune was in 440, which gauges your perception for the rest of your life?
Comment: What an interesting point, Stefan.  I heard DECCA every evening for a few years, aged around 12-15.  But always there was radio music at home, from birth, so that might have established a 440 tradition earlier still.


copy one role permission to another role

Question: I have come across a scenario where I need to assign same set of permissions to multiple roles (Note: Roles are of same type like Regular-Regular etc).
For example: I have created three regular roles "test_role_A", "test_role_B" and "test_role_C". These three roles would have same permissions.
So I am wondering If I can assign test_role_A role permissions to test_role_B and test_role_C at one go.
Is there any configuration in Liferay to do so?
Answer: After some research I observed that RoleLocalServiceUtil has methods like reassignPermissions, mergePermissions which are not fully satisfying my requirement.
However, I found that

ResourcePermissionLocalServiceUtil.addResourcePermissions(resourceName,
  roleName, scope, resourceActionBitwiseValue)

would help my cause.
So here is the beanshell script which will help you copy one role permission to another role of same type.
import com.liferay.portal.util.PortalUtil;
import com.liferay.portal.service.ResourcePermissionLocalServiceUtil;
import com.liferay.portal.model.ResourcePermission;
import java.util.List;
import com.liferay.portal.service.RoleLocalServiceUtil;
import com.liferay.portal.model.Role;

long companyId = PortalUtil.getCompanyId(actionRequest);
Role fromRole = RoleLocalServiceUtil.getRole(companyId, "<fromRoleName>");
List resourcePermissionList = ResourcePermissionLocalServiceUtil.getRoleResourcePermissions(fromRole.getRoleId());
//array of role names to which permissions needs to be copied
String [] copyToRoles = new String [] {"<ToRoleName1>", "<ToRoleName2>"};
for(String copyToRoleStr: copyToRoles){
    try{
        Role copyToRole = RoleLocalServiceUtil.getRole(companyId, copyToRoleStr);
        try{
            for(int i=0;i< resourcePermissionList.size();i++){
                ResourcePermission rp = resourcePermissionList.get(i);
                ResourcePermissionLocalServiceUtil.addResourcePermissions(rp.getName(), copyToRole.getName(), rp.getScope(), rp.getActionIds());
            }
            out.println("Successfully Assigned permissions of "+fromRole.getName()+" to "+copyToRole.getName());
        }catch(Exception e){
            out.println("Error occured while adding resource permission against role - "+copyToRoleStr+" : "+e.getMessage());
        }
    }catch(Exception e){
        out.println("Error occured while fetching role - "+copyToRoleStr+" : "+e.getMessage());
    }
}


Find a basis for a subspace.

Question: How can I solve for a basis for this subspace $W$?
$W = \{ \sum_{i=0}^n a_ix^i \in F[x] | \sum_{i=0}^k a_i =0 \text{ and } \sum_{i=k+1}^n a_i =0  \}$, where $n$ is an integer such that $n=2k \geq 4$. $F$ is a field.
Thanks. :D
Comment: ops sorry. that was a mistake. but i edited it already :D @egreg
Answer: Consider the map $f\colon V\to F^2$ defined by
$$
f(a_0+a_1x+\dots+a_nx^n)=
\begin{bmatrix}
\sum\limits_{0\le i\le k}a_i
\\[5px]
\sum\limits_{k+1\le i\le n}a_i
\end{bmatrix}
$$
where $V$ is the vector subspace of $F[x]$ consisting of polynomials of degree at most $n$.
Then it should be clear that $f$ is linear and surjective; moreover $W=\ker f$, so
$$
\dim W=(n+1)-2=n-1
$$
What about a basis? If a polynomial belongs to $W$, then $a_k$ is determined by $a_0,\dots,a_{k-1}$ and $a_n$ is determined by $a_{k+1},\dots, a_{n-1}$.
The matrix of the linear map with respect to the basis $\{1,x,\dots,x^n\}$ is
$$
\left[\begin{array}{cccc|cccc}
1 & 1 & \dots & 1 & 0 & 0 & \dots & 0 \\
0 & 0 & \dots & 0 & 1 & 1 & \dots & 1
\end{array}\right]
$$
where the first block is $2\times(k+1)$ and the second block is $2\times(n-k)$.
Thus the equations are
$$
\begin{cases}
a_0=-a_1-a_2-\dots-a_k\\
a_{k+1}=-a_{k+2}-\dots-a_n
\end{cases}
$$
so a basis of $W=\ker f$ is given by
$$
\{1-x,1-x^2,\dots,1-x^k,x^{k+1}-x^{k+2},\dots,x^{k+1}-x^n\}
$$
Comment: the basis is in terms of $a_i's$? not polynomial?
Comment: @mogs What are you saying? Of course the basis should be a set of polynomials.

Find a basis for a subspace.