Citing articles as references for the Leave-One-Out Method

Question: I wish to know which work(s) should be cited as reference(s) for the Leave-One-Out method.
Comment: The Elements of Statistical Learning by Hastie, Tibshirani and Friedman is a well known and standard reference.  It covers many aspects of the cross validation method, including leave one out, in detail in chapter 7.
Comment: You mean leave one out cross validation?
Comment: @MatthewDrury , Yes.
Comment: @MatthewDrury, why not make that an official answer? Otherwise, this thread will be counted as unanswered.
Comment: @gung Sure, will do!
Answer: The Elements of Statistical Learning by Hastie, Tibshirani and Friedman is a well known and standard reference. It covers many aspects of the cross validation method, including leave one out, in detail, in chapter 7.


SVN: Where is my repository?

Question: I'm trying to backup my SVN respository.
My problem: I forgot where I put my repository on the hard disk.
How can I find this?
FYI: I'm using TortoiseSVN if that helps.
Answer: Easy answer is to run:
find / -name 'svnserve.conf'

svnserve.conf is a file that should reside in ${REPO}/conf.
Hard(er) answer is to find out if svn has a command to find out what URL you used for you initial checkout.
Edit: for a command line svn client, the way to find the repository root (the place you checked your code out from) would be running this inside your working copy:
svn info

The line in the output that reads 'Repository Root' shows the place you checked out from and thus to the place where you repo is.
Take some time to get familiar with svn on the command line here.
Comment: @David: no it doesn't actually. It works just fine with the repo I checked out through file:// here. It does, however, assume we are using Linux, I realized later :-)
Comment: This assumes you're using svn://.
Answer: You have several different ways of accessing a subversion repository and finding the repository is different for each:
file://
This is the easiest. This means it's on the local machine where the path says it is.
svn://
This is using svnserve. Look for the process command line on the remote server and the path specified with the -r option is prepended to the path in the url to find the repository. If there is no -r option then the path in the url is the file system path.
svn+ssh://
This is svn over ssh. The path in the url is the filesystem location on the remote server.
http:// or https://
This is svn over http or https. You need to find the virtual host in your apache configuration and look for a <Location> section that matches your url path and look for a SVNPath or SVNParentPath config line. This will tell you the location of your subversion repository.
Comment: I had installed svnserve as a service, and so was able to look in there for the command, in case that helps anyone.
Comment: Another option is to try to check if there's a VisualSVN Server installed on the server machine which hosts the repository.
Answer: In Tortoise, choose the properties option. The top line will list the file path of the repository assuming it is direct file access.
Answer: From your top-level SVN checkout folder right click and select Tortoise SVN | Repo-browser.  The URL to your repository will be in the box at the top of the window.
Answer: $ find -iname .svn

it should find all working copies on host

SVN: Where is my repository?


what happens to memory when reassign array in C#?

Question: In the following code, what happens to the memory myArray initially pointed to once it's reassigned in the 2nd line?  Is that memory lost, or does the C# garbage collector take care of it?
static void Main(string[] args)
{
   double[] myArray = new Double[10];
   myArray = new Double[3];
}

As far as I've been reading, there's no way to explicitly free up the memory, so I'm hoping C# automatically does it.
Comment: _garbage collector will take care of it_
Comment: Data still remains in memory for a while. Because you don't have any pointer to that data anymore, GC will collect that space when necessary.
Answer: Of course, C# automatically releases memory that was associated with myArray prior to the assignment. This does not happen right away, but as soon as garbage collector realizes there are no remaining references to that Double[10] array object*, the memory allocated to the object gets reclaimed.
If you change your program slightly to create a second reference to the same array, like this
double[] myArray = new Double[10];
double[] myArray2 = myArray; // Second reference
myArray = new Double[3];

garbage collector would not release the object, as long as a reference to it remains accessible to your program.
* Sometimes your program finishes execution before garbage collector gets to complete its analysis. The memory still gets released, though.
Comment: _as soon as garbage collector realizes there are no remaining references_ .- depend on application size/using this can hapenned after application is closed
Comment: @Fabio That's fair. I added a footnote to mention this. Thanks!
Answer: When your variable goes out of scope and the memory is no longer required then it becomes eligible for garbage collection.
MS explains all, as seen here:  https://msdn.microsoft.com/en-us/library/aa691138(v=vs.71).aspx

C# employs automatic memory management, which frees developers from manually allocating and freeing the memory occupied by objects.
Automatic memory management policies are implemented by a garbage
collector. The memory management life cycle of an object is as
follows:

When the object is created, memory is allocated for it, the constructor is run, and the object is considered live.
If the object, or any part of it, cannot be accessed by any possible continuation of execution, other than the running of
destructors, the object is considered no longer in use, and it becomes
eligible for destruction. The C# compiler and the garbage collector
may choose to analyze code to determine which references to an object
may be used in the future. For instance, if a local variable that is
in scope is the only existing reference to an object, but that local
variable is never referred to in any possible continuation of
execution from the current execution point in the procedure, the
garbage collector may (but is not required to) treat the object as no
longer in use.
Once the object is eligible for destruction, at some unspecified later time the destructor (Section 10.12) (if any) for the
object is run. Unless overridden by explicit calls, the destructor for
the object is run once only.
Once the destructor for an object is run, if that object, or any part of it, cannot be accessed by any possible continuation of
execution, including the running of destructors, the object is
considered inaccessible and the object becomes eligible for
collection.
Finally, at some time after the object becomes eligible for collection, the garbage collector frees the memory associated with
that object.

If you want to go into more detail then you can look here:  https://learn.microsoft.com/en-us/dotnet/standard/garbage-collection/index
or I'm sure you can also find some blogs on the topic.


How to vectorize double summation in matlab

Question: I'm trying to produce vectorized code for this equation (and multiple others of the same form):

This will be evaluated in ode45 and the only part that changes is Rmn(t) so I am pre-computing the Bessel and sin functions. At the moment my code looks like this:
for iR = 1:nR
    p(:, iR) = sum(Rmn.*J0m(:, :, iR),1)*sinanz;
end

M, N are the number of terms in my summation and R, Z are the number of r and z coordinates I'm using. Rmn is an M*N matrix. J0m is a M*N*R array. It is an M*R matrix that is repeated N times. sinanz is an N*Z matrix. J0m and sinanz are pre-computed and don't change.
This works but is slow and so I am trying to optimize it. I figured the first step would be to reduce J0m so it is only m*R but I can't figure out how. I am looking for any suggestions on how to do this and any suggestions on how to optimize this in general.
Comment: Have you pre-allocated `p`? That is, put a `p = zeros(Z,nR);` before the loop?
Comment: Yes I have done that. Thanks also for fixing the image and `code`.
Answer: As you probably already know, you should pre-allocate p before the loop:
p = zeros(Z, nR);

this prevents the array p from growing at each iteration, speeding up the loop tremendously.
You could vectorize the whole thing by virtue of bsxfun:
% C ≣ M×N×R array of all products Rmn·J0m
C = bsxfun(@times, Rmn, J0m);

% D ≣ M×N×R×Z array of all products C·sinanz
D = bsxfun(@times, C, permute(sinanz, [3 1 4 2]));

% Sum in M and N directions, and re-order
p = permute(sum(sum(D,1),2), [4 3 1 2]);

but I doubt it's going to be faster; MATLAB (read: BLAS) is pretty fast with 2D matrices, but usually not terribly good with more-D arrays.
I suggest you read about bsxfun; it's also the way to reduce the J0m array to M×R in the way you describe.
Of course, you can get rid of the permutes by properly defining your variables, so let's do a small test in the 'ideal' versions of both the looped and vectorized code:
%% Initialize some variables

% Number of tests
TESTS = 1e4;

% Your dimensions
M = 5;   nR = 4;
N = 2;   Z  = 3;

% Some dummy data
Rmn    = rand(M,N);
sinanz = rand(N,Z);
J0m    = rand(M,nR); % NOTE: J0m doesn't have to be 3D anymore by using bsxfun

%% Test 1: your own version, optimized

% Start test
start = tic;

% Repeat the test a couple of times to get a good average
for ii = 1:TESTS
    p1 = zeros(Z,nR);
    for iR = 1:nR
        p1(:, iR) = sum( bsxfun(@times,Rmn,J0m(:, iR)), 1 )*sinanz;
    end
end
stop = toc(start);

% Average execution time
fprintf(1, 'Average time of looped version: %f seconds.\n', stop/TESTS);

%% Vectorized version, using 4D arrays:

% Don't make the permutes part of the test
J0m    = permute(J0m, [1 3 2]);
sinanz = permute(sinanz, [3 1 4 2]);

% Start test
start = tic;

% Repeat the test a couple of times to get a good average
for ii = 1:TESTS
    C  = bsxfun(@times, Rmn, J0m);
    D  = bsxfun(@times, C, sinanz);
    p2 = sum(sum(D,1),2);

end
stop = toc(start);

% Average execution time
fprintf(1, 'Average time of vectorized version: %f seconds.\n', stop/TESTS);

%% Check for equality

maxDifference = max(max(p1 - squeeze(p2)'))

Results:
Average time of looped version    : 0.000054 seconds.
Average time of vectorized version: 0.000023 seconds.
maxDifference =
    4.440892098500626e-016

which seems pretty good! However, using
M = 50;   nR = 40;
N = 20;   Z  = 30;

you get
Average time of looped version    : 0.000708 seconds.
Average time of vectorized version: 0.009835 seconds.
maxDifference =
    8.526512829121202e-014

so the vectorized version just got an order of magnitude slower than the looped variant.
Of course, your mileage may vary, but the take home message is that you should expect this difference to get worse and worse for increasing dimensions.
So, in conclusion:

if your dimensions are large, stick with the loop. If they are small, also use the loop - it's just SO much easier on the eyes than the vectorized version :)
make sure to pre-allocate the p variable
use bsxfun to reduce the memory footprint (but it won't increase the speed much)
Comment: Thanks for your help. I tried Test 1 and it takes about twice as long to run as my original version (stays roughly the same as the dimensions change), so it looks like I have the optimal solution for now. I was aiming to run the model with M=N=50 so the second one isn't going to help either. It's good to see the alternatives though.


drupal multisite

Question: I already have a Drupal 7 website. I want to extend it to a second one. The first one is stored in a subfolder. I want to create a second subfolder to store the second one. At the base I will have an intro
page which will redirect to either the 1st or the 2nd one. I do not care for different domain names. I just want different content (different menus) and slightly different template. Also same users same admin. But I would like the search to be site related.
What is the easiest way to do this? Should I create a 2nd databale or use the Domain Access module? If I create a 2nd database how can I have shared logged users? If I have the same database how will the admin
be able to update each one separately (assign nodes to sitesa) and should I override the search module to only query site-related content?
Answer: I don't think you need multisite for you needs. In fact it may work against you.
Cant you have
<code> / = intro page
 /sitea/ = menu and templates for site a
 /siteb/ = menu and templates for site b
</code>
Comment: Yes but I want users to be common between them. I don't want a user to have to register twice. This could mean same database or updating the users table in 2 databases. However it wouldn't make sense for the search to return results from both sites. So what I want is common database for users but different for content maybe. Maybe there is a single sign in solution?
Answer: There are a number of different ways to setup a drupal multisite.
Domain Access works well as its one code base, one database and one user base. Content can also be shared easily between each of the sites.
Domain Access is only one of the easiest to deploy and create any number of sub sites. Any node can be associated with as many sites as you have and node edits apply across all domains. You can also set different themes for each site to give them a different look and feel.
However in your case as previous poster pointed out it might be an overkill.
I would suggest you use Context module (https://drupal.org/project/context) to have ability to load different content and menu's. This would give you different "landing pages" while keeping single instance of Drupal and single user base.
Answer: Try the Single Sign-On module. I am not sure about the name, but it shares user table across Multiple sites.

Single Sign-On (SSO) means that if you are logged onto the master
  site, you will automatically be logged into any slave site that you
  visit.

drupal multisite


Scale log-likelihood as MCMC sampler advances, to improve acceptance rate

Question: I am working with a rather noisy and multi-modal likelihood. I've found that in order to obtain reasonable results from my Bayesian MCMC sampler (emcee, an affine invariant MCMC ensemble sampler), I need to scale the log-likelihood "on the fly" to ensure the acceptance rate is within the 0.25-0.5 range.
The steps in pseudo-code are:
# Initial log-likelihood scale factor
lkl_scale = 1.

# Run the MCMC N times
for i in 1..N

    # One sample from the MCMC sampler, passing the log-likelihood
    # and log-prior functions, and the log-likelihood scale factor
    sample = mcmc_sampler(log_lkl, log_prior, lkl_scale)

    # Obtain the acceptance rate for this step using some function
    accpt_rate = accpt_rate_func()

   if accpt_rate < 0.25
       # If the acceptance rate is too low, reduce the scale factor
       lkl_scale = lkl_scale * 0.5
   else if accpt_rate > 0.5
       # If it is too large, increase it
       lkl_scale = lkl_scale * 2.

The line sample = mcmc_sampler(log_lkl, log_prior, lkl_scale) basically returns the sampled $\theta_i$ parameters and the associated lkl_scale * log_lkl + log_prior values (for each chain) for this step.
The lkl_scale factor scales the log-likelihood simply as lkl_scale * log_lkl. This way if lkl_scale is small, it will flatten the log-likelihood which results in larger acceptance rate values, and vice-versa. Notice I do not modify the priors ever.
I've been advised in a previous question that this approach is correct, and that it is similar to annealing or parallel tempering. But this method as far as I understand is based on "swapping between multiple Markov chains run in parallel at different temperatures to accelerate sampling" (Gupta et al. 2018) which I am not doing.
In another question it is stated that if you "flatten" the posterior, then you are effectively sampling from a different posterior and your samples need to be weighted to make sense. Since I am only modifying the log-likelihood, I'm not sure this applies to my case.
My question is then: is this approach statistically reasonable? If not, then: what if I were to scale my log-likelihood just once before launching the MCMC sampler (using a value that I know produces good results) Would it be reasonable then?
Answer: In short, this method is not statistically valid without further steps.
The function
sample = mcmc_sampler(log_lkl, log_prior, lkl_scale)

should be detailed because, as presented, the method does not appear to be justified. Adding a constant to the likelihood modifies the probability model MCMC simulates from the posterior distribution to a geometric mixture of the prior and posterior distributions. The target of the MCMC is indeed the mixture
$$\pi_c(\theta|x)\propto \pi(\theta)f(x|\theta)^c$$This means that the output has to be reweighted by $f(x|\theta)^{1-c}$ or subsampled by accepting only with probability proportional to $$f(x|\theta)^{1-c}$$ Or by using parallel tempering as in Neal (1999).
Furthermore, the constant change of the weight $c$ according to the acceptance rate implies that the target is evolving along time, meaning the Markov chain is no longer time homogeneous, hence that the convergence of the MCMC sampler is in jeopardy. To be validated the algorithm must see the rate of update of the constant c decrease with the iteration index (condition of diminishing adaptation).
Comment: I've edited the question to add this information Xi'an. Please let me know if I should be more specific. I've also added a question regarding a simpler alternative approach (scaling the log-likelihood just once).
Comment: Thank you very much for the updated and detailed answer Xi'an. The weird thing is that the method *appears to work*, which makes it even more deceptive.
Comment: What do you exactly mean by "the method appears to work"? I would be highly surprised were it to reproduce the posterior density.
Comment: I mean that (using a uniform prior) it gives back reasonable values for my model parameters (ie: what I would expect physically) I don't know the shape of the posterior, so I can't really check anything other than the "reasonability" of the means of the parameters' distributions.


Making the area of the quadrilateral and the area of a triangle the same

Question: $ABCD$ is a quadrilateral and $X$ is a given point on AD. Find a point Y in AB such that the area of the $\triangle AXY$ is equal to that of $ABCD$. Hence show how to divide the quadrilateral $ABCD$ into three equal parts by straight lines drawn through $X$.

I cannot for the life of me solve the first problem. No matter what I try, what parallel lines I draw, I cannot make a triangle which has the same area of the triangle in this specific configuration. If the point $Y$ was opposite to point $X$, i.e, if it was on $BC$, then I can solve. But I just am unable to transfer this intuition to the case with $AB$. And also I fail to see how the first part is related to the 2nd part of the question. Could anyone explain it to me?

This is how I solved the 2nd part of the problem.
I joined $BX$ and $CX$. $AE$ and $DF$ are drawn parallel to $BX$ and $CX$ respectively. Then I join $XE$ and $XF$. The triangle $XEF$ has the same area as the quadrilateral $ABCD$. And we divide $EF$ in the ratio $\frac{1}{3}$. This point $P$ divides the quadrilateral into two regions having area ratio $1:2$. Making another line is essentially the same process.
.
Comment: Y can not be between A an B. Y can be on an extension to AB. There is no solution even if Y on BC.
Comment: What are the three equal area?
Comment: @Moti Can you tell why there will be no solution even if $Y$ is on the extension on $BC$?
Comment: If on extension it will work.
Comment: In your figure $ABCD$ is a rectangle, but the text is about a generic quadrilateral. Which is right?
Comment: @Aretino generic quadrilateral
Answer: Even if not explicitly stated, I'll assume quadrilateral $ABCD$ is convex.
Draw from $C$ the parallel $CC'$ to $BD$, intersecting $AB$ at $C'$. Triangle $ADC'$ has the same area as $ABCD$ (because triangles $BDC$ and $BDC'$ have equal altitudes with respect to common base $BD$).
Draw from $D$ the parallel $DY$ to $CX$, intersecting $AB$ at $Y$. Triangle $AXY$ has the same area as $ADC'$ (because triangles $YXC'$ and $DXC'$ have equal altitudes with respect to common base $XC'$).
Hence $AXY$ has the same area as $ABCD$.

EDIT.
To divide quadrilateral $ABCD$ into three polygons of equal area with three lines drawn through $X$ one can then proceed as follows.
Draw point $P$ on $AY$ such that $AP={1\over3}AY$: the area of triangle $APX$ is then ${1\over3}$ the area of $ABCD$ and if $P$ lies on side $AB$ then $APX$ the first desired first polygon.
If $P$ is outside $AB$ (as in figure below) then $PX$ cuts $BC$ at some point $E$, but the area of $ABEX$ is less than the desired area, by an amount equal to the area of triangle $BPE$. But we can easily construct a triangle $QEX$ with the same area as $BPE$: just construct on $EC$ point $Q$ such that
$$
QE:BE=PE:XE.
$$
Quadrilateral $ABQX$ is then the desired first polygon.
To construct a second polygon we can repeat everything on line $DC$: first of all construct $Y'$ on line $DC$ such that triangle $DXY'$ has the same area as $ABCD$, then divide $DY'$ into three parts, to find $P'$ (and $Q'$ if needed).
The third polygon is what is left of $ABCD$ by the other two.

Of course I assumed $Q$ to lie on side $BC$, but that might not be the case: in that event we should modify $ABQX$ to a pentagon, following the same path discussed above.
Comment: Thank you!!! But could you show the relation between this and dividing the quadrilateral into equal areas?
Comment: I don't see a relation for the general case. Are you sure the problem hinted at a generic quadrilateral?
Comment: @PopularPower I've found the source (Challenge and Thrill of Pre-College Mathematics, by Krishnamurthy and Pranesachar) and they indeed refer to a generic quadrilateral. I'll think about it.
Comment: Yes, there is a relation to a certain extent. You can use the triangle $AXY$ to obtain the desired division of the quadrilateral $ABCD$. This means that the division depends on the position of $X$. This division consists of one triangle and two quadrilaterals. I don't know whether that is acceptable to you.  I hope to post my answer few hours to from now. .
Comment: @PopularPower I've just edited my answer to address the second request.
Comment: @YNK The problem is we cannot be certain to get two triangles and a quadrilateral: see my edited answer. Taking into account all possibilities is quite cumbersome.
Comment: @Aretino but is my process correct? I also assumed that the point which divides $EF$ in the ratio 2:1 lies on BC. According to me, this will not apply to the genric quadrilateral.
Comment: @PopularPower Yes your method successfully divides the given rectangle into three equal parts, a triangle and 2 quadrilaterals. Did you try to apply your method to a generic quadrilateral?
Comment: @Aretino I agree with you. I want you to read the last paragraph of my answer. My construction depends on the centroid $G$ of $AXY$. If this centroid lies outside the quadrilateral, we have to device a totally new construction procedure.$$ $$ I answered a question addressed to you by Popular Power. I hope you don't mind.
Comment: @PopularPower Your method is fine but works only for a rectangle.
Comment: @YNK I originally intended for it to be a generic quadrilateral. Sorry if it looks like a rectangle, it is not. And it seems to work for all the quadrilaterals I have tried on Geogebra. However I also chanced upon the condition that Aretino mentioned when I tried some weird looking quadrilateral. I do think my method also applies outside of a rectangle though.
Answer: $\underline{\mathrm{Part\space I}}:\space \mathrm{Determination\space of\space point\space }\mathbf\it{Y}$

Our construction has two steps. In the first step, a triangle is found, which has the area of the given quadrilateral and shares one of the sides with it. In the second step, the sought triangle is constructed, so that it has the area of the triangle found in the previous step. Please note that we decided to leave the proof of the construction up to OP.
$ABCD$ shown in $\mathrm{Fig.\space 1}$ is a quadrilateral, the four sides of which were chosen arbitrarily, and $X$ is any point lying on its side $AB$. First, a line is drawn to join the opposite vertices $A$ and $C$ of $ABCD$. Next, a line parallel to $AC$ is constructed through the vertex $D$ to intersect the extended side $BC$ at $E$. Now we have the triangle $ABE$ which has the area of the quadrilateral $ABCD$ and the two figures share the side $AB$. This is a pretty standard construction and can be found in any good textbooks on geometry.
$\mathrm{Fig.\space 2}$ illustrates how to obtain the desired triangle with the area of the triangle $ABE$ and has $AX$ as one of its side. To begin the construction, draw two perpendicular lines to $AB$ through $X$ and $B$. Next, a parallel line to $AB$ is drawn through $E$, one of the vertices of the triangle $ABE$, to intersect the perpendiculars through $X$ at $F$. After joining $A$ and $F$, extend it to meet the perpendiculars through $B$ at $H$. Finally, complete the construction by drawing a line parallel to $AB$ through $H$ to cut the extended side $AD$ of the quadrilateral $ABCD$ at $Y$. Since the triangle $AXY$ has the area of $ABE$, both $ABCD$ and $AXY$ have equal areas.
We have noticed that, as $X$ moves from $A$ to $B$, $Y$ travels from $\infty$ to $Z$ (see $\mathrm{Fig.\space 3}$).
$$ $$
$\underline{\mathrm{Part\space II}}:\space \mathrm{Partition\space of\space a\space quadrilateral\space into\space three\space equal\space parts\space}$

In this 3-step-construction described below, we try to show how the first part is related to the division of the quadrilateral. This, we do by exclusive use of the triangle $AXY$.
First, we construct the point of concurrence of the medians $G$ known as the centroid of $AXY$. As shown in $\mathrm{Fig.\space 4}$, three sub-triangles $AXG$, $XYG$, and $YAG$, which are formed by the lines joining $G$ to the vertices of $AXY$, have the same area. Therefore, the area of each of the sought subdivisions of the quadrilateral is also equal to the area of a sub-triangle. We make use of this fact to obtain one of the subdivisions, namely $AXP$, by drawing a line parallel to $AB$ through $G$ to cut $AD$ at $P$. It is evident that the areas of the triangles $AXG$ and $AXP$ are the same, because they both have the same base and height.
Next, as shown in $\mathrm{Fig.\space 5}$, we draw a line parallel to $DG$ through $Y$ to cut $CD$ at $K$. To obtain the second and the third subdivisions, two lines are drawn to join $G$ to $K$ and $X$. The area of $PXG$ is equal to that of $PAG$ , because they both have the same base and height. In similar manner, the areas of the two triangles $DGK$ and $DGY$ are equal. Since the area $PGD$ is common to both the triangle $YAG$ and the pentagon $PXGKD$, their areas are equal. Therefore, the remaining pentagonal subdivision $XBCKG$ has the same area of either $AXP$ or $PXGKD$.
Only two of the three lines $PX$, $GX$, and $GK$ drawn to partition the quadrilateral goes through $X$. In order to rectify this problem, as shown in $\mathrm{Fig.\space 6}$, we need to continue the construction by drawing a line parallel to $KX$ through $G$ to cut $CD$ at $Q$. By doing this, we exchanged two equal areas $GXM$ (which was a part of the pentagon $XBCKG$) and $KQM$  (which was a part of the pentagon $PXGKD$) to obtain what we want.
It should be noted that there is no unique construction protocol capable of partitioning all quadrilateral to satisfy the given requirements. Therefore, we have to devise a construction as demanded by the case at hand.


Understanding logic behind Integer.highestOneBit() method implementation

Question: Java Integer class has the static method highestOneBit method which will return a value with a single one-bit, in the position of the highest-order one-bit in the specified value, or zero if the specified value is itself equal to zero.
For example input of int 17 will return 16; As 17 can be represented in binary as 10001 so it will return the furthest bit left which is equal to 16.
And in Integer class it has the following implementation in Java doc.
    public static int highestOneBit(int i) {
    // HD, Figure 3-1
    i |= (i >>  1);
    i |= (i >>  2);
    i |= (i >>  4);
    i |= (i >>  8);
    i |= (i >> 16);
    return i - (i >>> 1);
}

I just want to know the logic behind implementing it this way and the logic behind using shift operations
. Can any one put some light on it.
Answer: This algorithm calculates for a given i whose binary representation is:
0..01XXXXXXX...XXXX

the value
0..011111111...1111

That's what the 5 |= operators do.
Then, in the return statement, it subtracts from it that value shifted right by one bit
0..001111111...1111

to get the result
0..010000000...0000

How does it work:
The highest possible 1 bit the the 32nd (left most) bit. Suppose the input number has 1 in that bit:
1XXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX

You or that value with the value shifted right by 1 (i >>  1) and get
11XXXXXX XXXXXXXX XXXXXXXX XXXXXXXX

Then you or that new value with the value shifted right by 2 (i >>  2) and get
1111XXXX XXXXXXXX XXXXXXXX XXXXXXXX

Then you or that new value with the value shifted right by 4 (i >>  4) and get
11111111 XXXXXXXX XXXXXXXX XXXXXXXX

Then you or that new value with the value shifted right by 8 (i >>  8) and get
11111111 11111111 XXXXXXXX XXXXXXXX

Finally you or that new value with the value shifted right by 16 (i >>  16) and get
11111111 11111111 11111111 11111111

If the highest 1 bit is smaller than the 32nd bit, these operations still turn all the bits to the right of it to 1 and keep the remaining (higher bits) 0.
Comment: can you please link you answer with code and which portion is doing what .. ?
 i |= (i >>  1);   and this goes to 16 and in return statement   return i - (i >>> 1);  i am not being able to connect the dots.
Answer: The i |= statements help to compute a sequence of ones that is the same length as i. For example, for 101011 it computes 111111. I've explained how it works in this answer (I can't retype it right now since I am on mobile).
So basically, once you have the string of ones, subtracting itself shifted right one bit gives the H.O. bit.
111111 - (111111 >>> 1) = 111111 - 011111 = 100000

Answer: The first five lines (i |= (i >> x)) will set every bit below the highest 1-bit to 1. Then, the final line will subtract every 1-bit below the highest one, so that only the highest 1-bit will remain.
For simplicity, let's pretend an int was 8 bits. The code would in that case be like this:
public static int highestOneBit(int i) {
    i |= (i >>  1);
    i |= (i >>  2);
    i |= (i >>  4);
    return i - (i >>> 1);
}

Now, let's say we have the value 128 (10000000). This is what would happen:
// i == 10000000
i |= (i >>  1); // i = 10000000 | 11000000 = 11000000
i |= (i >>  2); // i = 11000000 | 11110000 = 11110000
i |= (i >>  4); // i = 11110000 | 11111111 = 11111111
return i - (i >>> 1); // 11111111 - 01111111 = 10000000

The >> is an arithmetic right shift, so it will preserve the signed bit.
The last >>> is a logical right shift, which will not preserve the signed bit. It will always insert zeroes on the left side.
Now, let's try with 64 (01000000):
// i == 01000000
i |= (i >>  1); // i = 01000000 | 00100000 = 01100000
i |= (i >>  2); // i = 01100000 | 00011000 = 01111000
i |= (i >>  4); // i = 01111000 | 00000111 = 01111111
return i - (i >>> 1); // 01111111 - 00111111 = 01000000


Linear Combination of multivariate t distribution

Question: I am looking for a resource where i can find derivation of the linear combination of multivariate t distribution. Does anyone here know any good site or place (s)he can point me to? I am trying to see if the linear combination of multivariate t distribution will give a multivariate t distribution. In other words, what is the distribution of the linear combination of two or more multivariate t distribution?
Does anyone here have any idea on this?
thanks.
Comment: Yes, the components are independent. Sorry about the algorithm tag.
Comment: One thing I should have asked you to clarify before. When you say 'linear combination of distribution' I have assumed you meant to take a linear combination of the *random variables* (where 'sum' is really a convolution of pdfs/probability functions in the independent case), not of the distribution functions (i.e. a mixture model).
Answer:
I am trying to see if the linear combination of multivariate t distribution will give a multivariate t distribution.

In general, no, this is not the case, even with univariate t's (see here and here for example; note that the difference of two t-random variables is the sum of two t-random variables, but with the second component having its mean that of the original random variable multiplied by -1)
In some very particular cases, yes. Consider:
(i) the limiting case of infinite degrees of freedom, linear combinations of multivariate normals are multivariate normal;
(ii) if the component t-variables are perfectly dependent their sums will be multivariate-t;
(iii) in the univariate case, sums of independent Cauchy random variables will be Cauchy. I haven't checked, but this may well extend more to subsets of the multivariate case than vectors of independent Cauchy (and the perfectly-dependent case mentioned above);
(iv) in the limit of very large numbers of components, where none of the components dominates variance-wise (that is, where the coefficient of each component times the variance of that component doesn't become too large), you may be able to invoke a version of the central limit theorem.

In the case where the weights on the components are equal (effectively converting it to a scaled sum) and you're dealing with standard t (rather than ones with general means and variances), this paper has some information. Extending it to the case of a general mean is straightforward but it doesn't deal with the general case of arbitrary scales, or equivalently arbitrary linear combinations.
Comment: You can defend that by counterexample (univariate with 2df for example, either by comparing characteristic functions of any claimed answer with the product of cfs, or even by direct comparison of the convolution integral with the claimed answer), but in any case in general the way to approach an undemonstrated general claim is to let the other person claim it is the case and then ask them to prove their claim (i.e. to say 'why should anyone think it is the case?' and then to show them their error).
Comment: I know that multivariate normals are normals when u add the independent components. How comes the linear combination of multivariate t are not multivariate t? Multivariate t are instances of the multivariate norma( i mean combinations).
Comment: What do you mean by the sentence "multivariate t are instances of the multivariate normal"?
Comment: Sorry, i meant to say, the multivariate t distribution is a family of continuous probability distributions with different degree of freedom. Say when df=1, you get Cauchy.So maybe i should have said with different df, you can get instances of the multivariate t distribution.
Comment: Yes, but now how does this relate to what you just asked me in comments a few minutes ago? The fact that you know a limiting case has a certain property doesn't imply that the general case shares that property. Your question there implies that you expect it should be the case. What is the basis for such an expectation?
Comment: I base it on the fact that the big picture(here multivariate t) will inherit the charactersitics of the instances( Say cauchy). this is just a feeling, it maybe be a total rusbish. So i just came here to seek advice from experts like you so that i can be enlighten. So if i have say two independent multivariate t distribution, when i add them up, will the results give me a multivariate t distribution as in the case of multivariate Gaussian distribution?
Comment: "*the fact that the big picture ... will inherit the characteristics of the instances*" -- this sounds like the [fallacy of composition](http://en.wikipedia.org/wiki/Fallacy_of_composition) ("what's true of the part is true of the whole"), but in any case it doesn't work like that. "*this is just a feeling*" isn't a sufficient basis on which to expect it to be so (results in probability don't generally accord with intuition), and in this case it isn't so - even in the super-simple case of independent, univariate t, with equal coefficients in the sum, and constant d.f., it doesn't hold.
Comment: thanks for the answer. Now that i know that it does not hold, i just want you to explain to me why it does not hold with some examples so that  can learn from it and be able to defend my statement anywhere incase i did mentioned it somewhere
Comment: Sorry, which specific statement do you need to defend?
Comment: That is the "linear combination of multivariate t distribution will not in general give a multivariate t distribution".
Answer: It is true that the components of a multivariate-$t$ vector and linear combination thereof are $t$-distributed. But linear combinations of arbitrary $t$-variables are not necessarily $t$-distributed. In fact, linear combinations of independent $t_\nu$ variables are not $t$-distributed.
The comment by Joram Soch starts with a correct result but then makes a subtle error by claiming that two independent t-distributed vectors have a joint t-distribution. To illustrate that this is not so, let $X$ and $Y$ be scalar random variables, independent, and $t_\nu$ distributed. Then their joint density is the product of their densities and thus is proportional to $((1+x^2/\nu)(1+y^2/\nu))^{-(\nu+1)/2} = (1+x^2/\nu+y^2/\nu + x^2y^2/\nu^2)^{-(\nu+1)/2}$. But the term in parentheses is not a quadratic form in $x$ and $y$ and therefore is not a multivariate $t$-density.
This example demonstrates also that the components of a multivariate-$t$ variable are always dependent.
Comment: This is more like a comment. But it is a good comment.
Comment: Thanks for this comment! You're right, so I've edited my answer.
Answer: Please have a look at
Walker, Glenn A., and John G. Saw. "The distribution of linear combinations of t-variables." Journal of the American Statistical Association 73.364 (1978): 876-878.
The resulting PDF is described as a weighted sum of student-t distribution, and the paper shows how to obtain the weight. The author started from observing for odd degrees of freedom the characteristic function of a student-t r.v. is expressible in closed form, i.e. proportional to the modified Bessel function of the third kind. It seems to me that the paper proposes the solution for all odd t-distribution cases, and no even degree-of-freedoms should be involved.
Comment: Welcome to the site. At present this is more of a comment than an answer. You could
expand it, perhaps by giving a summary of the information in the paper, or we can
convert it into a comment for you.
Answer: P15 of Multivariate T distribution and their application (Kotz and Nadarajah) says "
If X has the p-variate t distribution with degrees of freedom v, mean vector p, and correlation matrix R, then, for any nonsingular scalar matrix C and for any a, CX + a has the p-variate t distribution with degrees of freedom v, mean vector Cp+a, and correlation matrix CRC'. This result is of importance in applications and is similar to the corresponding result for the multivariate normal distribution.
"
edit:
In a univariate language, e.g., considering $aT_1+bT_2$ where $T_i$ is t distributed with mean $m_1,m_2$, scale (in same order as variance) $S_1,S_2$ and with the same degree of freedom. Then define $T:=[T1, T2]'$ which is a bivariate T distribution with zero covariance by stacking the $T_1$ and $T_2$ into a vector. Then $aT_1+bT_2=[a, b]T$ . Applying above conclusion, $aT_1+bT_2$ is univariate t distributed with the same degree of freedom, whose mean is $am_1+bm_2$ and scale is $a^2S_1+b^2S_2$.
An R simulation that supports this theory is:
<code>require('mas3321')
n=10000
sample=c()
mean1=.6
mean2=1.2
scale1=.5
scale2=1
p1=10
p2=20
samples_combt=p1*rgt(n, 10, mean1, scale1)+p2*rgt(n, 10,mean2 , scale2)
hist(samples_combt,probability = T)
mean_comb= p1*mean1+p2*mean2
scale_comb=p1^2*scale1+p2^2*scale2
curve(dgt(x, 10,mean_comb, scale_comb), add=TRUE)
</code>
Answer: Student-t distribution is a special case of the Generalised Hyperbolic Distribution which is closed under affine transform according to wikipedia page (all linear transforms are affine transforms). Hence I would think all linear transformations of student-t random variables (with same degree of freedom) are student-t distributed.
I believe if $$X \sim t_d(\nu, \mathbf{\mu}, \Sigma)$$ then $$\mathbf{w}^T X + c \sim t_1(\nu, \mathbf{w}^T \mathbf{\mu} + c, \mathbf{w}^T \Sigma \mathbf{w})$$
This is just my understanding after reading the below references and adds to answer posted by @user31575.
https://en.wikipedia.org/wiki/Generalised_hyperbolic_distribution
Also in "Quantitative risk management: Concepts, techniques and tools" section 2.3.1 equation 2.31
Also referenced in this paper Hu, W. and Kercheval, A.N., 2010. Portfolio optimization for student t and skewed t returns.
Comment: hence I think the addition of two uncorrelated student-t random variables (which isn't the same as independent) is also student-t distributed
Answer: If $X$ follows a multivariate t-distribution, then any linear combination of $X$ also follows a multivariate t-distribution with the same degrees of freedom:
$$
X \sim t(\mu, \Sigma, \nu) \quad \Rightarrow \quad Y = AX + b \sim t(A\mu + b, A\Sigma A^\mathrm{T}, \nu) \; .
$$
EDIT: Following @GeorgiBoshnakov's answer, I learned that everything enclosed between horizontal bars is not correct, because falsely assuming that independent multivariate t-variates have a joint multivariate t-distribution. What follows, therefore only holds in the limiting case $\nu \to \infty$ or for very large $\nu$, i.e. when the multivariate t-distribution becomes or can be approximated by a multivariate normal distribution. In this case, there is in fact a theorem stating joint normality with zero covariance under marginal normality and statistical independence.

Thus, the intended combination only works, if the two multivariate t-distributions have the same dimensions and degrees of freedom. Let us assume that
$$
\begin{split}
X_1 &\sim t(\mu_1, \Sigma_1, \nu) \\
X_2 &\sim t(\mu_2, \Sigma_2, \nu)
\end{split}
$$
where $X_1$ and $X_2$ are independent $n \times 1$ random vectors. If that is the case, we have:
$$
X = \left[ \begin{array}{c} X_1 \\ X_2 \end{array} \right] \sim t\left( \left[ \begin{array}{c} \mu_1 \\ \mu_2 \end{array} \right], \; \left[ \begin{array}{cc} \Sigma_1 & 0_{nn} \\ 0_{nn} & \Sigma_2 \end{array} \right], \; \nu \right) \; .
$$
The random variable you seem to have in mind is probably this:
$$
Y = c_1 X_1 + c_2 X_2 \; .
$$
Note that $Y$ can be emulated from $X$ by specifying an appropriate linear combination:
$$
A = \left[ \begin{array}{c} c_1 I_n & c_2 I_n \end{array} \right], \; b = 0_{n} \quad \Rightarrow \quad Y = AX + b = c_1 X_1 + c_2 X_2 \; .
$$
Thus, we can apply the linear transformation theorem from above:
$$
Y \sim t\left( \left[ \begin{array}{c} c_1 I_n & c_2 I_n \end{array} \right] \left[ \begin{array}{c} \mu_1 \\ \mu_2 \end{array} \right] + 0_{n}, \; \left[ \begin{array}{c} c_1 I_n & c_2 I_n \end{array} \right] \left[ \begin{array}{cc} \Sigma_1 & 0_{nn} \\ 0_{nn} & \Sigma_2 \end{array} \right] \left[ \begin{array}{c} c_1 I_n & c_2 I_n \end{array} \right]^\mathrm{T}, \; \nu \right) \; .
$$
This gives:
$$
Y \sim t\left( c_1 \mu_1 + c_2 \mu_2, \; c_1^2 \Sigma_1 + c_2^2 \Sigma_2, \; \nu \right) \; .
$$

Fun Fact: The above theorem can also be used to prove the relationship between the multivariate t-distribution and the F-distribution.
Comment: *"If X follows a multivariate t-distribution, then any linear combination of X also follows a multivariate t-distribution with the same degrees of freedom"* This is only true for a linear combination of components of a multivariate t-distribution. It is not true for a linear combination of two independent multivariate t-distributed vectors.


Using C#, how can I read the content of dynamic created textboxes?

Question: Hy,
I have created some dynamic textboxes with standard content.
Does anyone know how can I read the content of these textboxes (assuming that user modified the standard content) when I press one button?
Thanks a lot.
Jeff
Update
This is how I am creating the textboxes:
foreach (string name in listOfNames)
{
   TextBox tb = new TextBox();
   tb.Text = name;
   tb.BorderStyle = BorderStyle.None;
   tb.BorderWidth = 0;
   tb.Font.Name = "Arial";
   tb.Font.Size = 8;
}

Comment: WinForms, WPF, ASP.NET, Silverlight, XNA, Windows Phone 7, XBox 360, Zune, ...???? Try to be a little more specific, there are textboxes and buttons everywhere. Also define *dynamic*. How are those textboxes created, ideally provide a code snippet.
Comment: What technology are you using? ASP.NET? Win-Forms? WPF/Silverlight?
Comment: In Addition to Darin's comment, how where they added?  DO you know the names?
Comment: I am using ASP.NET, I have updated the tags, sorry
Comment: @Jeff, tags are OK, now, the next part is the code snippet you had to provide about how are those textboxes created.
Comment: You don't seem to be adding the controls to the page at all. How are they added? To what controls collection?
Answer: The specific will vary depending on the technology you are using. However the concept would remain very similar, though for ASP.NET it will be a little more interesting.
WinForms/WPF/Silverlight
Maintain a list of the dynamically created textboxes and when the button is pressed you can run through the list of textboxes and read the Text property to get the user input.
ASP.NET - After the tag update it seems this section is most appropriate to your requirement.
For ASP.NET you will need to create the textboxes in an override of the OnInit method, this should happen on each postback. Then in the Button.Click event you can read the user input from the textboxes that you created in the OnInit function. You need to ensure that the controls are created with the same ID on each post back.
Comment: +1 - for "Same ID"...without a consistent ID, dynamic controls will render but all sorts of problems occur (like events that only fire when you click the element twice).
Answer: You need to ensure that the text boxes are recreated on every postback.
If you do not recreate them, you will not be able to access their properties or events.
The best place to create dynamic controls is the page Init event handler.
I suggest reading up on the ASP.NET page life cycle.

Update (following updated question)
Make sure to set an ID (and a different one, at that) for the text boxes, so you can refer to them later on.
I can't see where you are adding these controls to the page either.
Comment: @Richard Friend - I never did that, but then again, I could be wrong. Can you point me to any articles that say this?
Comment: I know creating in Init works, however a lot of controls rely on EnsureChildControls() which inturn looks at ChildControlsCreated which in turn calls CreateChildControls.  see http://www.singingeels.com/Articles/Dynamically_Created_Controls_in_ASPNET.aspx
Comment: @Richard Friend - He also says (almost the very start of the article) that the page Init event handler is the best place. Using `CreateChildControls` is used for addressing a specific issue with exceptions.
Comment: He does say that but the summary at the end states to use CreateChildControls. again this is an area which seems unclear, however ms states thats what its for http://msdn.microsoft.com/en-us/library/system.web.ui.control.createchildcontrols.aspx
Comment: http://stackoverflow.com/questions/1504193/when-must-i-use-asp-net-createchildcontrols
Answer: Request.Form is a collection of Key-Value pairs that represents all of the data coming back from the ASP.NET request. If you access that you can get any value whether its control is specified in the ASPX code or dynamically created in the code behind file.
You can also get their values placed back in them automatically if you recreate them during the init part of the page lifecycle. If you do this, their values will be set when ASP.NET recreates the state of the page and applies the values from the form.
Answer: You should be able to access them as you would a normal control, you just need to get a reference to the control.
Remember to re-create all controls on each postback.
this is a useful article on dynamic controls in asp.net


Where do folds come from on a surfboard and are they bad?

Question: Why are there sometimes folds on the top of a surfboard and where do they come  from? Do they mean the board is in bad shape (or more exactly, is the board less usable or less effective)?
Answer: Those are creases. They effect performance as the board will flesh (bend) more around that point. The board is much more likely to break in half at that point as well. Creases can be fixed, can be expensive for epoxy boards ($200-$300 minimum) and requires a pro to do it right. A bad fix can weaken the board further.


Ridge Regression: changing tuning parameter

Question: I know for ridge regression, we add a penalty term with a fixed constant λ for all β the as tuning parameter. Just wonder why choosing a fixed λ instead of varying its value for different β?
Like this:
Answer: If you varied the penalty for every $\beta$ the effective number of parameters would be $2\times p$ and you would have overfitting once again.  Essentially the model would be non-identifiable.  Penalization works by borrowing information or tilting estimates towards a preconceived bias in a systematic way.  If you don't connect the $\beta$s in some way you are not borrowing information.  What would work would be to group the $X$s using prior knowledge about the likely importance of them, and to give a different $\lambda$ to each group, assuming every group has $> 1$ predictor in it.  The pre-grouping would not be allowed to use $Y$.


How to define an optional argument to Route callback

Question: I want define a simple controller action, which receive optional argument:
<?php

namespace Drupal\hello\Controller;

class SimpleCallback extends ControllerBase
{
    public function sayHiAction($name = 'there')
    {
        return new Response('Hi ' . $name, 200);
    }
}
?>

If I define the routing like this
# file @hello.routing.yml
hello.sayHi:
  path: '/hello/{name}'
  defaults:
    _title: 'Say Hi'
    _content: 'Drupal\hello\Controller\SimpleCallback::sayHiAction'
  requirements:
      _permission: 'access content'

GET /hello just gives me 404 response.
Follow official documentation, I updated my route:
# file @hello.routing.yml
hello.sayHi:
  path: '/hello/{name}'
  defaults:
    _title: 'Say Hi'
    _content: 'Drupal\hello\Controller\SimpleCallback::sayHiAction'
    custom_arg: 'there'
  requirements:
      _permission: 'access content'

But nothing changed.
Answer: The docs you link to say:

Finally, if any static arguments should be passed to the controller, provide those in the defaults array as well. Name them to match the arguments of your controller method. See below.

Emphasis mine.
So your code should be:
# file @hello.routing.yml
hello.sayHi:
  path: '/hello/{name}'
  defaults:
    _title: 'Say Hi'
    _content: 'Drupal\hello\Controller\SimpleCallback::sayHiAction'
    name: 'there'
  requirements:
      _permission: 'access content'

See also Required and Optional Placeholders in the Symfony routing docs.
Comment: "Name them to match the arguments of your controller method". Yeah there's some magic that won't work if you name it differently.


Cross Domain SSO with DotenetOpenAuth

Question: I have integrated DotenetOpenAuth to login using google account.
Its working perfect.
Can i achieve this with it ?
Once user logged in site1.com if user browse to site2.com the user get automatically logged in.
Answer: DotNetOpenAuth includes a Samples directory. There are OpenIdWebRingSsoProvider and OpenIdWebRingSsoRelyingParty projects which, I believe, do exactly what you want to achieve. Though, you have to implement your own central SSO provider.
Comment: Have you tried looking into example projects? Well, my point is that for site2 to **know for sure** that a user is logged in to site1, I believe you have to implement some authentication providing party. I think you cannot just ask Google from site2 - "is this user logged in to site1?". That would be insecure. So when a user visits site2, it should redirect a user to site1's SSO provider which is similar to the one in the example project. Then SSO provider would check if the user is indeed who he claims to be and if he is logged in. On success, it will redirect the user back to site1.
Comment: can you please elaborate `central SSO provider`
Comment: does `DotenetOpenAuth` provides both `authentication` and `authorization` ?
Comment: If the answer helped you, mark it as accepted. Otherwise, consider asking more questions in the comments.

Cross Domain SSO with DotenetOpenAuth


Como mudar a imagem de uma ImageView de forma dinâmica

Question: Olá, preciso de ajuda para mudar uma imagem de forma dinâmica
Segue meu código:
st = (ImageView) findViewById(R.id.imageView4);
int RID = this.getApplicationContext().getResources().getIdentifier(Bundle_img, "drawable", getPackageName());
st.setImageResource(RID );

o código acima não está substituindo a imagem anterior.
Comment: Amigo Obrigado, já encontrei o problema. Cometi um FO falha operacional. :D Minha imagem está com ic_situacao_+(Bundle_img) OBrigado
Comment: Coloca a solução aqui pra ajudar os outros no futuro. obg
Comment: Como está vindo a String `Bundle_img` ?
Comment: Bundle_img = "red"
Answer: Tente usar getResources().getDrawable(R.drawable.nome_imagem).
Exemplo:
Drawable drawable= getResources().getDrawable(R.drawable.nome_imagem);
img.setImageDrawable(drawable);


If a vocabulary and a view have the same machine name, will they conflict when exporting to a same feature?

Question: In other word, does the machine name of every type of Drupal object has its own scope? Do we need use some prefix?
Answer: If it's in a different table it's in a different scope. If it allowed you to set a similar machine name you are obviously in a different scope. If you go to a node page and try to create similar machine names you will get an error while adding the field.
taxonomyTable.same_machine_name is different from viewTable.same_machine_name
No they wont conflict when exported from features. on the features page it shows different sections like taxonomy terms, nodes, strongarm, image styles ect.. anything that could be related then it auto detects anything related and you can select what you want to go with it. That's why it's important to have unique machine names so you know right away what you are working with. When using features try to avoid using the same machine names; but you can get away with it
Answer: A vocabulary is an entity; before Drupal 8, a View is not. So ID collisions will never be a concern, they're different types of object, and treated as such. If a View was an entity (like in D8), then its properties would be unique to its entity-type, so there's still no concern.
Addressing this specifically:

does the machine name of every type of Drupal object has its own scope?

If you consider a "Drupal object" to be an entity provided by the core API, then yes. Views doesn't use it, but its data is in a separate DB table so there's nothing to worry about. YMMV with different contrib modules, but usually if a features integration has been written, you can pretty much assume it's going to work at that most basic level.


O que é "Idempotency Messages", "idempotente", "idempotência"?

Question: Tenho uma pergunta sobre o assunto aqui, e um artigo sobre o assunto até mesmo na Wikipédia, mas ainda não compreendi o assunto, portanto as perguntas são,

O que é Idempotency Messages?
Como e onde deve ser utilizado?
Exemplo simples de uso?
Answer: Uma operação idempotente pode ser entendida como algo que estabelece um valor ou estado ao invés de modificá-lo.
Um exemplo simples pode ser o saldo bancário. Imagine o extrato abaixo:
Data             Valor        Operação
01/01/2017      100,00        Abertura de conta + Depósito inicial
02/01/2017      -14,99        Netflix
03/01/2017      -20,00        Refeição
04/01/2017      -30,00        Táxi
01/02/2017        5,01        Saldo Atual
02/02/2017      +14,99        Extorno - Netflix

Se você quer saber o saldo desta conta no dia 06/01/2007 você não pode apenas observar o último valor processado, porque ele é um modificador (-30 reais no dia 4/01). Você precisa levar em consideração todas as modificações até encontrar o primeiro valor definido por uma operação idempotente - neste caso, a abertura de conta com o valor de 100 reais.
A mensagem no dia 01/02 também pode ser considerada uma operação idempotente: Muitos sistemas se utilizam deste mecanismo para facilitar avaliações de estado. No exemplo acima você não precisa recalcular todos os valores até a primeira operação para determinar o saldo atual.
Comment: @MarcelodeAndrade sim, se sua operação estabelece um estado (ao invés de modificá-lo) ela pode ser consideda como idempotente também.
Comment: Uma operação desse tipo necessariamente deve ser um valor inicial? Digo, se no teu exemplo adicionarmos um lançamento chamado `saldo atualizado`, com o valor das modificações, seria também um `idempotente`?

Exemplo: `04/01/2017      35,00        Saldo atual`
Answer: - O que é Idempotency Messages?
Primeiro é necessário compreender o conceito de idempotência: uma operação ou mensagem que pode ser aplicada diversas vezes sem alterar o resultado final.
- Como e onde deve ser utilizado?
Pode ser aplicado em comunicações assíncronas e/ou por mensagens, por exemplo. Uma mensagem recebe um identificador. O destinatário deve entender que uma segunda mensagem com mesmo identificador não deve alterar o estado final
- Exemplo simples de uso?
Imagine um sistema que recebe a mensagem:
{
"Identificador" : "789456",
"Comando" : "CancelarPedido",
"CodigoPedido" : "A258963"
}

Suponha que o sistema recebe essa mensagem pela primeira vez, ele cancela o pedido. Quando este sistema receber uma segunda mensagem como mesmos Identificador, Comando e CodigoPedido deve ser aceita como válida e retornar o mesmo resultado que a primeira ocorrência dela teve. O resultado final da mensagem, o cancelamento do pedido, não será alterado por mensagens subsequentes.
Há um episódio do Podcast da Lambda3 sobre Filas e Mensagerias que discute cenários sobre o tema e como esse conceito deve ser aplicado.
Answer: Vamos entrar na teoria sobre a idempotência antes de aplicá-la em um exemplo prático, até mesmo porque após compreender o que é a idempotência, você pode OU não aplicá-la dependendo da sua situação.
O que é idempotência?
Idempotência é a propriedade que permite uma operação ser aplicada mais de uma vez, sem que o resultado seja alterado.
Onde aplicar idempotência se torna interessante?
Quando existir a possibilidade de processamento duplicado de informação, onde irá causar problemas de inconsistência (principalmente no negócio).
No caso de comunicações assíncronas, utiliza-se o termo de Idempotency Messages, onde juntamente com a mensagem é enviada uma propriedade (um ID ou um token), que irá unicamente identificar a mensagem.
Você pode verificar através dessa propriedade se a mensagem já foi processada (ou não verificar), e tratar de acordo (apenas ignorar a mensagem ou reprocessar)
Outro exemplo seria quando houve uma falha na requisição, e você não tem certeza se a sua solicitação foi de fato atendida pelo servidor, imagine o seguinte cenário:
1) (Saldo total) R$1000,00
2) Transferência de R$100,00 para minha conta (POST /transfers?amount=100)
3) 200 OK | (Saldo total) R$900,00
4) Transferência de R$900,00 para minha conta (POST /transfers?amount=900)
5) CONNECTION TIMEOUT | (Saldo total) ?
6) Transferência de R$900,00 para minha conta (POST /transfers?amount=900)
7) 200 OK | (Saldo total) -R$900,00

Quando houve um timeout na conexão, não tivemos a resposta se a operação foi realizada ou não, no nosso caso o servidor já havia processado e descontado R$900,00, deixando nossa conta zerada. Uma nova requisição realizou uma outra trasferência de R$900,00, deixando nossa conta negativa.
Para este exemplo, caso houvesse um token de idempotência implementado, o fluxo ficaria desta forma:
Antes de realizar a operação, meu token de idempotência é gerado com o valor 98s0a9F

1) (Saldo total) R$1000,00
2) Transferência de R$100,00 para minha conta (POST /transfers?amount=100&tk=98s0a9F)
3) 200 OK | (Saldo total) R$900,00

Nova operação, meu token de idempotência é gerado com o valor f019a8f

4) Transferência de R$900,00 para minha conta (POST /transfers?amount=900&tk=f019a8f)
5) CONNECTION TIMEOUT | (Saldo total) ?
6) Transferência de R$900,00 para minha conta (POST /transfers?amount=900&tk=f019a8f)
7) 200 OK | (Saldo total) R$0,00

Para este segundo exemplo, o servidor vai identificar que para o token f019a8f uma operação já foi realizada, e somente vai enviar o resultado da operação que ocorreu com sucesso, sem realizar uma nova transferência.
Certo e onde eu consigo ver essa idempotência funcionando?
A Stripe possui um mecanismo bem interessante de idempotência para garantir que nenhuma operação seja realizada sem que de fato se tenha a intenção de realizá-la.
É importante destacar também que no caso do HTTP, existem métodos que são idempotentes por padrão e não necessitam deste tratamento.
Comment: Perdi o prazo de aplicar a recompensa e o sistema deu automaticamente metade pra outra nova mais votada, mas eu acho que esta aqui ficou melhor elaborada. Os exemplos e descrição estão bons. Só quis mencionar para que ninguem pense que os 25 foram na outra por uma escolha voluntária. De qq forma, no mínimo deixo meu +1 aqui
Comment: Não esquenta @Bacco :) fica pra próxima


SQL Update Parent ID from Same table

Question: Is it possible to update self referencing parent id with Update From query.
Tables I have
Acc
AccID    Name  ParentAccID  AccNumber
01       A       Null        A001
02       B       Null        A002
03       C       Null        A003
04       D       Null        A004
05       E       Null        A005

AccRef
AccNumber       MasterAcc    Name
A001             A001         A
A002                          B
A003             A002         C
A004             A005         D
A005             A009         E

the way it should be joined is with AccNumber and then need to updat AccID in ParentAccID based on MasterAcc of that Acc. Only for existing AccNumber and if AccNumber and MasterAcc is not same (sorry if its big confusing)
Result table should look like
Acc
AccID    Name  ParentAccID  AccNumber
01       A       Null        A001
02       B       Null        A002
03       C       02          A003
04       D       05          A004
05       E       Null        A005

Thanks
Comment: Why for AccID=05 your ParentAccID is null in your expected result?
Comment: AccId=05 is from AccNumber A005 which has MasterAcc = 'A009' which doesn't exist in Acc table (if you join on AccNumber)
Answer: Try this,
DECLARE @Acc Table (AccID varchar(10),Name  varchar(200),ParentAccID  varchar(200), AccNumber varchar(200))
Insert into @Acc Values
('01','A',Null,'A001'),
('02','B',Null,'A002'),
('03','C',Null,'A003'),
('04','D',Null,'A004'),
('05','E',Null,'A005')

DECLARE @AccRef Table (AccNumber varchar(10),MasterAcc varchar(200),Name  varchar(200))
Insert into @AccRef Values
('A001','A001','A'),
('A002',Null,'B'),
('A003','A002','C'),
('A004','A005','D'),
('A005','A009','E')

Update A set ParentAccID = M.AccID From @Acc A
Inner Join @AccRef R on R.AccNumber = A.AccNumber and R.AccNumber <> R.MasterAcc
Inner Join @Acc M on M.AccNumber = R.MasterAcc

Hope this is what you are looking for.
Finally,
Select * from @Acc

Comment: that's exactly what i was looking for. Thanks

SQL Update Parent ID from Same table


Backbone Collection

Question: I was trying to make a Backbone Application with Django at its backend. I was following a Backbone tutorial. I used the following code:
Code
<!doctype html>

    <meta charset = "utf-8">
    <title>IstreetApp</title>
    <link rel="stylesheet" href = "https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/2.3.1/css/bootstrap.min.css">

Book Manager

Since the collection is not defined, the success code doesn't execute. I suppose the collection data should come from the server through Django but I am not sure how and in what form. Kindly help. I am pretty much new to backbone and Django.
Comment: Why don't you put an error handler on your _fetch_ call?
Comment: @svetoslavnedkov how can i do that?
Comment: it uses the same format as the success handler, just put ',error: function (data) {alert(data);}' after your sucess handler.
Comment: It doesn't alert me with any error. I haven't asked server to send the data while fetching the `index` template. Should I do that?
Comment: Actually I tried the code you have and what I get is a 404 from the server, so the problem I think is server side. You must put in place a request handler that returns books for GET requests for /books.
Answer: When you call fetch on your collection, it makes an AJAX request to:
 http://backbonejs-beginner.herokuapp.com/books

However, there is no API set up there.  Either one of two things needs to happen:
1) you need to modify your code to point to a different URL, one that does have an existing API (perhaps whatever tutorial you are using has such an API)
2) you need to create such an API yourself on yoursever.com (and then make your Backbone code point to that API's URL instead)
Without a server to support it, operations like save and fetch and such in Backbone simply cannot function.
As a side note, Django is a web site framework.  While you can use it to create server-side APIs, that's not really Django's focus.  Because of this, several good third party libraries exist for doing RESTful APIs (ie. the kind that Backbone likes) in Django; personally I'd recommend either Django REST Framework (I use it and it works great) or TastyPie (never used it, but it's very popular).
Comment: That makes sense. I was waiting for such answer. Can pystache(Mustache) be used for the same purpose?
Comment: You *could*, but it'd be a bad idea.  What I mean is, you can use a templating system (either pystache or Django's) to make a JSON response, with a template that's something like: `{id: {{id}}, someField: {{valueOfSomeField}}, someString: '{someStringValue}}'}` and a Django View that provides those values. However that gets ugly fast.  Building the JSON in the View instead (with TastyPie/Django REST Framework), will almost work better; for example `response = JSONResponse(); response.id = id; response.someField = someField; response.someString = someStringValue; return response;`.
Comment: P.S. I know semi-colons are not very Pythonic, but comments don't support newlines, so ...
Comment: Thanks alot. I am using Django REST Framework and its really good.
Comment: Glad I could help.  BTW, for any future readers I meant to say "will almost *certainly* work better" in my previous comment :-)
Answer: When using a backbone collection you need to return a json array of objects from your api url (http://backbonejs-beginner.herokuapp.com/books)
Example
{[{"name":"bookname", "publisher": "penguin"}, {"name":"bookname", "publisher":"penguin"}]}

You'll also want a model for your collection. A model would look like this
Example:
var Book = Backbone.Model.extend({
defaults: {
"name":  "",
"publisher": ""
}
});

The way that the collection works is by parsing the json array and turning each object in the array in to a model that you specific (in this instant an individual book with values for the name and publisher).
When you make a .fetch() on your model you are making a GET request, so make sure that your http://backbonejs-beginner.herokuapp.com/books url is prepared to receive GET requests and respond with the json data in the format I specified up top.

Backbone Collection


Generate the output on a particular cell ID in CSV

Question: What I am trying to achieve here is to extract the values from  tags HTML to a CSV file which I am succesful. What I want to is to get some control over the output being generated.
HTML values are:
<input type=hidden id=batterystatus value="Charging" >
<input type=hidden id=batterylevel value="80 %" >
<input type=hidden id=signalStrength value="4" >
<input type=hidden id=ulCurrentDataRate value="5 Kbps" >
<input type=hidden id=dlCurrentDataRate value="136 Kbps" >
<input type=hidden id=ConnectionTime value="00:15:33:10" >
<input type=hidden id=noOfClient value="2" >

Now, I wrote the script using perl.
#!usr/bin/perl

use strict;
use warnings;
use HTML::TokeParser::Simple;

my $url = 'http://192.168.225.1';
my $parser = HTML::TokeParser::Simple->new(url => $url);
my %tags;
while (my $tag = $parser->get_tag('input')) {
           my $id=$tag->get_attr('id'); # get id attribute value
           my $value = $tag->get_attr('value'); # get value attribute value
           $tags{$id}=$value;
 }
my @tags;
my @values;
for (keys %tags){
   push (@tags, $_);
   push (@values, $tags{$_});
}

open(my $OUTFILE, ">", "test.csv")
    or die "Unable to open testfile for writing : $!";
print $OUTFILE join(",",@tags)."\r\n";
print $OUTFILE join(",",@values)."\r\n";

The data that is generated in CSV file is:

and then again when I re-ran this code I found something like this:

Questions:

How do I fix the cell id, such that I always get IMSI in Cell A1
and also the other output in particular cell?
How to I rename each cell header? For example batterylevel to be renamed as Battery Level or connectedStatus to be renamed as Connected Status.
Comment: Sidenode: Are you aware that the `id` of an `` is not the identifier that is used by forms? Those use the `name`. Just saying. :)
Comment: @simbabque - Question has been edited. Added the sample HTML code
Comment: I wonder what kind of page this is if there are no names on the inputs. Might it be the Kies Air status page or something similar where a mobile device has a webserver and you can access it from the network? The point is, those columns are not sorted, but they are all present.
Comment: @simbabque - I did not copy the entire html page since I am more concerned with picking out the values from there. If you want to have a look into let me know I will share it as well.
Answer: You are using a hash to store the data and are not sorting it when accessing the values. A hash in Perl is not sorted. The order of the keys is random and (at least in recent Perls) you cannot rely on the order. You need to sort the keys yourself. That is a feature.
The easiest way to do that is to just hardcode the order, since it looks like you are always expecting the same input. For renaming the columns, just use a hash as a lookup table.
I used Text::CSV because it is well-tested and pretty good at handling even arcane border cases.
use strict;
use warnings;
use HTML::TokeParser::Simple;
use Text::CSV;

my $html = <<HTML;
<input type=hidden id=imsi value="406E+14" >
<input type=hidden id=batterystatus value="Charging" >
<input type=hidden id=batterylevel value="80 %" >
HTML

my $parser = HTML::TokeParser::Simple->new( string => $html );

# mapping for the column headers
my %names = (
    imsi          => 'IMSI',
    batterylevel  => 'Battery Level',
    batterystatus => 'Battery Status',
);

my %tags;
while ( my $tag = $parser->get_tag('input') ) {
    my $id    = $tag->get_attr('id');       # get id attribute value
    my $value = $tag->get_attr('value');    # get value attribute value
    $tags{$id} = $value;
}

# use meaningfull names and sort in the loop
my @headers;
my @values;
foreach my $tag (qw(imsi batterylevel batterystatus)) {
    push @headers, $names{$tag};
    push @values,  $tags{$tag};
}

# create a T::CSV handle with Win line endings
my $csv = Text::CSV->new({
  sep_char => ',',
  eol => "\r\n",
  quote_char => q{},
});

# write out headers and values
$csv->print(\*STDOUT, \@headers);
$csv->print(\*STDOUT, \@values);

Comment: I am getting an error `Global symbol "@tags" requires explicit package name at htmlparse_test.pl line .
Execution of htmlparse_test.pl aborted due to compilation errors.`
Comment: But I am using it, since I need to write the data on csv file. Can you look into that pastebin code?
Comment: Feedback to your pastebin link: move the `open` above the `Text::CSV->new`, replace `\*STDOUT` with `$OUTFILE` (which is better named `$fh`, all-caps names suggest global variables, and it's a handle not a file) and get rid of the `print` `join` line.
Comment: In my answer, Text::CSV is taking care of the CSV file for you. It already does what you want. You can still do your `print` yourself of course. All I did was rename the variable to a more meaningful name. :)


Which distribution do I get?

Question: Be $X\sim N(\mu,1)$ and $Y\sim Inverse-Gamma(\alpha,\beta)$.
For the Inverse-Gamma, I usually use the parameterization which leads to the following probability distribution function for Y:
$f(y;\alpha,\beta)=\frac{\beta^{\alpha}}{\Gamma (\alpha)}(\frac{1}{x})^{\alpha+1}e^{-\frac{\beta}{x}}$
I need to find the distribution of $T=X\sqrt{Y}$.
According to my calculations, T is not a non-central Student's t-distribution , it is a non-standardized Student's t instead with $2\alpha$ degrees of freedom, location parameter $\mu$ and a scale parameter $\sqrt{\frac{\beta}{\alpha}}$.
Is it correct?
Thank you.
EDIT:
These are my calculations:
Comment: There are lots of competing definitions as to what an InverseGamma(a,b) distribution is or isn't ... so the question is not defined until you specify the functional form of the pdf you are assuming.
Comment: As what appears to be fairly routine book-work, this question should carry the `self-study` tag (see its [tag wiki](http://stats.stackexchange.com/tags/self-study/info) for the special way such questions may be dealt with). Since you can only have 5 tags, I'd suggest replacing the `distributions` tag. Perhaps you could (i) explain why you expect it to be non-central-t (since you raise that possibility), and (ii) show your work (i.e. why you now think it isn't), so that we can give more specific guidance.
Comment: We can't really give 'yes' or 'no' - or even one sentence - as answers, so it would also help in that way if you could give details we could respond to. But as far as that goes, I think it should be non-central t
Comment: The distribution of your T will not generally be symmetrical. By contrast, the Student's t pdf is symmetrical. Therefore, your T is not distributed as Student's t.
Comment: Check your calculations: @Glen_b is correct.
Comment: Yes, it's clear from your question you think it looks like a (central) t. The question is, what is your basis for thinking so? You refer to calculations -- what did you do to conclude that it was central t?
Comment: I just added the missing information. @ Glen_b: The reason why I'm asking this question is because in the "Prior distributions for variance parameters in hierarchical models" the author says that you get a non-central t-distribution but it looks more like a non-standardized Student's t to me
Comment: I just added my calculations. There must be a mistake somewhere. Thank you all for the interest.
Answer: You have a term:
$$\frac{X}{\sqrt{\frac{\chi^2_{2\alpha}}{2\alpha}}}$$
which you then say is $\sim$ student-$t(2\alpha)$
But for that to be the case, $X$ would need to have mean 0.
When it has mean other than 0, you have non-central $t$.
Call the denominator $R$ and consider writing it as
$\qquad\frac{X-1+1}{R} = \frac{X-1}{R}+\frac{1}{R}$
The left hand term is a (central) t-distribution, while the other term is right skew (and the two terms are dependent because the variable on the denominator is shared).
Informally, it seems like the sum of the symmetric and the right-skew term might be right skewed. It turns out to be the case.
Note that having $X$ with mean other than zero does not result in the same distribution as a location-shifted $t$
Comment: You are right. I didn't notice it. So $\mu$ is the non-centrality parameter. The problem is that now I can't lead back the formula to the general definition of a non-central Student's t-distribution as $T=\frac{Z+\mu}{\sqrt{\frac{V}{v}}}$ where $Z$ is a standard normal and $V$ is a Chis-squared with $v$ degrees of freedom, according to [this](https://en.wikipedia.org/wiki/Noncentral_t-distribution#Characterization) definition. Where do I put $\sqrt{\frac{\beta}{\alpha}}$? Is it a generalization of the non-central student's t? Thank you.
Comment: Yes, I think you have a scaled non-central t.

Which distribution do I get?


Is turning off Perfect Forward Secrecy a security threat?

Question: I had to disable PFS in the IPsec rules on one of our ASAs to allow Android Gingerbread clients to connect. Does doing so compromise the security of the VPN? Communications are AES-256 or AES-128 with group password. From my research, it seems like PFS is considered by some to be questionable, and to tell you the truth I never really noticed it before now, so I just don't know what to do here.
ETA: since this is apparently relevant, the ASA in question is used only for Remote Access, no lan-to-lan.
Answer: It depends a bit on how you've got the VPN configured. If you're using pre-shared keys, then PFS isn't doing much anyway. If you're using certificates, capturing a session initialization and the secret key of a device is compromised (say a phone is stolen, making both of these trivial) then it's significantly easier to calculate the key on the ASA (making it easier for someone who can capture any VPN session to decrypt the session).
In any case it would still take significant computational time to crack the keys and ability to eavesdrop on VPN connections.


Change the Source DB for migrating via Drush

Question: A few days ago I originally ran the full migration via the UI to get the site up and running and cleaning up any bugs from a DB instance that is a copy of the current live site. However, now that the site is up and running, I would like to re-migrate just the users from a NEW copy of the current live site in order to have an up-to-date data before going live (instead of losing any changes users made over the last few days).

I don't want to re-migrate through the UI because I just need to migrate users
I tried adding a $databases['migrate']['default'] setting in the settings.php file but migrate-import doesn't have any option like --legacy-db-key
drush migrate-status gives me an error because I deleted the backup db after I successfully migrated, but can't figure out a way to change the source DB

How can I change the source DB for the migrate drush commands in order to run a migration on just users via the drush migrate-import command?
Answer: in settings.php add the connection to the source database (that contains users you want to migrate)
$databases['myuserssourcedb']['default'] = [
  'database' => '',
  'username' => '',
  'password' => '',
  'host' => '',
  'port' => '',
  'driver' => '',
  'prefix' => '',
  'collation' => '',
];

in your migration yml file reference the source database as 'key' in the 'source'
id: migration_of_users
label: 'Migrate users from old database'
migration_group: migration
source:
  plugin: mypluginforusers
  key: myuserssourcedb  <-----This is the important line
process:


Does an AWS RDS Endpoint FQDN change when making an instance Private?

Question: I am using AWS RDS, accessed by some servers in AWS EC2, all connected to the same VPC.
One AWS RDS instance we have apparently was set to "Publicly Accessible", and we want to correct this.
My question: If I change the Instance's settings from "Publicly Accessible: Yes" to "Publicly Accessible: No", will that change the Endpoint FQDN?
Answer: No, it won't change the endpoint. Of course you should take a manual snapshot just in case something goes wrong, so you can restore. Manual snapshots are never deleted by AWS.
The great thing about AWS is things are so cheap you can try them yourself. Create yourself a t2.micro RDS instance that's publicly accessible, get it going. Copy the FQDN. Now make the change you're planning, and check the FQDN again. Total time: 10 minutes. Total cost, $0.10.
You can also set your security groups / Network ACLs to close the port externally, then it won't be publicly accessible. I have an RDS instance that is deliberately publicly accessible, I just changed my security group to remove my home IP, and I could no longer log into the database.


Voice Leading with Octave Leaps

Question: In a piece I'm working on, the climax of the melody line consists of an upward octave leap followed by another step upward--from G4 to G5 and then to A5. (I realize this is generally discouraged, but I like the sound of it.)
My question is what to do with the alto line, as I can see three options (discounting any alteration of the melody):
1) Overlap S/A
2) Have S and A more than an octave apart, and
3) Merge the S and A voices on the note before the octave leap.
Which of these is generally best in this type of situation?
Answer: Here's the thing: voice-leading rules aren't universal.  They are based on the assumptions of a particular style.  In particular, the kinds of concerns brought up in the question sound like OP is worried about "chorale style" (SATB?) voice-leading, derived from principles used mostly in 19th century hymnody.
If the soprano leaps up an octave (G4 to G5) and then continues upward to A6, that's simply a pattern that would never occur in 19th-century hymnody.  The rules of voice-leading for that style don't apply, as you're no longer writing in that style.
Instead, what should be considered is the reason for trying to maintain a "chorale-like" texture, and what effect breaking various voice-leading "rules" will have.  To understand that, one should consider the rationale behind the rules in the first place.  So, let's take the options in turn:
1) Overlap S/A
I'm not sure whether you mean a voice overlap (which sometimes means to move one voice past the range of another voice's previous note, in this case skipping the alto up past G4 when the soprano takes G5) or a true voice crossing.  If it's a voice overlap, the effect in a large ensemble can be losing track of which line connects with which, and also a feeling of two voices taking larger leaps in the same direction.  But voice overlaps of this sort occur all the time in non-chorale style textures, like an instrumental texture with bass voices and two upper voices that have a section where they travel in parallel thirds (including leaps).  It's a textural effect, primarily.
On the other hand, true voice crossing (where the alto sings/plays higher than the soprano) can undermine the sense of melody.  That can also work in instrumental pieces with proper orchestration, but should be used with caution elsewhere.  If voice crossing of the melody is allowed, it's probably best not to make it a "one-off" note that occurs once, which might sound weird.  Instead, you might have an inner voice take a sort of "descant role" for several notes or even an entire phrase.
2) Have S and A more than an octave apart
This again is a textural choice.  Many instrumental textures frequently will have a solo melodic voice floating high above the other voices for an extended period.  The general rule of keeping upper voices within an octave only helps to give a sense of an "ensemble" that are part of the same texture, rather than have a "solo voice" high above the others.  It can be workable, depending on the style you're after.  But again use caution in having it be a "one-off" event.  Switching to a different texture where the top voice becomes more like a solo may be useful for effect for a phrase or section, but having an unusual spacing for a single chord may just sound odd.  It's going to be a judgment call.
3) Merge the S and A voices on the note before the octave leap
There's no voice-leading rule against this even in strict chorale style, so there's no reason to avoid having any two adjacent voices come together on a unison for a single pitch.  Having two voices together might tend to make that note sound louder, so I suppose that's a potential concern if that's not an effect you want.

In general, the best compositional advice is to avoid something that stands out and potentially sounds odd in context.  You generally want to adopt a consistent set of principles for whatever texture/ensemble you're working with, and you want to avoid something that sounds disruptive.  Your goal and rationale for incorporating such a leap in the first place may help guide what solution will work best in your particular situation.
Answer: The question is slightly opinion based I guess. The answer depends on what the Tone of your piece is, Theme and also whatever you wish to convey through your piece.These decide the answer.
Merging might be a good option if you want to make this section bolder.
Overlap might be good if you want to add some flare to the harmonic texture.
You might want to separate S and A if you want a clear distinction between lead and Harmony.
If your message is strong enough, you might wanna bring in some dissonance here. It is easier for the singers if you keep some seperation between S and A. Make use of dynamics. Try making Alto softer but somewhat close to S.
Generally Best - Nothin like that exists like a general thing to compose music. It's about you getting creative.
Answer: It isn't clear what you think the problem is.
If the problem is purely a melodic concern - the octave leap plus step which I assume you mean is in the soprano, changing the alto won't fix a bad line in the soprano.
If your concern is the wide spacing between alto and soprano and chorale harmony norms, then fix the part writing. Abandon the octave leap and write like a chorale.
Finally, if there isn't a melodic problem, and you don't care about chorale style, then embrace what you are really doing. Don't treat the alto like an apology to chorale style. Do with it what you need it to do. I image that should be complete the next chord with motion that makes sense with the rest of the music.
It's hard to say any more, because you gave no musical context other than "G4 to G5 and then to A5" and there is an alto part.

Voice Leading with Octave Leaps


Why don't you need to use multiple comparison adjustments when you choose to look at one out of many possible comparisons?

Question: I was explaining multiple comparison adjustments to a colleague and explained how, if you test one hypothesis alone (call it hypothesis A), it's appropriate to use a certain alpha (say .05) but if you test that same hypothesis A among many (say B-K), it's appropriate to use a stricter alpha (e.g. .005 with Bonferroni). Their question was, "why does looking at hypotheses B-K change the amount of evidence I need to reject the null for hypothesis A? If any of these 10 possible hypotheses could be spurious, shouldn't we adjust the alpha for hypothesis A to .005 even if it's the only one we look at?"
I understand this intuitively but found myself at a loss for a helpful explanation. Does anybody want to give this a shot? I'm looking for something intuitive and not very technical.
Thanks!
Answer: You're in luck, I was just grappling with a similar problem.
The key concept here is the Familywise Error Rate (FWER), which is the probability that at least one of a "family" of tests incorrectly rejects the null hypothesis.  The Bonferroni Correction controls this rate.
What is a "family"? I'll quote Wikipedia which is in turn quoting Hochberg: "any collection of inferences for which it is meaningful to take into account some combined measure of error".
But you asked for intuitive, so here's an analogy.  You are playing Russian Rolette with a revolver with 1000 chambers.  You fire the gun ten times, you want to control the chance you get shot even once (FWER).  So based on the number of times you pull the trigger (# of hypotheses tested), you set the probability that a given chamber is loaded correspondingly lower (Bonferroni Correction to alpha).
But I think your colleague is on to something.  We find a similar objection in the widely cited "What's Wrong with Bonferroni", Pernegger 1998, under the section "Inference Defies Common Sense".

Bonferroni adjustments imply that a given comparison will be interpreted differently according to how many other tests were performed. For example, the difference in remission rates between two chemotherapeutic treatments could be interpreted as statistically significant or not depending on whether or not survival rates, quality of life scores, and complication rates were also tested. In a clinical setting, a patient's packed cell volume might be abnormally low, except if the doctor also ordered a platelet count, in which case it could be deemed normal. Surely this is absurd, at least within the current scientific paradigm. Evidence in data is what the data say—other considerations, such as how many other tests were performed, are irrelevant.

I'd recommend looking into controlling the False Discovery Rate (FDR) as an alternative to controlling the FWER.  See various work by Hochberg, Benjamini, and others.
Back to the Russian Rolette analogy, the FDR is measuring the probability that you get shot any time that you pull the trigger.  Some folks think this is the more sensible thing to control.  Of course, it depends on the circumstance.  I'd go with the FWER if I were dealing with real guns, or equally dangerous hypotheses.
Comment: That's a good analogy! If I'm playing Russian roulette 10 times, it could still be accurate to say each shot has a 5% chance of killing me, but what I'm really concerned with is being killed at all. If I'm trying to prove that a drug is effective and test 10 outcomes, it's true that for each individual outcome there's 5% chance of the effect being spurious, but the implicit argument I'm making is that the drug is "effective," not that it has any one specific effect.
Comment: Another potential analogy, since I think the rub is really when it comes time to advertise your findings. If I'm a unskilled archer and I shoot at a target and miss, nobody is mistaking me for Robin Hood. If I set up 20 targets side by side on a wall and shoot and hit one, that's still not impressive. If I take the one target I hit down from the wall and showed it off, I'd be misleading the public about my abilities.


Read and print Line between two tags

Question: I stuck into another problem of reading text between two tags through shell script. The file be
**# Enter IP Details, File Name of Machine here.
[SERVER]
TESTTAM
[/SERVER]

[SCRIPTNAME]
hdw.sh
[/SCRIPTNAME]**

and many such tags. Could anyone through some light on how to read lines between tags [SERVER] and [/SERVER] and store in a file. Later I will read this output file with some other script.
I also want to to do the same for other tags.
Answer: Using GNU sed, matching the pattern and printing the only the line without the headers.
sed -n "/\[SERVER\]/,/\[\/SERVER\]/p" file | sed '1d;$d'
TESTTAM

(and)
sed -n "/\[SCRIPTNAME\]/,/\[\/SCRIPTNAME\]/p" file | sed '1d;$d'
hdw.sh

Comment: That works really good..Thanks Inian...by the way, I do I remove the last empty line created after sed operation in the file.
Comment: That works really good..Thanks Inian...by the way, How do I remove the last empty line created after sed operation in the file.
Comment: @Ram: It is not producing an empty line for me, how is it present?
Comment: I already upvoted, but it says I should have more than 15 reputation to so that the upvote will be viewed publicly. By the way, I trying to run the commands in Windows with cygwin.

Read and print Line between two tags


Uniqueness of the valuation for a valuation ring

Question: From Hartshorne I.6:

If $v$ is a valuation, then the set $R = \{x \in K | v(x) \geq 0\} \cup \{0\}$ is a subring of $K$, which we call the valuation ring of $v$... A valuation ring is an integral domain which is the valuation ring of some valuation of its quotient field.

What I am confused about is when someone makes a statement like:

If $R$ is a Noetherian regular local domain of dimension 1, then $R$ is a discrete valuation ring with valuation $v$

Couldn't there be many possible valuations $v$ which give rise to $R$?
For example, if one is talking about a projective variety $X$ and a hypersurface $Y$ with generic point $\eta$ and uses the above fact on $\mathcal{O}_{X,\eta}$ to obtain the valuation $v_Y$ associated to the order of vanishing of a rational function along $Y$, how do we know that there aren't any other valuations taking different values that give back the same ring?
Comment: Abstractly the valuation doesn't need to be real valued, it can take its values in any ordered group. $R$ is a valuation ring iff $\{ a R, a\in Frac(R)^*\}$ is an inclusion-ordered group. Then the map $a\in R \mapsto aR$ is the valuation, which is unique up to isomorphism of the value group.
Answer: EDIT: I just learnt that it might not be a good practice to post images of pages of textbooks on MSE. Sorry: I don't have the time now to OCR everything in this answer (nor I possess the sufficient amount of mathpix snips—which became limited since they turned into freemium software). Either some generous editor might convert the images into text (please delete this paragraph if you do so) or me myself in the future along with some fully free OCR+LaTeX software I'll do it. In the meantime, in-text I referenced the sections in Bourbaki.

I think I prefer Bourbaki's definition to Harsthorne's one for valuation rings (the following is Bourbaki's Commutative Algebra, Ch. VI, §1, no. 1,2):

We will argue below the equivalence between Bourbaki's and Hartshorne's definitions for valuation ring. From now on we will adhere to Bourbaki's one.
Let's explore the relation of valuation rings with valuations. I will let Bourbaki state the definition for a general valuation and the result for valuations on fields. (The following is ibid., § 3, no. 1,2.)

Note that (v) means that $A$ is a valuation ring of $K$. This result means that Hartshorne's definition for valuation ring implies Bourbaki's one (it's not difficult to show that (v) implies $K=\operatorname{Frac}A$).

Hence, if $K$ is a field and $A$ is a valuation ring of $K$, then if a surjective valuation of $K$ whose ring is $A$ were to exist, it would be unique up to unique isomorphism (note that any valuation $v$ on $K$ gives a surjective one by restricting to the image of $v$). Answering your question "how do we know that there aren't any other valuations taking different values that give back the same ring?": if there are, then they will be isomorphic (after restricting to the image of the valuations).
Existence is argued by Bourbaki after Proposition 3:

Indeed, since the ordered group $\{Ax\mid x\in K\}$ (isomorphic to $K^*/U(A)$) contains as an ordered submonoid the monoid of principal ideals in $A$ (call it $P(A)$), we deduce that $K^*/U(A)$ totally ordered implies $P(A)$ totally ordered, i.e., $A$ is a valuation ring of $K$ (Definition 2 above). Conversely, suppose $P(A)$ is totally ordered, and let $x, y \in K$. Write $x=a/b, y=c/d$, where $a,b,c,d\in A$. We have that either $Aad \subset Acb$ or $Aad\supset Acb$. Dividing by $bd$, we get $Ax\subset Ay$ or $Ax\supset Ay$.

Hence, Bourbaki's definition for valuation ring implies Hartshorne's one.
The existence plus uniqueness up to unique isomorphism result entitles us to speak about the (onto) valuation of a valuation ring of a field. On the other hand, to each valuation ring $A$ we find attached an invariant: its value group (by definition, if $v$ is a valuation on $K=\operatorname{Frac}A$ with ring $A$, its value group is $v(K\setminus\{0\})$); it is an totally ordered group uniquely determined by $A$ up to unique isomorphism of ordered groups.
Answer: We can take $v$ as $v(x)=n$  iff $x\in {\frak m}^n\backslash {\frak m}^{n+1}$ for local ring $(R,{\frak m})$.
And in your example, $X$ is 1-dimensional so $Y$ is just one point.


Biased/Unbiased estimation of circular parameters

Question: How could one show, that sample circular mean direction $\bar{\theta}$ is unbiased estimate of (theoretical) circular mean direction $\mu$ ?
Even more, i was looking for precise definition of biased/unbiased estimates of circular parameters but failed to found even one. A lot´s of book just say that this one is unbiased estimate of this and that´s all, without any proper defition. So i was wondering if someone could give me clue how to define it?
Thanks for any help.
Good day.
Answer: The usual distribution on the circle is the Von-Mises Distribution, and if you go to this section they discuss some estimators for the radius and mean along the circle.
The article mostly discusses things as if the circle is defined on the complex plane, but you can map that back to the real plane fairly easily. They talk a little bit about that in the section on the moments.