HTTP request whose headers can be controlled and is automatically decompressed

Question: I'm trying to send HTTP requests in C# that look like HTTP requests from a certain software. I wanted to use System.Net.HttpWebRequest but it doesn't give me the control I need over its headers: their letter-casing can't be changed (e.g. I want the Connection header to be keep-alive and not Keep-Alive), I don't have full control over the headers ordering, etc.
I tried using HttpClient from CodeScales library. Unfortunately, it doesn't decompress responses automatically (see HttpWebRequest.AutomaticDecompression). I decompressed it myself with System.IO.Compression.GZipStream and DeflateStream, but it didn't work when the response had the header Transfer-Encoding: chunked.
System.Net.Http.HttpRequestHeaders seems to give more control over headers than HttpWebRequest, but still not enough.
How can it be done?
Edit: I know that HTTP accepts those headers as valid anyway, but I'm working with a server that validates the headers and refuses to respond if they're not exactly what it expects.
Answer: To set some headers in the HTTPWebRequest class, you have to either use an attribute from the class (for example HttpWebRequest.KeepAlive = true), or you have to add the custom header to the request by calling the add method to the request headers.
Something important is that is you try to add the header (in a custom way) while it's already an attribute of the request, it'll send you an error.
    objRequest.Headers.Add("Accept", "some data");

is incorrect. You'd rather say.
    objRequest.Accept = "some data";

In your case you can :
    objRequest.KeepAlive = true;

Don't worry to much for the letter-casing, it doesn't matter as far as you're sending the appropriate headers to the server.
Comment: see edit: I'm working with a server that validates the headers and refuses to respond if they're not exactly what it expects.
Comment: Your server will look at the content of the header, not the name of the header as far as it's the attribute of the request. Have you tested with the KeepAlive attribute?
Comment: I've tested it with fiddler. It works if the headers are the same as the ones created by the application I'm mimicking and doesn't otherwise, even if the only difference is in their order.
Comment: Hey, I found a great link for you to watch! [About HTTP Headers](http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2). Field name are case insensitive, meaning that using Keep-Alive or keep-alive makes no difference.
Comment: By the way, the order in which headers are received makes no difference either.
Comment: Please reread my edit and my comments. The server I'm working with doesn't use HTTP properly. Either someone implemented HTTP by himself when he wrote it and (s)he didn't do a very good job, or they're using headers to make sure that the request wasn't sent by a bot like the one I'm writing. It wouldn't be the first server that does that.
And as I've told you, I tested it. The only difference was (and I retested it) in the headers.
Comment: What 'control' are you talking about in your initial question? What would you like to do, if not ordering headers (unnecessary), and put some headers in capital letters (unnecessary too)? Let me know I won't let you down until it's solved :D


How do I estimate survival probabilities using datasets that cover different amounts of time?

Question: I'm looking for help classifying a problem that I don't yet have the statistical terminology for, and for help thinking about possible approaches to work on the problem. Below, I give an analogous version of the problem and provide an example of the kind of data I am trying to apply it to.
Analogous problem
Imagine that we have a population of widgets that stop functioning over time. We can't observe the widgets directly but instead need to use experiments with subsets of them to estimate the rate at which the widgets remain functional. The basic experimental approach we take is to collect fully functional widgets and divide them into N trials, each with the same sample size (sampleSize). After some time (say 1 year) we collect the widgets from your experiment and count the number that are still functional. We are unable to track the widgets during the year, and the process of recovering/counting them is destructive so this is the end of this particular 'cohort' of the experiment.
At the same time as we started the first experimental cohort, we started a second cohort with the same number of trials and sample sizes. We leave this cohort for some additional time (say 1 more year, for a total of 2 years) before we collect the widgets from and count the number that are still functional. We are again unable to track the widgets and the recovery/count is destructive.
Based on these 2 cohorts of the experiment, we can estimate the proportion of widgets that remain functional after 1 year ($\theta_1$), and the proportion of widgets that remain functional after 2 years ($\theta_2$). But we are also interested in the survival of widgets from the end of year 1 to the end of year 2. Given the data that might result from these experiments that I describe below, how can I estimate this survival rate?
Two possible outcomes
The first case below makes sense to me; if the number of functional widgets declines through time, the survival to year 2 would be the product of survival to year 1 and survival from year 1 to year 2.
## Case 1

# number of samples
N = 100

# number of trials per sample
sampleSize = 1000

# survival rates
theta_1 = 0.8
theta_2 = 0.64

year_1 = rbinom(n = N, size = sampleSize, prob = theta_1)
year_2 = rbinom(n = N, size = sampleSize, prob = theta_2)

plot(c(1,2),c(mean(year_1),mean(year_2))/1000,xlim=c(0,3),ylim=c(0,1),
     xlab='Year',ylab='Mean proportion survived',pch=16,cex=2)

# survival rate from year 1 to year 2
theta_2/theta_1
(mean(year_2)/100)/(mean(year_1)/100)

The second case is what leads to confusion; if the number of functional widgets is higher in year 2 than in year 1, how can I incorporate both datasets to obtain an estimate for the survival rate from year 1 to year 2?
## Case 2

# number of samples
N = 100

# number of trials per sample
sampleSize = 1000

# survival rates
theta_1 = 0.2
theta_2 = 0.3

year_1 = rbinom(n = N, size = sampleSize, prob = theta_1)
year_2 = rbinom(n = N, size = sampleSize, prob = theta_2)

plot(c(1,2),c(mean(year_1),mean(year_2))/1000,xlim=c(0,3),ylim=c(0,1),
     xlab='Year',ylab='Mean proportion survived',pch=16,cex=2)

# survival rate from year 1 to year 2
theta_2/theta_1
(mean(year_2)/100)/(mean(year_1)/100)

Questions

How can I combine the datasets from experiments covering 1 year, 2 year, etc. if the 2 year dataset does not have data from year 1 because sampling is destructive?
I have not discussed sampling error here but how do I properly combine error from these 2 different datasets when trying to get estimates of survival rate?
Comment: What you have is an interval-censored survival problem (see Kaplan Meier plots and Cox Proportional Hazards models). Not a full answer, but you can leverage later time points for analysis of earlier time points. Suppose you test 100 widgets at Year 1 and find 20 survived (20%), and another 100 at Year 2 and find 30 survived (30%) - you can infer that *at least* 50 widgets of the total 200 (25%) survived to Year 1, since the surviving Year 2 widgets must have survived Year 1. The trend reversal by Year 2 ought to be in the range of statistical noise, since widgets can't come back to life.
Comment: I would say that if your year 2 dataset has a higher rate than year 1, when there is no difference in the study population, these two results are likely to be too far off to be useful. Just my two cents. It's also very hard to get interpretable results from this during survival analysis with only one time point, let alone two trials with two different time points. I'll let someone else answer who may have a suggestion.
Answer: As Nuclear Hoagie says in a comment, "what you have is an interval-censored survival problem," with a strong flavor of discrete-time survival analysis if you only have data for a handful of years. For the cases sampled after 1 year, all you know is that the events occurred between 0 and 1 years while the others had no events through 1 year; for the cases sampled after 2 years, all you know is that the events happened between 0 and 2 years while the others had no events through 2 years; and so on. The event times are interval-censored while the non-event times are right-censored.
A proper interval-censored analysis will pool information from all cases and events. The random sampling variability that would lead to your second case ("widgets can't come back to life," as Nuclear Hoagie says) might mean higher than desired uncertainty in estimates. Nevertheless, you should still get a useful overall analysis of survival, as a function of time and any covariates you include in the model. See this page for an introduction to ways to proceed.


List from a Graph

Question: I have experienced with LightGraphs package in Julia and I have builded a random graph using erdos_renyi(n,p). I would like to know how can I generate a list of edges from this graph. For example:
If I do:
p=erdos_renyi(3,3)
I would like to have a function, which lists me the edges of this graph. That is to say:
 List(p)=[1,2],[2,3],[1,3]]
Thank you so much for your collaboration.
Answer: Normally you should do just:
edges(p)

it will not create you a list of edges, but a lazy edge iterator. The reason why it is preferred is that the number of edges can be very large and it could use a lot of RAM.
If you need to have a vector of edges you can collect this iterator with:
collect(edges(p))

Comment: Just do `adjacency_matrix(p)`. Is this enough for you?
Comment: Thank you so much. Do you know how to create the adjacency matrix from this collection?

List from a Graph


respond_with ArgumentError (Nil location provided. Can't build URI.):

Question: I have a controller that responds_with JSON for all of the RESTful actions, index, create, update etc,
class QuestionsController

 respond_to :json

   def index
     respond_with Question.all
   end

end

However, I also have other actions in the controller. For example, in one method, it checks whether a response was correct and then tries to return a variable with a boolean true or false
respond_with correct_response  #either true or false

However, this is giving me the error
ArgumentError (Nil location provided. Can't build URI.):

There will also be other methods that I'll wish to respond with multiple values. In Sinatra, you can do this to respond with json
{:word => session[:word], :correct_guess => correct_guess, :incorrect_guesses => session[:incorrect_guesses], :win => win}.to_json

How would I do that in Rails?
So, two questions, what's the proper way to write this
respond_with correct_response

and how to respond_with multiple values in a way similar to the example I showed from a Sinatra app.
Thanks for your help.
Answer: You want ActionController::Base#render, not respond_with. The proper way to do what you're trying to achieve here is:
render json: {word: session[:word], correct_guess: correct_guess, incorrect_guesses: session[:incorrect_guesses], win: win}

Comment: It has nothing to do with POST. `respond_with` is built for responding with ActiveRecord objects (ActiveModel really, but that's a different point). You are responding with a boolean, which is not meant to be handled by respond_with. POST has nothing to do with it- you can use respond_with on any type of HTTP action as long as the object you're responding with acts like ActiveRecord.
Comment: It's a shame using `respond_with` in POST calls results in this. I prefer it over `render`. But thanks anyway! :D
Comment: ok, thanks, but if I want to render multiple values, in the same way that that hash is rendered in the example I provided, how do I do it then?
Comment: I also faced the same for the POST request. render solved my issue. Thank you @Chris Cashwell.
Answer: respond_with is actually OK for this scenario--it just happens to do some magic for you and relies on having access to info it needs; take a look at Rails 4.1.9's actionpack/lib/action_controller/metal/responder.rb.
In your case, ArgumentError (Nil location provided. Can't build URI.) is actually telling the truth--it's trying to determine a URL to use from the location header setting but isn't able to figure it out.  I'd wager you could get your code working if you gave it one:
class QuestionsController
  respond_to :json

  def index
    respond_with Question.all, location: questions_url
  end
end


Are Bayesian methods robust to violations of normality?

Question: Consider the simple case
$$x|\sigma^2 \sim N(0,\sigma^2)$$
$$\sigma^2\sim IG(\alpha,\beta)$$
Then, marginally, $f(x) \propto (\beta + x^2/2)^{-\alpha}$, is a t-distribution.
Does this mean that the current model (and generally similar Bayesian models) are robust to violations of normality?
Comment: No, this marginal $t$ distribution is a consequence of integrating out the Normal density against the random $\sigma$, hence has nothing to do with "robustness". When $X$ is not Normal, but one assumes $X$ is Normal as in the question, the Bayesian approach is misspecified and not automatically robust.
Answer: The Bayesian approach is robust only in that few Bayesians would feel right about using the model you have posed, and Bayes makes it relatively easy to use more general models that make fewer assumptions.  The most common and most simple extension to the Gaussian model you posed is to use a $t$ distribution with unknown degrees of freedom $\nu$ for the raw data.  A prior distribution for $\nu$ could favor normality ($\nu > 20$) if you had such knowledge, and as $n \uparrow$, $\nu$ would "follow the data" to allow tails to be as heavy as needed.
To get a sense of how massive a departure this is from the classical approach, textbooks urge students to assess normality and to try different data transformations to achieve this.  The result is falsely narrow confidence intervals and much subjectivity.  Lack of admission of the subjectivity, i.e., not accounting for model uncertainty, is what makes the intervals too narrow.  A real Bayesian approach does not think dichotomously about normality, and when the analysis is finished, instead of asking "what if I am wrong?" you obtain a posterior probability of normality, e.g., $\Pr(\nu > 20)$.
If you do not believe the data distribution to be symmetric, you can use a similar process but based on a skew $t$ distribution.  Bayes encourages us to be honest about what we do not know by including a parameter for what we don't know.  If you only knew that a distribution was continuous you may need 4 parameters (location, scale, skewness, kurtosis).


WooCommerce add Read more button for Product list

Question: I use WooCommerce and in my product lists, the user will link to the product page after they click a picture or title.
I want to add " read more " button right after my short description in the product list, something like read more button for blogs
I found what page I must add PHP code, I also add this code to my functions.php
function read_more() {
    global $product;
    if ($product){
        $url = esc_url($product->get_permalink() );
        echo 'read_more';
    }
}

then I add this code to my product list template :
add_action('woocommerce_after_shop_loop_item_title','read_more');

but nothing adds.
Answer: Add this code in your active theme functions.php file
add_action( 'woocommerce_after_shop_loop_item_title', 'read_more' );

function read_more() {
    global $product;
    if ( $product ) {
        $url = esc_url( $product->get_permalink() );
        echo 'read_more';
    }
}


What is the time complexity when uniformly sampling $b$ entries without replacement from $n$ entries?

Question: Given $n$ entries, I uniformly sample $b$ entries without replacement among them.
My question is that I want a very fast algorithm to do such things, and what is its time complexity? Is the constant in the time complexity $O()$ big?
Comment: this might help http://stackoverflow.com/questions/196017/unique-non-repeating-random-numbers-in-o1#196065
Comment: @spaceisdarkgreen Looking at your link, that answer's coincidentally exactly the one I programmed below, except that he starts at "max" and counts down, whereas I start at zero and count up. And like I said below, I just googled the algorithm and implemented it, so it's perhaps no big coincidence. Maybe this is the absolutely standard technique that anyone googling it will find. I googled it several years back, and don't recall what else google might have coughed up. But I do recall being struck by how efficient it is -- much better than you might naively guess the problem could be solved.
Answer: I'm sure your question has been thoroughly researched and published (did you google it?). I haven't personally needed exactly what you're asking for, but I have needed random permutations of $1\ldots n$, for which I wrote the following little function, based on googling the algorithm it implements...
<code>#include <stdio.h>
#include <stdlib.h>
/* ==========================================================================
 * Function:    int permutation ( int seed, int n, int *p )
 * Purpose:     Returns p[0]...p[n-1] containing 1...n arranged randomly
 * --------------------------------------------------------------------------
 * Arguments:   seed (I)    int optionally containing seed for srandom(),
 *                          if 0 or negative, srandom() not called
 *              n (I)       int containing rank of permutation
 *              p (O)       &int returning 1...n arranged randomly
 * Returns:     ( int )     n=success, 0=failure
 * --------------------------------------------------------------------------
 * Notes:     o
 * ======================================================================= */
/* --- entry point --- */
int permutation ( int seed, int n, int *p ) {
  /* --- Allocations and Declarations --- */
  int   i=0;                /* p[] index */
  /* --- Initialization --- */
  if ( n<1 || p==NULL ) { n=0; goto end_of_job; } /* input error */
  if ( seed > 0 ) srandom(seed);    /* initialize random() */
  for ( i=0; i<n; i++ ) p[i] = i+1; /* initialize p[0...n-1] = 1...n */
  /* ---
   * Generate random permutation
   * ------------------------------ */
  for ( i=0; i<n-1; i++ ) {
    int j = i + ((random())%(n-i)); /* random i <= j <= n-1 */
    int pi = p[i];          /* swap p[i] <--> p[j] */
    p[i] = p[j];            /* p[i] = p[j] */
    p[j] = pi; }            /* p[j] = p[i] */
  end_of_job:
    return ( n );           /* back to caller */
  } /* --- end-of-function permutation() --- */

/* ---
 * Test Driver
 * -------------- */
#if defined(TESTDRIVE)
int main ( int argc, char *argv[] ) {
  int   n    = (argc>1? atoi(argv[1]) : 16 ), /* rank of permutation */
        seed = (argc>2? atoi(argv[2]) : 7654321 ); /* srandom() seed */
  int   permutation(), p[9999],     /* random permutation of 1...n */
        i=0;                        /* p[] index */
  if(n<0)n=16;  if(n>999)n=999;     /* sanity check */
  permutation(seed,n,p);            /* generate permutation */
  for ( i=0; i<n; i++ )             /* print results */
    printf("%5d%s",p[i],((i+1)%10==0||i==n-1?"\n":", "));
  } /* --- end-of-function main() --- */
#endif
</code>
Now, I think you'd just want the first $b<n$ entries, taking $p[0]\ldots p[b-1]$ as your random sample without replacement. And in that case, I think you could just stop the loop at $b-1$ (instead of at $n-1$ for the entire permutation). And then the complexity's clearly just linear in $b$, which I suppose is pretty much as good as you can hope for.


Python Decrement Dictionary

Question: can I decrement a value in a dictionary ? I have a dict with string key and int value, how to decrement all value's key ?
d={'A': 0, 'B' : 1, 'C' : 2, 'D' : 3}
d.pop('B')
##How to decrement values after 'B' ?

Comment: What exactly do you mean by "After 'B'"? What version of python is this?
Comment: I have a dict where keys are name , and values are ID ...  Maybe it's better to reverse things?
Comment: "Increment all values after B", dictionaries are unsorted in some versions of python.  How do we know which values come after B?
Comment: Version is 3.6.5 , when I delete 'B' , I have to decrement  'C' and 'D'
Comment: Do you want to decrement the keys that are _alphabetically_ after "B"?
Comment: No, I want decrement value of alphabetical key
Comment: Well we don't really know what you're trying to achieve. Keys must be immutable so it's unlikely that you want to reverse the dictionary.
Answer: Assuming that 'after B' means the keys lexicographically following 'B' in this case, this snippet should work:
d={'A': 0, 'B' : 1, 'C' : 2, 'D' : 3}
d.pop('B')
for k in d.keys():
    if k > 'B':
        d[k] -= 1

Comment: `for k in d` iterates over the keys.
Comment: Also, any reason to use `dict.pop` instead of `del d['B']`? We aren't storing the value returned.
Comment: I thought this might be a little clearer for this instance but you're right DyZ
Comment: Only that I copied the first two lines from the question.
Answer: d={'A': 0, 'B' : 1, 'C' : 2, 'D' : 3}
e={'dfg': 3, 'dec': 2, 'aaa': 0, 'cbbb': 1}

def remElem(l, i):
    del l[i]
    [l.update({k: v-1}) for k, v in l.items() if k > i]

print(d)
remElem(d, 'B')
print(d)

print(e)
remElem(e, 'cbbb')
print(e)

Gives:
{'A': 0, 'C': 2, 'B': 1, 'D': 3}
{'A': 0, 'C': 1, 'D': 2}

{'dfg': 3, 'dec': 2, 'aaa': 0, 'cbbb': 1}
{'dfg': 2, 'dec': 1, 'aaa': 0}

Comment: Why iterate over items if you don't use the value? Surely better to just iterate over keys?
Comment: It works ! But wait, In dictionary I have names not single character .. `if  k > i: ` decrement in alphabetic way ?
Comment: see extra example

Python Decrement Dictionary


Wireless clients getting ips my router is not assigning

Question: I'm running a small network with a peplink router and unifi access points. The wireless network is a vlan than has no access to the main network.
For some reason that I can not figure out, some random devices are getting a different subnet of ips assigned to them.
The only thing I can figure is that something is connecting wirelessly to the network and acting as a dhcp server. But how do I find it if changing the password is not an option?
Answer:
Put one or more clients on the network.
Start up a packet capture on them.
Perform DHCP requests and observe the IP of the DHCP server.
Repeat from various locations on the network until you have seen a reply from an unexpected IP.
Once you have the IP, you can get the MAC of the offending system and then track it down using the CAM tables (or equivalent) of your switch and wireless gear.

Note: Consider enabling DHCP snooping on your switches - with this you can ensure that only authorized servers can respond to DHCP requests.


How to tune the following query?

Question: This query gives me the desired result but i can't run this query every time.The 2 loops is costing me.So i need to implement something like view.But the logic has temp tables involved which isn't allowed in views as well.so, is there any other way to store this result or change the query so that it will cost me less.
DECLARE @Temp TABLE (
    [SiteID] VARCHAR(100)
    ,[StructureID] INT
    ,[row] DECIMAL(4, 2)
    ,[col] DECIMAL(4, 2)
    )
DECLARE @siteID VARCHAR(100)
    ,@structureID INT
    ,@struct_row INT
    ,@struct_col INT
    ,@rows_count INT
    ,@cols_count INT
    ,@row INT
    ,@col INT

DECLARE structure_cursor CURSOR
FOR
SELECT StructureID
    ,SiteID
    ,Cols / 8.5 AS Cols
    ,Rows / 11 AS Rows
FROM Structure
WHERE SellerID = 658 --AND StructureID = 55

OPEN structure_cursor

FETCH NEXT
FROM structure_cursor
INTO @structureID
    ,@siteID
    ,@struct_col
    ,@struct_row

SELECT @rows_count = 1
    ,@cols_count = 1
    ,@row = 1
    ,@col = 1

WHILE @@FETCH_STATUS = 0
BEGIN
    WHILE @row <= @struct_row
    BEGIN
        WHILE @col <= @struct_col
        BEGIN
            --PRINT 'MEssage';
            INSERT INTO @Temp (
                SiteID
                ,StructureID
                ,row
                ,col
                )
            VALUES (
                @siteID
                ,@structureID
                ,@rows_count
                ,@cols_count
                )

            SET @cols_count = @cols_count + 1;
            SET @col = @col + 1;
        END

        SET @cols_count = 1;
        SET @col = 1;
        SET @rows_count = @rows_count + 1;
        SET @row = @row + 1;
    END

    SET @row = 1;
    SET @col = 1;
    SET @rows_count = 1;

    FETCH NEXT
    FROM structure_cursor
    INTO @structureID
        ,@siteID
        ,@struct_col
        ,@struct_row
END

CLOSE structure_cursor;

DEALLOCATE structure_cursor;

SELECT * FROM @Temp

Comment: Please post sample data and expected output.
Comment: i am actually converting the table 1 as in image above to table 2.So that i can use the second table further in my query to join and get the output
Answer: Do this with a set-based operation.  I think you just want insert . . . select:
INSERT INTO @Temp (SiteID, StructureID, row, col)
    SELECT StructureID, SiteID, Cols / 8.5 AS Cols, Rows / 11 AS Rows
    FROM Structure
    WHERE SellerID = 658;

You should avoid cursors, unless you really need them for some reason (such as calling a stored procedure or using dynamic SQL on each row).
EDIT:
Reading the logic, it looks like you want to insert rows for based on the limits in each row.  You still don't want to use a cursor.  For that, you need a number generator and master..spt_values is a convenient one, if it has enough rows.  So:
with n as (
      select row_number() over (order by (select null)) as n
      from master..spt_values
     )
INSERT INTO @Temp (SiteID, StructureID, row, col)
    SELECT StructureID, SiteID, ncol.n / 8.5 AS Cols, nrow.n / 11 AS Rows
    FROM Structure s JOIN
         n ncol
         ON ncol.n <= s.struct_col CROSS JOIN
         n nrow
         ON nrow <= s.struct_row
    WHERE SellerID = 658;

Answer: You can generate the number of rows and columns and then CROSS APPLY with those, like below. I've left out your SellerID condition.
;WITH Cols
AS
(
    SELECT StructureID, SiteID, CAST(Cols / 8.5 AS INT) AS Col
    FROM Structure
    UNION ALL
    SELECT s.StructureID, s.SiteID, Col - 1
    FROM Structure s
        INNER JOIN Cols c ON s.StructureID = c.StructureID AND s.SiteID = c.SiteID
    WHERE Col > 1
)
, Rows
AS
(
    SELECT StructureID, SiteID, CAST(Rows / 11 AS INT) AS Row
    FROM Structure
    UNION ALL
    SELECT s.StructureID, s.SiteID, Row - 1
    FROM Structure s
        INNER JOIN Rows r ON s.StructureID = r.StructureID AND s.SiteID = r.SiteID
    WHERE Row > 1
)
--INSERT INTO @Temp (SiteID, StructureID, row, col)
SELECT s.SiteID, s.StructureID, r.Row, c.Col
FROM Structure s
    CROSS APPLY Cols c
    CROSS APPLY Rows r
WHERE s.StructureID = c.StructureID AND s.SiteID = c.SiteID
    AND s.StructureID = r.StructureID AND s.SiteID = r.SiteID

Comment: That's a recursive CTE. You set in the anchor the highest number for col/row (Rows/11 and Cols/8.5), then in the recursive member you keep on subtracting until you get to 1.
Comment: Thanks!! this helped me to create view as well..can you elaborate how can you do this Col - 1 in CTE above ..
Answer: We can do this by using CROSS APPLY and CTE.
CREATE TABLE Structure(SiteID varchar(20), StructureID int,
Cols decimal(18,2), [Rows] decimal(18,2))

INSERT INTO Structure (SiteID, StructureID, Cols, [Rows])

VALUES

('MN353970', 51,17,22),
('MN272252', 52,17,11)

;WITH RowCTE([Rows]) AS
(
    SELECT 1
    UNION ALL
    SELECT 2
),
ColCTE(Cols) AS
(
    SELECT 1
    UNION ALL
    SELECT 2
)

SELECT SiteID, StructureID, R.Rows, C.Cols
FROM Structure s
CROSS APPLY
    (
        SELECT Cols FROM ColCTE
    ) C
CROSS APPLY
    (
        SELECT [Rows] FROM RowCTE
    ) R

Sql Fiddle Demo
Comment: Hi selva..i looked at your code..Logic seems ok but looking at siteID MN272252..it should just create 2 rows not 4 rows because its ,Cols / 8.5 AS Cols and  Rows / 11 AS Rows.Which means 2x1 should be the output.The output should be dynamic.

How to tune the following query?


Why the joint hypothesis (F-test) cannot be substituted by multiple individual hypothesis (T-test)

Question: In the book Basic Econometrics (Page254), the author writes:

In testing the significance of β2_hat under the null hypothesis, it was assumed tacitly that the testing was based on a different sample from the one used in testing the significance of β3_hat under the null hypothesis that  β3 = 0.

And the author said that is the reason why the F-test cannot be substituted by multiple t-tests since any single hypothesis is 'affected' by the information in the other hypothesis.
However, I am still not sure about what does it mean by tests are independent of each other. Does it assume that there is no multicollinearity in the variables so that the result of one test does not affect the result of the other one?
May someone gives an example of how the test can be dependent of each other and explain why exactly the joint hypothesis cannot be substituded by multiple individual hypothesis? Thanks!
Answer: When the $t$-tests are performed, they assume that the other variables are already in the model.
For example, suppose you were building a model where the dependent variable was the weight of a book, and the independent variables were $x_2$ (the number of pages in the book) and $x_3$ (the thickness of the book).
If you fit a model with both of these variables, and did t-tests for their coefficients, it's possible that you would get high p-values for both, because there is collinearity.  The number of pages in a book is highly correlated with the thickness of a book.
So when you do a t-test to see if $x_2$ is needed in the model (in other words, if $B_2 = 0$), you may fail to reject, which makes sense because $x_3$ is already providing the information that $x_2$ would provide.  And when you do a t-test to see if $x_3$ is needed in the model (in other words, if $B_3 = 0$), you may fail to reject as well, because $x_3$ is already providing the information that $x_2$ would provide.
However, that does not mean that not mean that it's likely that you don't need either variable in the model.  You should probably have one of those variables in the model, since how thick the book is (whether measured in inches or in the number of pages) will definitely affect how much the book weighs.  So even though you may fail to reject $B_2 = 0$ and $B_3 = 0$ independently, it's unlikely you would fail to reject the test $B_2 = B_3 = 0$, since one of those variables should be in your model.
Comment: Thanks! Is it saying that the multicollinearity problem will not cause an increase of type 2 error rate for the joint hypothesis?


Moderator analysis in multilevel meta-analysis of dependent effect sizes

Question: I'm conducting a meta-analysis with a data set that is somewhat similar in structure to this post, where there are multiple effect sizes per study. Specifically, most studies I'm analyzing contain multiple experiments, and the effect sizes from these experiments are dependent (i.e., are derived from the same participants). Here's a depiction of what a small part the data looks like:
esid  studyid  sampleid  testtype
   1        1         1         a
   2        2         2         b
   3        2         3         b
   4        3         4         a
   5        3         4         a
   6        3         4         b

esid = effect size identifier
studyid = study identifier (same number = effect size from same study)
sampleid = sample (correlated effect) identifier (same number = effect size from same participants)

I'm planning on using metafor to fit a three-level model (similar to Konstantopoulos, 2011), including a random effect at the studyid level (i.e., paper level) since effect sizes are nested within studies:
ml.mod <- rma.mv(yi, vi, random = ~1 | studyid/esid, data = dat)

Then, using the robust function to account for the dependency between some effect sizes (clustering at the sampleid level):
robust(ml.mod, cluster = dat$sampleid, adjust=TRUE)

As far as I can tell, this is basically what Dr. Viechtbauer suggests here. Assuming this approach looks reasonable to deal with this data structure (and please, let me know if it doesn't!), my question has to do with how to best investigate whether testtype influences the estimate. Because testtype varies both within- and between-studies (i.e., some studies use both testtype a and b - and these estimates are dependent - others only a, others only b), is there a best approach to dealing with this question? For example, would it be reasonable to directly compare the testtypes:
ml.mod <- rma.mv(yi, vi, mods = cbind(testtype), random = ~1 | studyid/esid, data = dat)
robust(ml.pub, cluster = dat$sampleid, adjust = TRUE)

Or would it be more appropriate to conduct separate meta-analyses looking at the effect when using testtype a and then separately using testtype b (and not directly compare them):
ml.mod.a <- rma.mv(yi, vi, random = ~1 | studyid/esid, data = dat, subset=(testtype=="a")
robust(ml.mod.a, cluster = dat$sampleid, adjust = TRUE)

ml.mod.b <- rma.mv(yi, vi, random = ~1 | studyid/esid, data = dat, subset=(testtype=="b")
robust(ml.mod.b, cluster = dat$sampleid, adjust = TRUE)

Any suggestions would be extremely appreciated by this newbie! Thank you!
Answer: With respect to the clustering:
For the cluster-robust method, you should not cluster at the sampleid level, as that would treat the second and third effect as independent, even though they come from the same study (although different samples). You should cluster at the outermost grouping level, which would be studyid. So use:
robust(ml.mod, cluster = dat$studyid, adjust=TRUE)

As for the main question:
The standard approach would be to just include testtype as a moderator. But please use the formula syntax, so use:
ml.mod <- rma.mv(yi, vi, mods = ~ testtype, random = ~1 | studyid/esid, data = dat)

As you correctly point out, you have a mix of between- and within-study evidence about the difference between levels a and b of the testtype moderator. Generally speaking, between-study evidence is weaker, since it is more likely to be confounded by other variables that vary between studies. Therefore, besides the model above, I would suggest to also examine what happens when you only include studies that provide within-study evidence. So, create a dataset that is a subset of dat that only includes those studies that include both levels of testtype. Then compare the results and see if conclusions are consistent.


ASP.NET C# (3.5) Membership, How to get Approved Users only?

Question: I need a list of "Approved" Users only, but don't know how to figure it out.
lstUsersNotInRole.DataSource = Membership.GetAllUsers(); // Get Approved Users Only!?!
lstUsersNotInRole.DataBind();

Any kind help would be highly appreciated.
Kardo
Answer: Instead of delving into the database and altering/creating stored procedures (and since you didn't specify which database you are using), you can do this by using Linq. Try this:
Membership.GetAllUsers().Cast<MembershipUser>().Where(u => u.IsApproved == true).ToList();

Comment: Thanks for your so perfect and simple answer :-)
Answer: You need to filter the approved users in your stored procedure and return the result to your application.
CREATE PROCEDURE dbo.Membership_GetActiveUsersInRoles

    @RoleName nvarchar(256)

AS

    BEGIN

        DECLARE @RoleId uniqueidentifier

        SELECT @RoleId = RoleId
        FROM dbo.aspnet_Roles
        WHERE LOWER(@RoleName) = LoweredRoleName

        SELECT u.UserName, u.UserId
        FROM dbo.aspnet_Users u, dbo.aspnet_UsersInRoles ur, dbo.aspnet_Membership m
        WHERE
        u.UserId = m.UserId AND
        u.UserId = ur.UserId AND
        @RoleId = ur.RoleId and
        m.IsApproved = 'true'
         -- You can also specify m.IsLockedOut = 'false' for other purposes

END

Answer: If you use the SqlMembershipProvider, you could select them manually, either as plain sql-statement(e.g. via ADO.NET) or by creating a custom stored-procedure which  also supports paging and the ApplicationId and where you can parameterize IsApproved.
SELECT  m.IsApproved, u.UserName,
        m.Email, m.PasswordQuestion, m.Comment,
        m.CreateDate, m.LastLoginDate, u.LastActivityDate,
        m.LastPasswordChangedDate, u.UserId,
        m.IsLockedOut, m.LastLockoutDate
FROM   dbo.aspnet_Membership m, dbo.aspnet_Users u
WHERE  u.UserId = m.UserId
AND    m.IsApproved = @IsApproved
ORDER BY u.UserName

Just have  a look at dbo.aspnet_Membership_GetAllUsers, copy paste the relevant parts and modify it accordingly.


Submeter formulário no python

Question: Estou tentando submeter um formulário no site https://www.loskatchorros.com.br/ucp/login.php ,porém já procurei diversas maneiras aqui no stackoverflow de como submeter um formulário, e nao achei nenhuma maneira que solucionasse este problema.
Este é um exemplo de código que peguei aqui no fórum e não funcionou.
import requests
headers = {'User-Agent': 'Mozilla/5.0'}
payload = {'login':'*******','senha':'******'}

session = requests.Session()
r = session.post('https://www.loskatchorros.com.br/ucp/login.php',headers=headers,data=payload)

Ao colocar a linha:
print (r.text)

O programa retorna o mesmo código-fonte da página https://www.loskatchorros.com.br/ucp/login.php, isso da a entender que o formulário não foi submetido com exito, e queria saber o que estou fazendo de errado.
Se precisarem do login e senha verdadeiros peçam que eu mando sem nenhum problema
Answer: import requests

account = {'login': 'email@example.com', 'senha': 'muze123'}
abs_domain = 'https://www.loskatchorros.com.br/' #1
form = 'ucp/global/verifica_login.php' #2
absolute_form_URL = abs_domain + form

requests_session = requests.Session() #3
r = requests_session.post(url=absolute_form_URL, data=account) #4

text = r.text[-79:] if r.status_code == 200 else None #5
print('Status code [%s].' % r.status_code, text, sep='\n')

#1 Domínio absoluto
#2 Caminho para o form de autenticação
#3 "Instância a sessão"
#4 Faz o método POST com o objeto session
#5 Obtém uma fatia do texto recebido caso a autenticação seja bem sucessida

>>> Status code [200].
>>> Usuário ou senha incorretos, verifique e tente novamente!

Comment: Você fornecendo suas credencias aqui, é muito provável que apaguem sua conta. Tente utilizar Selenium nesses casos de redirecionamento. Me parece que ele está tentando abrir uma janela com cloudfire.
Answer: Você tem que enviar o post com os dados a página de destino do formulário, não para  a página que gera o formuláriom em HTML.
A URL para essa página é a que está no campo "action" da tag form. Nesse caso, global/verifica_login.php (e não login.php).
Em frameworks modernos web em geral a URL que gera o html de um formulário e o que processa o formulário é o mesmo - e isso facilita a criação da interface de validação de dados. Mas o autor de um site pode por os endereços que quiser, e se for um sistema feito a mão, sem estar aproveitando código de um framework, é mais simples ter páginas separadas.

Submeter formulário no python


Manage content items using Views

Question: I'm creating a user-upload photo gallery, and I require admin approval on all user uploads. Photos will be a content type with just a few fields (Image, Taxonomy, Tags). I'd like to be able to view a grid of all uploaded photos at a glance, and then check a box next to each one and publish/unpublish all that are selected. Is there a method by which I an do this using Views? If not, is there another method/module that can provide this type of functionality?
Thanks!
Answer: Create a content view, filter by your gallery content type, add the image field, add the field  "Content: Published" (to let you know if content is published). Then add  "Bulk operations: Content" (you need to install Views Bulk Operations module). In the field settings you should select the option "Publish content" and "Unpublish content". You should now be able to bulk publish and unpublish the image galleries.

Manage content items using Views


Image not fitting properly into Circular Image View

Question: I've a rectangular image, which I want to display into a circular border image view:
I made a drawable like this :
<?xml version="1.0" encoding="utf-8"?>
<shape xmlns:android="http://schemas.android.com/apk/res/android"
    android:shape="rectangle">
    <corners android:radius="100dp" />
    <solid android:color="#41ba7a" />
    <stroke
        android:width="3dip"
        android:color="#f9f9f9" />
    <padding
        android:bottom="4dp"
        android:left="4dp"
        android:right="4dp"
        android:top="4dp" />
</shape>

I'm using it inside a ImageButton :
<ImageButton
                android:id="@+id/service_image_on_orderpage"
                android:layout_width="120dp"
                android:layout_height="120dp"
                android:background="@drawable/profile_circular_border_imageview"
                android:elevation="5dp"
                android:padding="20dp"
                android:scaleType="centerCrop"
                android:layout_marginBottom="20dp"/>

This looks like this :

But when I load the image into it then it appears like this :

It is not completely covering the ImageButton. I want that the picture loaded should cover the ImageButton in circular way.
I've tried changing all possible options of android:scaleType, but nothing suits the requirement, I even have tried changing the ImageButton into ImageView but still the problem persists.
Please help.
Comment: Has already been answered here: http://stackoverflow.com/q/39423549/6816052
Comment: Use This Library https://github.com/hdodenhof/CircleImageView
Comment: @NikunjParadva : can't use this, because I've to change ImageButton at many places in my project.
Comment: @Amit Upadhyay Instead of `ImageButton` take `ImageView` as `Nikunj` say and then set it.
Comment: @AmitUpadhyay I think Circular Imagebutton is not Possible but Use Imageview I hope This  Library is works for you
Comment: @Ironman : I've tried using ImageView also(mentioned in second last line of question), but the result is same.
Comment: @AmitUpadhyay Use those library n then check https://github.com/hdodenhof/CircleImageView
Comment: @AmitUpadhyay post that code which `Nikunj` has mention..and make sure that you have add dependency for that.
Comment: @AmitUpadhyay Your Problem is solved or not?
Answer: use this code
  <ImageButton
    android:id="@+id/service_image_on_orderpage"
    android:layout_width="120dp"
    android:layout_height="120dp"
    android:background="@drawable/profile_circular_border_imageview"
    android:elevation="5dp"
    android:padding="20dp"
    android:scaleType="fitCenter"
    android:layout_marginBottom="20dp"/>

and after this also if you are getting same problem change the scaleType to another.
Comment: as mentioned in the question I've tried every possible options for `scaleType` and nothing seems to help
Comment: to load image what are you using ? android:src or glide?
Answer: Hey @Amit Upadhyay you should try this awesome library for a fast circular ImageView perfect for profile images.This is based on RoundedImageView from Vince Mi which itself is based on techniques recommended by Romain Guy
For using this library you need to add dependency in your gradle like below:
dependencies {
...
compile 'de.hdodenhof:circleimageview:2.1.0'
}

and for its usage add the widget in your layout xml file like below:
<de.hdodenhof.circleimageview.CircleImageView
xmlns:app="http://schemas.android.com/apk/res-auto"
android:id="@+id/profile_image"
android:layout_width="96dp"
android:layout_height="96dp"
android:src="@drawable/profile"
app:civ_border_width="2dp"
app:civ_border_color="#FF000000"/>

this will help you for sure!!!


Adicionar Animate ou classes ao rolar a página e chegar em div

Question: Estou trabalhando em um projeto em Vue e preciso que quando o usuário rolar a página e chegar em uma especificada div, seja adicionado um efeito ou uma classe nessa div, semelhante ao ScrollReveal
Alguém já precisou fazer algo semelhante no Vue?
Segue o código css do que eu possuo e que preciso que seja executado ao chegar em div
O efeito do título em h2 deve ser executado quando ele é exibido na tela.

@keyframes slideInFromLeft {
  0% {
    transform: translateX(-100%);
  }
  100% {
    transform: translateX(0);
  }
}

h1 {
  animation: 1s ease-out 0s 1 slideInFromLeft;
}

h2 {
  animation: 1s ease-out 0s 1 slideInFromLeft;
}

Hello World!

World Hello!

Answer: Você precisa usar Javascript para detectar a localização do elemento na tela e ativar a animação css quando ele estiver visível.
A seguir um passo a passo no qual você pode se basear para conseguir o resultado que você quer.
1. Crie o elemento HTML que você que verificar se está na tela (no seu caso o h2).
2. Carregue os arquivos jQuery JS e JQuery Viewport Checker.

3. Chame o plugin no seu h2

Espero que ajude em algum sentido.
Aconselho colocar alguma classe ou id nesse h2 para que fique mais facil identifica-lo.
Abraço.


Laplace Transform of Squared Function

Question: I have the following differential equation that I am trying to solve:
$$g(u,v,t) = a + bg(u,v,t) - c\frac{dg(u,v,t)}{dt} - k\times g(u,v,t)\frac{\partial(g(u,v,t))}{\partial t},$$
where $a,b,c,k$ are constants, and the variables $u$ and $v$ are independent of $t$.
I replaced $g(u,v,t)\frac{\partial(g(u,v,t))}{\partial t}$ with $\frac{1}{2}\frac{d(g(u,v,t))^2}{dt}$ in the last term on the RHS.
My initial idea was to compute the Laplace transform on both sides:
$$G(u,v,s) = \frac{a}{s} + bG(u,v,s) - csG(u,v,s) - k/2\times s\mathcal{L}(g^2(u,v,t)), $$
where $s$ is the Laplace transform variable and $G(s)$ is the Laplace transform of $g(u,v,t)$.
But now I cannot solve for G because of the $\mathcal{L}(g^2)$ term.
I am stuck now and will appreciate any ideas to proceed further.
Comment: Laplace transforms are usually useful for linear differential equations.  The $ff'$ term means the equation is not linear. Also, I propose that you streamline the equation ( since the $u,v$ are not used) to something like $a+bg+cg' +d(gg')=0$. In this form you may be able to find a solution on the web
Answer: This equation can be rewritten in the form
$$
\frac{dg}{dt}=\frac{(b-1)g+a}{kg+c}.
$$
This is a first order differential equation that can be solved by quadratures. Can you go on from here?
Answer: Laplace transform generally doesn't help with nonlinear equations.
Since $u$ and $v$ don't appear anywhere except in $g(u,v,t)$, you might as well get rid of them and just write $g(t)$.
This is a separable differential equation.

Laplace Transform of Squared Function


Angular 5 animate combined with *ngFor

Question: I am attempting to animate a number of cards that are based on an array of items. The number is dynamic. I have put together some code that animates but it animates all of them. I cannot see how to animate each one individually.
Here is my code so far:
import { Component, OnInit, trigger, state, style, transition, animate } from '@angular/core';

@Component({
  selector: 'app-root',
  template: `

{{item.item}} Front

{{item.item}} Back

  `,
  styles: [
    `
    .tp-box {
      position: relative; width: 200px; height: 100px; margin: 3rem auto;
      transform-style: preserve-3d; transition: transform 1s;
    }
    .tp-box__side {
      width: 100%; height: 100%; position: absolute; backface-visibility: hidden;color: #fff;
      text-align: center; line-height: 100px; font-size: 24px; font-weight: 700; cursor: pointer; user-select: none;
    }
    .tp-box__front {
      background: #f30d36;
    }
    .tp-box__back {
      background: #23262d;
      transform: rotateY(179.9deg);
    }

    `
  ],
  animations: [
    trigger('flipState', [
      state('active', style({
        transform: 'rotateY(179.9deg)'
      })),
      state('inactive', style({
        transform: 'rotateY(0)'
      })),
      transition('active => inactive', animate('500ms ease-out')),
      transition('inactive => active', animate('500ms ease-in'))
    ])
  ]
})
export class AppComponent {

  flip: string = 'inactive';
  constructor() {}

  items: any[] = [
    { "item": "Item 1"},
    { "item": "Item 2"},
    { "item": "Item 3"},
    { "item": "Item 4"},
    { "item": "Item 5"}
  ];

  toggleFlip() {
    this.flip = (this.flip == 'inactive') ? 'active' : 'inactive';
  }

}

Clearly each time the user clicks on the box it sets the flip property from active to inactive and back again. The problem is that the property is not specific to one box and I am uncertain how to get the animation specific to one box.
Here is a plunkr
Answer: You can add a property to the item itself and in the toggleFlip function, and send the item as an argument. something like this :
template: `

{{item.item}} Front

{{item.item}} Back

  `,

TS :
items: any[] = [
    { "item": "Item 1", 'flip' : 'inactive'},
    { "item": "Item 2", 'flip' : 'inactive'},
    { "item": "Item 3", 'flip' : 'inactive'},
    { "item": "Item 4", 'flip' : 'inactive'},
    { "item": "Item 5", 'flip' : 'inactive'}
  ];

  toggleFlip(item) {
    item.flip = (item.flip == 'inactive') ? 'active' : 'inactive';
  }

here is a working plunkr
Comment: Thank you! So simple when you know how.
Answer: I had a similar problem, I had a switch inside an ngFor, which had a css transition. Whenever I clicked the switch, the whole ngFor would re-render and no transition would occur, I used the track-by directive to help with that problem see article here.
I looked for a solution about animations in ngFor for hours, but found not answer on stackoverflow, I hope this saves someone alot of time!

Angular 5 animate combined with *ngFor


Distributed leftover bandwidth testing

Question: Faced with the task of testing the leftover bandwidth capacity for 100+ locations multiple times a day, I asked myself if there is some sort of software package you can install on these computers and have them make upload/download tests, then report back to a central server automatically. Does anyone know of a solution that can do that? Bonus points for a solution that runs on Windows and doesn't require me to install extra runtimes.
Answer: I was going to provide a rather long and involved answer, but reading your question again makes me think that your interpretation of "bandwidth" testing is radically different from mine.
There is no tool available which does not require you to "install extra runtimes".
Perhaps if you gave a bit more information about how you currently test and what you think it tells you might be helpful.
Certainly I'd recommend strongly that you move away from active testing and use passive testing instead.
Do have a look at Network Vantage, pastmon, mrtg and Scutinizer
C.
Comment: Of course the reason to monitor the bandwidth is a preparation period just to estimate average leftover bandwidth availability in already crowded networks. It would help admins at the respective sites to make informed decisions whether they have to upgrade their connection lines . Passive testing is thus not an option, as estimates have to rely on real life down/upload performance. Thanks for the suggested tools, I will check them out immediately.
Comment: If you want to measure actual bandwidth usage then host based metrics are a waste of time and the **only** way you should be monitoring these is using passive monitors (i.e. network sniffing) or measuring from routers/switches directly.
Comment: Exactly what symcbean said. How can you test "leftover" bandwidth by measuring a new download running on a client machine? Run something like MRTG against all of your gateway devices, and you'll know exactly how much bandwidth you're using. Bonus points for making the graphs to scale against your provider's bandwidth you've purchased, and not the max speed of your router's physical WAN interface.

Distributed leftover bandwidth testing


Making a custom form element AJAX aware

Question: How should I go about making my custom Drupal 7 Form element support AJAX events, just like other standard FAPI elements?
For example, with a drop-down list, I would do this:
$element['book'] = array(
  '#type' => 'select',
  '#title' => t('Book'),
  '#options' => $books,
  '#default_value' => $default_book,
  '#ajax' => array(
    'wrapper' => 'replace-me',
    'callback' => 'replacement-callback',
    'method' => 'replace',
  ),
);

But now that '#type' is my own custom element, the '#ajax' property is ignored. This is what I have implemented thus far:
function mymodule_element_info() {
  return array(
    'customelement' => array(
      '#input'          => TRUE,
      '#process'        => array('customelement_process'),
      '#theme'          => array('customelement'),
      '#theme_wrappers' => array('form_element'),
      '#tree'           => TRUE,
    ),
  );
}

function mymodule_theme($existing, $type, $theme, $path) {
  return array(
    'customelement' => array(
      'render element' => 'element',
    ),
  );
}

function theme_customelement($variables) {
  // render some elements
  return $output;
}

function form_type_customelement_value($element, $input = FALSE, &$form_state) {
  // manipulate form values
  return $input
}

function customelement_process($element, &$form_state, $complete_form) {

  // build the compound form array
  $element['#tree'] = TRUE;

  $element['book'] = array(
    '#type' => 'select',
    '#title' => t('Book'),
    '#options' => $books,
    '#required' => true,
  );

  $element['chapter'] = array(
    '#type' => 'textfield',
    '#title' => t('Chapter'),
    '#size' => 3,
    '#maxlength' => 3,
    '#required' => true,
  );

  return $element;

}

Using my custom element in a form goes like this:
function mymodule_menu() {
  $items = array();

  $items['path/to/page'] = array(
    'title'            => 'Test form',
    'type'             => MENU_LOCAL_TASK,
    'weight'           => 30,
    'page callback'    => 'drupal_get_form',
    'page arguments'   => array('mymodule_callback_testform'),
    'access callback'  => 'user_access',
    'access arguments' => array('mymodule_admin'),
  );

  return $items;
}

function mymodule_callback_testform() {
  $form = array();

  $form['test_element'] = array(
    '#type' => 'customelement',
    '#title' => 'Custom element test',
    '#required' => TRUE,
    '#ajax' => array(
      ... // this has no effect :(
    ),
  );

  return $form;
}

But unfortunately it doesn't "magically" support AJAX. I would like to trigger a Drupal AJAX event when any sub-element of my custom element has triggered certain events.
Answer: It has no effect because Drupal doesn't know what event to fire on.
If you look at the switch statement in ajax_pre_render_element() you can see that it has default events for all core element types. But it it doesn't know about your element type, and no event is provided, so it doesn't know how to continue. It returns the element as-is rather than attempt to make a decision its not qualified to make.
To fix you should just be able to pick an appropriate event for the code to fire on, e.g.
$form['test_element'] = array(
  '#type' => 'customelement',
  '#title' => 'Custom element test',
  '#required' => TRUE,
  '#ajax' => array(
    'event' => 'click',
    ...
  ),
);

Comment: This seemed like such an obvious solution that I accepted the answer before testing. But it still does not seem to trigger any AJAX activity.
Comment: @rudolfbyker Hmmm, I confess I didn't test this so there must be more to it. You can un-accept the answer, that way people will know you're still looking for a solution. I'll try to come back to this later and have another look, bit busy today
Comment: I'm curious if you got anywhere with this?  If so, care to share?  I'm struggling with the same issue.
Comment: Nope. This is still in my issue queue.
Answer: The custom element in the example is not a direct form element and act more as a wrapper (like a fieldset or item element) to create a compound form field. It's the same thing as putting an #ajax key on a fieldset, which does not make any sense for the form api. So, assuming that the trigger would be the select list, the only thing you need to do for it to work is to transfer the #ajax definition to the inside book element in the process function like this :
function customelement_process($element, &$form_state, $complete_form) {

  // build the compound form array
  $element['#tree'] = TRUE;

  $element['book'] = array(
    '#type' => 'select',
    '#title' => t('Book'),
    '#options' => array(1 => 1, 2 => 2),
    '#required' => true,
  );

  //If ajax exists, put the definition on the list element
  if(isset($element['#ajax'])) {
    $element['book']['#ajax'] = $element['#ajax'];
    unset($element['#ajax']);
  }

  $element['chapter'] = array(
    '#type' => 'textfield',
    '#title' => t('Chapter'),
    '#size' => 3,
    '#maxlength' => 3,
    '#required' => true,
  );

  return $element;
}

And then it will behave as expected. For an element that is not a compound element but totally custom, you have to set the event key of the #ajax definition inside the process function of your element. You will also have to add the ajax_process_form function. Here is an example :
function mymodule_element_info() {
  $elements = array();

  $elements['customelement'] = array(
    '#input' => 1,
    //Note the added ajax_process_form
    '#process' => ('customelement_process', 'ajax_process_form'),
    ....
  );

  return $elements;
}

function customelement_process($element) {
  // ... do whatever is needed to build the element

  if(isset($element['#ajax'])) {
    //Set the event to whatever is appropriate for the element
    $element['#ajax']['event'] = 'click';
  }
}

It's important to set the event key before the ajax_process_form because the latter function tries to figure out which event is best suited for the element and if it does not recognize the element it will return it without including the javascript libraries needed to perform the ajax call.
Answer: It seemed to work for me to add 'ajax_process_form' to the #process tag of the element definition. Like this.

function mymodule_element_info() {
   return array(
      'customelement' => array(
      '#input'          => TRUE,
      '#process'        => array('customelement_process', 'ajax_process_form'),
      '#theme'          => array('customelement'),
      '#theme_wrappers' => array('form_element'),
      '#tree'           => TRUE,
    ),
  );
}

Making a custom form element AJAX aware


Getting extra space while using SpanningString

Question: I am creating a CustomView. I need a string to be a spannable string, to change the colour of a part of the string. The code compiles, but there is a long space between the string and the part that is to be coloured.
Here is the code :
String string = mStat1 + "" + mStatValue1;
SpannableString spannableString = new SpannableString(string);
if(mStatValue1<=0) {
     spannableString.setSpan(new ForegroundColorSpan(Color.rgb(183,108,100)), mStat1.length(), string.length(), Spanned.SPAN_EXCLUSIVE_EXCLUSIVE);
}
else {
    spannableString.setSpan(new ForegroundColorSpan(Color.rgb(117,205,132)), mStat1.length(), string.length(), Spannable.SPAN_EXCLUSIVE_EXCLUSIVE);
}
TextPaint textPaint = createTextPaint(mTextColor, mStatSize, Paint.Align.CENTER);
mDynamicLayout = new DynamicLayout(spannableString, textPaint, mWidthPix/2, Layout.Alignment.ALIGN_NORMAL, 0.0f, 0.0f, false);

Here is the image of the view that i get :

I need to remove the gap between the coloured part of the string and the remaining string. Thanks.
Comment: What do you mean by remaining string? Is it the path between 1 and 6 for example?
Comment: In the 'Percentage Change', the value is far off as compared to the label.
Comment: The space between the value and 'Percentage Change' is unwanted.
Comment: And it is in the same TextView? Why not to have two separate TextView in a LinearLayout or RelativeLayout?
Answer: I think your problem is related with this lines:
TextPaint textPaint = createTextPaint(mTextColor, mStatSize, Paint.Align.CENTER);
mDynamicLayout = new DynamicLayout(spannableString, textPaint, mWidthPix/2, Layout.Alignment.ALIGN_NORMAL, 0.0f, 0.0f, false);

You are only changing the color of the text so you can use the method TextView#setTextColor(int color); using two separate TextView views.
Comment: This is a custom view, so I am not using textviews. I'm drawing these strings on the canvas. Therefore concatinating the string and the value and displaying so that the entire string can be centrally aligned.
Comment: I think you are overkilling your approach by using the draw method. Why not simply extend a, for example LinearLayout, and add the views inside via xml inflation, and finally providing some kind of API from this custom view to set the data? Writting a onDraw method is not so simple and involves so many things. Are you sure it is what you want?
Comment: It needs to be a custom view that can be used later for other modules, hence all the hard work :P
Comment: But what I am proposing to you is also a custom view, but built differently. You can build a custom view as a macro (View + xml) and redistribute it too in a library project.
Comment: Thats what I did now. Had to give up on the idea of drawing everything on the canvas.
Comment: I have done so many views, and it is a hard issue to paint a really complex one without having errors here and there. It is better to not reinvent the wheel and keep using the default views.


Form API: Autocomplete Field in Chrome and Safari

Question: I'm working on a custom webform component to essentially look up entity properties in an autocomplete field.
All works perfectly in Firefox, but I'm just seeing an plain old textfield in Chrome and Safari. Nothing is printed to the console, and nothing is printed to watchdog.
This is what the field looks like on Chrome/Safari:

This is what is looks like in Firefox:

I'm wondering if this has something to do with my code, so I've extracted the relevant snippets below:
Form Definition:
  $ac_field_id = drupal_html_id('field_autocomplete');
  $element['field_autocomplete'] = array(
    '#type' => 'textfield',
    '#title' => $component['name'],
    '#description' => t($component['extra']['description']),
    '#default_value' => $value['field_autocomplete'],
    '#required' => $component['required'],
    '#autocomplete_path' => 'entity-auto-comp/' . $field_entity_type .
                                '/' . $field_entity_prop . '/' . $field_number_of_items_to_return .
                                '/' . $search_type . '/autocomplete',
    '#attributes' => array(
      'data-entity-auto-id' => $ac_field_id,
    ),
  );

How Autocomplete Path is being Handled:
 /**
  * Implements hook_menu() - this provides a URL route for our
  * entity data source
  */
 function entity_auto_comp_menu() {
    $items['entity-auto-comp/%/%/%/%/autocomplete'] = array(
      'page callback' => 'entity_auto_comp_autocomplete',
      'page arguments' => array(1, 2, 3, 4),
      'access arguments' => array('view published content'),
      'type' => MENU_CALLBACK,
    );
    return $items;
}

/**
 * Implements hook_autocomplete() - which allows us to return data
 * to the form field as JSON.
 */
function entity_auto_comp_autocomplete($entity_type, $entity_prop, $max_results = 5, $search_type = 'STARTS_WITH', $string = '') {

  //Get all relevant entity IDs
  $efq = new EntityFieldQuery();
  $efq->entityCondition('entity_type', $entity_type, '=');
  $efq->propertyCondition($entity_prop, $string, $search_type);
  $efq->range(0, $max_results);
  $efq_result = $efq->execute();
  $efq_keys = array_keys($efq_result[$entity_type]);

  //Load entities from database
  $entities = entity_load($entity_type, $efq_keys, false);

  $prop_values = array();
  foreach($entities as $entity){
    $prop_values[$entity->$entity_prop] = $entity->$entity_prop;
  }

  //If we couldn't find anything, say so
  if(count($prop_values) == 0){
    $prop_values[""] = "No results were found...";
  }

  drupal_json_output($prop_values);

}

Some JS I'm using to detect changes and populate a hidden form field:
(function ($) {
    Drupal.behaviors.ol_webform_manage_map_default = {
        attach:function (context, settings) {

          //Get the autocomplete field as a jQuery object
          var $autocomplete_field = jQuery('input[data-entity-auto-id="'+Drupal.settings['ac_field_id']+'"]', context);
          var $autocomplete_value = jQuery('#'+Drupal.settings['ac_field_id'], context);

          //Event listener for option selection
          $autocomplete_field.bind('autocompleteSelect', function(event) {
            $autocomplete_value.val(jQuery(this).val());
          });

        }
    };
})(jQuery);

I'm stumped. So I would appreciate any help you can provide!
Thanks,
Strainy
Update
Scratch that. Nothing to do with browsers. It's got something to do with how I'm handling permissions at either the form field level or the autocomplete path. Realised that I can only use autocomplete if I'm logged in!
Comment: Actually, you don't need to be logged in, as long as the anonymous role has the 'view published content'  permission   Depending upon what data you're exposing, this may be completely reasonable.
Comment: The anonymous role does have that perm however. Otherwise they wouldn't even be able to see the webform.
Answer: If it works on Firefox, I suspect that the issue is with the javascript.  Here, the question you should be asking is: do you really need to write your own.
In Drupal 7, there's a default implementation of the javascript you need for this in misc/autocomplete.js.  In fact, by defining your callback via '#autocomplete_path', you are likely asking for trouble: both your javascript handler from your .js file and the javascript from misc/autocomplete.js may well be completing with each other to handle the textfield. I would not be in the least surprised if the behavior you saw was browser dependent.
Unless there's a specific reason you need the javascript you've written, I'd just use the default javascript, which will get automatically loaded for you when you use the '#autocomplete_path' Form API key.  AFAICT, your callback and your hook_menu definitions are just fine.
However, you can find a nice sample in the Examples Module (https://www.drupal.org/project/examples). Where your implementation differs, the simpler implementation in the Examples code may well be more cross-browser compatible.
Comment: Hi Torenware, I'm merely attaching the js to the form and assigning a Drupal core event listener - so this shouldn't be an issue. See my update, it's a permissions issue.
Comment: I see your implementation of `Drupal.behaviors.ol_webform_manage_map_default`.  Just be aware that simply by using the '#autocomplete_path' key in a Form API array that you are causing two things to happen in addition to your javascript: first, your textfield widget will get the CSS class 'autocomplete' added to its classes array, and as a consequence, the javascript code in   Drupal.behaviors.autocomplete will also want to manipulate the widget. My guess is that behavior in that case will be somewhat unpredictable.
Answer: The problem was with my menu hook. I was using 'view published content' as an access argument, where the argument should have actually been 'access content'.
 /**
  * Implements hook_menu() - this provides a URL route for our
  * entity data source
  */
 function entity_auto_comp_menu() {
    $items['entity-auto-comp/%/%/%/%/autocomplete'] = array(
      'page callback' => 'entity_auto_comp_autocomplete',
      'page arguments' => array(1, 2, 3, 4),
      'access arguments' => array('access content'),
      'type' => MENU_CALLBACK,
    );
    return $items;
}

This, and more about access arguments, can be found in the Drupal documentation:
https://www.drupal.org/node/553368


composer create-project from private repo

Question: I have a private project hosted on Bit Bucket. I have an SSH key setup. Is there a way I can use the php composer create-project vendor/name path command in the same way as If it was on Packagist?
Answer: Well there are different ways to accomplish this one being the use of a composer repository that is used instead of packagist.org, which is a better more centralized way to manage your private composer packages. The other method is to use a composer.json that incorporates your private repos within your environments, per environment.
First
Composer allows you to use private repositories to create projects.
Like so...
composer create-project vendor/name path --repository-url=http://repo.yourcomposerrepo.com

Since you won't submit a private package to packagist. That url just needs a packages.json file at minimum, you could use satis or your own packagist if you want a more dynamic solution to the packages.json.
The method for using composer.json applies to already created projects that will use custom repositories for private packages, not for creating new projects from private repositories. Use the next method if you want to go down a similar route.
Second
Configure your private repository into your config.json globally for your environment. Then like normally..
composer create-project vendor/name path

Comment: In composer the term repository refers to satis, packagist, or other which basically just generate a package.json file. Configure the project source in the package.json and you can create project by calling on it like I did in the above example.
Comment: This wants a package.json and not repo.
Comment: in a composer.json you can have a repo though ala "repositories": [ { "type" : "vcs", "url" : "firstname.lastname@example.com:blah.git" } ] and note the key is repositories. This is how we handle our private things at my company.
Comment: Well, if you are talking about placing that repository setting within the global config.json then that would be fine. But otherwise the create-project would normally be used to create a project that doesn't exist and there would not be a composer.json as of that moment. I will add the global config.json method though. Thanks because that is another valid alternative. :)
Answer: Yes, Composer allows you to add private projects as 'repositories' to your composer.json file. So therefore you can include private projects into another project.
It provides support for GitHub and Bitbucket (as well as SVN and Mercurial).
You need to modify your composer.json file to look something like this:
{
    "repositories": [ {
        "type": "package",
        "package": {
            "name": "TheShiftExchange/test",
            "version": "1.0.0",
            "source": {
                "url": "https://github.com/TheShiftExchange/test.git",
                "type": "git",
                "reference": "master"
              }
         }
    }],
    "require": {
        "laravel/framework": "4.0.*",
        "TheShiftExchange/test": "1.0.*"
    },
}

Comment: Note that `"type": "package"` is for packages that don't support composer (using it will impact autoloading for example) https://getcomposer.org/doc/05-repositories.md#package-2
Answer: The way I used to:
composer create-project vendor/name path --repository="{\"url\": \"https://bitbucket.org/user/project.git\", \"type\": \"vcs\"}" --stability=dev --remove-vcs

Reference: https://getcomposer.org/doc/03-cli.md#create-project
Answer: We have Toran Proxy (https://toranproxy.com/) installed as a private packagist, and for that we are able to create projects using command below
composer create-project vendor/framework --repository-url=http://your-toran-repo-url/repo/private/ --stability=dev project name

Stability version we use if the project is not tagged or you looking for bleeding edge version.
--stability=dev

Answer: Since this post has some traction, I thought I'd add another solution which I use.
Open up ~/.bash_profile
and add something like
function _cmsname {
    composer create-project vendor/package --repository-url=http://private.repo.url.co.uk/ --stability=dev "$1"
}
alias cmsname=_cmsname

and the just type cmsname projectname in terminal.


Web API - Token

Question: Uma aplicação MVC faz acesso a serviços WEB API.
Para acessar o WEB API é necessário obrigatoriamente informar um "token".
Na aplicação WEB API existe um botão que gera o token.
Como fazer que o WEB API aceite apenas o token gerado através da aplicação MVC sem usar banco de dados?
Eu fiz o MVC gerar o token (um GUID + data) e passo este token para o WEB API que valida se a data esta dentro de um período de 30s. Se estiver dentro do período considero que o token é valido.
<code>byte[] data = Convert.FromBase64String(token);
DateTime when = DateTime.FromBinary(BitConverter.ToInt64(data, 0));

if (when < DateTime.UtcNow.AddSeconds(-30))
{
    return false;
}
</code>
Isto funciona, entretanto, qualquer GUID que for informado concatenado de uma data será válido. Preciso fazer com que o meu WEB API saiba exatamente qual token foi gerado pela aplicação MVC.
Comment: Você pode substituir o GUID por uma chave que só o projeto MVC conhece. Mas não entendi porque tem um botão para gerar o token se só um outro projeto pode conhecer o token.
Comment: Olá Alberto, tente explicar melhor o que você necessita.
Comment: Obrigado pela ajuda. Estive pesquisando um pouco e acredito que o que preciso seja um OTP. Ou seja, o MVC gera um token e o outro projeto sabe qual foi o token gerado.
Comment: Já pensou em usar o `OAuth` que já faz tudo isso pra você? Dá uma olhada aqui: http://www.leonardohofling.com/blog/web-api-adicionando-autenticacao-oauth/
Comment: Recomendo que você utilize JWT / OAuth. https://github.com/IdentityServer/IdentityServer3 Caso queira utilizar o Azure AD para isto, eu gravei um video que talvez te ajude: https://www.youtube.com/watch?v=PSkY1PbkrfA&list=PL3tw-QzEqu8QS8lopRA_KpTs5DJvFm1ks
Answer: Existem diversas soluções para o seu problema...

Ao gerar o token na aplicação MVC, envie ele para a WebAPI (Através de um endpoint restrito, que só você tenha acesso), ao receber o token, sua API vai armazenar ele na memória em uma lista/dicionario estático. O problema é que você não pode matar o processo ou reiniciar o servidor onde está a WebApi e quando a Pool (se estiver usando IIS) reiniciar, todos os dados serão "perdidos". (Neste caso, ficaria mais fácil ainda você criar o token direto na WebAPI e criar um endpoint restrito que só o MVC acesse para obter o token, a cada acesso gera novo token e exclui os inválidos da memória [collection/list/dictionary/hashset])
Adicione um SALT (uma string aleatória, pré-definida, de conhecimento exclusivo das aplicações) ao seu token, codifique ele com SHA1 ou outro algoritmo de encriptação (neste caso, você nem precisa do GUID...)
Uma solução nada segura, porém eficaz para o seu problema seria, um SALT + GUID + data, tudo isso codificado em Base64, porém como disse, isso não é nada seguro, e qualquer um com conhecimento intermediário, que visualizar o token vai perceber que é uma string base64 e pode passar ela em um decodificador de base64 e ver o padrão original sem maiores complicações...
Implemente OAuth ou JWT (mas tenho impressão que para isso, obrigatoriamente irá precisar de um banco de dados).

Porque você não pode utilizar um banco de dados ? Nem um noSQL está autorizado ? Porque não faz isso utilizando o Redis ?

Web API - Token


Open Id Connect SSO on cross platform sites

Question: I have 3 sites(all were sub domain sites)

Server(id.test.com) - .NET
Client1(client1.test.com) - Drupal
Client2(client2.test.com) - .NET

Am using OpenId Connect for login. My login & logout using drupal site is successful. But when i login through Client2 and navigates to Client1 site automatic login was not made, when i initiates a click in "Login using Generic" button, login occurs.
Is that possible to login automatically? I have checked OpenId Connect SSO module which supports only for Drupal sites. Is there is any other way to achieve this?
Answer: The OpenID Connect module is designed only to login your users when they use the login button(s).
The SSO module will propagate the logins to your Drupal sites, if you are using Drupal as authentication provider/clients.
If you are using a cross-platform setup, you will have to create your own auto-login propagation in a custom module. Depending on your authentication provider, you may (or not) be able to define post-login actions to use. Or you may yourself utilize post-login hooks of your Drupal site to trigger logins on the remaining sites.
Unfortunately, the solution you are asking for not only depends heavily on your authentication provider, but also is far out of the scope of Drupal Answers.
Comment: Thanks, i have updated my query with an link. Could you check that too?
Comment: Your second question is very likely to be closed for being too broad. Drupal allows many different ways for solving certain problems. Drupal Answers is intended to helping you out with distinct hints/answers when you have questions/issues AFTER you did the research yourself and decided for a way to go but got stuck while doing so. You will most probably not receive fully-grown solutions or startup guidance.
Comment: After research only i have splitted my query in two. I have checked in all drupal sites i found none. also it an customization to be made in it. So i thought of asking for an idea. If you are unsatisfied with my reply, can i modify my query on my facts?
Comment: It's not my satisfaction that matters here. ;) - It's the rules of this QA group. We don't provide ideas, we provide answers to narrowly scoped to-the-point Drupal related questions that are useful to you and others. We won't teach how to use Drupal, nor tell approaches to solve complex problems/realize complex use-cases. That kind of creative work is what you as developer have to do on your own before you ask questions here.
Comment: Okay. my apologies you can go ahead of closing my second question :)


Is DAS OK for under-utilised virtual machines?

Question: We're a little behind in the virtualization game, I'll have to admit. We currently have nothing virtualized, but I'm trying to change that.
We want to get our feet wet with virtualization and get a feel for it, so I am going to propose for next year we get a server and virtualize 3 rather under-utilised (and incredibly old) non-critical servers.
I know virtualization is all about SANs, but unfortunately we don't have one (all our servers are DAS). The question I have is if I get a server (probably a HP DL380 of some description) with DAS, are the guest VMs (under-utilised don't forget) going to suffer badly as a result? My current thinking is to put Hyper-V Server 2008 R2 as the host, if it makes any difference.
I'll give a quick run down of what the 3 servers do below. I know this isn't an exact science, but roughly speaking - do you think DAS will be OK?
Server 1
Runs a legacy HR system which is accessed maybe once or twice a week. This has a SQL Server 2000 backend which also runs on this server. This thing is archaic (I don't know when HP stopped branding ProLiants Compaq, but this proudly displays the Compaq logo on the front).
Server 2
Runs some emulated printers for our AS/400 and prints the output to some real printers. This runs happily on an oldish PC with W2K3 and 2GB RAM.
Server 3
Runs a server for a BI product, however all it actually does is at 4:00AM imports a ton of data from our AS/400 and compiles it into a file (this takes about half an hour). About 15 users then periodically (maybe once every few days) download this compiled file (~100MB) to their local computer. The download is user initiated, so there's no real defined access pattern.
This is also on a PC running 2K3 but with 3GB RAM.
Update
We've looked at virtualization several times now, but because of our lack of SAN and the many advantages it brings, we have dismissed virtualization as an option each time.
I know some of the pitfalls of putting these VMs on DAS (lack of HA etc) - and that is OK for these machines. If the host were to fail, it would be an inconvenience rather than a full blown disaster.
I'm looking at this more of a consolidation project rather than anything else. The 3 servers are slated for replacement next year anyway, and like I said they are getting rather old, sit mostly idle and are big tower machines using a lot of space and electricity. In my opinion, 3 new servers (however basic) is still overkill for machines that sit doing very little all day.
Comment: I don't have an answer for you but I have an opinion on virtualization: Virtualization is not a neccessity. There are many reasons an organization may choose to virtualize their IT infrastructure but doing it because everyone else is doing it or virtualizing just for the sake of virtualizing aren't valid reasons in my book. Virtualization should be viewed like any other business endeavor: Is there a valid business or technical reason for it, does it give us some competitive advantage, does it reduce our TCO, does it increase our ROI, etc., etc. No offense intended.
Comment: In addition, your question implies that you haven't thought this through from a business or technical perspective and haven't validated your reasons for virtualization. You're considering virtualization on non-optimal hardware and you're guessing at which servers would be good candidates for virtualization, which tells me this is a "throw it against the wall and see if it sticks" attempt at virtualization. My advice would be to read some of the many white papers regarding virtualization and conduct an analysis on your environment to see if virtualization can actually benefit your organization.
Comment: @Ben: As in my first comment, no offense is intended by any of my comments (including this one). It's my opinion that as a sysadmin I need to pursue projects that have real world benefits for the business and are targeted to address specific, identified needs. I don't implement things just because I can, or because I think they're cool, or because everyone else is doing it.
Comment: Storage performance is unlikely to be any kind of an issue for this sort of "low-hanging fruit" virtualization project.  You will need to look at your actual performance requirements (refer to Jake's answer below), but, it is likely that something ludicrously cheap/simple (e.g. four 2TB 7.2k SATA drives in RAID10) would be more than adequate for these machines' VHDs.  It is unclear whether you meant internal or external DAS; if you meant external, be sure to consider the possibility that you may be able to satisfy your storage capacity/performance requirements within a single 2U server case.
Comment: @joeqwerty - no offence taken, I appreciate everybody's opinion. I have added an update to hopefully better clarify what I'm trying to achieve. I assure you it's not just to "keep up with the Jones'", I believe that I present a valid business case that will save us some money (on the cost of 3 servers, maintenance contracts and electricity) going forward.
Comment: @Ben: Based on your edit it looks like you have though it out and do have some valid reasons for going down the virtualization road. My concern was that it was a "keep up with the Jones" project, which to me wouldn't have been a valid reason for it. Good luck with it.
Answer: Your question can be answered partly by looking at the performance counters for I/O in the machines that you're trying to migrate to virtual.  Look at how many I/O operations per second (IOPS) each requires.  Remember that each physical disk is good for a few tens of IOPS.  Now size your virtualization system to support that level of I/O.  You'll know how many "spindles" you need.  Use a RAID controller to spread the load across the disks.
Note that none of this said anything about a SAN.  SANs are great, but they mostly provide things that you didn't mention: high availability, backup, storage tiering, fault tolerance, cross-site disaster recovery, rapid deployment, etc.
The most notable part of all of that is that if you use DAS, the failure of the physical server that the DAS is attached to constitutes a failure of all the VMs that were hosted on it.  With a SAN, using a cluster of compute hosts all attached to the SAN, you can survive the failure of the physical host by moving the VM to a new host.
Another strategy for surviving failure of that physical host is to back up the apps and OSes within the VMs frequently, or at least frequently enough that you won't mind losing things that happened after the last backup.
Your choice.  DAS can give you the performance you need (given what you have described.)
Comment: +1 - In some cases questions about performance may be orthagonal to the DASD vs SAN question. Lower-end SAN solutions (iSCSI on SATA disks) may not be able to out-perform 15K RPM SAS disks on a hardware RAID controller directly-attached.
Comment: nearly +1. Discs are good for a few HUNDRED IOPS. Depending on the disc.
Answer: 1. Performance
In my experience, SAN storage and DAS storage have different characteristics which make performance hard to measure objectively, since workloads tend not to rely on just one metric (e.g. access time), but rather a variable blend of access time, throughput, read vs write, etc. It's a very complex picture. The main benefits of a SAN, IMO, are having your data available to several hosts simultaneously (allowing for clustering/high availability), and unifying the storage of your data allowing for easier management and maintenance. So long story short - I wouldn't sweat performance all that much for DAS.
2. SAN Availability & Cost
You don't have to go with off-the-shelf discrete SAN solutions. Many folks are building their own since a collection of regular server parts can get you a fully functional SAN system. Here's one route to that:

Pick the server that you own that can take the most disks with the highest total capacity
Strip out all but a couple of Gb of RAM
Install OpenFiler
Configure some volumes and expose some iSCSI LUNs
Hook the server into an isolated gigabit VLAN
Plumb your virtual hosts into the same storage network
Point the hosts at the target LUNs
Start adding VMs to your SAN

Vendor support can be purchased for OpenFiler, and your hardware, if your company insists on these things. You can also configure multiple OpenFiler hosts with DRDB to give you high availability and redundancy options.
We've run one of these in production for over a year now, and it's been very solid.
Answer: DAS normally, should be faster than some FC SANs, and definitely faster than iSCSI SANs. You're getting a SAS link speed (3Gbps and in the newer boxes - 6Gbps)
The downside to DAS is not speed or stability, it's scalability, because the amount of ports on a DAS is limited, so you can't connect too many hosts to it.
Answer: When we started with virtualization in 2007 we had drbd in mind. So we started with a server-pair that had 5 internal 3,5" SAS-disks (configured as RAID 5 providing 500 GB netto storage). Since our database-servers run in a different cluster (SAN-attached) our virtual systems don't do much disk io - which makes this a possible scenario - even with 100 MBit Ethernet for drbd replication.
With that setup we were able to provide over 20 HA-VMs (XEN paravirtualized).
Disk IO or disk performance was never a problem. DAS is even faster nowadays than local storage. So for your setup with just three low performance, low disk-io servers I would say that even DAS is oversized.
Since you use mainly W2K you could play with Linux KVM or Windows Hyper-V. Or Windows-Admins love the latter (and use DAS attached to two servers themselves). With Linux KVM I would use DRBD as ha-storage backend (DAS or internal disks).
Just a note on SAN/DAS:
We attached some of our Virtualization Servers to SAN - for those guests that now run databases with heavy IO. But: SAN is not faster than the internal disks - it just distributes io to more disks - the same can be true for DAS.
I see the real advantage on SAN in a different area: It uses its own switching fabric, which has (at my work) nothing to do with the IP routers and switches - and is much more stable. From a cluster/ha point of view you get another - really independent - connection.


A generic error occurred in GDI+ saving

Question: I'm trying to save file from one server to another. Users would be upload a jpeg file to server. In my controller I'm getting file data with "HttpPostedFileBase". If I want to upload this file on local server, it has no problem (Web-based Admin application IP is xxx.xxx.xxx). But if I want to upload data to another server (IP is yyy.yyy.yyy) I'm getting this error that I shown on title.
I also doing impersonation to reach interested folder. The folder has IIS_USER,NetworkService,Everyone users full admin rights on (yyy.yyy.yyy) machine.
Also I have tried to do this code with File Copy, move in exe file and I havent got any error. So this code giving error just in ASP.Net MVC Application.
You can check partial code below :
if (isImpersonated)
        {

            //File.ResizeImage(1000, 1000).Save(Server.MapPath(Path.Combine(WebConfigurationManager.AppSettings["PicturePath"], fileName)));
            //File.ResizeImage(500, 500).Save(Server.MapPath(Path.Combine(WebConfigurationManager.AppSettings["MarinaPicturePath"], string.Format("{0}-medium{1}", imgName, Path.GetExtension(File.FileName)))));
            //File.ResizeImage(250, 250).Save(Server.MapPath(Path.Combine(WebConfigurationManager.AppSettings["MarinaPicturePath"], string.Format("{0}-small{1}", imgName, Path.GetExtension(File.FileName)))));

            //File.Copy("C:\\test\\asd.txt", "\\\\126.96.36.199\\C$\\test\\asd123.txt");
            //System.IO.File.Copy("C:\\test\\asd.txt", "\\\\188.8.131.52\\C$\\test\\asd123.txt");

            //File.ResizeImage(1600, 1200).Save(Server.MapPath(Path.Combine(WebConfigurationManager.AppSettings["PicturePath"], fileName)));

            File.ResizeImage(1000, 1000).Save(Path.Combine("\\\\220.127.116.11\\C$\\test\\", fileName));
            File.ResizeImage(500, 500).Save(Server.MapPath(Path.Combine("\\\\18.104.22.168\\C$\\test\\", string.Format("{0}-medium{1}", imgName, Path.GetExtension(File.FileName)))));
            File.ResizeImage(250, 250).Save(Server.MapPath(Path.Combine("\\\\18.104.22.168\\C$\\test\\", string.Format("{0}-small{1}", imgName, Path.GetExtension(File.FileName)))));

            GC.Collect();
            //var asd = StreamToByteArray(File.InputStream);
            //System.IO.File.WriteAllBytes(Path.Combine("\\\\188.8.131.52\\C$\\test\\", fileName), asd);

        }

Answer: I have found the answer! If somebody would get this error, should be check Impersination class and just set const int LOGON32_LOGON_INTERACTIVE = 9

A generic error occurred in GDI+ saving


How to get SQL SERVER 2008 r2 data in a column

Question: I have two different tables :
Tank & FillingStations
In which one tank can be attached to many fillingstations.
Suppose:
SrNo TankID  TankName   TANK_Balance    FillingStation_ID       FS_NAme  BALANCE
1     1      Tank1       5000                A11                FSA11     1545
2     1      Tank1       5000                A12                FSA12     1000
3     1      Tank1       5000                A13                FSA13     800

And i want to get a report as like :
SrNo TankID  TankName  TANK_Balance   A11    BAL1  A12    BAL2  A13   BAL3 TOTAL
1     1      Tank1     5000           FSA11  1545  FSA12  1000 FSA13  800   3345

Comment: This question should be improved: Add your actual table's structures and what you've tried so far. Best was a [Minimal-Complete-Verifyable-Example](http://stackoverflow.com/help/mcve). Use *copy'n'pasteable* code, use `DECLARE @Tank Table(...)` or `CREATE TABLE #Tank(...)` to define the structure, fill it with sample data (`INSERT INTO`). You might create a [SQL-Fiddle](http://www.sqlfiddle.com). In this case you'll get answers in minutes...
Comment: Can you please add more light on the question... details of business rules etc.
Answer: Try like this,
DECLARE @Table TABLE (
    SrNo INT
    ,TankID INT
    ,TankName VARCHAR(10)
    ,TANK_Balance INT
    ,FillingStation_ID VARCHAR(10)
    ,FS_NAme VARCHAR(10)
    ,BALANCE INT
    )
insert into @Table values
 (1,1,'Tank1',5000,'A11','FSA11',1545)
,(2,1,'Tank1',5000,'A12','FSA12',1000)
,(3,1,'Tank1',5000,'A13','FSA13',800)

SELECT min(SrNo) as SrNo, TankID
    ,TankName
    ,TANK_Balance
    ,max([A11]) AS [A11]
    ,max([BAL1]) AS [BAL1]
    ,max([A12]) AS [A12]
    ,max([BAL2]) AS [BAL2]
    ,max([A13]) AS [A13]
    ,max([BAL3]) AS [BAL3]
    ,max([BAL1])+max([BAL2])+max([BAL3]) as Total
FROM (
    SELECT SrNo,TankID
        ,TankName
        ,TANK_Balance
        ,FillingStation_ID
        ,replace(FillingStation_ID, 'A1', 'BAL') AS bFillingStation_ID
        ,FS_NAme
        ,BALANCE
    FROM @Table
    ) s
pivot(Max(FS_NAme) FOR FillingStation_Id IN (
            [A11]
            ,[A12]
            ,[A13]
            )) t
pivot(Max(BALANCE) FOR bFillingStation_Id IN (
            [BAL1]
            ,[BAL2]
            ,[BAL3]
            )) t1
GROUP BY TankID
    ,TankName
    ,TANK_Balance


How to include the comment page number in views "rewrite results"?

Question: I built a custom "new comments" notification system that doesn't rely on the history module (and the super slow queries when viewing nodes with new comments).
When I show nodes with new comments I need to rewrite the link to jump to the first new comment. "node/[nid]#new" does that. However, when I have a lot of comments and the first new comment is on the 2nd page (or higher) the link doesn't work. It would need to be "node/[nid]?page=1#new", etc.
I don't think the "page" value is available in views. Any tricks I can use?
Answer: I used a "Global PHP" field and added this code:
<?php
$new = $row->comment_count - $row->nc_count;
$page = floor($new / 50);
$link =  'nid;
if ($page > 0){
  $link = $link . '?page=' . $page;
}
$link = $link . '#new>' . $row->nc_count . ' new' ;
print $link;
?>

I don't like to use PHP in a view since it prevents caching. This view won't be cached, so it's not a problem this time. The other issue is that it doesn't generate the "pathauto" urls and requires an additional 301 redirect. I tried to use a "node link" variable but they don't seem to get changed to a pathauto link before the PHP field runs.
Any idea how I can render the /node/123 link to a pathauto link?


Add custom node content to cache

Question: I have added a few custom nodes to my website via the admin CMS. I wanted to know if I would have to explicitly add them to cache or if the nose_save or one of the internal drupal methods would add it to cache? I am using memcache for my caching purposes.
Answer: Generally, Drupal caches pages for anonymous users. This is true for all nodes. You can turn this, together with some other caching settings, on and off at admin/config/development/performance
If memcache handles the data that otherwise goes into cache_page, then you will benefit from memcache in returning nodes.

Add custom node content to cache