Determine sample size for each cluster

Question: Suppose I have a large population and I want to test if installing a new special light bulb can reduce energy consumption. Since I have a large population, I'll have people that usually consume high and some that consume less energy on a daily basis; therefore I would like to know the impact of the new special light bulb depending on their previous behavior. So I want to cluster my population into groups based on their energy consumption. Suppose that I get K groups after clustering.
How can I determine the minimum sample size for each of my K+1 groups (+1 because I'm counting with the group that doesn't have the new light bulb, i.e, control group) to ensure some statistical rigor (for example some confidence level, etc..). I've only encountered literature about stratified sampling and cluster sampling, but none of those are what I want to do. In stratified sampling, they divide the population into K groups and then take elements from all of the groups to make sure every group will have elements in the (only) sample. In clustering sampling, they divide the population into K groups and only consider some of those groups for the sample.
EDIT 1: I found about one-way ANOVA: it can determine sample sizes for multiple groups but assumes as a null hypothesis that all of the groups have equal means (which is not the case here). In this case, the null hypothesis would have to be something like "there's no changing on the mean (energy consumption) on any of the groups with the new special light bulb". Even if one-way ANOVA could be used, I would have to give as an input a guess/estimation of the means for each group after getting the new special light bulb?
EDIT 2: I'd like to thank StatsStudent for his answer, which is very detailed and informative. However, his answer is not what I'm actually looking for. To be more detailed, I have the data on the energy consumption of the whole population (without the new special light bulb). And after clustering, I know there are K different types of behavior (for example, K=3: low, medium, and high energy consumption behaviors among the population). I want to know if installing the light bulb will affect the energy consumption of any of the K groups. Also, one aspect that would also like to include is to have more than one special light bulb, for example, three different light bulbs: A, B, and C. Can I use Multi-Way ANOVA to determine the sample size needed? I've read that this test is an extension of ANOVA, where there is more than one category, but only one category is of
interest. The other category/categories are things that need
to be controlled for (blocking/nesting/random effects/etc.). If I'm interested in determining if there is a
difference in energy consumption due to light bulbs (A, B, and C) while
controlling for energy behavior (low, medium, high) (K=3:groups from clustering) can and should I use two-way ANOVA? This will result in $3 \times 3 = 9$ groups.
Comment: I'm a little confused by your description and hope you can clarify.  Is each household measured before the introduction of the new bulb and then you introduce the new bulb to every household at the same time and then want to determine if there is a reduction in energy usage after it's introduction?  Or is only some subset of households getting the new bulb?
Comment: If it's not too late, and you haven't conducted the experiment yet, I'd suggest a simple paired sample t-test in which you measure, for say, over the course of a year the energy usage by each household.  Then introduce the bulb to every household and for another year measure energy usage.  Then each Household serves as it's own control which will reduce your variance and make your test more powerful.  You'd simply take the difference in energy usage before and after the intervention of the bulb is introduced.  Then carry out a simple t-test on the differences. Of course (continued)
Comment: Each household is measured BEFORE the introduction of the new light bulb. I have that data and I use clustering to get K groups from it. Now I want to know the minimum sample size for the K+1 groups (including the control group) to see if there is a statistically significant difference, i.e, I want to know if the new light bulb really changed the energy consumption in any of the groups.
Comment: this design would suffer from potential time effects (e.g. other, external factors during the second year could lead to the reduction in energy usage or even weather patterns.  So one could modify this design with a regression model and half of the households could randomly receive the new bulb treatment first and the other receive it in the second measurement period.
Comment: OK.  There's really no need to do any clustering.  Use each household as it's own cluster in some sense.  I'll explain in an answer more fully below.
Comment: Even in that suggestion, I need to get a sample size. Assuming installing the light bulb has high costs!
Comment: Wait, you say you have the data, so does this mean the experiment has already been carried out or did you mean that you have data as in a list of households/potential participants and energy measurements before introducing the new bulb?
Answer: You essentially have a before and after experiment in which each participating household is provided the new bulb at a point in time and measurements are taken on energy usage before the "intervention" of the bulb is introduced and then additional measurements are taken for the same amount of time after the bulb is introduced.  In this case, you'll want to treat each household as its own cluster really and perform a paired-sample t-test (there are more complex designs that aim at reducing bias, but this is a good starting point):
Under the paired sample $t$-test, each household is measured for say, 1 year.  After 1 year, the bulb is introduced and for the next year energy consumption is measured.  Denote the energy consumption of household $i$ before the bulb is installed in time-period 1 as $Y_{i,1}$ and the energy consumption of the household $i$ after the bulb is installed at time-period 2 as $Y_{i,2}$, for $i=1, 2, ..., n$, where $n$ is the total number of participating households.
Then you'll compute the difference between the two time periods for household $i$ as:
$Y_{i,diff}=Y_{i,1}-Y_{i,2}$.
You'll then compute the simple average of of these $n$ differences. Call this $\bar{Y}_{diff}$.  In addition, you'll compute the variance as $SD_Y^2={{\sum_{i=1}^n(Y_{i,diff}-\bar{Y}_{diff})^2}\over{n-1}}$ and compute the standard deviation as $SD_Y=\sqrt{SD_Y^2}$.
You will then want to test the Null Hypothesis, $H_0$, that the mean difference between time periods is equal to zero vs. the Alternative Hypothesis, $H_1$, that the mean difference between period 2 and 1 is greater than 0 (you're expecting the bulb introduced in time period 2 will result in less energy than in time period 1).  In other words, you'll want to test:
$H_0:  \mu_{diff}=0$ vs $H_1: \mu_{diff}>0$.
To test this hypothesis, You'll compare the test statistic:  $T={\bar{Y}_{diff}\over{SD_Y/\sqrt{n}}}$ with $n-1$ degrees of freedom to the standard $t$-distribution with $n-1$ degrees of freedom.  To perform this test at the 99% confidence level for example, you'd compare the value of your test statistic $T$ with $t_{.01}$ with $n-1$ degrees of freedom.  If $T >t_{.01, n-1}$, you'd reject the null hypothesis and conclude that there is sufficient evidence at the 99% confidence level to conclude there is a reduction in the mean energy consumption.
Now, the question remains how large of a sample do you need?  The answer depends on a number of factors.  It's helpful to know what your budget is in planning this type of experiment because often time budgets dictate how much you can spend so sometimes you may be advised to sample as many households are you budget allows.  Disregarding cost for the moment, you should try to select a sample size that provides you with sufficient power to detect true differences in mean energy consumption when they exist.  Many statisticians select 80% power, which means there is an 80% chance of rejecting the null hypothesis when the alternative hypothesis is true (i.e. there is reduced energy consumption from the new bulb).  Then you'll have to make some educated guesses on how much energy, on average the homes actually consume in time periods 1 and 2 (call these $\mu_1$ and $\mu_2$).  You might be able to get estimates from your existing data and experiments with your bulb connected to a energy meter in the lab for example.  You'll also need to determine an estimate of the standard deviation of the differences in energy consumption $\sigma_Y$.  You'll then use these parameters for determining the sample size, $n$.  The easiest way to compute this is to use any of the available online sample size calculators.  I'll use this one here:  https://www.stat.ubc.ca/~rollin/stats/ssize/n2.html.  There is a reference on the web page to the formula used to compute this if you are interested in that.
So for example, let's assume $\mu_1=52$ and after introducing the bulb you expect the mean energy consumption to fall to $\mu_2=50$ and there is a standard deviation of $4$.  Let's also say you'll want to test your hypothesis at the 99% confidence level.  Then you simply input these parameters into screen as shown below and click calculate:

After doing that, you'll see that the sample size required is 63.  You'll then need to sample 63 households.  Keep in mind that this assumes 100% participation and compliance with every home in your experiment.  If you suspect 10% of the participants might drop out, you should inflate this sample size estimate by 10%.
That all being said, this test assumes that there are no confounding variables between the two time periods.  This may not be the case.  For example an extremely hot summer in time period 2 may mean that more households run their AC units more often in time period 2 than in time period 1 and this is actually what caused the increase in energy in time period 2, rather than the bulb.  To get around this, I'd recommend using a regression model and randomly assign half the households to get the bulb in time 1 and the other half in time 2.  Then you can take this into consideration in building a regression model to better control for these types of external factors, but this is beyond the scope of my answer.
You may also find this reference quite detailed and provide methods for computing sample sizes in R: https://stats.idre.ucla.edu/r/dae/power-analysis-for-paired-sample-t-test/
Answer: Since no one, besides StatsStudent, provided an answer, I guess I'll answer it myself. I think I understand it now.
If I'm interested in determining the minimum sample size needed for the mentioned example with three light bulbs $A, B$ and $C$ and with the $K$ behaviors obtained from clustering (let us assume for this example that $K=3$ and that we label it as $k_1,k_2$ and $k_3$), the Multi-way ANOVA should be considered, more specifically a two-way ANOVA. Why? Because we have two factors: one for the light bulb $\{A,B,C\}$ and another with behaviors $\{k_1,k_2,k_3\}$ and two-way ANOVA is an extension of the one-way ANOVA that examines the influence of two different categorical independent variables (these two factors previously mentioned) on one continuous dependent variable (energy consumption).
For this calculation, softwares like $\texttt{G*Power}$ or $\texttt{R}$ can be used. For the latter one, the function $\texttt{wp.kanova}$ from the $\texttt{WebPower}$ package can be used. To determinate the minimum sample size, one has to give for input: Numerator degrees of freedom, Effect size, Number of groups, Significance level, and Statistical power. All of these inputs are directly known except for the effect size. For the effect size, one needs a previous trial study to calculate it or we can simply assign a guess value for it (carefully). The $\texttt{R}$ function will return the total number of samples, that we will need to divide with the number of groups and, thus, we will have the number needed for each group (in this case there are $3 \times 3 =9$ groups).

Determine sample size for each cluster


Delphi - Load PFX Certificates without installing it on Windows using CAPICOM

Question: I'm using CAPICOM to load a certificate needed by a WebService client.
I need to have the certificate installed in Windows, and then open it from the certificate file itself, which I think is "kinda stupid".
Is there any way to either (in order of preference):

Save the PFX File contents to a memo field in the database, and load it from there, without installing it on Windows?
Load the PFX File from the file itself, without installing it on Windows

I'm using Delphi XE3, LibEay32 and Capicom 2.0
Answer: If you want to install the certificate in Windows you will ALWAYS get a popup asking the user for permission, unless the certificate comes from a root that is already trusted. If you don't want that the only option is to use the PFX contents from a memo field each time you need the certificate.
I have no experience with Capicom, but from what I read it is a Microsoft DLL that you use?
If you are having problems with Capicom and LibEay32 you might want to check out the Eldos Secure BlackBox components. I have very good experiences with those and their support is great.
Comment: Capicom is obsolete, see [http://msdn.microsoft.com/en-us/library/cc778518.aspx](http://msdn.microsoft.com/en-us/library/cc778518.aspx). To use WebServices with PFX files, I've sligtly modified SoapHTTPTrans.pas unit to support Synapse (and OpenSSL)
Comment: I second using Eldos Secure BlackBox. No more messing around with OpenSSL, capicom, etc. It provides everything you need for crypto and then some, and their support is stellar (even if they may appear a little rough at the beginning!)
Comment: Thank you for the replys!
Providing I'm using THTTPReqResp to establish comunications, how would I load the PFX from a TMemo?
That would be ideal, since it would be interesting to save the pfx contents in my SQL database.
Comment: What I understand from looking at THTTPReqResp.ClientCertificate code is that it only allows you to select certificates that are installed in the store. It looks impossible to use another method to load a certificate. I think you should consider abandoning THTTPReqResp.
Comment: What would be the alternative? THTTPRIO?
Comment: It seems THTTPRIO has the same problem: it only works with certificates from the store. The only thing I can recommend then is the Eldos components I mentioned earlier. They are not free, but worth the investment if this is the kind of stuff you need!
Comment: If you are still looking for a solution based on OpenSSL, I can post you my SoapHTTPTrans modification from 2008 using [Synapse](http://ararat.cz/synapse/doku.php) as a secure communication library.
Comment: pf1957: That would be great.
Someone else told me about Synapse and I began today looking into it, but couldn't yet find how to do it!
Thank you
Comment: @nunopicado: if this the case than send me your email, pls (mine is in my SO's profile)


How to evaluate $\int \frac { \sin x+\cos x }{ \sin^4 x+\cos^4x}\, dx$?

Question: How can one find $$\int  \frac { \sin x+\cos x }{ \sin^4 x+\cos^4x}\, dx?$$
Comment: What is the source of this problem, and what's its motivation?  (One can easily get the result by writing it as a sum of two integrals and applying substitutions $u=\sin x$ or $u=\cos x$ as appropriate, but that's almost certainly not the most elegant approach)
Answer: HINT:
$\sin^4x+\cos^4x=(\sin^2x+\cos^2x)^2-2\sin^2x\cos^2x=1-\dfrac{\sin^22x}2$
Set $\dfrac{d(\cos x+\sin x)}{dx}=-\sin x+\cos x=u\implies u^2=1-\sin2x$
Answer: Split the integral as
$$
\int\frac {\cos x}{\sin^4 x + (1 - \sin^2 x)^2} dx
+ \int \frac{\sin x}{(1 - \cos^2 x)^2 + \cos^4 x} dx
$$
Compute the integrals with the substitutions $u = \sin x$ and $u = \cos x$ respectively.
Answer: Let
\begin{align}
 I &= \int {\frac{{\sin x + \cos x}}{{\sin ^4 x + \cos ^4 x}}dx}  \\
  &= \int {\frac{{\sin x}}{{\sin ^4 x + \cos ^4 x}}dx}  + \int {\frac{{\cos x}}{{\sin ^4 x + \cos ^4 x}}dx}  \\
  &= \int {\frac{{\sin x}}{{\sin ^4 x + \cos ^4 x}}dx}  + \int {\frac{{\sin \left( {{\textstyle{\pi  \over 2}} - x} \right)}}{{\sin ^4 x + \sin ^4 \left( {{\textstyle{\pi  \over 2}} - x} \right)}}dx} ,\,\,\,\,\,\, (\text{since}\,\,
{\sin \left( {{\textstyle{\pi  \over 2}} - x} \right)}=\cos x)
\end{align}
Substituting $u=\frac{\pi}{2}-x$ in the second integral we get
\begin{align}
I&= \int {\frac{{\sin x}}{{\sin ^4 x + \cos ^4 x}}dx}  - \int {\frac{{\sin u}}{{\sin ^4 \left( {{\textstyle{\pi  \over 2}} + u} \right) + \sin ^4 u}}du}
\\
 &= \int {\frac{{\sin x}}{{\sin ^4 x + \cos ^4 x}}dx}  - \int {\frac{{\sin u}}{{\cos ^4 u + \sin ^4 u}}du} ,\,\,\,\,\,\,\,\,\,\,\,\, (\text{since}\,\,
{\sin \left( {{\textstyle{\pi  \over 2}} + u} \right)}=\cos u)
\\
&=0
\end{align}
Comment: If I understand it correctly, this does no prove that the integral is zero. Let $G(x)$ be a primitive function to $(\sin x+\cos x)/(\sin^4x+\cos^4x)$ and $F(x)$  be a primitive function to $\sin x/(\sin^4x+\cos^4x)$. It seems that you have shown $G(x)=F(x)-F(\pi/2-x)$ or something similar.


priors for Gamma shape and scale parameters

Question: I have a random variable $X$ that is Gamma distributed with unknown parameters $\alpha$ and $\beta$:
$$
X\sim \text{Gamma}(\alpha, \beta)
$$
I now want to estimate $\alpha$ and $\beta$ from samples $x_i$ in a Bayesian setting using STAN.
What is the recommended (weakly informative) prior distribution for these parameters?
In STAN $\alpha$=shape and $\beta$=inverse scale.
I read that the half-Cauchy distribution should be used for scale parameters, so is
<code>  shape ~ cauchy(0,2.5);
  scale ~ cauchy(0,2.5);
</code>
the recommended prior for both?
Answer: Alternatively, the reference prior for the ordering $\alpha$, $\beta$ is (http://www.stats.org.uk/priors/noninformative/YangBerger1998.pdf page 13):
$$
\Pi(\alpha,\beta) \propto \frac{\sqrt{\alpha PG(1,\alpha)-1}}{\sqrt{\alpha}\beta}
$$
where $PG(1,x)=\sum_{i=0}^{\infty} (x+ i)^{-2}$ is the polygamma function. It results in proper posteriors.
Comment: Watch out, they use the location / scale parametrization but denote it with $\alpha$ and $\beta$.
Answer: Half-Cauchy family was recommended instead of the inverse-gamma prior for scale parameters. Gelman (2006) recommended this because the inverse-gamma prior could be sensitive in inference problems if the variance estimates are close to zero. The density function of half-Cauchy is as follows (it only takes one parameter, $d$):
$$
f(x|d) = \frac{2d}{\pi\left(d^{2} + x^{2}\right)}, \quad x>0, d>0
$$
Therefore, you can use half-Cauchy(2.5) for parameters greater than zero.


How to determine the Radius of convergence?

Question: I have the function $$ \sum_{n=1}^{\infty} \frac{(-3x)^n}{n^2}$$
I´m not realy sure where to begin and how to determine the radius of convergence. Could someone provide a nice explanation?
THX
Answer: $$\lim\limits_{n\to\infty}\left|\frac{a_{n+1}}{a_n}\right|={}\lim\limits_{n\to\infty}\left|\frac{\frac{(-3x)^{n+1}}{(n+1)^2}}{\frac{(-3x)^{n}}{(n)^2}}\right|=\lim\limits_{n\to\infty}3|x|\left(\frac{n}{n+1}\right)^2=3|x|$$ So if $|x|<\frac{1}{3}$ the series convergent. On the other hand if $x=\frac{1}{3}$ and $x=\frac{-1}{3}$
series converges, too. So domain of convergence is $\left[\frac{-1}{3}, \frac{1}{3}\right] $ and radius of convergence is $R=\frac{1}{3}$.
Answer: Here is the formula
$a_n={(-3)^n\over n^2}$
${1\over R}=\lim\sup|a_n|^{1\over n}$


DateTimePicker, obter primeiro e último dia do mês

Question: Como faço para pegar o primeiro e o último dia do mês anterior, para preencher um DateTimePicker em tela?
Ex:
Data Inicial 01/07/2015
Data Final   31/07/2015

Comment: Poste o que já tentou fazer.
Comment: Você quer fazer algo especificamente no controle `DateTimePicker` ou só quer ter esta informação em uma data qualquer? Coloque o que fez, explique melhor onde você quer chegar, qual o resultado deseja.
Answer: A pergunta não está muito clara mas deve ser algo assim:
var data = picker.Value; //pega a data que está no controle
var mesAnterior = data.AddMonths(-1);
var primeiroDia = new DateTime(mesAnterior.Year, mesAnterior.Month, 1);
var ultimoDia =  new DateTime(mesAnterior.Year, mesAnterior.Month,
        DateTime.DaysInMonth(mesAnterior.Year, mesAnterior.Month));

Veja funcionando no ideone. E no .NET Fiddle. Também coloquei no GitHub para referência futura.
Tem inúmeras outras fórmulas de fazer isto. Cada uma pode ter uma vantagem em relação a outra mas em geral fará pouca ou nenhuma nenhuma diferença em coisas simples. Fiz um teste abaixo usando métodos de extensão, assim pode usar como se fosse parte do tipo.
Comment: Muito Obrigado Funcionou perfeitamente .
Answer: Existe uma resposta aqui
Vê se isso te ajuda também:
DateTime dtUltimoDiaMes = new DateTime(DateTime.Now.Year, DateTime.Now.Month, 1).AddMonths(1).AddDays(-1);

Lembre-se disso. O último dia do mês anterior é o primeiro_dia_mês_corrente menos_um_dia(AddDays(-1)). Essa é a fórmula.
Comment: "Alguém sabe me dizer como faço para pegar o primeiro dia e o ultimo dia do **mês anterior** ?"
Comment: Só estava a apontar que o código calcula o ultimo dia do mês currente, enquanto que a pergunta pede o ultimo dia do mês anterior. Um pequeno (mas importante) detalhe.
Comment: @dcastro, entendi, mas se você ver o código que eu postei, ele pega um determinado mês e volta um dia, o que vai ser o último dia do mês anterior. Acho que é isso que ele pediu, pelo que eu li.


Image resize to fit screen

Question: I created this code to resize photos/images to fit the screen, considering the space available for the nav bar.
The script fires on image load and on navigation click.
Does anyone have suggestions as to making this better and ensuring browser compatibility?
HTML
$(document).ready(function(){
 $("#photo").load(function(){
  resize();
 });

 $(".navigation img").click(function(){
  var imgPath = $(this).attr("src");
  $("#photo").attr({ src: imgPath });
  resize();
  return false;
       });
   });

Javascript
resize = function() {

    var borderVt=150; //value based on css style. bottom bar + padding of photoContain
    var borderHz=40; //value based on css style photoContain padding

    $("#photo").css("width", "auto").css("height", "auto"); // Remove existing CSS
    $("#photo").removeAttr("width").removeAttr("height"); // Remove HTML attributes

    var origSizeW = $("#photo").width();
    var origSizeH = $("#photo").height();
    var ratioVt=(origSizeW/origSizeH);
    var ratioHz=(origSizeH/origSizeW);
    var winW = $(window).width();
    var winH = $(window).height();
    var screenSizeW=Math.round(winW-borderHz);
    var screenSizeH=Math.round(winH-borderVt);

    if (origSizeW>=origSizeH){

     var newHeight = Math.round(screenSizeW*ratioHz);
     if (newHeight <= screenSizeH){
      $("#photo").css("width", screenSizeW); // Set new width
      $("#photo").css("height", newHeight);
      }
     else{
      $("#photo").css("height", screenSizeH);
      }

    }
    else{
    $("#photo").css("height", screenSizeH); // Set new height
    }
  };

Answer: I know this question is well old, but here's a solution (although I'm sure the OP's works fine too):
This jQuery plugin seems to do exactly what you need:
http://code.google.com/p/jquery-imagefit-plugin/
if you perform it on a 100% height, 100% width element:
http://www.tutwow.com/htmlcss/quick-tip-css-100-height/

(demo: http://jsfiddle.net/nottrobin/9Pbdz/)
Comment: I have up voted it, but since I am not the one, who has created the question in the first place, I can't accept it.
Comment: Just a question, does this method reduce to data amount to be sent to the client, when the image has a smaller size?
Answer: Try using the jQuery-Backgrounder plugin. You might be able to tweak it to do what you need. Here is an example:

[...]

Answer: I wrote a plugin!
jQuery.fn.positionMe = function () {
    var oH = $(window).height();
    var oW = $(window).width();
    var iH = this.outerHeight();
    var iW = this.outerWidth();

    // When the image is too small so, do nothing
    if(iW>oW && iH>oH){

        // Otherwise, proportionate the image relative
        // to its container
        if(oH/iH > oW/iW){
            this.css("width", oW);
            this.css("height", iH*(oW/iW))
        } else {
            this.css("height", oH);
            this.css("width", iW*(oH/iH));
        }

    }
    return this;
}

Usage:
$("#photo").positionMe();

Answer: Here is how I do it:
 jQuery(document).ready(function($) {
      $('.wp-post-image').height($(window).height());
 });

Answer: Looks good to me, but I would suggest attaching your resize function to jQuery's Window Resize Event handler. Then the image will stretch and shrink with the page.
Answer: var w=window.innerWidth||document.documentElement.clientWidth||document.body.clientWidth||document.body.offsetWidth||window.screen.availWidth;
var h=window.innerHeight||document.documentElement.clientHeight||document.body.clientHeight||document.body.offsetHeight||window.screen.availHeight;

function resize() {

    a=document.getElementsByClassName(' scale');
    for(var i=0; i=1.5){
            aW=w;
            aH=w-(w*(34/100));
            mT=(aH/2)*-1;
            mL=(aW/2)*-1;
        } else {
            aH=h;
            aW=h+(h*(66/100));
            mT=(aH/2)*-1;
            mL=(aW/2)*-1;
        }

        a[i].style.height=aH+'px';
        a[i].style.width=aW+'px';
        a[i].style.marginTop=mT+'px';
        a[i].style.marginLeft=mL+'px';
    }
}

Comment: above code is pure javascript. Center-Fit the images in all browsers. Just find out the ratio of ur image i.e. image-size 1500 x 1000= 1.5 now use this ratio to compare.
Comment: While this may answer the question it's always a good idea to put some text in your answer to explain what you're doing. Read [how to write a good answer](http://stackoverflow.com/help/how-to-answer).

Image resize to fit screen


how MIX subtractions works with "packed" words

Question: Im reading the Knuth's book TAOCP. And im just learning a simple math operations with registers. And there is an example of subtraction operation:
rA before: - | 1234 | 0| 0| 9
Cell 1000: - | 2000 |  150| 0
SUB 1000
rA after:  + | 766  | 149 | ?

i understand that -1234-(-2000) = 766
but how (0 | 0)  - 150  = 149 ??
and why 9 - 0 = ?
These are "packed" words. And maybe i need to read more about them. Or can anyone explain ?
Answer: The problem is that in MIX the byte size is not fixed. In | 2000 | 150| 0, although we know that the (3:4) field contains 150 we don't know what the (3:3) and (4:4) fields are individually.
A 6-bit-per-byte computer sees the low end of the computation as
$$150\times 64 - 9 = 9591 = 149\times 64 + 55,$$so 149 is stored in the (3:4) field and 55 in the (5:5) field.
A 7-bit-per-byte computer sees the low end of the computation as
$$150\times 128 - 9 = 19191 = 149\times 128 + 119,$$ so 149 is stored in the (3:4) field and 119 in the (5:5) field.
Thus the (3:4) field is 149 both times but the (5:5) field may vary and so is shown as unknown.
Note also the the default F-code (0:5) is being used with the SUB instruction, so the whole word is being used, and that fact that there appear to be packed values involved doesn't alter that.


Alternating Least Squares (ALS) - Why two different kinds of setting for $\lambda$ and $\alpha$?

Question: I am trying to use Spark ALS to do recommendation with implicit feedback. However, I found there are two totally different kinds of settings available:
The first one is the setting used by the original paper: Collaborative Filtering for Implicit Feedback Datasets. They use $\lambda={\sim}150$ and $\alpha={\sim}40$
The other is the setting suggested by Sean Owen in this discussion, which uses $\lambda=1$ and $\alpha=40$.
Through I don't very clearly understand the underlying mechanism of implicit feedback recommendation. I have to admit that the recommendations made from Owen's setting is much better than those from the original setting.
Can someone explain what makes this difference?
Answer: The term $\lambda$ is used for regularization here and $\alpha$ is a constant controlling the rate of increase of the confidence matrix $C_{ui}$. The confidence matrix is your confidence in observing a particular preference a user, $u$, has on an item, $i$.
Remember that regularization is to help reduce overfitting by penalizing $X_u$ and $Y_i$ when minimizing the overall loss function.
Comment: Is there a guideline for dummy that if the value of lambda and alpha is smaller or larger, the more the accuracy increase?
Comment: @tom10271 There is no clear cut values to start with as you always have to tune these problems to your particular dataset. Having said that, starting with values similar to the paper is a good place to start.


Content Type and Sub Pages?

Question: I have a drupal 6 instance and I want to make it so when I create a content type (for example movies) it would create 2 sub pages (reviews, and photos). So if I went to
www.site.com/movie-BLAHBLAH/ it would load the node content type info
and
www.site.com/movie-BLAHBLAH/reviews/ it would display a page where I could theme the page.tpl.php file to do what i want it to do. and same with the other one.
Is this possible somehow?
Answer: For Drupal 6, there is the CCK Fieldgroup Tabs module.

Display CCK fieldgroups in tabs. Enables splitting up content onto tabs in both editing and display.
Requires Tabs module as well as Content and Fieldgroup modules, part of CCK. For Drupal 5.x, Tabs module is part of Javascript Tools. For Drupal 6.x, Tabs is a separate module.

That project is seeking new maintainers and no longer developed by its maintainers. It does not have any release for Drupal 7 or more recent Drupal releases, which means that, as the project page for that module suggests, a Drupal 7 or a Drupal 8 site should use the Field Group module, which is now compatible with Drupal 9 and Drupal 10.

Fieldgroup will, as the name implies, group fields together. All fieldable entities will have the possibility to add groups to wrap their fields together. Fieldgroup comes with default HTML wrappers like vertical tabs, horizontal tabs, accordions, fieldsets or div wrappers.
Answer: Yes you can do this. Just like the node module has create an url where all nodes are visible: node/[nid] you could create path for node/[nid]/review and create a custom page that display the reviews for that node.
This is done by the use of hook_menu.
Answer: It sounds to me like what you really ought to do is create 3 content types, movies, reviews & photos. And use something like references or relation to keep them related to each other.

Content Type and Sub Pages?


jQuery progress bar using bluimp library

Question: I am trying to implement progress bar while uploading the files using bluimp library, however there is no enough documentation available on there site.
Also, in there demo they have shown remaining time, bytes, percentage etc while uploading the files. However, I am unable any sort of explanation on net and onto there site.
So anyone can please provide me some guideline to do this. I am trying this from many days.
Below is my html

        0% complete

and this is my json
$(document).ready(function () {

  $('#fileupload').fileupload({
    url: '/eSignature/Document/UploadFiles',
    forceIframeTransport: false,
    autoUpload: true,
    progressInterval: 100,
    bitrateInterval: 500,
    singleFileUploads: false,
    recalculateProgress: true,
    maxFileSize: 1024,
    done: function (e, data) {
        $("#divResult").html(data.result);
    },
    progressall: function (e, data) {

        $("#divProgress").show();

        var progress = parseInt(data.loaded / data.total * 100, 10);

        $('#divProgress .progress-bar').css(
            'width',
            progress + '%'
        );

    }
  });
});

Comment: where is the initialization script? and what is the problem you are facing?
Comment: @CerlinBoss I didn't get you for initialization script ?? I am facing problem while showing percentage,time,uploaded bytes etc and how to stop the progress bar after upload finished.
Comment: the initializing script will look something similar to `$('#fileupload').fileupload()`
Answer: Not sure how your Requests get handled, but for the MVC4 devs.
You need the following libraries (Get from NuGet in Visual Studio or console):

jQuery
jQuery UI
jQuery UI Core : Widget
jQuery.FileUpload

Add this in your BundlesConfig.cs
             bundles.Add(new ScriptBundle("~/bundles/bluimp").Include(
                        "~/Scripts/jquery-ui-1.11.2.min.js",
                        "~/Scripts/jQuery.FileUpload/jquery-2.1.1.js",
                        //"~/Scripts/jquery.ui.widget.min.js",
                        //"~/Scripts/cors/jQuery.FileUpload/jquery.xdr-transport.js",
                        //"~/Scripts/cors/jQuery.FileUpload/jquery.postmessage-transport.js",
                        //"~/Scripts/jQuery.FileUpload/jquery.fileupload-angular.js",
                        //"~/Scripts/jQuery.FileUpload/jquery.fileupload.js",
                        //"~/Scripts/jQuery.FileUpload/jquery.fileupload-image.js",
                        //"~/Scripts/jQuery.FileUpload/jquery.iframe-transport.js",
                        //"~/Scripts/jQuery.FileUpload/jquery.fileupload-jquery-ui.js",
                        //"~/Scripts/jQuery.FileUpload/jquery.fileupload-ui.js",
                        //"~/Scripts/jQuery.FileUpload/jquery.fileupload-process"));

jquery.xxx.js should be included on your _Layout.cshtml page. Add the following to your Upload page at the bottom somewhere.
    @section Scripts {
    @Scripts.Render("~/bundles/bluimp")
}

Then finally make sure it's an asynchronous (async) call, so make use of a WebApi call (and not a vanilla controller. Then also add the the following to the style (BundlesConfig.cs with a StyleBundle):
                      "~/Content/jQuery.FileUpload/css/jquery.fileupload",
                  "~/Content/jQuery.FileUpload/css/jquery.fileupload-ui.css",
                  "~/Content/jQuery.FileUpload/css/jquery.fileupload-ui-noscript.css",
                  "~/Content/jQuery.FileUpload/css/jquery.fileupload-ui.css",

Hope it helps (someone)


Regex to split PascalCase string into separate words

Question: Previously to split a PascalCase string into separate words I used the regular expression below:
(.)([A-Z]|\d+)

This will work for inputs such as Photo100, PhotoMax, PhotoMaxOrig. But when faced with a string like Photo100Orig my regular expression above doesn't split that string into three parts.
How can I more elegantly modify my regular expression to support strings like Photo100Orig?
Comment: Try `(.)([A-Z]|\d*(?=\d))`
Comment: If your string contains only ascii letters and digits, you can use `\B([A-Z]|\d+)` with ` $1` as replacement.
Answer: Assuming C# as your language,
var pattern = @"(?<!^)(?=[A-Z]|(?<!\d)\d)";

can be used in
var words = Regex.Split(input, pattern);

to get an array of the word components.


Generate random float number in column as element

Question: Is there any way I can generate random float array as element in 'Price' column?
import pandas as pd
data = pd.read_csv("C:\\users\\Hp\\Downloads\\Stock.csv")
data = data.rename(columns={'Date': 'Exportation Date'})
data['Exportation Month'] = data['Exportation Date'].str[6:7]

data['Price'] = 1.2

data.head()

Output (I deleted some columns due to large size that couldn't fit the question section):
Exportation Month   Price
0   2               1.2
1   6               1.2
2   1               1.2
3   8               1.2
4   8               1.2

Output I wanted (Assume those numbers in 'Price' are randomly generated):
Exportation Month   Price
0   2               5.4
1   6               2.5
2   1               4.4
3   8               17.1
4   8               2.3

Answer: Assuming you want a price between 0 and 100 with 1 decimal, you can do it that way:
from random import uniform

data['Price'] = [round(uniform(0, 100), 1) for i in range(len(data))]

Answer: I'd do something like this:
from random import randrange

randObj = f'{randrange(99)}.{randrange(99)}'
data['Price'] = float(randObj)

Comment: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answer: You can just use numpy.random
import pandas as pd
import numpy as np

# Fake Data
df = pd.DataFrame({"a": [1, 2],
                   "b": [3, 4]})
# Add Price
df["Price"] = np.random.uniform(low=0, high=10, size=len(df))

# Eventually use 2 decimal places only.
df["Price"] = df["Price"].round(2)


How to Hide post that have same category

Question: is there someone who knows how to hide post that have same category?
I have Post 1, Post 2, Post 3 that have same category. Can i just show the latest post ( post 1) at my homepage, and hide other post ( post 2 and post 3 )
Answer:
You can achieve this by alter the query of home page throught pre_get_posts hook.

Here is the code:
function txt_domain_get_distinct_post_id()
{
    $post_ids = [];

    //getting all non empty category
    $categories = get_terms('category', array(
        'orderby' => 'count',
        'hide_empty' => 0,
    ));

    foreach ($categories as $category)
    {
        $args = array(
            'posts_per_page' => -1,
            'category' => $category->term_id,
            'exclude' => $post_ids,
            'orderby' => 'date',
            'order' => 'DESC',
            'post_type' => 'post',
            'post_status' => 'publish'
        );
        //getting post from a specific category
        $postlist = get_posts($args);
        foreach ($postlist as $post)
        {
            if (!in_array($post->ID, $post_ids))
            {
                //this will only select one post form each category
                $post_ids[] = $post->ID;
                break;
            }
        }
    }
    return $post_ids;
}

add_filter( 'pre_get_posts', 'txt_domain_filter_get_posts' );

function txt_domain_filter_get_posts($query)
{
    if (is_home() && $query->is_main_query())
    {
        $post_ids = txt_domain_get_distinct_post_id();
        $query->set('post__in', $post_ids);
    }
    return $query;
}

Code goes in function.php file of your active child theme (or theme). Or also in any plugin php files.

Reference:

is_home()
get_terms
get_posts
Answer: One way of doing that is to have three separate wp_queries.
//Query 1
$args1 = array(
    'cat'                    => '1',
    'nopaging'               => true,
    'posts_per_page'         => '1',
);    $query_1 = new WP_Query( $args1 );
if ( $query_1->have_posts() ) {
    while ( $query_1->have_posts() ) {
        $query_1->the_post();
        // put content here
    }
} else {
    // no posts found
}
wp_reset_postdata();

//Query 2
$args2 = array(
    'cat'                    => '2',
    'nopaging'               => true,
    'posts_per_page'         => '1',
);    $query_2 = new WP_Query( $args2 );
if ( $query_2->have_posts() ) {
    while ( $query_2->have_posts() ) {
        $query_2->the_post();
        // put content here
    }
} else {
    // no posts found
}
wp_reset_postdata();

//Query 3
$args3 = array(
    'cat'                    => '3',
    'nopaging'               => true,
    'posts_per_page'         => '1',
);    $query_3 = new WP_Query( $args3 );
if ( $query_3->have_posts() ) {
    while ( $query_3->have_posts() ) {
        $query_3->the_post();
        // put content here
    }
} else {
    // no posts found
}
wp_reset_postdata();

Answer:     <?php
    $args = array(
        'numberposts' => 1,
        'offset' => 0,
        'category' => 0,
        'orderby' => 'post_date',
        'order' => 'DESC',
        'post_type' => 'post',
        'post_status' => 'draft, publish, future, pending, private',
        'suppress_filters' => true
    );

    // the query
    $the_query = new WP_Query( $args );?>

<?php if ( $the_query->have_posts() ) : ?>

        <?php while ( $the_query->have_posts() ) : $the_query->the_post(); ?>

<?php the_title(); ?>

        <?php endwhile; ?>

        <?php wp_reset_postdata(); ?>

    <?php else : ?>

<?php _e( 'Sorry, no posts matched your criteria.' ); ?>

    <?php endif; ?>

Add your category ID to 'category'.
reefer these links for more information.
https://codex.wordpress.org/Class_Reference/WP_Query
https://codex.wordpress.org/Function_Reference/wp_get_recent_posts


how to convert 1,000,000 gifs into a video file using ffmpeg

Question: i have 1 million gifs in a folder that i would like to convert into a 60 frames per second video file using ffmpeg. Here is my code:
ffmpeg -r 60 -pattern_type glob -i *.gif -c:v libx264 out.mp4

my gifs don't follow a %07d.gif pattern like 0000001.gif 0000002.gif but they are indeed in sequential order. like this: img1.gif, img2.gif ... img10.gif ... img10000.gif ... img1000000.gif.
When i try to execute my code i get this:
-bash: /usr/local/bin/ffmpeg: Argument list too long

please help. thank you.
Answer: You need to avoid the pattern being expanded by your command shell, and instead pass it to the program as is. This can be achieved with quotes.
The man page has this example
 ffmpeg -r 10 -f image2 -i 'img-%03d.jpeg' out.avi

So in your case, you could try
 ffmpeg -r 60 -i '%07d.gif' -c:v libx264 out.mp4

Comment: Try `img%d.gif` then.
Comment: still doesn't like it - img%d.gif: No such file or directory
Answer: The shell is performing file name expansion on the given commandline, namely *.gif.
Look here on how to prevent it using quotes.
Also, the sequence pattern type looks more useful in your case. Something like the following should work:
ffmpeg -r 60 -pattern_type sequence -i "img%d.gif" -c:v libx264 out.mp4

If I understand correctly, the problem with globbing is that it will sort your file in alphabetical order i.e. img-1.gif, img10.gif, ..., img2.gif, img20.gif...


How to mass copy the title of each node to a taxonomy term?

Question: I need to copy the title of nodes to its taxonomy term.
I have the following content type

Title - text
Body - text
Taxonomy term - (currently empty)

I want to copy the "Title" ---> "Taxonomy term" for each node.How can I do this?
I understand that VBO with som php-snippet could solve this but I am no programmer. Any suggestions about a working snippet or other solution?
Answer: You could use a combination of the Rules module and the VBO module to create a custom rule component, which you then use as a custom VBO action.
For way more details on this combination, have a look at my answer to "How to bulk update select list fields?".
In your case, the custom Rules Component should look pretty similar to this:
{ "rules_set_an_initial_value_for_a_term_field" : {
    "LABEL" : "Set an initial value for a term field",
    "PLUGIN" : "rule",
    "OWNER" : "rules",
    "REQUIRES" : [ "rules" ],
    "USES VARIABLES" : { "node" : { "label" : "Node", "type" : "node" } },
    "IF" : [
      { "entity_has_field" : { "entity" : [ "node" ], "field" : "field_yourfield" } }
    ],
    "DO" : [
      { "data_set" : { "data" : [ "node:field-yourfield" ], "value" : "HereGoesTheTitle" } }
    ]
  }
}

Some more details to further explain the above example:

adapt the machinename of field_yourfield to the machine name for your taxonomy term field.
replace the (hardcoded) HereGoesTheTitle value in the Rules Action by the token like node:title (which for sure wil be available).
I'm assuming your taxonomy term is a single value field, if it is a multi value field, then replace the action with a similar one to "Add an item to a list" (related to your multi value field).

PS: no need at all for any custom PHP code when using this approach.


How to check the values from the array in the loop

Question: <?php $callout = array(
    '1',
    '3',
    '5'
); ?>

<?php foreach($callout as $call) { ?>

<?php echo get_the_title($call); ?>

different text goes here...

<?php } ?>

Is it possible - if $call=1 then echo some static text, if 2 echo something else, and so on?
Comment: refer to this .I guess it will help you  http://codepad.org/0mZIkJuB
Comment: use php `if-else` statement or `switch`
Answer: Problem solved. Thanks to @swapnesh
http://codepad.org/0mZIkJuB
<?php $callout = array(1,2,3); ?>

<?php foreach($callout as $call) { ?>

        <?php
            if ('1' == $call)
                echo "a";
            elseif ('2' == $call)
                echo "b";
            elseif ('3' == $call)
                echo "c";
            else
                echo "d";
        ?>

<?php } ?>


mysql find all the possible combinations of one column

Question: I have the table UserContacts with a column like
contact_id

1
5
6

and I want to get the following result (which is all the possible pair combinations of the values of that column)...
(1,5),(1,6),(5,6)
is it possible?
Answer: Sure, make a self-join:
SELECT a.contact_id a, b.contact_id b
FROM   UserContacts a
  JOIN UserContacts b ON b.contact_id > a.contact_id

See it on sqlfiddle.
Comment: @Edward: you'd need to join the table once for each column, and relate the columns in turn eg `a > b` and `b > c` etc.
Comment: yeah, that's perfect.. it was so simple. Thanks
Comment: I know this is an old answer but how would you extend this solutions to create unique combinations of more than two unique columns  -  to three, four, or five columns?
Answer: This should work
SELECT t2.contact_id,t1.contact_id FROM UserContacts t1 JOIN UserContacts t2
WHERE t1.contact_id <> t2.contact_id

Answer: USE CROSS JOIN
SELECT a.contact_id
FROM   UserContacts a
  CROSS JOIN UserContacts b


Bijection from $\mathbb{R}^2$ to $\mathbb{R}^2 \setminus \{ (x, 2x+3) : x \in \mathbb{R}\}$

Question: So, I need to construct some bijection from $\mathbb{R}^2$ to $\mathbb{R}^2 \setminus \{ (x, 2x+3) : x \in \mathbb{R}\}$.
First I tried constructing $f : \mathbb{R}^2 \to \mathbb{R}^2 \setminus \{ (x, 2x+3) : x \in \mathbb{R}\}$, but I couldn't find a way to "remove" the line $y = 2x+3$ from range of $f$.
Then I tried constructing $f : \mathbb{R}^2 \setminus \{ (x, 2x+3) : x \in \mathbb{R}\} \to \mathbb{R}^2$, but I couldn't get it to have the entire $\mathbb{R}^2$ as range. Explicitly, I tried $f(x,y) = (\frac{1}{2x+3-y}, y)$, but obviously $(0, x) \notin rng(f), \forall x \in \mathbb{R}$, where $rng$ is range.
Answer: It is easy to move the line you have to $l=\{(0,y)\mid y\in\mathbb{R}\}$. Now, note that $\mathbb{R}^2\setminus l=\mathbb{R}^*\times\mathbb{R}$ and we are searching for a bijection from this set to $\mathbb{R}\times\mathbb{R}$ so it suffices to find one $f:\mathbb{R}\longrightarrow\mathbb{R}^*$. So let's define $f(x)=
\begin{cases}
x\quad\text{if } x\notin\mathbb{N}\\
x+1\quad\text{if }x\in\mathbb{N}
\end{cases}$
You could obviously compose the three steps to get an actual expression for the wanted bijection, but thought this way was more clear.
Answer: You can use the idea of a bijection between $\Bbb R$ and $\Bbb R \setminus \{0\}$, just to push it down the line.  For the $1D$ case, you just send $0$ to $1, 1$ to $2, 2$ to $3$ and so on.  Here we can just send the line upward by one unit, then send that line upward, and so on.  So $$f(x,y)=\begin {cases} (x,y+1)&y=2x+2+k, k\in \Bbb N\\(x,y)&\text{otherwise} \end {cases}$$ where I have taken $0 \not \in \Bbb N$


make operation between to unsigned number when variables can take negative value

Question: How I make one operation Between when I do queries like these:
types.init and types.end are an unsigned number, and for this case B.init=0 and B.end=0
SET @init:=20;
SET @end:=100;

SELECT
  A.`type_id`, B.`type_id`,count(A.id)
FROM
  `types` A ,`types` B
WHERE
  A.`init` BETWEEN B.`init`- @init
  AND B.`init` +  @init
  AND  A.`end` BETWEEN B.`end`-  @end
  AND B.`end` +  @end

then this error occurs:
 #1690 - BIGINT UNSIGNED value is out of range in '(`database`.`B`.`init` - (@init))'

Comment: You should probably provide a data sample and show what you want the results to be for the row(s) with `B.init=0` and/or `B.end=0`. Or just tell us without the example how the beginning bounds should be calculated when `@init > types.init` and/or `@end > types.end`.
Answer: you either need to use CAST to convert to SIGNED, or use CASE to check if B.init >= @init.
For the former case your query will look like this:
WHERE
  CAST(A.`init` AS SIGNED) BETWEEN CAST(B.`init` AS SIGNED) - @init
  AND CAST(B.`init` AS SIGNED) +  @init
  AND CAST(A.`end` AS SIGNED) BETWEEN CAST(B.`end` AS SIGNED) -  @end
  AND CAST(B.`end` AS SIGNED) +  @end

For the latter:
WHERE
  A.`init` BETWEEN CASE WHEN B.`init` >= @init THEN B.`init`- @init ELSE 0 END
  AND B.`init` +  @init
  AND  A.`end` BETWEEN CASE WHEN B.`end` >= @end THEN B.`end`-  @end ELSE 0 END
  AND B.`end` +  @end


HAProxy - How to raise the log level of responses with status code above 400

Question: Here's my sample configuration:
frontend www-http
    bind *:80
    acl status_error status ge 400
    http-response set-log-level err if status_error
    default_backend www-backend

backend www-backend
    server backend 127.0.0.1:8080

With this config, all responses with status 200 are being logged but 400 and below isn't. If I remove the acl and http-response line, then everything is logged.
I've added those lines to the default config, and I didn't touch any rsyslog configuration. I've tested this in HAProxy 1.5.8 (Debian 8) and 1.6.3 (Ubuntu 16.04).
Comment: I think this may be a bug.  Judging from [a patch submitted earlier today on the mailing list](http://permalink.gmane.org/gmane.comp.web.haproxy/28328), it looks like there may be a fairly subtle error in the configuration parser that results in the log-level being set to "silent" no matter what you supply.
Comment: @michael-sqlbot Yeah, I read that too. So I think I'll just wait for it to be fixed. In the meantime, I used `option log-separate-errors` with `rspdeny . if { status ge 400 }` to transform 4xx errors into a 502 error with log level err.
Answer: (It's four years later, but perhaps still relevant.)
Using HAProxy 2.2 this approach works for me to have conditional and HTTP-only requests/connection logging on one frontend:
global
  # 'notice' level as global minimum (does not include http request logs)
  log /dev/log local0 notice
  [...]

defaults
  log global

frontend myhttpsfrontend
  bind [...]
  mode http

  # enables logging of all HTTP requests at info level
  option httplog

  # raise loglevel from info to notice with status code >= 400
  http-response set-log-level notice if { status ge 400 }

I chose to have notice level globally, as TCP backends could be very noisy if the global level is info. Note that notice is a higher severity level than info in syslog; that was one part confusing me at first.
The condition if { status ge 400} is an inline expression as an example, but you could also use an ACL or some other condition of course.
Relevant basics on HAProxy logging in this blog: haproxy.com/blog: Introduction to HAProxy Logging.


Fitting Large Matrix Calculations into Memory when using Tensorflow

Question: I am attempting to build a model which has two phases.
The first takes an input image and passes it through a conv-deconv network. The resulting Tensor has entries corresponding to pixels in a desired output image (same size as the input image).
To calculate the final output image I want to take the value generated at each pixel location from the first phase and use it as an additional input to a reduction function that is applied over the entire input image. This second step has no trainable variables, but it does have computation/memory costs that grow exponentially with the size of the input (each output pixel is a function of all input pixels).
I'm currently using the tf.map_fn to calculate the output image. I'm mapping the output pixel calculation function onto the results from the first phase. My desire is that tensorflow would allocate the memory to store the intermediate tensors needed for each pixel calculation and then free that memory before moving on to the next pixel calculation. But instead it seems to never free the intermediate calculations causing OOM errors.
Is there someway to tell tensorflow (either explicitly or implicitly) that  it should free the memory allocated to hold the data of a Tensor that is no longer needed in the calculation?
Comment: Are you partitioning your computation into multiple tensors? (ie, TensorFlow won't deallocate a part of a tensor object)
Comment: Yes. I did originally try to do giant matrix operations, but it makes more sense to me why that wasn't working. This is what led me to tf.map_fn() to create tensors for each individual pixel calculation.
Answer: TensorFlow deallocates memory for the tensor as soon as the tensor is no longer needed for any future calculations. You can verify this by looking at memory deallocation messages as shown in this notebook.
It's possible you are running out of memory because TensorFlow executes nodes in a memory inefficient order.
As an example, consider following computation:
k = 2000
a = tf.random_uniform(shape=(k,k))
for i in range(n):
    a = tf.matmul(a, tf.random_uniform(shape=(k,k)))

The order in which it is evaluated can be shown below

All the circles (tf.random_uniform) nodes are evaluated first, followed by squares (tf.matmul). This has O(n) memory requirement compared to O(1) for the optimal order.
You can use control dependencies to force a specific execution order, ie, using helper function as below:
import tensorflow.contrib.graph_editor as ge
def run_after(a_tensor, b_tensor):
    """Force a to run after b"""
    ge.reroute.add_control_inputs(a_tensor.op, [b_tensor.op])

Comment: Thanks, it's good to hear that if I can get dependencies right this should be taken care of automatically. In my case I'm using the map_fn so it doesn't seem immediately apparent to me how to force the operations to be done sequentially (setting parallel_iterations=1 didn't seem to solve my problem like I thought it would, btw). Does building a collection of output tensors created by the mapping function then stringing them together with dependencies as you suggested seem like the proper approach?
Comment: Hm, map_fn works on slices of tensors, it doesnt seem possible to deallocate memory for unneeded slices, why not use regular Python map?
Comment: That does make sense. I think I just reflexively tended towards using tensorflow ops, but I suppose it might be overkill. I'm not worried about the size of the inputs/outputs to tf.map_fn though, just the intermediate tensors generated within the mapping function (if that makes sense). In the python map case I would still need to build dependency chains though right?
Comment: When I pass a python function that builds a portion of a graph (and returns a Tensor) as the map_fn fn argument I assumed all tensorflow does is stamp down lots of instances of that graph structure connected to the tensor slices, right?
Comment: No, it's more efficient than that
Comment: OK, I will try python control then. Does my earlier statement about still requiring control dependencies make sense?
Comment: Yes you'll need control deps to force nodes to run in particular order


Maximum Likelihood Estimator for equicorrelation model

Question: Consider the equicorrelation model for multivariate normal. Let $X_1, \dots, X_n\sim \mathbf{N_p(\mu, \Sigma)}$, where $\mathbf{\Sigma}=\sigma^2((1-\rho)\mathbf{I_p}+\rho\mathbf{J_p})$ where $\mathbf{J_p}$ is a matrix of all $1$'s. Now how do we find the maximum likelihood estimator of $\sigma^2, \rho$?. Estimating $\mathbf\mu$ is easier, it comes out to be $\overline{X}$. Can someone help me out? If I take inverses and try to differentiate the likelihood it gets quite messy. Is there a slick method? Thanks a lot.
Comment: This kind of covariance is named "compound symmetric" matrix, which can help in searches.
Comment: You use both $n$ and $p$. It appears wrong.
Answer: The useful formulas have been given in the answer by @Alecos
Papadopoulos, which I will refer to. Note that $\mathbf{\Sigma}$ is
positive definite iff $\rho_{\text{min}} < \rho < 1$ with
$\rho_{\text{min}} := - 1/(p-1)$.
For the ML estimation you can concentrate the vector $\boldsymbol{\mu}$
and the scalar $\sigma^2$ out of the log-likelihood function by replacing them by
their ML estimates, namely $\widehat{\boldsymbol{\mu}} = \bar{\mathbf{X}}$ and
$$
    \widehat{\sigma}^2 = \frac{1}{np} \, \sum_{i = 1}^n
    \left[\mathbf{X}_i - \bar{\mathbf{X}} \right]^\top \boldsymbol{\Sigma}^{\star -1}
      \left[\mathbf{X}_i - \bar{\mathbf{X}}\right]
$$
where $\boldsymbol{\Sigma}^\star:= \sigma^{-2} \boldsymbol{\Sigma}$. Thus the estimation
boils down to the one-dimensional optimisation of a function of $\rho$.
An interesting point is that for a vector $\mathbf{z}$ of length $p$ we have
$$
  \mathbf{z}^\top  \boldsymbol{\Sigma}^{\star -1} \mathbf{z} = \frac{1}{1-\rho}
  \left[\mathbf{z}^\top \mathbf{z} - \nu (\mathbf{1}_p^\top \mathbf{z})^2 \right]
$$
where $\mathbf{1}_p$ is a vector of $p$ ones and $\nu:= \rho\, / \, [1 + (p-1) \rho]$.
So taking $\mathbf{z}_i:= \mathbf{X}_i - \bar{\mathbf{X}}$ we can use the sums
$$
   A := \sum_i \mathbf{z}_i^\top \mathbf{z}_i, \qquad
   B := \sum_i (\mathbf{1}_p^\top \mathbf{z}_i)^2
$$
which do not depend on the parameter.
The concentrated log-likelihood obtained after replacing $\boldsymbol{\mu}$ and $\sigma^2$
by their estimate is
$$
  \ell_{\text{c}}(\rho) = -\frac{np}{2} \, \log(2 \pi) -
\frac{n}{2}\,\log \widehat{\sigma}^2 -
 \frac{n}{2}\,\log|\boldsymbol{\Sigma}^\star| -
  \frac{np}{2}.
$$
which is easily maximised.
<code>##' MLE of a normal vector with compound symmetric covariance.
##' @title Compound symmetry
##' @param X A matrix with n row and p columns.
##' @return A list of ML Estimates.
MLE <- function(X) {

    n <- nrow(X)
    p <- ncol(X)
    p1  <- p - 1
    rho.min <- - 1 / p1

    ## substract column means. We could use 'scale' as well
    xbar <- apply(X, 2, mean)
    Xcent <- sweep(X, MARGIN = 2, STATS = xbar, FUN = "-")

    ## sum of 'n' crossprods and sum of 'n' squared sum of 'p'
    ## components
    A <- sum(apply(Xcent, MARGIN = 1, FUN = crossprod))
    B <- sum(apply(Xcent, MARGIN = 1, sum)^2)

    ## concentrated (or profiled) deviance = - 2 log L, to be
    ## minimised
    deviance  <- function(rho1) {
        nu <- rho1  / (1.0 + p1 * rho1)
        sigma2.hat <- (A - nu * B) / n / p / (1.0 - rho1)
        dev <- n * (p * log(sigma2.hat) + p1 * log(1.0 - rho1)
                    + log(1.0 + p1 * rho1) + p)
        attr(dev, "sigma2.hat") <- sigma2.hat
        dev
    }

    opt <- optimise(deviance,  interval = c(rho.min, 1.0))
    list(mu = xbar,
         sigma2 = attr(opt$objective, "sigma2.hat"),
         rho = opt$minimum,
         loglik = -opt$objective[1] / 2)
}

## Now try the function change/remove set.seed for other examples
set.seed(123)
p <- 10L; n <- 1000
rho.min <- -1 / (p - 1)

## draw 'rho', 'mu' and 'sigma2'
rho <- runif(1, min = rho.min, max = 1)
mu <- rnorm(p)
sigma2 <- rexp(1)

## build a p * p covariance matrix 'Sigma'
Sigma <- matrix(rho, nrow = p, ncol = p)
diag(Sigma) <- 1
Sigma <- sigma2 * Sigma

## build the matrix 'X'
G <- chol(Sigma)
X <- matrix(rnorm(n * p), nrow = n, ncol = p) %*% G
X <- sweep(X, MARGIN = 2, STATS = mu, FUN = "+")

##
MLE(X)
</code>
Answer: One can verify that the inverse of this matrix equals
$$\mathbf{\Sigma}^{-1} =\frac {1}{\sigma^2(1-\rho)}\left(\mathbf{I_p}-\frac{\rho}{1+(p-1)\rho}\mathbf{J_p}\right)$$
Its determinant is
$$|\mathbf{\Sigma}| = \sigma^{2p}\cdot [1+(p-1)\rho]\cdot (1-\rho)^{p-1}$$
(see Tong 1990, p.104)
I don't think it gets simpler than that (and it is not very messy either).
Beware of asymptotics (which are always prominent in maximum likelihood estimation), since the matrix may become singular at the limit. And MLE with a sigular covariance matrix is a whole different ball-game. If I remember correctly, the existing theoretical results are acrimonious to follow in such a case.
Comment: Note that the variance is positive definite iff $-1/(p-1) < \rho < 1$. The parameter $\boldsymbol{\mu}$ and $\sigma^2$ can be concentrated out of the log-likelihood, leading to a one-dimensional optimisation problem.
Answer: I found this question in an exercise of Seber's Multivariate Observations (1984).
I don't have a proof but apparently the maximum likelihood estimators of $\sigma^2$ and $\rho$ are given by
$$\hat\sigma^2=\frac1p\sum_{j=1}^p s_{jj}$$
and $$\hat\rho=\frac1{\hat\sigma^2}\sum_{j\ne k}\frac{s_{jk}}{p(p-1)}\,,$$
where $s_{jk}$ is the $(j,k)$th element of $\boldsymbol S=\frac1{n-1}\sum\limits_{i=1}^n (\boldsymbol X_i-\boldsymbol{\overline X})(\boldsymbol X_i-\boldsymbol{\overline X})^T$.
The relevant extract from page 95 of the book:

The hint given in the exercise is to find $\boldsymbol \Sigma^{-1}$ (which exists for $\rho\in \left(-\frac1{p-1},1\right)$ as pointed out by @Yves) and $\det\boldsymbol \Sigma$, as given in the answer by @Alecos Papadopoulos.
Answer: There exists explicit (closed-form) solutions for the MLE's you are struggling to find.  In fact, a little intuition, after the re-parametrization in step 1 shown below, reveals that the average of the diagonal elements of xx' is a reasonable estimator for the variance and the average of the off-diagonal elements of xx' is a reasonable estimator for the covariance.  But to be specific, follow these steps.  For simplicity, let the mean vector be equal to the zero vector.
1) Parameterize the problem so that the diagonal elements of the variance-covariance matrix are each "a" and the off-diagonal elements are each "b".
2) Use well known formulas for the inverse and determinant of the matrix constructed in step 1 to set-up the likelihood equation.
3) Re-parameterize the problem again by letting c = a - b and d = a + b*(p-1).
4) Find the MLE of c and d (very easy): you should obtain c = (x'x)/(n-1) - (x'j)^2/(n(n-1) and d = (x'j)^2/n
5) Use the invariance property of the MLE to show that a = ((n-1)*c + d)/n and b = (d-c)/p


How do I write code to handle dealing with averaging replicates with NA values in R?

Question: I am currently working on a scientific project where each sample has two or three replicates.
The data (we'll call the dataframe 'raw') looks something along the lines of:
| SampleID | Var1     | Var2     | Var3     |...
| Unknown 1 |    NA    |    12    |    NA    |
| Unknown 1 |    NA    |    19    |    23    |
| Unknown 2 |    NA    |    NA    |    50    |
| Unknown 2 |    18.5  |    NA    |    32    |
| Unknown 2 |    8.12  |    NA    |    20    |
...

Where unknown 1 has two replicates and unknown 2 has three replicates. The NAs are for when values could not be obtained since the signal was too strong/too low.
I want to create a new dataframe called 'averages' that contains the averages of the replicates for each group (Sample ID) over all the Var# columns so that it looks more like this:
Averages of each sample over different variables
| SampleID  | Var1     | Var2     | Var3     |...
| Unknown 1 |    NA    |   15.5   |    23    |
| Unknown 2 |  26.62   |    NA    |    51    |
...

What I am struggling with is handling the NAs the way I want to when calculating the averages. In the new averages dataframe, I still want it to show:

'NA' for samples where ALL replicates were 'NA'. If I use any function that omits NAs when calculating the means, any sample with replicates that contain ALL NAs in at least one variable are not added to the new averages dataframe, which is not what I want. I still want to see all the samples listed in the new dataframe, just with their replicate/triplicate values combined into averages.

In the case where (1 of 2) or (2 of 3) replicates contained NA values, I want those NA values to be omitted. For instance, in the table above, 'Unknown 2' has triplicate values and in 'Var1', there is one triplicate value that is NA whereas the other triplicate values are "18.5" and "8.12". Thus, I want the NA value to be omitted and the mean calculation to instead look like (18.5 + 8.12)/2 even though there is supposed to be 3 replicate values.

#Approach 1 - worked only for one column at a time, omits the samples whose replicates ALL have NA values
averages <- aggregate(raw$'Var1', list(raw$SampleID), FUN=mean, na.rm=TRUE)
#Approach 2 - did not not omit the NAs in mean calculation
averages <- aggregate(.~SampleID, raw, mean, na.rm=TRUE)
#Approach 3 - #tried putting the colnames into a list, then calling the list in the following code. Same issue with samples being omitted if they have replicates with NA values.
avgcolnames <- colnames(raw[,sapply(results,is.numeric)])
averages <- aggregate(cbind(avgcolnames) ~ SampleID, data = raw,
                        FUN = mean, na.rm = TRUE)

#Approach 4 - # worked on only on one column at a time and have to specify new column names
averages <- raw %>%
  group_by(SampleID) %>%
  summarize(Var1 = mean(Var1, na.rm = TRUE))

Answer: Approach #4 is along the way, next step is to make it generic, which we can do with tidy-select functionality and an anonymous function:
library(tidyverse)

iris %>%
  group_by(Species) %>%
  summarize(across(where(is.numeric), ~ mean(.x, na.rm=TRUE)))

## # A tibble: 3 × 5
##   Species    Sepal.Length Sepal.Width Petal.Length Petal.Width
##   <fct>             <dbl>       <dbl>        <dbl>       <dbl>
## 1 setosa             5.01        3.43         1.46       0.246
## 2 versicolor         5.94        2.77         4.26       1.33
## 3 virginica          6.59        2.97         5.55       2.03

Here we select all numeric columns with across(where(is.numeric), <fn>) and the ~ mean(.x, na.rm=TRUE) is an anonymous function where .x is the argument and computes the mean with NAs excluded.


Showing two cardinalities are equal

Question: The title is vague because im not sure what to call this problem:
Verify:
$|AN|$=$\frac{|A||N|}{|A \cap N|}$
for N a normal subgroup or G and A an arbitrary subgroup.
I think I can probably use Lagranges theorem and show the number of left cosets are equal, but I have no idea how to do it.
Thanks in advance!
Comment: Use the second isomorphism theorem.
Comment: Hmm..Im not sure where to start (My book calls it the diamond theorem). Im just not sure how this theorem can indicate the order of a group (say AN)  I see it says that AN is isomorphic to A/(A$\cap$N) but what does this tell me abouts AN's order?
Answer: The Second Isomorphism Theorem says that $AN/N\cong A/A\cap N$. Therefore, the order of both sides is the same. Now use the fact that if $G$ is a finite group and $N\lhd G$, then $|G/N| =|G|/|N|$.

Showing two cardinalities are equal


Dividir array em grupos menores

Question: Tenho um array nesse formato:
meuArray = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", ...];

Gostaria de transformá-lo em um array multidimensional com javascript/jquery deixando-o assim:
meuArray = [
             [0] => ["0", "1", "2", "3", "4"];
             [1] => ["5", "6", "7", "8", "9"];
              ...
           ]

A quantidade de chaves por array irá variar, porém para facilitar a resposta, podemos considerar sempre 5 chaves por array.
Comment: Porque pode variar, então, a regra deveria ser melhor explicado, qual é a verdadeira regra de criar vários arrays a partir de um?
Comment: O corte será feito de acordo com a quantidade de resultados que couber na resolução do usuário, se for 5 vou cortar de 5 em 5 e criar uma linha com esses 5 e assim por diante
Comment: Se o número de item for 6 como é dividido?
Comment: Isso é o que eu quero saber, se for de 6 em 6, eu vou atualizar a divisão dentro do sucess e passar o loop que alguém irá me responder, vou testar com as respostas atuais pra ver se consigo o que quero.
Answer: Você pode usar o método Array#slice que vai dividir o array com base no índice de início e final, iterando o array por um valor de corte, levando em conta que seu array pode possuir n tamanho total.

var meuArray = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]
  , novoArray = []
  , corte = 5;

for (var i = 0; i < meuArray.length; i = i + corte) {
  novoArray.push(meuArray.slice(i, i + corte));
}

console.log(novoArray);

Mesmo se o array possuir outro tamanho ou corte diferente de 5, o novo array será cortado. O último array ficará com o resto dos elementos.
Answer: Eu faria da seguinte forma:

Percorreria os itens do array;
Utilizaria uma variável para controlar o grupo em que o item será colocado;
Incrementaria a variável caso o resto da divisão dela pelo máximo fosse 0. (No seu caso em 5, 10, ... 25, ...);

O resultado é o seguinte:

var arrayBase = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10"];
var dados = [];

function separar(base, maximo) {
  var resultado = [[]];
  var grupo = 0;

  for (var indice = 0; indice < base.length; indice++) {
    if (resultado[grupo] === undefined) {
      resultado[grupo] = [];
    }

    resultado[grupo].push(base[indice]);

    if ((indice + 1) % maximo === 0) {
      grupo = grupo + 1;
    }
  }

  return resultado;
}

dados = separar(arrayBase, 5);
console.log(JSON.stringify(dados));

Com ES6 utilizando Math.floor, reduce e Sintaxe de Espalhamento:

const separar = (itens, maximo) => {
  return itens.reduce((acumulador, item, indice) => {
    const grupo = Math.floor(indice / maximo);
    acumulador[grupo] = [...(acumulador[grupo] || []), item];
    return acumulador;
  }, []);
};

// Teste de execução
const itens = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10'];
console.log(JSON.stringify(separar(itens, 5)));
Comment: @LeoLetto Vou fazer uma leve alteração para que passe o array como parâmetro também. Fica mais útil.
Comment: e para acessar os resultados obtidos?
Fiz isso dados = JSON.stringify(separar(5));
Porém quando tento acessar dados[0] não da em nada.
Comment: @LeoLetto `dados = separar(5);` O `JSON.stringify` é pra transformar em `string`
Comment: Essa função leva em consideração caso o meuArray tenha menos que 5 chaves? pois testei aqui e quando tem só 2 chaves por exemplo, ele retorna no array apenas das chaves obtidas no meuArray
Comment: Obrigado, funcionou do jeito que eu precisava :)
Answer: Recomendo você dar uma olhada no método push de Array: http://www.w3schools.com/jsref/jsref_push.asp
Mas um exemplo bem básico seria assim:
meuArray = [];
meuArray.push([1, 2, 3, 4, 5]);
meuArray.push([6, 7, 8, 9, 10]);

Ate a quantidade que vc precisar, recomendo usar um loop.
Answer: Usando um laço de repetição simples, você pode usar o Array.Slice para separar o array e o Array.Push para criar um novo com os elementos extraídos.

function separarArray(arr, tamanho) {
  var novoArray = [];
  var i = 0;
  while (i < arr.length) {
    novoArray.push(arr.slice(i, i + tamanho));
    i += tamanho;
  }
  return novoArray;
}

console.log(separarArray(["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10"], 5));
Answer: Caso use em vários pontos, sugiro adicionar protótipos de objetos.

var splitList = function (list, limit) {
  var originalList = [];

  var loop = Number((list.length / limit).toFixed(0)) + (!!(list.length % limit) ? 1 : 0);

  for (var index = 0, position = limit; index < loop; index++, position += limit) {
    var grup = list.slice(index * limit, position);
    if (grup.length) originalList.push(grup);
  };

  return originalList;
};

console.log(splitList([1,2,3,4,5,6,7], 4)); ///[[1,2,3,4],[5,6,7]]

Dividir array em grupos menores


Intervention Analysis Coding in R TSA Package

Question: I am studying intervention analysis in time series with the Cryer and Chan book and am looking at trying to understand how to code the step response interventions. One question I had is how to differentiate between these two models:

It appears the only difference is in (c) the value of $\delta$ is constrained to be 1. How can this constraint be added to the arimax function? I believe that (b) uses the coding
transfer = list(c(1,0))
Is there a way to constrain $\delta$ =1?
Answer: This is pretty straight forward if you use a tsoutlier package in R. This was not possible in $R$ until Thanks to @javlacalle created tsoutlier package. See the question that I posted earlier.
With regards to incorporating regressors like intervention analysis that you posted, you could use outlier.effects in the tsoutlier package to create regressors in ARIMAX model. See below for an example. This is similar  to what you have asked. You could change the $\delta$ values in the temproary change to obtain desired shape of the curve. In the example below, I have left it to be default value for $\delta$ to be 0.7. You can consult package manual for further detail. tsoutlier package is great because it works with auto.arima and automatically identifies outliers and lets you code this arimax model.
In the example below I have shown you how to incorporate Level shift and Temporary change (which is what you are looking for). The outlier package identifies a level shift at 12 and temproary change at 20 which I both created as regressors using outliers.effects function. Temporary change has a decay effect which is nicely captured in this example.
library(tsoutliers)
library(expsmooth)
library(fma)

## Identify Outliers

outlier.chicken <- tsoutliers::tso(chicken,types = c("AO","LS","TC"),maxit.iloop=10)
outlier.chicken
plot(outlier.chicken)

n <- length(chicken)

## Create Outliers Regressors for ARIMAX
## Two type of outliers Level Shift (LS) and Temprory Change (TC)

mo.ls <- outliers("LS", 12)
ls <- outliers.effects(mo.ls, n)

mo.tc <- outliers("TC", 20)
tc <- outliers.effects(mo.tc, n)

xreg.outliers <- cbind(ls,tc)

## Create Arimax using Outliers as regressor variables.

arima.model <- auto.arima(chicken,xreg=xreg.outliers)
arima.model

output from outlier detection
Series: chicken
ARIMA(0,1,0)

Coefficients:
         LS12     TC20
      37.1400  36.3763
s.e.  11.8641  10.9382

sigma^2 estimated as 140.8:  log likelihood=-264.19
AIC=534.38   AICc=534.75   BIC=541.08

Outliers:
  type ind time coefhat tstat
1   LS  12 1935   37.14 3.130
2   TC  20 1943   36.38 3.326

output from  auto.arima incorporating outliers as xreg
series: chicken
ARIMA(0,1,0) with drift

Coefficients:
        drift     LS12     TC20
      -2.7450  39.8850  36.3763
s.e.   1.3997  11.6267  10.6414

sigma^2 estimated as 133.2:  log likelihood=-262.32
AIC=532.64   AICc=533.26   BIC=541.58

Hope this helps
Answer: @forecaster gave a great answer using a package that I will be checking out. This is the answer to my question using the arimax function. The trick is to see that (assuming the event occurs at T=200):

So, we can create the covariate as 0 for t < 200 and then 1,2,3....200 and use transfer=list(c(0,0))
We could also use this variable directly as a regressor via xreg.
Comment: not quite sure, if r=0 and s=0, doesn't that translate to $\omega BS_{t}$ ?
Answer: use a fixed vector to fix delta = 1.
fixed=c(NA,NA,1,NA)
you need to figure out where the NA's should be
located in the vector.See the docs for arima. I first
estimate arimax() without the fixed vector to find the
correct positions. Then add the fixed vector to the command
and rerun.
Comment: This approach would be appropriate to fix some of the coefficients of the ARIMA model (or of the external regressors). In this case, the parameter $\delta$ is not the coefficient of the regressor, it is a parameter to generate the values of the regressor (the intervention variable). The question may mislead you because it mentions constrain $\delta=1$. Nevertheless, I think it is usually more likely that we want to fix the coefficients of the ARIMA model rather than the external regressors. In the latter case we could simply fit a model for $y_t-\kappa x_t$, being $\kappa$ the fixed coeff.


Manipulate SVG in HTML with JS

Question: I'm having an problem with construction of a svg element using javascript in an html-embedded svg image. I created two files that should be exactly the same but one of them is being constructed with js.

SVG.html
<!DOCTYPE HTML>

  <meta charset="utf-8" />
  <title>Pozadí</title>

  <svg
    id="pozadi"
    xmlns="http://www.w3.org/2000/svg"
    version="1.1"
    height="200"
    width="200"
   >
   <path
     d="M 0,0 L 150,150 L 100,150 L 150,150 L 150,100"
     style="stroke: #000; stroke-width: 2px; stroke-linecap: round; fill: none;"
    >
    <animate
      from="M 0,0 L 150,0 L 115,35 L 150,0 L 115,-35"
      to="M 0,0 L 150,150 L 100,150 L 150,150 L 150,100"
      dur="10s"
      begin="5s"
      attributeType="XML"
      attributeName="d"
     >
    </animate>
   </path>
  </svg>

JS.html
<!DOCTYPE HTML>

  <meta charset="utf-8" />
  <title>Pozadí</title>

  <svg
    id="pozadi"
    xmlns="http://www.w3.org/2000/svg"
    version="1.1"
    height="200"
    width="200"
   >
  </svg>

The second file, JS.html, is just white and plain.
I'm asking, how to correct it? Thanks, m93a
Answer: var path = document.createElementNS('http://www.w3.org/2000/svg/','path');

is wrong. You want
var path = document.createElementNS('http://www.w3.org/2000/svg','path');

Note the lack of a trailing / character. Same problem with
var anim = document.createElementNS('http://www.w3.org/2000/svg/','animate');

If you fix these two lines you should see the arrow (I do in Firefox), here's a jsfiddle to prove it.
Answer: One of the problems you may have is that you run the script as soon as you load page, instead of waiting for all the elements to load.
So basically document.getElementById('pozadi'); might be null when you run it, and since you have an external request (http://www.w3.org/2000/svg) there is a good chance that happens.
Try adding an onload listener calling a function with your current script. If you have jQuery this is what you are looking for http://api.jquery.com/ready/. If not... make it manually.
Comment: document.getElementById('pozadi') does not depend on layout so you can run it any time. You don't need to use onload here.
Comment: Omg... since when? Put script on top and see the difference.
Comment: If the svg is "above" the script element, svg's dom should be ready to be used. If it is below, nothing but waiting can be done.
Comment: Since forever. getBBox does depend on layout for instance so you can't call that before onload but there's nothing in this case that does.
Comment: Whatever... encourage bad practices, that's the way to do it.

Manipulate SVG in HTML with JS


I want to open another chrome window in full screen mode when clicking on a hyperlink

Question: I have a html page open in a chromewindow.
The html page contains some hyperlinks. When user clicks on the hyperlink i want it to open in a new chrome window in full screen mode. please help how to achive it. I tried different JavaScript code and chrome extension but it is giving me error in console like.
Failed to execute 'requestFullscreen' on 'Element': API can only be initiated by a user gesture.
Comment: [see this question](https://stackoverflow.com/questions/172748/how-to-show-fullscreen-popup-window-in-javascript)
Comment: Possible duplicate of [How to show fullscreen popup window in javascript?](https://stackoverflow.com/questions/172748/how-to-show-fullscreen-popup-window-in-javascript)
Comment: Well, you can't. The error message is pretty self-explanatory.
Answer: You can do it like this in below code to force the browser to open a new window instead of new tab
<code>window.open('page.php', '', 'width=1000');
</code>


could not Convert legacy permission algorithm to 5 while migrating from liferay 5.2.3 to 6.0.6 tomcat with PostgreSQL

Question: <code>     Unable to process message {destinationName=liferay/convert_process, responseDestinationName=null, responseId=null,
     payload=com.liferay.portal.convert.ConvertPermissionAlgorithm, values=null}

     com.liferay.portal.convert.ConvertException: org.postgresql.util.PSQLException: ERREUR: erreur de syntaxe sur ou pr+¿s de -½ XSS -+  Position-á: 429
     at com.liferay.portal.convert.ConvertProcess.convert(ConvertProcess.java:53)
    at com.liferay.portal.convert.messaging.ConvertProcessMessageListener.doReceive(ConvertProcessMessageListener.java:47)
    at com.liferay.portal.convert.messaging.ConvertProcessMessageListener.receive(ConvertProcessMessageListener.java:32)
    at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:63)
    at com.liferay.portal.kernel.messaging.SerialDestination$1.run(SerialDestination.java:56)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
</code>
After this exception the system falls and shows this message:
The system is currently undergoing maintenance. Please try again later.
ث, 18 نو٠2014 10:00:37 +0000 Executing com.liferay.portal.convert.ConvertPermissionAlgorithm
Comment: Looks like an erreur de syntaxe: `ERREUR: erreur de syntaxe sur ou pr+¿s` Could be encoding. Please check the postgres log for the actual offending statement.
Answer: the problem was solved :
I followed this link
http://vetal-liferay.blogspot.com/2014/01/liferay-migration-form-523-to-62.html
but instead of changing the permission.user.check.algorithm in liferay 6.0.6 i have changed it in 6.1.1.


Ajax function not returning any html

Question: I'm trying to develop an Ajax process in a Drupal Module. Here's the code :
function My_Module_menu() {
  $items = array();

  $items['route/to/ajax/%'] = array(
    'page callback' => '_MY_MODULE_processing',
    'page arguments' => array(3),
    'type'  => MENU_CALLBACK,
    'access arguments' => array('acces content'),
    'delivery callback' => '_MY_MODULE_listing'
  );

  return $items;
}

function _MY_MODULE_processing($region='110') {

  // Executing some operations to retrieve a list of NIDs
  // retrieving the nodes
  $nodes = node_load_multiple ($nidList);
  return ($nodes) ;
}
/**
 * Here is the templating part
 * @param unknown $nodes
 */
function _MY_MODULE_listing ($nodes){
  $build = array();
  $nodeList='';
  foreach ( $nodes as $node ) {
    $nodeList = $nodeList.'<option value='.$node->nid.'>'.$node->title.'</option>';
  }
    $build['content'][]=array(
      '#prefix'=>'',
      '#suffix'=>'',
      '#markup'=>$nodeList,
    );

  return ($nodeList);
}

I don't get any response with this. I've run debugging function and it goes inside every function. First _MY_MODULE_processing then _MY_MODULE_listing.
How can i return a correct response ?
Answer: Got it. I returned the answer with ajax_deliver :
function mij_add_aide_list_proposition($nodes){
    // Magic stuff goes here (construction of $nodeList)
    $build['content'][]=array(
      '#prefix'=>'',
      '#suffix'=>'',
      '#markup'=>$nodeList,
    );

  //$test=drupal_render($build);
  ajax_deliver($build);
}

If someone finds another way, I'm open to any change !

Ajax function not returning any html


('42S02', "[42S02] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Invalid object name 'app_poll'. (208) (SQLExecDirectW)")

Question: I am getting following while running python django app(using python-odbc-azure lib) on azure web app.
    ProgrammingError at /

     ('42S02', "[42S02] [Microsoft][ODBC Driver 13 for SQL Server][SQL
     Server]Invalid object name 'app_poll'. (208) (SQLExecDirectW)")

       Method:  GET
       Request URL:     http://retailgenieredsky.azurewebsites.net/
       Django Version:  1.11
       Exception Type:  ProgrammingError
       Exception Value:

       ('42S02', "[42S02] [Microsoft][ODBC Driver 13 for SQL Server][SQL
       Server]Invalid object name 'app_poll'. (208) (SQLExecDirectW)")

       Exception Location:  D:\home\python354x64\lib\site-
       packages\sql_server\pyodbc\base.py in execute, line 545
       Python Executable:   D:\home\python354x64\python.exe
       Python Version:  3.5.4
       Python Path:

       ['.',
       'D:\\home\\python354x64\\python35.zip',

     'D:\\home\\python354x64\\DLLs',
     'D:\\home\\python354x64\\lib',
     'D:\\home\\python354x64',
     'D:\\home\\python354x64\\lib\\site-packages',
     'D:\\home\\site\\wwwroot']

    Server time:    Fri, 22 Dec 2017 13:10:47 +0000
    Error during template rendering

    In template D:\home\site\wwwroot\app\templates\app\layout.html, error at line 5
    42S02
    1   <!DOCTYPE html>
    2
    3
    4       <meta charset="utf-8" />
    5       <meta name="viewport" content="width=device-width, initial-scale=1.0">
    6       <title>{{ title }} - Django Polls</title>
    7       {% load staticfiles %}
    8       <link rel="stylesheet" type="text/css" href="{% static 'app/content/bootstrap.min.css' %}" />
    9       <link rel="stylesheet" type="text/css" href="{% static 'app/content/site.css' %}" />
    10
    11
    12
    13
    14

    15

I tried following ways.
1. Remove
5
But it given same error for next line.

I tried to remove layout.html its still give same error

This is deeper level of exception.
chunked_fetch

False

cursor

<django.db.backends.utils.CursorDebugWrapper object at 0x00000064038E4F28>

params

()

result_type

'multi'

self

<sql_server.pyodbc.compiler.SQLCompiler object at 0x00000064038E4668>

sql

('SELECT TOP 5 [app_poll].[id], [app_poll].[text], [app_poll].[pub_date] FROM '
 '[app_poll] ORDER BY [app_poll].[pub_date] DESC')

It seems this code is able to connect mssql database but it's not able to fetch data. Is my understanding right??
Also I ma very beginner to django , mssql and azure. Any lead is highly appreciated.
Thanks
Answer: Exception was because azure sql server has not created tables. So needed to database migration to create table on azure to resolve this.
Comment: Hey, I know it's a bit late, but can you please elaborate on your solution.
Comment: @ascii_walker, please elaborate on your solution. It will be help for us.


Why is this not LL(1) grammar?

Question: S -> 1S2 | S0 | epsilon
I thought it would be LL(1) because it is possible to determine.
For example if the next input symbol was 0 i'd know it was S -> S0
Does the epsilon mean that it cannot be LL(1)?
Comment: you may want to add a second tag to direct it around - for instance NLP
Comment: By your logic you would always choose `S0` when you saw 0. Since you never use up the 0 wouldn't you recurse forever into `S0` and never find epsilon?
Comment: So because S -> S0 -> S00 -> S000 -> e000 ?
Which is wrong because the epsilon should be like 000e?
Answer: This grammar is not LL(1) just because it has a left recursion on it. When you will try to build the parser table, you will came up with a conflict.
http://en.wikipedia.org/wiki/LL_parser

Why is this not LL(1) grammar?


How to get hostname for ip on a local network?

Question: I'm trying to get the hostname for a given ip of a computer in a local network.
The current test setup I have is a few single board computers(odroids) connected via Ethernet and a switch to a laptop which assigns IP addresses (this will get replaced by a server later).
These single board computers run Ubuntu 15 and have avahi-daemon running.
I can ping/ssh these using their hostnames just fine.
The issue is I can't seem to get the hostname if I know the IP address.
So far I've tried host,arp, nslookup, dig but no joy.
For example:
host 192.168.1.11
Host 18.104.22.168.in-addr.arpa. not found: 3(NXDOMAIN)

As a hacky workaround plan I was thinking of putting a script together that initially pings a list of known hostnames of computers expected to be on the network, creating associations between IPs and hostnames or trying to get the DHCP table from a router to which the computers are connected, but I was wondering if there's a more elegant solution.
Comment: I guess it could fit on SuperUser, but honestly it's pretty obvious that you are out of your depth. I'm not trying to be rude, but it appears that you don't know the first thing about name resolution. You should probably start by doing some reading / research about how DNS works.
Comment: This question doesn't belong on Server Fault.
Comment: My bad. I'll delete it then. Can you please recommend the correct StackExchange website I should ask this on ? Thank you
Comment: how did I automatically get a -1 as soon as I posted ? (What did I do wrong ?)
Comment: Thank you for putting it so politely :) I tried to explain I have no idea what I'm doing in the first line. I'll look into DNS and how that works on local networks.
Comment: What I was after actually was something like Bonjour/MDNS that allows me to ping a device on a local network based on its hostname which would resolve to the android device's local IP address. It boiled down to advertising `_workstation._tcp.local` on port `9`. On Android 9 recently I've realised it also requires multicast permissions(`ACCESS_WIFI_STATE`,`ACCESS_NETWORK_STATE`,`CHANGE_WIFI_MULTICAST_STATE`). [`javax.jmdns`](https://search.maven.org/search?q=g:javax.jmdns) worked in 2016, on P this [jmdns](https://search.maven.org/artifact/com.github.rickyclarkson/jmdns/3.4.2-r353-1/jar) worked
Answer: To get reverse DNS lookup working your will have to set up your nameserver / DNS-server in a way, that it is authoritative for 192.168.1.* and create zones / zonefiles with the DNS data (ip => hostname).
I guess you are under the (false) impression, that a nameserver that knows "hostname => ip" magically knows "ip => hostname" either. You have to setup both ways individually.
Comment: You're hunch is bang on the buck. Thank you for your contribution(+1). However, I will delete this question as it appears to be off topic


Unity Referencing an Instantiated (cloned) object on a Button via Script

Question: My Problem:
I cant get it to work, that every Building has its own SpawnPoint.
I'm working on an RTS like build mechanism. I have set it up that after pressing a key I can place buildings in the scene. The buildings get instantiated from a prefab and after they get instantiated they get a script called "BuildingScript" attached to them. Now i want to implement, so that i get a individual spawn point for every building (for now just next to it). I got a UI set up with a button, which by pressing spawns a unit at the building.
I had the "BuildingScript" attached to the prefab, but when i set the spawn point for one Building, it set it for every Building in the Scene. So a Unit was Spawned always at the same Buidling.
I want to set it up, that every Building has its own spawn point. Thats why I want to give every Building the script "BuildingScript" when Instatiated, because I hope, that this way every script gets handled individually. Is that right? Or will it still set the same point for every building, because the script is still the same?
Also I wanna reference the current placed building to the button, so when its clicked, it will run only the code of the last placed building (for now). I think I cant do this by using "On Click()" Of the Button, because my clone isnt Instatiated yet, so I have to reference the clone to the button somehow via Script, so the button works with the clone.So my problem is, that I need to set a reference from my cloned Building to the Button, after I placed the clone.
I googled a lot on how to do this, but didnt found any answers to my problem besides this https://forum.unity.com/threads/controlling-instantiated-game-objects-from-ui-buttons.332005/.
But I cant get it to work and I think it will not work because my clone is an Object and not a GameObject, so I could never set reference to it to call the funktion SpawnUnit(), because GetComponent only works for a GameObject.
Now I'm really at a point where I just don't know how Unity handles these kind of things.
BuildingScript on the Instantiated Building
public class BuildingScript : MonoBehaviour
{
    public bool SavePos = false;

    public Vector3 SpawnPoint;
    public Vector3 BuildingPos;
    public GameObject Unit;

    void Start()
    {
        FindObjectOfType<SpawnButtonReference>().GiveReference(this);
    }

    public void SpawnUnit()
    {

//I did this because if a building gets instatiated i wanted it to save its
Position to Spawn Units from it (doesnt really work though).
"MousePos" ist the last Position of the Mouse in the PlacementScript, before klicking to place the building.

        if (SavePos == false)
        {
            BuildingPos = GameObject.FindObjectOfType<GroundPlacementController>().GetComponent<GroundPlacementController>().MousePos;
            Debug.Log(BuildingPos);

            SavePos = true;
        }

        float PosX = BuildingPos.x + 2;
        float PosZ = BuildingPos.z + 2;
        SpawnPoint = new Vector3(PosX, 0, PosZ);
        Debug.Log("Spawn" + SpawnPoint);

        Instantiate(Unit, SpawnPoint, Quaternion.identity);
    }
}

Script on the Button
public class SpawnButtonReference : MonoBehaviour
{
    public GameObject objectReference = null;

    internal void GiveReference(BuildingScript Object)
    {
        objectReference = Object;
    }

    void OnButtonPress()
    {
        if (objectReference != null)
        {
            objectReference.GetComponent<BuildingScript>().SpawnUnit();
        }
    }
}

Comment: Please don't add "solved" or similar phrases to your question when you've solved the problem. If someone gave you an answer, [accept it](/help/someone-answers). If you found a solution on your own, [post an answer](/help/self-answer) (you can also accept your own answer after two days). Accepting answers is also the way to show a question has been solved - editing "solved" into the question is not.
Comment: ok didnt know that sorry :)
Answer: So I solved it myself with a little workaround. Instead of trying to reference the clone, I wrote a script on the Spawn Button which searches all Objects with the "BildingScript" then if they are Selected (which can only be one Building) it spawns a Unit at its Spawn point.
The Building itself saves his spawn point when being placed (so when Input.GetMouseButtonUp(0))
works very well for me :)


How can I take an Image from SD card? And How to convert to an String (base64)

Question: I need Access Images from the sd card. I have their path. How could I Take them and convert to an String base 64?
Answer: Do something like this... then, you can read the file as you want.
File dir = Environment.getExternalStorageDirectory();
File yourFile = new File(dir, "path/to/the/file/inside/the/sdcard.ext");

try this.
Answer: You can use the following code.
File dir = Environment.getExternalStorageDirectory();
File yourFile = new File(dir, "path/to/the/file/inside/the/sdcard.ext");
String encodeFileToBase64Binary = encodeFileToBase64Binary(yourFile);

private static String encodeFileToBase64Binary(File fileName) throws IOException {
    byte[] bytes = loadFile(fileName);
    byte[] encoded = Base64.encodeBase64(bytes);
    String encodedString = new String(encoded);
    return encodedString;
}

private static byte[] loadFile(File file) throws IOException {
    InputStream is = new FileInputStream(file);

    long length = file.length();
    if (length > Integer.MAX_VALUE) {
        // File is too large
    }
    byte[] bytes = new byte[(int) length];
    int offset = 0;
    int numRead = 0;
    while (offset < bytes.length && (numRead = is.read(bytes, offset, bytes.length - offset)) >= 0) {
        offset += numRead;
    }

    if (offset < bytes.length) {
        throw new IOException("Could not completely read file " + file.getName());
    }

    is.close();
    return bytes;
}