Glissando vs Arpeggio

Question: What is the difference between an arpeggio and glissando? Is a glissando just another form of an arpeggio? I still don't exactly understand.
Answer: Adding to Aaron's answer, a glissando doesn't have to be - in fact often isn't - comprised of any particular notes. In fact, on piano, it's usually deployed by sliding along all the white keys, when maybe there are certain sharps or flats in the key concerned. The notes are played quickly, so there's little chance of discerning the 'wrong' ones!
Portamento is another variation, which is glissando taken to the nth degree, every note available, and also those in between! Works well on instruments such as violins and trombones.
Arpeggios are sometimes called 'broken chords', as that's exactly what they are - using only notes from a particular chord, either top to bottom, or, more usually, bottom to top. The word is gleaned from the way a harp could be played, plucking the notes of a chord in order. Playing all the strings consecutively would constitute a glissando.
Comment: Way too early to have chosen a preferred answer. Better ones may well surface later! So choosing early may deter them bothering.
Comment: @CarlWitthoft - in my part of the world, the only chirps seem to come from sparrows.
Comment: Arpeggios are bottom-to-top starting around the middle classical period to present.  Prior to that, the default was top-to-bottom.  So if you're playing, e.g. J.S. Bach, it's the reverse of what you'd expect from later composers.  (A lot of Bach-era ornaments are flipped from what we'd expect in later music.)
Comment: @DarrelHoffman "Arpeggio" is a broader term that you're considering. They appear up and down in every era. Chords marked with an arpeggio line, yes, downward first, then upward later.
Comment: FWIW, in the Engineering World,  a portamento is equivalent to a "chirped frequency,"  as opposed to any frequency changes of discrete step size.  (except of course in the digital domain a chirp may be limited by the smallest 1-bit transition size).
Comment: @Aaron - is 'an arpeggio *line*' still the squiggly one, regardless?
Comment: @Aaron Yes, they can go either way in any era, and sometimes there's an arrow on the arpeggio line to specify, but *without* the arrow, the default reading is low-to-high in classical-to-modern music, but was high-to-low in Bach's day.
Comment: @DarrelHoffman I'm saying that limiting arpeggios only to chords marked with an arpeggio line is too much. Arpeggios are any ascending or descending broken chord, including those explicitly written out note by note and not played as ornaments but as an integral part of the music.
Comment: @Aaron Ah, gotcha.  This was specifically about the reading of the arpeggio *ornament*, not arpeggios in general.
Answer: 'Arpeggio' is easy to define.  It's a 'broken chord'.  The notes of a chord played one after another rather than all at once.
'Glissando' is the equivalent for a scale.  But does it step or slide between notes?   Some people lock in to the terminology of the once ubiquitous Yamaha DX7 synth, which called discrete notes (like a piano or harp can do) 'glissando', a sliding pitch change 'portamento'.   But hold on, trombones have been calling their characteristic sliding pitches 'glissando' for hundreds of years!  And singers and string players have their own definitions of 'portamento'.
So don't be too pedantic about 'glissando' and 'portamento'.  But you can be confident in distinguishing either of them from an 'arpeggio'.
Comment: Would you specify that an arp. is a broken chord sequentially played, all notes going in one direction? Broken chords are just that - chords with their notes broken into often random order.
Answer: An arpeggio is a chord broken up into individual notes. Each individual note is heard as a discrete unit.
The video below shows a person practicing some basic arpeggios.

A glissando is a smooth slide from one pitch to another. Individual notes are not articulated as discrete pitches; they're just points along a continuous line.
The video below demonstrates a classic glissando from George Gershwin's Rhapsody in Blue (from 0:06 to 0:09).
Comment: This answer implies that it is impossible to play a glissando on a discrete-pitched instrument like a piano or xylophone, which I disagree with.
Comment: @NuclearHoagie That's a fair criticism. I'm open to suggestions on how to better explain.
Comment: That is a common misused of "glissando."  See "portamento."   And as to breaking a chord into individual notes:  the risk is that something like an "arm-bar" could be claimed to be a chord, and then where are you?
Comment: @CarlWitthoft Every single definition here refers to "broken chord". Tone clusters are pushing it. A glissando is a "glide" or "slide"; "portamento" depends on the instrument being played: it's a slide on strings or voice, but for piano, for example, it's a synonym for "portato".
Answer: According to Classical-Music (by BBC Music Magazine), a glissando is a musical slide, from one pitch to another. In other words, a smooth continuous glide going up or down between two notes. Arpeggios, on the other hand, are the notes of a chord played in rapid succession, either ascending or descending (Oxford Languages).
Basically, arpeggios are broken chords where the notes are played successively, while glissandos are more of a smoother glide.

Glissando vs Arpeggio


Displaying Image caption with only HTML and CSS

Question: I am trying to display caption on images.
This is my HTML

Our philosophy and Vision

Our philosophy and Vision

Our philosophy and Vision

Our philosophy and Vision

This is my CSS :
.image-hover {
    overflow: hidden;
}

.image-hover .image-block {
    width: 234px;
    height: 100px;
    display: inline-block;
    border: 1px solid #666666;

}

.image-hover .image-block img {
    width: 230px;
    height: 100px;
    border: 2px solid #fff;
}

.details, .image-backdrop {
    float: left;
}

.details h4:after {
    display: inline-block;
  content:" ";
  width: 0;
  height: 0;
  border-top: 6px solid transparent;
  border-bottom: 6px solid transparent;
  border-left: 8px solid #666666;
  margin: 0 10px;
}

This is my JsFiddel
My problem is how can I get image's caption to above of image?
Any comments are greatly welcome.
Thank you.
Answer: you should use position:relative; to the container and position:absolute; to the child container whom you want to on image it is styled as you want
http://jsfiddle.net/ADMvd/2/
Answer: Add this to your CSS
.details h4 {
    color: white; /* just to make it look pretty */
}
.details {
    position: absolute;
}
.image-overlay {
    position: relative;
}

DEMO
EDIT
Note that you are using div tag inside your a tag so don't forget to add display: block to your a.image-overlay tag because inline elements can't contain any other type of elements. It does work but will cause many weird problems.
For starters height:100% won't work for div inside a


Quando uso PathParam ou QueryParam?

Question: Eu sei a diferença, porém quando eu faço uso do PathParam ou QueryParam? É correto a seguinte explicação?

If there is a scenario to retrieve a record based on id, for example
  you need to get the details of the employee whose id is 15, then you
  can have resource with @PathParam.

GET /employee/{id}

If there is a scenario where you need to get the details of all employees but only 10 at a time, you may use query param

GET /employee?start=1&size=10

Comment: Isso não é algo nativo do Java, certo? Assumi dada [essa pergunta semelhante no SOen](http://stackoverflow.com/q/5579744/520779) que se trata da framework Jersey, mas se eu estiver enganado favor reverter minha edição.
Comment: Obrigada, fiz a correção! A tag correta é "web-service"
Answer: Segundo essa resposta no SOen, PathParam serve para acessar o caminho enquanto QueryParam serve para acessar a query string. Esclarecendo, uma URL completa* possui as seguintes partes:
http    :// google.com : 80    /foo/bar/baz ?a=10&b=20     #seção

esquema :// origem     : porta caminho      query_string   fragmento

Não conheço o Jersey, então não posso dizer quando é apropriado usar um ou outro, exceto que depende de onde a informação que você quer está (no caminho ou na query string). Do ponto de vista do REST, o mais comum é colocar nomes e ids de recursos no próprio caminho, enquanto parâmetros de buscas arbitrárias podem vir na query string - de modo que considero correta a explicação citada.
* Bem, quase completa: ainda é possível ter usuario:senha@ antes da origem, mas nunca vi isso sendo usado na prática...
Answer: Como já foi explicado pelo @mgibsonbr, cada anotação lê os parâmetros de locais diferentes.
Porém, é importante compreender que o framework Jersey implementa a API JAX-RS, que é a API padrão do Java para RESTful Web Services. Portanto o correto é seguir sempre os padrões da arquitetura REST.
Nesse caso a explicação está correta, pois no REST entende-se que os identificadores de recursos ficam no caminho da URL, identificando aquele recurso de forma única.
Se o empregado (employee) é um recurso do sistema, então é correto acessá-lo pela requisição GET /employee/{id}.
Parâmetros adicionais, exemplificados na pergunta como dados de paginação, são passados como parâmetros de URL porque são informações secundárias.

Quando uso PathParam ou QueryParam?


Significant interaction and not significant correlation

Question: I am currently working on my master thesis and need help.
Basically, the correlation is not significant in most of the cases (except the one between the 2 DV).
Despite this, I got a significant interaction between the IV and Moderator using the PROCESS macro for SPSS.
My assumption is that the insignificant correlation is due to the fact that it does not take into account the influence of the Moderator (e.g. correlation between IV-DV). Respectively, in my case, one moderator influences the IV-DV relationship in a positive way and another in a totally opposite way. So I assume it cancels each other out and that's why there is not correlation.
What do you think about that and how can I prove it? Or better justify my reasoning?
I plotted the lines of the two-way interaction effects. It looks almost symmetrical. Would it be an argument to explain my reasoning that on average the correlation canceled each other out?
Comment: I think it would be easier to talk about this in terms of means and simple effects rather than correlations. A simple description is that your simple effects were in different directions resulting in strong evidence of an interaction (put F, df, and p value here)  and no evidence for a main effect.
Comment: Updated my answer since you posted your output.
Answer: I agree with David Lane's comment that you should talk about these in terms of means and simple effects, not correlations. Did you manipulate the portrayal and the emotions of love or fear? If so, you are under selling your data by talking about it in terms of correlations instead of effects.
This is called a crossover interaction. There are no main effects, because the effects of the IV on the DV is opposite for each level of the moderator. I don't know what you mean by "prove" this, but you would report the F statistic and p value associated with both main effects, and then talk about the significant interaction.
You could then follow-up this discussion by talking about the simple main effects, which PROCESS Model 1 reports for you automatically. If you post the output from PROCESS, we can show you where those results are if you are stuck with that.
The main effect is pretty much just pretending the moderator doesn't exist—it collapses across both levels of the moderator. And since the relationship is positive in one condition and negative in the other, these balance out. I describe an intuitive example of a crossover interaction in this CrossValidated post; see the basketball example with the graph that looks like yours.
After looking at the PROCESS output:
When WPort is 0, then there is a positive, significant effect of ForLove on AadTOT, p = .0042. When WPort is 1, there is a negative, but not statistically significant, effect of ForLove on AadTOT, p = .0758. Those are the simple main effects.
Comment: That's right, I did manipulate the emotions and portrayal (2x2 between-subjects design).
Comment: Then yes, do not discuss it it in terms of correlation. The stats are the same (i.e., the stats don't know if you manipulated the factor or not), but treat it as main effects, interaction effects, and simple main effects (where simple main effects are the effects *within* a condition). Discuss the main effects, then the interaction, then the simple main effects.
Comment: Now it makes sense why I should not talk about the correl.
I also thought that it is not relevant in my case as the effects are opposite and cancels each other out when we just look from the IV-DV perspective.

However I am still a bit confused in regards tot the main/simple effects.

Will  attach the Process output.
Comment: **Main effect:** The effect of the IV on DV, ignoring the influence of the Moderator. **Simple main effect:** The effect of the IV on the DV, *within* a level of the Moderator.


how to find slant height using radius and angle for a truncated cone

Question: I asked this question a while ago but it got closed. I'm trying to find the slant height or just height for a truncated cone with only the radius and the angles for a hobby project that i've been doing. Its called how to develop, build, and test small liquid-fueled rocket engines by rocket lab. The truncated cone is for the nozzle throat. The base radius (R=1.50225 in) and the top radius (r=0.50075 in). I have looked everywhere but it keeps saying volume and height and surface area. Not learning in 9th grade eather.
<code>      r=0.50079
      ________________ 30˙
     /                \    |
    /                  \   |
   /                    \  |h= unknown
  /                      \ |
 /                        \|
 -------------------------- 60˙    looking for slant height or height   R=1.50225
</code>
Comment: Welcome to Math SE. You have stated the base radius $R$ and top radius $r$ in units of "in$^2$" which usually represents area instead of a length. Did you mean just "in" instead?
Comment: I´m using in as a filler for the unit of measurement.
Comment: Thanks for the reply, but I'm not clear why you're using "in$^2$" instead of "in", as a filler or otherwise. If you truly don't want to use a filler rather than a specific unit of measurements, why not use no units at all or, if you really want to say something there, why not just state something like "$R = 1.50225$ units" instead? I believe the generic word "units" is a much more appropriate filler term to use.
Comment: please ignore the ²
Answer:
See (rather terrible) diagram above.
The half-angle of the cone (from the vertical axis) is $\theta$.
By simple right-angle trigonometry, you know that $l = \frac r{\sin \theta} = \frac r{\cos(90^{\circ} - \theta)}$. I'm giving you both forms because I'm not sure which angle you have.
Similarly, $L =  \frac R{\sin \theta} = \frac R{\cos(90^{\circ} - \theta)}$
Find $L-l$ to get the truncated slant height.
Answer: Here's a method that I think is especially easy to apply if you just know a little trigonometry.

The diagram is a cross-section of your truncated cone. The vertical line in the middle is the axis (centerline) of the cone.
The other vertical line is parallel to the axis and meets the edge of the circle at the upper end of the truncated cone.
So we now have a right triangle with base $R - r = 1.50225 - 0.50075 = 1.0015.$
The height $h$ is $(R - r) \tan 60^\circ$ and the slant height is
$(R - r) \sec 60^\circ$.
You can put any other angle in place of $60$ degrees for a different shape of cone.
But the $60$ degree angle is particularly nice because
$\tan 60^\circ = {\sqrt3}$ and $\sec 60^\circ = 2.$


Função DELETE com php e mysql

Question: deleta.php:
<?php
// quero transformar esse script em função e chama-lo no link
// deleta um unico post
        include("config.php");
        $id = $_GET['id'];
        $sqlInsert = "DELETE FROM conteudo WHERE id = '$id'";
        $deletaPost = mysqli_query($conecta,$sqlInsert);

    if($deletaPost) {
            header("Location: ".$_SERVER['HTTP_REFERER']."");
            exit;
    } else {
        echo mysqli_error($conecta);
    }

 // o script abaixo deleta através de um checkbox, e elimina varios de uma vez.
// quero que esse seja o padrão, e o script acima seja uma função que só funcione quando chamada.
    if( !empty( $_POST['deletar'] ) ) {
       $groups = array_chunk( $_POST['deletar'], 50 );
       foreach ( $groups AS $group ) {
    $group = implode('\',\'', $_POST['deletar']);
            $query = 'DELETE FROM conteudo WHERE id IN (\''. $group .'\')';
           $deleta = mysqli_query($conecta, $query);
          // executa a query
       }

        if($deleta) {
           header("Location: ".$_SERVER['HTTP_REFERER']."");
           exit;
    } else {
        echo mysqli_error($conecta);
    }

    } else {
        header("Location: ".$_SERVER['HTTP_REFERER']."");
           exit;
    }

    ?>

HTML:
<a href="deleta.php?id=<?php echo $row['id']; ?>">Excluir

Como devo fazer para transformar o script de deletar em uma função, e em seguida chama no link?
 Quero fazer isso porque na pagina deleta.php eu já tenho outro script, e os dois estão dando conflitos. Com isso, decidi transformar o script em função e chama-la no link. Como agir
Comment: você pode deletar mais de um registro por link?
Comment: Não, com um checkbox em uma outra parte do código.
Answer: Crie uma função ela pode receber dois argumentos, o primero é a conexão e o segundo o id a ser excluído, faça a validação se entra enviada pelo usuário é número, caso positivo efetue o delete e retorne true, pega o retorna da função e redirecione o usuario em caso de sucesso.
No link, você deve continuar a chamar o arquivo delete.php e nele fazer a chamada da função.
delete.php
include 'conexao.php';
function delete($conexao, $id){
   //código a ser desenvolvido
}

$id = isset($_GET['id']) && ctype_digit($_GET['id']) ? $_GET['id'] : null;
if(delete($conexao, $id)){
   header('delteSucesso.php');
}else{
   echo 'erro ao excluír o registro';
}

Comment: E como ficaria meu link chamando a função? A estrutura da função ficaria assim?: function deleta($conecta, $row['id']) {...} e o link: href="deleta.php?id=
Comment: @ThiagoBarros fica a mesma coisa
Comment: Como eu transformo em função então? É que na pagina deleta.php tem o script de deletar por id, e logo abaixo tem um script que funciona com um checkbox, que eu uso para deletar varios registros de uma vez..
Comment: Eu deixei sem transformar em função e tentei deletar, mesmo existindo os dois scripts. Quando tentei deletar, funcionou, mas ao invez de redirecionar, eu recebi a mensagem: Erreur de syntaxe pr�s de '' � la ligne 1. Ao tentar utilizar o script abaixo, não funcionou. Será que é melhor criar uma pagina pra cada script?
Comment: Parece ser erro do banco, o ideal que delete.php só tenho código php e nenhum html.
Comment: Não tem nenhum HTML. Eu irei atualizar o post com o meu deleta.php completo. O que eu quero é transformar o primeiro script que deleta por ID em uma função e chama-la no link.

Função DELETE com php e mysql


Trig derivatives

Question: I am working through the review sections of Stewart's Calc 7 book and I am at this question
$$\frac{d}{dx} (\tan^2 x) = \frac{d}{dx} (\sec^2 x)$$
It is true or false, I said false but the book says true and I do not understand why.
Comment: (Continuity check): On chat you said you thought it was $2\tan x \sec^2 x$. That's exactly right. But that's not quite everything.
Comment: @Whoops! I misread the tex part - I'm sorry.
Answer: It is simply a matter of trignometric identities.
I hope you know that
$$\sin ^2 x + \cos ^2 x = 1$$
Dividing through $\cos ^2 x$ gives
$$\sin ^2 x + \cos ^2 x = 1$$
$$\eqalign{
  & \frac{{{{\sin }^2}x}}{{{{\cos }^2}x}} + \frac{{{{\cos }^2}x}}{{{{\cos }^2}x}} = \frac{1}{{{{\cos }^2}x}}  \cr
  & {\tan ^2}x + 1 = {\sec ^2}x \cr} $$
This means that
$${\sec ^2}x - {\tan ^2}x = 1$$
Since these differ by a constant, $1$, their derivatives are equal:
$$\eqalign{
  & \frac{d}{{dx}}{\sec ^2}x - \frac{d}{{dx}}{\tan ^2}x = \frac{d}{{dx}}1  \cr
  & \frac{d}{{dx}}{\sec ^2}x - \frac{d}{{dx}}{\tan ^2}x = 0  \cr
  & \frac{d}{{dx}}{\sec ^2}x = \frac{d}{{dx}}{\tan ^2}x \cr} $$
Note it is not necessary to calculate the derivatives explicitly, although it might prove useful to do so.
Answer: You could just calculate.
Using the chain rule:
$$
(\tan^2 x)'= [ (\tan x)^2]' =2 (\tan x)^1 \cdot\color{maroon}{ (\tan x)'}=2\tan x\, \color{maroon}{\sec^2 x  },
$$
and
$$
(\sec^2 x)' = [ (\sec x)^2]'  =2(\sec x)^1 \cdot
\color{maroon}{(\sec x)'}=2\sec x\cdot\color{maroon}{ \sec x\tan x }=2\tan x\, \sec^2 x.
$$

Trig derivatives


How to config DATETIME_INPUT_FORMATS to accept timestamp?

Question: Is there anyway to accept timestamp as an input for DateField and DateTimeField fields by configuring DATETIME_INPUT_FORMATS?
I can convert all my outputs to timestamp using :
REST_FRAMEWORK = [
   "DATETIME_FORMAT": "%s"
]

But it seems having %s for inputs does not work:
REST_FRAMEWORK = [
   "DATETIME_FORMAT": "%s",
   "DATETIME_INPUT_FORMATS": ["%s"],
]

It raises this validation error:
{
    "my_field_name": [
        "Date has wrong format. Use one of these formats instead: %s."
    ]
}

UPDATE
I can create a custom DateTimeField or DateField field. But it's really cumbersome as I have to change many fields all over the project. not to mention may of these fields are auto generated by ModelSerializer and I don't really like to overwrite them manually.
Answer: It uses this table, %S should work but there is no directive with a lowercase s.
Comment: I've seen that table. Uppercase `%S` means second between [00-61]. Sadly it's not related to timestamp.
Comment: You need to use the different options like. For example. 'DATETIME_FORMAT': "%Y-%m-%dT%H:%M:%S"
Comment: Well, As the question suggests... I'm looking for a way to make it accept timestamp.


Best practice when testing module updates

Question: A problem I face on a regular basis is trying to test sites that have recently had a lot of module updates. My usual process is to take each module on a case by case basis and test site features related to that module.
For instance, if the wysiwyg module is updated, I'll find a node that implements it in a field and do exploratory testing. But the reality is that the issues can be more subtle - perhaps certain user roles have lost permissions, allowing them to embed media when they were previously restricted.
Sometimes there are so many factors to bear in mind it is impossible to test them all, and invariably the client, at UAT phase, discovers major issues that I may have been blind to for various reasons.
I would be interested to hear of any advice or approaches towards testing sites that have had comprehensive module updates - what can I do to ensure quality, and even, what kind of line should I take with our clients to prepare them for issues that do invariably arise?
Answer: A solution that I would very much like to test, but haven't so for, is combining a branching model like this one, not necessarily with git, with selenium automated testing. Every night, a tool like Jenkins pulls the development branch, runs drush make, drush site-install, and runs your selenium test suite.
I also remember reading an empirical study about finding bugs in software, that I unfortunately can not find a link to right now. The findings were, that no single way of finding bugs was "good enough", and that at least two separate models, such as "Test-driven", "User beta testing", "Code review", etc, were necessary to reach a high, +75%, level of found bugs, prior to release.
Comment: Sorry, spent most of my lunch trying to find it again, but neither google, nor slashdot, where I probably saw the study, helped me find it. If I think of something, I will post back.
Comment: I would love to see that study. Part of the battle is winning the team over to automated testing - i.e. getting the budget for it!


Jackson JSON Parser - Specifying delimiter to serialize List of String

Question: By default, a list of string List<String> will be serialized as "a,b,c", I want to serialize the list as "a;b;c" with semicolon as the delimiter. This only has to be on one property in a class. How can I do this?
Comment: You would have to join the `List` yourself and send it as a `String`. The JSON spec says arrays are csv.
Comment: You'll probably need to `@JsonSerialize(contentUsing=someClass.class)` here
Comment: Thanks. I am going to try @JsonSerialize annotation
Answer: Try to use String.join(";", list)


sum of this simple series

Question: How do you calculate the sum of this series?
$$ \sum_{n=1}^\infty 0.5n \left(\frac{2}{3}\right)^n $$
Comment: https://math.stackexchange.com/questions/647587/sum-of-a-power-series-n-xn
Comment: Please use MathJax as images are not accessible to people who are using screen readers.
Comment: Do you know calculus and derivatives?
Answer: Note that we can pull out the $0.5$ from the summation, and focus on
$$\sum_{n=1}^\infty n \left( \frac 2 3 \right)^n$$
This bears resemblance to a geometric series. We know that, for a geometric series with ratio $r \in (-1,1)$,
$$\sum_{n=0}^\infty r^n = \frac{1}{1-r}$$
We want to manipulate this expression into the desired form. First, we see the derivative is given by
$$\sum_{n=0}^\infty nr^{n-1} = \frac{1}{(1-r)^2}$$
and multiplication by $r$ gives
$$\sum_{n=0}^\infty nr^{n} = \frac{r}{(1-r)^2}$$
Since the first summand of the sum ($n=0$) is itself $0$, we can start the summation at $n=1$ if we choose, which would match the form of the sum we choose to focus on. Take $r=2/3$ and we have the "important" sum; once you have the result, multiply by $1/2$ again to account for the pulling out of it at the start, and we're done!
Comment: thank you for your answer! My question is, that the derivative(with respect to 'r') of 1/(1-r) is not -1/(1-r)^2? Because (1-r)^(-1) is a composite function. Thank you
Comment: Gotta remember the chain rule, I keep making this exact mistake. XD If we let

$$f(r) = \frac{1}{1-r} = (1-r)^{-1}$$

then

$$\frac{df}{dr} = -1\cdot (1-r)^{-2} \cdot \frac{d}{dr} (1-r) = (-1) \cdot (1-r)^{-2} \cdot (-1) = (1-r)^{-2}$$
Comment: thank you, great, you are totally right!!
Answer: A solution withouth calculus
We want to calculate the sum:
$$\sum_{n=1}^{m} n a^n $$
Notice that:
$$\sum_{n=1}^{m} n a^n=1a^1+2a^2+3a^3+...+m a^m =$$ $$=(a^1+a^2+a^3+...+a^m)+(a^2+a^3+...+a^m)+...+(a^{m-1}+a^m)+(a^m)=
a^1(1+a^1+...+a^{m-1})+a^2(1+a^1+..+a^{m-2})+...+a^m(1)$$
By geometric series formula this becomes:
$$a^1 \frac{a^{m}-1}{a-1}+a^2\frac{a^{m-1}-1}{a-1}+...+a^m\frac{a^{1}-1}{a-1}=\sum_{n=1}^{m} a^n \frac{a^{m+1-n}-1}{a-1}$$
Now we're almost done:
$$\sum_{n=1}^{m} a^n \frac{a^{m+1-n}-1}{a-1}=\frac{1}{a-1}\sum_{n=1}^{m}  (a^{m+1}-a^{n})=\frac{1}{a-1}\left[\sum_{n=1}^{m}  a^{m+1}-\sum_{n=1}^{m}a^{n}\right]=\left[ma^{m+1}-\frac{a^{m+1}-1}{a-1}+1\right]=\left[\frac{ma^{m+2}-ma^{m+1}-a^{m+1}+1+a-1}{(a-1)^2}\right]$$
As $m$ tend to infinity all the exponential terms tend to $0$ since $|a|<1$, so it becomes:
$$\frac{a}{(a-1)^2}$$
If $a=\frac 23$ this is equal to:
$$6$$
Your sum presents also a factor $0.5$ so the final answer is .
$$3$$
:)
Comment: wow, great! Thank you Eureka!
Answer: There is a simple trick, for $a < 1$:
$$\sum\limits_{n=0}^\infty n a^n = a\partial_a\sum\limits_{n=0}^\infty a^n = a\partial_a \frac{1}{1-a} = \frac{a}{(1-a)^2}$$
Comment: thank you denklo!
Answer: Hint:
$$\color{green}{\sum_{n=1}^\infty na^n}=\sum_{n=0}^\infty(n+1)a^{n+1}=a\color{green}{\sum_{n=1}^\infty na^n}+\sum_{n=0}^\infty a^{n+1}.$$

 $$|a|<1\implies(1-a)S=\frac a{1-a}.$$
Comment: Thank you Yves Daoust!

sum of this simple series


translate mongodb aggregation query to spring-data-mongodb

Question: is it possible to translate this mongo shell aggregation query to spring-data?
db.getCollection("X").aggregate([
    {
        $group: {
            _id: {
                year: {
                    $year: "$happenedAt"
                },
                month: {
                    $month: "$happenedAt"
                },
                day: {
                    $dayOfMonth: "$happenedAt"
                }
            },
            count: {
                $sum: 1
            }
        }
    }
])

Thanks
Answer: Criteria c=new Criteria(Criteria.where("year").is(y).and("month").is(m).and("day").is(d));
Aggregration a=newAggregration(match(c),group("year","month","day").count().as("total"));

That is it.


Adding Discount entity to ERD of Online Shop?

Question: I want to add Discount entity to my ERD of online shop, but as we can see in my ERD there is a loop between discount, cart and cart item. As you know we have 2 kinds of discount, one that is calculated on each cart item and one that calculated on whole basket total price. Could you please help how can i change this ERD that avoids loop?
Comment: It seems to me that in your model, a *discount* is really just an attribute of a **Cart Item** and a **Shopping Cart**. It makes no sense to model it as a separate entity.
Comment: Thanks! but  my scenario was a little complicated to have just one attribute of discount.
Answer: What do you mean with discount?

is it just an amount to be deducted ?
is it a discount code with a percentage for calculation, e.g. ("Promotion xyz", -5%, "vegetables")?

The solution will depend on this and on what you want to express with your loop:

Is it just the amount to be taken off on some items and some additional overall discount amount on the cart? Then consider, as Janez suggested in the comments, to have two different discount attributes, one in each entity.
Or is it even the amount to be taken off on some items, and the aggregated total calculated for the cart?  Then have an item discount attribute only,  and calculate the toal on the fly for the cart.
Is it a discount code and the same discount code is reminded at cart level?  Then simply remove the link from cart to discount: it can be found back from the items whenever needed.
Is it different discounts, one calculated on individual items (e.g. "damaged item"), and the other applying to whole carts (e;g. "special member discount"),  then you may consider two distinct discount entities,  since the data has a different semantic and different rules and is used in a different way in each case.
Now in the last case,  you could keep the visual loop: in reality it's a 0..1 cardinality on the discount side:  so one discount could apply only to one item whereas another discount could apply only to the total; the loop is only optical and not necessarily in the data.
Comment: Thanks for your complete answer, i forgot to mention the scenario, and the third scenario was my issue. The best way is to have 2 discount entities, one related to cart, and one which is normalized and is product-discount entity. for each product we have different and special kind of discount.
Comment: @SarahGhanei I see. But fir the cart, is it just a summary of the other discounts, or is it an independent discount calculated on the total of the cart?


Can bootstrap resampling be used to calculate a confidence interval for the variance of a dataset?

Question: I know that if you re-sample from a data set many times and calculate the mean each time, these means will follow a normal distribution (by the CLT). Thus, you can calculate a confidence interval on the mean of the data set without making any assumptions on the probability distribution of the data set.
I was wondering if you could do something similar for the variance. That is, if I were to re-sample from a data set many times and calculate the variance each time, would these variances follow a certain distribution (regardless of what the original probability distribution of the data set was)?
I know that if that original data set is normal, then the variances would follow a chi-squared distribution. But what about in the case that it is not normal?
Answer:
Can Bootstrap Resampling be used to Calculate a Confidence Interval for the Variance of a Data Set?

Yes, just as with many other statistics.

I know that if you re-sample from a data set many times and calculate the mean each time, these means will follow a normal distribution (by the CLT).

It is not always the case that if you bootstrap a mean the bootstrap means will follow a normal distribution, even for distributions for which the CLT applies.
Here's an example where I resampled the mean for a sample of size $n=100$, where I resampled 10000 times:

It's not remotely normal.
The original sample consists of ninety-seven '0' values, and a '1', a '2' and a '100'.
Here's the (R) code I ran to generate the plot above:
<code> x <- c(rep(0,97),1,2,100)
 y <- replicate(10000,mean(sample(x,replace=TRUE)))
 plot(table(y),type="h")
</code>
The problem is that in this case the sample size (100) is too small for the CLT to apply with this sort of distribution shape; it doesn't matter how many times we resample it.
However, if the original sample size is much larger, the resampling distribution of sample means for something like this will be more normal-looking (though always discrete).
Here are the ecdfs when resampling the above data (black) and for values in the same proportions but with ten times as many values (red; that is, n=1000):

As we see, the distribution function when resampling the large sample does look much more normal.

if I were to re-sample from a data set many times and calculate the variance each time, would these variances follow a certain distribution

No, for the same reason it's not necessarily true for the mean.
However, the CLT does also apply to the variance*; it's just that you can't argue that the CLT applies to bootstrap resampling simply by taking many resamples. If the original sample size is sufficiently large, that may (under the right conditions) tend to make the resampling distribution of means (and higher moments, if sufficiently-higher-order moments are finite) relatively close to a normal distribution (relative to its distribution in smaller samples, at least).
* that the CLT usually applies to the variance (assuming appropriate moments exist) is intuitive if you consider $s_n^2 = \frac{1}{n}\sum_{i=1}^n (x_i-\bar x)^2$. Let $y_i = (x_i - \bar x)^2$; then $s_n^2 = \bar y$, so if the CLT applies to the $y$-variable, it can be applied to $s_n^2$. Now $s_{n-1}^2$ is just a scaled version of $s_n^2$; if the CLT applies to $s_n^2$ it will apply to  $s_{n-1}^2$. This outline of an argument is not completely solid, however, and there are some exceptions that you might not expect at first.


Quantas tabelas serão necessárias no MySQL?

Question: Estou começando um projeto que visa gerenciar notas e matérias dos alunos.
As informações que precisarei armazenar são:

Matérias totais (Todas as matérias que o aluno precisará estudar)
Matérias vigentes (São as matérias liberadas para o semestre, ex. 7)
Notas Prova 1, Prova 2, Prova-Exame e Média
Alunos (Nome completo e CPF)
Datas das Provas

O correto seria organizar essas informações como? Criaria 5 tabelas e FK pra todo mundo? Depois puxaria cada item quando precisasse?
Tentei fazer com 2 tabelas, mas parece impossível...
Obs. Só preciso de como seria a boa prática, não precisam mandar o código, somente o modelo relacional mesmo.
Comment: A resposta resolveu sua dúvida? Acha que pode aceitá-la? Veja o [tour] se não souber como faz. Isso ajudaria muito indicar que a solução foi útil para você. Também pode votar em qualquer pergunta ou resposta que achar útil no site todo
Answer: A parte mais difícil é coletar os requisitos e analisar profundamente o que precisa. Pode ser que você já fez isso bem, pode ser que não. Se não está bem feito, nós não temos como ajudar precisamente.
Algum motivo para criar o mínimo de tabelas possível? Não vejo razão e parece muito errado. EU já vi sistemas complexos inteiros em uma única tabela, foi ao mesmo tempo umas das coisas mais trágicas e cômicas que eu já vi em TI.
Conhece normalização? Precisa conhecer para modelar tabelas de banco de dados.
Por cima, e não sei se isto é útil, seriam realmente as que destacou, mas só porque foi descrito assim:

Alunos
Disciplinas
Disciplinas X Semestres
Provas
Notas

Estou considerando que só tem um curso e os semestres não possuem informações relevantes e são considerados implicitamente sem precisa de cadastro, a aplicação tratará disso.
Os dados serão obtidos de acordo com a normalização, em muitos casos usando JOIN considerando a FK. Na verdade este não parece ser um problema difícil de resolver, que exija algo excepcional, a não ser que a coleta de requisitos e análise esteja errada.
Falou de boa prática para a pessoa errada, eu odeio isso, e considero que causa um estrago na mente das pessoas, porque no fundo a pessoa quer uma receita de bolo universal que resolva o problema dela sem precisar pensar. Em desenvolvimento de software se isso existir não precisa contratar pessoas só pra aplicar isso, já teria tudo pronto. Tem que aprender os fundamentos, entender tudo como funciona, os contextos, analisar profundamente e aplicar cada conhecimento básico para encontrar a melhor solução para o problema específico dentro daquele contexto.
De qualquer forma é isto que dá para responder, o modelo depende e seria fazer para você, o código nem faria sentido porque nada indica que tenha código nisso. Por isso que não gosto de boas práticas, parece que a frase foi formulesca, não considerou o contexto.
Se tiver algum problema específico pode perguntar.
Comment: Maniero, sua resposta foi super completa. Na verdade eu estou tentando "resumir" algo que não deveria ser resumido, tentando economizar nas tabelas, e isso me prejudicaria até, caso esse projeto aumentasse.

Li a explicação sobre Normalização, muito bacana, e me esclareceu.

Comentei sobre código, pois já tentei tirar essa duvida antes e confundiram com a codificação das tabelas. Devo estar me expressando errado.

As informações que preciso são somente estas mesmo, algo simples e objetivo.
Gostaria na vdd, apenas de saber como alguém mais experiente faria.
Muito obrigada!
Comment: @Flávia A resposta resolveu sua dúvida? Acha que pode aceitá-la? Veja o [tour] se não souber como faz. Isso ajudaria muito indicar que a solução foi útil para você. Também pode votar em qualquer pergunta ou resposta que achar útil no site todo


Fast test if directory is empty

Question: What is the fastest way to test if a directory is empty?
Of course I can check the length of
list.files(path, all.files = TRUE, include.dirs = TRUE, no.. = TRUE)

but this requires enumerating the entire contents of the directory which I'd rather avoid.
EDIT: I'm looking for portable solutions.
EDIT^2: Some timings for a huge directory (run this in a directory that's initially empty, it will create 100000 empty files):
system.time(file.create(as.character(0:99999)))
#    user  system elapsed
#   0.720  12.223  14.948
system.time(length(dir()))
#    user  system elapsed
#   2.419   0.600   3.167
system.time(system("ls | head -n 1"))
# 0
#   user  system elapsed
#  0.788   0.495   1.312
system.time(system("ls -f | head -n 3"))
# .
# ..
# 99064
#    user  system elapsed
#   0.002   0.015   0.019

The -f switch is crucial for ls, it will avoid the sorting that will take place otherwise.
Comment: This question http://stackoverflow.com/questions/18685576/php-what-is-the-best-and-easiest-way-to-check-if-directory-is-empty-or-not?rq=1  has a comment recommending `rmdir` which "should" fail if the directory is non-empty.  Check your permission level!
Comment: I just removed my comments (on linux only solutions) but You're right portability is a nice feature to have.
Comment: Yeah, but... is it faster for empty or "not-huge" directories, and which are you more likely to run across?  :-(
Comment: @CarlWitthoft: Yes, and bubble sort is faster than quicksort for "not-huge" data... I've wondered, I'm curious, I've posted a question. Let's wait and see if there is a good answer.
Answer: How about if(length(dir(all.files=TRUE)) ==0)  ?
I'm not sure what you qualify as "fast," but if dir takes a long time, someone is abusing your filesystem :-(.
Comment: The OP mentionnend that he wants to avoid enumerating the entire contents of the directory (and `dir` == `list.files`) and without `all.files = TRUE` you will miss hidden files
Comment: @dickoa good point, sorry.  But I fail to see what's so bad about enumerating (since I'm not returning an object to the environment)
Comment: Not that bad actually but may be the OP is just looking for a more efficient alternative to this solution. I use linux and for this kind of task if I want to do it fast I use the shell through `system`.
Comment: @CarlWitthoft: I'm really nitpicking here. Enumerating perhaps is not noticeable performance-wise expect if a slow file system is used and the directory contains many entries. (You call it "abuse", but it might happen.)
Comment: `dir` is much faster than calling the operating system yourself.  Compare `system.time(dir())` with `system.time(system("ls", TRUE))`.
Comment: But you still get `.` and `..` when you get all files.
Comment: @James --- Oh, well, guess it's still operating-system specific.
Comment: @RichieCotton: I have edited my question to add a comparison for a huge directory that shows that `system("ls | head -n 1")` can be faster than `dir()`.
Comment: `head` doesn't exist in the Windows command shell. I tried the 100k files tests under Win7 with a local SSD, and I still find `dir` to be a little quicker.  On a network drive, most of the time involves passing data across a network and it doesn't matter which solution you use.  So as with many performance questions, there are no guaranteed wins; it depends on your setup.  Interesting question though.
Comment: @RichieCotton: Sorry, it has to be `ls -f` to avoid sorting.
Comment: S/He wants to be portable, and this does not work under Windows unless you add `no.. = FALSE`

Fast test if directory is empty


InnoDB: Fatal error: cannot allocate memory for the buffer pool

Question: MySQL Seems to be crashing quite often, I'd say every hour or so.
The only thing running on this server that is using MySQL is a Drupal installation that I'm working on, there's only one person that's using it and it literally has around three pages - from what I can gather this seems to be a memory issue with innoDB? I'm not great at this stuff so I'm unsure.
Any guidance? Here's the relevant parts from my error.log
<code>140216 14:20:01 [Note] Plugin 'FEDERATED' is disabled.
140216 14:20:01 InnoDB: The InnoDB memory heap is disabled
140216 14:20:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140216 14:20:01 InnoDB: Compressed tables use zlib 126.96.36.199
140216 14:20:01 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(135987200 bytes) failed; errno 12
140216 14:20:01 InnoDB: Completed initialization of buffer pool
140216 14:20:01 InnoDB: Fatal error: cannot allocate memory for the buffer pool
140216 14:20:01 [ERROR] Plugin 'InnoDB' init function returned error.
140216 14:20:01 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
140216 14:20:01 [ERROR] Unknown/unsupported storage engine: InnoDB
140216 14:20:01 [ERROR] Aborting

140216 14:20:01 [Note] /usr/sbin/mysqld: Shutdown complete

140216 14:20:02 [Note] Plugin 'FEDERATED' is disabled.
140216 14:20:02 InnoDB: The InnoDB memory heap is disabled
140216 14:20:02 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140216 14:20:02 InnoDB: Compressed tables use zlib 126.96.36.199
140216 14:20:02 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(135987200 bytes) failed; errno 12
140216 14:20:02 InnoDB: Completed initialization of buffer pool
140216 14:20:02 InnoDB: Fatal error: cannot allocate memory for the buffer pool
140216 14:20:02 [ERROR] Plugin 'InnoDB' init function returned error.
140216 14:20:02 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
140216 14:20:02 [ERROR] Unknown/unsupported storage engine: InnoDB
140216 14:20:02 [ERROR] Aborting

140216 14:20:02 [Note] /usr/sbin/mysqld: Shutdown complete
</code>
EDIT
The server is the lowest-spec DigitalOcean Droplet with 512MB RAM. This particular Droplet is running NodeJS, MySQL and Apache. There's a low-traffic static site, a Ghost blog and this Drupal site I'm developing.
Should I just upgrade the RAM, or is there something I can do to fix this?
Comment: It is a memory issue. Just how much memory don't you have?
Comment: It's the lowest-spec DigitalOcean Droplet, 512MB RAM. Running NodeJS, Apache and this Drupal site. Very low-traffic, used mainly for development.
Answer: MySQL doesn't start because it cannot allocate 128M for the InnoDB buffer pool.
You have to either set lower innodb_buffer_pool_size or limit number of Apache processes so they don't consume all memory. I had the same problem and described it in a blog
Comment: I see... It's strange though, because it runs fine at 128M initially, then after around an hour (I don't even have to be working on the Drupal site) it will crash. I run my personal site from the same server, and it is very, very low traffic and is just a simple HTML site, the blog is using Ghost which runs using NodeJS so surely that bypasses Apache all together? I'll stick innodb_buffer_pool_size down to 64M and see if that helps. Cheers.
Comment: Why strange? MySQL needs 128 continuous megabytes for the buffer pool. You have 512M, it's not a problem to allocate 128 if number of httpd processes is low. But over time, number of idle httpd processes increases and it becomes not possible to find 128M anymore. But check why MySQL crashes. That's worrying
Comment: You may also have other memory consumers. Check that too.
Comment: I would guess the oom-killer was why MySQL crashed in the first place.  MySQL is a favourite of the oom-killer.  It logs what it does and why.  Grep for "oom" or "Killed" in your logs.
Comment: Mod_php and the Apache prefork MPM will chew up a lot of RAM.  CGI mode with a different MPM (or a different http daemon altogether) will help.


Reduced effective sample size in balanced population after inverse probability treatment weighting

Question: I am new to inverse probability treatment weighting (IPTW) and I am trying to understand it. However, I am not a statistician and I am having some troubles with advanced statistics concepts.
As far as I understand, with IPTW you can achieve a randomization-like effect. First you calculate the propensity scores and afterwards, depending on the group of the patients (treated vs non treated) you weight each patient.
The main difference with propensity score matching (PSM) is that with IPTW you don't lose patients.
Thus the question: why does it happen that in my dataset or in others (example below), there is a reduced effective sample size in the balanced group? Moreover, is it possible to obtain in the balanced population the number of patients for each baseline variable?
For instance
<code>data("lalonde", package = "cobalt")
library("WeightIt")
W.out <- weightit(treat ~ age + educ + race + married + nodegree + re74 + re75,
                  data = lalonde, estimand = "ATT", method = "glm")
W.out
</code>

In this example I balanced for the following covariates: age, educ, race, married, nodegree, re74 and re75
My questions are:

Why is the effective sample size of the balanced population reduced? Does this mean that the frequencies of the variables have changed compared to the unbalanced population?
Is there a way to obtain the number of patients in the balanced population in the group of educ, race, married, nodegree...? Since the weighted population is reduced I assume that there have been some changes also in the patients for each variable.

Please try to explain in a simple way, I am really new to this.
Answer: First let's talk about weighting. Weighting involves estimating a weight for each unit in the sample and then estimating the treatment effect in a way that incorproates the weights, such as weighted linear regression, weighted maximum likelihood, or a weighted difference (or ratio, etc.) in means. The weights also enter the balance statistics (e.g., standardized mean difference and KS statistics).
Separate from IPTW, it's important to understand that weighted samples have lower precision than the corresponding unweighted sample. By precision, I mean the standard error of the estimate of the desired quantity (e.g., the causal mean or difference between causal means). If we think of uncertainty arising due to the sampling of individuals, the estimate will change a lot if a unit with a large weight is swapped out for another unit in the population. Similarly, having a unit with a weight of 2 is not the same as having 2 units; choosing to use one unit's information twice doesn't give you more information about your sample; you are just changing the importance of a unit's contribution to your estimates.
The ratio of the variance (i.e., square of the standard error) of the estimate of a mean from a weighted sample to the estimate of a mean from an unweighted sample is known as the "design effect" (i.e., the effect of the "design" on the precision of the estimate) and was derived by Shook-Sa and Hudgens (2020) to be
$$
\text{deff}_w = \frac{N \sum_{i}w_i^2}{(\sum_{i} w_i)^2}
$$
where $N$ is the sample size of the group and $w_i$ is the weight for unit $i$. The effective sample size (ESS) is defined as
$$
\text{ESS} = \frac{(\sum_i w_i)^2}{\sum_i w_i^2} = \frac{N}{\text{deff}_w}
$$
and represents the size of an unweighted sample that contains the same precision as a weighted sample. When the weights are scaled to have an average of 1 (i.e., and a sum of $N$), the ESS can be equivalently written as
$$
\text{ESS} = \frac{N}{1 + \text{Var}(w^*)}
$$
where $\text{Var}(w^*)$ is the variance of the scaled weights computed using the population formula. This latter formula makes it easy to see that as the variance of the weights increases, the ESS gets smaller.
The ESS isn't the same as the sample size, but it functions like it. That is, if you have a sample of 1000 units but after weighting the sample has an ESS of 500, then you will have (approximately) the same precision as if you only had 500 units in an unweighed sample. That is the price we pay for using the weights to remove confounding. In this way, matching and weighting both function the same way, which is to trade precision for unbiasedness. The fact that weighting retains all units doesn't mean the estimates will be as precise as had you not done weighting at all; in fact, the ESS from IPTW can be smaller than the remaining sample size after matching.
The frequencies of events/conditions in the dataset is not a useful way to think about the changes caused by the weighting. The frequencies are the same as they prior to weighting; you will still have the same number of events, the same number of patients with conditions, etc., but those patients will contribute different amounts of information to the estimation of the effect and change the precision accordingly. It is useful, though, to think about weighted means and proportions of variables, since these represent balance in the weighted sample. So while the proportion of married units in one group may change from .2 to .4, that doesn't mean the actual number of married units has changed. It means the weighted sample is meant to represent a population in which 40% are married.
So, to sum up:

The size of the weighted sample and the number of units with each condition (e.g., the number of events) is the same as the unweighted sample. Weighting does not change the sample size or number of events; it changes the relative contribution of each individual to the estimation of the treatment effect
The effective sample size (ESS) is a measure that approximately captures the degree of precision remaining in the sample after weighting; it is not a literal sample size. It is a diagnostic statistic used for analysts to decide whether their weights have degraded their precision to an unacceptable degree. It should be reported in papers but very rarely is.
The frequency of events is not a useful way to think about prevalence in a weighted sample. The frequency is the same in the weighted and unweighted samples, but the amount of information contained in each event changes depending on its weight. The weighted proportions/rates capture this information. Balance should be assessed on the proportion of events, not their frequency, so computing the weighted mean of a binary variable is the appropriate way to characterize prevalence after weighting, not a frequency count.
Comment: @arielhasidim There is no rule of thumb except the same rules you woudl apply to a regular sample size. That means you should od a powr analysis before the weighting to see the smallest ESS you can have to detect the effect size of interest. I would guess 100 units in each group would be too small to detect many meaningful effect sizes. I explain in the `WeightIt` [vignette](https://ngreifer.github.io/WeightIt/articles/estimating-effects.html) on estimating effects how to validly estimate standard errors and confidence intervals (and p-values) after weighting.
Comment: @arielhasidim P-values don't consider sample sizes. They come from a formula computed from the estimate and its standard error. The standard error is computed correctly incorporating the weights if a robust SE is used as recommended in the `WeightIt` vignette. Please read the document because it explains the various modes of estimating SEs and CIs and their advantages and disadvantages.
Comment: Thank you! You have been incredibly clear. I have a question: as far as I understand in order to avoid extreme weights you can either stabilize the weights (stabilize=TRUE in WeightIt) or you can perform "trimming" with trim(). However, trimming from what I understand can mean either removing the observations responsible for the extreme weights or - as in trim() - "setting all weights higher than that at a given quantile..." If I choose to "trim" by deleting observations, how can I obtain the trimmed sample size of my population? Is it better in your opinion to use trim() or stabilize weights?
Comment: Stabilizing weights doesn't really do anything for a single time point treatment. It doesn't change balance or the ESS. It can slightly change the effect estimate if you use a certain kind of model. You can use `trim()` and just use `summary()` or `bal.tab()` to view the ESS after trimming. No units are dropped when trimming, so the sample size is the same. If you want to drop units from the sample, you can use `estimand = "ATOS"` or drop them manually, but I don't recommend this. Use a weighting method that yields good properties off the bat instead of trimming or dropping observations.
Comment: Thank you so much! I will mark as answered and upvote! Thank you again
Comment: Thank you @Noah! I'm also pondering the consequences of significantly reduced ESS due to significant weighting, and I'm worried about the limit. Let's say I have 100 treated cases and 1K controls, and I use full matching to estimate ATT. Is there a rule of thumb about the ESS? Should I aspire not to go under a 1:1 ratio with the treated? And maybe more importantly, is the p-value from marginaleffects::avg_comparisons considering the ESS or the actual sample size?
Comment: @Noah Thanks again. And the last question? ("is the p-value from marginaleffects::avg_comparisons considering the ESS or the actual sample size?")


Show that for each $x \in X$ there exists $ a \in A$ with $d(x,a) = dist(x,A)$

Question: Let $A$ be a compact set. Show that for each $x \in X$ there exists $ a \in A$ with $d(x,a) = dist(x,A)$. Show that if A is only closed then this is not valid in general.
Should I use the convexity of A to prove this? (since every bounded sequence in X has a convergent subsequence).
Comment: See also: [For any point $a$ of a compact subset $S$ of a metric space, prove that there exists a nearest point $ c $ to $ a $.](https://math.stackexchange.com/q/333479) and [Distance from a compact subset need not be attained in a metric space?](https://math.stackexchange.com/q/1767670)
Answer: I suppose that $(X,d)$ is a metric space and that $A$ is a compact subset of $X$.
Let $x \in X$. Then $dist(x,A)= \inf_{a \in A}d(x,a)$. Hence there is a sequence $(a_n)$ in $A$ such that $d(x,a_n) \to dist(x,A)$.
A is compact, hence $(a_n)$ contains a convergent subsequence $(a_{n_k})$ with limit $a \in A$.
Then we have:
$d(x,a)= \lim_{k \to \infty}d(x,a_{n_k})=dist(x,A)$.
Answer: If you fix $x$, then the map $a \mapsto d(x, a)$ is continuous. Restrict it to a compact set $A$, and the resulting set, being the continuous image of a compact set, is also compact in $\mathbb{R}$. In particular, it must have a minimum.
Answer: (1) Assume that $X$ is a metric space. $f: A\rightarrow \mathbb{R},\
f(a )=d(x,a)$. So $f$ is continuous so that since $A$ is compact,
$f$ has a minimum at $a_0$.
Hence $f(a_0)=d(a_0,x)\leq d(a,x)$ for all $a\in A$.
(2) If $X_i=[0,1]$ and $
 Y=\bigcup_i\ X_i$ is disjoint union, then
$X$ is a quotient of $Y$ by identifying $0$ in $X_i$.
Consider $a_i=\frac{1}{2}+\frac{1}{2i}\in X_n$ and let $A=\{ a_i|
i\in \mathbb{N}\}$. Then $A$ is closed.
Here $d(0,a_i)\rightarrow \frac{1}{2}$ but there is no $a\in A$ s.t.
$d(0,a)=\frac{1}{2}$
Answer: Note that if $A$ is compact, then it is bounded and therefore, $\mathrm{d(x,a)}$ is finite. This could fail if $A$ were only closed.
By definition,$$\mathrm{d(x,A)=\inf\{\mathrm{d(x,a)}: a\in A\}}$$
Therefore, if $\mathrm{d(x,a)}=\alpha$, for any $\epsilon>0$, there exists $a_{\epsilon} \in A$ such that $$\alpha \leq d(x,a_{\epsilon})<\alpha+\epsilon$$
Now take $\epsilon=\frac{1}{n}$ and you will get a sequence in $A$ and because $A$ is compact and therefore closed, the limit must be in $A$.
Therefore, $\lim_{n\to \infty}{a_n}=a_* \in A$ and $$\lim_{n\to \infty}\mathrm{d(x,a_n)}=\mathrm{d(x,\lim_{n\to \infty}a_n)}=\mathrm{d(x,a_*)}$$
The second equality holds because metric is continuous in both components.


C# regex to validate filename

Question: I want to replace these characters with string.Empty:'"<>?*/\|  in given Filename
How to do that using Regex
I have tried this:
Regex r = new Regex("(?:[^a-z0-9.]|(?<=['\"]))", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);
                 FileName = r.Replace(FileName, String.Empty);

but this replaces all special characters with String.Empty.
Comment: http://mattgemmell.com/2008/12/08/what-have-you-tried/
Comment: possible duplicate of [How to remove illegal characters from path and filenames?](http://stackoverflow.com/questions/146134/how-to-remove-illegal-characters-from-path-and-filenames)
Answer: You could use the Regex.Replace method. It does what its name suggests.
Regex regex = new Regex(@"[\\'\\""\\<\\>\\?\\*\\/\\\\\|]");
var filename = "dfgdfg'\"<>?*/\\|dfdf";
filename = regex.Replace(filename, string.Empty);

But I'd rather sanitize it for all characters that are forbidden in a filename under the file system that you are currently using, not only the characters that you have defined in your regex because you might have forgotten something:
private static readonly char[] InvalidfilenameCharacters = Path.GetInvalidFileNameChars();

public static string SanitizeFileName(string filename)
{
    return new string(
        filename
            .Where(x => !InvalidfilenameCharacters.Contains(x))
            .ToArray()
    );
}

and then:
var filename = SanitizeFileName("dfgdfg'\"<>?*/\\|dfdf");

Answer: look here how to do it:
How to remove illegal characters from path and filenames?
remember to use Path.GetInvalidFileNameChars()

C# regex to validate filename


XPath for expressions on same attributes connected by "or"

Question: I need to write a xpath expression to satisfy the following requirement:
EventID with 4357 or EventID with 8473 or EventID with 3689 should be resulted.
I wrote it as follows:
<code>"*[System/EventID = 4357 or EventID = 8473 or EventID = 3689]"
</code>
it results event id with 4357 only. It means it results only results that satisfies first expression. I don't have any idea about how to write this query. How to write such queries?
Answer: Using XPath 1.0, you can either do it this way :
<code>*[System/EventID = 4357 or System/EventID = 8473 or System/EventID = 3689]
</code>
or this way :
<code>*[System[EventID = 4357 or EventID = 8473 or EventID = 3689]]
</code>
Comment: It supports maximum 23 events


Error with the following dynamic sql statement

Question: I am having an error I cannot see in the following dynamic SQL statement:
  SET @sql_statement =
    "INSERT INTO " +
      @archive_db_name + ".dbo._TEST_TB_ACTIVE_ORDERS
    SELECT " +
      @reporting_db_name + ".dbo._TEST_TB_ACTIVE_ORDERS.*
    FROM " +
      @reporting_db_name + ".dbo._TEST_TB_ACTIVE_ORDERS
    WHERE " +
      @reporting_db_name + ".dbo._TEST_TB_ACTIVE_ORDERS.ORDER_ID
    IN
      (
        select #tmp_table_order_ids.order_id
        from #tmp_table_order_ids
      )"
  EXEC(@sql_statement)

The error is:
Msg 102, Level 15, State 1:
Server 'server_name', Line 5:
Incorrect syntax near 'IN'
But I cannot see any syntax errors at the location the error specifies. Could someone please kindly point out why I am getting this error?
Comment: I don't use Sybase, but does PRINT @sql_statement look okay?  Do you have SET QUOTED_IDENTIFIER OFF?
Answer: Try changing the double quotes to single quotes
  SET @sql_statement =
    'INSERT INTO ' +
      @archive_db_name + '.dbo._TEST_TB_ACTIVE_ORDERS
    SELECT ' +
      @reporting_db_name + '.dbo._TEST_TB_ACTIVE_ORDERS1.*
    FROM ' +
      @reporting_db_name + '.dbo._TEST_TB_ACTIVE_ORDERS1
    WHERE ' +
      @reporting_db_name + '.dbo._TEST_TB_ACTIVE_ORDERS1.ORDER_ID
    IN
      (
        select #tmp_table_order_ids.order_id
        from #tmp_table_order_ids
      )'
  EXEC(@sql_statement)

Answer: There is no error :
declare @reporting_db_name varchar(50)
declare @archive_db_name varchar(50)

declare @sql_statement varchar(500)

select @reporting_db_name ='PHILARIS_RD'

select @archive_db_name ='PHILARIS_RD'
SET @sql_statement =
    "INSERT INTO " +
      @archive_db_name + ".dbo.A
    SELECT " +
      @reporting_db_name + ".dbo.A.*
    FROM " +
      @reporting_db_name + ".dbo.A
    WHERE " +
      @reporting_db_name + ".dbo.A.ID
    IN
      (
        select B.ID
        from B
      )"
 EXEC(@sql_statement)


pointwise converging sequence of continuous functions: bounding oscilation from below

Question: Assume It is known know that oscilation $\omega(f_n, U) > \epsilon$ for continuous functions $f_n$ and some open set $U$ starting from some $N$.
Also it is known that $f_n \to \varphi$ pointwise.
It is also safe to assume that $\overline{U}$ is compact..
Here $$\omega(f,U) = \sup_{x,y \in U} \Big|f(x) - f(y)\Big|$$
Intuitively, it seems true that  that $\omega(\varphi,U) \ge \epsilon$.
But is it actually true?
I thought that it must be possibele to find points $x_n,y_n \in \overline{U}$ such, that
$$
  \omega(f_n,U) \le \Big|f_n(x_n) - f_n(y_n) \Big|
$$
Then compute limits of converging subsequences, which exist by compactness  of $\overline{U}$. Name them $Y$ and $X$.
And Then to show that $|\varphi(Y) - \varphi(X)|$ is substantially  big by convergence and continuity arguments.
But I don't know how to do it as both argument and function depend on $n$.
Answer: To answer the question as is: No. We do not necessarily have that $\omega(\varphi,U) \ge \epsilon$.
For example, take
$$f_n : (0,1) \rightarrow \mathbb{R} : x \mapsto \frac{1}{nx}$$
Then, for all $n$, we have that $\, \omega(f_n, U) \ge \epsilon \,$, but since $\, f_n \xrightarrow{\text{p.w.}} 0 \,$, $\,\omega(\varphi,U) = 0$.
To me it seems that the more interesting question would be to replace $U$ with $\overline{U}$.
Comment: Ok.  I will reask my question. I think that to have $f_n$ uniformly bounded is enough to avoid this kind of results. Your answer is undoubtedly  still correct.
Comment: https://math.stackexchange.com/questions/2539518/pointwise-converging-sequence-of-continuous-functions-on-a-compact-bounding-osc
Comment: Now I think you were right about closure.


Rdlc report viewer in not shwoing report in a WPF applicaiton

Question: I am trying to view a rdlc report in a WPF application I have used the WindowsFormHost to host the rdlc report viewer but when i run the application no error at all but also noting is showing in report-viewer not even the report-viewer is showing.
Here is the xaml code
<code><WindowsFormsHost Name="windowsFormsHost" Grid.Row="2" >
            <rv:ReportViewer x:Name="reportViewer"/>
        </WindowsFormsHost>
</code>
And the c# code for report loading
<code> List<Furniture> Purchases = null;
            using (IDbConnection db = new SQLiteConnection(ConfigurationManager.ConnectionStrings["cs"].ConnectionString))
            {
                if (db.State == ConnectionState.Closed)
                {
                    db.Open();
                }
                string Command = "Select I.Id,I.Title,I.Category,I.Description,I.VoucherNumber,I.Cost,I.PurchasedDate,B.Title AS BudgetHead,D.Name AS Distributer, F.Type,F.Color,F.Condition,I.Status,I.CreatedBy FROM Items I INNER JOIN Furniture F ON I.Id=F.EntryId INNER JOIN BudgetHeads B ON I.BudgetHeadId=B.Id INNER JOIN Distributers D ON I.DistributerId=D.Id";
                Purchases = db.Query<Furniture>(Command,null, commandType: CommandType.Text).ToList();
            }
            reportViewer.LocalReport.DataSources.Clear();
            var DataSource = new ReportDataSource() { Name = "FurnitureDataSource", Value = Purchases };
            reportViewer.LocalReport.DataSources.Add(DataSource);
            string Path = @"D:\ADiTM\Products\Stock Management\Software\Reports\Purchase.rdlc";
            reportViewer.LocalReport.ReportPath = Path;
            reportViewer.Dock = DockStyle.Fill;
            reportViewer.Refresh();
            reportViewer.RefreshReport();
</code>
Comment: i think you are missing the `reportViewer.SetDisplayMode(DisplayMode.Normal)`
Comment: @styx I have used the display mode but still not working,as i mentioned in the question not even the report-viewer is showing
Comment: did you initialize the `DataContext` in your window?
Comment: I have that the problem is with the windows host, which is not showing at run-time and  have even added a simple MaskedTextBox to check but it is not that as well in run-time. Can any one please comment?
Answer: I got the solution, actually the allow transparency was set to true in main window which s causing the windows host not showing any thing at run-time.


DC.js - barChart not filtered on click of geoChoroplethChart

Question: I have a choropleth map of counties, and a bar chart showing population of each county on the map. When I click on the bar chart the map is filtered and redrawn to show the selected bar, but when I click on the a county in the choropleth map the bar chart is not filtered to show the population data. I don't understand why it would be filtering one way but not the other. Any help is appreciated!

    Population by counties (color: total population)
    reset
     | Current filter:

    Population by county
    reset
     | Current filter:

    Reset All

var iowaMap = dc.geoChoroplethChart("#iowa-map");
var populationChart = dc.barChart("#population-chart");

d3.csv("iowaCountiesPop.csv", function (data) {

      data.forEach(function (d) {
        d.county = d.county;
        d.popByCounty = +d.e2015;
      });

      var data = crossfilter(data);

      var counties = data.dimension(function (d) {
        return d.county;
      });

      var counties2 = data.dimension(function (d) {
        return d.county;
      });

      var popByCounty = counties.group().reduceSum(function (d) {
        return d.popByCounty;
      });

    d3.json("IowaCounties.json", function (countiesJson) {
        iowaMap.width(990)
                .height(500)
                .dimension(counties)
                .group(popByCounty)
                .projection(d3.geo.mercator()
                    .translate([495, 250])
                    .rotate([93 + 20 / 60, -41 - 60 / 60])
                    .scale(7900))
                .colors(d3.scale.quantile().range(colorScheme[quantiles]))
                .colorDomain([0, 430640])
                .overlayGeoJson(countiesJson.features, "NAME", function (d) {
                    return d.properties.NAME;
                })
                .title(function (d) {
                    return d.key + " County \nTotal Population: " + numberFormat(d.value);
                })
                .on('renderlet', function(map) {
                    map.selectAll("path").on("click", function(d) {
                    //console.log("click!", d)
                     map.filter(d.properties.NAME)
                        .redrawGroup();
                    })
                });

         populationChart.width(width)
            .height(height)
            .dimension(counties2)
            .group(popByCounty)
            .x(d3.scale.ordinal().domain(counties))
            .xUnits(dc.units.ordinal)
            .margins({top: 0, right: 0, bottom: 70, left: 70})
            .yAxisLabel(["Population Values"])//,[12])
            .xAxisLabel("County Names")
            .barPadding(0.1)
            .outerPadding(0.05)
            .elasticY(false)
            //.turnOnControls(true)
            .on('renderlet', function(chart) {
                chart.selectAll('rect').on("click", function(d) {
                    //console.log("click!", d)
                    chart.filter(d.data.key)
                        .redrawGroup();
                })
                chart.selectAll("g.x text")
                    .style("text-anchor", "end")
                    .attr("dx","-8")
                    .attr("dy", "5")
                    .attr("transform", "rotate(-50)");
            });

        dc.renderAll();

    });

});

Comment: I realize this is is almost a complete example, but if you could provide a fiddle of it ([including the data](http://stackoverflow.com/a/22896088/676195)), or another running example, it will be much easier for us to diagnose.
Comment: Well here is my attempt at putting it in jsfiddle - https://jsfiddle.net/bahanson/xo5dw3m6/ but it's not actually working, so I doubt it will actually be helpful.
Comment: ftfy: https://jsfiddle.net/gordonwoodhull/28qsa0jr/4/ - just the usual version mismatches, and your data was actually tab-separated not comma separated. Of course, I could probably have stared very hard at your code and figured this out, since it's probably The Most Frequently Asked question, but it did help to use the fiddle, thanks!
Answer: So what is the the Most Frequently Asked Question Ever for dc.js?
Why is my chart not filtering?
And what is the most frequent answer?
Your charts are on the same dimension, or more specifically, your second chart's group is observing the same dimension that the first chart is filtering. Which means that it will not see the changes, because groups do not observe their own dimensions.
It looks like you started to go in this direction, but only duplicated the dimension. Both charts are observing the same group, and since that group is produced from the dimension for the map, it will not observe filtering on the map.
iowaMap
   .dimension(counties)
   .group(popByCounty)

populationChart
   .dimension(counties2)
   .group(popByCounty)

Instead:
function pop_by_county(dim) {
  return dim.group().reduceSum(function(d) {
    return d.popByCounty;
  });
}

var popByCounty = pop_by_county(counties),
  popByCounty2 = pop_by_county(counties2);

populationChart
  .dimension(counties2)
  .group(popByCounty2)

https://jsfiddle.net/gordonwoodhull/28qsa0jr/7/


Convergence for $\frac{1}{n} {\bf p}^H {\bf D}{\bf q}$ using law of large numbers

Question: Suppose ${\bf p} = [p_1, \dots, p_n]^T$ and ${\bf q} = [q_1, \dots, q_n]^T$ be mutually independent $\mathbb{C}^{n \times 1}$ vectors whose elements are i.i.d zero-mean and unit variance random variable (RVs), i.e., $\mathbb{E}\{|p_i|^2\} = \mathbb{E}\{|q_i|^2\} = 1$. Then from law  of large numbers we have $\frac{1}{n} {\bf p}^H {\bf q} \to 0$.
Now let ${\bf D} \in \mathbb{R}^{n \times n}$ be a diagonal matrix with different real entries, i.e., $d_{n}$. Then can I use law of large numbers to find convergence for:
\begin{equation}
\frac{1}{n} {\bf p}^H {\bf D}{\bf q}
\end{equation}
Comment: It depends on how $D$ depends on $n$.
Comment: Hi Kimchi. $D$ is a diagonal matrix with real entries in diagonal and each element is different. I think if the elements of $D$ are same we can directly use law of large numbers. But in this case the elements are different.
Comment: But how do the entries in $D$ change as $n$ changes? If the $k$th diagonal entry in $D$ is $k!$, for example, most of the oomph in your sum comes from the last term, and LLN behavior will not hold.
Answer: I am interpreting the question as follows:

Let $(d_{ij})_{i,j\in\mathbb N}$ be a deterministic doubly infinite
  sequence. Let $(p_i)_{i,\in\mathbb N}$ and $(q_i)_{i,\in\mathbb N}$ be
  independent iid sequences whose elements have mean $0$ and variance
  $1$. Let $X_n=\sum_{i,j=1}^np_id_{ij}q_j$. Is it true that $\frac{X_n}{n}$ converges to $0$ in probability?

Before answering the question, I just want to point out that a routine calculation using independence of the $p_i$ and $q_j$ shows that that $\mathbb EX_n=0$ and
$
\mathbb E[X_n^2]=\sum_{i,j=1}^nd_{ij}^2=\|D_n\|_2^2.
$
This shows that $n$ is the wrong scaling in the statement above, rather it should be $\|D_n\|_2^2$ in general (note that $\|D_n\|_2^2=n$ when $D_n$ is the identity matrix). For example, when $d_{ij}=1$ for all $i,j$ the quantity $X_n/n$ has infinite variance in the limit. In fact, in this case the variance of $X_n$ grows like $n^2$, not $n$, since we are summing $n^2$ independent quantities (unlike the case of the identity matrix, when there are only $n$).
Now let's answer the corrected version of the question.

Let $X_n$ and $\|D_n\|_2$ be as above. Assume that $\lim_{n\to\infty}\|D_n\|_2=\infty$. Then $\frac{X_n}{\|D_n\|_2^2}$ converges to $0$.

Proof:
By Chebyshev's inequality,
$$
\mathbb P\Bigl(\frac{X_n}{\|D_n\|_2^2}>\epsilon\Bigr)\leq \frac{1}{\epsilon^2\|D_n\|_2^2}.
$$
By assumption, this tends to $0$ as $n\to\infty$, for all $\epsilon >0$, proving the claim.
Note, the assumption $\|D_n\|_2\to\infty$ is necessary. For example, if $d_{ij}=0$ whenever $i$ or $j>N$ then $X_n$ stays random in the $n\to\infty$ limit, so no convergence is possible.
Note. When $D_n$ is assumed to be a diagonal matrix, it may still not be the case that $\|D_n\|_2\to\infty$, for example if $d_{n,n}=n^{-2}$. Likewise, it can also be the case that $\|D_n\|_2$ grows asymptotically faster than $n$. So assuming the matrix is diagonal does not reduce the number of assumptions one has to make.
Comment: Hi. I just wanted to clarify a calculation based on your answer. So  I am now calculationg the value for $X = {\bf p}^H{\bf D}{\bf p}$. Now, based on your answer, $E[X] = \sum_i d_{i,i}$, where I have used the fact that ${\bf D}$ is a diagonal matrix. Now with $E[X] = \sum_i d_{i,i}$, can I claim $\frac{1}{n}{\bf p}^H{\bf D}{\bf p} \to \sum_i d_{i,i}$, as $n \to \infty$.
Comment: No, $\mathbb E[X_n]=0$ since the $p_i$ and $q_i$ have mean zero. It is essentially irrelevant whether $D_n$ is diagonal or not. And as I wrote above, normalizing using $n$ instead of $\|D_n\|_2^2$ means that the limit could either be $0$, or it could be $\infty$, or it could oscillate around in between and stay random.
Comment: Will $\mathbb{E}({\bf X}_n)$ be still be $0$ if ${\bf X}_n = {\bf p}^H {\bf D} {\bf p}$ instead of ${\bf X}_n = {\bf p}^H {\bf D} {\bf q}$, i.e., $\mathbb{E}[{\bf X}_n] = \mathbb{E}[ {\bf p}^H {\bf D} {\bf p}] = \mathbb{E}[\sum_i p_i^* d_{i,i} p_i] = \sum_i \mathbb{E}[|p_i|^2] d_{i,i} = \sum_i d_{i,i}$ given $\mathbb{E}[p_i^2] = 1$.
Comment: You are changing the question in the comments and it is hard to see (I misread your $q$ as a $p$ in the last comment). Just ask another question instead of doing this in the comments....
Comment: Hi thank you for the suggestion. I will post this clarification as a separate question.


Return Top Track Spotify API

Question: I am trying to create a helper file that will return the top track(s) of any artist's related artists. All I want to do use 1 artist URI to surface their related artists' name, popularity, and top track. And I want to separate out the top track functionality into a separate file that I can call whenever.
But, I can't seem to figure out how to properly return the top track of the related artists.
In  my "get-toptrack.js" file:
   require([
      '$api/models',
      '$api/toplists#Toplist'
    ], function(models, Toplist) {
      'use strict';

      var doGetTopTrack = function(uri, num) {
        var artistURI = uri;
        var artist = models.Artist.fromURI(artistURI);
        var artistTopList = Toplist.forArtist(artist);

        artistTopList.tracks.snapshot().done(function(snapshot){

          snapshot.loadAll('name').done(function(tracks) {
            var i, num_toptracks;
            num_toptracks = num;

              for(i = 0; i < num_toptracks; i++){
                console.log("top track: " + tracks[i].name);
                // WHERE DO I RETURN THE TOP TRACKS??
              }
          });
        });
      };

      exports.doGetTopTrack = doGetTopTrack;
    });

In my "artist.js" file":
require([
    '$api/models',
    'scripts/get-toptrack'
  ], function(models, getTopTrack) {
    'use strict';

    var showRelated = function() {
      var artist_properties = ['name', 'popularity','related', 'uri'];

      models.Artist
        .fromURI('spotify:artist:11FY888Qctoy6YueCpFkXT')
        .load(artist_properties)
        .done(function(artist){

          artist.related.snapshot().done(function(snapshot){
            snapshot.loadAll('name').done(function(artists) {

              for(var i = 0; i < artists.length; i++){
                var u, p, n, t, listItem;

                // store artist details
                p = artists[i].popularity;
                n = artists[i].name;
                u = artists[i].uri;

                // am I missing something here?
                t = getTopTrack.doGetTopTrack(u, 1);

                listItem = document.createElement("li");
                listItem.innerHTML = "Name: " + n.decodeForText() + " | Popularity:  " + p + " | Top Track: " + t;

                // undefined name
                $('#artistsContainer').append(listItem);

              }
            });

          });
        });
    };

    exports.showArtists = showArtists;
  });

And in the 'main.js' file, I call the artists function to begin.
require([
  '$api/models',
  'scripts/artist'
], function(models, initArtist) {
  'use strict';

  initArtist.showRelated();
});

Answer: can't comment on the multiple .js files as I haven't yet refactored my app into multiple files.
As far as returning goes, you're working in an asynchronous application, so you can't.  You have to use a callback, or a Spotify Promise might make your api more congruous with Spotify's.  Look at the documentation for Promise.each().
Here is an implementation of the callback method.  Changed a few things to make it easier for me to test.  Made life a bit easier to pass the artist in instead.  Also, there is no guarantee of the order they will come out since the second artist's toplist could come back faster than the first.  You'll need to add more code if you want to keep order.
function doGetTopTrack(artist, num, callback) {
    var artistTopList = Toplist.forArtist(artist);

    artistTopList.tracks.snapshot(0,num).done(function (snapshot) { //only get the number of tracks we need

        snapshot.loadAll('name').done(function (tracks) {
            var i, num_toptracks;
            num_toptracks = num; //this probably should be minimum of num and tracks.length

            for (i = 0; i < num_toptracks; i++) {
                callback(artist, tracks[i]);
            }
        });
    });
};

function showRelated() {
    var artist_properties = ['name', 'popularity', 'related', 'uri'];

    models.Artist
      .fromURI('spotify:artist:11FY888Qctoy6YueCpFkXT')
      .load(artist_properties)
      .done(function (artist) {

          artist.related.snapshot().done(function (snapshot) {
              snapshot.loadAll('name').done(function (artists) {

                  for (var i = 0; i < artists.length; i++) {
                      // am I missing something here?
                      doGetTopTrack(artists[i], 1, function (artist, toptrack) {
                              console.log("top track: " + toptrack.name);

                              var p = artist.popularity;
                              var n = artist.name;
                              var u = artist.uri;

                              //listItem = document.createElement("li");
                              console.log("Name: " + n.decodeForText() + " | Popularity:  " + p + " | Top Track: " + toptrack.name);

                              //// undefined name
                              //$('#artistsContainer').append(listItem);
                      });
                  }
              });

          });
      });
};
showRelated();

Comment: Thank you for this. Explains alot!
Comment: Is there a particular reason to retain 'order' from grabbing the tracks? If so, can you point me in the right direction of what to look up in the Spotify Docs?
Comment: In this case, I don't see any reason to maintain order.  But if you really wanted to make sure your artist list entries were in the same order as 'related', then you'd need to redo to logic to store each track and output them all at the end.
Comment: If I wanted to create a 'list' of these top tracks, would I have to completely refactor my code above?
Comment: http://stackoverflow.com/questions/19781876/spotify-api-create-temp-playlist-not-loading

Return Top Track Spotify API


weka: train and test set in different format (arff and text format)

Question: i have an arff file that is built with stringtowordvectorand contains features and it's TFIDF wight like this:
@relation 'sss'
-weka.filters.unsupervised.attribute.StringToWordVector-R-W100-prune-rate-1.0-C-T-I-N0-S-stemmerweka.core.stemmers.NullStemmer -tokenizerweka.core.tokenizers.WordTokenizer -delimiters \" ؟،؛\\r\\t\\n.,;:\\\'\\\"()?!-><#$\\\%&*+/@^_=[]{}|`~0123456789\"'

@attribute @@class@@ {mis,pol}
@attribute water numeric
@attribute start numeric
@attribute government numeric

{2 0.285724,6 0.338022,7 0.517187,8 0.164801,9 ...}
{7 1.191401,8 0.560813,9 0.904039,10 0.322267....}
..
....
{0 pol,6 1.276448,36 0.702977,...}

now i have a test folder that contain 2 class text.(such as train set:pol and mis ).
and i want to classify this test and evaluate my train set.i know that for this purpose i should use batch filter  so i read this link :
http://weka.wikispaces.com/Use+WEKA+in+your+Java+code#Filter-Batch%20filtering
based on this link , my test and train set should be in the same format (simple text format) .i don't know what  should i do when my train set is in arff format and my test set is in text format.(i don't have train set in text files format )
Answer: You can do the following:

Take your previous training set file in ARFF format without applying the StringToWordVector filter.
Generate a test set file using TextDirectoryToARFF.
Now you have two ARFF files with text in plain format. Thus apply the StringToWordVector filter to both of them in batch mode.
Comment: my problem is :i don't have training set in text format,i just have arff file with mentioned format.
Comment: It seems strange you say that you do not have the training set in text format, as you are posting an ARFF file which header is an original relation named `'sss'` and after a `StringToWordVector` filter has been applied. In my answer I mean taking the original `@relation 'sss'` file.
Comment: let me describe the problem :i have tree dataset in ARFF format that their attribute did not match(i asked about these problem in this link :http://stackoverflow.com/questions/21067439/how-to-match-attributes-order-of-two-instances-in-weka  ).I built these ARFF file with 'StringToWordVector' **separately** .now i have an arff file in format that i describe.now i want to test an external test set ,but i have not the original text files.is that possible to do that?


How do you take a weighted mean where values farther from a given number are weighted less?

Question: Say, I have a vector v = {5,2,8,9,2}
I then want a weighted mean where values of v closer to a given number (which could be mean(v), or something else) are weighted more (proportionally). So 5 (or v(1)) should be weighted more heavily than 9 (which should be given low weight).
The given number could be the mean, but it could also be any other value. For sake of simplicity though, you can give the answer as if you were giving it for the distance from the mean.
Comment: A working example appears at http://gis.stackexchange.com/questions/35159, where weights are based on distance in the plane.
Comment: Sorry to ask, but: why was my answer un-accepted?
Answer: The mean known as Dodonov's mean seems to fulfill what you are asking for: in this, every data-point is assigned a weight based on the inverse average distance to each of the other data points.
Let $w_i = {{n - 1} \over {\sum^{n}_{j-1}|x_i-x_j|}}$ be the weight for the i'th datapoint. Then the Dodonov-mean is defined as
$\mu_D = {{\sum^n_{n=1}w_ix_i} \over{\sum^n_{n=1}w_i}}$
The paper by the Dodonovs is located here.


How to prove this is a basis for $\mathbb{R}^3$ even after being rotated

Question: I have $\{\vec{v_1},\vec{v_2},\vec{v_3}\}=\{(1,0,0)^T,(0,1,0)^T,(0,0,1)^T\}$ as a basis for $\mathbb{R}^3$. I know intuitively that $\{A\vec{v_1},A\vec{v_2},A\vec{v_3}\}$ is also a basis for $\mathbb{R}^3$ where $A$ is the transformation represented by rotating 30 degrees counterclockwise about the axis spanned by $(1,1,1)^T$ because the second basis simply represents a rotated version of the original coordinate system of $\mathbb{R}^3$.
How would I go about proving this though? I'm not sure how to rigorously show that this second basis is still a basis for $\mathbb{R}^3$ even though I know this is true geometrically.
Comment: Just show that your 2nd basis is a set of linearly independent vectors.
Answer: The key thing here is that $A$ is invertible; as such, linear independence of $\{\vec{v}_1,\vec{v}_2,\vec{v}_3\}$ implies linear independence of $\{A\vec{v}_1,A\vec{v}_2,A\vec{v}_3\}$.
Why?  Suppose that $\{A\vec{v}_1,A\vec{v}_2,A\vec{v}_3\}$ is not a linearly independent set.  Then we can write
$$
\vec{0}=\alpha_1A\vec{v}_1+\alpha_2A\vec{v}_2+\alpha_3A\vec{v}_3=A(\alpha_1\vec{v}_1+\alpha_2\vec{v}_2+\alpha_3\vec{v}_3),
$$
for some $\alpha_1,\alpha_2,\alpha_3$ which are not all zero. Invertibility of $A$ then implies
$$
\alpha_1\vec{v}_1+\alpha_2\vec{v}_2+\alpha_3\vec{v}_3=\vec{0},
$$
contradicting linear independence of $\{\vec{v}_1,\vec{v}_2,\vec{v}_3\}$.
So, as long as you can argue that $A$ is invertible (which shouldn't be too hard), you can also argue that this new set is linearly independent. The fact that $\mathbb{R}^3$ is 3-dimensional gets you the rest of the way there.
Answer: You could get the matrix $A$ for the rotation and show that it's nonsingular. (This may be a little tricky; I remember doing a problem like this.  First move $(1,1,1)$ to the $z$-axis,  say, by a couple rotations.  Then rotate about the $z$-axis.  Then move back.)
Alternatively,  note that the kernel of such a rotation is necessarily $0$.  Therefore it's nonsingular and takes a basis to a basis.