How can I Set State of a Value that is inside some other Function

Question: I am Trying to Create a React App That Detects Age of Pictures Using Clarifai API .
I am Able to Console.Log Detected Age but I Want To Display The Age on My Webpage . Help me With Setting The AgeDectect State so I Can Use it on my Webpage
             //Value Displayed On Console
              //39

            //App.js Code That Console.Logs Age
              class App extends Component {
            constructor(){
             super();
             this.state = {
             input : '',
             imgURL : '',
             AgeDetect : ''
             }
            }

            onInputChange = (event) => {
              this.setState({input : event.target.value});
             }

            onClickEvent = () => {
              this.setState({imgURL : this.state.input})

              app.models.predict(Clarifai.DEMOGRAPHICS_MODEL ,
                                 this.state.input).then(
                      function(response) {
                           const A =response.outputs[0].data.regions[0].
                                  data.face.age_appearance.concepts[0].name
             //This Line of Code Displays Age on Console
                                  console.log(A);
                                  this.setState({AgeDetect : A});
                             },
          //Having Problem SettingState ,this.state.AgeDetect isnt
                                                //doing anything
         render(){
                return (<AgeDetection AgeDetect={this.state.AgeDetect}/>
                      )
                   }
             //AgeDetection.js file
      import React from 'react' ;
      const AgeDetection = ({AgeDetect}) => {
      return(

              {AgeDetect}

           );
         }
         export default AgeDetection;

Comment: You could propably just write a script that inserts the data into an html div that looks something like:   document.write('Data ' +  listOfData + '');
Answer: Sort your array that is returned by the value and set the first object or the whole array to your state and then you can use it in your app very easily.  Use an arrow function inside your predict then block to bind to the class.
class App extends Component {
  constructor() {
    super();
    this.state = {
      input: '',
      imgURL: '',
      AgeDetect: ''
    };
  }
  onInputChange = event => {
    this.setState({ input: event.target.value });
  };

  onClickEvent = () => {
    this.setState({ imgURL: this.state.input });
    app.models.predict(Clarifai.DEMOGRAPHICS_MODEL, this.state.input).then(
      response => {
        const A =
          response.outputs[0].data.regions[0].data.face.age_appearance
            .concepts[0].name;

        this.setState({ AgeDetect: A });
      },
      function(err) {
        // there was an error
      }
    );
  };

  render() {
    console.log(this.state);
    return (

        <Navigation />
        <Logo />
        <ImageLinkForm
          onInputChange={this.onInputChange}
          onClickEvent={this.onClickEvent}
        />
        <FaceRecognition imgURL={this.state.imgURL} />
        <AgeDetection AgeDetect={this.state.AgeDetect} />

    );
  }
}

export default App;

Comment: I Filtered My Console.Log to only Array[0].name (i.e : Age Displayed by 0th Element of Array ) . How Can I display that only Item on my Webpage
Comment: Not sure what you mean.  Sort the array and then set a value to the state.  If its just the name that you want then set just the name to a value in the state. After that you can use that state value anywhere in your component.  Lets say you set the value to `myValue` in your state.  Then you can do something like `{this.state.myValue}` anwhere in your components jsx
Comment: const A = response.outputs[0].data.regions[0].data.face.age_appearance.concepts[0].name //outputs a number (which is age i want to display ) . Then this.setState=({AgeDetect : A }) . This State returns me an empty value
Comment: If A has a valid output then setting that output the state should work.  I'm not sure what you are doing.  Here is a simple example of this working https://codesandbox.io/s/bitter-surf-rcdkc
Comment: Now how are you trying to display it in your page this part looks fine.
Comment: using this line of code in render function  . And in using these lines of code in AgeDectection.js file .. import React from 'react' ;

const AgeDetection = ({AgeDetect}) => {
    return(

         {AgeDetect}

    );
}
export default AgeDetection;
Comment: I'm not sure it may have something to do with the function your running it in but it doesn't seem likely.  Idk what to tell you just from what I can see it should be working.  Without being able to see your code and debug it myself its hard to see what the problem is.  Is imgURL being added to the state properly inside your onClickEvent?  You can maybe try using lower case letters and renaming things.
Comment: onClickEvent is working properly as my .then(function.....  console.logs the age , I think I am not able to setstate because the Function in which in I am setting state lies inside onClickEvent Function , here is link to full code and app :https://github.com/cyumair/smartbrain
Comment: Use an arrow function to bind the function to the class then you can set the state.  I edited my answer to show this.
Comment: Displays Age in the console , not on webpage ... And on 2nd click displays TypeError: this.setState is not a function
Comment: Its working for me I downloaded your code and it works fine.  I edited the answer again with what I have here just copy it and let me know.  I am console logging the state as well to show the state change.  I took the particles out though.


Documentation on how to create Debian init.d scripts

Question: I'm looking for some documentation/tutorial on how to create good init.d scripts for different services in Debian.
This article is nice introduction: http://www.debian-administration.org/articles/28 but I'm looking for more details.
Thanks!
Comment: Not very detailed but the debian policy manual has a section on what a "good" init script should contain. http://www.debian.org/doc/debian-policy/ch-opersys.html#s-sysvinit
Comment: @topdog this is nice, can you move you comment to an answer, so I could accept it ?
Answer: Not very detailed but the debian policy manual has a section on what a "good" init script should contain. http://www.debian.org/doc/debian-policy/ch-opersys.html#s-sysvinit


Update ComboBox with VBA and multiple WHERE condition SQL

Question: I try to update a combo box in a Form using VBA and SQL with multiple WHERE conditions. If I use only one condition it works, if I add others it will show no results
Private Sub Form_Load()
Dim strSQL As String
If Forms!frmLogin!txtLogin = "some text" Then
strSQL = "SELECT table.field1 FROM table WHERE table.field2=1; "
End If
With Forms!frmForm1!subfrmForm2!cboField
.RowSource = strSQL
.Requery
End With
End Sub

is working fine
strSQL = "SELECT table.field1 FROM table WHERE table.field2=1 And table.field2=2 And table.field2=3; "

is not working
Comment: The multiple conditions are not the issue. Please post your actual code with actual field names and parameters.
Answer: A field cannot hold three different values. Try with:
… where table.field2 in (1, 2, 3);


Viterbi Algorithm

Question: Can someone explain why

Is it because for the left argument, we would find the most likely sequence of states given observations and for the right argument, we would eventually find the most likely sequence of states amongst all sequences of states and observations. And because of this, we find the most likely sequence of states for both arguments so they are equal?
Comment: Please add some details about the meaning of the various symbols.
Comment: Viterbi Algorithm has been maturely used in digital communications for long, especially used for soft decoding of error-correction code, e.g. convolution code, trellis code, turbo code etc. You may easily find detailed explanation and examples in a typical textbook on digital communications or error-correction coding (or channel coding).
Answer: We can express that conditional probability as:
$$P(q|O,\lambda)=\frac{P(q,O|\lambda)}{P(O|\lambda)}$$
Since $P(O|\lambda)$ does not depend on $q$, the $\arg\max_q$ of this fraction will be the same as the $\arg\max_q$ of the numerator alone.

Viterbi Algorithm


Atualizar valores após alteração de dropdown

Question: Não sei quase nada de JavaScript mas consegui usar o Chosen para mudar minhas selects.
O que não estou conseguindo é usar o trigger que o site indica para atualizar o campo.
Quero que cada vez que clique no campo select e o dropdown list acione, ele atualize os valores.
Sei que o site diz para colocar $("#form_field").trigger("chosen:updated"); mas não sei onde, também não sei o que é o #form_field. É o id do formulário? Do campo? Tenho que criar algum evento com onselect ou onclick?
Código:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" type="text/css" href="chosen/chosen.css">

<title>Cadastro de Filmes</title>

Cadastro de Filmes

 <form action='' method='post' enctype='multipart/form-data' name='form1' id='form1'>

País
    <select name='pais[]' data-placeholder=' ' multiple class='chzn-select')
      <?php
        //LAÇO PARA BUSCAR TODOS OS DADOS DO BANCO MARCANDO AQUELES QUE ESTÃO SELECIONADOS
        $j=0;
        $tam=sizeof($pais_sel);
        for($i=0;$i<sizeof($pais);$i++){
         if($pais[$i]==$pais_sel[$j]){
          echo "<option value='$pais[$i]' selected>$pais[$i]</option>";
         if($j<$tam-1){
          $j++;
         }
        }else{
          echo "<option value='$pais[$i]'>$pais[$i]</option>";
         }
       }
       ?>
    </select>

    <input name='enviar' type='submit' id='enviar' value='Enviar' />

  </form>

Comment: Podes colocar o código que tens aqui?
Comment: Como coloco código aqui?
Comment: Pode [edit] a pergunta e colocar o código na pergunta. Se precisar de ajuda a formatar o código nós ajudamos.
Comment: Coloquei o código, mas nem todo apareceu.
Comment: Olá, amigos. Vou tentar ser mais claro.
Tenho um formulario de cadastro de filmes em que tem um campo país, por exemplo, para selecionar os países que co-produziram esse filme. Vou criar ao lado desse campo um botão novo para cadastrar novos países. O que preciso é que o campo país do primeiro formulário atualize com o novo país que acabei de cadastrar sem ter que dar um refresh na página, pois assim perderia os dados digitados nos outros campos. Creio que seja com a função update do chosen que estou usando, mas não sei usar direito.

Agradeço a atenção.
Comment: Tem erros aqui `` tem `);"` a mais. Tem algum erro na consola?
Comment: Olá, Sergio. Já notei o erro, foi de digitação, mas o problema não está nesse código. Quero saber o que tenho que acrescentar nele para atualizar a tag select com o chosen. Abraços
Comment: Seria ideal fazer um jsFiddle com o problema. Mas vou dar uma olhada. Se puder entretanto coloque o HTML renderizado, melhor para testar.
Comment: Olá, Sergio. Desculpe, não sei usar jsFiddle. Quer que coloque capturas da página?
Comment: Pode explicar o que falta aqui: __http://jsfiddle.net/rq7fkzpk/__
Comment: Conseguiste resolver este problema?
Answer: No seu caso:
$('.chzn-select').trigger("chosen:updated");

Agora,mudando de assunto
Agora seu problema em atualizar o pais, eu sugiro que você use o knockoutjs e "link" seu select a um array, após salvar o novo país via callback, volte a acrescente um ítem no array, e seu problema estará resolvido.
Comment: Ola, amigo. Não sei bem onde colocar esse código.
Answer:

Comment: Seria melhor voce editar sua resposta original acrescentando este trecho em vez de postar 2 pedaços.
Comment: Olá, Penihel. Agradeço pela resposta.
Como disse antes não entendo nada de Java, apenas copiei o que vi num fórum para fazer funcionar o chosen. O que quero, se possível, é que se eu atualizasse o banco de dados não fosse preciso dar um refresh na página do formulário para atualizar o campo. Infelizmente não entendi seu código. Abraços
Answer: Basta você criar uma função que atualiza o select Chosen, adicionando um novo option ao select. Fiz uma implementação no Fiddle do Sergio que mostra como funciona, aí basta você adaptar conforme queira: http://jsfiddle.net/rq7fkzpk/3/


bounded self-adjoint operator has non-empty spectrum

Question:
Show that the spectrum of a bounded self-adjoint linear operator on a
  complex Hilbert space $H\neq\{0\}$ is not empty.

If possible, let the spectrum $\sigma(T)=\emptyset$. So its resolvent set $\rho(T)$  equals $\mathbb{C}$. So for all $\lambda\in\mathbb{C}$ we have a $c>0$ such that $||T_\lambda(x)||\geq c||x||$ where $T_\lambda=T-\lambda I$. Dividing both sides by $||x||$ and taking supremum, we have $$||T_\lambda||\geq c \ \ \ \ \ \forall \ \ \ \lambda\in\mathbb{C}$$ which contradicts that $T$ is a bounded linear operator. So $\sigma(T)\neq\emptyset$. Is my proof correct? Any help is appreciated.
Comment: (1) Doesn't $c$ depend on $\lambda$?  (2) How does this contradict $T$ being a bounded operator?
Comment: Yes $c$ depends on $\lambda$, but since $\lambda$ takes value in all of $\mathbb{C}$, so $||T_\lambda||\geq c$ for all $\lambda\in\mathbb{C}$, so eventually unbounded on whole $\mathbb{C}$.
Comment: $c$ can for example be $0$ or just $1$, you will not get a contradiction in this way.
Comment: In fact, every bounded linear operator on a complex (!) Hilbert space has a non-empty spectrum. However, even a self adjoint one may have empty point spectrum.
Comment: @s.harp I have already written that $c>0$, so how can $c$ be zero I don't understand. Also why $c=1$ is not contradiction?
Comment: @am_11235..., didn't you be embarrassed that you have never used self-adjointness in your proof? In addition, why should lambda and $c$ be directly related? Why not there be an inverse relationship?
Comment: @thing, the line I have wrote above "..we have a $c>0$ s.t. $||T_\lambda(x)||\geq c||x||$.." holds only for bounded self-adjoint linear operators. It's a standard theorem for resolvent sets of bounded self-adjoint linear operators. That is, if the operator weren't bounded self-adjoint, I could never write that line.
Comment: Take a look at $T=\begin{pmatrix}1&0\\ 0 &-1\end{pmatrix}$, for this $T$ you have a bound $\|T_\lambda\|≥ c(\lambda)$, where here you may set $c(\lambda)=1$ for all $\lambda$, this is not contradiction to the boundedness of $T$.
Comment: A way to achieve the result is to consider the sub-algebra of $B(H)$ generated by $\Bbb1$ and $T$. Every element of this algebra except for $0$ is invertible, because $\sigma(p(T))=p(\sigma(T))=\emptyset$ for any polynomial $p$. Further it is a commutative algebra. Hence you have here a _normed field_ containing $\Bbb C$, Compare this with [the Gelfand-Mazur theorem](https://en.wikipedia.org/wiki/Gelfand–Mazur_theorem) to get a contradiction.
Comment: @am_11235...If $\lambda$ is a point of the resolvent set, then the statement $\|T_\lambda x\|\geq c\|x\|$ is true for any bounded operator (in fact, there is no even boundedness needed, linearity enough). Self-adjointness is needed here if this statement were applied in the opposite direction.
Answer: Let $M=\sup\limits_{\|x\|=1}(Ax,x)$, $m=\inf\limits_{\|x\|=1}(Ax,x)$ and let for definiteness $0\leq m\leq M$, then $\|A\|=\sup\limits_{\|x\|=1}|(Ax,x)|=M$. We show that $M$ is point of the spectrum. By the property of the supremum, there exists a sequence $x_n\in H$ such that $\|x_n\|=1$ and $(Ax_n,x_n)\to M$. Moreover, $\|Ax_n\|\leq\|A\|\|x_n\|=M$.
Further, $\|Ax_n-Mx_n\|^2=\|Ax_n\|^2-2M(Ax_n,x_n)+M^2\|x_n\|^2\leq 2M^2-2M(Ax_n,x_n)$. By $n\to\infty$ we have $\|Ax_n-Mx_n\|\to0$, so $M$ -- is the point of spectrum.
Other cases are treated similarly.
Comment: If $A$ happens to be a complex self adjoint operator. Can we show that either $m$ or $M$ is in the point spectrum.
Comment: You mean to show that these numbers are eigenvalues? This don't follow from anywhere unless the operator is compact.
Comment: The last equality in this sentence "Moreover, $\|Ax_n\| \leq \|A\| \|x_n\| = M$" does not follow without proof. What follows is  $\|Ax_n\| \leq \|A\| \|x_n\| = \|A\|$. But then the proof fails.
Comment: In this case everything is correct and follows from the equality $\|A\|=\sup\limits_{\|x\|=1}|(Ax,x)|=M$.
Comment: concur, if we suppose that $\|A\| = \sup_{\|x\|= 1} \langle Ax, x\rangle$ (which is true), then  I believe your proof holds. My proof of  $\|A\| = \sup_{\|x\|= 1} \langle Ax, x\rangle$ uses the fact that the spectrum is nonempty. John Conway's Functional Analysis book has a proof of this fact in Proposition 2.13 https://www.datascienceassn.org/sites/default/files/A%20Course%20in%20Functional%20Analysis%20-%20Conway.pdf. Conway's proof does not use the spectrum, it uses the parallelogram equality.
Answer: Suppose, for contradiction, that $(L-\lambda I)^{-1} \in \mathcal{L}(H)$ for every $\lambda \in \mathbb{C}$.
Then $f:\mathbb{C} \to \mathcal{L}(H)$ by $f(\lambda) = (\lambda I-L)^{-1}$ is a non-constant, analytic function on (all of) $\mathbb{C}$.
This function is uniformly bounded on $\{|\lambda|  \leq \|L\| + 1 \}$ because it is continuous on a compact set, and for $|\lambda| > \|L\| + 1$, we see $\|f(\lambda)\| = \|(L/\lambda - I)^{-1}\|/|\lambda| \leq  1$.
But a uniformly bounded analytic function on $\mathbb{C}$ is constant, which is a contradiction.


Create interface for custom Adapter and DataTable

Question: I am learning to use interfaces and there is a problem when using the generated dataset.
I created two similar tables in the database, then created two data adapters in the dataset based on these tables. Then I created classes that will describe the adapter, datable and necessary methods.
My DataSet
class DB
{
    public class Table1 {
        public DataSetTableAdapters.Table1TableAdapter adapter;
        public DataSet.Table1DataTable dataTable;

        public void Init()
        {
            adapter = new DataSetTableAdapters.Table1TableAdapter();
            dataTable = new DataSet.Table1DataTable();
        }
    }

    public class Table2
    {
        public DataSetTableAdapters.Table2TableAdapter adapter;
        public DataSet.Table2DataTable dataTable;

        public void Init()
        {
            adapter = new DataSetTableAdapters.Table2TableAdapter();
            dataTable = new DataSet.Table2DataTable();
        }
    }
}

I tried to implement, for example, like this
interface ISecondaryTable
{
    DataTable dataTable { get; set; }
    IDataAdapter adapter { get; set; }
}

And I tried many other options. But without results.
Does anyone know their common class?
Comment: See [smarter ways to use TableAdapter with base-class, interface or partial class](https://stackoverflow.com/questions/1847193/smarter-ways-to-use-tableadapter-with-base-class-interface-or-partial-class).
Answer: If you open the Object Browser in visual studio (Ctrl+Alt+J in 2017, used to be Ctrl-W,J as a chord) you can see more info about the tableadapters in your project:

All TableAdapters inherit from Component. They "HAVE-A" DataAdapter, they are not "IS-A" DataAdapter. For example, they look like this:
public class XTableAdapter: Component{

  private DataAdapter _da;

}

They do not look like this:
public class XTableAdapter: DataAdapter

All this said, I'm not sure why you want to treat them this way or encapsualte them along with the data table. Data is stored in the datatable, tableadapters push it between db and datatable. I don't think i've ever seen someone do, for example, a class that wraps a StreamWriter (thing that writes a file) and a String (the content of the file):
class CombinedFileContentAndWriter{

  StreamWriter sw = newStreamWriter(@"C:\temp\x.txt");

  string content = "Hello World";

  void DoIt(){
    sw.Write(content);
  }
}

It's not to say you can't, it's just weird. TableAdapters are supposed to be short-to-medium life things that are called upon to move data; they don't need pairing up inseparably from that data. One tableadapter can readwrite hundreds of different instances of a datatable. TableAdapters can be created and thrown away on demand and they don't need to remain paired with the data they downloaded in order to function. You can:
var dt = new XTableAdapter().GetDataByName();

//manipulate dt in a 30 minute operation

new XTableAdapter().Update(dt); //a different tableadapter sends the data back to the DB

There's not much point trying to find a generic way to refer to tableadapters, becawuse they are all customized exactly to a specific datatable, and have names of methods that are variable, and hence not overloadable:
SchoolDataset ds = new SchoolDataSet();
new StudentTableAdapter.FillByStudentId(ds, 123);
new StaffTableAdapter.FillByStaffLastName(ds, "Smith");
new ClassroomTableAdapter.FillByYearDesignation(ds, "First grade");

This is about as deep as you need to go with tableadapters and typed datatables/sets.
The question Karen linked to is a smart resource, for sure, but it's worth noting that the lead answer chose to create a generic way to address tableadapters so they could easily enroll each adapter in a transaction, whereas Microsoft intended that to enroll tableadapter operations in a transaction they should be executed inside a TransactionScope


Interactive decision trees

Question: I was wondering if there is a free tool to build a decision tree in interactive fashion like in SAS Enterprise Mining. I'm used to work with Weka. But nothing fits to my needs. I would like that before splitting every node, the program asks to user which attribute (maybe from a list of the "best" attributes) to choose. I saw that in SAS it is implemented. Should I write some code to get what I want?
Thanks
Answer: Try Orange Canvas, it will give you option to build interactive decision tree.
Answer: Try the examples under dendrogram.   You can make it as interactive as you want.
require(graphics); require(utils)

hc <- hclust(dist(USArrests), "ave")
(dend1 <- as.dendrogram(hc)) # "print()" method
str(dend1)          # "str()" method
str(dend1, max = 2) # only the first two sub-levels

op <- par(mfrow= c(2,2), mar = c(5,2,1,4))
plot(dend1)
## "triangle" type and show inner nodes:
plot(dend1, nodePar=list(pch = c(1,NA), cex=0.8, lab.cex = 0.8),
      type = "t", center=TRUE)
plot(dend1, edgePar=list(col = 1:2, lty = 2:3),
      dLeaf=1, edge.root = TRUE)
plot(dend1, nodePar=list(pch = 2:1,cex=.4*2:1, col = 2:3),
      horiz=TRUE)

Edit 1 ====================================
The interactivity depends on what you want to do.   It all comes down to the structure of the data that goes to plot.    To make it easier to see what's going on, I'll only use the first 3 lines of data from the above example:
#Use only the first 3 lines from USArrests
(df <- USArrests[1:3,])

#Perform the hc analysis
(hcdf <- hclust(dist(df), "ave"))

#Plot the results
plot(hcdf)

#Look at the names of hcdf
names(hcdf)

#Look at the structure of hcdf
dput(hcdf)

The next segment is the output of the above dput statement.   This structure tells plot how to draw the tree.
structure(list(merge = structure(c(-1L, -3L, -2L, 1L), .Dim = c(2L,
2L)), height = c(37.1770090243957, 54.8004107236398), order = c(3L,
1L, 2L), labels = c("Alabama", "Alaska", "Arizona"), method = "average",
   call = hclust(d = dist(df), method = "ave"), dist.method = "euclidean"),
.Names = c("merge", "height", "order", "labels", "method", "call", "dist.method"),
class = "hclust")

You can easily change the data and see what plot does.  Just copy/paste the structure statement from your screen and assign it to a new variable, make your changes, and plot it.
newvar <- structure(list(merge = structure(c(-1L, -3L, -2L, 1L), .Dim = c(2L, 2L)), height = c(37.1770090243957, 54.8004107236398), order = c(3L, 1L, 2L), labels = c("Alabama", "Alaska", "Arizona"), method = "average",     call = hclust(d = dist(df), method = "ave"), dist.method = "euclidean"), .Names = c("merge", "height", "order", "labels", "method", "call", "dist.method"), class = "hclust")

plot(newvar)

As far as making the clustering more interactive, you'll have to explore the different methods and determine what you want to do.
http://cran.cnr.berkeley.edu/web/views/Cluster.html
http://wiki.math.yorku.ca/index.php/R:_Cluster_analysis
http://www.statmethods.net/advstats/cluster.html
http://www.statmethods.net/advstats/cart.html
Comment: This is a nice example showing how little code is required in R to visualize decision trees in different ways. But, it is not clear to me how this would be made "...as interactive as you want". For the sake of people like me who are not strong R coders, can you say more about how the interactivity is accomplished?
Comment: @Josh, I updated my answer above.

Interactive decision trees


Confirming proofs of properties of preimages

Question: I do not think that I made terrible mistakes, nevertheless a conforming word would be good for me. Thank you!

Let $f\colon X\to Y$ be a map and $(B_i)_{i\in I}$ a family of subsets of $Y$ ($I$ is any index set). Overmore consider any $A,B\subset Y$. Show:
    \begin{align}
&(1)~f^{-1}(\bigcup_{i\in I}B_i)=\bigcup_{i\in I}f^{-1}(B_i)\\
&(2)~f^{-1}(\bigcap_{i\in I} B_i)=\bigcap_{i\in I}f^{-1}(B_i)\\
&(3)~(f^{-1}(A))^C=f^{-1}(A^C)\\
&(4)~f^{-1}(A\setminus B)=f^{-1}(A)\setminus f^{-1}(B)\\
&(5)~f^{-1}(A\Delta B)=f^{-1}(A)\Delta f^{-1}(B)
\end{align}

My proofs:
(1) "$\subset$":
\begin{align}
x\in f^{-1}(\bigcup_{i\in I}B_i) &\Rightarrow\exists~j\in I: f(x)\in B_j\\&\Rightarrow x\in f^{-1}(B_j)\\&\Rightarrow x\in\bigcup_{i\in I}f^{-1}(B_i)
\end{align}
"$\supset$":
\begin{align}
x\in\bigcup_{i\in I}f^{-1}(B_i) &\Rightarrow\exists j\in I: x\in f^{-1}(B_j)\\&\Rightarrow f(x)\in B_j\\&\Rightarrow f(x)\in\bigcup_{i\in I}B_i\\&\Rightarrow x\in f^{-1}(\bigcup_{i\in I}B_i)~~~\Box\end{align}
(2) "$\subset$":
\begin{align}
x\in f^{-1}(\bigcap_{i\in I} &\Rightarrow f(x)\in\bigcap_{i\in I}B_i\\&\Rightarrow f(x)\in B_i~\forall~i\in I\\&\Rightarrow x\in f^{-1}(B)~\forall~i\in I\\&\Rightarrow x\in\bigcap_{i\in I}f^{-1}(B_i)
\end{align}
"$\supset$":
\begin{align}
x\in\bigcap_{i\in I}f^{-1}(B_i) &\Rightarrow x\in f^{-1}(B_i)~\forall~i\in I\\&\Rightarrow f(x)\in B_i~\forall~i\in I\\&\Rightarrow f(x)\in\bigcap_{i\in I}B_i\\&\Rightarrow x\in f^{-1}(\bigcap_{i\in I}B_i)~~~\Box
\end{align}
(3) "$\subset$":
\begin{align}
x\in (f^{-1}(A))^C=X\setminus f^{-1}(A) &\Rightarrow x\in X\wedge x\notin f^{-1}(A)\\&\Rightarrow x\in X\wedge f(x)\notin A\\&\Rightarrow x\in X\wedge T(x)\in A^C\\&\Rightarrow x\in X\wedge x\in f^{-1}(A^C)\\&\Rightarrow x\in X\cap f^{-1}(A^C)=f^{-1}(A^C)
\end{align}
"$\supset$":
\begin{align}
x\in f^{-1}(A^C)=Y\ &\Rightarrow f(x)\in A^C=Y\setminus A\\&\Rightarrow f(x)\in Y\wedge f(x)\notin A\\&\Rightarrow x\in f^{-1}(Y)=X\wedge x\notin f^{-1}(A)\\&\Rightarrow x\in X\setminus f^{-1}(A)=(f^{-1}(A))^C~~~\Box
\end{align}
(4) "$\subset$":
\begin{align}
x\in f^{-1}(A\setminus B) &\Rightarrow f(x)\in A\wedge f(x)\notin B\\&\Rightarrow x\in f^{-1}(A)\wedge x\notin f^{-1}(B)\\&\Rightarrow x\in f^{-1}(A)\setminus f^{-1}(B)
\end{align}
"$\supset$":
\begin{align}
x\in f^{-1}(A)\setminus f^{-1}(B) &\Rightarrow x\in f^{-1}(A)\wedge f(x)\notin f^{-1}(B)\\&\Rightarrow f(x)\in A\wedge f(x)\notin B\\&\Rightarrow f(x)\in A\setminus B\\&\Rightarrow x\in f^{-1}(A\setminus B)~~~\Box
\end{align}
(5) "$\subset$":
\begin{align}
x\in f^{-1}(A\Delta B) &\Rightarrow f(x)\in (A\setminus B)\cup (B\setminus A)\\&\Rightarrow f(x)\in (A\setminus B)\vee f(x)\in B\setminus A
\end{align}
I. $f(x)\in A\setminus B$:
\begin{align}
&\Rightarrow f(x)\in A\wedge f(x)\notin B\\&\Rightarrow x\in f^{-1}(A)\wedge x\notin f^{-1}(B)\\&\Rightarrow x\in f^{-1}(A)\setminus f^{-1}(B)\\&\Rightarrow x\in (f^{-1}(A)\setminus f^{-1}(B))\cup (f^{-1}(B)\setminus f^{-1}(A))=f^{-1}(A)\Delta f^{-1}(B)
\end{align}
II. $f(x)\in B\setminus A$:
\begin{align}
&\Rightarrow f(x)\in B\wedge f(x)\notin A\\&\Rightarrow x\in f^{-1}(B)\setminus f^{-1}(A)\\&\Rightarrow x\in f^{-1}(A)\Delta f^{-1}(B)
\end{align}
"$\supset$":
\begin{align}
x\in f^{-1}(A)\Delta f^{-1}(B) &\Rightarrow x\in (f^{-1}(A)\setminus f^{-1}(B))\vee (f^{-1}(B)\setminus f^{-1}(A))
\end{align}
I. $x\in f^{-1}(A)\setminus f^{-1}(B)$:
\begin{align}
&\Rightarrow x\in f^{-1}(A)\wedge x\notin f^{-1}(B)\\&\Rightarrow f(x)\in A\wedge f(x)\notin B\\&\Rightarrow f(x)\in A\setminus B\\&\Rightarrow x\in f^{-1}(A\setminus B)\cup f^{-1}(B\setminus A)\overset{\text{with}}{\underset{\text{(1)}}{=}}f^{-1}((A\setminus B)\cup (B\setminus A))=f^{-1}(A\Delta B)
\end{align}
II. $x\in f^{-1}(B)\setminus f^{-1}(A)$:
\begin{align}
&\Rightarrow x\in f^{-1}(B)\wedge x\notin f^{-1}(A)\\&\Rightarrow f(x)\in B\wedge f(x)\notin A\\&\Rightarrow f(x)\in B\setminus A\\&\Rightarrow x\in f^{-1}(B\setminus A)\\&\Rightarrow x\in f^{-1}(B\setminus A)\cup f^{-1}(A\setminus B)\overset{\text{with}}{\underset{\text{(1)}}{=}}f^{-1}((B\setminus A)\cup (A\setminus B))=f^{-1}(A\Delta B)~~~\Box
\end{align}
Comment: You can find links to some other posts about (som of) these identities [here](http://math.stackexchange.com/questions/359693/overview-of-basic-results-about-images-and-preimages).
Answer: Nothing blatantly wrong that I can see (apart from a few typos, like missing "$B_i)$" in the $\subset$ direction of part 2, "$T(x)$" in the $\subset$ direction of part 3, and a random "$=Y$" in the $\supset$ direction of part 3), but you're making it a bit too complicated in your proofs of the last two. Indeed, $$f^{-1}(A\setminus B)=f^{-1}(A\cap B^c)\overset{(2)}{=}f^{-1}(A)\cap f^{-1}(B^c)\overset{(3)}{=}f^{-1}(A)\cap(f^{-1}(B))^c=f^{-1}(A)\setminus f^{-1}(B),$$ and $$\begin{align}f^{-1}(A\triangle B) &= f^{-1}\bigl((A\setminus B)\cup(B\setminus A)\bigr)\\ &\overset{(1)}{=}f^{-1}(A\setminus B)\cup f^{-1}(B\setminus A)\\ &\overset{(4)}{=}\bigl(f^{-1}(A)\setminus f^{-1}(B)\bigr)\cup\bigl(f^{-1}(B)\setminus f^{-1}(A)\bigr)\\ &= f^{-1}(A)\triangle f^{-1}(B),\end{align}$$ with no need for element-chasing, double-inclusion, or breaking into cases.
Also, you could do parts 1 through 3 as $\iff$ all the way, rather than a double-inclusion proof.
Comment: You are very welcome. Kudos and +1 for showing your work and your thoughts, by the way.


Python 2.7 OS X installer, mod_wsgi and Apache

Question: The Python 2.6 OS X installer didn't support 64 bit architectures, necessitating some chicanery to get mod_wsgi, Python 2.6 and Apache2 working on 10.5.
However, the Python 2.7 OS X installer does build a 'fat' Python install, so I'd like to know if anyone's tried this setup before I ditch my current Python 2.6 local setup, rebuild mod_wsgi with Python 2.7 and restore 64 bit Apache2? Given the cautions, that is (although I don't use IDLE).
Comment: I might add that this is to test Django apps locally with mod_wsgi.
Comment: I found this in the mod_wsgi list archives, which is slightly helpful: http://email@example.com/msg04854.html
Answer: How to solve your problem:

upgrade OS X to 10.6
install macports and install any python distribution you like (you can have multiple installed)

PS. I use Python 2.7 with mod_wsgi on Apache (64bit)
Comment: @Dave I am aware about HomeBrew but I do not trust it for several reasons: the homepage is nice but is missing critical project information, like community links and bug tracker. Also on github the project is not migrated to an organization (multiple contributors). So I don't want to reply on one man maintaining an OS-like project.
Comment: Thanks Sorin - is that a local testing setup? Snow Leopard and Python 2.7 are inevitable (although Q was Oct 2010 and I *still* have one machine on 10.4!), but I tried and didn't like Macports - I'll probably use Homebrew (since I sometimes install to /usr/local already) http://www.engineyard.com/blog/2010/homebrew-os-xs-missing-package-manager/


Is modulation used as a tool to create climax?

Question: And the further you modulate away from the tonic key, the more tension you create?
I've been looking over both of Rachmanioff's op. 33 and 39 etude tableau and notice regular climax in his music.
So far I have judged his climax section to used:
-Many accidentals(modulation)
- melodic units are usually repeated and sometimes shortened when approaching climax
- usually approaches the high end of the piano range
Comment: Do you mean "And the further you modulate away from the **home** key"?
Answer:
the further you modulate away from the tonic key, the more tension
  you create?

If you mean "tonic key" as in Sonata form https://en.wikipedia.org/wiki/Sonata_form
then I don't know, I tried to listen to some of those examples, and I'd say, if such a phenomenon exists, then it fades away within seconds and isn't particularly strong. I hadn't even heard such a term before. What I understand from Sonata form, the keys that are modulated to and form aren't arbitrary like you could pick any number of semitones 1-11, the keys have something "structural" to do with each other.
Talking about modulations, the terms that are used are tonic note, sometimes tonic chord.
https://en.wikipedia.org/wiki/Tonic_(music)
Modulation means that the tonic note moves. When a modulation happens, you forget the old tonic within a few seconds, so whatever tension or other feelings were caused by the modulation, it's not permanent. The feelings fade away quite quickly.
Accidentals don't mean modulation, at least a permanent one. Individual notes with accidentals only mean very short temporary changes to the scale-, and usually that means some kind of a change in the prevailing mode, i.e. harmony around the tonic. Or a note with an accidental could be simply a chromatic passing tone or other embellishment that is so brief that it doesn't change the feeling of mode even temporarily. A key signature change might mean a modulation, because key signature changes are larger events that can be used between sections, or at least nobody bothers using them for mode changes lasting only a couple of bars. In some jazz tunes that constantly jump between keys and modes, the notation might just use individual accidentals all the way through. Or a player might improvise her own mode changes, which of course isn't notated at all.
Comment: "There is no such thing as "tonic key" " - you might want to change how this is formulated. In sonata form there IS a "tonic key". It just has another function
Comment: @NickQuant Thanks, I had never heard of that. I should have checked it first. Anyway, I tried to listen to Beethoven's Waldstein sonata, but I couldn't retain a sense of previous tonics over long periods of time.


Evaluate $\lim\limits_{(x,y)\to(0,0)}\frac{|x|}{|x|+|y|}$

Question: Evaluate $\lim\limits_{(x,y)\to(0,0)}\dfrac{|x|}{|x|+|y|}$
The limit does not exist.
From the x-axis (y=0), we have $\lim\limits_{x\to 0} \dfrac{|x|}{|x|+0}=1$
From the y-axis (x=0), we have $\lim\limits_{y\to 0} \dfrac{0}{0+|y|}=0$
Therefore, since $1\neq 0$, the limit does not exist.

But what if we apply the squeeze thereom.
We know that $\dfrac{|x|}{|x|+|y|}\leq \dfrac{|x|}{|x|}=1$, and so the limit is $1$. Why is this incorrect? What are the circumstances that I can use the squeeze thereom, and why isn't this one of them
Comment: You have not squeezed inasmuch as the left-hand side is not bounded below.
Comment: you haven't "sqeezed" your function, unless you can also find a lower bound, and then show that they are equal to one another.
Comment: And from the diagonal "axis" (where $x=y \equiv z$):  $\lim\limits_{z\to0} {|z| \over |z| + |z|} = 1/2$.
Answer: No, the Squeeze Theorem just gives you in this case that the limit, if it exists, can be at most 1. You really want to use the Squeeze Theorem when you can bound it from both sides and the limit on each side is the same. Otherwise, you really aren't squeezing anything but merely setting 'good boundaries' for your misbehaved limit.


More preliminaries of the Martingale Convergence Theorem

Question:

Really struggling with this lemma.
Not sure about the general structure of the proof. Why have we chosen g to be orthogonal to all functions of the form 4.3.1?
Why should $G(\lambda)=0$, does it have something to do with the inner product on $L^2$.
What gives us the justification of the analytic extension and why would we want to do this?
I cannot find any justification in my book that if $G=0$ on $\mathbb{R^n}$ that as $G$ is analytic $G$ would be equal to $0$ on $\mathbb{C^n}$.
Next I cannot understand why we are using the fourier inversion.
Finally I cannot see how the last paragraph justifies that the subset is dense. Why is $g=0$ significant? I think this may be help me understand the general structure of the proof.
Answer: The proof is based on this standard fact from functional analysis:

Theorem. Let $H$ be a Hilbert space and let $E \subset H$ be a linear subspace.  Let $E^\perp := \{ x \in H : \langle x,y \rangle = 0 \text{ for all } y \in E\}$.  Then $E$ is dense in $H$ iff $E^\perp = \{0\}$.

So for this proof, $H = L^2(\mathcal{F}_T, P)$ and $E$ is the linear span described in the statement.  We choose an arbitrary $g \in E^\perp$ and show we must have $g=0$.  This will show that $E^\perp = \{0\}$ and hence $E$ is dense.
Next verify that if $h \in L^2([0,T])$ is a step function, having a jump of size $\lambda_i$ at the point $t_i$, $i=1,\dots,n$, and constant in between, then $\int_0^T h(t)\,dB_t = \lambda_1 B_{t_1}  + \dots + \lambda_n B_{t_n}$.  (This is immediate from the definition of the stochastic integral.)  Then $$\exp\left(\int_0^T h(t)\,dB_t - \frac{1}{2} \int_0^t h(t)^2\,dt\right) = \exp\left(-\frac{1}{2} \int_0^T h(t)^2\,dt\right) \cdot \exp(\lambda_1 B_{t_1} + \dots + \lambda_n B_{t_n})$$
which is a scalar multiple of $\exp(\lambda_1 B_{t_1} + \dots + \lambda_n B_{t_n})$, which is therefore in $E$.  Since $g$ was assumed to be in $E^\perp$, we have
$$\int_\Omega \exp(\lambda_1 B_{t_1} + \dots + \lambda_n B_{t_n}) g\,dP = 0$$
which is (4.3.2).
The fact that $G$ has a complex analytic extension should be explained in Exercise 2.8(b).  But rather than using facts about real analytic functions, it is probably easier to verify explicitly that $G$ is complex differentiable by differentiating under the integral sign (hopefully familiar from a measure theory course).  You will need some estimates to justify this which are presumably the same ones from Exercise 2.8(b).  Alternatively, use differentiation under the integral sign to show that $G$ satisfies the Cauchy-Riemann equations.
Now it is a standard fact from complex analysis that if $G$ is analytic on $\mathbb{C}^n$ and vanishes on $\mathbb{R}^n$, then $G=0$.  When $n=1$ you can say more: if the zeros of $G$ have a limit point (which $\mathbb{R}^1$ certainly does) then $G=0$.  You can find this as Theorem 6.9 in Bak and Newman.  Our statement for general $n$ can then be proved by induction.  Suppose it holds for $n-1$,  and suppose $G : \mathbb{C}^n \to \mathbb{C}$ is analytic and vanishes on $\mathbb{R}^n$.  Fix $w \in \mathbb{R}$ and consider the function $G_w(z_1, \dots, z_{n-1}) = G(z_1, \dots, z_{n-1}, w)$.  Then $G_w : \mathbb{C}^{n-1} \to \mathbb{C}$ is analytic and vanishes on $\mathbb{R}^{n-1}$ so by the inductive hypothesis $G_w = 0$.  Next fix $z_1, \dots, z_{n-1} \in \mathbb{C}$ and let $f(u) = G(z_1, \dots, z_{n-1}, u)$, so that $f : \mathbb{C} \to \mathbb{C}$ is analytic.  For $u \in \mathbb{R}$ we have $f(u) = G_u(z_1, \dots, z_{n-1}) = 0$, so $f$ vanishes on $\mathbb{R}$ and by the $n=1$ case we have $f = 0$.  Thus $G(z_1, \dots, z_{n-1}, u) = 0$ for all $u \in \mathbb{C}$.  But $z_1, \dots, z_{n-1}$ were arbitrary so we must have $G=0$.
Finally, why do we use the Fourier transform?  Well, because it works.  Once we have the complex extension of $G$, we know that for any $y_1, \dots, y_n$ we get $\int_\Omega \exp\left(i y_1 B_{t_1} + \dots + i y_n B_{t_n}\right) g\,dP = 0$.  But, roughly speaking, the Fourier inversion formula says that any reasonable function $\phi$ of $B_{t_1}, \dots, B_{t_n}$ is an "infinite linear combination" of functions of the form $\exp(\lambda_1 B_{t_1} + \dots + \lambda_n B_{t_n})$ (think of the integral in the inverse Fourier transform as a sort of infinite sum; in fact, it really is the limit of finite sums).  So it is not surprising that we then get $\int_\Omega \phi\left(B_{t_1},\dots, B_{t_n}\right) g\,dP = 0$.
On the other hand, it had previously been shown (presumably Lemma 4.3.1) that the subspace $F$ spanned by functions of the form $\phi\left(B_{t_1},\dots, B_{t_n}\right)$, where $n$ ranges over all integers, was dense in $L^2(\mathcal{F}_T, P)$, and the conclusion of my previous paragraph says that $g \in F^\perp$.  By the other direction of the theorem at the top of this answer, we conclude that $g=0$, which is what we needed.
Comment: @1234: Actually I was mistaken - the statement I gave before is not actually correct for $n > 1$.  But the statement we need is still true.  See my edit.
Comment: "Now it is a standard fact from complex analysis that if the zeros of an analytic function have a limit point, the function must be identically zero. Certainly $\mathbb{R^n}⊂\mathbb{C^n}$ has a limit point (in fact every point of $\mathbb{R^n}$ has a limit point) so we conclude that $G=0$ on $\mathbb{C^n}$." Where is this result? I cannot find it in my text on complex analysis (Bak and Newman)


Writing values to already existing worksheet

Question: I already have an Excel worksheet. How do I write some values to that worksheet from my VB application?
Answer: Note: This is an Excel-VBA solution; however, someone has changed the question tag to VB.NET since.
<code>Sub MySub()
    Set xlApp = CreateObject("Excel.application")
    xlApp.Visible = True
    Set xlBook = xlApp.Workbooks.Open("C:\MyBook.xls")
    Set xlSheet = xlBook.ActiveSheet

    xlSheet.Range("A1").Value = "Hello, world!"
End Sub
</code>
Comment: That is the full working code! Just tested it myself. Put it in a `Sub`. Heck, I'll spell it out for you. Note that if you are using `Option Explicit` (which is good practice) then you will have to declare the objects first.


How to write a sequence

Question: I want to write this sequence in R,
1, 1/3, 1/5,1/7,...,1/21
I have:
x <- seq(1,21,by=2)
y<- 1/x
print(y)

I got:
1.00000000 0.33333333 0.20000000 0.14285714 0.11111111 0.09090909 0.07692308 0.06666667 0.05882353 0.05263158 0.04761905

which is the same as the sequence,
Now is there another way to get the numbers as fraction without using the fraction package?
Comment: `paste0("1/",seq(1, 21, 2))`
Answer: Try using the as.fractions in the MASS package
 x <- seq(1,21,by=2)
  y<- 1/x
  MASS::as.fractions(y)
  [1]    1  1/3  1/5  1/7  1/9 1/11 1/13 1/15 1/17 1/19 1/21

Comment: Didnt know this function - quite kool : `MASS::as.fractions(y) + sqrt(2)` even works
Comment: That only does approximations.. Maybe if there is a surd fuction I do not know about that yet
Comment: ive not properly looked, but probably the print  (and fractions) method  - the fractions that we see - are approx. But the underlying numeric, *I think* will be the same (`all.equal(MASS::as.fractions(y)  + sqrt(2), y + sqrt(2), check.attributes=FALSE)`) (could wrap fractions in as.numeric to remove message)

How to write a sequence


sort() with a cascade it works but using map() doesn't work and printing list show same thing again

Question: In dart when I use sort() with a cascade it works but using map() doesn't work and the printing list returns the same thing again. Why?
 void main(){
        list<dynamic> list= [4,1,3,2]..sort();
        print(list);  //outputs [1,2,3,4]

        var list1= [4,1,3,2]..map((e)=>e+1);
        print(list1); //again outputs [1,2,3,4]
       }

Comment: *"In dart when I use sort() with a cascade it works but using map() dosen't work and printing list returns same thing again.Why?"* - because `sort` modifies the list itself, while `map` creates an new `Iterable`
Comment: Ok thank you,so is it like it tries to return a new object but due to cascade it returns the receiver itself?
Comment: You can use dart.pad to test your code. If you do so, you´ll see that the outputs are not identical! ( list = [1, 2, 3, 4] whereas list1 = [4, 1, 3, 2])
Comment: sorry,yes it outputs still same thing though [4,1,3,2] ,I hope you get what I am trying to say
Answer: If you consult the dart documentation, you can find the following example of how the sort() method works in detail. In your example you used the cascading operator to discard the result of sort(), which is void, and instead return the original receiver, the list, which you then conveniently store back into the variable. Consequently, the first output prints the sorted list, as expected.
The map method, as you can read here, will return a new lazy Iterable with elements that are created by calling toElement on each element of this Iterable in iteration order. This result is now discarded by the cascade operator and instead the original receiver will be returned.
And here is the problem, whereas the sort method modified the original list, which then is returned by the cascade operator (as desired), the map method will create a new Iterbale, which due to the cascade operator will be discarded. Instead the original receiver (the un-modified list) will be returned, it still prints the un-modified list.


Defined variables losing value after using drupal_bootstrap

Question: We're trying to use drupal_bootstrap to be able to delete a drupal user when a user is inactivated in our internal (non-Drupal) system. It appears that all defined variables set prior to the bootstrap call are losing value as soon as the call is made. Here's the weird part: it only happens on our live server - everything runs perfectly on dev.
Here's the code; I've stuck a whole bunch of error_log statements in so I can see exactly where the variables are disappearing.
<code>//email address is empty; delete institute record - but not if it's user 0 or 1
if($institute_uid > 1) {
  error_log("institute uid right before bootstrapping drupal: $institute_uid");
  //previous line prints and the $institute_uid is correct

  //set the working directory to your Drupal root - necessary for delete user function
  chdir('/home/www/institute');

  //require the bootstrap include
  require_once './includes/bootstrap.inc';

  //Load Drupal
  drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
  //(loads everything, but doesn't render anything)

  error_log("institute uid immediately after bootstrap = $institute_uid");
  //previous line prints but the $institute_uid value is blank

  $test = "123";
  error_log("test = $test");
  //previous line prints and $test value is correct

  user_delete(array(), $institute_uid); //$vuid = institute_uid from new_people
  $body .= '<br />Institute account deleted';

}
</code>
Any ideas? We're new to the bootstrap stuff; maybe there's a server setting, or line in a .ini file somewhere in Drupal that needs to change on our live server?
Answer: When "register_globals" is enabled, Drupal unsets all global variables before boostrap.
Disable the "register_globals" option in your live server.
Comment: That's correct. Another solution, if you don't want do disable register_globals (e.g. used in your old PHP3 site) is to use $_REQUEST['institute_uid'] instead of $institute_uid.


Comparing noisy data sequences to estimate the likelihood of them being produced by different instances of an identical Markov process

Question: (Prompted to some extent by the answers already given by Shane and Srikant, I've rewritten this to try to clarify what I'm getting at, if only to myself.)
Suppose we have several similar systems, each with behaviour that approximates a continuous time Markov process. That is, there are some number of discrete states the system can be in and associated probabilities of transitioning from one state to another at any instant, depending solely on the current state. For now, consider the processes to be stationary, ie the transition probabilities do not change over time, and unaffected by seasonality or other external considerations.
Unfortunately, we cannot measure the state of any system directly, but have instead to measure a proxy quantity, which varies with state but is not discrete and is subject to various sources of noise and error.
The principal question is this:
Q1: Given two data sequences produced independently from two such systems, how can we decide whether the underlying Markov processes are the same?
Now, it may be that the best way to approach this is as two separate problems:

Convert the imperfect proxy sequence into an idealised time series of (categorical) states
Determine whether the state sequences correspond

On the other hand, such a separation might involve discarding some information from the data in step 1 (eg, about its variability) that would be useful in step 2. Which leads to:
Q2: Does it make sense to decompose the problem in this way or is it better to compare the proxy data directly?
If such a decomposition does make sense, that opens up a whole other issue about how to do the idealisation, but that's definitely a question for another day.
Shane, below, mentions goodness-of-fit and distributional tests such as Anderson-Darling, and that seems like a promising approach. But I'd like to check I'm understanding the idea correctly.
Given sufficient samples in a sequence, we would expect the proportion of time spent in each state to tend to the stationary distribution. So one could test the distributions of occupancies in the two sequences for similarity. (I have the vague sense a two-sample Kolmogorov-Smirnov might suit for this, but please set me right about that.)
The thing is, I'm not sure how good this can be as evidence. If the distributions are very different, that seems like a reasonable strike against the underlying processes being the same, but what if they're very similar? Can we draw conclusions in that direction?
Q3: Does a good fit of occupancy distributions tell us anything useful?
It seems like there could be an infinite number of processes that will tend to the same stationary distribution. I think this is very unlikely in practice, and that different systems will tend to have distinctly different behaviour, but still it's worth considering.
Finally, we will often have a model of the underlying process that we are looking for, although it may not be perfect. So we could compare each sequence with the expected behaviour of the model, instead of with each other. We may also have more than two sequences to test.
Q4: Is it better to compare multiple sequences to a single model, even an approximate one, or are we better off comparing data directly?
Comment: I was interested in using a Kolmogorov-Smirnov test for a similar purpose.  As far as I can tell, there is no variant on a K-S test that's appropriate for comparing time series.  (I am very, very far from being an expert on such things, however.)
Answer: You can perhaps use a hidden markov model (HMM). I know that there is a R package that estimates HMMs but cannot recall its name right now.
Comment: You can use the HMM package: http://cran.r-project.org/web/packages/HMM/index.html.
Answer: A few thoughts:

Can you not just use a goodness-of-fit test?  Choose a distribution and compare both samples.  Or use a qqplot.  You may want to do this with returns (i.e. changes) instead of the original series, since this is often easier to model.  There are also relative distribution functions (see, for instance, the reldist package).
You could look at whether the two series are cointegrated (use the Johansen test).  This is available in the urca package (and related book).
There many multivariate time series models such as VAR that could be applied to model the dependencies (see the vars package).
You could trying using a copula, which is used for dependence modeling, and is available in the copula package.

If the noise is serious concern, then try using a filter on the data before analyzing it.
Comment: Thanks for the suggestions. I'll have to look into them in more detail, but one thing that occurs immediately is that the series do not share a time frame, so cointegration and VAR are probably not relevant. I've edited the question to try to clarify this, apologies for being misleading before. Noise filtering brings up some other issues, but those are for another question and another day!
Comment: Ok.  Well, looking at the distributions may be the way to go.  You can also try a distribution test such as Anderson–Darling (as in this question: http://stats.stackexchange.com/questions/1645/appropriate-normality-tests-for-small-samples).


How to indent XML text and preserve line breaking in C#?

Question: My goal is to indent manually written XML document programmatically using C# code.
At the moment I'm indenting it using XmlWriter as specified here.
But I still have two problems:

After indentation all line breaks disappear (I do want to preserve artificial line break inside the XML document).
Xml elements which has both text node and sub-elements - are merged to one line. For example:

<element>text
  <subElement></subElement>
</element>

will be indented to this:

<element>text<subElement></subElement></element>

I guess both of the above problems are actually the same problem.
Comment: Why do you want this? Just to look "good"? There's no semantic reason for such formatting. Line breaks are interpreted as whitespace.
Comment: To preserve the XML readability: It is written by humans and also read by them. Sometimes it is much easier to understand XML with line breaks.
Comment: You would be better giving the humans a pretty-printer and leaving the actual document alone.
Answer: Inserting whitespace into an XML document, in general, risks changing the semantics of the document.
However, if you really want to indent lines by XML nesting (with no reflow), the serializers that come with most XML parsers can be configured to do it.
Or you could put the document through an XSLT stylesheet which is configured to do indentation.
Either of those approaches would be simpler than writing it yourself, and unless I've missed something in your problem description they should do what you need.
Answer: If you are using XmlWriter you can create a new XmlWriterSettings object to pass your settings. Here you can set the indentation, NewLineHandling, and many other useful things.
        XmlWriterSettings set = new XmlWriterSettings();
        set.NamespaceHandling = NamespaceHandling.OmitDuplicates;
        set.OmitXmlDeclaration = true;
        set.DoNotEscapeUriAttributes = false;
        set.Indent = true;
        set.NewLineChars = "\n";
        set.IndentChars = "\t";
        set.NewLineHandling = NewLineHandling.None;
        // Make any other changes... up to you

        XmlWriter writer = XmlWriter.Create(Console.Out, set);
        // You can use your writer with the passed settings


Sort a View by whether content has a user reference

Question: I have a view that lists all users that have flagged a piece of content (using the Flag module). Also, for that content, I have certain user references attached (using the References module).
So:

User can flag a piece of content.
Content can reference user.

I want to be able to have the view show all users that have flagged the piece of content (I have that working correctly), but also show them in the order of:

Users that are referenced by the piece of content.
Username, alphabetically.

I think I just have a mental block from getting this done. Any pointers or advice to help me get this kind of sort working? Note: I also need to have everything done in one display, since it needs to be sortable and filterable.
Answer: I have an idea!  Use your current block to limit the list to users who are referenced in the node, and then add an ATTACHMENT to the block listing users who have flagged it who weren't referenced in the node.
That way you can separate them our and sort independently, but they'll still appear as if they're displayed in the same view.
Here are the exact steps:

Add an additional relationship that is the node user reference field, but reversed:

I added the 'Content: nid' field with the above relationship to my view just while testing, and that was helpful, so I'm including it as a step...
Add that field 'Content: nid' using the user-reference relationship to your set of contextual filters too, if you're using it, to make sure the content they're referenced in is the current node (don't want to sort them first just because they're referenced in SOME node on the site, right? ;-))

Add an attachment to your view, and set it to display directly after your existing page or block.  In this one, you'd want to do the exact same as above, however, when you're setting the contextual filter, select to 'Exclude' the matching answers (you'll find that check box in the collapsed 'More' section at the bottom.
Comment: This is a good solution for simply displaying all the users on a page—however, I still need to be able to sort and filter (using exposed filters) all the users together, so this won't work for this site.
Comment: Oh, your question only mentions the display...
Comment: I'll update it, then :)
Comment: Given your new requirement, I suspect you'll have to actually modify the SQL in your view to accomplish something like that.  If no one else chimes in to say differently, the samples in this question may help you (using hook_views_query_alter()): http://drupal.stackexchange.com/questions/73/custom-query-in-views.
Comment: I was hoping there was a way to do it without the query_alter hook—I've been banging my head against the wall with that, too (it's hard to conceptualize the joins and compound order-by clause I'll need). But I'll give it another go.
Comment: Maybe you can use the views php module (http://drupal.org/project/views_php) to compare the Content:nid that's referenced to the current node and return 1 or 0 based on that comparison as one of your php sorting fields?
Comment: I'm doing that for now, but the problem is that there are a few instances where there are over 100,000 rows, and PHP now spends about 1-2 min. just sorting everything—on every page load.
Comment: Maybe caching can help.  No need to re-compute each time the page is looked at, right?
Comment: It could help somewhat, but this particular list is updated fairly regularly, and has some unique per-row rendered bits for each user :(


Automatic EC2 Role Assignment

Question: Trying to understand AWS IAM resources/concepts a little better. I know there is a way to configure an EC2 (either possibly via its underlying AMI or a launch template) so that when it launches for the first time it is automatically assigned the right role/permissions for things that instance should have access to. For instance if I know that the software server running on that instance will need S3 read/write access, I should be able to configure things so that I don't need to SSH onto that instance and configure an ~/.aws/credentials file.
My understanding is that Users and Roles are IAM resources that make up authenticatable identities. Meaning you can gives a User/Role a set of credentials and those credentials can then authenticate against AWS and identify themselves. I believe it is the recommended practices to give human beings a User resource and to give software a Role resource. So I would have my own AWS IAM User, and that User would have credentials that I would punch in to the web console or CLI to gain access to AWS. And my server (which would be deployed to AWS) would be given an AWS IAM Role such as (myapp-server-dev) and that Role would have credentials that the server could then present to the AWS API for authentication.
My understanding is that Groups are just collections of Users and/or Roles, used for the purpose of assigning those Users/Roles to specific permission sets.
My understanding is that IAM Policies are those permission sets, and that you attach/bind Policies to Users, Roles and/or Groups so as to give them permission to various AWS services/resources.
My understanding is that IAM Instance Profiles somehow bind an EC2 instance to a Role, but herein lies my main confusion.
Finally, it is my understanding that in order to make it so that:

I create an EC2 instances; and
That EC2 instance automagically has access to the correct AWS services/resources without having to manually install credentials files on the instance via SSH...

...in order to make that happen, I need to:

Create a Role for the EC2 instance and any other instances that will be running homogenous server software on them (that have the same authorization needs)
Create an Instance Profile for the EC2 instance that is bound to this Role
Attach a Policy to that Role (or any Group where the Role is member of) that gives it access to the appropriate services/resources

So first, if anything I've stated above is inaccurate, please begin my providing some course correction/clarification for me! Assuming my understandings are correct, then my question here is: how and where do I associate a given EC2 instance with an IAM Instance Profile, and how/where do I associate that Profile with a Role?
Answer: An "IAM Instance Profile" is a Role if you're using the console.
You associate one when you create the instance, in the "Advanced details" section.

Behind the scenes, AWS is creating an instance profile for you. If you're using the command line or API, you can create them separately if you like; see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html for details.
Comment: Interesting @ceejayoz +1 and thanks! What would I do from inside an autoscale group managed by CloudFormation? Meaning, I want to define and manage an autoscaling target group of EC2 instances from inside a CF template. I assume there's an CF/EC2 field/property that allows me to specify the role/instance profile to use for all the EC2 instances managed by that autoscaling group? Thanks again!
Comment: @hotmeatballsoup The same dropdown is available when creating an autoscaling launch configuration.

Automatic EC2 Role Assignment


Dummy variables when not all categories are present across multiple features & data sets

Question: I want to ask an extension of this question, which talks about adding a label to missing classes to make sure the dummies are encoded as blanks correctly.
Is there a way to do this automatically across multiple sets of data and have the labels automatically synched between the two? (I.e. for Test & Training sets). I.e. the same columns but different classes of data represented in each?
E.g.:
Suppose I have the following two dataframes:
df1 = pd.DataFrame.from_items([('col1', list('abc')), ('col2', list('123'))])
df2 = pd.DataFrame.from_items([('col1', list('bcd')), ('col2', list('234'))])

df1
   col1   col2
1   a     1
2   b     2
3   c     3
df2
   col1    col2
1   b      2
2   c      3
3   d      4

I want to have:
df1
    col1_a col1_b col1_c col1_d col2_1 col2_2 col2_3 col2_4
1   1      0      0      0      1      0      0      0
2   0      1      0      0      0      1      0      0
3   0      0      1      0      0      0      1      0

df2
    col1_a col1_b col1_c col1_d col2_1 col2_2 col2_3 col2_4
1   0      1      0      0      0      1      0      0
2   0      0      1      0      0      0      1      0
3   0      0      0      1      0      0      0      1

WITHOUT having to specify in advance that
col1_labels = ['a', 'b', 'c', 'd'], col2_labels = ['1', '2', '3', '4']
And can I do this systematically for many columns all at once? I'm imagining a fuction that when passed in two or more dataframes (assuming columns are the same for all):

reads which columns in the pandas dataframe are categories
figures out what that overall labels are
and then provides the category labels to each column
Does that seem right? Is there a better way?
Answer: I think you need reindex by union of all columns if same categorical columns names in both Dataframes:
print (df1)
  df1
1   a
2   b
3   c

print (df2)
  df1
1   b
2   c
3   d

df1 = pd.get_dummies(df1)
df2 = pd.get_dummies(df2)

union = df1.columns | df2.columns
df1 = df1.reindex(columns=union, fill_value=0)
df2 = df2.reindex(columns=union, fill_value=0)
print (df1)
   df1_a  df1_b  df1_c  df1_d
1      1      0      0      0
2      0      1      0      0
3      0      0      1      0
print (df2)
   df1_a  df1_b  df1_c  df1_d
1      0      1      0      0
2      0      0      1      0
3      0      0      0      1

Comment: This is awesome! I tested this on df's with multiple columns and it still worked great, will update the question to reflect what I was actually trying to do


Nome do construtor sendo exibido no array ao acessá-lo via console

Question: Estou com uma pequena dúvida, diria até que é mais uma curiosidade, sobre a exibição do nome da função construtora (como se fosse um título) ao acessar no console um array com objetos instanciados por esta função.
Quando adiciono um objeto em um array, e consulto ele no console, tenho seguinte resposta:
var biblioteca = [];
biblioteca.push({titulo: "De primatas a astronautas"})

Obtenho no console:
[{...}]
0: {titulo: "De primatas a astronautas"}

E quando adiciono um objeto, criado por uma função construtora, no array e consulto ele no console, tenho como resposta:
var biblioteca = [];
var livro = new Livro("De primatas a astronautas");
biblioteca.push(livro);

Obtenho no console:
[Livro]
0: Livro {titulo: "De primatas a astronautas"}

No segundo exemplo, é exibido um "título" com o nome da função construtora (Livro) para o objeto. Este título é exibido somente como uma informação sobre qual foi a função que instanciou aquele objeto?
Sei que é uma dúvida simples, mas é a primeira vez que vejo esta informação sendo exibida assim no console... então fica a dúvida.
Answer: O nome no lado do objeto indica que ele tem um construtor, e isso indica que pode haver mais propriedades no objeto, estando no seu protótipo, por exemplo:

Como funcionam protótipos em JavaScript?

class Livro {
  constructor(titulo) {
    this.titulo = titulo
  }

  qualquerCoisa() {
    return 'Qualquer Coisa';
  }
}

const livro = new Livro('Exemplo');

console.log(livro);
console.log(livro.qualquerCoisa);
console.log(livro.__proto__.qualquerCoisa);

Apesar do primeiro console.log mostrar apenas a propriedade titulo, há também o método qualquerCoisa, "escondido" (no sentido que não é mostrado no console diretamente) no seu protótipo
Isso pode ser observado também nas APIs nativas:

const promise = new Promise(() => {});

console.log(promise);
console.log(promise.then);
console.log(promise.__proto__.then);

Lembrando que o console é do navegador e o que ele mostra ou não pode variar de acordo com cada implementação. Também há o console do NodeJS que é um pouco diferente
Comment: Respondeu sim, sou meio novato aqui, não sabia sobre a questão de marcar como respondida. Vou fazer já...
Comment: Muito obrigado!!!
Comment: @Mantuan se respondeu tua pergunta, marque o "v" na resposta pra que a pergunta seja considerada respondida


masterpage with findcontrol return null

Question: I have a master page with a label... When i try to find this label on backend it returns me null... Anyone can help me?

            <button type="button" class="close" data-dismiss="alert" aria-label="Close">
                ×</button>
            Atenção!
            <asp:Label ID="lbAlert" for="Alert" runat="server" />
            <asp:LinkButton ID="lbkbtnAtivar" for="Alert" runat="server" />

and backend.
        Label lbAlert = (Label)this.Master.FindControl("lbAlert");
        LinkButton lbkbtnAtivar = (LinkButton)this.Master.FindControl("lbkbtnAtivar");

If anyone know, help me please!! :)
Answer: Remove "Master"
Label lbAlert = (Label)this.FindControl("lbAlert");
LinkButton lbkbtnAtivar = (LinkButton)this.FindControl("lbkbtnAtivar");

You are already in the appropriate scope for the class. Therefore, "this" refers to the masterpage.
Comment: I can use Label lbAlert = (Label)Master.FindControl("lbAlert"); too?
Comment: Sure. "This" refers to the instance and is redundant when using Master. Otherwise, you are calling a parent Master page of the current master page. Which is NULL.

masterpage with findcontrol return null


Strange rendering error in Visual Studio 2022

Question: A few days ago I tried to start Visual Studio and it glitched out showing only some of the buttons and ignoring my entire project list. When I try opening a solution file, it also glitches out, showing the core components and everything being stuck in a bright white screen.
VS Menu:
Visual Studio Menu
VS Editor:
Visual Studio Solution
This has happened before but the only way it was resolved was by having to reinstall windows entirely.
So far I have reinstalled Visual Studio 3 times, manually installed / reinstalled all packages and libraries, Updated all graphics drivers, and searched through almost every Microsoft document and forum page for Visual Studio errors I could find.
Is there any way to fix this without having to lose all of my apps and files again?
(this only happens to 2022, I would use other versions but my projects seem to break when using them)
Comment: This isn't the first time I've seen this kind of question asked, and there doesn't seem to be a concrete way to resolve it.  All I can think of is make sure any display drivers are up to date.
Comment: Very strange. Have you filed a feedback ticket about this? That will include diagnostic information that MS engineers can use to try and diagnose the problem (in VS, _Help | Provide Feedback | Report a Problem_). Probably also worth updating your video drivers.
Comment: I already tried updating my video drivers, but after many hours of debugging I put all of my files onto an external drive and downgraded from windows 11 to windows 10
Answer: I hit a similar issue with a WXwidgets project I'm working on, VS debug solution started to render off screen and all the buttons and text fields were all jacked, and my undo buttons weren't available.
I resolved this issue by making my file smaller, removing a bunch of commented code, white spaces about 1400 lines to 1100 lines. Idk why this fixed the issue, I just read somewhere about some issues with visual studio and files being to long.


Orientation and first homology

Question: Let $M$ be a differentiable manifold. From the fact that a vector bundle $\pi:V \rightarrow M$ is orientable iff its orientation covering $Or(V) \rightarrow M$ is trivial, one can conclude that if the first integer homology group of $M$ vanishes, then any (differentiable) vector bundle over $M$ is orientable, in particular $M$ is then an orientable manifold.
1) What is the geometric intuition behind this statement? It seems surprising to me that there is a one dimensional phenomenon which is even a stronger property than orientability.
2) Do we have a similar result for topological manifolds?
3) Does the conversion also hold, i.e. if every (differentiable) vector bundle over a (differentiable) manifold $M$ is trivial orientable, do we have $H_1(M;\mathbb{Z})=0$ ?
Edit: Sorry, it seems that i wasn't really awake when i wrote this first. So here is some tidying up. Of course in 3) i meant all bundles to be orientable and not trivial. Also 2) is clearly to be answered with yes since we can define an orientation of a vector bundle in the same manner as one can for smooth ones, i.e. choose orientations of each fiber and make sure they are compatible with a chosen bundle atlas (meaning that if $o_p \in V_p$ is an orienation and $\varphi : \pi^{-1} (U) \rightarrow U \times\mathbb{R}^n$ a bundle chart of this atlas then the iso $\varphi_x:V_x := \pi^{-1}(x)  \cong \mathbb{R}^n$ maps the orientation $o_x$ to the canonical standard orientation of $\mathbb{R}^n$ for all $x \in U$). So we can build the orientation covering $Or(V) = \coprod_\limits{p \in M} Or(V_p) \rightarrow M, (p,o_p) \mapsto p$ and proceed as usual.
Comment: @Tyrone: The claim was about homology, not cohomology; note that $H_1(\mathbb{RP}^2; \mathbb{Z}) \cong \mathbb{Z}_2$.
Comment: Ok, granted I can't read before coffee, but my second statement holds. For the same reasoning as above we have $H^1(M;A)=[A,BA]=0$ for all discrete abelian groups $A$. Hence the universal coefficient theorems tell you that $H_1(M;\mathbb{Z})=0$.
Comment: I'm not sure your opening statement is correct. Consider $\mathbb{R}P^2$, for which $H^1(\mathbb{R}P^2;\mathbb{Z})=0$ yet is non-orientable.
Comment: Your statement $3)$ is actually correct, since elements of $H^1(M;\mathbb{Z})$ may be thought of as homotopy classes of maps $M\rightarrow K(\mathbb{Z},1)\simeq S^1$. Since $K(\mathbb{Z},1)\simeq B\mathbb{Z}$, these homotopy classes classify principal $\mathbb{Z}$-bundles over $M$, or equivalently vector bundles over $M$ with structure group $\mathbb{Z}$. Since these bundles are all trivial by assumption, there are no non-trivial homotopy classes of maps $M\rightarrow B\mathbb{Z}\simeq K(\mathbb{Z},1)$. Thus $H^1(M;\mathbb{Z})=0$.
Comment: Yes, that's nice. I've just realised that the orientation covering is also defined in the topological case. So if we'd restrict to show that $M$ is orientable Dan Rust's answer in https://math.stackexchange.com/questions/786483/simply-connected-manifolds-are-orientable applies perfectly if you remove his last argument ($\pi_1 = 1$) and notice $Hom(\pi_1(M), \mathbb{Z}) =Hom(H_1(M),\mathbb{Z})=0$ by Hurewicz which gives the same contradiction.
Answer: So after getting a little more into vector bundle theory, i guess this should work:
Claim Let $M$ be a connected manifold. Then $H^1(M;\mathbb{Z}_2) = 0$ if and only if any vector bundle over $M$ is orientable.
This claim shows that my statement 3) does not hold, e.g. pick the Lens space $L_{3,1} = S^3/\mathbb{Z}_3$ which has $H_1(L_{3,1};\mathbb{Z}) = \mathbb{Z}_3 \neq 0$ but $H^1(L_{3,1};\mathbb{Z}_2) = 0$ by universal coefficients, hence any bundle over $L_{3,1}$ is orientable.
Proof of the claim. Let $H^1(M;\mathbb{Z}_2) = 0$ and let $V \rightarrow M$ be a vector bundle. Consider the orientation covering $\rho : Or(V) \rightarrow M$ which is a double cover and trivial iff $V \rightarrow M$ is orientable. Asumme it is not trivial, so $Or(V)$ is connected, hence path connected (as a locally path connected space). By covering theory we then have an index $[\pi_1(M), \rho_* \pi_1(Or(V)] = 2$, so $\rho_* \pi_1(Or(V)$ is a normal subgroup of $\pi_1(M)$ which leads us to the epimorphism
$$\pi_1(M) \rightarrow \pi_1(M) / \rho_* \pi_1(Or(V) \cong \mathbb{Z}_2$$
But since $Hom(\pi_1(M),\mathbb{Z}_2) \cong Hom(H_1(M;\mathbb{Z}),\mathbb{Z}_2) \cong H^1(M;\mathbb{Z}_2) = 0$ by Hurewicz and universal coefficients this is a contradiction.
On the other hand if $H^1(M;\mathbb{Z}_2) \neq 0$, consider the first Stiefel-Whitney class $w_1$ which is an isomorphism $w_1 : Vect^1(M) \rightarrow H^1(M;\mathbb{Z}_2)$ from isomorphism classes of line bundles on $M$ to the first mod 2 cohomology. We have $w_1(L) = 0$ iff $L \rightarrow M$ is orientable and so there is a non orientable line bundle on $M$.

Orientation and first homology


One problem with imputing the missing value by the sample mean

Question: Today I read an interesting statement, which I am not sure about its correctness.
Assume we are looking at a data set with one column $H$ that is numeric. It has a bunch of missing values. Now let's say I just replace the missing value in column $H$ by the mean of this column (calculated by excluding the missing value). This method is good in the sense that it won't change the mean of column H. But it also has some problems. I was told that this will lessen the correlation with other numeric columns.
This is something I don't understand. I think the extreme case. Assume that column H only has 2 non-missing value, then I use the average of these 2 numbers to replace all the missing values in column H, then this will make the column H almost like a constant, which has variance close to 0. Then the correlation between this almost constant variable and other numeric column will be infinity? or undefined? or zero?
Answer: Consider the formula for computing the sample correlation between two variables, $x$ and $y$:
$$r_{x,y}=\frac{\sum_n^N(x_n-\bar x)(y_n-\bar y)}{(N-1)s_xs_y},$$
where $s_x$ and $s_y$ are non-zero corrected sample standard deviations. Now, if $x$ is the imputed variable, in all the imputed samples the term $(x_n-\bar x)$ will evaluate to 0, reducing the value of the numerator and thus of the correlation.
In your specific example, the the correlation between $H$ and some other numeric column would depend on the correlation between the values in non-imputed rows. It will always be in the range $[-1;1]$, unless one of the columns has zero sample variance, in which case it will be undefined.
Comment: If neither column x and y don't have missing data, then $r_{x,y}$ is the sample correlation. Now let's say in x column, we missed a couple of data and we fill them with the sample mean computed from the remaining non-missing data in column x (which is almost equal to the original sample mean). Now, for those filled row, the term $x_n-\bar{x}$ is 0. So the numerator shrinks. But for the denominator, when we compute $s_x$, it also gets smaller, right? Because $(x_n-\bar{x})^2$ for those filled rows are zero as well. So both numerator and denominator gets smaller. How can we tell about quotient?
Comment: The important part is that the zero terms in the nominator ($x_n-\bar x$) also cancel out the $y$-part of the nominator (sum of products). In the denominator, those variances are computed separately (product of sums), so the variance in $y$ is not cancelled out.
Comment: I agree with you. But if the row with missing data of x has a small value of $y_n-\bar{y}$ and that row originally has a big value of $x_n-\bar{x}$, then the shrinkage in the numerator is kind of small but the shrinkage of the denominator is relatively large, as it is squared, right? Sorry for my being picky. I just want to be a little bit more precise. I agree with you in general. I just want to make sure that mathematically, there are some extreme corner case such that something else could happen


How to add vocabulary terms to a taxonomy category

Question: I'm trying to get my module to install a vocabulary on enable/install but something about adding terms to it isn't working. any idea why this function isn't working?
function _vocabulary_add_items() {
  $vocabulary = taxonomy_vocabulary_machine_name_load('usertype');
  if($vocabulary === TRUE) {
    $terms = array(
        (object) array(
            'vid' => $vocabulary->vid,
            'name' => t('Economy'),
            'description' => t('Basic user account'),
            'format' => '', // default
            'weight' => 0, // default
            'vocabulary_machine_name' => $vocabulary->machine_name, // optional
        ),
        (object) array(
            'vid' => $vocabulary->vid,
            'name' => t('Deluxe'),
            'description' => t('Deluxe user account'),
            'format' => '', // default
            'weight' => 0, // default
            'vocabulary_machine_name' => $vocabulary->machine_name, // optional
        ),
        (object) array(
            'vid' => $vocabulary->vid,
            'name' => t('Ultimate'),
            'description' => t('Ultimate user account'),
            'format' => '', // default
            'weight' => 0, // default
            'vocabulary_machine_name' => $vocabulary->machine_name, // optional
        )
    );
    foreach ($terms as $term) {
        taxonomy_term_save($term);
    }
  }
}

Answer: taxonomy_vocabulary_machine_name_load() returns an object or FALSE. You're checking for exactly TRUE by using ===.
Any of the following should work (sorted by my preference):
if ($vocabulary) {
if (is_object($vocabulary)) {
if ($vocabulary !== FALSE) {
if ($vocabulary == TRUE) { // only two equal signs here

Comment: I've tried each of those options but it seems not to create the terms still.
Comment: Could it be the way i have the terms nested in the array?
Comment: I just tried your code using `if ($vocabulary) {` and it worked great. `hook_install()` is only called on first installation. So if you're just disable/enable to test w/o uninstalling first, it won't work.
Comment: I've been using the devel reinstall option. I believe this uninstalls and then installs.
Comment: Great thanks rocketeerbkw seems to work now. I think it was my conditional statement in a previous function that was preventing it from looping over the results.


Create heatmap with values from matrix in ggplot2

Question: I've seen heatmaps with values made in various R graphics systems including lattice and base like this:

I tend to use ggplot2 a bit and would like to be able to make a heatmap with the corresponding cell values plotted.  Here's the heat map and an attempt using geom_text:
library(reshape2, ggplot2)
dat <- matrix(rnorm(100, 3, 1), ncol=10)
names(dat) <- paste("X", 1:10)
dat2 <- melt(dat, id.var = "X1")
p1 <- ggplot(dat2, aes(as.factor(Var1), Var2, group=Var2)) +
    geom_tile(aes(fill = value)) +
    scale_fill_gradient(low = "white", high = "red")
p1

#attempt
labs <- c(apply(round(dat[, -2], 1), 2, as.character))
p1 +  geom_text(aes(label=labs), size=1)

Normally I can figure out the x and y values to pass but I don't know in this case since this info isn't stored in the data set.  How can I place the text on the heatmap?
Comment: [this](http://stackoverflow.com/questions/11599023/r-ordering-the-axis-labels-in-a-ggplot-geom-tile-plot) question has an example of using text with geom_tile that may be useful. Move your aes() call from geom_tile to geom_text
Comment: try `heatmap.2`. refer to similar post http://stackoverflow.com/questions/3789549/display-a-matrix-including-the-values-as-a-heatmap
Comment: @TylerRinker Had just hopped on the bus, couldn't provide a proper answer but glad to see you were able to get it sorted.
Comment: Does this help? http://socialdatablog.com/heatmap-tables-with-ggplot2-sort-of/
Comment: @Puriney It seems to me that heatmap.2 is a more powerful option for creating heatmaps than ggplot2, would you agree with this?  I read that "There is no specific heatmap plotting function in ggplot2" on SOURCE: http://learnr.wordpress.com/2010/01/26/ggplot2-quick-heatmap-plotting/
Answer: Key is to add a row identifier to the data and shape it "longer".
edit Dec 2022 to make code reproducible with R 4.2.2 / ggplot2 3.4.0 and reflect changes in tidyverse semantics
library(ggplot2)
library(tidyverse)
dat <- matrix(rnorm(100, 3, 1), ncol = 10)
## the matrix needs names
names(dat) <- paste("X", 1:10)

## convert to tibble, add row identifier, and shape "long"
dat2 <-
  dat %>%
  as_tibble() %>%
  rownames_to_column("Var1") %>%
  pivot_longer(-Var1, names_to = "Var2", values_to = "value") %>%
  mutate(
    Var1 = factor(Var1, levels = 1:10),
    Var2 = factor(gsub("V", "", Var2), levels = 1:10)
  )
#> Warning: The `x` argument of `as_tibble.matrix()` must have unique column names if
#> `.name_repair` is omitted as of tibble 2.0.0.
#> ℹ Using compatibility `.name_repair`.

ggplot(dat2, aes(Var1, Var2)) +
  geom_tile(aes(fill = value)) +
  geom_text(aes(label = round(value, 1))) +
  scale_fill_gradient(low = "white", high = "red")

Created on 2022-12-31 with reprex v2.0.2
Comment: Still useful! I received a warning (in ggplot2 v. 2.2.0) `Ignoring unknown aesthetics: fill`. A bit of tinkering showed that the penultimate line doesn't need a fill value. So this produces the same output for me, without any warning: `geom_text(label = round(dat2$value, 1)))`
Comment: @Gabriel the whole thing was outdated so I updated it
Comment: brilliant, thank you very much. There's some syntax in there I don't recognise... Time to get googling.
Comment: @TylerRinker Thanks much. How can I upgrade this graph to look like this one:
Comment: @TylerRinker: Awesome intro, brother. Any tips how I can bump the game to make the heatmap look like   http://imgur.com/gallery/X2Zxf?
Comment: @Pradeep not sure but that would be nice.  Maybe adding annotations on top and/or combining plots (some text only) using gridExtra, cowplot, and/or ggtree packages.  In any event this is a new question in and of itself that I'd like to know the answer to.  Could you start a new thread?
Comment: it is important to have `geom_text` after `geom_tile` otherwise you get no label and wonder why.
Comment: But the heatmap you ploted is **not** in the same order as in the original `dat` dataframe that you created. How can you make it to look **exactly** like that. I thought that is what OP was asking, and what I thought I would find here. Thanks :)
Comment: The answerer and OP are one and the same.  How is it not in the same order?  If you don't get the same order and you need to make a factor out of the x/y vars and set the levels.
Comment: How would you add a small white border around each number?
Comment: @skan That is a new question worthy of it's own thread.  There are a few approaches that might be possible.
Answer: There is another simpler way to make heatmaps with values. You can use pheatmap to do this.
dat <- matrix(rnorm(100, 3, 1), ncol=10)
names(dat) <- paste("X", 1:10)
install.packages('pheatmap') # if not installed already
library(pheatmap)
pheatmap(dat, display_numbers = T)

This will give you a plot like this

If you want to remove clustering and use your color scheme you can do
pheatmap(dat, display_numbers = T, color = colorRampPalette(c('white','red'))(100), cluster_rows = F, cluster_cols = F, fontsize_number = 15)

You can also change the fontsize, format, and color of the displayed numbers.


Any tips for reading sequencer piano roll notation?

Question: I'd like some hints reading piano roll notation in Cubase LE. At the moment when I see the vertical piano keyboard on the left-hand side of the window I'm sort of turning my head, imagining it horizontal.
Seems that everything that's on a stave can be represented in piano roll, e.g. note pitch, duration, bends, articulation and so on. I'm comfortable with using visual cues in stave notation, so for instance if a three note chord has notes on three adjacent lines (or three adjacent spaces) I know it's probably a root position triad. Or I can see if two notes are a third or a fifth or an octave apart. And so on.
What should I be concentrating on when I'm reading (or writing) piano roll? Are there corresponding visual cues that I can use to speed up my reading. It's really slow at the moment.
Comment: @Dekkadeci You can see note velocities via a vertical bar graph below the piano roll.
Comment: @Todd Wilcox She's 73 [https://en.wikipedia.org/wiki/Kelly_Green_(musician)], but a gentleman wouldn't mention it.
Comment: Yes, it is definitely slow. Personally I find the horizontal staff line of notation a closer visual metaphor than the vertical piano keys. The only problem is that accidentals all get their own line, making it hard to eyeball intervals...
Comment: Wait, the Cubase piano roll notation displays dynamics? (I find this to be a traditional display weakness in piano roll and MIDI notation such as Synthesia.)
Comment: @Dekkadeci Logic displays velocity in piano roll with color coding of the notes. Cool colors are low velocity, warmer colors are higher. Kelly green is approx 64.
Comment: The thing this question made me think of is how the traditional keyboard layout reflects the layout of the stave: they both contain the principle of the diatonic scale, that is to say they are organised on a diatonic rather than chromatic basis; whereas the piano roll accords equal space to all 12 tones. There's no reason really why one should be better/worse than the other, but perhaps the diatonic organisation was more /useful/ at certain points historically, I don't really know. I might attempt a proper answer to the question asked later.
Comment: Piano roll doesn't impart the harmonic context - for instance it can tell you to play the left-hand black note in a group of three black notes. But it doesn't tell you whether to think of that black note as an F# or a Gb. Stave notation would tell you.
Comment: @BrianTHOMAS - Famitracker renders all black notes as sharps, so at some level (and with 12TET being ubiquitous in these parts), accidental notation doesn't matter at all.
Comment: @JudyN. "perhaps the diatonic organisation was more /useful/ at certain points historically": it was certainly more useful in the middle ages before the chromatic scale had been invented.
Answer: One thing I find helpful it to think in terms of interval classes. This allows me to focus on the guide lines rather than the absolute pitches, thus largely ignoring the piano keyboard.
I found it fairly quick to train myself to see intervals in terms of the number of grid lines lay between pitches. So a major chord would look, vertically, like <0,4,3>, and a minor chord would be <0,3,4>. It takes a little while to get used to thinking of chords this way, but one starts to see "stacks of thirds", even when inverted, in a manner analogous to the way one learns to see them on a keyboard.
Octaves are especially easy to see, since each note will be equidistant from the "C" markers above and/or below. And seconds (1s and 2s) are also an easy starting place.
In general, I found harmonic intervals to be the best starting place. Once I began to see them more easily, my feel for melodic intervals followed.


Cancel file transfers while transferring file to destination

Question: I have used Winscp for file transferring and got details file transfer progress like below link,
File transfer details binding continuously until file transfered in window using WPF
and transfer code link below
https://winscp.net/eng/docs/library
But i need to cancel file transferring. Please give your suggestion.
Answer: To my understanding the only way to achieve this is to abort the current session.
link: https://winscp.net/eng/docs/library_session_abort
Comment: How to define Abort method? @William Cross
Comment: if you follow the getfiles example here:
https://winscp.net/eng/docs/library_session_getfiles - you will see that the getfiles method is invoked inside a new session (in this example, named session). If you were to abort the session that getfiles is contained within you would interrupt the file transfer.
Comment: I have canceled file transferring with help of Abort();..Thanks @William Cross
Answer: You can use FileTransferProgressEventArgs.Cancel:
bool cancel = false; // set to true to cancel the transfer

session.FileTransferProgress +=
    (sender, e) =>
    {
        if (cancel)
        {
            e.Cancel = true;
        }
    };

session.PutFiles(localPath, remotePath).Check();


Include dynamically generated filenames in grunt-concat build step

Question: As part of our Grunt build we are transpiling some typescript code into a few separate files, and then concatenating the resulting javascript along with all the rest of our javascript code. Unfortunately I can't get the dynamic buildup of filenames to work along with grunt-concat.
This is the relevant snippet from our Gruntfile - see the part on generatedTypeScriptFiles, which doesn't work.
var jsfiles = [
    'js/external_libraries/inherit.js',
    'js/external_libraries/modernizr.js',
    'js/baz.js'
];

grunt.initConfig({
  gitinfo : {}, //will be populated with values from Git
  options : {
    "packageName"         : nconf.get("name"),
    "frameworkVersion"    : nconf.get("version"),
    "frameworkOutputPath" : nconf.get("frameworkOutputPath"),
    "workerOutputPath"    : nconf.get("workerOutputPath"),
    "sourceMapPath"       : nconf.get("sourceMapPath")
  },

  typescript : {
    foo: {
      src     : ['js/Foo/*.ts'],
      dest    : 'generated/Foo.js',
    },
    bar : {
      src     : ['js/Bar/*.ts'],
      dest    : 'generated/Bar.js',
    }
  },

  generatedTypeScriptFiles : {
    all : function () {
      var tsf = [];
      for (var key in this.typescript) {
        if(this.typescript[key].dest) {
          tsf.push(this.typescript[key].dest);
        }
      }
      return tsf;
    }()
  },

  scriptfiles : {
    hybrid : function () {
      return jsfiles.concat('<%= generatedTypeScriptFiles.all %>');
    }(),
    web    : function () {
      return jsfiles.concat('<%= generatedTypeScriptFiles.all %>');
    }()
  }

  concat : {
    web    : {
      options : {
        separator : ';',
      },
      src     : '<%= scriptfiles.web %>',
      dest    : '<%= options.frameworkOutputPath %>'
    }
  }
}

I am guessing that for our case where we actually know all the resulting filenames in our typescript build step, we could just build up the filenames beforehand - outside of grunt.initConfig. That should fix things, right? Or is there another way?
Answer: Maybe I misunderstand the problem or what you're trying to do, but why don't you just glob all .js files from the "generated" directory?
Like this:
var jsfiles = [
    'js/external_libraries/inherit.js',
    'js/external_libraries/modernizr.js',
    'js/baz.js',
    'generated/*.js'
];
grunt.initConfig({
  typescript : {
    foo: {
      src     : ['js/Foo/*.ts'],
      dest    : 'generated/Foo.js',
    },
    bar : {
      src     : ['js/Bar/*.ts'],
      dest    : 'generated/Bar.js',
    }
  },
  concat : {
    web    : {
      options : {
        separator : ';',
      },
      src     : jsFiles,
      dest    : '<%= options.frameworkOutputPath %>'
    }
  }
});

Comment: That is absolutely a valid solution. On second thought, I guess what I was wondering was if there was a way of building up values during one step of a Grunt build and use those values in a later stage/step. That is a more general question than what I asked, so I guess the wording of my original question was somewhat unfortunate.
Comment: Sure, you can store them in the config object (grunt.config.set) during one task and then retrieve them in another task. Here's more info: http://gruntjs.com/api/grunt.config
Comment: After taking another look at your code, I assume what goes wrong is that `generatedTypeScriptFiles.all` isn't substituted, right? That's because the templating function isn't called, but you could do that manually: http://gruntjs.com/api/grunt.template


Dates and rules

Question: I've got a rule with php condition. I'm trying to compare 2 dates. the problem comes from the node date.
Here is a part of my script:
<code>$date= date("Y-m-d");
$node_date= ("[node:field_event_date]");

$myDateTime = DateTime::createFromFormat("Y-m-d H:i:s", $node_date);
echo $myDateTime->format("d-m-Y");
</code>
I would like to convert my date and time to a different date format (d-m-Y was set on purpose to test the script, but eventually the format would be Y-m-d).
The date i'm pulling out has this format: yyyy-mm-dd H:i:s (i did an echo $node_date )
but i can't convert it using "createFromFormat".
Even though when i enter a date with the same format that "$node_date", instead of the variable, it works...
So i don't really know any more what to do...i feel i got the solution, but cannot make it.
Thanks for your help
Answer: I don't know if my solution is actually going to help, but I had a similar issue not with the node date but with a date field in a content type.
For some reason I was not able to properly expose the date field for comparison, and there are some slightly relevant issues about this in Rules issue queue. So what I did was to expose the field using the Views Rules module.
Although this is just a work-around really, with a performance penalty, it turned out to me that with this way the exposed field could then be used directly in comparisons.
Comment: Thanks for the advice, i'll check out and let you know !

Dates and rules