Converter Json em XML

Question: Tenho esse código:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Windows.Forms;
using System.Diagnostics;
using System.Xml;
using Newtonsoft.Json;
using System.Net;
using System.Xml.Linq;
using Newtonsoft.Json.Linq;

namespace PowderToyUpdater
{
    static class Program
    {
        ///
        /// The main entry point for the application.
        ///
        [STAThread]
        static void Main()
        {
            Application.EnableVisualStyles();
            Application.SetCompatibleTextRenderingDefault(false);

            WebClient web = new WebClient();
            var value = web.DownloadString("http://powdertoy.co.uk/Browse/Comments.json?ID=1681176&Start=0&Count=20");

            XmlDocument xdoc = (XmlDocument)JsonConvert.DeserializeXmlNode(value);

            XmlWriter w = XmlWriter.Create("comments_.xml");
            xdoc.WriteTo(w);
        }
    }
}

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace PowderToyUpdater
{
    public struct Comment
    {
        public string Username;
        public string Userid;
        //public string Gravatar;
        public string CommentContent;
        public string Timestamp;
        public string FormattedUsername;
    }
}

Mas quando o JSON é convertido para XML e armazenado em um arquivo, parte do XML fica faltando.
<?xml version="1.0" encoding="utf-8"?><User><root><Username>Sandwichlizard</Username><UserID>105412</UserID><Gravatar>/Avatars/105412_40.png</Gravatar><Text>front page.</Text><Timestamp>1424281388</Timestamp><FormattedUsername>Sandwichlizard</FormattedUsername></root><root><Username>ndutoit</Username><UserID>133746</UserID><Gravatar>http://www.gravatar.com/avatar/88ff6ca3f7391896d88ea1f6ac46c3f8?s=40&amp;d=http%3A%2F%2Fpowdertoy.co.uk%2FDesign%2FImages%2FAvatar.png&amp;r=pg</Gravatar><Text>what is fp?</Text><Timestamp>1424279967</Timestamp><FormattedUsername>ndutoit</FormattedUsername></root><root><Username>Lord_Bowserinator</Username><UserID>130740</UserID><Gravatar>/Avatars/130740_40.png</Gravatar><Text>fp again</Text><Timestamp>1417569227</Timestamp><FormattedUsername>Lord_Bowserinator</FormattedUsername></root><root><Username>Sandwichlizard</Username><UserID>105412</UserID><Gravatar>/Avatars/105412_40.png</Gravatar><Text>sure.  I will make a tutorial save.  that will be easier than just talking about it here.  I will leave a link in this save when it is ready.  I will also publish it.</Text><Timestamp>1417561239</Timestamp><FormattedUsername>Sandwichlizard</FormattedUsername></root><root><Username>Anonymous100</Username><UserID>92366</UserID><Gravatar>/Avatars/92366_40.png</Gravatar><Text>Sandwich would you mind explaining to me the basic of all thesse fancy filt drives? I know that it has something to do with the fact that a color of filt when beamed at dtec with aray changes the filts color that is next to the dtec, but I dont understand how that info is decoded, and how it is written in the first place.</Text><Timestamp>1417541787</Timestamp><FormattedUsername>Anonymous100</FormattedUsername></root><root><Username>tombattraw</Username><UserID>85950</UserID><Gravatar>http://www.gravatar.com/avatar/a9a4ec31425b097ae496b8f027cdf016?s=40&amp;d=http%3A%2F%2Fpowdertoy.co.uk%2FDesign%2FImages%2FAvatar.png&amp;r=pg</Gravatar><Text>try recording a pong game</Text><Timestamp>1417535992</Timestamp><FormattedUsername>tombattraw</FormattedUsername></root><root><Username>Sandwichlizard</Username><UserID>105412</UserID><Gravatar>/Avatars/105412_40.png</Gravatar><Text>@mecha-man.  sure. I have already started.  I am working on a new printer save using this drive and I realized I needed some things and it forced me into a redesign.  I will PM you.</Text><Timestamp>1417488946</Timestamp><FormattedUsername>Sandwichlizard</FormattedUsername></root><root><Username>mecha-man</Username><UserID>48475</UserID><Gravatar>/Avatars/48475_40.png</Gravatar><Text>@Sandwichlizard: Wanna work together to make it more compact?</Text><Timestamp>1417482098</Timestamp><FormattedUsername>mecha-man</FormattedUsername></root><root><Username>159819</Username><UserID>78051</UserID><Gravatar>/Avatars/78051_40.png</Gravatar><Text>is it posable to ither make a biger screen or smaler pixels?</Text><Timestamp>1417478468</Timestamp><FormattedUsername>159819</FormattedUsername></root><root><Username>0356459</Username><UserID>121388</UserID><Gravatar>http://www.gravatar.com/avatar/56261703f558a3503a1ab238fd4f4dac?s=40&amp;d=http%3A%2F%2Fpowdertoy.co.uk%2FDesign%2FImages%2FAvatar.png&amp;r=pg</Gravatar><Text>sorry man......but hey front page for a long time! :)</Text><Timestamp>1417412356</Timestamp><FormattedUsername>0356459</FormattedUsername></root><root><Username>Sandwichlizard</Username><UserID>105412</UserID><Gravatar>/Avatars/105412_40.png</Gravatar><Text>If someone would like to make a new video for this setup I would happily update the save with credit to the author of said video</Text><Timestamp>1417399984</Timestamp><FormattedUsername>Sandwichlizard</FormattedUsername></root><root><Username>Sandwichlizard</Username><UserID>105412</UserID><Gravatar>/Avatars/105412_40.png</Gravatar><Text>the crappy video is just to demo the drive.  video made way better video's than mine for sure.  thank you for your input.</Text><Timestamp>1417399892</Timestamp><FormattedUsername>Sandwichlizard</FormattedUsername></root><root><Username>0356459</Username><UserID>121388</UserID><Gravatar>http://www.gravatar.com/avatar/56261703f558a3503a1ab238fd4f4dac?s=40&amp;d=http%3A%2F%2Fpowdertoy.co.uk%2FDesign%2FImages%2FAvatar.png&amp;r=pg</Gravatar><Text>video did it better 2 times before. -1</Text><Timestamp>1417318433</Timestamp><FormattedUsername>0356459</FormattedUsername></root><root><Username>zsdrfty</Username><UserID>100327</UserID><Gravatar>/Avatars/100327_40.png</Gravatar><Text>Minecraft is more insane because it has analog computers. oog</Text><Timestamp>1417281395</Timestamp><FormattedUsername>zsdrfty</FormattedUsername></root><root><Username>Imperator</Username><UserID>123921</UserID><Gravatar>/Avatars/123921_40.png</Gravatar><Text>It's so cool that people can actually make a functioning computer within a game.</Text><Timestamp>1417269917</Timestamp><FormattedUsername>Imperator</FormattedUsername></root><root><Username>handicraftsman</Username><UserID>108835</UserID><Gravatar>/Avatars/108835_40.png</Gravatar><Text>Sandwichlizard, you can use my tablets without credits ) It's for you ) id:1677416</Text><Timestamp>1417268811</Timestamp><FormattedUsername>handicraftsman</FormattedUsername></root><root><Username>Ilog123</Username><UserID>104909</UserID><Gravatar>/Avatars/104909_40.png</Gravatar><Text>Try Making a Movie maker so you can copy/paste your film and check em out</Text><Timestamp>1417251254</Timestamp><FormattedUsername>Ilog123</FormattedUsername></root><root><Username>electronic_steve</Username><UserID>96362</UserID><Gravatar>/Avatars/96362_40.png</Gravatar><Text>id:1683229 108,3... i won!</Text><Timestamp>1417237206</Timestamp><FormattedUsername>electronic_steve</FormattedUsername></root><root><Username>zsdrfty</Username><UserID>100327</UserID><Gravatar>/Avatars/100327_40.png</Gravatar><Text>16.7k? We're over 1/10 of the capacity of a single-sided, single-density 5.25" floppy disk!</Text><Timestamp>1417229000</Timestamp><FormattedUsername>zsdrfty</FormattedUsername></root><root><Username>Fulchrum</Username><UserID>95857</UserID><Gravata  <= Aqui falta o restante do xml.

E depois de converter o JSON para XML, como faço para utilizar XmlTextReader ou XmlReader?
Comment: No titulo da pergunta é para converter json para objeto, no corpo da pergunta você está falando sobre um XML cortado. Compõe a pergunta corretamente.
Comment: Por que você precisa dos dados como XML? Não seria melhor armazena-los como JSON já que ele está sendo obtidos do web service dessa forma?
Comment: Nathan, por qual motivo minha resposta não resolveu seu problema? Eu testei o que eu disse nela muito bem antes de postar aqui, e mesmo assim estou com -1. Isso não me parece nada justo.
Answer: É recomendado que você utilize o método XmlDocument.Save para gravar esse documento XML em um arquivo. A utilização dele é bem simples, veja como alterei o trecho do método Main do seu código para utiliza-lo:
WebClient web = new WebClient();
var value = $"{{ user: {web.DownloadString("http://powdertoy.co.uk/Browse/Comments.json?ID=1681176&Start=0&Count=20")} }}";

XmlDocument xdoc = JsonConvert.DeserializeXmlNode(value, "root");

xdoc.Save("comments_.xml");

Note também que fiz uma alteração no JSON obtido do web service para que ele se tornasse válido para conversão pelo método JsonConvert.DeserializeXmlNode. De acordo com a documentação do Json.NET para que você converter um documento JSON para XML, é necessário que esse documento JSON possua um único objeto na sua raiz. O JSON retornado pelo web service não seguia esse padrão e vinha no formato de array. Essa página da documentação fala sobre isso.
Sobre a segunda parte da sua pergunta onde você pergunta sobre a utilização do System.Xml.XmlReader ou System.Xml.XmlReader.
Conforme descrito na documentação do System.Xml.XmlReader, sua utilização não é mais recomendada e o ´System.Xml.XmlReader` deve ser utilizado no seu lugar.
Como não valeria a pena que eu demonstrasse tudo que é possível fazer com o System.Xml.XmlReader, e sua pergunta também não explicitou exatamente o que você gostaria de fazer com ele, vou te recomendar ler a documentação dessa classe na MSDN, já você poderá encontrar vários exemplos de utilização bem explicados. Documentação do System.Xml.XmlReader.
Answer: Esse código abaixo vai gerar um XML sem cortes, fiz um fiddler para você com o código:

WebClient web = new WebClient();
var value = web.DownloadString("http://powdertoy.co.uk/Browse/Comments.json?ID=1681176&Start=0&Count=20");

            string json = @"{
   '?xml': {
     '@version': '1.0',
     '@standalone': 'no'
   },
   'root': {
     'minhaclasse':
[
" + value.Replace("'","\'") + @"

]
  }
}";

XmlDocument xml = (XmlDocument)JsonConvert.DeserializeXmlNode(json);

xml.Save(@"c:\caminho\teste.xml");

Fiz um exemplo básico com XmlReader para você, o código abaixo também roda aqui no fiddler.

public class Program
{
 public static void Main()
 {

  String xmlString =
   @"<bookstore>
    <book genre='autobiography' publicationdate='1981-03-22' ISBN='1-861003-11-0'>
     <title>The Autobiography of Benjamin Franklin</title>
     <author>
      <first-name>Benjamin</first-name>
      <last-name>Franklin</last-name>
     </author>
     <price>8.99</price>
    </book>
   </bookstore>";

  // Create an XmlReader
  using (XmlReader reader = XmlReader.Create(new StringReader(xmlString)))
  {
   reader.ReadToFollowing("book");
   reader.MoveToFirstAttribute();
   string genre = reader.Value;
   Console.WriteLine("The genre value: " + genre);

   reader.ReadToFollowing("title");
   Console.WriteLine("Content of the title element: " + reader.ReadElementContentAsString());
  }
 }
}

Ainda existe a opção de jogar o Json direto para objeto, digo isso pelo titulo da sua pergunta, que você alterou, caso interesse você pode pegar o link direto por webClient e transformar em um objeto qualquer criado por você. Nesse caso não teria que fazer nada com XML.
ATUALIZAÇÃO
Conforme solicitado no comentário eu fiz um código juntando tudo.
Esse código não grava em arquivo, ele pega o Json, converte para XML, joga em uma String e faz a leitura. Aqui (Fiddle) você pode ver o código rodando.

            WebClient web = new WebClient();
            var value = web.DownloadString("http://powdertoy.co.uk/Browse/Comments.json?ID=1681176&Start=0&Count=20");

            string json = @"{
   '?xml': {
     '@version': '1.0',
     '@standalone': 'no'
   },
   'root': {
     'minhaclasse':
[
" + value.Replace("'", "\'") + @"

]
  }
}";

            XmlDocument xml = (XmlDocument)JsonConvert.DeserializeXmlNode(json);

            var xmlString = xml.InnerXml;

            using (XmlReader reader = XmlReader.Create(new StringReader(xmlString)))
            {
                reader.MoveToContent();
                while (reader.Read())
                {
                    if (reader.NodeType == XmlNodeType.Element)
                    {

                        if (reader.Name == "Username")
                        {
                            XElement el = XNode.ReadFrom(reader) as XElement;
                            if (el != null)
                            {
                                Console.Write("Username : ");
                                Console.WriteLine(el.Value);
                            }
                        }
                    }
                }
            }
}
Comment: Por favor poderia colocar do mesmo jeito mais com esse json aqui?
http://powdertoy.co.uk/Browse/Comments.json?ID=1681176&Start=0&Count=20
Comment: O exemplo de converter o json com para XML é com esse link aí.
Comment: Fiz um código que eu acho que é o que você quer. Se atender não esqueça de marcar a resposta.

Converter Json em XML


How to find a function that is the upper bound of this sum?

Question: The Problem
Consider the recurrence
$ T(n) =
\begin{cases}
c  & \text{if $n$ is 1} \\
T(\lfloor(n/2)\rfloor) + T(\lfloor(n/4)\rfloor) + 4n, & \text{if $n$ is > 1}
\end{cases}$
A. Express the cost of all levels of the recursion tree as a sum over the cost of each level of the recursion tree
B. Give a function $g(n)$ and show that it is an upper bound on the sum
My Work
I was able to do part A. I drew the first six levels of the recursion tree and expressed the expressed the cost of all levels as $\sum_{i=0}^{log_2n} \frac{4n}{2^i}f(i+2) $ where $f(n)$ is the $n$th term in the Fibonacci sequence(0, 1, 1, 2, 3, 5, 8)
How would I come up with a function that would be an upper bound of this sum?
Answer: Suppose we start by solving the following recurrence:
$$T(n) = T(\lfloor n/2 \rfloor) + T(\lfloor n/4 \rfloor) + 4n$$
where $T(1) = c$ and $T(0) = 0.$
Now let $$n = \sum_{k=0}^{\lfloor \log_2 n \rfloor} d_k 2^k$$
be the binary representation of $n.$
We unroll the recursion to obtain an exact formula for $n\ge 2$
$$T(n) = c [z^{\lfloor \log_2 n \rfloor}] \frac{1}{1-z-z^2}
+ 4 \sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
[z^j] \frac{1}{1-z-z^2}
\sum_{k=j}^{\lfloor \log_2 n \rfloor} d_k 2^{k-j}.$$
We recognize the generating function  of the Fibonacci numbers, so the
formula becomes
$$T(n) = c F_{\lfloor \log_2 n \rfloor +1}
+ 4 \sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} \sum_{k=j}^{\lfloor \log_2 n \rfloor} d_k 2^{k-j}.$$
We now compute lower and  upper bounds which are actually attained and
cannot  be improved upon.  For the  lower bound  consider a  one digit
followed by a string of zeroes, to give
$$T(n) \ge c F_{\lfloor \log_2 n \rfloor +1}
+ 4 \sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} 2^{\lfloor \log_2 n \rfloor-j}
\\ = c F_{\lfloor \log_2 n \rfloor +1}
+ 8 \times 2^{\lfloor \log_2 n \rfloor}
\sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} 2^{-j-1}.$$
Now since $$|\varphi|=\left|\frac{1+\sqrt{5}}{2}\right|<2$$
the sum term converges to a number, we have
$$\frac{1}{2} \le
\sum_{j=0}^{\lfloor \log_2 n \rfloor-1} F_{j+1} 2^{-j-1}
\lt \sum_{j=0}^{\infty} F_{j+1} 2^{-j-1}
= 2.$$
For an upper bound consider a string of one digits to get
$$T(n) \le c F_{\lfloor \log_2 n \rfloor +1}
+ 4 \sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} \sum_{k=j}^{\lfloor \log_2 n \rfloor} 2^{k-j}
\\ = c F_{\lfloor \log_2 n \rfloor +1}
+ 4 \sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} (2^{\lfloor \log_2 n \rfloor+1-j} - 1)
\\ = c F_{\lfloor \log_2 n \rfloor +1}
- 4 (F_{\lfloor \log_2 n \rfloor +2} -1)
+ 4 \sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} 2^{\lfloor \log_2 n \rfloor+1-j}
\\ = c F_{\lfloor \log_2 n \rfloor +1}
- 4 (F_{\lfloor \log_2 n \rfloor +2} -1)
+ 4  \times 2^{\lfloor \log_2 n \rfloor+1}
\sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} 2^{-j}
\\ = c F_{\lfloor \log_2 n \rfloor +1}
- 4 (F_{\lfloor \log_2 n \rfloor +2} -1)
+ 16 \times 2^{\lfloor \log_2 n \rfloor}
\sum_{j=0}^{\lfloor \log_2 n \rfloor-1}
F_{j+1} 2^{-j-1}.$$
The same  constant appears as in  the lower bound. Now  since the term
$F_{\lfloor  \log_2   n  \rfloor}$  is   asymptotically  dominated  by
$2^{\lfloor  \log_2   n  \rfloor}$  (we  have   $F_{\lfloor  \log_2  n
\rfloor}\in o(2^{\lfloor \log_2  n \rfloor})$ because $F_{\lfloor  \log_2  n
\rfloor} \in\Theta(\varphi^{\lfloor  \log_2   n  \rfloor}))$ joining the
upper  and  the  lower  bound  we  get for  the  asymptotics  of  this
recurrence that it is
$$T(n)\in\Theta\left(2^{\lfloor \log_2  n \rfloor}\right)
= \Theta\left(2^{\ \log_2  n}\right) = \Theta(n),$$
which, let it be said, could also have been obtained by inspection.

Remark. The evaluation of the constant is done by noting that
the generating function of
$$F_{j+1} 2^{-j-1}\quad
\text{is}\quad\frac{1/2}{1-z/2-z^2/4}$$
which at $z=1$ evaluates to $\frac{1/2}{1-1/2-1/4} = 2.$
We have a certain flexibility as to what power of two to use in the constant but this does not affect the asymptotics.

This MSE link has a similar calculation.
Comment: So the upper bound function is O(n)? I looked on here and saw the fibonacci sequence upper bound is O(2^n). - http://stackoverflow.com/questions/360748/computational-complexity-of-fibonacci-sequence   Is this just a different application?
Comment: Oh ok thanks. that makes sense!
Answer: In light of the detailed solution of @Marko Riedel, if you only want to show that $T(n)=O(n)$, you can make the (somewhat easier computation):
Base case: (ok)
Inductive Hypothesis: $T(k)\leq Ck$ for $k<n$ for some constant $c>0$.
Then, prove the inductive step:
\begin{align*}
T(n) &= T(\lfloor(n/2)\rfloor) + T(\lfloor(n/4))+4n \\
     &\leq T(n/2)+T(n/4) + 4n & \text{by monotonicity} \\
     &\leq C(n/2)+C(n/4) + 4n \\
     &=(3C/4+4)n \\
     &=Cn & \star\star
\end{align*}
Where $\star\star$ is true if $(3C/4+4)=C$ which works when $C=16$.
So, the moral of the story is that if you can guess that $T(n)=O(n)$ it is not hard to prove it by induction.
EDIT: One final note, it wasn't asked to get a $\Theta$ bound on $T(n)$, but it is pretty clear (by the recurrence) that $T(n)=\Omega(n)$ since $T(n)\geq 4n$.  Then, combining this with the previous argument shows that $T(n)=\Theta(n)$.
Comment: Without going through  @Marko Riedel, is there a way to show if g(n) = n is an upper bound on the sum itself? There were actually two questions.  Show that it is an upper bound on the sum and state and prove by induction a theorem showing T(n) in O(g(n)).
Comment: @committedandroider, if you just guess that $T(n)=O(n)$ then the proof by induction I gave shows that this is correct.  There are a number of "heuristics" outlined in the Cormen et. al "Algorithms" that would lead you to guess that it is $O(n)$.  The bound $T(n)=\Omega(n)$ is trivial from the recurrence.


debian set eth0 autoneg off speed 1000 duplex full: Invalid argument

Question: I use debian for my things,but when i config speed 1000 and duplex full the error occur
the exactly command i use:
ethtool -s eth0 autoneg off speed 1000 duplex full
  Cannot set new settings: Invalid argument
         not setting speed
         not setting duplex
         not setting autoneg
  Cannot set new settings: Invalid argument
         not setting speed
         not setting duplex
         not setting autoneg

And this is my eth0 information.
   Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                        100baseT/Half 100baseT/Full
                        1000baseT/Full
     Supported pause frame use: No
     Supports auto-negotiation: Yes
     Advertised link modes:  10baseT/Half 10baseT/Full
                        100baseT/Half 100baseT/Full
                        1000baseT/Full
     Advertised pause frame use: No
     Advertised auto-negotiation: Yes
     Speed: 100Mb/s
     Duplex: Full
     Port: Twisted Pair
     PHYAD: 1
     Transceiver: internal
     Auto-negotiation: on
     MDI-X: Unknown
     Supports Wake-on: g
     Wake-on: d
     Link detected: yes

      driver: bnx2
      version: 2.1.11
      firmware-version: bc 3.5.12 ipms 1.6.0
      bus-info: 0000:04:00.0
      supports-statistics: yes
      supports-test: yes
      supports-eeprom-access: yes
      supports-register-dump: yes
      supports-priv-flags: no

Could any one tell me which is the wrong place?
Comment: FYI, you should almost never be forcing the link speed/duplex on a Gig link.  If I can find the link someone had a really good blog post pointing go various specs saying that forcing the duplex for gig links is actually harmful most of the time.  This may have been it...  http://etherealmind.com/ethernet-autonegotiation-works-why-how-standard-should-be-set/
Comment: While that blog post is interesting, and mostly valid, what he's saying is condradicted by many of the comments. Indeed, at the company I work, it is often necessary to force the speed link on Gig links. Advised by such equipment manufacturers Cisco, Oracle, HP. Clearly we had issues that did not fall into the 'most of the time' category, but I think that many visitors here will fall into this category too.
Comment: Incidentally, I know my answer is 1.5 years late, but, the bnx2 driver does not allow disabling autonegotiation with speeds of 1 or 2.5 Gb/s. See bnx2.c:bnx2_set_settings() in the linux kernel source code for details.
Comment: Please post the exact command you're using to set the speed.
Comment: sorry, i just add it.
Answer: Autonegotiation is a requirement for using 1000BASE-T according to Section 28D.5 of IEEE 802.3-2002 specification, so you cannot disable autonegotiation with 1000BASE-T.
Try -
sudo ethtool -s eth0 speed 1000 duplex full autoneg on
Comment: While it's required by the specification, some vendors provide an option to disable autonegotiation while operating at gigabit speeds. This was relevant a decade ago when gigabit NICs were less reliable, but the non-standard configuration option still exists in certain NIC drivers, switches, and other equipment.
Answer: Did you try,
sudo ethtool -s eth0 speed 1000 duplex full autoneg off

but I support Zoredache's comment, you should probably not be forcing the link speed, and unless it doesn't work without it, you should leave the machine set to autoneg, and ensure the network port is set to the right speed.
Comment: -1 how exactly are your ethtool parameters different from the OP's?
Comment: i follow your answers.But i'm sorry.It's not usefull
Comment: @vladr the OP edited the question after I posted my answer, and included the command, it wasn't in the question when I originally wrote my answer.


Compare elements and make new column

Question: I want to compare the i and i+1 elements of second column(Close price), and if i<i+1 place 1 in third column. But i have no idea how to do this.
require(quantmod)
getSymbols("GOOG",src="yahoo")
data<-last(GOOG,40)
newdata<-data.frame(data[,4])
newdata

Comment: Question is not very clear, maybe try this example: `x <- 1:10; xdf <- data.frame(x=x, x_lag=dplyr::lag(x)); xdf$compare <- xdf$x == xdf$x_lag`
Comment: @zx8754 should it not be `xdf$x < xdf$x_lag` ?
Comment: @Tensibai yes, comparison logic was added after edit.
Answer: Try this:
newdata$closePrice = c((diff(newdata[,1])<0)*1,0)

Explanation:
diff(newdata[,1]) will return the difference of rows i+1 and i
The <0 will give a true or false array that can be converted to 1s or 0s
Since the result will be one element short, concatenate a 0 at the end
Comment: FWIW, the `*1` is unnecessary as the concatenation with a number will coerce logical to integer, i.e: `c(TRUE,FALSE,TRUE,0)` give `[1] 1 0 1 0`
Answer: Using dplyr lead/lag function:
library(dplyr)

#dummy data
set.seed(123)
mydf <- data.frame(price = runif(5))

#result
mydf %>%
  mutate(price_lead = lead(price),
         compare = ifelse(price < price_lead, 1, 0))

#output
#       price price_lead compare
# 1 0.2875775  0.7883051       1
# 2 0.7883051  0.4089769       0
# 3 0.4089769  0.8830174       1
# 4 0.8830174  0.9404673       1
# 5 0.9404673         NA      NA

Compare elements and make new column


Record video from front facing camera in iphone

Question:
Possible Duplicate:
Accessing the front facing camera. iPhone/iPod 4

How to record the video from front facing camera of the device in iPhone.
To use the front facing camera,We use this:
<code>picker.cameraDevice = UIImagePickerControllerCameraDeviceFront;
</code>
But how to create the video from that?
Answer: you can record Video from front facing camera in iphone like:-
<code>-(IBAction)cameraLibraryButtonClick:(id)sender{
    if ([UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera]) {
        UIImagePickerController *videoRecorder = [[UIImagePickerController alloc]init];
        videoRecorder.delegate = self;
        NSArray *sourceTypes = [UIImagePickerController availableMediaTypesForSourceType:videoRecorder.sourceType];
        NSLog(@"Available types for source as camera = %@", sourceTypes);
        if (![sourceTypes containsObject:(NSString*)kUTTypeMovie] ) {
            UIAlertView *alert = [[UIAlertView alloc] initWithTitle:nil
                                                            message:@"Device Not Supported for video Recording."                                                                       delegate:self
                                                  cancelButtonTitle:@"Yes"
                                                  otherButtonTitles:@"No",nil];
            [alert show];
            [alert release];
            return;
        }
        videoRecorder.cameraDevice=UIImagePickerControllerCameraDeviceFront;
        videoRecorder.sourceType = UIImagePickerControllerSourceTypeCamera;
        videoRecorder.mediaTypes = [NSArray arrayWithObject:(NSString*)kUTTypeMovie];
        videoRecorder.videoQuality = UIImagePickerControllerQualityTypeLow;
        videoRecorder.videoMaximumDuration = 120;

        self.imagePicker = videoRecorder;
        [videoRecorder release];
        [self presentModalViewController:self.imagePicker animated:YES];
        newMedia = YES;
    }
    else {
        [self displaysorceError];
    }

}
</code>


Diagonal terms of the inverse of positive definite matrix

Question: I have a matrix $A = I + G$ where $G_{ii} = 0$ for all $i$, and $G_{ij} \in \{0, a\}$ with $a \in [0, 1[$, $G$ is symmetric, and such that $A$ is positive definite.
We know that the inverse of $A$, call it $B$, is also symmetric positive definite, and also that $B_{ii} > 1$ for all $i$.
My question: is it true that $B_{ii} > B_{ij}$ for all $i, j$?
Any help welcome! Thanks
Answer: The answer is no.  For a random counter-example, consider
$A:= \left[\begin{matrix}1 & \frac{5}{9} & 0 & 0 & 0\\\frac{5}{9} & 1 & 0 & \frac{5}{9} & \frac{5}{9}\\0 & 0 & 1 & \frac{5}{9} & \frac{5}{9}\\0 & \frac{5}{9} & \frac{5}{9} & 1 & \frac{5}{9}\\0 & \frac{5}{9} & \frac{5}{9} & \frac{5}{9} & 1\end{matrix}\right]\implies A^{-1}= \left[\begin{matrix}\frac{1053}{103} & - \frac{1710}{103} & - \frac{1125}{103} & \frac{2025}{206} & \frac{2025}{206}\\- \frac{1710}{103} & \frac{3078}{103} & \frac{2025}{103} & - \frac{3645}{206} & - \frac{3645}{206}\\- \frac{1125}{103} & \frac{2025}{103} & \frac{1503}{103} & - \frac{1260}{103} & - \frac{1260}{103}\\\frac{2025}{206} & - \frac{3645}{206} & - \frac{1260}{103} & \frac{9999}{824} & \frac{8145}{824}\\\frac{2025}{206} & - \frac{3645}{206} & - \frac{1260}{103} & \frac{8145}{824} & \frac{9999}{824}\end{matrix}\right]$
(check row 3)
Comment: what's wrong with row 3? I saw $5/2$ dominates all off-diagonal entries.
Comment: @Zhanxiong last minute bug.  Thanks.  Now fixed.
Comment: Thank you very much for this counter-example. In fact my problem is even more precise but I don't think there would be a simple answer. My problem is as described above, except that $A^{-1}. 1 \geq 0$ (i.e. the sum of rows is positive). This is not verified by your counter-example where rows 2 and 3 add up to negative numbers.


Jordan blocks, similarity class

Question: Are matrices with the same Jordan blocks similar over $\Bbb C$? This seems to be the case, since say:
$$\begin{array}{cc|c}0&1&0\\0&0&0\\\hline0&0&0 \end{array} \sim \begin{array}{c|cc}0&0&0\\\hline0&0&1\\0&0&0 \end{array}$$
by conjugation from $\begin{bmatrix}0&0&1\\1&0&0\\0&1&0\end{bmatrix}$.

"Any square matrix has a Jordan normal form if the field of coefficients is extended to one containing all the eigenvalues of the matrix. In spite of its name, the normal form for a given M is not entirely unique, as it is a block diagonal matrix formed of Jordan blocks, the order of which is not fixed;" -Wiki

That text is meant to simply mean that the Jordan form doesn't uniquely identify a matrix, without requiring some other rules, such as having eigenvalues listed in decreasing order from the top. But the matrices are still uniquely classified into a similarity class, and if they have the same blocks, they belong to the same class, right?
Answer: Right, if they have the same blocks, then they are similar.

Jordan blocks, similarity class


Font Awesome Icons only printing in Black

Question: I am developing a website that must print a page that contains Font Awesome icons. The issue is that when you go to print, the Font Awesome icons will not print in color.
In the browser they show up in color, but when the page is printed the icons are solid black. Is there anyway to make the Font Awesome icons print in color? Perhaps through CSS with @media print { }?
EDIT: Also, I am developing in firefox.
Answer: It turns out that the item that you actually need colored isn't the i itself but its :before element. Thus:

Answer: If you are using bootstrap you need to edit its CSS since it's specifies black color for "@media print"
Comment: Surely there's a way to do it without modifying bootstrap?
Comment: Thank you. Yes, I am using bootstrap. I should have said that in my question. Thank you for your help!
Answer: I just put together a simple HTML example using font-awesome and it seems to work fine in Chrome and Firefox for me. I see the icon in red onscreen and it also prints in red without any further action. Having said that, creating a separate media CSS is a good idea if your HTML page warrants it, since it can provide a better user experience (onscreen view isn't always ideal for printing).
Are you sure you don't have a printer setting modified to not print in color? Is the printer out of that color and thus defaults back to just plain black? Have you tried it in another browser (Chrome, Safari, Opera, IE)?
Here is the simple code I used for testing:

        <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css">
  <title>Fontawesome Test</title>

    Testing color  fa-camera-retro


TypedFactoryFacility: Passing null argument

Question: Consider the following test:
    [Test]
    public void Create_ServiceWithDynamicDependency_Created()
    {
        // arrange
        IWindsorContainer container = new WindsorContainer();

        container.AddFacility<TypedFactoryFacility>();

        container.Register(Component.For<IServiceFactory>().AsFactory());

        container.Register(Component.For<ServiceWithDynamicDependency>().LifeStyle.Transient);
        container.Register(Component.For<SomethingStatic>().LifeStyle.Transient);

        var factory = container.Resolve<IServiceFactory>();

        // act
        ServiceWithDynamicDependency serviceWithDynamicDependency = factory.Create(null);

        // assert
        Assert.That(serviceWithDynamicDependency, Is.Not.Null);
    }

This fails with the following exception: Could not resolve non-optional dependency for 'Testing.Windsor.Factory.ServiceWithDynamicDependency' (Testing.Windsor.Factory.ServiceWithDynamicDependency). Parameter 'somethingDynamic' type 'System.String'
If I replace the assert part with this:
        ServiceWithDynamicDependency serviceWithDynamicDependency = factory.Create("foo");

The component is resolved as expected. Does anyone know of a workaround for this or do I have to hand-roll factories which accepts null arguments?
Answer: Although you didn't specify it I'm assuming your component has one constructor that looks like this:
public ServiceWithDynamicDependency(string somethingDynamic) {}

The behavior you're observing is not a bug - this is by design, and here's why.
If a value is required an actual value has to be provided, and null is not considered to be a correct value. Otherwise Windsor would just be passing null in for every constructor instead of bothering to find actual dependencies for that.
null is a special value in .NET and it means "no value". So even though you explicitly provide null as an argument Windsor will ignore it.
To solve this, instead of working around be explicit that the value is optional and provide a second constructor that does not include it.
Windsor is smart enough to pick the right constructor and if the value is not provided (or null as in your case) it will pick the no-argument constructor, and when the value is there (is not null) it will pick the one with the value.


Is there a musical term for melodic density?

Question: Is there a musical term for melodic density? By density i simply mean the amount of melodic (as opposed to harmonic) notes within a given time measure.
Example: 40 notes within a bar has higher melodic density than 10 notes within a bar.
(I realize tempo is also a factor for perceived density).
Comment: Compulsory YouTube link https://www.youtube.com/watch?v=AUFwk1Ibwkc
Comment: It feels like the definition misses harmony vs melody, it could be a single 40 note chord.
Comment: I see your point, but i did write *melodic* density. Edited for clarity.
Comment: @Tetsujin There can **never** be too many notes
Comment: I'd suggest starting with a comparison to digital communications. Bandwidth = number of notes per second, maybe? Similar concept to data rate such as kbps.
Answer: Melodic density, while not a "term of art", would be generally understood to mean as you intend.
In 1962, Milton Babbitt coined the term time-point interval, which can be used to describe the linear density of pitch onsets. A time point is the instantaneous moment at which a note begins (its onset). The time-point interval is the distance between onsets. It can be measured according to note-values rather than absolute time, and thus is tempo-independent.
For example, in 4/4 time, a quarter note followed by a quarter rest and then a half note would have a time-point interval (also called interonset interval) of a half note — the time from the initiation of the first quarter note to the initiation of the half note. An eighth note followed by a dotted quarter rest and then a half note would also have a time-point interval of a half note.
Thus, a measure with 40 linear (melodic) onsets would have a higher average interonset interval (call it interonset density) than a measure with 10 linear onsets.


Hibernate: a different object with the same identifier value was already associated with the session

Question: I did see all other similar questions but none of the solutions solves my problem.
I have two classes: Person and PersonSynonym. A person can have many synonyms. The synonyms property in the Person class looks like this:
@OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.ALL)
@JoinColumn(name = "person_id")
private Set<PersonSynonym> synonyms = Sets.newHashSet();

and the PersonSynonym class:
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String name;
... (getters, setters, equals, hashcode, ...)

updating the Person objects works fine if I'm adding only ONE synonym to a person. As soon as I try to add a second synonym and try to update the person object I'm getting this exception:
Caused by: org.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session: [my.package.model.PersonSynonym#6]

the synonym with the id 6 is the one existing already. Apparently hibernate tries to insert the existing one again... why?
Update:
moving the JoinColumn to PersonSynonym doesn't help, Person:
@OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.ALL, mappedBy = "person_id")
private Set<PersonSynonym> synonyms = Sets.newHashSet();

PersonSynonym:
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;

private String name;

private Long person_id;

@ManyToOne
@JoinColumn(name = "person_id", referencedColumnName = "id", insertable = false, updatable = false)
private Person person;

Comment: While it may not get to the root of the issue, have you tried different GenerationType strategies? Does the same error occur if you do?
Comment: I did, no difference.
Answer: The @JoinColumn attribute is on the wrong side. It has to be in the PersonSynonym class with the member (probably personId or getPersonId) which references to the parent class.
Comment: Changing it doesn't help, updating person still triggers an insert. I've updated the question.


Xen PV, Xen HVM and KVM

Question: Am I correct that you can boot up any operating system with either of these virtualization types as they act as a bare metal machine and compile your own kernel?
I am looking to switch over from OpenVZ to a Type-1 (is it?) hypervisor, so more OSes can be booted. I was just wondering if Xen PV, Xen HVM or KVM was a better choice for hosting clients or if it just comes down to personal preference?
I recently bought an E3-1230V3 with 32GB of RAM and maxed it out with a Hardware RAID card for RAID-10 on disks.
So from what I've gathered.

XEN HVM = bare metal. like physical machine
XEN PV = *nix based and kernel compile
KVM = *nix based and kernel compile
Answer: First about differences:
KVM is a kernel based virtual machine. So virtualization becomes a job for the standard kernel of the host. There is no additional hypervisor in between.
XEN is a hypervisor running underneath a ccontrolling VM, called Dom0 (priviledged VM). The hypervisor can run modified, XEN-aware Linux VMs in PV mode. ParaVirtualized VMs have direct, not emulated access to the hypervisor, which can make some operations faster. Fully virtualized VMs use a emulation layer, normally provided by qemu, to access hypervisor resources. You can mix PV and HVM to your liking.
IMO XEN is much more stable than KVM, especilly when running non-linux VMs. But I am a XEN-friend any way.
For personal use I can also recommend Oracle Virtual Box. A user-space virtualization running atop of most common OSes, including Win7, Debian, RedHat.
Answer: The latter. For normal workloads there is nothing you can only do with one or another. If you come from OpenVZ I would suggest KVM since it's architecture is more similar to what you know. Xen is a completely different concept.
Comment: Tim, thanks for your reply. Can you explain or give me some links on how Xen is different over KVM? Thank you very much.
Answer: Xen HVM is the only type of Xen that will support a different OS. KVM will run anything of course.
As for which to choose, that's offtopic on SF.net

Xen PV, Xen HVM and KVM


Text Processing

Question: I have a few folders named folder1 folder2 etc.. under a root folder. In each folder there are many xml files. In each xml file there are number of occurence of text patern  (in case of under folder1) folder1.xxx folder1.yyy . I want to replace all the folder1.xxx with just xxx and folder2.yyy with yyy etc.
I know first I need to get all the folder list and process each files under each folder and replace the texts accordingly. I using Ubuntu linux distro I tried sed but didnt succeed. Could you show me a qucik way to do this no matter whether it perl or bash or any other script. Thanks
Answer: (Edited as per your comment):
for dir in folder1 folder2; do
  sed -i "s/$dir\.//" $dir/*
done

What this will do is for each directory you list (the folder1 folder2 bit) it will do a simple "stream edit" (sed) on every file in that directory and search for the directory name followed by a dot and replace it with nothing.
Comment: Womble xxx is any string (I want to replace "folder1." with "") and folders names are changing script must get all the folder names. Thanks
Comment: especially sed "-i" option what I looked for thanks. You saved my day.
Answer: If you have mmv installed, you might try something like
for dir in *; do
    mmv "$dir/$dir*" "$1";
done

mmv should be part of most distributions, but is not installed by default some cases. I'm not sure if it is in Ubuntu.

Text Processing


Depended CCK Fields

Question: There are two Fields, Image and Video
Users have to use only one field, image or video.
The problem is how do I remove video field if user upload a image ?
is there any custom validation for this ?
Thanks
Answer: The Conditional fields module can do this. It allows to set up conditions and dependencies between multiple files (for example, only show field X if field Y is filled/checked).
Comment: I'm already using Conditional fields but it's only show and hide fields. if user upload image and edit the node and upload a new video both of them is allowed to upload. I need a validation of this.
Comment: I guess the problem here is that images/files are kinda special. Once you've uploaded a file, there is no form field anymore so your conditional field rule does not match anymore. Maybe you can inverse it (Instead of hide the video field if image is filled, change it to *show* the video field as long as the image field is  empty). Untested. You could also ask *that* (how to make a rule that works for this) as a question.
Comment: @kLezer This is a perfect example why you need to put all relevant information in the question and explain more what your problem is. I'd have been able to give you the comment above as an answer to your question initially... With the given information, there was no way for me to know that you are already using conditional fields.
Comment: Thank you Berdir, I'll create a custom module for validation. first trial was successful...
Answer: You have to use hook_form_alter to add your own validate handler to your content type form. For example code View this question.

Depended CCK Fields


Strange Oracle SQLPlus host echo error

Question: I am trying to run a script.  We have 184.108.40.206 installed on most of our machines.  I was asked to upgrade to 22.214.171.124 on some of them.  I upgraded in my Windows environment locally on my Windows 7 box.  Everything worked fine.  Then I was given a virtual with Windows Server 2008 (not R2) 32 bit.  So, I downloaded the 32 bit client.
When I tried to run our script, I was getting some odd errors. I looked into it, and it turns out that SQL*Plus will not pass the environment variables
So, if I do host echo %COMPUTERNAME% I get a response of %COMPUTERNAME%
Any ideas?
Comment: If I type echo %COMPUTERNAME% from the command line, it gives me the name of the computer.  And yes, we're using VMWare, but it's centrally controlled, so I can't administer the box.
Comment: What happens when you use "echo %COMPUTERNAME%" from the command line?  It might be a misconfigured VM. Are you using VMware?
Comment: What happens if you just use "host" by itself? If it does open a command shell, which variables does "set" show?
Answer: I imagine this is because Oracle is having trouble with the %. There's no need to use the HOST command to do this though. You can use the TERMINAL parameter of the USERENV namespace for SYS_CONTEXT instead:
SQL> select SYS_CONTEXT('USERENV', 'TERMINAL') from dual;

SYS_CONTEXT('USERENV','TERMINAL')
---------------------------------------------------------

ELZAR

The documentation describes TERMINAL as:

The operating system identifier for the client of the current session.
  In distributed SQL statements, this attribute returns the identifier
  for your local session. In a distributed environment, this is
  supported only for remote SELECT statements, not for remote INSERT,
  UPDATE, or DELETE operations. (The return length of this parameter may
  vary by operating system.)

If you must use the HOST command I can get round the problem by enclosing in backticks
SQL> host `echo %computername`
ELZAR

Comment: "host echo %COMPUTERNAME%" works fine on my system without quotes.
Comment: It doesn't on mine @Chris or the OPs... The central part of my answer is that there is a way to do this in Oracle, without resorting to external OS commands though.
Comment: I currently have a project that has over 150 SQL scripts in it.  To change each one to have a different call when it is supposed to work, and in fact does on another machine, doesn't work for the environment.  I just need a way to fix my VM that has this problem.
Comment: You didn't put any of that in the question @paul... Personally, I would take the hit and change it as your current method is OS dependent when it needn't be.

Strange Oracle SQLPlus host echo error


Compare dynamic arrays of string angular 6

Question: I have two arrays of strings. One of those will be a dynamic list of checkboxes. I want to check if the item exists in the other array. How can I do it dynamically using angular 6?
This is the current situation of the list of checkboxes...
The checked need to be dynamic...

      <mat-checkbox class="example-margin secondary-text"
        [checked]="false" >p</mat-checkbox>

Comment: What is the **other array**? How does the structure of the `p` looks like?
Comment: @Senal The 'p' looks like

Array(34)
0: "ITEM_1"
1: "ITEM_2"
2: "ITEM_3"
3: "ITEM_4"
...

And the other array has the same structure
Comment: why not to have one array (called people) where object is a  person like this: [ { name: "whatevername", checked: false }, ... ]. If you do have already the arrays and you can not changed them please share as part of the question
Answer: You could do something like this if you want to check the items which are also existing inside the otherArray. This is assuming that these arrays are string arrays as you mentioned.
// In the template

  <mat-checkbox class="example-margin secondary-text"
    [checked]="isInOtherArray(p)" >p</mat-checkbox>

// In the component method
public isInOtherArray(person) {

   return this.otherArray.indexOf(person) > -1;
}

Or you could directly use it on your template

  <mat-checkbox class="example-margin secondary-text"
    [checked]="otherArray.indexOf(p) > -1" >p</mat-checkbox>


Coded UI : How to skip a test case

Question: Let say I have multiple test case: TC1, TC2, TC3, TC4
If TC2 failed, I wanna skip TC3 and move on to TC4 instead.
I know there is 'Continue After Failure' in Ordered Test, but that function will just continue to TC3.
Thanks.
Answer: you can organize one ordered test inside another. Combine TC2 and TC3 in one ordered test and set it NOT to continue on failure. Then pack this ordered test inside your bigger one, which should continue on failure. Your main ordered test will look like:

TC1
OrderedTest-TC2-TC3
TC4

Coded UI : How to skip a test case


How do you sample from a multivariate distribution with conditions?

Question: First, I need to sample 10,000 vectors $Y_{i}=\begin{bmatrix}Y_{1i} & Y_{2i} & Y_{3i}\end{bmatrix}^{T}$ that are multivariate normal $N(\mu,\Sigma)$ with $\mu = \textbf{0}$ and
$$\Sigma = \begin{bmatrix}1 & 4.5 & 9\\ 4.5 & 25 & 49\\ 9 & 49 & 100\end{bmatrix}.$$
I don't need help here. I know that you can draw iid standard normal vecotrs $Z_{i} = \begin{bmatrix}Z_{1i} & Z_{2i} & Z_{3i}\end{bmatrix}^{T}$ and set $Y_{i} = CZ_{i}$ where $C = \Sigma^{\frac{1}{2}}$.
My question is, how do you sample 10,000 variables $Y_{1i}$ from the distribution $P(Y_{1i} \mid Y_{2i} = Y_{3i} = 1)$?
I'm drawing the samples in R, but I'm more interested in the theory than the code. I don't know how to approach this.
Comment: $Y_{1i}$ given $(Y_{2i},Y_{3i})$ has a univariate normal distribution.
Answer: Given that $Y_{i}=\begin{bmatrix}Y_{1i} & Y_{2i} & Y_{3i}\end{bmatrix}^{T}$ has a multivariate distribution, $Y_{1i}$ is also distributed normally (this is by definition).
Now, when you condition on $Y_{2i}=Y_{3i}=1$, you can think of it as if you have "observed" the values for $Y_{2i}$ and $Y_{3i}$ and you just take into account that they are now static in value when looking at the probability distribution of $Y_{1i}$.
To think of it geometrically I think it's easier explained in a 3-d space. Imagine $X=(Z_1,Z_2)$ is bivariate normal. Suppose each $Z_i$ has a standard normal distribution. They will have the following graph (ignore the values on the axes):

Imagine now that you "observe" that $Z_1=0.5$. You now fix $Z_1$ at $0.5$ and slice the graph so you see what values does the density take when you consider the values of $Z_2$. Here's how this would approximately look like (ignore the values on the axes):

This picture shows what's the distribution of both of the variables, but the idea is the same.
Answer: It turns out, the answer to my question was on Wikipedia all along. For the trivariate case, you can partition $X$ into $X_{1}$ and $\begin{bmatrix}X_{2} & X_{3}\end{bmatrix}^{T}$ then use the formulas below to solve for $E[X_{1} \mid X_{2}=X_{3}=1]$ and $Var(X_{1} \mid X_{2}=X_{3}=1)$. And then you can sample from a one-dimensional normal distribution with those mean and variance.


How to filter journalctl output by process name?

Question: When a process logs to systemd, its process name is recorded and displayed as identifier in the journalctl output.
$ systemd-cat echo "test"
$ journalctl -f
-- Logs begin at Sat 2018-02-24 12:13:24 CET. --
...
Jul 25 13:52:26 mycomputer echo[25098]: test

Filtering on that name, however, doesn't seem to work.
$ journalctl -f -t echo
-- Logs begin at Sat 2018-02-24 12:13:24 CET. --
$ journalctl -f -u echo
-- Logs begin at Sat 2018-02-24 12:13:24 CET. --

When I manually specify an identifier, the identifier is shown instead of the process name and filtering works.
$ systemd-cat -t myapp echo "test"
$ journalctl -f
-- Logs begin at Sat 2018-02-24 12:13:24 CET. --
Jul 25 13:56:08 mycomputer myapp[25213]: test
$ journalctl -f -t myapp
-- Logs begin at Sat 2018-02-24 12:13:24 CET. --
Jul 25 13:56:08 mycomputer myapp[25213]: test

How can I filter journalctl output on process name?
Answer: Just figured this out. You can use any systemd journal field as filter by specifying <FIELD_NAME>=<VALUE>.
The following fields are useful for this question:

_COMM=, _EXE=, _CMDLINE=
The name, the executable path, and the command line of the process the journal entry originates from.

So in order to filter on the command name, use journalctl -f _COMM=<command-name>
$ journalctl -f _COMM=echo
-- Logs begin at Sat 2018-02-24 12:13:24 CET. --
Jul 25 13:52:26 mycomputer echo[25098]: test
Jul 25 13:56:08 mycomputer myapp[25213]: test


How can i find firewall rules and settings via command line in linux

Question: I'm writing a script to find all the firewall details, routing information, and important network related via command line. I can fetch the firewall configuration with iptables -nvL, but I'm not sure about the rest. What commands should I use to gather everything else?
Answer: You can get the interface status by:
ifconfig

Or even better, use the "ip" command, for example:
ip addr

To get the route info, use one of these:
route
route -n #no ptr resolving
ip route

For firewall rules, use:
iptables -L # you can add -n to prevent name resolving

For IPv6 you can add "-6" to "ip" commands:
ip -6 addr
ip -6 route
...

You can also check "ethtool" to get link status and speed/duplex:
ethtool eth0

Answer: routel

List lots of information, albeit in a not really readable form, but it is a full dump of all routing tables of the host, and gives enough information to reconstitute the IP configuration on the host.  This information is not complete enough to reconstitute policy routing. If you happen to use that, you also need ip rule. Anyway, as an example, for IPv4 (even if IPv6 is also listed):
# list all interfaces addresses (unicast/broadcast/anycast)
routel | grep "local"
# list all (except ipv6 link-local) addresses of interfaces
routel | grep "local" | grep "host"
# list all broadcast addresses of interfaces
routel | grep "local" | grep "link"
# list all routing entries
routel | grep -v "local" | grep -v "unspec"
# list all on-link routing entries
routel | grep -v "local" | grep -v "unspec" | grep "link"
# list all on-link routing entries that are added by the kernel,
# (which also matches the netmask of the IP addresses).
routel | grep -v "local" | grep -v "unspec" | grep "link" | grep "kernel"
# list non-on-link routes (aka gatewayed or routed)
routel | grep -v "local" | grep -v "unspec" | grep -v "link"

Contrary to ip route or the 1999's route, routel list all routing tables, while the other only list the main table. If someone uses policy routing, you will get no clues when using ip route, while routel will spell you the name of routing tables.
Also, you may want to dump the sysctl knobs, to know if someone enabled forwarding, ARP proxying or some other funky stuff. Just dump sysctl -a
If you want more complete flags or interface informations, also dump ip addr. Really special configuration might need more, like ip link or ip tunnel.
As for firewall configuration, there is iptables-save or iptables-save -c if you also want counters. This will list all tables (and not only filter like iptables -L or iptables -S) You can even use iptables-restore that takes the output of iptables-save and restore its configuration.
Comment: ...just dont try the "routef" command (mentioned at the end of the routel man page) on a remote machine :) (don't ask which idiot did it just now, without reading what it does) ... :)
Comment: @mulaz: `routel` works as a user, routef does not ;)
Comment: Awesome !! thanks a lot :-) It is really helpful :-)
Answer: The command to print existing iptables rules is like this:

iptables -L
Comment: Be sure to add `-v -n` as well if you want them all (chains, nats etc.)
Answer: You can see if there is CSF Firewall installed.

/etc/csf/csf.conf
Comment: I have tried in different linux boxes, i couldn't able to see this file itself. Can you share which OS may have this ?


What is the correct term for "front swing" or "front shuffle"?

Question: Swing and shuffle are generally defined as moving the off-beat closer to the end of the measure.  What's the reverse of this?
What is it called when the off/up beat is moved closer to the beginning of the measure?
Answer: I often use the word "anticipating" when talking about this, e.g. "The Viennese waltz anticipates the 2nd beat slightly, placing it a bit early." In less formal circles I might also call it "pushing" the beat.
Note, I'm looking here for terms that (like swing and shuffle) try to describe a slight offset that doesn't neatly align with a rhythmic value. There's also a phenomenon in many works in which major motion happens earlier than "expected," like chord changes every measure but on the "and of 4" rather than the downbeat, like in this song that I can't get out of my head lately from Matilda: The Musical:

For this phenomenon, I'd definitely use the word "anticipation," e.g. "The chord changes are anticipated by half a beat," as long as it's clear that we're not using the meaning of anticipation related to non-harmonic tones in a melodic line.
Comment: (...and as a result, "Hotel California" has been stuck in my head too...)
Comment: Which part of Hotel California is a result of Revolting Kids?  I've always thought of it (and played it) straight.
Comment: @Tim The chord progression of the main "We are revolting children" section. The same "chromatically descending bass line underpins something that 'fills in the gaps' of an Andalusian cadence." https://youtu.be/nnicGKX3lvM
Answer: The general term for swing with the short note on the beat is reverse swing.

A particular type of reverse swing is the Scotch snap (AKA Lombardic rhythm), which is characterised by beat subdivision in this form:

The Scotch snap is quite common in Scottish music such as Strathspeys, but it is also somewhat popular in hip-hop.
Comment: Do you have a source for "reverse swing"? It's not a term I'm familiar with.
Comment: @Aaron I haven't found an "official" source using the term *reverse swing*, but there are lots of unofficial sources that use it, such as:
[Advanced Mathematics 4: Notes inégales](https://canthisevenbecalledmusic.com/advanced-mathematics-4-notes-inegales/),
[Swing Metronome](https://www.swingmetronome.com),
[String Theory - I Got Rhythm, Part 18: Triplets and Swing Eighth Notes](https://youtu.be/5YToM31fLyU?t=448),
[Microtiming: Degrees of Swing, Reverse Swing, and Compound Swing](https://jackashley.co.uk/2020/08/12/microtiming/)
Comment: Fair enough. Nice research.


Determinant of orthogonal matrix

Question: If $A$ is orthogonal. how do I show that $\det(A-2I)\not=0$.
I tried writing $A-2I=A-2AA^T=A(I-2A^T)=A(A^TA-2A^T)=AA^T(A-2I)$
but it seems that I am just doing loops after loops.
Comment: If $\det(A-2I)=0$, what can you say about the eigenvalues of A?
Comment: $\lambda=2$ is an eigenvalue.
Comment: That is right, so now see if you can show that A cannot have 2 as an eigenvalue.
Answer: If $2$ is an eigenvalue, then $Ax = 2x$ for some $x\in \Bbb R^n \setminus \{0\}$. Then, $\|x\|^2 = (x,x) = (A^TAx, x) = (Ax, Ax) = (2x,2x) = 4(x,x) = 4\|x\|^2$. Contradiction.
Comment: Also prove that any eigenvalue $\lambda$ of $A$ must salsify $|\lambda| = 1$
Comment: Why $(A^TAx, x) = (Ax, Ax)$?
Comment: @EugenCovaci Of course I am assuming that the setting takes place in $\Bbb R^n$. We have for any matrix$A$ and vector $x$, $(Ax,x) = (x,A^Tx)$

Determinant of orthogonal matrix


Erro ao executar teste em Python 3.6.7

Question: Esta dando erro ao executar a def teste() para testar a def questao().
Porque estaria dando erro no retorno da função teste?
def questao():
    lista_nomes = []
    nome = ''
    x = 0
    reader = csv.DictReader(arquivo)

    for linha in reader:
        nome = linha['full_name']
        lista_nomes.append(nome)
        x += 1

        if  x == 20:
            break

    return lista_nomes

Teste
def teste():
    r = questao()
    assert (
        isinstance(r, list) and
        all(isinstance(y, str) for y in r)
    )

Erro abaixo:
C:\Users\perei\devpy\python-1\teste.py:25: AssertionError: assert [] == ['C. Ronaldo'..., 'Robert Pererira', ...]

Comment: Pelo que parece você tem dois tipos de teste, o primeiro para saber se `r` é uma lista e outra para saber se cada elemento da lista tem _strings_. Eu separaria estes testes, algo como `assert isinstance(r, list) == True` e `assert all(isinstance(y, str) for y in r) == True` pois são validações distintas.
Answer: Você tem certeza que essa falha do teste é dessa função de teste mesmo?
Por que ela não compara isso - e a mensagem de erro do assert mostra de fato uma lista populada com os nomes lidos - que deve ser o retorno correto da função questao.
Voce está usando pytest, certo? (O pytest modifica o assert do Python para dar mensagens de erro com os valores comparados - o assert sozinho não mostraria os dois valores que estão sendo comparados).
Na saida completa que você tem no terminal deve ter o nome e o ponto certinho da função de teste que está falhando - possivelmente é uma outra.
Se não for, por favor, atualize a sua pergunta com o código do modulo da sua função de teste -inclusive com as importações. E confirme se está usando o  pytest ou outra biblioteca terceira para rodar os testes.
Comment: Desculpe, mas não tenho acesso ao arquivo de teste.
Tem alguma diferença utilizar ou não o virtualenv para fazer os testes?
Comment: Não há diferença alguma em usar ou não o virtualenv para os testes.

Erro ao executar teste em Python 3.6.7


Automated test Python/selenium gspread

Question: For example, to view a Google Sheets document I can use gspread:
import gspread

gc = gspread.login("your login", "your password")
wks = gc.open("document name").worksheet("sheet name")

cellValue = wks.acell("A1").value
print "Cell 'A1' has a value of %s" % cellValue

Is there a way I can access Google Sheets and try to edit it in an automated test using Python and Selenium?
Comment: please clarify " access google sheets and try to edit it in an automated test using python and selenium "
Answer: https://developers.google.com/people/quickstart/python

Go to the above link and create a project by following the steps given with your google account.
From the above link you will download OAuth client ID which will give you client_secret.json.
In this example OAuth client ID is not used and it will be necessary while using other api.
In addtion to above you have to download Service Account Key which will be below OAuth client ID. Download it as json.
The path to Service Account Key has to be mentioned in the fourth line of the following code.

Code:
import gspread
from oauth2client.service_account import ServiceAccountCredentials
#scope is necessary if we are accesing gdrive
scope = ['https://spreadsheets.google.com/feeds','https://www.googleapis.com/auth/drive']
credentials = ServiceAccountCredentials.from_json_keyfile_name('/home/username/Downloads/MyProject.json', scope)
gc = gspread.authorize(credentials)

#to create a new gsheet
sh = gc.create('Sheet_1')

#to open existing gsheet
sh = gc.open('LYKE')

# to share to other users
sh.share('email@example.com', perm_type='user', role='writer')

#to create a worksheet in the gsheet
worksheet = sh.add_worksheet(title='title', rows="500", cols="50")

#Define a cell range and update cells
cell_list = worksheet.range('A1:A10')
for i,cell in enumerate(cell_list):
    cell.value=i
worksheet.update_cells(cell_list)

Automated test Python/selenium gspread


Conditional probability, Bernouilli's trial alternative method..

Question: So I got this question on an practice test, and my first thought was to use Bernouilli's trial to solve for the answer. My question is how to possibly solve it without Bernouilli's trial, in a manner simply using conditional probabilities. I don't really know how to approach it in that case, however I figure it has to be extremely simple.
Customers at a store independently decide whether to pay by credit card or with cash. Suppose the probability is 70% that a customer pays by card. Find the probability that 3 out of 5 customers pay by card.
Comment: You don't need to use conditional probabilities, because every customer pays by card with probability 0.7, *regardless* of what the other customers are doing.
Comment: Thank you, that makes sense, it was under the "conditional probabilities" section of questions which is what was misleading me.
Comment: That's probably because you need to understand what the "independently" in the question meant, and that comes under the topic of conditional probability.
Answer: $p = 0.7$, and $q = 1 - p = 1 - 0.7 = 0.3$. Thus:
$P(x = 3) = \binom{5}{3}(0.7)^3\cdot (0.3)^2 = 0.3087 \approx 0.3$


Baroque ornamentation is not consistent across editions of the same piece

Question: Here are two versions of the Quodlibet from Bach's Goldberg Variations. As you can see in the attachments,  the ornaments and the recommendations as to how to play them are exactly opposite. One shows playing the trill (or shake) starting on the note above, while the other shows starting from below.
Why are both correct?
If I have a piece that does not show the annotation, how would I know which way to play the ornament?
Answer:
Why are both correct?

Maybe they're not.
In general, in everything, when someone tells you something, it pays to consider who is telling you that thing and why.  Does the person have an agenda that might influence what they're saying or how they say it?  What are the person's beliefs, and do you agree with those beliefs?
The same is true of music editors. If the edition include a section of critical notes explaining the editor's choices, read it.  Perhaps the editor includes a discussion of this very question.  Consider when the edition was prepared and when the editor was educated: what was the prevalent understanding of baroque ornamentation at the time?  Does today's practice agree with it?
In particular, the practice of beginning a trill on the upper note fell out of fashion for a couple of centuries, so I suspect (also from the engraving) that the Russian edition is rather older and predates the revival of the fashion for using upper-note trills when performing music from the 17th and 18th centuries.  The difference might also be geographical rather than temporal.  That is, the editions could be roughly the same age but reflect a difference in opinion between Russia and Western Europe about the best way to play baroque music in general or Bach's keyboard music in particular.
Another possibility is that the two editions represent different schools of thought that have coexisted (and indeed still coexist) throughout the world.  You can certainly find recordings of this piece with both forms of trill on the internet today.  Do your research and make your choice.
Answer: Often, Baroque composers left out ornaments so these may be the choice made by each editor. In addition, ornamental notation wasn't standard so different editors may interpret an ornament differently.
In addition, Baroque key signatures for minor keys were not consistent. Often one flat was left out or even one sharp.
Comment: Baroque musicians rarely dealt with editors. Music publishing was uncommon. However, they did leave the ornamentation to the performer.
Comment: The incomplete key signatures are addressed in [that question](https://music.stackexchange.com/questions/50938/why-is-the-key-signature-not-the-same-as-the-listed-key).
Answer: To collect some facts:

In baroque every player was prepared to supply his or her own ornaments. So they were either left out entirely, or reduced to a single + to suggest a place for some.
Especially for J. S: Bach this does not apply, however, since he provided a pretty detailed translation table (see Wikipedia, section baroque music or below)
Today few amateurs have the background knowledge to supplement ornamentation, let alone on the fly. So editors jump in suggesting reasonable choices. Unfortunately this heavily depends on the state of research and the willingness to reflect it, parameters significantly changing over the centuries This is the reason for the different examples.
Currently there are urtext editions, giving the pure intentions of the composer (indispensable for any professional preparing a recording or concert) and edited ones; modern editions clearly indicate, what was added by the editor, so the performer has the choice to deviate from it.
Comment: Regarding the Wikipedia link you provided: I suggest that you copy and paste the picture of that ornament table as that gives a clear answer to the question, and the answer should be posted here and not on an external page. External pages can certainly be linked to as further reference, but as I said the answer to the question should appear here.


Convergence in probability of a product of RVs

Question: I have found the following very useful theorem and I would appreciate some help comprehending it fully.
Theorem  Let $\{X_n \} $ be a sequence of random variables bounded in probability and let $ \{Y_n \} $ be a sequence of RVs which converge to $0$ in probability. Then
$$X_n Y_n \rightarrow 0 \quad \text{in probability} $$
And its proof:
For a given $\epsilon$ since $X_n$ is bounded in probability we can choose $N$ and a constant $B$ such that
$$n \geq N \Rightarrow P \left[ |X_n| \leq B \right] \geq 1-\epsilon$$
Then
$$ \overline{\lim_{n \to \infty}}P \left[|X_n Y_n| \geq \epsilon \right] \leq \overline{\lim_{n \to \infty}} P \left[ |X_n Y_n| \geq \epsilon, |X_n| \leq B \right]+ \overline{\lim_{n \to \infty}}P \left[|X_n Y_n| \geq \epsilon, |X_n| > B \right] \leq \overline{\lim_{n \to \infty}} P\left[|Y_n| \geq \epsilon/B \right] +\epsilon=\epsilon  $$
If someone could help me understand how the last inequality is derived, given the information we have, I would be grateful. I understand the second to last inequality comes from the subadditivity property of limitsuperior so I merely need to comprehend the last one. Thank you in advance.
EDIT: I think it is because $$P(A) \times P(B|A) \leq P(A) $$ and the complement of the event that is implied by the bound in probability.
Comment: Could you provide a reference for this theorem?
Comment: I think it is still correct if we replace "in probability" by "a.s." in those to places.
Comment: The theorem appears to state
$$O_p(1)\cdot o_p(1) = o_p(1)$$. This is a rather old result, for example in White's Asymptotic Theory for Econometricians (1984), it is stated as an exercise.
Answer: First you should show that: $$P[|X_n Y_n| \geq \epsilon,|X_n|\leq B]\leq P[|Y_n|\geq \epsilon/B]$$ Hint: If $A\subset B$ then $P(A)\leq P(B)$.
For the 2nd part, again by using above property, you can say that: $$P \left[|X_n Y_n| \geq \epsilon, |X_n| > B \right]\leq P[|X_n|>B]$$ Now try to use $P[|X_n|\leq B]\geq 1-\epsilon$.
Comment: For the second case, that is easy to see so no problems there. The problem is for the first case, because we are also replacing $X_n$ with its upper bound. I find that somewhat hard to digest. I'll think about it some more. Thank you.
Comment: Could you provide a reference for this theorem?
Comment: Is it correct if we assume that $Y_n$ converges a.s. to 0 then the $X_nY_n$ converges a.s. to 0?


in C, Reading a text file into an array of structs then writing a new text file with the array

Question: I am fairly new to C and programming in general so I am unsure of my problem. Whenever i run the code and open the new output file, the struct data is only zeros. Here is my code.... I have put the text file contents at the bottom of the question. Thank you in advance.
#include <stdio.h>
#include <stdlib.h>

typedef struct {
int id;
char description[40];
double mancost;
double sellprice;
} ItemInfo;

int readInfo( ItemInfo input[] );
void listData(ItemInfo input[], int i);

int main(){

ItemInfo input [200];
int i;
i = readInfo( input );
listData (input, i) ;
return 0;
}

int readInfo( ItemInfo input[] ) {
FILE *fin;
int i = 0;

fin = fopen( "C:/Users/ecrooks98/CLionProjects/untitled5/p6-initial.txt", "r" );

if( fin == NULL ) {
    printf( "File cannot be opened\n" );
}
else {
    while( !feof(fin)) {
        fscanf( fin, "%d\t", &input[i].id);
        fscanf( fin, "%39[^\t\n]s\t", input[i].description);
        fscanf( fin, "%lf\t", &input[i].mancost);
        fscanf( fin, "%lf\n", &input[i].sellprice);
        i++;
    }

    fclose( fin );
}

return i;
}
void listData(ItemInfo input [], int i) {
FILE *fout;
int j;

fout = fopen("output.txt", "w");

for (j = 0; j < i; j++) {

        fprintf(fout, "%d\t", input[i].id);
        fprintf(fout, "%s\t", input[i].description);
        fprintf(fout, "%lf\t", input[i].mancost);
        fprintf(fout, "%lf\n", input[i].sellprice);
    }
}

My Text file i am inputting also looks like this
 2250 shoes 3.00 80.00
 2251 hat .50 20.00
 2252 iphone .10 600.00
 2253 grill 80.00 100.00

Comment: Besides the other problems, in your `listData ` function, you need to print `input[j].id` and the rest, not `[i]`.
Answer: You should check the return value of fscanf() to make sure, that your conversions were successful.
When using the bracket matcher %39[^\t\n] the trailing s is not needed and seen as separate match character. If your input consists just of a single word, it is better to just use %39s, otherwise use  %39[^\t].
Furthermore, there is no need to split it into four fscanf()-calls, you can make it all in one. And you shouldn't try to match the exact type of blank, a simple space should do the trick. This leads to:
int st;

do {
    st = fscanf(fin, "%d %39s %lf %lf ",
                 &input[i].id,
                  input[i].description,
                 &input[i].mancost,
                 &input[i].sellprice);
    i++;
} while (st == 4);

This should about do what you need.
Furthermore, as SteveSummit pointed out in the comments, you have to output input[j].* in your listData()-function, since j is the counter variable of your loop (good catch, Steve!).


How do I speed up `drush test-run` (compared to the SimpleTest UI)?

Question: I have a number of Feature modules for which I've written tests to assure they work as expected. For performance reasons, I've overridden the parent's setUp() method in each test case so that it doesn't create a brand new database each time.
When I run my tests in the SimpleTest admin UI, all of my Feature modules' tests take about 15-20 seconds total to complete. When I run them via drush test-run [feature-group-name], they take about 75 seconds total. I noticed that drush seems to pause in between test cases to output something like the following:
Contact page 7 passes, 0 fails, and 0 exceptions  [ok]
No leftover tables to remove.                     [status]
No temporary directories to remove.               [status]
Removed 1 test result.                            [status]
 Group  Class  Name

If you're curious, my setUp() function looks like this:
public function setUp() {
  $this->setup = TRUE;
  // Prevent PHP warnings about a missing public_files_directory variable:
  $this->public_files_directory = variable_get('file_public_path');
}

Any ideas why drush is so much slower than the admin UI? I'd really like to get drush more performant, as I want to integrate these tests into a continuous integration environment.
Thanks!
Answer: Consider using "php scripts/run-tests.sh" instead of "drush run-test". The former supports parallel test execution which improves runtime (as long as you have multiple test-classes). Compare:
$ php scripts/run-tests.sh --url http://d45.l User
...
Test run duration: 11 min 45 sec
$ php scripts/run-tests.sh --url http://d45.l --concurrency 3 User
...
Test run duration: 4 min 13 sec
$ php scripts/run-tests.sh --url http://d45.l --concurrency 6 User
...
Test run duration: 3 min 7 sec


Interpreting the entropy of a Dirichlet distribution

Question: I was looking for a measure to interpret the "spikiness" of categorical histograms. So, if it becomes unnaturally skewed towards a certain value at a given time, I want a metric that will show some kind of a spike at that time. I considered a variety of metrics for this purpose and finally settled on the Entropy of a Dirichlet distribution (considering the histogram of counts as a sample from a Dirichlet and using the corresponding Entropy as my metric). For this, I used the formula for Entropy in the Wikipedia article .
$$H = logB(\alpha) + (\alpha_0 - K)\psi(\alpha_0) - \sum(\alpha_i-1)\psi(\alpha_i) $$
Here, $\alpha$ is the vector of counts in various categorical bins of the histogram, $\alpha_0 = \sum\alpha_i$, $B$ is the multivariate beta function, $\psi$ is the digamma function. I implemented this in C# (implementation pasted below) and am having some problem with interpreting the results. I would expect an $\alpha$ with a flat distribution (uniformly spread across its categories) to have a higher entropy than one that is spiked towards a given category and this holds true. The gap in interpretation arises with various kinds of flat distributions. My expectation would be that an alpha described by the array {x,x,x} would have increasingly higher values of entropy as x increased. The reason for this belief is that a higher sample would mean we are increasingly sure that the distribution is flat and should increase the Entropy. What I see in practice is this -
x: 0.1, Entropy: -13.025
x: 0.6, Entropy: -4.82
x: 1.0, Entropy: -0.693
x: 1.6, Entropy: -0.8164
x: 2.1, Entropy: -0.967
As you can see, there seems to be a maxima at x=1. This goes against my intuition. Can anyone help me interpret this or let me know if these results are rubbish.
<code>private static double regularizer = 0.1;
    /// <summary>
    /// Based on the formula for total information entropy given here - https://en.wikipedia.org/wiki/Dirichlet_distribution.
    /// </summary>
    /// <param name="alpha">alpha: The parameters of the Dirichlet distribution. These correspond to a histogram with counts.</param>
    /// <returns>The Entropy of a Dirichlet distribution.</returns>
    public double Entropy(double[] alpha)
    {
        _2_gammafamily g = new _2_gammafamily();
        double alpha_0 = 0, H = 0;//The sum of coefficients (normalizing factor) and final entropy term respectively.
        int K = alpha.Length;
        for (int i = 0; i < K; i++)
        {
            alpha[i] += regularizer;//Before doing anything else, we regularize the parameters which is equivalent to a uniform prior.
            alpha_0 += alpha[i];
            H += g.Gammaln(alpha[i]);//Positive part of normalization constant (which is the log of a multivariate beta distribution).
            H -= (alpha[i] - 1) * g.Digamma(alpha[i]); //The contribution from each of the alphas.
        }
        H -= g.Gammaln(alpha_0);//Negative part of normalization constant.
        H += (alpha_0 - K) * g.Digamma(alpha_0);//The contribution from the normalizing factor.
        return H;
    }
</code>
Comment: "Wrong" is such a strong word
https://en.wikipedia.org/wiki/Differential_entropy
Comment: That formula is wrong. Entropy can't be negative.
Answer: In case people are interested and for my own reference, I found the answer in the Wikipedia article on Dirichlet distributions. As we increase the parameter $\alpha$ of a symmetric Dirichlet distribution, we start getting samples that are more likely to be all equal to each other. For a 3-d Dirichlet for example, we will be more likely to see (.33,.33,.33). Since this becomes more likely, we can predict with larger certainty what the sample is going to look like. Hence the reduction in Entropy. When $\alpha$ is below 1, we are more likely to see sparser samples (where most are zero or very small). In these cases as well, the Entropy becomes lower. The case of $\alpha = 1$ most closely resembles white noise and so, the Entropy is highest for this case.


Starting a mysql server in my ${HOME}?

Question: I'm not root on my server (linux) and this server doesn't currently run a mysql server as a daemon.
Can I start a mysql daemon that would only work for my account and where the data/files would be stored somewhere in my ${HOME}/... ? How can it be done  ?
Thanks,
Pierre
Answer: You should definitely try MySQL Sandbox :
To run a simple MySQL service in your $HOME, you just need to :

Download a MySQL binary tarball from MySQL's website
run "make_sandbox  /path/to/tarball.tar.gz"

You will get a complete MySQL server running in the directory of your choice.
The full documentation is there.
Answer: Most probably your option would be to compile from source and setting the install directories to be in your home. And then you need to choose some port above 1024, as by default normal (non-root) users can not open ports below 1024.
Answer: To configure to compile in your directory:
<code>./configure --prefix=/path/to/my/dir
make && make install
</code>
I believe you can specify your my.cnf file under your home directory.
/path/to/mysql/bin/mysqld_safe &
I haven't needed to do this, but it should work.
Answer: As Sunny stated, using the normal ports will not be an option for you but it should run otherwise.  For a non-root user, though, it would almost always be better to use a DB product like SQLite which is designed to be used in this way.
Answer: You can take the generic tar.gz/tar.bz2 binaries from the MySQL site and set at least the following options either using a my.cnf (which has to be specified with --defaults-file to mysqld) or directly as command line parameter when starting the mysqld:
<code>--with-datadir - this configures where MySQL will store the database files
--socket       - Where to put the unix domain socket file
</code>
in a my.cnf you might also set the listen option.
Please refer to the documentation on http://dev.mysql.com/doc/refman/5.1/en/server-options.html for details.

Starting a mysql server in my ${HOME}?


Memory warning message

Question: I keep getting this error message:

Warning: require_once(): Unable to allocate memory for pool. in Database::loadDriverFile() (line 1742 of /home/site/public_html/includes/database/database.inc).

I am not sure what the problem might be. It started when I installed Varnish 3.0; I couldn't get it to work, so I reverted back to my previous configuration with only APC. Now, I keep getting the message, and it is also visible to anonymous users. I have disabled the system help block from all themes, and turned off error reporting, but it still appears.
I also have mod-pagespeed enabled.
Answer: Seems to definitely be an APC issue, there's a post about it on StackOverflow which links through to a bug report on PHP.net for precisely this issue.
Comment: @acouch You're not being dense, I linked to that post as a whole but someone else edited these settings into my answer. According to the upvotes on the other answer (not the one that was accepted) in that question setting `apc.shm_size=x` and `apc.ttl=x` is what fixes the problem so kudos to you :)
Comment: hey Clive: I might be being dense, but what is the prescribed solution for Linux boxes? Just to specify one of the mmap solutions above?
Answer: When we've had that same type of warning, increasing the APC memory limit has helped. Try adding
<code>; APC Memory limit size
apc.shm_size=64
</code>
to the php.ini file and restarting apache. And obviously try increasing if that doesn't solve it.

Memory warning message


Counting Ordered Pairs

Question: Let $A$ be a finite set with $n \geq 4$ elements and let $\rho$ be an equivalence relation on $A$. Suppose that there are exactly $4$ equivalence classes, $C_1$, $C_2$, $C_3$, $C_4$. Moreover we know that $|C_1| = |C_2| = 1$. Let $a\in A$ be a fixed element that we know is in $C_3$.
What is the maximum number of ordered pairs of $(x, y) \in \rho$ in which a can occur (meaning $a = x$ or $a = y$ or $a = x = y$)?
Comment: This is what I currently have: In order to maximize the number of pairs that contain $a$, $|C_4|$ must also be equal to 1. If $n$ = 4, there is only one possible pair that contains $a$, which is ($a, a$). When $n$ = 5, you can add the fifth element, say $b$, to [$a$]. This would add the pairs ($a,b$) and ($b,a$). Therefore, each time you are adding two more possible pairs containing $a$. The answer ends up being $2n - 7$.
Comment: +1 for showing your thoughts on the question.  It makes it much easier to give a(n |I hope) useful answer.
Answer: Hint:  You are correct that to maximize the number of pairs you should have $|C_4|=1$, which gives $|C_3|=n-3$  But when you add the third element to $C_3$ you add more than two ordered pairs.  $c$ can be paired with each of $a$ and $b$ in either order, so you add four pairs.  Can you determine what happens for larger $n$?
Comment: I thought $a$ had to be in the pair. Therefore, it only adds two more pairs each time an element is added to $C_3$.

Counting Ordered Pairs


Percent deviation for linear regression

Question: I have 2 sets of experimental data to which I applied a linear fit using Matlab. I can use the slope value to compare between both of them.
My question is: can I use the following percent deviation equation for the linear fit to compare between these 2 sets?
$$ \%\text{ deviation} = \frac{\text{last point - first point}}{\text{last point}}\times 100$$
The first point of the linear fitting when (x=0), the last point where my data end (x=311 in this example)

In this example the slope is -0.01569 and the %deviation is -4.063%.
Comment: You can do whatever you want, but maybe better tell us what are you trying to achieve with this..? If you want to compare the slopes, why not just do this?
Answer: The results are different from the slope results.
Example:
Let's say your sets both consist of 2 points.
Set a: y1=0 y2=1 Set b:y1=4 y2=5
Slopes are both equal to 1. But a%=100% b%=20%
Edit: Another problem arises if the two sets are not of equal length.
Comment: @MustafaMahmood  The slope shows the absolute increase per unit of the x-axis. The %deviation shows the percentual increase per unit of the x-axis.
Comment: Thanks for your answer, The linear regression slope also have the same problem if the lengths are not equal. in my opinion (I don't know if it's correct or not) the %deviation shows how much the %change between the start condition and the last condition. if your presented data are representing 2 cars speed then the first one has changed 100% from 0 speed to 1, while the second car changed speed by only 20% from 4 to 5, the slope shows the increasing or decreasing while the %change shows the change from the start condition. (forgive my bad English).

Percent deviation for linear regression


Python3: how to slice a list to two part in a loop, one is 20%, the left is another part

Question: Hi I want to slice a large list data like the following list, one part is 20%(consecutive) part of the list, another is left. I just know to get the 20% part but how to get the left part of list in a loop. Thanks my code is following.
for example, when X1 = [ '1', '2'], X2 = ['3','4','5','6', 7','8','9','10']. then
X1 = ['2','3'], and X2 =  ['1','4','5','6', 7','8','9','10'], .........
X = ['1','2','3','4','5','6', 7','8','9','10']
index = 0
for i in range(9):
     X1 = X[i:(i+2)]
     X2 = X[]

Answer: You can fetch the second part by telling from what index you would like to start untill the end.
X = ['1','2','3','4','5','7','8','9','10']
index = 0
for i in range(9):
     X1 = X[i:(i+2)]
     X2 = X[:i] + X[(i+2):]

Comment: thanks for your comments, sorry for my missing some information about my question, and I have updated my questions
Comment: @tktktk0711 , Now it's more clear :) In that case you can do what I did in my updated answer.
Answer: I would like to offer a similar, but more robust solution. You say you need to get 20% of the large list. While slice size of 2 works for a list with 9 members, you will obviously need to increase it for a larger list. If you do not know how many members a list will have, it can be particularly tedious and error prone. Hence, I would suggest something like this:
import math

X = ['1','2','3','4','5','7','8','9','10']
# Calculate how many members a 20% slice should have
twenty = int(math.ceil(len(X) *  20 / 100.0))

for i in range(len(X)):
     X1 = X[i:(i+twenty)]
     X2 = X[:i] + X[(i+twenty):]

What this does is calculate how many members ~20% slice should have, and then apply this number to list slicing. Note, however, that math.ceil() always rounds up, so if you have something like 2.1, it would end up being 3.0 (meaning that some slices will have slightly more members than 20%, e.g. 22%). If you want to round in a more familiar fashion (2.49 -> 2; 2.5 -> 3), round() would be a good choice.


Same note on two staves in a piano score

Question: These are bars 56–59 in Patrik Pietschmann's arrangement of Hans Zimmer's "Interstellar Main Theme"

How does one deal with the Es that appear in both staves?
The first two bars I can play by holding the LH's E and releasing it to play the RH's one, and then hitting it again (so that the left hand ends up playing three sequences of an eighth note and  an eighth note rest) This sounds nice, and is the approach suggested by answers to this question A well known example of this is the first Gymnopédie
For the next two bars, though, that does not really work, as far as I can tell.
Comment: How about holding down the eb an octava lower without playing the tone? Its harmonics will be heard and resound on each attack of the higher notes played by the right hand.
Comment: While waiting for answers, here is a somewhat related one: https://music.stackexchange.com/questions/120683/bach-invention-2-bar-13-overlapping-keys-for-both-hands-while-playing-mordent
Answer: The piece is intended to be played with the sustain (damper) pedal. Just keep the pedal down through the entire passage. Pedal use can be heard throughout his own recording of the piece.
However, more telling is, in fact, his recording, in which he doesn't play the left hand E at all. Here is the video, timed to measure 58 (the measure numbers appear in small print at the left of the video).
Answer: In the first bar you show, you press the sostenuto (middle) pedal then play the first note in the treble and bass clefs; both the treble and bass Es will sustain, but that should work fine. If you don't have a sostenuto pedal on your piano, you'll have to figure out a workaround.
Comment: Yes. I am asking for a workaround. That's the question.
Comment: Electronic or acoustic workaround?
Comment: @MarianoSuárez-Álvarez - no mention of a workaround in the question! However, **IF** your piano has a middle pedal which is sostenuto, then empty's answer is slightly more applicable. If it only has a sustain pedal, go with Aaron's answer - there's no real alternative way to play it. Most pianos on the left of the Atlantic will have said sostenuto, as will most good electronic pianos. Not so with acoustics on the right.


JXA set window bounds weird issue

Question: script1.js:
function run() {
  const chromeApp = Application('Google Chrome');
  const window = chromeApp.windows[0];
  console.log(window.name());
  window.bounds = {
    x: 0,
    y: 0,
    width: 500,
    height: 500,
  };
  chromeApp.activate();
}

Run:
osascript -l JavaScript script1.js

And it works
script2.js:
function run() {
  const systemEvents = Application('System Events');
  const ap = systemEvents.processes().find(ap => ap.name() === 'Google Chrome');
  console.log(ap.name());
  const window = ap.windows[0];
  console.log(window.name());
  window.bounds = {
    x: 0,
    y: 0,
    width: 500,
    height: 500,
  };
}

Run:
osascript -l JavaScript script1.js

It does NOT work:
script2.js: execution error: Error: Error: Can't set that. (-10006)

But I really need to get script2.js work. Because in my real application, I don't know the application name in advance and I need to fetch the process dynamically base on user interaction. Because I don't know application name, I cannot use script1.js.
Any input is appreciate!
Comment: Window bounds are normally a list of 4 integers, not a record.
Comment: @foo It's not the root cause because script1.js works.  And the code is JavaScript instead of AppleScript.
Comment: Did you try it? Every "AppleScriptable" app implements its own Apple event interface how it sees fit. You can't assume that what works on one app will work the same on others. Also, JXA is pants; it mostly just obfuscates Apple event behaviors so that when it does fail you've no idea if the problem is with the app or if it's JXA that's broken. Honestly, you're best to solve the problem in AppleScript, because at least it works right, so if something fails you know the issue is with the app.
Comment: The problem isn't JXA vs. AppleScript.  `bounds` is a property of `window` elements of an AppleScriptable application.  The `window` class object of _System Events_ doesn't have a `bounds` property, and instead has `size` and `position` properties.  The `window` class of a sciptable app and the `window` class of _System Events_ may share a name, but they are completely different objects: the former is a child element of an `application` class object; the latter is a kind of `UI element` class object; thus their properties are not interchangeable.
Answer: function run() {
  const systemEvents = Application('System Events');
  const p = systemEvents.processes().find(ap => ap.frontmost() === true);
  const ap = Application (p.bundleIdentifier());
  const window = ap.windows[0];
  window.bounds = {
    x: 0,
    y: 0,
    width: 500,
    height: 500,
  };
  }

Comment: Thanks. It's a good idea. But as I tested `const ap = systemEvents.processes().find(ap => ap.frontmost() === true)` doesn't work if frontmost app is Visual Studio Code. As I said I don't know the app name in advance, I need to position the frontmost app. It's better to make it work by app ID instead of by app name.
Comment: I don't think AppleScript vs JavaScript is the root cause, please check my answer. I could translate your code to JavaScript and it still works.
Comment: @Tyler Long: yes, you are right, it works as pure JXA. I updated my answer to application be identified by ID instead of the name.
Comment: Thanks. I accepted your answer. But I've turned to AppleScript. Because most documents/samples available online is AppleScript.
Answer: It's years since I've had the displeasure of using GUI Scripting so didn't immediately spot the real problem.
The real problem is that System Events is a bloated badly designed mess that shoehorns a dozen libraries' worth of functionality into one #BigBallOfMud.
SE includes Cocoa Scripting's Standard Suite window class definition, including a standard bounds property, but doesn't actually implement it (since SE has no windows of its own). So while SE's dictionary claims windows elements have a bounds property, trying to get/set that property throws an error. It is very confusing.
Therefore, ignore the window class definition in SE's Standard Suite, and only look at the window class definition in its Process Suite (aka GUI Scripting). It doesn't have a bounds property; instead it has separate position and size properties:
position (list of number or missing value) : the position of the window

size (list of number or missing value) : the size of the window

Here is code that works:
tell application "System Events"
    set ap to first process whose name is "Firefox"
    set win to window 1 of ap
    -- set bounds of win to {100, 100, 600, 600} -- this doesn't actually work
    set position of win to {100, 100}
    set size of win to {500, 500}
end tell

In other words, just because an app's dictionary says it supports something doesn't mean it actually does. Trying to figure out an app's brokenness while also wrestling with JXA's brokenness is at best a Sisyphean' exercise.
Figure out how you get your code working in AppleScript first; if it doesn't work there then you know it's the app that's the problem. If you still want to use JXA, then you can attempt to port your working code to that afterwards (although generally I don't recommend wasting time on JXA as it's crippled and abandoned, and the whole AppleScript stack is slowly dying anyway).
Comment: Also bear in mind that using GUI Scripting requires the user to enable Accessibility access in System Preferences' Security & Privacy settings (which is rightly locked down by default as it's a huge potential security hole).
Comment: I appreciate your answer! Yes as you said it is kind of buggy. And Apple doesn't release any updates to JXA in the recent years either. Some one said Swift will be the future language for macOS automation but nobody knows. For now we have to be used to what we have right now, sadly.
Comment: Don't expect any updates to JXA (or to AppleScript) either: Apple disbanded the Mac Automation team and fired the project manager responsible back in 2016. The future of end-user automation across Apple platforms is most likely Siri with Shortcuts as an extensible backend providing a degree of customization. So it goes.
Answer: Inspired by Robert's answer, I found that the following works:
function run() {
  const systemEvents = Application('System Events');
  const p = systemEvents.processes().find(ap => ap.name() === 'Google Chrome');
  console.log(p.name());
  const app = Application(p.name());
  const window = app.windows[0];
  window.bounds = {
    x: 0,
    y: 0,
    width: 500,
    height: 500,
  };
  app.activate();
}

So I can convert a Process into Application by name: const app = Application(p.name()); Then script2.js become script1.js.
But it doesn't always work. For example, the following doesn't work:
function run() {
  const systemEvents = Application('System Events');
  const p = systemEvents.processes().find(ap => ap.frontmost() === true);
  console.log(p.name());
  const app = Application(p.name());
  const window = app.windows[0];
  window.bounds = {
    x: 0,
    y: 0,
    width: 500,
    height: 500,
  };
  app.activate();
}

If the frontmost application is Visual Studio Code. Because the process name will be "Electron" and convert process to application will be wrong.
I'd like to find a way to convert process to application by ID. I am still investigating and I will post updates later.
Update: please check the accepted answer.
Answer: You can set the position and size to a two-dimensional array. Because Application("Name of App") doesn't always seem to work, it's best to go through System Events.
let sys = Application("System Events")
let app = sys.applicationProcesses
                    .whose({ name: "AppYouWant"})[0]
app[0].windows[0].size = [200, 200]
app[0].windows[0].position = [200, 200]

JXA set window bounds weird issue