How to set a static route for an external IP address

Question: I'm putting a new router on our network to help manage our private network connections and to better control routing to the outside. I've decided to simplify this question by removing references to a second private network on our router (but please be aware that there are reasons for this question so the answer is not to leave the old router the default gateway).
I have the following routing in my iptables on our router:
# Allow established connections, and those !not! coming from the public interface
# eth0 = public interface
# eth1 = private interface #1 (18.104.22.168/25)
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -m state --state NEW ! -i eth0 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow outgoing connections from the private interface
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT

# Masquerade (NAT)
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

# Don't forward any other traffic from the public to the private
iptables -A FORWARD -i eth0 -o eth1 -j REJECT

This configuration means that users will be forwarded through a modem/router that has a public address - this is all well and good for most purposes, and in the main it doesn't matter that all computers are hidden behind the one public IP.
However, some users need to be able to access a proxy at 126.96.36.199:8080 - and the proxy needs to identify this traffic as coming through a gateway at 184.108.40.206 (the old router) - it won't respond otherwise.
I tried adding a static route on our router with:
route add -host 22.214.171.124 gw 184.108.40.206 dev eth1

I can successfully ping 18.104.22.168 from the router. When I trace the route, it lists the 18.104.22.168 gateway, but I just get *** on each of the following hops (I think this makes sense since this is just a web-proxy and requires authentication).
When I try to ping this address from a host on the 18.104.22.168/25 network it fails.
Should I do this in the iptables chain instead? How would I configure this routing?
Network diagram

Here is the interfaces configuration for the router:
auto eth0
iface eth0 inet static
    address     220.127.116.11
    netmask     255.255.255.248
    gateway     22.214.171.124

auto eth1
iface eth1 inet static
    address     220.127.116.11
    netmask     255.255.255.128

And here is the routing table (without my static route added):
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         eth1202.sa.adsl 0.0.0.0         UG    100    0        0 eth0
localnet        *               255.255.255.0   U     0      0        0 eth1
22.214.171.124       *               255.255.255.128 U     0      0        0 eth1

To restate - I want traffic from 220.127.116.11 (for example) to now route through our router (184.108.40.206). But this router needs to then forward 8080 requests with the destination of 18.104.22.168 - which is somewhere the other side of the old router (22.214.171.124 - which is not manageable by us).
Comment: could you provide the routing tables (`ip route`) and the output of `ifconfig` as well? There is some confusion about how routes and interfaces are configured in your case - the numbers do not quite match up with the drawing you've posted.
Comment: maybe there's something I'm not getting with the idea of static routes - when I configure the static route on the client (10.2.2.7) it works fine... I thought that this would just be the same on the router?
Answer: What you are attempting to do is entirely possible and indeed is a quite often used technique. You also have got most of the setup right already. What you might be missing is an "ACCEPT" rule at "new router" allowing traffic flows from eth1 to eth1. Just add it using
iptables -I FORWARD -i eth1 -o eth1 -s 188.8.131.52/25 -j ACCEPT
iptables -I FORWARD -i eth1 -o eth1 -d 220.127.116.11/25 -j ACCEPT

It should be sufficiently safe not to circumvent any of the rules and security measures in place and yet be generic enough to allow any of your network's clients and protocols. Once you've done setting up and troubleshooting, you could restrict the rule to the protocols and destinations you are going to use:
iptables -I FORWARD -i eth1 -o eth1 -s 22.214.171.124/25 -d 220.127.116.11 -p icmp -j ACCEPT
iptables -I FORWARD -i eth1 -o eth1 -d 220.127.116.11/25 -s 184.108.40.206 -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT

iptables -I FORWARD -i eth1 -o eth1 -s 18.104.22.168/25 -d 184.108.40.206 -p tcp --dport 8080 -j ACCEPT
iptables -I FORWARD -i eth1 -o eth1 -d 126.96.36.199/25 -s 188.8.131.52 -p tcp --sport 8080 ! --syn -j ACCEPT

Note that in the typical case, you hardly would see any traffic flow through the "new router" for 126.96.36.199. Linux happily would issue ICMP redirect messages informing your subnet's hosts to use "old router" for traffic to 220.127.116.11 at the first packet occurrence. And most hosts would follow (assuming they got the message - i.e. it has not been filtered anywhere) and send all subsequent packets to 126.96.36.199 directly via "old router".
Comment: in hindsight, this really does make sense... thanks a lot
Answer: If you're routing to a specific host, I don't think that using netmask is correct. Specifying 0.0.0.0 would imply a default gateway for all traffic since that mask would match ALL IPv4 IPs.
In fact, a full 255.255.255.255 is kind of necessary for a single host matching rule.
And perhaps it is obvious, but you must specify a gateway that is on a directly-connected network covered by your existing routing table.
In working with iptables and routing, I'd suggest setting up a LOG rule to send dropped packets to a log. That will tell you if you're dropping packets that might only appear to be routing problem.
This sets up a LOGDROP chain.
-A LOGDROP -j LOG --log-prefix "LOGDROP "
-A LOGDROP -j DROP

And then at the end of various chains that you want to log, you jump to the LOGDROP chain at the end:
-A INPUT -j LOGDROP

Instead of simply dropping the packets. Various LOGDROP chains with different labels will allow you to get more granularity in understanding what you're seeing.
Comment: When I specify a netmask of 255.255.255.255 I get the message that it isn't compatible with the route - probably because the proxy (192) isn't on the same network as the gateway (129) I want to route through
Comment: If you're specifying -host, I'm not sure why you'd even want to specify mask. Have you tried simply leaving the mask off?
Comment: yeah - but the same results
Comment: Honestly, your partial IP obfuscation is making this more difficult. So IS that desired gateway on a connected network or not? I'm pretty sure you can't route packets to gateways that aren't on a directly-connected network unless it's going through the default gateway.
Comment: The gateway at 129.n.n.126 is provided to us along with the whole subnet - we don't have control over it. I guess that there is some routing somewhere in the supernet that that gateway connects to which allows users to access the proxy - the problem is, all outgoing traffic on my new gateway at 129.n.n.125 is now routed through a completely different provider - this is what I want, except for this one specific address - so I need to forward requests  for 192.n.n.n through 129.n.n.125 to 129.n.n.126 and let that supernet worry about the rest.
Comment: I de-obfuscated the IPs, and tried to clear up some of my explanation and intention a bit more.
Comment: Appreciate the de-obfuscation. I just think you're insane now that I can understand what you're trying to do.
Comment: quite possibly  - but i think i was driven to it by the infrastructure


Sibelius seven - adjusting the length of one bar/measure

Question: I want to shorten the length of the final bar in the line, without any of the other bars moving. Is this possible?
Answer: Turn on View: Handles.  You'll see a marker just past the end of the last stave.  Drag it across.
You can do this without making the handle visible, once you know where it is.
The other bars in that line will re-size to accommodate the shorter line length.  If you want just one bar compressed,  Shift-Alt-LeftArrow and Shift-Alt-RightArrow will compress or expand the spacing of just a selected bar or bars.
http://www.laurencepayne.co.uk/sibelius.html#u
Comment: Shift-Alt-LeftArrow and Shift-Alt-RightArrow will compress or expand the spacing of a selected bar or bars.

http://www.laurencepayne.co.uk/sibelius.html#u
Comment: thank you Laurence. I am looking for a way to adjust the length of only the final bar. As far as I can see moving this handle moves all of the bars in the lin.


Veryfing basis and finding dual basis

Question: Lets say I have a vector space $V$ and it's basis $B=(b1,b2,b3)$. How do I verify that for any $t$, the vectors $$c_{1,t} := (b1), c_{2,t} := (b1+b2), c_{3,t} := (t*b1+b2+b3)$$
are a basis of $V$ as well? Furthermore, if I then want to find the dual basis of $C$, where do I begin?
I tried to start off by proving linear independence of the $c$ vectors. I came up with the following equation: $$a_1*c_1+a_2*c_2+a_3*c_3 = 0$$
If I could verify that $a_1,a_2,a_3$ must all be zero, then I would have at least verified that they are linearly independent. But I cannot seem to solve this, even if I split it up into three seperate equations. I think I have to solve this another way. Any hints?
Edit: I tried to apply the hint I got, so I ended up with the coordinate matrix:
$$\langle B^*,C_t\rangle = \begin{pmatrix}1\ 1\ t\\ 0\ 1\ 1\\ 0\ 0\ 1 \end{pmatrix}$$
I can see now that they are indeed linearly independent. How do I find the dual basis now though?
Comment: Express $C$ as coordinate vectors in basis $B$. Write them in a matrix, you will see it has determinante $\neq 0$ and so they are linear independent, do you know how the dual basis changes when we have a basis change in the vector space?
Comment: Edit: Nevermind, I think you can also see that they are linearly independent by looking at the collumns of the matrix, right?
Answer: Hint: To find the dual basis, go back to its definition: If $(\mathbf e_1,\dots,\mathbf e_n)$ is a basis for $V$, then its dual basis is a set of vectors $(\mathbf\varepsilon_1,\dots,\mathbf\varepsilon_n)$ such that $\mathbf\varepsilon_i[\mathbf e_j]=\delta_{ij}$ for $i,j=1..n$. If we represent the $\mathbf\varepsilon_i$ as row vectors and the $\mathbf e$ as column vectors, this can be expressed in matrix form as $$\pmatrix{—&\mathbf\varepsilon_1&—\\\vdots&\vdots&\vdots\\—&\mathbf\varepsilon_n&—}\pmatrix{\mid&\cdots&\mid\\\mathbf e_1&\cdots&\mathbf e_n\\\mid&\cdots&\mid}=I.$$ Does this product suggest anything to you?
Comment: I think I found it now. By matrix transformations I could find the matrix which, multiplied by my basis matrix produces $I$... Thanks!
Answer: If you have a basis $e_i$ for a inner product space $(V, < , >)$,
then the dual basis in $V^*$ will be the functions that map $v$ to $<v,e_i>$

Veryfing basis and finding dual basis


How do I go about practicing this arpeggio?

Question: I'm an experienced guitar player, and I'm finally getting around to spending some time self-studying the piano.  I've been learning a few songs by ear, and I'm wondering how to go about properly playing the arpeggio/melody combination in this tune that starts at about 0:20.
It's a descending D minor-C major-Bb major-A minor arpeggio.  I'm assuming the way it's supposed to be played is 5-2-1 with the left hand and then bringing the 3rd finger over to hit the high 3rd of the chord? Unfortunately, I've found that doing this while playing a melody in addition is at the limits of my abilities.  Is there an optimal way for a beginner to practice this type of piece to "get up to speed"?
Answer: First, isn't that D minor, C major, Bb (without third), A minor?
In addition to practicing slowly, a good way (in general) to practice is to practice hands separately until they can do their own thing well and only then start playing them together. Also try practicing just a single measure over and over again. When you can play every measure separately start practicing connecting them. This applies to both the hands separate and hands together phase.
Now there are lots of ways to finger these arpeggios. You could try to play some of the notes with the right hand but it might be a bit tricky with the melody (maybe even impossible, I didn't actually try it). Here are some ways to finger the arpeggios which suit my hand, each with their own strengths and weaknesses:

5-5(4)-2-1-2-5(4)-2-1 or similar for every arpeggio. That is, you play the first note with 5, then jump to a position where the chord fits comfortably in your hand and play the arpeggio. This will probably bring the base line out a bit, which may or may not be what you want. A strength here is that most of the arpeggio will be easy. A weakness is that the jump might not be easy, especially if you want the arpeggio to be very even.
D minor, C major: 5-2-1-2-3-5-2-1, Bb: 5-2-1-1-3-5-3-1, A minor: 5-1-2-1-3-5-3-1. This is probably the most "fluid" version I could come up with, but it's also the least straightforward (so it might even require some practice just to get the fingers to play in the right order). It also has a jump in the Bb arpeggio with which you need to be careful with.
All chords except A minor: 5-3-2-1-2-4(5)-1, A minor the same as in the previous one. This should work quite well if your hand is largish but might be problematic for smaller hands, though even then if you are able to play the required intervals you can practice a kind of wrist rotation technique to make it work.

Of course you can play one arpeggio with a fingering from (1) and another with one from (2) and so on, and you can modify these; I just tried to group them according to some common idea.
Comment: Had a brain fart on the chords, sorry! I've edited my post.  Thanks for all your suggestions.


Confused on how to implement asynchronous callback in node.js

Question: ytdl(url, { filter: function(format) { return format.container === 'mp4'; } })
    .pipe(fs.createWriteStream('./songs/Battle-Scars.mp4'));

I am new to node.js and am trying to implement a youtube-mp3 converter, and I don't know how to make the following code run ONLY after the code above finishes running.
var proc = new ffmpeg({ source: './songs/Battle-Scars.mp4'})
    .withAudioCodec('libmp3lame')
    .toFormat('mp3')
    .saveToFile('./songs/Battle-Scars.mp3', function(stdout, stderr) {
        console.log('file has been converted succesfully');
    });

I appreciate all the help and thanks in advance.
Answer: Since you are using streams, you might want to take a look at the finish event which you can trigger once your write stream has completed writing to disk.
Try something like:
var writeStream = fs.createWriteStream('./songs/Battle-Scars.mp4');

// Set up event listener
writeStream.on('finish', function () {
  var proc = new ffmpeg({ source: './songs/Battle-Scars.mp4'})
    .withAudioCodec('libmp3lame')
    .toFormat('mp3')
    .saveToFile('./songs/Battle-Scars.mp3', function(stdout, stderr) {
      console.log('file has been converted succesfully');
    });
});

ytdl(url, { filter: function(format) { return format.container === 'mp4'; } })
.pipe(writeStream);

Comment: It works!, Thanks for all the help, should've looked closer to the documentation.


Criar um elemento com uma lista de valores informados pelo usuário

Question: Tenho um programa simples, que pega o valor de uma caixa de texto id1 e após pressionado o botão OK joga esse texto (função Enviar1) para uma outra caixa de texto id2. O que eu preciso fazer é eliminar essa id2 e mostrar o texto em itens.

function Enviar1() {
  var valor = document.getElementById('id1').value;
  document.getElementById('id2').value = valor;
}
<input id="id1" type="text" />
<input id="id2" type="text" />
<input id="botao" type="button" value="Ok" onclick="Enviar1()">

Gostaria de cada vez que digitar um texto em id1 e pressionar OK, ele fosse criando novos itens, sem apagar o primeiro, para criar itens um embaixo do outro.
Agradeço pela ajuda! :)
....
Já atualizando o conteúdo com a ajuda dos colegas...
só pra ficar mais claro, o que tenho até agora está assim.
Gostaria de poder editar os itens já escritos e enviados agora, e até apagá-los se possível. Obrigado pela ajuda!! :D

    Atualização com JS puro

 Teste ServiceReportMaker

var menu_dropdown = document.getElementById("selecao");

menu_dropdown.addEventListener("change", function(){

    var valor_selecionado = menu_dropdown.options[menu_dropdown.selectedIndex].value;

    if(valor_selecionado == "personalizar"){
        document.getElementById("div_personalizar").style.visibility= "";
    } else {
        document.getElementById("caixa1").value = valor_selecionado;
        document.getElementById("div_personalizar").style.visibility = "hidden";
    }

});

function Enviar1()
{
  var ul = document.getElementById("itens");
  var li = document.createElement("li");
  var valor = document.getElementById('id1').value;
  li.appendChild(document.createTextNode(valor));
  ul.appendChild(li);
}
<select id="selecao">
    <option value="" selected > Selecionar empresa </option>
    <option value="1ª opção" >Primeira opção aqui</option>
    <option value="2ª opção">Segunda opção aqui</option>
</select>

<input id="caixa1" type="text" />

<input type="text" id="id1" />
<input type="button" value="OK" onclick="Enviar1()" />

Comment: Eduardo, acredito que esta nova função deve ser alvo de uma nova pergunta aqui so SOpt. Como você não especificou todo o problema inicialmente, as respostas dadas anteriormente a esta alteração passariam a não fazer muito sentido, pois responderiam apenas parte da pergunta. Sinta-se livre para criar outra pergunta, colocando o código que tem, descrevendo a necessidade de exclusão e alteração dos itens.
Answer: Você pode resolver esse problema adicionando itens dinamicamente em uma lista do HTML (ul / li):

function Enviar1()
{
  var ul = document.getElementById("itens");
  var li = document.createElement("li");
  var valor = document.getElementById('id1').value;
  li.appendChild(document.createTextNode(valor));
  ul.appendChild(li);
}
<input type="text" id="id1" /> <input type="button" value="OK" onclick="Enviar1()" />

Comment: Pode adicionar `document.getElementById('id1').value = '';` ao final da função para remover o conteúdo do `input` assim que o botão OK for pressionado, facilitando o cadastro de novos valores.
Comment: Olá Marcell. Funcionou perfeitamente!! Deixa eu aproveitar e te pedir mais uma coisa: Isso que quero fazer é para relatar vários problemas de equipamentos e gerar um relatório técnico. Como eu faço para editar os itens que já enviei, pressionando o OK? Tem jeito?
Answer: Para a funcionalidade de exclusão de itens, sugiro introduzir um link ao lado de cada um deles, e escrever uma função que remove aquele item específico:

var menu_dropdown = document.getElementById("selecao");

menu_dropdown.addEventListener("change", function(){

    var valor_selecionado = menu_dropdown.options[menu_dropdown.selectedIndex].value;

    if(valor_selecionado == "personalizar"){
        document.getElementById("div_personalizar").style.visibility= "";
    } else {
        document.getElementById("caixa1").value = valor_selecionado;
        document.getElementById("div_personalizar").style.visibility = "hidden";
    }

});

function Enviar1()
{
  var ul = document.getElementById("itens");
  var li = document.createElement("li");
  var valor = document.getElementById('id1').value;
  li.appendChild(document.createTextNode(valor));
  li.appendChild(document.createTextNode(" "));
  var iconeRemover = document.createElement("a");
  iconeRemover.appendChild(document.createTextNode("X"));
  var hrefAttr = document.createAttribute("href");
  hrefAttr.value = "#";
  iconeRemover.setAttributeNode(hrefAttr);
  var clickEvent = document.createAttribute("onclick");
  clickEvent.value = "RemoverItem(this.parentNode)";
  iconeRemover.setAttributeNode(clickEvent);
  li.appendChild(iconeRemover);
  ul.appendChild(li);
}

function RemoverItem(li)
{
  var ul = document.getElementById("itens");
  ul.removeChild(li);
}
<select id="selecao">
    <option value="" selected > Selecionar empresa </option>
    <option value="1ª opção" >Primeira opção aqui</option>
    <option value="2ª opção">Segunda opção aqui</option>
</select>

<input id="caixa1" type="text" />

<input type="text" id="id1" />
<input type="button" value="OK" onclick="Enviar1()" />


zfs zpool degraded drive missing

Question: I have a degraded zpool called storage of 4 6TB Drives in 2 mirrors. One of the drives on the second mirror is unavailable. All cables seem to to be plugged in physically. Did my device id change?
zpool status -v storage
NAME                                          STATE     READ WRITE CKSUM
storage                                       DEGRADED     0     0     0
  mirror-0                                    ONLINE       0     0     0
    sdc                                       ONLINE       0     0     0
    sdd                                       ONLINE       0     0     0
  mirror-1                                    DEGRADED     0     0     0
    ata-WDC_WD60EFRX-68MYMN1_WD-WX91D65355TK  ONLINE       0     0     0
    15417011366624190108                      UNAVAIL      0     0     0  was /dev/disk/by-id/ata-WDC_WD60EFRX-68MYMN1_WD-WX91D6535J36-part1

ls -l /dev/disk/by-id
total 0
lrwxrwxrwx 1 root root  9 Feb 22 10:56 ata-Samsung_SSD_840_EVO_250GB_S1DBNEADA00610E -> ../../sdb
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-Samsung_SSD_840_EVO_250GB_S1DBNEADA00610E-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-Samsung_SSD_840_EVO_250GB_S1DBNEADA00610E-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-Samsung_SSD_840_EVO_250GB_S1DBNEADA00610E-part3 -> ../../sdb3
lrwxrwxrwx 1 root root  9 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX11DA40H4ZK -> ../../sdc
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX11DA40H4ZK-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX11DA40H4ZK-part9 -> ../../sdc9
lrwxrwxrwx 1 root root  9 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX51D6410414 -> ../../sdd
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX51D6410414-part1 -> ../../sdd1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX51D6410414-part9 -> ../../sdd9
lrwxrwxrwx 1 root root  9 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX91D65355TK -> ../../sda
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX91D65355TK-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 ata-WDC_WD60EFRX-68MYMN1_WD-WX91D65355TK-part9 -> ../../sda9
lrwxrwxrwx 1 root root  9 Feb 22 10:56 wwn-0x50014ee20b713dd3 -> ../../sdd
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50014ee20b713dd3-part1 -> ../../sdd1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50014ee20b713dd3-part9 -> ../../sdd9
lrwxrwxrwx 1 root root  9 Feb 22 10:56 wwn-0x50014ee20c912489 -> ../../sda
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50014ee20c912489-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50014ee20c912489-part9 -> ../../sda9
lrwxrwxrwx 1 root root  9 Feb 22 10:56 wwn-0x50014ee260a4bd75 -> ../../sdc
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50014ee260a4bd75-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50014ee260a4bd75-part9 -> ../../sdc9
lrwxrwxrwx 1 root root  9 Feb 22 10:56 wwn-0x50025388500a0d02 -> ../../sdb
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50025388500a0d02-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50025388500a0d02-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Feb 22 10:56 wwn-0x50025388500a0d02-part3 -> ../../sdb3

edit
fdisk -l
Disk /dev/sda: 6001.2 GB, 6001175126016 bytes
256 heads, 63 sectors/track, 726751 cylinders, total 11721045168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdb: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xcb1189ee

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1   488397167   244198583+  ee  GPT

WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdc: 6001.2 GB, 6001175126016 bytes
256 heads, 63 sectors/track, 726751 cylinders, total 11721045168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.

WARNING: GPT (GUID Partition Table) detected on '/dev/sdd'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdd: 6001.2 GB, 6001175126016 bytes
256 heads, 63 sectors/track, 726751 cylinders, total 11721045168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.

edit2
Didn't realize one disk wasn't even showing up. Looks like 2nd drive in mirror-1 is completly dead. The out put from sudo lshw -class disk helped me see that.
 *-disk
       description: ATA Disk
       product: WDC WD60EFRX-68M
       vendor: Western Digital
       physical id: 0.0.0
       bus info: scsi@0:0.0.0
       logical name: /dev/sda
       version: 82.0
       serial: WD-WX91D65355TK
       size: 5589GiB (6001GB)
       capabilities: gpt-1.00 partitioned partitioned:gpt
       configuration: ansiversion=5 guid=e4026c6a-61c5-9a4a-adaf-ebcdf8bba68c sectorsize=4096
  *-disk
       description: ATA Disk
       product: Samsung SSD 840
       physical id: 0.0.0
       bus info: scsi@3:0.0.0
       logical name: /dev/sdb
       version: EXT0
       serial: S1DBNEADA00610E
       size: 232GiB (250GB)
       capabilities: gpt-1.00 partitioned partitioned:gpt
       configuration: ansiversion=5 guid=8d48ca17-2e53-4ee4-a040-e80362e68eca sectorsize=512
  *-disk
       description: ATA Disk
       product: WDC WD60EFRX-68M
       vendor: Western Digital
       physical id: 0.0.0
       bus info: scsi@4:0.0.0
       logical name: /dev/sdc
       version: 82.0
       serial: WD-WX11DA40H4ZK
       size: 5589GiB (6001GB)
       capabilities: gpt-1.00 partitioned partitioned:gpt
       configuration: ansiversion=5 guid=5811c203-99cd-5342-bb8f-c838f896b64e sectorsize=4096
  *-disk
       description: ATA Disk
       product: WDC WD60EFRX-68M
       vendor: Western Digital
       physical id: 0.0.0
       bus info: scsi@5:0.0.0
       logical name: /dev/sdd
       version: 82.0
       serial: WD-WX51D6410414
       size: 5589GiB (6001GB)
       capabilities: gpt-1.00 partitioned partitioned:gpt
       configuration: ansiversion=5 guid=c24e5950-cd95-f94b-826a-21a004221d32 sectorsize=4096

Comment: this question misses `fdisk -l` output. and, after all, what I see is very weird. Half a pool is coposed from dedicated disk, which, in the same time, seem to have partitions on them.
Comment: I have included fdisk -l output. Like the question says I have 4 6TB Drives in a mirrored pairs. Does the output not seem that way? The mirror-1 has two disks. The second on is all the way on the right side of the output. The partitions confuse me also.
Comment: Your disk could be dead.  For a quick test, try `dd if=/dev/sdb1 of=/dev/null bs=1m` and see if you can actually read from the disk.
Comment: @AndrewHenle here is the out put of dd        nathan@davis-server:~$ sudo dd if=/dev/sdb1 of=/dev/null
^C709345+0 records in
709344+0 records out
363184128 bytes (363 MB) copied, 3.3554 s, 108 MB/s
Comment: @NathanTheGr8 Don't break the `dd` - let it run to see how much data it can actually read from the disk.  It may take a while, but it will tell you if there are disk errors, and if more than 250GB can be accessed.
Comment: @AndrewHenle 1048576+0 records in
1048576+0 records out
536870912 bytes (537 MB) copied, 2.49903 s, 215 MB/s  Also people seem to be confused with the 250GB drive. That is an ssd that the os is installed on. It is not managed by zfs
Answer: Well, the output clearly states that one of the drives is 250GB. Either it was replaced, or it was corrupted in a way when it started report it's space erroneously. Take smartmontools and examine it with smartctl, it will show all of the info about it.
Comment: the 250GB disk is the boot disk. It is not managed by zfs.
Comment: I see only 4 disks so far. Either it's not powered on, either it's completely dead electrically.
Comment: yep, you are right. well just ordered a new one. Hope I can rebuild in time.

zfs zpool degraded drive missing


Cannot upload an App Bundle on Google Play

Question: I tried to upload an App Bundle (for a new app) on the Google Play. Google asked me to enable App Signing. Fine, I choose to enable it, I download their PEPK tool and use it to generate a ZIP (with a keystore that I have created to export my App Bundle). The ZIP contains two files : "encryptedPrivateKey" and "certificate.pem". I upload that ZIP file on Google Play and now it says "Google App signing successfully enabled".
So now, I export my App Bundle using the same keystore and try to upload it on Google Play and the result is "You uploaded an APK that is signed with a key that is also used to sign APKs that are delivered to users.". Okay, so I create a new keystore but now I have an "Incorrect SHA1" error.
My question is : what was my mistake and how can I solve this problem ?
Thanks for your help.
Comment: I do not think to create an app bundle is tricky. Simply, create an app bundle via Android studio, create or enter the previous password and just create a new release putting app bundle in the required box.
Answer: I have finally managed to upload my App Bundle on Google Play. The solution was to delete my release and create another one. Then, I uploaded my App Bundle signed with the first keystore and it worked.


How do I stop Fields being rendered twice for a node with Paragraph Types using UI Patterns?

Question: I'm using UI Patterns within Paragraph Types which is resulting in the Fields being rendered twice in the theme. I'm using a sub theme of the bootsrap5 contrib theme. I have a custom layout twig:-
{#
/**
 * @file
 * Theme override to display a two-column layout.
 *
 * Available variables:
 * - in_preview: Whether the plugin is being rendered in preview mode.
 * - content: The content for this layout.
 * - attributes: HTML attributes for the layout
.
 */
#}

{%
    set classes = [
        'col-12',
        'col-md-9',
        'col-md-3',
    ]
%}

{% if content %}

            {% if content.first %}

                    {{ content.first }}

            {% endif %}

            {% if content.second %}

                    {{ content.second }}

            {% endif %}

{% endif %}

Screenshot of the result:-

The resulting HTML is below:-

Heading

This is the heading!

Message

Hello! I'm a message!

      <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>

Card image

Heading

This is the heading!

Message

Hello! I'm a message!

      <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>

Card image

Answer: So this turned out to be a face palm and a half!
The Field in question had the Display set to "Rendered Entity" with no "UI Limit".
Changing the "UI Limit" to 1 fixed it.
However, as this is a Paragraph Type the Display should really be "Paragraph Type".
Hope this helps anyone experiencing this problem!
Comment: I have this issue on Drupal 10, and there is no admin field to adjust this. :(


How to solve this problem in Python Importing Stocks data?

Question: I'm sorry that I'm a begginer, so this might be something simple but I don't know what might be wrong.
import pandas as pd
from pandas_datareader import data as wb
tickers = ['F','MSFT','BP']
new_data = pd.DataFrame()
for t in tickers:
    new_data[t] = wb.DataReader(t, data_source='yahoo', start='2015-1-1')

This is the error I get:

ValueError                                Traceback (most recent call last)
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py in _ensure_valid_index(self, value)
   3524             try:
-> 3525                 value = Series(value)
   3526             except (ValueError, NotImplementedError, TypeError):

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    312
--> 313                 data = SingleBlockManager(data, index, fastpath=True)
    314

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\managers.py in __init__(self, block, axis, do_integrity_check, fastpath)
   1515         if not isinstance(block, Block):
-> 1516             block = make_block(block, placement=slice(0, len(axis)), ndim=1)
   1517

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\blocks.py in make_block(values, placement, klass, ndim, dtype, fastpath)
   3266
-> 3267     return klass(values, ndim=ndim, placement=placement)
   3268

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\blocks.py in __init__(self, values, placement, ndim)
   2774
-> 2775         super().__init__(values, ndim=ndim, placement=placement)
   2776

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\blocks.py in __init__(self, values, placement, ndim)
    127                 "Wrong number of items passed {val}, placement implies "
--> 128                 "{mgr}".format(val=len(self.values), mgr=len(self.mgr_locs))
    129             )

ValueError: Wrong number of items passed 6, placement implies 1318

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-11-6f48653a5e0e> in <module>
      2 new_data = pd.DataFrame()
      3 for t in tickers:
----> 4     new_data[t] = wb.DataReader(t, data_source='yahoo', start='2015-1-1')

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py in __setitem__(self, key, value)
   3470         else:
   3471             # set column
-> 3472             self._set_item(key, value)
   3473
   3474     def _setitem_slice(self, key, value):

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py in _set_item(self, key, value)
   3546         """
   3547
-> 3548         self._ensure_valid_index(value)
   3549         value = self._sanitize_column(key, value)
   3550         NDFrame._set_item(self, key, value)

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py in _ensure_valid_index(self, value)
   3526             except (ValueError, NotImplementedError, TypeError):
   3527                 raise ValueError(
-> 3528                     "Cannot set a frame with no defined index "
   3529                     "and a value that cannot be converted to a "
   3530                     "Series"

ValueError: Cannot set a frame with no defined index and a value that cannot be converted to a Series

I imagine this has got something to do with missing information? Reading int as text or something? Can someone help?
This is actually from the answer key of an exercise I am doing from an online course, and it's returning this error.
Comment: does `DataReader` return a dataframe? but the issue is you can not set a value like that on the DataFrame
Comment: Welcome to StackOverflow! There's no need to worry if the question is simple or not. Try to avoid that kind of sentence in your questions. The only thing you need to worry about is duplicates, so please do some research before posting. Have a nice time here.
Answer: You can only assign a column with in a dataframe with data of the same size: a column or Series (in Pandas terms). Let's look at what DataReader returns:
wb.DataReader('F', data_source='yahoo', start='2015-1-1').head()

Date        High    Low     Open    Close   Volume        AdjClose
2015-01-02  15.65   15.18   15.59   15.36   24777900.0  11.452041
2015-01-05  15.13   14.69   15.12   14.76   44079700.0  11.004693
2015-01-06  14.90   14.38   14.88   14.62   32981600.0  10.900313
2015-01-07  15.09   14.77   14.78   15.04   26065300.0  11.213456
2015-01-08  15.48   15.23   15.40   15.42   33943400.0  11.496773

It returns a dataframe. If you'd like to pick one column from here - say, Open - you can assign that to your new table.
tickers = ['F','MSFT','BP']
new_data = pd.DataFrame()
for t in tickers:
    new_data[t] = wb.DataReader(t, data_source='yahoo', start='2015-1-1')['Open']
new_data.head()

Date          F         MSFT       BP
2015-01-02  15.59   46.660000   38.209999
2015-01-05  15.12   46.369999   36.590000
2015-01-06  14.88   46.380001   36.009998
2015-01-07  14.78   45.980000   36.000000
2015-01-08  15.40   46.750000   36.430000

Answer: You could assign to a dictionary;
import pandas as pd
from pandas_datareader import data as wb
tickers = ['F','MSFT','BP']
new_data = {}
for t in tickers:
    new_data[t] = wb.DataReader(t, data_source='yahoo', start='2015-1-1')

print(new_data['F'])

you are trying to assign a dataframe to each column of a dataframe


Get text from an element that is not yet in the DOM

Question: I need to get the text from the element IDed "list" but this element doesnt not exist yet when the page first loads, it takes like 3 seconds to appear so if I use this code:
var lists = document.getElementById("list").textContent;
console.log(lists)

I get an error that says TypeError: document.getElementById(...) is null
What can I do so I get the text once the element appears?
UPDATE
document.addEventListener("DOMContentLoaded", function(event) {
// Select the node that will be observed for mutations
var parentOfMyList = document.body;

// Options for the observer (which mutations to observe)
var config = { attributes: true, childList: true, subtree: true };

// Callback function to execute when mutations are observed
var callback = function(mutationsList) {
    for(var mutation of mutationsList) {
        if (mutation.type == 'childList') {
            var lists = document.getElementById("list").textContent;
            console.log(lists)
            console.log(parentOfMyList)
        }
    }
};

// Create an observer instance linked to the callback function
var observer = new MutationObserver(callback);
observer.observe(parentOfMyList, config);
});

DIFFERENT APPROACH
function fuckobserver() {
            var fuckobs = new MutationObserver(function (mutations, observer) {
                $.each(mutations, function (i, mutation) {
                    var lists = document.getElementById("topcmm-123flashchat-sound-messages-contents").textContent;
                    console.log(lists)
                    });
                });
            var parentOfMyList = document.body;
            fuckobs.observe($(parentOfMyList)[0], {childList: true, subtree: true});
        }

        if (document.body.length) {
            // body is already in the DOM
            fuckobserver();
        } else {
            // body is not in the DOM yet, add an observer to detect its addition
            new MutationObserver(function (mutations, outineObserver) {
                if (document.body.length) { // body is finally in the DOM
                    outineObserver.disconnect();
                    fuckobserver();
                }
            }).observe($(document.body)[0], {childList: true, subtree: true});
        }

Comment: Use https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers/onload
Answer: is #list in your html or is it injected using javascript? If the former is the case, you can use the DOMContentLoaded event (#1) to wait until you html is loaded. If it is injected by a javascript function you have control over, you can implement a callback (#2). If you do not have control over the javascript function (e.G. a third-party-library or an external script), you can use MutationObserver (#3).
#1 DOMContentLoaded
https://caniuse.com/#search=DOMContentLoaded
This will wait for your browser to tell the script that it finished loading your HTML. Wrap your assignment and your log like this:
document.addEventListener("DOMContentLoaded", function(event) {
    var lists = document.getElementById("list").textContent;
    console.log(lists)
});

#2 Callback
With this approach, you basically pass a function that will be executed at the end of another function.
// add callback parameter
function createList (callback) {
   // code that creates the list element you want to log
   callback()
}

// put a function that console.logs your element into the parameter
createList(function () {
    var lists = document.getElementById("list").textContent;
    console.log(lists)
});

#3 MutationObserver (taken from MDN, slightly modified)
https://caniuse.com/#search=mutationobserver
MutationObserver is a relatively new API to watch for changes ('mutations') in the DOM. It will fire every time it detects a change to a specific target node.
// Select the node that will be observed for mutations
var parentOfMyList = document.body;

// Options for the observer (which mutations to observe)
var config = { attributes: true, childList: true };

// Callback function to execute when mutations are observed
var callback = function(mutationsList) {
    for(var mutation of mutationsList) {
        if (mutation.type == 'childList') {
            if (mutation.target.id === 'topcmm-123flashchat-sound-messages-contents') {
                // do something with your element mutation.target.id

            }
        }
    }
};

// Create an observer instance linked to the callback function
var observer = new MutationObserver(callback);
observer.observe(parentOfMyList, config);

Comment: Its javascript injected I think, I will try with each one of your methods.
Comment: If you cannot edit the script injecting the list, then you will most probably need the MutationObserver approach. You can't really implement a callback if you don't have control over the function and it won't be loaded when Document is Ready. If you need help with the implementation, feel free to ask :)
Comment: Im getting this error with the #3 method: TypeError: Argument 1 of MutationObserver.observe is not an object.
Comment: I think the problem is that parent is not loaded yet either. Can i have body observed instead?
Comment: sure, just use `document.querySelector('body')` instead of `document.getElementById('listParent')` for your parentOfMyList
Comment: I still get the error which is weird because body is always present. Does the parent of the element has to be exactly the parent that is 1 level above or can be any?
Comment: Even weirder, if I enclose everything in "setTimeout(function(){}, 3000);" then it works...
Comment: you could try setting `subtree: true` in the config object. Then it will watch all descendands of the body. Also, console.log parentOfMyList to make sure it's a DOM-Node
Comment: I did. It points body as the parent but unless I leave "setTimeout(function(){" on it wont give me any output.
Comment: Any further suggestions please?
Comment: don't have any suggestions without seeing more code. Can you reproduce the behaviour in a fiddle and link it here please?
Comment: I cant reproduce it here but I can give you the link to the actual site if you wish.
Comment: sure go ahead! I'll see what I can do
Comment: thank you, this is the link http://saintseiyaforos.net:35555/htmlchat/123flashchat.html?init_room=1&init_user=invitado&init_lang=en first you will see a loading page with only a logo in the middle, then you will see the actual site but the code is supposed to load from the loading page.
Comment: I'd suggest wrapping the whole block of text inside the example of  **#1 DOMContentLoaded**, if it still doesn't work try using `var parentOfMyList = document.body`. This is a shot in the dark though, since my browser debugger is broken somehow
Comment: I tried `parentOfMyList = document.body` but I get this error `Argument 1 of MutationObserver.observe is not an object.`
Comment: Wrap everything in `DOMContentLoaded` listener then and see if it works
Comment: You mean put the whole #3 inside #1?
Comment: I updated my question with the modifications to the code I think you mean. This time I get the logs right although it produces logs not once but every 5 seconds or so. Please check it out.
Comment: this just means it can't find an element with the ID `#topcmm-123flashchat-sound-messages-contents`. I will update my answer soon for an implementation which waits for exactly that element
Comment: @snksaint I updated the callback in example #3. Try using this instead vor defining the lists variable
Comment: This time I get `Argument 1 of MutationObserver.observe is not an object.` and no log is displayed. I think there is no object with ID listParent.
Comment: just use `document.body` again
Comment: sorry again but I get the same error and no logs are shown. If you need any additional details please let me know.
Comment: No errors but no logs are shown either. I tried a different approach based on another code I have confirmed it works within the same page but its not working here for some reason. Please look at the very bottom of my question.
Comment: should I just give it up?
Comment: I don't have much time at hand at the moment but I'll try to look into it later
Comment: Okay, no problem. I can wait. I will keep on trying meanwhile.
Answer: You can use setTimeout as follows
setTimeout(function(){
    var lists = document.getElementById("list").textContent;
    console.log(lists);
 }, 3000);

3000 --> 3secs
You can provide how much time you required.
Comment: This isn't a good solution.  You're just guessing how long it will take to load the DOM.
Comment: Actually its so simple its very nice however I just found out the element wont appear on a timer but on a user click which renders it unusable but thank you.


Nesting / hierarchical modeling in a pre-post study

Question: We are conducting a study which has a pre-post design.  The unit of the study is the outpatient visit. In each visit, we study how patients and physicians interact.
Each physician will see between 3-6 patients and most of these patients (ie, except for dropouts) will be seen twice: once in the pre-phase, once in the post-phase.
Because the relation of patient to physician in any given visit is symmetric, it would seem to lead to two distinct ways to model the hierarchical structure. Starting from the root vertex of the (directed) tree:
A) study $\to$ physician $\to$ patient $\to$ pre/post
B) study $\to$ pre/post $\to$ physician $\to$ patient
Is there a scientific reason or stat. methodological issues to prefer one nesting over the other? What differences in results can we expect?
Any pointers to the literature are appreciated.
Answer: Yes, there is a good reason to prefer (A) over (B).
(A) correctly reflects the design of the study, while (B) does not. The two models are not equivalent, and the choice is not arbitrary.
Model (B) posits that Physicians are nested under Pre/post, as are Patients (by virtue of being nested under Physicians), but clearly neither of these is true. To say that Physicians are nested under Pre/post would be to say that each Physician is observed under one and only one level of Pre/post... that is, we have some Physicians that are "Pre Physicians", and other Physicians that are "Post Physicians", but no Physician has both Pre and Post observations (according to model B).
But in actual fact, each Physician does have some Pre observations and some Post observations. And because the Pre/post observations under one Physician are different from the Pre/post observations under another Physician, we can see that the order of nesting is actually the reverse... Pre/post is nested under Physician, not vice versa. And of course, Patients are in the middle of this hierarchy.
The difference here is not merely conceptual. There are tangible statistical consequences that follow from the choice of (A) or (B). Perhaps most saliently in this case, in model (A) the denominator of the test for the Pre vs. Post difference is free of variance due to both Physician and Patient, because these are held constant when comparing Pre-scores to Post-scores. (This becomes slightly more complicated in the presence of unbalanced/missing data, but we'll ignore that for now.) In model (B), Physician and Patient variance are both (wrongly) thrown in the denominator for testing the Pre vs. Post difference. This is "wrong" in the sense that it leads to a test of the Pre vs. Post difference that is overly conservative.
Comment: Jake, +1 for the explanation. Before I accept, however, can you explain what you mean in the last paragraph by "post difference is free of variance"? Two issues: (1) we are not necessarily hypothesis testing, but primarily estimating correlation structure. (2) We observe variation in captured data across visits even with the same physician and patient, because it is a distinct visit with distinct issues and workflow. What is "held constant"?
Comment: I don't think you parsed that sentenced the way I intended. Putting it differently: the *t*-ratio for testing the difference between Pre-scores and Post-scores does not contain Physician variance (i.e., variance due to Physician means) or Patient variance (i.e., variance due to Patient means). This is what I mean when I say that Physicians and Patients are "held constant" (not allowed to vary) when testing the Pre vs. Post difference. The *t* denominator does, of course, still contain other sources of variance, but overall the denominator of this *t* is smaller in (A) than it is in (B).
Comment: What scores do you mean? We are studying detailed interaction at the level of process data, eg second-by-second time-motion study combining video cameras and computer interface data capture. Every summary measure we have looked at so far varies greatly from visit to visit, eg length of visit, ratio of physicians' gaze to computer vs. gaze to patient, amount of computer activity, vocalization and verbal communication patterns. I'm not sure what pre-scores and post-scores refer to. Do you mean, regardless of the measure, the averages (across physicians) are constant in each phase?
Comment: A "score" refers to an individual value of whatever outcome measure you are looking at in each visit, i.e., the dependent variable. So maybe for one model it is the gaze ratio you mentioned, or maybe a value representing amount of computer activity, or whatever. A "pre-score" is a value of this outcome variable during the "pre" phase, and likewise a "post-score" is a value in the "post" phase. Sorry if this confused you, it is pretty standard terminology in my field. What I mean about holding things constant is this:
Comment: If I want to know how scores change when moving from the "pre" phase to the "post" phase I can, conceptually speaking, compare each patient's pre-score to *that same patient's* post-score. Because this comparison is not influenced by the mean for that Patient or for that Physician (i.e., it doesn't matter if the patient is an overall "high responder" or "low responder"; you can add an arbitrary constant to both the pre-score and post-score and still get the same answer regarding change), we can therefore say that the comparison of pre to post is free of Patient variance and Physician variance.
Comment: This is *NOT* to say that the scores don't vary at all! Obviously they do, as you noted. There is probably still a lot of *error variance* left over even after accounting for Patient variance and Physician variance. Anyway, I think I have now answered your original question more than sufficiently. Just go with model (A).
Answer: I think you want to go for something along option A). Obviously, patients are nested within physicians, i.e. you want to account for the effect of physicians on your measurements. Finally, you are dealing with repeated measures on the patient level.
Approaching the multilevel/hierarchical modeling this way guarantees the correct shrinkage towards the appropriate levels of hierarchy. You want the individual time courses to tend towards the 'average' time course for this particular physician, which in turn shrinks toward the sample 'average'.
Comment: Thanks. However, patients are nested within physicians in both A. and B. So that doesn't seem to resolve it.
Comment: Well B) is sort of implying that there is a 'pre- and post-physician' in addition to the 'pre- and post-patient'. In other words, option A) implies that there is something unique about a particular physician, while option B) implies that there is something unique about the time point (pre/post) at which all physicians sees their patients.


How to get coordinates of element during animation in SVG

Question: I have such code:
<defs>
    <g id="g0">
        <circle cx="0" cy="1" r="16" fill="blue" id="pr0" stroke-width="0" stroke="#2986dc"/>
        <circle cx="0" cy="0" r="14" fill="red" id="pp0" />
    </g>
</defs>
<use x="603" y="484" id="u0" href="#g0">
    <animateTransform attributeName="transform" attributeType="XML" type="rotate" id="t0" dur="30s" repeatCount="indefinite" from="0 494 494" to="360 494 494"/>
</use>

How can i get coordinates of group during animation using JavaScript?
Answer: The SVG 1.2 Micro DOM provides the get*PresentationTrait*() class of methods for accessing animated values, but I don't think they are supported yet by browsers. So maybe the easiest way would be to resort to JavaScript animation.
Comment: I guess I was slightly off with my answer, but I found there actually *is* a way to find the current position using `getScreenCTM()`, which returns the animated value of the transformation matrix. I currently don't have a working example, though.
Comment: JavaScript animation is not smooth :( It is looks good only if I animate vector objects. But when I try to animate raster images or  shapes filled by raster images (PNG, JPG) it looks very bad. Shapes have jittering and it looks unacceptable. But using native SVG animation such as animateMotion or animateTransform i get smooth good animation :)


Teaching unmotivated students

Question: I teach 2 ten year old twin boys piano, but they both show no motivation to learn. Every time they come over they have a new excuse for why the couldn't practice at all in the last 2 weeks since I have seen them. Lots of these are valid excuses ("We moved and the piano is not set up yet." or "We went on a vacation") but most of the excuses are complete baloney. ("I couldn't figure out how the metronome works so I didn't practice at all.") Most weeks, they have practiced for a total of 20 minutes. Frequently they will not have practiced at all.
I have been teaching them for over a year, but I don't think they have made any significant progress. At this point, I feel like I could continue teaching them for another 2-3 years and that they would learn very little. Obviously, this is a waste of my time, their time and their parents' money.
Is this indicative of poor teaching on my part? Most of our lessons are working through their books, and I could understand how they might consider music from a book "boring". However, I try very hard to make music fun and interesting, which is why I am always showing them cool music, working with them on songs that they enjoy and choose, and give them tips and ideas on songs that they are writing.
I have asked them if they want to learn piano or if it's just their parents forcing them to take lessons, and they tell me that they love music, and they dream of being a composer one day, and that they like taking lessons from me. However, I don't really believe them. I have also explained to their parents the lack of practice several times, and each time the same thing happens. "We'll work with them on that" and they practice a little bit more for the next couple weeks before falling back into the same habits.
I'm not sure what I should do from here.

Am I the problem? Do I just need to find a better way to motivate and teach them? I am sure that part of the problem is bi-weekly lessons, rather than weekly lessons. However their parents don't want lessons more than once every two weeks.
Should I set a standard? If you are to continue taking lessons from me, you must practice a minimum of X minutes per week?
Should I stop teaching them? If this continues much longer, I must say this at some point, especially since for me, I teach out of a desire to bring the love of music to passionate people that are dedicated to learning about music, not to help parents enforce their vision of "Every child must be enrolled in hundreds of extracurricular activities".  I know that this is possible at their age (my hardest practicing student so far was 9, and I would practice for hours a day when I was 14) even if it's not common.
Comment: @Matthew Read - the clue's in the title - Music-Practice and Theory. Of course it could be construed that the question could relate to any subject, however, it doesn't. Motivating in music specifically, can be very different from motivating in other subjects
Comment: You may want to look at this [question and answer](http://music.stackexchange.com/questions/24570/how-to-make-people-interested-in-what-were-teaching). It's very similar in nature.
Comment: I won't make this a real answer right now in case this is deemed to be a duplicate: I agree with the linked answer that catering to the student can help - I ask students what music they want to learn and teach that while working in "important" stuff like theory on the sly. Also, sometimes things outside your control will rule the day. When I was 13 I was very depressed and quit everything, including piano lessons. No teacher or teaching technique would have been able to stop me that in my case.
Comment: One more note: when I taught myself guitar it was easy because I never felt like I was practicing anything, I was just playing guitar (very badly for the first few years). I would miss classes because I was playing guitar instead. I've never felt that way about another instrument. Loving music is good, loving the instrument is unstoppable.
Comment: On a personal note - I was much like that age 13, & eventually gave up my piano lessons age 15, much to my chagrin now as an adult. I did, however pick up guitar & bass & eventually made a career of it. They may love music, but not like their piano lessons. It doesn't make you a bad teacher, it just makes them 'teenagers'
Comment: Nothing about this is specific to music.
Answer: I actually had a piano teacher politely ask me if I was sure I wanted to keep going, as I wasn't really that interested in playing, but sort of went (I was about 10 years old). I regret that, as a few years later I picked up the guitar and loved it.
I think the thing was, I was playing stuff I just couldn't relate to - at all. 'Oh when the saints go marching in' really didn't peak my interest.
I would suggest teaching them songs for 6 months or so, no theory, no scales, not even with sheet music if you can help it (maybe chord charts for reference with the lyrics to songs). Also, the songs which you teach, only teach them songs that they love listening to. I know if I were taught Michael Jackson songs I would probably still be playing piano today.
As an additional insentive, always have a new song 'in the wings' so you can say 'once you have got this right, we will learn this'.
Eventually, when they get stuck because something is too complicated, that might be the time to introduce some scale practice to help.
Finally, you could get one to sing the song they love while the other plays, and then swap around (or perhaps one plays left hand and the otherright). This way, they learn to play together - as right now, if one is practicng, what is the other one doing? sitting around bored waiting for their turn, or worse on an Xbox distracting the other twin from their practice! I would be suprised if the house has two pianos!
Comment: I like this answer too. I have tried doing this, but not quite to the extent that you recommend.
Answer: Life's too short for such nonsense. You aren't enjoying the situation and they probably aren't feeling too good about maintaining the charade either.
Kids are often much better at 'being diplomatic' and telling people what they want to hear than adults are. Or maybe it's true that they are serous about music, but piano isn't their thing - it was never mine, and I love music (and still want to be a composer :) or it may be that for whatever reason, your teaching style isn't right for them.
If you've already brought up the issue several times, I wouldn't bother with any further ultimatums or crisis meetings - the danger is that you end up carrying on with more ill-feeling. Just say that for whatever reason, they're not making progress, and you absolutely can't keep taking their money. Perhaps suggest a break for a couple of months and, if the kids actually decide that the lessons were a good thing, you can consider taking them back on your terms (maybe weekly, and with strict practice rules).
Comment: Yeah, I'm inclined to agree, especially as this can be done quite diplomatically.
Comment: @Patrx2 - exacltly - making an executive to finish it on good terms is going to be better than forcing the issue, which may make the kids *actually* start hating the piano!
Comment: I like this answer. It makes it clear (to them and their parents) that it is not working, but still leaves the door wide open for them continuing to take lessons from me if they get a little more serious.
Answer: One aspect of this that I don't think has been addressed (unless I missed it in the other answers!) is that, in my opinion, teachers should protect themselves from being "blamed" for lack of student progress in these cases. If one teaches kids as a business, it's only a matter of time before one comes across a parent that says some version of, "What am I paying you for!? It's been a year and my kid still sounds like he hasn't had a lesson!"
One concrete way to do this, especially for kids of this age, is to be very specific about what you want them to practice, write it down at the end of each lesson, and have the students record each day (in either a notebook, or on a sheet that you print and give to them) what they practiced. There are many possible versions of this. If you know that the students have parents or guardians that have the time and availability, you could even have the parents just record each day whether or not the student practiced for at least 20 minutes (or whatever.) If they can't practice on a given day, just have them jot down the reason.
Having a discussion with the parents is a good idea as well. Suggest to them that setting a specific practice time each day might be a good idea. Maybe an analogy with say, soccer practice, could be useful. Soccer practice takes place at a certain time, in a certain location. Just as a parent might ensure that their child gets to and from soccer practice, so too they should ensure their child gets "to and from" piano practice. (Not all kids have parents that are able to be super involved in the details of their children's activities, so you will have to use some judgement/care/tact here! I certainly would avoid language that tells a parent what they "should" do.)
Of course, these things can sometimes be good ideas for the students even if they aren't having practice issues, and it does more than just protect the teacher from being blamed for non-progressing students. Sometimes adding a little structure can increase productivity.
Answer: I would have a nice little sit down with the parents and discuss the situation with them. They may not be aware of exactly how there little tyrants are taking them for a ride.
If you gently let them know that you are not getting co operation and that yes you can still take there money if they so wish but it is not fair on them to pay for a teaching service that does not actually teach their children anything.
You should also consider asking there parents, au pair or some sort of parental figure to sit in on the lessons.
Answer: Even if you can't teach them piano, at least you can try to teach them (and their parents!) that that what the kids (don't) do has consequences. Next time they admit they haven't done any practice, end the lesson right there and show them the door. Make it clear that you aren't going to spend any more time with them unless they do their "homework".
If this results in a visit from their parents, you are in a better position to have a meaningful discussion about the situation than the "yeah, I hear what you say" response when got when you tried to talk to them.
Comment: I agree, but I think this might make them dislike lessons (from me or otherwise) even more.
Answer: How carefully do you map out your instructions for home practice?  Some students need things written down in a very explicit way.
Catch the child displaying the desired behavior and reward it.  (To figure out how to reward the child, you have to use some intuition, experimentation and observation.  For example, on a really good day, my son's teacher allowed him to fool around on her electric organ for a few minutes at the end of the lesson.)
How involved are the parents?  How good a communication channel do you have with them?  Is it possible to involve them more?  (The extreme example of this is the Suzuki parent coach, but you might be able to use them less intensively.)  Would they be willing to hire a teenaged practice buddy?  Homeschoolers are often interested in working in this way.
In the worst case, you may want to do as a viola teacher I know did for one student.  The student had done good work with her viola in previous years, but in high school her ballet schedule became extremely intense.  The teacher was basically the practice buddy in the lessons.  Of course, the student did not progress very quickly during those periods, but there was still a benefit of continuing the lessons.
Perhaps you can get these twins playing piano more at home by assigning simple duets.  Alternatively, you could have one of them "accompany" the other with some percussion.
Answer: I am new to this forum, hence not sure of the proper parameters or protocol. However,  I do have a personal experience that may apply  to 'talented' yet unmotivated students. I began playing piano on my own at age 4. No one forced me. As some of the answers suggested, around age 12 or 13 I had other interests, and resented being forced to travel on the subway for my lesson.  I played stickball, then hid in the closet till my teacher informed my parents that I never showed up. This may not be the right way, but my folks actually kicked me out of the house at dinner time and had to eat in a diner. Needless to say, this was 1970 and it was not as dangerous to be out at that age in NYC. In any case, I learned the hard way to continue, as I was interested in music in any case and have not regretted the decision they made as I did go on and majored in music in college and continue to enjoy it.
Perhaps this seems harsh but it made an indelible impression on me and I'm glad.
Maybe there are other ways to see how motivated students are, certainly that incident got me back on track. I realize there are more 'toys' etc that interest music students these days, however that worked for me.

Teaching unmotivated students


looking for a complete documentation for Drupal's database schema - need to have some overview

Question: well dear buddies,
i am pretty new to drupal. Just set up a drupal commons
now i 'd like to add some data to my drupal, and i dont really understand the database model/scheme. Can somebody point me to some documentation? For example i need to understand the role of node, vocabulary, and all the tables and what is tid,vid,nid fieds... Then i could add data the sql way.
Of course I don't think complete documentation for Drupal's database schema exists. Well does any body have a  very very first attempt:
An ongoing attempt with comments for Drupal 6 can be seen here
http://drupal.org/node/79874
An analysis with SchemaSpy i found here
http://webdevgeeks.com/schemaspy/index.html
and overview i have seen here.
http://upsitesweb.com/sites/upsites.co/files/drupal7_model_0.png
well who can give more hints - i love to hear from you -
Again: i want to have some overview on the options i have to extend the group
(running on http://www.schulcenter.org - and see here http://schulcenter.org/?q=group/schulen )
How can i configure / extend the groups (/on the database-level)?
How can i configure / extend the groups (/on the database-level)?
i wanna creat the following fields - (derived from the following shedule https://scraperwiki.com/scrapers/austrian-schools/ ) see the names of the collumns

name
adresse
schultyp
homepage
ergänzender schulname
beschreibung
schwerpunkte

can this be done best in groups !? Can i this set in the groups. Or should i do that in the so called - organic-groups
Note: as you see the site schulcenter.org should give an overview on schools
with a certain amount of data from the profile: the adress, the website..

a little image (a thumbnail of the website - shown in a block,)
a little description (approx 200 characters) what the site /the school does - some special programmes or something like that...

How to achieve this !? Love to hear from you any and all help will greatly be appreciated
love to hear from you
zero
Answer: Check Drupal's Schema API. Here is the link to it. Also the Book "Pro Drupal Development" has an excellent chapter on Working with Databases.You might find that one useful too.
Comment: hello dear aeozeppelein thx - i do so
Answer: There are two answers to your question.
1) Open up a drupal installation in a file browser. Go to drupal/modules/node. The core node module does two things: a) It implements an entity under the entity API (not to be confused with the Entity API contrib module), and it creates the Node API. Most people trying to create content types in code will use the Node API. Adventurous people will use the Entity API.
So... Look at /drupal/modules/node/node.install. This file implements hook_schema(), which shows an abstracted version of the DB schema used by node, helpfully documented (to some degree anyway). If you want to hack around with nodes in SQL, then this is a good place to start learning the system.
2) Disregard answer #1. Use the Node APIs and Field APIs to manipulate Drupal data. This gives you the benefit of everyone's eyes on bugs, and not needing to update your code (as much) for future updates and upgrades. Poke around node_example, nodeapi_example, and field_example in the Examples project. This gives you good code to emulate/copy/paste.
HTH.
Comment: hello dear Paul_m - many many thanks for the great hints and the input. Your help is greatly apprecated - the idea to benefit of everyones eye on bugs is great - i will digg into it and come back and report all my findings - greetings zero
Answer: My favorite resource on Drupal's schema is Padraig O'Sullivan's Digging into Drupal's Schema:
http://posulliv.github.io/2012/08/02/drupal-er-diagram/


How can I add a dynamic redirect on a 404 hit?

Question: I Created a site and URL that returns a 404 Not Found status on www.xyz.com This creates a loss of link equity to the domain and negatively impacts the ability to compete in organic search results.
I wanted to Redirect the URLs returning a 404 status to the defined closest alternative, or parent category, to reclaim link equity from external domains.
Example: Suppose I am using URL http://www.mysite.com/about/xyz/ and it return 404 so I wanted to create any module or system that smartly redirect to nearest parent path like "http://www.mysite.com/about/"
I am not able to find any specific documentation which would point me in the right direction.
Comment: Search 404 module?
Comment: @Kevin Thanks for your comment actually  it's a nice idea I tried that module but it does not fulfill my Exact requirement.
Answer: Try creating a custom event subscriber. If the page is 404, grab the original URL and redirect accordingly.
To do so, create a new module ("response_test" in this example), with the usual .info.yml and .module file.
Then register your event subscriber in the module's services file... (see EventSubscriber Example for a full example.
./response_test.services.yml
services:
    response_test.response_subscriber:
      class: Drupal\response_test\404RedirectResponseSubscriber
      tags:
        - { name: event_subscriber }

Then create a class to handle the event
./src/EventSubscriber/404RedirectResponseSubscriber.php
<?php

namespace Drupal\response_test;

use Symfony\Component\HttpKernel\Event\FilterResponseEvent;
use Symfony\Component\HttpKernel\KernelEvents;
use Symfony\Component\HttpKernel\HttpKernelInterface;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpFoundation\RedirectResponse;

class 404RedirectResponseSubscriber implements EventSubscriberInterface {

  public function onRespond(FilterResponseEvent $event) {
    if ($event->getRequestType() !== HttpKernelInterface::MASTER_REQUEST) {
      return;
    }

    $response = $event->getResponse();
    if ($response->getStatusCode() == 404) {

      // Get the old path somehow
      // @see https://stackoverflow.com/q/11047305
      $oldPath = ''; // Try using $response->getUri() or $response->headers->get('location')

      // Manipulate the $oldPath to remove the last part
      // @see https://stackoverflow.com/a/51573313/5771750
      $redirectPath = dirname($oldPath);

      $new_response = new RedirectResponse($redirectPath );

      // Prepare a new Symfony\Component\HttpFoundation\Response and use
      $event->setResponse($new_response);

    }
  }

  public static function getSubscribedEvents() {
    $events[KernelEvents::RESPONSE][] = array('onRespond');
    return $events;
  }

}

Note: This answer is partly based of Clive's answer here: https://drupal.stackexchange.com/a/86622
Comment: Thanks @Chris :D really helpful for me.


Criar um ficheiro bath que realize o DIR de uma pasta ou conjuntos de pastas que o utilizador escolha

Question: Queria criar um ficheiro em bath onde o utilizador escolhesse o local onde realizar o comando DIR.
Eu tenho isto feito:
@echo off

set /p pasta=digite uma pasta: dir %pasta%>>c:\conteudo2.txt

if exist %pasta% goto existe

if not exist %pasta% goto nao

:existe

echo file found

goto fim

:nao

echo file not found

:fim pause

O meu problema é esse:

" Crie a partir da linha de comandos um ficheiro com o nome Listar2.bat, que permita guardar dentro de um ficheiro com o nome Conteudo2.txt, no ambiente de trabalho, o conteúdo de um diretório fornecido pelo user. "
Comment: Explique melhor como você quer que seja feita essa escolha, e mostre o que você já tem pronto. Senão a pergunta fica ampla demais para o formato deste site (dê uma olhada no [tour]).
Comment: Por que excluir a pergunta, @LaércioLopes?
Answer: Seu código está correto, só precisa pular a linha antes do comando dir e do pause.
Ficando assim:
@echo off

set /p pasta=digite uma pasta:

dir %pasta%>>c:\conteudo2.txt

if exist %pasta% goto existe

if not exist %pasta% goto nao

:existe

echo file found

goto fim

:nao

echo file not found

:fim

pause

Comment: Não é mais fácil usar `dir [caminho]` direto? Pra que o batch nesse caso?
Comment: Verdade né? mas ela quer um batch...
Comment: A pergunta não está clara. Pode ser que ela queira isso, pode ser que queira algo bem mais complexo como o midnight commander.
Comment: Pra mim parece bem clara apesar de ser muito simples.
Comment: Não estou entendendo esses seus pedidos de excluir a pergunta.
Comment: Por acreditar que dificilmente essa pergunta ajudará outros usuários.
Answer: deixo aqui o meu script ...
@echo off
mode 70,8
:main

echo.&echo. Arraste uma pasta e dˆ enter.
set /p caminho=^>:

if not defined caminho (cls &goto main)
set caminho=%caminho:"=%
if not exist "%tmp%\Lista-dir" (md "%tmp%\Lista-dir")

cls
echo.&echo. Escolha uma Op‡Æo: &echo.

echo. [1] - Listar somente pastas
echo. [2] - Listar somente pastas e Subpastas
echo. [3] - Listar pastas, Subpastas e arquivos.

CHOICE /c "123" /n /M:

if %errorlevel% equ 1 (dir /b "%caminho%">%tmp%\Lista-dir\listagem.txt)
if %errorlevel% equ 2 (tree /a "%caminho%">%tmp%\Lista-dir\listagem.txt)
if %errorlevel% equ 3 (tree /a /f "%caminho%">%tmp%\Lista-dir\listagem.txt)

explorer %tmp%\Lista-dir\listagem.txt

Answer:
@echo off && setlocal enabledelayedexpansion

type nul >"%userprofile%\desktop\conteudo2.txt" & set /p "_pasta=digite uma pasta: "

2<nul dir "!_pasta!" | find ".." >nul || goto :_nao

:_existe
rem :: copia p/ seu "Crtl+c" o conteudo do arquivo "%userprofile%\desktop\conteudo2.txt"
dir !_pasta!>>"%userprofile%\desktop\conteudo2.txt" && clip < "%userprofile%\desktop\conteudo2.txt"
echo file found & goto :_fim

:_nao
echo file not found

:_fim
echo/Pressione qualquer tecla para continuar. . . && timeout /t -1 >nul


How do I modify the breadcrumb?

Question: I want to modify my bread crumb ( remove one link from bread crumb and add one link to bread drumb).
I tried,
/**
 * Implements hook_system_breadcrumb_alter().
 */
function modulename_system_breadcrumb_alter(\Drupal\Core\Breadcrumb\Breadcrumb &$breadcrumb, \Drupal\Core\Routing\RouteMatchInterface $route_match, array $context) {
// to get current path and alias
  $current_path = \Drupal::service('path.current')->getPath();
  $result = \Drupal::service('path.alias_manager')->getAliasByPath($current_path);

// get current route name
  $route_name = \Drupal::service('current_route_match')->getRouteName();
// check route is an quickinfo content create in group
  if ($breadcrumb && $route_name == 'entity.group_content.group_node__quickinfo.create_form') {
// get bread crumb links
    $breadcrumb_links = $breadcrumb->getLinks();
// fetch titile of the current path
    $title = \Drupal::service('title_resolver')->getTitle(\Drupal::request(), $route_match->getRouteObject());
    if (!empty($title)) {
// change third link of a breadcrumb
     $breadcrumb_links['2'] = Link::createFromRoute(t('RZ-schnellinfo'), '<none>');
    }
//    set links again
   $breadcrumb->setLinks($breadcrumb_links);
  }
}

SetLinks() ==> work when only bread crumb links doesn't exist, i.e.. we can create new links by using addLinks(). Since we need to remove a link we can use this function .
Then I followed https://www.drupal.org/node/2026025  but I am getting below error.

exception
  'Symfony\Component\DependencyInjection\Exception\LogicException' with
  message 'Service                                    [error]
  'xxxxx.breadcrumb_xxxx_create' for consumer 'breadcrumb' does not
  implement Drupal\Core\Breadcrumb\BreadcrumbBuilderInterface.' in
Answer: As comment said above, you have to implement Drupal\Core\Breadcrumb\BreadcrumbBuilderInterface. Here is a working snippet that I use to implement those Breadcrumbs :
It appends a custom routename to all 'Nodes' pages :

<?php

namespace Drupal\Your_Module_Namespace\Breadcrumb;

use Drupal;
use Drupal\Core\Breadcrumb\Breadcrumb;
use Drupal\Core\Breadcrumb\BreadcrumbBuilderInterface;
use Drupal\Core\Link;
use Drupal\Core\Routing\RouteMatchInterface;

class Your_Module_NamespaceBreadCrumbBuilder implements BreadcrumbBuilderInterface {

  /**
   * {@inheritdoc}
   */
  public function applies(RouteMatchInterface $attributes) {
    $parameters = $attributes->getParameters()->all();
    // I need my breadcrumbs for a few node types ONLY,
    // so it should be applied on node page ONLY.
    if (isset($parameters['node']) && !empty($parameters['node'])) {
      return TRUE;
    }
  }

  /**
   * {@inheritdoc}
   */
  public function build(RouteMatchInterface $route_match) {
    $breadcrumb = new Breadcrumb();
    $breadcrumb->addLink(Link::createFromRoute('Home', '<front>'));

    $node = Drupal::routeMatch()->getParameter('node');
    $node_type = $node->bundle();

    // Put the needed logic here

    // addLink example
    $breadcrumb->addLink(Link::createFromRoute('My Page Name', 'drupal_internal.routename'));
    $breadcrumb->addCacheContexts(['route']);

    return $breadcrumb;
  }
}

Answer: Above in the start you should use this class .
use Drupal\Core\Breadcrumb\BreadcrumbBuilderInterface;

How do I modify the breadcrumb?


how to show NSLog in handleWatchKitExtensionRequest

Question: I have put a bunch of NSLog statements in handleWatchKitExtensionRequest in my iOS app delegate. When I initiate an action from apple watch that calls handleWatchKitExtensionRequest, it does the action but all of the print outs that I have put for debugging purposes never show up in Xcode. I believe this is because the simulator is running watch app at that time. It is frustrating not knowing what going on at the iphone end of things.
Is there a way to show those debug statements?
Answer: To look the NSLogs in your console, you should attach your iOS app also going to Debug->Attach to process and select your iOS app which should be running at that time.

If you follow the above steps, you will see both iPhone app and applewatch running in debug navigator, and you will see the NSLogs.
Comment: It's worth noting that attaching to a process doesn't redirect console output.
Comment: i did the exact steps above with my watch app. but it does not work
Comment: do you see both iPhone app and Applewatch running in the the debug navigator?
Comment: yes I do. This is what I did, i launched the app on the watch simulator first. once it came up and then did a debug attach to iOS simulator. i saw both of them like you had shown. But the Xcode log console was empty. It stopped showing any logs from watch or iphone
Comment: handleWatchKitExtensionRequest is launched when you call  openParentApplication. Are you sure you are calling [WKInterfaceController openParentApplication:dictionary reply:^(NSDictionary *replyInfo, NSError *error){....}];   from Applewatch?
Comment: I couldn't see the nslogs / use breakpoints of the iPhone app,  until i attached the iPhone app process.   See this link: http://www.raywenderlich.com/94672/watchkit-faq   "How can I run and debug both the iPhone app and the Apple Watch app at the same time using the simulators?"
Answer:
Run watch app.
Open from the simulator you iOS app
Attach the debugger to the iOS app
Finally execute the code that calls handleWatchKitExtensionRequest on your watch.

It's all in executing in the correct order.


Pivot a table in DAX and sum column by groups

Question: I read in data from an ODBC source, and created measures based on the values of various columns in the data source using DAX formulas. I then created a new table to try to pivot it from this wide version

to this.

I've tried this code and may variations of it but I only get something that aggregates by indicator, not by indicator, month, and Code.
risk - MTD = UNION(
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[CV] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "CV"),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[Closed] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "CP"),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[EF] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "EF"),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[IOF] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "IOF"),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[MRP] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "MRP"),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[MR] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "MR"),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[OF] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "OF"),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[AP] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "AP"))

How can I adjust my code to get the desired output?
Answer: Try:
risk - MTD = UNION(
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[CV] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "CV", "Value", 'RSQL'[CV]),
SELECTCOLUMNS(FILTER('RSQL', 'RSQL'[Closed] >=0), "FacilityCode", 'RSQL'[FacilityCode], "Month", 'RSQL'[Month], "IndicatorName", "CP", "Value", 'RSQL'[Closed]),
...,
...
)


What algorithm would be appropriate to find nearest neighbors based on transaction history

Question: I have two data sets with transaction history of customers by date and product (de-identified). These are from two different sources and have different capture rate (e.g.: One might have 5 transactions for a customer and the other may report 7). I also have information like ZIP, age, gender for these customers.
What machine learning algorithm can I leverage to find same customers across the sources based on similarity in transnational history?
I was thinking if somehow collaborative filtering may be of help here.
Thanks in advance!
Answer: Nearest Neighbours is already a clustering model that automatically will cluster items within a data set.
https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
Another similar algorithm is K-Means, very effective also that requires low computing power.
The key point here I can guess is that you will need to make some research on data normalization and data scaling. Yoy may have to transform your data converting categorical features into binary ones and other stuff.
The first step is deciding which algorithm you will use and the second one is being aware of what type of data it expects.
Comment: Thanks for your response Ginzalo. KNN is definitely one way of doing it, but will it also consider the date stamps of these transactions and the sequence of transactions into consideration? I have sequence of events for each customer in the table. I need to match the customer based on similarity of event occurrence in a sequence
Comment: Collaborative filtering won't capture the sequential aspect. You could look into time series modeling.
Comment: You may need to use the timestamps for joining the two tables, but you should not use timestamps within your statistical models.

Instead of that create derived items such as days differences, differences in weeks, day of the week (using binomial columns), holiday days, working days, quarter (again using binomial columns), etc...
Comment: I forgot to mention, if you want some non-linear classification tool you may also check Random Forest and Support Vector Machines as they can be used for classification.

Neural networks are also good for that purpose but they're much more complex to configure and the results not necessarily would be better.


Reference to angular distance metric properties.

Question: I know that the Angular Distance is a proper metric but I'm struggling to find a reference that states that and proves all the properties for that distance. In the book Mining of Massive Datasets (Page 95) the authors briefly argue that the properties hold for what they call "Cosine Distance", which is just the angle between two vectors.
However, I've read that the Angular Distance must be normalised in order to be considered a metric.

Does the Angular Distance have to be normalised for it to be a metric?
Can you point me to a reference that proves that the Angular Distance is a proper metric?
Answer: Metric
Let $X$ be an arbitrary set. A function $d: X \times X \rightarrow \mathbb{R}$ is called a metric on $X$, iff
        the following properties hold true for all $x, y, z \in X$:

identity of indiscernibles: $\forall x, y \in X: d(x, y) = 0 \Leftrightarrow x = y$
symmetry: $\forall x, y \in X: d(x, y) = d(y, x)$
triangle inequality: $\forall x, y, z \in X:d(x, z) \leq d(x,y) + d(y, z)$

Cosine Distance
Cosine distance is defined as:
$$d_{cos}(x, y) := 1 - \cos(\theta) = 1 - {\mathbf{A} \cdot \mathbf{B} \over \|\mathbf{A}\| \|\mathbf{B}\|} = 1 - \frac{ \sum\limits_{i=1}^{n}{A_i  B_i} }{ \sqrt{\sum\limits_{i=1}^{n}{A_i^2}}  \sqrt{\sum\limits_{i=1}^{n}{B_i^2}} }$$
Cosine distance is not a metric in $\mathbb{R}^n$, because the identity of indiscernibles does not hold true:
$$d(0.5, 1) = 1 - \cos(0°) = 1 - \frac{1}{1} = 0\text{, but } 0.5 \neq 1$$
To the question if cosine distance is a metric on $S_n = \{x \in \mathbb{R}^n: ||x|| = 1\}$:
\begin{align}
x &= y\\
\Rightarrow d(x,y) &= d(x,x)\\
&= 1 - \sum_{i=1}^n x_i^2\\
&= 0
\end{align}
So that looks good.
We also know that $\cos(\theta) = 1 \Leftrightarrow \theta \in \{0°, 180°\}$. This means it still does not have the identity of indiscernibles attribute. We have to reduce the space to the non-negative unit sphere $S_n^+ = \{x \in \mathbb{R}^n: ||x|| = 1 \land x^{(i)} \geq 0\}$.
Comment: The cosine distance is not even a distance for vectors of uni length because it violates the triangle inequality. Also, the question was not about the cosine distance but about the angular distance `d_a(x,y) = \theta(x,y)` which would also violate the identity of indiscernibles, but the triangle inequality should hold.
Comment: Can you show the triangle equation for the angular distance? Geometrically it is clear, but I am not able to formalize it.


Constructing $\sqrt{\sqrt{1+x^2}-1}$ to be smooth (cancelling the square)

Question: $\DeclareMathOperator{\sign}{sign}$
Is there a way to rewrite $f(x)=\sign(x)\sqrt{\sqrt{1+x^2}-1}$ using (smooth) elementary functions?
As far as I can see the function seems infinitely differentiable, but is composed of two non-differentiable factors. Is there a way to write the same function (equal on $\mathbb{R}$), but composed using purely smooth functions? (So without $sign(\cdot)$.)
(This function was my attempt at making a function with linear behaviour near $x=0$, and $\mathcal{O}(\sqrt{x})$ behaviour for large $x$.)

How far I've got
I realised that the non-differentiable point comes into existence whenever you take the square root of a function for which its first two Taylor series terms are zero, i.e $\sqrt{g(x)}$ for $g \approx \mathcal{O}(x^2)$.
For example:
For $g(x) = 1 - \cos{2x}$, we have
$$\sign(x)\cdot \sqrt{g(x)} = \sqrt{2}\sin{x}$$
on the domain $[-2\pi, 2\pi]$.
This is due to the identity  $\cos{2x} = 1 - 2\sin^2{x}$.
Thinking about it this way, the question thus becomes whether there exists an odd elementary function $h(x)$ for which $h(x)^2 = \sqrt{1+x^2}-1$.
I have found the answer while typing this up; will post soon.
Answer: $\DeclareMathOperator{\sign}{sign}$
I have found the answer before even posting the question; let me know if that is not allowed, but I found the answer quite interesting, so I thought I would share. Try it yourself before looking at the answer if you want!

The solution is $$h(x) = \frac{x}{\sqrt{\sqrt{1+x^2}+1}} = \sign(x)\sqrt{\sqrt{1+x^2}-1}$$
Inspired by my success with the sine/cosine identities, I tried a substitution $x = \sinh{u}$, and indeed, it works wonderfully! I found $h(x) = \sqrt{2}\sinh{\frac{u(x)}{2}}$, and from there a inverse substitution with some algebra (assisted by Wolfram Alpha) got me the answer in no time.
In hindsight, an easier (but harder to come up with, perhaps) way of finding the answer is by recognising $\sqrt{1+x^2}-1$ as a factor of $x^2$, along with its conjugate $\sqrt{1+x^2}+1$.
Comment: @808GroundState Well, I already had the question written out, and figured that maybe other people might be interested? Though I suppose this too obscure a topic to be a good wiki question.
Comment: Apparently I can't accept my own answer until after 2 days...
Comment: You can indeed answer your own question, this is well within house rules. However, if you found an asnwer to your question before posting the question, I have to ask *why* you did that?
Comment: That's a fine response.


SQL Query to find Parent-Child Child-Parent relationships?

Question: o_O
If I have the following records in a table:
Parent     Child
1          2 <--
2          1 <--
3          2
3          4

etc...
And I want to identify records that are both the parent of their child AND the child of their parent such as the 2 records identified by arrows above, how would I accomplish this?
I am trying to run some recursive SQL on this table, but these items are causing an infinite loop. I would like to identify these items so they can be addressed manually.
My brain is fried-enough from messing with recursive queries, I have nothing left to solve this one. Please help :)
Answer: If understood you well, you don't need recursion at all:
 SELECT a.parent, a.child
 FROM table1 a
 INNER JOIN table1 b ON (b.child=a.parent and a.child = b.parent)

You might want to use LEFT JOIN instead of INNER if you also need to display rows that don't satisfy condition .
Answer: The following query will work in your example case. If it needs more you'll have to extend the demonstration information
;WITH CTE_DATA AS (
    Select Parent = 1, Child = 2
    union Select Parent = 2, Child = 1
    union Select Parent = 3, CHild = 2
    union Select Parent = 3, Child = 4
)
select
    d1.*
from
    CTE_DATA d1
    join CTE_DATA d2 on d1.Child = d2.Parent and d2.Child = d1.Parent

Answer: DECLARE @YourTable TABLE (Parent INT, Child INT)

INSERT INTO @YourTable
SELECT 1, 2
UNION
SELECT 2, 1
UNION
SELECT 3, 2
UNION
SELECT 3, 4

SELECT *
FROM @YourTable A
INNER JOIN @YourTable B
ON A.Parent = B.Child AND A.Child = B.Parent


Optimize a function with $30,000$ variables. Possible? Reasonable?

Question: I have a nonlinear optimisation problem of the form:
$$ \min \:~ (a - \rho_{a})^2 + (b - \rho_{b})^2 + (c - \rho_{c})^2 $$
where $a,b,c$ are constants, and $\rho$ are different functions of around $30,000$ variables, say $x_{i}$. The nominator fo these functions includes a weighted sum of most of them, plus a weighted sum of their pairwise combinations, including squares. The denominator includes a type of "norm", this is the squared root of the squared sum of each of them. The key difference between the three $\rho$ functions is the subset of variables included. But there is huge overlap.
I also have lower and upper bounds for each of these variables.
Is there any hope I can achieve a meaningful optimisation? If so, would you recommend a particular language/optimisation tool? I'm open to anything really.
Comment: Instead of describing in English what the $\rho$ functions look like, $\LaTeX$ them. Parsing natural language is painful.
Comment: I think it does not make much difference what the function actually is (beyond that is is non-linear, with variables on the nominator and denominator). But if more people suggest that, I will.
Comment: It sounds like you have large co-variance matrix.  Which means that is should be pretty easy to calculate your objective function.  If you wanted to be analytical with it, you would find the principle components.  Likely a very small subset of your variables drive the changes to the objective function.  The other approach would be to perturb the variables, find the sensitivity of your objective to each variable.  Optimize the variable with the greatest sensitivity, then step to the next most significant.
Comment: @DougM The formulas actually are the correlation between three vectors of around 80,000 observations where in total around 30,000 are missing. I think that most of the variables are of importance, and there are not many "outliers" that could drive the relationship. PCA might not work. I cannot find a reasonable introduction to perturbation methods, or implementations in e.g. Matlab. Can you elaborate on that please?
Comment: It would be helpful to see the analytical form of the functions. This might help in writing  an analytical upper bound or lower bound to the objective function that will help in improving the convergence time of the optimization algorithm.
Answer: For that number of variables, the trick is to be careful and not choose an algorithm that requires the computation of a dense Hessian.  There may be a better way to cache things, but a matrix-free inexact Newton method should work.  Basically, it's Newton's method where the optimality system is solved inexactly using some kind of Krylov method like conjugate-gradient.  Both trust-region and line-search variants exist, which should guarantee convergence.  As far as the performance, it will depend greatly on how the eigenvalues of the Hessian are distributed.  If they're well clustered, then it should work well.  If they're not, it'll perform poorly.  Figuring this out can be, well, a pain because for that many variables it's unlikely that you could compute the dense Hessian and just check.  Really, the best option is to just code the derivatives and try the algorithms to see if they work.
As far as software, Optizelle implements the requisite algorithms and has interfaces to C++, Python, and MATLAB.  It's also BSD licensed.  Use the development version as it works much better.  One of these days, I'll post some installers, which'll make installation easier.  If you ask on the support forum, I'll do it sooner rather than later.
In any case, 30000 variables by itself isn't a problem.  I've scaled certain problems out to half a billion variables, but they had a relatively compact spectra and I had a lot of computation to calculate the requisite derivatives.