Como setar SSL no cookie e exibir no navegador?

Question: Meu site no local usa SSL, carrego por https://, então, ele deveria setar o protocolo SSL, minha dúvida é porque quando acesso pelo console do Chrome, em Application > Storage > Cookies, onde tem o cookie, o item Secure não está checado como os cookies remotos do google por exemplo?
public function renewCookiePublic($token,$user)
{

     $protocoloSSL = FALSE;
     if ((isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https') ||
        (isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] == 'on')) {
            $protocoloSSL = TRUE;
     }

     setrawcookie("_token_access", $token, time()+10800,'/', '',  $protocoloSSL);
     setrawcookie("_user_access", trim($user), time()+10800,'/', '',  $protocoloSSL);

 }

Comment: Veja [isso](https://www.php.net/manual/pt_BR/function.setcookie.php).
Answer: Primeiramente eu não usaria:
isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https'

Isso pode ser manipulado facilmente, já que vem via headers da requisição HTTP, podendo ser usado por algum servidor com alguma configuração em cima do Fast-CGI no próprio back-end, ainda sim ela pode ser envia via "front-end", o que poderiam fraudar facilmente a requisição fazendo algo assim:
curl -i -H "X-Forwarded-Proto: https" http://site/teste

Na verdade ele é mais usado para um "controle" de "proxy" como https://www.nginx.com/resources/wiki/start/topics/examples/forwarded/#how-to-use-it-in-nginx, mas realmente se o $_SERVER['HTTPS'] não esta "populado" creio que seja uma má configuração no "proxy reverso" ou "load balancer", eu não entendo muito.
Esses headers (de Forwarded) se eu não me engano são voltados pra configurações nestes tipos de "proxies" (tudo no servidor apenas) e não deveriam ser expostos ao cliente e nem ser misturado com o que vem do lado cliente, mas muita gente configura nas coxas isso.
Posso estar enganado, mas do que entendo de servidores, se estiver tudo configurado ok só isso sempre deve funcionar:
if (isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] == 'on') {
   ...
}

No seu script não vai "afetar", porque na verdade ele esta meio sem sentido, o próprio uso da IF esta desnecessário (mas isso vou falar mais abaixo)

Sobre o problema do cookie sem HTTPS
Tem que ter certeza de que você não criou acidentalmente já os cookies fora do HTTPS, porque ai será outra coisa, agora se tratando de setar no momento que acessa HTTPS
Porque pelo seu código, se executar com HTTPS e sem HTTPS os cookies vão ser gerados sempre, então provavelmente ambos existem, se a intenção é só autenticar em HTTPS então apenas isto bastaria:
public function renewCookiePublic($token,$user)
{
     setrawcookie("_token_access", $token, time()+10800, '/', '', true);
     setrawcookie("_user_access", trim($user), time()+10800, '/', '', true);
}

Nem precisaria checar se é HTTPS, porque como a propria documentação diz sobre o quinto parametro da função setcookie e setrawcookie:

Indica que o cookie só poderá ser transmitido sob uma conexão segura HTTPS do cliente. Quando configurado para TRUE, o cookie será enviado SOMENTE se uma conexão segura existir. No lado do servidor, fica por conta do programador enviar esse tipo de cookie somente sob uma conexão segura (ex respeitando $_SERVER["HTTPS"]).

Então provavelmente no teu storage existe o cookie sem HTTPS checado porque você realmente gerou, seja por ter acessado sem HTTPS e depois com HTTPS ou por simplesmente esta tua IF talvez estiver errada e o teu servidor Apache ou Ngnix estiver configurado errado.


Standard sheet music file

Question: I am working on a project that converts raw sheet music into MIDI, mp3 files.  I did this because right now many of the sheet music players I have heard are very bad and I think I can do better.  In any case I was wondering if there was a standard for storing sheet music.  I know a lot of sheet is stored in pdfs.  Is there another way people save sheet music something that is easier to read if you are a computer?
Thanks in advance!
Comment: Pretty much nope. There is MusicXML which is an open standard but it's mostly used for import/export between music apps.
Comment: @AndyBonner Real sorry but I think this bunch of comments are not really adding to the question here but are more questioning if the question is sound (pun intended) and talking the op out of his/her/its goal. IMO the question is sound but questioning why someone would ask it is not really the subject..
Comment: You've got quite a challenge there—you're talking about OCR, Optical Character Recognition, and  it's a lot harder for printed music than for verbal text. But people have already been working on it for decades. There's probably something you can drop into your project rather than reinvent the wheel.
Comment: 'many are very bad'… that's pretty much inherent in anything as hard-quantised & emotion-free as a raw score, just like a raw midi file. BTW, this project sounds rather like you want to make a better English/Chinese dictionary… all you need to do first is learn Chinese.
Comment: Oh yeah, and I don't mean to talk you out of a learning opportunity, but large companies have already put lots of money into this issue and have solutions that are acceptable to the industry, so the idea that one person in their own spare time will "build a better mousetrap" is unlikely. Maybe a better motivation is "Because I want to explore musical OCR and translation to MIDI for personal curiosity." Part of the problem that Testsujin is getting at is that only 50%-80% of a pleasing musical performance is represented on the page....
Comment: ... Humans, reading sheet music, add variations of volume and timing that would be too detailed to show in the printed material, just like humans reading language add vocal inflection and timing. Text-to-speech synthesis has worked for decades to make it more lifelike, but they have rules of grammar and meaning to follow, while musical interpretation is less predictable. I'm not aware of *any* "sheet music readers," but if they're "bad" it's probably first because they use cheap midi libraries for playback, and secondly because they lack these interpretive details.
Comment: Meanwhile, any industry need for good-sounding music that isn't generated by humans probably skips the sheet music entirely and just works with midi data from the start.
Comment: @AndyBonner  Thanks so much.  This is some great background. Thanks for the help!
Answer: There are many parts to this problem. A PDF can be both a raster image, but it can also be made up of vector paths and text (as exported from notation software). But is does not contain direct information about music context. The task of turning the former into the latter is what is usually called OMR (as pendent to OCR for text). Note that this is a much more complicated task than OCR, because music notation is has more dimensions.
There are quite a few solutions that work fine, but you need to provide sufficient quality and the notation should not be too complex. Some software explicitly works with the output from notation software (containing vector paths and text), other software will even need to try to identify different musical symbols.
Apart from PDF the raster content can be any raster image format (e.g. jpeg), and the vector output can be any vector format, although here you'd usually have pdf.
Once you are past that step and once you have data of musical semantics there are many options of how to encode and store that data. MIDI is a format mainly for controlling synthesizers and thus does not encode a lot of information. It basically gives chromatic pitch, velocity and some control options. MusicXML is a format intended to be an open standard to store notation. This will be able to encode more than you need for playback, like exact positioning of elements. You can of course use any file format for any notation software, or even devise your own code.
So what would need to be done now is to estimate things like dynamics, articulation, phrasing, timing &c. from this representation and then use this to perform synthesis.
If you are looking for a "format that is easier to read for a computer" you'd automatically land at a format with musical context. Which probably makes sense, as you won't probably want to do your own OMR system.
I'd suggest to draw an outline of what you want to priorise. E.g. you might not want to implement OMR or synthesis, as for both there do exists decent solutions. Rather maybe you might want to implement something that takes something like a musicXML and creates from this playback information, which could then be fed into a synthesizer.
Comment: Thank you so much this is great!
Answer: If you're starting from a printed page of music PDF is the standard.
Anything 'easier to read' will already have been at least partially converted into computer-playable form.  Your job is done!
But if you're asking what format you should use as output from your 'music OCR' program, it's pretty much got to be MusicXML.

Standard sheet music file


Hide fields from a view

Question: I am not sure how to explain this as I have very little experience with Drupal. I built a site with Drupal 7 and now I am trying to recreate it in Drupal 8. I have a problem with the Content area.
I have set up a view that looks like this.

The arrow points to a title name link.
When I click the link, I want to see additional detail not shown in the view. This works. However, there is as an additional view that looks like a "teaser" view that I do not want to be displayed. The area at the top in the rectangle I want and is part of the "Entity View (Content) with a category of Chaos Tools. The bottom part near the x is the teaser part and is a duplicate of the top part I don't want. How can I remove the bottom part?
Answer: man. Have you solved yet? There's this custom module that even someone with basics of programming could do. But it is not perfect, you will have to make an extra view and stuff... https://atendesigngroup.com/blog/removing-duplicate-content-across-multiple-drupal-views

Hide fields from a view


How can i use the ionic 2 's components in angular 2 project?

Question: I want to use the ionic 2 in a feature module of my angular 2 project. I want to use the ionic's component or tag in some component of mine. Every things I have done,but I got this error
EXCEPTION: Expected to not be in Angular Zone, but it is!
Can someone tell me how to use the ionic 2 's components in angular 2 project but not to use the ionic-cli to create a new one?
Answer: First create an angular 2 project using
ng new myapp --style=scss

then
cd myapp
npm install --save ionic-angular ionicons

You may get a few warnings such as
npm WARN email@example.com requires a peer of @firstname.lastname@example.com but none was installed.
npm WARN firstname.lastname@example.com requires a peer of @firstname.lastname@example.com but none was installed.
...

It will work even with the warnings but if you prefer to getting rid of all those warnings you can check what peerDependencies are declared by ionic-angular with:
npm view ionic-angular peerDependencies

Then update the versions in package.json accordingly and npm install.
You could also remove @angular/router since Ionic provides its own navigation mechanism.
npm uninstall --save @angular/router

Then ng serve
The next step to use Ionic in your project is to change src/app/app.module.ts to import the IonicModule instead of Angular's BrowserModule, and bootstrap IonicApp:
import { NgModule } from '@angular/core';
import { IonicApp, IonicModule } from 'ionic-angular';

import { AppComponent } from './app.component';

@NgModule({
  imports: [ IonicModule.forRoot(AppComponent) ],
  declarations: [ AppComponent ],
  bootstrap: [ IonicApp ]
})
export class AppModule { }

The IonicApp component has ion-app as its selector, so you also need to modify src/index.html replacing  with .
To keep the project structure somewhat similar to what the Ionic CLI generates, create a src/theme folder, then download and put into that folder these two files: ionic.scss and variables.scss.
Finally include them in build process by editing angular-cli.json to load variables.scss first in the styles inside app:
"styles": [
  "theme/variables.scss",
  "styles.scss"
],

Note that you'll have to restart ng serve for this change to take effect.
The final step for having a basic Ionic app is to create a page component, and load it in AppComponent using ion-nav.
To do that create a src/pages folder with a home.page.ts file containing e.g.
import { Component } from '@angular/core';

@Component({
  selector: 'page-home',
  templateUrl: 'home.page.html'
})
export class HomePage {

  message = 'Welcome!';

}

and the corresponding template in home.page.html with a basic Ionic page, including header and content:
<ion-header>
  <ion-navbar>
    <ion-title>Ionic App</ion-title>
  </ion-navbar>
</ion-header>
<ion-content padding>
  {{ message }}
</ion-content>

You can get the full documentation from ionic2-with-angular-cli .Hope it helps.


Time zone effect on Date to LocalDate conversions for dates far in the future

Question: The following test illustrates something I ran across will investigating Date to LocalDate conversions.
For my setup, Java 7 & joda-time 2.7, the conversion of Date to LocalDate and back appears to get a bit off starting around 2038-3-22.
For me, test_2038_3_21 passes, but test_2038_3_22 does not.
Is this a known problem, and if so, is there a work around?
package com.example;

import static org.junit.Assert.*;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.TimeZone;

import org.joda.time.DateTime;
import org.joda.time.DateTimeZone;
import org.joda.time.LocalDate;
import org.junit.Test;

public class TimeZoneTest {

    @Test
    public void test_2038_3_21() {
        exerciseDate(2038, 3, 21);
    }

    @Test
    public void test_2038_3_22() {
        exerciseDate(2038, 3, 22);
    }

    static void exerciseDate(int year, int month, int day) {
        TimeZone defaultTimeZone = TimeZone.getDefault();
        String defaultUserTimezone = System.getProperty("user.timezone");
        DateTimeZone defaultDateTimeZone = DateTimeZone.getDefault();

        try {
            for (String timeZoneId : timeZoneIds()) {
                System.setProperty("user.timezone", timeZoneId);
                TimeZone.setDefault(TimeZone.getTimeZone(timeZoneId));
                DateTimeZone.setDefault(DateTimeZone.forID(timeZoneId));

                LocalDate ld = new LocalDate(year, month, day);
                DateTime dt = ld.toDateTimeAtStartOfDay();
                Date d = ld.toDate();
                Date dd = dt.toDate();

                LocalDate ld2 = LocalDate.fromDateFields(d);
                DateTime dt2 = ld2.toDateTimeAtStartOfDay();
                Date d2 = ld2.toDate();
                Date dd2 = dt2.toDate();

                assertEquals(timeZoneId, ld, ld2);
                assertEquals(timeZoneId, dt, dt2);
                assertEquals(timeZoneId, d, d2);
                assertEquals(timeZoneId, dd, dd2);
                assertEquals(timeZoneId, d, dd);
            }
        } finally {
            DateTimeZone.setDefault(defaultDateTimeZone);
            System.setProperty("user.timezone", defaultUserTimezone);
            TimeZone.setDefault(defaultTimeZone);
        }
    }

    static List<String> timeZoneIds() {
        List<String> timeZones = new ArrayList<>();
        for (String timeZoneId : TimeZone.getAvailableIDs()) {
            try { // only use time zones that DateTimeZone thinks are valid
                DateTimeZone.forID(timeZoneId);
                timeZones.add(timeZoneId);
            } catch (IllegalArgumentException e) {
                // ignore
            }
        }
        return timeZones;
    }
}

Answer: You are likely hitting a DST transition for one of the time zones.
From the docs for LocalDate.toDate:

Converting to a JDK Date is full of complications as the JDK Date constructor doesn't behave as you might expect around DST transitions. This method works by taking a first guess and then adjusting the JDK date until it has the earliest valid instant.

Do note though that future transitions are really just a best guess based on current information.  Time zone rules change regularly.  For something that far out, there could easily be a change between now and then.


What does "TAG for Three-part voices" mean?

Question: Reading some three-part a cappella music, I see "TAG for Three-part voices" above the staff. Any idea what that might mean?

19 is the measure number. There are 27 measures total in the song, so this is the start of the last 8 measures. There's an alternate ending for two voices that has written above the staff: "TAG for Two-part voices".
UPDATE: Besides the posted answers below, see http://en.wikipedia.org/wiki/Tag_(barbershop_music).
Comment: I suppose it is probably not "Tenor/Alto Glockenspiel" which is the only thing I can think to fit, heh.  Good question.
Comment: That's not a piano part, it's a piano reduction for use in rehearsal only -- that's why it's in small notes.
Comment: What about Tenor, Alto, Ground? The barbershop answer (a capella) seems to mismatch the piano combination below.
Answer: Tags are idiomatic to barbershop music (which is typically four a cappella voices), and refer to the ending chords of a song, usually coincident with the last line of lyrics.
Sometimes singers will get together for the sole purpose of singing tags, since they are easier to learn than an entire piece of music and usually consist of harmonic cadences that are very fun to sing.
And you can find a whole bunch of them here, including a public domain book of 125 classic tags.
Comment: Thanks! I think the thing that threw me off was that TAG was in all caps, so I kept searching for some acronym.
Answer: "Tag" is a synonym for "Coda", or the final section of a song.
Comment: in that example?  Seems odd to see that at the beginning of the piece.
Comment: Like NReilingh said, this is an arrangement for the ending section of a piece, which means the coda. But for some reason it has been published separately, unless the number 19 in the box is a measure number.
Comment: I thought about 19 being the measure but it looks like the beginning of the song and is the opening line of the song so I hazarded a guess . . .
Comment: Sorry for not specifying this earlier: 19 is the measure number. There are 27 measures total in the song, so this is the start of the last 8 measures. There's an alternate ending for two voices that has written above the staff: "TAG for Two-part voices".


Alternative to Traversing Hasmap of list Values?

Question: I have a hashmap that contains a list as its values:
HashMap<Integer,ArrayList<<Children>> parent

I used this because it allowed for an O(1) lookup of children belonging to the parent (by using the Parent ID as the key). However, the problem is when I need to find the parent of a child. This requires traversing the whole Hashmap and looking at all the values in each individual map. I'm not against doing this but is there anything faster that could be done?
Comment: Have you considered storing parent reference and/or id within children?
Comment: Alternatively, you could build a second map for those relationships.
Comment: Suggest you have a `Map` for quick lookup of any node, then add `Node parent` and `List children` as fields of the `Node`. You can now find anything fast, and easily traverse the tree up and down. If you also need to traverse sideways easily, replace `children` with `Node firstChild`, `Node nextSibling`, and `Node prevSibling`. *(Where have I seen this before? Oh yeah, XML DOM)*
Answer: Instead of having a Map<Integer, List<Children>> you could implement a Guava MultiMap as defined here. The MultiMap would have the structure MultiMap<Integer, Children> now this doesn't solve your current problem. However if you take a quick look around stack overflow you can find an implementation of a MultiBiMap like this, a MultiBiMap would have the advantage of two way lookups, so you could lookup by the ID of the parent or inverse the Map and lookup by the Child object (**The child would need a HashCode and Equals implementation for the lookup).
PS: I am aware this is hideously complex but if you are looking purely for efficiency this should be pretty efficient.


SSH tunneling is faster than OpenVPN, could it be?

Question: Logically, VPN should be faster than SSH for tunneling, because:

It's running on UDP and not TCP (so no TCP over TCP)
It has compression

However, today I tested Redis replication over both methods.
I ran the test over an Ireland AWS VM, connecting to a US-East AWS VM.
Since my test case is Redis replication, this is exactly what I tested - I ran a blank Redis server, and after it finished loading, I executed slaveof the other server, and measured the time between  Connecting to MASTER and MASTER <-> SLAVE sync: Finished with success. In between, I used
while 1; do redis-cli -p 7777 info | grep master_sync_left_bytes;sleep 1; done

To get a crude estimation of the speed.
SSH won by a long shot: ~11MB/s compared to OpenVPN's ~2MB/s.
Does that mean that all of what I reaserched was wrong, or have I grossly misconfigured my setup?
Update
I've made several test with the same dataset, and got these results:

OpenVPN

TCP:
compression: 15m
no compression: 21m
UDP:
compression: 5m
no compression: 6m

SSH
defaults: 1m50s
no compression: 1m30s
compression: 2m30s

Update2
Here are the iperf results, with bidirectional tests (except SSH, where no return path is available)
| method           | result (Mb/s)|
|------------------+--------------|
| ssh              | 91.1 / N.A   |
| vpn blowfish udp | 43 / 11      |
| vpn blowfish tcp | 13 / 12      |
| vpn AES udp      | 36 / 4       |
| vpn AES tcp      | 12 / 5       |

Technical specs
I'm running CentOS 6.3 (server), CentOS 6.5 (client).
OpenVPN version is 2.3.2 (same as in Ubuntu 14.10, so no moldy version there)
My SSH tunnelling looks like:
ssh -f XXXX@XXXX -i XXXX -L 12345:127.0.0.1:12345 -N

My configuration file looks like:
server
port 1194
proto udp
dev tun0
topology subnet
log /var/log/openvpn.log

ca XXXX
cert XXXX
key XXXX
dh XXXX
crl-verify XXXX

cipher AES-256-CBC

server XXXX 255.255.255.0

ifconfig-pool-persist /etc/openvpn/ipp.txt
keepalive 10 120
comp-lzo
status /var/log/openvpn-status.log
verb 3
tun-mtu 1500
fragment 1300

persist-key
persist-tun

client
client

remote XXXX 1194

proto udp
dev tun
log /var/log/openvpn.log
comp-lzo

cipher AES-256-CBC
ns-cert-type server

# the full paths to your server keys and certs
ca XXXX
cert XXXX
key XXXX

tun-mtu 1500 # Device MTU
fragment 1300 # Internal fragmentation

persist-key
persist-tun
nobind

Comment: @Nitz TCP tunnels in ssh do not use any TCP over TCP. In fact the ssh client is usually run with insufficient privileges to even do it. And no, ssh does not strip any TCP headers from packets, because it never even touches a TCP packet. ssh is just an application making use of the TCP stack in the kernel, like any other application. Data travels through one TCP connection from some program to the ssh client. The ssh client encrypts the data and send it through the TCP connection to the server. The server decrypts it ans send it through the third TCP connection to a program at the other end.
Comment: SSH supports compression as well, so that isn't necessarily something different between OpenVPN and SSH.  Have you tried disabling compression on both side?  When you perform the transfer over OpenVPN, run top or something on your client/server.  Are there any obvious signs that you are maxing your CPU/Memory/etc with the VPN connection?
Comment: It seems unlikely for a AWS hosted system, but there is a small possibility that UDP is getting rate limited or something.  Have you tried doing OpenVPN over TCP?
Comment: Sure there might be a little more overhead with OpenVPN because it has the extra IP/TCp headers.  But that shouldn't make a difference of 4-10 times slower.  If the difference was in the 5-10% slower range I might be tempted to blame that. The reason you might want to still investigate is that this could be a symptom of some underlying problem that might be impacting other things in a way that is less obvious.
Comment: @Nitz If I understand you correctly, you are saying that the unencrypted packets entering the virtual interface are 1424 bytes, but the encrypted packets send on the physical interface are only 160 bytes. That would indicate a pretty extreme fragmentation happening at the VPN layer or the UDP/IP layer beneath it. That could certainly explain the performance problem. The packets on the virtual interface should be something on the order of 1300-1400 bytes. The packets on the physical interface should be something on the order of 1400-1500 bytes.
Comment: Following Zoredache's line of thought, net neutrality is dead and gone, and various providers artifically retard various services as they please.  It may well be that someone, somewhere in the chain, is de-prioritising OpenVPN traffic.  Unless they're doing it via deep packet inspection, though, you could test that by moving your OpenVPN to a different port number, and trying the test again.
Comment: @Nitz Yes, that's how TCP tunnels work. Additionally there is of course some multiplexing, compression, and encryption going on.
Comment: This is even more interesting because openssh (which I presume you're using) has some well-known performance problems.  Maybe the systems you're using have it patched already, I'm told some linux distros do that.  See https://www.psc.edu/index.php/hpn-ssh/640  .  Also, your "fragment 1300" means that every TCP packet is getting fragmented at 1300 bytes.  To optimize throughput, I would want to (1) only have openvpn fragment when absolutely necessary, which is probably more like 1450  (2) see if i could get the tcp stack to set its MTU appropriately, which would be this same value.
Comment: (1) OpenVPN should use the largest MTU available on your network path.  Probably 1500 bytes (maybe not). (2) openVPN interface needs to have MTU based on this. calculate by subtracting the openVPN overhead from original MTU.   (3) kernel's TCP software ("stack") needs to know about this MTU, and set the TCP MSS based on this value.  If you are lucky, this all happens automatically.  But there are many MTU options in the docs so it probably doesn't, not always.  To test, check your tcpdump.  Are all the packets in a bulk xfer the same size?  Or are they alternating long and short?
Comment: @Nitz The most extreme stack of headers I could possibly imagine for a VPN still only came out as 248 bytes. Assuming the MTU is at least 1280 bytes, that is still less than 20%. And the 5-10% suggested by Zoredache is a much more likely amount of overhead being added. The amount of bytes used for headers simply isn't the problem. The problem when TCP is used multiple times in the stack isn't space used for headers, it has to do with the timeouts, retransmissions, flow-/congestion-control, and buffering happening in TCP. TCP was not designed to exist more than once in the protocol stack.
Comment: @kasperd and how do I do that? Currently, I commented out the framgnet/mtu commands and put `mtu-test` instead.
Got 1541 MTU, and iperf gives out 21.4 Mb/s, still much lower than SSH
Comment: Don't forget the overhead of encryption, I'd bet SSH is a bit more optimized (as it's been around for longer, especially during times when bandwidth was much more restricted, and CPU power).
Comment: @Zoredache thanks for the tips. I will attempt to manually disable compression and move OpenVPN to TCP. I'm now attempting to find a explanation to how ssh tunnelling actually works - it can't be keeping the package intact (since it redirects to another port), so I'm starting to guess its stripping the header and rebuilding it on the other side, avoiding the TCP-on-TCP issue
Comment: @kasperd so SSH just opens sockets on both sides, and moves the data from the sockets from one side to another?
I've been poking around the code but I can't seem to understand what happens there
Comment: Wow your test results comparing SSH and OpenVPN give results I would not expect at all.  If you are interested in digging deeper I would use a different tool to test instead of redis.  So I would try running iperf tests over your ssh/openvpn tunnels, and get the bandwidth latency results.  Past that, I would probably fire up tcpdump and see if there is anything happening while you are using OpenVPN.  Lots of retransmits, unusual fragments, any errors, etc.
Comment: @Zoredache I was shocked too. My current theory is that kasperd is right, and SSH doesn't transfer headers, only content, so it beats OpenVPN's overhead of TCP over UDP. I'm still thinking about whether that's the reason. I might do your testing, although my boss is starting to think I'm obsessing about this too much :)
Comment: Did you try with blowfish cipher rather than AES?
Comment: @gparent no. Didn't think of it, but it'll be unacceptable as a solution - I don't know my crypto that well, but it's vulnerable in long sessions / short keys. Don't think it's the crypto's fault anyway.
Comment: blowfish is the default cipher shipping with nearly every OpenVPN install to date. If you think it is vulnerable, you should report it to their developers, but I suspect that's not the case.
Comment: Was not aware of that. When choosing a key, I saw [this](http://en.wikipedia.org/wiki/Blowfish_%28cipher%29#Weakness_and_successors) and figured I'd better avoid it
Comment: @gparent blowfish improved things by a bit, but SSH is still winning. Updated chart
Comment: Finished reading the `tcpdump` output. There are occasional black patches of duplicate packets, but 90% of the output is good, so that's no explanation.
Comment: @DanPritts I must admit I'm afraid to touch these values - the current form is "magic numbers" I picked up along the way. You're right though, I'll give it a shot. By "TCP Stack" you mean the actual server's ethernet card, no the openvpn one, right?
Comment: @kasperd if you're referring to my OpenVPN performance issues, it still happens when I use UDP, so it can't be blamed on TCP on TCP
Comment: @Nitz A different issue that can show up, even if there is no TCP on top of TCP has to do with how packets are fragmented. PMTU discovery can easily be broken by clueless administrators anywhere along the path. Even if PMTU discovery isn't broken, the presence of a VPN means there are two different layers in the stack, which could do fragmentation. But you don't want fragmentation at all. If TCP exists anywhere in the stack, you are much better off relying on segmentation by TCP. The best workaround for that tend to be to clamp the MSS on TCP-SYN packets at one end of the VPN connection.
Comment: If either end of the VPN connection is a Linux machine routing the traffic, then this should work: `iptables -A FORWARD -p tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1220`. If you already have other iptables rules, you would have to adjust accordingly. If you want to use it on an endpoint rather than on a router on the path, then you need to insert rules in both the INPUT and OUTPUT chains.
Comment: @kasperd as far as I can understand MSS, as long as I let OpenVPN discover the MTU by itself, MSS isn't needed. Am I wrong?
Comment: @Nitz In order for MTU to work as intended, first the endpoints of the VPN connection need to detect the PMTU between themselves. Then they have to use that to set the correct MTU on the virtual link. That virtual link will be just one of many links on a path on which PMTU has to work as well. Even if PMTU works on one of those two levels, it isn't sufficient. PMTU has to work on both levels, and that will usually be outside of your control. MSS on the other hand can be adjusted by any single router forwarding the unencrypted packet, and as long as the new value is low enough, it will work.
Comment: @kasperd I added `fixmss`, and made sure using `ip route get to ` that the mss (`1460`) is lower than the interet-path MTU (`1500`). No improvement in transfer rate. I guess that's not it
Comment: @Nitz 1460 is surely too high. You need to subtract two IP headers, one TCP header, and the VPN overhead from the path MTU. Assuming the PMTU actually is 1500, then that still means your MSS need to be lower than 1440. 1400 might be low enough, but I wouldn't be certain. When tweaking MSS, I recommend first trying with 1220. I don't know the `fixmss` setting, does it modify MSS on packets in transit, or does it only affect packets originated by that host?
Comment: @kasperd when inspecting the encrypted OpenVPN traffic, I can see that the actual OpenVPN packets are very small (~160). This stays true after every configuration tweak I make. I very few retransmits or failures, and the packet going into the `tun` device seem decently sized (almost all are 1424).
I'm thinking about leaving it that way and staying with SSH. Do you have any experience with cross-continent traffic showing my performance is inadequate?
Comment: In practice, if I were you I'd just stick with SSH, but I agree with kasperd.  160 byte packets on the wire is a recipe for poor performance.
Answer: Thanks to kasperd's comment, I learnt that SSH doesn't suffer from TCP-over-TCP since it only moves packet data.
I wrote a blog post about it, but the most interesting thing is the netstat output, proving that SSH indeed doesn't preserve Layer 3,4 data:
after tunneling, before connecting
backslasher@client$ netstat -nap | grep -P '(ssh|redis)'
...
tcp        0      0 127.0.0.1:20000             0.0.0.0:*                   LISTEN      20879/ssh
tcp        0      0 10.105.16.225:53142         <SERVER IP>:22              ESTABLISHED 20879/ssh
...

backslasher@server$ netstat -nap | grep -P '(ssh|redis)'
...
tcp        0      0 0.0.0.0:6379                0.0.0.0:*                   LISTEN      54328/redis-server
tcp        0      0 <SERVER IP>:22              <CLIENT IP>:53142           ESTABLISHED 53692/sshd
...

after tunneling and connecting
backslasher@client$ netstat -nap | grep -P '(ssh|redis)'
...
tcp        0      0 127.0.0.1:20000             0.0.0.0:*                   LISTEN      20879/ssh
tcp        0      0 127.0.0.1:20000             127.0.0.1:53142             ESTABLISHED 20879/ssh
tcp        0      0 127.0.0.1:53142             127.0.0.1:20000             ESTABLISHED 21692/redis-cli
...

backslasher@server$ netstat -nap | grep -P '(ssh|redis)'
...
tcp        0      0 0.0.0.0:6379                0.0.0.0:*                   LISTEN      54328/redis-server
tcp        0      0 127.0.0.1:6379              127.0.0.1:42680             ESTABLISHED 54328/redis-server
tcp        0      0 127.0.0.1:42680             127.0.0.1:6379              ESTABLISHED 54333/sshd
tcp        0      0 <SERVER IP>:22              <CLIENT IP>:53142           ESTABLISHED 52889/sshd
...

So I'm going to use SSH tunneling, since it seems that my OpenVPN isn't misconfigured or anything, just not the right tool for the job.
Answer: It depends what you are trying to achieve and what your priorities are.  VPN connects you to a network and SSH to a machine.  VPN is a bit more secure with the encapsulation, which SSH does not do.
Also, VPN allows all the traffic to easily go through it, versus SSH where you will have to force the applications.
Are you going to use AD at all?  Because VPN will let you do that with much more ease.
I prefer SSH for speedy necessities and VPN for critical applications where I should spare the extra time.
Depending on the situation, I have used SSH in a VPN in case the VPN was compromised.  This way someone probing would have to get through the SSH tunneling.
Comment: I'm running redis over the tunnel, so a single port suffices to me. I was just amazed by the fact that VPN is not always the best solution for tunneling network traffic
Answer: SSH port forwarding is not tunneling, because protocol stack wrapping doesn't happen. OpenVPN does wrap, so SSH port forwarding won't suffer from TCP-over-TCP issue.
For example in OpenVPN, protocols stack will be similar to:
Redis
TCP
IP
OpenVPN (tun mode)
UDP
IP
Ethernet

IP (a network layer protocol) appeared here twice, that's we calling it a tunnel.


Show that $\sum x_n y_n$ converges when $\sum x_n$ is bounded for any $n \in \mathbb{N}$ and $\sum (y_n - y_{n+1})$ converges absolutely

Question: Show that $\sum_{n=n_0}^{\infty} x_n y_n$ converges when $\sum_{j=1}^{n} x_j$ is bounded for any $n \in \mathbb{N}$ and $\sum (y_n - y_{n+1})$ converges absolutely.
Does this actually require an added condition which is "$y_n \to 0 ~\text{as}~ n\to \infty$"? I don't think that it is at all necessary, or did I miss something?
Comment: What if $y_n=1$ for all $n$?
Comment: If you don't think it is necessary, does that mean you have a proof of this problem? You could post that.
Comment: $y_n \to 0$ is necessary as there is a term in the summation by parts that has a $y_n(\sum{x_k})$ form
Comment: @clark It is implicitly assumed that $\sum x_n$ is convergent, so $y_n=1$ is not a counterexample.
Comment: Can it be proven without the 'summation by parts' as Kabo Murphy writes in his answer below?
Answer: The hypothesis assumes that $\sum_j x_j$ is convergent. Since $\sum |y_n-y_{n+1}| <\infty$ it is easy to see (by triangle inequality that $(y_n)$ is a Cauchy sequence. If $y_n \to l$ then we can prove the result with $y_n$ change to $y_n-l$ and the use convergence of $\sum x_j$ to finish the proof.
For the benefit of other readers let me point out that the result follows when $y_n \to 0$ by a simple 'summation by parts' argument.
Comment: Could you please prove it without the 'summation by parts' as you claim here? Additionally, the hypothesis doesn't assume $\sum x_j$ to be convergent. How about $\{x_n\}^{\infty} = \{1,-1,1,-1,\cdots\}$?
Comment: The hypothesis about $x_n$s is boundedness of the  partial sums $ S_n=\sum\limits_{k=1}^{n} x_k$. So you have to write $x_n$ as $S_n-S_{n-1}$ to use this hypothesis. I don't see how you can avoid summation by parts.
Comment: I somehow don't see how taking $S_n - S_{n-1}$ makes more sense. $|S_n| < 2$ $\forall$ $n\in \mathbb{N}$ yet $\{S_n\}$ isn't bounded in the counterexample I stated.


recurrent neural network ValueError: Found array with dim 3. Estimator expected <= 2

Question: I am running an LSTM, GRU and bilstm model using the following code
# Create BiLSTM model
def create_model_bilstm(units):
    model = Sequential()
    model.add(Bidirectional(LSTM(units = units,
              return_sequences=True),
              input_shape=(X_train.shape[1], X_train.shape[2])))
    #model.add(Bidirectional(LSTM(units = units)))
    model.add(Dense(1))
    #Compile model
    model.compile(loss='mse', optimizer='adam')
    return model

# Create LSTM or GRU model
def create_model(units, m):
    model = Sequential()
    model.add(m (units = units, return_sequences = True,
                input_shape = [X_train.shape[1], X_train.shape[2]]))
    model.add(Dropout(0.1))
    #model.add(m (units = units))
    #model.add(Dropout(0.2))
    model.add(Dense(units = 1))
    #Compile model
    model.compile(loss='mse', optimizer='adam')
    return model

# BiLSTM
model_bilstm = create_model_bilstm(20)

# GRU and LSTM
model_gru = create_model(50, GRU)
model_lstm = create_model(20, LSTM)

# Fit BiLSTM, LSTM and GRU
def fit_model(model):
    early_stop = EarlyStopping(monitor = 'val_loss',
                                               patience = 100)

    history = model.fit(X_train, y_train, epochs = 700,
                        validation_split = 0.2, batch_size = 32,
                        shuffle = False, callbacks = [early_stop])
    return history

history_bilstm = fit_model(model_bilstm)
history_lstm = fit_model(model_lstm)
history_gru = fit_model(model_gru)

This all runs smoothly and prints out my loss graphs. but when it comes to predictions i run the following code
# Make prediction
def prediction(model):
    prediction = model.predict(X_test)
    prediction = scaler_y.inverse_transform(prediction)
    return prediction

prediction_bilstm = prediction(model_bilstm)
prediction_lstm = prediction(model_lstm)
prediction_gru = prediction(model_gru)

and i get the following error
   ValueError Traceback (most recent call last)
<ipython-input-387-9d45f01ae2a2> in <module>
      5     return prediction
      6
----> 7 prediction_bilstm = prediction(model_bilstm)
      8 prediction_lstm = prediction(model_lstm)
      9 prediction_gru = prediction(model_gru)

<ipython-input-387-9d45f01ae2a2> in prediction(model)
      2 def prediction(model):
      3     prediction = model.predict(X_test)
----> 4     prediction = scaler_y.inverse_transform(prediction)
      5     return prediction

                         ...

    ValueError: Found array with dim 3. Estimator expected <= 2.

I am assuming this has something to do with my X_test shape based on other posts i have read so i tried to reshape it to 2d but got another error telling me "expected bidirectional_3_input to have 3 dimensions, but got array with shape (62, 36)" on line 7 again.
What am i doing wrong and how can i fix it?
Data Explanation:
So I am trying to predict discharge rates (target variable) using groundwater levels (34 features), precipitation and temperature as input which gives me a total of 36 features. My data is in monthly resolution. I am using 63 observation for my test (5 year pred) and the rest for my train.
Comment: can you please tell the shape of **X_test** and **X_train** aswell
Comment: @SrikarManthatti `X_test.shape = (62, 1, 36)` and `X_train = (316, 1, 36)`
Comment: I believe it is surely with the shape of the data since for bilstm you are training with input dimensions (316,1) and while predicting you are passing (62,1,36). May I know on what data you are working on, what does those second(1) and third dimension (36) mean in your data? If it is okay you can paste a picture of data and talk about it.
Comment: @SrikarManthatti i have added an explanation for my data. Based on your comment does this mean my reshape is wrong?
Comment: Great!!! so as per your explanation I suggest that the shape of your X_train should be (316,36), y_train (316,1), and the test should be X_test(63,36) and Y_test should be(63,1)
Comment: @SrikarManthatti but for lstm, gru and bilstm the data is expected to be in 3D is it not? If i make it 2D the models won't accept it
Comment: Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/223026/discussion-between-srikar-manthatti-and-denise).
Answer: What are you doing wrong? Let's assume your input data has shape X_train.shape = [d0,d1,d2], then after setting up your BiLSTM-model like
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Bidirectional,LSTM,Dense

model = tf.keras.Sequential()
model.add(
   tf.keras.layers.Bidirectional(
      tf.keras.layers.LSTM(
         units = 10,
         return_sequences=True),
         input_shape=(d1, d2)
         )
      )
model.add(Dense(1))
model.compile(loss='mse', optimizer='adam')

we can check the input- and output-shapes your model expects by
>>model.input.shape
TensorShape([None, d1, d2])
>>model.output.shape
TensorShape([None, d1, 1])

So your model expects input of shape (n_batch,d1,d2), where n_batch is the batch size of the data, and returns a shape (n_batch,d1,1), thus a 3d-tensor.
Now if you provide a 3d-tensor to your model, the model.prediction-method will succesfully return a 3d-tensor, however sklearn.preprocessing.StandardScaler.inverse_transform only works for 2d-data, thats why it says
ValueError: Found array with dim 3. Estimator expected <= 2.

On the other hand, if you first reshape your data to be 2d, then model.prediction complains, because it is set up to expect a 3d-tensor.
How can you fix it? For further help on how to fix your code, you will need to provide us with more detailled information on what you expect your model to do, especially what output-shape you want your BiLSTM-model to have. I assume you actually want your BiLSTM-model to return a scalar for each sample, so an additional Flatten-layer might do the trick:
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Bidirectional,LSTM,Dense,Flatten

model = tf.keras.Sequential()
model.add(
   tf.keras.layers.Bidirectional(
      tf.keras.layers.LSTM(
         units = 10,
         return_sequences=True),
         input_shape=(d1, d2)
         )
      )
model.add(Flatten()) #<-- additional flatten-layer
model.add(Dense(1))
model.compile(loss='mse', optimizer='adam')


Private filesystem does fire 403 on every role and file

Question: Like the title might expect I am not able to view any file that i am  delivering through the private file system. Only if I bypass the access check via role settings.
I tested it a minute ago with a fresh d8.9 installation. Here images are are delivered (statuscode: 200). I my main project I receive a 403. Why is that so ?
Drupal does create the private files and as admin (bypassing) I also can see the file. As non admin (authenticated user) I cannot.
How can I debug this issue?
A theory I have read said something about varnish setting... I searched  the whole project without finding any script mentioning "varnish". I spend now good 6 hours... checked roles, scripts and so on. I could now start a crazy adventure by deinstalling 80+ modules until it might work or somebody could give me a hint how I debug this madness?
Thanks in advance

Guests on the left | authenticated users on the right
Answer: If you need to debug access at the Drupal application level, then look at FileDownloadController::download; it's the controller method for the /system/files/{filepath} path; you can use your tool of choice to inspect the execution flow from request to the returned response object.
If the issue is outside the Drupal application level (e.g. web server, varnish) then it will require further stack troubleshooting.


Iteração dá erro de ConcurrentModificationException ao incluir mais de um botão

Question: Tenho este pequeno jogo que estou a desenvolver em que a personagem passa por botões que vão retirar barreiras para que esta possa aceder a outras zonas do labirinto.

Eu criei duas listas, uma para as cercas e outra para os botões.
listaCercas = new ArrayList<Arbusto>();
listaCercas.add(new Arbusto(this,105, 496));
listaCercas.add(new Arbusto(this,660, 210));

listaBotoes = new ArrayList<Itens>();
listaBotoes.add(new Itens(this, 625, 460));
listaBotoes.add(new Itens(this, 200, 40));

E coloquei o seguinte código na parte das colisões:
Iterator<Itens> itbotao = listaBotoes.listIterator();
    Iterator<Arbusto> it6 = listaCercas.listIterator();
    while(it6.hasNext() && itbotao.hasNext()) {
        Arbusto cerca = it6.next();
        Itens botao = itbotao.next();
        Rectangle rBotao = new Rectangle(botao.x, botao.y, botao.botaoL, botao.botaoA);
        Rectangle rCerca = new Rectangle(cerca.x, cerca.y, cerca.cercaL, cerca.cercaA);
        if(rBoneca.intersects(rCerca)) {
            switch(direcao) {
                case 1:
                    bonecaSprite.y -= 5;
                    break;
                case 2:
                    bonecaSprite.x -= 5;
                    break;
                case 3:
                    bonecaSprite.x += 5;
                    break;
                case 0:
                    bonecaSprite.y += 5;
                    break;
            }
        }
        if((rBoneca.intersects(rBotao) && rBotao.getX() == 625) && !On_Off) {
            On_Off = true;
            listaCercas.remove(1);
        } else if((rBoneca.intersects(rBotao) && rBotao.getX() == 625) && On_Off) {
            On_Off = false;
            listaCercas.add(new Arbusto(this,660, 210));
        }
        if((rBoneca.intersects(rBotao) && rBotao.getX() == 200) && !On_Off) {
            On_Off = true;
            listaCercas.remove(0);
        } else if ((rBoneca.intersects(rBotao) && rBotao.getX() == 200) && On_Off) {
            On_Off = false;
            listaCercas.add(new Arbusto(this,105, 496));
        }

O meu código funciona se eu tiver só um botão, mas quando ponho o outro botão e o respectivo código o Java lança-me o erro: java.util.ConcurrentModificationException.
Eu estive a pesquisar na net e a tentar compreender melhor o problema e vi que era sobre a parte da iteração mas não sei como o resolver. Como criar mais botões nos níveis seguintes e não ter que definir cada botão com uma variável?
Comment: Não analisei a lógica para entender a razão de você tentar remover sempre os elementos `0` ou `1`, mas se a ideia for remover o elemento atual da iteração você pode usar o método [`remove()`](http://docs.oracle.com/javase/7/docs/api/java/util/Iterator.html#remove\(\)) do próprio `Iterator`.
Answer: O problema é que você está alterando uma coleção sobre a qual está iterando (como em listaCercas.remove(1) e listaCercas.add(new Arbusto(this,660, 210))).
Em muitos casos pode acontecer de o iterador "se perder" quando um elemento da lista é removido.
A exceção é lançada quando há mais de um botão na coleção porque é o caso em que o código it6.next() será executado após a alteração da lista original.
Uma possível solução é clonar a lista que será alterada e iterar sobre o clone, removendo os elementos da lista original.
List<Arbusto> cloneCercas = new ArrayList<>(listaCercas);
Iterator<Arbusto> it6 = cloneCercas.iterator();
while(it6.hasNext() && itbotao.hasNext()) {
    Arbusto cerca = it6.next();
    Itens botao = itbotao.next();
    // código original...
    if(/* ... */) {
        listaCercas.remove(1);
    }
}

Outra possibilidade é criar uma lista com os elementos que devem ser removidos e, somente após o laço ter terminado, remover os elementos com o método listaCercas.removeAll(cercasARemover).
Os mesmos conselhos valem para a adição de elementos as listas.
Cabe a você avaliar se será necessário levar em consideração as alterações da lista durante a iteração. :)


Como ver os valores das variáveis em tempo de execução?

Question: Como posso fazer para saber os valores das variáveis de uma aplicação em tempo de execução?
Quero saber qual o valor de algumas variáveis pra saber se o programa está agindo da forma correta.
Answer: O normal é aparecer isso quando está debugando, se não está aparecendo no seu Visual Studio pode usar CTRL + ALT + V e depois A (apenas as variáveis que estão sendo manipuladas neste momento) ou CTRL + ALT + V e depois L (mostra todas variáveis em escopo), conforme a documentação. Só funciona quando estiver em execução do debug.
Você poderá navegar pelas variáveis, abrir seu conteúdo nos casos de não escalares, como mostra esta imagem:

Dá para ir acompanhando as mudanças de valores (fica vermelho) em cada passo.
Coloquei no GitHub para referência futura.
Comment: Lembrando que para conseguir acompanhar as mudanças são necessários pontos de interrupção para parar o programa e olhar as condições naquele momento que esbarrou no ponto. Se esbarrar no mesmo novamente, interrompe novamente. Não usei muito, mas se não me engano é isso.


How to create custom Android radio buttons?

Question: I've got a couple radio button options in my android app, but I want them to look totally different. More something like below (quick mockup) in which you simply click the words you want and it makes them bold and underlined.

Does anybody know how I can achieve something like this? All tips are welcome!
Comment: If you got the answer for this the please share.
It would be a great help
Answer: Generally speaking, to override the look of default widgets, you'll need to create a drawable folder and put all of your xml definitions in that folder.  Then reference that xml file within the RadioButton block of your layout.
Here's a good blog post on how to do all that:
http://blog.devminded.com/posts/custom-android-radiobutton
Answer: I know it might be late, but it is not a reason to keep the solution for myself.
1) You need to implement the .XML layout for the RadioGroup and it's RadioButtons. Set the RadioGroup children orientation with Horizontal value to display the button side by side. Setting the RadioButton button with @null value to hide the default selector

As following:
<RadioGroup
        android:id="@+id/my_radiogroup"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:orientation="horizontal">

        <RadioButton
            android:id="@+id/i_like_radiobutton"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:button="@null"
            android:padding="10dp"
            android:text="I Like"
            android:textColor="@android:color/holo_orange_light" />

        <RadioButton
            android:id="@+id/i_dont_like_radiobutton"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:button="@null"
            android:padding="10dp"
            android:text="I Dont Like"
            android:textColor="@android:color/holo_orange_light" />
    </RadioGroup>

2) In your Activity class, initialize them and set their listener. The listener should keep track of the changes of the RadioButton changes and set the UI changes according to the state either select or unselect. As following:
    RadioGroup myRadioGroup = (RadioGroup) findViewById(R.id.my_radiogroup);
    RadioButton likeRadioButton = (RadioButton) findViewById(R.id.i_like_radiobutton);
    RadioButton dontLikeRadioButton = (RadioButton) findViewById(R.id.i_dont_like_radiobutton);

    //Like button listener
    likeRadioButton.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
        @Override
        public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
            if (isChecked) {
                //Make the text underlined
                SpannableString content = new SpannableString(getString(R.string.like_text));
                content.setSpan(new UnderlineSpan(), 0, content.length(), 0);
                buttonView.setText(content);
                //Make the text BOLD
                buttonView.setTypeface(null, Typeface.BOLD);
            } else {
                //Change the color here and make the Text bold
                SpannableString content = new SpannableString(getString(R.string.like_text));
                content.setSpan(null, 0, content.length(), 0);
                buttonView.setText(content);
                buttonView.setTypeface(null, Typeface.NORMAL);

            }
        }
    });

    //Don't Like button listener
    dontLikeRadioButton.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
        @Override
        public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
            if (isChecked) {
                //Change the color here and make the Text bold
                SpannableString content = new SpannableString(getString(R.string.like_text));
                content.setSpan(new UnderlineSpan(), 0, content.length(), 0);
                buttonView.setText(content);
                buttonView.setTypeface(null, Typeface.BOLD);
            } else {
                //Change the color here and make the Text bold
                SpannableString content = new SpannableString(getString(R.string.like_text));
                content.setSpan(null, 0, content.length(), 0);
             buttonView.setText(content);
             buttonView.setTypeface(null, Typeface.NORMAL);

            }
        }
    });

3) Now, the RadioButton will change it's color and it's TextStyle according to it's state automatically. You can add more customization if you want.
4) For performing the required action when the user select any of the Buttons, we need to override the setOnCheckedChangeListener method for the RadioGroup as following:
genderRadioGroup.setOnCheckedChangeListener(new RadioGroup.OnCheckedChangeListener() {
        @Override
        public void onCheckedChanged(RadioGroup group, int checkedId) {
            if (checkedId == R.id.i_dont_like_radiobutton) {
                //Do some actions
            } else if (checkedId == R.id.i_like_radiobutton){

            }
        }
    });

The final output will be very similar the the question image except the separator.

I hope it helps.


Show that when $p$ is a principal type, it must contain the formula that implies it

Question: In my model theory lecture course, a partial type $p$ is defined to be principal when there is some $ϕ$ such that $T \models ∃vϕ(v)$ (for any n-place variable $V$) and for any $ψ ∈ p$, $T \models ∀v(ϕ(v) → ψ(v))$.
[and we define a partial type of a theory $T$ in variables $x$ to be a satisfiable set $P$ of
formulas in the variables $x$, containing $T$ and closed under logical deduction.]
I'm asked to show that when $p$ is a principal (complete) type, the formula $ϕ$ which features in the definition above must be in $p$.
It's clear that since $T \models ∃vϕ(v)$, and $p$ must be closed under logical deduction, that $∃vϕ(v) \in p$ , but I don't see how it follows that $ϕ(v) \in p$, which is what I'm being asked to prove.
Any suggestions you could give me would very much be appreciated.
Comment: Is the principal type $p$ in question supposed to be complete?  Otherwise this isn't true.  (This definition of "principal type" is also quite weird, and I would consider it nonstandard when applied to incomplete types.)
Comment: @EricWofsey yes it should be, you're right - in our course if we just say 'type' we mean a complete one (I'm not sure how standard this is).
Comment: @yoasdf That last bit is standard, but it can get confusing when you are also talking about partial types.
Answer: As pointed out in the comment this question only makes sense for complete types. So let $p$ be a principal type, with principal formula $φ \in F_n$. Since p is complete, $φ \not\in p \implies ¬φ \in p$. Add new constant symbols $c = (c_1, c_2, ..., c_n)$. The principality condition gives us that $T \cup \{φ(c)\} \models ψ(c)$ for all $ψ(x) \in p$, and so $T \cup \{φ(c)\} \models ¬φ(c)$. It follows that $T \models ¬φ(c)$, and therefore $T \models ¬\exists x φ(x)$. This contradicts the fact that $φ$ is realised in all models of T, i.e. $Τ \models \exists x φ(x)$. Thus $φ \in p$.


Bandwidth Selection for Kernel Density Estimation

Question: Are there any heuristics for selecting the bandwidth for kernel density estimation? In other words, is a spiky curve better or a smooth one?
Comment: It depends on what you are using the KDE for.  Do you have a specific dataset or problem setting in mind?  At https://stats.stackexchange.com/a/428083/919 I provide examples of how the bandwidth affects the KDE and how to exploit that to find modes.
Answer: I assume your sample is from a continuous distribution. Then the situation is somewhat like it is for making histograms. If you have enough data, you want to choose the width of histogram bars to be just thin enough to suggest the shape of the population density curve without the 'raggedy' look that comes from neighboring bars of very different heights (maybe interspersed with some empty bins).
If the population density is multi-modal (as for a mixture of several distributions), then you don't want bars
so thick that you can't see the individual modes. [The link, provided by @whuber while I was writing this answer, pays particular attention to detecting modes.]
Using the KDE (kernel density estimator) of density in R,
I have found that the default bandwidth is usually about right. Sometimes I halve it and sometimes I double it, depending on the task at hand.
Sampling from a normal distribution, you may need a sample
as large as a thousand to get a useful density estimator.
set.seed(2921)
x1 = rnorm(500, 100, 15)
x2 = rnorm(1000, 100, 15)
x3 = rnorm(5000, 100, 15)

The three panels below compare R's default histogram bins,
KDE badwiths (dotted brown), and population density (blue)
for samples of sizes 500, 1000, and 5000. (Maybe the histogram at the right could use thinner bars, but eh default KDE bandwidth seems about right.)

R code for figure:
par(mfrow=c(1,3))
hist(x1, prob=T, ylim=c(0,.03), col="skyblue2")
 curve(dnorm(x, 100, 15), add=T, lwd=2, col="blue")
 lines(density(x1), lwd=3, lty="dotted", col="brown")
hist(x2, prob=T, ylim=c(0,.03), col="skyblue2")
 curve(dnorm(x, 100, 15), add=T, lwd=2, col="blue")
 lines(density(x2), lwd=3, lty="dotted", col="brown")
hist(x3, prob=T, ylim=c(0,.03), col="skyblue2")
 curve(dnorm(x, 100, 15), add=T, lwd=2, col="blue")
 lines(density(x3), lwd=3, lty="dotted", col="brown")
par(mfrow=c(1,1))

Now we look at a bimodal population with (for simplicity, exactly) a 50:50
mixture of distributions with means sufficiently far apart
to make bimodal populations.
set.seed(2921)
x1 = c(rnorm(250, 80, 15), rnorm(250, 120, 15))
x2 = c(rnorm(500, 80, 15), rnorm(500, 120, 15))
x3 = c(rnorm(2500, 80, 15), rnorm(2500, 120, 15))

Here again, we use defaults in R. The histogram at the left
should probably have fewer bars, but the KDE bandwidth seems OK. Often, the KDE bandwidth is less 'fussy' than the width of histogram bars.

par(mfrow=c(1,3))
hist(x1, prob=T, ylim=c(0,.02), col="skyblue2")
 curve(.5*dnorm(x,80,15)+.5*dnorm(x,120,15), add=T, lwd=2, col="blue")
 lines(density(x1), lwd=3, lty="dotted", col="brown")
hist(x2, prob=T, ylim=c(0,.02), col="skyblue2")
 curve(.5*dnorm(x,80,15)+.5*dnorm(x,120,15), add=T, lwd=2, col="blue")
 lines(density(x2), lwd=3, lty="dotted", col="brown")
hist(x3, prob=T, ylim=c(0,.02), col="skyblue2")
 curve(.5*dnorm(x,80,15)+.5*dnorm(x,120,15), add=T, lwd=2, col="blue")
 lines(density(x3), lwd=3, lty="dotted", col="brown")
par(mfrow=c(1,1))

Note: If the population density has restricted support (for example, $[0,\infty)$ for gamma distributions, $[0,1]$ for beta distributions, the KDEs in R are often positive just
outside the region of support.


Crontab not sending e-mail with output

Question: I'm having a strange problem here with crontab. I have set up the crontab correctly to run the cron job as follows:
[user@user ~]$ crontab -e
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
firstname.lastname@example.com
HOME=/

2 * * * * php usr/share/nginx/html/site/script.php

The script runs, I know this because the script is just a basic test that uses the mail() function to let me know it ran.
But I do not get the e-mail from crontab to let me know the script has ran. I should be getting an output e-mail to the MAILTO address above but I am not. I have even tried with quotes around the address.
There's nothing wrong with Sendmail because otherwise the script wouldn't be able to send an e-mail. There's nothing wrong with the e-mail address either because the mail() code in script.php sends to the same e-mail address.
Technically I could just send the e-mail output myself this way rather than rely on crontab but I know that's not the right way to do it.
Any idea where I am going wrong?
PS. This is a Centos 6.3 server.
Comment: Does your script produce any output to stdout and/or stderr? If not, then cron will not send an email.
Answer: What you can do is to remove that MAILTO and then just use a root forwarder.
/root/.forward  <- just place your email in there, nothing else. Assuming you are running the cron job as root. You can also do it via /etc/aliases, add the line: user: firstname.lastname@example.com
Comment: I put my e-mail in that file and removed the MAILTO and still didn't get an e-mail. Haven't tried the second thing you said yet though.
Comment: The second solution didn't work either.
Answer: I would check cron and mail log to see whether there is any error reported. Could that be also possible the email is landed in your junk mail folder or recipient mail server rejected the mail for invalid sender name or hostname etc.,?
Comment: Please do a live tail of mail log when your cron runs and see what is happening. You should see where is sendmail relaying to. tail -f /var/log/maillog
Comment: I've no idea why but it started working around 5pm yesterday. I think I did reboot my server late afternoon so maybe that changed something. Or the MAILFROM parameter I added.
Comment: There's nothing in the cron log, just a line saying the command was ran. It's not a junk folder issue either, been checking that and I've also tried directing the output to an e-mail address at a different provider.

I did have to edit the mail config to bypass local e-mail check though as shown here http://serverfault.com/questions/65365/disable-local-delivery-in-sendmail because php mail(); wasn't working at all on my the server's own domain, I wonder if that broke it.
Comment: Checking the mail log (in my case `/var/log/exim4/mainlog`) allowed me to spot the problem which was that I had a stale `root: myuser` entry in `/etc/aliases` and the user `myuser` didn't exist anymore.

Crontab not sending e-mail with output


Expression of probability

Question: Given the following question...

I'm still unclear why it's
$$P(T|L)$$
and not
$$P(L|T)$$

Appreciate any advice please.
Answer: I'm not sure where the difficulty lies. You should probably clarify why you think what you do.
$P(T|L)$ is "Probability test yields oil given low reserve"
The questions states "if an area has a small oil reserve then the probability that the test drills will yield oil is 0.2"
The first part specifies the condition (an area has a small oil reserve) ($L$)
The second part specifies the probability of the outcome. (Probability of $T$)
So when the condition ($L$) holds, $P(T)$ is $0.2$. You write that as $P(T|L)=0.2$.

Expression of probability


Significance test for yes/no data in R

Question: I have the following data format:
id     drug_type     result
--     ---------     ------
1      A             Yes
2      B             No
3      A             No
4      Placebo       Yes
.
.

How do I test the result column for significance of each drug_type that is not Placebo, against the Placebo. Usually I would take subsets of the table based on drug_type and do:
t.test(drugA,placebo)

however I know I should not be using a t-test for categorical data. The prop.test handles categorical variables, but to me does not seem to be able to compare each non-Placebo drug group to the placebo, but merely run significance test on the proportion of successful tests within each drug_type.
any suggestions?
Comment: You can run Fisher's exact test for each treatment vs placebo. The results will (almost) match the one from the suggested logistic regression.
Answer: How about logistic regression -- especially if you have other factors that you need to control?  You can simply set up a contrast to test differences in drug types.
For example:
id<-seq(1, 100)
drug_type<-as.factor(sample(c(LETTERS[1:2], "Placebo"), size=100, replace=TRUE))
result<-as.factor(sample(c("Yes","No"), size=100, replace=TRUE))
mydf<-data.frame(id, drug_type, result)
mydf2<-cbind(mydf, result2=as.numeric(mydf$result)-1)
head(mydf2)

mymodel<-glm(result2~drug_type, data=mydf2, family="binomial")
summary(mymodel)

install.packages("aod")
library(aod)
L<-cbind(0, 1, -1)
#For example to test Test Drug B against Placebo
wald.test(b = coef(mymodel), Sigma = vcov(mymodel), L = L)

Comment: thank you @StatsStudent - this is suitable for my analysis. would you be able to explain where in the `wald.test` does it pick out drug B being tested against Placebo? I can seem to pick out how drug A is not used for example.
Comment: In the statement right before wald.test, I create a vector L, which sets up the contrast. It sets the intercept term to 0 since the intercept term drops out when subtracting the Drug B effect from the Placebo effect. The next 1 is setting the coefficient for Drug B to +1 and the -1 is setting the coefficient of the Placebo to -1 so you obtain $\hat\beta_{B}-\hat\beta_{placebo}$ which measures the mean difference between the two treatments. So it does pick out drug B being tested against the Placebo. I'm not sure what you mean by "I can seem to pick out how drug A is not used for example."
Comment: To be more precise, I should have said that $\hat\beta_{B}-\hat\beta_{placebo}$ is the $estimated$ mean difference between the two treatments.
Comment: thanks @StatsStudent. I think what I am getting at is when you say `The next 1 is setting the coefficient for Drug B to +1`, to me I see no way in the code of how the second parameter of `L =1` relates to `drugB`. I assume that each parameter of the vector L represents each *ordered* unique drug type. For example if I had drugs `A,B,C,D....placebo` then L would look like `L = c(drugA,drugB,drugC,drugD,......placebo)`. Have I interperated this correctly?
Comment: No, but, it depends on how you parametrized your model.  In the example I have used, I used the default R parameterization, which sets the first ordered level to 0 (in this case drug A).  If you run a model.matrix on the model (in my example head(model.matrix(mymodel)), you will see that the the estimated coefficients are, in order, (Intercept), drug_typeB and finally drug_typePlacebo.  The contrast vector corresponds to this parameterization.
Comment: I see. Brilliant I understand now. Thanks for being patient!
Comment: Glad you got it figured out!  Best of luck to you.  I recommend reading the following on parameterization and contrasts:  http://thirteen-01.stat.iastate.edu/wiki/stat430/files?filename=lec18h.pdf
Answer: I think a Chi-square test would be appropriate for your situation, provided that you have in each cell more than 5 observations (i.e., in your case ID's).
What you need to do is make a 2 x 2 contingency table of drug type (A vs B) and Result (Yes vs No) and count the occurrences in each cell. You can than test if there is an association between the two variables. See for examples of medical research the following link: http://ocw.jhsph.edu/courses/fundepiii/pdfs/lecture17.pdf
In R the function chisq.test(table) can be used.
Comment: This doesn't address @brucezeppelin 's question about comparing the two drugs to his placebo condiion. A Chi square statistic could work, but not in the 2x2 contingecy table as you have suggested here.

Significance test for yes/no data in R


Gitk does not show HEAD of master?

Question: This is on Ubuntu 14.04, git version 1.9.1 (same for gitk). I'm doing this:
$ cd /tmp
$ git clone https://github.com/underdoeg/ofxLibRocket.git ofxLibRocket-git
Cloning into 'ofxLibRocket-git'...
remote: Counting objects: 1013, done.
remote: Total 1013 (delta 0), reused 0 (delta 0), pack-reused 1013
Receiving objects: 100% (1013/1013), 16.85 MiB | 326.00 KiB/s, done.
Resolving deltas: 100% (631/631), done.
Checking connectivity... done.
$ cd ofxLibRocket-git
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean
$ git log -1 --decorate
commit 14360ed8fd3428aa0a2e052f6363e5daa0bf0d75 (HEAD, origin/master, origin/HEAD, master)
Merge: 06dd209 801877c
Author: Philip Whitfield <firstname.lastname@example.com>
Date:   Fri Feb 22 03:16:37 2013 -0800

    Merge pull request #6 from neilmendoza/master

    Fixed OSX libs and example
$ gitk --all .

So far so good - but, if I run gitk --all . in this directory, then the commit of current HEAD is not indicated in the gitk display? This is what I get:

This is the weird thing: the only commit containing the word OSX shown in gitk, which I've selected manually:

has the hash 801877c48390324dc7d600936306cd7c00486dcc, unlike what git log shows for current head, 14360ed8fd3428aa0a2e052f6363e5daa0bf0d75
The 14360ed8 is the "merge pull request" by one committer, the 801877c the pull request by another committer, for the same patch/commit, "Fixed OSX libs and example"
The 14360ed8 is not listed in any commit of gitk
There is no yellow dot to indicate HEAD, nor a "master" branch label, anywhere in gitk (and certainly not next to 801877c).

So, why doesn't gitk in this case show the HEAD and the 14360ed8 commit? I have tried doing:
git checkout master
git reset --hard HEAD

... but it seems to make no difference in terms of display in gitk... How can I get the proper display of HEAD and the master branch in this case?

EDIT: So, I installed gitg (Version 0.2.7), and it seems to make more sense visualisation-wise:

So is this a bug in gitk, or does gitk need some special setup (say, command line switches) to show this state of the repository?
Answer:
gitk --all without path specification gives the expected result, exactly as gitg
It seems that gitk <some path> completely ignores --all switch and displays the first commit which modifies a given file and doesn't collect any references. So again I wouldn't call it a bug, just a, hmm, way it's implemented. Also I should mention that specifying a file in the command line for gitk is a unusual thing for me.
Comment: Many thanks, @user3159253 - indeed, without the dot for current directory, `gitk --all` shows the yellow dots for HEAD etc. Re: "_specifying a file in the command line for `gitk` is a unusual thing_" - if you mean my inclusion of the current dir dot (as in `gitk --all .`), I simply saw it somewhere on this site when I looked up why I cannot see all branches in `gitk`, and I had remembered it verbatim. Cheers!

Gitk does not show HEAD of master?


L is a complete lattice, prove that there exists $x \in L$, which is a fixed point of $f$ and is the smallest $y$ that $f(y) \le y$.

Question: I have such task:
Let $(L, \le)$ be a complete lattice and $f: L \to L$ a monotonic function. Proof that there exists $x \in L$ which is a fixed point of $f$ and is the smallest $y$ that $f(y) \le y$.
I believe it has much to do with the Knaster–Tarski theorem. I think that by proving the Knaster–Tarski theorem I show that there in fact exists such element. But I have problems understanding the theorem's proof. Or perhaps there's a better and easier way to prove that.
If anyone could explain that in simple words I would be very thankful.
Comment: You're entirely correct. In fact, the theorem you're looking to prove _is_ the Knaster-Tarski theorem as stated (and proved) in Davey and Priestley's _Introduction to Lattices and Order_. That said, it's currently very hard to answer your question -- you say you have problems understanding the theorem's proof, but _where_ are you having problems? Can you be more precise about what parts of the proof you do/don't understand? As for your goal, again, what have you tried? Once we have a better idea of exactly where you're struggling, we can help you better ^_^
Answer: Indeed, this is the Knaster-Tarski Fixed Point Theorem.
Forget about the fixed-point part for a bit. Look at the set
$$
A:=\{y\in L: f(y)\le y\}
$$
We'd like to look at the minimum of $A$, but this need not exist in a general lattice. So, we take the next best thing, namely $x:=\inf (A)=\bigwedge A$.
Fix $y\in A$. We know that $x\le y$, so $f(x)\le f(y)\le y$. This shows that $f(x)$ is a lower bound for $A$, hence that $f(x)\le x$. Applying $f$ again, $f(f(x))\le f(x)$, so $f(x)\in A$. Since $x$ is a lower bound for $A$, $x\le f(x)$. Therefore, $f(x)=x$.
We've argued that $x$ is a fixed point of $f$. Moreover, we've shown that $x=\bigwedge A\in A$, so that we actually have $x=\min(A)$, which is what we wanted to show.
Comment: Thank you very much, It doesn't seem that hard when you explained it.
Comment: Why do you say $x = min(A)$? You showed $x$ is the inf of the prefix points and that $x$ is a fixpoint…. oh duh because fixed points are prefix points…
Comment: @Hans I showed that $x:=\inf(A)\in A$, which shows that $A$ has a smallest element, namely $x$.


Mostrar informações de um dataframe em um mapa shapefile

Question: Tenho uma tabela .dta que já importei para o R. Uma das colunas desse arquivo chama-se "mun" e tem o código dos municípios do Brasil, um número de 7 digitos.

Baixei o Shapefile do site do IBGE
que vem todos os arquivos: .cpj, .dbf, .prj, .shp, .shx
Usei o maptools e importei o mapa para o R. O mapa já possui as informações dos códigos dos municípios como: CD_GEOCMU
require(maptools)
mapa<-readShapeSpatial(file.choose())

Meu arquivo tem mais de 4 milhões de dados com seus respectivos códigos dos municípios. Eu preciso colocar estes dados no mapa com uma escala de cor para poder ver legal a distribuição destes dados no Brasil, mas não tenho ideia de como fazer isso. Agradeço muito se alguém puder me ajudar.
Answer: Acredito que a função geom_map, da biblioteca ggplot2, é justamente que você está procurando. Tem uns exemplos similares na documentação que linkei. Grosso modo, basicamente você basicamente usa o parametro fill para definir qual variável usar para preencher as regiões.
Este artigo explica passo-a-passo como tratar os dados de modo a se adequarem melhor à lógica da ggplot2.


How to compare two datagridview's

Question: I need help with my litle project.
So what i need is to compere two datagridview's.
I have made something like this
<code>(dataGridView1.DataSource as DataTable).DefaultView.RowFilter = string.Format("test= '{0}'", textBox4.Text);

(dataGridView2.DataSource as DataTable).DefaultView.RowFilter = string.Format("test= '{0}'", textBox4.Text);
</code>
and it works for manual search.
But i need to look "test" in dgw1 and "test" in dgw2 and look if values are same.
Comment: It is not clear what you are asking.
Answer: You can use linq for this as follows:
<code>var idsNotInB = (dataGridView1.DataSource as DataTable).AsEnumerable().Select(r => r.Field<int>("id"))
        .Except(dataGridView2.AsEnumerable().Select(r => r.Field<int>("id")));
</code>
Comment: @user2768054 It is `(dataGridView1.DataSource as DataTable).AsEnumerable()`... not `dataGridView2.AsEnumerable()`.
Comment: .Except(dataGridView2.AsEnumerable().Select(r => r.Field("id")));

How to compare two datagridview's


What is the number of elements in a set if the ratio of the number of subsets to the number of elements is $\frac {32}{3}$?

Question: What is the number of elements in a set if the ratio of the number of subsets to the number of elements is $\frac {32}{3}$?
At first, I thought that
$$2^n = \frac {32}{3}$$
I see that
$$\text {s} = 2^n$$
Where am i going wrong or where to start?
Comment: you forgot the number of elements in your ratio
Comment: @J.Sadek Thank you so much :)
Comment: You're welcome.
Answer: $\frac{2^n}n = \frac {32}{3}$
n=6
Answer: Hint: Use the fact that for a set with $n$ elements ("number of elements" in the question), there are $2^n $ subsets ("number of subsets" in the question).


Rotate about a point or an axis?

Question:
English is not my native language. Why math text uses the preposition "about" while referring rotation in the context of a point, an axis or a plane? Is not the preposition "around" or "on" more natural to use in this context?
Is matrix multiplication of NON-square matrix and identity matrix commutative?
Scale applies along an axis. For example, if you scale along x-axis, the horizontal length of the shape will increase/decrease in 2D. Why the author of the following para (whole para is pasted below to clear the context) is saying that it is applied about the perpendicular axis? Also, why did he use the preposition "about" here?

Scaling along the Cardinal Axes
The simplest scale operation applies a separate scale factor along each cardinal axis. The scale along an axis is applied about the perpendicular axis (in 2D) or plane (in 3D). If the scale factors for all axes are equal, then the scale is uniform; otherwise, it is nonuniform.

Please help if you can answer any one of the three questions.
Thanks.
Comment: Obviously, around and about are synonyms in this context. You can't swap a rectangular and a square matrix in a product for obvious dimension compatibility reason.
Comment: Thanks. Can we call following multiplication commutative: IA=AI where A is rectangle matrix and I is a square matrix and dimension compatibility requirement is met?
Comment: Please given an example of $A$ that would make this comparison possible.
Comment: A (2x3): A1 (first row) = [1 2 3] A2 = [4 5 6]. I (3x3) identity. AI=A. I (2x2) then IA=A. Can we say it commutative even if dimension of I is different is both cases?
Comment: Is $I$ $2\times2$ or $3\times3$ ?
Comment: I is 2 x 2 for IA and 3 x 3 for AI.
Comment: Come on, this is nonsense, $I\ne I$.
Comment: Got it. Thanks.
Answer: "around" is pretty common too; this is more a question for the English language stacks, but I know that "about" was more commonly used in the past the way "around" is now; see for instance this passage from Shakespeare's Macbeth.

O well done! I commend your pains;
And every one shall share i' the gains;
And now about the cauldron sing,
Live elves and fairies in a ring,
Enchanting all that you put in.

A matrix that isn't square can only be multiplied on both sides by another matrix that isn't square: a $2\times3$ matrix can only be multiplied on both sides by a $3\times2$ matrix.  The results of these two multiplications are both square matrices, of different sizes: if $A$ is $2\times3$ and $B$ is $3\times2$, then $C = AB$ is a $2\times2$ matrix, and $D = BA$ is a $3\times3$ matrix.  Because they are different sizes, $C$ and $D$ are obviously not equal; because they are not square, neither $A$ nor $B$ can be an identity matrix.

The final passage is ... not how I would put things, but I think what he means is - a scaling along a particular axis fixes its perpendicular: in 2 dimensions, a scaling along one axis leaves a line perpendicular to the axis unchanged, and in 3 dimensions it similarly leaves a plane perpendicular to the axis unchanged.  This sense of "about" is shared with rotation, in that in both cases these operations leave the things they are "about" unchanged.
Comment: Excellent answers. It was a treat to read answer of the 3rd question. Thanks. You know better but  as per my knowledge a non-square matrix can be multiplied with a square matrix: AB is possible if  A = 2 x 3 and B is 3 x 3, AB is also possible if A is 2 x 2 and B is 2 x 3. Please clarify why non-square matrix can not be multiplied with a square matrix on either side?
Comment: Those two matrices, the 2x2, and 3x3, are *different matrices*, even if they're both technically identity matrices.

Rotate about a point or an axis?


Secure email routing

Question: We have relationships with some banks that are trying to implement some secure email. Is there any method that would force the server to send emails via TLS when sending email to a specific domain? This is a cpanel/WHM server.
We basically need to require that an email be secure from
user -> mail server
and from
mail server -> bank's server.
I'm not entirely familiar with what happens after the email leaves my email program. If I send a message over TLS or SSL outgoing connection, does the email leave the mail server also over a secure connection? I have no problem implementing TLS from the client to the mail server. It's after it gets to the mail server that I am unclear on.
Any help on this would be appreciated.
Answer: Others can talk about the cpanel-specific issues, but I can talk to SMTP TLS.
A TLS connection is negotiated between two mailers as part of the ESMTP negotiation process. Most mailers can be configured to prefer TLS connections over plain ones (or not, encryption does cost CPU cycles), and the major Linux mailers also have the ability to set per-domain mail policies that include TLS settings. So you can specify wellsfargo.com gets TLS, but gmail.com doesn't for instance. Mailers are pretty good these days about stamping Received-By: headers with (TLS) if it received the mail over a TLS connection.
This will encrypt the mail in transit between actual mailers. Encrypting it between the sender and the mailer will require the sender to use an encrypted method to send, either TLS or SSL. Requiring login to send mail and only allowing logins over encrypted methods will accomplish this.
If you require the mail to be encrypted while in mail queues, then you'll have to encrypt the mails themselves with either S/MIME or PGP/GPG.
Answer: Check the TLS/SSL chapter in the Exim Specification.  If you have TLS enabled, it will be used on outgoing connections to servers which advertise STARTTLS.  TLS will be required for hosts listed in the hosts_require_tls option.  This should be set at the beginning of the Exim configuration file.
Answer: Personally (and for management), I'd not trust this, if for no other reason than if something went wrong you might not know about it without strict monitoring of the logs and it depends on not just your server's config, but all the other organizations staying on board and you have no control over their updates/alterations/politics. I'd rather keep more control and do it with PGP on the user mail clients. You'll be sure that the mails are encrypted end to end that way, even if the mail server is compromised, not just in the transit channel.

Secure email routing


Updating cell in Google Spreadsheets returns error "Missing resource version ID" / "The remote server returned an error: (400) Bad Request."

Question: I would like to update a cell value in Google Spreadsheets but unfortunatelly an error is received:
Google.GData.Client.GDataRequestException was unhandled
  HResult=-2146233088
  Message=Execution of request failed: https://spreadsheets.google.com/feeds/cells/1nW8nxoS2l9pbj6dctreEfKHNXmsfbbsCAvOd7TIj4Bo/od6/private/full/R1C1
  Source=Google.GData.Client
  ResponseString=Missing resource version ID
  StackTrace:
   at Google.GData.Client.GDataRequest.Execute()
   ...
   at System.Threading.ThreadHelper.ThreadStart()
InnerException: System.Net.WebException
   HResult=-2146233079
   Message=The remote server returned an error: (400) Bad Request.
   Source=System
   StackTrace:
        at System.Net.HttpWebRequest.GetResponse()
        at Google.GData.Client.GDataRequest.Execute()

My code is very simple and is based on sample downloaded from https://developers.google.com/google-apps/spreadsheets/?csw=1#changing_contents_of_a_cell:
        SpreadsheetsService service = new SpreadsheetsService("MySpreadsheetIntegration-v1");

        // TODO: Authorize the service object for a specific user (see other sections)
        service.setUserCredentials("...", "...");

        // Instantiate a SpreadsheetQuery object to retrieve spreadsheets.
        SpreadsheetQuery query = new SpreadsheetQuery();

        // Make a request to the API and get all spreadsheets.
        SpreadsheetFeed feed = service.Query(query);

        foreach (SpreadsheetEntry spreadsheet in feed.Entries)
        {
            if (spreadsheet.Title.Text == "Test01")
            {
                // Get the first worksheet of the first spreadsheet.
                WorksheetFeed wsFeed = spreadsheet.Worksheets;
                WorksheetEntry worksheet = (WorksheetEntry)wsFeed.Entries[0];

                // Fetch the cell feed of the worksheet.
                CellQuery cellQuery = new CellQuery(worksheet.CellFeedLink);
                cellQuery.MinimumRow = 1;
                cellQuery.MaximumRow = 10;
                cellQuery.MinimumColumn = cellQuery.MaximumColumn = 1;
                cellQuery.ReturnEmpty = ReturnEmptyCells.yes;
                CellFeed cellFeed = service.Query(cellQuery);

                // Iterate through each cell, updating its value if necessary.
                foreach (CellEntry cell in cellFeed.Entries)
                {
                    cell.InputValue = "Foooooo!";
                    cell.Update();
                }

            }
        }

The error is raised on the following line:
                    cell.Update();

I use Google.GData version 184.108.40.206 (http://code.google.com/p/google-gdata/).
Do you know what could cause this problem?
[Edit] This issue has also been reported in the gdata python client. Hope it gets fixed soon.
http://code.google.com/p/gdata-python-client/issues/detail?id=692&sort=-opened&colspec=Opened%20Stars%20ID%20Type%20Status%20Priority%20Component%20Summary
Thank you!
Comment: I had this error. Was just a case of add it in. It used to be optional, but no longer is.
Comment: What exactly do you mean by "add it in"?
Comment: Sorry was in a rush. Looks like you have the ID I was thinking of, i.e. SpreadsheetsService("MySpreadsheetIntegration-v1"); .... Are you using the old style spreadsheet (400,000 cell limit). Last I read, it does not work with new style spreadsheet, is work in progress. .... Also I use batch update as using single cell updates is slow.
Comment: Update: As on Apr 1st all gdata writes stopped working. I get "Unusual traffic from your computer network" as described here, but reads work just fine. https://support.google.com/websearch/answer/86640
Comment: I'm using new Spreadsheets. It works fine in old Spreadsheet. I think that "Unusual traffic" is a different problem not connected to this one.
Comment: So is the problem caused by using new style spreadsheets? ..... Agree, the "Unusual traffic" problem is not connected.
Comment: I think so, the code works fine with old spreadsheet.
Comment: Just tested, new style sheets work with Java. But someone said it fails with Python3, see: http://stackoverflow.com/questions/20599425/is-an-api-available-for-the-new-google-sheets-spreadsheets
Comment: I'm seeing the same thing as oneee.  Was working fine until the forced update to the new spreadsheet.  Anyone able to resolve this?
Comment: Not yet, fortunately we still can use old spreadsheet. Who forced you to update to the new spreadsheet? Google?
Answer: We hit this same issue about a week ago when it seems that Google flipped over all spreadsheets to the "new" format.
This goes for new ones created via the GData api as well.  400 errors everywhere.
I dug into the reports across the GData variant libraries (Python, Java, .Net etc) and eventually found this little nugget:
https://stackoverflow.com/a/23438381/1685090
Setting the Etag property to "*" is the answer :)
To be clear, we're setting WorksheetEntry.Etag when we want to run Update() on a worksheet, and we're also setting CellEntry.Etag when doing batch updates via SpreadsheetsService.Batch().
So far it seems to work fine with Google's "new" spreadsheets.
One catch with this approach is that any concurrent/merging operations will be foregone - essentially you are telling Google that your update must blow away any other concurrent prior value in the cells.
Comment: @Paul DB , can you please insert a short script with setting that Etag property in PHP/Zend? I would really appreciate it! Thanks!
Comment: I'm not sure how it would work in PHP, in the .Net version I believe the ETag property maps to a 'If-Match': '*' header being added to the HTTP request that goes over the wire to Google.
Comment: I hit this working on a python library I'm writing (http://sheetsync.readthedocs.org/en/latest/)

I wrote this monkey patch https://gist.github.com/mbrenig/3d007fb8f5f98468a228  perhaps you can do something similar?
Comment: Has anyone found a way to do this in PHP? Looking to fix my spreadsheet software which cannot import data anymore...
Comment: @Laci have you found a solution?
Answer: Another way I solved this is through adding an additional HTTP header
If-Match: *

Which says overwrite anything.


If Goldbach's conjecture is false, is it possible that there are only a finite number of failing cases?

Question: If Goldbach's conjecture is false, is it possible that there are only a finite number of failing cases?
I know it is probably unknown, but any reference to something addressing this question would be apreciated.
Comment: Currently, every finite number of failures is possible (of course most likely is "$0$ failures") and I think we cannot even rule out inifnite many failures.
Answer: You may wish to have a look at section 3.2 of Vaughan's The Hardy-Littlewood Method, where he proved that
Theorem: Let $E(x)$ denote the number of even numbers $\le x$ that cannot be written as a sum of two primes. Then for all $A>0$, there exists a constant $C(A)>0$ such that for large $x$:
$$
E(x)<C(A){x\over\log^Ax}
$$
This might not be the best possible bound as the book was published in the 1980s.


Solving the system of recurrence relatioin

Question: Given the recurrence system:
\begin{equation*}
 \begin{cases}
   T_n = T_{n-1} + S_n, &\\
   S_n = T_{n-1} + S_{n-1} &
 \end{cases}
\end{equation*}
And we know $T_0 = 1, S_0 = 0$.
I tried to solve it the following way:
Using generating functions I represent the system as
\begin{equation*}
 \begin{cases}
   \mathfrak{T}(t) - 1 = \mathfrak{T}(t) + \mathfrak{S}(t), &\\
   \mathfrak{S}(t) = \mathfrak{T}(t) + \mathfrak{S}(t) &
 \end{cases}
\end{equation*}
But solving obtained system of equation I conclude that $\mathfrak{T} = 0$ and $\mathfrak{S} = -t$.
It seems wrong for me. Possibly is my way of solving wrong?
Comment: By the way, it looks like Fibonacci numbers
$F_0 = S_0, F_1 = T_0, F_2 = S_1, F_3 = T_1$, and so on
Answer: Use generating functions, $S(z) =  \sum_{n \ge 0} S_n z^n$ and similarly $T(z)$. Shift indices:
$\begin{align}
  S_{n + 1} &= S_n + T_n \\
  T_{n + 1} &= S_{n + 1} + T_n
\end{align}$
Multiply by $z^n$, sum over $n \ge 0$ and recognize the resulting sums:
$\begin{align}
  \frac{S(z) - S_0}{z}
    &= S(z) + T(z) \\
  \frac{T(z) - T_0}{z}
    &= \frac{S(z) - S_0}{z} + T(z)
 \end{align}$
Plug in initial values $T_0 = 1, S_0 = 0$ and solve:
$\begin{align}
   S(z)
     &= \frac{z}{1 - 3 z + z^2} \\
   T(z)
     &= \frac{1 - z}{1 - 3 z + z^2}
\end{align}$
Next step is to split into partial fractions (ugly, quadratic irrationalities) and read off the coefficients.


Probability question with a deck of cards

Question: Just started to learn maths, so I'm sorry if this an elementary question.
The question is: here is a deck with 40 cards ; 10 cards, each 10 with one of the 4 shapes (hearts, diamonds, clubs and spades). All the cards are numbered from 1 to 10. Every card has the same probability to be drawn.
We start drawing cards from the deck. If the 11th card which was drawn is the first one with number 10 on it, what is the probability that the next card is a clubs?
What I have tried:
We need to find P(A|B),
where A is the probability that the 12's card is a clubs, and B is the probability that the 11's card is a first ten.
I chose the sample space to be picking 12 cards from 40.
$P$($\Omega$) = $\binom{40}{12}$
$P(B)$= $\binom{36}{10}\binom{4}{1}\binom{29}{1}$. Where in $P(B)$, i chose 10 cards from 36 cards(without 10's), the picking 1 card from the the 10, the choosing from the rest.
$P(A\cap B)$ = $\binom{9}{9}\binom{26}{1}(\binom{3}{1}\binom{1}{1} +\binom{1}{0})$ $+$ $\binom{9}{8}\binom{26}{2}(\binom{3}{1}\binom{2}{1}+\binom{1}{1})$ $+$ $\binom{9}{7}\binom{26}{3}(\binom{3}{1}\binom{3}{1} + \binom{2}{1})$ $+$ $\binom{9}{6}\binom{26}{4}(\binom{3}{1}\binom{4}{1}+\binom{3}{1})$ $+$ $\binom{9}{5}\binom{26}{5}(\binom{3}{1}\binom{5}{1}+\binom{4}{1})$ $+$ $\binom{9}{4}\binom{26}{6}(\binom{3}{1}\binom{6}{1}+\binom{5}{1})$$+$ $\binom{9}{3}\binom{26}{7}(\binom{3}{1}\binom{7}{1}+\binom{6}{1})$$+$ $\binom{9}{2}\binom{26}{8}(\binom{3}{1}\binom{8}{1}+\binom{7}{1})$$+$ $\binom{9}{1}\binom{26}{9}(\binom{3}{1}\binom{9}{1}+\binom{8}{1})$$+$ $\binom{9}{0}\binom{26}{10}(\binom{3}{1}\binom{10}{1}+\binom{9}{1})$
. Here, $\binom{9}{8}$ is the possibility that we choose 8 from 9 clubs ( we dont want to pick 10), $\binom{26}{2}$ is completing the picking of the rest 10 cards, $\binom{3}{1}\binom{3}{1}$ is picking 1 10 from 3 and picking a club from the 3 that left, $\binom{1}{1}$ is picking the ten of clubs and then picking the only left clubs. The rest of the calculations are the same.
So, $P(A|B)  =  \frac {P(A\cap B)}{P(B)} = \frac {5271351254}{29485675300} $ $ \approx $ 0.178
The correct answer is $ \frac 14$. My questions are: am I wrong, and, if so, is there a "nicer" way to compute $P(A\cap B)$(or generally solving this exercise)?
*Sorry if there are mistakes in grammar.
Comment: The probability is the same for the 12th card whatever the suit specified, so it should be clear what the probabability is for the 12th card being a club
Comment: What do you mean by "If the 11's card who was drawn is the first ten"
Comment: @C.Koca We pull out 10 cards from the deck ( we didn't pick a  card with a number of 10), and only in the 11's card we pull  a card with a number of 10.
Comment: @Arthur Vause If i understand, beacuse every card has the same possibility to be drawn and there are an even amount of card ( ten from each shape) , then we can say the probability is 1/4?
Comment: @Yoxbox my argument is the same as C. Koca has described in his answer
Comment: @Yoxbox It is too hard to follow your logic in the combination argument. I tried to find where you might have gotten wrong but it will take a long time :) I have to ask this though: If you remove all the clubs and all the 10s, you are left with 27 cards, not with 26 cards. Could this be the case?
Comment: @C.Koca I've tried with 27 as well ( I wasn't sure if it's 26 or 27 ). Should have mentioned it, sorry. The final answer ( when 26 is replaced with 27 ) is ~ 0.27.
Comment: @Yoxbox Then there might be some other logical or computational error as well. You should get the same probability with your approach, it would just take a long time.
Comment: @C.Koca Yes i guess.  Thanks for the help!
Answer: The unknown developments prior to the selection do not affect the probability of the outcome of that selection.
The key point is that you have no knowledge of the symbol of the cards for the first 11 draws. So, you cannot have any idea about the symbol of the 12th card. So, all the developments, related to the symbol of the cards are unknown to you and the initial probability do not change.
Knowing the first 10 being the 11th card however would have an impact on the number of the 12th card.


How to create a circular heatmap in R shiny?

Question: I want to create a circular heatmap in R shiny. However, I cannot create a circular heatmap that I created in R in R shiny.
library(circlize)

# data
data <- matrix(rnorm(100), ncol = 10)

# Create a circular heatmap
circos.clear()
circos.par("track.height" = 0.2)
chm <- circos.heatmap(data, col = colorRamp2(c(-2, 2), c("blue", "red")))

# Display the circular heatmap
circos.trackPlotRegion(chm)

Answer: There shouldn't be anything special. You can wrap the heat map inside a renderPlot and display it using plotOutput. Here is a minimal example.

library(circlize)

set.seed(0)

ui <- shinyUI(fluidPage(headerPanel("Circular Heatmap"),
                        fluidRow(column(
                            12,
                            plotOutput("heatmap")
                        ))))

server <- function (input, output, session) {
    data <- reactive({
        matrix(rnorm(100), ncol = 10)
    })

    output$heatmap <- renderPlot({
        circos.clear()
        circos.par(track.height = 0.2)
        circos.heatmap(data(), col = colorRamp2(c(-2, 2), c("blue", "red")))
    })
}

shinyApp(ui = ui, server = server)