Byte skipping when casting occurs

Question: I have a struct that looks like this
struct message_header
{
    unsigned long msg_num : 32;            //0-3    message id
    unsigned long msg_len : 32;            //4-7    message length
    unsigned long hardware_version : 16;   //8-9    hardware version
    unsigned long sender_location : 32     //10-13  location
    unsigned long message;                 //14 ... messages
};

message_header * msg_ptr;

After receiving the character array (recvbuf) from recvfrom function, i will do
reinterpret_cast,
msg_ptr = reinterpret_cast<message_header*>(recvbuf)
However, after my simulator (from linux, VM) sends the data, and my receiver (the struct is in the receiver) (on windows), the output data does not tally.
Assuming the data send from the simulator is:
msg num : 1010
msg len : 20
hardware version: 1
location: 25
messages: //rest of the bytes
Packets from the wireshark shows:
00 00 03 F2
00 00 00 14
00 01 00 00
00 19 00 00
...
The output print is:
msg num : 1010
msg len : 20
hardware version: 1
location: 1638400
After much debugging, I noticed that during the cast, after hardware version, the 00 00 after the 01 was either discarded, or skipped, i am not sure which as I can't find a way to determine it, and the next 4 bytes of 00 19 00 00 was cast into my sender_location.
All the message types and length, from the sender is build base on the sender's design specification, and the hardware version on the sender is an unsigned short.
I have read up on the following question and answers
Fields in a struct skipping bytes
#pragma pack effect
And i have tried but to no avail, please advise.
Comment: Have you checked that the data in the buffer right after you received it is correct? Have you checked the data in the buffer directly before the cast? If it's correct at either point then it might be a compiler bug. If the data is incorrect when cast it, but correct when receive it, then it's modified somewhere between and you have to look at that code. It could also be problems with threads overwriting data perhaps? There could be thousands of reasons behind the problem, but without a [Minimal, Complete, and Verifiable Example](http://stackoverflow.com/help/mcve) it's impossible to say more.
Comment: Are you using an old compiler that doesn't support [C99 fixed width integer types](http://en.cppreference.com/w/c/types/integer) or can pack structures? Is there some other reason you have a bitfield?
Comment: I am using VS 2015, and i am using bitfield to limit on the amount of bytes getting cast into the per variables, i am a junior programmer and using bitfield is "recommended" by my senior, i was initially using memcpy per field but was told that casting it into a struct would be less complicated.
Comment: As for your problem, have you tried to allocate the message structure separately and then using `memcpy` to copy the data (the whole structure)?
Comment: gonna try that now and let you know the result
Comment: @Someprogrammerdude yes, memcpy, works just fine
Comment: Then I suggest you use that for now, while you try to figure out why the casting doesn't seem to work (it should, if the buffer is large enough and you don't modify it directly any other way).
Comment: i will have to change my entire class, which might be a bit of work, and yes, i do not understand why are my bytes discarded, or not being cast into the variables as well.
Comment: Yes, i have verified when the message arrives on the machine, via wireshark, that they are correct, i have checked byte by byte myself to make sure what i sent from the simulator sender is correct, and i have printed the entire character array out, in hex form, from the `recvfrom` function, to make sure it is correct as well, and this is when i noticed that my `00 00` were skipped/discarded.
Comment: You should look into how to mock a program and post it as a [mcve]. Chances are, you will have found the problem while doing this
Answer:
The alignment of a member will be on a boundary that is either a multiple of n or a multiple of the size of the member, whichever is smaller.

I did not understand what this means at first, till i re-read over and over again, i realize that my hardware version is using a long of 4 bytes, rather than i should be using a short of 2 bytes instead, another question comes to mind, which is why is the bitfield limitation not working, or i  may have understand it wrongly.
So after changing it from long to short, and with #pragma pack(1), the cast works fine now.
And how i found out was by using offsetof function to check
cout << offsetof (message_header, msg_num) << endl;
cout << offsetof (message_header, msg_len) << endl;
cout << offsetof (message_header, hardware_version) << endl;
cout << offsetof (message_header, sender_location ) << endl;
cout << offsetof (message_header, message) << endl;
And the printout is, then i realize the bitfield was not working at all.
0
4
8
12
16
After changing hardware_version, from long to short, AND without #pragma pack(1), the printout is still similar to the one above, however, once #pragma pack(1) is enabled, the printout is
0
4
6
10
14

Byte skipping when casting occurs


Repassar dados capturados de um código Python para Shell Script

Question: Gostaria de saber o que fazer para pegar esses dados captados de um sensor DHT11 (Código em Python [Temperatura e Umidade]) e passá-los para um script em Shell
if umid is not None and temp is not None:
     print("Temperatura = {0:0.1f}Umidade{1:0.1f}\n").format(temp,umid);

Comment: Você quer passá-los um por vez, ou múltiplos por vez?
Answer: Faça assim:
from subprocess import call

call(["sh", "caminho_do_script.sh",  temp, umid])

Você precisará usar um list. O primeiro item desse list é o comando, e o restante, os argumentos.
No exemplo acima, é como se eu tivesse chamando direto do Bash (supondo os valores de temp e umid):
sh caminho_do_script.sh 30.6 10.2

Existem várias formas de fazer isso em Python, como por exemplo usando os.system:
import os
os.system("sh caminho_do_script.sh {0:0.1f} {1:0.1f}".format(temp,umid))

Se quiser mais opções, pode dar uma olhada nessa resposta do Stackoverflow  English


Manipulando classe dentro de outra classe e salvar em array

Question: Criei uma classe Pessoa e nela um atributo endereco, só que fiz uma classe Endereco para cadastrar mais de um endereco por pessoa.
Nessa outra classe Endereco tem id e nome da rua, e não consigo manipular elas da classe Pessoa.
Quero salvar os dois junto em um mesmo array.
public static void main(String[] args) {

        ArrayList<Pessoa> pessoa = new ArrayList<>();

        Pessoa c1 = new Pessoa();
        c1.setId(1);
        c1.setNome("Pedro");
        c1.setEndereco(endereco);//Erro, nao consigo manipular para por o nome da rua e nem o id

        System.out.println(c1.getEndereco());

        pessoa.add(c1);
}

public class Pessoa {

    private int id;
    private String nome;
    private Endereco endereco;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getNome() {
        return nome;
    }

    public void setNome(String nome) {
        this.nome = nome;
    }

    public Endereco getEndereco() {
        return endereco;
    }

    public void setEndereco(Endereco endereco) {
        this.endereco = endereco;
    }

}

public class Endereco {

    private int id;
    private String rua;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getRua() {
        return rua;
    }

    public void setRua(String rua) {
        this.rua = rua;
    }

}

Comment: Você declara a variável `endereco` aonde na função main? Não teria que criar um ArrayList de endereço e um objeto de Pessoa?
Comment: Tipo, quero um array que possa cadastrar varios endereco se precisar. Mais o Endereco que esta em outra classe possa ser salvo na sequencia certa do array de pessoa.
Comment: A resposta resolveu sua dúvida? Acha que pode aceitá-la? Veja o [tour] se não souber como faz. Isso ajudaria muito indicar que a solução foi útil para você. Também pode votar em qualquer pergunta ou resposta que achar útil no site todo
Answer: Primeiro vamos combinar que o que você chama de atributo na verdade se chama campo.
Você tem que criar um objeto Endereco no seu código e aí passar esse objeto para o objeto Pessoa, nem precisaria criar uma variável que você criou mas não só não colocou um valor nele, mas nem a declarou. Claro que na forma que está fazendo a variável será necessária, mas só porque todo o código está conceitualmente errado, por exemplo deveria usar construtor (não vou entrar sobre o abuso de getter e principalmente setter, mas pesquise aqui mesmo no site sobre o assunto).
Seria mais ou menos assim:
var endereco = new Endereco();
endereco.setId(1); // sei lá se isso deveria existir e se deveria ser livre assim
endereco.setRua("rua");
c1.setEndereco(endereco);

Coloquei no GitHub para referência futura.
Endereço não tem só rua, este é outro conceito bastante errado. Claro que isso importa pouco para exercício mas é bom saber. Na verdade as pessoas erram muito sobre endereços.
Se deseja um objeto para cadastrar uma lista de pessoas, então a variável deveria se chamar pessoas, certo?
Se deseja cadastrar mais de um endereço por pessoa então terá que ter um campo com uma lista de endereços, o que faz o código ser muito diferente deste que está usando, e aí precisa rever esses setters e provavelmente getters. Por isso que eu digo para nunca usar um conceito que não entende, só porque as pessoas estão usando, talvez elas também não entendem porque usaram, ou podem entender e fazerem certo, enquanto os demais não conseguirão o mesmo.
É bem complicado manter a coesão e baixo acoplamento com o que quer fazer mantendo preceitos de OOP. Então a classe Pessoa só deveria lidar com esse array, de forma indireta para não expor detalhes de implementação e a classe Endereco cuidar de todo o resto dele.
Comment: Minha ideia era fazer assim
Pessoa c1 = new Pessoa(1, "Pedro",endereco);
Porem travei como por o endereco que esta em outra classe e poder modificar ele.
Comment: é, seria melhor assim, mas não foi o que fez.


ParseError parsing empty valued xml doc

Question: There are lots of articles pertaining to parsing xml with elementtree. I've gone through a bunch of them and read through the docs but I can't come up with a solution that works for me. I'm trying to supplement info thats created by another app in a nfo file but i need to preserve the conventions in the file.
Here is an example of how the file is laid out
<title>
  <name>Test Name</name>
  <alt name />
  <file local="C:\file\file1.doc" type="word">http://filestore/file1.doc</file>
  <file local="" type="excel">http://filestore/file2.xls</file>
  <file local="C:\file\file3.xls" type="excel" />
  <file local="" type="ppt" />
</title>

Note: Elements are not closed properly e.g...
<alt name /> should be <alt name></alt name>

This is what I'm running...
import xml.etree.ElementTree as ET
tree = ET.parse('file.nfo')
root = tree.getroot()

The error I'm getting is...

xml.etree.ElementTree.ParseError: not well-formed (invalid token):

I've tried...
myparser = ET.XMLParser(encoding='UTF-8')
tree = ET.parse('file.nfo', myparser)

Also tried, xmlparser, opening with codecs but i'm pretty sure its the formatting. I'm guessing the immediate issue is non-escaped > but i suspect ET needs opening/closing?
I'm sure i could open this file and go through it with regex but i was hoping to use ElementTree.
The end goal is to have the details from the nfo as a dictionary that looks like...
dict = {'title': [{'name': 'Test Name',
    'alt name': '',
    'file': [{'local': 'C:\file\file1.doc', 'type': 'word', 'url': 'http://filestore/file1.doc'},
        {'local': '', 'type': 'excel', 'url': 'http://filestore/file2.xls'},
        {'local': 'C:\file\file3.xls', 'type': 'excel', 'url': ''},
        {'local': '', 'type': 'ppt', 'url': ''}]
    }]}

I'm sure there is a better (more pythonic) way to do this but I'm pretty new to python.
Any help would be appreciated
EDIT: I'm also trying to avoid using 3rd party libraries if possible
Comment: The error message is not surprising since the sample document is not well-formed. The problem is the `` bit. An XML element simply cannot look like that. If it is changed to `` (for example), your code would work.
Comment: Note that changing from `` to `` would not help. It would still be ill-formed.
Comment: Thanks for your response, problem is i need to maintain the poor formatting so that the existing app that creates/uses it continues to work. Using  works flawlessly (with or without value) in my tests.
Comment: The "poor formatting" means that the document is not XML. That in turn means that you cannot expect to process it with XML tools. And I don't know what tests you do, but `` is still bad.
Comment: Thanks for the input, i **don't** disagree with you and i don't know enough about it to comment further. My hands are kind of tied is usage though. What i can say however is that the ElementTree.write() method formats it this way
Comment: I thought that the problem was about **parsing** XML. There is nothing about **writing** (using `ElementTree.write()`) in the question. Sorry, but I'm not sure what else I can say. It is too bad that your hands are tied.
Comment: Btw, what exactly is "info thats created by another app in a nfo file"?
Comment: I'm also re-writing the nfo file with additional information that i'm gleaning from outside of the app, that part i'm not having a problem with. So i didn't include it in my post
Answer: So I ended up creating a customer parser of sorts, its not ideal but it works. It was suggested to me that lxml and html.parser may parse malformed xml better but i just went with this.
I'm also still very interested in any feedback whether it be on this or using any other method.
import re

def merge_dicts(*dict_args):
    result = {}
    for dictionary in dict_args:
        result.update(dictionary)
    return result

def make_dict(str_arg, op):
    result = {}
    result = dict(s.split(op) for s in str_arg.split(","))
    return result

'''
Samples
lst = r'  <name>Test Name</name>'
lst = r'  <alt name />'
lst = r'  <file local="C:\file\file1.doc" type="word">http://filestore/file1.doc</file>'
lst = r'  <file local="" type="excel">http://filestore/file2.xls</file>'
lst = r'  <file local="C:\file\file3.xls" type="excel" />'
lst = r'  <file local="" type="ppt" />'
'''
def match_pattern(file_str):
    #<description>desc blah</description>'
    pattern1 = r'''(?x)
    ^
    \s*                                             # cut leading whitespace
    (?P<whole_thing>
        < (?P<tag_open> (\w+?|\w*\s\w+?)+) \b       # word boundary, so we can
        >                                           # skip attributes
        (?P<tag_body> .+? )                         # insides
        </ (?P<tag_close> (\w+?|\w*\s\w+?)+) >      # closing tag, nothing interesting
    )
    $'''

    #<alt name />
    pattern2 = r'''(?x)
    ^
    \s*
    (?P<whole_thing>
        < (?P<tag_open> (\w+?|\w*\s\w+?)+) \b
        \s/>
    )
    $'''

    #<file local="C:\file\file1.doc" type="word">http://filestore/file1.doc</file>'
    pattern3 = r'''(?x)
        ^
        \s*
        (?P<whole_thing>
            < (?P<tag_open> (\w+?|\w*\s\w+!=?)+) \b
            \s
            (?P<tag_attrib1> (\w*\=.*?))            # 1st attribute
            \s
            (?P<tag_attrib2> (\w*\=.*))             # 2nd attribute
            .*? >
            (?P<tag_body> .+? )
            </ (?P<tag_close> (\w+?|\w*\s\w+?)+) >
        )
        $'''

    #<file local="" type="ppt" />
    pattern4 = r'''(?x)
        ^
        \s*
        (?P<whole_thing>
            < (?P<tag_open> (\w+?|\w*\s\w+!=?)+) \b
            \s
            (?P<tag_attrib1> (\w*\=.*?))            # 1st attribute
            \s
            (?P<tag_attrib2> (\w*\=.*))             # 2nd attribute
            \s/>
        )
        $'''

    pat_str = 'pattern'
    pat_val = 1
    return_dict = {}
    while (pat_val <= 4):
        pattern = pat_str+str(pat_val)
        matchObj = re.match(eval(pattern), file_str, re.L|re.M)

        if matchObj:
            #for k, v in matchObj.groupdict().items():
            #    print('matchObj.group({!r}) == {!r}'.format(k, v))
            if pat_val == 1:
                body = matchObj.group('tag_body')
                return_dict = {matchObj.group('tag_open'): body}
            elif pat_val == 2:
                return_dict = {matchObj.group('tag_open'): ''}
            elif pat_val == 3:
                attr1 = make_dict(matchObj.group('tag_attrib1'), '=')
                attr2 = make_dict(matchObj.group('tag_attrib2'), '=')
                body = {'url': matchObj.group('tag_body')}
                attrib = merge_dicts(attr1, attr2, body)
                return_dict = {matchObj.group('tag_open'): attrib}
            elif pat_val == 4:
                attr1 = make_dict(matchObj.group('tag_attrib1'), '=')
                attr2 = make_dict(matchObj.group('tag_attrib2'), '=')
                body = {'url': ''}
                attrib = merge_dicts(attr1, attr2, body)
                return_dict = {matchObj.group('tag_open'): attrib}
            return return_dict
        else:
            pat_val = pat_val + 1
            if pat_val > 4:
                print("No match!!")

#print(match_pattern(lst))

def in_file(file):
    result = {}
    with open(file, "r") as file:
        data = (file.read().splitlines())
        for d in data:
            if data.index(d) == 0 or data.index(d) == len(data)-1:
                if data.index(d) == 0:
                    print(re.sub('<|/|>', '', d))
            elif d:
                lst = []
                dct = {}
                if 'file' in match_pattern(d).keys():
                    for i in match_pattern(d).items():
                        if 'file' in result.keys():
                            lst = result['file']
                            lst.append(i[1])
                            dct = {i[0]: lst}
                            result = merge_dicts(result, dct)
                            #print(result['file'])
                        else:
                            dct = {i[0]: [i[1]]}
                            result = merge_dicts(result, dct)
                else:
                    result = merge_dicts(result, match_pattern(d))
                    print('else', match_pattern(d))
    return result

print(in_file('C:\\test.nfo'))

NOTE: I dropped the top most dictionary from the original post

ParseError parsing empty valued xml doc


What are the rules of Third Stream composition?

Question: In 1961, Gunther Schuller defined Third Stream as "a new genre of music located about halfway between jazz and classical music".
Schuller stated what Third Stream music is not

It is not jazz with strings.
It is not jazz played on "classical"
instruments.
It is not classical music played by jazz players.
It is not inserting a bit of Ravel or Schoenberg between be-bop changes—nor
the reverse.
It is not jazz in fugal form. It is not a fugue played
by jazz players.
It is not designed to do away with jazz or classical
music; it is just another option amongst many for today's creative
musicians.

An example of Schuller's vision of Third Stream can be found on the album "Abstractions" composed by Schuller and Jim Hall.
Saying what something is not does not define what it is.
What are the rules of Third Stream composition?
Comment: I'm not sure we can fully define Third Stream music and exactly where between classical music and jazz it lies. The Wikipedia article you linked to says that improvisation is an important part of Third Stream music, then it mentions Nikolai Kapustin as a Third Stream composer...who eschews improvisation pretty much entirely.
Comment: "Saying what something is not does not define what it is" -- structuralists might disagree with you on this point
Answer: Schuller also said: "by designating the music as a 'separate, Third Stream', the other two mainstreams could go about their way unaffected by the attempts at fusion".    'Third Stream' gives classicists permission to use jazz elements, it gives jazzers permission to use classical elements.
As stated, this freedom has been used in various ways, both with and without improvisation.  'Third Stream' has no rules or charter.  It's just a label.  Stick it where you choose - on music that you might otherwise be tempted to criticise as not being 'pure jazz' or not being 'pure classical'.


Can't call method "getFirstName" on an undefined value at ConsManagement.pl

Question: Boa tarde.
Comecei recentemente a aprender Perl.
Estava a praticar com um programa simples orientado a objetos.
Esta é a minha classe
<code> package Consultant;

sub new {
    my $class = shift;
    my $self = {
        _firstName => shift,
        _lastName=> shift,
        _age=>shift,
        #_dataofBirth=>shift,
        _country=>shift,
        _salary=>shift,
        _manager=>shift,
        _contract=> shift,
        _experience=> shift,
    };

    print "Created a Consultant\n";
    bless $self, $class;
    return $self;

}

sub getFirstName {
    my( $self ) = @_;
    return $self->{_firstName};
}

sub getLastName {
     my( $self ) = @_;
    return $self->{_lastName};
}

sub getAge {
   my( $self ) = @_;
   return $self->{_age};
}

sub getCountry {
   my( $self ) = @_;
   return $self->{_country};
}

sub getContract {
   my( $self ) = @_;
   return $self->{_contract};
}

sub getSalary {
   my( $self ) = @_;
   return $self->{_salary};
}

sub getManager {
   my( $self ) = @_;
   return $self->{_manager};
}

sub getExperience {
   my( $self ) = @_;
   return $self->{_experience};
}

sub setFirstName {
   my ( $self, $firstName ) = @_;
   $self->{_firstName} = $firstName if defined($firstName);
   return $self->{_firstName};
}

sub setLastName {
   my ( $self, $lastName ) = @_;
   $self->{_lastName} = $lastName if defined($lastName);
   return $self->{_lastName};
}

sub setAge {
   my ( $self, $age ) = @_;
   $self->{_age} = $age if defined($age);
   return $self->{_age};
}

sub setCountry {
   my ( $self, $country ) = @_;
   $self->{_country} = $country if defined($country);
   return $self->{_country};
}

sub setSalary {
   my ( $self, $salary ) = @_;
   $self->{_salary} = $salary if defined($salary);
   return $self->{_salary};
}

sub setManager {
   my ( $self, $manager ) = @_;
   $self->{_manager} = $manager if defined($manager);
   return $self->{_manager};
}

sub setContract {
   my ( $self, $contract ) = @_;
   $self->{_contract} = $contract if defined($contract);
   return $self->{_contract};
}

sub setExperience {
   my ( $self, $experience ) = @_;
   $self->{_experience} = $experience if defined($experience);
   return $self->{_experience};
}
</code>
Ao tentar eliminar membros de um array com o ciclo
<code>case "3" {
                print "Deseja eliminar:\n";
                print "1-Consultor\n";
                print "2-Funcionário\n";
                my $opcao3=<STDIN>;
                chomp($opcao3);
                switch($opcao3) {
                        case "1" {
                        print "Primeiro Nome:\n";
                        my $firstNameDespedir=<STDIN>;
                        chomp($firstNameDespedir);
                        print "Ultimo Nome:\n";
                        my $lastNameDespedir=<STDIN>;
                        chomp($lastNameDespedir);
                        for my $i (0 .. $#consultores) {
                        my $object=$consultores[$i];
                        my $firstName1= $object->getFirstName();
                        my $lastName1= $object->getLastName();

                        if(($lastName1 eq $lastNameDespedir) && ($firstName1 eq $firstNameDespedir)) {
                        splice @consultores, $i, 1;
                        }
                        else {
                        print "Continua\n";
                                }

                            }
                        }
                    }
                }
</code>
O erro so ocorre se tentar eliminar um membro do array que não seja o primeiro.
Answer: Está resolvido.
O que estava a acontecer era que o iterador detectava prematuramente uma igualdade e eliminava, após isso, nao era possivel chamar o metodo.
Resolveu-se adicionando um last; dentro do if de igualdade de strings.


jQuery .load() post issue

Question: This code is working perfectly:
<code>$(document).ready(function() {
  $("#submit").click(function(){
    $("#output").load("extract.php", {query: $("#input").val()} );
  });
});
</code>
However, when I change the link from extract.php to the same exact file on an external server, the script stops working:
<code>$(document).ready(function() {
  $("#submit").click(function(){
    $("#output").load("http://example.com/extract.php", {query: $("#input").val()} );
  });
});
</code>
According to firebug, the latter script is posting to the external file, but there is no response for some reason. Any thoughts? Greatly appreciate it.
Comment: You cannot sent POST requests to an external server, it's called the [Same Origin Policy](http://en.wikipedia.org/wiki/Same_origin_policy).
Comment: ajax same origin policy - duplicate of a million other questions
Answer: To quote @Rocket:

You cannot sent POST requests to an external server, it's called the Same Origin Policy.

What you can, however, do, is use some server-side code (such as PHP) hosted locally to make that POST request for you. Here is one such post describing one way to do it: jQuery AJAX Proxy (with POST data) in PHP

jQuery .load() post issue


What would prevent the NFL from expanding to a 19 game season?

Question: There has long been team discomfort in Thursday night and international games because teams have a reduced rest period.  A 2 bye-weeks schedule was attempted in 1993, but was dropped, it seemed in large part due to the excessive disruption within each team's schedule.  Thoughts of expanding to an 18 game schedule (likely removing two preseason games) were hinted at by Roger Goodell during the 2016 season, but were tabled for the time being.  The big barrier was likely player agreement given the increase in games played and wear-and-tear on the bodies, but an additional issue was that the scheduling would become imbalanced relative to the symmetric format used now.
However, if the NFL instead added 4 extra weeks to the season made up of 3 games and a second bye week, it would

Allow teams to maintain a symmetric schedule (either by adding a second intra-conference division or matchups with equal position teams in the opposite conference)
Increase the overall game-to-rest ratio
Allow the NFL to schedule bye weeks for teams the week before playing in Thursday night games, as well as adjacent to international games.
Allow weeks of the preseason to be eaten up, and also expand the season further into one of the seemingly-ripest lull periods (particularly in the less bustling cool season) in American sports, January and February.

Are there any complications, other than the player concerns of adding more games, that would hinder this scheduling change option?
Comment: @Nij By "international games," OP is referring to games that are played outside the U.S., like the Cardinals/Rams game last weekend in London.
Comment: @Nij  I can understand where that might be confusing.  Unfortunately even the NFL itself [often uses the phrase](http://www.nfl.com/international) ("the NFL will play five international games in 2017.")  Note that [true International games also may or may not on the near horizon](https://www.si.com/nfl/2017/09/21/nfl-london-team-international-series-europe-football), but indeed, apologies for the term, but just following the general usage unfortunately :-/
Comment: As the NFL has teams only within the USA, mentioning international games does not seem to have any relevance, since just one country is involved.
Comment: Then the question should say "games played outside the USA" instead of a phrase that means something different to the majority of sports and sports fans.
Comment: Idiosyncratic usage isn't general usage, though, that's the entire point.
Answer: At the moment (2017) the agreement of the NFL Player's Association (NFLPA) is the only roadblock that would preclude changes in the structure of the season.
The reason is that the union negotiated the limits to the regular season in the last Collective Bargaining Agreement (CBA). Article 31 of the 2011-2020 NFL/NFLPA CBA states that (link):

ARTICLE 31 ADDITIONAL REGULAR SEASON GAMES
The  League  and/or  Clubs  may  increase  the  number  of  regular
  season  games  per  Team above the standard of sixteen (16) only with
  NFLPA approval, which may be withheld at the NFLPA's sole discretion.

Therefore, the change to 18 regular season games scheduled above can't happen unless the NFL negotiates with the NFLPA and gains the union's agreement, as stipulated in the CBA.
The players and the union might agree to a longer regular season schedule only if they gain additional concessions from the league (e.g., more money, better pensions, etc.). Most likely, any changes to the schedule will come with changes to the next CBA (see this article as one example):

With 256 current regular-season contests drawing fewer viewers, one
  very simple way to pad the total numbers will be to add another 32
  games, which would carry with it two more weekends of prime-time
  action. Even if there are fewer total viewers or the same number of
  viewers watching less football (I still can't figure out which one is
  worse), giving them more football increases the chances that any given
  prime-time game will be more compelling, more competitive, more
  exciting, and generally less Jaguars-Titany.
Ultimately, it won't happen unless the union agrees. Although the
  official position for years has been one of total disinterest and at
  times outright refusal, the NFLPA realizes that everything is
  negotiable. With, for example, larger rosters, a bigger piece of the
  pie to pay all players, and maybe a few other concessions (neutral
  arbitration, anyone?), the players could be willing to accept the
  terms.
Comment: "the NFLPA negotiates with the NFLPA" should probably be "the NFL negotiates with the NFLPA"?
Comment: @RolazaroAzeveires yep, looks that way.  Looked pretty clear, so I went ahead and changed it as it was pretty clear.  Don't be afraid to make such changes yourself if you're confident.  It takes some getting used to, but it's what we're really supposed to be about, especially on the science side of SA... finding the best answers, not getting the most points :-)  I'd always welcome any real improvements or error corrections to my answers!
Comment: @RolazaroAzeveires, thanks for catching that typo!
Comment: @JeopardyTempest sure, I have no problem editing. But it was not a serious mistake plus the reply was very recent, so JW8 might still want to add something more.


Exploratory Factor Analysis (EFA) followed by Confirmatory Factor Analysis (CFA)

Question: How meaningful or informative is it to follow an Exploratory Factor Analysis with Confirmatory Factor Analysis that uses exactly the same indicators and factors?
My goal is to use various CFA methods (such as ML, Satorra-Bentler, Yuan-Bentler, Bollen-Stine, WLS and DWL) to demonstrate in detail how well the model specified by the EFA fits the data.
Do you think this is a good idea? Or is it unnecessary?
Please advise.
Thanks.
Answer: An EFA has all indicators load on all factors and will always fit the data better than a CFA which constrains some loadings to 0. CFA is thus a stricter test and if it fits well enough, will provide stronger evidence than EFA.
It is common to use EFA to select an appropriate number of factors and to identify the pattern of loadings, followed by CFA with the same number of factors and with all the EFA loadings under some threshold (e.g., 0.3 or 0.4) constrained to 0. This CFA is sometimes done on the same data as the EFA and is sometimes done on independent data. It is more rigorous to use independent data and this helps to attenuate "overfitting" but there is still some value in using the same data as this will show how robust the model is to the aforementioned constraints.


Non-Movable C++17 Unique Pointer

Question: I came across this answer Prevent moving of a unique_ptr C++11. However while trying it out on a compiler online, this works with C++11(std::move compiler error) but with C++17, I'm seeing that the std::move below is successful. Shouldn't the compiler throw an error on that line? Also if some semantics have changed in C++17, what is the correct way to create a non movable unique_ptr in C++17 and onward.
template <typename T>
using scoped_ptr = const std::unique_ptr<T>;

int main()
{
    auto p = scoped_ptr<int>(new int(5));
    auto p2 = std::move(p); // should be error?
    std::cout << *p2 << std::endl; // 5
    return 0;
}

You can try it online here.
Answer: p is not const.  See here for it to fail the way you expect.
auto deduces like a template<class T>void foo(T) does.  T is never deduced as const, and neither is auto p=.
Meanwhile, the auto p = line works because you compiled it in c++17 mode.  In c++11 it does not compile.  This is because how prvalues differ in 17; some call the difference guaranteed elision.
If you want an immobile unique ptr:
template<class T, class D>
struct immobile_ptr:private std::unique_ptr<T, D>{
  using unique_ptr<T>::operator*;
  using unique_ptr<T>::operator->;
  using unique_ptr<T>::get;
  using unique_ptr<T>::operator bool;
  // etc

  // manually forward some ctors, as using grabs some move ctors in this case
};
template<class T, class...Args>
immobile_ptr<T> make_immobile_ptr(Args&&...args); // todo

an alternative might be to take a unique ptr with an immobile destroyer.
template<class X>
struct nomove_destroy:std::destroy<T>{
  nomove_destroy(nomove_destroy&&)=delete;
  nomove_destroy()=default;
  nomove_destroy& operator=(nomove_destroy&&)=delete;
};
template<class T>
using nomove_ptr=std::unique_ptr<T,nomove_destroy<T>>;

But I am uncertain if that will work.
Comment: If I read [overload #6](https://en.cppreference.com/w/cpp/memory/unique_ptr/unique_ptr) correctly, the deleter trick should work. Neat!
Answer: Note that p is declared as non-reference type, the const part of the argument scoped_ptr<int>(new int(5)) is ignored in type deduction. Then the type deduction result for p is std::unique_ptr<int>, not const std::unique_ptr<int> (i.e. scoped_ptr<int> as you expected).
What you want might be
auto& p = scoped_ptr<int>(new int(5)); // p is of type const std::unique_ptr<int>& now

Comment: `auto&` doesn't make it of non-reference type `const std::unique_ptr`, does it?
Comment: @ShadowRanger You mean `const std::unique_ptr&` ?
Comment: Works  across versions this way https://coliru.stacked-crooked.com/a/aef0f48df5604e38
Answer: Welcome to the world of type deduction in C++. Try
auto & p = scoped_ptr<int>(new int(5));

or
auto && p = scoped_ptr<int>(new int(5));

instead. This lecture may be helpful: https://www.youtube.com/watch?v=wQxj20X-tIU
Comment: Thanks for the suggested lecture - I think I need to watch this and read the relevant material from Modern Effective C++ again :)

Non-Movable C++17 Unique Pointer


proving $\lim\limits_{n\to\infty} \int_{0}^{1} f(x^n)dx = f(0)$ when f is continous on [0,1]

Question: $$\lim\limits_{n\to\infty} \int_{0}^{1} f(x^n)dx = f(0)$$
when f is continuous on $[0,1]$
I know it can be proved using bounded convergence theorem but,
 I wanna know proof using only basic properties of riemann integral and fundamental theorem of calculus and MVT for integrals ...
 Thank you.
Comment: $\int_{0}^{1-\xi}f(x^n)dx+\int_{1-\xi}^1f(x^n)dx$
Comment: Actually i did it. and using MVT for integral and proved first term of equation be f(0) but then I coudln't dealt with limits and etc. So could you gimme some more hints?
Comment: I think that's enough.Can you explain your problems in this method?
Answer: take any $\epsilon$, choose $\delta > 0$ such that $|f(x) - f(0)| < \epsilon$ on $[0,\delta]$. choose $n$ big enough such that $(1-\epsilon)^n < \delta$ then
$$ |\int_0^1 f(x^n) dx - f(0) |=  |\int_0^{(1-\epsilon)} [f(x^n) - f(0)]|dx + \int_{1-\epsilon}^1 [f(x^n) - f(0) ]dx |  \leq $$
$$ \int_0^{(1-\epsilon)} |f(x^n) - f(0)|dx + \int_{1-\epsilon}^1 |f(x^n) - f(0) |dx $$
first factor is smaller than $\epsilon(1 - \epsilon)$ thanks to the choice of $\delta$ and $n$, second one is smaller than $\epsilon \cdot 2 \sup |f|$ because length of your interval of integration is $\epsilon$ so the result follows since $\epsilon$ was arbitrarily small
Answer: Hint:
$f(0)=\int_0^1 f(0) dx$
$x^n\to 0$ for $0\leq x<1$
and $f$ is continuous.
Comment: Technically you need that $f$ is uniformly continuous to pull the limit inside the integral (which is itself a limit).  In this case, that is guaranteed by continuity of $f$ on $[0,1]$ and compactness of $[0,1]$.  A detail which might be okay to omit for a student far enough along to know the dominated convergence theorem, but I feel it is worth noting it for those who do not.
Comment: @zibadawatimmy that's what i meant:)
Answer: Handsome tip:
Prove it for $g\left(x\right)=f\left(x\right)-f\left(0\right)$ where
$g\left(0\right)=0$ and then make use of $\int_{0}^{1}f\left(x^{n}\right)dx=\int_{0}^{1}g\left(x^{n}\right)+f\left(0\right)dx=\int_{0}^{1}g\left(x^{n}\right)dx+f\left(0\right)$.


Pandas Oracle query gives "ORA-00911: invalid character"

Question: Hello I am trying to execute the following Oracle Query. I confirmed i can successfully connect to the database using cx_Oracle but my query is not executing. This is a large table and i am trying to limit the number of rows to 10
query1 = """
select *
  from
(select *
   from some_table
  )
where rownum < 10;
"""

df_ora1 = pd.read_sql(query1, con=connection1)

I am getting the following error but cant figure out what the invalid character is!
DatabaseError: ORA-00911: invalid character
Please help!
Answer: Remove the semi-colon from the SQL statement.  Semi-colons are not part of SQL.
query1 = """ select * from (select * from some_table ) where rownum < 10 """


Figured bass question: When to use #6 in minor key?

Question: So I'm transcribing a piece by Giuseppe Valentini, one of his Allettamenti da Camera (1714), but my figured bass is a bit rusty. Here's an excerpt that illustrates my question:

So in the end of first full bar, it's pretty clear to me that the x over the G# indicates to play the #3 (B#), but what about the 5th? "The rule" (which can be found in Aldwell and Schachter, for example) is that the figures follows the key - however, in this case the melody suggests D# instead of the D-natural of the key. That chord is then forming the V of V (G# into C#- in the third measure), and maybe the next bar is just avoiding/extending that cadence or something.
But, that next bar does confuse me a little. The 5 6-slash over D# should be D# F# A B# (the 3 and 5 of the key with the raised 6), going to C# E G#, and then....I want to say the last two eighths are just outlining a G# major chord, but I just don't see it in the figure.
What I see is 7 over B#, which is B# D F# A (no modifications to the figure from key signature), and then X over G# is G# B# D (raised third, 5th in key). I guess if I was going to arrange that carefully, I would make do D B# resolving outwards to octave C#, but still...a much simpler solution would be to imply the D# here, making it another V of V.
So I guess what I'm asking about is the rule "the figure keeps the key signature" in minor keys, when the 6th note of the scale could be raised. How do you know when to raise it? What would they have done in 1714?
Answer: It's reasonably clear that what's happening harmonically is that a first inversion B# diminished seventh chord is moving through a passing-tone C# in the pass to a root position B# diminished seventh chord.
An experienced continuo player, seeing a 6-5 indication of a B# diminished chord, followed by a 7 over a B#, would recognize the change in inversion, so not need an explicit figure to sharp the D.
Comment: @cduston Yes, I understanding the final G# as the root of a V7 chord leading to C# minor in the next bar.
Comment: So you're saying B# D# F# A over the B# (makes sense with the A in the melody), but what about the G#, last note in the bar? Make that G# under B# D#, turns into a 7th chord with that descending line in the melody? The accompaniment could hold the F# too.


LESSCSS: interpolation of a mixin argument

Question: I have the following code:
@red_SOPRA: orange;
@red: red;
.gradient(@color) when (@color = red)
{
  background: linear-gradient( 180deg, @color 0%,yellow 100%);
}

div
{
  .gradient(red);
}

Compiled into:
div {
  background: linear-gradient(180deg, #ff0000 0%, #ffff00 100%);
}

I would like to add "_SOPRA" to the end of variable "@color" present in background definition, in order to obtain an interpolated and dynamic name of variable used in mixin.
I tried with @@ and @{color} definitions but without success.
How to obtain a generated background like this (with value "#ffa500" - @red_SOPRA value - instead of "#ff0000")?
div {
  background: linear-gradient(180deg, #ffa500 0%, #ffff00 100%);
}

Comment: wont this do: .gradient(@red_SOPRA); ?
Comment: Because I'll have also "@blu_SOPRA".... I need to have a "dynamic" name.... practically, color value used in gradient definition should be a "merging" between string passed through @color variable and "_SOPRA" one
Comment: In current Less versions the standard colour keywords (e.g. `red`) are *immediately* converted to their hex representation (e.g. `#ff0000`) so `"@{color}"` would actually be expanded into "#ff0000" (so instead of `red_SOPRA` you get `#ff0000_SOPRA`). To prevent this you have to use `red` with quotes (e.g. `"red"` or `~"red"` depending on where you use it).
Answer: Option #1
Unless I'm missing something from what you are trying to achieve, then simply removing the guard expression when (@color = red) will get a "dynamic" value output for @color, allowing it to be called directly by @red_SOPRA or @blu_SOPRA (whatever you pass).
LESS
@red_SOPRA: orange;
@red: red;
.gradient(@color)
{
  background: linear-gradient( 180deg, @color 0%, yellow 100%);
}

div {
  .gradient(@red_SOPRA);
}

CSS Output
As you wanted...
div {
  background: linear-gradient(180deg, #ffa500 0%, #ffff00 100%);
}

Option 2
You can set up the mixin to perform a "'merging' between string passed" and the "_SOPRA" suffix, something like this:
LESS
@red_SOPRA: orange;
@blu_SOPRA: blue;
.gradient(@pre; @SOPRA: ~'@{pre}_SOPRA'; @color: @@SOPRA)
{
  background: linear-gradient( 180deg, @color 0%,yellow 100%);
}

div
{
  .gradient('red');
}

.test {
  .gradient('blu');
}

CSS Output
div {
  background: linear-gradient(180deg, #ffa500 0%, #ffff00 100%);
}
.test {
  background: linear-gradient(180deg, #0000ff 0%, #ffff00 100%);
}

This will not fail gracefully if @color does not resolve to a valid @@SOPRA value.
Answer: First of all, I would like to thank ScottS for his answer, especially second option that is what I needed.
Secondly, I add solution that finally yesterday I found by myself, very similar in philosophy but little different and (maybe) easier to understand by newby, due to a temp variable.
@red_SOPRA: orange;
@red: red;

.gradient(@color) when (@color = 'red')
{
  @stop-1:~"@{color}_SOPRA";

  background: linear-gradient( 180deg, @@stop-1 0%,yellow 100%);
}

div
{
  .gradient('red');
}

Philosophy is the same, but with this "double passage" I think that could be more "talking"... :-)
UPDATE:
After correct comment of  seven-phases-max, I updaded my solution. Please note that parameter must be passed as string with 'red'
Comment: The only problem is that it's not working: `SyntaxError: variable @#ff0000_SOPRA is undefined in ...` (you need to use `'red'` instead of  `red` :). Btw., `@stop-1: ~"@{color}_SOPRA";` can be just `@stop-1: "@{color}_SOPRA";` (i.e. no need for `~` there).


Getting the time as file name

Question: I have a program that grabs a video using Direct-show. My problem is that I want the saved video to have the time as name. Here is what I 've done :
CoInitialize(NULL);
CComPtr<IGraphBuilder> graph;
graph.CoCreateInstance(CLSID_FilterGraph);
time_t rawtime;
struct tm * timeinfo;
time (&rawtime);
timeinfo = localtime (&rawtime);
wchar_t pathstuff[50];
swprintf(pathstuff, L"D:\\BMDvideos\\%hs.avi", asctime(timeinfo));
LPCOLESTR x = pathstuff;
HRESULT hr = BuilGraph(graph,x);
..........

This doesn't work, any idea how can I get this done?
Comment: @bash.d A more likely problem is that he's under Windows.  `asctime` uses colons in its formatting, and Windows doesn't allow colons in filenames (except for special purposes, like the initial device specifier or secondary streams).
Comment: Have you checked, if your `timeinfo*` contains valid data?
Comment: `asctime` has spaces and a newline in its output. This could be one of the problems.
Comment: You're trying to solve three problems - getting the time, formatting a filename, writing a file. Break it down to make sure each works independently.
Comment: In addition, I don't think Engine is operating with LPCOLESTRs (BSTRs) correctly.
Comment: @OlafDietsche One might think so.  However: I'm not sure about the newline, but Windows definitely accepts spaces.  (I'd avoid them, however, because they will confuse so many useful tools.)  On the other hand, `asctime` definitely does contain colons, which Windows _doesn't_ accept.
Answer: The function for formatting time is strftime, so you could do
something like:
char filename[1000];
size_t l = strftime( filename,
                     sizeof(filename),
                     "D:/BMDvideos/%Y-%m-%dT%Hh%Mm%S.avi",
                     timeinfo );
assert( l != 0 );

Several comments:

You do not want wide character filenames.  They're not
standard, and not widely supported.  (The universal standard for
Unicode filenames is UTF-8, which does work everywhere.  In
fact, you do not want wide characters anywhere outside your
program.)
You can't use any standard formatting functions, like
asctime, as they will almost certainly use colons (':') in
the time format, and colons are illegal characters in a filename
under Windows.  You must specify the exact format yourself,
using strftime.
There are probably a lot of other characters you should avoid
as well.  I tend to limit my filenames to alphanumerics,
underscore, hyphen and dot, and I'm not that sure concerning the
portability of hyphen, or a name with more than one dot.  If the
names are not for human consumption, I'd just use
"%Y%m%d%H%M%S", with no separators.
Finally, it's useful to use the ISO ordering (year, month,
day) for dates, as this means lexicographical sorting
corresponds to a sort on the dates.

EDIT:
Since you say that you must use wchar_t filenames (which is
a very bad policy): you can still use the above to generate the
filename, then convert it to std::wstring by simply using
std::wstring( filename, filename + strlen( filename ) ).
Comment: @Engine Why?  Windows will accept both on a local disk, but if you ever want to backup (perhaps over NFS), or do anything else which may result in the files being seen by a non-Windows system, you're going to be in trouble.  (This works both ways: I've had problems with filenames containing colons, generated under Linux, which caused no end of problems when the filesystem was mounted on a Windows box.)  Still, if you must: just copy the generated string to a `std::wstring` (using the two iterator constructor).  If the filename is pure ASCII, this will work.
Comment: thanks for your answer, it work but, I still need wide character filename
Answer: asctime adds '\n' character at the end of returned string which is not valid for a file name

The call ctime(t) is equivalent to asctime(localtime(t)).  It
  converts the calendar time t into a null-terminated  string  of  the
         form
          "Wed Jun 30 21:49:08 1993\n"

try using strftime isntead
char name[100]
strftime(name, 100, "%F_%R", timeinfo);

Comment: Do _not_ use `"%R"`.  This will insert colons, which aren't legal characters in a filename under Windows.
Answer: wchar_t * and LPCOLESTR aren't compatible types, at runtime. Have you read this BSTR manual? It'll tell you how to allocate your BSTRs correctly.

Getting the time as file name


Does anyone know what this musical symbol means?

Question: Here is an extract from the very first measures of Le Marteau sans maître from Pierre Boulez. My problem is, I can't figure out what these strange vertical lines mean. (And I'm not talking about the barlines ;) )

Does anyone have an idea?
Answer: That is likely a musical separator. Typically, they are used as a sort of punctuation, to illustrate phrasing or motivic structure and sequences. In some contexts they might be akin to phrase markings. It is doubtful the composer wrote them in himself (at least it is for anything before contemporary music) and could be an editor's addition.


Test all models possibles is a good manner to choose the "best" model?

Question: I have programmed a function in python to test all possibles linear regression models that I can do with 5 variables. I choose the "best" model in base its AIC and BIC. These models are heteroscedastic and with normal errors. But I would like to know if this procedure is valid to choose the best model.
Comment: What is your ultimate goal; inference or prediction?
Comment: Inference is my ultimate goal.
Comment: I think the best procedure for inference is to prespecify a hypothesis and evaluate that hypothesis from that data rather than do some sort of model selection.  If you think all 5 variables have an effect on the outcome, then you should include them all in the model.  Use your background knowledge on the problem.
Answer: Though I'm not sure, Wikipedia says that this is how AIC is used in practice.  I think it depends on if you're interested in doing inference, or if you want to do prediction.
Answer: Part of this depends upon what you mean by the word "best."
The various information criteria can be derived either from information theory or Bayesian theory.  Assuming your likelihood function is correct, the various information criteria are good approximations of the Bayesian posterior density under relatively flat prior densities unless the values are quite close.
Your statement regarding heteroskedasticity is not a problem as long as your likelihood function doesn't assume homoskedasticity.  If you would assume a normal distribution with a fixed variance, then it is possible that a given criterion could generate an incorrect solution.
The value of the information criterion will generally be proportional to the posterior probability found using a Bayesian method.  The largest difference is about how parameters are understood.  The information criterion is driven by the supremum of the likelihood of the parameter estimates, whereas the Bayesian posterior over the model space integrates out the uncertainty over the entire parameter space.  The Bayesian method removes the consequence of making a specific choice for a parameter.
In practice, most models are far apart in posterior probability from the most likely model.  It does happen that a few models will be close in probability.  I have had that happen.  I have had the case where one model had about 54% posterior probability; another had about a 46% posterior probability and the remaining seventy-six models had a total posterior probability of around 1/10,000 when their posterior probabilities were added together.
I did not have a reason to calculate an information criterion but had I, seventy-six would not be close to the best two, and the best two would be marginally different.  What I did instead was to exclude the worst 76 and used model averaging on the two remaining models.  Tests against the hypothesis of concern showed that the joint model was better than either apart.  As they did not share variables, the difference was substantial.  Interestingly, both models made theoretical sense, after the fact, but also made it clear something was missing in the models and that further research was needed.
There is an argument, that I do not find persuasive, that using the information criterion could be problematic using combinatoric solutions if none of the models is the true model in nature.  In essence, every choice is a misspecified model.
The problem with the argument is that you can never validate this state of nature unless someone else comes along and validates via a theoretical construction a model that is not in your set.  Nonetheless, you need to work with what you have today.
Two other potential issues could happen if there is a variable or set of variables that must be present for theoretical reasons, but removing them is optimal.  That can happen with some form of collinearity.  The solution is to force mandatory variable into every model.  It could also happen if you cannot fully capture causal variables and marks the difference between prediction and inference.
Imagine you were looking at readmission rates at a hospital.  You found the single best variable was having been in the ICU, for predictive purposes.  It would be life-threateningly dangerous to then ban people from going to intensive care.
Imagine the true best predictor for readmission was skipping doctor visits six months prior to the hospitalization and non-compliance with medical orders, but you couldn't capture or fully capture this.  Imagine a hospital where there are three readmissions per month, all of whom were admitted to the ICU.  When you ban admission to the ICU it goes up to eighteen readmissions per month.
The presence of the ICU as a variable may be the best single predictor, but it is not valuable for inferential purposes.  In that case, an information criterion could give a misleading result.
Logic is always your greatest protection against bad models.


Levi-Civita connection acting on tensor density

Question: Consider an arbitrary $d$-dimensional (pseudo)-Riemannian manifold $(\mathcal{M},g)$ with Levi-Civita connection and some covariant $2$-tensor density $T$. More precisely, we consider $T$ to be given by
$$T=T^{\prime}\otimes\mathrm{vol}_{g}\in\Gamma^{\infty}(T^{\ast}\mathcal{M}^{\otimes 2}\otimes{\bigwedge}^{d}T^{\ast}\mathcal{M})\cong\Gamma^{\infty}(T^{\ast}\mathcal{M}^{\otimes 2})\otimes_{C^{\infty}(\mathcal{M})}\Omega^{d}(\mathcal{M}),$$
where $\mathrm{vol}_{g}$ denotes the volume $d$-form of $(\mathcal{M},g)$ and $\Omega^{d}(\mathcal{M})$ denotes the space of top-degree differential forms on $\mathcal{M}$. In other words, we consider a density $T$ with tensor part $T^{\prime}$ and density part $\mathrm{vol}_{g}$.
Now, I try to figure out what $\nabla_{X}T$ is for some vector field $X$. Using the Leibniz rule, this can be written as
$$\nabla_{X}T=(\nabla_{X}T^{\prime})\otimes\mathrm{vol}_{g}+T^{\prime}\otimes\nabla_{X}\mathrm{vol}_{g}.$$
Maybe its trivial, but is there are general way to compute $\nabla_{X}\mathrm{vol}_{g}$? By definition of the Levi-Civita connection, $\nabla g=0$. Does this imply $\nabla_{X}\mathrm{vol}_{g}=0$? I am not sure, since there is the square root of $g$ appearing ing $\mathrm{vol}_{g}$...
Comment: I don't know who is Mr or Mrs Leipniz, but the differentiation rule is named after Lei**b**niz!
Answer: The Riemannian volume form of an oriented Riemann manifold is parallel: $\nabla \mathrm{vol}_g=0$.
Indeed, let $\{E_1,\ldots,E_n\}$ be a local positively oriented orthonormal frame.
For $j\in \{1,\ldots,n\}$, let $\theta^j = g(\cdot,E_j)$.
Then $\theta^1\wedge\cdots\wedge \theta^n$ is a $n$-form, and agrees with $\mathrm{vol}_g$ on $\{E_1,\ldots,E_n\}$: it follows that locally,
$$
\mathrm{vol}_g = \theta^1\wedge\cdots\wedge \theta^n.
$$
Therefore, for any $X$, locally,
$$
\label{star}
\nabla_X\mathrm{vol}_g = \sum_{j=1}^n \theta^1\wedge\cdots\wedge \nabla_X\theta^j \wedge \cdots \wedge \theta^n. \tag{$\star$}
$$
Fix $j\in \{1,\ldots,n\}$.
From $\|E_j\|^2 = 1$, one has $g(\nabla_XE_j,E_j)=0$, and then, $\nabla_XE_j \in \mathrm{span}\left\{ E_k \mid k\neq j\right\}$.
It follows that
$$
\nabla_X\theta^j = g(\cdot,\nabla_XE_j) \in \mathrm{span}\left\{\theta^k \mid k\neq j\right\}.
$$
Hence, any term in the sum of equation \eqref{star} is then zero, and $\nabla_X\mathrm{vol}_g = 0$.
Answer: Just for completeness, a proof in coordinates.
Let $vol_g=\sqrt{\det g}\,dx^1\wedge\cdots\wedge dx^n$, where $g$ denotes the Gram matrix $\langle\tfrac\partial{\partial x_i},\frac\partial{\partial x_j}\rangle$, and $n=$dimension of the Riemannian manifold.
Then, for any vector field $X$,
\begin{align*}
\nabla_X vol_g={}&X(\sqrt{\det g})\, dx^1\wedge\cdots\wedge dx^n
\\
&+\sqrt{\det g}\,(\nabla_X dx^1)\wedge\cdots\wedge dx^n
\\
&\cdots
\\
&+\sqrt{\det g}\,dx^1\wedge\cdots\wedge(\nabla_X dx^n).
\end{align*}
We have
$$
\nabla_X dx^i=\Gamma^i_{jk}d x^jX^k
$$
hence
$$
(\nabla_X dx^1)\wedge\cdots\wedge dx^n=\Gamma^1_{1k}X^kdx^1\wedge\cdots\wedge dx^n
\\
\cdots
\\
dx^1\wedge\cdots\wedge (\nabla_X dx^n)=\Gamma^n_{nk}X^kdx^1\wedge\cdots\wedge dx^n
$$
so it remains to check that
$$
\frac{X(\sqrt{\det g})}{\sqrt{\det g}}=\sum_{i=1}^n\Gamma^{i}_{ik}X^k.
$$
This follows from the following identity for Christoffel symbols of the Levi-Civita connection:
$$
\sum_{i=1}^n\Gamma^i_{ik}=\frac{\partial}{\partial x^k}\log\sqrt{\det g}.
$$
The latter can be proved as follows:
$$
\sum_{i=1}^n\Gamma^i_{ik}=\frac 12g^{il}\left(\frac{\partial g_{lk}}{\partial x^i}+\frac{\partial g_{li}}{\partial x^k}-\frac{\partial g_{ki}}{\partial x^l}\right)=\frac 12g^{il}\frac{\partial g_{li}}{\partial x^k}=\frac{\partial}{\partial x^k}\log\sqrt{\det g}.
$$


In sequent calculus, what's going on with sequents with multiple formulae in the succedent?

Question: The sequent proof systems I learned only allowed one formula on the right hand side of the sequent, and $\phi_1, \ldots, \phi_n \Rightarrow \psi$ (or ... $\vdash \psi$) is understood as saying that $\psi$ is a logical consequence of (or provable from) $\phi_1, \ldots, \phi_n$.
Now I'm seeing sequent calculus systems with right weakening:

$$\frac{\Gamma\vdash \Delta}{\Gamma\vdash \Delta,A} \text {RW}$$

So I guess $\Gamma\vdash \Delta,A$  has to be saying that at least one of $\Delta,A$ is provable from $\Gamma$ and not that both are. Why is this used?
I assume the notation has some payoff I'm missing, since it seems like sequences of wffs have different meanings based on whether they're in the antecedent or succedent.
Answer: The sequent calculus is based on the notation $\Gamma \Rightarrow  \Delta$ (or $\Gamma \vdash \Delta$), with $\Gamma, \Delta$ finite (possibly empty) sequences of formulas, called a sequent.
The intuitionistic sequent calculus is obtained with the restriction that $\Delta$ consists of at most one formula.
For the semantics for sequents, see Gaisi Takeuti, Proof Theory (2nd ed - 1987), page 9:

intuitively a sequent $\gamma_1, \ldots, \gamma_m \vdash \delta_1, \ldots, \delta_n$ means :

"if $\gamma_1 \land \ldots \land \gamma_m$, then $\delta_1 \lor \ldots \lor \delta_n$".

Then we have [page 41] :

A sequent $\Gamma \vdash \Delta$ is satisfied [in an interpretation $\mathfrak I$] if either some formula in $\Gamma$ is not satisfied by $\mathfrak I$, or some formula in $\Delta$ is satisfied by $\mathfrak I$. A sequent is valid it it satisfied in every interpretation.
Comment: @Hungry - yes; the original Gentzen's formulation of sequent calculus derive the intuitionistic version from the classical one only with that restriction. See Takeuti's book for a detailed exposition. There are now many "variants"; see S.Negri & J.von Plato, [Structural Proof Theory](https://books.google.it/books?id=ZvACGkn9138C&printsec=frontcover).
Comment: I thought that any sequent proof system was considered a sequent calculus, but now I'm thinking that's wrong and is the source of my confusion. I've used a classical sequent proof system that only allowed one formula on the right hand side, and an intuitionistic sequent proof system that also only allowed one formula on the right hand side but had different rules. Are you saying that you get an intuitionistic sequent calculus by *just* adding the restriction that $\Delta$ consists of at most one formula and not changing the rules beyond that?
Comment: An old post you made about Getzen sequent calculus rules being "upside down" tableaux rules was ENORMOUSLY helpful to me. Funny how tableaux rules seem so natural and Getzen rules seemed bewildering. When I returned to the book I learned tableaux from (Bostock's Intermediate Logic) I realized it actually explicitly goes through the conversion later in the book.

And nicely answers my question re: the benefits of allowing multiple formulae on the right—all the combos of rules for signs are complete for all sequents containing only those signs. Otherwise would need neg rules!


Join common columns from different tables and select first most filled

Question: I have three tables: A, B, and C.
Records in B and C contain columns data1 and data2 which nullable, and id_A.
I need to receive any for both data1 and data2 per record in A (all columns from A).
Table A (other columns omitted):
id
1
2
3
4
5

Table B:
id | id_A | data1 | data2
1    1      b11     null
2    1      null    b12
3    2      null    b22
4    5      b51     null

Table C:
id | id_A | data1 | data2
1    1      c11     c12
2    2      c21     null
3    3      c31     null
4    5      c51     null

Result:
id | data1 | data2
1    b11     b12
2    c21     b22
3    c31     null
4    null    null
5    b51     null

Answer: I think you want left join and aggregation:
select a.id, coalesce(max(b.data1), max(c.data1)) as data1,
       coalesce(max(b.data2), max(c.data2)) as data2
from a left join
     b
     on a.id = b.id_a left join
     c
     on a.id = c.id_a
group by a.id

Answer: you can use - COALESCE (val1, val2, val3...)
select a.id, coalesce(b.data1,c.data1), coalesce(b.data2,c.data2) from Aleft join B on .id=b.id_a left join C on a.id=c.id_a


Please recommend a complete guide/book on MVVM for silverlight

Question: Although there are many MVVM for silverlight demo app/article online, but all of them are very simple and not cover mvvm with depth and extension for LOB app.
I am looking for a complete MVVM book/demo in depth related to such as WCF Ria Services, Command, etc.
Answer: Take a look at Advanced MVVM by Josh Smith.
Answer: Advanced MVVM is a good book about basic MVVM concepts, but it does not address the issues for LOB apps.  Keep looking.
Answer: For beginners : Take a look at the following article on MVVM basics


OfficeJS Addin office-store validation fails only in office online

Question: I have a Manifest that points to home.html hosted in some web server. It also has X-Frame-Options as SAMEORIGIN. For security reasons I can't change this configuration. I believe due to this SAMEORIGIN configuration in word online addin fails to load.
It works fine though in Windows and MAC native version of word. It is a taskpane addin.
I don't have a requirement to support word online but seems to validate the addin it must work in word online also.
Any suggestion how can I load home.html in word online when X-FRAME-OPTIONS is set to SAMEORIGIN.

Here is how I refer home.html in manifest.xml
This server will always return x-frame-options: sameorigin with all responce header.
<code><Hosts>
    <Host Name="Document" />
</Hosts>
<DefaultSettings>
        <SourceLocation DefaultValue="https://<somedomain>.com//App/Home.html" />
</DefaultSettings>
</code>
Comment: can you please  provide a pointer to your manifest?
Answer: It seems that your browser just respects X-Frame-Options: sameorigin header and saying I'm not going to load your Task pane iframe because it is not from the same domain as office online application. You can minimize your security risk by making exception for this rule in a web proxy (if you are using one or install and configure one) for requests coming from your office online domain.
Comment: Thanks Dalibor, Perhaps it can work that way to some extent. There are two problems with that. First - after home page it will redirect to different page and domain also. Practically it is impossible add office online domain to all those for security reasons. Second - as per this [stackoverflow thread](https://stackoverflow.com/questions/48334839/x-frame-options-for-outlook-web-add-ins) allow-from is not preferable way of configuring office online domains.


A successful connect to Oracle using Hibernate is 3 minutes

Question: Friends tell me the reason for why it takes three minutes to connect using Hibernate.
In the method of obtaining the factory:
sessionFactory = new Configuration().configure().buildSessionFactory();

hibernate.cfg.xml
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE hibernate-configuration PUBLIC
        "-//Hibernate/Hibernate Configuration DTD 3.0//EN"
        "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
    <session-factory>
        <property name="hibernate.connection.driver_class">oracle.jdbc.driver.OracleDriver</property>
        <property name="hibernate.connection.url">jdbc:oracle:thin:@<IP>:1521:sbboldev</property>
        <property name="hibernate.connection.username">login</property>
        <property name="hibernate.connection.password">password</property>
        <property name="hibernate.dialect">org.hibernate.dialect.Oracle10gDialect</property>
        <property name="hibernate.default_schema">login</property>
        <property name="show_sql">true</property>

        <mapping resource="Developer.hbm.xml"/>

    </session-factory>
</hibernate-configuration>

database_module.log
2019-02-08 12:06:08 DEBUG JdbcEnvironmentInitiator:72 - Database ->
       name : Oracle
    version : Oracle Database 11g Enterprise Edition Release 184.108.40.206.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
      major : 11
      minor : 2
2019-02-08 12:06:08 DEBUG JdbcEnvironmentInitiator:83 - Driver ->
       name : Oracle JDBC driver
    version : 220.127.116.11.0
      major : 11
      minor : 2
2019-02-08 12:06:08 DEBUG JdbcEnvironmentInitiator:389 - JDBC version : 11.2
2019-02-08 12:06:08 INFO  Dialect:158 - HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
2019-02-08 **12:06:08** DEBUG DefaultSchemaNameResolver:53 - Unable to use Java 1.7 Connection#getSchema
2019-02-08 **12:09:15** DEBUG Namespace:51 - Created database namespace [logicalName=Name{catalog=null, schema=null}, physicalName=Name{catalog=null, schema=null}]
2019-02-08 12:09:15 DEBUG ModelBinder:239 - Mapping class: net.proselyte.hibernate.example.Developer -> HIBERNATE_DEVELOPERS

Hibernate-core version: 5.4.1.Final.
Java version: 1.8.0_161.
Answer: Can you use the latest 18.3 JDBC driver ?
Comment: I tried - no positive result. Is it possible to slow down if the database contains 2000 tables?


When managing fields for a content type, fields of type "list (integer)" can't be shared?

Question: In the drop-down for "add existing field" not all fields are available for reuse. Specifically I would like to reuse a field of type "list (integer)" because I will be adding 300 of them to my content type with exactly the same settings (just different labels and machine names). Doing  the manually is going to take me over eight hours since it takes about 1.5 min to add a field and configure it's settings. Is it possible to reuse fields of type "list (integer)"?
Thanks!!
ps: if you are unsure what I mean by reusing a field see here: Documentation for Field-UI and look at the section titled, "Reusing Fields".
Comment: All fields normally can be shared between entity types.

If you would like to add many instances of one field to one content type you can use API field_create_field() & field_create_instance() functions.
Comment: Thank you, Oleg, for your reply. At the moment I have one content type that I am working with. I have added 40 fields of type "list (integer)" one-by-one. I agree with what you said, "all fields normally can be shared between entity types." I wonder why the field does not show up in the drop-down for "add existing field". Could perhaps fields of type "list (integer)" not be able to be shared?
Comment: Because I have already added 40 similar fields to the content type should I delete 39 of them and then use the function "field_create_instance()" to add 299 instances the field so that they are actually "shared"? Because the field has already been created through the GUI will "field_create_field()" still be needed?
Answer: Here's the answer to this question:
I cannot use the same field within one content type. Only one instance of a field may be used within each content type. This is why the field which I have created is not showing up in the drop-down for "add existing field" - it is not available.
Answer: As you say, you can only have one instance of a field in each content type. To share a list of options between multiple fields of a content type (say, a list of colours for 'background colour' and 'foreground colour' fields), I think your best bet would be to set up a separate content type ('Colour' in this example) and then create multiple entity reference fields that point to it.


In what sense does a NN model a distribution over outputs?

Question: In Goodfellow, Bengio, and Courville's book Deep Learning they state some variation of the following at several points (22.214.171.124, 10.2.3):

In general, we can think of the neural network as representing a function $f(x; \theta)$. The outputs of this function are not direct predictions of the value $y$. instead, $f(x; \theta)$ = $\omega$ provides the parameters for a distribution over $y$. Our loss function can then be interpreted as $-\log p(y;\omega(x))$. [p. 182]

I take this to mean that although a basic, standard neural network (i.e. feedforward with one output unit) outputs a single value, the training process and the suggested use of the maximum likelihood principle mean that inference is equivalent to sampling from some probability distribution whose structure is specified by the input and the network's parameters.
Am I interpreting this correctly? If so, what exactly is the relationship between the probability distribution and the network or the network's parameters? What is the relationship between $\omega$ and the network or the network's parameters?
Or are the distribution and parameters $\omega$ just hypotheticals and not meant to be interpreted so literally?
Comment: Check out this [Medium](https://medium.com/hal24k-techblog/a-guide-to-generating-probability-distributions-with-neural-networks-ffc4efacd6a4) article.
Answer: The specific part that you quote is dealing with the scenario when your network is not trained to predict the value of $y$ (which is the most common use-case of neural networks these days), but it predicts the parameters of the distribution of $y$. Let's say that we know that $p(y|x)$ is distributed normally, but we don't know parameters of the Gaussian. However, we know they depend on $x$ (if $x$ is age, maybe $y$ has higher variance in some age groups). So instead of letting $y=f(x,\theta)$, we let $\omega=f(x, \theta)$ and $y\sim \mathcal{N}(\omega)$.
As stated in the book, you can train such model using the maximum likelihood principle. The trained model gives you the parameters of the distribution that allow you evaluating $p(y)$.
To answer your question:

what exactly is the relationship between the probability distribution and the network or the network's parameters? What is the relationship between $\omega$ and the network or the network's parameters ($\theta$)?

Network parameters $\theta$ parametrize the network function $f(x; \theta)$. Output of this function, $f(x;\theta)=\omega$,  are the parameters of the distribution of $y$. In the example case I gave above, it is the mean and the variance of the normal distribution.


Should missing definition of object / function with non-reserved identifier lead to diagnostics?

Question: Sample code:
void accept(int x);

int main(void)
{
        accept(0);
        return 0;
}

Invocations:
$ gcc t719.c -std=c11 -pedantic -Wall -Wextra
<nothing>

$ clang t719.c -std=c11 -pedantic -Wall -Wextra
<nothing>

$ cl t719.c /std:c11 /Za
t719.obj : error LNK2019: unresolved external symbol accept referenced in function main

$ icc t719.c -std=c11 -pedantic -Wall -Wextra
<nothing>

Consider that the user forgot to define accept. We see that no diagnostics may be produced. Is diagnostics required?
UPD: Another example:
extern int y0;

int main(void)
{
        return y0;
}

# linux (begin)
$ gcc t719.c -std=c11 -pedantic -Wall -Wextra
undefined reference to `y0'

$ clang t719.c -std=c11 -pedantic -Wall -Wextra
undefined reference to `y0'

$ icc t719.c -std=c11 -pedantic -Wall -Wextra
<nothing>
# program returned: 243
# linux (end)

# windows (begin)
# gcc in cygwin
$ gcc t719.c -std=c11 -pedantic -Wall -Wextra
<nothing>
# program returned: 255

# clang in cygwin
$ clang t719.c -std=c11 -pedantic -Wall -Wextra
<nothing>
# program returned: 255

$ cl t719.c /std:c11 /Za
unresolved external symbol _y0 referenced in function _main

$ LLVM/12.0.0/bin/clang t719.c -std=c11 -pedantic -Wall -Wextra
<nothing>
# program returned: 72

$ icl -Qstd=c11 t719.c
<nothing>
# program returned: 65
# windows (end)

Comment: Nope — diagnostics are not required when the symbol is satisfied by the standard libraries linked by the compiler.  WIth GCC and Clang (on Unix-like systems) `accept()` is a function that accesses a system call.  The standard library provides it, so there's no need for a diagnostic.  Was `cl` running on Unix or Windows?  Is `cl` the Microsoft C compiler or something else?
Comment: change 'accept' to 'froodle' and see what happens, accept is the name of a unix/linux system call
Comment: `y0` is one of the Bessel functions in the maths library.
Comment: Umm, [`accept()` doggone well **is** reserved on a lot of systems](https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/functions/accept.html).
Comment: @AndrewHenle: The fact that system-provided libraries provide a function does not mean the identifier is reserved, particularly if no related headers are included.
Comment: @EricPostpischil [Here it does](https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html): "All functions and external identifiers defined in XBD Headers are reserved for use as identifiers with external linkage."  `accept()` is a reserved identifier on POSIX systems.
Comment: @JonathanLeffler Yes, `cl` is the Microsoft C compiler. It was running on Windows. Question: what `y0` is used for (see UPD)?
Comment: @AndrewHenle: Well, they tried. [1.1](https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap01.html#tag_14_01) indicates "alignment" to the C standard (ISO/IEC 9899:1999) and says "any conflict… is unintentional." Consider that a strictly conforming program may use `accept` for its own purposes, and a conforming implementation must accept and faithfully execute strictly conforming programs. So POSIX cannot reserve additional identifiers, other than those reserved for implementations, while conforming to the C standard.
Comment: @EricPostpischil Since the C standard does not have `accept`, the user expects that there are zero external definitions for the identifier `accept`. Hence, the user expects a diagnostics. Does it mean that the `-std=c11` is violated?
Comment: @pmor: The C standard does not require a diagnostic for this.
Comment: @EricPostpischil OK. Why "there shall be exactly one external definition for the identifier" is in Semantics, rather than in Constraints?
Comment: @EricPostpischil Meaning that an implementation is free to silently fail to resolve all external object and function references (phase 8).
Comment: @pmor: No, it is not free to resolve references. It is free not to diagnose missing definitions or multiple definitions.
Comment: @EricPostpischil Not to forget that "If a ''shall'' or ''shall not'' requirement that appears outside of a constraint or runtime constraint is violated, the behavior is undefined". If "multiple definitions" is OK (some policy may be used to select the definition), then what about "missing definitions"? How an implementation can behave after _not_ diagnosing missing definitions?
Answer: C 2018 §6.9 ¶5 says:

… If an identifier declared with external linkage is used in an expression (other than as part of the operand of a sizeof or _Alignof operator whose result is an integer constant), somewhere in the entire program there shall be exactly one external definition for the identifier;…

Since a "shall" is violated, but it is not in a constraint paragraph, C 2018 §4 ¶2 makes this undefined behavior:

If a "shall" or "shall not" requirement that appears outside of a constraint or runtime-constraint is violated, the behavior is undefined…

The clause on diagnostics, §18.104.22.168, does not require a diagnostic for this:

A conforming implementation shall produce at least one diagnostic message (identified in an implementation-defined manner) if a preprocessing translation unit or translation unit contains a violation of any syntax rule or constraint, even if the behavior is also explicitly specified as undefined or implementation-defined. Diagnostic messages need not be produced in other circumstances.


How to output a required number of significant digits for a number

Question: I have below JSON which I have to display on browser ...
{
  "testDoubleArr": [1, 2.0, 3.45],
  "testStr3": "SHA",
  "testDouble1": 12.56,
  "testDouble2": 1.0,
  "testStr1": "OMG"
}

On browser it get display in below format ...
{
  "testDoubleArr": [1, 2, 3.45],
  "testStr3": "SHA",
  "testDouble1": 12.56,
  "testDouble2": 1,
  "testStr1": "OMG"
}

It has removed all zeros if the value only contains zeros after the decimal, while I have to show them without converting it into string.
How do I specify the number of significant digits for JavaScript numbers?
Comment: They are the same thing - there is only one `Number` type in javascript.
Comment: _"I have to show them without converting it into string"_ And how do you expect to do that?
Comment: Also, if you are just trying parrot out the input string, you may be best off not parsing the JSON prior to the point that you are displaying this string.  If you never parse the string into memory, the string will remain intact for display.
Comment: I want to show them with zero means "testDouble2": 1.0  and not "testDouble2": "1.0"
Comment: @PrabhatNigam I think the point being made in other comments is that you are absolutely required to convert to string in some capacity in order to display to user. Currently you are setting the numerical values as Number types in javascript. When outputting these, you will drop insignificant digits unless you do something to specifically specify the string representation you are looking for.  If you are concerned about matching the exact input values without transforming the display (or performing any calculations on the numbers), then you should use string fields as input.
Answer: You can specify the number of significant digits to decimal.
var num = 1;
num.toFixed(2); // returns the string "1.00"

However, if the number has MORE significant digits, they will be dropped:
var num = 1.123456789;
var num.toFixed(2) // returns the string "1.12"

Comment: @ Chad, this solution is not valid for all the scenarios. First at all it will drop remaining numbers after point. And second thing it will convert the integer number to double number too. eg 1 will be shown as 1.00, which is not reasonable.
Comment: All JavaScript numbers are floating point. There is only one number type. Your question doesn't really explain all of your requirements.
Comment: My only requirement is, if value is 1 then it should show 1 not "1" , if value is 1.0 then it should show 1.0 not "1.0", while displaying it in json on UI.
Comment: I want json should be printed on UI as it works in http://jsonformatter.curiousconcept.com/#about, this tool prints JSON as it is irrespective of the type of value. for the sample json you can use below within that tool ...
Comment: {
  "data" : {
    "testDoubleArr" : [ 1.3 ],
    "testDouble2" : 0.0,
    "InterestRate" : 0.0,
    "testInt1" : 0,
    "testBool3" : true,
    "testInt2" : 0,
    "testBool2" : true,
    "testInt3" : 0,
    "testBool1" : true,
    "Month" : "UMG",
    "testStr3" : "SHA",
    "testStr2" : "HAMP",
    "testDouble3" : 0.0
  },
  "header" : {
    "modelName" : "Test Tenant Version",
    "majorVersion" : 1,
    "transactionId" : "2",
    "minorVersion" : 0,
    "date" : "2015-01-30T11:18:00.000"
  }
}


Logistic regression and success probability: Bayesian statistics

Question: I have a Bayesian statistics homework question that I'm not sure of:
Running this in R, I got completely flat lines at 0 and 1 so I was thinking this meant that the success probability does not change with this particular prior. Can anyone shed some more light on this problem?
mu0    = 0
sig2_0 = 100
n      = 10000
b0     = rnorm(n, mean=mu0, sd=sqrt(sig2_0))
x      = seq(from=0, to=10, length.out=100)
res    = matrix(0, length(x), 3)
for(i in 1:length(x)){
  z       = b0
  z1      = exp(z) / (1+exp(z))
  res[i,] = quantile(z1, c(0.025, 0.5, 0.975))
}

plot( x, res[,2], type='l', ylim=range(res), ylab='Success probability', xlab='Covariate')
lines(x, res[,1], lty=3)
lines(x, res[,3], lty=3)

Comment: I'm not sure I follow your comment about getting flat lines at 0 & 1 when you ran it in R. Can you add the figure to your question, or include your code?
Comment: sure @gung This is the code we were given to work with in class (I edited it to fit this particular problem...I hope it's right) `mu0=0
sig2_0=100

n=10000
b0=rnorm(n,mean=mu0,sd=sqrt(sig2_0))

x=seq(from=0,to=10,length.out=100)
res=matrix(0,length(x),3)
for (i in 1:length(x)){
 z=b0
 z1=exp(z)/(1+exp(z))
 res[i,]=quantile(z1,c(0.025,0.5,0.975))
}

plot(x,res[,2],type='l',ylim=range(res),ylab='Success probability',xlab='Covariate')
lines(x,res[,1],lty=3)
lines(x,res[,3],lty=3)`
Comment: I'm not sure I follow your code. What do you want it to accomplish?
Comment: This particular code was meant to "show how the success probability varies as a function of our covariate and, at the same time, show the 95% interval for this relationship." Honestly I'm not 100% sure this is all we were given to work with from the notes I took in class. I'm not even sure if you need R to solve this problem @gung
Comment: Note that questions about 'how do I do `_____` in R?', or 'what's wrong w/ my r code?' belong on [Stack Overflow](http://stackoverflow.com/), not here. Note further that our approach to HW questions is to provide hints *only* (see the `[self-study]` tag's [wiki](http://stats.stackexchange.com/tags/self-study/info)). Since I think there is a legitimate statistical question here, I provided hints below. There are also some issues w/ your code, however. 1st, your code doesn't include any covariates. 2nd you generate your pseudorandom data in line 4; each pass through the loop uses the same data.
Comment: Sorry I'm new to this, but this wasn't a question on whether my R code was right or not. I wasn't sure how to solve the problem, and I thought possibly this R code that my professor showed me in class might be the way to do it (obviously it isn't). I was merely wondering what the best way to go about starting to answer this question was, and I was only asking for hints not a flat out answer. I apologize for any confusion @gung
Comment: There's no need to apologize, that was just FYI. (You may want to take our [tour](http://stats.stackexchange.com/about) to learn a little bit more about CV.) I think the code, in its present form, will not help you. The hints below should get you started, though.
Answer: For some initial hints, consider a logistic regression in a non-Bayesian context.  If the true $B_0$ in some situation were $0$, what would the probability of success be when all $X$ variables were $0$?  Note that $B_0$ is on the logit scale, so you have to convert this into a probability.
Now, if your prior for $B_0$ were centered on $0$, that implies that in some sense (n.b., there are different positions on what Bayesian priors are supposed to mean) you believe the true value of $B_0=0$ (or at least did, before seeing your data).
The width of a prior represents your certainty about your prior belief, with wider priors implying greater uncertainty.  The posterior can be understood (very loosely) as a weighted average of your prior and what your data tell you, with the weights being proportional to the width of your prior.  So the result / mean of your posterior for $B_0$ will be partway between $0$ and what the data suggest, being closer to $0$ the narrower your prior and closer to what the data suggest the wider your prior.
Comment: You're welcome, @C.Thurman. You can ask for further hints if this doesn't get you over the hump.
Comment: hey @gung not sure if you are online, but I'm still slightly confused. If you have any further hints that would be great
Comment: What have you come to understand & what are you still stuck on, @C.Thurman?
Comment: Honestly I'm so lost, I think I understand that since my width of my prior is large it shows greater uncertainty (vague). I'm still not seeing the relation to the success probability @gung
Comment: Start w/ the 1st paragraph of my answer above. In a non-Bayesian setting, the true value of $B_0$ is $0$. Note that this is on the log odds scale. Can you convert this into a probability?
Comment: Wouldn't it just be 1?
Comment: No, it isn't 1. Try reading my answer here: [interpretation of simple predictions to odds ratios in logistic regression](http://stats.stackexchange.com/questions/34636//34638#34638).
Comment: I looked at it, and if I was supposed to use the equation you provided in the chat, I got 1/2. This was assuming the other coeffients were 0 as well. Again I'm not sure, math and I are  not the best of friends @gung
Comment: So if the answer is 1/2, then what is the probability of success when all covariates =0? Moreover, if your prior is centered on 0, what do you believe the probability of success is when all covariates =0? & so, "What does this prior imply in relation to the success probability π?"
Comment: SO does this imply that the success probability of this prior is 50% (so it falls directly between 0 and 100)?
Comment: It does seem like 1/2=50%, which falls between 0% & 100%.
Comment: hmmm okay @gung
Comment: sorry to bother you again, but looking again at the data...if the prior is zero wouldn't that just mean that the data would not transform and the success probability would also be centered on zero? @gung
Comment: $B_0=0$ is on the log odds scale. Try converting it into a probability & see if it ends up equaling 0.
Comment: okay nwm .5 was correct the first time @gung


Is there a simple way to simulate $N$ binary random variables with marginal probability $p$ such that they all have correlation $\rho$?

Question: I am wondering if there a simple way to simulate $N$ binary random variables $X_1, \ldots, X_N$ each with marginal probability $p$ such that they all have the condition of:
$$
Corr(X_i, X_j) = \rho \qquad \forall \ \ i \neq j
$$
? I found a case for $N=2$, but am wondering if this can be extended for all $N$. Any thought would be greatly appreciated.
Answer: If $\rho\in[0,1]$ the following method works: with probability $\alpha$ pick the $X_i$ iid, with probability $1-\alpha$ pick the $X_i$ all equal.  If $\alpha=0$ the $X_i$ are perfectly correlated, if $\alpha=1$ they are perfectly uncorrelated and if $\alpha=1-\rho$ the $X_i$ have the desired correlation of $\rho$.
The case $\rho<0$ is trickier.  In any case, since the covariance matrix of the $X_i$ must be positive semidefinite, we have $\rho\ge -1/(N-1)$, which for $N>2$ restricts the $\rho$ for any collection of rvs, binary or not. Even in the $N=2$ case the restriction to binary rvs makes things delicate.  The pair $X_1=\mathbb 1[U\le p],$ $X_2=\mathbb1[1-U\le p],$
where $U$ is uniformly distributed on $[0,1]$ achieves the most negative $\rho$, which is $\rho_p= -(1-p)/p$ in the case $p\ge1/2$ and $\rho_p=-p/(1-p)$ if $p\le 1/2$.  If the target $\rho$ does not lie in the interval $[\rho_p,0]$, no such binary random variables exist; if it does, stochastically mixing this recipe with the iid recipe will do the job.
I do not know how to generalize to the  negative $\rho$ and $N>2$  case.
Comment: Do you have more references where to find this equation? thanks
Comment: I do not have references yes, but have added some detail which might make you happy.


3-level meta-analysis: Level 3 (study level) has a variance of 0, should I drop this level?

Question: I am doing a 3-level meta-analysis.
First, I fit a theoretically-driven model, where level 1 is the sampling variance, level 2 is the effect level, level 3 is the study level. This method is well known (e.g. see Cheung, 2019). I found that the level 3 variance is basically 0. Hence, I dropped this level and fit a 2-level model (level 1 is the sampling variance, level 2 is the effect level). Comparing model fit, the 2-level model has a slightly better fit (based on AIC and BIC), but only very slightly (e.g. AIC: -8 vs -12)
Question:
Should I go with the 2-level or 3-level model? Another way of asking is: should the model structure be based on theory or model fit?
Thanks in advance!
Ref:
Cheung, M.WL. A Guide to Conducting a Meta-Analysis with Non-Independent Effect Sizes. Neuropsychol Rev 29, 387–396 (2019).  10.1007/s11065-019-09415-6
Answer: Theory, particularly for a meta-analysis and where the fit is comparable as it is here. If the additional level is reasonable eg. where there is a clear nesting of studies according to the random effect and it seems reasonable to account for this, then I would stick to the three-level error structure regardless of the fit statistics.
If the findings differ substantially between the two, you can always report the results of both models.