Upload files to azure blob with async using python get stuck

Question: I have a directory of images which I want to upload to azure blob.
I use the following python code:
    from multiprocessing import Pool
    def upload_file_to_blob(file_path, dest):
         file_name = os.path.basename(file_path)
         blob_client = BlobClient.from_connection_string(connection_string, container_name, os.path.join(dest, file_name))
         with open(file_path, "rb") as data:
             blob_client.upload_blob(data, overwrite=True)

    with Pool(processes=10) as pool:
         for file_ind, file_path in enumerate(file_list):
             pool.apply_async(upload_file_to_blob, args=(file_path, file_dest))
         pool.close()
         pool.join()
    print(f"done")

It seems that all of the files were indeed uploaded, but the program is stuck at finish and the final print is not printed to the terminal. What am I doing wrong?
Comment: Azure does have an aioBlobClient package specifically for this use, in linux that may be the best choice.

however, for me, in windows it had so many bugs it was better to use the normal library with asyincio.to_thread() function for the operation that sends the blob.

Is not a optimal solution (till the guys of microsoft solve their bugs) so make sure to measure the performance difference for your use case as it may depend on the size and number of files that this solution is worth it.
Comment: As per doco in https://docs.python.org/3/library/multiprocessing.html - "If the optional argument timeout is None (the default), the method blocks until the process whose join() method is called terminates."  May be try with a timeout to debug further?
Comment: @Anand Pool.join() does not take any arguments. I guess you are refering to Process.join() which does
Comment: why using multiprocessing in first place for an I/O constrained operation, for me it seems more like a async operation?
Comment: Which operation would you use? Multithread? @ZiurOlpa
Answer: Using the below code I am able to Upload multiple files from the folder to Azure Blob storage.

import os
from multiprocessing import Pool
from azure.storage.blob import BlobClient

# Replace these placeholders with your Azure Blob Storage connection information
connection_string = ""
container_name = " "
local_file_path = " "  # Replace with your local file path
file_dest = "your_destination_folder"

def upload_file_to_blob(file_path, dest):
    file_name = os.path.basename(file_path)
    blob_client = BlobClient.from_connection_string(connection_string, container_name, os.path.join(dest, file_name))
    with open(file_path, "rb") as data:
        blob_client.upload_blob(data, overwrite=True)

if __name__ == "__main__":
    with Pool(processes=1) as pool:  # Using 1 process for a single file
        result = pool.apply_async(upload_file_to_blob, args=(local_file_path, file_dest))
        result.get()  # Wait for the task to complete

    print("done")

Output:

Code:
import os
from multiprocessing import Pool
from azure.storage.blob import BlobClient

# Replace these placeholders with your Azure Blob Storage connection information
connection_string = ""
container_name = " "
local_file_paths = ["C:/Users/Newfolder13.zip", "C:/Users/DotNet6.zip"]  # Put the file paths in a list
file_dest = "your_destination_folder"

def upload_file_to_blob(file_path, dest):
    file_name = os.path.basename(file_path)
    blob_client = BlobClient.from_connection_string(connection_string, container_name, os.path.join(dest, file_name))
    with open(file_path, "rb") as data:
        blob_client.upload_blob(data, overwrite=True)

if __name__ == "__main__":
    with Pool(processes=10) as pool:  # Use 10 processes for parallel uploads
        for file_path in local_file_paths:
            pool.apply_async(upload_file_to_blob, args=(file_path, file_dest))
        pool.close()
        pool.join()  # Wait for all processes to complete

    print("done")

with processes=10

For more details refer to this SO.
Comment: @Sampath why using muiltiprocessing and not the aioBlobClient?
Comment: Won't using result.get() in a loop (as I mentioned, the point is to upload multiple files in parallel) destroy the point of parallel?
Comment: @gozi updated please check


Problema com ("select username, pontos from data order by pontos desc limit 10");

Question: Estou tentando fazer um sistema de ranking, onde aparece o nome de usuário e a pontuação. Estou fazendo da seguinte forma:
$select = $mysqli->query("select username, pontos from data order by pontos desc limit 10");
  $row = $select->num_rows;
  $get = $select->fetch_array();
  $top_1 = $get[0];
  $top_1_pontos = $get[1];
  $top_2 = $get[2];
  $top_2_pontos = $get[3];

Usando um echo no $top_1 ele exibe o nome do usuário com a maior pontuação.
Usando um echo no $top_1_pontos ele exibe a pontuação do usuário.
A partir do $top_1_pontos ele não exibe mais nada... o quê eu fiz de errado?
Qual a forma correta pra se obter o ranking buscando todos os usuários e listando os 10 melhores?
O MySql é mais ou menos assim:
INSERT INTO `data` (`id`,`username`,`pontos`) VALUES
(1,'Joao',9321),
(2,'Maria',151000),
(3,'Pedro',21201),
(4,'Jose',31252),
(5,'Antonio',131145),
(6,'Adriano',13211),
(7,'Marcelo',6897),
(8,'Juliana',53144);

Comment: Hey, mas ai você pediu pra exibir só username e pontos, e nada mais... porque não fez um foreach() pra listar os registros com as devidas pontuações? Vc pode colar seu SQL aqui, e montamos um exemplo prático
Comment: Está só a selecionar o username e os pontos na sua query, ou seja $get[2] já não tem nada. Até devia ter dado erro `undefined index...`
Comment: Quantos registros tem na tabela? Você já experimentou dar um print_r no $get para ver tudo que ele trás?
Comment: Editei com os dados André, ela é mais ou menos daquele jeito.
Answer: É necessário 1 fetch para ler cada registro. Portanto, repita o fetch após mostrar os dados do top_1. (e use os índices 0 e 1 novamente)
Se quiser mostrar todos os 10, é mais prático colocar seu fetch dentro da condição de um while.
EDIT: como tu atualizou tua pergunta, atualizo minha resposta. Segue o trecho de código em questão:
while($get = $select->fetch_array()) {
  $nome = $get[0];
  $pontos = $get[1];
  //faz algo com $nome e $pontos
}

Isso vai ler os nomes e pontuação dos 10 primeiros.
Comment: O responsável pelo downvote poderia explicar?
Comment: Bacana, funcionou!


breadcrumb containing a link to a path that's deeper than the one of the page it's on

Question: On my site i have the weird problem that for one of the menu pages the breadcrumb shows as one lvl deeper than the page it's on.
So it's showing: Home > Page i'm on > subpage i'm not on
instead of: Home > Page i'm on
In addition, contrary to other pages i have, the bottom level of the breadcrumb is clickable which makes no sense, as that's supposed to be the page i'm on (wel it's not due to the above described error, but anywho).
I've no breadcrumb-modules installed.  I do have a hook to modify the breadcrumbs like so:
function mytheme_breadcrumb(&$variables) {
  if (isset($variables['breadcrumb']) && !empty($variables['breadcrumb'])) {
    $crumbs = '
';
    $breadcrumb = $variables['breadcrumb'];
    $array_size = count($breadcrumb);
    $i = 0;
    while ( $i < $array_size) {
     $crumbs .= '
' . $breadcrumb[$i] . '
';
     $i++;
    }
    $crumbs .= '
';
    return $crumbs;
  }
}

Doesn't look like the theme_breadcrumb function is responsible, as it uses the values it's getting from drupal by standard.
Anybody any ideas as to what might be causing this?
Answer: Is it possible that there is an old link to this page in your menu that is nested in the tree like.
home
-page
--MY-PAGE
Because breadcrumbs seem to use the menu tree to build the breadcrumb items.
Comment: This was indeed it: one of the descendants of that item (two levels down) had a link to this item still from when i entered it as dummy content.  Changing the url of that item resolved it :-)

Much appreciated!  I'd upvote you, but sadly my reputation isn't high enough yet :(


Criar relatorio com listas aninhadas

Question: Gostaria de saber por favor como criar relatório no Ireport levando em conta que a figura presente nesse post tem três coleções aninhadas.
Onde uma tabela tem N vigências e uma vigência tem N faixas.
Gostaria de saber por favor como fazer este tipo de relatório no Ireport.
Classe Persistent só tem id do tipo Long, não vai alterar em nada.
Classe TabelaTaxasComissoesEmprestimo
public class TabelaTaxasComissoesEmprestimo extends Persistent {

    private String nome;
    private Boolean desativada;
    private Boolean tabelaNaoPodeSerVinculadaLojas;
    private Boolean tabelaExclusivaParaCPGarantia;
    private Boolean tabelaUtilizadaParaFinanciamentoVeiculos;
    private Short anoFabricacaoVeiculoMin;
    private Short anoFabricacaoVeiculoMax;
    private Float financiamentoPermitidoSobreValorVeiculo;
    private Float endividamentoPermitidoSobreRenda;
    private List<VigenciaTaxasComissoesEmprestimo> vigencias;

    private VigenciaTaxasComissoesEmprestimo vigenciaCorrente;
    private VigenciaTaxasComissoesEmprestimo vigenciaProxima;

    // getters e setters
}

Classe VigenciaTaxasComissoesEmprestimo
public class VigenciaTaxasComissoesEmprestimo extends Persistent {

        private Date dataInicial;
        private Date dataFinal;
        private List<FaixaTaxaComissaoEmprestimo> faixas;

        // getters e setters
    }

Classe FaixaTaxaComissaoEmprestimo
public class FaixaTaxaComissaoEmprestimo extends Persistent {

    private Short prazoMinimo;
    private Short prazoMaximo;
    private Float taxaJuros;
    private Float taxaDiferenciadaParaBanco;
    private Float comissao;
    private Float complementoComissao;

    // getters e setters
}

Obrigado
Answer: Estou supondo que você já tem os conhecimentos básicos em iReport. Você não colocou seus modelos de objeto, então vou imaginar um conforme sua descrição (na verdade, você deve usar os mesmos objetos do domínio da sua aplicação; algumas pessoas tem uma certa dificuldade para trabalhar deste modo, e muitos vezes fazem um de/para dos dados antes de mandar um objeto Java para o Jasper renderizar). Eu me baseei em JRBeanCollectionDataSource mas é facilmente adaptável para outros tipos de DataSources (XML, Connection, ResultSet ...).
Resolução do problema
Como você já deve saber, a melhor maneira de "iterar" sobre uma lista, é através de um sub-report. Ainda sem sair deste conceito, para ter um relatório/arquivo de fácil manutenção e que consequentemente suporte evolução, você deve imaginar o seu layout no iReport de maneira modular e que seja coeso com seu modelo de negócio. Observe a imagem descrita por você, começando pelo modelo Tabela:

Novamente, sabendo que já é de seu conhecimento que a banda "detail" no ireport irá repetir para cada objeto no DataSource (ou cada objeto enviado na lista do JasperReport) eu passo para o sub-report de Vigencia as vigencias da Tabela atual, layout:

O mesmo conceito será aplicado aos itens da Faixa:

A chamada no java, fica mais ou menos assim:
    Tabela tabela = new Tabela();

    tabela.setDescricao("Taxa 1%");
    tabela.setStatus("Ativa");

    Vigencia vigencia = new Vigencia();
    vigencia.setDescricao("Vigencia a partir de 31/07/2014");

    tabela.getVigencias().add(vigencia); // forçando repetir
    tabela.getVigencias().add(vigencia);

    Faixa faixa = new Faixa();

    faixa.setPrazo("37");
    faixa.setTaxaDeJuros("1.00%");
    faixa.setTaxaParaBanco("0.50%");
    faixa.setComissao("4.55%");
    faixa.setComplemento("6.00%");

    vigencia.getFaixas().add(faixa); // forçando repetir
    vigencia.getFaixas().add(faixa);
    vigencia.getFaixas().add(faixa);

    List<Tabela> tabelas = Arrays.asList(tabela);

    Map<String, Object> parameters = new HashMap<String, Object>();

    /**
     * isto é como eu resolvo os subreports, mas existem outras maneiras
     * mais elegantes sem usar FileResolver
     */
    parameters.put(JRParameter.REPORT_FILE_RESOLVER, new FileResolver() {

        public File resolveFile(String fileName) {

            return new File(Main.class.getResource("/jasper/" + fileName).getFile());
        }
    });

    JasperPrint jasperPrint = JasperFillManager.fillReport(Main.class.getResourceAsStream("/jasper/tabela.jasper"), parameters, new JRBeanCollectionDataSource(tabelas));
    ...

O que nos gera a saída:

E atendendo sua complexidade (o que de fato não é complexo):

Fonte dos Arquivos
tabela.jrxml:
<?xml version="1.0" encoding="UTF-8"?>
<jasperReport xmlns="http://jasperreports.sourceforge.net/jasperreports" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://jasperreports.sourceforge.net/jasperreports http://jasperreports.sourceforge.net/xsd/jasperreport.xsd" name="tabela" pageWidth="595" pageHeight="842" columnWidth="555" leftMargin="20" rightMargin="20" topMargin="20" bottomMargin="20">
    <property name="ireport.zoom" value="1.0"/>
    <property name="ireport.x" value="0"/>
    <property name="ireport.y" value="0"/>

    <queryString>
        <![CDATA[]]>
    </queryString>
    <field name="descricao" class="java.lang.String">
        <fieldDescription><![CDATA[descricao]]></fieldDescription>
    </field>
    <field name="status" class="java.lang.String">
        <fieldDescription><![CDATA[status]]></fieldDescription>
    </field>
    <field name="vigencias" class="java.util.List">
        <fieldDescription><![CDATA[vigencias]]></fieldDescription>
    </field>
    <background>
        <band splitType="Stretch"/>
    </background>
    <title>
        <band height="50" splitType="Stretch">
            <staticText>
                <reportElement style="borda" x="0" y="0" width="75" height="50"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Tabela]]></text>
            </staticText>
            <staticText>
                <reportElement style="borda" x="75" y="0" width="75" height="50"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Vigência]]></text>
            </staticText>
            <staticText>
                <reportElement style="borda" x="150" y="0" width="405" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Taxa e comissões por prazo]]></text>
            </staticText>
            <staticText>
                <reportElement style="borda" x="150" y="25" width="81" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Prazo (em meses)]]></text>
            </staticText>
            <staticText>
                <reportElement style="borda" x="231" y="25" width="81" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Taxa de juros]]></text>
            </staticText>
            <staticText>
                <reportElement style="borda" x="312" y="25" width="81" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Taxa para banco]]></text>
            </staticText>
            <staticText>
                <reportElement style="borda" x="393" y="25" width="81" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Comissão]]></text>
            </staticText>
            <staticText>
                <reportElement style="borda" x="474" y="25" width="81" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <text><![CDATA[Complemento]]></text>
            </staticText>
        </band>
    </title>
    <detail>
        <band height="25" splitType="Stretch">
            <textField>
                <reportElement style="borda" stretchType="RelativeToBandHeight" x="0" y="0" width="75" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <textFieldExpression class="java.lang.String"><![CDATA[$F{descricao} + "\n" + "(" + $F{status} + ")"]]></textFieldExpression>
            </textField>
            <subreport>
                <reportElement x="75" y="0" width="480" height="25"/>
                <dataSourceExpression><![CDATA[new net.sf.jasperreports.engine.data.JRBeanCollectionDataSource($F{vigencias})]]></dataSourceExpression>
                <subreportExpression class="java.lang.String"><![CDATA["vigencia.jasper"]]></subreportExpression>
            </subreport>
        </band>
    </detail>
</jasperReport>

vigencia.jrxml:
<?xml version="1.0" encoding="UTF-8"?>
<jasperReport xmlns="http://jasperreports.sourceforge.net/jasperreports" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://jasperreports.sourceforge.net/jasperreports http://jasperreports.sourceforge.net/xsd/jasperreport.xsd" name="vigencia" pageWidth="480" pageHeight="842" columnWidth="480" leftMargin="0" rightMargin="0" topMargin="0" bottomMargin="0">
    <property name="ireport.zoom" value="1.0"/>
    <property name="ireport.x" value="0"/>
    <property name="ireport.y" value="0"/>

    <queryString>
        <![CDATA[]]>
    </queryString>
    <field name="descricao" class="java.lang.String">
        <fieldDescription><![CDATA[descricao]]></fieldDescription>
    </field>
    <field name="faixas" class="java.util.List">
        <fieldDescription><![CDATA[faixas]]></fieldDescription>
    </field>
    <background>
        <band splitType="Stretch"/>
    </background>
    <detail>
        <band height="25" splitType="Stretch">
            <textField isStretchWithOverflow="true">
                <reportElement style="borda" stretchType="RelativeToBandHeight" x="0" y="0" width="75" height="25"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <textFieldExpression class="java.lang.String"><![CDATA[$F{descricao}]]></textFieldExpression>
            </textField>
            <subreport>
                <reportElement x="75" y="0" width="405" height="25"/>
                <dataSourceExpression><![CDATA[new net.sf.jasperreports.engine.data.JRBeanCollectionDataSource($F{faixas})]]></dataSourceExpression>
                <subreportExpression class="java.lang.String"><![CDATA["faixa.jasper"]]></subreportExpression>
            </subreport>
        </band>
    </detail>
</jasperReport>

faixa.jrxml (81px * 5 = 405px):
<?xml version="1.0" encoding="UTF-8"?>
<jasperReport xmlns="http://jasperreports.sourceforge.net/jasperreports" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://jasperreports.sourceforge.net/jasperreports http://jasperreports.sourceforge.net/xsd/jasperreport.xsd" name="faixa" pageWidth="405" pageHeight="842" columnWidth="405" leftMargin="0" rightMargin="0" topMargin="0" bottomMargin="0">
    <property name="ireport.zoom" value="1.0"/>
    <property name="ireport.x" value="0"/>
    <property name="ireport.y" value="0"/>

    <queryString>
        <![CDATA[]]>
    </queryString>
    <field name="comissao" class="java.lang.String">
        <fieldDescription><![CDATA[comissao]]></fieldDescription>
    </field>
    <field name="complemento" class="java.lang.String">
        <fieldDescription><![CDATA[complemento]]></fieldDescription>
    </field>
    <field name="prazo" class="java.lang.String">
        <fieldDescription><![CDATA[prazo]]></fieldDescription>
    </field>
    <field name="taxaDeJuros" class="java.lang.String">
        <fieldDescription><![CDATA[taxaDeJuros]]></fieldDescription>
    </field>
    <field name="taxaParaBanco" class="java.lang.String">
        <fieldDescription><![CDATA[taxaParaBanco]]></fieldDescription>
    </field>
    <background>
        <band splitType="Stretch"/>
    </background>
    <detail>
        <band height="20" splitType="Stretch">
            <textField>
                <reportElement style="borda" x="243" y="0" width="81" height="20"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <textFieldExpression class="java.lang.String"><![CDATA[$F{comissao}]]></textFieldExpression>
            </textField>
            <textField>
                <reportElement style="borda" x="324" y="0" width="81" height="20"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <textFieldExpression class="java.lang.String"><![CDATA[$F{complemento}]]></textFieldExpression>
            </textField>
            <textField>
                <reportElement style="borda" x="0" y="0" width="81" height="20"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <textFieldExpression class="java.lang.String"><![CDATA[$F{prazo}]]></textFieldExpression>
            </textField>
            <textField>
                <reportElement style="borda" x="81" y="0" width="81" height="20"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <textFieldExpression class="java.lang.String"><![CDATA[$F{taxaDeJuros}]]></textFieldExpression>
            </textField>
            <textField>
                <reportElement style="borda" x="162" y="0" width="81" height="20"/>
                <textElement textAlignment="Center" verticalAlignment="Middle"/>
                <textFieldExpression class="java.lang.String"><![CDATA[$F{taxaParaBanco}]]></textFieldExpression>
            </textField>
        </band>
    </detail>
</jasperReport>

Como é chamado o sub-relatório ?
Na aba "Paleta", selecione o elemento "Subreport", na propriedade "Data Source Expression" coloque:
new net.sf.jasperreports.engine.data.JRBeanCollectionDataSource(/*expressao que resultado em uma referencia para java.util.Collection (geralmente um java.util.List) */)
Em "Subreport Expression", coloque o nome do seu sub-relatório: relatorioABC .jasper
Modelos usados

Tabela.java

public class Tabela implements Serializable {

    private static final long serialVersionUID = 203356217547759664L;

    private String descricao;

    private String status;

    private List vigencias;

    public String getDescricao() {
        return descricao;
    }

    public void setDescricao(String descricao) {
        this.descricao = descricao;
    }

    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }

    public List getVigencias() {
        if (vigencias == null) {
            vigencias = new ArrayList();
        }
        return vigencias;
    }

    public void setVigencias(List vigencias) {
        this.vigencias = vigencias;
    }
}

Vigencia.java

public class Vigencia implements Serializable {

    private static final long serialVersionUID = -4743305406353605507L;

    private String descricao;

    private List faixas;

    public String getDescricao() {
        return descricao;
    }

    public void setDescricao(String descricao) {
        this.descricao = descricao;
    }

    public List getFaixas() {
        if (faixas == null) {
            faixas = new ArrayList();
        }
        return faixas;
    }

    public void setFaixas(List faixas) {
        this.faixas = faixas;
    }
}

Faixa.java

public class Faixa implements Serializable {

    private static final long serialVersionUID = -1713006520166696857L;

    private String prazo;

    private String taxaDeJuros;

    private String taxaParaBanco;

    private String comissao;

    private String complemento;

    public String getPrazo() {
        return prazo;
    }

    public void setPrazo(String prazo) {
        this.prazo = prazo;
    }

    public String getTaxaDeJuros() {
        return taxaDeJuros;
    }

    public void setTaxaDeJuros(String taxaDeJuros) {
        this.taxaDeJuros = taxaDeJuros;
    }

    public String getTaxaParaBanco() {
        return taxaParaBanco;
    }

    public void setTaxaParaBanco(String taxaParaBanco) {
        this.taxaParaBanco = taxaParaBanco;
    }

    public String getComissao() {
        return comissao;
    }

    public void setComissao(String comissao) {
        this.comissao = comissao;
    }

    public String getComplemento() {
        return complemento;
    }

    public void setComplemento(String complemento) {
        this.complemento = complemento;
    }
}
Comment: Editei e coloquei o fonte dos modelos. Qual seria exatamente sua dificuldade ?
Comment: Cara, estou tendo um pouco de dificuldade, vc teria como postar o código fonte por favor?
Comment: Cara como você disse, eu dei mole porque deveria ter postado a estrutura de classes mas já foi, estou tendo problemas ao gerar, talvez seja na parte onde se pega o relatório na pasta.
Comment: Você pode usar as que eu anexei, elas seguem 1 Tabela para N Vigencias que tem N Faixas. Mas comente aqui sua estrutura para chegarmos a um concenso :)
Comment: Vou postar lá acima as classes mas não vai mudar muito, mas é o seguinte, eu vi que você colocou isso `/jasper/tabela.jasper`, eu tenho que ter um arquivo deste tipo?
Comment: `/jasper/` é apenas um diretório para meus arquivos jasper, os *.jasper são os compilados (dos *.jrxml) gerados pelo iReport. Se quiser compilar em tempo de execução, você deve fazer algo como: `JasperCompileManager.compileReport(Main.class.getResource("/jasper/tabela.jrxml").getFile());`. Mas eu não recomendo este metodo, afinal, você estaria compilando toda vez que fosse gerar um relatório.
Comment: Vamos [continuar esta discussão no bate-papo](http://chat.stackexchange.com/rooms/17533/discussion-between-macario1983-and-wryel).

Criar relatorio com listas aninhadas


Which linux server distrabution best to run lamp server for production enviroment?

Question: Which Linux server distribution the best to run lamp server for production environment?
1. sharing hosting
2. virtual hosting
3. dedicated hosting
(Ubuntu server, redhat,....)
Thanks,
Yosef
Answer: Whichever you're most comfortable with, as long as it's a serious distribution with good track record for security updates. For servers, long support time tends to be nice as well.
I'd say RHEL (or clones thereof such as CentOS), SLES, Ubuntu LTS, and Debian are all good choices.
Answer: I prefer Ubuntu server mainly for the vast amount of support you can receive via Ubuntu forums. I've solved many a problem there. However, if this is a production server you will most likely want to purchase a proper support package... but free advice never hurt.
Comment: Ubuntu have also enterprise support
Answer: Most of the servers run on CentOS Linux (clone of RedHat Linux). Ubuntu/Debian is also good choice. Depends.. on which platform you are good at. All are good.
Answer: The answer to this answer is: it depends :) In your place, I'd choose the one I was most comfortable with. Every distro has its pros and cons. I've experience with a few Linux distros (RedHat, Debian, CentOS, Ubuntu, Caixa Mágica) and my favorite one is Debian.
I think that it's the most "clean" Linux distro and that it has the best packaging system in the world. On the other side, some commercial software doesn't offer support to Debian. But Debian is the distro that I'm most comfortable with and it suits all my needs :) If you need commercial support, I'd recommend RedHat.
So, choose the one that you're more comfortable with and the one that suits all your needs!


Should I use Docker to create Linux OS within a Linux OS?

Question: I recently started to learn Docker, and know it creates and runs Ubuntu within a container with just a simple command.
docker run -i -t ubuntu:14.04 /bin/bash

I also know that docker-machine uses VirtualBox to create Linux OS in a very handy way.
So what's the difference between them?
Answer: So docker run -i -t ubuntu:14.04 /bin/bash uses docker engine  to create containers(ubuntu container in this case) and will use your Host OS to manage them. On the other hand docker machine will use virtualBox and create VMs(Linux) which will serve as docker hosts running docker engine on them. There are a few links you can refer to :

https://dougwells.gitbooks.io/docker-notes/content/what_is_docker/what_is_difference_between_docker-machine_and_dock.html
https://docs.docker.com/machine/overview/
https://docs.docker.com/engine/
Answer: The first command using docker run is to start a new container. Docker containers can be run anywhere - on your local machine, within a VM (Virtualbox, VMWare etc), in an instance in the cloud, on bare metal or even on your smartphone. All this requires is to have docker installed and running as a daemon / service
docker-machine is a tool used to mimic running docker containers locally using a VM. This is only because earlier versions of docker were not available on MacOS & Windows natively. As such a Linux OS is made available insider a virtual machine with docker installed. On this VM it was possible to run docker commands and docker containers as though it was running docker natively.
You should check out Docker for Mac and Docker for Windows if these are compatible with your setup.


is it possible to connect yamaha psr s970 and Yamaha p115 digital piano?

Question: Is there any way to play my s970 sounds in Yamaha p115 by midi'ng them both? I feel that the s970 piano tones will sound good when it played in weighted keys. Is it possible to achieve that setup with these two keyboards? If so, what should I have to do? How do I connect them? Thanks in advance.
Answer: The P115 does not have 'real' MIDI connectors; it has only a USB port.
And USB makes a strict distinction between a device and a host, so the P115's "USB TO HOST" port works only when connected to a host (a PC), not to another device.
The PSR S970 actually can act as a host, but its "USB TO DEVICE" port works only with flash drives.
To connect the P115 to the PSR S970, you have to put a PC/laptop in the middle.
(And most Yamaha USB MIDI devices are not class compliant, so the Kenton MIDI USB Host does not work either.)
Answer: Yes, but not by a direct connection.  There needs to be a computer running a DAW program in the middle.


CORS erro ao usar fetch API js - destino origem

Question: Bom, estou tentando usar Js puro e html, para consumir uma api, rodando no localhost. Segue os códigos:
Java

JavaScript(rodando em um servidor local)

porém ele retorna um erro de cors, sendo que eu estou colocando origem e destino no header de destino, e no header de retorno. Quando não uso cors, funciona, porém ele não me retorna o body. Quando habilito o cors, ele retorna o seguinte erro
Answer: O problema foi resolvido com uma simples anotação, no lado do backend:


Sistema Academico Java

Question: Olá..
Preciso de ajuda para montar um programa em Java com as seguintes especificações:
1) Ler, do usuário, os seguintes itens:
Nome(String), Idade(int), gênero(char[M ou F]), telefone(String), endereço(String), cpf(String), rg(String), bairro(String), cidade(String), uf(String), ra (se for um aluno)(String), curso (se for aluno)(String), rp(se for professor)(String), disciplina ministrada(se for professor)(String), salário(se for professor)(double).
2) É necessário criar uma classe chamada "Aluno", que possua os atributos de alunos. - Classe criada
package SistemaAcademico;

public class Aluno extends Pessoa {

    public Aluno(String nome, int idade, char genero, String telefone, String endereco, String cpf, String rg, String bairro, String cidade, String uf) {
        super(nome, idade, genero, telefone, endereco, cpf, rg, bairro, cidade, uf);
    }

    String ra;
    String curso;
}

3) É necessário criar uma classe chamada "Professor", que possua os atributos de professores. - Classe Criada
package SistemaAcademico;

public class Professor extends Pessoa {

    public Professor(String nome, int idade, char genero, String telefone, String endereco, String cpf, String rg, String bairro, String cidade, String uf) {
        super(nome, idade, genero, telefone, endereco, cpf, rg, bairro, cidade, uf);
    }

    String rp;
    String disciplinaMinistrada;
    double salario;
}

4) É necessário criar uma classe chamada "Pessoa", que possua os atributos comuns entre Alunos e Professores. - Classe criada
package SistemaAcademico;
import java.util.Scanner;

public class Pessoa {

    private String nome;
    private int idade;
    private char genero;
    private String telefone;
    private String endereco;
    private String cpf;
    private String rg;
    private String bairro;
    private String cidade;
    private String uf;

    public Pessoa(String nome, int idade, char genero, String telefone, String endereco, String cpf, String rg, String bairro, String cidade, String uf) {
        this.nome = nome;
        this.idade = idade;
        this.genero = genero;
        this.telefone = telefone;
        this.endereco = endereco;
        this.cpf = cpf;
        this.rg = rg;
        this.bairro = bairro;
        this.cidade = cidade;
        this.uf = uf;
    }

    public String getNome() {
        return nome;
    }

    public void setNome(String nome) {
        this.nome = nome;
    }

    public int getIdade() {
        return idade;
    }

    public void setIdade(int idade) {
        this.idade = idade;
    }

    public char getGenero() {
        return genero;
    }

    public void setGenero(char genero) {
        this.genero = genero;
    }

    public String getTelefone() {
        return telefone;
    }

    public void setTelefone(String telefone) {
        this.telefone = telefone;
    }

    public String getEndereco() {
        return endereco;
    }

    public void setEndereco(String endereco) {
        this.endereco = endereco;
    }

    public String getCpf() {
        return cpf;
    }

    public void setCpf(String cpf) {
        this.cpf = cpf;
    }

    public String getRg() {
        return rg;
    }

    public void setRg(String rg) {
        this.rg = rg;
    }

    public String getBairro() {
        return bairro;
    }

    public void setBairro(String bairro) {
        this.bairro = bairro;
    }

    public String getCidade() {
        return cidade;
    }

    public void setCidade(String cidade) {
        this.cidade = cidade;
    }

    public String getUf() {
        return uf;
    }

    public void setUf(String uf) {
        this.uf = uf;
    }
}

5) A classe Pessoa deve possuir um método que sobrescreva o método "toString", de modo que imprima todos os dados de Pessoa.
6) Utilize o polimorfismo, para que a classe Aluno possa sobrescrever o método "toString", de modo que delegue à superclasse a escrita dos atributos da classe Pessoa e faça a impressão dos métodos exclusivos da classe Aluno.
7) Utilize o polimorfismo, para que a classe Professor possa sobrescrever o método "toString", de modo que delegue à superclasse a escrita dos atributos da classe Pessoa e faça a impressão dos métodos exclusivos da classe Professor.
(Aonde devo utilizar o toString em cada classe ?)
8) Crie um método com o nome de identificação na classe Principal, que possa ser chamado sem a necessidade de instanciar um objeto (ou seja, diretamente da classe). Esse método não deve receber parâmetros e deve imprimir a seguinte mensagem: "Nome do Aluno - RA do Aluno".
9) O usuário, ao inserir os dados, deve digitar 1 para inserir um Aluno ou 2 para inserir um Professor. Após selecionar o que quer inserir, deve-se apresentar os campos a serem inseridos do respectivo cadastro, ou seja, se for 1, todos os dados de Aluno, se for 2, todos os dados de professor. Os dados devem ser inseridos em uma variável do tipo selecionado. - Criado
package SistemaAcademico;
import java.util.Scanner;

public class Principal {

    public static void main(String[] args) {

        Scanner scan = new Scanner(System.in);
        int opcao = 0;

        do {
            System.out.println("##ESCOLHA UMA OPÇÃO\n");
            System.out.println("1 - Cadastrar Aluno");
            System.out.println("2 - Cadastrar Professor");
            System.out.println("3 - Sair\n");
            System.out.println("Digite uma opção: ");
            opcao = scan.nextInt();

            switch (opcao){
                case 1:
                    cadastro.cadastrarAluno();
                    break;
                case 2:
                    cadastro.cadastrarProfessor();
                    break;
                case 3:
                    break;
                default:
                    System.out.println("Esta não é uma opção válida!");
            }
        } while (opcao !=3 );
    }
}

10) Após inserir os dados, o sistema deve imprimir os mesmos.
11) Regras a serem seguidas:
a) Não é permitido o cadastro de alunos menores de 18 anos. Deve-se mostrar uma mensagem solicitando que entre com a idade aceitável
b) Não é permitido o cadastro de professores menores de 21 anos. Deve-se mostrar uma mensagem solicitando que entre com a idade aceitável;
(Não sei aonde devo inserir estas regras)
Comment: Qual é dúvida em especifico? vc conseguiu resolver algum exercicio? Edite a pergunta e coloque em parte especifica vc tem problema ou dúvida.
Comment: Bem vindo ao SOpt. Estamos aqui para ajudar, mas considere nas próximas vezes fazer perguntas mais específicas. Voc~e pode fazer quantas perguntas quiser. Fazendo-as separadas ajudar organizar e classificar o conteúdo, além de aumentar a chance de ajudar outras pessoas. Contamos com sua colaboração para aproveitar todo o potencial de ajuda do site.
Comment: A sua dúvida é no número 11 então?
Answer: 1) Você pode usar o showInputDialog() nas variáveis para fazer aparecer um dialog com um campo para digitar. Essa seria a maneira mais simples de receber os dados (sem ser pelo prompt, claro). Caso seja pelo prompt, use um objeto do tipo Scanner
String name = JOptionPane.showInputDialog("Digite o nome do aluno");

Com Scanner:
in = new Scanner(System.in);
System.out.print("Digite um nome: ");
String input = in.nextLine();

2 e 3) Procure pela estrutura básica de uma classe Java, também conhecida como POJO. Você ganharia muito tempo se também pudesse usar uma IDE como NetBeans ou Eclipse para isso.
4) Para isso, crie primeiro uma classe Pessoa, que você estará considerando como a classe pai de Aluno e Professor. O ideal é que dados como "nome, idade, gênero..." (todos que ambos Alunos e Professores podem possuir) estejam nessa classe. Procure por exemplos de herança em Java.
5) Para sobreescrever toString, apenas adicione o metódo na classe desejada (no caso, Pessoa). Basta usar this.Atributo para referenciar os atributos desejados no retorno.
public String toString() {
return "Name: '" + this.name + "', Height: '" + this.height + "', Birthday: '" + this.bDay + "'"; }

6) Você escreveu métodos, mas creio que você esteja se referindo à atributos. Basta criar um toString() dentro de Aluno onde você apenas referenciará os atributos que a classe Pessoa não tem. (Por exemplo, RA).
7) Mesma coisa, agora com os dados/atributos exclusivos da classe Professor.
8) Sabe quando você começa a escrever o método? Para que seja possível chamá-lo sem instanciar um objeto, adicione static no cabeçalho.
public static int MeuMetodo() {}

E para chamar, basta chamá-lo a partir da classe.
ClasseAluno.MeuMetodo()

9)Isso é lógica de programação pura. Você só precisa criar linhas de execução para cada caso. Exemplo bem simples de um if:
if(numero == 1) {
   // Faça o recebimento de aluno aqui
} else if(numero == 2) {
   //Faça o recebimento do professor aqui
}else {
   System.out.println("Digite 1 ou 2!")
}

10)Use o System.out.println em um for ou while, desde que ele passe por todos os atributos de quem está sendo inserido/foi inserido. Tranquilo.
11)Lembra do if que eu te mostrei na questão 9? Simplesmente adicione essas restrições no fluxo do programa, idealmente após o recebimento da variável de idade.
Espero ter ajudado. Creio que com essas informações e, considerando que são apenas esses os requisitos, você conseguirá terminar tudo rapidinho. (usando, é claro, if)

Sistema Academico Java


Getting "ImportError: No module named 'MySQL'" and I don't know why

Question: This issue has been bugging me for a while. So, I am using Python 3.9.10 on my Windows 10 computer. To install mysql.connector, I used this command in the command prompt:
pip install mysql-connector-python

And, it returns this:
Requirement already satisfied: mysql-connector-python in c:\users\aksha\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (8.0.24)
Requirement already satisfied: protobuf>=3.0.0 in c:\users\aksha\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from mysql-connector-python) (3.15.6)
Requirement already satisfied: six>=1.9 in c:\users\aksha\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from protobuf>=3.0.0->mysql-connector-python) (1.15.0)

Next, I went to the python shell and entered this code:
import mysql.connector

And, got this error:
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import mysql.connector
ModuleNotFoundError: No module named 'mysql'

I then went back to the command prompt and entered this command:
pip3 install mysql-connector

This is what I get:
Using cached mysql-connector-2.2.9.tar.gz (11.9 MB)
  Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for mysql-connector, since package 'wheel' is not installed.
Installing collected packages: mysql-connector
    Running setup.py install for mysql-connector ... done
Successfully installed mysql-connector-2.2.9

I go back to the python shell and enter the the same code, and it gives back the same error.
I have no idea what I am doing wrong and why this error keeps coming. I have already gone through posts like "ImportError: No module named 'MySQL'", but I am still facing the same issue. I have also checked if I have any python scripts named mysql.py since I have heard they can cause some problems, but there aren't any.
Any help is greatly appreciated!
Edit 1:
Just did pip freeze in the command prompt. This is the result:
mysql-connector==2.2.9
mysql-connector-python==8.0.24
protobuf==3.15.6
six==1.15.0

Not sure what to do now.
Comment: What IDE are you using?  It's possible your IDE is using a Python distribution separate from the one where you installed it.
Comment: I am using IDLE Shell 3.9.10. for anything code related for Python.
Comment: What do you get when you do a "pip freeze"? Do you see mysql module there?
Comment: So, I did just did a pip freeze on command prompt now. This is what I get: ```mysql-connector==2.2.9
mysql-connector-python==8.0.24
protobuf==3.15.6
six==1.15.0 ```
Comment: Enter "import sys; sys.executable" in your python shell, what do you get?
Comment: So, this is what i got: ```'C:\\Users\\aksha\\AppData\\Local\\Programs\\Python\\Python39\\pythonw.exe'```
Answer: This worked for me:
First run this command in the IDLE shell.
import sys; sys.executable

From this you'll get a python path like: C:\\Users\\prat\\AppData\\Local\\Programs\\Python\\Python39\\pythonw.exe
Then use this path with the pip install like:
entire_path -m pip install package-name

Comment: Hey! So, here's what I typed in the command prompt: ```C:\\Users\\aksha\\AppData\\Local\\Programs\\Python\\Python39\\pythonw.exe -m pip install mysql-connector-python```. Is this correct?
Comment: Yeah did it work?
Comment: Entered the command and checked whether it worked in IDLE using ```import mysql.connector```. The same error is still coming. Not sure why.


Shared subset of users in a multi-site instance

Question: Can I have a shared subset of users in a multi-site instance? In other words, I have the same list of administrators across all sites. Can these administrators be admins for all sites through a shared database rather than producing the same list of admins for each site?

I am aware that each site in a multi-site instance has its own database, which contains content and users specific to said site.
I am aware that content and users can be shared by configuring a shared database. However, I only want to share administrators among sites and keep content and other users with site-specific roles and permissions specific to its respective site.
Answer: As you mentioned yourself you can share users across multiple site but that are all users. So you need an external 'database' for your admin users that magically exists for all sites.
For this you could use https://www.drupal.org/project/ldap which could contain you administrators.
Normal users can login as normal aka without use the LDAP directory.
Note LDAP can work with MS Active Directory too.
Comment: If I'm reading this correctly, this would require users (in my case, administrators) to be on an LDAP instance, then authenticated on all sites in a multi-site instance using said LDAP instance? This may be the only way to go about it, so your suggestion is appreciated, but 1) I was hoping to keep it in-house (ie, within what is capable with Drupal) and 2) adding this external component seems to add complexity. Moreover, I'm not sure if adding this external component is possible in my case.
Comment: Bakery sound much easier for sure. For SSO maybe https://www.drupal.org/project/ldap_sso helps.
Answer: You might be able to use what Drupal.org uses(d?), the bakery module.  This requires the sites to be on the same sub-domain by default.
Comment: This has promise. Although users will be per site, SSO would allow my admins to sign in once on one site, then access another site without having to sign in again. Not sure this is the ultimate solution, but this does provide a way to connect admins across all sites.


Analytical function of this landscape

Question: Hey I'm looking for an analytical (or at least with an analytical inverse) continuous function that has this "landscape" please, and I welcome even differential equations that this function would respect :

Meaning : $$\lim_{r\rightarrow \pm\infty} f(r)=0$$, $$\lim_{r\rightarrow \pm\infty} f'(r)=0$$ and $f(0)=0$ and $f(\pm a)=b$ with $a>0$ and $b>0$, if we make it simple : $$f(\pm 1)=\pm 1$$.
Thank you in advance
Comment: Did you mean $f(1)=\color{red}-1?$
Answer: Try something like
$$
f(x)=-\frac{x}{1+x^2}
$$
Or
$$
f(x)=-x\exp(-\lambda x^2),\quad\lambda>0.
$$

Analytical function of this landscape


Erro com switch em swift

Question: Galera, estou tendo um erro no seguinte código. O erro esta no switch, aparece o seguinte erro:

expected declaration.

O Que fazer?
import UIKit

class ViewControllerAnalseOP1: UIViewController, UITextViewDelegate {
    var indice: Int?
    var string1 = "Primeira String"
    var string2 = "Segunda String"

    @IBOutlet weak var descricao: UITextView!

    switch indice {
        case 0:
            descricao.text = string1
        case 1:
            descricao.text = string2
        default:
            descricao.text = "Not Found"
    }

    func textViewShouldBeginEditing(descricao: UITextView) -> Bool {
        return false
    }

    override func viewDidLoad() {
        super.viewDidLoad()
        descricao.delegate = self
    }
}

Comment: Vlw pela ediçao, n vi que estava escrito tao errado
Answer: O erro de sintaxe aparece pois o switch está declarado no corpo da classe. O switch deve ser declarado dentro de algum método, ex:
class ViewControllerAnalseOP1: UIViewController, UITextViewDelegate {
    var indice: Int = 0
    var string1 = "Primeira String"
    var string2 = "Segunda String"

    @IBOutlet weak var descricao: UITextView!

    func textViewShouldBeginEditing(descricao: UITextView) -> Bool {
        return false
    }

    override func viewDidLoad() {
        super.viewDidLoad()
        descricao.delegate = self

        switch indice {
            case 0:
                descricao.text = string1
            case 1:
                descricao.text = string2
            default:
                descricao.text = "Not Found"
        }
    }
}

Comment: Muito Obrigado deu certo

Erro com switch em swift


Criar uma propriedade para uma variavel bool que trate string e int

Question: Estou criando um projeto em que busco as informações do banco de dados, no caso é um arquivo DBF, e o campo é do tipo Logical. Porem quero criar uma propriedade que possa receber uma variável (seja int, string ou bool) e grave na variável privada bool.
Já tenho métodos que recebem int (0/1) e transformam em (True/False) e que recebem string (T/F | V/F | S/N) e transforma em (True/False). Mas gostaria de fazer o tratamento quando estivesse "setando" o valor do campo.
Tentei criar 3 propriedades com cada tipo de dados, mas quando vou utilizar ele fala que há mais de uma propriedade com o mesmo nome. (Que seria nesse caso).
public override int Vip
{
    get { return clsGeneric.convertFromBool(vip); }
    set { vip = clsGeneric.convertToBool(value); }
}
public override string Vip
{
    get { return clsGeneric.convertFromBool(vip,clsGeneric.TypeRetBool.TF); }
    set { vip = clsGeneric.convertToBool(value); }
}
public override bool Vip
{
    get { return vip; }
    set { vip = value; }
}

E agora está assim:
public override TipoGenérico Vip
{
    get { return vip; }
    set
    {
        if (value.GetType() == typeof(string))
        {
            vip = clsGeneric.convertToBool((string)value);
        }
        else if (value.GetType() == typeof(int))
        {
            vip = clsGeneric.convertToBool(value);
        } else {
            vip = value;
        }
    }
}

Queria fazer com que a propriedade aceitasse tanto Int, quanto string e Bool. Há algum meio de fazer isso?

Edit: O resultado ficaria dessa forma?

Class DBPort
{
    private bool vip;
    public LogicalValue Vip
    {
        get { return vip; }
        set { vip = value.Value; }
    }
    public DBPort()
    {
        Vip = false;
    }

}
Comment: Dessa forma não funciona?
Answer: Não é possível fazer a sobrecarga(overload) de propriedades.
Terá de criar uma classe que encapsule esse comportamento, qualquer coisa assim:
public class LogicalValue
{

    public bool Value { get; }

    public LogicalValue(int value)
    {
        Value = clsGeneric.convertToBool(value);
    }

    public LogicalValue(string value)
    {
        Value = clsGeneric.convertToBool(value);
    }

    public LogicalValue(bool value)
    {
        Value = value;
    }
}

Nota: Deverá adicionar aos construtores algum tipo de validação.
A sobrecarga do construtor permite construir objectos LogicalValue a partir de um int, string ou bool.
O seu código ficaria assim:
Class DBPort
{
    public LogicalValue Vip { get; set;}

    public DBPort()
    {
        Vip = new LogicalValue(false);
    }

}

A utilização seria assim:
DBPort dpPort = new DBPort();
dbPort.Vip = new LogicalValue(true);ou (1) ou ("S")

bool valorVip = dbPort.Vip.Value;// valorVip é igual a true

Comment: Eu não entendi como posso utilizar essa classe para setar o valor na propriedade
Comment: A sua propriedade em vez de ser do tipo `bool` passa a ser `LogicalValue`. Para "setá-la" faça `nomeClasse.NomeProp = new LogicalValue(aqui pode ser ser bool ou int ou string);` para obter o valor faça `bool valor = nomeClasse.NomeProp.Value;`
Comment: Peço desculpa pela ignorância, mas continuo sem entender, na questão eu coloquei o código (desculpe-me caso não esteja nos padrões daqui)
Comment: Veja a edição à resposta.
Answer: Você pode criar duas propriedades uma do tipo Objct e outra bool, a primeira servirá de base para seta o value da outra.
namespace stackoverflow
{
    public partial class AlterarTipo : System.Web.UI.Page
    {
        private bool vip { get; set; }
        public  object Vip
        {
            get { return vip; }
            set
            {
                if (value.GetType() == typeof(string))
                {
                    vip = Convert.ToBoolean(value);
                }
                else if (value.GetType() == typeof(int))
                {
                    vip = Convert.ToBoolean(value);
                }
                else {
                    vip = Convert.ToBoolean(value);
                }
            }
        }

        protected void Page_Load(object sender, EventArgs e)
        {
            Vip = "false";
            Vip = 1;
            Vip = true;
            Vip = false;
        }
    }
}

Comment: Esta resposta não é boa. Ela viola o SOLID e o `get` é inseguro.
Comment: @CiganoMorrisonMendez, faça uma melhor, terei o prazer de ver como pode ser feito.
Comment: [Já foi feito](http://pt.stackoverflow.com/a/145488/2999). Minha sugestão é tipar fortemente a propriedade `Vip`, ao invés de um `object`.


Preencher option com dados do Flask - Front x Backend

Question: Tenho pouco conhecimento em frontend e estou com precisando fazer o seguinte:
No meu Backend tenho uma função ao qual executo em python e que retorna uma lista conforme print.
app = Flask(__name__)

@app.route('/')
def info():
    return 'Navegar para os dados'

@app.route('/Teste')
def mensagem():
    fd='var_fd'
    dt='var_dt'

    _engine = sqlalchemy.create_engine("minha_string_de_conexao")
    sql = """
    select data
        from minha_tabela
    where codigo like '%{0}%'
    and data={1!r}
    order by 1 desc;
    """.format(fd, dt)

    pos = pd.read_sql_query(sql,_engine)

    value = str(pos.iloc[0:]['data'].tolist())
    return str(value)

if __name__ == '__main__':
    app.run()

    <select class="form-control" style="width:120px">
    <option selected value="">hh:mm:ss</option>

Preciso que o retorno desses dados seja populado na combobox. Acredito que com jquery seja uma boa opção, no entanto, não estou conseguindo realizar essa demanda.
Como mencionei inicialmente, não tenho grande conhecimento em front. Grato!
Alguém poderia me ajudar?
Answer: Pode usar um AJAX simples com jQuery. Coloque um id no select para ter uma referência:
<select id="horas" class="form-control" style="width:120px">
   <option selected value="">hh:mm:ss</option>
</select>

AJAX:

Como o retorno virá uma array em forma de string, a opção "json" irá converter a string em array.
Comment: Aqui: `$.get("http://127.0.0.1/Teste", function(data){`, a primeira informação entre aspas é a URL de onde virá o retorno do back-end, ou seja. `http://127.0.0.1/Teste`.
Comment: Eu não entendo de Phyton, mas no arquivo onde é gerado a array que vc mostrou no print, você deve retornar essa array para o AJAX... No PHP seria um `echo`.
Comment: Como o AJAX é uma requisição HTTP, é como se vc fizesse um arquivo em Phyton que imprimisse na tela apenas essa array, e o AJAX irá puxar ela em forma de string.
Comment: Isso. Basicamente vc deve gerar um retorno para o AJAX, algo que gere uma string. Acho que no Phyton seria um tipo de `print()`.
Comment: Pelo que vejo, a função do seu Phyton tem um return... vc deveria imprimir esse retorno para que o AJAX o receba.
Comment: Procure marcar ✔ na resposta das suas perguntas que resolveu a questão. Estava olhando seu histórico de perguntas e vi que nenhuma foi finalizada com ✔. O esforço dos usuários em responder perguntas é recompensado quando vc marca ✔ na melhor resposta ou vota positivo nas que vc achou útil. Abs!
Comment: Muito obrigado por seu retorno. Acabei ficando com uma dúvida. Em qual momento o AJAX chama a função que está no Python? Eu devo adaptar o código que você me enviou?
Comment: Perfeito Sam. Pelo que entendi, a etapa que você me explicou "pega" os dados da URL que representa o retorno do back-end. No entanto, para que essa URL seja gerada, eu preciso chamar a função no Python que no caso é a def mensagem().


Permutation test for variance

Question: I have two arrays of data and I wish to test whether or not they are from some distribution with the same variance (whether or not they have the same mean is not important). I don't know what distribution the data are from and certainly have no particular reason to assume it is normal. I was thinking of using a permutation test.
My question is; can I simply re-sample without replacement to create a distribution of difference in variance and then compare my true difference to this distribution? This seems almost too intuitive. Are there any other conditions that apply.
I think this question is probably related to Permutation tests: criteria to choose a test statistic .
Answer: While you could certainly use a statistic of the difference in variance to test equality of variance, some people might feel it would make more sense to have a statistic of the ratio of variance (where equality implies a ratio of 1).
However, this resampling scheme (randomly dividing the whole of the data randomly into two groups) relies on an assumption that the means are identical. If that's not the case, you're not really testing for a difference in variance.
[I'm not sure that "being intuitive" is really something that you can have too much of, but yes, permutation tests tend to be quite intuitive in simple cases. ]
Comment: Re your second paragraph: probably one can explicitly center both groups separately & then do the permutation test as described?
Comment: @amoeba I think the test will no longer be distribution free. It should still perform fairly well though.
Comment: Thanks, I think in reality assuming equal means is reasonable. It would be satisfying, if unnecessary pragmatically speaking, to relax it.
Comment: @amoeba I should have been clearer: you no longer have exact exchangeability (under the null, errors would be but residuals would not), but as sample sizes become large the residuals will tend to behave more and more like the errors, so if the samples are not so small it will be nearly distribution free)

Permutation test for variance


Detecting Differential Item Functioning (DIF) using ordered logistic regression

Question: I am in a situation in which I want to investigate whether test items of a questionnaire appear to have DIF.
The items are polytomous and ranges from 1 to 5, with 5 being the most positive answer. I'm a rookie in the area of psychometrics, but I've tried to read the literature about the various methods of how to detect DIF. Based on what I've read, I decided to use the approach outlined below:
1) Using all items of the questionnaire, I estimate the Graded Response Model and obtained estimates of the latent trait parameter, $\hat{\theta}$, for all individuals.
2) For each item, run three ordered logistic regressions. The first model includes only the trait estimate as covariate, the second model includes also a group-dummy, and finally the third model also features a interaction between the trait estimate and the group-dummy.
3) An item is considered to show uniform-DIF if the LR-test between model 1 and 2 is significant; nonuniform-DIF if the LR-test between model 2 and 3 is significant; and overall-DIF if model 3 significantly outperforms model 1 in terms of log-likelihood.
Basically, what I would like to know, is whether this approach is valid? Does it make any sense?
Comment: Thanks for your answer. I recently came across the R-package "lordif". The package is supposed to be able to decect various type of DIF, using different criterias. Does anyone have sort of experience with this package? If this is the case, how do you decide which one of three criterias Chisqr, R2, or change in coefficients, that you use in application?
Comment: There is an associated JSS paper which might be worth a read there. Also check out the `difR` and `mirt` packages for alternative ways of doing DIF with IRT models.
Comment: could you define what is Differential Item Functioning (or add a link to a definition)?
Comment: My "definition" goes as follows: An item is said to exhibit DIF if individuals from different groups respond differently to the item, given that their trait level is identical.
Comment: Do anyone have an opinion about the approach outlined above?
Comment: Seems rather cumbersome if you ask me, better methods exist for these kinds of data that don't require two-stage approximations and don't initially pool the groups. Also, this approach doesn't account for the (un)-reliability of the $\theta$ estimates, so it will be very dependent on the initial test length.
Answer: Your approach is sound. You did not mention this, but you will need to assess unidimensionality of the response using a factor analysis. Basically, if the test is multidimensional, the item characteristic slopes may end up being of opposite sign which violates the assumption that the probability of greater sum score response is monotonic in ability.
It's slightly more complicated than your explanation. lordif does employ graded response models, which is a hybrid of latent variable modeling and proportional odds models. Just like latent variable modeling uses multivariate normal likelihood and expectation-maximization to model (in a missing data framework) the relationship between multiple exogenous variables and a unifying endogenous variable, lordif does the same with polytomous exogenous variables and a single continuous endogenous variable having a logistic probability model.
Proportional odds models arise from maximum likelihood of logistic RVs thresholded to assume ordinal categories. Just as with normal ICC curves, the logistic slope of this model measures "difficulty" of items. Also just as before, statistically significant differences in thresholds are taken to be evidence of uniform DIF but differences in logistic slopes are evidence of non-uniform DIF.


Trying to scrape this page with python but returns gibberish

Question: I tried this tutorial from here:
https://www.youtube.com/watch?v=XQgXKtPSzUI&list=WL&index=93
And this is the script which I am trying to scrape a steemit post:

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'https://steemit.com/test/@bitcoinfree/test-4'

uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html,'html.parser')
print(page_soup.prettify("utf-8"))

currently the code is outputting gibberish.
I don't know how to get the pure html source.
What am I doing wrong ? :(
Comment: check the gzip option of your accept headers
Answer: got it.
import requests
from bs4 import BeautifulSoup

url = 'https://steemit.com/test/@bitcoinfree/test-4'
r = requests.get(url)
soup = BeautifulSoup(r.content, "html.parser")

print(soup.prettify())


How does one produce a drumline using software?

Question: I have read several threads (see, e.g. Why might one want to record a drum accompaniment electronically without using regular or electronic drumheads and cymbals to drum on?) about creating a drumline using software.  How does one go about that?  I have seen software where one can input notes by clicking with the mouse, creating note values, etc., using a normal computer keyboard; or using a small piano-like (musical) keyboard connected to the computer.  Once the tune is inputted, then you can choose a playback feature.
I can't imagine that's the procedure used for creating a drum part.
So how is it done, and what software do people use for that?  I'm just curious.  I'm sure my son would find it intriguing, but I do NOT want to steer him away from playing his acoustic drumset!  It is very therapeutic for him.  (He has Tourette Syndrome and sensory integration disorder.)
Comment: I made an entire album worth of drum parts using the exact method of pointing and clicking with a mouse that you mention in your question. Yes, it took months of tedious work. Of course, copy and paste was used a lot. Plus, this was in ProTools which has extensive randomization features that I was able to use to create a more organic sounding drum part. Music production can be very labor-intensive.
Comment: @ToddWilcox - So no one has brought artificial intelligence to this problem?
Comment: The definition of "artificial intelligence" that I know is something that doesn't exist, even though it may be possible to create it one day. Like a Bussard ramjet.
Comment: @ToddWilcox - I was imagining a program that would give you some suggestions to start from.
Answer: You're pretty spot on with what you said about clicking and adding notes with the mouse. However this is just one way of programming drums. First and foremost there are hardware drum machines with their own layout and software built in which allow you to create drum tracks with sampled drums. You can also write out drum tracks in MIDI using software such as Guitar Pro (which is what I do) or as you said, you can do it directly in the DAW itself.
If you want to take a peek into this, you can try out some good drum VSTi's such as EZDrummer, Superior Drummer, Addictive Drums and Kontakt. I'm sure most of these have demo versions that you can try out, and also with the Toontrack products (EZ and Superior), they come with a massive library of pre-made MIDI drum beats which you can use when recording, or take a copy of the MIDI and butcher it to your hearts content!


$\int_{[a,b]} z \mu(dz)=\int_a^b \mu([x,b])dx$

Question: Let $\mu$ be a measure on $[0,1]$.
Show that
$$\int_{[0,1]} z \mu(dz)=\int_0^1 \mu([x,1])dx.$$
It looks simple, but I am unable to prove it.
Comment: The tag description of ```self-learner``` specifically says: "Don't use this tag just because you were self-studying when you came across the mathematical question you're asking; it is only for when the fact that you're self-studying is what your question is about." Please edit accordingly.
Answer: $\int_0^{1} \mu [x,1]dx=\int_0^{1}\int_{[x,1]} 1d\mu (z) dx=\int \int_0^{z}dx d\mu(x)=\int zd\mu(z)$. The second equality follows by Fubini's Theorem.
Comment: The fact that $\mu$ and $m$ may be different measures is irrelevant- Draw the picture of a square, $m$ going along the $x$ axis, $\mu$ along the $y$ axis. Draw the diagonal from $(0,0)$ to $(1,1).$ You are integrating over the upper triangle. Now apply Fubini.
Comment: I understand that $\mu([x,1])=\int_{[x,1]}\mu(dz)$. Now I realize what you did with the integral limits in the second equlity. But how to apply Fubini's Theorem when one integral is w.r.t. $\mu(z)$ and the other w.r.t. $x$?
Comment: @Matematleta The Fubini's theorem I learned assumes two measure spaces. In this case, one space has measure $\mu$, and what is the measure behind the other space?
Comment: The second measure is Lebesgue measure on $[0,1]$.


Community website - User rating based on votes in Node + Statistics

Question: I am planning to build a community based website where users would be able to rate the content uploaded by others. I have gone through many blog posts and Stack Exchange questions too, but what I want could not be answered in any of it.
What I got:

User Points module: To associate points to author of a node (content) if the node (content) is up-voted or downn-voted - This, as I understand, can be done using Rules.
Rate module or Fivestar module - To provide rating mechanism for a particular node (content) provided by the author.

I would want to extract statistics like:

Highest rated article for a period of time - Like 1 month or between 'from' and 'to' date etc.
User who has accumulated highest points within a period of time
Etc.

This basically lets me evaluate and find out the most 'liked' or 'popular' user or most 'liked' or 'popular' article within a period of time to award them some goodies.
So, how can I achieve this?
I have done some homework, stumbled upon some threads but could not find what exactly I needed:

How to implement a reputation based system with badges in Drupal similar to StackExchange sites?
Radioactivity: http://www.webomelette.com/drupal-radioactivity-module
Answer: I haven't used User Points Module but for fivestar module or module based on Voting api you can create a view of content and add a relationship with vote results and you will get the aggregation you need i.e average/total number/total score  for that node then you can add a Vote results :Timestamp filter to get the results in between dates. You can add sort and limit to display the way you want.
Comment: It is a field, if you are providing a static value then you can create a filter for this field and set the value or if you are going to execute this view through code then you can set the value using `$view->exposed_input`. But if you want to change this dynamicaly then add this as a field instead of a filter then write `hook_views_query_alter(&$view, &$query)` to add the condition for this field
Comment: Just to be clear "Vote results :Timestamp" is this something which has to be done in code? Or, is it a filter feature provided by the module?
Answer: If you're looking for a working module example with user voting, ratings and statistics, you may want to have a look at the Answers module (which enables the content types Question and Answer, whereas Answers are displayed below the Question they relate to). Together with (at least) 2 of its submodules, i.e.:

Answers Voting, to implement a system to upvote or downvote either Questions and/or Answers (which uses the Rate module).
Answers Userpoints, to increase or decrease "reputation points" (which uses the Userpoints module) on events such as up- or down-voting on a question or answer, or marking an Answer as the "Best Answer".

Head over to its project page for more details.
To extract nodes related statistics, just use the Views module, similar to what is shown on the Answers demo site (in read only mode for anonymous users). It includes Views based lists of:

Questions.
Answers.
Rankings (reputation based).

Refer to my answer to "How to get the node ids of answers for a question in Drupal Services (REST)?" for more details about these views.
Quite a lot of the "business logic" of the Answers module is implemented via all sorts of configurable rules (using the Rules module). As an example, if you don't agree with the out-of-the-box rules about how much "userpoints" are granted to an upvoted (or downvoted) question (or answer), you just use the Rules UI to tune the amount of such points to your requirements.
Disclosure: I'm a (co-)maintainer of the Answers module.


Using factories to create multiple objects when using Dependency Injection

Question: I am trying to figure out how to create multiple objects when using Dependency Injection. As far as I understand the standard approach is to inject a Factory which is then used to create the objects. The part I struggle with is how the Factory creates the objects. So far I see two possible solutions:
The Factory just uses new() to create the object.

Isn't DI supposed to free me of the use of new for non value objects?
What happens if the Object to be created has dependencies that could be resolved by the IoC?

Use the Container as Serviclocator

solves the problems of just newing objects at the cost of introducing an antipattern or is it no longer an antipattern if the use of the serviclocater is constraind within the factories?

It feels like i can coose between a bad and a bad solution. Is there something I am missing or do I understand somthing wrong here?
Edit Currently I am not using an Ioc at all but thinking about Ninject. Although the Autofac DelegateFactories sound very promising.
Comment: Your factories can resolve dependencies from the container.  You don't specify a container.  However, I can tell you that Autofac has explicit support for this scenario.  See http://code.google.com/p/autofac/wiki/DelegateFactories
Comment: What container are you using?
Comment: I have no experience of Ninject so could not comment on that.  Autofac is a *very* good container.
Answer: For starters, I don't consider using a container as service locator in factories an anti-pattern. There are genuine circumstances where it is entirely appropriate. Come to think about it, container aware factories are really container extensions, and those seem to be excluded from service locator bashing. Even the most pure IoC frameworks like AutoFac or Ninject have extensive extension capabilities. A most typical use case for this pattern is resolving to different implementations based on where the service is used.
With regards to using new to create instances inside factories, that is acceptable as well. The IoC/DI message got a bit distorted there and never using new is really a side effect, rather than the goal of DI. The first imperative of Dependency Injection is to externalise creation of dependencies from the component. A factory satisfies that imperative as long as it itself gets injected into component. The questions you need to ask yourself when evaluating such scenarios are:

Does the component itself create its dependencies? A: No, the factory does.
Can you make the component work with different dependencies without modifying it? A: Yes, by injecting a different factory.

I said this before, IoC containers are just factories on steroids. For 80% use case they work out of the box. The other 20% might require tweaks of the above two varieties. I tend to use container aware factories when I want to create components that require both registered dependencies and some input at run-time and new-ing factories when I create Domain objects that don't have dependencies on other services, but take all their construction parameters at run time.
Answer: Although the interface for your factory will be defined at the application level, you would typically define the implementation of that factory class close to your DI configuration, thus as part of your composition root. Although calling the container directly from your code is an implementation of the Service Locator anti-pattern, any code that is defined inside the compostion root is merely mechanics and is therefore not Service Locator. As long as newing up objects or calling into the container is done inside (or very close to) the composition root, this is not a problem, because the application will still be clean from any locator / container.
In other words: use the factory approach. Whether or not you need to new up objects directly inside your factory or make use of the container, depends on the objects. Letting the container create the objects is preferable, especially when they got dependencies on their own, but not all objects can be created by the container. In that case you need to revert to the new operation. Both are fine when the code is part of the composition root and not of the application. The factory itself can have dependencies of its own. This should not be a problem. You can let the container wire-up the factory instance.
Comment: DI containers can build factories for you. In this way you don't have to create code that uses the container. Example from Windsor: http://docs.castleproject.org/Windsor.Typed-Factory-Facility-interface-based-factories.ashx#Custom_codeITypedFactoryComponentSelectorcodes_15


python twisted cred use

Question: i am having a small doubt here regarding twisted for ex when i was browsing throug examples of twisted( which are very well written) i saw an example of using cred. i have to mention that this curiosity is because the more i use twisted ,more i am liking it and trying to wrap my head around it and understand its functionality in detail . now my question is why in the first place i would use cred? if it is for simple user pass credentials cant a simple check to database can be used like its done in most web applications? if it for making async then couldnt async version of db clients be used to achieve the same ?what benefits do i get by using relam , avatars and portals? i mean in which scenario these things could be used. I know it has something to do with my undersatnding of twisted cred so please help.  below is the ex i am talking about
from __future__ import print_function

import sys
from zope.interface import implementer, Interface

from twisted.protocols import basic
from twisted.internet import protocol
from twisted.python import log

from twisted.cred import error
from twisted.cred import portal
from twisted.cred import checkers
from twisted.cred import credentials

class IProtocolUser(Interface):
    def getPrivileges():
        """Return a list of privileges this user has."""

    def logout():
        """Cleanup per-login resources allocated to this avatar"""

@implementer(IProtocolUser)
class AnonymousUser:
    def getPrivileges(self):
        return [1, 2, 3]

    def logout(self):
        print("Cleaning up anonymous user resources")

@implementer(IProtocolUser)
class RegularUser:
    def getPrivileges(self):
        return [1, 2, 3, 5, 6]

    def logout(self):
        print("Cleaning up regular user resources")

@implementer(IProtocolUser)
class Administrator:
    def getPrivileges(self):
        return range(50)

    def logout(self):
        print("Cleaning up administrator resources")

class Protocol(basic.LineReceiver):
    user = None
    portal = None
    avatar = None
    logout = None

    def connectionMade(self):
        self.sendLine("Login with USER <name> followed by PASS <password> or ANON")
        self.sendLine("Check privileges with PRIVS")

    def connectionLost(self, reason):
        if self.logout:
            self.logout()
            self.avatar = None
            self.logout = None

    def lineReceived(self, line):
        f = getattr(self, 'cmd_' + line.upper().split()[0])
        if f:
            try:
                f(*line.split()[1:])
            except TypeError:
                self.sendLine("Wrong number of arguments.")
            except:
                self.sendLine("Server error (probably your fault)")

    def cmd_ANON(self):
        if self.portal:
            self.portal.login(credentials.Anonymous(), None, IProtocolUser
                ).addCallbacks(self._cbLogin, self._ebLogin
                )
        else:
            self.sendLine("DENIED")

    def cmd_USER(self, name):
        self.user = name
        self.sendLine("Alright.  Now PASS?")

    def cmd_PASS(self, password):
        if not self.user:
            self.sendLine("USER required before PASS")
        else:
            if self.portal:
                self.portal.login(
                    credentials.UsernamePassword(self.user, password),
                    None,
                    IProtocolUser
                ).addCallbacks(self._cbLogin, self._ebLogin
                )
            else:
                self.sendLine("DENIED")

    def cmd_PRIVS(self):
        self.sendLine("You have the following privileges: ")
        self.sendLine(" ".join(map(str, self.avatar.getPrivileges())))

    def _cbLogin(self, result):
        (interface, avatar, logout) = result
        assert interface is IProtocolUser
        self.avatar = avatar
        self.logout = logout
        self.sendLine("Login successful.  Available commands: PRIVS")

    def _ebLogin(self, failure):
        failure.trap(error.UnauthorizedLogin)
        self.sendLine("Login denied!  Go away.")

class ServerFactory(protocol.ServerFactory):
    protocol = Protocol

    def __init__(self, portal):
        self.portal = portal

    def buildProtocol(self, addr):
        p = protocol.ServerFactory.buildProtocol(self, addr)
        p.portal = self.portal
        return p

@implementer(portal.IRealm)
class Realm:
    def requestAvatar(self, avatarId, mind, *interfaces):
        if IProtocolUser in interfaces:
            if avatarId == checkers.ANONYMOUS:
                av = AnonymousUser()
            elif avatarId.isupper():
                # Capitalized usernames are administrators.
                av = Administrator()
            else:
                av = RegularUser()
            return IProtocolUser, av, av.logout
        raise NotImplementedError("Only IProtocolUser interface is supported by this realm")

def main():
    r = Realm()
    p = portal.Portal(r)
    c = checkers.InMemoryUsernamePasswordDatabaseDontUse()
    c.addUser("auser", "thepass")
    c.addUser("SECONDUSER", "secret")
    p.registerChecker(c)
    p.registerChecker(checkers.AllowAnonymousAccess())

    f = ServerFactory(p)

    log.startLogging(sys.stdout)

    from twisted.internet import reactor
    reactor.listenTCP(4738, f)
    reactor.run()

if __name__ == '__main__':
    main()

Answer: twisted.cred provides an abstraction which allows your application to be developed independent of the specific means of validation some credentials and lets your credentials database be developed independent of your application.
In your example, note how your application code (Protocol) is unaware of the fact that credentials are stored purely in memory.  It would likewise be unaware if you replaced InMemoryUsernamePasswordDatabaseDontUse with a PAM backend, an /etc-passwd-style database, or something using PostgreSQL.
Furthermore, those username/password backends can all be developed independently of your application logic - allowing each of your applications to re-use them, whether they are your work or work someone else has shared with you.
Comment: thanks for replying but it is still not clear to me i mean i can get that sort of abstraction using a class which checks for user validation ( suppose i am making a web app). is this not a suitable thing for a web app ?
Comment: I don't really understand.  Can you implement something equivalent to twisted.cred from scratch?  Of course you can.  twisted.cred is just a pile of Python code.  You could also implement your own HTTP server and your own filesystem and your own TCP/IP stack.  It's all just more software.  The point of libraries is so that not everyone has to re-implement every thing for every program.
Comment: i didn't mean that. i know libraries exist for some reason that is why in my question above i mentioned `I know it has something to do with my undersatnding of twisted cred` . if possible can you please provide me some directions over where i can look for more info on `twisted.cred` maybe then i will get it
Comment: Sorry, I still don't really understand what you're asking.  I assume you've read the twisted.cred docs on twistedmatrix.com?

python twisted cred use


Fitting and comparing distributions based on diverse summary statistics

Question: I have a bunch of samples, about 35, drawn from a fat-tailed distribution. I think it is reasonable to assume that the samples are all drawn from distributions from the same distributional family, though the parameters will vary from sample to sample.
The top end of the sample is censored. The sample itself is random. (Well, really it's probability weighted. But let's ignore that for purposes of this question).
I have a couple hundred observations for short distance below the censorship threshold, enough to estimate the average value and the slope at the threshold point. I do not think they are spread out enough to estimate, e.g. curvature.
For the interval between the censorship threshold and a second, higher value equal to about five times the threshold value, I have the number of observations and the average value.
I have the number of observations above the higher value just described.
So, that is five pieces of information for each sample.
I have several candidate three- and four-parameter distributional families. I would like to use the data I have to estimate the parameters and select between the distributions.
The best thing I have been able to think of is to use a basic hill-climbing optimization algorithm on the parameters of each distribution to minimize, say, the product of the squared differences between the observed and estimated value of the five parameters (the product because the observations are in three different units, so they can not be added). But this procedure is admittedly very ad-hoc, and I would prefer something more principled. Also, I'm really not sure I should be treating the slope the same way as the other values. Finally, I'd just be comparing fit, like selecting the model with the highest R squared. That does not seem ideal
Is there a better way to do this? Is there any principled way to do it? I have seen maximum likelihood approaches for binned data, but I do not know how to implement them with such diverse summary information, especially when I am comparing different functional forms.
Answer: Let's call the bounds $u_1$ and $u_2$, let's further assumed your distributions have pdf $f(x)$ and cdf $F(x)$. Except for one thing, the easiest thing would seem to be to use maximum likelihood with contribution $f(x_i)$ for observations with $x_i \leq u_1$, $F(u_2)-F(u_1)$ for observations between the two bounds and $1-F(u_2)$ for observations beyond the upper bound. The one thing that does not fit perfectly into this framework is that you know the mean of the values falling between $u_1$ and $u_2$. It sort of feels wrong to throw away the available information in the average, so one alternative possibility is to use approximate Bayesian computation.
What is the process for having that situation (with knowing the average)? The only type of data where this tends to happen is, if the data is really available (someone must have calculated the average), but somehow you cannot get it all (e.g. anonymized tax data). Perhaps looking into the literature of how other people deal with this type of data would be helpful.
Comment: Dear  Bjorn -- yes, it's anonymized survey data. Good call. The average over the threshold is supplied by the agency. Except it is really the average between the two thresholds, with observations above the upper threshold included in the average as if they were at that threshold. This is not even mentioned in the standard documentation, and so far as as I can tell only one other person has used the information. And I don't think they did it correctly. Seems like most people either believe the average (it's the Census, after all!) or disbelieve it and use something Pareto-like.


Formato da data e hora na conversão xls/csv

Question: tenho um codigo que funciona, no entanto preciso de ajuda no formato da data. Na conversão de xls para csv converte a data para mm/dd/aaaa e hora em h:mm e eu queria que fosse dd-mm-yyyy e a hora em hh:mm. O que preciso no meu código?
exemplo da conversão data e hora: 5/28/2018 0:15 e eu precisava que você fizesse assim: 28-05-2018 00:15
if WScript.Arguments.Count < 2 Then
    WScript.Echo "Please specify the source and the destination files. Usage: ExcelToCsv <xls/xlsx source file> <csv destination file>"
    Wscript.Quit
End If
Set objFSO = CreateObject("Scripting.FileSystemObject")
src_file = objFSO.GetAbsolutePathName(Wscript.Arguments.Item(0))
dest_file = objFSO.GetAbsolutePathName(WScript.Arguments.Item(1))
Dim oExcel
Set oExcel = CreateObject("Excel.Application")
Dim oBook
Set oBook = oExcel.Workbooks.Open(src_file)
oBook.SaveAs dest_file, 3
oBook.Close False
oExcel.Quit
Set objFile = objFSO.OpenTextFile(dest_file, 1)
strText = objFile.ReadAll
objFile.Close
strNewText = Replace(strText, " ", ";")
Set objFile = objFSO.OpenTextFile(dest_file, 2)
objFile.WriteLine strNewText
objFile.Close

Comment: Ele está usando o formato de Data/Hora configurado no painel de controle do computador?
Answer: Antes de
oBook.SaveAs dest_file, 3

faça assim:
oBook.Worksheets(0) _
    .Columns("letra da coluna que tem a data/hora a mudar") _
    .NumberFormat = "dd-MM-yyyy hh:mm"


How to search value in ngselect?

Question: From Api, I get the countryName and code as like below
eg:-
  countryName        code
  -----------------------
  India              IND
  United States      USA
  Aruba              ABW

I want to use the ngSelect dropdown and want to display the value like countryName (code) in dropdown.
eg: India (IND)
Question:- I want concatenate the countryName and code. And want to filter the dropdown value based on both countryName and code.
Below is the code which I tried. But its not working properly
component.html
  <ng-select [items]="countries" bindLabel="description" bindValue="code"
           clearAllText="Clear" formControlName="countryCode"
          [searchFn]="customSearchFn">
      <ng-template ng-label-tmp let-item="item">
          {{ item.countryName + ' (' + item.code + ')' }}
      </ng-template>
      <ng-template ng-option-tmp let-item="item" let-search="searchTerm" let-index="index">
           {{ item.countryName + ' (' + item.code + ')' }}
      </ng-template>
   </ng-select>

component.ts
getCountry() {
this.lookupService.getCountry().subscribe(
  data => {
    this.countries = data;
  });
 }

customSearchFn
customSearchFn(term: string, item: Lookup) {
  term = term.toLocaleLowerCase();
  return item.code.toLocaleLowerCase().indexOf(term) > -1 || item.countryName.toLocaleLowerCase() === term;
   }

Can anybody give me a solution for Searching?
Comment: Can anybody help me?
Answer: I took your code and put it into a Stackblitz. It actually almost worked!
I just made a minor change in the search function:
customSearchFn(term: string, item: any) {
    term = term.toLocaleLowerCase();
    return item.code.toLocaleLowerCase().indexOf(term) > -1 ||
           item.countryName.toLocaleLowerCase().indexOf(term) > -1;
}

(both countryName and code use indexOf instead of item.countryName.toLocaleLowerCase() === term)
The only other thing I removed was formControlName="countryCode", that gave me an error "No provider for NgControl". The error makes sense, I have not implemented an NgControl.
Now, ABW gives you Aruba and Aru gives you Aruba too. (That's what you want, right?)
Comment: Thank you Jeremy. Actually, the value fetches only when I type the countryName completely. I need to get value simultaneously when I start to type. Eg: When I type AR (Aruba) it should fetch all the values related to AR but it is fetching when I complete the word . Can u help me to solve this.?
Comment: Ah, now I see. I corrected the customSearchFunction and updated my answer, now it works in the Stackblitz
Comment: Thank you so much, Jeremy. Its working good. can you explain to me the concept?
Comment: Well, you in [items] you give an array of any type of object. Then you tell ng-select what to display as label and what is the code with bindLabel and bindValue. customSearchFunction called for each object in your array for each search term. You get the search term and each object and you have to return if that item does match the search term or not.
Comment: I need to use the  " customSearchFn() " concept in multiple places. Can I control all the dropdown by writing a single function ( "customSearchFn() ") in common ts file? And if yes, how can I call and control the multiple drop downs by common customSearchFn()?
Comment: I don't know. I think it's best to create a new stackoverflow question for that (but search first if someone else asked that already)
Answer: I solved this problem by declaring this array in my ts:
countries = [
    { countryName: "India", code: "IND" },
    { countryName: "United States", code: "USA" },
    { countryName: "Aruba", code: "ABW" }];

I added the ng-select to my html like this, along with the class:
<ng-select
    bindLabel="description"
    class="searchable_select"
    (change)="changed($event)">
    <ng-option *ngFor="let item of countries" [value]="item">
        {{ item.countryName + ' (' + item.code + ')' }}
    </ng-option>
</ng-select>

The changed function is just to display the value of the selected item.
And the most important thing, the SCSS class:
.searchable_select {
  .ng-input {
    display: flex;
    align-items: center;
  }
}

You can also add it as CSS, of course:
.searchable_select .ng-input {
  display: flex;
  align-items: center;
}

How to search value in ngselect?


MYSQL vs. InnoDB

Question: mysqltuner output.
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MEMORY +MRG_MYISAM +MyISAM +PERFORMANCE_SCHEMA
[--] Data in InnoDB tables: 11G (Tables: 44100)
[--] Data in MyISAM tables: 458G (Tables: 52114)
[!!] Total fragmented tables: 13

Here is my.cnf
[mysqld]
symbolic-links=0
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
log-error = /var/log/mysql/error.log
log-warnings = 2
symbolic-links=0
lc-messages-dir = /usr/share/mysql
skip-external-locking
key_buffer              = 2G
thread_stack            = 192K
thread_cache_size       = 350
myisam-recover         = BACKUP
max_connections        = 1500
query_cache_limit       = 100M
query_cache_size        = 512M
general_log             = 0
long_query_time = 10
bind-address = 0.0.0.0
old_passwords=1
skip-secure-auth
innodb_buffer_pool_size  = 30G
innodb_stats_on_metadata=0
table_open_cache = 4096
skip-external-locking
skip-name-resolve
ft_min_word_len = 2
innodb_buffer_pool_instances = 30
innodb_file_per_table=0
innodb_checksum_algorithm=INNODB
binlog_checksum=NONE

I want to cache as much data as possible for MyISAM data-set as well. Currently have dedicate innodb_buffer_pool_size  = 30G for innodb but its only 11 GB in size, which looks me a waste of RAM at all.
What is the best way to cache MyISM data..
My System is suffering from 100% usage on IO, whereas I have plenty of RAM available on system
I'm not able to utilise more than 15GB of RAM from 62GB on a 24 Core Hardware box with 5 TB of RAID storage.
Comment: Unlike InnoDB, MyISAM relies on operating system disk caching.
Answer: 96K tables !!  That can lead to all sorts of problems.
query_cache_size        = 512M -- Having the QC that large hurts performance.  Don't go over 50M.
If you have 62GB of RAM and are using both MyISAM and InnoDB, I would recommend:
innodb_buffer_pool_size = 16G  -- (also noticing that you have only 11G of data)
key_buffer_size = 8G  -- This only for MyISAM indexes

The unused space in RAM is used for OS buffers.  MyISAM lets the OS do the caching.  So, that will leave about 35G for MyISAM data caching.

suffering from 100% usage on IO

Often that is caused by inefficient queries, such as doing table scans
Or maybe it is opening lots of tables, so see if it will let you set this setting to this high:
table_open_cache = 20000

Also, check open_files_limit and ulimit -n.
Comment: my open files limit is 1024.. should I go for increasing it `open files                      (-n) 1024`... what is the best value for a dedicated MySQL server serving higher loads...
Comment: @NasirMahmood To allow more than 1024 Open Files in Linux, from OS Command Prompt, ulimit -n 30000 and press Enter.  To make this value persist across OS shutdown/restart, use 30000 rather than 500000 as in this URL guide. https://glassonionblog.wordpress.com/2013/01/27/increase-ulimit-and-file-descriptors-limit/   Suggesting 30000 since you want to use 20000 just for MySQL activities.
Comment: @NasirMahmood Please post your complete MySQLTuner report and we can then better advise you on the size of key_buffer_size (this is what supports MyISAM indexes).  Also please post to pastebin.com TEXT results of A) SHOW GLOBAL STATUS; and B) SHOW GLOBAL VARIABLES; for workload analysis.
Comment: That 1024 is a common problem.  Yes increase it.  Perhaps 20000.  (The value does not matter a lot -- I think it is there to prevent run-away processes.
Comment: MyISAM data will never by stored in MySQL cache.  Until you ALTER the table to be managed by INNODB - then the data could be a part of innodb_buffer_pool_size storage with reduced access time.  As you move tables to Innodb, adjust your innodb_buffer_pool_size to accomodate the additional data storage.'

MYSQL vs. InnoDB


Combining the conclusions from multiple Diebold-Mariano tests

Question: Suppose I have two forecasting models, A and B. Moreover, suppose I have multiple datasets and I use each of these sets to perform Diebold-Mariano tests. My aim is to find out which of the two models is best overall. So, I perform these tests and get a p-value for each.
What is the right method of combining the conclusions of each of my tests into an overall conclusion? More specifically, what is wrong (if anything) with the following two simple methods:

Sum the z-scores across all tests and divide by the number of tests.
Sum the p-scores across all tests and divide by the number of tests.

Intuitively, 1 seems better to me since it does not work directly on probabilities; the average would finally be converted to an overall p-value. But it also seems to me to be missing something, namely that greater frequencies of extreme observations should provide some kind of "bonus".
Comment: It may be useful to look into meta analysis, e.g. https://stats.stackexchange.com/questions/243003/can-a-meta-analysis-of-studies-which-are-all-not-statistically-signficant-lead
Comment: And although @ChristophHanck is too modest to say so his answer there should give you insight into the literature on combining p-values.
Answer: The first method you suggest is similar to Stouffer's method
Assume we have $k$ studies each giving rise to a $p_i$ and $z()$ is the normal deviate.
Stouffer's method computes
$$\frac{\sum_{i=1}^k z(p_i)}{\sqrt{k}}$$
Which gives a normal deviate $z(\alpha)$ Note the division is not by $k$
The second method you suggest is similar to Edgington's method
$$\frac{(S)^k}{k!}
- {k - 1 \choose 1}\frac{(S - 1)^k}{k!}
+ {k - 2 \choose 2}\frac{(S - 2)^k}{k!} - \dots$$
where $S = \sum_{i=1}^k p_i$ and the summation continues until the term in the numerator $(S-i)$ goes negative. As can be seen this relies on alternating large positive and negative numbers so be careful if you choose to program it yourself.
There are a number of other methods of which the most popular is Fisher's method
$$\sum_{i=1}^{k} - 2 \log p_i$$
which is referred to a $\chi_{2k}(\alpha)$.