How to simulate a browser's back action with HTTP POST reload in Cypress?

Question: I'm using cypress. I want to automate a flow where an HTML page is loaded by an HTTP POST to display a bunch of links. Each of the links loads new pages. On clicking browser's back button, browser performs reload with a POST again (usually comes from a disk cache). Subsequently, I want to navigate to other links on the page and perform the same back.
I'm using cy.go('back') to simulate the back operation. However, the automation gets stuck after the back operation with the browser displaying:
This web page requires data that you entered earlier in order to be properly displayed. You can send this data again, but by doing so you will repeat any action this page previously performed.

I'm using Chrome 117 for the test.
Here is my code:
describe('test', () => {
  const visit = () => {
    cy.visit('https://example.org');
    // Click button performs a POST request and reloads the page
    cy.get('#button').click();
  };

  it('Iterate over all links and back', () => {
    visit();
    cy.get('a.link')
      .as('alias')
      .each((el, index) => {
        cy.get('@alias').eq(index).click();
        /* Do my thing */
        cy.go('back');
      });
  });
})

If I replace cy.go('back') with visit(), it works. However, it defeats the purpose of simulating browser's back navigation.
How do I make cy.go('back') work in this scenario?
EDIT: Re-POST in itself is not an issue for me as it works the same way while simulating this manually in a browser. The browser serves the request via disk cache. Cypress network logs show the failure reasons as net::ERR_CACHE_MISS in Chrome and NS_ERROR_DOCUMENT_NOT_CACHED in Firefox. I also added Cypress proxy to trust store as suggested here to enable disk caching but the issue persists.
Answer: It's not a bug, it's normal for situations where the cache has expired - ERR_CACHE_MISS.
If you want to avoid the message, try adding configuration to the browser statup.
cypress.config.js
module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      on("before:browser:launch", (browser = {}, launchOptions) => {
        if (browser.family === 'chromium' && browser.name !== 'electron') {
          launchOptions.args.push('--disable-prompt-on-repost')
        }
      })
      return config
    })
  }
})

Comment: `ps -ef` tells me that the browser is already running with `--disable-prompt-on-repost`. That's default behavior.
Comment: If it's default, why have it - can you use it to turn it off? If it's already running, why are you seeing the message?
Answer: I am not clear on what the test is testing, but if re-POST-ing is the issue, the second POST can be blocked.
Using the same cy.intercept() twice within the loop, the first one is allowed to proceed, and the second one is blocked.
This works because the last intercept set is the one that will be used.
Something like this:
cy.get('a.link')
  .as('alias')
  .each((el, index) => {
    cy.intercept('POST-url-here', req => req.continue())  // this one goes through
    cy.get('@alias').eq(index).click()
    ...
    cy.intercept('POST-url-here', req => req.destroy())  // this one is blocked
    cy.go('back')

    cy.get('#button').should('be.visible')
  })

To make sure the loop is running at a sensible pace for the page refresh, add a line after the cy.go('back') that asserts something unique to the "back" page is visible.
Otherwise, the test might overrun itself.
Depending on what you need to assert WRT go('back'), you can change req.destroy() to stubbing, and cy.wait('@post')
    ...
    cy.intercept('POST-url-here', {stub-data-here}).as('post')  // stub and alias
    cy.go('back')

    cy.wait('@post').then(interception => {
      // test interception request here
    })
  })

The use of alias to prevent "detached-from DOM" is a nice touch, you would also be able to avoid it by for-looping (so you don't depend on actual link elements, just the count.
cy.get('a.link').then($links => {
   const count = $links.length
   for (let index = 0; index < count; index++) {
     cy.intercept('POST-url-here', req => req.continue())
     cy.get('a.link').eq(index).click()
     ...
     cy.intercept('POST-url-here', req => req.destroy())
     cy.go('back')
     cy.get('#button').should('be.visible')
   }
})

This works because the for-loop variables are in the test  and the refresh from cy.go('back') occurs in the app
Comment: Indeed, but that's not the issue - I think the re-POST is blocked because of the error you first posted so `cy.intercept()` will never see the re-POST.
Comment: `cy.go('back')` is the one causing the re-POST. Re-POSTing is not an issue but somehow the browser/cypress doesn't allow the re-POST and instead displays the error message - 'This web page requires data that you entered earlier in order to be properly displayed. You can send this data again, but by doing so you will repeat any action this page previously performed.'
Comment: IIUC, you're proposing that I intercept re-POST. However, that's not my intention. I want the POST to go through just like it does when simulating this manually with a browser (disk cache). Cypress network logs show the failure reasons as `net::ERR_CACHE_MISS` in Chrome and `NS_ERROR_DOCUMENT_NOT_CACHED` in Firefox.


php recursive key renaming for inner arrays only

Question: i am in need of bit direction here for renaming my inner child keys who inner depth is unknown. Original array is parent child multidimensional array created via id and parent_id relation. Here's a one example of array

$testArray = array (
    "name" => "Test name",
    "someValue1" => 834.69,
    "someMoreValue" => 33.4,
    "someCode" => 6668,
    "child" => array
        (
            array
                (
                    "name" => "Some name",
                    "someValue1" => 471.05,
                    "someMoreValue" => 18.84,
                    "someCode" => 7064,
                    "child" => array
                        (
                            array
                                (
                                    "name" => "Yet another name",
                                    "someValue1" => 438.62,
                                    "someMoreValue" => 17.56,
                                    "someCode" => 7065
                                ),
                            array
                                (
                                    "name" => "Da name",
                                    "someValue1" => 4444,
                                    "someMoreValue" => 44,
                                    "someCode" => 7044
                                )
                        )
                ),
                array
                    (
                        "name" => "name",
                        "someValue1" => 2222,
                        "someMoreValue" => 22,
                        "someCode" => 7022
                    )
        )
);

here's my trying so far, i am able to rename the first level of childs only.
function keyprefix($keyprefix, $keyprefix2, Array $array) {
    foreach($array as $k=>$v){
        $array[$keyprefix.'-'.$k.'-'.$keyprefix2] = $v;
        unset($array[$k]);
    }
    return $array;
}
function test($array) {
    $newArr = array();
    foreach ($array as $key => $value) {
        // $newArr[] = is_array($value) ? test($value) : $value;
        // $newArr[] = is_array($value) ? $array : array_merge( keyprefix("$name","Alt Danışman", $array[$key]) );
        $index = 0;
        $name = $array['name'];
        if (is_array($value)) {
            //if (is_array($value)) {
            //    test($value);
            //}
            $newArr[$key] = array_merge(
                keyprefix("$name","Under Child", $array[$key])
            );
        }
        else {
            $newArr[$key] = $value;
        }
    }
    return $newArr;
}

echo highlight_string(var_export(test($testArray), true));

Any ideas on how I can approach this? I feel that I am close just need a little guide in the right path. I have seen many examples of stack-overflow etc but need some help. I know recursion is not my best skill yet.
Comment: What do you want to rename them too? Something based on the depth, or the parent name, or ???
Comment: hey @Nick i added pic of my code, as you can see i renamed them based on there respective parent name followed by some text. the `keyprefix` is actually renaming all keys of provided array
Comment: Rename array with key child only or rename every inner array?
Comment: Only key name child have inner arrays @shingo
Answer: function TryRenameChildren(array $array)
{
    if(isset($array['child']))
    {
        $array['child'] = keyprefix($array['name'], 'prefix2', $array['child']);

        foreach($array['child'] as $key => $value)
            if(is_array($value))
                $array['child'][$key] = TryRenameChildren($value);
    }

    return $array;
}

$testArray = TryRenameChildren($testArray);

Comment: daymm! thank you @shingo i don't get it, when i will be able to understand recursion in depth. sometimes i just suck :(


Por que uma atribuição cujo valor usa o operador vírgula nem sempre resulta na expressão mais à direita?

Question: Estou com uma dúvida sobre o operador vírgula em C.
No código abaixo, levando em conta alguns textos que li (1) e (2) e vendo a resposta postada aqui, eu esperava que o valor de c seria 20 (expressão mais à direita) e não 10 (expressão à esquerda). O valor de d é 20, conforme esperado.
int main(void)
{
    int a, b, c, d;
    a = 10;
    b = 20;
    c = a, b;       // c = 10
    d = (a, b);     // d = 20
    return 0;
}

Também tentei fazer a atribuição conforme indicada nessa pergunta  e obtive o erro "expected identifier or '(' before numeric constant"
int x = 1, 2, 3, 4; //expected identifier or '(' before numeric constant

O programa foi compilado com o gcc 9.4.0.

Onde está o meu erro?
Meu erro estaria na questão da precedência do operador de atribuição, que é maior que a do operador de vírgula?
Qual a razão do erro na atribuição de "x"?

(1) "Considere a expressão e1, e2. O tipo e o valor da expressão são o
tipo e o valor de e2; o resultado da avaliação de e1 é descartado. O
resultado será um valor l se o operando à direita for um valor l." -
(Microsoft)

(2) "As vírgulas usadas aqui [exemplo de uso de for], na realidade são
operadores de vírgula, que garantem que as listas de expressões sejam
avaliadas da esquerda para a direita. O valor e o tipo de uma lista
separada por vírgulas são o valor e o tipo da expressão mais à direita
na lista" (Deitel, p. 83, C: como programar)
Comment: De fato, faltou parênteses lá, está corrigido agora, me desculpe pela falha e obrigado por perceber isso, mostra que foi útil para você e responde para pessoas como você. E o hkotsubo já respondeu bem que no contexto a precedência cria o problema, em outros contextos não tem esse problema. Mesmo colocando os parênteses é possível que algum compilador, ou configuração gere *warning*.
Answer: Para entender a diferença, basta analisarmos a AST (Abstract Syntax Tree) que cada uma das declarações gera.
No caso, usei o clang (versão 10.0.0-4ubuntu1), com as seguintes opções:
clang teste.c -Xclang -ast-dump

Com isso ele mostra toda a AST, mas vamos ver especificamente a atribuição da variável c:
|-BinaryOperator 0x12b19c0 <line:11:5, col:12> 'int' ','
| |-BinaryOperator 0x12b1968 <col:5, col:9> 'int' '='
| | |-DeclRefExpr 0x12b1910 <col:5> 'int' lvalue Var 0x12b1728 'c' 'int'
| | `-ImplicitCastExpr 0x12b1950 <col:9> 'int' <LValueToRValue>
| |   `-DeclRefExpr 0x12b1930 <col:9> 'int' lvalue Var 0x12b1628 'a' 'int'
| `-ImplicitCastExpr 0x12b19a8 <col:12> 'int' <LValueToRValue>
|   `-DeclRefExpr 0x12b1988 <col:12> 'int' lvalue Var 0x12b16a8 'b' 'int'

Ou seja, o operador vírgula tem dois operandos: a atribuição c = a e a expressão b (que neste caso não faz nada, é como se eu tivesse uma linha contendo somente b;, que é uma expressão válida, mas sem efeito algum). Ou seja, c = a e b são duas expressões separadas. O operador vírgula continua agindo, mas c = a é o primeiro operando, então ele executa primeiro (atribuindo o valor de a em c), e depois ele "executa b" (mas como é uma expressão sem efeito, não "acontece nada").
No fundo, essa expressão é interpretada como (c = a), b, já que o operador = tem precedência maior que o operador ,. A expressão equivale à árvore abaixo:

Caso a imagem não carregue:
        +-----+
        |  ,  |
        +-----+
        /     \
       /       \
    +-----+  +-----+
    |  =  |  |  b  |
    +-----+  +-----+
    /     \
   /       \
+-----+  +-----+
|  c  |  |  a  |
+-----+  +-----+

Agora a atribuição da variável d é diferente:
|-BinaryOperator 0x12b1ab0 <line:12:5, col:14> 'int' '='
| |-DeclRefExpr 0x12b19e0 <col:5> 'int' lvalue Var 0x12b17a8 'd' 'int'
| `-ParenExpr 0x12b1a90 <col:9, col:14> 'int'
|   `-BinaryOperator 0x12b1a70 <col:10, col:13> 'int' ','
|     |-ImplicitCastExpr 0x12b1a40 <col:10> 'int' <LValueToRValue>
|     | `-DeclRefExpr 0x12b1a00 <col:10> 'int' lvalue Var 0x12b1628 'a' 'int'
|     `-ImplicitCastExpr 0x12b1a58 <col:13> 'int' <LValueToRValue>
|       `-DeclRefExpr 0x12b1a20 <col:13> 'int' lvalue Var 0x12b16a8 'b' 'int'

Veja que agora, por causa dos parênteses, a expressão (a, b) é avaliada primeiro, e só depois o seu valor é atribuído a d. Isso porque no início temos o operador de atribuição (=) com dois operandos: a variável d e o resultado do operador , – e este, por sua vez, tem dois operandos: a e b.
No caso, a expressão equivale à árvore abaixo:

Caso a imagem não carregue:
        +-----+
        |  =  |
        +-----+
        /     \
       /       \
    +-----+  +-----+
    |  d  |  |  ,  |
    +-----+  +-----+
             /     \
            /       \
         +-----+  +-----+
         |  a  |  |  b  |
         +-----+  +-----+

Por que int x = 1, 2, 3, 4; não compila?
Quanto ao caso citado, de int x = 1, 2, 3, 4; não compilar, vamos fazer alguns testes para entender. Fiz os exemplos abaixo com somente dois valores, para simplificar, mas a ideia é a mesma.
Primeiro vamos testar com:
int x = 1, 2;

Compilando tanto com gcc 9.4.0 e clang 10.0.0-4ubuntu1, dá erro:
$ gcc teste.c
teste.c: In function 'main':
teste.c:7:16: error: expected identifier or '(' before numeric constant
    7 |     int x = 1, 2;

$ clang teste.c -Xclang -ast-dump
teste.c:7:16: error: expected identifier or '('
    int x = 1, 2;
               ^
teste.c:7:15: error: expected ';' at end of declaration
    int x = 1, 2;

No caso do clang, ele também mostra a AST que conseguiu montar:
|-DeclStmt 0x14dafb0 <line:7:5, col:16>
| `-VarDecl 0x14daf28 <col:5, col:13> col:9 x 'int' cinit
|   `-IntegerLiteral 0x14daf90 <col:13> 'int' 1
`-ReturnStmt 0x14dafe8 <line:15:5, col:12>
  `-IntegerLiteral 0x14dafc8 <col:12> 'int' 0

Repare que ele interpreta isso como um DeclStmt (declaration statement), ou seja, uma declaração de variável (no caso, x), cujo valor atribuído é 1. O 2 não consegue ser interpretado corretamente: repare que o próximo item da AST já é o return 0;, que no meu programa era a próxima linha.
Isso é porque em um declaration statement a vírgula serve para separar as declarações de duas ou mais variáveis - por exemplo, int a, b, c; ou int a = 1, b = 2, c = 3; ou ainda int a = 1, b, c = 3; (a e c são declaradas e já inicializadas, b é somente declarada). Neste caso, a vírgula não é o operador, e sim um separador de declarações de variáveis.
Ou seja, ali ele estava esperando um identificador (no caso, um nome de variável), daí a mensagem "expected identifier". Mas como 2 é um literal numérico, e não um identificador válido, dá erro de compilação.

Por outro lado, se adicionarmos os parênteses:
int x = (1, 2);

Agora ambos (gcc e clang) compilam, e podemos ver na AST que a expressão entre parênteses é executada primeiro, e só depois o seu valor é atribuído a x:
|-DeclStmt 0x23f2010 <line:7:5, col:19>
| `-VarDecl 0x23f1f28 <col:5, col:18> col:9 x 'int' cinit
|   `-ParenExpr 0x23f1ff0 <col:13, col:18> 'int'
|     `-BinaryOperator 0x23f1fd0 <col:14, col:17> 'int' ','
|       |-IntegerLiteral 0x23f1f90 <col:14> 'int' 1
|       `-IntegerLiteral 0x23f1fb0 <col:17> 'int' 2

Pois agora é uma declaração de variável, cujo valor atribuído é o resultado da expressão entre parênteses.

Mas isso só acontece se estivermos declarando e atribuindo a variável ao mesmo tempo. Curiosamente, se separarmos a declaração da atribuição, aí o código compila e executa:
int x;
x = 1, 2;

Pois como já explicado acima, agora a segunda linha não é mais uma declaração, e sim uma operação. Mas como o operador vírgula tem menor prioridade, a atribuição x = 1 é executada primeiro:
|-DeclStmt 0x2407100 <line:7:5, col:10>
| `-VarDecl 0x2407098 <col:5, col:9> col:9 used x 'int'
|-BinaryOperator 0x2407198 <line:8:5, col:12> 'int' ','
| |-BinaryOperator 0x2407158 <col:5, col:9> 'int' '='
| | |-DeclRefExpr 0x2407118 <col:5> 'int' lvalue Var 0x2407098 'x' 'int'
| | `-IntegerLiteral 0x2407138 <col:9> 'int' 1
| `-IntegerLiteral 0x2407178 <col:12> 'int' 2

E agora o 2 é interpretado como um literal numérico (IntegerLiteral), pois neste caso a segunda linha não é mais uma declaração de variável (que espera um identificador), e sim o operador vírgula, que aceita qualquer expressão válida.
E como já explicado acima, o segundo operando (o literal numérico 2) acaba não tendo nenhum efeito aqui. Curiosamente, se compilarmos com a opção -Wall, é mostrado o warning que diz justamente isso (a mensagem do gcc é mais específica):
$ gcc -Wall teste.c
teste.c: In function 'main':
teste.c:8:10: warning: right-hand operand of comma expression has no effect [-Wunused-value]
    8 |     x = 1, 2;

$ clang -Wall teste.c
teste.c:8:12: warning: expression result unused [-Wunused-value]

Por fim, só para deixar completo, podemos separar a declaração da atribuição e incluir os parênteses:
int x;
x = (1, 2);

Isso gera a AST:
|-DeclStmt 0xd5b100 <line:7:5, col:10>
| `-VarDecl 0xd5b098 <col:5, col:9> col:9 used x 'int'
|-BinaryOperator 0xd5b1b8 <line:8:5, col:14> 'int' '='
| |-DeclRefExpr 0xd5b118 <col:5> 'int' lvalue Var 0xd5b098 'x' 'int'
| `-ParenExpr 0xd5b198 <col:9, col:14> 'int'
|   `-BinaryOperator 0xd5b178 <col:10, col:13> 'int' ','
|     |-IntegerLiteral 0xd5b138 <col:10> 'int' 1
|     `-IntegerLiteral 0xd5b158 <col:13> 'int' 2

Que faz com que o operador vírgula seja executado primeiro, e o seu resultado (no caso, 2) seja atribuído a x.


Como criar uma condição "se todos os elementos de uma lista então dentro de uma lista de sets" (Python)?

Question: Estou tentando dizer: "Se todos elementos da uma lista estão em uma lista de conjuntos".
a= [0,1,2,3,4]
b=[ 3,4,5,6,7]
c=[0,1,8,9,10]
d=[0,1]

e= set(a).intersection(b)
f = set(a).intersection(c)

g = [e,f]
print (g)

if all(d) in g : # aqui está meu problema
    print ('done')

Já que a lista d é igual a [0,1] e a lista de conjuntos (sets) g é igual a [{3, 4}, {0, 1}], por que 'done' não é impresso?
Answer: Sua pergunta foi: Como a lista d é igual a [0,1] e a lista de conjuntos (sets) "g" é igual a [{3, 4}, {0, 1}], porque 'done' não é impresso?
A resposta para a pergunta é: "Como d é uma lista e g é uma lista de sets, o Python compara se eles são o mesmo tipo de objeto antes de comparar os elementos. Também é importante notar que igualdade entre listas (list_a == list_b) só é verdadeira se ambas possuírem os mesmos elementos e na mesma ordem."
'''python
a= [0,1,2,3,4] # Isso é uma lista, lista são conjuntos ordenados de dados
b=[ 3,4,5,6,7] # isso também é uma lista
c=[0,1,8,9,10] # também
d=[0,1]        # idem

e= set(a).intersection(b) # Aqui você criou um set a partir da lista a e tirou a interceção com b, o resultado é um set
f = set(a).intersection(c) # Sets são conjuntos não ordenados de dados

g = [e,f] # aqui você criou G como uma lista de sets [{3,4},{0,1}]
print (g)

#if all(d) in g:
#    print ('done')
print(all(d)) # resulta em False
print(False in g,all(d) in g) #resulta em False tambem.
print(d in g) # Também é False, d é uma lista com 2 inteiros, g é uma lista com 2 sets
print(set(d) in g) # Aqui você converte d de lista para set antes de fazer a procura, então ele retorna True

if set(d) in g:
    print('done')

'''
- Não é muito bom, e em geral não funciona, fazer comparações entre dois tipos diferentes a não ser que exista alguma forma dentro do objeto que o compare...

Se g fosse um set e não uma lista de sets você poderia usar issubset() ou issuperset() (referência https://www.w3schools.com/python/python_sets.asp)
Outra alternativa é, você quiser que: lista d = [0,1] seja considerada dentro de algo como h = [{0,1,2},{3,4,5}] pois d está contido no conjunto 1 da lista h

Logo:
d_set = set(d) # convertendo o tipo

contido = d_set.issubset(h) # cria uma lista com a resposta se d está contido em algum conjunto de d

if any(contido): # ou, juntando as linhas: if any(set(d).issubset(h)):
   print('done')

O canal Ignorância Zero (em português) no youtube explica muito bem isso nas seguintes aulas:
Aula 32: Comparação entre listas Aula 117: Sets
Comment: Obrigado! :D
Aproveitei pra adicionar algumas referências de estudo de sets e listas.
Comment: Eu conheço Markdown, HTML e LaTex o meu problema real é escrever de forma bonitinha usando o que eu sei de Markdown e HTML kkkkkkkkkk
Mas obrigado pela dica, vou dar umas olhadinhas pq o Markdown varia de plataforma pra plataforma (Git, Jupyter, etc)
Comment: Como vai, Pedro? Considerando que o site é em português, traduza as partes em inglês da sua resposta. Você pode ainda sugerir uma edição na pergunta traduzindo-a para português. Olhando o perfil do AP fica bem claro que ele é brasileiro e, como normalmente acontece, confundiu o site que fez a pergunta com o SOen. ;D
Comment: Feito, LipESprY eu disse que ia responder em português e me contrariei no final kkkkkkk
Comment: Pode tirar a parte em inglês da sua resposta. A pergunta dele TEM que de ser traduzida. Vai ficar um "furo" na sua resposta. kk
Comment: Pronto, sugeri a edição na pergunta dele mas ela é um pouquinho ambígua. Dei resposta para todas as interpretações que eu pude pensar.
Comment: Agora tá filé! ;)
Comment: Depois dá uma lida em [Formatação](https://pt.stackoverflow.com/help/formatting) que vai alavancar as suas respostas e perguntas. ;)
Comment: Obrigado, funcionou. E vlw pelas referencias tbm !
Comment: De nada @VitorOliveira ! Se puder aceitar a resposta eu agradeço (estou tentando construir uma reputação pra pelo menos poder comentar livrement no stackoverflow :3 )


Confirmation of terminology: Solvable ideal in a Lie algebra

Question: I am (slowly, but steadily) reading J-P. Serre's book on semisimple Lie algebras. The term "solvable ideal" (in a Lie algebra) has been introduced.
I'm interpretting this to mean that if I is a solvable ideal, I just consider it solvable as a Lie subalgebra, in the sense that I usually know Lie algebras to be solvable.
This makes sense, I'm just slightly wary as other terms which have been introduced didn't quite have the initial meaning I presumed them to have. Can anyone confirm that the above is definitely what we mean by solvable ideal?
Answer: Yes, your interpretation is correct. That's what "solvable ideal" means.


How can I parse my json file from asset in kotlin?

Question: How can I parse my json file in kotlin. There are many solution for this on the internet but all of them are same however I couldnt find to parse my json file in kotlin.
my json file
{
    "A": [
        {
            "collocation": "above average",
            "meaning": "more than average, esp. in amount, age, height, weight etc. "
        },
        {
            "collocation": "absolutely necessary",
            "meaning": "totally or completely necessary"
        },
        {
            "collocation": "abuse drugs",
            "meaning": "to use drugs in a way that's harmful to yourself or others"
        },
        {
            "collocation": "abuse of power",
            "meaning": "the harmful or unethical use of power"
        }
    ],
    "B": [
        {
            "collocation": "back pay",
            "meaning": "money a worker earned in the past but hasn't been paid yet  "
        },
        {
            "collocation": "back road",
            "meaning": "a small country road "
        },
        {
            "collocation": "back street",
            "meaning": "a street in a town or city that's away from major roads or central areas"
        },
        {
            "collocation": "back taxes",
            "meaning": "taxes that weren't paid when they were due"
        },
        {
            "collocation": "bad breath",
            "meaning": "breath that doesn't smell nice"
        }],

this way it goes up to the letter W.
I want to parse them all by letter.
my data classes
data class Collocations(

    val tag:String,
    val collocation: List<Collocation>

)
data class Collocation(
    val collocation: String,
    val meaning: String
)

But I think these data classes are not correct in my opinion or not effective.
Answer: You should just need the Collocation class that you provided here. Then the top-level object could be a Map<String, List<Collocation>> so that the letters are dynamic keys instead.
Using Kotlinx Serialization, it could look like this:
import kotlinx.serialization.Serializable
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.json.Json

@Serializable
data class Collocation(
    val collocation: String,
    val meaning: String,
)

fun main() {
    val json: String = TODO("get your JSON string here")
    val collocations = Json.decodeFromString<Map<String, List<Collocation>>>(json)

    val collocationsForA = collocations["A"]
    println(collocationsForA)
}

Comment: can you show code snippet ?
Comment: can you show code snippet ? so as I understand from you said Collocations class just has Map> and hasnt tag and collocation also Collocation data class will stay the same. right ?
Comment: @NewPartizal the code depends on the serialization library you're using. Which one is it? Kotlinx serialization?
Comment: I dont use any serialization library right know because I couldnt  understand the logic. But if we assume that I use kotlinx serialization, how can I do what you said. By the way which serialization library do you think is better to use ?
Comment: I would advise Kotlinx Serialization. It's a good option, because it is an official library meant for Kotlin. It handles Kotlin constructs out of the box, like sealed classes and the likes, and generates serializers at compile-time which is nice (IMO). Jackson with the Kotlin module is also a good option because it's very popular. Gson is not great for Kotlin, and also it's oudated (not really actively developed), and if you go that route you should prefer Moshi (its successor).
Comment: I applied it to my code as you said, but I ran into a problem. The problem is "kotlinx.serialization.json.internal.JsonDecodingException: Expected start of the object '{', but had 'EOF' instead
    JSON input: collocations.json" what's wrong ?
Comment: Are you sure you're passing a JSON that looks like the one in the question? It's looking for a `{`, but it seems it reaches the end of the file unexpectedly (EOF). How are you reading the file and passing the JSON?


Generate a Hash from installed apk file in memory

Question: I am using this code but it is showing the hash code of the same application.
Please Help me out of it .
File file = new File(getApplicationContext().getPackageCodePath());
     String outputTxt= "";
        String hashcode = null;

    try {

        FileInputStream input = new FileInputStream(file);

        ByteArrayOutputStream output = new ByteArrayOutputStream ();
        byte [] buffer = new byte [65536];
        int l;

            while ((l = input.read (buffer)) > 0)
                output.write (buffer, 0, l);

            input.close ();
            output.close ();

            byte [] data = output.toByteArray ();

            MessageDigest digest = MessageDigest.getInstance( "SHA-1" );

            byte[] bytes = data;

            digest.update(bytes, 0, bytes.length);
            bytes = digest.digest();

            StringBuilder sb = new StringBuilder();

            for( byte b : bytes )
            {
                sb.append( String.format("%02X", b) );
            }

            hashcode = sb.toString();

    } catch (FileNotFoundException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (NoSuchAlgorithmException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

Now I want to read the installed application file of my current android application(assuming it will be *.apk), read byte array from that file and produce the Hash value.
Thanks in Advance.
Answer: This is the best solution for this.
First u have to pass the path of the .apk file.
          private CharSequence getHash(String sourceDir)  {
          // TODO Auto-generated method stub

          File file = new File(packageInfo.applicationInfo.sourceDir);
         String outputTxt= "";
           String hashcode = null;

        try {

        FileInputStream input = new FileInputStream(file);

        ByteArrayOutputStream output = new ByteArrayOutputStream ();
        byte [] buffer = new byte [65536];
        int l;

              while ((l = input.read (buffer)) > 0)
                  output.write (buffer, 0, l);

                  input.close ();
              output.close ();

                byte [] data = output.toByteArray ();

                MessageDigest digest = MessageDigest.getInstance( "SHA-1" );

            byte[] bytes = data;

            digest.update(bytes, 0, bytes.length);
            bytes = digest.digest();

            StringBuilder sb = new StringBuilder();

            for( byte b : bytes )
            {
                sb.append( String.format("%02X", b) );
            }

            hashcode = sb.toString();

         } catch (FileNotFoundException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
         } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
         } catch (NoSuchAlgorithmException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
       }

          return hashcode;
         }

Answer: public void buttonAppClick() {
    final PackageManager pm = getActivity().getPackageManager();
    //get a list of installed apps.
    List<ApplicationInfo> packages = pm.getInstalledApplications(PackageManager.GET_META_DATA);
    outputTextView.setText("");
    for (ApplicationInfo packageInfo : packages) {
        try {
            String packageName = packageInfo.packageName;
            outputTextView.append("Apk Path : " + packageInfo.sourceDir + "\n");
            PackageInfo pi = pm.getPackageInfo(packageName, PackageManager.GET_SIGNATURES);

            Signature sig = pi.signatures[0];
            String md5Fingerprint = doFingerprint(sig.toByteArray(), "MD5");
            Log.d(TAG_HOME, "MD5 : " + packageInfo.sourceDir + md5Fingerprint);
            outputTextView.append("MD5 : " + md5Fingerprint + "\n");
            outputTextView.append("\n");
        }
        catch (Exception e) {
            Log.e(TAG_HOME, e.getMessage());
        }
    }
}

protected static String doFingerprint(byte[] certificateBytes, String algorithm)
        throws Exception {
    MessageDigest md = MessageDigest.getInstance(algorithm);
    md.update(certificateBytes);
    byte[] digest = md.digest();

    String toRet = "";
    for (int i = 0; i < digest.length; i++) {
        if (i != 0)
            toRet += ":";
        int b = digest[i] & 0xff;
        String hex = Integer.toHexString(b);
        if (hex.length() == 1)
            toRet += "0";
        toRet += hex;
    }
    return toRet;
}

First i got the signatures from the PackageInfo
Convert that signature to byte array and then use the doFringerPrint function to get the MD5 of the android app.
Comment: This does not calculate the APK hash, but the MD5 of the signing certificate.


Best practices to increase performance of Plone sites behind Apache

Question: I have a small number of Plone sites running in virtual hosts of an Apache 2.4 server. I have the strong feeling that the performance could be much better.
Let's assume a pretty basic setup for a Plone site:
<VirtualHost *:80>
    ServerAdmin email@example.com
    ServerName www.example.com
    CustomLog /var/log/httpd/vhost-example.log combined
    ErrorLog /var/log/httpd/vhost-example.error.log
    RewriteEngine on

    DocumentRoot /opt/zope/instances/example/htdocs
    ErrorDocument 502 /error-500.html
    ErrorDocument 503 /error-500.html

    RewriteEngine On
    RewriteRule ^/(.*) http://localhost:8080/VirtualHostBase/http/%{HTTP_HOST}:80/example/VirtualHostRoot/$1 [L,P]
</VirtualHost>

In this setup, almost everything is done by the Zope server, listening to port 8080, which runs the Plone site in the virtual folder /example (which lives in the object database).
Now this server provides some static resources, e.g. images and stylesheets (which live in the filesystem), and some pseudo-static resources, like compilations of scripts or stylesheets with a timestamped name (which don't have a filesystem location).
Some static resources have paths which give hints where the physical resource is located (e.g. /++resource++browsername/name-of-script.js, where browsername is the hint); others are taken from so-called skins (i.e. from the first skin which contains them), with no easy way to tell the filesystem location from the skin name (which would not be known from the response anyway).
Such requests would not need to be handled by the underlying Zope server, using one of its threads. What are the recommended strategies to keep away from Zope what Apache could do itself (more efficiently, and without blocking Zope's threads), and to cache contents?
So, a complete solution might include

mapping of distinct paths of static resources to known file system location, where possible
trying several locations for static resources which are taken from skins
caching of generated resources (e.g. HTML pages, or compilations of CSS or Javascript)

and, with ZEO setup,

load balancing (distribution of requests to a number of ZEO clients which in turn talk to the ZEO server)

Any configuration guides anywhere?
Answer: The main purpose of Plone is to provide a multi-user content management system that allows to edit websites collaboratively. The price is dynamic creation of the web content presentation and some loss of speed. If you want small, fast websites which you mostly edit yourself then you might be better off with a generator software that creates static websites for you. Quite a number of them are available both commercially and open source. Having said this, in principle you can combine both dynamically and statically generated content from all sorts of sources behind apache, e.g. http://httpd.apache.org/docs/2.4/urlmapping.html, at any leaf node of the tree structure of the site.
Comment: I didn't say the content is static (it isn't); but the most resources are static (javascripts, stylesheets, icon images) or rarely changed after creation (which will be true for many dynamic sites).  Thanks for the link; nice to know, but this doesn't help much, because it is just a quick overview.


Customização de mensagem Validator Laravel

Question: Tenho o seguinte validador cpfValidacao.php:
<?php

namespace App\Utils;

class cpfValidacao
{
    public function validate($attribute, $value, $parameters, $validator)
    {
        return $this->isValidate($attribute, $value);
    }

    protected function isValidate($attribute, $value)
    {
        $c = preg_replace('/\D/', '', $value);
        if (strlen($c) != 11 || preg_match("/^{$c[0]}{11}$/", $c)) {
            return false;
        }
        for ($s = 10, $n = 0, $i = 0; $s >= 2; $n += $c[$i++] * $s--);
        if ($c[9] != ((($n %= 11) < 2) ? 0 : 11 - $n)) {
            return false;
        }
        for ($s = 11, $n = 0, $i = 0; $s >= 2; $n += $c[$i++] * $s--);
        if ($c[10] != ((($n %= 11) < 2) ? 0 : 11 - $n)) {
            return false;
        }
        return true;
    }
}

Chamada no AppServiceProvider.php:
/**
* Bootstrap any application services.
*
* @return void
*/
public function boot()
{
   Validator::extend('cpf', '\App\Utils\cpfValidacao@validate');
}

Chamada no controller:
 /* validação de parâmetros */
 $validator = Validator::make($request->all(), [
     'email' => 'required|min:1|max:255',
     'cpf' => 'required|min:1|max:11|cpf',
     'name' => 'required|min:1|max:255',
     'password' => 'required|min:8|max:255'
     //'profile_picture_path' => 'required|min:1|max:255'
 ]);
 if($validator->fails()){
     $errors = $validator->errors();
     return response()->json(['error' => true, $errors], 422);
 }
 /* fim */

Retorno:
{
    "error": true,
    "0": {
        "cpf": [
            "validation.cpf"
        ]
    }
}

Quero customizar a mensagem "validation.cpf". Como faz?
Laravel 6.2
Comment: O ideal seria você criar uma Rule mesmo sem precisar estender o service provider, lá você pode definir a mensagem que quiser, https://laravel.com/docs/7.x/validation#custom-validation-rules
Comment: Certo obrigado irmão
Answer: Como comentado pelo Thiago, o ideal seria você criar uma Rule da seguinte forma:
Rodando o comando para criar o arquivo de validação em app/rules
php artisan make:rule CpfValidation

Na função passes colocar sua lógica:
public function passes($attribute, $value)
{
    $c = preg_replace('/\D/', '', $value);
    if (strlen($c) != 11 || preg_match("/^{$c[0]}{11}$/", $c)) {
        return false;
    }
    for ($s = 10, $n = 0, $i = 0; $s >= 2; $n += $c[$i++] * $s--);
    if ($c[9] != ((($n %= 11) < 2) ? 0 : 11 - $n)) {
        return false;
    }
    for ($s = 11, $n = 0, $i = 0; $s >= 2; $n += $c[$i++] * $s--);
    if ($c[10] != ((($n %= 11) < 2) ? 0 : 11 - $n)) {
        return false;
    }
    return true;
}

E na função mensage, a mensagem que você retornar caso dê erro de validação:
public function message()
{
    return 'Cpf inválido.';
}

Um exemplo de como usar a validação feita:
$validator = Validator::make($request->all(), [
 'email' => 'required|min:1|max:255',
 'cpf' => ['required', 'min:1', 'max:11', new CpfValidation],
 'name' => 'required|min:1|max:255',
 'password' => 'required|min:8|max:255'
 //'profile_picture_path' => 'required|min:1|max:255'
 ]);

Comment: @LuizRobertoFurtuna senão tiver a ver com o CPF não é legal colocar junto porque aí você está desrespeitando o principio da responsabilidade única, sempre tente deixar as coisas separadas. então se precisar de outra regra, é só criar outra de novo
Comment: Entendi. Fiz aqui e já funcionou. É possível adicionar mais de uma função no mesmo "`rules`"?


Computationally feasible way to fit data generated by sum of log normal and normal variables

Question: I often run into data that has effects that go across a wide range of scales and are roughly log-normal, yet also have normal error terms added in, occasionally making some terms negative. Is there some way to fit a distribution to this data? If I can do this, I can check whether this model makes sense numerically and theoretically, which may make this data more interpretable.
More specifically, if $Z_1$ and $Z_2$ are uncorrelated standard normal variables, then I believe samples $X$ are drawn from
$$X=e^{\mu_1+\sigma_1Z_1} + \mu_2 + \sigma_2Z_2$$
I am most specifically considering this in R, but I am open to other methods. I have considered writing out the likelihood function explicitly and optimizing with the four parameters and nlm, but the integration (convolution) was too complicated to run efficiently. I also considered using nls to fit the empirical cumulative distribution to the cumulative distribution, but I didn't find an easy way to specify that cumulative distribution.
Edit: two realistic cases I have encountered have been roughly:

N=10000, $\mu_1 = -3$, $\sigma_1 = 3$, $\mu_2 = 0$, $\sigma_2 = 10^{-3}$
N=1000, $\mu_1 = -9$, $\sigma_1 = 3$, $\mu_2 = 0$, $\sigma_2 = 10^{-6}$,
Comment: Maybe you could look for $\mu_1, \sigma_1, \mu_2, \sigma_2, (L_i), (N_i)$ which satisfy $L_i+N_i=X_i$ and maximize the likelihood $\prod P[L_i| \mu_1, \sigma_1]P[N_i|\mu_2, \sigma_2]$. Your first step would be guessing a decomposition of the $X$'s, then in even steps you'd optimize the parameters given the decomposition, while in odd steps you'd optimize the decomposition given the parameters, alternating until convergence.
Comment: Those are such enormous values of $\sigma_1$ and small values of $\sigma_2$ as to make the problem unsolvable with any reasonable precision.  If these values are realistic, you might just as well throw away the nonpositive values in the data and fit a Lognormal distribution.
Comment: It might not make a whole lot of difference but have you considered a model that is a "mixture" of a log normal and a normal rather than the sum of log normal and a normal?  (https://en.wikipedia.org/wiki/Mixture_distribution)  That would depend on how you believe what the process is that generates the data.
Comment: @Jim Because the Normal term is intended to model measurement error, a mixture model is not indicated.
Comment: @whuber My usual approach is to add some small constant term such that the distribution is roughly log-normal (sometimes doing several runs with different parameters as a robustness check). I've done that for five or six versions of problems like this over the past few years. It felt like an inelegant hack, so I thought I'd explore fitting what I really think is going on.
Comment: @whuber I stand corrected.  I should have thought a bit more on that suggestion of a mixture distribution.
Answer: Here is an approach using stochastic optimization to perform distribution matching. The idea is to generate a random a random vector, $Y$, using the parameter guesses at each iteration and then compare $Y$ to $X$ using a goodness-of-fit test statistic, which is returned to the stochastic optimizer.
The benefits to this approach are that it is fairly simple to code, and it converges to good estimates. The drawbacks are that it is somewhat computationally expensive, and each parameter must be within the upper and lower search bounds provided to the optimizer.
The R code below demonstrates the approach using the following:

The stochastic optimization function StoSOO from the OOR package
The Kolmogorov-Smirnov (KS) test statistic
The two cases provided by the OP

Setup:
library(OOR)
set.seed(890018898)

# simulate case 1
n <- 1e4L
x <- exp(rnorm(n, -3, 3)) + rnorm(n, 0, 1e-3)

x <- sort(x)
pos <- which.max(x > 0)

First, get initial guesses for the parameters.
m1 <- mean(log(x[pos:n]))
s1 <- sd(log(x[pos:n]))
m2 <- 0
s2 <- sqrt(mean(x[1:(pos - 1)]^2))

To reduce the variability in the test statistic, sample $Y$ to have more elements than $X$. Also, have the function that returns the test statistic accept $\log(\sigma_1)$ and $\log(\sigma_2)$ to keep the guesses positive for the scale parameters.
os <- 100L # y oversampling for the KS statistic
nos <- n*os
w <- rep.int(c(-1, 1/os), c(n, nos))

f <- function(p) {
  # returns a value proportional to the KS statistic comparing x to a randomly
  # generated vector of size "nos" using the parameters in "p"
  y <- exp(rnorm(nos, p[1], exp(p[2]))) + rnorm(nos, p[3], exp(p[4]))
  max(abs(cumsum(w[order(c(x, y))])))
}

Run the stochastic optimizer.
system.time(
  res1 <- StoSOO(
    numeric(4), f, lower = c(m1 - abs(m1)/2, log(s1/2), -1, log(s2/2)),
    upper = c(m1 + abs(m1)/2, log(s1*2), 1, log(s2*2)), nb_iter = 1e3
  )
)
#>    user  system elapsed
#>  193.17    7.47  200.88

View the results.
res1$value
#> [1] 59.485
params1 <- setNames(res1$par, c("mu1", "sigma1", "mu2", "sigma2"))
params1[c(2, 4)] <- exp(params1[c(2, 4)])
params1
#>           mu1        sigma1           mu2        sigma2
#> -3.0227077504  2.9517279929  0.0000000000  0.0009123462

StoSOO doesn't always converge tightly, so re-run it with tighter bounds in an attempt to find a better fit.
system.time(
  res2 <- StoSOO(
    numeric(4), f, lower = res1$par - abs(res1$par/4),
    upper = res1$par + abs(res1$par/4), nb_iter = 1e3
  )
)
#>    user  system elapsed
#>  198.06    9.16  207.51

res2$value
#> [1] 49.55
params2 <- setNames(res2$par, c("mu1", "sigma1", "mu2", "sigma2"))
params2[c(2, 4)] <- exp(params2[c(2, 4)])
params2
#>           mu1        sigma1           mu2        sigma2
#> -3.0227077504  2.9914362045  0.0000000000  0.0009946919

Compare the values of the test statistic for the estimated parameters to those for the actual parameters. We find the estimated parameters fit $X$ better than the actual parameters because $X$ is a random sample.
mean(replicate(100, f(c(-3, log(3), 0, log(1e-3)))))
#> [1] 82.9047
mean(replicate(100, f(res2$par)))
#> [1] 51.9628

Repeat the exact same procedure for the second case.
# simulate case 2
n <- 1e3L
x <- exp(rnorm(n, -9, 3)) + rnorm(n, 0, 1e-6)

x <- sort(x)
pos <- which.max(x > 0)
# get initial guesses
m1 <- mean(log(x[pos:n]))
s1 <- sd(log(x[pos:n]))
m2 <- 0
s2 <- sqrt(mean(x[1:(pos - 1)]^2))

os <- 100L # oversampling for the KS statistic
nos <- n*os
w <- rep.int(c(-1, 1/os), c(n, nos))

system.time(
  res1 <- StoSOO(
    numeric(4), f, lower = c(m1 - abs(m1)/2, log(s1/2), -1, log(s2/2)),
    upper = c(m1 + abs(m1)/2, log(s1*2), 1, log(s2*2)), nb_iter = 1e3
  )
)
#>    user  system elapsed
#>   19.42    1.31   20.75

res1$value
#> [1] 14.33
params1 <- setNames(res1$par, c("mu1", "sigma1", "mu2", "sigma2"))
params1[c(2, 4)] <- exp(params1[c(2, 4)])
params1
#>           mu1        sigma1           mu2        sigma2
#> -9.071491e+00  2.956245e+00  0.000000e+00  4.889863e-07

system.time(
  res2 <- StoSOO(
    numeric(4), f, lower = res1$par - abs(res1$par/4),
    upper = res1$par + abs(res1$par/4), nb_iter = 1e3
  )
)
#>    user  system elapsed
#>   19.40    1.17   20.61

res2$value
#> [1] 13.2725
params2 <- setNames(res2$par, c("mu1", "sigma1", "mu2", "sigma2"))
params2[c(2, 4)] <- exp(params2[c(2, 4)])
params2
#>           mu1        sigma1           mu2        sigma2
#> -9.052825e+00  2.962846e+00  0.000000e+00  4.889863e-07

mean(replicate(100, f(c(-9, log(3), 0, log(1e-6)))))
#> [1] 21.1452
mean(replicate(100, f(res2$par)))
#> [1] 14.1672

Answer: One feasible approach is moment-matching: finding parameters to match the first four moments of the data.
In this case $X=L+N$ with independent lognormal $L$ and normal $N$ normal, so (for example)
\begin{align}
E[X^2]&=E[L^2]\phantom{\mu_1+2\sigma_1^2)}+2\,E[L]\,E[N]\phantom{\mu_1\sigma_1^2/2)}+E[N^2]\\
&=\exp(2\mu_1+2\sigma_1^2)+2\mu_2\exp(\mu_1+\sigma_1^2/2) + (\mu_2^2+\sigma_2^2)
\end{align}
which uses standard formulas for the moments of the lognormal and normal summands. This gives an equation in $\mu_1,\sigma_1,\mu_2,\sigma_2$, saying that the above expression should equal the average of the squares of the datapoints.
This and similar equations for $E[X], E[X^3], E[X^4]$ can then be solved numerically for $\mu_1, \sigma_1, \mu_2, \sigma_2$.
Comment: Or, if we can set $\mu_2=0$ for an unbiased error, we could use just three moments and get more stability.
Comment: It would be nice to see a realistic example, because my intuition is that this is a very unstable numerical problem and that moment matching might not even succeed without very large amounts of data.
Comment: It would be nice to see some realistic parameters and sample sizes from the OP!
Comment: Agreed.  We do have a hint: the presence of some negative values implies $e^{\mu_1},$  $\mu_2,$ and $\sigma_2$ are of similar magnitudes.  But a lot hinges on how large $\sigma_1$ might be.
Comment: I just added some parameters and sample sizes. Thanks for the suggestion!
Answer: To find the maximum likelihood estimates of the parameters we need the probability density function, $f(x)$, for $X=Z_1+Z_2$:
$$f(x)=\int_0^\infty \frac{\exp \left(-\frac{(x- z_1-\mu_2)^2}{2 \sigma_2^2}-\frac{(\log (z_1)-\mu_1)^2}{2 \sigma_1^2}\right)}{2 \pi  \sigma_1 \sigma_2 z_1}dz_1$$
So with (likely) no nice closed-form for this integral we need to evaluate it numerically.  And as @whuber more than hints at in a comment a direct attack results in numerical instability because of underflow while evaluating the integrand.  We can avoid this and end up with a reasonable approximation of the density by determining the limits of integration not associated with underflow and even avoiding areas where the integrand is "close enough to zero."
Some R code below gives functions for the maximum likelihood estimates along with estimates of the associated standard errors.  Note that to get all parameters roughly the same order of magnitude I'm using $\log(\sigma_1)$ and $\log(\sigma_2)$ rather than $\sigma_1$ and $\sigma_2$, respectively, which also automatically enforces the restriction that the estimates of $\sigma_1$ and $\sigma_2$ will be positive.
f <- function(z1, x, mu1, logsigma1, mu2, logsigma2)  {
  # Integrand for determining pdf of X = Z1 + Z2
  exp(-logsigma1 - logsigma2 - (x - z1 - mu2)**2/(2*exp(2*logsigma2)) -
    (-mu1 + log(z1))^2/(2*exp(2*logsigma1)))/(2*pi*z1)
}

logpdf <- function(x, mu1, logsigma1, mu2, logsigma2) {
  # Log of the pdf of X = Z1 + Z2
  lower <- max(10^-20, x - 4*exp(logsigma2))
  upper <- 4*exp(logsigma2) + max(0, x)
  result <- integrate(f, lower, upper, x=x, mu1=mu1, logsigma1=logsigma1, mu2=mu2,
    logsigma2=logsigma2, subdivisions=100)[1]
  if (result$value == 0) {
     result$value <- -100
  } else {
     result$value <- log(result$value)
  }
  result$value
}

logL <- function(parms, d) {
  mu1 <- parms[1]
  logsigma1 <- parms[2]
  mu2 <- parms[3]
  logsigma2 <- parms[4]
  sum(unlist(lapply(d, logpdf, mu1=mu1, logsigma1=logsigma1, mu2=mu2, logsigma2=logsigma2)))

}
mle <- function(d, inits) {
  # Function that finds maximum likelihood estimates and standard errors
  mle.optim <- optim(inits, logL, d=d, control=list(fnscale=-1), hessian=TRUE)
  # Covariance matrix
  covmat <- -solve(mle.optim$hessian)
  list(mle=mle.optim, covmat=covmat, sd=diag(covmat)^0.5)
}

To use those functions consider the OP's two examples:
# Generate a random sample
  n <- 10000
  set.seed(12345)
  d <- exp(rnorm(n, -3, 3)) + rnorm(n, 0, 10^-3)

# Find maximum likelihood estimates
  system.time(results1 <- mle(d, c(-3, log(3), 0, log(10^(-3)))))
  #    user  system elapsed
  #   88.16    1.74   90.14
  # Estimates of mu1, sigma1, mu2, and sigma2
  with(results1$mle, c(par[1], exp(par[2]), par[3], exp(par[4])))
  # [1] -2.987227e+00  2.974923e+00 -4.195435e-05  1.006057e-03
  # Standard errors of mu1, logsigma1, mu2, and logsigma2
  results1$sd
  # [1] 0.0311040500 0.0089383883 0.0000552333 0.0445809578

# Generate a random sample
  n <- 1000
  d <- exp(rnorm(n, -9, 3)) + rnorm(n, 0, 10^-6)

# Find maximum likelihood estimates
  system.time(results2 <- mle(d, c(-9, log(3), 0, log(10^(-6)))))
#   user  system elapsed
#   8.37    0.10    8.47
# Estimates of mu1, sigma1, mu2, and sigma2
  with(results2$mle, c(par[1], exp(par[2]), par[3], exp(par[4])))
# [1] -8.967050e+00  1.099405e+00 -1.212325e-07 -1.380606e+01
# Standard errors of mu1, logsigma1, mu2, and logsigma2
# [1] 9.576075e-02 2.406230e-02 4.335842e-06 1.345771e-01

I have taken the liberty to use the true values of the parameters for starting values.  Your mileage may vary.
If either amount of computer time is larger than needed (as opposed to desired), then one can take advantage of the fact that the pdf of $X$ is relatively smooth.  One could use interpolation from a subset of values in the first 90% of the data and a regression with another subset of the upper 10% of the data (with the regression being of the form $a_0+a_1 \log(x) + a_2 (\log(x))^2$).  That would considerably reduce the number of calls to the numerical integration routine.


memory size error linux container

Question: container name: vm1
config:
lxc.tty = 4
lxc.pts = 1024
lxc.rootfs = /var/lib/lxc/vm1/rootfs
lxc.mount.entry = sysfs /var/lib/lxc/vm1/rootfs/sys sysfs defaults  0 0
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.veth.pair = veth_vm1
lxc.cgroup.memory.limit_in_bytes = 512M
lxc.cgroup.memory.memsw.limit_in_bytes = 1G

lxc.utsname = vm1

lxc.cgroup.devices.deny = a
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 254:0 rwm
lxc.cgroup.devices.allow = c 10:200 rwm

I get the following error when I start the container? Could it be the memory setting?
$ lxc-start -n vm1

// error
lxc-start: cgroup is not mounted
lxc-start: failed to setup the cgroups for 'vm1'
lxc-start: failed to setup the container
lxc-start: invalid sequence number 1. expected 2
lxc-start: failed to spawn 'vm1'

Answer: You need to have the cgroup filesystem mounted. If this is RHEL/CentOS, make sure you have libcgroup installed.

memory size error linux container


How to retrieve the client certificate from a request in a REST service using Quarkus in two way SSL

Question: Following Quarkus Getting Started guide and enabling SSL the next step I wanted to do was to get the client certificate chain.
I would like to do something like this:
<code>    private X509Certificate extractCertificate(HttpServletRequest req) {
        X509Certificate[] certs = (X509Certificate[])
        req.getAttribute("javax.servlet.request.X509Certificate");
        if (null != certs && certs.length > 0) {
            return certs[0];
        }
        throw new RuntimeException("No X.509 client certificate found in request");
     }
</code>
Following the getting started guide injecting HttpServletRequest is not straight forward as described in this issue
How would be the way to have access to the client certificate chain then?
Comment: Sure @chrylis-onstrike- , and what I need to do is under a very rare, specialized circumstance.
Comment: You are aware that client certificates are only used in rare, specialized circumstances?
Answer: After doing some research I ended up with the following solution using class io.vertx.core.http.HttpServerRequest
<code>private X509Certificate extractCertificate(HttpServerRequest req) throws SSLPeerUnverifiedException {
    X509Certificate[] certs = req.connection().peerCertificateChain();
    if (null != certs && certs.length > 0) {
        return certs[0];
    }
    throw new RuntimeException("No X.509 client certificate found in request");
}
</code>


ADB não reconhece Android 4.2.2 no ubuntu

Question: Estou a trabalhar em ubuntu 13.04, e acabei de receber um tablet com Android 4.2.2 diretamente dum fabricante e não estou a conseguir que o ADB reconheça o tablet, e consequentemente não consigo fazer debug.
Já tentei:

reiniciar o Android e o PC;
reiniciar o ADB (alias quase todos os comandos do ADB);
o Android tem a depuração USB ligada;

Já testei, com diferentes SDK, em ubuntu, em Windows XP, Windows 7 e Windows 8 e nada.
Em windows se for ao gestor de dispositivos ele aparece com o ponto de exclamação.
Vi aqui que precisava de definir as regras undev e segui estes passos:
# lsusb

Bus 001 Device 011: ID 1f3a:100b

# sudo gedit /etc/udev/rules.d/99-android.rules

SUBSYSTEM=="usb", ATTR{idVendor}=="1f3a", ATTR{idProduct}=="100b",
  MODE="0666", OWNER="meu-login"

# sudo service udev restart

Mas continua sem reconhecer o dispositivo.
Fonte ubuntu-precise-install-android-sdk
Alguém imagina o que poderá ser?
Comment: Tipo, a tabela do Google não fornece o `idVendor` pra RFTECH. Tente entrar em contato com o suporte deles pra ver se eles disponibilizam esse `id` pra você. Me parece que o problema pode ser esse.
Comment: Tentou criar o arquivo com o prefixo `51` em vez de `99`? Caso negativo, tente criar mesmo que fique duplicado.
Comment: Sim já tentei com os dois, 51 e 99.
Comment: Sim eu sei, provavelmente será mesmo esse o caso, mas estou na esperança que possa ser outra coisa.
Comment: Tenta realizar os passos recomendados no windows, se não funcionar talvez só entrando em contato com o suporte mesmo.
Answer: 1. Ubuntu
Você deve ir como sudo até o diretório /etc/udev/rules.d/ e editar o arquivo 51-android.rules (se não existir, crie).
Adicione essa linha:
SUBSYSTEM=="usb", ATTR{idVendor}=="0bb4", MODE="0666", GROUP="plugdev"

Substituindo o {idVendor} pelo valor na tabela que ele fornece no site.
Ao final, use: chmod a+r /etc/udev/rules.d/51-android.rules, tire e plug o device novamente.
Além disso, verifique que está rodando tudo com as versões mais novas.
2. Windows*
Se ao instalar o Driver do fabricante e não funcionar, você vai precisar realizar alguns passos.
No gerenciador de dispositivos, procure a seção Android Phone ou Telefone Android e expanda.
Clique com botão direito em Android Composite ADB Interface e selecione Update Drivers.
Selecione Install from a list or specific location (Instalar de uma lista ou local específico)
Escolha o Driver do Google (Google USB Driver) que está localizado em caminho_para_o_sdk\extras\google\usb_driver\ e continue.
*Alguns desses passos podem ser um pouco diferentes dependendo da versão, qualquer coisa consulte a fonte com instruções específicas para cada versão
Fonte: Using Hardware Devices e OEM USB Drivers.
Comment: Ainda agora mesmo coloquei o teu primeiro ponto na edição, ainda nem tinha visto a tua resposta, mas a questão é que continua sem funcionar.
Comment: @JorgeB., qual o fabricante do tablet?
Comment: [RFTECH](http://rftech.com.cn) rsrsrs


Why does certbot suddenly fall back to http-01 challenge instead of tls-sni-01 (https)?

Question: I have a well-tested bash script that was running fine until a few weeks ago.
It seems that certbot challenge defaults now to http instead of https. I can't figure out the reason.
echo 'rsa-key-size = 2048
renew-hook = /usr/sbin/ipsec reload && /usr/sbin/ipsec secrets
' > /etc/letsencrypt/cli.ini

certbot certonly --non-interactive --agree-tos --email $EMAIL --standalone -d $VPNHOST

Firewall settings:
Firewall stopped and disabled on system startup
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere             state NEW recent: UPDATE seconds: 60 hit_count: 12 name: DEFAULT side: source mask: 255.255.255.255
           all  --  anywhere             anywhere             state NEW recent: SET name: DEFAULT side: source mask: 255.255.255.255
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:https
ACCEPT     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             state INVALID
ACCEPT     udp  --  anywhere             anywhere             udp dpt:isakmp
ACCEPT     udp  --  anywhere             anywhere             udp dpt:ipsec-nat-t
DROP       all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  ip-10-10-10-0.eu-west-2.compute.internal/24  anywhere             policy match dir in pol ipsec proto esp
ACCEPT     all  --  anywhere             ip-10-10-10-0.eu-west-2.compute.internal/24  policy match dir out pol ipsec proto esp
DROP       all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Certbot:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for vpn-1.duelify.com
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. vpn-1.duelify.com (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://vpn-1.duelify.com/.well-known/acme-challenge/BsPR5NK4JVb8rgIyS2VR4VTOv4Bc1F5TNppjoo95O70: Timeout

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: vpn-1.duelify.com
   Type:   connection
   Detail: Fetching
   http://vpn-1.duelify.com/.well-known/acme-challenge/BsPR5NK4JVb8rgIyS2VR4VTOv4Bc1F5TNppjoo95O70:
   Timeout

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

Answer: It could be due to the recent vulnerability announced with the tls-sni-01 verification method, Let's Encrypt disabled this until it's been fixed patched.
Until then, they have advised using http for verification.
Lets Encrypt tls-sni-01 bug details


use web form master page as master page for ASP.Net MVC views programatically

Question: We have a running solution contains many web forms application with the same master page, we created a class library project for master page and its controls and reference it as a dll inside each web form application and assign master page programatically
Now we will add two new application to our solution but we decided to use ASP.Net MVC 3 for building these new applications.
The problem is, we must use the same master page for the new applications, so how can we set the master page for our views using code?
Take care to remember our master page is a class library project.
Answer: Look at this articles to get an overview about mixing Webforms with MVC:

Mixing Razor Views and WebForms Master Pages with ASP.NET MVC 3
Using Razor Pages with WebForms Master Pages

Another interesting project is Zeus when it comes to a mixed Webforms/MVC solution.
It contains helpers to use HtmlHelper and UrlHelper on Webform pages.
Comment: thanks for great posts but it talk about master page inside web project not as class library project(dll).
Comment: This post talks about sharing a master page between ASPX engine and Razor Engine. Will this be the same approach to share a regular ASP.Net Mater page with ASP.Net MVC 3.0 Razor Engine project?
Comment: @HaBo: I don't understand your question exactly. The posts talk about using an Webforms Masterpage with ASP.NET MVC Razor Views.
Comment: Sorry for Confusing. I am clear now. Thanks for the information
Comment: I tried "Using Razor Pages with WebForms Master Pages" and it works, but it is a .aspx page, My pages are in cshtml(MVC views).

How can I use web form master page(Site.Master) in MVC views cshtml page.
Answer: If you really want to go the MVC route moving forward then you should probably invest some time to convert your master page to a layout. Everything else would be a hack/workaround.
Answer: Amir, I uploaded a working sample at here. Steps,
1) Downloaded and converted a sample application from here.
2) Add a MVC 3 application in this solution and Add reference the class library of above solution.
3) Specify the MasterPage in MVC view(either in view, MasterPageFile="~/MasterPageDir/MasterPage.master" or in controller return View("Index",masterName: MasterPageVirtualPathProvider.MasterPageFileLocation);).
4) Put these lines in global.asax.
MasterPageVirtualPathProvider vpp = new MasterPageVirtualPathProvider();
HostingEnvironment.RegisterVirtualPathProvider(vpp);

Comment: +1 for you answer but it depends on `VirtualPathProvider`. unfortunately our old project doesn't use it :(. It seems I have to copy master page Html into my `_layout` view and make the styles and images refer to production version(online).


Computing Mallows' Cp with SAS

Question: I have a question about SAS:
How do you get the parameter estimate for Mallows' Cp procedure in SAS?
Comment: Anything missing in the [PROC REG](http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_reg_sect007.htm) documentation?
Answer: You can use the CP option on the MODEL statement in PROC REG. This writes the Cp statistic to the OUTEST= data set:
<code>proc reg data=sashelp.class outest=est;
model weight = height / cp;
run;
proc print data=est; run;
</code>
See http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_reg_sect013.htm

Computing Mallows' Cp with SAS


c#-minor Vs E-major for Moonlight Sonata

Question: I see Moonlight sonata often referred to as being in the key signature of c#-minor.
I am beginning to learn music and just learning about key signatures and I would have referred to the key signature as E-major.
Is there a difference (i.e. they are en-harmonic)?  Does it matter which reference is used?
How do you tell which should be used?
Comment: The easiest way to see whether a key is Major or minor is to look at the leading tone of the relative minor and see if it is raised, In this case to we see a B# or a B natural.
Comment: related question: [Hard to tell major from minor](https://music.stackexchange.com/questions/15866/hard-to-tell-major-from-minor)
Answer: Both C# minor and E major keys have the same key signature, so there is no difference there. This relationship is called 'relative key'. Each major key has a relative minor one, with the same key signature (to find it, descend a minor 3rd or ascend a major 6th from your tonic). Similarly for the minor key.
To sum up the difference: These two keys have the same key signature but different tonics.
The E major key would revolve around the note E, whereas the C# minor key would revolve around C#.

Does it matter which reference is used?

Of course it matters. A minor key sounds different than a major one, no matter what notes are included. So, you need to revolve around the minor harmony and not the major.
The relative key is not to be confused with the parallel key. The latter is when two scales have the same tonic, but one is a major one and the other is a minor one.

How do you tell which should be used?

Υοu see the harmony and the melody. In a song that is in C# minor, you are bound to see B# (leading tone) leading to C# (melody line). On the chords (harmony line) of the song you'll see V-i relationships, like G#-C#m etc. Things that will establish that the song is in C#minor.
Moreover, I'd like to add something concerning your enharmonic reference. Enharmonic note (or key signature)

is a note, interval, or key signature that is equivalent to some other note, interval, or key signature but "spelled", or named differently

A simple example: This example would be really easy to understand on a guitar or piano. C and D are a tone apart. If you ascend a semitone from C, you get C#; if you descend a semitone from D, you get Db. But both ways got you to the same note, each time spelled different. These two notes (C# and Db) are enharmonic notes.
Comment: Thanks for your answer, just to make sure I have understood: Using just the key signature it could be either scale.  By looking at the composition of the music I can deduce that in this case it is c#-minor.  Getting this wrong doesn't affect playing the piece (the notes will still all be correct - sharpened by the key sig) but is relevant to understand the intent of the composition?
Comment: Yes, you can play the song without knowing what scale it belongs in, as long as you play the notes correctly. But understanding the theory behind the piece (including but not limited to what scale the piece is in), is like understanding what a book is about and not simply reading words that are in a book.


comparing gaussian mixture models with and without regularization

Question: I am using Gaussian mixture models for clustering a bunch of data sets.
On some data sets a regularization value is often used/suggested (see the MATLAB example for Fisher's Iris data which sets RegularizationValue = 0.01).
I could not find a systematic way to set such a regularization value for a given data set.
My question is, is it valid to compare two Gaussian mixture models (e.g. using their negative log-likelihood or criterion like AIC/BIC) one of which uses a non-zero regularization value and another does not?
Answer: If I recall correctly, Matlab's GMM regularization works by adding a multiple of the identity matrix to the covariance matrices.
Using regularization can be viewed as placing a prior on the covariance matrices. As the strength of regularization increases, the model becomes less flexible. As a consequence, the training set log likelihood may be lower for more strongly regularized models. Because of this, training set log likelihood isn't a great way to compare models with different levels of regularization.
AIC and BIC penalize more flexible models, so this is a step in the right direction. They do this by measuring the number of free parameters, and are straightforward to use for comparing models with different number of mixture components and different types of covariance matrices (tied, diagonal, full, etc.). In these cases, the number of free parameters can simply be counted. The situation is more complicated for regularized covariance matrices, because model flexibility is governed by a continuous regularization parameter, rather than a change in the number of parameters. It may be possible to find an expression for the 'effective degrees of freedom' as a function of the regularization parameter. For example, this can be done in the case of ridge regression.
Two alternatives come to mind (I'm assuming you're holding the number of components, type of covariance matrix, etc. fixed, and only changing the regularization parameter). 1) Train each model on a subset of your data, then compute the log likelihood on the remaining test set (better yet, use cross validation). Choose the regularization parameter that maximizes test set likelihood. 2) Use Bayesian model comparison. Choose the regularization parameter that maximizes the evidence. This is a more complicated approach, because you'd have to define priors on all parameters, derive an expression for the evidence, and find a way to compute or approximate it.
One reason to prefer cross validation/Bayesian model comparison over AIC/BIC is that AIC/BIC rely on asymptotic approximations, which may or may not be appropriate for your data. Note that cross validation is not a valid strategy for choosing the number of mixture components, because the test set likelihood may increase monotonically with the number of components.
Comment: thanks for the answer. I have two additional queries (perhaps they will qualify as a separate question). (1) Can we count regularization as reducing number of free parameters by some amount? (2) The MATLAB regularization (and any other similar tools) how is the regularization implemented, does it add it to the diagonal before every inversion?
Comment: @user20160 : you specifically mentioned that CV shall not be used to choose the number of mixture components. How would you suggest to choose the number of mixture compolents?


Send Confirmation Email with Google Forms / Spreadsheets

Question: I know that Google Spreadsheets includes a mass mailout option with its MailApp.sendEmail function, but, as far as I know, it can only run once a form is opened, or by manually clicking "run".
I have a Google form which includes an area for someone to put their email address.  Is there a simple solution to send this data to its Google Spreadsheet and send it as a confirmation to the specified email address?  It would be great if I could somehow incorporate MailApp.sendEmail into the form, instead of into the spreadsheet (which would be after the fact and manual, instead of instant and automatic).
Also, note that I will be posting with jQuery.
Any ideas?
Answer: I created a php script to process the php form fields and send them back to google.  On the php code you can add add captcha, field validation, email confirmation messages, even attachments.
http://www.jazzerup.com/blog/item/googleforms
Answer: The solution is to have multiple actions with Javascript.
I found this on http://www.codeproject.com/KB/scripting/multiact.aspx. Here's the code:
<form name=Form1 action= "login1.php" method=post>
Username <input type="text" name="username">
Password <input type="password" name="password">
<input type="submit" value="Send" name="button1"
                     onclick="return OnButton1();">

One of the iframes can point to a Google form and one can point to your own PHP file that can handle the data (i.e. send an email to the person).  Doing it this way can also allow you to handle the "Thank you" page too, instead of sending them to Google's thank you page.


Grails cascade validation for domain objects not mapped to a table

Question: Please can you help me with the problem below:
I have 2 domain classes(Parent, Child) which I do not want to be mapped to a table, so I put mapWith=none. However, when I do parent.validate() I want the validation to be cascaded to the child. How can I enable cascade validation for domain objects which are not mapped to a table?
Many thanks in advance!
Answer: Don't know if you can by design.
You could optionally add a customValidation() method on the parent object to initiate a check which is looping over the children and invoking their validate() methods. Any errors on the children could then be added to the errors object of the parent object.
boolean cascadedValidation() {
    this.validate();

    children.each {
        if (!it.validate()) {
           it.errors.allErrors.each { err ->
               // Bind somewhere on parent object
           }
        }
    }

    return this.hasErrors();
}

Comment: thanks a lot for your solution! I was thinking to do something similar  too, but I am not very happy as it's more a bandaid.... :( I saw in GormValidationApi an if statement which tests if the current validator is instance of CascadeValidator...I'm trying to find a way to inject an instance of CascadeValidator to that validate Gorm method so that it will force to cascade.
Comment: there must be an workaround or a solution for this, cannot believe you can't cascade validation unless you map the domain to a table!
Comment: Maybe this post is of use: http://stackoverflow.com/questions/1040596/grails-validation-of-a-list-objects


Diferença entre get_context_data e get_queryset e melhoria de códigos (Django)

Question: eu queria saber a diferença entre get_context_data e get_queryset.
E gostaria de saber a opinião de vocês sobre os meus códigos.
O que está certo e o que está errado neles?
https://github.com/rg3915/vendas/blob/master/vendas_project/vendas/views.py#L121-L132
class SaleDetailView(DetailView):
    template_name = 'vendas/sale/sale_detail.html'
    model = Sale

    def get_context_data(self, **kwargs):
        s = Sale.objects.get(pk=self.kwargs['pk'])
        sd = SaleDetail.objects.all().filter(sale=s)
        context = super(SaleDetailView, self).get_context_data(**kwargs)
        context['count'] = sd.count()
        context['Sale'] = s
        context['Itens'] = sd
        return context

https://github.com/rg3915/vendas/blob/master/vendas_project/vendas/views.py#L74-L88
def get_queryset(self):
    p = Product.objects.all()
    q = self.request.GET.get('search_box')
    # buscar por produto
    if q is not None:
        p = p.filter(product__icontains=q)
    # filtra produtos em baixo estoque
    if self.request.GET.get('filter_link', False):
        p = p.filter(stock__lt=F('stock_min'))
    return p

O que pode ser melhorado nesses códigos?
Answer: Esses métodos têm propósitos totalmente diferentes:

get_context_data fornece os dados do contexto a ser usado ao renderizar um template. Isso não envolve necessariamente o ORM, de modo que você poderia por exemplo criar um view que não consultasse nenhuma tabela do seu BD, apenas reunisse alguns parâmetros de alguma outra fonte e os colocasse num formato apropriado para serem usados no template.
get_queryset tem como objetivo criar um QuerySet "básico" que pode ou não ser "refinado" pelo código consumidor. Ou seja, se você tem alguma operação que quer fazer a todos os query sets antes dos mesmos serem usados (em um projeto passado eu pré-filtrava os modelos baseados num campo "tenant") você pode fazê-lo nesse método. O que será feito com o resultado depois, depende do caso (em um ListView, ele vai acabar sendo usado também em um template, mas outros usos sem envolver templates também seriam possíveis).

Eu não tenho experiência com views genéricas para opinar sobre seus códigos, mas à primeira vista não há nenhum problema nos mesmos, nem nada que pudesse ser melhorado. Talvez fosse possível usar uma única query para consultar a venda e seus itens, em vez de duas queries como está sendo feito, mas não tenho certeza se é possível e/ou se traria alguma vantagem significativa na performance. E o primeiro exemplo me parece que falhará se não houver venda com a chave primária consultada, você está tratando essa possibilidade de alguma forma? (i.e. para a exceção não propagar e resultar num erro 500)
Comment: Obrigado pela opinião @mgibsonbr


Getting exception during calling API by Eureka server

Question: I've created the microservices with spring boot getting the exception during calling the URL http://localhost:8765/CURRENCY-EXCHANGE/currency-exchange/from/USD/to/INR from the local server
exception details:
java.net.UnknownHostException: failed to resolve 'DESKTOP-HG47DMC' after 2 queries
at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: Error has been observed at the following site(s):
|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
|_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
|_ checkpoint ⇢ HTTP GET "/CURRENCY-EXCHANGE/currency-exchange/from/USD/to/INR" [ExceptionHandlingWebHandler] Stack trace:
    at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:966) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:414) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.resolver.dns.DnsResolveContext.onResponse(DnsResolveContext.java:625) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.resolver.dns.DnsResolveContext.access$400(DnsResolveContext.java:63) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:458) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) [netty-common-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) [netty-common-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) [netty-common-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) [netty-common-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) [netty-common-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:605) [netty-common-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) [netty-common-4.1.63.Final.jar:4.1.63.Final]
    at io.netty.resolver.dns.DnsQueryContext.trySuccess(DnsQueryContext.java:201) ~[netty-resolver-dns-4.1.63.Final.jar:4.1.63.Final]

here is my application.properties
spring.application.name = api-gateway
server.port = 8765
eureka.client.serviceUrl.defaultZone =http://localhost:8761/eureka
spring.cloud.gateway.discovery.locator.enabled= true

Answer: Add this code snippet to your application.properties file of your microservices.
eureka.instance.hostname=localhost

Answer: Add this code snippet to your application.properties file of all your microservices(Naming server, API-Gatway and currency-exchange).
eureka.instance.hostname=localhost
if you get 404 response then please wait for 30 - 60 sec.


WinForm application behaves differently in different system

Question: I have winform application with many controls but the main control is tree on form. The application is working fine in my system but when this was tested in another system this behaves differntly. Tree size got changed and overlapped other controls and hence some of the controls are not visible. This is surprising for me. What could be the issue? I verified DPI setting and msinfo32 but could not see any differnce in both the systems.
Comment: Can you give more info about it? What size are you using for your form? Are you using a custom font? Those can cause problems.
Comment: Install .net framework 4.7.  this will correct the behavior.  Re-target to that version recompile.
Comment: If there is a problem with .NET version then I could see the issue in both the systems and not in one system.I am using VS 2013 and all my projects are built with .NET Framework 4. Both the systems have same configuration. Tree control is 3rd party control.
Comment: Form font is Microsoft Sans Serif, 8.25pt and size is 535, 648.
Comment: Any comment from anyone?
Comment: `...when this was tested in another system this behaves differntly` you need to provide detailed descriptions of both systems. What's different about the second? Also, you need to add more info on the problem itself: which controls malfunction? how? Ideally, you need to provide a [reproducible example](https://stackoverflow.com/help/mcve)
Comment: I solved this issue. This was problem with DPI. The application was developed with DPI 125% and hence controls are overlapping wherever DPI was 100%. Any app must be developed with 100% DPI to avoid such issues.
Answer: I found the issue with DPI only. Application was developed with 125% DPI and that's why controls were overlapping in other system where default DPI was 100%.


Custom Verify.js validation rule not applying

Question: I'm using a new plugin to validate form fields called Verify.js, everything has been working great, until I started trying to create my own custom validation rules.
Here is the link to their documentation, where it explains how to create custom validations: http://verifyjs.com/#custom-rules
Even more helpful, is this example posted on JSfiddle by the plugin author: http://jsfiddle.net/jpillora/R4t84/1/ I copied the code format here almost precisely, yet it still doesn't work.
Right now, I am trying to get the form to not submit if the user has not selected the input field #YesExact. It is a very simple form validation, just trying to get this to work so I get the hang of writing custom validation rules with Verify.js
The HTML markup for the form field is like this:
p class="regular-content option-label">Do you know the exact size of your order?

    <input name="YesExact" id="YesExact" data-validate="checkExact" type="button" class="btn btn-tca padding-button" value="Yes">

The javascript I wrote that has the custom validation rule is written like this:
// custom form validation
$.verify.addRules({
    checkExact: function (r) {
        if ($('#YesExact').val() === "") {
            $('#YesExact').notify("Please tell us you know the exact amount!", "error");
            return "Please tell us you know the exact amount!" false;
        }
        return true;
    }
});

This looks like it should totally work, I followed the example very closely. However, I have one major clue as to why it doesn't work... the javascript console says:
verify.js: Missing rule: checkExact
So how am I supposed to fix this? Where Verify.js can find the rule?
The webpage is HERE if you want to go check it out for yourself.
Thanks in advance for your help.
Answer: You have a syntax error at:
return "Please tell us you know the exact amount!" false;

change to return false;
You should also make sure to only set its value when clicked, as well as the "No" button, and then check if either are set, otherwise your validation will always return true. Though a better way in my opinion would be to have their clicks trigger the setting of a hidden field to either true or false which you could easily pull as a boolean server side.
Answer: I fixed this problem by including the verify.js script at the bottom of the file just before the closing body tag, assuming that your jquery file is also included before these below statements, and assuming that inputs are properly enclosed in <form> tag


Basic Authentication with htaccess but without password file

Question: Can I use htaccess to restrict access to my site and specify the username/password within the .htaccess, so I don't have to worry about password files?
Answer: You can absolutely use .htaccess to restrict access, but I don't know of any way to avoid creating an equivalent .htpasswd file if you want to control access via users and passwords, rather than whitelisted IP addresses or similar.
Now, you can create any number of .htpasswd files; so if you wanted a specific one for a specific directory, that would be easy to set up by creating the .htpasswd file using:
<code>htpasswd /usr/local/etc/apache/.your_htpasswd_filename_here username_here
</code>
then specifying it using the "AuthUserFile" directive in your .htaccess file.
Answer: It would not be that wise to have the users and passwords inside the .htaccess file (or the apache-config itself) for min. two main reasons.
See, the .htaccess lies inside a directory of your website. The password file should never lie there, because everyone could then download it and has a complete list of valid usernames. And can brute force their passwords offline.
Another reason could be that you could have a large number of users/groups for a BASIC auth protected website. You do not want to mix server config and authentication database.
Comment: Apache should never let anyone download the .htaccess file, or you have problems. Normal httpd.conf has a directory match for all .htaccess files with deny all. How easy is it to get the .htaccess file from outside the machine?

If you only need one user/password for a given path, it won't be hard to maintain.

If the username is not a shell account to the machine then the web data it protects is the only risk if the .htaccess file is compromised.


Modularised terraform code wants to destroy and recreate an S3 bucket

Question: This is a simplified example of my problem, for the sake of understanding.
My team used to manage an S3 bucket through the module1, which invoked module2, which contained the code of the bucket.
production_account/ --(invokes)--> module1/ --(invokes)--> module2/s3.tf

But then they decided to refactor the code, and directly invoke module2.
production_account/ --(invokes)--> module2/s3.tf

As a result, when I terraform plan now I get:
module.module1.module.module2.aws_s3_bucket.bucket will be destroyed {
...
}

module.module2.aws_s3_bucket.bucket will be created {
...
}

In short, terraform wants to recreate the s3 bucket. The problem with this is that all data will be lost.
Is there a way to import the existing bucket to the state and to manage it through module 2?
terraform import module.module1.module.module2.aws_s3_bucket.bucket

solves the first half of my question.
Answer: Yes. Use terraform state mv not import.
terraform state mv module.module1.module2.aws_s3_bucket.bucket module.module2.aws_s3_bucket.bucket

Comment: Did you already `import` it?
Comment: If so, you'll need to use `terraform state rm module.module2.aws_s3_bucket.bucket` to forget it from that state location first.
Comment: If you do `terraform state list | grep "aws_s3_bucket.bucket"`, does it exist in your state anywhere? If so, you should be able to move it.
Comment: Okay, but the resource should still be there, so as long as it is in neither the old state address or the new state address, you should be clear to import it into the new location and plan.
Comment: That is correct. Thank you @theherk
Comment: Did it and I get `module.module2.aws_s3_bucket.bucket must be replaced`. "Replace" in Terraform's terminology means to destroy the existing object and create a new one to replace it. Doesn't seem to solve the problem.
Comment: I did not import it before executing the `state mv`, no. Now, I executed `terraform state rm module.module2.aws_s3_bucket.bucket` as you mentioned and it just wants to create a new bucket without destroying the old nowmore
Comment: It does not exist! I used `terraform state rm`.
Comment: I executed the `state mv` command and it asked to replace. Then I executed `terraform rm` and now its gone.
Comment: Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/238512/discussion-between-dimitris-and-theherk).


Understanding the Limitations of GEE

Question: Review of GEE: Generalized Estimating Equations (GEE) are used to estimate the parameters of a generalized linear model. The main advantage of GEE is that they essentially allow you to model a multivariate correlated random variable, without knowing the exact form of this random variable (this is really useful because its really difficult to write a multivariate correlated non-random variable). GEE works through the Quasi-Likelihood function - by only knowing some relative moment based properties between the response and the covariates, we can still estimate model parameters, and these model parameters will still be consistent, unbiased and asymptotic normal (even we pick the wrong correlation structure!). The only drawback is however that GEE can only estimate population level parameters and not individual level parameters. I am trying to understand why this is.
The quasi-likelihood function for GEE is given by:
$$ Q(\beta; y) = \sum_{i=1}^{n} (y_i - \mu_i(\beta))^T V_i^{-1}(\beta, \alpha) (y_i - \mu_i(\beta))$$
where:

$y_i$ is the response for the $i$-th subject,
$\mu_i(\beta)$ is the mean response for the $i$-th subject,
$V_i(\beta, \alpha)$ is the variance function for the $i$-th subject,
$\beta$ are the fixed effects parameters, and
$\alpha$ are the dispersion parameters.

The GEE are obtained by solving (where $D_i$ is the derivative of $\mu_i(\beta)$ with respect to $\beta$):
$$\frac{d}{d\beta} Q(\beta; y) = U(\beta) = \sum_{i=1}^{n} D_i^T V_i^{-1} (y_i - \mu_i(\beta)) = 0$$
Question: However, reading the above equations, it is unclear to me what happens if I would trick/force GEE to estimate individual level parameters instead of population level parameters. I tried to create a proof-of-concept problem to see what can go wrong if I use GEE to estimate individual level parameters.
Suppose we start with a Simple Longitudinal Model for the $i$-th person in the $j$-th group ( $y_{ij}$ is the response, $x_{ij}$ is the covariate, $b_{0ij}$ and $b_{1ij}$ are the individual-specific intercept and slope, and $\epsilon_{ij}$ is the error term):
$$y_{ij} = b_{0ij} + x_{ij}b_{1ij} + \epsilon_{ij}$$
$$b_{0i} = b_{0} + u_{0i}$$
If we write the GEE :
$$\mu_{ij}(\beta) =  b_{0ij} + x_{ij}b_{1ij}$$
$$D_{ij} = \frac{\partial \mu_{ij}(\beta)}{\partial \beta} = (1, x_{ij})^T$$
$$U(\beta) = \sum_{i=1}^{n} D_i^T V_i^{-1} (y_i - \mu_i(\beta)) = 0$$
$$U(\beta) = \sum_{i=1}^{n} \sum_{j=1}^{m} (1, x_{ij})^T V_{ij}^{-1} (y_{ij} - b_{0ij} - x_{ij}b_{1ij}) = 0$$
Problem: Perhaps I can show that GEE provides unbiased Population estimates but biased Individual estimates?

Population Level Bias: As bias is linked to the Expected Value, we can write:

$$U(\beta) = \sum_{i=1}^{n} D_i^T V_i^{-1} (y_i - \mu_i(\beta)) = 0$$
$$E[U(\beta)] = E\left[\sum_{i=1}^{n} D_i^T V_i^{-1} (y_i - \mu_i(\beta))\right]$$
Substituting $y_i = \mu_i(\beta) + \epsilon_i$, where $\epsilon_i$ is the error term, we get:
$$E[U(\beta)] = E\left[\sum_{i=1}^{n} D_i^T V_i^{-1} \epsilon_i\right]$$
If $\epsilon_i$ is independent of $D_i^T$, then $E[U(\beta)] = 0$ and the estimates are unbiased.

Individual Level Bias:

$$E[U(\beta)] = E\left[\sum_{i=1}^{n} D_i^T V_i^{-1} (y_i - \mu_i(\beta))\right]$$
Substituting $y_{ij} = b_{0ij} + x_{ij}b_{1ij} + \epsilon_{ij}$ and $b_{0i} = b_{0} + u_{0i}$, we get:
$$E[U(\beta)] = E\left[\sum_{i=1}^{n} D_i^T V_i^{-1} \epsilon_{ij}\right]$$
If $\epsilon_{ij}$ is independent of $D_i^T$, then $E[U(\beta)] = 0$ and the estimates are STILL unbiased???
Thus it seems like GEE is just as suitable for estimating Individual parameters compared to Population parameters (contradiction). Where have I made a mistake?

Note: Independence of $\epsilon_i$ and $D_i^T$ justification:

$$E[U(\beta)] = E\left[\sum_{i=1}^{n} D_i^T V_i^{-1} \epsilon_i\right]$$
$$E[U(\beta)] = \sum_{i=1}^{n} E[D_i^T V_i^{-1} \epsilon_i]$$
Since $\epsilon_i$ is independent of $D_i^T$, we can say ($D_i$ and $V_i$ are fixed quantities):
$$E[D_i^T V_i^{-1} \epsilon_i] = D_i^T V_i^{-1} E[\epsilon_i]$$
Assuming that $E[\epsilon_i] = 0$, we get:
$$E[U(\beta)] = \sum_{i=1}^{n} D_i^T V_i^{-1} \cdot 0 = 0$$
Comment: When you have a nonlinear link function, GEEs give coefficients with a population-averaged interpretation, whereas mixed models give a subject-specific interpretation. More details on this here: https://stats.stackexchange.com/questions/365907/interpretation-of-fixed-effects-from-mixed-effect-logistic-regression/365918#365918
Answer: There are two sorts of problems that will arise: general ones and boring technical ones for specific cases.  I will start with the general one, where we can simplify to the independence working model and the identity link -- linear regression.
Suppose you have longitudinal data with $t=1,\dots,T_n$ observations on person $n$, and people $n=1,\dots,N$.  You have an outcome $Y_{nt}$ on person $n$ at time $t$ and a single predictor $X_{nt}$. You might fit a model A
$$E[Y_{nt}|X_{nt}]=\alpha+\beta X_{nt}$$
In this model, $\beta$ is an average difference in $Y$ per unit difference in $X$, averaged over times and people, and $\alpha$ is an intercept averaged over people.
You could also fit a model B
$$E[Y_{nt}|X_{nt}]=\alpha_n+\beta X_{nt}$$
that has a separate intercept for each person (still just linear regression; no random effects or anything). Or you could even fit a model C
$$E[Y_{nt}|X_{nt}]=\alpha_n+\beta_n X_{nt}$$
that has separate slope and intercept for each person.
As long as you have at least two distinct $X$ values for each person all these models can be fitted and will give unbiased estimates of the parameters they estimate (because it's just linear regression).
However, the last model will give absolutely terrible unbiased estimates in most settings, because each $\beta_n$ is estimated only from $T_n$ points. Unless $T_n$ is large, the variance of $\hat\beta_n$ will be large and poorly estimated, so you won't learn much by fitting the model. Your estimates of $\beta_n$ won't improve as $N$ increases, only as $T_n$ increases
If you want a good estimate of $\beta_n$, and $T_n$ isn't big enough to get one from model C, you need to accept some bias in your estimates.  A reasonable approach is to take a weighted combination of $\hat\beta$ from model A and $\hat\beta_n$ from model C.
What weighted combination do we want? Well, one approach is to minimise the expected mean squared error, which depends on the bias of $\beta$ and on the variances of $\hat\beta$ and $\hat\beta_n$. We can estimate the variances (it's just linear regression) and we can estimate $E[(\beta-\beta_n)^2]$, so that's all feasible.   We'll give more weight to $\hat\beta$ when $T_n$ is small (so $\hat\beta_n$ is bad) or when $E[(\beta-\beta_n)^2]$ is small, so that $\hat\beta$ is good.
After all this we have good (though not unbiased) estimates of individual-level slopes, done with just linear regression and no random effects.  Except, it turns out that the estimates we get this way are exactly the linear mixed model estimates! So, it works, but it doesn't change anything.
The same is basically true for log links: Ulrike Grömping wrote a paper on using Poisson GEE to fit Poisson mixed models. However, the $\hat\beta_n$ are no longer exactly unbiased -- they can be infinite -- and the pooling approach doesn't work very well.  Having unbiased estimating equations (as you still do) isn't enough on its own to guarantee unbiased estimators  (you need information going to infinity to get consistent estimators, and even they aren't unbiased outside linear models)
For logistic regression the whole thing goes horribly wrong. Even in model B, with separate intercepts and a common slope, $\hat\beta$ is badly biased for $\beta$ (conditional logistic regression was invented to work around this problem).
Getting back to the original question: yes, you can fit individual parameters by ordinary regression or GEE.  You probably won't have much information about each parameter, and this will be a problem. The type of problem it is varies depending on which model you're fitting.  You're still probably going to need the regularisation that a random-effects model gives you if you want individual-level parameter estimates.
Comment: Great answer! I will try to unpack it ... I really like the way you describe mixed effects models as performing regularization .... I am actually self-learning (rather self-convincing myself) that mixed models perform regularization : https://stats.stackexchange.com/questions/637627/why-do-mixed-effects-regression-models-shrink-parameter-estimates , https://stats.stackexchange.com/questions/637869/why-does-partial-pooling-occur-in-mixed-effects-regressions . These is the analysis I thought of

Understanding the Limitations of GEE