The same message delivered multiple times on postfix

Question: I have a problem with the postfix server I manage.
Server was working fine for over 4 years, but recently I have one issue with it.
Some e-mail messages are delivered on "my '' server many times. Those multiplied messages come from some big providers like amazon ses or zoho mail.
I have found a similar thread on zoho forum, but it ends with a private message to zoho admins and I can't see how this story ends.
It seems like the sender server doesn't know the message was delivered and repeats it many times, and every repeated message is delivered making some kind of flood.
I thought that it was related to greylisting, but amazon ses is on whitelist...
I think " my '' postfix server doesn't confirm message delivery "enough" and the sender repeats it. Configuration on "my '' server didn't change recently. I don't have any idea where to look for the problem... postfix itself, postgrey or something else. I would be grateful for any help.
My configuration:
main.cf (without hashed lines):
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

append_dot_mydomain = no

readme_directory = no

compatibility_level = 2

smtpd_tls_cert_file = /etc/postfix/smtpd.cert
smtpd_tls_key_file = /etc/postfix/smtpd.key
smtpd_use_tls = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
myhostname = <hidden>
alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
alias_database = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
myorigin = /etc/mailname
mydestination = $myhostname, localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
virtual_alias_domains = proxy:mysql:/etc/postfix/mysql-virtual_alias_domains.cf
virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman, proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, proxy:mysql:/etc/postfix/mysql-virtual_alias_maps.cf, proxy:mysql:/etc/postfix/mysql-virtual_email2email.cf
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf
virtual_mailbox_base = /var/vmail
virtual_uid_maps = proxy:mysql:/etc/postfix/mysql-virtual_uids.cf
virtual_gid_maps = proxy:mysql:/etc/postfix/mysql-virtual_gids.cf
sender_bcc_maps = proxy:mysql:/etc/postfix/mysql-virtual_outgoing_bcc.cf
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes
smtpd_sasl_authenticated_header = yes
smtpd_restriction_classes = greylisting
greylisting = check_policy_service inet:127.0.0.1:10023
smtpd_recipient_restrictions = permit_mynetworks, reject_unknown_recipient_domain, reject_unlisted_recipient, permit_sasl_authenticated, reject_non_fqdn_recipient, reject_unauth_destination, check_recipient_access proxy:mysql:/etc/postfix/mysql-virtual_recipient.cf, check_recipient_access mysql:/etc/postfix/mysql-virtual_policy_greylist.cf, check_policy_service unix:private/quota-status
smtpd_tls_security_level = may
transport_maps = hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
relay_domains = proxy:mysql:/etc/postfix/mysql-virtual_relaydomains.cf
relay_recipient_maps = proxy:mysql:/etc/postfix/mysql-virtual_relayrecipientmaps.cf
smtpd_sender_login_maps = proxy:mysql:/etc/postfix/mysql-virtual_sender_login_maps.cf
proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $sender_bcc_maps $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $smtpd_sender_login_maps $virtual_uid_maps $virtual_gid_maps $smtpd_client_restrictions $smtpd_sender_restrictions $smtpd_recipient_restrictions $smtp_sasl_password_maps $sender_dependent_relayhost_maps
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, check_helo_access regexp:/etc/postfix/helo_access, permit_sasl_authenticated, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, check_helo_access regexp:/etc/postfix/blacklist_helo, reject_unknown_helo_hostname, permit
smtpd_sender_restrictions = permit_mynetworks, check_sender_access proxy:mysql:/etc/postfix/mysql-virtual_sender.cf,  permit_sasl_authenticated, reject_non_fqdn_sender, reject_unlisted_sender
smtpd_client_restrictions = check_client_access proxy:mysql:/etc/postfix/mysql-virtual_client.cf, permit_inet_interfaces, permit_mynetworks, permit_sasl_authenticated, reject_rbl_client zen.spamhaus.org, reject_unauth_pipelining, permit
smtpd_client_message_rate_limit = 100
maildrop_destination_concurrency_limit = 1
maildrop_destination_recipient_limit = 1
virtual_transport = dovecot
header_checks = regexp:/etc/postfix/header_checks
mime_header_checks = regexp:/etc/postfix/mime_header_checks
nested_header_checks = regexp:/etc/postfix/nested_header_checks
body_checks = regexp:/etc/postfix/body_checks
owner_request_special = no
smtp_tls_security_level = dane
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_protocols = !SSLv2,!SSLv3
smtp_tls_protocols = !SSLv2,!SSLv3
smtpd_tls_exclude_ciphers = RC4, aNULL
smtp_tls_exclude_ciphers = RC4, aNULL
dovecot_destination_recipient_limit = 1
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
message_size_limit = 0
smtpd_milters = inet:localhost:11332
non_smtpd_milters = inet:localhost:11332
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
milter_default_action = accept
smtpd_etrn_restrictions = permit_mynetworks, reject
smtpd_data_restrictions = permit_mynetworks, reject_unauth_pipelining, reject_multi_recipient_bounce, permit
smtpd_tls_mandatory_ciphers = medium
tls_medium_cipherlist = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA
tls_preempt_cipherlist = yes
address_verify_negative_refresh_time = 60s
enable_original_recipient = no
smtpd_forbidden_commands = CONNECT,GET,POST,USER,PASS
address_verify_sender_ttl = 15686s
smtp_dns_support_level = dnssec
smtpd_reject_unlisted_sender = no
sender_dependent_relayhost_maps = proxy:mysql:/etc/postfix/mysql-virtual_sender-relayhost.cf
smtp_sasl_password_maps = proxy:mysql:/etc/postfix/mysql-virtual_sender-relayauth.cf, texthash:/etc/postfix/sasl_passwd
smtp_sender_dependent_authentication = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous, noplaintext
smtp_sasl_tls_security_options = noanonymous
authorized_flush_users =
authorized_mailq_users = nagios, icinga

smtpd_discard_ehlo_keyword_address_maps = cidr:/etc/postfix/esmtp_access

content_filter = amavis:[127.0.0.1]:10024
smtpd_proxy_options = speed_adjust

and master.cf
smtp      inet  n       -       n       -       -       smtpd

submission inet n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_client_restrictions=$mua_client_restrictions
  -o smtpd_helo_restrictions=$mua_helo_restrictions
  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING
  -o content_filter=amavis:[127.0.0.1]:10026

smtps     inet  n       -       n       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=$mua_client_restrictions
  -o smtpd_helo_restrictions=$mua_helo_restrictions
  -o milter_macro_daemon_name=ORIGINATING
pickup    unix  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
relay     unix  -       -       n       -       -       smtp
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d vmail ${extension} ${recipient} ${user} ${nexthop} ${sender}
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix  -       n       n       -       2       pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman   unix  -       n       n       -       -       pipe
  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
  ${nexthop} ${user}

dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${nexthop}

amavis unix - - n - 2 smtp
        -o syslog_name=postfix/amavis
        -o smtp_data_done_timeout=120
        -o smtp_send_xforward_command=yes
                -o smtp_bind_address=

127.0.0.1:10025 inet n - n - - smtpd
        -o syslog_name=postfix/10025
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_client_restrictions=
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o smtpd_end_of_data_restrictions=
        -o mynetworks=127.0.0.0/8
        -o strict_rfc821_envelopes=yes
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
        -o smtp_send_xforward_command=yes
        -o disable_dns_lookups=yes

127.0.0.1:10027 inet n - n - - smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_client_restrictions=
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o smtpd_end_of_data_restrictions=
        -o mynetworks=127.0.0.0/8
        -o strict_rfc821_envelopes=yes
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
        -o smtp_send_xforward_command=yes
        -o milter_default_action=accept
        -o milter_macro_daemon_name=ORIGINATING
        -o disable_dns_lookups=yes

Logs:
May 31 13:41:12 webh postgrey[2921214]: action=pass, reason=triplet found, client_name=<hidden>, client_address=<hidden>, sender=<hidden>, recipient=<hidden>
May 31 13:41:12 webh postfix/smtpd[2983539]: DF3A613F931: client=<hidden>
May 31 13:41:13 webh postfix/cleanup[2984139]: DF3A613F931: message-id=<hidden>
May 31 13:43:15 webh postfix/qmgr[2932048]: DF3A613F931: from=<hidden>, size=21379, nrcpt=1 (queue active)
May 31 13:43:15 webh postfix/smtpd[2983539]: disconnect from <hidden> ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
May 31 13:43:15 webh dovecot: lda(<hidden>)<2985193><WrCwN1P/lWLpjC0A/liSRw>: sieve: msgid=<hidden>: stored mail into mailbox 'INBOX'
May 31 13:43:15 webh postfix/pipe[2985192]: DF3A613F931: to=<hidden>, orig_to=<hidden>, relay=dovecot, delay=124, delays=124/0.03/0/0.08, dsn=2.0.0, status=sent (delivered via dovecot service)
May 31 13:43:15 webh postfix/qmgr[2932048]: DF3A613F931: removed

Comment: Please provide your Postfix configuration using commands shown in the tag description. Gather logs noting the receipt of the messages in question. Add them [into the question](https://serverfault.com/posts/1102098/edit). You may [mask private details that identify you (globally routable IP addresses, DNS names)](https://meta.serverfault.com/questions/963/what-information-should-i-include-or-obfuscate-in-my-posts/6063#6063), but make sure it stays consistent.
Comment: Hello anx, I have added configuration to my question. In the meantime I have asked my business partner about some logs. There was "uncathegorised bound/Read time error", so I have investigated a little. I have found, that the time e-mail is processing may be too long for some servers (as you have pointed). I can see in the logs that e-mail is processed in the phase (a) about 2 minutes. Every e-mail is processed similar (123-124 seconds in first phase). I am sure that is the reason for sending such e-mail once again and again. So now the point is to find why the first phase lasts so long.
Comment: Thanx anx once again. I have moved my answer from comment to answer, to be more clear for others.
Comment: Specifically, I would be looking into clues about *time spent*, whether that is time Postfix spent waiting for a misbehaving recursive resolver, or some milter/filter software failing to update postfix about its progress in scanning mail. [Increase verbosity](https://www.postfix.org/DEBUG_README.html#verbose) if the timestamps of your current log lines do not already pinpoint at which point Postfix is spending more time than the sender is presumably willing to wait.
Comment: I have also run all processes with "-v" switch, I get very detailed log, but during that 2 minutes there was nothing... simply waiting.
Answer: I have figured out what was the reason. It was that redis server (used by rspamd) was misconfigured. In rspamd logs there was an entry:
cannot get ANNs list from redis: timeout while connecting the server

The answer was found on https://blog.christosoft.de/2019/11/rspamd-redis-connection-refused
I had two entries missed:
write_servers = "127.0.0.1:6379";
read_servers = "127.0.0.1:6379";

I've added it in /etc/rspamd/local.d/redis.conf and delays are now like that: delay=4.7, delays=0.79/0/0.01/3.9 :-)


LogCat - Não aparece

Question: Estou fazendo o Exemplo5 do cap 3 do livro Google Android 3ºed(Ricardo R.Lecheta), onde aplicando um filtro para exibir no logcat. só que não está me retornando nada.
<code>package br.livro.android.cap3;

import android.app.Activity;
import android.os.Bundle;
import android.util.Log;

public class Exemplo5 extends Activity {
    private static final String CATEGORIA = "livro";

    @Override
    public void onCreate(Bundle icicle) {
        super.onCreate(icicle);

        setContentView(R.layout.main);

        // Verbose
        Log.v(CATEGORIA, "log de verbose");

        // Debug
        Log.d(CATEGORIA, "log de debug");

        // Info
        Log.i(CATEGORIA, "log de info");

        // Warn
        Log.w(CATEGORIA, "log de alerta");

        // Error
        Log.e(CATEGORIA, "log de erro", new RuntimeException("teste de erro"));
    }
}
</code>
Comment: Eu estou sem o Eclipse aqui (usando o [Android Studio](https://developer.android.com/sdk/installing/studio.html)), mas você já tentou rodar o logcat na mão? (`adb logcat`). Talvez o IDE esteja redirecionando a `stdout` ou algo assim ([Documentação Oficial da Google](http://developer.android.com/tools/debugging/debugging-log.html)).
Comment: Testei em outro projeto e funcionou :), deve ser algum erro que fiz na aplicação, obrigado a todos.
Comment: Faça como disse o @AnthonyAccioly a ver se resulta.
Comment: utilizando o comando (`adb logcat`) me retorna algumas informações mais nada do que eu digitei na minha classe, tem que utilizar filtro ou deveria aparecer?
Comment: @Josinaldo quando você tira o filtro e fica usando o emulador, os logs ficam aparecendo?
Answer: Tente abrir a perspectiva do DDMS (em cima a direita), e selecionar o device que você deseja visualizar o LogCat Output.
Exemplo: selecionando o device "015DC1F50600" para exibir o LogCat.
Comment: nada :(, pior que nem posso pular está parte do livro porque é essencial.

(http://s16.postimg.org/eynkqshnp/Screenshot_erro_eclipse2.png)

LogCat - Não aparece


PHP e Javascript - Automatizando função de impressão de uma página a partir de outra página

Question: O meu site possui uma tela de consulta de eventos cadastrados. Para cada um, tenho a opção de Editar Evento que me redireciona para um formulário:
    function editaEvento(val){
        var valor = val;
        location.href = "http://meusite.com.br/edicaoeventos?valor="+valor;
        // Função de edição pra abrir a página de formulário, semelhante ao do Index, mas que somente fará a edição
    }

Dentro do formulário de edição há um botão pra imprimir esse formulário. Estava querendo criar uma nova função pra Imprimir o evento a partir da consulta de eventos cadastrados.
Como eu posso fazer? Pensei em algo do tipo: repetir a função editaEvento e adicionando um valor de $_SESSION['imprime'] igual a 1. Aí quando a página de edição carregar, ela faz uma verificação do valor da session. Se continuar como 1, ele ativa a impressão do formulário automaticamente.
Agradeço quem puder me ajudar.
Answer: Fala chará.
Você pode criar no botão de Impressão onclick="window.print()"
E no CSS usar a imaginação do css com o @media print
Acho que o caso da session é complicado pois a verificação é mais complexa, tenta utilizar o print() mesmo e ocultar os campos que não devem ser mostrados na impressão.
Ou senão criar um eventosimpressao.php?idevento=
e nesse arquivo configurar os moldes. Só que é mais complicado.
Ex:

        @page {
            size: A4;
        }
@media print {
    body{max-width: 100%}
    thead {display: table-header-group;}//bug do firefox
    #btnSave{display:none;}
    #btnCancel{display:none;}

}


Google Tag Manager JavaScript Compiler Error

Question: The Error
Validate Container
The container has the following errors:
Type    Location    Description
JavaScript compiler error
Keyvendors_whastaap

Error at line 6, character 27: This language feature is only supported
for ECMASCRIPT_2015 mode or better: arrow function.

The Code

whats the error how I fix it ?
trying to add whatsaao widget throough GTM in all pages except one page. but the tag have an error.
Answer: Your mistake is that you use an arrow function in your code. You can't do that in GTM.
Rewrite this:
window.onload = () => {
  _waEmbed(wa_btnSetting);
};

into this:
window.onload = function(){
  _waEmbed(wa_btnSetting);
};


Translating without google translator

Question: I have used google translator. And it has horrible grammatical problems. I am looking for some library, or something that is more versatile then that. Does anyone know anything about it?
I once heard that pear has a translator package. Does anyone know how efficient it is?
Comment: Any translation tool **will have** horrible grammatical problems. Specially if the two translating languages are extremely different (i.e.: English -> Japanese). But most of the time, it's acurate enough to understand the underlying message.
Comment: What languages do you want to translate?
Comment: Yes voyager, you are right. I am just asking that what is the best method two translate text from one language to another. (Not manually :)
Comment: Google Translate is among the best automatic translation systems existing today. If it's not good enough for you I don't believe you'll find a system which is. Machine translation has been one of the toughest problems tackled with digital computers since they were invented.
Comment: I'd stick with Google definitely. Though it is not always as good still it produces the **best** results. And Google is working on the semantics recognization to serve even better results each day...
Comment: While looking into Machine Translation for a project of mine, I came across Moses. I have never used it, but it sounds pretty cool, and is open source. feed it a library of texts translated manually between the two target languages, and it constructs a translation database on its own. It supports strict word-for-word translation and intelligent grammatical tree-based translation too. It's a C++ application, so if you want to use it on the web, you may need to write a wrapper for it in PHP or Node or your preferred server-side language with commandline capabilities. http://www.statmt.org/moses/
Comment: Yandex's translation service is an alternative for translating websites. After a moderate search, I didn't find very many services that do it, anymore. If you're not looking to translate websites specifically, there are a lot more alternatives.
Answer: It's unlikely you'll find a much better solution. There are two primary methods to machine translation - rule based and statistical. Rule based translation has the problem that a great deal of text involves idiomatic expressions, grammar, or vocabulary that are outside the set of rules. Statistical methods can sometimes provide better solutions if there is a large dataset (and large data sets are Google's business), but also has trouble with idiomatic expressions and differences in word order between languages. For more information, check the Wikipedia Article on Machine Translation, and more specifically the articles on Inter-Lingual and Statistical Machine Translation methods.
Answer: Try babelfish: http://babelfish.yahoo.com/
Comment: I have tried that too. Its not that effective.
Answer: You will not find any package that can automatically translate a text gramatically correct. Computers simply can't understand text well enough to translate it.
The main problem with automatic translation is understanding the text that is translated, the actual translation part is trivial in comparison and could be done by a computer.
Take for example a simple sentence like "When the going gets tough, the tough gets going.". It has ambigous words, implicit references and cultural undertones, which is very hard to write an algorithm for deciphering.
I once read about a project of developing an independent, unambigous intermediate translation language, which sounded promising. You could translate your text (partly manually) to this intermediate language, which then could be automatically translated into any language. I haven't heard anything more about it, though...
Comment: This seems intuitive but isn't actually true. The best automatic translators today like Google Translate don't even try to understand the text at all, they use statistics instead. In fact Fred Jelinek (in)famously said "Every time I fire a linguist my system improves" about the machine translation he was working on.
Comment: I tend to agree but the consensus seems not to agree \-: Though everybody agrees computers will never be able to translate as well as people. "Context" would be the bigger obstacle though. Lack of general knowledge used to be cited as well but I don't see this so much now that large knowledge bases exist and machine translation still sucks.
Comment: @hippietrail: That is only true to a certain level, i.e. statistics can only get translations from *horrible* to *bad*, to get a good translation you need to understand the text that you translate.

Translating without google translator


German resources loaded from resource only dll when system locale is french

Question: In an MFC application, there are German, French, Spanish resource files. These resource files have resources to be loaded for all sub-languages.
Say for German:
#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_DEU) || defined(AFX_TARG_DES) || defined(AFX_TARG_DEA) || defined(AFX_TARG_DEL) || defined(AFX_TARG_DEC)
LANGUAGE LANG_GERMAN, SUBLANG_NEUTRAL
#pragma code_page(1252)

Resource DLL is loaded using LoadLibrary. OS is Windows 7 Enterprise English version. In clock -> language -> region from Control Panel, it is "France".
What is the reason for German resources to be loaded when MFC application is opened?
Comment: maybe you set a priority or something similar.
Comment: Any idea how to set priority?
Comment: see this [link](https://msdn.microsoft.com/en-us/library/cc194810.aspx)
Comment: This worked. Thanks a lot. I used setlocale which loaded the proper resources from resource only dll.
Comment: OK i add this as a answer then
Answer: As we discuss in comments you can use setlocale here is the solution.


In how many ways can i partition a stick

Question: I have a stick (or a ruler) of 10 cm length.
I want to cut the stick into pieces.
Each cm (1 cm, 2cm, 3cm, ... 0 cm) is a possible cutting point.
In how many different ways can i cut the stick?
One way would be to cut at every cm, so I get 10 parts.
Another would be to cut two 1cm pieces, and the rest 2cm pieces = 6 parts
Is there a formula for this? All i know is that it's not a normal permutation...
edit:
A 1cm piece cut at 1cm is different from a 1cm piece cut from 9cm, even when the rest is partitioned the same way.
Comment: If every integer point 1cm through 9cm is either a cut or not a cut then there are 2^9 == 512 ways to make the cuts.
Comment: Wrong website :) _Mathematica_ != _Mathematics_
Comment: `IntegerPartitions`?
Comment: @Öskå oh  no. thanks for pointing it out. will never happen again!
Comment: @Mr.Wizard Can you explain the underlying systematic behind it? (If you make a post, ill accept it as an answer)
Comment: This is actually a difficult problem, and has an entire topic devoted to it: https://en.wikipedia.org/wiki/Partition_%28number_theory%29
Comment: I'm not sure in what sense it's a difficult problem: you can count the partitions of 10 by hand.
Comment: You did not specify the problem fully enough: is it the same for you to make 1 cut at 1 cm or 1 cut at 9 cm (you obtain a 1-cm and a 9-cm piece), or do you consider these as two different ways to cut the stick? If you choose the latter, then @Mr.Wizard is correct.
Comment: @KevinCarlson I mean the general problem is difficult, and there's no elementary formula for it.
Answer: I do not think this is a partitions problems. In partitions the order of the summands do not matter. That is $1 + 8 + 1 = 10$ and $8 + 1 + 1 = 10$ are counted as the same partition, but they correspond to different ways of cutting the ruler. The first $1 + 8 + 1 = 10$ corresponds to cutting a 1cm and 9cm while the second $8 + 1 + 1 = 10$ corresponds to cutting a 8cm and 9cm. The questions as posed is how many ways to cut the ruler, this is actually a compositions problem. Like Mr.Wizard said the solution is $2^9 = 512$. This is because of each of 1cm, 2cm, ..., 9cm there can either be a cut or not. Thus we have two options (to either cut or not) and we do this 9 times. Therefore we get $2^9$.
See below for more on compositions. Compositions are a much easier problem than partitions.
https://en.wikipedia.org/wiki/Composition_(combinatorics)
Comment: So If I chose to either cut, not cut or delete a piece (like throw it away without cutting, making the stick smaller), i get the solution with 3^9?
Comment: You have to be careful with that. We are cutting at the marks 1 cm, 2 cm, etc. But we would be throwing away actual parts of the stick. So, if you choose to "delete" at 3 cm for example what are you deleting? The part before 3 cm or the part after 3 cm? So, it is not as simple as just adding another option and making it $3^9$.
Answer: There are two interpretations of your problem: if cutting into $[0,1]$ and $[1,10]$ is different than cutting into $[0,9]$ and $[9,10]$, then as was mentioned in the comments this is a counting principle problem: you decide independently which of nine points to cut or not cut at, in the same way as given five shirts, three pairs of shoes, and two pairs of trousers you have thirty choices of outfits.
But what seems to be a more natural interpretation is that the two cuttings I mentioned are the same, in which case this is a partition problem, the general type being to determine how many ways we can write an integer $n$ as a sum of smaller integers. Your problem is a partition problem because the pieces of ruler you come out with must have integer lengths summing to 10.
The easiest way to compute partition numbers by hand is to use Euler's pentagonal number theorem, which is described on the Wikipedia page. It's also possible, though it'll probably take a little while, to compute this partition number directly: you have partitions $10,9+1,8+2,8+1+1,7+3,7+2+1,7+1+1+1,6+4,...,5+4+1,...,5+1+1+1+1+1,4+4+2,...,1+1+1+1+1+1+1+1+1+1$.
The thing you must be careful about is not to list any partitions twice: this is why I wrote them down with pieces in decreasing order, which will guarantee you don't write, say, both $6+4$ and $4+6$. You should end up with the answer to life, the universe, and everything, so it's surely a worthwhile problem.


powers in the group of integers modulo n - Zn

Question: Let $\mathbb Z_n$ be the group of integers modulo $n$, and define power of elements in the group as following: $$[a]^{[b]} = \left[a^b\right].$$
I need to prove that this definition of power is not well defined, because it can lead to $\left[a ^ b\right]$ not equal to $\left[a^{b + n}\right] or \left[(a + n)^b\right]$.
I found one example in $\mathbb Z_3$, where $$[2]^{[1]} = [2^1] = [2] \ne [2]^{[1]} = [2]^{[4]} = [2^4] = [1].$$ But most of the other examples I tried fit well into the definition and do not cause inequalities.
What is the rule that decides which $a$, $b$ fit into the definition and which $a$, $b$ do not?
Thank you!
Comment: Use mathjax to make your post more readable :)
Comment: [Here's a MathJax tutorial](https://math.meta.stackexchange.com/q/5020/104041) :)
Comment: Please consider using MathJax. To familiarize yourself with math Jax you can use the link above. This is considered a standard I'm this site.
Comment: sorry guys, new to this website :) I will definitely use it next time!
Comment: I've added MathJax to your post - you can take a look at the changes that have been made so you're familiar with it for your next question!
Comment: @kipf Thank you!
Comment: You began by talking about the group $Z_n$ (which is a group with the addition modulo $n$) but then used mainly multiplication modulo $n$. You probably should talk about the *ring* of integers modulo $n$.
Answer: This is a good question. The thing is the power does not behave like addition or multiplication. You have several cases:

$a \in \mathbb{Z}/n\mathbb{Z}$ is nilpotent, i.e. $\exists k \in \mathbb{N}$ such that $a^k = 0$. It is the case for $[2] \in \mathbb{Z}/4\mathbb{Z}$.
Then you have necessarily some $k \in \mathbb{N}$ such that $k = 1 \mod{n}$, which means $a^k = 0$ and $a^1 = a$ which is a different result if $a \neq [0]$.
Thus this doesn't work.
If $a$ is not nilpotent (like if $n$ is prime). You can chose $a$ invertible (but non nilpotent does not mean invertible, e.g. $[2]$ \in $\mathbb{Z}/6\mathbb{Z}$).
Then $a \in \mathbb{Z}/n\mathbb{Z}^\times$ a group of order $\varphi(n)$. Necessarily you have $a^{\varphi(n)} = [1]$. Then for such elements you have $a^k = a^{k + m \varphi(n)}$, so the power is well defined but not for $\mathbb{Z}/n\mathbb{Z}$, but for $\mathbb{Z}/\varphi(n)\mathbb{Z}$. You can even replace $\varphi(n)$ by the order of $a$.
Answer: You can explicitly expand the two options you mentioned -
$$a^{b+n} = a^b a^n,$$
$$(a+n)^b = \sum_{k=0}^b\frac{n!}{k!(n-k)!}a^kn^{b-k}.$$
Therefore, for the first term to be $a^b$, we mut have $a^n=1$, and for the second to be $a^b$ we must have $$\sum_{k=0}^{b-1}\frac{n!}{k!(n-k)!}a^kn^{b-k} = 0.$$
Since every element of the sum is $0$ modulo $n$, (since $k < b$), the second condition always holds. The first may or may not hold, depending on the values of $a, n$.
Comment: Use `$$y=x^2$$` for $$y=x^2$$ and `$y=x^2$` for $y=x^2$.
Comment: thank you very much! really helpful.
Comment: Correct, thanks! I edited my comment.


Is it possible that a person will finds the number what his friend thinks in my mind. the number should between 1-100 via java code

Question:
the person will think a number between 1-100. The Questionnaire can ask n number of
condition, i.e the number is even or odd, the number is perfect
square or prime number, sum of squares etc.
Answer: It is certainly possible to find a number between 1 and 100 by asking 100 questions, namely:

Is the number one?
Is the number two?
etc., etc., etc.

It is possible to do it with 7 questions by binary search. Writing the number in base 2, ask:

Is the units bit one?
Is the twos bit one?
Is the fours bit one?
etc., etc., etc.

I don't know anything about Java code, but that's for a different website, anyway.
Comment: Indeed, even 7 questions of the form "Is the number greater than $x$?" will do. (Hint: start with $x=50$.) The binary expansion technique does, however, have the advantage that you can decide all the questions you will ask in advance.


Django model form and objects database id

Question: I have a complex django object, which has properties of other class types. This gets like this:
class Order:
   contractor - type Person
   some other fields....

In my form I'd like to be able to either choose existing Person object from the dropdown or add a new one with a form. I've managed to create forms and appropriate workflow, but the problem is with saving the Order itself, I simply can't get the id of a saved Person instance. I'm doing sth like this:
def make_order(request):
  if request.method == 'POST':
    parameters = copy.copy(request.POST)
    contractor_form = ContractorForm(parameters)
    if contractor_form.is_valid():
      contractor_form.save()
      parameters['contractor'] = ???
    form = OrderForm(parameters)
    if form.is_valid():
      form.save()
      return HttpResponseRedirect('/orders/')
  else:
    form = OrderForm()
    contractor_form = ContractorForm()

  return render_to_response('orders/make_order.html', {'order_form' : form, 'contractor_form' : contractor_form})

So, if POST request reaches this method I first check if ContractorForm have been filled - I assume that if the form is valid, it is meant to be used. If yes, than I save it and would like to assign the saved object's database id to appropriate field for OrderForm to find it.
All my forms are ModelForms.
The questions are:

Are there better ways to do this? (choose from dropdown or add in place) - better or more pythonic ;-)
How can I get saved objects id when using ModelForms?

Edited
My ContractorForm is:
class ContractorForm(ModelForm):
  class Meta:
    model = Contractor

Nothing fancy.
Comment: On a side note, use reverse (http://docs.djangoproject.com/en/dev/topics/http/urls/#reverse) instead of referencing the '/orders/' url directly in your `HttpResponseRedirect`... it'll save you tons of headaches in the future, trust me :)
Comment: Ha! You're right, I should've done that earlier.
Answer: save() should return the newly created instance.
if contractor_form.is_valid():
  instance = contractor_form.save()
  parameters['contractor'] = instance

where id would be instance.id, or even better instance.pk.
pk vs. id:

Regardless of whether you define a
  primary key field yourself, or let
  Django supply one for you, each model
  will have a property called pk. It
  behaves like a normal attribute on the
  model, but is actually an alias for
  whichever attribute is the primary key
  field for the model. You can read and
  set this value, just as you would for
  any other attribute, and it will
  update the correct field in the model.

Follow-up on comment:
Well it does work by default, so there must be something else wrong.
models.py
class Category(models.Model):
    name = models.CharField(max_length=70)
    slug = models.SlugField()

forms.py
from django import forms
from models import Category

class MyModelForm(forms.ModelForm):
    class Meta:
        model = Category

Test in shell:
In [3]: from katalog.forms import MyModelForm
In [4]: data = {'name':'Test', 'slug':'test'}
In [5]: form = MyModelForm(data)
In [6]: instance = form.save()
In [7]: instance
Out[7]: <Category: Test>
In [8]: instance.id
Out[8]: 5L

Comment: This was my first guess, but unfortunately it doesn't work like this. ModelForm.save() returns None. There should be a way of doing this with overloaded save() method, but this seems overcomplicated and messy.
Comment: Well, I've tested this again, and your solution works. I'll however test it more thoroughly with my specific case.


is $(x+y)^2\neq (x^2+y^2)$ correct if we consider the integers mod $2$?

Question: In general $(x+y)^2  \neq (x^2+y^2)$. Is this still correct in the case of mod $2$? Can anyone share some insight?
Comment: @therapist I think user133281 is trying to point out that you are proving the statement with the wrong quantifier. You need to prove $\forall$ but your example proves $\exists$. I.e when we say $(x+y)^2\neq x^2+y^2$ we mean there $\exists x,y$ s.t. $(x+y)^2\neq x^2+y^2$. So to disprove this you must show $\forall x,y\; (x+y)^2=x^2+y^2$
Comment: @therapist That way of reading the question makes no sense, since for no field is it true that $\forall x,y\;\;(x+y)^2\neq x^2+y^2$, as user 133281 points out. Thus the only sensible reading of the question is "Is it true for fields of characteristic 2 that $\exists x,y\;\;(x+y)^2\neq x^2+y^2$?" It sounds strange due to the phrasing which uses negation.
Comment: The *generalized* answers given below are great, but it may help to consider a particular example to solidify why the negated equivalence doesn't hold in $\mod 2$. Consider when $x=y=1$. Then $$(x+y)^2=(1+1)^2=(0)^2=0,$$ and $$(x^2+y^2)=(1^2+1^2)=(1+1)=0.$$ Thus, here is a simple counterexample to your claim that $(x+y)^2\neq(x^2+y^2)$ for the integers $\mod 2$.
Comment: And then $(0+0)^2 = 0^2 + 0^2$ is a counterexample for the general case ... ?
Comment: @user133281 If your comment is directed to me, then please indicate this. I'm not sure what you're trying to say.
Comment: @DRF Maybe I'm reading the question wrong then. It seems like OP is claiming the negated equality holds for all integers mod $2$. My counterexanple was to show this is not true.
Answer: Not so because $x^2 + y^2 = x^2 + 2xy + y^2 = (x+y)^2 \pmod 2$ as $ 2xy = 0 \pmod 2$
Answer: That statement is false in the case of mod 2. In fact the case of $(x+y)^p=x^p+y^p$ is true mod p for all prime values of $p$.
Comment: maybe you can write why is that true...
Comment: @Ale: see pew's answer.
Comment: @TonyK i knew the answer, i only thought that askask answer could have been improved, since it has been posted before pew's answer...
Answer: By the binomial theorem
$$(x+y)^n=x^n+y^n+\sum_{k=1}^{n-1}{n\choose k}x^k y^{n-k}$$
Now if $n$ is prime, $n$ divides ${n\choose k}$ if $0<k<n$ because the factor $n$ appears in the numerator of
$${n\choose k} = \frac{n\times (n-1) \times \dots \times (n-k+1)}{k!}$$
but not the denominator (as $k<n$ and $n$ is prime) (Source).
Therefore, for $n$ prime, ${n\choose k}$ is a multiple of $n$ and under the operation $\pmod n$ the rightmost sum in the first formula vanishes, meaning
$$(x+y)^n \equiv x^n+y^n \pmod n$$
Answer: If you want to read about this, I suggest you to take the book Abstract Algebra in the Chapter 13 (Field Theory, third edition) and check about characteristic of a field. Here prove that ASKASK says.


Potential leak of an object stored into CGImageRef

Question: Running my code under Xcode Analyze i've stumbled into the following block
<code>- (UIImage *)imageWithFilter:(CIFilter *)filter
{
    CIContext *ctx = [CIContext contextWithOptions:nil];
    CGImageRef imageRef = [ctx createCGImage:filter.outputImage fromRect:CGRectMake(0, 0, self.size.width, self.size.height)];
    return [UIImage imageWithCGImage:imageRef];
}
</code>
Xcode complains about a potential memory leak:

What is going on? And how would I go about fixing it?
Answer: The following looks like a fix, still not sure if this is the best way of handling this over retained reference?
<code>- (UIImage *)imageWithFilter:(CIFilter *)filter
{
    CIContext *ctx = [CIContext contextWithOptions:nil];
    CGImageRef imageRef = [ctx createCGImage:filter.outputImage fromRect:CGRectMake(0, 0, self.size.width, self.size.height)];
    UIImage *image = [UIImage imageWithCGImage:imageRef];
    CGImageRelease(imageRef);
    return image;
}
</code>
Comment: This is the best way. I can  you :D


Why we should use uppercaseStringWithLocale of NSString to get correct uppercase string?

Question: I have a very simple task: from server I get UTF-8 string as byte array and I need to show all symbols from this string in upper case. From this string you can get really any symbol of unicode table.
I know how to do it in a line of code:
NSString* upperStr = [[NSString stringWithCString:utf8str encoding:NSUTF8StringEncoding] uppercaseString];

And seems to me it works with all symbols which I have checked. But I don't understand: why we need method uppercaseStringWithLocale? When we work with unicode each symbol has unique place in unicode table and we can easily find does it have upper/lower case representation. What trouble I might have if I use uppercaseString instead uppercaseStringWithLocale?
Comment: In certain regions of the world apparently there's a difference in what character is considered "upper case" of what.  Generally if you're dealing with European languages `uppercaseString` should work fine (though I suppose there's some chance of a problem with, say, German).
Answer: Uppercasing is locale-dependent. For example, the uppercase version of "i" is "I" in English, but "İ" in Turkish. If you always apply English rules, uppercasing of Turkish words will end up wrong.
Answer: The docs say:

The following methods perform localized case mappings based on the
  locale specified. Passing nil indicates the canonical mapping.  For
  the user preference locale setting, specify +[NSLocale currentLocale].

Assumedly in some locales the mapping from lowercase to uppercase changes even within a character set. I'm not an expert in every language around the globe, but the people who wrote these methods are, so I use 'em.
Comment: If you don't know the locale then certainly the generic method is the one to call.
Comment: Seems you are right. But the worst in current situation is when you are working with unicode as a byte array - you are to able to determine what local you should use with this byte array.


É possível criar uma aplicação híbrida multiplataforma com PhoneGap?

Question: Com o advento da possibilidade de criação de aplicações mobile para diversas plataformas utilizando PhoneGap, como exemplo, encontrei no próprio site da Adobe PhoneGap a chance de criar um app híbrido, utilizando tanto componentes nativos da plataforma mobile escolhida como componentes Cordova. Fiquei um pouco confuso a respeito de um ponto:
É possível criar uma aplicação híbrida para multiplataformas utilizando o mesmo código gerado?
Segue abaixo a fonte de onde encontrei sobre o assunto:
http://docs.phonegap.com/develop/1-embed-webview/android/
Answer: Os aplicativos híbridos são parcialmente nativos e parcialmente web apps. Como os nativos, eles devem ser baixados através de um aplicativo de loja (como Google Play do Android e App Store da Apple), ficam armazenados na tela principal do dispositivo e podem aproveitar todas as funcionalidades do dispositivo (câmera, GPS, acelerômetro, gestos etc). Como web apps, eles podem ser baseados em HTML5 e exibidos através de um navegador embutido no aplicativo, tendo parte ou conteúdo total carregado da web.
Você tera um codebase bem parecido de uma forma geral, porém, haverá peculiaridades pra cada plataforma, como por exemplo o código pra UI. O UI no Android é diferente do IOS, dessa forma você terá que separar o codebase de UI, ou assets, se preferir.
O que podemos aproveitar em ambas as versões, é a forma como você vai tratar o dado vindo do seu WS, via Restful, SOAP, RTC, etc. Esse codebase pode ser o mesmo, você vai tratar os dados com o pattern que você escolher e reutilizar esse código em ambas as plataformas de mobile.
Podemos então chegar à conclusão que o código será parcialmente reaproveitado e parcialmente refeito, porém é mais vantajoso que desenvolver nativamente, pois nesse último caso não há reaproveitamento nenhum de código, sendo assim mais lento para desenvolver.


GCM function only get registartion ID on few device

Question: I got a program including GCM service and the GCM service performs perfectly on some of my devices
including:
1)android emulator (Google API 17 - 4.2.2)
2)LG Phone (Android 4.1.2)
3)Google phone (Android 4.2.2)
However, on the following device it seems the GCM function stops at registration status and
onRegistered() was not called.
including:
1)Sony (Android 4.0.4)
2)HTC (Android 4.0.3)
3)SAMSUNG Tablet (Android 3.0.3)
Since it can run on some of my devices, I am pretty sure the problem is not in my code or the Android manifest permission settings, but I don't know what can I add or modify to solve this.
I knew that GCM function for version 4.0.4 and lower needs an active Google account, but I tried it and it's still not working.
Meanwhile I tried the solution suggested in these two questions, but it didn't help
I can not get registration ID from Android GCM
Android GCM : GCMRegistrar gives empty registration ID
Can anyone give me some advice how to solve this problem?
Any other way I can perform server push notification function?
Here are my code
    //GCM part
    cd = new ConnectionDetector(getApplicationContext());

    // Check if Internet present
    if (!cd.isConnectingToInternet()) {
        // Internet Connection is not present
        alert.showAlertDialog(this,
                "Internet Connection Error",
                "Please connect to working Internet connection", false);
        // stop executing code by return
        return;
    }

    // Getting name, email from intent
    Intent i = getIntent();

    // Make sure the device has the proper dependencies.
    try{
        GCMRegistrar.checkDevice(this);
        // Make sure the manifest was properly set - comment out this line
        // while developing the app, then uncomment it when it's ready.
        GCMRegistrar.checkManifest(this);
    }catch(Exception e){
        Toast.makeText(getApplicationContext(), "Version too old" + e.toString(), Toast.LENGTH_LONG).show();
    }
    registerReceiver(mHandleMessageReceiver, new IntentFilter(
            DISPLAY_MESSAGE_ACTION));

    // Get GCM registration id
    final String regId = GCMRegistrar.getRegistrationId(this);
    // Check if regid already presents
    if (regId.equals("")) {
        // Registration is not present, register now with GCM
        try{
            GCMRegistrar.register(this, SENDER_ID);
            Toast.makeText(getApplicationContext(), "Trying to register", Toast.LENGTH_LONG).show();
        }
        catch(Exception e){
            Toast.makeText(getApplicationContext(), "Registration failure", Toast.LENGTH_LONG).show();
        }
    } else {
        // Device is already registered on GCM
        if (GCMRegistrar.isRegisteredOnServer(this)) {
            // Skips registration.
            Toast.makeText(getApplicationContext(), "Already registered with GCM", Toast.LENGTH_LONG).show();
        } else {
            // Try to register again, but not in the UI thread.
            // It's also necessary to cancel the thread onDestroy(),
            // hence the use of AsyncTask instead of a raw thread.
            final Context context = this;
            mRegisterTask = new AsyncTask<Void, Void, Void>() {

                @Override
                protected Void doInBackground(Void... params) {
                    // Register on our server
                    // On server creates a new user
                    ServerUtilities.register(context, userId, "123", regId);
                    return null;
                }

                @Override
                protected void onPostExecute(Void result) {
                    mRegisterTask = null;
                }

            };
            mRegisterTask.execute(null, null, null);
        }
    }

And also Manifest
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.test.activity"
android:versionCode="1"
android:versionName="2" >

<uses-sdk
    android:minSdkVersion="10"
    android:targetSdkVersion="17" />

<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

<uses-permission android:name="android.permission.INTERNET" />

<uses-permission android:name="android.permission.GET_ACCOUNTS" />

<uses-permission android:name="android.permission.WAKE_LOCK" />

<permission
    android:name="com.test.activity.permission.C2D_MESSAGE"
    android:protectionLevel="signature" />

<uses-permission android:name="com.test.activity.C2D_MESSAGE" />

<uses-permission android:name="com.google.android.c2dm.permission.RECEIVE" />

<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />

<uses-permission android:name="android.permission.VIBRATE" />

<application
.
.
.

     <receiver
        android:name="com.google.android.gcm.GCMBroadcastReceiver"
        android:permission="com.google.android.c2dm.permission.SEND" >
        <intent-filter>

            <action android:name="com.google.android.c2dm.intent.RECEIVE" />

            <action android:name="com.google.android.c2dm.intent.REGISTRATION" />

            <category android:name="com.test.activity" />
        </intent-filter>
    </receiver>

    <service android:name="com.test.activity.GCMIntentService" />
</application>
</manifest>

Answer: I'm not sure if it's related to your problem, but this :
<uses-permission android:name="com.test.activity.C2D_MESSAGE" />

Should be changed to this :
<uses-permission android:name="com.test.activity.permission.C2D_MESSAGE" />


PCA & LDA in dimension reduction

Question: I have a dataset split into training and testing dataset. Then I preprocess the training data using scaling, PCA and LDA. As I recalled, I have to use the centre and standard deviation I get from training data on the testing data. Is it the same too for PCA and LDA? (i.e use the transformation matrix of PCA and LDA from training data?)
Answer: Yes, PCA and LDA does not differ from other ML methods. Test data should not be used while standardizing.

PCA & LDA in dimension reduction


SSH tunneling puzzle

Question: I am trying to tunnel vnc through putty (on win vista) to an ssh server on fedora 14. Normal vnc connections work but the tunnel just terminates immediately. My server config file:
<code>VNCSERVERS="2:Robert 3:Christina 4:Lan 5:Simon"
 VNCSERVERARGS[2]="-geometry 1280x800 -localhost"
 VNCSERVERARGS[3]="-geometry 1280x800 -localhost"
 VNCSERVERARGS[4]="-geometry 1280x800 -localhost"
 VNCSERVERARGS[5]="-geometry 1024x600 "
</code>
Nothing of significance is being been reported in any of the logs except for the sshd log on the server which prints out one of the following lines: sshd[8993] error connect_to 192.168.1.64 port 5902: failed except I'm actually not trying to connect to 5902. whichever port I try to tunnel to gives the same error.
Comment: How is the tunnel configured in Putty?
Answer: Problem solved! It turned out that having the having the standard vnc ports forwarded as well as the ssh ports forwarded was causing the issue somehow. (I don't understand how or why at the moment)

SSH tunneling puzzle


IIS only processing one request at a time

Question: We have an asp.net web application that works fine in our environment.  One of our partners tries to the exact same code in their environment and can only process one request at a time.  For example, there is only page that calls a long running stored procedure (5 minutes).  If you call that page and the try to open a new page, the new page won't serve until after the first page has completed.  We can see the request back up by looking at the worker processes for their application pool.   They are running IIS 8.5 on a windows 2012 server.  They're also running SQL Server 2016.
The database connection is structured the same as as in our environment.  The stored procedure is not wrapped in a transaction.  Any ideas what might be causing this behavior?
Comment: If you are using sessions, session access is serialized on a per-user-basis. Solution: Do not use session or do not use long running requests.
Comment: Some years back, I summed this up in a (German only, sorry) developer [forum post](https://entwicklergate.de/t/session-zugriffe-in-asp-net-sind-langsam/219?u=uwekeim). Maybe the links from there help you understanding the possible issue and find alternatives. (You may use the Google Chrome built-in translation feature to translate the forum posting to English).
Answer: IIS also can work in the Debug mode, when it is enabled, Asp is limited to processing one request at a time in a single-threaded manner.
Open "InetMgr" or IIS manager, double click ASP under the IIS section of the website. Check the below settings in Asp section.

See the below discussion for more information.
IIS7 - only serves up one page at a time. It's a making me crAzY!


Notation Question: Times with a subscript

Question: What does the notation $\times_\chi$ mean in this context?
Is it semidirect product?
Context: $\pi_n(E)\cong \text{coker}\times_\chi\ker$, for a cohomology class $\chi\in H^2(\ker,\text{coker})$.
Paper: http://www.unirioja.es/cu/anromero/AAECC-ACA11.pdf Page 9
Thanks for any help.
Answer: From the nLab:

Definition 5. (central extension associated to group 2-cocycle)
For $[c] \in H^2_{\mathrm{Grp}}(G,A)$ a group cohomology class represented by a cocycle $c \colon G \times G \to A$, define a group
  $$G \times_c A \in \mathrm{Grp}$$
  as follows. The underlying set is the cartesian product $U(G) \times U(A)$ of the underlying sets of $G$ and $A$. The group operation on this is given by
  $$(g_1,a_1) \cdot (g_2,a_2)≔(g_1 \cdot g_2, a_1+a_2+c(g_1,g_2))$$
Comment: The main point being that any central extension can be written as such, so there is such a $\chi$ exists.


Is the U factor in LU decomposition for rectangular matrices always in row echelon form?

Question: I have come across the following rectangular 5 x 10 matrix and carried out a LU decomposition of it, in the form PA = LU. The following matrices were obtained by function scipy.linalg.lu from module scipy to Python language. At first I thought was a computational issue, but after a contributor obtained the same result in both matlab and R, I decided to post it here (See: https://stackoverflow.com/questions/25186560/is-the-upper-triangular-matrix-in-function-scipy-linalg-lu-always-in-row-echelon)
\begin{equation}
    A = \begin{bmatrix}
        1 & 1 & 1 & 1 & 0 & 1 & 1 & 1 & 1 & 0 \\
        1 & 1 & 0 & 0 & 1 & 0 & 1 & 0 & 1 & 1 \\
        1 & 1 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 \\
        0 & 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 & 1 \\
        1 & 1 & 0 & 0 & 1 & 1 & 1 & 1 & 1 & 1 \end{bmatrix}
\end{equation}
\begin{equation}
    U = \begin{bmatrix}
        1 &  1 &  1 &  1 &  0 &  1 &  1 &  1 &  1 & 0 \\
        0 &  1 &  0 &  1 &  1 &  0 &  1 &  0 &  0 &  1 \\
        0 &  0 & -1 & -1 &  0 &  0 &  0 & -1 & -1 &  0 \\
        0 &  0 &  0 &  0 &  1 & -1 &  0 &  0 &  1 &  1 \\
        0 &  0 &  0 &  0 &  1 &  0 &  0 &  1 &  1 &  1
      \end{bmatrix}
\end{equation}
Notice that U is upper triangular, though it is not in row echelon form. Element U[5,5] = 1, but it should be zero, since the pivot for row 4 is immediately above it. According to Strang (1988, p.72), the U factor of a LU decomposition of a rectangular matrix should be in row echelon form, with the following characteristics:
"(i) The nonzero rows come first - otherwise there would have been row exchanges - and pivots are the first nonzero entries in those rows. (ii) Below each pivot is a column of zeros, obtained by elimination. (iii) Each pivot lies to the right of the pivot in the row above. This produces the staircase pattern."
STRANG, G. Linear Algebra and its applications. 3rd Ed., Thomson, 1988.
The L factor and permutation matrix P are below. Notice in P that Gaussian elimination swapped rows 2 and 4.
\begin{equation}
    L = \begin{bmatrix}
         1 &  0 &  0 &  0 &  0 \\
         0 &  1 &  0 &  0 &  0 \\
         1 &  0 &  1 &  0 &  0 \\
         1 &  0 &  1 &  1 &  0 \\
         1 &  0 &  1 &  0 &  1
    \end{bmatrix}
\end{equation}
\begin{equation}
    P = \begin{bmatrix}
         1 &  0 &  0 &  0 &  0 \\
         0 &  0 &  0 &  1 &  0 \\
         0 &  0 &  1 &  0 &  0 \\
         0 &  1 &  0 &  0 &  0 \\
         0 &  0 &  0 &  0 &  1 \\
    \end{bmatrix}
\end{equation}
Say one tries to cure the problem by cancelling out element U[5,5] = 1, by subtracting row 5 from row 4, and finally producing the row echelon form. L will not be modified, since L[5,5] will remain equals 1, and $PA \neq LU$. I suspect that Gaussian elimination does not always produce the row echelon form.
Comment: If we write the decomposition you've already done as $A=PLU$ (i.e. our "L" is lower triangular up to permutation of rows), then clearing below the last pivot amounts to writing $A=PLE^{-1}EU$, where $E$ is the elementary matrix representing the row operation, which is the identity plus a matrix which is zero everywhere except at [5,4] and -1 at [5,4]. Then $E^{-1}$ is the identity except for a 1 at [5,4], which inserts a 1 into (PL)[5,4]. So the next step does change $L$.
Comment: Being clearer, we have
$A=\begin{bmatrix}
1 & 0 & 0 & 0 & 0 \\
1 & 0 & 1 & 1 & 0 \\
1 & 0 & 1 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 \\
1 & 0 & 1 & 1 & 1
\end{bmatrix}
\begin{bmatrix}
1 & 1 & 1 & 1 & 0 & 1 & 1 & 1 & 1 & 0 \\
0 & 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 & 1 \\
0 & 0 & -1 & -1 & 0 & 0 & 0 & -1 & -1 & 0 \\
0 & 0 & 0 & 0 & 1 & -1 & 0 & 0 & 1 & 1 \\
0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 \end{bmatrix}$
Answer:
According to Strang (1988, p.72), the U factor of a LU decomposition of a rectangular matrix should be in row echelon form,

What you can conclude from this is that in the 1988 book by Strang, the LU decomposition is defined so that $U$ is in row echelon form. By picking up another linear algebra book, you may find LU described differently. Also, none of those books constitute the documentation of numerical algebra routines.
The LAPACK routine dgetrf (which is probably used by the aforementioned software) promises the following output:

The factorization has the form $A = P  L  U$  where $P$ is a permutation matrix, $L$ is lower triangular with unit  diagonal elements (lower trapezoidal if $m > n$), and $U$ is upper  triangular (upper trapezoidal if $m < n$).

No promise is made to produce row-echelon form. A simpler example than yours is
$$
A = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix}
$$
for which the aforementioned software returns $U=A$.
This may happen for square matrices, too: e.g., $U=A$ is also returned for
$$
A = \begin{pmatrix} 1 & 0 & 0  \\ 0 & 0 & 1  \\ 0 & 0 & 1  \end{pmatrix}
$$
Comment: Thanks, I did not know there was no standard LU decomposition. I have always seen in textbooks the form described by Strang.


Loops and Vector for generating URLs; digits in loop

Question: I want to produce a vector of URLS that correspond to each day of the year, however can't seem to figure out two things: i) I would like to produce one vector of urls with all of the dates but keep getting a matrix that produces one vector for each month (a vector length 372 rather than 12 columns of 31); and ii) I would like to have the single digits concatenate with 0, so that instead of 1,2,3 I get 01,02,03. Code below and many thanks to all responses.
n <- 12
q <- 31
url_vec <- matrix(NA, ncol = 1, nrow = q)
full_url_vec <- as.data.frame(rep(NA, q))

for(j in 1:n){

            url_year <- paste("http://www.parliament.go.ug/hansard/hans_view_date.jsp?dateYYYY=2009&dateMM=", j, "&dateDD=", sep = "")
        for(i in 1:q){
        url <-  paste(url_year, "&dateDD=", i, sep = "")
        url_vec[i,] <- url
    }

full_url_vec[,j] <- url_vec

}

dim(full_url_vec)
full_url_vec

Answer: You dont even need a loop.  paste is vectorized
Also, paste0 saves you the step of using sep=""
i <- 1:31

urlBase  <- "http://yourwebiste.com/?dfdfsdfdfd."
urlSuffix <- "apikey=d323j43k43"

paste0(urlBase, i, urlSuffix)

as far as concatenating with 0's, here is a function, fw0() on github that does that:
LINK: https://github.com/rsaporta/pubR/blob/gitbranch/fw0.R
Answer: Here is one approach: Use sprintf in conjunction with paste. The key here is to make sure that your month vector (j) cycles appropriately, so for that, I used rep.
j <- rep(sprintf("%02d", 1:12), each = 31)
i <- sprintf("%02d", 1:31)
url_year <- paste("http://some/siteMM=", j, "&dateDD=", i, sep = "")
length(url_year)
# [1] 372
head(url_year)
# [1] "http://some/siteMM=01&dateDD=01" "http://some/siteMM=01&dateDD=02"
# [3] "http://some/siteMM=01&dateDD=03" "http://some/siteMM=01&dateDD=04"
# [5] "http://some/siteMM=01&dateDD=05" "http://some/siteMM=01&dateDD=06"
tail(url_year)
# [1] "http://some/siteMM=12&dateDD=26" "http://some/siteMM=12&dateDD=27"
# [3] "http://some/siteMM=12&dateDD=28" "http://some/siteMM=12&dateDD=29"
# [5] "http://some/siteMM=12&dateDD=30" "http://some/siteMM=12&dateDD=31"

Comment: Thanks so much for the help - that's fantastic and so much more efficient.


Optimal harmonic audiation techniques

Question: I am trying to learn to audiate harmonies in order to be able to compose without instruments. I can already audiate melodies, but harmony is a real beast. (Edit: To be clear, I can recognize chords, as well as discern the tones within them, but I cannot synthesize them).  I can just about vaguely hear two tones at the same time; what I hear is devoid of physically heard harmony's richness, unless I have just heard it whereby it is in my short-term memory (though I guess then it is not audiation, but mere repeating of a memory).
My main practice technique is simply to painstakingly go through two lines of Bach's chorales, trying to synthesize each harmony. (I of course also conduct general ear training too). I may be improving, but as is common, it is a tremendously slow process, and I am often disheartened by my incompetence. What other methods would be more efficient, or is it even possible to cultivate it to excellence beyond childhood?
Comment: A brief explanation of what you mean by 'audiation' might be useful - it's not a very commonly-used bit of terminology.
Answer: I'm not sure if I understand exactly what you are asking, but it seems like you want to learn how to recognize the chord degrees within a key, am I right?

I can just about vaguely hear two tones at the same time

Imho, it's more important to identify harmony by their respective characteristics, rather than their individual notes, because it's easier to identify a chord degree by its distinctive sound rather than hearing every single note and then figure out what chord that really is, just like it's easier to identify intervals by their sound, rather than figuring out which exact two notes are played and then measure their distance.

What other methods would be more efficient

There are plenty of programs that can help you recognizing chord degrees within a specific key. EarMaster, for example, has a chord progression training, and with smartphones nowadays, there are plenty of apps that can train you in recognizing chord progressions and thus learn to recognize the characteristics of certain chord degrees.
If you'd follow the order of EarMaster, it's best to start with chord identification - identifying whether you hear a minor/major/augmented/diminished/seventh/etc. chord - followed by learning to recognize these chords in their different inversions and eventually learning to recognize chord progressions.

is it even possible to cultivate it to excellence beyond childhood?

Ear training is something (nearly) everyone can learn, and obviously as we grow older it becomes harder, but almost everyone has the ability to develop a sense of it. The best way to do this is by practicing every day for a few minutes. Whether it's possible to become entirely proficient in ear training is probably something that differs from person to person, but I guess that if there's a will, there's a way?
A few tips that can help you practice:

Learn to recognize the scale functions. So let's say you listen to a tonal melody and you pause it at some point, try to figure out by ear which tone of the scale that last tone was by singing the descending scale to the root, starting from that tone and counting the amount of tones sung.
For recognizing inversions: try to sing the chord you're hearing or playing in an ascending arpeggio from the bass note. Try to find out the interval between the first and second note, and then from the second to the third note (and third to fourth in case of four notes, etc.). Using your theoretical knowledge, you can figure out what inversion it is, and practicing this a lot will make you more familiar with the sound until you can actually hear the inversions.

I hope this more or less answers your question, or at least pull you one step further through your journey!
Comment: I edited the question to dispel ambiguity. But I appreciate these tips; I should have got them when I began my ear training journey a couple years ago. :)

Optimal harmonic audiation techniques


fluent-ffmpeg How to make video rendering synchronous with code?

Question: I want to make the code run only after ffmpeg has finished rendering the videos. Currently, the code runs faster than the videos can render.
videos.forEach((vid, i) => {
  ffmpeg(`${process.cwd()}/video/tmp/${vid}`)
  .outputOptions(['-r 30', '-filter:v scale=w=720:h=1280', '-crf 20'])
  .save(`${process.cwd()}/video/tmp/${vid}`)
  .on('end', ()=> console.log(`Video ${i} rendered`));
});
console.log("Fully Completed");

The console shows:
Fully Completed
Video 0 rendered
Video 1 rendered

The execution should be opposite. How can I make the code wait for video to finish rendering before continuing?
Answer: the code below will help you:
async function processVideos(){
    let videos = [vid1.mp4, vid2.mp4, vid3.mp4];
    for(vid of videos){
        await processVideoSync(vid)
    }
}
function processVideoSync(vid){
    return new Promise((resolve,reject)=>{
       ffmpeg(`${process.cwd()}/video/tmp/${vid}`)
       .outputOptions(['-r 30', '-filter:v scale=w=720:h=1280', '-crf 20'])
       .save(`${process.cwd()}/video/tmp/${vid}`)
       .on('end', ()=>{
          console.log(`Video ${i} rendered`)
          return resolve()
        })
        .on('err',(err)=>{
            return reject(err)
        })
    })
}

o loop for of é o unico capaz de esperar o async await.


Django: change admin FileField output?

Question: I've made a model with file that is uploaded to custom path (not in MEDIA_ROOT). So it's some kind like protected file.
Now I need to change it's representation in admin details. It shows a path relative to MEDIA_URL. I need to change that, to show a URL to an application view which generates a proper URL.
So, what is the best way to display link, and  only in objects details in admin?
Answer: Here is the way I did it:
models.py
class SecureFile(models.Model):
    upload_storage = FileSystemStorage(
        location=settings.ABS_DIR('secure_file/files/'))

    secure_file = models.FileField(verbose_name=_(u'file'),
                                   upload_to='images', storage=upload_storage)

widgets.py
from django import forms
from django.utils.translation import ugettext_lazy as _
from django.core.urlresolvers import reverse
from django.utils.safestring import mark_safe

class AdminFileWidget(forms.FileInput):
    """A FileField Widget that shows secure file link"""
    def __init__(self, attrs={}):
        super(AdminFileWidget, self).__init__(attrs)

    def render(self, name, value, attrs=None):
        output = []
        if value and hasattr(value, "url"):
            url = reverse('secure_file:get_secure_file',
                          args=(value.instance.slug, ))
            out = u'{}
{} '
            output.append(out.format(url, _(u'Download'), _(u'Change:')))
        output.append(super(AdminFileWidget, self).render(name, value, attrs))
        return mark_safe(u''.join(output))

admin.py
class SecureFileAdminForm(forms.ModelForm):
    def __init__(self, *args, **kwargs):
        super(SecureFileAdminForm, self).__init__(*args, **kwargs)
        self.fields['secure_file'].widget = AdminFileWidget()

    class Meta:
        model = SecureFile

class SecureFileAdmin(admin.ModelAdmin):
    form = SecureFileAdminForm

Django: change admin FileField output?


R Leaflet renders only 6 out of 650+ points on map

Question: I have been using Leaflet before for a different dataset, which was from a CSV file. This dataset however, is in a Data Frame, which has 675 rows (some lat and lon are NA values, but most are there).
The problem is that I would expect there to be at least 500 or more points on the map, but it seems to render only about 6. Anyone experienced this before?
renderUiMeetup <- function(data){
  return (leaflet(data) %>% addTiles('http://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}.png') %>%
            setView(5, 52.37, zoom = 11) %>%
            addCircles(~results.venue.lon, ~results.venue.lat, weight = 1, radius=2,
                       color="#000", stroke = TRUE, fillOpacity = 0.8))
}

When I check the data, it is indeed the dataframe with 675 rows. This function is called in a reactive functions, if that matters (I assume not, since each time I see the dataframe inside the function still returns what it should)
Any help appreciated, do not know where this is coming from ..
EDIT: Some data (meetup data)
                  results.venue.address_1                 results.venue.name results.venue.lon results.venue.id
1                      Amstel 1 Amsterdam        Raadzaal Stadhuis Amsterdam          4.900240         24631310
2                               16 Notweg                     FitClub Bergen          4.699218         24632049
3                               16 Notweg                     FitClub Bergen          4.699218         24632049
4                            Nieuwmarkt 4                       Waag Society          4.900210         23744775
5                            Overtoom 301         Anamorphic Studio in OT301          4.865707         16443062
6                        Langs de Akker 3                          Emergohal          4.879670         23816542
7                                    <NA>                               <NA>                NA               NA
8                 Goeman Borgesiuslaan 77                 Ibuildings Utrecht          5.113864         19126412
9                          Amstelstraat 2 Vapiano Restaurant - Rembrantplein          4.910965         12762372
10                    Amstelveenseweg 130                        All4Running          4.855405         23844308
11                           Overtoom 301                              OT301          4.865604          6736512
12 Ceintuurbaan 338, 1072 GN Amsterdam |                   The Rialto Cinema          4.893891         24690797
13                       Weteringschans 6                           Paradiso          4.884215           698679
14                   Singel 165A 1012 WE                       Het Spinhuis           4.889594         24224032
15                    Recht Boomssloot 52                    De Boomsspijker          4.903769          1110122
16                          Oosterpark 10                       Bar Bukowski          4.917633         21271582
17                Nieuwe Achtergracht 170                               Crea          4.912870          9152402
18                    James Wattstraat 10             Basisschool 'Spectrum'          4.925560         23633711
19                      Volmolengracht 13                  MakerSpace Leiden          4.493560         23445132
20                 Lizzy Ansinghstraat 88                   Sporthal de Pijp          4.893200         23594478
21                      ketelhuisplein 41                    WG cursusruimte          4.869719          1500646
22                              Jansdam 3          Mick O'Connells Irish Pub          5.122060           702533
23                             Spui 25-27                            Spui 25          4.889688          2549081
24          Arthur van Schendelstraat 650                        HNK Utrecht          0.000000         24251942
25       Vondelpark 6B, 1071 AA Amsterdam                  Kinderkook cafe            4.864780         23666288
   results.venue.lat results.venue.repinned results.venue.phone results.venue.address_2 results.rating.count
1           52.36768                  FALSE                <NA>                    <NA>                    0
2           52.67305                  FALSE                <NA>                    <NA>                    0
3           52.67305                  FALSE                <NA>                    <NA>                    0
4           52.37271                  FALSE                <NA>                    <NA>                    1
5           52.36014                  FALSE                <NA>                    <NA>                    0
6           52.29020                  FALSE                <NA>                    <NA>                    0
7                 NA                     NA                <NA>                    <NA>                    8
8           52.10300                  FALSE                <NA>                    <NA>                    1
9           52.34731                  FALSE                <NA>                    <NA>                    0
10          52.35427                  FALSE                <NA>                    <NA>                    0
11          52.36013                  FALSE                <NA>                    <NA>                    0
12          52.35298                  FALSE                <NA>                    <NA>                    0
13          52.36223                  FALSE      020 - 626 45 2                    <NA>                    1
14          52.37452                   TRUE                <NA>                    <NA>                    0
15          52.37190                  FALSE        020-626-4002                  1011EC                    0
16          52.35798                  FALSE                <NA>                    <NA>                    0
17          52.36328                  FALSE                <NA>                    <NA>                    0
18          52.35243                  FALSE                <NA>                    <NA>                    0
19          52.16282                  FALSE                <NA>                    <NA>                    1
20          52.34951                  FALSE                <NA>                    <NA>                    0
21          52.36355                  FALSE                <NA>                    <NA>                    0
22          52.09256                  FALSE        030 236 8466                    <NA>                    0
23          52.36859                  FALSE                <NA>                    <NA>                    6
24           0.00000                  FALSE                <NA>                    <NA>                    2
25          52.35920                  FALSE                <NA>                    <NA>                    0

Comment: Is there a chance you can share your data? It's kinda hard to guess what's going on...
Comment: Sorry, but without the original data I don't think I can help you any further.
Comment: I seem to recall that **leaflet** has problems with NAs in the coordinate columns. Does it work if you delete those rows with no coordinates?
Comment: @TimSalabim I see more points indeed, I used `vals <- apply(cbind(complete.cases(data$results.venue.lon), complete.cases(data$results.venue.lat)), 1, all)`, which leaves 580 rows. Still not even close to that amount of points rendered unfortunately

***Edit: I now see this replaces those with 0.0000 values, probably has problems with those too***
Comment: @TimSalabim Sure, it's meetupdata. Might be shitty formatted now but I selected some entries. As you see there is one there record which has NA for `results.venue.lon` and `results.venue.lat`. I'm guessing this is where it goes wrong. With the function I applied above these values are replaced by 0.0000 but that won't help
Comment: how about you create an index of the NA coordinates `ind <- is.na(data[["results.venue.lon"]]) | is.na(data[["results.venue.lat"]])` and then delete those rows from your data `data <- data[!ind, ]`
Comment: @TimSalabim Seems to return the same as first, so actually more points than 6 but way less than it should
Comment: @TimSalabim Thanks anyways, I don't think this is the problem .. all the NA and 0.000000 values are filtered out now, but it's still not good so
Comment: use dput() to add a subset of your dataset to your post.
Answer: Leaflet will stop parsing data points when it hits the first error. To see all the points on your map, you will need to remove all of the NA values:
data <- data[!is.na(data$lat), ]


how to append filename.pdf to url to browser

Question: My controller action method returns pdf file as
public FileContentResult GetPDF(string filename)
        {FileContentResult filecontent= new FileContentResult(Contents, "application/pdf");
               HttpContext.Response.AddHeader("Content-Disposition", "inline; filename=" + filename);
               return filecontent;
}

Now, it is not opening in a browser tab , so i want to return to url as filename.pdf at end that is how i can open pdf in browser tab.So how can i add that to my existing action method.
thanks,
 michaeld
Answer: Changing the browser url from server and at the same time downloading file content is impossible. Browser expects results to be received from the url it makes request to, so you cannot change it after you serve the file. So you have to have the handler for the url ending with ".pdf", and make the browser request files from that url, already ending with ".pdf". With asp.net mvc, that is simple. In your route configuration, you should now have something similar to
routes.MapRoute("PdfFile",
                "DownloadFile/GetPDF/{fileName}",
                new { controller = "DownloadFile", action = "GetPDF" });

And now, change the route to match only when url ends with ".pdf"
routes.MapRoute("PdfFile",
                "DownloadFile/GetPDF/{fileName}.pdf",
                new { controller = "DownloadFile", action = "GetPDF" });
//note the ending .pdf at the end of the route

And if you have designed your system to generate links authomatically using ActionLink or other helpers, and not have written links by hand, you should already get the desired result - your links that download file now would automatically be generated to end with .pdf.
There's some technique in http when browser(or generally who makes the request) can be told that the resource it requests has been moved to some other url. That is supported by asp.net mvc too, by returning RedirectToAction or similar actionResults. When the browser receives redirect response, it makes another request to that new url. In your case, this could have been used to redirect browser to url ending with ".pdf". But the key point here is that even in case of redirection of course you have to handle that url ending with ".pdf" :). So its better to get rid of those redirections and initially generate links that already point to desired url, by means changing the route configuration.


Converting few indexes of the list to hex

Question: I have an integer list arr with five elements/indexes ranging from arr[0] to arr[4]
The last three elements are customer number which ranges from 01 to 250,000. What I need is to convert them to hex in such a way that the array should look like:
For customer 1,
arr = [00,00,01,00,00] - cust num is the last three bytes
For customer 2,
arr = [00,00,02,00,00] - cust num is the last three bytes
For customer 250,000,
arr = [00,00,90,d0,03] - cust num is the last three bytes, 250,000 in hex = 3D090
I know I can use hex() function to individually convert the indexes to hex or can convert the whole array to hex, but how do I convert the value to hex and place it in last three bytes as in the above format?
Comment: So you have a nested list?
Comment: This is just a sample of what I'm trying to achieve. If I get to know how do I convert the last three bytes to hex in the above format, I can modify the code accordingly :) Yes, the original program has a nested list..
Comment: I think what's going on here is a misunderstanding of what "hex" is. It is but a way of representing a number, therefore you don't really need to store it in hex, you can just convert it on the go, and as to how to split it into numbers representing 2 digits... 1 hex digit can represent 16 numbers (0-15), therefore 2 represent 16x16=256 (0-255) so you get last 2 by 250 000 % 256 = 144 (90 in hex), next 2 you get by (250 000/256)%256 and so on. Just be careful with division, you always want it rounded down here.
Answer: This will work:
i = 250000

constant = ['00', '00']

result = [constant + [first + second
                      for first, second in zip(string[::2], string[1::2])][::-1]
          for string in (f'{i:X}'.zfill(6) for i in range(1, 250001))]
print(result[0])
print(result[-1])

For Python <= 3.5:
i = 250000

constant = ['00', '00']

result = [constant + [first + second
                      for first, second in zip(string[::2], string[1::2])][::-1]
          for string in ('{:X}'.format(i).zfill(6) for i in range(1, 250001))]
print(result[0])
print(result[-1])

Output:
['00', '00', '01', '00', '00']
['00', '00', '90', 'D0', '03']

Explanation:
First, we recognise that we can use string formatting to convert each number into a string, which is then padded to length 6 with 0 if needed.
Next, we chunk the string into two-character blocks and reverse it so that it has the representation required.
Lastly, we add two '00' strings to each inner list.
Comment: Precisely what I was looking for, but when I try to compile this, I get the following error: for string in (f'{i:X}'.zfill(6) for i in range(1, 250001))]
                          ^
SyntaxError: invalid syntax. Am I doing something wrong?
Comment: @AmithPV You're probably using Python <= 3.5. I'll include a modification for earlier Python versions.
Comment: Thanks! I am using Python 2.7.9 and your modified code worked. Thanks a lot!
Answer: Might help

import re
def split_hex_rev(number):
    arr = re.findall('..',hex(number).replace('x',''))
    while len(arr)<5:
        arr.append('00')
    arr.reverse()
    return arr


Significant result with one-way anova test but not with Scheffe test?

Question: I conducted one-way anova test and get a significant result, but when I conducted Scheffe test there was no significant result. Is this possible? How should I proceed or report the results?
Comment: @whuber Thanks, yes, that's relevant. There's also [this](http://stats.stackexchange.com/questions/29248/strange-result-of-post-hoc-test), and it can also happen [the other way around](http://stats.stackexchange.com/questions/35175/non-significant-group-effect-but-lsd-post-hoc-is-significant-why) (though you wouldn't normally do a post hoc in that case).
Comment: @Glen_b One of them might be http://stats.stackexchange.com/questions/14500.
Comment: Yes, it's possible. There are many questions here on related issues. I'll see if I can dig one or two up.
Answer: Scheffe's test allows you to do more than compare group means. You can compare the average of groups A and B with the average of groups C, D and E. Etc. The only real reason to choose Scheffe's test for multiple comparisons is to do that kind of contrast. If you only compare group means, this test has less power than others (Tukey, Dunnett).
The Scheffe's test (unlike Tukey etc.) guarantees you that if the overall ANOVA is statistically significant then at least one contrast will be too (page 217, Maxwell and Delaney, 2nd edition). But that contrast may not be a simple comparison of group means, but rather may be a complex comparison involving many groups.


How to create an end date field in a hook_form_alter

Question: I've succeeded creating a simple date_popup field with the following code :
<code>$form['date_start'] = array(
  '#type' => 'date_popup',
  '#title' => t('Start date'),
  '#default_value' => '',
  '#date_format' => 'Y-m-d',
);
</code>
Now, is there any way to directly create the "End date" field without having to create a second field declaration ? Is there any bundled widget I can use ? I've seen something about date_combo but I'm not sure to understand it well.
Thanks for help !
Answer: there is no way to do it with the standard field widgets, you have to craft your own, the examples module may help you
Comment: this may help you https://api.drupal.org/api/drupal/modules!field!field.api.php/group/field_widget/7


Dovecot LDA (deliver) and LDAP groups

Question: I have a LDAP and a Dovecot running on my server, the Dovecot uses LDAP as userdb and authdb, postfix is using LDA (deliver) to deliver the mails to the user. Now I want to have group email addresses like firstname.lastname@example.com. This group is in my ldap like this:
objectClass: extensibleObject
objectClass: groupOfUniqueNames
objectClass: top
cn: group1
uniqueMember: uid=user1,ou=people,dc=example,dc=com
uniqueMember: uid=user2,ou=people,dc=example,dc=com
(...)
mail: firstname.lastname@example.com

How can I get my dovecot LDA to except emails sent to email@example.com and then send the mail to the uniqueMembers of the group?
Thanks
Philip
Answer: This is not the job of Dovecot (or it's LDA). This is the job of Postfix to expand the user. And therefore there are the virtual_alias_maps that do the expanding/rewriting. Each expanded user will then get the mail as before.
You just have to set the virtual_alias_maps to get the mapping from LDAP as described here, but you probably are familiar with that as you already get your normal users from LDAP.
Comment: Thanks, that helps for a start but I have two problems with it: First all mail: attributes will be used and not just the first and second: I get a "user not found" bounce from the server even though the mail gets delivered to all users.

Dovecot LDA (deliver) and LDAP groups


Can I make partial screenshot of page by chrome browser?

Question: Can I make partial screenshot of the page after load in Chrome browser? It's the best if I can make screenshot of the chosen DOM element. I think about normal client JavaScript browser code, without any extensions.
Comment: Did you make any research at all? Because a simple Google search lead me to [html2canvas](https://html2canvas.hertzen.com/)
Comment: Wow, thanks it's should be great. Yes I was looking for that, but unfortunately didn't find anything working.
Answer: In your Google Chrome Browser press F12 followed by Ctrl+Shift+p and search for capture full size and click enter.
An in-built full-size screenshot tool provided by Chrome
Comment: I believe OP wants a screenshot made with JS, for their users, not for himself.


If $X$ and $Y$ are random variables, is $Z=(X, Y)$ a random variable?

Question: What definition do we use for random variable? How do we show that $Z=(X,Y)$ is a random variable?
Comment: What do you mean by the brackets?
Comment: It is a bivariate distribution for a random vector.
Answer: We would call $Z$ a random vector (but yes, it is random).
Random variables are defined within a probability space that includes a sample space $\Omega$.  Within such a space, real-valued random variables $X:\Omega \rightarrow \mathbb{R}$ and $Y:\Omega \rightarrow \mathbb{R}$ are functions$^\dagger$ from the sample space to Euclidean space.  You have $Z \equiv (X,Y)$ so $Z(\omega) = (X(\omega), Y(\omega)) \in \mathbb{R}^2$ for all $\omega \in \Omega$, which means that the latter is a mapping $Z:\Omega \rightarrow \mathbb{R}^2$.$^\dagger$  This means that it is a two-element real random vector.

$^\dagger$ Formally, random variables must be measurable functions, which is a formal requirement from measure theory.  If you are new to dealing with random variables, just ignore this part; I mention it for completeness.  It can be shown that measureability of $X$ and $Y$ implies measureability of $Z$, so this requirement is met in this case.  (Hat-tip to whuber in the comments for noting the importance of the latter property.)
Comment: Since you're being formal--which is fine--consider including a remark about the key idea, which is that the measurability of $X$ and $Y$ implies the measurability of $Z$ with respect to the product measure on $\mathbb{R}^2.$
Comment: There's no such general theorem simply because $Z$ has different mathematical properties than $X$.  For instance, values of $Z$ cannot be ordered whereas values of $X$ can be, so any theorem that refers to ordering values of $X$ couldn't be applied to $Z.$  The solution to your problem is obtained simply by applying the definitions of measurability and product measure.
Comment: @Zhanxiong You lost me, because measure is defined without reference to any random variable.  Measurability of a random variable is with respect to the Borel sigma field on $\mathbb{R}$ and measurability of a 2D random variable is with respect to the Borel sigma field on $\mathbb{R}\times\mathbb{R}$. I don't see how you can even talk about a 2D "random variable" without having a measure on $\mathbb{R}^2$ in the first place.
Comment: @whuber Zhanxiong: My understanding of this is the same as [whuber](https://stats.stackexchange.com/users/919/whuber) - i.e., you need to be able to form a sigma-field on $\mathbb{R}^2$.  I have edited my answer to move all the measurability stuff to a footnote, so that the OP is not scared off with a bunch of formal measure theory.
Comment: @Zhanxiong: Sorry, I am probably not explaining myself well.  My understanding is that we want to ensure that $Z = (X,Y)$ is a measureable function, which means that we want to ensure that the class $\{ Z(\mathcal{E}) | \mathcal{E} \in \mathscr{F} \}$ is a sigma-field (where $\mathscr{F}$ is the sigma field on $\mathbb{R}^2$ in the original probability space).  So long as you can ensure that this class is a sigma-field (which it is), you can say that $Z$ is a measureable function to $\mathbb{R}^2$ and then you're fine.
Comment: If we have a theorem which is true for a random variable, e.g., it is true for $X$ and $Y$, is it true for a random vector $Z=(X,Y)$? Can we apply the theorem to this random vector?
Comment: @whuber Regarding measurability only, we do not need introduce any measure. Specifically, not product measure. To my knowledge, product measure refers to the case that $X$ and $Y$ are independent, while here OP clearly doesn't restrict to be that case.
Comment: @whuber To talk about measurability, what you only need is a measurable space, which doesn't need to be equipped with any measure. Please refer to Section 13 of Billingsley's book. I certainly can talk about a 2D random variable without having a measure on $\Omega$ or $\mathbb{R}^2$ in mathematical sense (though it is less interesting in application). After all, random variables/vectors are nothing but functions/mappings.
Comment: @Zhanxiong I agree with all that: the issue is one of pulling back the Borel sigma algebra, pure and simple.  My understanding of the term "measurable" in this context does not include that a measure *is* defined, but that the collections of measurable sets (that is, the sigma algebras) on $\Omega$ and $\mathbb R$ are determined.
Comment: @Ben Just make it clear, to form a sigma-field on $\mathbb{R}^2$ doesn't need to specify a measure --- that's all my point, I didn't say we don't need a sigma-field.
Answer: To be formal, you need to set up a probability space $(\Omega, \mathscr{F}, P)$ (to discuss the measurability only, the probability measure $P$ is even not required, as the argument below shows). What you want to confirm is that given $X$ and $Y$ are two $\mathscr{F}$-measurable mappings from $\Omega$ to $\mathbb{R}^1$, then $Z$ is an $\mathscr{F}$-measurable mapping from $\Omega$ to $\mathbb{R}^2$.
This is ready to check by noting that for any $(x, y) \in \mathbb{R}^2$, we have
\begin{align}
& \{\omega: Z(\omega) \leq (x, y)\} \\
= & \{\omega: X(\omega) \leq x, Y(\omega) \leq y\} \\
= & \{\omega: X(\omega) \leq x\} \cap \{\omega: Y(\omega) \leq y\} \in \mathscr{F}
\end{align}
as both sets after the last equality sign are in $\mathscr{F}$ by measurability of $X$ and $Y$ and that the $\sigma$-field $\mathscr{F}$ is closed under intersection.
Note also the converse of the statement is also true: if $Z$ is a random vector, then each of its component is a random variable.


iOS Can't Get PDF to Appear in UIWebView

Question: I'm having issues displaying a PDF in a UIWebView - something that should be really trivial (even for me beginner level). However I'm not seeing anything.
I have a UIWebView correctly connected as an IBOutlet, then in the viewDidLoad, i have the following code:
NSURL *targetURL = [NSURL URLWithString:@"/Users/tom........../PDF_6.pdf"];
NSURLRequest *request = [NSURLRequest requestWithURL:targetURL];
[_pdfWebView loadRequest:request];

Any ideas what I'm doing wrong??
Thanks.
(Apologies if this is a really duplicated question i just can't find the answer).
Answer: Since it seems you want to load the PDF from a file, you need to create a file URL:
NSURL *targetURL = [NSURL fileURLWithPath:@"/Users/tom........../PDF_6.pdf"];

BTW - that path is only going to work in the simulator, not on a real iOS device.
Comment: Thanks!! I really need to pay more attention !
Comment: Haha no worries mate - was waiting for the 6 mins :)


Efficient Stopwatch

Question: Hi I'm programming a stopwatch utility in javascript and I have a question about efficiency and overhead. There are two ways I have considered making the stopwatch:
1.Store a start Date and constantly measure the number of milliseconds it has been since that date.
2.Create an integer and increment its value at a set interval.
I want to know which is most efficient. Also, I'm not sure if option #2 would be very accurate, if anyone has any input about this that would be awesome as well.
Comment: #1 is both more efficient AND #2 will not work, because you can't guarantee the time between 'ticks'.
Comment: `performance.now()` is available in modern browsers https://developer.mozilla.org/en-US/docs/Web/API/Performance/now
Answer: Option 2 will not be accurate, especially if you have a page with additional javascript for other purposes.  Go with the first approach.
Comment: However, with option #1 a *clock change* should be accounted for. That is, the expected change and the actual change should be compared (with an appropriate margin of error) -- and corrected if needed. Granted, it's a very edge-case.
Answer: As others have said, go with #1. If you want a clock that ticks each second (or minute or whatever) you should estimate the time to the next "tick" so that setTimeout is called a few ms after the right time, e.g. to run just after the next whole second:
<code>var d = new Date();
var interval = 1020 - d.getMilliseconds();
setTimeout(fn, interval);
</code>
That way if execution for one call is delayed by the system being busy, the next one should still be called about 20ms after the next whole second.
Comment: This can still miss timeouts (e.g. computer goes to sleep); a delta inside the timeout is still required, depending upon what is required.
Comment: Of course anything in a browser is dependent on the browser executing the script. What do you mean by a "delta inside the timeout"?
Comment: The point about making sure the clock is updated at the right time is interesting. However, without this the clock will still display the right time even if it is behind on the display, won't it?
Comment: @Drew - it will display the right time *when it updates*, but it may show the wrong time (for a shorter or longer period, depending on the regularity of the update) before that. The idea is to have it update as regularly as possible — i.e. as close to the "tick" as possible — as even minor lag or jerkiness in regular updates will be apparent to the user and cause them to lose confidence in the timer.

Efficient Stopwatch