Uso spring-boot e @component com @scheduled e ele trava a cada 12 horas

Question: Tenho um projeto feito com JHipster. Ele usa spring-boot. Uso esta aplicação como plataforma de pedidos onde consumo vários webservices de um ERP e também soncronizo os dados com dispositivos android na versão mobile da plataforma.
Para consumir os WSs do ERP, utilizo uma classe com annotation @Component e @Scheduled(fixedDelayString = "90000") para que funcione como um job a cada 90 segundos. Ele consome 11 webservices. O problema é que esse Job para de funcionar a aproximadamente cada 12 horas.
Tenho um outro Job semelhante rodando no mesmo projeto, mas que não consome nenhum recurso externo, apenas faz cálculos.
Os 2 Jobs param ao mesmo tempo e apenas alguns metodos REST funcionam.
Alguém faz ideia do que possa ser?
segue classe:
<code>package br.inf.gati.gts.job;

@Component
public class SyncErpJob {

    private final Logger log = LoggerFactory.getLogger(SyncErpJob.class);

    @Autowired
    private EmpresaFilialRepository empresaFilialRepo;

    @Autowired
    private ClienteRepository clienteRepo;

    @Autowired
    private ProdutoRepository produtoRepo;

    @Autowired
    private CondicaoPagtoRepository condPagtoRepo;

    @Autowired
    private TransportadoraRepository transpRepo;

    @Autowired
    private TabelaPrecoRepository tabPrecoRepo;

    @Autowired
    private TabelaPrecoProdutoRepository tabPrecoProdRepo;

    @Autowired
    private ExcecaoFiscalRepository excecaoRepo;

    @Autowired
    private OperacaoFiscalRepository operacaoRepo;

    @Autowired
    private ParamImpostosRepository impostosRepo;

    @Autowired
    private PedidoVendaRepository pedidoRepo;

    @Autowired
    private ContasReceberRepository contasRepo;

    @Autowired
    private MailService mailService;

    @Value("${gts.webservice.soap.ativo}")
    public Boolean ativo;

    @Value("${gts.webservice.soap.erp.date.format}")
    public String dateFormatPattern;

    @Value("${gts.webservice.soap.erp.date.timezone}")
    public String timeZoneParam;

    @Value("${gts.webservice.soap.erp.url}")
    private String url;

    @Value("${gts.webservice.soap.erp.port}")
    private String port;

    @Scheduled(fixedDelayString = "${gts.webservice.soap.erp.time}")
    public void ConsumoSOAPErp() {

        if(ativo){
            log.info("SEJ-MSG001: Percorrendo Webservices de consumo de dados do ERP!");

            AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(SoapClientConfig.class);

            List<EmpresaFilial> listaFiliais = empresaFilialRepo.findAll();
            for (EmpresaFilial filial : listaFiliais){
                log.info("SEJ-MSG003: Empresa:["+filial.getCdEmpresa()+"] Filial:["+filial.getCdFilial()+"]");

                consomeCliente(filial, context);
                consomeProduto(filial, context);
                consomeCondPagto(filial, context);
                // demais WS
            }

            context.close();

        }else{
            log.debug("Consumo de werbservices inativo!");
        }
    }

    private void consomeCondPagto(EmpresaFilial filial, AnnotationConfigApplicationContext context) {

        try{
            CondPagtoWsClient condPagtoListaWs = context.getBean(CondPagtoWsClient.class);

            condPagtoListaWs.setUrl(url);
            condPagtoListaWs.setPort(port);
            condPagtoListaWs.setDefaultUri(url+":"+port+"/WSDLCPGTO.apw");

            CONDSLSTRESPONSE responseList = condPagtoListaWs.getCondPagtoLstResponse(filial.getCdEmpresa(), filial.getCdFilial());
            List<STCPGTO> wsCondPagto = responseList.getCONDSLSTRESULT().getAREGS().getSTCPGTO();
            for (STCPGTO stCond : wsCondPagto){
                log.info("SEJ-MSG020: Cond.Pagto: cdCod:["+stCond.getCE4COD() +"] "
                        + "Cond. Pagto:["+stCond.getCE4COND() +"]"
                        + "dsCondPagto:["+stCond.getCE4DESCRI()+"]");

                try{
                    BigInteger regNumErp = stCond.getCE4RECNO();

                    CondicaoPagto cond = null;
                    try{
                        cond = condPagtoRepo.findByFilialIdAndCdCondPagto(filial.getId(),
                                StringUtils.truncate(stCond.getCE4COD().trim(), 3));
                    }catch (NullPointerException e) {}

                    if (cond == null){
                        cond = new CondicaoPagto();
                        cond.setFilial(filial);
                        cond.setCdEmpresa(          StringUtils.truncate(filial.getCdEmpresa(),2));//c2
                        cond.setCdFilial(           StringUtils.truncate(filial.getCdFilial(),2));//c2
                        cond.setCdCondPagto(        StringUtils.truncate(stCond.getCE4COD(),3));//c3
                    }

                    cond.setCdCondicao(         StringUtils.truncate(stCond.getCE4COND(),60));//c60
                    cond.setDsCondPagto(        StringUtils.truncate(stCond.getCE4DESCRI(),60));//c60
                    cond.setNrDiasPrazoMedio(   new BigDecimal(stCond.getCE4PRAZO()));//float14,2
                    cond.setDtSincronizacao(    ZonedDateTime.now());
                    cond.setIdAutorCriacao(     Constants.SERVER);

                    CondicaoPagto condSalvo = condPagtoRepo.save(cond);

                    if (condSalvo != null){
                        CONDSRETRESPONSE responseRet = condPagtoListaWs.setCondPagtoRetResponse(filial.getCdEmpresa(), filial.getCdFilial(), regNumErp);

                        if (responseRet != null){
                            if (responseRet.getCONDSRETRESULT().getCRET().trim().equalsIgnoreCase(Constants.RET_SUCESSO)){
                                log.info("SEJ-MSG021: Cond. Pagto. Registro ["+regNumErp+"] integrado e notificado com successo!");
                            }else if (responseRet.getCONDSRETRESULT().getCRET().trim().equalsIgnoreCase(Constants.RET_FALHA)) {
                                log.error("SEJ-MSG022: Cond. Pagto. Registro ["+regNumErp+"] integrado e notificado com falha!");
                            }else {
                                log.error("SEJ-MSG024: Cond. Pagto. Registro ["+regNumErp+"] integrado e notificado com falha no retorno!");
                                log.warn("SEJ-MSG025: msg["+responseRet.getCONDSRETRESULT().getCMESSAGE()+"]");
                                log.warn("SEJ-MSG026: ret["+responseRet.getCONDSRETRESULT().getCRET()+"]");
                            }
                        }else{
                            log.error("SEJ-MSG018: Sem retorno da notificação ao ERP!");
                        }
                    }else{
                        log.error("SEJ-MSG019: Falha ao integrar Cond. Pagto. registro ["+regNumErp+"]");
                    }
                }catch (Exception e) {
                    log.error("SEJ-ERR005: Erro no consumo de condição de pagamento", e);
                }
            }
        }catch (RuntimeException e) {
            log.error("SEJ-ERR006: Falha de comunicação no consumo de condição de pagamento", e);
        }
    }

    private void consomeProduto(EmpresaFilial filial, AnnotationConfigApplicationContext context) {

        try{
            ProdutoWsClient produtoListaWs = context.getBean(ProdutoWsClient.class);

            produtoListaWs.setUrl(url);
            produtoListaWs.setPort(port);
            produtoListaWs.setDefaultUri(url+":"+port+"/WSDLProd.apw");

            PRODUTOLSTRESPONSE responseList = produtoListaWs.getProdLstResponse(filial.getCdEmpresa(), filial.getCdFilial());
            List<STPROD> wsProdutos = responseList.getPRODUTOLSTRESULT().getAREGS().getSTPROD();
            for (STPROD stProd : wsProdutos){

                log.info("SEJ-MSG012: Produto: cdProduto:["+stProd.getCB1COD() +"] "
                        + "dsProduto:["+stProd.getCB1DESC()+"]");

                try{
                    BigInteger regNumErp = stProd.getCB1RECNO();

                    Produto prod = null;
                    try{
                        prod = produtoRepo.findByCdEmpresaAndCdFilialAndCdProduto(filial.getCdEmpresa(),
                                filial.getCdFilial(),
                                StringUtils.truncate(stProd.getCB1COD().trim(),15));
                    }catch (NullPointerException e) {}

                    if (prod == null){
                        prod = new Produto();
                        prod.setFilial(filial);
                        prod.setCdEmpresa(          StringUtils.truncate(filial.getCdEmpresa().trim(),2));//c2
                        prod.setCdFilial(           StringUtils.truncate(filial.getCdFilial().trim(),2));//c2
                        prod.setCdProduto(          StringUtils.truncate(stProd.getCB1COD().trim(),15));//c15
                    }

                    prod.setDsProduto(          StringUtils.truncate(stProd.getCB1DESC().trim(),120));//c120
                    prod.setCdFamilia(          StringUtils.truncate(stProd.getCB1FPCOD().trim(),2));//c2
                    prod.setCdUnidadeMedida(    StringUtils.truncate(stProd.getCB1UM().trim(),2));//c2
                    prod.setCdGrupo(            StringUtils.truncate(stProd.getCB1GRUPO().trim(),4));//c4
                    prod.setCdGrtrib(           StringUtils.truncate(stProd.getCB1GRTRIB().trim(),6));//c6
                    prod.setPcIpi(              new BigDecimal(stProd.getCB1IPI()));//float5,2
                    prod.setlImportZnFranca(    StringUtils.truncate(stProd.getCB1IMPZFRC().trim(),1).equals(Constants.SIM) ? true : false );//c1
                    prod.setCdProdutoSubst(     StringUtils.truncate(stProd.getCB1ALTER().trim(),15));//c15
                    prod.setVlCmvStandard(      new BigDecimal(stProd.getCB1CUSTD()));//float16,6
                    prod.setVlCmvRazao(         new BigDecimal(stProd.getCCMVRAZAO()));//float16,6
                    prod.setIdOrigem(           StringUtils.truncate(stProd.getCB1ORIGEM().trim(),1));//c1
                    prod.setCdClassFiscal(      StringUtils.truncate(stProd.getCB1POSIPI().trim(),10));//c10
                    prod.setQtEstoqueDisp(      new BigDecimal(stProd.getCESTOQUE()));//float16,0
                    prod.setPcReducPis(         new BigDecimal(stProd.getCB1REDPIS()));//float5,2
                    prod.setPcReducCofins(      new BigDecimal(stProd.getCB1REDCOF()));//float5,2
                    prod.setPcPis(              new BigDecimal(stProd.getCB1PPIS()));//float5,2
                    prod.setPcCofins(           new BigDecimal(stProd.getCB1PCOFINS()));//float5,2
                    prod.setIdSitTrib(          StringUtils.truncate(stProd.getCB1SITTRIB().trim(),1));//c1
                    prod.setDsFichaTecnica(     StringUtils.truncate(stProd.getCFICHADESC(),200));//c200

                    //                  campo nr_aliq_icms_interna criado em 30/09/2016 após reunião com cliente para definir os cálculos
                    prod.setNrAliqIcmsInterna(  new BigDecimal(stProd.getCICMSNCM()));//float5,2);
                    //                  campo nr_aliq_icms_externa  criado em 30/09/2016 após reunião com cliente para definir os cálculos
                    //                  Este campo não será usado atualmente, porem foi sugerido manter pelo Analista Fiscal
                    prod.setNrAliqIcmsExterna(  new BigDecimal(stProd.getCICMSNCM()));//float5,2);

                    prod.setBlobFichaTecnica(   stProd.getCFICHATEC().getBytes());
                    if (stProd.getCFICHATEC().length() > 0 ){
                        log.error("[[TESTE]] tamanho do blob maior que zero stProd.getCFICHATEC()[ " + stProd.getCFICHATEC() + " ]");
                    }

                    prod.setBlobFichaTecnicaContentType(Constants.CONTENT_TYPE_PDF);

                    Boolean lAtivo  = StringUtils.truncate(stProd.getCB1ATIVO().trim(),1).equals(Constants.SIM) ? true : false; //c1
                    Boolean lEnvGts = StringUtils.truncate(stProd.getCB1ENVGTS().trim(),1).equals(Constants.ATIVO) ? true : false; //c1;

                    prod.setlAtivo(             (lAtivo && lEnvGts));

                    try{
                        prod.setCfRentabilidadeprod(new BigDecimal(stProd.getCB1COEFICI().trim()));
                    }catch (NullPointerException e) {
                        prod.setCfRentabilidadeprod(new BigDecimal(0.0));
                    }catch (NumberFormatException e) {
                        prod.setCfRentabilidadeprod(new BigDecimal(0.0));
                    }

                    prod.setDtSincronizacao(ZonedDateTime.now());
                    prod.setIdAutorCriacao(Constants.SERVER);

                    Produto prodSalvo = produtoRepo.save(prod);

                    if (prodSalvo != null){
                        //                  if (prodSalvo != null || !prod.islAtivo()){
                        PRODRETRESPONSE responseRet = produtoListaWs.setProdRetResponse(filial.getCdEmpresa(), filial.getCdFilial(), regNumErp);

                        if (responseRet != null){
                            if (responseRet.getPRODRETRESULT().getCRET().trim().equalsIgnoreCase(Constants.RET_SUCESSO)){
                                log.info("SEJ-MSG013: Produto Registro ["+regNumErp+"] integrado e notificado com successo!");
                            }else if (responseRet.getPRODRETRESULT().getCRET().trim().equalsIgnoreCase(Constants.RET_FALHA)) {
                                log.error("SEJ-MSG014: Produto Registro ["+regNumErp+"] integrado e notificado com falha!");
                            }else {
                                log.error("SEJ-MSG015: Produto Registro ["+regNumErp+"] integrado e notificado com falha no retorno!");
                                log.warn("SEJ-MSG016: msg["+responseRet.getPRODRETRESULT().getCMESSAGE()+"]");
                                log.warn("SEJ-MSG017: ret["+responseRet.getPRODRETRESULT().getCRET()+"]");
                            }
                        }else{
                            log.error("SEJ-MSG018: Sem retorno da notificação ao ERP!");
                        }
                    }else{
                        log.error("SEJ-MSG019: Falha ao integrar Produto registro ["+regNumErp+"]");
                    }

                }catch (Exception e) {
                    log.error("SEJ-ERR003: Erro no consumo de produtos", e);
                }
            }
        }catch (RuntimeException e) {
            log.error("SEJ-ERR004: Falha de comunicação no consumo de produtos", e);
        }
    }

    private void consomeCliente(EmpresaFilial filial, AnnotationConfigApplicationContext context) {

        try{
            ClienteWsClient clienteListaWs = context.getBean(ClienteWsClient.class);

            clienteListaWs.setUrl(url);
            clienteListaWs.setPort(port);
            clienteListaWs.setDefaultUri(url+":"+port+"/WSDLCLI.apw");

            CLIENTELSTRESPONSE responseList = clienteListaWs.getClientLstResponse(filial.getCdEmpresa(), filial.getCdFilial());
            List<STCLIENTE> wsClientes = responseList.getCLIENTELSTRESULT().getAREGS().getSTCLIENTE();
            for (STCLIENTE stCli : wsClientes){

                log.info("SEJ-MSG004: Cliente: CdCliente:["+stCli.getCA1COD() +"] "
                        + "CdLoja:["+stCli.getCA1LOJA()+"] "
                        + "Nome:["+stCli.getCA1NOME()+"]");
                try{
                    BigInteger regNumErp = stCli.getCA1RECNO();

                    Cliente cliente = null;
                    try{
                        cliente = clienteRepo.findOneByFilialIdAndCdClienteAndCdLoja(   filial.getId(),
                                StringUtils.truncate(stCli.getCA1COD().trim(),6),
                                StringUtils.truncate(stCli.getCA1LOJA().trim(),4));
                    }catch (NullPointerException e) {}

                    if (cliente == null){
                        cliente = new Cliente();
                        cliente.setFilial(filial);
                        cliente.setCdEmpresa(           StringUtils.truncate(filial.getCdEmpresa().trim(),2)); //c2
                        cliente.setCdFilial(            StringUtils.truncate(filial.getCdFilial().trim(),2));//c2
                        cliente.setCdCliente(           StringUtils.truncate(stCli.getCA1COD().trim(),6));//c6
                        cliente.setCdLoja(              StringUtils.truncate(stCli.getCA1LOJA().trim(),4));//c4
                    }

                    cliente.setNmCliente(           StringUtils.truncate(stCli.getCA1NOME().trim(),60));//c60
                    cliente.setNmClienteReduc(      StringUtils.truncate(cliente.getNmCliente(),20));//c20
                    cliente.setDsEndereco(          StringUtils.truncate(stCli.getCA1END().trim(),80));//c80
                    cliente.setCdTipoCli(           StringUtils.truncate(stCli.getCA1TIPO().trim(),1));//c1
                    cliente.setCdGrptrib(           StringUtils.truncate(stCli.getCA1GRPTRIB().trim(),3));//c3
                    cliente.setSgUf(                StringUtils.truncate(stCli.getCA1EST().trim(),2));//c2
                    cliente.setNmCidade(            StringUtils.truncate(stCli.getCA1MUN().trim(),60));//c60
                    cliente.setNmBairro(            StringUtils.truncate(stCli.getCA1BAIRRO().trim(),30));//c30
                    cliente.setCdCep(               StringUtils.truncate(stCli.getCA1CEP().trim(),8));//c8
                    cliente.setNrDdd(               StringUtils.truncate(stCli.getCA1DDD().trim(),3));//c3
                    cliente.setNrTelefone(          StringUtils.truncate(stCli.getCA1TEL().trim(),15));//c15
                    cliente.setDsEmail(             StringUtils.truncate(stCli.getCA1EMAIL().trim(),120));//c120
                    cliente.setNrCnpjCpf(           StringUtils.truncate(stCli.getCA1CGC().trim(),14));//c14
                    cliente.setNmContato(           StringUtils.truncate(stCli.getCA1CONTATO().trim(),15));//c15
                    cliente.setNrInscr(             StringUtils.truncate(stCli.getCA1INSCR().trim(),18));//c18
                    cliente.setCdRegiao(            StringUtils.truncate(stCli.getCA1REGIAO().trim(),3));//c3
                    cliente.setCdSuframa(           StringUtils.truncate(stCli.getCA1SUFRAMA().trim(),12));//c12
                    cliente.setlContribIcms(        StringUtils.truncate(stCli.getCA1CONTRIB().trim(),1).equalsIgnoreCase("1") ? true: false);//c1
                    cliente.setIdSimplesNacional(   StringUtils.truncate(stCli.getCA1SIMPNAC(),1));//c1
                    cliente.setVlSaldoLimite(       new BigDecimal(stCli.getCLIMCRED()));//float17,2
                    cliente.setInadimplente(        StringUtils.truncate(stCli.getCSITCLI().trim(),1).equalsIgnoreCase(Constants.SIM) ? true: false);//c1
                    cliente.setDtSincronizacao(ZonedDateTime.now());
                    cliente.setIdAutorCriacao(Constants.SERVER);

                    Boolean lLiberado   = StringUtils.truncate(stCli.getCATIVO().trim(),1).equals(Constants.SIM) ? true : false; //c1

                    cliente.setlLiberado(               lLiberado);

                    Cliente clienteSalvo = clienteRepo.save(cliente);

                    if (clienteSalvo != null){
                        CLIRETRESPONSE responseRet = clienteListaWs.setClientRetResponse(filial.getCdEmpresa(), filial.getCdFilial(), regNumErp);

                        if (responseRet != null){
                            if (responseRet.getCLIRETRESULT().getCRET().trim().equalsIgnoreCase(Constants.RET_SUCESSO)){
                                log.info("SEJ-MSG005: Cliente Registro ["+regNumErp+"] integrado e notificado com successo!");
                            }else if (responseRet.getCLIRETRESULT().getCRET().trim().equalsIgnoreCase(Constants.RET_FALHA)) {
                                log.error("SEJ-MSG006: Cliente Registro ["+regNumErp+"] integrado e notificado com falha!");
                            }else {
                                log.error("SEJ-MSG007: Cliente Registro ["+regNumErp+"] integrado e notificado com falha no retorno!");
                                log.warn("SEJ-MSG008: msg["+responseRet.getCLIRETRESULT().getCMESSAGE()+"]");
                                log.warn("SEJ-MSG009: ret["+responseRet.getCLIRETRESULT().getCRET()+"]");
                            }
                        }else{
                            log.error("SEJ-MSG010: Sem retorno da notificação ao ERP!");
                        }
                    }else{
                        log.error("SEJ-MSG011: Falha ao integrar Cliente registro ["+regNumErp+"]");
                    }

                }catch (Exception e) {
                    log.error("SEJ-ERR001: Erro no consumo de clientes", e);
                }

            }
        }catch (RuntimeException e) {
            log.error("SEJ-ERR002: Falha de comunicação no consumo de clientes", e);
        }
    }
}
</code>
Answer: Ocorre alguma exception? Ou simplismente o scheduler não inicia novas threads?
Sugiro conectar o JVisualVM no momento em que a aplicação estiver travada para inspecionar as threads on the fly.
Sugiro este material para detalhes de como usar a ferramenta.
Comment: Descobri. A causa do problema não é minha aplicação, mas o server soap. Quando eu faço a conexão ClientHttp de resposta, o soap server faz um select interno e trava, assim ele nunca encerra a conexão http o meu ClientHttp fica esperando infinitamente. O TimeOut funciona corretamente, porém o readTimeOut é infinito. Agora preciso saber como setar isso usando spring-ws e setando a propriedade via arquivo application.yml.
Answer: Verifique o sistema operacional/container/servidor de aplicação que você está utilizando.
Se por exemplo, estiver usando uma conta free do Heroku, esse comportamento é aceitável. O Heroku coloca sua máquina para hibernar após um tempo.
Comment: Descobri. A causa do problema não é minha aplicação, mas o server soap. Quando eu faço a conexão ClientHttp de resposta, o soap server faz um select interno e trava, assim ele nunca encerra a conexão http o meu ClientHttp fica esperando infinitamente. O TimeOut funciona corretamente, porém o readTimeOut é infinito. Agora preciso saber como setar isso usando spring-ws e setando a propriedade via arquivo application.yml.


Can DPM 2010 backup Exchange 2007 LCR enabled storage groups?

Question: My organization is currently evaluating DPM 2010 RC for backing up Exchange 2007 storage groups. When selecting group members from the "Create New Protection Group" dialog, the only storage groups that render selectable are those that do not have LCR enabled. Does DPM 2010 support backing up Exchange 2007 LCR enabled storage groups?
Here's our environment:

Single Exchange Server
LCR enabled for all exchange databases
Windows Server 2008 SP2
Exchange 2007 SP2, Rollup 2
All databases hosted from a SAN array
Answer: DPM 2010 absolutely does support Exchange 2007 LCR storage groups. Here are a couple leads to get you started troubleshooting:
http://technet.microsoft.com/en-us/library/bb795725.aspx (written for DPM 2007 but the issues are exactly the same in 2010)
http://msexchangeteam.com/archive/2008/08/25/449684.aspx
Answer: I asked the same question over email on the technet DPM blog and received the following response. We tried the registry setting change and found it worked. Thank you DPM!
http://blogs.technet.com/dpm/
To display the Exchange LCR-enabled storage groups in the DPM New Protection Group Wizard, use the following set key command to disable the (LCR) VSS replica writer.
set key EnableVssWriter under
HKLM\Software\Microsoft\Exchange\Replay\Parameters to 0 DWORD type
Note: After setting the registry key, you must restart the Microsoft Exchange Replication Service.
For more information: http://technet.microsoft.com/en-us/library/bb795725.aspx


Setting Realm Relationships

Question: I am integrating Realm but after some testing it seems that a relationship has to be set (and I assume managed) on both ends.
Category.java
private String name;
private RealmList<Subcategory> subcategories;
// Accessors not shown

Subcategory.java
private String name;
private Category category;
// Accessors not shown

When I set category on a subcategory and then do a query, I can see the category I just set. However, if I then query the Category, I cannot see the subcategories.
Am I making a mistake or do I need to set a relationship on both ends? If I do, then I presume the removal of a relationship should also be done on both ends?
Comment: No, you don't need to do it in both ends. Show us how you're trying to retrieve the objects...
Comment: Using the generated accessors - `getSubcategories()`
Comment: try using `getSubcategories().first()` is it null?
Comment: getSubcategories returns an empty list.
Comment: probably is the way you are setting the subcategories... Can you show the code where you add the subcategories to the category?
Answer: Turns out relationships in Realm have to be set on both sides. It also means removing a relationship has to be done on both sides, adding significant complexity to our code. This is such a serious limitation when working with all but the simplest of database schemas that we wont be able to use Realm anymore.
Comment: Backlink is on our todo list and tracked here https://github.com/realm/realm-java/issues/607 . However, it needs some time to be well implemented in our underlying storage engine. Sorry for the inconvenience.

Setting Realm Relationships


Feature selection + classification in Caret

Question: I'm using Caret to apply a bunch of different machine learning algorithms for phenotype prediction from gene expression data. With about 20,000 genes, I'd like to perform filter feature selection before training my classifiers. How can this be achieved? I've been through all the Caret documentation and vignettes, but the sbf and rfe feature selection methods seem to have the classification algorithms built in, for example
output = sbf(x,y,sbfControl = sbfControl(rfSBF, method = "repeatedcv", repeats = 5))

gets me a list of features for each fold and repeat, optimized according to the RF classifier accuracy.
Ideally, I'd like to pass the features selected in each fold along with the indices used into the "train" function, so that I'm only training on a small feature subset for each fold and repeat. The steps I want are:

Split data into 10 folds
Select features using 9 folds
Train model on the same 9 folds and feature subset from step 2
Evaluate performance on remaining fold
Repeat 2-4 over all folds
Repeat 1-5 with different fold splits

The "train" function takes care of everything but step 2 internally, so I can't figure out how to insert the feature selection using the same fold split. I assure you I've been through all the documentation I can find, so any other help would be much appreciated!
Answer: You should be able to accomplish everything you want with the sbf function instead.  I originally assumed it worked the same way you are, but the functionality given by sbf is apparently more like a super set of what's available in train.
For example, something like this sounds like what you're getting at:
fit <- sbf(
  form = response ~ .,
  data = d, method = "glmnet",
  tuneGrid=expand.grid(.alpha = .01, .lambda = .1),
  preProc = c("center", "scale"),
  trControl = trainControl(method = "none"),
  sbfControl = sbfControl(functions = caretSBF, method = 'cv', number = 10)
)

This would run 10 outer folds and fit a single glmnet model to each, using only a feature subset.  You could also specify some number of cv folds for trControl and a parameter grid to do training on inner folds.


WordPress job and resume board not working

Question: i just developed a job and resume upload code for my website hoping it will work. Am using code snippets plugin to add the code.
after adding the code, i get menu shortcut on my dashboard menu: illustrated in the pic.
enter image description here
how do i develop short code to add to a page.
attached is the developed code:
// This file contains the code needed to create a job and resume plugin for your WordPress site.
// First, we need to create a function that will register the custom post type for our job and resume posts.
function register_job_resume_post_type() {
    $labels = array(
        'name'               => 'Jobs & Resumes',
        'singular_name'      => 'Job & Resume',
        'add_new'            => 'Add New',
        'add_new_item'       => 'Add New Job & Resume',
        'edit_item'          => 'Edit Job & Resume',
        'new_item'           => 'New Job & Resume',
        'all_items'          => 'All Jobs & Resumes',
        'view_item'          => 'View Job & Resume',
        'search_items'       => 'Search Jobs & Resumes',
        'not_found'          => 'No jobs & resumes found',
        'not_found_in_trash' => 'No jobs & resumes found in Trash',
        'parent_item_colon'  => '',
        'menu_name'          => 'Jobs & Resumes'
    );
   $args = array(
        'labels'             => $labels,
        'public'             => true,
        'publicly_queryable' => true,
        'show_ui'            => true,
        'show_in_menu'       => true,
        'query_var'          => true,
        'rewrite'            => array( 'slug' => 'job-resume' ),
        'capability_type'    => 'post',
        'has_archive'        => true,
        'hierarchical'       => false,
        'menu_position'      => null,
        'supports'           => array( 'title', 'editor', 'author', 'thumbnail', 'excerpt', 'comments' )
    );

    register_post_type( 'job-resume', $args );
}
// We also need to create a function to register the custom taxonomies for our job and resume posts.
function register_job_resume_taxonomies() {
    // Add new taxonomy, make it hierarchical (like categories)
    $labels = array(
        'name'              => 'Job & Resume Categories',
        'singular_name'     => 'Job & Resume Category',
        'search_items'      => 'Search Job & Resume Categories',
        'all_items'         => 'All Job & Resume Categories',
        'parent_item'       => 'Parent Job & Resume Category',
        'parent_item_colon' => 'Parent Job & Resume Category:',
        'edit_item'         => 'Edit Job & Resume Category',
        'update_item'       => 'Update Job & Resume Category',
        'add_new_item'      => 'Add New Job & Resume Category',
        'new_item_name'     => 'New Job & Resume Category Name',
        'menu_name'         => 'Job & Resume Categories'
    );

    $args = array(
        'hierarchical'      => true,
        'labels'            => $labels,
        'show_ui'           => true,
        'show_admin_column' => true,
        'query_var'         => true,
        'rewrite'           => array( 'slug' => 'job-resume-category' ),
    );

    register_taxonomy( 'job-resume-category', array( 'job-resume' ), $args );

    // Add new taxonomy, NOT hierarchical (like tags)
    $labels = array(
        'name'                       => 'Job & Resume Tags',
        'singular_name'              => 'Job & Resume Tag',
        'search_items'               => 'Search Job & Resume Tags',
        'popular_items'              => 'Popular Job & Resume Tags',
        'all_items'                  => 'All Job & Resume Tags',
        'parent_item'                => null,
        'parent_item_colon'          => null,
        'edit_item'                  => 'Edit Job & Resume Tag',
        'update_item'                => 'Update Job & Resume Tag',
        'add_new_item'               => 'Add New Job & Resume Tag',
        'new_item_name'              => 'New Job & Resume Tag Name',
        'separate_items_with_commas' => 'Separate job & resume tags with commas',
        'add_or_remove_items'        => 'Add or remove job & resume tags',
        'choose_from_most_used'      => 'Choose from the most used job & resume tags',
        'menu_name'                  => 'Job & Resume Tags'
    );

    $args = array(
        'hierarchical'          => false,
        'labels'                => $labels,
        'show_ui'               => true,
        'show_admin_column'     => true,
        'update_count_callback' => '_update_post_term_count',
        'query_var'             => true,
        'rewrite'               => array( 'slug' => 'job-resume-tag' ),
    );

    register_taxonomy( 'job-resume-tag', 'job-resume', $args );
}

// Finally, we need to register our custom post type and taxonomies with WordPress.
 add_action( 'init', 'register_job_resume_post_type' );
 add_action( 'init', 'register_job_resume_taxonomies', 0 );

Assist me on how to sort it out.
step on how to make it work and develope shortcode for easy installation
Answer: You would use the add_shortcode() function. This function takes two parameters: the name of the shortcode and then the callback function to display the shortcode content. See the code below.
function namespace_cpt_shortcode() {
    $args = [
        'post_type' => 'job-resume'
    ];

    $the_query = new WP_Query( $args );

    // The Loop
    if ( $the_query->have_posts() ) {
        while ( $the_query->have_posts() ) {
            $the_query->the_post();

            // Save the data you wish to show in the post
            $postTitle = get_the_title();
            $postContent = get_the_content();

            //Save the data to the variable that will be returned in the shortcode.
            $result = "

$postTitle

$postContent

";

        } // End while loop
    } else {
        $result = "No posts found.";
    }
    /* Restore original Post Data */
    wp_reset_postdata();

    return $result;
}
// Use add_shortcode() to create the shortcode. First parameter is the name of the shortcode and the second is the function name.
add_shortcode( 'job-listing', 'namespace_cpt_shortcode' );

If you want to add more post information to the layout, you would add it into the $result variable as that is what is being returned in the function because shortcodes require a returned value.


Cannot schedule task to run as user on Server 2012 R2

Question: I'm trying to create a scheduled task on Server 2012 R2 to run as a domain service account. I select "Run whether the user is logged on or not", and select the domain service account, click on OK, enter the credentials, and then I get this error message:

An error has occurred for task LPAEmail. Error message: The following error was reported: A specified logon session does not exist. It may have already been terminated.

Things I've already tried:

Checked that Network access: Do not allow storage of passwords and credentials for network authentication policy is disabled.
Logging on to the server as the user.
A different domain account.
Tried using the local administrator account.
Adding the domain account to the local Administrators group.
Changing "Configure for" to Vista, Windows 7, and Server 2012 R2. None of them made a difference.
Setting to a local account, and then entering credentials for a domain account when prompted.
Verified "Password never expires" is checked for the service account.
Tried creating a completely new task.

Does anyone have any suggestions on what else to try?
Note: I discovered if I check "Do not store password. The task will only have access to local computer resources.", it works. But this is not an acceptable solution, because the task needs to access remote resources.
Comment: Would anyone care to explain why this was down voted?
Comment: Did you verify the `storage of password` setting with GPEdit or RSOP? GPEdit will show the local setting. RSOP will be the effective setting. Downvote was not me.
Comment: @Craig620, through RSOP.
Comment: Create an new empty task, and try tu put again that user in. I wonder if the task's config are corrupted. (Strange error happen when the file (where the task is saved) got invalid/corrupted entry)
Comment: @yagmoth555, I already tried that. Same error. Thanks though.
Comment: Does the problem occur with another server in the same OU?
Comment: Did you try enabling the job option to `run with highest privileges`? Is the LPAEmail service account or any groups it belongs to listed beneath: User Rights Assignment / Deny logon as batch job? Try using a dif service account to run a simple test job `echo "hello world" > c:\temp\hello.log` Does that work?
Comment: @Craig620, Run with highested privelages results in the same error. Looking under User Rights Assignment in RSOP has all options (including Deny logon as batch job) set to `Not Defined`. I have another Server 2012 R2 instance that also has all of it's User Rights Assignments set to `Not Defined` and it works on that server. Different service account with simple test job also results in the same error.
Comment: @druciferre - does the problem occur with another server in the same OU?
Comment: @Bill_Stewart, yes. Both are in the same OU, and both are only members of the Domain Computers group. As far as I've been able to tell they have the same RSOP.
Comment: Try this: `Control Panel / Credential Manager / Windows Credentials / Generic Credentials` Remove the credentials the problem task uses. Then Reconfigure the credentials on the task.
Comment: @Craig620, after clicking on Windows Credentials, I get the same error... "A specified logon session does not exist..."
Comment: Is the `Credential Manager` service running? Do the NTFS perms at `C:\ProgramData\Microsoft\Vault` on the broken system resemble the perms on the working system?
Comment: As @Craig620 notes, this must be a problem related to the credential manager on the broken server.
Comment: @Craig620, the Credential Manager service is running on both servers, and the permissions on the Vault folder match on both.
Answer: Turns out the issue was the Credential Manager was disabled via the following registry settings:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa]
"DisableCredMan"=dword:00000000
"disabledomaincreds"=dword:00000000

Both values must be set to 0 in order for the Credential manager to work with domain accounts.
Comment: Note: There was no mention of "Credential" or "CredMan" on the output of `gpresult /h report.html`.
Comment: Don't you mean `DisableCredMan` was set to `1` and you set it to `0` to enable it?
Comment: @Bill_Stewart, I don't see any contradiction between what I wrote and what you said
Comment: Sorry, your answer says the credential manager is disabled by the zeros. Actually both must be set to 0 (as you say after, but it seems contradictory).


Question regarding my multiple adjustment technique

Question: I am computing several correlations tests using cor.test in R. Say I have $n$ distinct cities, and for each of these cities I have $4$ variables ($W$, $X$, $Y$, $Z$) and one variable of interest $A$. For each city I make a correlation test between $A$ and $W$, $A$ and $X$, $A$ and $Y$, and finally $A$ and $Z$.
What's the "factor" I should adjust for here? I'd go with $4$ as we're considering $n$ independent cities and testing $4$ times the same variable per considered city, but maybe I am mistaken? Do I need to adjust by the total number of tests done ($4*n$), even though the cities are independent?
Comment: To be clear, these variables have multiple observations for each city, and you're correlating all observations within city 1, 2, ..., n ? Or there's one observation per city, and the correlation is all values of A (i.e. all n cities) vs. another variable?
Answer: Correlational Approach
(After writing all of this, I realized you actually had 5 variables, but the interpretation of the answer below is still basically the same).
It seems to me, based off the question, that you are interested in some relationship between four key variables, but are cognizant of the fact that this varies by city. I don't think you can meaningfully summarize that much by-group variation in your variables. Even with a correlation matrix or heatmap, there is simply too much information to sort through. Here I simulate some data to emulate your situation first.
#### Load Packages and Set Seed ####
library(lmerTest)
library(correlation)
library(tidyverse)
set.seed(123)

#### Number of Cities and Obs Per City ####
n_city <- 50
n_obs <- 100

#### Create Data Frame ####
df <- data.frame(
  city = rep(1:n_city, each = n_obs),
  x1 = rnorm(n_city * n_obs),
  x2 = rnorm(n_city * n_obs),
  x3 = rnorm(n_city * n_obs)
)

#### Random Intercepts/Slopes for Cities ####
random_intercepts <- rnorm(n_city)
random_slopes_x1 <- rnorm(n_city)
random_slopes_x2 <- rnorm(n_city)
random_slopes_x3 <- rnorm(n_city)

#### Create Outcome Variable ####
df$y <- with(df,
             random_intercepts[city]
             + random_slopes_x1[city]
             * x1
             + random_slopes_x2[city]
             * x2
             + random_slopes_x3[city]
             * x3
             + rnorm(n_city * n_obs))

If we group these correlations by city:
#### Check Correlations ####
data %>%
  group_by(city) %>%
  correlation()

we get a massive correlation matrix (I only show the first 5 cities here)...
# Correlation Matrix (pearson-method)

Group | Parameter1 | Parameter2 |         r |         95% CI |    t(98) |         p
-----------------------------------------------------------------------------------
1     |         x1 |         x2 |      0.01 | [-0.18,  0.21] |     0.12 | > .999
1     |         x1 |         x3 |      0.16 | [-0.03,  0.35] |     1.65 | 0.408
1     |         x1 |          y | -9.78e-03 | [-0.21,  0.19] |    -0.10 | > .999
1     |         x2 |         x3 |     -0.15 | [-0.33,  0.05] |    -1.46 | 0.443
1     |         x2 |          y |      0.38 | [ 0.20,  0.54] |     4.10 | < .001***
1     |         x3 |          y |     -0.60 | [-0.71, -0.45] |    -7.35 | < .001***
2     |         x1 |         x2 |      0.07 | [-0.13,  0.26] |     0.71 | > .999
2     |         x1 |         x3 |     -0.04 | [-0.23,  0.16] |    -0.39 | > .999
2     |         x1 |          y |      0.32 | [ 0.13,  0.49] |     3.34 | 0.005**
2     |         x2 |         x3 |      0.02 | [-0.17,  0.22] |     0.23 | > .999
2     |         x2 |          y |      0.64 | [ 0.51,  0.75] |     8.30 | < .001***
2     |         x3 |          y |     -0.47 | [-0.61, -0.31] |    -5.32 | < .001***
3     |         x1 |         x2 |     -0.02 | [-0.22,  0.18] |    -0.20 | > .999
3     |         x1 |         x3 |     -0.06 | [-0.25,  0.14] |    -0.55 | > .999
3     |         x1 |          y |     -0.75 | [-0.82, -0.65] |   -11.15 | < .001***
3     |         x2 |         x3 |      0.08 | [-0.11,  0.28] |     0.84 | > .999
3     |         x2 |          y |     -0.03 | [-0.23,  0.17] |    -0.31 | > .999
3     |         x3 |          y |     -0.38 | [-0.54, -0.20] |    -4.11 | < .001***
4     |         x1 |         x2 |      0.11 | [-0.09,  0.30] |     1.07 | > .999
4     |         x1 |         x3 |      0.04 | [-0.15,  0.24] |     0.42 | > .999
4     |         x1 |          y |      0.87 | [ 0.81,  0.91] |    17.13 | < .001***
4     |         x2 |         x3 |      0.08 | [-0.12,  0.28] |     0.83 | > .999
4     |         x2 |          y |     -0.09 | [-0.29,  0.10] |    -0.94 | > .999
4     |         x3 |          y |     -0.21 | [-0.39, -0.01] |    -2.13 | 0.178
5     |         x1 |         x2 |     -0.09 | [-0.29,  0.10] |    -0.94 | 0.900
5     |         x1 |         x3 |     -0.13 | [-0.32,  0.07] |    -1.27 | 0.835
5     |         x1 |          y |     -0.25 | [-0.43, -0.06] |    -2.56 | 0.059
5     |         x2 |         x3 |     -0.06 | [-0.25,  0.14] |    -0.57 | 0.900
5     |         x2 |          y |     -0.75 | [-0.82, -0.64] |   -11.07 | < .001***

We could employ a heatmap and perhaps facet this by city, but even this would be difficult to summarize meaningfully given you would have to compare 50 groups somehow.
Mixed Model Approach
What could be more meaningful but potentially more difficult is to simply fit a mixed model, where your three IVs and one DV are fit in a regression, but we model out each city by it's own unique intercept (random intercept), and potentially include more sophisticated random effects based on our needs. The syntax is similar to lm in R, but we use (1|city) to include the random intercept of cities.
#### Simple Model ####
model.int <- lmer(y
              ~ x1
              + x2
              + x3
              + (1 | city),
              data = df)

Here this fits a normal regression, but the response variable y varies by city. Running ranef(model.int) shows how much each city fluctuates around the conditional mean of y:
$city
   (Intercept)
1  -1.03268009
2  -0.35127375
3  -1.06261293
4   0.97840227
5   0.40144410
6   1.36994080
7  -1.56389063
8   0.54683387
9   0.50180444
10  0.22833614
11  0.50076443
12 -0.98574989
13  0.71349714
14 -0.16301217
15  0.45231931
16 -0.33856598
17  0.78105347
18  0.46048970
19 -0.07553787
20  0.53894904
21  1.54668930
22 -2.20445323
23  0.27170677
24  1.83835442
25 -0.06014703
26 -0.67720596
27  0.44845990
28 -0.60372979
29 -1.38065676
30 -0.39841245
31 -2.14597392
32 -0.73881666
33  0.34478326
34  0.92780038
35  0.05273206
36 -0.11864168
37  0.28281417
38  0.62924181
39 -1.33870775
40 -0.42792721
41  1.05921028
42 -0.30089312
43 -1.01100523
44  0.14459701
45  0.54592963
46  0.27505794
47  0.45523254
48 -0.95264744
49  1.04945728
50  0.58664008

with conditional variances for "city"

However, it seems you believe the relationships between your IVs and DVs vary by city, so a more plausible model may be one which also includes random slopes, shown now with the new syntax below, which replaces the 1 with all our predictor terms.
#### Fit Full Model ####
model.slp <- lmer(y
              ~ x1
              + x2
              + x3
              + (x1 + x2 + x3 | city),
              data = df)

The results show our fixed effects and random effects:
Linear mixed model fit by REML. t-tests use
  Satterthwaite's method [lmerModLmerTest]
Formula: y ~ x1 + x2 + x3 + (x1 + x2 + x3 | city)
   Data: df

REML criterion at convergence: 15085.8

Scaled residuals:
    Min      1Q  Median      3Q     Max
-3.3798 -0.6444 -0.0102  0.6398  3.5897

Random effects:
 Groups   Name        Variance Std.Dev. Corr
 city     (Intercept) 0.8647   0.9299
          x1          1.0907   1.0443   -0.26
          x2          1.5222   1.2338   -0.01 -0.12
          x3          1.0200   1.0100    0.18  0.00  0.04
 Residual             0.9918   0.9959
Number of obs: 5000, groups:  city, 50

Fixed effects:
             Estimate Std. Error        df t value Pr(>|t|)
(Intercept) -0.113360   0.132278 49.013180  -0.857  0.39562
x1          -0.034227   0.148405 48.978220  -0.231  0.81856
x2           0.001188   0.175072 49.023218   0.007  0.99461
x3           0.392522   0.143551 48.986373   2.734  0.00868

(Intercept)
x1
x2
x3          **
---
Signif. codes:
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
   (Intr) x1     x2
x1 -0.252
x2 -0.009 -0.119
x3  0.173 -0.002  0.037

It seems that x3 is the most influential predictor, with the slopes and intercepts varying a fair amount. Plotting this may be helpful to visualize what is going on. First we can just plot the main effect each predictor has on the outcome. You can see that x3 has the most meaningful association while the other two are "flat."
#### Average Effect ####
df %>%
  pivot_longer(cols = contains("x"),
               names_to = "predictor",
               values_to = "value") %>%
  ggplot(aes(x=value,
             y=y))+
  geom_point(alpha = .4)+
  geom_smooth(method = "lm",
              se = F,
              color="darkred")+
  facet_wrap(~predictor)+
  theme_bw()+
  labs(x="Predictor",
       y="Outcome",
       title = "Variation in Associations")

Changing the code to group these relationships by city:
#### Mixed Effects ####
df %>%
  pivot_longer(cols = contains("x"),
               names_to = "predictor",
               values_to = "value") %>%
  ggplot(aes(x=value,
             y=y,
             group=city))+
  geom_point(alpha = .4)+
  geom_smooth(method = "lm",
              se = F,
              color="darkred")+
  facet_wrap(~predictor)+
  theme_bw()+
  labs(x="Predictor",
       y="Outcome",
       title = "Variation in Associations")

We see a lot of variation by city with each drawn red line:

However, it may still be useful to determine which cities vary and by how much. We can inspect this by using ranef(model.slp).
$city
   (Intercept)          x1          x2          x3
1  -1.12361729  0.14943285  0.44747548 -1.15497954
2  -0.43709314  0.47399377  1.22579927 -1.22660879
3  -0.65738154 -1.53823824 -0.02145161 -1.31895491
4   1.06980044  2.05322178 -0.40419307 -0.91479656
5   0.59671297 -0.57717809 -1.27544608 -0.72552255
6   1.34369518  1.31518731  0.06934293  0.79403958
7  -1.29504010  1.39637595  2.11980123  1.41176362
8   0.69223549 -1.33581170 -0.26478080 -0.86068739
9   0.44541878 -0.36270692 -0.50919090  1.34586408
10  0.45828743  0.18996906 -1.35617567  0.50143210
11  0.58201366 -0.64549233 -0.19783999  1.33064489
12 -1.53029548  0.23184807 -1.79466973  0.82793902
13  0.66171310 -0.15688078  0.72645412  0.05202828
14 -0.04114744 -0.52765143  0.11056928 -2.65074821
15  0.21583041 -0.75066041  2.81112427 -0.11766525
16 -0.39410493  0.99711620  0.27424943 -1.31582034
17  0.81448286  0.36394147  0.96287237 -0.42553170
18  0.98695079 -1.14153773  2.74777699  0.59604367
19 -0.17933276  1.71202544 -0.89578123 -0.20632712
20  0.41355221  1.39158426 -0.48646080 -0.14346558
21  1.69276989 -0.01854201 -1.16190375  0.71941689
22 -2.10928083  2.09567804 -0.85410721 -1.05082693
23  0.16316791 -1.13063844  0.03519864  1.35422199
24  1.91866086 -1.56306471 -0.78274663 -0.56400815
25 -0.25367920 -2.52803739 -0.16185180 -0.78553432
26 -0.56933064 -0.72966704  0.58411171 -0.62920627
27  0.24995033 -0.64659081  1.19652123  0.66381377
28 -0.72941233  1.15206239 -2.22565484  0.47706728
29 -1.58047610  1.28933050 -1.18468098 -0.70305121
30 -0.36896070  0.26067152  0.52120332 -0.05824251
31 -2.26889745 -0.22295834  2.14988513  0.53696272
32 -0.83068029 -0.77176304 -1.11081479 -0.73564387
33  0.40156810  0.88879666 -0.91023373 -0.35571429
34  0.84288070 -0.70007611  1.43424610 -0.79530596
35  0.05426781 -0.16027823 -1.28165407 -0.02569450
36 -0.21029419 -0.28184057  1.31386627 -1.05483523
37  0.21553144 -0.46654427 -1.29550639  1.18219448
38  0.65273488 -0.01010801 -2.28514697 -0.38031978
39 -1.31586299  0.12389335 -0.59569529 -0.47694292
40 -0.22938452  0.45478125  1.38420746  0.42221215
41  0.81449534 -0.62740591  0.17606884  1.60942705
42 -0.39689880 -1.43312372 -0.15138616  0.05949651
43 -0.99970458  1.80014644  0.37314972  1.80678709
44  0.20111604  0.69005048 -0.02573371 -0.58859110
45  0.46823891  1.11354875  2.44106689  0.26116380
46  0.39602197 -0.92561265  0.63261421 -0.50121419
47  0.70324700 -0.92006151  0.45509325  1.77054083
48 -1.11917294  0.07739495 -1.07474629 -0.48226780
49  0.98691437 -0.86063974 -0.42122574  2.40351259
50  0.59778937  0.81205966 -1.46361994  0.12193459

with conditional variances for "city"

The intercepts in the first column show how much the conditional mean of y changes by city. For example, City 50 has a conditional mean that is about .598 above the conditional mean of the full model, meaning that $y$ is generally higher for City 50. The three columns after show the slope of each predictor on the outcome for that city. City 50 has the following slopes:

$x_1: .812$
$x_2: -1.463$
$x_3: .122$

We can also summarize everything in this list with a dot plot, which makes it easier to look at how each city compares to the average.
library(lattice)
dotplot(ranef(model.slp))


If $\log_35=a$ and $\log_54=b$, what is $\log_{60}70$?

Question: One student sent me this question:

If $\log_35=a$ and $\log_54=b$, what is $\log_{60}70$?

Question asks the value of $\log_{60}70$ in terms of $a$ and $b$. Equations for $a$ and $b$ involved $2$, $3$ and $5$. So I am not sure how to deal with $7$ in $\log_{60}70$ by using logarithm identities. I may be missing something very obvious.
Comment: @Smath but that is exactly what the question is about!
Comment: Maybe the change of base formula for logarithms will be useful!
Comment: @Smath, as i stated in the question, i cannot get rid of $7$ in that way.
Comment: I think the answer expected will have log(7) as a constant term in it.
Answer: If there is an algebraic function $f$ with
$$
\log_{60}70 =f(a,b)= f\left(\frac{\log 5}{\log 3}, \frac{\log 4}{\log 5}\right)$$
then since
$$
\log_{60}70 = \frac{\log 70}{\log 60} = \frac{\log 7+\log 2+\log 5}{2\log 2+\log 3+\log 5},$$
we would have
$$
\log 7 = f\left(\frac{\log 5}{\log 3}, \frac{\log 4}{\log 5}\right)(2\log 2+\log 3+\log 5) - \log 2 - \log 5
$$
so $\log 7$ would be algebraic in $\log 2$, $\log 3$, and $\log 5$.
But according to Schanuel's conjecture, the logarithms of the primes are algebraically independent.
Comment: This solves it.


WCF assembly causing circular dependencies

Question: I have a silverlight library which I have added a wcf service reference. Call this library A.dll. This library contains other functions and I prefer to have the wcf code in another library.
I now have created another silverlight class library which I have added the same wcf service reference (call this library B.dll) and want the first silverlight library to reference the wcf service.
If I reference A.dll in B.dll I do not get a circular reference.
I still have the wcf reference code in A.dll but with a different namespace than the wcf reference code in B.dll (not sure if this related to the problem).
Any ideas why I get this error when I reference B.dll in A.dll?
JD.
Ps. I know that I could create an interface library to provide another design but at the moment I am not sure why this error is happening.
Comment: @Dan: A week ago I added the wcf service reference code to the first library. Today, I decided to put the wcf service code in another library and then reference the library from the first one. I have just removed the wcf code from the first library and I still get circular reference errors. Yes, the error is the dialog that is shown when adding the reference.
Comment: Do you get this error as a pop-up dialog when you try to add the project B reference to project A? It is a little unclear to me exactly what you are trying to do. My understanding is that you have 2 silverlight class libraries and each has a service reference to the same WCF service. You then want one project to reference the other. I see no circular reference here as long as they both aren't trying to reference each other.
Comment: Thanks for clarifying... I'm just trying to set up a new set of projects to try to repro your problem. I'm not sure if it has any effect or not but did you generate your WCF service proxy by doing Add Service Reference... or by running slsvcutil.exe and adding the generated code to the project?
Comment: @Dan: Thanks for trying. What I have done now is edit the csproj for A.dll in notepad and add B.dll to the projectReferences tag. When I now build it all works fine. However, I see a Warning Icon (triangle) in next to the referenced project (B.dll) in the solution explorer. So it looks like VS misbehaving in some way. With regards to adding the proxy, I followed Sahil Malik's excellent article (http://blah.winsmarts.com/2008-7-SilverLight_WCF_References_in_SharePoint_-_The_right_way.aspx) to get the proxy without all the config baggage.
Comment: @Dan : Looks like it was VS that got in a mess. I had to manually fix the vs solution and the project file and it is all cleared up now. Not sure how it go in to this state. I created a couple of solution projects and dragged the projects into them. Not sure if that confused the ide. Anyway, thanks for all the help.
Comment: @JD: I guess looking at the Project Dependencies window of each project in your solution would have done the trick. I had the same problem. (See my comment at Mark Cooper's answer)
Answer:
I still have the wcf reference code in A.dll but with a different namespace than the wcf reference code in B.dll (not sure if this related to the problem).

I think that even though the namespaces are different, as you have already identified you have created a circular assembly reference.
VS builds dependent (referenced) assemblies first. If you ahve a circular reference, VS won't know which order to build the projects, hence the error you are getting.
Comment: +1. Thanks for the hint with the build order. I wanted to add a reference to one of my assemblies and VS was complaining about a circular reference although there was no circular depedency in the code. The problem was that I once referenced the projects the other way around and visual studio kept the depedency / reference in the Projekt Dependencies window. I hope this will help someone in the future.


Sylow $p$-subgroups of $\mathbb{Q}/\mathbb{Z}$

Question: I want to prove that $\mathbb{Q}/\mathbb{Z}=\bigoplus_{p} S(p)$ where $S_{p}$ is the Sylow $p$-subgroup of $\mathbb{Q}/\mathbb{Z}$ $$S(p)=\lbrace [q]\in\mathbb{Q}/\mathbb{Z}\mid \exists k\in\mathbb{N}~\text{such that } p^{k}[q]=[0]\rbrace.$$
At first instance I thought to look at a prime decomposition of rationals $\frac{a}{b}=p_{1}^{\alpha_{1}}\cdots p_{r}^{\alpha_{r}}$ for integers $\alpha_{j}$ but this doesn't seem to work since we have addition $+$ in $\mathbb{Q}/\mathbb{Z}$ and not a product operation.
Comment: Is $S_p$ the same thing as $S(p)$?
Comment: I was just about to ask that, @ArturoMagidin :)
Answer: Let $T(p)$ be the subgroup of $\mathbb{Q}$ consisting of all rationals that can be written with a denominator which is a nonnegative power of $p$.
Note that this is indeed a subgroup: it contains $0=\frac{0}{p^0}$; and if $\frac{a}{p^n}$ and $\frac{b}{p^m}$ lie in $T(p)$, then $\frac{a}{p^n}-\frac{b}{p^m} = \frac{p^{m}a - p^nb}{p^{n+m}}\in T(p)$.
I claim that $\mathbb{Q}$ is the sum (not direct sum, just sum) of the $T(p)$ with $p$ ranging over all primes. Indeed, let $\frac{a}{b}\in\mathbb{Q}$, with $b\gt 0$. If $b=1$, then  write $b=p^0$ and the element lies in $T(p)$. If $b\gt 1$, take a prime factorization of $b$, $b=p_1^{r_1}\cdots p_k^{r_k}$, with $r_i$ positive integers. I want to find integers $q_i$ such that
$$\frac{a}{b} = \frac{q_1}{p_1^{r_1}}+\cdots + \frac{q_k}{p_k^{r_k}}.$$
Let $x_i = \frac{b}{p_i^{r_i}}$, $i=1,\ldots,k$. Then the right hand side is
$$\frac{q_1}{p_1^{r_1}}+\cdots + \frac{q_k}{p_k^{r_k}} = \frac{x_1q_1}{b}+\cdots \frac{x_kq_k}{b} = \frac{x_1q_1+\cdots+x_kq_k}{b}.$$
Thus, we want to find integers $q_1,\ldots,q_k$ such that $x_1q_1+\cdots+x_kq_k = a$.
Because $\gcd(x_1,x_2,\ldots,x_k)=1$, this is possible. Thus, we can find $q_1,\ldots,q_k$ satisfying the equality, and this proves that $\frac{a}{b}$ can be written in the desired form. That is, every rational is equal to a sum of rationals whose denominators are prime powers.
We now project down to $\mathbb{Q}/\mathbb{Z}$. Because $\mathbb{Q}$ is the sum of $T(p)$, then $\mathbb{Q}/\mathbb{Z}$ is the sum of the images of the $T(p)$; note that the $T(p)$ contain $\mathbb{Z}$.
In fact, the image $T(p)$ is precisely your $S(p)$. Indeed, if $q=\frac{a}{p^k}\in T(p)$, then $p^kq\in\mathbb{Z}$, so $p^k[q]=[0]$ in $\mathbb{Q}/\mathbb{Z}$, proving that $T(p)/\mathbb{Z}$ is contained in $S(p)$. And if $[q]\in S(p)$, then there exists $k$ such that $p^kq\in\mathbb{Z}$; thus, $q = \frac{a}{p^k}$ for some $a\in\mathbb{Z}$, proving that $q\in T(p)$, so $[q]\in T(p)/\mathbb{Z}$.
This proves that $\mathbb{Q}/\mathbb{Z}$ is the sum of the $S(p)$.
To verify the sum is direct, assume that we have pairwise distinct $p_1,\ldots,p_n$, and that $\frac{a_1}{p_1^{r_1}}\in S(p_1)\cap (S(p_2)+\cdots+S(p_n))$. Then there exist integers $z,a_2,\ldots,a_n$, and nonnegative integers $r_2,\ldots,r_n$ such that
$$\frac{a_1}{p_1^{r_1}}  = \frac{a_2}{p_2^{r_2}}+\cdots+\frac{a_n}{p_n^{r_n}} + z.$$
Letting $x=p_2^{r_2}\cdots p_n^{r_n}$, this means that we can write
$$\frac{a_1}{p_1^{r_1}} = \frac{y}{x}$$
for some integer $y$. This means that $a_1x = p_1^{r_1}y$, and thus that $p_1^{r_1}\mid a_1x$. Since $\gcd(p_1,x)=1$, it follows that $p_1^{r_1}\mid a_1$, and therefore that $\frac{a_1}{p_1^{r_1}}\in\mathbb{Z}$. That is, $[\frac{a_1}{p_1^{r_1}}] = [0]$, proving that the intersection is trivial and therefore that the sum is direct.
Thus, $\frac{\mathbb{Q}}{\mathbb{Z}} = \oplus_p S(p)$. What remains is to show that the $S(p)$ are Sylow $p$-subgroups of $\mathbb{Q}/\mathbb{Z}$. But indeed, these are the maximum $p$-subgroups of $\mathbb{Q}/\mathbb{Z}$: if $[\frac{a}{b}]$ has order a power of $p$, then $\frac{p^ka}{b}\in\mathbb{Z}$, so $\frac{a}{b}$ can be written as $\frac{n}{p^k}$ for some $n\in\mathbb{Z}$. Therefore, $[\frac{a}{b}] \in T(p)/\mathbb{Z}=S(p)$. Since $S(p)$ contains every element of order a power of $p$ of $\mathbb{Q}/\mathbb{Z}$, it is the Sylow $p$-subgroup, being a maximum (and hence maximal) $p$-subgroup.


Subtitles won't display if closed caption is off in system settings

Question: I am trying to get subtitles to display regardless of what the device has set under the accessibilities. Currently, if the device is set to English with closed captioning enabled in settings, English subtitles will play, and if the device is set to Spanish, Spanish subtitles will play. I would like subtitles to play regardless of whether or not closed captioning is on.
I attempted to add this code from apple's documentation, but it did not help. It seems to be reading the options fine, detecting both the Spanish and English subtitles, but I cannot get them to display on the screen.
import UIKit
import AVFoundation
import AVKit

class ViewController : UIViewController
{
    override func loadView()
    {
        let view = UIView()
        view.backgroundColor = .white

        self.view = view
    }

    override func viewDidLoad()
    {
        super.viewDidLoad()
        guard let path = Bundle.main.path(forResource: "Lebron", ofType:"m4v")
        else
        {
            debugPrint("File not Found")
            return
        }

        let player = AVPlayer(url: URL(fileURLWithPath: path))
        player.appliesMediaSelectionCriteriaAutomatically = false

        let asset = AVAsset(url: URL(fileURLWithPath: path))

        for characteristic in asset.availableMediaCharacteristicsWithMediaSelectionOptions {
            print("\(characteristic)")

            // Retrieve the AVMediaSelectionGroup for the specified characteristic.
            if let group = asset.mediaSelectionGroup(forMediaCharacteristic: characteristic) {
                // Print its options.
                for option in group.options {
                    print("  Option: \(option.displayName)")
                }
            }
        }

        if let group = asset.mediaSelectionGroup(forMediaCharacteristic: AVMediaCharacteristic.legible) {
            let locale = Locale(identifier: "en")
            let options =
                AVMediaSelectionGroup.mediaSelectionOptions(from: group.options, with: locale)
            if let option = options.first {
                // Select Spanish-language subtitle option
                player.currentItem!.select(option, in: group)
                print("\(player.currentItem!.select(option, in: group))")
            }
        }

        let playerViewController = AVPlayerViewController()
        playerViewController.player = player

        addChild(playerViewController)
        playerViewController.view.frame = view.frame
        view.addSubview(playerViewController.view)
        playerViewController.didMove(toParent: self)
        playerViewController.player?.seek(to: CMTime(seconds: 0, preferredTimescale: 1))

    }
}

This is what is output to the console
2019-05-30 21:52:21.432911-0400 subtitleTest[6438:1450671] [Accessibility] ****************** Loading GAX Client Bundle ****************
AVMediaCharacteristic(_rawValue: AVMediaCharacteristicAudible)
  Option: Unknown language
AVMediaCharacteristic(_rawValue: AVMediaCharacteristicLegible)
  Option: Spanish
  Option: Spanish Forced
  Option: English
  Option: English Forced
()

The device should display whichever subtitle file matches the input locale, but instead it doesn't display any subtitles at all.
Comment: I also find this problem. the forced subtitle is selected, but it's not displayed at all.
Answer: I eventually got this working. This was the code I ended up using. Just need to change LebronTestProdNamed to the name of your video file, and .m4v to the filetype of your video file.
import UIKit
import AVFoundation
import AVKit

class ViewController : UIViewController
{
    override func loadView()
    {
        let view = UIView()
        view.backgroundColor = .white

        self.view = view
    }
    var test = AVPlayerMediaSelectionCriteria()

    override func viewDidLoad()
    {
        super.viewDidLoad()
        guard let path = Bundle.main.path(forResource: "LebronTestProdNamed", ofType:"m4v")
            else
        {
            debugPrint("File not Found")
            return
        }

        let player = AVPlayer(url: URL(fileURLWithPath: path))
        player.appliesMediaSelectionCriteriaAutomatically = false

        let asset = AVAsset(url: URL(fileURLWithPath: path))
        let playerItem = AVPlayerItem(asset: asset)

        for characteristic in asset.availableMediaCharacteristicsWithMediaSelectionOptions {
            print("\(characteristic)")

            // Retrieve the AVMediaSelectionGroup for the specified characteristic.
            if let group = asset.mediaSelectionGroup(forMediaCharacteristic: characteristic) {
                // Print its options.
                for option in group.options {
                    print("  Option: \(option.displayName)")
                }
            }
        }

        // Create a group of the legible AVMediaCharacteristics (Subtitles)
        if let group = asset.mediaSelectionGroup(forMediaCharacteristic: AVMediaCharacteristic.legible) {
            // Set the locale identifier to the value of languageID to select the current language
            let locale = Locale(identifier: "en-EN")
            // Create an option group to hold the options in the group that match the locale
            let options =
                AVMediaSelectionGroup.mediaSelectionOptions(from: group.options, with: locale)
            // Assign the first option from options to the variable option
            if let option = options.first {
                // Select the option for the selected locale
                playerItem.select(option, in: group)
            }
        }

        let playerViewController = AVPlayerViewController()
        playerViewController.player = player
        playerViewController.player?.replaceCurrentItem(with: playerItem)

        addChild(playerViewController)
        playerViewController.view.frame = view.frame
        view.addSubview(playerViewController.view)
        playerViewController.didMove(toParent: self)
        playerViewController.player?.seek(to: CMTime(seconds: 0, preferredTimescale: 1))

    }
}

Comment: For anyone trying to get this to work with AVPlayer/AVPlayerLayer, it doesn't.  Only works if embedded in AVPlayerViewController like @JoshHolme has done above.


Prove that $f(x) = \varphi (x) + P_{n+1}(x)$ given $f(x+T) = f(x) + Q_n(x)$ where $\varphi (x)$ is periodic with period $T$.

Question: The question below is an attempt to generalize this idea.

Given a function $f(x)$ there exist $T \ne 0$ and some polynomial $Q_n(x)$ of $n$-th power such that $\forall x \in D(f)$:
  $$
x+T \in D(f) \\
x-T \in D(f)
$$
And $f(x+T) = f(x) + Q_n(x)$.
Prove that:
  $$
f(x) = \varphi (x) + P_{n+1}(x)
$$
  Where $\varphi (x)$ is periodic with period $T$ and $P_{n+1}(x)$ is a polynomial of $(n+1)$-th power.

As shown in the linked question we may use the fact that $\varphi (x)$ is periodic. So it has been shown that:
$$
\varphi (x) = \varphi (x+T) = f(x) - {a\over T}x
$$
in case $f(x+T) = f(x) + a$.
Using the same idea I have shown that:
$$
f(x) = \varphi (x) + {a\over {2T}}\left(x^2 - Tx\right)
$$
in case $f(x+T) = f(x) + ax$.
And now i'm struggling to show that this idea may be generalized. I've tried to use the same technique suggested in the linked question.
I started with presenting $Q_n(x)$ and $P_{n+1}(x)$ in the form of a sum with some unknown coefficients, but that got ugly very quickly, since $(x+T)^k$ appears in the sum when applying $\varphi (x)$ to $x+T$.

How can i prove the above?

upd 1:
Here are some further thoughts. So i want $\varphi (x) = \varphi (x+T)$. Let's try to find coefficients of $Q_0(x)$ and $P_1(x)$ to see whether we could deduce the polynomials ($a_k$ is coefficient of $Q_n(x)$ and $b_k$ is for $P_n(x)$):
$$
f(x+T) = f(x) + a_1 \\
f(x) = \varphi (x) + b_1x + b_2 \\
\varphi (x) = f(x) - b_1x - b_2 \\
\varphi (x + T) = f(x+T) - b_1(x+T) - b_2 \\
\varphi (x+T) = \varphi (x) \\
f(x) + a_1 - b_1(x+T) - b_2 = f(x) - b_1x - b_2 \\
a_1 - b_1x -b_1T-b_2 = -b_1x -b_2
$$
This gives $b_2 = 0$ and $b_1 = {a_1\over T}$. So now if $a_1 = a$ we get $f(x+T) = f(x) + a$ and $f(x) = \varphi(x) + {a\over T}x$. The exact situation as in the linked post.
For $Q_1(x)$ and $P_2(x)$ i get that:
$$
2b_1Tx + b_1T^2 + b_2T = a_1x+a_2
$$
So $b_1 = {a\over 2T}$ and $b_2$ may be obtained only by the assumption that $a_2 = 0$. If $a_2 = 0$ then $b_2 = {a\over 2}$ and we get the case for $Q_1$ and $P_2$. But the assumption is taken from nowhere.
upd 2:
I've just tried to find $Q_2(x)$ and $P_3(x)$ and finally arrived at a system of equations for the coefficients:
$$
\cases {
b_1 = {a_1 \over 3T} \\
b_2 = {{a_2 - 3b_1T^2}\over {2T}} \\
b_3 = {{a_3 - b_1T^3 - b_2T^2} \over {T}}
}
$$
In this case I've tried to assume that $a_2 = 0$ and $a_3 = 0$ and therefore the following is true:
$$
f(x) = \varphi (x) + \frac{a}{3T} (x^3 - \frac{3x^2T}{2} + 2T^2x)
$$
in case $f(x+T) = f(x) + ax^2$. It seems like $Q_n(x)$ is in the form of $ax^n$. But $P_n(x)$ has somewhat more complicated form.
Comment: Are you familiar with vector spaces ? Your technique actually reduces the problem to a question exclusively about polynomials.
Comment: @Segipp Unfortunately no
Answer: Let $\mathcal P_n$ be the vector space of polynomials of degree $\le n$, and $T>0$ fixed. Then $p(x)\mapsto p(x+T)-p(x)$ is a linear map $\Delta_T\colon\mathcal P_n\to \mathcal P_{n-1}$ because the highest powers cancel. If $p\in\ker\Delta_T$, then $p$ is periodic and being, being a polynomail, $p$ is constant (and indeed, if $p$ is constant, then $\Delta_Tp=0$). We conclude that $\dim\ker\Delta_T=1$, hence from $\dim \mathcal P_n=n+1$, $\dim \mathcal P_{n-1}=n$, we conclude that $\Delta_T$ is onto.
Now let $f,Q$ as in the problem statement. From the surjectivity of $\Delta_T$, we obtain $P\in \mathcal P_{n+1}$ with $\Delta_TP=Q$. Let $\phi(x)=f(x)-P(x)$. Then $$\phi(x+T)-\phi(x)=f(x+T)-f(x)-P(x+T)+P(x)=Q(x)-Q(x)=0,$$
henec $\phi$ is periodic as desired.
Comment: Thank you for the answer. I wish i could understand this. But it's too far beyond my level of knowledge for now
Answer: Generalizing your computations, we can find (without the theory of vector spaces) a polynomial $P(X)=\sum_{k=0}^{n+1}b_kX^k$ such that $P(X+T)-P(X)=Q(X)=\sum_{k=0}^na_kX^k$ (and then we finish as in Hagen von Eitzen's answer).
Compute
$$ P(X+T)-P(X)=\sum_{k=0}^{n+1}\sum_{i=0}^kb_k\binom{k}{i}X^iT^{k-i} - \sum_{k=0}^{n+1}b_kX^k = \sum_{i=0}^{n+1}\left(\sum_{k=i+1}^{n+1}b_k\binom{k}{i}T^{k-i}\right)X^i $$
so that the condition is equivalent to having, for all $i$ from $0$ to $n$ :
$$ \sum_{k=i+1}^{n+1}b_k\binom{k}{i}T^{k-i} = a_i $$
which is, taking $j=i+1$ ($j$ goes from 1 to $n+1$) and taking out $k=j$ from the sum :
$$ b_j = \frac{1}{jT}\left(a_{j-1}-\sum_{k=j+1}^{n+1}b_k\binom{k}{j-1}T^{k-j+1}\right) = \frac{1}{j}\left(\frac{a_{j-1}}{T}-\sum_{k=j+1}^{n+1}b_k\binom{k}{j-1}T^{k-j}\right) $$
so the coefficients $b_j$ can be computed recursively by starting from $j=n+1$ and going down :

first $b_{n+1} = \frac{a_n}{(n+1)T}$
then $b_n = \frac{1}{n}\left(\frac{a_{n-1}}{T}-b_{n+1}\binom{n+1}{n-1}T\right) = \frac{1}{n}\left(\frac{a_n}{T}-\frac{(n+1)nb_{n+1}}{2}\right)$ which we can compute because we already computed $b_{n+1}$
then we can compute $b_{n-1}$ using $b_{n+1}$ and $b_n$
and so on until $b_1$ (as for $b_0$, from our analysis there is no condition on $b_0$, so any choice works)

This shows that given $Q_n$ we can always find a $P_{n+1}$ satisfying our requirements (and this $P_{n+1}$ is unique up to its constant coefficient).


REST won't work

Question: On a Drupal6 installation I have the REST service enabled (6.x-3.3).
All seems to setup correctly, but I cannot query any data and get always 404 Not Found.
What is the correct URL for retieve node #1:
<code>    http://local.host/rest/node/1
    http://local.host/services/rest/node/1
</code>
None of them returns any data.
<code>$endpoint = new stdClass;
$endpoint->disabled = FALSE; /* Edit this to true to make a default endpoint disabled initially */
$endpoint->api_version = 3;
$endpoint->name = 'rest';
$endpoint->server = 'rest_server';
$endpoint->path = 'rest';
$endpoint->authentication = array();
$endpoint->server_settings = array(
  'rest_server' => array(
    'formatters' => array(
      'json' => TRUE,
      'bencode' => FALSE,
      'jsonp' => FALSE,
      'php' => FALSE,
      'rss' => FALSE,
      'xml' => FALSE,
      'yaml' => FALSE,
    ),
    'parsers' => array(
      'application/json' => TRUE,
      'application/vnd.php.serialized' => FALSE,
      'application/x-www-form-urlencoded' => FALSE,
      'application/x-yaml' => FALSE,
      'multipart/form-data' => FALSE,
    ),
  ),
);
$endpoint->resources = array(
  'node' => array(
    'operations' => array(
      'retrieve' => array(
        'enabled' => 1,
      ),
      'create' => array(
        'enabled' => 1,
      ),
      'update' => array(
        'enabled' => 1,
      ),
      'delete' => array(
        'enabled' => 1,
      ),
      'index' => array(
        'enabled' => 1,
      ),
    ),
    'relationships' => array(
      'files' => array(
        'enabled' => 1,
      ),
      'comments' => array(
        'enabled' => 1,
      ),
    ),
  ),
  'system' => array(
    'actions' => array(
      'connect' => array(
        'enabled' => 1,
      ),
      'get_variable' => array(
        'enabled' => 1,
      ),
      'set_variable' => array(
        'enabled' => 1,
      ),
      'del_variable' => array(
        'enabled' => 1,
      ),
    ),
  ),
);
$endpoint->debug = 0;
</code>
Update
Get this running now:

reinstall Drupal
Put in a context (drupal)
http://local.host/drupal/rest/node/1.json respons with node#1 data in JSON format.
Answer: <code>$endpoint = new stdClass();
$endpoint->disabled = FALSE;
$endpoint->api_version = 3;
$endpoint->name = 'rest';
$endpoint->server = 'rest_server';
$endpoint->path = 'rest';
$endpoint->authentication = array();
$endpoint->server_settings = array(
  'rest_server' => array(
    'formatters' => array(
      'json' => TRUE,
      'bencode' => FALSE,
      'jsonp' => FALSE,
      'php' => FALSE,
      'rss' => FALSE,
      'xml' => FALSE,
      'yaml' => FALSE,
),
'parsers' => array(
  'application/json' => TRUE,
  'application/vnd.php.serialized' => FALSE,
  'application/x-www-form-urlencoded' => FALSE,
  'application/x-yaml' => FALSE,
  'multipart/form-data' => FALSE,
  ),
 ),
);
$endpoint->resources = array(
 'comment' => array(
  'operations' => array(
  'create' => array(
    'enabled' => '1',
  ),
  'retrieve' => array(
    'enabled' => '1',
  ),
  'update' => array(
    'enabled' => '1',
  ),
  'delete' => array(
    'enabled' => '1',
  ),
  'index' => array(
    'enabled' => '1',
  ),
),
'actions' => array(
  'countAll' => array(
    'enabled' => '1',
  ),
  'countNew' => array(
    'enabled' => '1',
  ),
),
  ),
'file' => array(
'operations' => array(
  'create' => array(
    'enabled' => '1',
  ),
  'retrieve' => array(
    'enabled' => '1',
  ),
  'delete' => array(
    'enabled' => '1',
  ),
  'index' => array(
    'enabled' => '1',
  ),
),
'actions' => array(
  'create_raw' => array(
    'enabled' => '1',
  ),
 ),
),
'node' => array(
'operations' => array(
  'retrieve' => array(
    'enabled' => '1',
  ),
  'create' => array(
    'enabled' => '1',
  ),
  'update' => array(
    'enabled' => '1',
  ),
  'delete' => array(
    'enabled' => '1',
  ),
  'index' => array(
    'enabled' => '1',
  ),
),
'relationships' => array(
  'files' => array(
    'enabled' => '1',
  ),
  'comments' => array(
    'enabled' => '1',
  ),
),
'targeted_actions' => array(
  'attach_file' => array(
    'enabled' => '1',
  ),
  ),
 ),
 'system' => array(
 'actions' => array(
  'connect' => array(
    'enabled' => '1',
  ),
  'get_variable' => array(
    'enabled' => '1',
  ),
  'set_variable' => array(
    'enabled' => '1',
  ),
  'del_variable' => array(
    'enabled' => '1',
  ),
 ),
 ),
 'taxonomy_term' => array(
'operations' => array(
  'retrieve' => array(
    'enabled' => '1',
  ),
  'create' => array(
    'enabled' => '1',
  ),
  'update' => array(
    'enabled' => '1',
  ),
  'delete' => array(
    'enabled' => '1',
  ),
  'index' => array(
    'enabled' => '1',
  ),
),
'actions' => array(
  'selectNodes' => array(
    'enabled' => '1',
  ),
),
),
'taxonomy_vocabulary' => array(
'operations' => array(
  'retrieve' => array(
    'enabled' => '1',
  ),
  'create' => array(
    'enabled' => '1',
  ),
  'update' => array(
    'enabled' => '1',
  ),
  'delete' => array(
    'enabled' => '1',
  ),
  'index' => array(
    'enabled' => '1',
  ),
),
'actions' => array(
  'getTree' => array(
    'enabled' => '1',
  ),
),
),
'user' => array(
'operations' => array(
  'retrieve' => array(
    'enabled' => '1',
  ),
  'create' => array(
    'enabled' => '1',
  ),
  'update' => array(
    'enabled' => '1',
  ),
  'delete' => array(
    'enabled' => '1',
  ),
  'index' => array(
    'enabled' => '1',
  ),
),
'actions' => array(
  'login' => array(
    'enabled' => '1',
  ),
  'logout' => array(
    'enabled' => '1',
    'settings' => array(
      'services' => array(
        'resource_api_version' => '1.0',
      ),
    ),
  ),
  'register' => array(
    'enabled' => '1',
  ),
 ),
),
);
$endpoint->debug = 0;
</code>
can you check with this code
Comment: The same, 404 at http://localhost/services/rest/node/1
Comment: can you check with browser localhost/services/rest/node/1.json
Answer: Take a look at Services Installation and Setup
Also you can read Testing resources on a test server to dig deeper.
Adding a REST server

Browse to admin/build/services
Click on the Add tab
Set the name of you rest server for example rest_server
 Select the  REST   to be the Server Type
Type the Path to endpoint, I set to rest_server_endpoint
  Authentication to be   Session authentication
Save
Switch to the Server tab and allow  application/x-www-form-urlencoded as a request paring type
Switch to the resources tab and select the resources you need to manipulate with rest client for example if you need to login check on users-> login
Save
Answer: After successful installation of rest service, and setup the endpoint url. If you still getting the page not found error means, try to change the path to end point and try . it will work fine !

REST won't work


AWS CloudFront Limit Origin Access

Question: I am averaging 12,000 page views per day, but my cloudfront is getting around 6 million requests for images and fonts every day.  I suppose it is possible that all of these requests are coming from my visitors, but I am having my doubts.
I know that I can enable keys and cookies so that only users of my website will be able to access the cloudfront servers, but I am not convinced yet that this is necessary.
Is there a way for me to check what percentage of the requests are from users that are being referenced from my server?  Before I go and change my code to work with the private origin access?
Comment: Did you enable logging?  Or look at the "top referers" report in the CloudFront console?  6M requests from 12k views seems sketchy.
Comment: Thank you that's exactly what I was looking for, looks like its all coming from the site, or at least 97%
Answer: Don't bother micromanaging and overanalyzing. Just use the keys and cookies now because that's good practice to protect your assets and bandwidth anyway. It's an easy decision to make, and you'll not have to spend four times the effort to track abusers.
Answer: The CloudFront section of the AWS console has several reports you can view or download, including a Top Referrers Report.

The CloudFront console can display a list of the 25 domains of the websites that originated the most HTTP and HTTPS requests for objects that CloudFront is distributing for a specified distribution. These top referrers can be search engines, other websites that link directly to your objects, or your own website.
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/top-referrers-report.html

You can also enable detailed logging of each individual request, which stores the logs in an S3 bucket for thorough analysis.

AWS CloudFront Limit Origin Access


IBM DB2 unable to access database from PHP after reboot

Question: System: Windows 8
I have been working on a project for school using IBM DB2 Express-C, Xampp, and PHP to access the database.
Everything has been working perfectly. Then one day I rebooted my computer.
IBM Database Manager returned the following error at startup:
SQL5005C The operation failed because the database manager failed to access either the database manager configuration file or the database configuration file.
I have tried running db2stop and db2start.
I am still able to access my database via the db2 command line terminal as if nothing is wrong.
However, when I try to access my database from PHP I now get the following error from my PHP file:
58031
[IBM][CLI Driver] SQL1031N The database directory cannot be found on the indicated file system. SQLSTATE=58031 SQLCODE=-1031
Answer: You might want to double check the db2diag.log to find the reason for SQL5005C. In the better case there was a temporary problem accessing the configuration files. However, in the worse case one of the DBM or DB configuration files may have become corrupt. In the case of a corrupt DBM CFG, I usually use db2idrop/db2icrt. A corrupt DB CFG can be "fixed" (as in "reset", you will lose all custom changes) by RESET DB CFG.
The subsequent SQL1031N tells me the database you are trying to access has been uncataloged. This is also odd, especially since you are telling me you are able to access the database from the DB2 command line. You might want to do LIST DB DIRECTORY from your command line to show your database entries, and also LIST NODE DIRECTORY to see which nodes the databases reside on.
Answer: The possible reason is that you lost the database files on your hard disk.

Use command to check.
db2diag -g db=database name > log.txt

open log.txt,you will see the message like below. In this case, I lost some db2 config files.

Create missing folder and copy the db2
config files from the backup

Using backup file to restore database.
Answer: When you try to interact with DB2 from a programming language (PHP, python or others) you use a library (example ibm_db for python) that interact with a db2 client application to connect/query your DB.
If you have more than one IBM Data Server client packages installed on your workstation you might not use the right one to access your DB.
So you have to verify which client your PHP api has using to do this job. To do this:

identify your ../clidriver/bin folder that contains your db2 client utilities (in my case it was in my python installation folder/site-packages/ibm_db-2.0.8-py3.6.egg/clidriver/bin
run command db2cli validate that will show you the client you are using
If you have more than one entry in List of all IBM Data Server client packages on the current workstation: and you are not using one of the IBMDBCL_ client, you must switch the client by double clicking on db2swtch.exe (run as administrator) and following the visual instructions to switch to the IBMDBCL_ item
if it doesn't work, uninstall and reinstall your library (in my case pip uninstall ibm_db and easy_install ibm_db)

You can find more information about the management of more db2 client on the same machine here


Question about notation - the differential

Question: In physics it is usually denote $dx^i$ by an "infinitesimal" variation of the variable $x^i$, and for example, the work could be written as:
$$\delta W=\sum_i Y^i dx^i$$
(1) This implies that in sense $\delta W$ is a 1-form?
(2) How is the conceptual relation between an "infinitesimal" variation of $x^i$ and the differential of the coorditate $x^i$?
Many thanks!
Comment: If you are envisioning the $x^i$ as being coordinates on the manifold, then the $\mathrm{d}x^i$ are coordinates on the tangent space.
(and it can be fruitful to envision the points of the tangent space at $P$ as being a collection of points infinitesimally close to $P$)
Answer: To answer your first question, yes $dW$ is a $1$-form based on this definition $dW = \sum Y^i\,dx^i$. The idea is that $dW$ itself is something to be integrated to find the total work $W = \int dW = \int\sum Y^i\,dx^i$.
To understand the conceptual relation between an infinitesimal variation of $x^i$ and the $1$-form $dx^i$, consider as in single-variable integration the object $dx$ as "a small change in $x$". In terms of work and forces, $dx$ is a small displacement of the particle, say. Hence, $W = \int dW = \int F\,dx$.
In the multivariable setting the idea is the same in that each of the $dx^i$ accounts for "small displacements" in each direction so that
$$
W = \int dW = \int \sum Y^i\,dx^i.
$$


Why do jamming musicians "trade fours" (rather than any other number)?

Question: I've run across the idea of "trading fours" when playing in (and listening to) jazz groups; i.e. where two or more musicians take it in turns to play four bar licks or solos. I understand that it can encourage creative interaction between musicians.
However, I was wondering why it seems to be specifically four bars? Would trading twos or eights (for example) be less effective?
Answer: It's just a name: it used to be based on four bars, which probably would comprise one set of chord changes (eg doowop, I vi IV V), but could just as easily be two or eight bars. It's like calling a song's bridge a 'middle eight', even though the number of bars may be different. The Beatles always called their bridges 'middle eights'.
Answer: In swing setups such (e.g. tenor sax battles), it is not uncommon so see "trades" of varying (typically decreasing) length : trade 16, then trade 8, trade 4 and sometimes even trade 2 then trade 1, each time building up the tension. Things could also end in both musicians improvising simultaneously. Nice example from Robert Altman's Kansas City:

As for the reason behind 4 bars, I think it comes from the structure of the typical 32 bar form AABA, in which the 8 bars of A are often divided in 4 bars (tension / question) often ending in half cadence (V7) + four bars (relief / answer) ending in IMaj7. This means having 4 bar solo by one musician followed by for bars by another is quite natural.
Answer: "Trading twos" is definitely a phrase people use. "Trading eights" might be.
More generally, you can call it "trading licks".
8 bars is long enough to be thought of as a whole solo. Get any longer than this and you're not really "trading" any more; you're just taking it in turns to solo.
When I say "trading", I mean that you base your lick on whatever the player before you played.
Answer: There are great answers already here, but perhaps the most important consideration is making sure no one gets cut short when the trading is over.
Consider a 12-bar blues. It wouldn't really work for the soloists to trade off every 8 bars, because whoever goes second in soloing would only be halfway through his/her turn when the 12-bar form ended. (In particular, the first soloist would take bars 1-8, and the second soloist would take bars 9-12 followed by bars 1-4.) Spilling over like that can undermine the listener's sense of the form itself. Our ears generally expect to hear the soloist change at the very top/beginning of the form. If that doesn't happen, it feels uneasy, and we might get the feeling that we don't actually know where the top of the form is.
Jazz songs very frequently use 12-bar forms, 16-bar forms, or 32-bar forms. Letting the soloists trade off every 4 bars is great because all of those forms are evenly divisible by 4. Thus no soloist's turn will be interrupted by the end of the form. Trading 2 bars at a time is also common, and this frequently comes near the end of the trading, after having spent some time trading 4 bars. Trading 8 bars is also pretty common, although only if the entire song has an integer multiple of 8 bars.
Answer: Generally it'll be even numbers, 2, 4 , 8 as a 'line' of a song tends to be that long, so phrasing sounds more balanced. It follows from songs usually being 8, 12, 16 or occasionally 24  bars long. Never heard in a band "Let's trade 7s." Although that could be interesting. Both to play and listen to...
Answer: In most tunes, four bars is the smallest unit you will find a complete phrase, musical idea, or melody. Although you will often see a complete chorus or bridge or A section of the tune in eight as well, but youll never see it in two because two bars is too short to express a complete musical idea.
Comment: I wasn't referring to "trading twos" I was explaining that you don't see choruses, verses, etc that are just two bars long.
Comment: "but you'll never see it in two" -- this is just wrong. Trading twos is quite common.
Comment: Right, but you don't need a chorus or verse to express a musical idea, and that is exactly what players are doing when they trade twos, fours, eights, etc., that is, they are expressing a musical idea for other players to respond to. You can fit a phrase into two bars, and players do this all the time.
Comment: you can fit a "phrase" into 1/2 bar, but it's not going to be meaningful. That's why I'm saying, in my opinion 4 bars is used because it's the smallest general-purpose amount to create a meaningful idea in.
Comment: Do you mean one or two bars instead of 1/2 bar? In any case, players _do_ trade 2 bar phrases. In particular, it is very common in traditional folk/old-timey/Americana circles for a group of players to cycle around trading twos for part of a song. I used to play with a mandolin player who would call "trade two bars", and everyone knew exactly what he meant. I don't know what constitutes a "meaningful" idea for you, but an idea to be offered and responded to by another player seems meaningful enough, and certainly the point of trading any number of bars.
Comment: Oh that's interesting i didn't know that. I've been in some jazz settings where it's almost always 4s. I guess if you trade twos, then if the next person keeps the motif going then it could sound more continuous.


Open Dynamics Engine não reconhece colisão entre cilindros

Question: Estou utilizando a biblioteca ODE (Open Dynamics Engine) para a simular física na minha aplicação. Para criar o bounding box e geometria do cilindro estou usando o seguinte código
dMass m1;
dMassSetZero(&m1);
dMassSetCappedCylinderTotal(&m1, mass, 2, lx/2.0, lz);
dBodySetMass(body, &m1);
dBodySetPosition(body, x0, y0, z0);
dBodySetRotation(body, R);

dGeomID geom = dCreateCylinder(PhysicsEngine::space, lx / 2.0, lz);
dGeomSetBody(geom, body);

Nenhum erro é gerado, e objeto é gerado corretamente. Porém ao executar o tratamento de colisões:
const int N = 100;
dContact contact[N];
int n = dCollide(o1, o2, N, &contact[0].geom, sizeof(dContact));

for (int i = 0; i < n; i++) {
    contact[i].surface.mode = dContactSlip2 | dContactSlip1;
    contact[i].surface.mu = dInfinity;
    contact[i].surface.slip1 = 0.1;
    contact[i].surface.slip2 = 0.1;
    dJointID c = dJointCreateContact(PhysicsEngine::world, PhysicsEngine::contactgroup, &contact[i]);
    dJointAttach(c, dGeomGetBody(contact[i].geom.g1), dGeomGetBody(contact[i].geom.g2));
}

A engine reconhece colisões de:

Plano e Cilindro
Cubo e Cilindro
Esfera e Cilindro

mas não reconhece Cilindro com Cilindro, como na imagem

Alguém tem alguma idéia do que pode ser? Estou utilizando a versão atual da biblioteca, disponível neste repositório e usei o seguinte comando para compila-lo
./premake4.exe --with-tests --with-libccd --with-demos vs2010

Comment: Dá uma olhada na documentação wiki da própria biblioteca, na seção [Collision Tests Supported](https://www.ode-wiki.org/wiki/index.php?title=Manual:_Collision_Detection#Collision_tests_supported). Lá tem uma nota (nota 3) dizendo que as colisões entre cilindros requerem que a [libccd esteja habilitada](https://www.ode-wiki.org/wiki/index.php?title=Enabling_the_libccd_colliders). Talvez seja isso que falte.
Comment: Só agora eu notei que na sua linha de comando você de fato já inclui a tal libccd. Bom, fora isso, talvez você esteja pegando as DLLs de um local equivocado? Outra coisa: o seu "tratamento" de colisões na verdade cria joints. O problema é na criação dos joints ou de fato a chamada de `dCollide` não tá funcionando? Depure pra ter certeza.
Comment: @LuizVieira Valeu, funcionou direitinho. Eu tinha recompilado, mas só atualizei o cabeçalhos (.h) e as bibliotecas (.lib), mas não atualizei o DLL. Como não apresentava erros pensei que estava tudo em ordem
Comment: https://imgur.com/a/xyjOWwT
Comment: O tratamento de colisão ocorre em loop,  e recupera todas as colisões com o dCollide, em seguida eu configuro as forças através do dCreateContact
Comment: Que bom que funcionou. Boa sorte.
Answer: Obrigado pela ajuda dos comentários. Para resolver este problema eu precisei recompilar a biblioteca do ODE (Open Dynamics Engine) junto com a aplicação libccd (Biblioteca para colisões entre objetos convexos). Acontece que o ODE, como depende atualmente de contribuições, não possui uma implementação nativa de colisão para cilindro para cilindro (por se tratar de um objeto convexo). Porém a biblioteca libccd pode ser compilado junto com o ODE para complementar a engine, além disso o libccd é utilizado por outras bibliotecas de simulação de físicas que você pode encontrar neste artigo.
Aproveito para criar um tutorial em Português para esta engine de simulação fisica. Eu vou explicar passo a passo como fazer isso para Microsoft Visual Studio (2017).
1. Primeiro você precisa ir no repositório do bitbucket do ODE e baixar o código-fonte (compactado em tar.gz) e extrair em alguma pasta.
2. Abra o prompt de comando e vá para a pasta build (cd ./pasta/do/ode/build)
3. Execute o comando para gerar o projeto para Visual Stuido, ativando a opção do libccd
./premake4.exe --with-tests --with-libccd --with-demos vs2010

(Observação, atualmente o ODE possui configuração para vs2010 para baixo, porém funciona no Visual Studio 2017)
4. O comando irá gerar o projeto para visual studio na pasta ./build/vs2010 e abra ode.sln
5. Dentro do Visual Studio compile todos os projetos (talvez seja necessário redirecionar o projeto para o SDK da sua IDE).
6. Na pasta principal vá para a :

\include, para adicionar as duas pastas com arquivos de cabeçalho ao seu projeto através das configurações do Visual Studio, em Diretórios VC++ > Diretório de Inclusão
\lib\DebugDoubleDLL, para adicionar os arquivos (menos os de teste), ao seu projeto através das configurações do Visual Studio, em Diretórios VC++ > Diretório de Biblioteca

7. Copie o arquivo ode_doubled.dll na pasta de saída do executável (para evitar erros), arquivo que esta na pasta \lib\DebugDoubleDLL.
8. No seu projeto, vá em Configurações do Projeto > C/C++ > Pré-processador > Definições de Pré-Processador e adicione o pré-processador dDOUBLE
9. No seu projeto, vá em Configurações do Projeto > Vinculador > Entrada > Dependências Adicionais e adicione a dependência ode_doubled.lib
10. Já está configurado o ODE, para testar use este exemplo do tutorial de demura e meu código na pergunta para criar um cilindro.
Resultado:


Asp.net Client-Server

Question: Hello i have a web api with individual user accounts that creates tokens and send them back to the client.
I created an mvc client in a separate project that gets this token from the web api using the following function.
    private async Task<Dictionary<string,string>> GetTokenAsync()
    {
        var client = new HttpClient();

        var post = new Dictionary<string, string>
        {
            {"grant_type","password" },
            {"username","email@example.com" },
            {"password","Panagorn18!" }
        };

        var response = await client.PostAsync("http://localhost:55561/token", new FormUrlEncodedContent(post));
        //response.StatusCode == HttpStatusCode.Unauthorized
        var content = await response.Content.ReadAsStringAsync();

        var json = JObject.Parse(content);

        var tkn = json["access_token"].ToString();
        var ex = json["expires_in"];
        var exp = new DateTime();
        exp.AddSeconds((long)ex);
        var ms = exp.ToUniversalTime().Subtract(
            new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).TotalMilliseconds;
        var dic = new Dictionary<string, string>
        {
            { "token", tkn },
            { "expires", ms.ToString() }
        };
        return dic;
    }

Now my questions are:
1. Where i have to save this token?
2. How can i keep the user loged in for example 30 days?
3. How can i check if the token expired and logout the user in the mvc project?
4. What configuration i have to put at startup class at mvc project to use this tokens?
Answer: 1. Where i have to save this token?

Server side: Session, Memory Cache, etc
Client side: cookie, localStorage, sessionStorage, etc
Others: maybe another cache server (Redis)
Database is also a good place to save
2. How can i keep the user logged in for example 30 days?

It's what token expiry date used for (check AccessTokenExpireTimeSpan)
3. How can i check if the token expired and logout the user?

A good way is implement your own AuthenticationTokenProvider, deserialize the token passed to server, check the expiry date and add the AccessTokenExpired to response header
Sample code:
// CustomAccessTokenProvider.cs
public class CustomAccessTokenProvider : AuthenticationTokenProvider
{
    public override void Receive(AuthenticationTokenReceiveContext context)
    {
        context.DeserializeTicket(context.Token);
        var expired = context.Ticket.Properties.ExpiresUtc < DateTime.UtcNow;
        if(expired)
        {
            context.Response.Headers.Add("X-AccessTokenExpired", new string[] { "1" });
        }

        base.Receive(context);
    }
}

// Startup.cs
public void Configuration(IAppBuilder app)
{
    app.UseOAuthBearerAuthentication(new OAuthBearerAuthenticationOptions
    {
        AccessTokenProvider = new CustomAccessTokenProvider()
    });
}

Comment: Can you please explain me more the Receive function how to use it, where and how to check the expiration?
Comment: @PanayiotisIrakleous: I updated my answer to include usage for CustomAccessTokenProvider
Comment: i put your code on the server side and when token is expired i see in the header the value. So now i have to make a request when the web app (mvc) start to check if the token has expired? and how to keep the user loged in with the expiration date from the token?

Asp.net Client-Server