How to only get file names matching a pattern

Question: I have this below function which looks for the files in the directory.
files = list(filter(lambda f: fnmatch.fnmatch(f, FILENAME +"*"), os.listdir(SRC_DIR)

Here is the example of how it looks
['data.txt',data.done,data_audit.done, data1.trans]

I want to exclude all the .Done files. How do I remove all the .done files from the list?
Thank you for your help in advance
Comment: Try this: `list(filter(lambda f: not f.endswith('.done'), os.listdir(SRC_DIR)))`
Comment: a little unclear because of `FILENAME` not being defined, is the goal to keep all files that begin with `data` but do not end with `.done`?
Comment: Side-note: If you need a `lambda` to use `map`/`filter`, don't. Just use an equivalent list comprehension or generator expression (which can inline the contents of the `lambda` and thereby run faster). `map`/`filter` are basically only appropriate when an existing function does *exactly* what you need, and only provide non-trivial performance benefits if said function is a built-in implemented in C.
Answer: You can use a list comprehension with a filter on a pair of conditions for what I would consider to be the simplest solution.
files = [f for f in os.listdir(SRC_DIR)
         if f.startswith('data') and not f.endswith('.done')]

Comment: Thank you so much.. But I am getting this error. I am on Python 2.7.  "Unicode object has not attribute 'endwith"
Comment: you tagged your question as python-3.x, but that is a Python 2 error. Are you intentionally running Python 2 (if so, change the tags)? Or are you trying to use Python 3?
Comment: Sorry Ryan my bad. By mistake I tagged as Python3.X, We are using python 2.7
Comment: It looks like you just typo'd **end**with instead of **ends**with
Answer: With Python 3.5+, consider using Pathlib. Your example then becomes:
import pathlib as pl
files=[fn for fn in pl.Path("/path_to_dir").glob("data*") if fn.suffix!='.done']

For the current working directory:
files=[fn for fn in pl.Path.cwd().glob("data*") if fn.suffix!='.done']

And the returned files will all be absolute.
If there is a possibility that non files match that glob, you can add:
files=[fn for fn in pl.Path("/path_to_dir").glob("data*")
                     if fn.suffix!='.done' and fn.is_file()]

Advantage over os and glob modules:

No need to import glob module to support file globbing;
The glob is handled by the os, so you do not need to filter ALL the files;
Consistent treatment of paths so there are fewer mistakes with relative paths;
Recursive searching and globbing is supported (similar to os.walk but a bit more simple).

Disadvantages:

Really should be Python 3.6+ since Path support was then added to os and shutil;
No native chdir -- you have to do from os import chdir to be able to change the working directory of the script.
Comment: Thanks! Do we need to use the "suffix" function? This should work too: `files=[fn for fn in pl.Path.cwd().glob("data*.done")]`
Comment: Thank you so much I am on python 2.7 :)
Comment: @arquillos: A glob works too! But the goal was to *exclude* `.done` files. Hence the suffix with the `!=`. Globs works better to *include* a pattern.


jQuery Sortable Newly appended elements not recognized

Question: How does jQuery UI sortable method refresh works. I have Multiple span tag with P tags inside. The P tags are sortable from one span to another. However when i add a new span with p tags inside, the new P tags are not recognized. The refresh method is supposed to make newly added recognizable. but it doesn't seem to be working or doing anything.  $('span').sortable('refresh').
Check http://jsfiddle.net/ySP96/2/
Answer: I believe the problem is that refresh is looking for child elements of only the containers you have already initialised with .sortable(). You're adding both child elements and containers, so you need to initialise those new containers with .sortable(), instead of using .sortable('refresh').
$('#span').click(function() {
    var x = '
NEW Span
';
    $(x).appendTo('#form').sortable({connectWith: 'span'});
});

See the updated demo: http://jsfiddle.net/ySP96/3/.
Comment: Thank you. But what is the purpose of refresh. Thats what i want to know. Give me an example of how refresh can be used.
Answer: You have to do it manually as below:
$("#MyList").sortable({
        update: function() {
            var postData = $(this).sortable("serialize");
            $.post("UpdateItemOrder.aspx", postData, function(theResponse) {
                $("#ResultsArea").html(theResponse);
            });
        },
        beforeStop: function(event, ui) {
            // Handle new items only
            if ($(ui.item).hasClass("new-item")) {
                // Refresh the list
                var postData = $("#MyList").sortable("serialize");
                    $.post("UpdateItemOrder.aspx", postData, function(theResponse) {
                        $("#ResultsArea").html(theResponse);
                    });
            }
        });
});


Evaluate $\int_1^{\infty} {\frac{\ln{x}}{(x-1)(2x-1)}\,dx}$

Question: Evaluate:
$$\int_1^{\infty} {\left(\frac{\ln{x}}{\left(x-1\right)\left(2x-1\right)}\,dx\right)}$$
It turns out that the value of the integral is exactly:
$$\frac{1}{12}\left(\pi^2+6\ln^2{2}\right)$$
as found by WolframAlpha, but Wolfram gives no indication of how it arrives at this curious result.
How would one solve the integral analytically? A first step might be the $u$-sub, $u=\frac{1}{x}$, which gives:
$$\int_1^{\infty} {\left(\frac{\ln{x}}{\left(x-1\right)\left(2x-1\right)}\,dx\right)}=-\int_0^1 {\left(\frac{\ln{x}}{\left(1-x\right)\left(2-x\right)}\,dx\right)}$$
Thanks!
Comment: Try expanding 1 over the denominator in partial fraction form, then break up the integral into two simpler ones.
Answer: Start by performing the substitution $u=\frac 1x$, $\text dx=-\frac {1}{u^2}\text du$:
\begin{align}\int_1^\infty\left(\frac{\ln x}{(x-1)(2x-1)}\right)\text dx&=\int_1^0\left(\left(\frac{\ln\left(\frac 1u\right)}{\left(\frac 1u-1\right)\left(2\frac 1u-1\right)}\right)\times \left(-\frac {1}{u^2}\right)\right)\text du\\
&=-\int_0^1\left(\frac{-\ln\left(\frac 1u\right)}{(u-2)(u-1)}\right) du\\
\end{align}
We can now write the integral as follows, using partial fraction decomposition:
\begin{align}-\int_0^1\left(\frac{-\ln\left(\frac 1u\right)}{(u-2)(u-1)}\right) du&=-\int_0^1\left(-\ln\left(\frac 1u\right)\left(\frac{1}{(u-2)(u-1)}\right)\right)\text du\\
&=-\int_0^1\left(-\ln\left(\frac 1u\right)\left(\frac{1}{u-2}-\frac{1}{u-1}\right)\right)\text du\\
&=-\int_0^1\left(\ln\left(\frac 1u\right)\left(\frac{1}{u-1}-\frac{1}{u-2}\right)\right)\text du\end{align}
This now leaves us with:
\begin{align}-\int_0^1\left(\ln\left(\frac 1u\right)\left(\frac{1}{u-1}-\frac{1}{u-2}\right)\right)\text du&=-\left(\left(\int_0^1\ln\left(\frac 1u\right)\frac1{u-1}\right)\text du-\left(\int_0^1\ln\left(\frac 1u\right)\frac1{u-2}\right)\text du\right)\\
&=\left(\int_0^1\ln\left(\frac 1u\right)\frac1{u-2}\text du\right)-\left(\int_0^1\ln\left(\frac 1u\right)\frac1{u-1}\text du\right)\\
&=\left(\int_0^1-\ln(u)\frac1{u-2}\text du\right)-\left(\int_0^1-\ln(u)\frac1{u-1}\text du\right)\\
&=\left(-\int_0^1\frac{\ln u}{u-2}\text du\right)-\left(-\int_0^1\frac{\ln u}{u-1}\text du\right)\\
&=\left(\int_0^1\frac{\ln u}{u-1}\text du\right)-\left(\int_0^1\frac{\ln u}{u-2}\text du\right)\end{align}
Can you continue from here? At a glance, I would recommend trying integration by parts.
Comment: Could you be more specific on the remaining two integrals? It seems that integration by parts or other standard tricks do not work. I would be glad to know alternatives without using the dilogarithm.
Comment: @pisco125 I'm afraid I don't know any more techniques - I just suggested integration by parts as it looked like it would work at first glance.
Answer: This is related to the dilogarithm function $\text{Li}_2(x)$ and the special value $$\text{Li}_2(\frac{1}{2}) = \frac{\pi^2}{12}-\frac{\ln^2 2}{2}$$
The dilogarithm has integral representation:
$$\text{Li}_2(z) = \frac{\pi^2}{6}+\int_0^{1-z} \frac{\ln t}{1-t} dt$$
For your question, we have
$$\begin{aligned}
\int_0^1 \frac{\ln{x}}{\left(1-x\right)\left(2-x\right)}dx &= \int_0^1 \frac{\ln x}{1-x}dx - \int_0^1\frac{\ln x}{2-x}dx \\
&= \int_0^1 \frac{\ln x}{1-x}dx - 2\int_0^{1/2}\frac{\ln 2x}{2-2x}dx \\
&= \int_0^1 \frac{\ln x}{1-x}dx-(\ln 2) \int_0^{1/2}\frac{1}{1-x}dx-\int_0^{1/2}\frac{\ln x}{1-x}dx \\
&= -\frac{\pi^2}{6}-\ln^2 2 -[\text{Li}_2(\frac{1}{2})-\frac{\pi^2}{6}] \\
&= -\frac{\pi^2}{12}-\frac{\ln^2 2}{2}
\end{aligned}$$


Fit VAR model with unknown order in Matlab

Question: I have a multivariate observed time series $Y_t$ and I want to find the best fitting VAR process for it.
I have the econometric toolbox in Matlab and can use 'vgxvarx' if I pre-specify an order for the VAR process. However I want the order of the process to be estimated as well. Is there a function to do this in Matlab?
Comment: How much data you have? How many dimensions?
Answer: There is no automatic order selection. You can run many different orders in the loop and compare fit statistics to choose the best order.
I would also run cross correlation for all variable pairs to get an idea of lead/lag relationships. ACF/PACF also helps to get an intuition on lag structure of the series themselves.
Answer: There may be some ways to do it (including p in the likelihood) but you are no more taking the advantage of VAR model (whis is a simple MLE).
I would suggest:

plot the acf of your time series, you will have an idea of p.
Estimate the VAR with p-1,p,p+1, and see if there is significant changes between coefficients.


Como deletar as noticias do banco MySQL?

Question: Neste link ensina a listar comentários (no meu caso noticias) usando banco de dados externo (tem o download do projeto).
Como posso deletar uma notícia ao selecionar, e ainda fazer com que apareça um Dialog perguntando se quero ou não deletar a noticia do banco de dados externo?
Esse é o código do INSERT:
public void chamaCadastrarNoticia(){
    setContentView(R.layout.cadastrar_noticia);

    editTituloNoticia=(EditText) findViewById(R.id.editTituloNoticia);
    editNoticia=(EditText) findViewById(R.id.editNoticia);
    btCadastrarNoticia=(Button) findViewById(R.id.btCadastrarNoticia);

    btCadastrarNoticia.setOnClickListener(new View.OnClickListener() {
        public void onClick(View arg0) {
            String vazios = "";
            if (editTituloNoticia.getText().toString().equals(""))
                vazios="Campo Titulo não pode estar vazio\n\n";

            if (editNoticia.getText().toString().equals(""))
                vazios="Campo Noticia não pode estar vazio\n\n";

            if  (editTituloNoticia.getText().toString().equals("") || editNoticia.getText().toString().equals(""))
                mensagemExibir("Erro:", ""+vazios);
            else {
                String urlPost="#url"; // URL
                String urlGet="#url   ?titulo="+editTituloNoticia.getText().toString()+"noticia="+editNoticia.getText().toString(); // URL
                ArrayList<NameValuePair> iparametrosPost = new ArrayList<NameValuePair>();
                iparametrosPost.add(new BasicNameValuePair("titulo",editTituloNoticia.getText().toString()));
                iparametrosPost.add(new BasicNameValuePair("noticia",editNoticia.getText().toString()));
                String respostaRetornada = null;

                try {
                    respostaRetornada = ConexaoHttpClient.executaHttpPost(urlPost, iparametrosPost);
                    String resposta = respostaRetornada.toString();
                    resposta = resposta.replaceAll("\\s+", "");

                    if (resposta.equals("1")) {
                        mensagemExibir("Cadastrado:", "Acesse Listar Noticias para ver a noticia cadastrada");
                    }

                    chamaIndexAdmin();
                }
                catch(Exception erro){
                    Log.i("erro", "erro = "+erro);
                    mensagemExibir("Erro", "Erro ao gravar: "+erro);
                }
            }
        }
    });
}

Comment: vc ja conseguiu fazer o `INSERT`?
Comment: sim, quero usar esse projeto para listar noticias(deu certo) mas nao consigo implementar para excluir elas
Comment: tem como vc me mostrar como faz para deletar do banco de dados, usando esse projeto la?  la tem para baixar
Comment: acho que eu nao vou precisar baixar; sua pergunta está um pouco ampla, mas se vc ja conseguiu fazer o insert o delete é mt parecido, poderia nos mostrar como vc faz o insert? edite sua pergunta para inserir o código
Comment: nao fiz o insert usando o projeto la, fiz uma mistura, porderia fazer separado mas preciso resolver com urgencia esse problema, quando baixei o projeto, consegui listar ai juntei o codigo com meu projeto antigo
Comment: la msm o cara fala que pode usar o onItemClick para futuras modificações mas ele nao ensina a deletar =(
Comment: pera minto, fiz o INSERT no meu projeto, vou enviar
Comment: @Math olha la na pergunta o codigo do INSERT
Comment: o codigo é totalmente diferente, nao uso JSON no codigo do insert
Comment: ajuda ai pessoal =(
Answer: Para fazer apresentar uma confirmação para o usuário você poderá utilizar o Dialog, colocando uma requisição ao servidor no botão de positivo.
Para deletar um comentário ( notícia, no seu caso ), acredito que a melhor opção seja criar um novo serviço ( removeComment.php, por exemplo ) que será chamado e passar como parâmetro algum identificador da notícia ( recomendo ser o próprio identificador da tabela do banco de dados )
A partir daí é só atualizar a lista no dispositivo
Comment: no meu caso criei o excluirNoticias.php mas tento implementar a função de excluir la no OnItemClick e sempre da erro. tem como vc baixar o projeto la e me mostrar como se faz? la so mostra pra listar, agora excluir nao =(  semana passada fez 1 mês que estou tentando e nada, me ajuda


How to implement a connected scatter graph in Chart.js

Question: I am trying to implement the chart shown in Chart.js - essentially a scatter graph with two sets of connected data-sets. I have coded a scatter graph with the correct point distribution but struggle to find the way to implement the connected lines. The API shows options.elements.lines, but can this be applied to the Scatter type?
Comment: Am I right in thinking that although Chart-plugins-annotations can be used to draw boxes and polygons - neither have the capacity for this use case?
Answer: You can use the showLine property and make sure your data is in the correct order to make the dots connected:

const options = {
  type: 'scatter',
  data: {
    datasets: [{
      data: [{
        x: 5,
        y: 6
      }, {
        x: 2,
        y: 9
      }, {
        x: 3,
        y: 4
      }, {
        x: 5,
        y: 6
      }],
      showLine: true
    }]
  },
  options: {
    scales: {}
  }
}

const ctx = document.getElementById('chartJSContainer').getContext('2d');
new Chart(ctx, options);

  <canvas id="chartJSContainer" width="600" height="400"></canvas>


How should I create signal for model with placeholder outside in django-cms?

Question: For example, I have model:
class Ad(models.Model):

    firm_title = models.CharField(u'Название фирмы', max_length=255)
    placeholder = PlaceholderField('ad_slot')

    changed_date = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        return self.firm_title

And I want to change auto the value of "changed_date" field when I updated placeholder (added or changed plugin). I need it for generate sitemap to use actual changed_date. Now, when I add or update plugin, the "changed_date" field is not updated auto.
As I understand, I should add post_save signal for Placeholder, but I don't know how.
Please, help me :)
Answer: You shouldn't need a signal for handling that, and the field definition should update the date when you save the object.
I regularly define a created & modified field on tables using these two fields to achieve this same thing;
created = models.DateTimeField(editable=False, auto_now_add=True)
modified = models.DateTimeField(editable=False, auto_now=True)

Comment: Unfortunitally, It doesn't work when added or changed plugins in placeholder, because obj of class Ad is not saved. Placeholder is saved...


Preciso alocar novamente uma estrutura caso eu limpe apenas uma parte dela?

Question: Tenho uma struct que possui nome e horário para marcação de consultas.
              typedef Struct{
               char nome[20];
               char data[20];
              }Dados;

Faço a declaração:
           Dados *dados_cliente[quantidade];

Faço o alocação (sempre que eu  inicio uma nova marcação para não alocar tudo):
         dados_cliente[i]=malloc(sizeof(Dados));

Agora estou trabalhando para o usuário editar a marcação, eu poderia utilizar uma estrutura auxiliar mas preferi fazer dessa maneira:
        // SÓ VOU COLOCAR CASO ELE QUEIRA MUDAR O NOME PARA NÃO FICAR ENORME
        Case 1:
                free(dados_clientes[i]->nome);
                printf("DIGITE O NOVO NOME:\n);
                fflush(stdin);
                scanf("%[^\n]",dados_clientes[i]->nome);

Agora minha pergunta, quando utilizei o free() apenas no nome, eu preciso alocar novamente? Ou devo esquecer isso tudo e utilizar uma estrutura auxiliar mesmo, e/ou sugestões para resolver isso, caso possível.
Comment: Você está equivocado no que o `malloc` e `free` significam. Aconselho a começar a revisar por ai, aprendendo bem o que cada um faz e quais as implicações.
Answer: Tem uma série de problemas nesse código e até na pergunta, mas vou focar no que perguntou.
Este free() não faz o que está imaginando. Você só pode dar free() em algo que você deu malloc(), exatamente a mesma coisa, não pode inventar outra coisa para dar free(). Você poderia apenas não ter colocado na pergunta o malloc() no nome, mas duvido, e se o fez está bem errado.
Se o nome é um dado inline na estrutura não tem o que alocar fora dela, portanto não tem porque dar um malloc() para ele, o que eu acho que você não fez. O dado já está dentro da struct, se alocar espaço para ela, todo o espaço já está alocado, e se precisa liberar o espaço, tem que ser feito para toda struct.
Se deseja ter o nome com tamanho variável e fora da struct então coloque ali um ponteiro apenas e aí aloque o nome sob demanda, mas aí já é outro problema, e não temos informação suficiente para falar sobre isso, na verdade isto já é até especulação.
Comment: É muita coisa e só com essas informações não vai adiantar muita coisa. E eu sempre postei aqui isto: https://i.stack.imgur.com/CQiXB.jpg. Você está aprendendo tudo errado porque acredita que funcionar quer dizer alguma coisa, e especialmente em C quer dizer nada útil. Funcionar é a pior coisa que pode acontecer para você.
Comment: Sim eu utilizei a estrutura, quando eu faço um cadastramento os dados são jogados para a estrutura(não coloquei aqui para não ficar grande a pergunta), após eu cadastrar uma estrutura eu queria apenas caso o usuário queira fazer alterações, se ele quiser mudar o nome eu limpo o nome na estrutura correspondente e faço a alteração.
Comment: Se possível pontuar os erros que você diz ter no código pois não consigo ver, sempre utilizei estruturas dessa forma e funcionou(usava estruturas auxiliares) por isso estou tentando buscar outras formas de fazer isso mais rápido mas acho que não tem jeito.
Comment: acabei me equivocando nas palavras, peço perdão, sei que minha programação está no básico, acabei meu primeiro período de c no fim do ano passado, eu apenas aprendi essa forma para alocamento de estruturas de ponteiros, eu aloco uma estrutura inteira e coloco free nela inteiracaso eu queria excluir ela inteira também, para alterações eu sempre usei estrutura auxiliar, o usuário digita o que ele quer alterar e joga para a auxiliar que passa(substitui) para a estrutura original, mas estava tentando melhorar essa forma para não precisar de auxiliar.
Answer: Em C, quando há um ponteiro, este aponta para o primeiro byte da memória relacionada àquela variável. Quando há um vetor estático, o nome do vetor sem os colchetes com um índice também é tratado como um ponteiro, apontando para o primeiro byte de memória do vetor. Portanto, ao analisar a definição do tipo Dados
typedef struct{
    char nome[20];
    char data[20];
}Dados;

pode-se perceber que, em um ponteiro para Dados, como dados_clientes[i], tanto dados_clientes[i] quanto dados_clientes[i]->nome apontam para o mesmo endereço da memória e, por isso não há falha de segmentação no comando free(dados_clientes[i]->nome), mas toda a estrutura é liberada, pois é a mesma coisa de free(dados_clientes[i]).
Para solucionar o problema, basta eliminar o trecho free(dados_clientes[i]->nome);, pois o membro nome é estático e, portanto, não deve ser liberado dinamicamente. Outra possibilidade seria utilizar ponteiros para caracteres
typedef struct{
    char *nome;
    char *data;
}Dados;

Dessa forma, porém, seria necessário alocar memória dinamicamente para os membros nome e data
Comment: Muito obrigado, você entendeu a minha pergunta e conseguiu responder!! Agradecido.
Answer:     typedef struct
    {
        char nome[20];
        char data[20];
    }   Dados;

    Dados *dados_cliente[quantidade];

O nome e a data em sua estrutura são strings, 20 bytes terminados por nulo. "Limpar" um campo desses pode ser apenas colocar um zero na primeira posição. Não precisa liberar memória e alocar de novo, e nem pode porque do modo como escreveu só pode alocar uma nova nova estrutura em dados_cliente[i], onde todos são Dados*. Se por exemplo nome em dados_cliente[i] era "vermelho" quer dizer que tem um zero depois do 'o'. E se vai trocar por "azul" basta gravar 'a' 'z' 'u' 'l' e '0' a partir de dados_cliente[i][0];
dados_cliente é um vetor de ponteiros, de tamanho fixo. E os ponteiros é que são alvos possíveis de malloc() , free() e realloc(). E o que você pode "liberar" é o que você aloca, no caso os ponteiros, e um a um. Você pode iniciar todos com zero, o tal NULL, e depois liberar todos. free() em um ponteiro NULL é inofensivo, mas imagino que mantenha controle de quantos desses quantidade ponteiros estão em uso.
Se quer alocar um número variável dessas estruturas aí é diferente, e você declara
Dados ** cadastro;

por exemplo, e usa alguma lógica para controlar a alocação, como a que eu descrevi em um exemplo ontem aqui mesmo nesse post onde tem um programa completo, Ou aloca um a um e usa uma lista ligada ou algum tipo de estrutura assim. Se for usar um bloco deles pode iterar como em argv[i] em main(), usando um vetor de ponteiros para estruturas, com o tamanho dado por quantidade, só que variável, como no caso do clássico par argc/argv de main()


Is there any method to get SEO friendly URL from a human readable string in Hippo CMS?

Question: On Hippo Site I am doing a business logic to persist a bean.
I want to convert the following human readable string to SEO friendly URL
Human readable string:
To get - developers @ up to speed with Hippo CMS quickly
SEO friendly URL:
to-get---developers-up-to-speed-with-hippo-cms-quickly
What is the Hippo utility method to do this conversion?
Answer: Internally Hippo does something like:
<code>ISettingsService settingsService = getPluginContext().getService(ISettingsService.SERVICE_ID,
            ISettingsService.class);
StringCodecFactory stringCodecFactory = settingsService.getStringCodecFactory();
return stringCodecFactory.getStringCodec("encoding.node");
</code>
You should be able to use that.
edit: Looking at Ilja Murasko's answer, I just realized I left out an important part. What I described gets you the proper StringCodec instance. You still have to call encode on that with the string. It is in practice the same class that Ilja is instantiating.
Comment: Which class has the implementation of  getPluginContext() method?
Comment: Its in AbstractRenderService, so any plugin has that. From the hst side I think you would have to use HippoServiceRegistry
Answer: import org.hippoecm.repository.api.StringCodecFactory.UriEncoding;
new UriEncoding().encode("To get - developers @ up to speed with Hippo CMS quickly");
the above line returns: "to-get---developers--at--up-to-speed-with-hippo-cms-quickly"
Comment: What I posted was actually showing how to get that class. I just realised I left out the part where you actually use it. Sow without your answer my answer is incomplete. I think it is better to get the service rather than instantiate the class but your way works.
Comment: I agree, it is better to access UriEncoding via service you suggested, as direct use of UriEncoding is discouraged


Commerce - delete products that do not reference a node

Question: I want to delete commerce products that are not referencing nodes. Either a delete process or report that shows which commerce products are not referencing nodes.
Answer: Try the Views Bulk Operations module.
You can create a View that lists Commerce Products
Use the Table format, showing fields.
Add a filter for the reference field and for the operator choose "Is empty (NULL)".
Add a field to the View, look for "Bulk operations: Commerce Product"
When the field settings come up, in the "Selected bulk operations" field set, check the "Delete Item" checkbox.
Arrange this field to be the first one in the list.
Save the view.
Now you will see all products that do not have a value for your reference field. The Bulk Operations: Commerce Product field will make a checkbox display for each row, and you can select one or more products to delete.
You can also check the text box in the table header, and that will "select all" products listed on the page. If you have more than one page worth of products, a button will appear that will let you select all products in the view.
Choose the "Delete Item" option from the drop down above the view and hit the button, and it will start the delete process, after a confirmation.


Convergence in probability of an estimator

Question: I'm asked to verify if the following estimator of $\mu$:
$$\hat{\mu}=\frac{1}{n+1}\sum_{t=1}^{n}y_t$$
converges in probability by computing its $p \lim$ (note that $E[y_t]=\mu$). I've made the following computations:
\begin{align}
p \lim(\hat{\mu})&=p\lim \left ( \frac{1}{n+1}\sum_{t=1}^{n}y_t \right)
\\ &=p\lim \left ( \frac{1}{n+1} \frac{n}{n}\sum_{t=1}^{n}y_t \right )
\\ &=p\lim \left ( \frac{n}{n+1} \frac{1}{n}\sum_{t=1}^{n}y_t \right )
\\ &=\frac{n}{n+1}\ p\lim \left ( \frac{1}{n}\sum_{t=1}^{n}y_t \right )
\\ &=\frac{n}{n+1}\ p\lim \left ( E[y_t] \right )
\\ &=\frac{n}{n+1}\ \mu
\end{align}
Can I say that for $n \to +\infty$ the fraction $n/(n+1)$ is equal to 1 and thus $p \lim(\hat{\mu})=\mu$ or must I leave the ratio in front of $\mu$?
Comment: In the first three lines of your calculations, "$n$" was a *bound variable*: you could have replaced it by any variable name you please without changing the meaning.  At the fourth line, where $n/(n+1)$ appears outside the limit, "$n$" suddenly appears unbound--and thereby is *meaningless*.  That line is invalid.
Answer: You can say that the limit of the fraction $\frac{n}{n+1}$ is $1$ and use the Law of Large Numbers for the sample mean, yes. It's a mistake however to take the limits sequentially. They have to be taken at the same time. Here for instance, you can use the fact that if $A_n \xrightarrow{P} A$ and $B_n \xrightarrow{P} B$ then $A_n B_n \xrightarrow{P} AB$. And you are done.


Can I configure Flyway to run without requiring ownership of pg_stat_statements_allusers? Postgres

Question: PostgreSQL
12.6

Flyway Maven Dependency
org.flywaydb
flyway-core
6.5.7

I'm using maven to kick off a flyway:clean run, specified to a remote database. I have DDL permissions, however flyway is giving the following message

ERROR: must be owner of view pg_stat_statements_allusers

Unfortunately, I am unable to get ownership over the Database view.
My question is, is there a way to configure flyway to ignore the pg_stat_statements_allusers during my run flyway:clean
Comment: Well, `pg_stat_statements` should not be dropped to begin with. It's part of the extension [pg_stat_statements](https://www.postgresql.org/docs/current/pgstatstatements.html) and typically owned by the superuser. It seems Flyway is trying to drop _everything_ rather than everything that the _user owns_.  (for that `drop owned by current user;` is a more efficient and easier approach anyway).
Answer: Thanks to @a_horse_with_no_name which made a good point in regards to the super user. I looked into the superuser privileges which helped me solve the problem. But I was unable to figure out how to "drop owned by current user".
It came down to an internal company policy that requires using the pg_stat_statements specifically in the public schema across all DB's, which was owned only by the superuser. Therefore by using a different schema than the public schema, I was able to avoid the locked pg_stat_statements unlike with the public schema and run flyway clean.


How do I access profile fields?

Question: I have installed the Profile module. I am trying to programmatically load profiles. The code works, but I am not able to access any field value.
class IdCardController extends ControllerBase {

  /**
   * Init id card download.
   */
  public function initDownload(User $user) {

    $user_profile = $this->entityTypeManager()->getStorage('profile')->loadByProperties([
      'uid' => $user->id(),
      'type' => 'personal_information',
    ]);
    kint($user_profile);
  }
}

Using kint(), I can see all methods are protected.
Answer: You could examine the Profile Entity class here profile / src / Entity / ProfileInterface.php, showing you a list of public methods including its getter methods, eg.:
$profile->getCreatedTime();

Additionally and similar as with Entities in general, you can also call data via either $entity->get('field_name')->getValue() or $entity->field_name->getValue().
See more on that here: Get information from an entity/ Entity methods
So if you have a Profile Entity with a field machine name address, then you could get its data like:
$profile = \Drupal\profile\Entity\Profile::load(123);
$profile->get('address')->getValue();
$profile->address->getValue();

Comment: OR `$entity->field_name->value;`

How do I access profile fields?


How to use permutation test to find significant outcomes from many fisher exact tests

Question: I am reading from this where they mention running 500K fisher exact test to find significant associations between different DNA changes and a phenotype.
They also mention that Bonferroni, Benjamini, and FDR can be used to set a significance threshold, which I think I understand.
What I don't get is that they say permutation can also be used to set a significance threshold. How can you use permutation on fisher exact test output to set a significance threshold?
Answer: It's difficult to speculate they mean without giving a citation or reference, but they may be referring to the Westfall-Young method which is a permutation based method.  I did just answer a question at CV where I suggested using permutation testing (empirical p-value testing), which is here.


Is training a deep neural network still referred as training using back propagation?

Question: I wa currently reading up on standard neural network and become a bit confused in the terms used relating training deep neural network versus a normal neural network.  Are they trained similarly or differently if so? what difference?
Comment: Depends what you mean by similarly. Deep networks are still trained using backpropagation, but adding more and more layers requires sophisticated ways to combat the vanishing/exploding gradient - careful weights initialization, batch normalization, identity connections, skip identity connections, complex optimizers etc.
Answer: In general, a deep neural network is a neural network with many layers. In practice, there are some qualitative differences, which I'll go into in a sec, but they're still trained using back propagation, in general.  As far as qualitative differences, deep networks tend to use layers which facilitate training:

convolutional layers have fewer parameters to learn, so can be stacked up relatively deep
skip-connections and residual networks facilitate the flow of gradients backwards through the network, so enable back-propagation through relatively large numbers of layers, up to 50-100, instead of maybe less than 10ish
dropout facilitates finding reasonably good minimum, not getting stuck in local minimum too much
batch normalization facilitates using high learning rates without the gradients becoming too extreme


Convert swf file to mp4 using ffmpeg

Question: I need to convert swf files to mp4 files using ffmpeg commands (command line). There is text in swf files which must be converted too.
But when I convert files using the following ffmpeg command text in swf file is not converted to mp4 / no text is visible in mp4 file:
ffmpeg -i file.swf video.mp4

Is there any other command to convert text?
Comment: After hours of searching, I found this app that does convert .swf to mp4 and html5 with sound and everything in sync and many other formats also in batch: `iDealshare_VideoGo_22.214.171.12438`
Answer: I had the same problem. My walk around was to use the raw-output of gnash.
The following bash script summarizes the single steps.
#!/bin/bash

SWFFILE=$1
MP4FILE=${SWFFILE%.*}.mp4
TMPFILE=/tmp/$(cat /dev/urandom | tr -dc 'a-zA-Z' | fold -w 32 | head -n 1).bin
TMPMP4=/tmp/$(cat /dev/urandom | tr -dc 'a-zA-Z' | fold -w 32 | head -n 1).mp4

# create raw-dump
GNASHCMD="dump-gnash -1 -r 1 -D $TMPFILE $SWFFILE"
OUTPUT="$(exec $GNASHCMD)"

# extract parameters
WIDTH="$(echo $OUTPUT | grep -o 'WIDTH=[^, }]*' | sed 's/^.*=//')"
HEIGHT="$(echo $OUTPUT | grep -o 'HEIGHT=[^, }]*' | sed 's/^.*=//')"
FPS="$(echo $OUTPUT | grep -o 'FPS_ACTUAL=[^, }]*' | sed 's/^.*=//')"

# create raw, uncompressed mp4 file
mplayer $TMPFILE -vo yuv4mpeg:file=$TMPMP4 -demuxer rawvideo -rawvideo fps=$FPS:w=$WIDTH:h=$HEIGHT:format=bgra

# create compressed mp4 (ffmpeg should work as well)
avconv -i $TMPMP4 $MP4FILE

# clean up
rm -rf $TMPFILE
rm -rf $TMPMP4

Comment: how to include sound into result MP4 in Your example?
Answer: ffmeg doesn't support shapes and sprites in swf. It can only convert video (Sorenson Spark, VP6), audio (pcm, adpcm, mp3) and images (jpeg) contained in a swf file. You'll have to look for another tool.
Comment: Thanks for your response! Do you know of any other tool which does the conversion?
Comment: Unfortunately I don't know any 3rd party tool. Adobe Flash had the option to export an animation as a video if I remember correctly.

Convert swf file to mp4 using ffmpeg


Can someone tell me how to read this equation in words?

Question: $$
P\left[\frac{\text{MTBF}\cdot2r}{\chi^2_{1-\alpha/2,2(r+1)}}
\le
\text{True MTBF}
\le
\frac{\text{MTBF}\cdot2r}{\chi^2_{\alpha/2,2r}}
\right]
\ge 1-\alpha
$$
Can someone please help me to understand this formula. I know/understand the variables but not its mathematical significance.
Comment: Could you give some context by adding what $MTBF$ and $r$ are?
Comment: Did you read this http://www.itl.nist.gov/div898/handbook/apr/section4/apr451.htm ?
Comment: MTBF - Mean Time Before Failure, r - number of failures
Answer: What you have is a probabilistic statement about the parameter sought (True MTBF) that says that the probability that the true parameter is between the two fractions is greater than or equal to 1-$\alpha$. This probably sound like a confidence interval, because it is.
For a given $r$ and a given $\alpha$, the stuff in brackets defines a set (a confidence set). Say that $\alpha=0.05$, as is typical, and maybe $r=50$: then you can find the corresponding points on the chi-square distributions
$$\chi^2_{0.975, 102} = 131.84 \ \ \ \text{and} \ \ \  \chi^2_{0.025, 100}=74.22$$
Notice that the first $\chi^2$ critical point is larger and the second is smaller; as they're in the denominators and the numerators are constant, this ensures that the confidence set is of the form $[a,b]$ with $a<b$ which is intuitive.
Now, let's say that the observed MTBF is $\hat{MTBF} = 5$ just for the sake of calculation. Then,
$$\text{Pr}\left[ \frac{5*2*50}{131.84}\leq MTBF \leq \frac{5*2*50}{74.22} \right] \geq 0.95 \iff $$ $$\text{Pr}(3.79\leq MTBF \leq 6.74) \geq 0.95 \iff$$ $$\text{95% CI is}: (3.79,6.74)$$
Comment: Given `r` is the number of trials, that's why your CI is so high.  I edited it to replace it with a reasonable number (50) and correct values for the various formulas, if that's acceptable.
Comment: @Joe thank you. Reasonable values add clarity to the example.
Answer: This is how to read this equation in words, somewhat simplified:

"The probability of the logical statement in the square brackets being true is greater than or equal to some high probability" (1.0 is the maximum probability, so this is only smaller than 1.0 by $\alpha$).
"In the square bracket, there is a logical statement that is either true or false: the True Mean Time Between Failures (MTBF) is somewhere between a lower value and a upper value."
"The formulas for 'lower value' and 'upper value' are confidence intervals:  the estimated MTBF (from your sample data) TIMES two times the number of failures $r$, DIVIDED by the Chi-squared value for the lower and upper limits, respectively, based on your confidence level $\alpha$ and $r$."

See @ScouserInTrousers answer for explanation of Chi-squared value, plus an example calculation.


Display all rows from table but first where id=x - PHP and MySQL

Question: I am new to PHP and MySQL.
I wanted to know that if its possible to show all the rows from the table but first where id='some value'. ie.
SELECT * FROM users ORDER BY id='some value'

Thanks for answering.
Comment: Yes, it's possible to do that. But I guess that's not the answer you were looking for. Please read [ask].
Comment: I don't understand.
Answer: You can make a set of queries, for example:
SELECT * FROM users WHERE id= 5 UNION SELECT * FROM users WHERE id != 5;

Or if you need something like this example:
SELECT * FROM users WHERE id >= 5;

Comment: Marcos David Huaranga Ortega, Thanks you very much!


python how to rename a list

Question: So I am currently working on a battle sim for a school project in which two tanks will battle each other.
every tank has to have the possibility of having multiple gun objects.
so I made a method to let the user create guns but I want to make sure that every time I call this method the generated list will get the name saved in the gunListName variable. is there a way to do this cause I can't find out how to do this.
Here is the method I'm talking about:
 def createguns(self,):
    gunList = []
    gunListName = input("please input the name of the tank that will use these guns: ")
    gunNumber = input("how many guns do you wish to give your tank? ")
    for x in range(int(gunNumber)):
        gunName = input("please input the name of your gun: ")
        gunRange = input("please input the range of your gun in Meters(example: 1000 CM would be 1KM range): ")
        gunMinRange = input("please input the minimum range of your gun: ")
        gunDamage = input("please input the damage value of your tank: ")
        gunReload = input("please input the reload time of your gun")
        gunName = Gun(gunName,gunRange,gunMinRange,gunDamage,gunReload)
        gunList.append(gunName)
    return gunList

Comment: Declare your variable outside your method and you will be sorted
Comment: Your `createguns` functions should be responsible for creating guns only, it's the responsibility of the code that calls it to do whatever it wants with it.
Answer: DO like this
def createguns(self):
    gunList = []
    gunListName = input("please input the name of the tank that will use these guns: ")
    gunNumber = input("how many guns do you wish to give your tank? ")
    for x in range(int(gunNumber)):
        gunName = input("please input the name of your gun: ")
        gunRange = input("please input the range of your gun in Meters(example: 1000 CM would be 1KM range): ")
        gunMinRange = input("please input the minimum range of your gun: ")
        gunDamage = input("please input the damage value of your tank: ")
        gunReload = input("please input the reload time of your gun")
        gunName = Gun(gunName,gunRange,gunMinRange,gunDamage,gunReload)
        gunList.append(gunName)
    return gunList, gunListName

gunList,name = createguns()
print(name)

python how to rename a list


CentOS iptables NAT, client cannot connect to WAN from LAN

Question: I have the following configuration:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:e0:4c:68:31:a8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.1/24 brd 192.168.3.255 scope global enp2s0
       valid_lft forever preferred_lft forever
    inet6 fe80::2e0:4cff:fe68:31a8/64 scope link
       valid_lft forever preferred_lft forever
3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:e0:4c:68:31:a9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.130/24 brd 192.168.1.255 scope global noprefixroute dynamic enp3s0
       valid_lft 85920sec preferred_lft 85920sec
    inet6 fe80::b8df:4b68:a402:677c/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

enp3s0 is WAN
enp2s0 is a local network connected to a client pc.

I want to make it so the client can access the internet. At the moment the client connected to enp2s0 can ping the server and visa versa.
I have run the command
echo 1 > /proc/sys/net/ipv4/ip_forward

and ensured it has been set and have been following the instructions here: https://www.revsys.com/writings/quicktips/nat.html
sudo iptables -t nat -A POSTROUTING -o enp3s0 -j MASQUERADE
sudo iptables -A FORWARD -i enp3s0 -o enp2s0 -m state  --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i enp2s0 -o enp3s0 -j ACCEPT

I have reconnected the client (not sure this is necessary) and tried to connect beyond the server:
$ ping google.com
ping: google.com: Temporary failure in name resolution

$ ping 184.108.40.206
connect: Network is unreachable

Other tutorials suggest only running the first line sudo iptables -t nat -A POSTROUTING -o enp3s0 -j MASQUERADE. I tried connecting before running the second two steps and it was no different.
Is there something else I need to do to create the NAT so that the client can access the internet?
edit: The default route is also correct:
$ ip route
default via 192.168.1.1 dev enp3s0 proto dhcp metric 100
169.254.0.0/16 dev enp2s0 scope link metric 1002
192.168.1.0/24 dev enp3s0 proto kernel scope link src 192.168.1.130 metric 100
192.168.3.0/24 dev enp2s0 proto kernel scope link src 192.168.3.1

The output of iptables -L is:
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
INPUT_direct  all  --  anywhere             anywhere
INPUT_ZONES_SOURCE  all  --  anywhere             anywhere
INPUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
FORWARD_direct  all  --  anywhere             anywhere
FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_IN_ZONES  all  --  anywhere             anywhere
FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_OUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
OUTPUT_direct  all  --  anywhere             anywhere

Chain FORWARD_IN_ZONES (1 references)
target     prot opt source               destination
FWDI_public  all  --  anywhere             anywhere            [goto]
FWDI_public  all  --  anywhere             anywhere            [goto]
FWDI_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_IN_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_OUT_ZONES (1 references)
target     prot opt source               destination
FWDO_public  all  --  anywhere             anywhere            [goto]
FWDO_public  all  --  anywhere             anywhere            [goto]
FWDO_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_OUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_direct (1 references)
target     prot opt source               destination

Chain FWDI_public (3 references)
target     prot opt source               destination
FWDI_public_log  all  --  anywhere             anywhere
FWDI_public_deny  all  --  anywhere             anywhere
FWDI_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain FWDI_public_allow (1 references)
target     prot opt source               destination

Chain FWDI_public_deny (1 references)
target     prot opt source               destination

Chain FWDI_public_log (1 references)
target     prot opt source               destination

Chain FWDO_public (3 references)
target     prot opt source               destination
FWDO_public_log  all  --  anywhere             anywhere
FWDO_public_deny  all  --  anywhere             anywhere
FWDO_public_allow  all  --  anywhere             anywhere

Chain FWDO_public_allow (1 references)
target     prot opt source               destination

Chain FWDO_public_deny (1 references)
target     prot opt source               destination

Chain FWDO_public_log (1 references)
target     prot opt source               destination

Chain INPUT_ZONES (1 references)
target     prot opt source               destination
IN_public  all  --  anywhere             anywhere            [goto]
IN_public  all  --  anywhere             anywhere            [goto]
IN_public  all  --  anywhere             anywhere            [goto]

Chain INPUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain INPUT_direct (1 references)
target     prot opt source               destination

Chain IN_public (3 references)
target     prot opt source               destination
IN_public_log  all  --  anywhere             anywhere
IN_public_deny  all  --  anywhere             anywhere
IN_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain IN_public_allow (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh ctstate NEW

Chain IN_public_deny (1 references)
target     prot opt source               destination

Chain IN_public_log (1 references)
target     prot opt source               destination

Chain OUTPUT_direct (1 references)
target     prot opt source               destination

Comment: Can you please add the routing table of the client, `ip route`? The above looks like the routing table of the gateway.
Answer: You need to ignore and undo the changes you made based on obsolete tutorials.
You're using CentOS 7 with firewalld, so all you need to do is to tell firewalld to enable masquerading on the zone corresponding to your external interface.
firewall-cmd --zone=public --add-rich-rule='rule family=ipv4 masquerade' [--permanent]

(See why I do not recommend using --add-masquerade)
You also need to assign the internal interface to a different zone other than public. Edit /etc/sysconfig/network-scripts/ifcfg-enp2s0 and set ZONE=internal or some other zone.
Comment: Thanks, I've tried that: Added ZONE=internal, ran firewall-cmd as above. I tried reloading network.service, flushing iptables and it didn't have any effect. I rebooted to clear iptables and force everything to reload, re-ran firewall-cmd as above, ran firewall-cmd --reload and still have the same issue.  At least I know to look for a more up to date tutorial.
Comment: It turned out that I also needed firewall-cmd --permanent --direct --passthrough ipv4 -I FORWARD -i enp2s0 -j ACCEPT


Only the current page can be accessed if it jumps from a specific page

Question: I have two pages. One is /a and the other is /b.
There is the a tag in the A page. It is B page.
My requirement is that users cannot directly access page B. they must click the a link from page A to access page B. how should I write my routing guard.
Comment: You would probably have to use [CanActivate guards](https://angular.io/api/router/CanActivate) and check if the previous route was `/a`
Comment: @MrCodingB Yes, but I can't get the information of the previous route
Comment: Well if there's no way to access the previous route with the `RouterStateSnapshot` and the `ActivatedRouteSnapshot`, you could subscribe to the activated routes or the router events in the guard and save the last route that way
Answer: I think you could setup a service that would listen to NavigationEnd events.  By doing this you could store previously activated url.
Then in the guard you'd just fetch previous url and check if it's /a or not.
Take a look at this example implementation: https://stackoverflow.com/a/72112335/9932919


How do you discover and Execute commands in a .Net vNext package

Question: So curious about discovering and executing dnx commands programmatically. Currently I have a scheduling program that will scan a folder for Jobs using MEF and then create a new AppDomain and run any jobs found on some schedule and destroy the appdomain when done.
I want to convert all of the jobs to dnx console apps and find a way to scan a folder for all packages and their corresponding dnx commands so that they can be executed from the scheduling application as well as from the command line in a one off fashion.
It seems that there must be some library that is already doing this as Visual Studio can discover your commands and call them to debug them
Comment: have you tried a google search for the following `C# dnx console apps` http://docs.asp.net/en/latest/dnx/console.html
Answer: DNX commands are just the name(s) of a Console Application(s) referenced by a given project.json.  project.json  then also lets you change the name that you use to refer to the referenced Console Application.  So the best way would be just to scan directories for project.json and grab the "commands" section.
If you are then interested in determining what arguments you can pass to the commands, well then that would be specific to how you have implemented those arguments/options inside your Console Application.
Answer: There are two options:

If you have a project.json file you can simply parse it and look for the commands property.
If you have a nupkg then you can look for App folder inside it. That will tell you which command are globally installable. However, there are a few commands that are ignored at pack time and they will not show up there. Take a look at the code for dnu commands: https://github.com/aspnet/dnx/tree/dev/src/Microsoft.Dnx.Tooling/Install

The VS code that does that is not public but it's just option #1 (looking at the project.json file).
Comment: Seems like parsing the project.json files is the better option.  Is there a better alternative to using System.Diagnostics.Process to execute one of the commands?
Comment: Nope. That's the only way.


strlen function incompatible w/loop, loop variable incompatible

Question: Ok I've been looking everywhere and I can't find the proper way to do this. I simply want to take in a string, put that string in the array and output the contents. However, I want to do it dependent on the size of the string which the user enters in. And I was getting weird errors such as incompatibility and I would like to know why please, thank you.
#include <iostream>
#include <array>
#include <string>
using namespace std;

int main()
{
    int x = 4000;
    string y;

    cout << "Enter value";
    getline(cin, y);
    array<char, strlen(y)>state;

    for(int i=0; i<strlen(y); ++i)
        cout << state[i] << ' ';

    system("PAUSE");
    return 0;
}

Comment: tip of the day: your `y` is only a char
Comment: Please post the errors too. It not only helps people trying to help  (who can figure out even without it) you but is also good for reference.
Comment: string y;

my bad,and thanks VusP

wow..it all of a sudden wants to work now, well the thing to learn is that char can't hold all that data, i guess :/

thank you
Comment: What is the *point* of the `state` variable? Can't you just use `y`?
Comment: x variable is redundant.
Answer: std::array needs a compile-time size, so cannot be instantiated with strlen. Furthermore, strlen does not work with std::string, it expects a pointer to char, pointing to the beginning of  null terminated string.
You could use an std::vector<char> instead:
std::string y;
std::cout << "Enter value";
std::getline(std::cin, y);

std::vector<char> state(y.begin(), y.end());

for(int i = 0; i < state.size(); ++i)
    std::cout << state[i] << ' ';

On the other hand, why not just use the string y directly? Do you really need the "array"?
for(int i = 0; i < y.size(); ++i)
    std::cout << y[i] << ' ';

Answer: i hope it will work..
#include "stdafx.h"
#include <iostream>
#include <array>
#include <string>

using namespace std;

int main()

{
int x = 4000;

  string y;

  cout << "Enter value";

  getline(cin, y );
  char *b = new char[y.length()];
  int j=y.length();

//array< char, strlen(y)>state;

for( int i = 0; i<j; ++ i )
{//whatever uu want
}
  //cout << state[i] << ' ' ;

system( "PAUSE" );
 return 0;
   }

Comment: The question is about C++, not C#.
Answer: std::array is a wrapper template class around a C static array. This means that it's dimensions must be known at build time (not runtime).
Here's a short version of a working code that's also a little faster since string::length() is called once.
#include <iostream>
#include <string>

using namespace std;

int main()
{
    string y;

    cout << "Enter value";
    getline(cin, y );

    for( size_t i = 0, yLen = y.length(); i < yLen; ++i )
        cout << y[i] << ' ';

    system( "PAUSE" );
    return 0;
}

If you like pointer tricks and take advantage that string's buffer is contiguous in memory (by the C++ standard) your code can look like:
int main()
{
    string y;

    cout << "Enter value";
    getline(cin, y );

    for( const char* p = &y[0]; *p; ++p )
        cout << *p << ' ';

    system( "PAUSE" );
    return 0;
}

Comment: Hmm, for the condition *p, how does that work? Shouldn't it be the last element in the loop, not the first? Like...I get the starting point, but the endpoint doesn't make sense to me.

-Thanks
Comment: sorry forgot to address:

@egur

btw how does the loop do a bounds check, like why doesn't that result in an infinite loop, because *p could go on forever, technically?
Comment: @Shinji-san: `*p` is either zero (end of string - false) else true. Every string end with zero. This will work when you are sure that this is a legal string. I'm sure it's legal since I'm accessing y's internal buffer and std::string store a standard NULL terminated C string.


Compare event count in two different time periods

Question: We have over 10 years of data on a number of hospitals that we regularly transport patients to.  This past February, a new hospital (1830) opened that has substantially altered our transport patterns, especially taking away from the previous primary destination (1247).
<code>Destination   Dec     Jan     Feb     Mar     Apr
1247          117     140     131      65      77
1017           75      69      54      45      31
1818           56      58      43      46      39
1830            -       -      30     125      86
</code>
I want to evaluate the overall impact (improved in-service times, reduced mileage/vehicle costs, etc...) of the new facility.  What is the appropriate manner to evaluate the new facility?  Should I compare equal periods prior to and after it opened?  Or, can I evaluate the prior year and the period since it opened?  Any help is appreciated.
EDIT:  I'm not sure that I have too much seasonal/cyclical variance in my long-term data.  Here's a chart that has the previous 5 year history (each point is a 2-week period).  I've highlighted total patients transported (with 26 period moving average), and the destination hospital in question.
Comment: I should have mentioned that I use 28-day "months" for reporting to avoid issues related to number of days in the reporting period.
Answer: As I understand you have some strong seasonal prevalence in your data. In this case I'd rather do the following things:

It is always better to compare the period you want to analyse (since February till now) to the same period last year to get rid of seasonal effect.
It is better to compare not the absolute values, but the percents (just divide everything by the total amount). This will get rid of year-to-year difference (for example average increase of costs).

Hope it will help.
Comment: Thanks for the feedback-I've added some longer term numbers to explore the seasonality angle.  I was concerned about using a prior year's period because of the impact of long-term trends, but you covered that in #2.
Comment: This data is surprisingly stable! I was almost sure that there should be seasonal effect. In this case I have seen reports that providing both numbers: "year-to-year period" comparison, and "period-to-the previous one". If both numbers (relative numbers) show the improvement - you have nothing to explain any more (you cover both short- and long-term improvement).


Extend convolution identity to $L^2$

Question: I am trying to prove the following statement.

if $f\in L^2$ and $g\in L^1$, $\mathcal{F}$ denotes fourier
  transformation,and$*$ denotes convolution, then
  $$\mathcal{F}(f*g)=\mathcal{F}(f)\mathcal{F}(g)$$

It should be extended from the case $f,g\in L^1$.
I followed the procedure starting from the lower part of page 72,
but I couldn't understand the last estimation
$$\int\int|f(x-y)-f_n(x-y)||g(y)|\,dx\,dy\\\le||f-f_n||_2||g||_1$$
Any hints? Thanks!
Answer: I think there might be a typo in how that inequality is written down. To show the $L_2$ limit of $\mathcal{F}(f_n∗g)$ is $\mathcal{F}(f∗g)$:
$$ ||f_n*g - f*g||_2 = ||(f_n-f)*g||_2 \leq ||f_n-f||_2 ||g||_1\to 0 $$
I guess this is what the written solution was going for.
Context:
We want to show $\mathcal{F}(f*g)=\mathcal{F}(f)\mathcal{F}(g)$
You know there's a sequence $f_n\in L_1\cap L_2$ tending to $f$ in the $L_2$ norm. You know that for each $n$,
$$\mathcal{F}(f_n*g)=\mathcal{F}(f_n)\mathcal{F}(g)$$
pointwise a.e. By (b) we know that the ($L_2$ and hence) pointwise a.e. limit of $\mathcal{F}(f_n*g)$ is $\mathcal{F}(f*g)$.
Therefore it suffices to prove that the pointwise a.e. limit of $\mathcal{F}(f_n)\mathcal{F}(g)$ is $\mathcal{F}(f)\mathcal{F}(g)$. But $\mathcal{F}(f_n)$ tends to $\mathcal{F}(f)$ in $L_2$ (Plancherel's theorem) and hence pointwise a.e.
Comment: Thanks for the reply. My main difficulty is the estimation I stated above. And why $\mathcal{F}(g)\in L_\infty$?
Comment: $|\mathcal{F}(g)(\zeta)|\leq ||g||_1$, but actually I don't think you even need that - I've modified my answer to give a full argument.

Extend convolution identity to $L^2$


Wait for all the threads to finish their job and then measure the time elapsed

Question: I am using Executor service in one of my Multithreaded code. I am trying to see whether all of my threads are finished doing their job then afterwards I need to do certain task.
Currently I need to measure the time elapsed, so I can only measure the time elapsed only after all the threads has finished executing there job. So I have this code currently? And I am measuring time elapsed in the finally block.
ExecutorService service = Executors.newFixedThreadPool(noOfThreads);

long startTime = System.nanoTime();
try {
    for (int i = 0; i<noOfThreads; i++) {
        service.submit(new Task());
    }
    service.shutdown();
    service.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);

    //Do I need this while block here?
    while (!service.isTerminated()) {

    }
} catch (InterruptedException e) {
    //Log exception here
} finally {
    long estimatedTime = System.nanoTime() - startTime;
    logTimingInfo(estimatedTime, noOfTasks, noOfThreads);
}

I am not sure whether I need while loop here or not? Is the way I am currently doing it right or not?
Updated Code:-
So below code should work fine. Right?
ExecutorService service = Executors.newFixedThreadPool(noOfThreads);

long startTime = System.nanoTime();
try {
    for (int i = 0; i<noOfThreads; i++) {
        service.submit(new Task());
    }
    service.shutdown();
    service.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
} catch (InterruptedException e) {
    //Log exception here
} finally {
    long estimatedTime = System.nanoTime() - startTime;
}

Answer: You are not required to re-invent the wheel. You can use http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/CompletionService.html
This works pretty fine. This should solve your problem statement too.
Answer: Q: Do you need the while?  A: No.
The previous awaitTermination call won't return until either the service has terminated, or (2^63 - 1) seconds have elapsed.  (That is a very long time.)

UPDATE - The updated version looks OK to me.
Comment: You don't really need a finally either.  You are only going to get an `InterruptedException` if some other thread interrupts the current one.  In that case, I hardly think you care what the timing was ... 'cos it won't be a meaningful measure of anything.
Comment: Thanks Stephen for the suggestion. So that means I won't be needing while loop right? And I can measure the time elapsed in the finally block then?


Different behavior between netbeans and shell windows

Question: I wrote a small program in Java using Netbeans 8.02 (JDK8; java version 1.8.0_45).
It reads an XML file (in UTF-8 encoding), finds and replaces the character "&", parses the file and signs it.
Executing this program from Netbeans it's all right.
Executing the same program from windows XP shell i meet errors about the encoding of the file to be signed.
Executing the same program from windows 7 shell every thing is fine.
somebody knows what cause this different behavior?
Answer: to minimize the distance between this different behaviors, and thus avoiding errors of BOM and charset, I solved in this way:
<code>f = new File(PatInputFile);
String sif = readfile(PatInputFile,StandardCharsets.UTF_8);
byte[] contentInBytes = sif.replaceAll("&", "&").getBytes("UTF-8");
FileOutputStream fop =new FileOutputStream(f);
fop.write(contentInBytes);
fop.flush();
fop.close();
</code>
where
<code> private static String readfile(String path, Charset encoding)  throws IOException
{
  byte[] encoded = Files.readAllBytes(Paths.get(path));
  return new String(encoded, encoding);
}
</code>
and so on parsing and signing the file "f" .


Deixar item do listview preenchendo a tela toda

Question: Tem como deixar um item de um ListView tomando a tela toda do celular?
Comment: Por "tela toda" você quer que suma a Action bar e Botton bar?
Comment: não , quero e tipo um fill_parent
Comment: Uma pergunta: para que um `ListView` para isso?
Comment: To querendo ter uma base para uma coisa que to pensando, mais se não tiver como não tem problema
Answer: Deixe o layout_width e layout_height setados com math_parent.
No xml da activity apague as seguintes linhas do código:

android:paddingBottom="16dp"
android:paddingLeft="16dp"
android:paddingRight="16dp"
android:paddingTop="16dp"

OU substitua o 16dp de cada linha por 0dp, assim:

android:paddingBottom="0dp"
android:paddingLeft="0dp"
android:paddingRight="0dp"
android:paddingTop="0dp"

Em styles.xml (app>res>values>styles.xml) você modifica a linha


Create 'n' circles along a mouse created vector line paper.js

Question: I would like to create 'n' number of circles along a mouse created vector line when the mouse is clicked and dragged.
This would result in a row of circles which would be a certain distance apart, depending on the number of circles in the line. The first and last circles would be at the point where the mouse was first clicked and then released.
Example: If the line is 1000px long, and 3 circles are required there would be a circle at 0px 500px and 1000px. If the line is 1000px long and 5 circles are required, there would be a point at 0px, 250px, 500px, 750px and 1000px.
I had an attempt at paper.js with the following results, but this is not quite what I need. In this example, the circles follow a free path and are spaced by a set number of pixels.
Click here to see
As an extra, it would be nice to see the path of the mouse line before the mouse is released.
Comment: [`getPointAt(offset)`](http://paperjs.org/reference/path/#getpointat-offset) is your friend
Comment: Thanks. That looks much simpler that my solution :) http://sketch.paperjs.org/#S/VVJNb8IwDP0rUS+kKpSh3Sg7MWmnaUjsNqYpNB5ELTZKUnpA/PfF6QeslzrvOc/Pdq4JqhMky2RbgS+PyTQpSfO5Ngg/nqgWLwKhFZ8hlGmxw5HICd+pcfBq1SEk/TZYekMoJFwAfSquO7woKw6WmnMv8sZxVOk455X1DpQP/KLoMEA9IE89ZKl14Twws/u94kGHM7hyrqnFDRn0d8WROz/gNWDAu7szzhrVgA0zPReSi6c9YwI8mCoba4PihtwgEttqj6YGIY1YRdvZIg5ChG+H8zn/2bfrKx3VGfrZbDnO18aWNUgGYgvyXiadimc2Ei/lzluqYE012SAw2deqrCYjq5VXOW928MZFMxP4uI5cab0ORrWM6az6r52HwyyOIySYLCu6Nm6DioUTXeAD+QXErd7C+9lbUFUcs0uWX9+3Pw==
Comment: You are still not using `getPointAtOffset` though
Answer: As @NicholasKyriakides wrote in his comment, this can be best implemented using a Path with getPointAt(offset).
I have implemented this in your code example and made it more generic, so you can use any path, not only a straight line.
 I have also replaced your circle items with instances of SymbolItem, which is more efficient, because they share a lot of data.
Here is the Sketch


Eclipse is saying "Missing builder" and there's no way to configure it

Question: I'm trying to configure my projects to build with jdk 1.6. The problem is that when I removed 1.7 from my computer, my projects show up as having "Missing builder (org.eclipse.vjet.eclipse.core.builder). And trying to build them shows "The specified JRE does not exist"
There's no option to configure which JRE to use in the builders, when I select the one that says missing, "Edit" is greyed out.
How can I fix this so my project can build?
Stack Overflow won't let me post images directly, so here are the links to them, as hosted on imgur.com:
Answer: The Missing Builder comes from not having VJet installed.  You can disable or remove the reference to it from the shown Property Page, or just install VJet. The other you might resolve by going to the Installed JREs preference page and making sure all of the values are correct, and then fixing any existing launch configurations (Run->Run Configurations...) that may be pointing to JREs that aren't around any more (such as by changing them to use an Execution Environment instead of a specific JRE by name).
Comment: I only had to do the second part of what you said, and then it started working. I guess VJet wasn't needed to build. Thanks


Hearing a sequence of tones but not a melody

Question: Normal cognitive sciences (neurosciences and psychology) try to explain what goes on when a person

hearing a sequence of beats perceives a rhythm
hearing a sequence of tones perceives a melody
hearing a simultaneous set of tones perceives a chord.

From the Wikipedia article on Music Psychology (redirected from Gestalt (music)):

»At the turn of the century, music psychology moved beyond the study
  of isolated tones and elements to the perception of their
  inter-relationships.«

My question goes into the other direction:
Has it been observed or theorized about that a person might not perceive

a sequence of beats as a rhythm
a sequence of tones as a melody
a simultaneous set of tones as a chord?

Have some people consciously reported to have heard

a sequence of beats but not a rhythm
a sequence of tones but not a melody
a simultaneous set of tones but not a chord

when "normal" people (test persons) did?
Can "normal" people get an idea how it is or feels like to do so, i.e. to hear (under normal conditions)

a sequence of beats but not a rhythm
a sequence of tones but not a melody
a simultaneous set of tones but not a chord?

What do normal cognitive sciences say about such phenomena? What goes "wrong"?
Comment: Do you mean something like tone deafness or similarly "rhythm deafness" or "harmony deafness"? Something like this perhaps https://en.wikipedia.org/wiki/Amusia
Comment: Sounds like [aphasia](https://en.wikipedia.org/wiki/Aphasia) to me. Cannot imagine such effects to be caused by anything other than structural damage to the brain.
Comment: @piiperiReinstateMonica:  I'm not an expert, but probably I mean something like that, while I would call the first not "tone" but "melody deafness": a melody deaf can perceive isolated tones but not melodies. The article on amusia is quite helpful (thanks for it) but it is not so easy to access. And it defines "melody deafness" quite differently than I do. The "inability to recognize familiar melodies" is something different than the "inability to perceive a sequence of tones as a melody".
Comment: Would hearing a sequence of tones as accompaniment instead of a melody count? For example, I've read descriptions saying that the 1st movement of Mozart's Symphony No. 40 in G Minor starts with accompaniment instead of melody.
Comment: @Dekkadeci: Thanks for this interesting hint. I have to think about it. I know what you mean: I perceive an accompaniment differently than a (lead) melody. (Same goes for [lounge music](https://en.wikipedia.org/wiki/Lounge_music) which is some kind of accompaniment.)
Comment: @Pyromonk: Since aphasia is a linguistic concept, maybe "muscial aphasia" would be better?
Comment: @Pyromonk: Furthermore, there are many symptoms of aphasia, only one being the "inability to perceive a sequence of words as a sentence" (which is the symptom I am asking for).
Answer: Music is a language. The hearing and reading of music (including perceiving sequences of notes as "phrases"), I imagine, is no different, which is why (Broca) aphasia was the first thing to come to mind. That's my opinion which seems to be supported by modern research ("the same areas of the brain process the syntactic information for both music and language").
A search on "music and aphasia" yields quite a few results, including these:

Facing the music: three issues in current research on singing and aphasia
A theoretical and clinical account of music and aphasia

Yes, "aphasia" is a bit of an umbrella term. I am not a neuroscientist (and have not worked as a linguist for a long time either) and can only base my opinion on the limited knowledge I have inherited from my ex-wife who was one. I cannot imagine what you describe to be caused by anything other than damage to Broca's area.
Comment: You suggest there will be no symptoms of linguistic aphasia without some symptoms of musical aphasia and vice versa. It would be interesting to know if this is true.
Comment: @Hans-PeterStricker, yes, that is indeed what I suggest. The first article links to a composer (Maurice Ravel) who suffered from Wernicke aphasia, whose biography might be worth investigating. It won't answer your question but might give you a better idea as to what's happening. Please take into account that I am not a professional in either music or neuroscience, my opinion is personal and stems from my experiences of learning languages (spoken, programming and the language of music).
Answer: In my own experience I have witnessed a series of tones that did not sound melodic at all. The lead break in Dr. Hooks version of "On the cover of the Rolling Stone" comes to mind. And at the risk of offending drummers everywhere, I've heard drum solos that seem to throw the concept of rhythm into the trash bin. Also, I'm not aware of a chord name for a tonic, minor second, major second, set of tones when played at the same time. This leads me to believe that these situations exist, but that we don't tend to apply the musical definition to them, choosing instead to just call them noise.
Answer: Let's consider all three one by one.
Sequence of tones: have you learned yet about twelve-tone music?  Create a twelve-tone row by using each of the twelve chromatic notes within the octave exactly once, with the order arranged such that the sequence does not strike the ear as sounding "tonal."  That is, choose the order such that the result doesn't sound like it's in one key or another, and doesn't have a cadence.  Once the composer has constructed a twelve-tone row in this way, that s/he finds interesting to listen to and work with, the next step is to make interesting music with it!
Rhythm: I haven't heard of this before, but I suppose it might be fun to try to create a sequence of note values such that no regular rhythm pattern, or groove, has been conveyed to the listener, in an analogous way to what I just described with the twelve-tone row.
Chord: Normally a chord is constructed by playing a triad with all notes in the triad sounding simultaneously.  (A triad is also called a "broken chord" -- it's every other note, ascending the scale.)  The simplest chord contains the first, third and fifth notes (or degrees) in the chosen scale (or mode, more generally).  It's a straightforward next step to add the seventh.  But in more recent history people have added other degrees, e.g. "added eleventh."  But note that you can optionally take that eleventh down an octave, if you want to.  Now you have a fat cluster of notes in your chord: third degree of the scale, fourth, fifth -- all sounding simultaneously.
But you can also have some fun juxtaposing two different chords together, at the same time.  Now you have some mud -- fun!  Effective in certain circumstances!  There are some lovely piano études by Waxman that can give the beginning to intermediate piano student some exposure to some of these more modern concepts.
I suggest you make some progress with playing and listening to "twentieth century music," and read some music theory books, to acquaint yourself with music that doesn't follow whatever stylistic paradigms you've been apparently limited to up until now  (Mozartian maybe?)  Expand your horizons -- and then come back after a year and reread your post.
Comment: I've listened to twelve-tone music. They were atonal and unpleasant to listen to, but I assure you they had melodies. It's also a bit debatable, but it's easier to perceive a tone cluster as a chord on visible sheet music (especially ones that span a sixth or more). Also be careful with the rhythm example: there are ways to construct quasicrystal rhythms that never repeat beyond a certain scale but that sound convincing enough.
Answer: A series of pitches that doesn't create a feeling of melody is called a tone row. A rhythm that doesn't give a pulse is called pointillistic or punctual rhythm.
Comment: But a tone row can be a melody, can't it?
Comment: Thanks, but this is not an answer to my question: I asked for the inability of perceiving a "normal" melody that is normally perceived as such (by normal listeners under normal conditions).


How can I Pay Azure Bill Before It's Due Day?

Question: I have an Azure Pay-as-you-go billing account. I've used resources with that account. Now I want to pay it before it's due date. Is it possible if so how can I do it? And what happens if it goes past due?
Answer: If your payment isn't received or if we can't process your payment, you will get an email and see an alert in the Azure portal telling you that your subscription is past due. If your default payment method is credit card, the Account Administrator can settle the outstanding charges in the Azure portal. If you pay by invoice (check/wire transfer), send your payment to the location listed at the bottom of your invoice.
If I forget to pay, The service is canceled and your resources are no longer available. Your Azure data is deleted 90 days after the service is terminated.
For how to pay bill check link.


Average multiple precision recall curves

Question: I have a binary classifier that I evaluate using cross validation. The classifier gives me a score for each observation and for each of the test sets I use a varying threshold to compute precision and recall values. Similar as to described in Threshold in precision/recall curve.
I can plot the precision recall values for each set as a curve by connection the entries/points with lines, but how would I average the different precision recall curves into one?
My first idea would be to use a for-loop to go through all recall values and try to (sub)sample precision values, but that seems to be very un-elegant.
How do you do it the right way?
Answer: I just ended up using an 'engineering' approach and used an optimized function for interpolation.
In my case:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.interp.html


Mobaxterm vs vim text copy

Question: Can I  somehow copy text between two vim sessions in 2 different terminals?
I use mobaxterm and can not scroll while marking text that I want to copy.
How can I do it? Edit 2 files on 2 different terminals (SSH sessions) with multiexec option and copy more text than I can see on terminal. I can not highlight it in any vim mode to copy it to my laptop notepad or other terminal SSH session window.
Comment: When copying more than screenheight, I usually use `cat` and select the content there...
Comment: This is, in fact, kind of solution :) but maybe someone can let us know if there is any better way to do it, i tried also other vim modes but i could not select text:)
Answer: In vim, you can use the r command to read a file or command from somewhere else. So, what I would do in this scenario is:

Dump what you want to copy to a file, either by redirecting the output of a command, or it's already a file so just save it.
Open vim in the other terminal and go to the file you want to paste into
Use :tabe from command mode to open a new tab
Use :r /path/to/file to read the content of that file that you created in (1) into the buffer
Then use yank/paste as normal, or use ctrl-V in command mode to do a visual select block and yank that.

Note you can pass a command to :r as well, but you have to escape some characters, so :r! date +\%s will get the current unix time and insert it into the file.
Lots more tips on the well known Grok Vim post.
You can also do things like turn on line numbering with :set number, and then read specific lines from a file: :r! sed -n 147,227p /path/to/foo/foo.c
Or, use the line numbering with relative numbering (:set relativenumber) to show you easily how many lines to yank with y. That way you just do 37y and then p where you want it.

Mobaxterm vs vim text copy


How to make singleton members threadsafe

Question: Let's say I have the following singleton:
public sealed class singleton
{
// fields, creation, etc.
private static IList<SomeObjectType> _objects = new List<SomeObjectType>();
private ManualResetEvent _updateCompleted = new ManualResetEvent(true);

private bool RefreshRequired()
{
    //true if records in DB newer than the objects in the list
}

private void RefreshList()
{
    _updateCompleted.Reset();

    //code to refresh the list

    _updateCompleted.Set();
}

public IList<SomeObjectType> Objects
{
    get
    {
        _updateCompleted.Wait();

        if (RefreshRequired())
            RefreshList();

        return _objects;
    }
}

..

This way I am trying to achieve that data stored in the list is always up to date before any client reads it. This mechanism is very simple, but it is working well so far. However, obviously it is not sufficient for multithreading scenarios.
If there were multiple threads accessing the Objects-Member, I wanted only the first one to check if data is up to date and, then update the List if necessary. While the refresh is in progress, all other threads should be forced to wait BEFORE even checking if an refresh is required.
I have read much ablut locks, BlockedCollections, and ManualResetEvents, but I am not sure about which concept to use.
Could you explain which one you would choose and how you would solve the described task?
Comment: The _objects variable does not need any help since it is created by the static constructor.  There is no simple way to make the Objects property thread-safe, you don't know when the client code stops using the return value.  Only the client code can do it correctly.  Passing the job off to a programmer that's least likely to get it right is one of the core reasons why writing thread-safe code is so difficult.
Comment: The whole concept of "thread safe" is itself [only vaguely-defined](http://blogs.msdn.com/b/ericlippert/archive/2009/10/19/what-is-this-thing-you-call-thread-safe.aspx). Your first step is to figure out what _exactly_ you actually mean by that. "Always up to date" is also similarly vague; there will always be race conditions...you need to decide what behavior you want in such cases. What if new data is made available while "the first one" is in the process of updating the list? What should other threads see then? That's just scratching the surface...you have a lot of work ahead.
Comment: Okay, maybe the term 'thread safe' is too much (or wrong) for what I actually want. After reading the linked article, Peter, I would say that I just want other threads to be blocked while the list is being refreshed.
Comment: I have also tried to figure out, what you mean with 'only the client code can do it correctly', Hans, but I do not get the point. Is it, that there could be one thread using the return value while another one finds out, that the list is not up to date?
Comment: You are not looking for a ThreadSave Singelton. You want to use a ThreadSave Collection ( A singelton would be a Single Instance of an class that provides you more then a Collection does ). You are looking for something like a [ConcurrentBag](https://msdn.microsoft.com/en-us/library/dd381779(v=vs.110).aspx). Also Possible( if you would like to stay by your Desgin), you could do the simple Lock and Wrap up every access that is made to your Collection if your are Intrested i post an example
Comment: Yes, JP, this is exactly what I want (and what I have written in the topic btw :) ). However, I am not sure if the ConcurrentBag or BlockingCollection can do the job since I also want to block other threads from entering RefreshRequired(). Maybe I am wrong, but I think the lock-statement would create a bottle-neck. I think, when there is no update in progress, everything is fine and no block is needed. This is why my first approach was to solve that by using a ManualResetEvent. This event was reset within the Update-Method and set when it was finished, but it didn't work :( How would you lock?
Comment: .. I have added the mentioned ManualResetEvent-approach to my code just to give you an idea of what I had tried. However, it seems to be too "slow" .. when the event is reset, other threads may have entered already. So my approach is insufficient.
Comment: Oh, and there is no need to keep that design .. if you could give an advice, that would be absolutely great :)
Answer: Best answer i can suggest is found here: http://csharpindepth.com/Articles/General/Singleton.aspx
Since I'll get yelled at by the masses if only posting a link, here are some samples taken from the article to ponder. The article largely has to do with performance, and appropriateness so please read it's descriptions of these samples.
As far as your Refresh method, the others commented on that fairly well. It's just as important knowing how it's intended to be consumed.
Hopefully the article gives you some food for thought.
Simple thread safety...
public sealed class Singleton
{
    private static Singleton instance = null;
    private static readonly object padlock = new object();

    Singleton()
    {
    }

    public static Singleton Instance
    {
        get
        {
            lock (padlock)
            {
                if (instance == null)
                {
                    instance = new Singleton();
                }
                return instance;
            }
        }
    }
}

Thread-safe without using locks...
public sealed class Singleton
{
    private static readonly Singleton instance = new Singleton();

    // Explicit static constructor to tell C# compiler
    // not to mark type as beforefieldinit
    static Singleton()
    {
    }

    private Singleton()
    {
    }

    public static Singleton Instance
    {
        get
        {
            return instance;
        }
    }
}

Fully lazy instantiation...
public sealed class Singleton
{
    private Singleton()
    {
    }

    public static Singleton Instance { get { return Nested.instance; } }

    private class Nested
    {
        // Explicit static constructor to tell C# compiler
        // not to mark type as beforefieldinit
        static Nested()
        {
        }

        internal static readonly Singleton instance = new Singleton();
    }
}

Using .NET 4's Lazy type...
public sealed class Singleton
{
    private static readonly Lazy<Singleton> lazy =
        new Lazy<Singleton>(() => new Singleton());

    public static Singleton Instance { get { return lazy.Value; } }

    private Singleton()
    {
    }
}

Comment: This is in no way solving the problem described in the question.
Comment: Servy is right, but thanks for your efforts, Atoms. I do know about the singleton pattern and its implementation and, thus, didn't mention it above .. if the timespan between the first thread entering and other ones is big enough, the ManualResetEvent should perform okay, but if there are threads entering almost simultaneously, this mechanism is just to slow - at least this is what it seems to me.


how to save excel file using vb.net?

Question: im stuck getting this error whenever i try to save excel in vb.. pleas someone help me what is wrong in this code:
<code>        xlWorkBook = xlApp.Workbooks.Add
        xlWorkSheet = xlWorkBook.Sheets("Sheet1")
        xlWorkBook.SaveAs(Filename:="C:\Users\PB\Documents\thesis\bookrecord", FileFormat:="xlCSV")
        xlApp.Visible = True
</code>
the save statement gets the error Exception from HRESULT: 0x800A03EC
Comment: Put that inside a try-catch block and see what the exception message tells you.
Answer: I made a simple test using the enum type for file format and it created the file without error. If this does nto work, please show a bit more of your coode:
<code>    xlWorkBook.SaveAs(Filename:="C:\Users\PB\Documents\thesis\bookrecord", FileFormat:=XlFileFormat.xlCSV)
</code>

how to save excel file using vb.net?


Replacing the corner of a piecewise continuous function with a bump function.

Question: I would like to share a problem that has been bugging me for a bit, which is on the general topic of bump functions.
The setting is that I have one piecewise function $p$, that comprises of a constant leg (suppose this is $g$) and one that looks parabolic (suppose this is $h$).
For reference,
\begin{equation}
p(t) = \begin{cases}
g(t) \\
h(t)
\end{cases} =
\begin{cases}
1\text{,  for }t<0, \\
\frac{1}{2} + \frac{1}{10} \sqrt{-80t^2+60t+25},
\end{cases}
\end{equation}
Let us consider $p$ inside $[-1,1]$ for the sake of simplicity.
They connect at $0$, and my goal is to smooth this out. The idea is to extract a stripe of length 2$\epsilon$, for some $\epsilon$ that is relatively small -- like $0.1$ for example, but not important at the moment -- and replace that part with a bump function that glues together the pieces into one, smooth function.
If I understand correctly, I should pick a bump function $f$, i.e.
\begin{equation}
f = \begin{cases}
\exp{{(-1/x^2)}}\text{,  for } x>0\\
0\text{,  otherwise,}
\end{cases}
\end{equation}
consider its integral (suppose it's $F$ but is still not clear to me why I need its integral). Then, in the $(-\epsilon, \epsilon)$ interval, I would replace said functions with $gF+h(1-F)$ and voila?
The issue is that this doesn't work so far. I have experimented with a few different variations for bump functions, and still something seems to evade me.
I would appreciate any insight on this. Ultimately, I want to understand the process exactly, in order to replicate this again in different points, using different functions.
All other similar questions simply provide the bump equivalent of the step function and I cannot completely see the connection.
Answer: The function $g$ is already very nice, so we just have to change $p$ in an interval $(0,\epsilon)$. The goal is to redefine it in that interval, so that the resulting function $\widetilde{p}$ is $C^\infty$ in $[-\infty,t_*)$, where $t_*$ is the smallest positive root of $h$.
Let $\varphi$ be a positive bump function in $(0,\epsilon)$,  that vanishes  with all its derivatives  outside $(0,\epsilon)$, and has integral $1$. For example, we can take
$$\forall x \in (0,\epsilon), \quad \varphi(x)=c_\epsilon \exp\Bigl(\frac{-1}{x(\epsilon-x)}\Bigr) \: \quad \text{and} \: \quad\forall x \notin (0,\epsilon), \quad \varphi(x)=0\,,$$
where $c_\epsilon$ is chosen to ensure that
$$\Phi(x):=\int_0^x \varphi(t) \,dt $$
satisfies $\Phi(\epsilon)=1$. Define
$$\widetilde{p}(x)=1+(h-1)\cdot \Phi(x)$$
for $x \in (0,\epsilon)$ and $\widetilde{p}(x)=p(x)$ for  $x \notin  (0,\epsilon)$.
Clearly $\widetilde{p}(0)=1=g(0)$ and $\widetilde{p}(\epsilon)=p(\epsilon)$.
By the Leibniz formula for the $n$'th derivative of a product (https://en.wikipedia.org/wiki/General_Leibniz_rule) we see that
$$\forall n \ge 1, \quad \widetilde{p}^{(n)}(0)=0$$
and
$$\forall n \ge 1, \quad \widetilde{p}^{(n)}(\epsilon)=h^{(n)}(\epsilon) \,,$$
as required.
Comment: Thank you very much for taking the time to explain. I believe I understand now.
Comment: Hey Yuval.

Thanks for your answer! There are a few things I would like to understand.

First of all, I would like to say that following your method, I ended up calculating a constant $c_\epsilon$ that is extremely large, i.e. $~10^{176}$, which is a bit troubling in itself.

Then, I would like to understand how your definition of $\tilde{p}$ emerged.
And, why do we need the integral of $\Phi$ to be equal to 1?

Please excuse my many questions -- I really appreciate you taking the time to help me out.
Comment: @TheConcernedScientist  1. This method does not attempt to optimize constants. In applications you would never try to match all the derivatives, the first two would suffice, and you would use splines https://en.wikipedia.org/wiki/Spline_(mathematics).
Comment: 2. The definition of $\widetilde{p}$ is natural in view of the Leibniz rule. When we compute the $n$th derivative of $(h-1)\Phi$ at $\epsilon$, the only term that does not vanish is $h^{n}(\epsilon)\Phi(\epsilon)$. That is why we want $\Phi(\epsilon)=1.$ It is not the integral of $\Phi$ but $\Phi$ itself that should equal 1 at $\epsilon$. Integrating $\varphi$ is just a convenient method to get a function with all derivatives vanishing at $0$ and at $\epsilon$, yet having different prescribed values at $0$  and at $\epsilon$.


Are there corner connectors for rj45?

Question: I cant find connectors or cables that have a turn at their end of the connector. (90 degrees)
This might be trivial to some. Do they exist at all?
Edit: I draw a picture for this
Comment: I dunno what you mean.  Can you draw a picture?
Comment: @TomO'Connor I added an image
Answer: VPI:

http://www.vpi.us/cat5e-down-angle.html
http://www.vpi.us/cat6-up-angle.html

BlackBox:

http://www.blackbox.com/resource/genPDF/SpaceGain-Brochure.pdf
http://www.blackbox.com/Store/Results.aspx/Cables/CAT6/Patch/90%C2%BA/Shielded/n-4294864837
http://www.blackbox.com/Store/Results.aspx/Cables/CAT6/Patch/90%C2%BA/Unshielded/n-4294952394
http://www.blackbox.com/Store/Results.aspx/Cables/CAT5e/Patch/90%C2%BA/Shielded/n-4294858109
http://www.blackbox.com/Store/Results.aspx/Cables/CAT5e/Patch/90%C2%BA/Unshielded/n-4294858172

Are there corner connectors for rj45?


Is it possible to apply Authcache caching to elements in page.tpl.php?

Question: In applying Authcache settings to an existing site I have encountered a problem where certain items hard-coded into the theme's page.tpl.php need to be personalised for each user. I could do it by moving those components into blocks and/or panels, but they are quite complex and I was wondering if a better approach would be to do it via the Authcache API using the authcache_usercookie.module example as a starting point.
Here is an example of the code I would like to personalise:
<?php if($page['topics_tab']): ?>

        <?php print render($page['topics_tab']); ?>

<?php endif; ?>

Is it possible (easy? difficult?) to modify page.tpl.php to make Authcache cache this snippet for each user, or should I redo the page using blocks/panels?
Answer: I recommend to redo the page with blocks (or panels). The Authcache User Cookie example is rather basic and does not demonstrate how to deliver fragments from the custom front controller.
Regrettably there is still not enough documentation on how to roll your own custom fragment renderer. Look at the api.php files and read through the numerous submodules to get the picture.
Last but not least, there are some (closed) support requests in the ticket tracker covering this issue, e.g. #2780485.


SQLite and Ruby rescue

Question: I'm following the pretty great SLite & Ruby tutorial over at ZetCode and have run into a silly question. What exactly is going on here?
rescue SQLite3::Exception => e

    puts "Exception occured"
    puts e

I conceptually get what's happening: if an exception is raised, it gets printed on the screen. But what is this syntax, specifically?
SQLite3::Exception => e

Is there another simple example of where I'd use the hash rocket like this?
Thanks!
Answer: The rescue statement has an unusual use for that notation. It means "capture the exception of type SQLite3::Exception into the variable e":
rescue SQLite3::Exception => e

You're not required to capture the exception into a variable, you can omit that part if you don't need it, and you don't need to specify the exception kind, either, which will default to the exception base class Exception.
The syntax is only coincidentally the same as the hash declaration presumably to promote consistency. The two are not otherwise related.
Comment: The statement is equivalent to Java's `} catch (SQLiteException e) {`

SQLite and Ruby rescue


Google firestore rest API: How to create documents within a transaction

Question: Using a transaction ref I can update multiple documents by creating an array/list of Write objects and then doing a CommitRequest.
However I can find no way of creating new documents within the transaction? firestoreApi.projects.databases.documents.createDocument(...) Doesn't provide have a transaction parameter. Write seems to for updating documents and not creating new ones.
Answer: Creating documents from a transaction is definitely possible. You would first need to begin a new transaction using the projects.databases.documents.beginTransaction endpoint. After using the beginTransaction endpoint, you will receive a transaction ID that can be used with the projects.databases.documents.commit endpoint. This endpoint does use a Write object, but the Write object has a currentDocument property which is a Precondition type. The Precondition object in turn has a property called exists, which needs to be set to false so that it can be declared that a document does not exist.

When it is set to true, the target document must exist. When it is set to false, the target document must not exist.

This allows a new document to be created from the transaction, and then the transaction is committed.


Como descobrir o negativo de uma cor em hexadecimal?

Question: Gostaria de saber qual é o cálculo que posso utilizar em linguagem como javascript, php ou python para descobrir o negativo de uma cor hexadecimal.
Exemplo: Se negativo de branco é preto, então eu pressuponho:
`#FFFFFF => #000000`

Ou
 0xFFFFFF => 0x000000

Answer: A cor inversa pode ser calculada via OU exclusivo (XOR) versus o valor máximo (#FFFFFF).
O formato de cores em hexadecimal já é uma dica de como realizar esta operação: cada um dos duplos octetos em #RRGGBB representa uma cor primária, cuja intensidade pode variar de #00 a #FF (255 em decimal).
Em outras palavras, cada cor primária é representada por 8 bits.
Para inverter uma cor, inverta os bits:
Cor     Valor HEX valor BIN                     BIN invertido                 HEX invertido
Preto   #000000   B00000000 00000000 00000000   B11111111 11111111 11111111   #FFFFFF
Branco  #FFFFFF   B11111111 11111111 11111111   B00000000 00000000 00000000   #000000
Azul    #0000FF   B00000000 00000000 11111111   B11111111 11111111 00000000   #FFFF00

Um exemplo de código em Angular/javaScript que realiza esta operação segue abaixo:

function SampleController($scope) {
  $scope.sourceValue = 0;
  $scope.inverseColor = function(){
    var srcVal = $scope.sourceValue;
    var valNumerico = parseInt(srcVal, 16);
    var mascara = parseInt('FFFFFF', 16);
    var dest = valNumerico ^ mascara; //Operação XOR
    return dest.toString(16);
  };
}

      Cor: #<input type='text' ng-model='sourceValue'>

      Inversa: {{ inverseColor() }}

Answer: Encontrei um exemplo em PHP:
function color_inverse($color){
    $color = str_replace('#', '', $color);
    if (strlen($color) != 6){ return '000000'; }
    $rgb = '';
    for ($x=0;$x<3;$x++){
        $c = 255 - hexdec(substr($color,(2*$x),2));
        $c = ($c < 0) ? 0 : dechex($c);
        $rgb .= (strlen($c) < 2) ? '0'.$c : $c;
    }
    return '#'.$rgb;
}

Testando:
// preto -> branco
print color_inverse('#000000');
// --> retorna #ffffff

Créditos: Link
Comment: Realmente, foi o que um amigo me disse: você tem que usar o valor em RGB pra fazer isso :|
Answer: Se for apenas para exibição você pode resolver isso no CSS
-webkit-filter: invert(100%);
filter: invert(100%);

Via Javascript tem a solução nessa outra pergunta.
https://stackoverflow.com/questions/1664140/js-function-to-calculate-complementary-colour