dealing with trailing slashes in angularjs

Question: I have a function that looks like this:
   $scope.deleteTodo = function(id) {
        var url = '/musicians/' + id;
        console.log("URL" + url);
        $http.delete(url, { 'id': id})
            .success(function(data) {
                $scope.todos = data;
                console.log(data);
            })
            .error(function(data) {
                console.log('Error: ' + data);
            });
    };

If Id is 123, I get /musicians123
However, I was expecting /musicians/123.
I straightaway get an error and it does not even print console.log statement. Since it it before $http.delete, I believe it should have been printed but it does not.
I have tried escaping / using \\/ but did not help.
Also, I tried to configure with:
.config(['$resourceProvider', function($resourceProvider) {
  // Don't strip trailing slashes from calculated URLs
  $resourceProvider.defaults.stripTrailingSlashes = false;
}]);

Stacktrace:
DELETE http://localhost:3000/musicians55accbce27e2cd2802de4894 404 (Not Found)$get.id @ angular.min.js:100n @ angular.min.js:96l @ angular.min.js:95$get.l.(anonymous function) @ angular.min.js:97$scope.deleteTodo @ VM416 core.js:43w @ angular.min.js:73(anonymous function) @ angular.min.js:145$get.e.$eval @ angular.min.js:89$get.e.$apply @ angular.min.js:89(anonymous function) @ angular.min.js:145x.event.dispatch @ jquery.min.js:5x.event.add.y.handle @ jquery.min.js:5
VM416 core.js:49 Error: Error: Not Found
    at Layer.handle (C:\Users\dev4\Desktop\Sample\server.js:86:13)
    at trim_prefix (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:226:17)
    at C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:198:9
    at Function.proto.process_params (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:251:12)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:189:19)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38)
    at next (C:\Users\dev4\Desktop\Sample\node_modules\express\lib\router\index.js:166:38

But no help. How do I fix this?
Comment: @Vineet that will not make any difference
Comment: You can try var url = '/base/' + id.toString()
Answer: There is nothing wrong with what you posted. Additionally, resourceProvider has nothing to do with how your slashes are parsed in a string. For a working example, see http://plnkr.co/edit/Y99E2pQA6RfVRkg1t6zZ?p=preview .
angular.module('app', [])
.controller('testCtrl', ['$scope', function($scope) {
  $scope.Demo = function(id) {
        var url = '/base/' + id;
        alert("URL: " + url);  //It gives `base123` if 123 is the id
        };
}]);

<!DOCTYPE html>

    Test Demo Function

Comment: I copied what you posted and it still works for me. Can you try

var url = "/musicians/" + id;

(double quotes)?


Cadastro sem duplicidade no SQLite

Question: Consegui evitar a duplicidade de cadastro do romaneio(nr_roman), porém fiz várias tentativas de fazer o mesmo com a plaqueta(plaq),de tal forma que, se já existir a plaqueta no banco de dados, ele persista na solicitação do nr da plaqueta, sem precisar voltar ao início do programa, porém sem sucesso.
def cad_romaneios():
    # conectando...
    con_romaneios = sqlite3.connect("romaneios.db")
    # definindo um cursor
    cursor = con_romaneios.cursor()

    # Se a tabela não existe, então é criada a seguir...
    cursor.execute('''CREATE TABLE IF NOT EXISTS romaneios
        (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
        nr_roman VARCHAR(4) NOT NULL,
        data_cad TEXT NOT NULL,
        qtde_cad INT NOT NULL,
        nome_tor VARCHAR(30) NOT NULL,
        nome_fre VARCHAR(30) NOT NULL,
        plac_01 VARCHAR(8),
        plac_02 VARCHAR(8),
        plaq VARCHAR(6) NOT NULL,
        espe VARCHAR(30) NOT NULL,
        dia1 INT NOT NULL,
        dia2 INT NOT NULL,
        comp FLOAT NOT NULL,
        desc FLOAT(2),
        vlrm3 FLOAT NOT NULL,
        volume FLOAT NOT NULL,
        vlrpg FLOAT NOT NULL);''')

    print("===========================")
    print("=  CADASTRO DE ROMANEIOS  =")
    print("===========================")

    continua_cadastro = True
    volume_total = 0.000
    total_pago = 0.00
    total_desconto = 0.000

    while continua_cadastro:
        p_nr_roman = input("Nº do romaneio  : ")
        while p_nr_roman == "":
            p_nr_roman = input("Nº do romaneio  : ")
        # Verifica se já existe romaneio com este número...
        sql_romaneio = "SELECT * FROM romaneios WHERE nr_roman = ?"
        for dados in cursor.execute(sql_romaneio, (p_nr_roman,)):
            if dados:
                beep(700, 50)
                print("========== A T E N Ç Ã O ==========")
                print("    O romaneio nº", p_nr_roman + " já existe!")
                print("===================================")
                # continua_cadastro = False
                time.sleep(.5)
                print("")
                cad_romaneios()

        # ... se o romaneio informado ainda não existe, continua o cadastro...
        p_data_cad = str(input("Data do romaneio: "))
        # A data é obrigatória...
        while p_data_cad == "":
            p_data_cad = str(input("Data do romaneio: "))
        p_qtde_cad = int(input("Quantidade toras: "))
        while not int(p_qtde_cad):
            p_qtde_cad = int(input("Quantidade toras: "))
        p_nome_tor = str(input("Toreiro/Vendedor: ")).upper()
        # Obriga o usuário a informar o nome do Toreiro/Vendedor/Projeto...
        while p_nome_tor == "":
            p_nome_tor = str(input("Toreiro/Vendedor: ")).upper()
        p_nome_fre = str(input("Nome do Freteiro: ")).upper()
        if p_nome_fre == "":
            p_nome_fre = "Não informado."
        p_plac_01 = str(input("Placa 01 veículo: ")).upper()
        # Obriga o usuário a informar uma placa pelo menos...
        while p_plac_01 == "":
            print("Informe uma placa...")
        p_plac_02 = str(input("Placa 02 veículo: ")).upper()
        print("===================================")

        for i in range(p_qtde_cad):
            if i + 1 <= p_qtde_cad:
                print("Cadastrando a tora nº", i + 1)
            else:
                print("Cadastrando última tora...")

A plaqueta não deve ter duplicidade. Isto que eu não consegui fazer. Tentei fazer igual ao p_nr_roman, porém sem sucesso.

            p_plaq = input("Nº plaqueta: ")
            while p_plaq == "":
                p_plaq = input("Nº plaqueta: ")**
            p_espe = input("Essência...: ")
            while p_espe == "":
                p_espe = input("Essência...: ")
            p_dia1 = input("Diâmetro 1 : ")  # int
            while p_dia1 == "":
                p_dia1 = input("Diâmetro 1 : ")
            p_dia2 = input("Diâmetro 2 : ")
            while p_dia2 == "":
                p_dia2 = input("Diâmetro 2 : ")  # int
            p_comp = input("Comprimento: ")  # float
            while p_comp == "":
                p_comp = input("Comprimento: ")
            # Verifica de houve desconto...
            p_desc = 0
            try:
                p_desc = int(input("Desconto ..: "))
            except ValueError:
                print("Não houve desconto!")
            p_vlrm3 = input("Valor do m3:")  # float
            while p_vlrm3 == "":
                p_vlrm3 = input("Valor do m3: ")

            p_vlrm3 = float(p_vlrm3)
            p_dia1 = int(p_dia1)
            p_dia2 = int(p_dia2)
            p_comp = float(p_comp)
            p_vlrm3 = float(p_vlrm3)

            # =====================================
            # CUBICANDO AS TORAS INDIVIDUALMENTE...
            # Cálculo do desconto...
            # ======================
            p_desc = ((p_desc * p_desc) * p_comp * .7854) / 10000.0

            # Cálculo do volume...
            # ====================
            p_volume = ((((p_dia1 + p_dia2) / 2) * ((p_dia1 + p_dia2) / 2) * .7854 * p_comp) / 10000.000) - p_desc

            # Cálculo do valor pago...
            # ========================
            p_vlrpg = p_vlrm3 * p_volume
            print("Custo da tora: R$ %.2f" % p_vlrpg)
            print("")

            # Aquí vai acumulando os totais do Volume / Valor pago / Desconto
            # ===============================================================
            volume_total = volume_total + p_volume
            total_pago = total_pago + p_vlrpg
            total_desconto = total_desconto + p_desc

            # Contador que controla o número de toras cadastradas...
            # ======================================================
            i += 1
            # GRAVANDO NO BANCO DE DADOS...
            # Informação: são 15 variáveis...
            cursor.execute("""INSERT INTO romaneios (nr_roman, data_cad, qtde_cad, nome_tor, nome_fre, plac_01, plac_02,
                                    plaq, espe, dia1, dia2, comp, desc, vlrm3, volume, vlrpg)
                                    VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""", (
                            p_nr_roman, p_data_cad, p_qtde_cad, p_nome_tor, p_nome_fre, p_plac_01, p_plac_02, p_plaq,
                            p_espe, p_dia1, p_dia2, p_comp, p_desc, p_vlrm3, p_volume, p_vlrpg))
            con_romaneios.commit()

        # Montagem e apresentação do Romaneio...
        # ======================================
        import os
        os.system("cls")  # Apaga porção da tela
        beep(700, 70)
        print("  ======================")
        print("  =    Cadastro OK!    =")
        print("  ======================")
        time.sleep(1)
        print("")
        # if i + 1 >= int(p_qtde_cad):
        print("Romaneio nº:", p_nr_roman + "  -  Data:", p_data_cad)
        print("Origem ....:", p_nome_tor + "  -  Freteiro:", p_nome_fre)
        print("Veículo ...:", p_plac_01 + " / ", p_plac_02)
        print("Quantidade :", int(p_qtde_cad), " tora(s)")
        print(29 * "=" + int(len(p_nome_tor + p_nome_fre)) * "=")
        print(" Volume total ...: %.3f" % volume_total + "m3")
        print(" Volume desconto : %.3f" % total_desconto + "m3")
        print(" Valor total pago: R$ %.2f" % total_pago)
        print("")
        print("   Estatística")
        print("  =============")
        print(" Média m3 por tora : %.3f" % (volume_total / int(p_qtde_cad)) + "m3")
        print(" Média pago p/ tora: R$ %.2f" % (total_pago / int(p_qtde_cad)))
        break
    con_romaneios.close()
    sys.exit(0)

principal()

Comment: Não li toda pergunta, provavelmente é está longa demais e nada disso é necessário, mas existe algum motivo para não ter usado `UNIQUE`? https://sqlite.org/lang_createtable.html
Comment: Olá caro Denis! grato pelo seu tempo.
Comment: ... vou me aprofundar nesse assunto do "UNIQUE". Grato Denis!
Comment: A resposta resolveu sua dúvida? Acha que pode aceitá-la? Veja o [tour] se não souber como faz. Isso ajudaria muito indicar que a solução foi útil para você. Também pode votar em qualquer pergunta ou resposta que achar útil no site todo (quando tiver 15 pontos).
Answer: Quando você precisa que um dado (uma ou mais colunas) seja único em uma tabela você usa a cláusula UNIQUE que proibirá que sejam cadastrados valores duplicatas de forma inequívoca e sem risco de uma condição de corrida como poderia ocorrer no código que estava tentando fazer.
Em geral é importante ter um índice adequado para facilitar esta verificação, caso contrário tem o risco de ficar lento.
Sempre que tentar cadastrar algo duplicata gerará um erro no banco de dados que deverá ser tratado pela aplicação.
Pode ser algo assim:
CREATE TABLE tabela (col1 INT, col2 INT, UNIQUE(col1, col2));

Coloquei no GitHub para referência futura.
Neste caso a conjunção dessas duas colunas precisa ser única.
Em alguns casos pode querer usar um ON CONFLICT REPLACE se desejar que na existência da chave ela seja atualizada no lugar de inserida. Ou pode escolher outras formas.
Pode ser que só queira ser informado do erro na aplicação. Cada linguagem e cada tecnologia de acesso ao SQLite pode trabalhar com uma forma diferente de tratar o erro. Exemplo em C#:
try {
    //seu código
} catch (SQLiteConstraintException e) {
    //o que fazer quando dá o erro, pode ser que queira ver se é UNIQUE mesmo
}

Comment: Boa tarde colegas!
Verifico que se eu não usar UNIQUE, posso criticar a rotina que faz a entrada de dados para não permitir a duplicidade; tenho feito isso.
No momento, minha dúvida está no  trecho de código que escrevi, ou seja: se for informado um nr de romaneio existente, o usuário é informado e o programa retoma a pergunta; vencido essa crítica, é informado o nr da placa; caso esta já exista, informo o usuário e o programa  tem que voltar ao ponto  em que esta última informação é solicitada. Aí é que não estou achando o caminho...
Comment: Ou seja, você vai insistir em fazer errado?
Comment: Ah!! quer dizer que posso usar o try/catch para solucinar essa questão. Sorry, não havia entendido.
Vamos lá então....
Muito agradecido!!

Cadastro sem duplicidade no SQLite


Methods for feature elimination of categorical variables?

Question: I have a dataset with 56 columns, 4 numeric and the target variable which is also numeric. I am trying to eliminate some of the categorical variables from my model and wanted to get some understanding of the methodologies to do that.
I am currently working in R, which is somewhat new to me.
Any help would be great!
Edit: I would like to eliminate some variables to increase my models performance. While the model is not performing poorly, I believe the model can perform better than it is now. Some of the variables I can take a look at and know right off the bat that this is something that I can remove. But, I would like to show proof that this is not an informative predictors.
Edit2: Here is information about my dataset.
Out of the columns that are categorical, there are roughly 5 variables that have levels greater than 2. My goal is essentially to remove variables that are the least informative to my model. My model is a regression model to predict the sale price of something. Using accuracy as a metric, I want to have the most accurate model and remove predictors (both numeric and categorical) that are not conducive to the predicting accuracy of my model.
In removing categorical variables, I would be removing the entire variable, not particular levels.
Comment: Why do you want to eliminate variables? Please provide more context.(welcome to the site btw. Please put any updates into the question, not as a comment)
Comment: @RobertLong Added the update!
Answer: The group LASSO was designed specifically to allow groups of predictors to be selected or omitted together, so that is what you would want to use in your case to keep all levels of a categorical predictor together. There are several implementations in R. This, unlike stepwise selection, provides a principled way to decrease the number of predictors; you can choose the LASSO penalty (and thus the number of maintained predictors) by minimizing cross-validation error to get a good tradeoff between parsimony of predictors and performance while providing reasonable application to new data samples. It seems that you do not want to omit any of your numeric predictors, so make sure that the implementation you use allows for forced inclusion of chosen predictors.
There is one unavoidable difficulty with LASSO or ridge regression with categorical predictors: whether/how to normalize those predictors before doing the analysis, which applies a single penalty to all predictors. Normalization is clearly necessary with numeric predictor values: you don't want the penalization to depend on the scale of measurement (e.g., millimeters versus miles). So programs may default to scaling all predictors to 0 mean and unit standard deviation for calculations, then back-transforming to the original scales for reporting coefficient values.
But is that what you want with a binary categorical predictor, where the mean and SD necessarily depend on the class prevalence? Does such normalization even make sense with a multi-level categorical variable? Depending on the implementation, your results with a multi-level categorical predictor might even differ depending on your choice of reference level. So spend some time thinking about what you want to accomplish, the details of how your chosen LASSO implementation works, and whether or not to specify normalization for the categorical predictors.
Comment: Thank you for the elaborate explanation, that makes sense to me. I would like to follow up by saying that I have no problem removing numeric predictors if they are in fact uninformative in the model. If both numeric and categorical variables can be used in this, I think this is the solution I was looking for. Any idea on where to look to implement this in R?
Comment: A quick web search for `group lasso r` shows 3 packages with this functionality: `grpreg`, `grplasso`, and `gglasso`. I don't have much experience with such packages, so read the documentation carefully.
Answer: Here are some ways to do this:
1) Lasso: Perform regularized regression with the lasso penalty. This will force most of your predictors to be 0.
2) Forward selection: At any point, greedily add the feature that adds most predictive value.
3) Backward selection: Start with all predictors and greedily remove the feature that if removed affects the prediction quality the least.
Comment: Note that forward and backward selection are typically [very poor choices](https://stats.stackexchange.com/q/20836/28500) in constructing a model. They tend to overfit, producing models that do not perform well on new data samples. The regularization provided by LASSO tends to avoid these difficulties, but there might be complications if some of the categorical variables have more than 2 levels.
Comment: @EdM Thanks for the response. I am interested in using the LASSO method. My data set does have levels greater than 2 although the majority of the categorical variables only have 2 levels.
Comment: Thank you for the post. I have used backward selection in the past and had some success but in the case of this data set, that does not seem to be working. Can the lasso method be used for categorical variables with levels greater than 2?


How to make my function loop every x second in javascript

Question: Can someone help me so that the function myFunction runs every X second. The function works perfectly if i only run it once, but when i begin using setInterval it is not run every x seconds. in the example below, it is sometimes executed after 5 sec and sometimes after 8 and so on. What should i change in my code? The api i´m using only grants 30 request per minute and i´m worried this function will exceed the limit as there are two getJSON.
setInterval(function () {
    myFunction();
}, 10000);

function myFunction() {

    $.getJSON("URL", function (data) {
        console.log(data);
        $.getJSON("URL", function (data2) {
            console.log(data2);

            variable1 = data.Departure;
            variable2 = data2.Departure;

            text = "";
            variable3 = variable1.concat(variable2);
            variable3.sort(function (a, b) {
                return new Date('2017/01/16 ' + a.time) - new Date('2017/01/16 ' + b.time);
            })

            console.log(variable3);
            for (var i = 0; i < variable3.length; i++) {
                text += variable3[i].name.substring(13, 19) + " departure " + variable3[i].time.substring(0, 5) + " from " + variable3[i].stop.substring(8) + "
";
            }
            document.getElementById("Textn").innerHTML = text;
            console.log(text);

        });
    });
}

myFunction();

Comment: setTimeout function runs every x seconds.
https://developer.mozilla.org/es/docs/Web/API/WindowTimers/setTimeout
Comment: How do you know it's taking 5 seconds, 8 seconds, etc?
Comment: @jaikl How would you know how much time your function is taking ?
Comment: setTimeout function run only once @Roy
Comment: @JamesMonger i counted when the function was running. It should take at least a minimum of 10 seconds so something is wrong.
Answer: Problem with the current implementation is that it uses asynchronous requests which may take more than 10 seconds(can't predict when the request is served).
The method setInterval will execute the myFunction irrespective of the previous requests were completed or not.
As a result next request will queue up to be executed. So you get it is sometimes executed after 5 sec and sometimes after 8 and so on.
You should use setTimeout() to recursively invoke the method instead of using setInterval
function myFunction() {
    $.getJSON("URL", function (data) {
        $.getJSON("URL", function (data2) {
            //Your existing code

            //Schedule it be executed after x * 1000 milliseconds
            setTimeout(myFunction, 10000);
        });
    });
}
myFunction();

Answer: Try This:

callfunction();
var callCount = 1;
var repeater = setInterval(function () {
  if (callCount <= 30) {
    callfunction();
    callCount += 1;
  } else {
    clearInterval(repeater);
  }
}, 1000);

function callfunction()
{
  console.log((new Date()).getSeconds());
}
Answer: Example setInterval implementation, This will run  every 1 second. Your code should call the function every 10 seconds, Couldn't find any issue in that.

setInterval(myFunction, 1000);

function myFunction(){
 console.log((new Date()).getSeconds());
};

myFunction();
Comment: Haven't answered reason behind _it is sometimes executed after 5 sec and sometimes after 8 and so on._


$M$ $\mathscr{R}$-submodule complete lattice $\iff$ $aL \subset M \subset a^{-1}L$, $a \in \mathscr{R}$

Question: My question may be nestled in the sense that there might be confusion about several notions, please let me know if it is the case.
In "The Arithmetic of Hyperbolic 3-manifolds" by Maclachlan and Reid, the proof of the following Lemma is left as an Exercise:

Let $L$ be a complete Lattice in $V$ and $M$ an $R$-submodule of $V$. Then $M$ is a complete Lattice $\iff$ there exists $a \in R$ such that $aL \subset M \subset a^{-1}L$

Note: $R$ is a Dedekind Domain whose Field of Fraction $k$ is either a number field or a $\mathscr{P}$-adic Field. An $R$-Lattice is a finitely generated $R$-module contained in a vector space $V$ of finite dimension over $k$. Completeness of an $R$-Lattice $L$ is defined through $L \otimes_R k \cong V$.
My attempt:
$\implies$ since both are complete, we have $L \otimes_R k \cong M \otimes_R k$ as vector spaces. Therefore, for $l_i,m_i$ bases of $L,M$, we have that $l_i \otimes_R 1$ and $m_i \otimes_R 1$ are bases of their respective spaces and we can can $\alpha_j^i \in k$ such that:
$$ l_i \otimes_R 1 = \sum_j \alpha_j^i \ m_j \otimes_R 1 $$
Since $k$ is the field of fraction, we can express each $\alpha_j^i = n_j^i/d_j^i$ with $n_j^i, d_j^i$ both in $R$.
At this point I get stuck. I figured I could use a method similar to the proof of the gaussian lemma (roughly speaking factoring out the denominators).
But I am falling short.
Answer: Have you tried a 2 by 2 example? The entries $\alpha^i_j$ form an invertible change of basis matrix. Mutiply both sides of the equation you wrote by the common denominator of all entries of the matrix, denoted by $a$, we have $$a \cdot l_i\otimes_R 1 = \sum_j (a \cdot \alpha^i_j ) m_j\otimes_R 1,$$ which implies $a\cdot L\subset M$. Invert your equation by the inverse of the matrix of $\alpha^i_j$, you will have $M \subset a^{-1}L$.


mean comparison using SD or 2*SD

Question: In many plots there are the group means and +/- 1 SD. Does it make sense to use the +/- 2*SD for making a simple, rough and quick visual comparison of the means?
For example, in this plot are 10 independent groups. We can see the mean and +/- 1 SD (we don't have standard error). Then, while I am presenting this plot in a meeting someone asks if the last group could be different compared to 1st group (comparison of these two groups only, we don't care for the rest).
Does it make sense to say, look the 2SD of 1st group and the 2SD of the last group are not overlapping so yes, the chances to be different is high.
Is this thought somehow valid? or the response should be something like.. we can not make this conclusion from this plot?
Comment: Generally this is not a good idea.  The situation is more complicated than that, but it is possible to [develop relatively simple rules of thumb to compare means](https://stats.stackexchange.com/a/18259/919) based on their individual standard *errors* (or, almost equivalently, their individual confidence intervals).  Comparing SDs of data is almost meaningless because you need to factor in the sizes of the datasets.
Comment: But the 95% of the data should be within 2 SD from the mean. Hence, somewhere inside this interval should be the "real mean" which the 95%CI try to find. But still, when a pair of 2SD are not overlapped we cannot expect to be different?
Comment: The 95% CI will typically be *much* smaller than the $\pm$2 SD interval, depending on how much data there are in the group.  Moreover, the standard error of the *difference* of means will be about 30% smaller than the sum of the standard errors of each mean (assuming the two SEs are about equal). Thus, when the $\pm$2 SD intervals do not overlap, it's usually fair to conclude the means differ.  But this is a truly severe test! You also need to account, in your application, for the fact you are doing many comparisons, not just one.
Answer: Are the groups independently and identically distributed? The standard deviation measures the dispersion of the distribution, while if you are only interested in comparing the means it may be better to plot the standard error of the mean (i.e. confidence intervals around the point-estimate of the sample mean - not the distribution of the data itself).
Answer: $$
\hat\mu \pm 2 \times \frac{\hat\sigma}{ \sqrt{n}}
$$
is roughly the normal 95% confidence interval for the mean, but notice that it uses standard error, not standard deviation.
You can check also the What is the difference between confidence intervals and hypothesis testing? thread.

mean comparison using SD or 2*SD


Football Player auctions

Question: I was wondering the scenario where a player is playing for a club and is to be transferred (free/purchase) to another club. What is the procedure?
For e.g.,

It is the player's wish to go to X Club and he talks to X club's manager
It is the Player's manager who takes the decision to move this player for money
a. He is not playing well
b. He is getting huge money from the other club for transfer

Cheers!!!
Answer: Transfer of a player can be initiated as the following situations:

Player or Player agent came to team manager with a proposal from another team.
Another team or an agent address a team with a proposal to buy player.
Player address his manager and ask to be released/sold from the team and if the manager accepts he will put the player in Transfer List (and then the player agent or scouts from another teams will try to find sell deal)
Club put player in Transfer List (and then the player agent or scouts from another teams will try to find sell deal)
Team want to move the player to another team as part of exchange deal.

In all of those deals the buying club should get to an agreement with the selling club about player price.
Then the buying club should get to an agreement with the player (or his agent) about player contract.
If this is an exchange deal (section 5) the selling club should also get to an agreement with the exchanging player (or his agent) about player contract.
I can think about some reasons for a player move/transfer:

Player had finish his contract at the club and both player or team won't want to continue the contract (or not come to an agreement about the details of the new contract).
Team want to release/sell the player to another team in order to reduce the wage balance of the team.
Team want to sell the player to another team in order to earn money on this player.
Team want to release/sell the player to another team נecause of incompetence professionally and \ or personally.
Player is asking to move to another team (better challenge, better wage or other reasons) and his current team approved it.
Team to sell a player as part of exchange transfer where it can get another player that will suit better for the team

If you can think on some other reasons you can add it to my answer..

Football Player auctions


install Drush on shared server

Question: please i need to use drush on  shared server
i allready use composer foe install drush,,
but if i use sudo command
sudo mv drush.phar /usr/local/bin/drush
, it is give me this massage

-bash: sudo: command not found

i allready contact the server customer serves
and the answer was
we cannot support and sudo commnds
any IDEAS please for using drush on shaired server
Comment: `sudo mv drush.phar /usr/local/bin/drush` moves the Drush executable to a path that lets you invoke it by calling `drush`. This was the case [for Drush 8 and older](https://docs.drush.org/en/8.x/install/) but is [no longer supported in Drush 9 and newer](https://www.drush.org/latest/install/). Instead, you install Drush locally with the project and invoke it via `vendor/bin/drush`.
Comment: You can still call `drush` when using the [Drush Launcher](https://github.com/drush-ops/drush-launcher). If you don't have sudo privilege see https://drupal.stackexchange.com/questions/311773/how-can-i-add-composer-to-an-existing-site how to install a binary in ~/bin.
Answer: Execute Drush out of the vendor directory as vendor/bin/drush. That is all you need to do. Shared hosting will never allow you to place executables in /usr/local.
Comment: yes thats what they say , im looking for anther way to use drush
Comment: Then you need to rewrite your question because we do not understand it.
Comment: I think this answer is pretty clear. There is no need to move Drush in a different directory that is only accessible from the root user.
Answer: i found the answer for my issue
we should use the command from the website directory
and use

vendor/bin/drush

not only

drush

after > vendor/bin/drush we can put the drush command and injoy by using it
thanks every one for help
Comment: That is what the already given answer said: *Execute Drush out of the vendor directory as `vendor/bin/drush`.*

install Drush on shared server


ether.js accessing public views

Question: I have 2 public views in my smart contract but when I try to access them with ether.js if I try with callStatic like that:
NVAContract.callStatic.calculateTotal(address)

it throws an error: callStatic undefined
If I try like that:
NVAContract.calculateTotal(address)

it asks for gas fees. (the contract and the abi are right, double checked the abi and the same contract object works when using write functions)
Answer: Fixed. I just realized I was using ethers.js 3, switching to version 4 fixed the problem.

ether.js accessing public views


composer require fails because it can't delete default.services.yml

Question: Using composer with Drupal (7, 8 & 9), composer require drupal/<package-name> returns the following error.

Installation failed, reverting ./composer.json to its original content.
[RuntimeException]
Could not delete web/sites/default/default.services.yml

How can I fix this error?
Comment: note to others: be aware of the security implications of the answers proposed so far as described in comments
Answer: Make your "local" machine work as intended, and maintain the security of your live site.
Since you want to use composer require command only on non-production environments, the following steps affect only on the machines you deliberately set up. Drupal's recommended configuration will be preserved on all other machines, including production servers.
Don't make this change to settings.php to avoid having it on PROD or propagating it silently to other developers' machines.
Steps

enable settings.local.php (here are the instructions from the docblock in settings.local.php):

 * To activate this feature, copy and rename it such that its path plus
 * filename is 'sites/default/settings.local.php'. Then, go to the bottom of
 * 'sites/default/settings.php' and uncomment the commented lines that mention
 * 'settings.local.php'.
 *
 * If you are using a site name in the path [... read about exceptions in file]

set file permissions to allow user and group to write to the file on your local machine:
$ chmod ug+w {name_of_docroot}/sites/default

How this approach matches up with the established community approach
By default, settings.local.php contains the following line:
$settings['skip_permissions_hardening'] = TRUE;

This line tells Drupal to avoid resetting permissions on the default folder only on your local, and allows you to work more smoothly with composer on your local without weakening permissions on any other copy of the codebase.
Comment: Makes sense. You should only make changes to web/default/ on your local, so it's logical to keep local permissions that allow changes to that directory while continuing to 'harden' permissions everywhere else.
Comment: credit to @robb-davis for chmod improvement included in this answer
Comment: The **skip_permissions_hardening** approach didn't work for me, while changing permissions did make it work as expected. I've used the workaround provided in the  [documentation](https://www.drupal.org/docs/develop/using-composer/starting-a-site-using-drupal-composer-project-templates#s-full-error). Nor sure why group write permission is needed as owner might generally be the user running update routines on the local system. A corresponding [issue](https://www.drupal.org/project/drupal/issues/3091285) also mentioned the **file-mapping** approach, though I'm not sure what happens with changes.
Comment: I'm having the same problem on my remote server I get the message "Could not delete default.services.yml". But you are talking about a local change, how will this affect my remote problem? Or doesn't it and should I do something else?
Comment: hi @Joost - since I avoid making changes directly on a remote server (and I believe most of the drupal community feels similarly), I may not be familiar with your need. Can you make changes on a local copy then deploy those changes to your remote?
Answer: This command will fix it temporarily:
chmod u+w web/sites/default

But Drupal will harden permission again soon. To turn it off add this to settings.php:
$settings['skip_permissions_hardening'] = TRUE;

Source: https://www.drupal.org/docs/develop/using-composer/starting-a-site-using-drupal-composer-project-templates#s-troubleshooting-permission-issues-prevent-running-composer
Comment: This answer adds some useful details - thanks for posting it. That said, naturally it doesn't seem like you'd want simply to turn off hardening as part of your generally applied solution. How about updating the answer to specify doing this in settings.local.php?
Comment: How about to discuss axiom that permission hardening has some sence? What are realistic cases where it protects something?
Comment: Having seen how hard the Drupal community works at determining and implementing best available security settings, I believe default features are good to preserve unless I have extensive expertise in the alternatives. Since much of Drupal QA assumes defaults, odds are even higher there's a big advantage in maintaining them. Whether this particular setting is extraneous, @bobik, I'm not qualified to take on, tho I certainly hear your point. Thanks for the follow-on discussion!
Comment: Good point, usually I use the same argument. But if I see that anywhere in docs the reason is not properly described I am skeptical.

In this case it seems also like some common created in ages of primitively secured shared hostings.

I also can imagine a lot of attack vectors what will work even with hardening and none what hardening can stop on current good hosting. And if someone use some primitively secured hosting than this hardening is probably his less important problem.
Comment: This seems to be a good answer, though it seems to mention that permission fix will only be a temporary one. From what I've just experienced with a local Drupal 9 that had similar issues is that `sides/default` is missing the **write** permission for the owner, while contained files like `default.services.yml` seem to have it set. Just changing the permission for the local system should work just fine, with compose being able to make all the necessary changes. If that change is not welcome for the live environment, then an update script could add that temporarily and then remove.
Answer: I think you need to use
chmod ug+w web/sites/default

Without the g (group write permission) twig can't create it's tmp directories in files/php/twig and the site will throw a warning.
Comment: Not sure why would the group permission be needed and it might be related to ownership of the files and directories. If the owner and the user running updates is the same, then group permissions shouldn't be required. Maybe it's worth mentioning.
Answer: The issue is due to write permission. I resolved it by running the following commands.
chmod u+w sites/default
composer update
composer require drupal/mail_safety


Python Voting Calculator

Question: I am trying to make a calculator that determines whether the user can vote after entering their date of birth. I have tried using some other posts to get where I am currently but I am now stuck.
I have this error:
TypeError: unorderable types: datetime.timedelta() >= int()

And my code is:
from datetime import date

print("Please enter your birthday")
bd_y = input("Year: ")
bd_m = input("Month (1-12): ")
bd_d = input("Date: ")

now = date.today()

birthdate = date(int(bd_y), int(bd_m), int(bd_d))

age =  now-birthdate

print("Your age is", age)

if age >= 6570:
    print("You can vote!")
else:
    print("Your not old enough! Sorry!")

Any help you can give would be great, thanks!
Answer: age is a Python datetime.timedelta and cannot be compared with an int directly.
age.days is the integer representation of the number of days. This can be compared to another integer.
It's worth noting that this does not include nor account for leap years, nor does it actually determine whether you can vote, as it takes the current time, rather than your age at a certain cutoff date. Additional avenues for improvement.
Answer: Try this:
    from datetime import date
print("Please enter your birthday")
bd_y = input("Year: ")
bd_m = input("Month (1-12): ")
bd_d = input("Date: ")

now = date.today()

birthdate = date(int(bd_y), int(bd_m), int(bd_d))

age = now-birthdate

print("Your age is", age)

if age.days >= 6570:
    print("You can vote!")
else:
    print("Your not old enough! Sorry!")

Answer: age is an object:
In [256]: age
Out[256]: datetime.timedelta(10188)

To get the total seconds, for example:
In [264]: age.total_seconds()
Out[264]: 880243200.0

To get the days:
In [269]: age.days
Out[269]: 10188

To see the attributes and methods of an object like age just do dir(age)
Answer: Why dont you make things simple and just subtract the age(year only) from the current year? See below how to get current year only:
import datetime

print("Please enter your birthday")
bd_y = input("Year: ")
bd_m = input("Month (1-12): ")
bd_d = input("Date: ")

currentyear = datetime.date.today().year

birthdate = bd_y

age =  currentyear - int(bd_y)

print("Your age is", age)

if age >= 6570:
    print("You can vote!")
else:
    print("Your not old enough! Sorry!")

Answer: The issue is your variable age is not simply the integer number of days. Please refer to here to learn more about the datatype.
The solution to your problem is choosing the correct attribute. Long story short:
if age.days >= 6570:
should resolve your immediate issue. Also it would be good practice to use variables to represent 6570 or at least make it clear what that number is (18 years * 365 days) in some way.
Comment: `type(age.days) = `, so typecasting is not necessary.

Python Voting Calculator


How do I know from which page the users are coming?

Question: I'm looking for a way to determine where the users are coming from to build the breadcrumb for a page.
Is there something I can do with hook_breadcrumb_alter(&$active_trail, $item), or should I use the Flag module or any activity tracker module?
Comment: you can use php for that : if(isset($_SERVER['HTTP_REFERER'])) {
    $previous = $_SERVER['HTTP_REFERER'];
}
that might not work on HTTPS
Answer: If you have control over the Urls linking to your target page, append a page-specific parameter. Suppose the page you are linking to is http://www.example.com/target-page:

On the first referring page, make this Url: http://www.example.com/target-page?referer=page-1
On the second referring page, make this Url: http://www.example.com/target-page?referer=page-2

Then in your hook_menu_breadcrumb_alter function, the referer will be available in the $_GET variable, and you will be able to do something like this in the template.php file for your theme:
function mytheme_breadcrumb(&$variables) {

  if (request_uri() == '/target-page') {
    switch($_GET['referer']) {
      case 'page-1':
        $variables['breadcrumb'][] = l('First referring page', '/page-1');
        break;
      case 'page-2':
        $variables['breadcrumb'][] = l('Second referring page', '/page-2');
        break;
    }
  }
}

If you don't want to add parameters to the Url, and are not using HTTPS, you could use the approach above except with $_SERVER['HTTP_REFERER'] in the switch statement. Or alternatively, you could set a cookie on the referring pages and read it on the target page.
The example above is probably the easiest way to manipulate breadcrumbs but strictly speaking, there should not be business logic in a theme function. See these questions for more detailed info on how to manipulate breadcrumbs:

Altering the breadcrumb contents
Move user profiles under a menu item
https://stackoverflow.com/questions/10087168/how-to-add-breadcrumb


Migration java8 and WLS 12c to Java11 and WLS 14c

Question: I have a java application that serves Soap xml web services, in the present case it is java8 app and running on weblogic 12c. I decided to upgrade it java11 and also weblogic 14c. As far as I read from Oracle's documentations Oracle removed jaxws and jaxb from JDK11 permanently.Based on a few articles I've read, all I have to do in pom.xml before migration;

Set java version => 11
Set maven-compiler-plugin version min => 3.8.0
Put jaxws and jaxb dependencies explicitly

I have done those things and I have added dependencies like
<code>        <dependency>
            <groupId>com.sun.xml.ws</groupId>
            <artifactId>jaxws-ri</artifactId>
            <version>2.3.1</version>
            <type>pom</type>
            <exclusions>
                <exclusion>
                    <groupId>org.glassfish.jaxb</groupId>
                    <artifactId>jaxb-runtime</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.sun.xml.bind</groupId>
                    <artifactId>jaxb-xjc</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.sun.xml.bind</groupId>
                    <artifactId>jaxb-jxc</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.eclipse.persistence</groupId>
            <artifactId>eclipselink</artifactId>
            <version>2.7.6</version>
        </dependency>
</code>
I have removed old jaxb implementation and added MOXY implementation. Every thing is fine until deployment to the wls 14c, a successfully compilation, I can see the jars in my war file;

jaxb-api-2.3.1.jar
jaxws-api-2.3.1.jar
jaxws-rt-2.3.1.jar
eclipselink-2.7.6.jar

When I try to deploy war file to wls 14c, I get weird stack trace error on wsl 14c console logs;
<code>weblogic.application.ModuleException: java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    com/sun/xml/ws/db/glassfish/JAXBRIContextFactory.getContext(Ljavax/xml/bind/Marshaller;)Lcom/sun/xml/ws/spi/db/BindingContext; @8: invokevirtual
  Reason:
    Type 'com/sun/xml/bind/v2/runtime/JAXBContextImpl' (current frame, stack[1]) is not assignable to 'javax/xml/bind/JAXBContext'
  Current Frame:
    bci: @8
    flags: { }
    locals: { 'com/sun/xml/ws/db/glassfish/JAXBRIContextFactory', 'javax/xml/bind/Marshaller' }
    stack: { 'com/sun/xml/ws/db/glassfish/JAXBRIContextFactory', 'com/sun/xml/bind/v2/runtime/JAXBContextImpl' }
  Bytecode:
    0000000: 2a2b c000 34b6 0035 b600 36b0

    at weblogic.application.internal.ExtensibleModuleWrapper.start(ExtensibleModuleWrapper.java:140)
    at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:124)
    at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:233)
    at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:228)
    at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:45)
    Truncated. see log file for complete stacktrace
</code>
I couldn't understand the error message, what am I missing ? Could you please help.
Answer: I found the solution, if you are running your app on wls14c, the only thing you have to do is adddin JAX-RT dependency with provided scope. Because in java11, jax-ws and jaxb are removed but not in wls14c.
<code>        <dependency>
            <groupId>com.sun.xml.ws</groupId>
            <artifactId>jaxws-rt</artifactId>
            <version>2.3.3</version>
            <scope>provided</scope>
        </dependency>
</code>
Comment: This is covered in https://docs.oracle.com/en/middleware/standalone/weblogic-server/188.8.131.52/wlupg/compat.html#GUID-575BFB76-FA68-44C8-A628-65616CB4A1EF which says:

The following Java EE classes that were in the Java SE runtime have been removed in Java SE 11: JAXB, JAX-WS/JWS/SOAP, JTA, Activation Framework, JSR 250 common annotation, CORBA.

New jar files have been added to the classpath to account for this.


How can I show tweets on a map in Processing?

Question: I have been trying to use a map to show locations of tweets (from Streaming API) in Processing. But I haven't found any good resource/tutorial. Will anyone help me please?
Comment: Please see my answer to your other question. And in the future, please edit your original question (and/or topic) rather than posting duplicate questions.
Comment: Also, if you try to give more elaborate questions that show research, links, source code, etc. people will be more likely to help you. FYI.
Answer: The best library for maps in Processing currently is Unfolding. You will find tutorials on their site.
Answer: Jer Thorp has done a fair bit of this... he has several tutorials and source code links on his site: http://blog.blprnt.com/source-code-tutorial
Comment: @Fari was this (or any other) answer useful ? Please consider upvoting one or more, and accepting one...
Answer: you might be interested in checking out my work on tweetedtrips.com.
This uses the streaming api to pull tweets, geocode those which don't already have location information attached to them and plot them on a Google Map.
Peter


How to increase fit of high values (generalised linear mixed model)?

Question: My dataset consists of ~800 observations. This is the distribution of my response variable:

I'd like to model the response with a generalised linear mixed model with 1 random effect and 8 fixed effects. I've tried different families and links but in any case when I compare observed vs. fitted values I can see that the few very high values of the response variable are modelled poorly.
Model fit with gaussian family and log link (best of all I tried):

What can I try to increase the fit?
Comment: Have you also tried non-linear associations (transforming variables [predictor or outcome] to for example, log scale; or fractional polynomials; or [restricted] cubic splines)?
Comment: On the residual plot, which axis are the predictions?
Comment: @IWS I tried transforming the response variable to no avail
Comment: I would recommend you to look at the non-linear association types I mentioned in my previous comment. This concerns the predictor variables (independent variables) instead of the response (==outcome==dependent variable).
Answer: Under your assumptions, it seems the data has a couple of data points that have a higher outcome than the model expects. You hope that after a logarithmic transformation the errors look Gaussian, but they do not.
You can think about what's causing the outliers. Perhaps there is something that's causing these? A nonlinear effect in one of the predictors? Or a missing predictor?
If you are confident the linear model is correct, you can try to wrestle the errors into a Gaussian. You mention you have used the log transform, that's a common one, but you can also try a couple of other transformations from the Box-Cox family, I've googled this link: https://www.isixsigma.com/tools-templates/normality/making-data-normal-using-box-cox-power-transformation/.


Syntax error while creating table in PostgreSQL 8.1

Question: I have a syntax error in this query:
CREATE TABLE test (LIKE original_table INCLUDING INDEXES);

ERROR : syntax error at or near "INDEXES"

I am using PostgreSQL 8.1 version. Where is the issue in my query?
Comment: Why you are you using such an outdated version? You should _really_ upgrade to a version that is under maintenance.
Answer: PostgreSQL 8.1 only supports INCLUDING DEFAULTS.
You'll either have to upgrade to at least 8.3 or create the indices manually.


Construct an object of fitobject (cfit) with a vector of parameters in MATLAB

Question: Is there a possibility to construct a cfit-object with a vector of parameters like:
param = [17,-77,95,112];
ft = fittype('poly3');
result = cfit(ft,param);

instead of:
ft = fittype('poly3');
result = cfit(ft,17,-77,95,112);

Thanks in advance!
Answer: use the comma-separated list operator : together with cell arrays::
param = {17,-77,95,112};
result = cfit(ft,param{:});

you can get param in cell form with mat2cell.
Comment: That was one of my ideas before I asked here. This does not work here.
Comment: OK I found the answer:

    param = {17,-77,95,112};
    result = cfit(ft,param{:});

Thanks!
Comment: I couldn't test it, I edited my answer. Thanks for correction!


How would I approach writing a function for the volume of this rectangular prism inscribed in a rectangular pyramid?

Question:
Hi all,
The answer is supposed to be v(x)=x(20-2x)^2.
I don't understand how to derive the (20-2x) in this equation.
Can anyone explain where this comes from?
Thanks.
Answer: Hint. The top plane of the prism creates a pyramid similar to the whole pyramid. You can find the height of that pyramid and then the side of its square base.
Comment: Thanks so much! I understand the problem now


MoveMouseAction is not working in Microsoft WebDriver (for Edge browser)

Question: MoveToElement action method is not working when trying to run the tests with Edge browser using Microsoft Webdriver.
Code snippet:
<code>new Actions(driver).moveToElement(driver.findElement(By.linkText("menu"))).perform();
</code>
Observed below exception:

org.openqa.selenium.UnsupportedCommandException: Unknown command received
  Command duration or timeout: 10 milliseconds
  Build info: version: '2.47.1', revision: 'unknown', time: '2015-07-30 11:02:44'
  System info: host: 'DESKTOP-R1AE8BA', ip: '10.40.14.9', os.name: 'Windows 8.1', os.arch: 'amd64', os.version: '6.3', java.version: '1.8.0_51'
  Driver info: org.openqa.selenium.edge.EdgeDriver
  Capabilities [{acceptSslCerts=true, browserVersion=20.10240.16384.0, platformVersion=10, browserName=MicrosoftEdge, takesScreenshot=true, pageLoadStrategy=normal, takesElementScreenshot=true, platformName=windows, platform=ANY}]
  Session ID: BC49145A-C839-49D7-890C-D06A13E904D8
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
      at java.lang.reflect.Constructor.newInstance(Unknown Source)
      at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206)
      at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158)
      at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:595)
      at org.openqa.selenium.remote.RemoteExecuteMethod.execute(RemoteExecuteMethod.java:35)
      at org.openqa.selenium.remote.RemoteMouse.mouseMove(RemoteMouse.java:89)
      at org.openqa.selenium.interactions.MoveMouseAction.perform(MoveMouseAction.java:39)
      at org.openqa.selenium.interactions.CompositeAction.perform(CompositeAction.java:50)
      at org.openqa.selenium.interactions.Actions.perform(Actions.java:371)

As per Microsoft blogs, all the functions are not yet implemented, List of commands implemented in Edge, Full list of commands
Is there any alternative way to perform mouse over operations?
Comment: Unfortunately, you've already found the answer. If it's not supported by the driver, there's no way to do it.
Answer: moveTo is not implemented yet, along with other methods such as "submit"
More details on the WebDriver implementation status can be found here: https://dev.modern.ie/platform/status/webdriver/details/


OS X Boot DVD (for imaging and recovery)

Question: It is possible to create a Boot CD/DVD for Mac OS X?
What I would like would be a disc I can put in and boot up into something with a few utilities.  We often, for example, have NetRestore NetInstall sets that you can boot up into over the network, and then you have a hugely stripped down system; it has Terminal and some CLI utilities, it has Disk Utility, and it has NetRestore (which is wonderful for imaging, even if it has been discontinued).  Could I make a CD that does that?
Some notes

This would be useful for occassions on which I don't have a netboot server running, and don't have a properly formatted external hard drive to boot off of.
The OS X install DVDs do most of what I want, but it would be nice to run NetRestore.  (Disk Utility works in a pinch).
There is a utility called "Boot CD" (scroll to the bottom of the page) but it doesn't work with Tiger or Leopard.
It would be nice to be able to customize the software on the disc.
Comment: FWIW, NetRestore was discontinued as a third party product as it is now a new feature of Mac OS X 10.6 Server (Snow Leopard Server).
Comment: Seriously?  That's great!
Answer: I still think a stripped down install on a external hard drive (I use a nice one that uses firewire or USB, and has no extra power cord...) or USB key would be superior, but it is possible to have a custom boot DVD.
Start with the install DVDs - these boot a version of the OS that doesn't assume a writable filesystem.
Make a new, writable disk image (CD/DVD Master) from the DVD using Disk Utility.
Mount the image and prune it down as much as possible to make room for your custom apps.
Remove the XCode installer and other extras from /System/Installation/Packages. Keep at least the file MacOSinstall.pkg.
Drop the .app for any custom programs you need in the root folder, or in an Applications folder.
Umount the image, and burn to a new DVD.
Comment: You can actually delete the entire /System/Installation/Packages folder; couple that with deleting /Optional Installs (and /Instructions and /Install Mac OS X.app) means the whole thing only takes about 1.4GB (down from 7.1GB).  I did copy over /usr/bin/open and /System/Library/CoreServices/Finder.app, but wasn't able to get the Finder to run.
Comment: Sounds great.  What do you do to make a bootable USB key?
Comment: http://blog.bradbergeron.com/2006/11/howto-install-and-boot-os-x-on-a-flash-drive/ looks like it covers installing onto a USB key.  I haven't tried your answer yet, but is sure looks right so I'm going to accept it.  Thank you!


hook_path_update (and insert and delete) not getting called

Question: I've added the following methods to my module, saved, and flushed all caches on my site. I have double checked that my module is enabled and working properly in all other respects.
<code>function xxxxxxx_path_insert($path) {
  kpr($path);
  var_dump($path);
}

function xxxxxxx_path_update($path) {
  kpr($path);
  var_dump($path);
}

function xxxxxxx_path_delete($path) {
  kpr($path);
  var_dump($path);
}
</code>
I want to receive notifications when content path aliases are updated so that I can store some info about them for my custom module. I just put these shell methods in place to ensure I was getting the hook calls and to double check the structure of the $path variable.
Then I edited an existing piece of content. I changed the title and saved the node. Since I'm also using pathauto, that module updated the content path alias to point to a new url based on the new title. I did not receive any of the calls.
Ok, so I figured maybe the auto generation of the alias was the problem, so I edited the piece of content, disabled the auto generation of the path alias and changed the path alias to something by hand. Then saved the node. I did not receive any of the calls to the above hooks.
What am I doing wrong? Why aren't these hooks getting called?
The hook documentation for your reference: path.api.php
Answer: sigh I flushed all caches a TON of times and it would never call my hooks. I had to completely disable my module, flush all caches, and then re-enable my module.
Then the system saw my hooks.
sigh


Inject ngMaterial using webpack, angular fullstack generator

Question: I want to build a web application and I have started building this by using the angular fullstack generator. This generator uses webpack and this is the first time I am using webpack.
I have been trying to add the ngMaterial dependency by installing it with:
npm install angular-material --save
I have added the dependency in the config.entry.vendor from the webpack.make.js file but when I tried to import the ngMaterial i received the following error:
ERROR in ./client/app/app.js
Module not found: Error: Cannot resolve module 'ngMaterial' in path\client\app
 @ ./client/app/app.js 21:18-39

How can I inject a dependency with webpack?
Answer: I assume you pointed incorrect dependency name in webpack configuration: ngMaterial instead of angular-material.
ngMaterial is angularjs module name, you point it as a dependency in your angular app module, like this:
angular.module('app', ['ngMaterial'])

But in webpack dependencies you have to point node.js module name, which is angular-material (it is located in node_modules folder after you install it via npm install command)
EDIT:
If needed in your app.js you also have to import that module via import 'angular-material' or require('angular-material'). But if you include this dependency in separate vendor.js file (via webpack), than you shouldn't ever include this module in your app.js, cause it will upload to the page in vendor.js. You'll include it 2 times in your project if you do so.
Comment: Thanks for your explanations, I solve the problem. Can you tell me where I can find more documentation about webpack so I can document myself on this subject? Thanks
Comment: Here are good youtube playlists of [webpack lessons](https://www.youtube.com/channel/UCblk3IlXm_ZXkR5OYLuYWaQ/playlists) by Matthew Hsiung. Basics and some advanced config examples.


error while passing the backend URL as a parameter in template.json file at

Question: I am trying to deploy the APIM instance by using ARM templates. So I want to pass the backend URL as a parameter. So for that I have added a parameter "backend_url" in parameters.json file and provided the URL as a value. Also I have added the same parameter in template.json file.
In the template.json file at the <set-backend-service base-url> element I am passing the parameter as below:
<set-backend-service base-url=\"(parameters('backend_url'))\"/>
When I deploy the templates getting

"Error in element 'set-backend-service' on line 76, column 6: Value is not a valid absolute URL." error.

So please let me know how to pass a parameter in <set-backend-service base-url> element while deploying APIM instance using ARM templates.
Comment:
Comment: Are you trying to define the backend api url policies from the ARM template?
Comment: yes..@SillyJohn
Answer: Im doing the same thing at the moment! There are two things to do:
1 - Define the parameter type you are using in the template file
2 - When defining params you need to signify the the start and end of the command using square brackets.
Try this:
<set-backend-service base-url=\"[parameters('backend_url')]\"/>

MS Resource:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-parameters


SwiftUI - How to perfectly align multiple Text with different font sizes in VStack?

Question:
I have 2 Text elements in a VStack. They are set to different font size.
How do I get the first glyph of the 2 Text elements to align perfectly left? Is that even possible?
Below is the code snippet:
extension Font {
    static let countDownTitle: Font = Font.system(size: 46, weight: .regular, design: .rounded).leading(.tight)
}

struct MyView: View {
    var body: some View {
        VStack(alignment: .leading) {
            Text("Hello!!")
            Text("20.49").font(.countDownTitle)
        }
     }
}

Answer: You have a bit of an artificial test going on here. Remember, each character takes up a different amount of room in a proportional font takes up a different amount of room. When they are laid out, they are put in a space that is controlled by the font designer, not us. You can see this is you cycle through different numbers. The 4 is pretty much right on, but the 5 is way off. This is one of those things that your ability to control it is lacking.
While, I am not at all recommending this, you could get perfect alignment like this, using a monospaced font and an .offset():
extension Font {
    static let countDownTitle: Font = Font(UIFont.monospacedSystemFont(ofSize: 46, weight: .regular))
}

struct MyView: View {
    var body: some View {
        VStack(alignment: .leading) {
            Text("Hello!!")
            Text("50.49").font(.countDownTitle)
                .offset(x: -2.2, y: 0)
        }
     }
}

But, this is really ugly.
Comment: Appreciate your help @yrb. Agree, it's ugly. But it seems like `.offset()` or `.alignmentGuide()` are the only solution. I tried out `Text.tracking()` but that's not helping.
Comment: `.offset()" is only going to work on monospaced fonts. On a proportional, unless you have an individual offset for each digit, they will be off somewhere.


Which consistency model do I have?

Question: I have implemented a replicated key/value store on top of Redis. I have passive replication in which all write and read requests are forwarded to the leader that always returns the last value written for the key. The system uses quorum. So it works even if there are nodes which are down or with a network partition. In this case, the value in those nodes are not consistent. But this does not prevent the system to return the last most updated value. Do I have an eventual consistency model or a strict one? Thanks
Answer: You mentioned that, it is a quorum based system, with one node as a leader. The read and write requests are always forwarded to the leader.
For the sake of simplicity, let's assume that, there are 5 nodes in the system and one of them is a leader. Other 4 nodes are secondaries.
Typically quorum based systems work on consensus protocols. So out of 5 nodes, if 3 of the nodes have the latest value, it is enough to always return the latest value.
This is how writes should work

Leader first updates the key/value in it's database
Forwards the request to remaining 4 nodes, which are secondaries
The leader waits for at least 2 of the secondaries to acknowledge that, they have updated the latest key/value in their database. It means out of 5 nodes, at least 3 nodes have the latest updated value.
If the leader does not get response from at least 2 of the secondary nodes within the specified time period (request times out), then leader returns failure to client and client needs to retry.

So, the write requests succeeds only if 3 out of 5 nodes have the latest value. At any point, 2 of the nodes may or may not have the latest value and may catch up later.
For the reads, the leader (which has the latest key/value), always returns the response.
What happens when a leader machine is unable to serve requests, due to some issue? (For e.g. network error)

Typically these systems will have a leader election protocol, when the current leader is unable to serve the requests due to some error.
The new leader will be chosen from one of the secondaries, which have the latest updates. So, the newly elected leader should have the latest updated state and should start serving read requests with the latest set of values.

Your system is strictly consistent.

Which consistency model do I have?


How to parse multiple tables using BeautifulSoup and save them to a csv file

Question: I've got a program, almost finished, it is only missing this last part where I'm struggliing. I need to scrap from a lot of webpages (If you need to see an example you need to go on this site http://www.pa.org.mt/page.aspx?n=63C70E73&CaseType=PA and fill the case number with 03732 and case year with 16 and click on the first submit.) the tables on the contentHolder div and write them to a csv file, to get something like this:
Case Status, Status Available, Case Number, PA/03732/16 Location of development: 40 ....
Something like this to all the tables on a webpage and for a lot of webpages.
I wrote some code trying to do this, but it is not working, when I run it, it makes this output on the csv file: https://gyazo.com/6557ac08ad5613a24b5432bfd9e4f2e6 And it doesn't even finishes all the pages because It returns an error in the middle:
Traceback (most recent call last):
  File "C:\PROJECT\pdfs\converterpluspa.py", line 93, in <module>
    csv.writer(f).writerow(answer)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 0: ordinal not in range(128)

Heres's my program's entire code so far:
import shlex
import subprocess
import os
import platform
from bs4 import BeautifulSoup
import re
import csv
import pickle
import requests
from robobrowser import RoboBrowser
import codecs

def rename_files():
    file_list = os.listdir(r"C:\\PROJECT\\pdfs")
    print(file_list)
    saved_path = os.getcwd()
    print('Current working directory is '+saved_path)
    os.chdir(r'C:\\PROJECT\\pdfs')
    for file_name in file_list:
        os.rename(file_name, file_name.translate(None, " "))
    os.chdir(saved_path)
rename_files()

def run(command):
     if platform.system() != 'Windows':
         args = shlex.split(command)
    else:
        args = command
    s = subprocess.Popen(args,
                          stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE)
    output, errors = s.communicate()
    return s.returncode == 0, output, errors

# Change this to your PDF file base directory
base_directory = 'C:\\PROJECT\\pdfs'
if not os.path.isdir(base_directory):
    print "%s is not a directory" % base_directory
    exit(1)
 # Change this to your pdf2htmlEX executable location
bin_path = 'C:\\Python27\\pdfminer-20140328\\tools\\pdf2txt.py'
if not os.path.isfile(bin_path):
    print "Could not find %s" % bin_path
    exit(1)
for dir_path, dir_name_list, file_name_list in os.walk(base_directory):
    for file_name in file_name_list:
        # If this is not a PDF file
        if not file_name.endswith('.pdf'):
            # Skip it
            continue
        file_path = os.path.join(dir_path, file_name)
        # Convert your PDF to HTML here
        args = (bin_path, file_name, file_path)
        success, output, errors = run("python %s -o %s.html %s " %args)
        if not success:
            print "Could not convert %s to HTML" % file_path
            print "%s" % errors
htmls_path = 'C:\\PROJECT'
with open ('score.csv', 'w') as f:
    writer = csv.writer(f)
    for dir_path, dir_name_list, file_name_list in os.walk(htmls_path):
        for file_name in file_name_list:
            if not file_name.endswith('.html'):
                continue
            with open(file_name) as markup:
                soup = BeautifulSoup(markup.read())
                text = soup.get_text()
                match = re.findall("PA/(\S*)", text)#To remove the names that appear, just remove the last (\S*), to add them is just add the (\S*), before it there was a \s*
                print(match)
                writer.writerow(match)
                 for item in match:
                    data = item.split('/')
                    case_number = data[0]
                    case_year = data[1]

                browser = RoboBrowser()
                browser.open('http://www.pa.org.mt/page.aspx?n=63C70E73&CaseType=PA')
                form = browser.get_forms()[0]  # Get the first form on the page
                form['ctl00$PageContent$ContentControl$ctl00$txtCaseNo'].value = case_number
                form['ctl00$PageContent$ContentControl$ctl00$txtCaseYear'].value = case_year

                browser.submit_form(form, submit=form['ctl00$PageContent$ContentControl$ctl00$btnSubmit'])

                # Use BeautifulSoup to parse this data
                answer = browser.response.text
                print(answer)
                soup = BeautifulSoup(answer)
                #print soup.prettify()
                status = soup.select('#Table1')
                print (status)
                with codecs.open('file_output.csv', 'a', encoding ='utf-8') as f:
                  for tag in soup.select("#Table1"):
                    csv.writer(f).writerow(answer)

EDIT: I tried to change the last line to csv.writer(f).writerow(answer.encode("utf-8")) and it didn't worked, it printed another error message :
Traceback (most recent call last):
  File "C:\PROJECT\pdfs\converterpluspa.py", line 93, in <module>
    csv.writer(f).writerow(answer.encode("utf-8"))
  File "C:\Python27\lib\codecs.py", line 706, in write
    return self.writer.write(data)
  File "C:\Python27\lib\codecs.py", line 369, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 25496: ordinal not in range(128)

And nothing channged on the final csv file.
Answer: You need to encode your output using UTF-8. Change the last line to:
csv.writer(f, encoding="utf-8").writerow(answer.encode("utf-8"))

Also change the import from import csv to import unicodecsv as csv
Comment: I tried it and it didn't worked I updated my question so you can see better what gone wrong
Comment: I've updated my answer to reflect the error that you've posted. Please give it a try now.
Comment: Now it says that unicodecsv isn't a module, it doesn't exist, keepin mind i'm using python 2.7


Partial sum of Stirling numbers of first kind

Question: After this question, that introduced me to Stirling numbers, I have the following question.
For the (signed) Stirling numbers of first kind $s(k,i)$ and $k>M$, I would like to know the value or a good bound depending on $M$ and $k$ of
$$ \Delta(k,M):= \sum_{i=0}^M s(k,i).$$
Equivalently, for the unsigned version of the Stirling numbers,
$$ \Delta(k,M)= \sum_{i=0}^M (-1)^{k-i} \biggl[{k \atop i}\biggr].$$
I know that $\Delta(k,k)=0$, from the definition, but other computations seem to require extra knowledge about these numbers.
Comment: There are two variations of the definition of the Stirling numbers of the first kind: Historically the alternating sign version has been favored, but today there is a clear tendency towards the unsigned variant. You might want to point out which version you are referring to. (Personally, I would also modify the question into using the unsigned variant...)
Answer: Let $S_k$ be the set of permutations on $\{1,\dots,k\}$.

Theorem: For all $k\ge 2$, $\Delta(k,m)$ is equal to $(-1)^{k-m}$ times the number of permutations in $S_k$ with exactly $m$ cycles and where $1$ and $2$ are in the same cycle.

Proof: Let $S_k^{m}$ be the set of permutations in $S_k$ with at most $m$ cycles. Since a permutation in $S_k$ with $i$ cycles has a sign of $(-1)^{k-i}$, the sum  $\Delta(k,m)$ is equal to the sum of the signs of all of the permutations in $S_k^{m}$. To simplify this summation, we shall partition $S_k^m$ into pairs and singletons, where the two permutations in each pair have opposite signs. These pairs cancel each other out in the summation, so we can sum over the singletons alone.
For each $\pi \in S_k^m$, we will pair $\pi$ with $(1\;\;2)\pi$, except in the case where $(1\;\;2)\pi$ has more than $m$ cycles. Note that $(1\,\,\,2)\pi$ has the opposite sign of $\pi$. This causes all permutations in $S_k^m$ to be paired up, where the only exceptions are the permutations with exactly $m$ cycles, such that $1$ and $2$ are in the same cycle. Since all of the unpaired permutations have a sign of $(-1)^{k-m}$, the entire sum equals $(-1)^{k-m}$ times the number of such permutations.
$\tag*{$\square$}$
As a corollary, we have
$$
\left|\Delta(k,m)\right| \le {k\brack m}
$$
There may be some combinatorial argument which allows you to get a more precise count on the number of permutations with $m$ cycles such that $1$ and $2$ are in the same cycle, and thus to get a better bound than above. However, the above is the best bound I can give.
Comment: I would like to use this, eventually, in a paper, how should I procede?
Comment: Just click the word "cite" that appears below my answer. For some more info, see https://math.meta.stackexchange.com/q/4259


Maximum likelihood vs generalized method of moments

Question: I am trying to understand how maximum likelihood (MLE) and generalized method of moments (GMM) are related to each other. In particular, I often see people saying that MLE can be written in terms of the GMM or some minimum-distance estimators. I am not sure how this is true in general.
Suppose my parameter of interest is $\theta \in \Theta \subseteq \mathbb{R}^k$, and my log-likelihood is given by $L(\theta) = \frac{1}{n} \sum^n_{i=1} f(x_i; \theta)$. Then, the first-order conditions for MLE is given by the system of $k$ equations
$$
\nabla_\theta L(\theta) = 0_{k}.
$$
Equivalently, this is to solve
$$
\frac{\partial L(\theta)}{\partial \theta_i} = 0
\quad \text{ for all $i = 1, \ldots, k$}.
$$
From the Wikipedia page, it suggests that MLE can be written as a form of GMM using the moment conditions formed by my first-order conditions. In my notation, the moment condition is
$$
E[g(\theta)] := E[\nabla_\theta F(\theta)] = 0_k.
$$
The GMM objective function is given by
$$
\min_{\theta \in \Theta} g(\theta)'W_n g(\theta),
$$
for some weighting matrix $W_n$. I think one way to pick $W_n$ here is to set it as the identity matrix.
Then, the first-order condition of the GMM objective is
$$
\nabla_{\theta} g(\theta) W_n g(\theta) = 0_k.
$$
Substituting back the definition of $g$ above, we have
$$
\nabla_{\theta\theta} L(\theta) W_n \nabla_{\theta} L(\theta) = 0_k.
$$
My question is, how is the GMM estimator related to the MLE estimator? By looking at the first-order conditions of the two estimators, I don't think they are the same unless we impose more conditions. For instance, the second derivative appeared in the GMM first-order condition, whereas only the first derivative appeared in the MLE first-order condition.
Any thoughts are very much appreciated.
Comment: The deep connection between MLE and GMM can be shown properly by looking at theory of extremum estimators in general. For now, it should be enough to note that the quadratic form which you see in the GMM objective is only needed if you have more moment conditions than parameters. Here the moment conditions are $k$ partial derivatives, and the number of parameters is $k$, so the sample analogue of $E[\nabla \log f(x_i, \theta)] = 0$ is exactly the GMM objective. In other words, you are solving $n^{-1}\sum_i \nabla \log f(x_i,\hat{\theta})] = 0$ for $\hat{\theta}$.
Answer: To expand on what I wrote in the comment, suppose we have data $\{X_i\}_{i=1}^n$ which have some distribution $F$ whose density $f_{\theta}(x)$ is known up to some parameters $\theta \in \Theta \subset \mathbb{R}^k.$ Assume that the "true" parameter is some fixed but unknown $\theta_0 \in \Theta$.
Suppose further that you know some function $g:\mathbb{R} \times \mathbb{R}^k \to \mathbb{R}^p$ of your data and unknown parameters such that
$$
\begin{equation}
\mathbb{E}g(X_i, \theta) = \int_{\mathbb{R}}g(x,\theta)f_{\theta_0}(x)dx = 0 \tag{1}
\end{equation}
$$
if and only if $\theta = \theta_0$. This is the point-identification assumption. It's an assumption because while we have a set of $p$ equations in $k$ unknowns, we don't know the equations themselves, since we need $\theta_0$ to compute the integral above. Thus we find the sample analogue of the equations, given by
$$ Q_n(\theta) := n^{-1}\sum_i g(X_i,\theta) = 0. \tag{2}$$
The identification assumption does not imply that equation (2) has a solution; it only lets us assume that equation (1) does. Indeed, what we have here is $p$ equations in $k$ unknowns; for $p < k$ neither (1) nor (2) have unique solutions, since that is an underdetermined system, so our assumption is actually invalid. For $p = k$, we are likely to have unique solutions to both systems, (in the case of linear systems we almost always have solutions, in a technical sense). For $p > k$, we assume that (1) has a unique solution, but (2) may not have a solution at all since it is an overdetermined system. This is where the quadratic form of the moments come in; we can recover estimates by minimizing $ \lVert Q_n(\theta)\rVert_2^2$ which is equivalent to making $Q_n$ as close to zero as possible. Note you can also try to minimize this quadratic form when $p = k$, but if equation (2) has solution $\hat{\theta}$, then $\lVert Q_n(\hat{\theta}) \rVert_2^2 = 0$ as well. This is exactly what is going on with your MLE example, where $g(X_i, \theta) = \nabla \log f(X_i, \theta) $.


Server loses Internet connectivity every hour on the hour

Question: I have a Server 2003 Small Business Server that loses Internet connectivity every hour on the hour, for about 15-20 minutes. I can't figure out what's causing it. The machine still has LAN connectivity, but it loses its Internet connection, and people who are trying to connect remotely (via RRAS) can't get in. Then, after about 15 min., it works again. I checked to see if any services are scheduled to run and are knocking it offline, but there doesn't seem to be anything. Checked Event Viewer and it looks normal. Any ideas as to what this might be? Thanks...
Comment: How is it connecting to the internet - via some kind of router on the LAN or some other method?
Comment: It connects through an AT&T U-verse gateway.
Comment: Are other computers that use the same gateway afected by this problem as well?
Comment: No, it's just the server. Thanks. One thing that just occurred to me is that the gateway is acting as a DHCP server, but so is the server that we're having problems with. Apparently, we need it to do that because we need it to give out WINS addresses, otherwise Outlook/Exchange doesn't work right, and the AT&T gateway doesn't give out WINS addresses. Could the fact that there are two DHCP servers on the network be causing this problem? And if so, how do we handle it? (Apologies if this is not well explained, my business partner is the one who knows more about this stuff but he is out today.)
Comment: Have any cron jobs or scheduled tasks that happen on the hour?
Answer: Without looking at what the AT&T one can do, you should be able to disable the DHCP on it and have the Win server hand out all the IP's.
I actually do it the other way and have my router hand out the IP's, but I can specify the DNS to use with the DHCP so the DNS it has the workstations pointing to is the Win server.  The Win server doesn't have a DHCP on it.  I don't use WINS, don't have the DHCP server on the Windows server running and my exchange clients are fine.  So you should be able to configure you network to work as well.
Answer: Regular intervals smells of DHCP problems. You're getting your IP from an external device, right? I'm no familiar with AT&T policies (different continent..) but sounds like your public IP address somewhat goes 'down'.
I think the problem might not even be on your side..
[updated]
2 dhcp servers DO fight if they are on the same subnet. It works this way: at regular intervals your server sends out a message saying "hi, can I keep my IP?" and both servers answer. Apparently, after some struggle, your server gains its ip back.
Solution:

try limiting the scope of the gateway DHCP server to just one IP (using MAC addresses - but beware when you change network card)

or

(simple solution, might not be your case) use different network interfaces: one for the server-going-out, one for the lan

or

get a nat box/router between the gateway and the lan. Have the nat box grab the ip from the gateway. Do port mapping to tunnel connection through nat box to server.

Your mileage may vary but please do not put two dhcp on the same subnet! :)
Comment: Yes, getting the IP from the AT&T device, but the server itself is also acting as a DHCP server. (See above for why.) Apparently, the way this AT&T U-verse thing works, it has to assign an address to the server via DHCP; we can't just set it to a static IP.


Mikrotik - Add second WAN IP and route specific traffic

Question: After 20 pages of SO results about Mikrotik and some more google results, I'm come here, down on my knees to request some enlightment.
I have a network with static IP and some public IP (248 mask).
So far I've configured the network to use one of those public IP and use it for 2 subnets (192.168.85.X and 192.168.5.X) wich are isolated from each other and both can access internet.
What I'm trying to achieve is to add a second WAN ip to the same interface, and route traffic to a specific server to use that IP.
What i mean is:
Any PC from 192.168.X.X should use WAN IP1 and use internet with it.
Any PC from 192.168.X.X trying to access INSERT RANDOM SITE/IP HERE should use WAN IP2 to browse and navigate to that website (while all the rest of the traffic goes through WAN IP1).
The device that gives me the WAN link only has 1 "out" port, so there is no way to put 2 cables from the "modem" to the Mikrotik, right now there is 1 cable going from the modem to the Mikrotik device. I've seen some forum post where the first part can be solved by just connecting 2 cables and then assign different IP for each interface.
As you may notice, I'm really raw in networking and routing, so any GUI/Winbox instruction is appreciated, but CLI commands would be just fine.
Answer: You should be able do this by using src-nat with destination address lists and setting to-address to the desired ip. for the default ip, copy the rule and don't set a dest. address list. For dns names you'll want to create a script to run using the scheduler which will update an address-list with the ip addresses the website uses. The firewall rules work using IP addresss rather than dns names.
A somewhat similar example would be 1 router with multiple subnets and different IP addresses per subnet but rather than use the src-address in the nat rule, you're basing it of the dst-address.
The info below should help get your on your way to what you're looking to do.
/ip address
add address=188.8.131.52/29 interface=ether1-gateway
add address=22.214.171.124/29 interface=ether1-gateway

/ip route
add dst-address=0.0.0.0/0 gateway=184.108.40.206/29

/ip firewall address-list
add address x.x.x.x list=alt-ip-list
add address y.y.y.y list=alt-ip-list
add address z.z.z.z list=alt-ip-list

/ip firewall nat
add action=src-nat chain=srcnat dst-address-list=alt-ip-list \
out-interface=ether1-gateway to-addresses=184.108.40.206
add action=src-nat chain=srcnat out-interface=ether1-gateway to-addresses=18.104.22.168

You can use src-nat instead on masquerade when you need to specify the external ip address. You'll need to make sure that the ip address you specify is set on the outbound interface. This is called IP aliasing.
For the script to update the address list you'll need a create one that can lookup the ip addresses used by whatever website you want to have use the alternate ip.
For facebook addresses, this would be the linux command for ipv4 addresses.
whois -h whois.radb.net -- '-i origin AS32934' | grep ^route:
For IPV6 you would use.
whois -h whois.radb.net -- '-i origin AS32934' | grep ^route6:


How to access Vertical Tabs from Contrib Modules in Panels?

Question: Drupal node forms have a vertical tabs area, which provides options such as "Revision Information", "Authoring Information", "Publishing Options" etc.
Contrib modules such as Node Hierarchy and XML sitemap can add custom tabs to this area.
How do you access these tabs from contrib modules in Panels.
In the past, with Drupal 7 & Panels, you could access them via the "Form" section of "Panels > Add Content".
However, I am creating a new Drupal 7 site and I can no longer access Vertical Tabs from Contrib modules. Only the tabs from Code Drupal modules are shown.
Steps I have tried to overcome the problem:
1) I have made a fresh install with just Panels and Node Hierarchy.
I have made a second fresh install with just Panels and Xml Sitemap.
In both cases, Vertical Tabs from Contrib modules are not accessible.
2) I have tried installing Node Vertical Tab Elements. It too only gives access to core tabs (and not tabs added by Contrib modules).
3) I have tried upgrading my older Drupal 7 sites to the latest versions of Drupal 7. In these older sites, the Vertical Tabs from Contrib modules are still accessible, so I am not sure if this means a special setting is required to access the tabs. I have disabled all custom themes and modules on the site, so I know the tabs are not coming from custom code.
Any help would be most appreciated.
Answer: Sounds like this should work.
In your 1) scenario, where you restricted your installation to the minimum amount of modules -- you might have run into a change/bug with a module version which you intentionaly have upgraded. Try not to test with "most recent versions".
Because you phrased your experience as "in the past you could ...".
Answer: I have figured this out. It is Ctools 7.13 that breaks it (Ctools 7.12 works fine).
I have raised an issue here:
https://drupal.org/node/2147191


How to order the basis vectors that put a matrix in jordan canonical form

Question: I have the following matrix A \begin{pmatrix} 1 & -3 & -1\\ 1 & 5 & 1 \\ -2 & -6 & 0 \end{pmatrix}
I want to find a basis that puts the matrix in Jordan canonical form. The matrix A has eigenvalue 2 with an algebraic multiplicity of 3 and a geometric multiplicity of 2. I choose the first two vectors in my basis to be two eigenvectors,
\begin{pmatrix} 1 \\ 0\\-1\end{pmatrix} \begin{pmatrix} 0 \\ 1\\-3\end{pmatrix} and find the last basis vector by solving the equation $(A-2I)^{2}\textbf{x}=0$, and take $\textbf{x}$ to be the follwing vector
\begin{pmatrix} 0 \\ 0\\1\end{pmatrix}
My problem is that when I construct the Matrix P as \begin{pmatrix} 1 & 0 & 0\\ 0 & 1 & 0 \\ -1 & -3 & 1 \end{pmatrix} and calculate the Jordan Canonical form using J=$P^{-1}AP$ I get that J is \begin{pmatrix} 2 & 0 & -1\\ 0 & 2 & 1 \\ 0 & 0 & 2 \end{pmatrix}. Why is there a -1 in the first row? And does the order I put my basis vectors in the matrix P matter? Thank you!
Answer: no. when everything is integers, I do this:  the minimal polynomial  is $(x-2)^2.$  The genuine eigenvectors make up a subspace. The, I choose  a  column vector $w$ for which $(A-2I)^2 w = 0,$  that one is automatic here, but $(A-2I) w \neq 0.$  Then name $v = (A-2I) w.$   This $v$ is a genuine eigenvector. Finally, pick a genuine eigenvector $u$ that is independent from $v.$    The change of basis matrix $P$ will have columns $u,v,w.$   Our $P$ will have integer elements, but its determinant may not be $\pm 1,$  so $P^{-1}  = \frac{1}{\det P}   \tilde{P}  $  where $ \tilde{P}  $ is another matrix of integers.  Finally $$ \frac{1}{\det P}   \tilde{P}  A  P = J $$  is in Jordan form
When possible, I like  $w = (0,0,1)^T$  (a column)  and that is possible here.
Answer: To have 1 at the right place in J, we must find U, V, W such that
AU=2U, AV=2V and AW=V+2W
i.e. (A-2I)U=0, (A-2I)V=0 and (A-2I)W=V
Without taking care of U, we must find W non-zero in $N((A-2I)^2)$ such that $V := (A-2I)W$ is non-zero (V will automatically be in $N(A-2I)$)
Example: I choose $W=\begin{pmatrix} 0 \\ 0  \\ 1 \end{pmatrix}$ such that $(A-2I)^2 W=0$
and I define
$V:=(A-2I)W=\begin{pmatrix} -1 \\ 1  \\ -2 \end{pmatrix}$


Calculating weight as 1/(stnd error) for weighted regression if stnd error = 0

Question: I'm assisting a colleague with a weighted regression of average length of stay (LOS) measured in days vs. inpatient admit rate in a dataset consisting of inpatient records from 30 hospitals. We've calculated the weight at each data point as the inverse standard error of patient's LOS at the hospital.
Is there a standard procedure for estimating the inverse stnd error weight where the stnd error = 0? In several hospitals with low patient volume the LOS is identical for all patients during our analysis time period, and therefore the stnd error = 0, producing a weight equal to infinity.
We could drop these data points from the regression (or avoid a weighted regression entirely), but in principle it seems there ought to be an accepted technique for calculating weights in special cases where the variance = 0. I haven't had any luck checking my stats textbooks.

Thank you for the advice, Michael and whuber. Most of the total error sum of squares can be attributed to measurement error in my case when I run a simple unweighted least squares regression (RSS=44.5, ESS=168.9, TSS=213.4).
So if I were to construct my own weighting scheme it might entail, at one extreme if variance=0 then the weight = # obs in that hospital, and at the other extreme, if variance=infinity, the weight=0.
Perhaps a handy formula could be weight_i = N_i/(N_i^CV_i), where weight_i = weight for hospital i, N_i = # obs for hospital i, and CV_i = Coefficient of Variation of observed LOS for hospital i?
Comment: This is, at bottom, an issue of identifying and quantifying separate *components of variance.* Your weighting is appropriate when *all* variation can be attributed to the measurement error in LOS. Because you're doing a regression, there will likely be residuals: they will include a separate (independent) variance component. If, eyeballing the scatterplot, it appears the regression residuals will be larger than typical LOS SE's, then you are probably OK not weighting anything. The problem is more challenging otherwise, so first it would be good to do this check! What does it tell you?
Comment: Just checking:  you are weighting by the standard error of the mean for the $i$th observation: $\hat{s_i}/\sqrt{n_i}$, right?  The $\sqrt{n_i}$ factor could be more important than the $\hat{s_i}$ factor
Comment: @AndrewM - Yes, weighting by stnd error, although it's been 3 years so I'd have to search thru my files for the SAS code to be absolutely certain.
Answer: The problem you have is that you are using the estimated standard error in the denominator of the weight.  The population standard deviation is not likely to be 0 in real situations.  I do not think there is a standard way that applies this particular weighting scheme and some other weight when to estimated standard error is 0.  The solution is to take a different weighting scheme.  There is no law that says that you must take the reciprocal of the standard errors as the weights.  Under certain assumptions those would be the optimal weights. But that is not the case here.
Answer: In other cases when I have tricky zeros that make my calculations hard, such as on the diagonal of a matrix I need to invert, I was taught something that was called regularisation: that is, adding a small value uniformly to the offending variable.
For instance, in my case where I had a covariance matrix that I need to invert, Q, I set Qhat = I*q for q = 10e-x for some x and for I the identity matrix of same size as Q. In your case, you could add a small value to the variance.
Comment: That's absolutely correct. The only answer that I can give is to use values that produce results are still good enough by a reasonable metric. The problem is that yes, it does introduce noise and another parameter to tune.
Comment: Although this can work, it would seem that the results might be exquisitely sensitive to the choice of regularization parameter.  (Arbitrarily small values will weight the zero-SD data by arbitrarily large amounts.) That kicks the question down the road: what value of that parameter should be used?
Comment: One could cast this as a hierarchical model, impose a prior on the LOS standard deviations, and use empirical bayes to estimate the shrinkage factor.


Outgoing mail from linux not being delivered

Question: I can't seem to send mail through my php scripts or through the linux console on my Centos 5.5 LAMP server, when the email is addressed to go to a domain that is hosted by my box.
I think it is something to do with the email routing internally, or the DNS servers that the box uses not reporting the correct MX records.
Basically my box doesn't host any mail, it's all hosted on google apps. My name servers are hosted by a 3rd party provider and I am using webmin. Webmin doesn't recognise the settings on the 3rd party provider.
I'm unsure how to fix this. Previously when I had this problem on a cpanel server, I would edit the remotedomains and localdomains files, moving domains from one file to another and it would fix the problem.
What information do I need to provide for anyone to work out what the issue is?
Thanks
Comment: I agree that it has something to do with the local domains. I am a postfix kind of guy though, so I cannot help you on how it works with sendmail.

However, if it detects the domain locally it will not try to route it externally.
Comment: Please give us informations about the "domain that is hosted on your box" and the domains you have configured in google apps. Basically what are the MX records for the "domain that is hosted on your box" ?
Answer: If you run sendmail -bt (test mode for sendmail) and then type $=w you will see that the domains that you have problem with are listed. This is because sendmail believes that the mail addressed to said domains must be delivered locally in the box, instead of sending it to Google.  So what you have to do is to modify your sendmail.mc in two places (or add them if they do not exist).
First in the LOCAL_CONFIG section add a map that looks up the best MX for a domain:
LOCAL_CONFIG
Kbestmx bestmx -T.TMP

Next, in the LOCAL_RULE_0 section where sendmail selects a delivery agent use the map above to route mail to Google:
LOCAL_RULE_0
R $* < @ example.com. > $*           $#esmtp $@ [$(bestmx example.com. $)] $: $1 < @ example.com. > $2

Note 1: replace example.com with the domain in question
Note 2: Do not copy paste the rules in your sendmail.mc. Type them for the left and the right hand side of the rules are not separated with spaces but with tab characters.
Note 3: After you've done editing sendmail.mc, you have to build sendmail.cf and then restart sendmail. I do not know how this is done in CentOS. I run Debian and execute sendmailconfig

You can read a more detailed explanation on the problem in my blog post here.
Comment: I edited my local domains hosts file and removed the domains and added them to the outgoing domains hosts file and it seems to work. Your answer didn't exactly work because my sendmail.mc file doesn't have a local config section or a local rule section. Also it has "dnl" at the start of every line.
Comment: Like I said, if said sections do not exist in your .mc, you can add them. dnl is an m4 directive that instructs to "delete from here until newline" so you can treat it as a comment identifier.
Comment: Its not clear to me where the LHS should be separated from the RHS. I have tried putting a tab in various spots, but none of them worked for me. ???
Comment: LHS: `R $* < @ example.com. > $*`