Concurrent execution of two processes sharing a Tesla K20

Question: I have been experiencing a strange behaviour when I launch 2 instances of a kernel in order to run at the same time while sharing the GPU resources.
I have developed a CUDA kernel which aims to run in a single SM (Multiprocessor) where the threads perform an operation several times (with a loop).
The kernel is prepared to create only a block, therefore to use only one SM.

simple.cu

#include <cuda_runtime.h>
#include <stdlib.h>
#include <stdio.h>
#include <helper_cuda.h>
using namespace std;

__global__ void increment(float *in, float *out)
{
    int it=0, i = blockIdx.x * blockDim.x + threadIdx.x;
    float a=0.8525852f;

    for(it=0; it<99999999; it++)
             out[i] += (in[i]+a)*a-(in[i]+a);
}

int main( int argc, char* argv[])
{
    int i;
    int nBlocks = 1;
    int threadsPerBlock = 1024;
    float *A, *d_A, *d_B, *B;
    size_t size=1024*13;

    A = (float *) malloc(size * sizeof(float));
    B = (float *) malloc(size * sizeof(float));

    for(i=0;i<size;i++){
            A[i]=0.74;
            B[i]=0.36;
    }

    cudaMalloc((void **) &d_A, size * sizeof(float));
    cudaMalloc((void **) &d_B, size * sizeof(float));

    cudaMemcpy(d_A, A, size, cudaMemcpyHostToDevice);
    cudaMemcpy(d_B, B, size, cudaMemcpyHostToDevice);

    increment<<<nBlocks,threadsPerBlock>>>(d_A, d_B);

    cudaDeviceSynchronize();

    cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);

    free(A);
    free(B);

    cudaFree(d_A);
    cudaFree(d_B);

    cudaDeviceReset();

    return (0);
}

So if I execute the kernel:
time ./simple
I get
real   0m36.659s
 user   0m4.033s
 sys    0m1.124s
Otherwise, If I execute two instances:
time ./simple & time ./simple
I get for each process:
real   1m12.417s
 user   0m29.494s
 sys    0m42.721s
real   1m12.440s
 user   0m36.387s
 sys    0m8.820s
As far as I know, the executions should run concurrently lasting as one (about 36 seconds). However, they last twice the base time. We know that the GPU has 13 SMs, each one should execute one block, thus the kernels only create 1 block.
Are they being executed in the same SM?
Shouldn't they running concurrently in different SMs?
EDITED
In order to make me clearer I will attach the profiles of the concurrent execution, obtained from nvprof:
Profile, first instance

Profile, second instance

Now, I would like to show you the behavior of the same scenario but executing concurrently two instances of matrixMul sample:
Profile, first instance

Profile, second instance

As you can see, in the first scenario, a kernel waits for the other to finish. While, in the second scenario (matrixMul), kernels from both contexts are running at the same time.
Thank you.
Answer: When you run two separate processes using the same GPU, they each have their own context. CUDA doesn't support having multiple contexts on the same device simultaneously. Instead, each context competes for the device in an undefined manner, with driver level context switching. That is why the execution behaves as if the processes are serialised -- effectively they are, but at a driver rather than GPU level.
There are technologies available (MPS, Hyper-Q) which can do what you want, but the way you are trying to do this won't work.

Edit to respond to the update in your question:
The example you have added using the MatrixMul sample doesn't show what you think it does. That application runs 300 short kernels and computes a performance number over the average of those 300 runs. Your profiling display has been set to a very coarse timescale resolution so that it looks like there is a single long running kernel launch, when in fact it is a series of very short running time kernels.
To illustrate this, consider the following:
This is a normal profiling run for a single MatrixMul process running on a Kepler device. Note that there are many individual kernels running directly after one another.

These are the profiling traces of  two simultaneous MatrixMul processes running on the same Kepler device:

Note that there are gaps in the profile traces of each process, this is where context switching between the two processes is occurring. The behaviour is identical to your original example, just at a much finer time granularity. As has been repeated a number of times by several different people in the course of this discussion -- CUDA doesn't support multiple contexts on the sample device simultaneously using the standard runtime API. The MPS server does allow this by adding a daemon which reimplements the API with a large shared internal Hyper-Q pipeline, but you are not using this and it has no bearing on the results you have shown in this question.
Comment: Matrixmul won't run kernels concurrently either from two separate processes. You are misinterpreting something. The apps may appear to run concurrently but individual kernels will not when originating from separate processes. The profiler may show some kind of overlap but this will represent API activity. The kernels themselves will not run concurrently.
Comment: @siserte: CUDA supports concurrent kernels *within the same context*. It absolutely does not support concurrent contexts. Also, please check who you are arguing with here - robert crovella is an NVIDIA employee who specialises in this stuff. He is perfectly correct, and you are not, I'm afraid
Comment: It is possible to capture cuda trace simultaneously from both processes using Nsight VSE or nvprof. This will clearly show that there is a context switch on the GPU between the two CUDA contexts. The only way to run two processes concurrently on CC 3.5 - CC5.x is to use the CUDA MPS server.
Comment: matrixMul launches *many* kernels.  You are showing the first two launches.  No conclusions can be drawn from that.  Furthermore, unless your applications are perfectly synchronized, so that the cuda kernel launch requests are issued at nearly the same time, you won't observe anything interesting in profiler traces that were captured independently.  These two instances of matrixMul might have attempted to use the GPU at slightly different points in time, leading to no overlap at all.  I think your comparison methodology is flawed.
Comment: @siserte: I have updated my answer to respond to your updates regarding your matrixMul example. I hope this demonstrates that you have just misinterpreted what is going on here and it doesn't contradict anything else you have been told in this discussion
Comment: I see, I didn't know about it, however, when I launch 2 concurrent instances of matrixMul from the SDK, they run concurrently. I assume that the increase in the execution time is due to SMs sharing, but the presented kernel is so small that it could be run without sharing SMs. We have profiled the executions and we have discovered that our kernel blocks other kernel execution, while matrixMul kernel doesn't block other executions. Do you know why?
Comment: Thank for the answer, but we are launching 2 instances of matrixMul, where are executed 2 kernels in each one. The profiles of both executions show that at the same time there are 2 kernels running at the same time. As far as I know, CUDA is able to handle several kernels concurrently.
Comment: @talonmies You are likely to be right in your explanation and I don't hesitate about the experience and knowledge of other users. Moreover I edited the question adding the profiles of the concurrent execution of our kernel and matrixMul, please check it. Besides, I have checked the MPS documentation [link](https://docs.nvidia.com/deploy/pdf/CUDA_Multi_Process_Service_Overview.pdf) at point 3.2 trying to understand the concurrent context execution while sharing the GPU, there it is shown that multiple contexts can be time sliced.
Comment: @RobertCrovella please check the question again, because it has been extended with execution profiles. Your feedback will be really apreciate. Thank you.


Moving only to the right and down, how many different pathways exist to get from A to Point B

Question:
The answer I think is 10 following the rules in my textbook am I correct?
Answer:

Comment: I think this is the correct answer Thanks Zen Chonoles I have awaraded you the correct answer
Comment: Glad I could help :)
Comment: Zev would you mind taking a look at my graphing equation...I have been trying to figure it out for ever.http://math.stackexchange.com/questions/425730/given-the-graphs-of-fx-and-gx-sketch-the-graph-of-f-gx-on-the-same-grid


Show that ... Has No Real Roots

Question: When $f(x) = 3x-4$ and $g(x) = \frac{5}{3-x}$,
Question 1: Find the value of x for which fg(x) = 5
Question 2: Show that the equation $f^{-1}(x) = g^{-1}(x)$ has no real roots.
I understand that in question 1 it simply is asking to plug 5 into the x and then solve for fg(x) =
5/(3-5) = -2.5
3(-2.5) - 4 = -11.5 (this is all right so far?)
I don't understand question 2, or how exactly I should answer it. How do I work out that there are no real roots? This is how the question is asked. Obviously the functions have to first be in the inverse forms.
I'm not an Einstein when it comes to working out roots - so if possible - please explain each step taken so that I may understand this type of question in the future.
Many thanks,
Student909.
Comment: Please edit your question to indicate  what have you attempted and where are you stuck.
Comment: Question 2 is quite easy because you can explicit the inverse in both cases: you have $x=\frac{y+4}{3}$ and $x=\frac{3y-5}{y}$ hence your resolvent equation has discriminant negative.
Answer: $$f(x)=3x-4$$
$$x=\frac{f(x)+4}3$$
$$f^{-1}(x)=\frac{x+4}3$$
$$g(x)=\frac{5}{3-x}$$
$$x=3-\frac{5}{g(x)}$$
$$g^{-1}(x)=3-\frac{5}{x}$$
$$f^{-1}(x)-g^{-1}(x)=\frac x3+\frac43-3+\frac 5x=\frac{x^2-5x+15}{3x}$$
The discriminant of $x^2-5x+15$ is negative. So, $x^2-5x+15\ne0$ for all $x\in\mathbb R$.
Thus, $$f^{-1}(x)-g^{-1}(x)\ne0$$ for all $x\in\mathbb R$.
As a result, $$f^{-1}(x)=g^{-1}(x)$$ has no real roots.

Show that ... Has No Real Roots


Javascript variables sent to modal

Question: I have PHP code that spits out an HTML table. In this row, I have an input type=\"checkbox\" that sends the id=\"$Row[ID_NUMBER]}\" to a modal window. Here is the echo statement that prints the row data with the checkbox.  Take notice of the extra data field in the row.
 echo "
<input type=\"checkbox\" id=\"{$Row[ID_NUMBER]}\" data-info=\"{$Row[BOL_NUMBER]}\" name=\"checkMr[]\" />
";

Here is the javascript the retrieves the id=\"{$Row[ID_NUMBER]}\" and displays it in the modal window:

Of course there is a button that calls the javascript function via onclick="modalForm();" that I don't think I need to show you.  However, here is where the modal prints the #idNumber:
 Container Selected
 <input disabled type="text" name="idNumber" id="idNumber" placeholder="no containers selected" />

What I need to do on top of retrieving the id from the datatable is retrieve the data-info=\"{$Row[BOL_NUMBER]}\" and send it to the modal window.  I have an input field for boNumber that goes right underneath idNumber:
 Bol Selected
 <input type="text" name="bolNumber" id="bolNumber" />

I know this can be done within the same javascript function above.  Any help would be appreciated.
Answer: $(':checkbox[name="checkMr[]"]:checked').each (function modalForm(){
    selectedID.push(this.id);
    // get the data-info from checkbox and set value in modal
    $("#bolNumber").val($(this).attr('data-info'));
});

Comment: I really have no idea what you are doing with modalForm since you are redeclaring it several times. If you want to use the function drop the function keyword.
Comment: I have multiple modal windows and I figured I will need to utilize several javascript functions.
Comment: Should there be another section in the javascript that sends the 'data-info' to a section in the modal?

Javascript variables sent to modal


How to compare percentages for populations of different sizes?

Question: I have several populations (of people, actually) which vary in size (from 5 to 6000). I would like to visualize the ratio of women vs. men in each of them so that they can be compared. I will get, for instance

case 1: 20% of women, size of the population: 6000
case 2: 20% of women, size of the population: 5
case 3: ...
...

Both percentages in the first cases are the same but a change of one person in each of the populations obviously changes percentages in a vastly different proportion.
Should I take that into account when presenting the data? I am working on a whole population, not samples, so I would tend to say no. I also have a gut feeling that the differences in the population size should still be accounted in some way.
What I am trying to achieve at the end is the ability to state "all cases are similar" or "case 15 is significantly different" - again with the constraint of wildly varying population sizes.
Comment: You could present the actual population size using an axis label on any simple display (e.g. bar chart) of women/men. A quite different plot would just be #women versus #men; the sex ratios would then be different slopes. Provided all values are positive, logarithmic scale might help.
Comment: @NickCox: this is a good idea. I wanted to avoid using actual numbers (because of the orders of magnitudes), even with a logarithmic scale (about 93% of the intended audience would not understand it :)). I was more looking for a way to signal this size discrepancy by some "uncertainty bars" around results normalized to 100%. It is just that I do not think it is possible to talk about any kind of uncertainty here, as all the numbers are known (no sampling). I will probably go for the logarythmic version with raw numbers then. if you do not mind could you please turn your comment into an answer?
Comment: Sure. In turn, if you would give your data, or a larger fraction of it, I could add authentic graphical examples.
Answer: You could present the actual population size using an axis label on any simple display (e.g. bar chart) of women/men.
A quite different plot would just be #women versus #men; the sex ratios would then be different slopes. Provided all values are positive, logarithmic scale might help. On logarithmic scale, lines with the same ratio #women/#men or equivalently the same fraction of women plot as parallel. An audience naive or nervous about logarithmic scale might be encouraged by seeing raw and log scale side by side.
Answer: The problem that you have presented is very valid and is similar to the difference between probabilities and odds ratio in a manner of speaking.
The percentage that you have calculated is similar to calculating probabilities (in the sense that it is scale dependent). I would suggest that you calculate the Female to Male ratio (the odds ratio) which is scale independent and will give you an overall picture across varying populations.
Comment: The odds ratio is also sensitive to small changes e.g. a shift from 1 to 2 women out of 5. It's difficult to see that this addresses the question at all. Nothing here on graphics.
Comment: You can try conducting a two sample t-test between varying percentages i.e. number of women expressed as a percent of total population.
Comment: What set-up here defines _two samples_?
Comment: You are working with different populations, I don't see any other way to compare your results. All the populations (5 - 6000) are coming from a population, you will have to trust your instincts to test if they are dependent or independent. But I would suggest that you treat these as separate samples.
Comment: I can't follow your comments at all. Note that the question is not mine,  but that of @WoJ. Perhaps we're reading the word "populations" differently. For the OP, several populations just define data points with differing numbers of males and females. Regardless of that, I don't see that you have addressed my query about what defines precisely two samples in this set-up.
Answer: Although your figures are for populations, your question suggests you would like to consider them as samples, in which case I think that you would find it helpful to illustrate your results by also calculating 95% confidence intervals and plotting the actual results with the upper and lower confidence levels as a clustered bar chart or perhaps as a bar chart for the actual results and a superimposed pair of line charts for the upper and lower confidence levels.


Tempo di Louie indication

Question: I am learning the piece Louie Louie as part of a jazz ensemble.
The tempo marking is only Tempo di Louie.

How fast/slow is this meant to be?
Is the piece meant to be swung?
Are there any unusual accenting patterns in this piece?
Comment: Why don't you just listen to the original recording?
Answer: "Tempo do Louie" is really just a joke; literally it means "Louie's Tempo," which really means nothing at all.
In cases like this, the best thing to do is consult the original, in which you'll hear that the song is roughly ♩ = 124bpm.
In the recording, you'll also hear that it is not meant to be swung. I don't hear any unusual accenting patterns.
Answer: Richard already provided a great answer, but I want to elaborate more on a comment from Carl Witthoft.
When learning any jazz tune, it is best to listen to as many recordings of it as you can. Listen to jazz greats playing the tune and how they add their own spin on things. Mimic these different styles and even try developing your own.
Here's an example of the same song by three different cats:

This really helps you get a understanding of the song that's certainly better than anything you'll get from reading sheet music (which is why it's great to learn songs by ear).

Tempo di Louie indication


jQueryUI extending Dialog

Question: I am trying with the code below to extend jQueryUI Dialog to get to something like this:http://www.schillmania.com/projects/dialog/png/
Everything works fine except the fact that when I click the button to close the dialog nothing happens.
(function( $, undefined ) {

    var uiDialogClasses = "ui-dialog ui-widget ui-widget-content roundbox";

    $.widget("custom.mydialog", $.ui.dialog, {

        _super_create: $.ui.dialog.prototype._create,
        _create: function() {
            this.originalTitle = this.element.attr( "title" );
            // #5742 - .attr() might return a DOMElement
            if ( typeof this.originalTitle !== "string" ) {
                this.originalTitle = "";
            }
            this.oldPosition = {
                parent: this.element.parent(),
                index: this.element.parent().children().index( this.element )
            };
            this.options.title = this.options.title || this.originalTitle;
            var self = this,
                options = self.options,

                title = options.title || " ",
                titleId = $.ui.dialog.getTitleId( self.element ),

                uiDialog = ( self.uiDialogOriginal = $( '
' ) )
                    //.addClass( uiDialogClasses + options.dialogClass)
                    // setting tabIndex makes the div focusable
                    .attr( "tabIndex", -1)
                    .keydown(function( event ) {
                        if ( options.closeOnEscape && !event.isDefaultPrevented() && event.keyCode &&
                                event.keyCode === $.ui.keyCode.ESCAPE ) {
                            self.close( event );
                            event.preventDefault();
                        }
                    })
                    .attr({
                        role: "dialog",
                        "aria-labelledby": titleId
                    })
                    .mousedown(function( event ) {
                        self.moveToTop( false, event );
                    })
                    .appendTo( "body" )
                    .wrap('

')

                    self.uiDialog = $('div.dialog');

                     $('div.dialog')
                       .prepend('
'+
                         '

')
                       .append('
'+
                         '

')
                       .addClass( uiDialogClasses + options.dialogClass);

                uiDialogContent = self.element
                    .show()
                    .removeAttr( "title" )
                    .addClass( "ui-dialog-content ui-widget-content" )
                    .appendTo( uiDialog ),

                uiDialogTitlebar = ( self.uiDialogTitlebar = $( "
" ) )
                    .addClass( "ui-dialog-titlebar  ui-widget-header  " +
                        "ui-corner-all  ui-helper-clearfix" )
                    .prependTo( uiDialog ),

                uiDialogTitlebarClose = $( "" )
                    .addClass( "ui-dialog-titlebar-close  ui-corner-all" )
                    .attr( "role", "button" )
                    .click(function( event ) {
                        event.preventDefault();
                        self.close( event );
                    })
                    .appendTo( uiDialogTitlebar ),

                uiDialogTitlebarCloseText = ( self.uiDialogTitlebarCloseText = $( "" ) )
                    .addClass( "ui-icon ui-icon-closethick" )
                    .text( options.closeText )
                    .appendTo( uiDialogTitlebarClose ),

                uiDialogTitle = $( "" )
                    .addClass( "ui-dialog-title" )
                    .attr( "id", titleId )
                    .html( title )
                    .prependTo( uiDialogTitlebar );

            uiDialogTitlebar.find( "*" ).add( uiDialogTitlebar ).disableSelection();
            //this._hoverable( uiDialogTitlebarClose );
            //this._focusable( uiDialogTitlebarClose );

            if ( options.draggable && $.fn.draggable ) {
                self._makeDraggable();
            }
            if ( options.resizable && $.fn.resizable ) {
                self._makeResizable();
            }

            self._createButtons( options.buttons );
            self._isOpen = false;

            if ( $.fn.bgiframe ) {
                uiDialog.bgiframe();
            }
        },
        _createButtons: function( buttons ) {
            var self = this,
                hasButtons = false;

            // if we already have a button pane, remove it
            self.uiDialogOriginal.find( ".ui-dialog-buttonpane" ).remove();

            if ( typeof buttons === "object" && buttons !== null ) {
                $.each( buttons, function() {
                    return !(hasButtons = true);
                });
            }
            if ( hasButtons ) {
                var uiDialogButtonPane = $( "
" )
                        .addClass( "ui-dialog-buttonpane  ui-widget-content ui-helper-clearfix" ),
                    uiButtonSet = $( "
" )
                        .addClass( "ui-dialog-buttonset" )
                        .appendTo( uiDialogButtonPane );

                $.each( buttons, function( name, props ) {
                    props = $.isFunction( props ) ?
                        { click: props, text: name } :
                        props;
                    var button = $( "<button type='button'>" )
                        .attr( props, true )
                        .unbind( "click" )
                        .click(function() {
                            props.click.apply( self.element[0], arguments );
                        })
                        .appendTo( uiButtonSet );
                    if ( $.fn.button ) {
                        button.button();
                    }
                });
                self.uiDialogOriginal.addClass( "ui-dialog-buttons" );
                uiDialogButtonPane.appendTo( self.uiDialogOriginal );
            } else {
                self.uiDialogOriginal.removeClass( "ui-dialog-buttons" );
            }
        }
    });

}( jQuery ) );

here is how I do use it:
and the issue seems to be that $(this).dialog('close'); dose not call the close function at all....
    var btns = {};
    btns[$.i18n.prop('buttons.ok')] = function() {
        $(this).dialog('close');
    };

    var opt = options || {};

    if(opt.open==undefined)
        opt.open = true;

    $dialog = $('

');
    $dialog.mydialog({
        title: opt.title || '',
        bgiframe: opt.bigFrame || true,
        modal: opt.modal!=undefined?opt.modal:glbmodal,
        overlay :{
            backgroundColor :'#000',
            opacity :0.8
        },
        buttons: opt.buttons || btns,
        width: opt.width || $(document).width()/2,
        height: opt.height || $(document).height()/2,
        position: 'center',
        autoOpen: opt.open,
        close: function(){
            $(this).dialog('destroy');
            $(this).remove();
            $('.dialog').filter(function(){
                if ($(this).text() == "")
                {
                    return true;
                }
                return false;
            }).remove();
        }
    });

    if(opt.content){
        $dialog.html(opt.content);
    }
    return $dialog;
};

Thanks!
Answer: I think your problem is that you are referencing the wrong variable with "this".
btns[$.i18n.prop('buttons.ok')] = function() {
    $(this).dialog('close');
};

The $(this) selector here is selecting the button from your array of buttons, which I don't think is what you are trying to accomplish as your buttons don't have a dialog attached.  You need to reference the dialog itself to close it, something like this.
  $($dialog).dialog('close');

Comment: This same code that you point out as not working works good with the jqueryui dialog thing, i mean, the original one...

jQueryUI extending Dialog


Does this infinite series converge?

Question: I need help proving that this infinite sequence converges:
$\sum_{n=2}^\infty(\frac{1}{1+\sqrt2})^n$
I am trying to do the ratio test and have gotten to this point:
$\lim_{n\to \infty}(\frac{(\frac{1}{1+\sqrt2})^n+1}{(\frac{1}{1+\sqrt2})^n})$
I'm stuck on what to do next
Comment: This is simply a geometric series.
Comment: Write $a=1/(1+\sqrt{2})$; then the ratio test asks you to find $\lim_{n\to\infty}\frac{a^{n+1}}{a^n}$ which is obviously $a$.
Answer: Hint: it's a geometric series. You can even calculate the sum, leave alone check convergence.
Answer: When you have complicated terms, use algebra. Set
$$
a=\frac{1}{1+\sqrt{2}}
$$
Then the series is
$$
\sum_{n=2}^{\infty}a^n
$$
The ratio test asks you to compute
$$
\lim_{n\to\infty}\frac{a^{n+1}}{a^n}
$$
and this should be easy enough, isn't it?
Now, can you prove that the limit is $<1$, so the ratio test guarantees convergence?
Answer: If you want to do the ratio test let $x=\frac{1}{1+\sqrt{2}}$, then we get $\lim_{n \rightarrow \infty}x^{n+1}/x^{n} = x = \frac{1}{1+\sqrt{2}}$
You could also note that $\frac{1}{1+\sqrt{2}} < 1$ and it's a geometric series, therefore it converges.

Does this infinite series converge?


jsonapi Cross-Origin Read Blocking (CORB) blocked cross-origin response in vue

Question: I got the data with postman in json:api at drupal.

 But I has error of CORB in  Vue-resource.get() with code blew:
<code> data(){
    return{
        cyclePic:[],
        rootUrl:this.$http.options.root,
        header:{headers:{Accept:'application/vnd.api+json',
            'Content-type':'application/vnd.api+json',
            Authorization:"Basic dG1wQWRtaW46VG1wQDEyMyM0NTY="}
            }
    }
},
created(){
    this.getCycleList()
},
methods:{
    getCycleList(){

        this.$http.get('jsonapi/node/banner?include=field_image',this.header).then(result=>{
            console.log(result)

            if(result.ok && result.status===200){
                this.cyclePic=result.body.included
                console.log(this.cyclePic)
            }else{
                Toast('Fail')
            }

        })
    }
</code>
what i miss some thing?
Answer: You need to set up your Cors settings
Look at your services.yml file then read ...
How do I set up CORS?
Comment: How to doing ? Drupal 8 restrict the Cors module .Or you can recommend the CORS module for me.thanks.
Comment: read the link aka https://drupal.stackexchange.com/questions/245903/how-do-i-set-up-cors
above then edit your "sites/default/default.services.yml" file !


Polynomial regression - correctness and accuracy

Question: I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients.
Now I'd like to add some check procedures to assert that everything is correct and that the regression model can be used with confidence. From Wikipedia I know that "This is the unique least squares solution as long as $X$ has linearly independent columns. Since $X$ is a Vandermonde matrix, this is guaranteed to hold provided that at least $m + 1$ of the $x_i$ are distinct (for which $m < n$ is a necessary condition)." So I guess that a good first step would be to check whether $m$ is indeed less than $n$... I could also tell the user that the degree of the regression shouldn't be too high, to avoid overfitting the data.
The thing is, everything should be hidden from the user... I can't ask him to perform cross-validation. On the other hand, I could write a little leave-one-out cross validation code which would be run on the whole training data every time a new model is created.
Any thought or suggestion ?
Thanks
Comment: Honestly: I do not see what you want to be checked. Is your question on how to check whether $(X'X)^{-1}X'y$ is calculated correctly? To this end you have to do checks: given $X$, you should obtain the $i$-th column of $(X'X)^{-1}X'$ by calculating $(X'X)^{-1}X'e_{i}$, where $e_{i}$ is the standard basis vector with zeroes except a one at entry $i$. The confidence put in a regression model (choice of $X$) is what defines "doing a regression". There is of course no general answer.
Comment: So is $X'$ the transpose? And shouldn't the expression be $(X' X)^{-1} X' y$ using your notation?
Comment: you're right, typo corrected, thank you ! (I use the correct formula in my code, of course)
Comment: And beside the text formating, any help ? thx !
Comment: Well, as I wrote in my post, there must be some guidelines to assert whether or not a regression model can be used with confidence ? Also, what about m < n ? I mean I asked precise questions, I don't get it why the question is downgraded...
Answer: 1- Let the matrices $X_{train}$ and $Y_{train}$ be your training data, and $X_{tune}$ and $Y_{tune}$ be the held-out data that is not included in the training data.
Solve $\hat{\theta}=(X_{train}^TX_{train})^{\dagger}X_{train}^TY_{train}$
Now you regression model is" $Y=X\hat{\theta}$.
Use this regression model to estimate the outputs for the held-out data:
$\hat{Y}_{tune}=X_{tune}\hat{\theta}$
Afterwards, calculate $corr(\hat{Y}_{tune},Y_{tune})$, the correlation coefficient between $\hat{Y}_{tune}$ and $Y_{tune}$, if the correlation is greater than some threshold, then you have found a good fit.
(Threshold=.8 may be a good guess).
2- A question that you might have is about the best number of polynomial terms to use when forming your input Vandermonde matrix. Here is my suggestion, let $k$ be the number of columns in $X$, then start from $k=1$ and record the resulting correlation, keep increasing $k$ until some maximum integer (say k=1000, but it really depends on the complexity of the underlying function that generates your data.) Finally choose the $k$ that yields the maximum correlation.
3-    By $(X_{train}^TX_{train})^{\dagger}$, I mean the pseudo-inverse of $(X_{train}^TX_{train})$. Note that $(X_{train}^TX_{train})$ is always PSD, but not necessarily PD. So if its determinant is zero you can either use the Moore–Penrose pseudoinverse, or use the Tikhonov regularization, which is the foillowing:
$(X_{train}^TX_{train})^{\dagger}\approx (X_{train}^TX_{train}+\epsilon I)^{-1}$.
You can also tune $\epsilon$ by tuning on the held-out data.
4- For choosing either $k$ or $\epsilon$, if your sample size is small, I recommend using cross-validation, rather than only working on one  held-out dataset.
Hope this helps.
Answer: This answer is based on a software development practice rather than anything from mathematics.
It sounds like you have two categories of question:
1) Is the approach mathematically correct?
and
2) Is the code producing the correct result?
You can answer both questions via non-mathematical means: write a test suite and run it each time you update your code.  E.g., for a polynomial you know, generate points for which the best fit is that polynomial.  Then check that your code finds the correct one.  You can programmatically test a few hundred or thousand cases -- and a good test suite will check edge and corner cases as well.
Comment: I would say that I have more a runtime problem rather than a unit test problem. I know how to test the stuff I programmed, what I want to know is some tricks on how to assert, at runtime, that the user entered correct data to perform regression on, and that the result makes sense. So far I have modified my code to create a test matrix as well as a data matrix, then I check the error between the actual test values and the results of the projection of the test data on  the regression model.


Manipular arquivo .txt em Ruby e pupular BD?

Question: Preciso abrir um arquivo .txt qualquer que tenha um sequência de strings separadas por  ";" e preciso colocar a cada ";" em um vetor para depois criar um laço e inserir no banco.
O que eu tenho:
require 'pg'
require 'active_record'

 = File.open('teste.txt')
f_lines = f.read.split(";")

Conexão com o banco
conn = ActiveRecord::Base.establish_connection(:adapter => 'postgresql',
                                                :username => 'root',
                                                :password => "123",
                                                :host => "127.0.0.1",
                                                :database => "db")

#query = "insert into students (name, registration, room, password) values "+ vetor[]
#simulated = ActiveRecord::Base.connection.execute(query)

PS.: Meu arquivo .txt terá várias linhas.
Answer: Você estava bem no caminho certo, faltou só o pulo do gato srsrs
Vou utilizar um arquivo de entrada para demonstração
grupos.txt
'Administradores'; 'adm'; true
'Redatores'; 'red'; false
'Editores'; 'edt'; false
'Leitores'; 'lei'; false

Aqui a conexão que você já fez:
require 'pg'
require 'active_record'

file = File.open('grupos.txt')

ActiveRecord::Base.establish_connection(:adapter => 'postgresql',
                                                :username => 'root',
                                                :password => "123",
                                                :host => "127.0.0.1",
                                                :database => "db")

E o pulo do gato:
file.each_line do |line|
  values = line.split(';').join(',')
  insert_sql = "insert into grupos (nome, slug, padrao) values ( #{values} )"
  ActiveRecord::Base.connection.execute insert_sql
end

Resultado:

OTIMIZANDO
Caso seu arquivo não seja pequeno, vale muito a pena usar transaction , esse mesmo exemplo com 1000 linhas demorou 26.221072s para rodar, utilizando transaction levou apenas  0.343375s
Exemplo com transaction:
   ActiveRecord::Base.connection.transaction do
    file.each_line do |line|
      values = line.split(';').join(',')
      insert_sql = "insert into grupos (nome, slug, padrao) values ( #{values} )"
      ActiveRecord::Base.connection.execute insert_sql
    end
  end

Comment: Caaaaaaaaara, Perfeito!!!!Muito Obrigado, ajudou muito mesmo
Comment: Opa que bom que ajudou @Juliano, se a resposta resolveu seu problema marque como resposta. Grande abraço. Precisando estamos ai!
Comment: Boaa @Juliano :D


A cycle and a cutset intersect in an even number of edges in a graph $G$

Question: In a series of lemmas to prove the correctness of a minimum spanning tree algorithm for a finite connected weighted non-directed graph $G$, I come across the statement in the textbook Algorithm Design by John Kleinberg, Eva Tardos:
A cycle and a cutset intersect in an even number of edges
The authors defines a cut of a graph as simply a subset $S \subset G$ of nodes (this may conflict with other definitions of a "cut"). Correspondingly, a cutset is defined as the subset of edges with exactly one endpoint in $S$. The author gives a proof/argument for this statement with a simple picture but I would like to write a more formal argument.
Thoughts:
Assume we have a cutset and a cycle whose intersection contains at least one edge. For each edge in the cycle that intersects the cutset, only one node can belong to the cut, so that any edge in the cycle where both nodes are in the cut is not in the intersection.
Consider the next edge in the cycle, either both nodes are in the cutset (an edge not in the intersection), or only one node is in the cutset. In the former case, this cannot continue indefinitely since eventually you would come across an edge from a cutset element to the original node not in the cutset, and so there must eventually be an edge from a cut node to a non-cut node.
Thus for every edge in the intersection entering the cut (an edge from non-cut to cut), there will be a corresponding edge leaving the cut (an edge from cut to non-cut)
Since this is true for each edge in the intersection of the cutset and our cycle, then the overall intersection must have an even cardinality.
Any insights how to clean up this argument appreciated.
Comment: If you colour the elements of $S$ red and the other vertices blue, then an edge both in the cycle and cutset corresponds to a change of colour. So why must the colour change an even number of times in a cycle?
Comment: What would happen it you went all the way around the cycle and had an odd number of colour changes?
Comment: @LordSharktheUnknown I have an idea why as per my arugment that after an edge both in the cycle and the cutset the next edge (assuming some ordering of the edges in the cycle) will either contain two elements in the cut, or one element in the cut and one not in the cut. In the former case, this can't continue indefinitely ( I think?)  and so there must be a corresponding edge that leaves the cut. Is this the idea?
Comment: @LordSharktheUnknown That is an elegant proof of the statement or maybe just a hint for a proof. Either way it should be an answer, not a comment.
Comment: @LordSharktheUnknown i'm not sure.
Comment: Here's another proof: https://ifarah.mathstats.yorku.ca/files/2020/03/3260-cycle-cutset.pdf?x92218
Answer: A cycle can be viewed as a path starting with a vertex $v$ and finishes in the same vertex $v$. according to your definition of a cut-set as the set of all edges with one endpoint at a subset of vertices $S$, color all the vertices of $S$ blue, color the rest of the vertices red. our vertex $v$ is either blue or red, let's assume for simplicity that it's blue and begin traveling
through our cycle. our cycle starts at $v$ with the color blue, it also finishes at $v$ with the color blue. our path changes and only changes color when passing through an edge in the cut-set, and since our path finishes at the same color it started at, it means we must have passed through edges of our cut-set an even number of times.


Trigger a behavior from another module in Javascript?

Question: The block_ajax module defines a behavior in its module:
https://git.drupalcode.org/project/block_ajax/-/blob/8.x-1.x/js/refresh_ajax_block_command.js
I want to initiate this call every 60 seconds so the contents of the block can refresh if there is new data from my own script file.
I tried two ways.
The first way I tried was setting up a route for a simple controller that returned an AjaxResponse:
  public function execute() {
    $response = new AjaxResponse();
    $response->addCommand(new RefreshBlockAjaxCommand('.my_block'));
    return $response;
  }

Hitting the route with Drupal.ajax results in 'context.querySelectorAll is not a function', but it appears that the block is replaced (but missing the block wrapper around it, oddly enough, which probably won't work calling it over and over).
The second way is I tried to call it from the browser console with Javascript, but I can't figure out between this answer and the linked Javascript file at the top of the question what to call.
Trying to loop the element and attachBehaviors results in this:
Answer: Turned out to be a couple of things.
One, I was getting the querySelectorAll error because the Block being rendered was being rendered with a theme and custom template file, which was missing an outer div. The first tag it was seeing was a


How can I send a Privatemsg using Rules and have the name of the site as the sender?

Question: I am using the Privatemsg and Rules module and I am trying to set up an automatically generated welcome message to new members registering on the site. This message will go into the new member's inbox.
I do not see a token that would allow the site's name to be the sender. All I see is the token for the logged in user. As expected, this doesn't trigger the message.
I have toyed with the [account] token. So when John creates an account, the message comes from John himself. This would be acceptable but it isn't consistent: Sometimes it works, sometimes it doesn't.
I know some tokens exist that are not always displayed. Does anyone have the name of the mysterious token? Or the magic to get this to work?
Answer: You could create an account for sending the messages and in rules, use direct input and enter its user id. That way, you could also see replies to welcome message.
Comment: Did you click "Switch to Direct Input Mode" in the rules action for "Send message" and enter the uid of the sender?
Comment: @NiallMurphy : even though it was not part of the question, you could take this a step further, using [Menu Badges](https://www.drupal.org/project/menu_badges) and its integration with the Privatemsg module ... to produce such "shiny" iOS style of (red) msg notification ...
Comment: This approach only works if the account is an administrator and creates a new account for a user (token is "logged-in-user".) But with a newcomer creating his own account, no message is sent. There isn't the possibility of assigning a certain user or the site as the sender.
Comment: Good idea! I've never seen that module before..
Answer: Here's the site name token: [site:name]
Full list of tokens: https://www.drupal.org/node/390482
Comment: Unfortunately, this token is not accepted by privatemsg.


VBA code does not print value from sheet

Question: I got the following VBA code that aims to get the value from cell 1,1 in a sheet named "risk_cat_1" and print it.
Therefore I wrote the following VBA code:
Sub get_prime_values()

 Sheets("risk_cat_1").Cells(1, 1).Value = value_1
 MsgBox (value_1)

End sub

When I run it I get an empty MsgBox however. Any thoughts what goes wrong here?
Comment: You are inverting destination and source, try `value_1 = Sheets("risk_cat_1").Cells(1, 1).Value`.
Answer: Flip your assignment from
Sheets("risk_cat_1").Cells(1, 1).Value = value_1

to
value_1 = Sheets("risk_cat_1").Cells(1, 1).Value


macos statically linking libstdc++ with g++ 4.7.2

Question: I've been looking at this issue for a number of weeks now with no joy so its time to ask for the wisdom of stack overflow...
For various reasons I need to link libstdc++ into my executable so it has no extra dependencies. Using g++'s -static-libstdc++ and -static-libgcc flags I was able to achieve this, however, no exceptions were being caught.
I produced the following test code to investigate the problem further. It seems the code works when I compile in 32 mode but not in 64 bit. I do not understand why the exception is not being caught and its rather frustrating.
Setup

Macos 10.7 64 bit
G++ 4.7.2

The proram
<code>#include <cstdio>
#include <stdexcept>

void myMethod() {
    throw std::invalid_argument("foo");
}

int main () {
    try {
            myMethod();
    } catch (const std::invalid_argument& ex) {
            printf("caught: %s\n", ex.what());
    } catch (...) {
            printf("caught it\n");
    }
    return 0;
}
</code>
32 bit mode
<code>$ g++ -m32 -o main Main.cpp -static-libgcc -static-libstdc++ && otool -L ./main && ./main
    ./main:
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
 caught: foo
</code>
64 bit mode
<code>$ g++ -o main Main.cpp -static-libgcc -static-libstdc++ && otool -L ./main && ./main
    ./main:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
Abort trap: 6
</code>
I have tried many different methods to try to solve this problem including:

Linking directly with the static libraries i.e. /usr/local/lib/libstdc++.a
Using -Wl,-bstatic -lstdc++  -lgcc_eh -bdynamic

But no avail.
Is there some part of the compiler configuration I need to check?
A flag I'm missing?
I know mac stopped supporting g++ at version 4.2.1 so it might be better to move over to using clang and hoping the binary still works on different versions of OSX.
Answer: I think exceptions are a bit broken with GCC on Mac OS X:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45486
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42159


What mode/scale are these notes in?

Question: I have created a short melody that uses these notes:

What mode contains these notes?
I have tried Dorian, Aeolian, Lydian, Phrygian and Mixolydian by starting at the scale in each mode that contains all naturals and working up via 5ths. But I can't find a scale that incorporates these notes
e.f. for Dorian, I started on D, then A, then E, etc. etc.
D  E  F G  A  B  C
A  B  C D  E  F# G
E  F# G A  B  C# D
B  C# D E  F# G# A
F# G# A B  C# D# E
C# D# E F# C# A# B
Answer: It is not any of the modes built on the Ionian scale but if B is your tonal center it is a part of the B harmonic or melodic minor scale.
Answer: The modes you've tried — the "diatonic" modes — do not fit the notes you've composed. However, there are scales that incorporate them.
In particular both the B harmonic and melodic minor scales includes those pitches.
X: 1
T: B Harmonic Minor
M: none
K: none
L: 1/4
"_↑"B, "_↑"^C "_↑"D E "_↑"^F G "_↑"^A B ||

X: 1
T: B Melodic Minor
M: none
K: none
L: 1/4
"_↑"B, "_↑"^C "_↑"D E "_↑"^F ^G "_↑"^A B ||

The Musical Scale Search Tool lists a number of other scales that also include those pitches.
For example:
F♯ Double Harmonic
X: 1
T: F# Double Harmonic
M: none
K: none
L: 1/4
"_↑"^F G "_↑"^A "_↑"B "_↑"^c "_↑"d ^^e ^f ||

Answer: For starters, you have used the notes A♯, B, C♯, D and F♯. The problem is that they won't belong to a diatonic major scale. The reason for that is that any key with an A♯ in it, will already have a D♯. And your tune has D♮.
We could look at those sharps as flats: B♭, C♭... - no, that's not going to help!
So, we look at minors, and that A♯ to B is a clue.It features in both the harmonic and melodic (rising) minors, where those two are the leading and tonic notes. Of B minor. The B minor scale goes B, C♯, D, E, F♯, G, A♯, B - or B, C♯,  D, E, F♯, G♯, A♯, B for the melodic. So, the key signature will be simply two sharps, as in B minor, and each subsequent A note will need a sharp before it. And B will most likely feel like 'home'. I hope there will be more of the tune than just five notes!
EDIT: just a thought, but if the A# was the anacrucis, putting more emphasis on the B at the start of the next bar, that would make it sound like it is in B minor. What do you think?
Comment: In the melody, B feels like the tonic and F# the fifth
Comment: So it's pointing more to being in key Bm. Unusual to start a melody on its leading note, on beat one.. Move the barline one beat to the right, and it seems to fit.

What mode/scale are these notes in?


An explanation on user_save() works

Question: I was writing on a module for drupal and i had to update some fields in the user-object. I first tried this code:
$account = user_load($user->uid); // Loading account
$edit = array(
  'gitlab_refresh_token' => $response_data['refresh_token'],
);
user_save($account, $edit);

This did not update the the field.
Then i found this code on the Internet:
$existingUser = user_load($user->uid);
$existingUser->data['gitlab_refresh_token'] = $response_data['refresh_token'];
user_save((object) array('uid' => $existingUser->uid), (array) $existingUser);

This code works. But the question is why?
The documentation for user_save() says, that $edit just needs to contain an array with key/value pairs of what you want to change and it will be put in the user's data field,
and that is what i did, right?
Answer: The first snippet doesn't work because Drupal doesn't use any 'gitlab_refresh_token' array index, and it doesn't know any field with that name. If you were trying to change the value of a field added with the field UI, the array needs to have a different structure, to be saved.
The second snippet works because Drupal effectively uses that array index, as also said in the documentation.

Key / value pairs added to the $edit['data'] will be serialized and saved in the {users.data} column.

// Prepopulate $edit['data'] with the current value of $account->data.
// Modules can add to or remove from this array in hook_user_presave().
if (!empty($account->data)) {
  $edit['data'] = !empty($edit['data']) ? array_merge($account->data, $edit['data']) : $account->data;
}

An explanation on user_save() works


Qual a diferença do uso do "equals" vs == no LINQ to Entities?

Question: Conforme citado acima, quando digo equals do LINQ to Entities estou me referindo especificamente à palavra reservada de C# e não ao Equals() do System.Object.
Fonte: equals (Referência de C#)
O exemplo a seguir retorna todos os produtos vermelhos.
String color = "Red";
using (AdventureWorksEntities context = new AdventureWorksEntities())
{
    var query =
        from product in context.Products
        where product.Color == color
        select new
        {
            Name = product.Name,
            ProductNumber = product.ProductNumber,
            ListPrice = product.ListPrice
        };

    foreach (var product in query)
    {
        Console.WriteLine("Name: {0}", product.Name);
        Console.WriteLine("Product number: {0}", product.ProductNumber);
        Console.WriteLine("List price: ${0}", product.ListPrice);
        Console.WriteLine("");
    }
}

Fonte: Exemplos de sintaxe de expressão de consulta: filtragem | Microsoft Docs
Dúvida: Qual a diferença ou impacto ao alterar == para equals ? Ao fazer essa alteração o select é montado da mesma forma?
Obs: não me refiro ao método .Equals()
Answer: Você não pode usar equals, daria erro porque o equals é uma cláusula do join que você não está usando.
Em um join você usa para fazer a ligação entre duas sequências de dados usando o campo de uma delas contra o campo da outra, desta forma fica claro que é isso e não uma comparação de igualdade pura e simples que retornaria um booleano como resultado, é um mecanismo diferente.
Comment: Obrigado pela resposta @Maniero, eu estava imaginando que o equals na verdade poderia sempre ser utilizado para criar uma consulta no linq, mas não havia me atentado ao fato de que ele é especificamente utilizado junto à clausula join.


Can we add security Realm to glassfish-resources

Question: Hi I am currently using JDBC realm in Glassfish 4.1.1 for authentication, and I wonder if I can add the configuration of jdbc realm to glassfish-resources.xml, so that when migrate, I can just call
 asadmin add-resources glassfish-resources.xml

to add all jdbc connection, datasources, and jdbcRealm to the new glassfish server?
Answer: No, you cannot configure the security realm in glassfish-resouces.xml. See the DTD, there is no element for that. There is a feature request open for that, see GLASSFISH-20874.
You can use a shellscript and the asadmin commands to do automatic configuration for all your resources or just for the security realm (create-auth-realm). Or use the REST-configuration interface.
Sources:
https://stackoverflow.com/a/19546431/5658642
https://stackoverflow.com/a/7855104/5658642


How to force ViewController to unload?

Question: I have an application that runs on background because it's needs to use Location Update. however, since i don't want to consume battery. I would like to make sure that all view controller is unloaded when app switch to run on background.
Answer: Forcing your view controller to unload is not going to save any battery life.  It's best to let the operating system worry about taking your app out of memory when it needs to.  If you're using the background mode that notifies your application when the user has travelled a significant amount you need to worry about this even less since your app is not actually running in the background but only given a chance to respond to location changes.  If you're using the constant location background mode (for GPS apps, etc.) this is a little bit more of an issue.
Comment: Thanks Paul ... i'm using constant location update because I need it.
Answer: You don't need to unload the view controllers, but you do need to turn off location updates. If you are starting location updates in your App Delegate, put in something like this:
- (void)applicationWillResignActive:(UIApplication *)application
{
    [myLocationManager stopUpdatingLocation];
}

You can also do this in the viewWillDisappear: method if your location manager is started in a different place than the App Delegate.

How to force ViewController to unload?


Does fiber product preserve limits?

Question: Let $\mathcal{C}$ be a category in which every fiber product exists.
Let $S' \rightarrow S$ be a morphism in $\mathcal{C}$.
Let $(\mathcal{C}\downarrow S)$ and $(\mathcal{C}\downarrow S')$ be the slice categories.
Let $F\colon (\mathcal{C}\downarrow S) \rightarrow (\mathcal{C}\downarrow S')$ be the functor defined by $F(X) =  X\times_S S'$.
Does $F$ preserve limits?
Answer: Yes. In fact, $(-) \times_S S'$ preserves limits because it is the right adjoint of the functor that takes an object $X \to S'$ in $(\mathcal{C} \downarrow S')$ to the object $X \to S' \to S$ in $(\mathcal{C} \downarrow S)$.
Comment: I don't really know what you're asking. This argument can be found in various category theory textbooks – certainly in Awodey (Ch. 9).
Comment: I understand how you and Makoto Kato are definng F, but I have a question: is your notation really correct? The way you write it $(-) \times_S S'$ appears to be a functor in $\mathcal{C}$ and not on the slice, and can be misleading, as shown below in Augusti Roig question/comment
Comment: How could it possibly be a functor on $\mathcal{C}$? There is no chance of confusion.
Comment: The source of my puzzlement is this: $F$ is a functor on a slice cat, whose objects are really morphisms $f$ in $\mathcal{C}$. So - by writing the functor $F$ in that way - you end up with expressions like $f \times_S S'$ where you have a fiber product between a morphism $f$ and an object $S'$. I wonder : is this standard notation? if yes, where?
Comment: The notation we use is completely standard in algebraic geometry, for example, and fairly common elsewhere. It is an accepted abuse of notation.
Comment: Thank you. Could you please suggest an algebraic geometry text where these functors/arguments are used?
Comment: Indeed it is in Awodey's text, but I was under the impression that this functor F is used in algebraic geometry. So I am asking for an algebraic geometry text, if it is possible. Or, more in general, a good introductory text on algebraic geometry which uses categorical language
Comment: This functor is used whereever Grothendieck's relative point of view is fruitful: algebraic geometry is just one of them. There is literally no algebraic geometry content to this question or its answer.
Answer: Yes and another explanation: your functor, being itself a limit, commutes with limits.
Comment: True, but _this_ functor is not a limit. The functor $(\mathcal{C} \downarrow S) \times (\mathcal{C} \downarrow S) \to (\mathcal{C} \downarrow S)$ that sends $(X \to S, Y \to S)$ to $X \times_S Y$ _is_, and it is right adjoint to the diagonal functor $\Delta : (\mathcal{C} \downarrow S) \to (\mathcal{C} \downarrow S) \times (\mathcal{C} \downarrow S)$.

Does fiber product preserve limits?


How do I disable the password recovery link?

Question: How can I disable, in the standard login form, the link for password recovery? I need it for my custom theme.
Answer: You can use hook_menu_local_tasks_alter().
E.g.
/**
 * Implements hook_menu_local_tasks_alter().
 */
function MY_MODULE_menu_local_tasks_alter(&$data, $route_name) {
  if($route_name == 'user.login') {
    $data['tabs'][0]['user.pass']['#access'] = FALSE;
  }
}

Note: Make sure to replace MY_MODULE with the actual name of your module.
Comment: I actually tested the above code so I know it works. This code should sit in a custom module. Make sure custom modules have info files and are enabled. You will also need to clear the cache before you'll be able to see the change.
Comment: It is not working. First a put it into myTheme.theme file and than I tried myModule.module file. But no chance. I still see the password recovery link
Comment: Not working. Drupal 9.4.8. Code is placed in module, cache cleared and it's executed. $data is there. Tried also unsetting whole $data['tabs'][0]['user.pass'], but link is still on login page.
Answer: For non-programmers, try No Request New Password module


Authentication via RADIUS : MSCHAPv2 Error 691

Question: I am working on setting up authentication into an Acme Packet Net-Net 3820 (SBC) via RADIUS. The accounting side of things is working just fine with no issues. The authentication side of things is another matter. I can see from a packet capture that the access-request messages are in fact getting to the RADIUS server at which point the RADIUS server starts communicating with the domain controllers. I then see the chain of communication going back to the RADIUS and then finally back to the SBC. The problem is the response I get back is always an access-reject message with a reason code of 16 (Authentication failed due to a user credentials mismatch. Either the user name provided does not match an existing user account or the password was incorrect). This is confirmed by looking at the security event logs where I can see events 4625 and 6273. See the events below (Note: The names and IPs have been changed to protect the innocent):
Event ID: 6273

Network Policy Server denied access to a user.

Contact the Network Policy Server administrator for more information.

User:
    Security ID:            NULL SID
    Account Name:           real_username
    Account Domain:         real_domain
    Fully Qualified Account Name:   real_domain\real_username

Client Machine:
    Security ID:            NULL SID
    Account Name:           -
    Fully Qualified Account Name:   -
    OS-Version:         -
    Called Station Identifier:      -
    Calling Station Identifier:     -

NAS:
    NAS IPv4 Address:       10.0.0.10
    NAS IPv6 Address:       -
    NAS Identifier:         radius1.real_domain
    NAS Port-Type:          -
    NAS Port:           101451540

RADIUS Client:
    Client Friendly Name:       sbc1mgmt
    Client IP Address:          10.0.0.10

Authentication Details:
    Connection Request Policy Name: SBC Authentication
    Network Policy Name:        -
    Authentication Provider:        Windows
    Authentication Server:      RADIUS1.real_domain
    Authentication Type:        MS-CHAPv2
    EAP Type:           -
    Account Session Identifier:     -
    Logging Results:            Accounting information was written to the SQL data store and the local log file.
    Reason Code:            16
    Reason:             Authentication failed due to a user credentials mismatch. Either the user name provided does not map to an existing user account or the password was incorrect.

Event ID: 4625

An account failed to log on.

Subject:
    Security ID:        SYSTEM
    Account Name:       RADIUS1$
    Account Domain:     REAL_DOMAIN
    Logon ID:       0x3E7

Logon Type:         3

Account For Which Logon Failed:
    Security ID:        NULL SID
    Account Name:       real_username
    Account Domain:     REAL_DOMAIN

Failure Information:
    Failure Reason:     Unknown user name or bad password.
    Status:         0xC000006D
    Sub Status:     0xC000006A

Process Information:
    Caller Process ID:  0x2cc
    Caller Process Name:    C:\Windows\System32\svchost.exe

Network Information:
    Workstation Name:
    Source Network Address: -
    Source Port:        -

Detailed Authentication Information:
    Logon Process:      IAS
    Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
    Transited Services: -
    Package Name (NTLM only):   -
    Key Length:     0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

So at first glance it would seem that the issue is merely a case of an invalid username or mismatched password. This is further confirmed in the packet capture where I can see the MSCHAPv2 response has an error code of 691 (Access denied because username or password, or both, are not valid on the domain). The thing is I know I am using a valid username and I have tried many usernames including new ones I created just for troubleshooting. I don't know how many times I have reset the password in an attempt to ensure it is not a mismatch password. I have even made sure to use passwords that are fairly short and contain only letters to ensure there was no terminal encoding issues (we connect to the SBC via SSH clients). I have also done this same thing with the shared secret used during communication between the SBC and the RADIUS server. I have tried prefixing the username with the domain name at login (though I don't think that should be necessary). I have also tried using the full UPN of the user to login. I have tried several RADIUS testing clients (NTRadPing, RadiusTest, etc.), but they either don't support MSCHAPv2 or only support EAP-MSCHAPv2. I have even created my own client using PHP's PECL RADIUS module. Still it always seems to fail with the MSCHAPv2 authentication with an error code of 691. Does anyone have any ideas as to why I always get an invalid username or bad password response when I have done everything possible to ensure that is not the case?
Here are the specs for our RADIUS configuration:

Windows Server 2012 R2
SQL Server 2012 Back End Database for accounting.
The server has been authorized on the domain and is a member of the "RAS and IAS Servers" group. For which that group does have access to the accounts we are testing with.
The accounts we are testing with do have the "Control access through NPS Network Policy" option checked under their "Dial-in" property tab.
RADIUS clients configured to simply match on the IP address which you can see from the events above that it is applying the client friendly name.
Connection Request Policy: The "SBC Authenication" policy is being applied as seen above. The only condition is a regex expression that does successfully match the friendly name.
Network Policy: As seen in events above, none are getting applied. For troubleshooting purposes I have created a Network Policy that is set to "1" for the processing order and its only condition is a Day and Time Restriction currently set to any time, any day.
The authentication method is set to only MSCHAPv2 or MSCHAPv2 (User can change password after it has expired). I have tried adding this to just the Network Policy and I have also tried adding this to the Connection Request Policy and setting it to override the authentication method of the Network Policy.
We do have other RADIUS servers in our domain that use PEAP to authenticate wireless clients and they all work fine. However, we need this to work with MSCHAPv2 only (No EAP).
All other configurations are set to the defaults.

The only other things of note to consider is the fact that in the events above you can see that the Security ID is "NULL SID". Now I know this is common especially among failed logons but given that this issue is stating an invalid username or bad password, perhaps it matters in this case. Also, this server has been rebuilt using the same computer account in Active Directory. I do not know if it would have worked before the rebuild. Essentially we built this server and only got as far as authorizing the server to the domain and adding SQL when we decided to separate out the SQL role onto another server. Rather than uninstalling SQL we just rebuilt the machine. However, before reinstalling Windows I did do a reset on the computer account. I don't think this should matter but thought I would point it out if there is some weird quirk where reusing the same SID of a previously authorized NPS server would cause an issue.
All in all it is a fairly basic setup and hopefully I have provided enough information for someone to get an idea of what might be going on. Apologies if my understanding of this seems a bit basic, after all, when it comes to RADIUS servers I guess you could say I'm the new guy here.
Edit 1:
In an attempt to further troubleshoot this issue I have tried bringing up additional servers for testing. Here are the additional tests I have performed.
Multiple Domains
I have now tried this in 3 different isolated domains. Both our test and production domains as well as my private home domain which has very little in the way of customizations aside from the modifications made for Exchange and ConfigMgr. All have the same results described above.
VPN Service
Using Windows Server 2012 R2 we brought up a separate server to run a standard VPN setup. The intent was to see if we could use RADIUS authentication with the VPN and if that worked we would know the issue is with the SBCs. However, before we could even configure it to use RADIUS we just attempted to make sure it worked with standard Windows Authentication on the local VPN server. Interestingly, it too fails with the same events getting logged as the RADIUS servers. The client machine being a Windows 8.1 workstation. Again I point out that we have working RADIUS servers used specifically for our wireless environment. The only difference between those RADIUS servers and the ones I am having problems with is that the working wireless servers are using PEAP instead of MSCHAPv2.
FreeRADIUS
Now I'm no Linux guru but I believe I have it up and running. I am able to use ntlm_auth to authenticate users when logged on to the console. However, when the radiusd service tries to use ntlm_auth to do essentially the same thing it fails and returns the same message I've been getting with the Windows server (E=691). I have the radiusd service running in debug mode so I can see more of what is going on. I can post the debug info I am getting if requested. The lines I am seeing of particular interest however is as follows:
(1) ERROR: mschap : Program returned code (1) and output 'Logon failure (0xc000006d)'
(1) mschap : External script failed.
(1) ERROR: mschap : External script says: Logon Failure (0xc000006d)
(1) ERROR: mschap : MS-CHAP2-Response is incorrect

The thing to note here is that while we are essentially still getting a "wrong password" message, the actual status code (0xc000006d) is slightly different than what I was getting on the Windows Servers which was (0xc000006a). From this document you can see what these codes mean: NTSTATUS values . The good thing about this FreeRADIUS server is that I can see all of the challenge responses when it is in debug mode. So if I can wrap my head around how a MSCHAPv2 response is computed I can compare it to see if this is simply a miscomputed challenge response. Update Was just noticing that the 6a code is just the sub-status code for the 6d code. So nothing different from the Windows Servers, I still wonder if there is a computation error with the challenge responses though.
Currently, I am working on bringing up a Windows Server 2008 R2 instance of a RADIUS server to see if that helps at all. However, I would be surprised if something with the service broke between W2K8 R2 and W2K12 R2 without anyone noticing until now. If this doesn't work I may have to open a case with Microsoft. Update: Same results with W2K8 R2.
Update 2
I opened a case with Microsoft and they worked on it for a couple of weeks. The first week I spent my time just trying to get them to understand this has nothing to with wireless and that the device we are trying to connect to does not support authentication against a RADIUS server using any form of EAP. Once they finally understood that we are trying to setup the authentication method as just MSCHAPv2 only, his initial reaction was simply "you can't do that". He said, that no matter what you always need some form of EAP or PEAP setup in the top box (even for PAP and other "Less secure methods" in that authentication method window).  This seemed highly unlikely to me so I asked for some documentation stating this for which he then change the subject and never provided any documentation. It became clear I was getting nowhere with Microsoft when they told me that the issue seems to be with the username and password. So it took them 2 weeks to come to that conclusion when the subject line of my premiere ticket had the E691 error code in it which means mismatched username and password and despite the very lengthy paragraph of me explaining that I have done everything possible to ensure it is not a bad username and password.
Unfortunately my project lead does not want to waste anymore premiere support hours on this so we have closed the case. We will likely just deal with the SBC local shared login and setup some other way to enforce accountability (there is only 4 of us that will have access).
I will point out that before I was told to drop the project I did get the FreeRADIUS server to work using MSCHAPv2 only but was only able to do this using accounts local to the FreeRADIUS server. That is plain text passwords stored in a FreeRADIUS config file somewhere. So obviously not a solution but it at least shows that the SBC is correctly communicating to a RADIUS server via MSCHAPv2. This and the other things I have mentioned above lead me to believe that the issue lies between the RADIUS server and the Domain Controllers. While NTLM authentication works fine on both the Windows RADIUS and FreeRADIUS servers while logged into the servers locally (Can login to the Windows RADIUS via the test account and can get successful authentication on the FreeRADIUS server when using ntlm_auth command with just a username and password), neither RADIUS server seems to authenticate via users when coming in as a RADIUS access request (FreeRADIUS uses the same ntlm_auth command I use when logged in locally but instead of providing a username and password to the command it provides a username and challenge response).
So I will end this thread here but I will keep an eye on this and it will notify me if anyone posts something. If someone posts a solution or has comments I will respond.
Comment: I'm impressed w/ your troubleshooting so far and in your description of the issue and the environment. I don't know that I can get you an answer, but I wanted you to know that you're definitely going about things in the right way. Bravo!
Comment: Can you set the Acme device to run PAP? (It would appear that the "authentication-methods" command under "security/authentication/radius-servers" will let you do that.) That'll put the password into an encoded plaintext on the wire, where you can at least see if the Acme is doing something dumb to the password (munging it up somehow).
Comment: @EvanAnderson Right now our SBCs are still have a few unrelated components that are being configured by the vendor consultant. So at the moment we temporarily only have read access to the configuration of the SBCs and the vendor consultant is out on vacation until next week. When he gets back we can have him try that and I will let you know what I see in the packet capture. I can say that with the ability to read the config I have checked multiple times and saw that the password was correct but yes, would be good to test whether it is being passed to the server as seen in the config.
Comment: The statement about a password "in the config" confuses me. The configuration will contain the RADIUS secret, but the password we're talking about would be the password supplied by a user attempting to authenticate to the SBC device.
Comment: My comment was poorly worded....I will do a packet capture to get the username/password combo from the test account attempting to authenticate. What I was referring to was that I had read somewhere that you can get the MSCHAPv2 E=691 message when the issue is actually a bad or malformed shared secret. I was just adding that it looked good in the config.
Answer: Solution
Wouldn't you know it, about a month after we had abandoned this approach to using a RADIUS server to control access to the SBC we find a possible solution. Of course at this point we are too busy with other projects to go back and try this solution out so I can't say 100% if this would fix the problem but once I explain you will probably agree this is the answer.
One of my colleagues was at a Microsoft conference having various discussions when it dawned on him that MSCHAPv2 relies on NTLM to generate the password challenges and responses. Now plain old MSCHAP and MSCHAPv2 (i.e. not EAP-MSCHAPv2 or PEAP) when used in Windows RAS services will use NTLMv1 by default.
As many of of you have already started to catch on, we, like many administrators, have disabled NTLMv1 on our DCs and as such the DCs will only accept NTLMv2 requests. This explains why the failure I continued to get was a "bad password" error. The password being sent to the DCs was in NTLMv1 format and was getting ignored.
Once we realized this, I was able to do some more research and I found the following article:
http://support.microsoft.com/kb/2811487
This article describes the same behavior I was experiencing including the E=691 error code I have mentioned. This article also provides a workaround to force RAS services to use NTMLv2 when building a MSCHAPv2 response. Funny how easy it is to find these articles after you know precisely what the issue is.
Again, I have yet to verify this is the issue as I haven't had time to rebuild the RADIUS servers that I have already decommissioned but I do plan on trying this out sometime if I get a chance. I just wanted to post this possible solution in case someone else stumbles across this issue. If anyone can confirm this before I do please let me know.
Edit - Confirmed
We were asked to take on a bigger role with these SBCs and as such we came back to this project and brought up a Windows RADIUS server again. This time we applied the registry key described in the link above. After taking a packet capture of the communication between the RADIUS server and the SBCs I can in fact now see "Access Accept" messages getting fired back at the SBCs. So I can now confirm at least in our scenario that the issue we were having is as described above.
TL;DR
NTLMv1 was disabled on the DCs. Setting a registry key to force the RADIUS server to use NTLMv2 fixed the issue.
Answer: From my experience MS NPS tells you if RADIUS pre-shared key doesn't match. Something like this:
Event 14: A RADIUS message was received from RADIUS client x.x.x.x with an invalid authenticator. This is typically caused by mismatched shared secrets. Verify the configuration of the shared secret for the RADIUS client in the Network Policy Server snap-in and the configuration of the network access server.

You'll get it in the event log.
That fact you see Access-Reject in your captured traffic also means that pre-shared key matches on both sides otherwise NPS just wouldn't reply and you would see no further RADIUS packets after Access-request.
What is also important in the logs you provided is:
Sub Status:     0xC000006A

This means that "user name is correct but the password is wrong". It's pretty explicit on my view.
I know that MS-CHAPv2 doesn't require to store password using reversible encryption but just to check this guess could you set this checkbox for the user account you use, re-set its password after that and try it again.
Comment: I would have to agree with you, I was a bit skeptical about the shared secret being the issue as well. My comment above was based on forum posts from people who swear that is what fixed their issue. I only included it in this post as I am starting to grasp at straws here to get this working.
Comment: I did try setting the reversible password option for the test account and then reset the password on it. Got the same E=691 error message. I also tried MSCHAPv1 and CHAP via my php "client" and it failed with the same error. That said, I don't know for certain I implemented my php client properly so that may not mean anything. Your sub status comment should help. I verified this by attempting to login with a username I know doesn't exist and the sub status code changed to 0xC0000064. So at least I know it is the password it is failing on. Now to figure out why.
Comment: I should clarify, the above test with CHAP returned a "Reason Code: 19". Essentially stating that it could not find a reversible password for the account despite the fact that I had enabled it on the account, waited for replication, and then reset the password to make sure there was a reversible password stored with the account.
Comment: Your last note is interesting. You enabled reversible encryption and re-set password but NPS says it can not find password encrypted with reversible encryption. It appears that your NPS uses wrong/out of sync AD server as backend.
Comment: I find that odd as well. It is definitely using the correct DCs, confirmed this in the packet captures. I did wait a good while after setting the option to reset the password. I supposed I could try again and just force AD replication to ensure it has updated everything. Though I do wonder if there is perhaps a default domain policy of some kind that prevents this insecure method of authentication, I haven't found any to that affect myself but I could be missing something.
Answer: FYI - After upgrading a working RADIUS + MSCHAP system, I was getting the same error.
"\000E=691 R=1"
Turns out there was a samba change of some sort that threw off the rights to the winbind socket.   Adding freerad to the winbindd_priv group fixed the issue.
/etc/group:
winbindd_priv:x:110:freerad
Comment: Yes, that was a common problem I came across when doing research on the issue. However, in my case all the proper permissions were setup correctly and in addition to that the main issue was trying to get this to work with a Microsoft RADIUS server. That said, we recently discovered what may actually be the problem in our case. I haven't had a time to verify it yet but once I do I had planned on coming back here to post the answer.
Answer: We had the same problem with authentication from our NPS servers after adding new 2012R2 domain controllers to an existing 2008R2 environment. The new 2012R2 domain controllers had NTLMv1 disabled where the 2008R2 domain controllers had it enabled.
The NPS servers (running 2008R2) where randomly denying access for users.
The following event was logged on the NPS servers:
Event ID 6273 (Security log)
Network policy server denied access to a user.
Reason code: 16
Reason: Authentication failed due to a user credentials mismatch. Either the user name provided does not map to an existing user account or the password was incorrect.
When the NPS servers connected to the 2008R2 dc's everything worked like a charm. But on the 2012R2 dc's access was denied. All because NTLMv1 was disabled on the new 2012R2 domain controllers.
I can confirm that the solution in the MS KB article (http://support.microsoft.com/kb/2811487) works perfectly!.
Thank you very much for this excellent article!!! It saved me a huge amount of time in solving this issue ; - )