website contact form (site sucker website)

Question: I have downloaded a website using site sucker and I have all the html files. The website was developed using a CRM (it's my new client's website which I have to upload to my server asap and they don't have credentials for the same). I have a contact form which I want to make it work
Form code:
<form name="contactform" method="post" class="dmRespDesignRow" locale="ENGLISH">

        Name:
        <input name="dmform-0" type="text" id="1049346235"/>
        <input name="name" value="Name" type="hidden" id="1808095320"/>

        Email:
        <input name="dmform-1" type="email" id="1874224805"/><input name="email" value="Email" type="hidden" id="1790388701"/>

        Phone:
        <input name="dmform-2" type="tel" id="1363423394"/><input name="phone" value="Phone" type="hidden" id="1562324064"/>

        Message:
        <textarea name="message" id="1731760981"></textarea>
        <input name="message2" value="Message" type="hidden" id="1794081407"/>

        <input class="" name="submit" value="Send" type="submit" id="1446531284"/>

    <input class="dmActionInput" name="action" value="form_email.php" type="hidden" id="1733868877"/><input name="dmformsubject" value="Form Message" preserve_css="true" type="hidden" id="1339178452" data-success-page=""/>
</form>

I tried creating a new file i.e form_email.php and adding as  action in the form but no luck. If I create a separate form test file (plain html without any classes) it works using that php file but doesn't work when integrating it with the actual form. I am not able to figure out what's the issue.
Answer: If you are not using JavaScript to intercept the form the action attribute should be added to the form tag, and not to an input. Example:
<form name="contactform" method="post" action="form_email.php" class="dmRespDesignRow" locale="ENGLISH">
In addition you must make sure that form_email.php is being correctly requested. If the file's location is not within the same directory the HTML file is (or the directory in which the CMS designated as such) you must fix the action attribute so that it points to the correct location. For example, if form_email.php is one level up (in the directory above the HTML file) you would reference it as such: action="../form_email.php".


How to make condition inside GridView Yii2

Question: I have problem here.
I want to add a condition inside the GridView
<?= GridView::widget([
    'dataProvider' => $dataProvider,
    'filterModel' => $searchModel,
    'columns' => [
        ['class' => 'yii\grid\SerialColumn'],
        'table_column1',
        'table_column2',
        //CONDITION HERE
        if(condition){
            //BUTTON HERE
        } else {
            //TOOLS COLUMN HERE
        }
    ],
]); ?>

How can I do that?
Comment: the grid layout is the same for all rows  oan change between each row ?  in others words you need  alternative gridview  .. or a configurable gridview  row by row ?
Answer: You can use $visible property if you want only show/hide some columns based on specified condition:
<?= GridView::widget([
    'dataProvider' => $dataProvider,
    'filterModel' => $searchModel,
    'columns' => [
        ['class' => 'yii\grid\SerialColumn'],
        'table_column1',
        'table_column2',
        [
            'name' => 'some_column1',
            'visible' => $condition,
        ],
        [
            'name' => 'some_column2',
            'visible' => !$condition,
        ],
    ],
]); ?>

Based on $condition, only one of two columns (some_column1 or some_column1) will be visible.
Comment: can you please see my new [question](https://stackoverflow.com/q/61341463/8919244) related to it ?
Answer: If you need  two gridview one alternative to the other  a simple way is based on alternate $column contents
if ( condition) {

    $columns  = [
        ['class' => 'yii\grid\SerialColumn'],
        'table_column1',
        'table_column2',
          [
            'attribute'=>'your_att',
            'header'=>'your_header',
            'format'=>'raw',
            'value' => function($model, $key, $index)
            {
                    return '<button class="btn green">Y</button>';
            },
        ],
        ],

    } else {
        $columns  = [
        ['class' => 'yii\grid\SerialColumn'],
        'table_column1',
        'table_column2',
         ['class' => 'yii\grid\ActionColumn',
        ],
    }
}

echo GridView::widget([
  'dataProvider' => $dataProvider,
  'filterModel' => $searchModel,
  'columns' => $columns
]); ?>

otherwise uisng the value function for a raw column type you could return the code you need  for each specific rows ..
Comment: It helped me a lot but I have another issue can you please see my related [question](https://stackoverflow.com/q/61341463/8919244)?
Answer: Here is simple if else condition inside gridview
   <?= GridView::widget([
    'dataProvider' => $dataProvider,
    'columns' => [
        ['class' => 'yii\grid\SerialColumn'],

        //'id',
        'event_id',
        //'img:ntext',
        //'path:ntext',
        //'type',
        //'description:ntext',
        [
            'attribute' => 'type',
            'value'=> function($data){
                if($data->type == 0){
                    $info = 'Empty';
                }
                if($data->type == 1){
                    $info = 'One';
                }
                if($data->type == 2){
                    $info = 'Two';
                }
                return $info;
            },
            'format'=>'html',
        ],

        ['class' => 'yii\grid\ActionColumn'],
    ],
]); ?>

Comment: Please explain your solution. Answers which do not have an explanation and are only code get flagged as low effort.
Answer:  [
            'class' => 'kartik\grid\EditableColumn',
            'attribute' => 'status',
            'label'=>'Status',
             'filterType' => \kartik\grid\GridView::FILTER_SELECT2,
                'filter' => ArrayHelper::map(\app\models\PropertyStatus::find()->all(), 'status', 'status'),
                'filterWidgetOptions' => [
                    'theme' => Select2::THEME_BOOTSTRAP,
                    'pluginOptions' => [
                        'allowClear' => true,
                    ],
                ],
                'filterInputOptions' => ['placeholder' => 'All...'],
                'format'=>'raw',
            //'options' => ['style' => 'width: 8%'],

            'editableOptions'=> function ($model, $key, $index, $widget) {
                if(Yii::$app->user->can('property/status')){
                    $appttypes = ArrayHelper::map(\app\models\PropertyStatus::find()->all(), 'status', 'status');
                    return [
                    'header' => 'Status',
                    'attribute' => 'status',
                    'asPopover' => false,
                    'inlineSettings' => [
                        'closeButton' => '<button class="btn btn-sm btn-danger kv-editable-close kv-editable-submit" title="Cancel Edit"></button>'
                    ],

                    'type' => 'primary',
                    //'size'=> 'lg',
                    'size' => 'md',
                    'options' => ['class'=>'form-control', 'placeholder'=>'Enter person name...'],
                    'inputType' => Editable::INPUT_DROPDOWN_LIST,
                    'displayValueConfig' => $appttypes,
                    'data' => $appttypes,
                    'formOptions'=> ['action' => ['/ajax/update-property-status']] // point to new action
                ];
                }
            },

        ],

Comment: Welcome to Stack Overflow. This might be an answer, but it is always appretiated to explain your answer rather than just pasting code.


What is a natural sharp?

Question: I'm looking the allstate music (as my class is required to pass it off) the piece is in concert c minor. I play flute and on my f(which is already sharp) there is a natural sign and then another sharp sign. I haven't seen this before so I'm hoping someone will know
Comment: A pic with all of the previous bar would make it better to understand. And what do you mean - my F is already sharp?
Comment: @phoog The reasons for the double accidental are different, but otherwise the questions is exactly the same: it's a (imho unnecessary) courtesy accidental in either case.
Comment: Are you sure it's not in C sharp minor?  C minor has a key signature of three flats; C sharp minor has four sharps.
Comment: @PiedPiper that question is about a note that is raised two semitones above the diatonic; this one is about cancelling an accidental to restore a note back to the diatonic from being raised one semitone.  In other words, I think the fact that we're differentiating from a double sharp here vs. from a single flat there makes the similarities rather abstract, perhaps too much so for many who are at the point in their studies where they are encountering this for the first time.
Answer: The natural is there as a cautionary - in the previous bar there may well have been an accidental for that same F note (guess it's treble clef), most likely an Fx, so it's just saying 'this F is an F♯ now'. Followed by an Fx (F double sharp, same pitch as G).
It's just a reminder - you play that F as an F♯.
Comment: Right, it's as if the natural "cancels out one of the sharps" of a double sharp. (I suppose it's even possible that it's using it as a courtesy gesture since the following note is double-sharp, though to my mind that would confuse more than clarify...)
Comment: @AndyBonner at the left edge of the image is the right half of a notehead on the F line.  That is surely an F double sharp.
Comment: So do you believe the natural is supposed to make the score easier to read? I'd say it doesn't fulfill this task well...
Comment: @user1079505 - it's not really about what I believe, it's about what is sometimes deemed 'a good thing'. Personally, I'm o.k. either way. But - it makes it very clear what note gets played. Not so good sight-reading, maybe, but for reassurance.
Comment: @phoog - if it is, the the G must be sharp, too. But those notes in key C minor..?
Comment: Oh, I didn't notice the statement that the piece is in C minor.  My first guess is that it's actually in C sharp minor.  F "which is already sharp" suggests that there is an F sharp in the key signature.
Comment: @phoog - and a G sharp, no doubt. You're not part of the Holmes family, are you?
Comment: @user1079505 it's standard practice to cancel a double sharp this way.  I suppose a lone sharp is seen as confusing since normally a lone sharp  means the note is a half-step higher than it would have been without the accidental, but here it's a half-step lower.
Comment: @phoog  It WAS standard practice.   See my answer, and its comments.
Answer: It's the old notation for a normal sharp when double-sharps (as on the following note) are in the vicinity.  Possibly the F in the preceding bar (which you have just avoided showing us!) was a double-sharp?   In that case, the 'natural-sharp' notation, though not strictly necessary, could be a useful cautionary.
The more modern style is just to write the sharp, even after a double-sharp in the same bar.  The natural is not used.   But you'll find plenty of older music that does it this way.   We could compare it with the old style of cancelling a key signature with naturals before writing the new one.  Now we just write the new one - naturals are only needed when moving into the open key with no sharps or flats!
Gould (2011) refers to 'traditional' and 'contemporary' practice. "When a double flat is cancelled by a single flat, and a double sharp by a single sharp, the traditional practice of placing a natural sign before these is redundant, since a single flat or sharp sign cannot mean anything else." Garner Read (1969) concurs. "To cancel a double sharp, one now merely writes a single sharp-sign."

Are you SURE this piece is in C minor?  I'd expect a key signature of at least three sharps in this context!   Perhaps C♯ minor?
Comment: Old notation? When did this change?
Comment: I can't give an exact date!  Gould (2011) just refers to 'traditional' and 'contemporary' practice.  "When a double flat is cancelled by a single flat, and a double sharp by a single sharp, the traditional practice of placing a natural sign before these is redundant, since a single flat or sharp sign cannot mean anything else." Garner Read (1969) concurs. "To cancel a double sharp, one now merely writes a single sharp-sign."

What is a natural sharp?


Convergence rate of an iterative scheme

Question: Given an iteration scheme of the form : $x_{k+1}=x_k+t_kd_k$. It is also known that there are $\delta,C>0,D\geq 0$ such that if $x_k \in B_{\delta}(x_*)$ holds we have the following estimate:
$ ||x_{k+1}-x_*||\leq C(||x_k-x_*||+D)||x_k-x_*||$.
What is the convergence rate of the iteration scheme if $C=D$ ?
I've tried to plug in $C=D$ inside this expression and tried to expand and see what the RHS looks like. I thought I might be able to factor out some term to make it look nice but was unable to do so. Any suggestions as to how to determine the rate of convergence of this scheme?
Answer: Let $d_k = ||x_k-x_*||$.
We are given
$d_{k+1} \le C d_k(D+d_k)
=
CD d_k + C d_k^2
$.
This becomes
$\begin{align}
d_{k+1}
&\le C d_k(d_k+D) \\
&\le CD d_k+C d_k^2\\
&\le C D(CD d_{k-1}+C d_{k-1}^2)+C (CD d_{k-1}+C d_{k-1}^2)^2\\
&= C^2D^2 d_{k-1}+C^2 D d_{k-1}^2+C^3d_{k-1}^2 ( D+ d_{k-1})^2\\
&= C^2 D^2d_{k-1}+C^2 Dd_{k-1}^2+C^3d_{k-1}^2 ( D^2+ 2D d_{k-1}+d_{k-1}^2)\\
&= C^2 D^2d_{k-1}+C^2 Dd_{k-1}^2+C^3D^2d_{k-1}^2 + 2C^3D d_{k-1}^3+C^3d_{k-1}^4\\
&= C^2 D^2d_{k-1}+C^2D(1+CD) d_{k-1}^2+2C^3Dd_{k-1}^3+C^3d_{k-1}^4\\
\end{align}
$
If $D=C$,
this becomes
$d_{k+1}
\le C^4d_{k-1}+C^3(1+C^2) d_{k-1}^2+2C^4d_{k-1}^3+C^3d_{k-1}^4
$.
It looks like
$d_{k+1} = C^{m+1}D^{m+1}d_{k-m} + O(d_{k-m}^2)$
for integer $m \ge 0$.
Looking at this,
there doesn't seem to me
to be any obvious reason
why $D=C$ would be special.
$D=0$ would certainly be.

Convergence rate of an iterative scheme


(solved: method factor) Does low cross-type covariances (compared to within-type cross-factor) imply a bad measure?

Question: I'm not sure about the right terminology, but here goes.
I have a questionnaire measure (30 likert-items) that has been traditionally divided into five six-item factors. Each factor has two different types of items, which represent two different approaches to formulate the questions (basically, one asks how you feel about the thing in principle, the other asks you to judge examples related to the thing).
I collected some data, and had a poor-ish fit for my SEM (RMSEA .078), which prompted me to look for what's wrong. I ended up trying a model that separates the two types for each factor into a factor of their own, resulting in a 10-factor model (with only three items per factor). I don't know much about psychometrics, but I would intuitively imagine that if the original factors are actually valid, the covariances between the two separated factors (that are supposed to represent the same theoretical construct) should be higher than, say, the covariances between different factors. I found the opposite:

(I've listed here all the factor-factor covariances and marked what kind of relationship the factors have: "group" means covariance between factors that are theoretically linked; "type" means the two types of questions, denoted as _j and _r in the factor names; w and x are within and cross.)
My intuition - without much expertise - says that there is something wrong with this measure. Is this kind of comparison meaningful at all? Do you agree with my intuition? If so, what could be wrong with the measure? Any suggestions for how to proceed?
Answer: It sounds to me like you might have some method factors that you need to account for. These are dealt with in CFA using multi-trait multi-method (MTMM) models.
I don't think it means you have a bad measure.
Comment: Thanks. Can you point me to a good source on method factors and MTMM?
Comment: https://www.socialresearchmethods.net/kb/mtmmmat.php


Why do I have to cast the returned value?

Question: To me it seams that the function I call already returns the type I have to cast it to. Why do I have to do that?
T[]leftPart=(T[]) sort(copyOfRange(a,0,middle));

The method copyOfRange that I created supposed to return the same type as the parameter a.
Here is the code:
    static public   <T extends Comparable<T>> Object[] sort(T[] a){
    int middle=a.length/2;

    T[]leftPart=(T[]) sort(copyOfRange(a,0,middle));
    }

static public<T> T[] copyOfRange(T[] original,int from,int to){
    int size=to-from;
    T[] retObject=(T[])new Object[size];
    for(int i=0;i<size;i++) retObject[i]=original[from+i];
    return retObject;
}

Comment: `sort` is declared as returning `Object[]`, not `T[]`.
Comment: I reduced the title to the core problem and moved the rest to the question statement. Try to keep the question title concise.
Answer: copyOfRange returned T[], but you put it in sort() and sort() returned a Object[].


Rails form inline errors

Question: I have a large form with a lot of fields, The way we get back errors in rails is that the they reside inside the model object. i want a clean way to show errors besides each field instead of showing the separately. Is there a gem that can do this and if now what can be the best way to do it.
I want each error to show besides the field the error is on.
Comment: Get the errors for each field (`@object.errors[field_name]`) and iterate over them in the form. I think some of the form gems do this, but I'm not sure which one(s).
Answer: Assuming you are using Rails 3+ and the Rails Form Helpers, you do not need any gems and you do not need to iterate over the errors collection.
Well, that's not entirely true, but Rails does it for you. You just need to provide a proc.
I add the following to  .../config/application.rb:
# wrap each form field that has an error with the following.
# note, condition for arrays and singletons.
config.action_view.field_error_proc = Proc.new do |html_tag, instance|
  if instance.error_message.kind_of?(Array)
    %(#{html_tag}).html_safe
  else
    %(#{html_tag} ).html_safe
  end
end

Make sure to define the CSS class validation-error. That should get it working, then you can tweak it to fit your needs. Note, your CSS can be specific to each field type, e.g., input, textarea, select, etc.
And of course restart the Rails app.
I actually learned this in another thread on SO, but I can't find it now so I can't provide the reference.
Comment: This is exactly what I was looking for but is there a way to pull this out into the lib folder as a class or something. Code in application.rb is not really in the best interest of maintenance
Answer: Try simple_form or formtastic to make this easier. With those, it'll automatically do what you're looking for.
If you want to make your views really long, or just don't want to use a gem, you can also do this:

<%= @object.errors[:field] %>

But that'll get icky fast.
Comment: It's not icky at all, but you should iterate over them (or use a helper) since it's a collection-per-field.

Rails form inline errors


Guitar bridge resonates wherever I play a C#

Question: I have an Ibanez Gio and no matter where I play a C# it causes my guitar bridge to resonate. Please see my example attached to sound cloud where I alternate playing a C# on the G and D string.
https://soundcloud.com/cameron-ward-150102703/brdige-sound/s-omNyk
If I place my hand on the bridge I can stop this from happening, I cant do this every time I play though so my question is is there anything I can adjust on my guitar to stop this from happening?
Many Thanks
Cameron
Answer: On the assumption it does this even without being amplified, there must be something loose on the bridge. I'd check screws and springs first, methodically working through each component. there should be nothing loose which will vibrate sympathetically.
Comment: Yes it does do this without amplification, I can't seem to find anything loose, when it makes this sound the whole bridge vibrates, the entirety of the metal. I can definitely feel this if I put my hand on it and if I press hard enough it stops.
Comment: And you have checked the springs in the back @cameron? They can easily resonate. If they do, wrapping a rubber band round them can help.
Comment: @DoktorMayhem Ooh I haven't tried that, would it matter where I place the bands or not? I think they are defiantly part of the problem tho as they do seem to resonate.
Comment: I don't think it should matter Cameron. They just help damp unwanted resonance. I use rubber bands on three of my guitars that have "wolf tones" - my Gibsons and a Charvel - none of the others have the problem.


Custom Authorization Filtering Minimal API .Net 6

Question: I'm exploring Minimal APIs in .Net 6, and trying to apply a custom Authorization Filter to the endpoint (via Attributes or Extensions).
But it seems to me, I am doing something wrong, or it's simply not designed to work in that way (and it's sad if so).
Couldn't find anything in the docs besides the default usage of [Authorize] attribute in Minimal APIs.
Here is the Filter
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class CustomAuthorizeAttribute : Attribute, IAuthorizationFilter
{
    //Checking tokens
}

And if I try to apply it at Controller level, it works fine
[CustomAuthorize]
public class CustomController : ControllerBase
{
    //Necessary routing
}

But if I switch to Minimap APIs notation and try to use attributes
app.MapGet("/customEndpoint",
        [CustomAuthorize] async ([FromServices] ICustomService customService, Guid id) =>
            await customService.GetCustomStuff(id));

or even an extension method
app.MapGet("/customEndpoint",
        async ([FromServices] ICustomService customService, Guid id) =>
            await customService.GetCustomStuff(id)).WithMetadata(new CustomAuthorizeAttribute());

It just doesn't work. The filter doesn't even being constructed.
What did I miss or did wrong?
Thx in advance
Comment: Why don't you create custom middleware for this, then that will validate every request
Comment: When you use controllers and attributes, the MVC framework is designed to use them the designated way. this does not apply to the custom classes you have created. You might want to research if you can inject the authorization layer in minimal API by some other mean.
Comment: https://auth0.com/blog/securing-aspnet-minimal-webapis-with-auth0/
Comment: Thx, I've got such middleware, actually, but I was curious about what I can reuse directly from MVC in Minimal APIs
Comment: @Beeeg ASP.NET Core MVC includes _a lot_ of plumbing and features that aren't
needed (and so _can't be used_) by minimal APIs - it seems `[Authorize]` is one of those things - _but that's okay_ because honestly the `[Authorize]` attribute itself is pretty awful (while declarative behaviour is nice, there's far too many moving-parts and frustrating restrictions with attribute and declarative-based auth), so you really are much better-off with using middleware for this.
Answer: You can write a custom authorization filter for Minimal API in .NET 6.0
Here is how I tend to approach it - by using Policy-based authorization in ASP.NET Core
Step 1: Create a Requirement
A requirement implements IAuthorizationRequirement
public class AdminRoleRequirement : IAuthorizationRequirement
{
     public AdminRoleRequirement(string role) => Role = role;
     public string Role { get; set; }
}

Note: A requirement doesn't need to have data or properties.
Step 2: Create a Requirement Handler
A requirement handler implements AuthorizationHandler<T>
 public class AdminRoleRequirementHandler : AuthorizationHandler<AdminRoleRequirement>
 {
    public AdminRoleRequirementHandler(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }
    protected override async Task HandleRequirementAsync(AuthorizationHandlerContext context, RoleRequirement requirement)
    {
        if (context.User.HasClaim(c => c.Value == requirement.Role))
        {
            context.Succeed(requirement);
        }
        else
        {

            _httpContextAccessor.HttpContext.Response.StatusCode = StatusCodes.Status401Unauthorized;
            _httpContextAccessor.HttpContext.Response.ContentType = "application/json";
            await _httpContextAccessor.HttpContext.Response.WriteAsJsonAsync(new { StatusCode = StatusCodes.Status401Unauthorized, Message = "Unauthorized. Required admin role." });
            await _httpContextAccessor.HttpContext.Response.CompleteAsync();

            context.Fail();

        }

    }
    private readonly IHttpContextAccessor _httpContextAccessor;
}

Note: HandleRequirementAsync method returns no value. The status of either success or failure is indicated by calling context.Succeed(IAuthorizationRequirement requirement) and passing the requirement that has been successfully validated or by calling context.Fail() to indicate AuthorizationHandlerContext.HasSucceeded will never return true, even if all requirements are met.
Step 3: Configure Your Policy in the Authorization Service
 builder.Services.AddAuthorization(o =>
 {
        o.AddPolicy("AMIN", p => p.AddRequirements(new AdminRoleRequirement("AMIN")));
 });

Step 4: Add Your Requirement Handler to DI
 builder.Services.AddSingleton<IAuthorizationHandler, AdminRoleRequirementHandler>();

Step 5: Apply Policy to Endpoints
 app.MapGet("/helloworld", () => "Hello World!").RequireAuthorization("AMIN");

Comment: The only thing wrong with this is the injecting of the IHttpContextAccessor. Authorization  handlers shouldn't be writing response...
Comment: returning StatusCode from httpContextAccessor is a little bit hacky. More proper is to add Step 6: AuthorizationMiddlewareResultHandler. Here is good example from @Ogglas: https://stackoverflow.com/questions/35656828/return-http-403-using-authorize-attribute-in-asp-net-core
Comment: Noted @davidfowl.
Answer: I think you won't be able to inject action filter in minimal api, you can use 3 alternative approches.

Create a custom middleware and inject it in startup class, it would check every request and do the intended work as you filter is doing. You can put a check for the request path there if you only need to validate a specific controller/endpoint.

The second approach is you can inject httpcontext in minimal api like this, from that extract jwt token and validate that, if found not ok reject that request.

 app.MapGet("/customEndpoint", async (HttpContext context, ICustomService service) =>
 {
     var token = context.Request.Headers["Authorization"].FirstOrDefault()?.Split(" ").Last();
     if (string.isNullOrEmpty(token) || <not a valid token>) return Results.Unauthorized();
     // do some work
     return Results.Ok(result);
 });

as @Dai suggested, you can extract token in this way also
AuthenticationHeaderValue.TryParse(context.Request.Headers["Authorization"], out var parsed ) && parsed.Scheme == "BearerOrWhatever" ? parsed.Parameter : null

You can register the filter globally from startup.cs.
Comment: Yes @Dai agreed, much better


C++ Template Classes and Matlab Mex

Question: I'm wondering if there is an elegant way to write a Matlab/Mex interface to templated C++ code.
Here is the situation: say I have written a very nice set of C++ template classes like so:
template<FooType T>
Foo{
    private:
        //Some data
    public:
        //Constructors, methods
};

template<BarType S>
Bar{
    private:
        //Some data
    public:
        //Constructors, methods
};

template<FooType T, BarType S>
MyClass{
    private:
        Foo<T> *D;
        Bar<S> *M;
    public:
        //Constructors, methods
};

If I have say, n different FooTypes and m different BarTypes, I have n*m different parameter choices for MyClass (FooType and BarType are custom typenames, incedentally).  Now, if I was writing a C++ program to use these classes, it would look very simple:
int main()
{
    Foo<fooType1> *F = new Foo<fooType1>(params);
    Bar<barType3> *B = new Bar<barType3>(params);

    MyClass<fooType1,barType3> *M = new MyClass(F,B);

    M->doThing();

    return 0;
}

I compile main.cpp and run it and rejoice.  This works because I have selected template parameters at compile time, as per C++ template design.  This works very well for me so far, and I like the design.
Now suppose I want to write the same type of program as main.cpp, but using Matlab as the scripting language and a Mex interface to my classes.hpp. The main reason for wanting to do this is that my code is an add-on to an exisiting matlab package.  Is there an elegant way to write my mex file so that I have access to every possible pair of template parameters?  What I have started to do is write my interface file with a lot of switch statements so that I can select FooType and BarType - essentially the Mex file compiles every possible (n*m) class instance and leaves them sitting there for Matlab to use.  This seems OK for now (I have n=3, m=2), but it also seems sloppy and difficult to maintain.
I have thought about making the "user" re-compile the mex file every time they want to choose a different FooType and BarType, but this also seems a bit irritating (to the average Matlab user anyway).  Thanks for your input!
Comment: If you were using Octave or Python, you should consider using SWIG. When using Matlab you should consider using a type library and the Matlab external interface instead of the mex interface. The mex interface requires you to write a lot of unnecessary boiler plate. What you could do is use to SWIG for wrapping C++ to plain C, which can be accessed directly using the external interface
Comment: How do you envision the user indicating what the two types are to the Mex file?
Comment: @JensMunk that could work, but I think I'm far enough down the Mex path that switching to SWIG now would be a huge pain.
Comment: @icurays. Fair enough. I worked for quite some time with the mex interface and actually managed to wrap C++ template objects with Matlab objects using reference counting and a templated static garbajge collector. It was pretty neat. A lot of macros for instantiating templates and validating input arguments. I published a paper about this, but the focus was ultrasound not the coding
Comment: @Praetorian I have been using code similar to [here](http://www.mathworks.com/matlabcentral/fileexchange/38964-example-matlab-class-wrapper-for-a-c++-class).  Basically I have written a Matlab class that mirrors the C++ class, then using Mex I am able to initialize my C++ variables directly from matlab, then call their methods via the Matlab class interface (if that makes sense).  So the user is doing things on the Matlab side like `x = Foo(params); y = Bar(params); z = MyClass(x,y); z.doThing()`.
Comment: @icurays1 Exactly, like I did
Comment: @JensMunk that sounds a lot like what I've done.  If you have example code you would be willing to share (e.g. via github or dropbox), I would be interested in seeing if we've done a similar thing.
Comment: Okay, I can wrap something up and put it on github. It is a bit more advanced that your reference. I'll upload it in about 20 mins
Comment: @JensMunk Awesome, thank you!
Comment: So the type information is hidden with an `mxArray`. You are probably restricted to doing something along the lines of what Jens describes. You should be able to automate the generation of a `switch` statement for the `n*m` cases by using Boost.Preprocessor. Within each `case` statement you'd instantiate the C++ type, and pass a *handle* (probably a `uint64_t`) that contains a pointer to the instantiated object back to the MATLAB world.
Comment: @Praetorian That sounds intruiging.  I will look in to that.  Thanks!
Comment: @icurays1 Here goes. https://github.com/JensMunkHansen/mexClass. It is a small part of a larger library for ultrasound signal processing.
Comment: @icurays1 I use RTTI for identifying objects and validation when objects are deleted using the static garbage collector. The entire library is a collection of 6 shared objects or DLL's, which share many resources.
Comment: @JensMunk very cool, I will take a look. Thanks again!  I'm working on medical imaging related things too =)
Comment: Okay, 10 years ago, I made a mex library using SSE2 for forward projectors for MRI, CT and PET imaging. It is a part of a toolbox made by Jeffrey Fessler from University of Michigan. I stole a lot of macros from Jeffrey Fessler
Comment: I can't really address the issue of runtime template definitions (but perhaps explicit instantiation of every conceivable type, or a clever polymorphism scheme with `dynamic_cast `), but I have a modern (i.e C++11) approach to persistent/stateful MEX files. Perhaps it could help along the way: https://github.com/chappjc/MATLAB/tree/master/cppClass.  It is a lightly tested experiment.  I did it this way because I disliked violating RAII and passing a pointer to MATLAB in a uint64.
Answer: I'm not familiar with Mex; I'm just a C++ user.  What you describe is not possible without direct MATLAB support for run-time library generation, which I don't think is even a thing that exists.  You will definitely need to pre-instantiate everything at compile time.  This is not simple.
The problem, as you have said, is that the templates are all generated at compile time, and they aren't used until runtime, when all the type information is gone.  The mangled names shouldn't be an issue, assuming MATLAB knows the mangling scheme.
So you'll have to solve this at the C++ level.  In C++, there's no type-safe way to get the name of a type as a string.  The only solution is to use macros to eliminate some of the bloat.  This is unfortunate, because if it were possible, a very elegant solution would be possible with a std::tuple of all your valid parameter types, and a little recursive template magic.

C++ Template Classes and Matlab Mex


Can't deep learning models now be said to be interpretable? Are nodes features?

Question: For statistical and machine learning models, there are multiple levels of interpretability: 1) the algorithm as a whole, 2) parts of the algorithm in general 3) parts of the algorithm on particular inputs, and these three levels split into two parts each, one for training and one for function eval. The last two parts are much closer than to the first. I'm asking about #2, which usually leads to better understanding of #3). (if those are not what 'interpretability' means then what should I be thinking?)
As far as interpretability goes, logistic regression is one of the easiest to interpret. Why did this instance pass the threshold? Because that instance had this particular positive feature and it has a larger coefficient in the model. It's so obvious!
A neural network is  the classic example of a model that is difficult to interpret. What do all those coefficients mean? They all add up in such complicated crazy ways that it is hard to say what any particular coefficient is really doing.
But with all the deep neural nets coming out, it feels like things are becoming clearer. The DL models (for say vision) seem to capture things like edges or orientation in early layers, and in later layers it seems like some nodes are actually semantic (like the proverbial 'grandmother cell'). For example:

(from 'Learning About Deep Learning')
This is a graphic (of many out there) created by hand for presentation so I am very skeptical. But it is evidence that somebody thinks that is how it works.
Maybe in the past there just weren't enough layers for us to find recognizable features; the models were successful, just not easy to post-hoc analyze particular ones.
But maybe the graphic is just wishful thinking. Maybe NNs are truly inscrutable.
But the many graphics with their nodes labeled with pictures are also really compelling.
Do DL nodes really correspond to features?
Comment: I don't see the premise of this question. That neural nets have become more complex and give better predictions doesn't make them any more  interpretable. The opposite is usually true: complexity/better prediction <-> simplicity/better interpretation.
Comment: @AdamO is exactly correct.  Because of that, regression trees (recursive partitioning) are only intepretable because the results are **wrong**.  They are wrong because they are volatile; get a new sample and the tree can be arbitrarily different.  And single trees are not competitive with respect to predictive discrimination.  Parsimony is often the enemy of predictive discrimination.  And to the original question, in the biomedical field, AI/ML results have not been interpretable.
Comment: The bold question in the body and the question in your title are very different. It looks like all of the answers, including mine, are addressing the question in the title. Perhaps you could ask the narrower question about nodes and features in its own thread? But before you do that, consider that you've already linked to a paper that answers your bold question in the affirmative, so consider what, precisely, you'd like to learn in an answer before asking.
Comment: See this article *AI researchers allege that machine learning is alchemy*   http://www.sciencemag.org/news/2018/05/ai-researchers-allege-machine-learning-alchemy
Comment: @Sycorax The link I just added is to a blog post, not a paper, and so I am very skeptical of the affirmative view. The variety of interpretability that I ask about DL in the title I consider to be the one in bold in the text.
Comment: Another perspective on the question is that historically NNs have been considered _entirely_ uninterpretable; doesn't treating nodes as features allow some interpretation? And if not, why not? (the adversarial examples are good here)
Comment: A recent line of research shows that neural networks make predictions by finding similar training instances to a test instance.
You can find more information in the following paper:
[GPEX, A Framework For Interpreting Artificial Neural Networks](https://arxiv.org/abs/2112.09820)
Answer: Interpretation of deep models is still challenging.

Your post only mentions CNNs for computer vision applications, but (deep or shallow) feed-forward networks and recurrent networks remain challenging to understand.
Even in the case of CNNs which have obvious "feature detector" structures, such as edges and orientation of pixel patches, it's not completely obvious how these lower-level features are aggregated upwards, or what, precisely, is going on when these vision features are aggregated in a fully-connected layer.
Adversarial examples show how interpretation of the network is difficult. An adversarial example has some tiny modification made to it, but results in a dramatic shift in the decision made by the model. In the context of image classification, a tiny amount of noise added to an image can change an image of a lizard to have a highly confident classification as another animal, like a (species of) dog.

This is related to interpretability in the sense that there is a strong, unpredictable relationship between the (small) amount of noise and the (large) shift in the classification decision. Thinking about how these networks operate, it makes some sense: computations at previous layers are propagated forward, so that a number of errors -- small, unimportant errors to a human -- are magnified and accumulate as more and more computations are performed using the "corrupted" inputs.
On the other hand, the existence of adversarial examples shows that the interpretation of any node as a particular feature or class is difficult, since the fact that the node is activated might have little to do with the actual content of the original image, and that this relationship is not really predictable in terms of the original image. But in the example images below, no humans are deceived about the content of the images: you wouldn't confuse the flag pole for a dog. How can we interpret these decisions, either in aggregate (a small noise pattern "transmutes" a lizard into dog, or a flagpole into a dog) or in smaller pieces (that several feature detectors are more sensitive to the noise pattern than the actual image content)?
HAAM is a promising new method to generate adversarial images using harmonic functions. ("Harmonic Adversarial Attack Method" Wen Heng, Shuchang Zhou, Tingting Jiang.) Images generated using this method can be used to emulate lighting/shadow effects and are generally even more challenging for humans to detect as having been altered.
As an example, see this image, taken from "Universal adversarial perturbations", by
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. I chose this image just because it was one of the first adversarial images I came across. This image establishes that a particular noise pattern has a strange effect on the image classification decision, specifically that you can make a small modification to an input image and make the classifier think the result is a dog. Note that the underlying, original image is still obvious: in all cases, a human would not be confused into thinking that any of the non-dog images are dogs.

Here's a second example from a more canonical paper, "EXPLAINING AND HARNESSING ADVERSARIAL EXAMPLES" by Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy. The added noise is completely indistinguishable in the resulting image, yet the result is very confidently classified as the wrong result, a gibbon instead of a panda. In this case, at least, there is at least a passing similarity between the two classes, since gibbons and pandas are at least somewhat biologically and aesthetically similar in the broadest sense.

This third example is taken from "Generalizable Adversarial Examples Detection Based on Bi-model Decision Mismatch" by João Monteiro, Zahid Akhtar and Tiago H. Falk. It establishes that the noise pattern can be indistinguishable to a human yet still confuse the classifier.
For reference, a mudpuppy is a dark-colored animal with four limbs and a tail, so it does not really have much resemblance to a goldfish.

I just found this paper today. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus. "Intriguing properties of neural networks". The abstract includes this intriguing quotation:

First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.

So, rather than having 'feature detectors' at the higher levels, the nodes merely represent coordinates in a feature space which the network uses to model the data.
Comment: I would also point out that even simple models such as logistic regression are vulnerable to adversarial attacks. In fact, [Goodfellow et. al.](https://arxiv.org/abs/1412.6572) points out that it is shallow models which lack the capability to resist such attacks. Yet we still claim to be able to interpret logistic regression.
Comment: Good points. But note that even in the simplest models (logistic, decision trees) it is obscure why any particular coefficient /threshild is what it is (but that's not the same interpretability I was asking about). Not totally unrelatedly, are there any good adversarial studies/examples for language/RNNs/LSTMs?
Comment: @DeltaIV the point is not that you can't notice the noise. Every jpeg that's been degraded too much has noise. The point is that the noise can be manipulated to make the DNN do crazy things, things which make no sense to a human observer even if the noise itself can be seen.
Comment: good answer, except for point 3 which is moot for two reasons. 1) It's very seldom the case that an adversarial image is ", to a human, indistinguishable from an unmodified image", unless that human has serious visual impairments. Nearly always you can notice that the image has some noise pattern added to it, especially in the background, aesthetically resembling so-called JPEG-noise (only visually: the actual statistical properties of the perturbation are different). What's surprising is not that the classifier is uncertain whether it's a _bona fide_ cat rather than, say, a corrupted 1/
Comment: 2/ image, but that's it's nearly certain that it's a bus. 2) How are adversarial examples related to interpretability? Linear models, generalized linear models and also decision trees are susceptible to adversarial examples. It's actually _easier_ to find an adversarial example which fools logistic regression, rather than one which fools ResNet. Notwithstanding this, we usually consider (G)LM to be interpretable model, so I wouldn't associate the existence of adversarial examples with the interpretability of a model.
Comment: There is a difference between interpretation and correctness.  Linear Regression models ARE easy to interpret, it is very clear how the model will change for different inputs.  It isn't important whether or not a human can tell apart the adversarial model inputs, what is important is that they are showing the chaotic properties of standard DL.  Fairly close inputs can give very different outputs and it isn't clear why.  With a linear model it is easy to pinpoint the variables that are shifting the output.  It isn't impossible in DL, but you sort of need to backtrace to input nodes at least.
Comment: @HongOoi of course: both I and Sycorax agreed on that. What I disagreed with was the specific sentence "an adversarial image is, to a human, indistinguishable from an unmodified image". Sycorax has since removed that sentence, so on this point I have no further objections. I'm still not convinced that adversarial examples are directly related to lack of interpretability, because, as shimao noticed, linear and even generalized linear models probably have adversarial examples. However, it's hard to prove that in general: for a proof in a specific case see https://arxiv.org/pdf/1801.02774.pdf
Comment: @DeltaIV There is one aspect of interpretability (or rather inscrutability, the opposite direction) that applies equally to all modeling methods. If you ask why a particular output corresponds to a particular input, even whiteboxing things, depends not only only the mechanics of the particular method, but also very significantly on the distribution of the training set. That is just as inscrutable for logistic regression as it is for SVM or random trees or any NN.
Comment: @mitch thanks, this is very interesting and in line with what I was trying to get at, i.e., that interpretability of GLMs for the classification task is overrated. I didn't know the term _inscrutability_.
Comment: @DeltaIV I've added an example of an adversarial image which has indistinguishable noise added, in the sense that there are no obvious artifacts visible to a human and re-instated my original claim that adversarial images can be indistinguishable to humans.
Comment: @Sycorax Re (1): oh, come on ;-) the classic "adversarial noise" is quite evident in nearly all of them. It's egregious in "Flagpole", "Baloon"  "Joystick",  "Tresher", "Whiptail lizard" - note how the only one where you really couldn't notice it ("Tibetan Mastiff"), is still correctly detected after adding noise. Anyway, +1 because of the nice image and cool reference. Overall the answer is very good, so I'll refrain from writing one myself :-)
Comment: interesting: you will note that the real difference between the first paper you posted, where the differences are obviously visible to humans, and the other two papers, is that the attack in the first paper doesn't have access to the loss function of the classifier, but only to the output labels. Instead, the only attacks in the other two papers  which can create indistinguishable (for humans) adversarial examples are gradient-based attacks, where the attack has access to the gradient of the model's loss function. This is particularly evident in Figure 3 of the last paper, where, even if 1/
Comment: 2/ the resolution is so absurdly low that it's nearly impossible to appreciate noise, still, it seems that the only attack which do not corrupt the input image significantly, are the two attacks based on the gradient of the loss function. Very interesting. PS just to be clear, I had already upvoted your answer.
Comment: Sure, but in light of your adamant insistence that all adversarial noise is obvious to humans, both of these images are striking counter-examples.  Your further argumentation of this point is an example of the "no true Scotsman" fallacy.
Comment: ? You're misunderstanding. I have already conceded (is this the right term in English?) that **not all adversarial examples are obvious to humans**. I was just noticing that it looks like this requires attacks which can access the model's "insights", not just the possibility of interrogating it with an input and recording the output, and I found this interesting. I think AE people call the first kind of attacks "white box attacks", and the second kind of attacks "black box attacks" (I'm not sure), but I've **never** said that "white box attacks are not valid" or whatever. 1/
Comment: 2/  So no "true" or "false" Scotman here. Take it easy :-) I already told you I like your answer...I changed my mind, you don't have to keep rubbing my nose in :-)
Answer: Layers don't map onto successively more abstract features as cleanly as we'd like. A good way to see this is to compare two very popular architectures.
VGG16 consists of many convolutional layers stacked on top of each other with the occasional pooling layer -- a very traditional architecture.
Since then, people have moved on to designing residual architectures, where each layer is connected to not only the previous layer, but also one (or possibly more) layers farther down in the model. ResNet was one of the first to do this, and has around 100 layers, depending on which variant you use.
While VGG16 and similar networks do have layers act in a more or less interpretable manner -- learning higher and higher level features, ResNets do not do this. Instead, people have proposed that they either keep refining features to make them more accurate or that they're just a bunch of shallow networks in disguise, neither of which matches the "traditional views" on what deep models learn.
While ResNet and similar architectures handily outperform VGG in image classification and object detection, there seem to be some applications for which the simple bottom-up feature hierarchy of VGG is very important. See here for a good discussion.
So given that more modern architectures don't seem to fit into the picture anymore, I would say that we can't quite say CNNs are interpretable yet.
Comment: @Mitch Some recent architectures such as Densenet seem to be slowly creeping towards the limit of having every layer connected to every other layer -- much like your "undesigned network". But surely, ResNet and Densenet have a more sophisticated design than VGG16, yet one could say they are less interpretable -- so no, I don't think more design means more interpretable. Possible, sparser connections means more interpretable.
Comment: Presumably the entirely unengineered/undesigned topology of a DL network would be a large random partial ordered set, input the sensors and output the desired function (that is, no attempt at layering at all, let the training figure it out). The nodes here would be _very_ inscrutable. But doesn't that sort of imply that the more designed a topology is, the more likely it does have some interpretability?
Answer: The subject of my Ph.D dissertation was to reveal the black-box properties of neural networks, specifically feed-forward neural networks, with one or two hidden layers.
I will take up the challenge to explain to everyone what the weights and bias terms mean, in a one-layer feed-forward neural network. Two different perspectives will be addressed: a parametric one and a probabilistic one.
In the following, I assume that the input values provided to each input neuron have all been normalized to the interval (0,1), by linear scaling ($x_{input}=\alpha \cdot x + \beta$), where the two coefficients $\alpha$ and $\beta$ are chosen per input variable, such that $x_{input} \in (0,1)$. I make a distinction between real-numbered variables, and enumerated variables (with a boolean variable as a special case enumerated variable):

A real-numbered variable is provided as a decimal number between $0$ and $1$, after linear scaling.
An enumerated variable, take the days of the week (monday, tuesday, etc.) are represented by $v$ input nodes, with $v$, being the number of enurable outcomes, i.e. $7$ for the number of days in a week.

Such a representation of your input data is required in order to be able to interpret the (absolute value) size of the weights in the input layer.
Parametric meaning:

the larger the absolute value of the weight is between an input
neuron and a hidden neuron, the more important that variable is, for
the 'fireing' of that particular hidden node. Weights close to $0$
indicate that an input value is as good as irelevant.
the weight from a hidden node to an output node indicates that the weighted
amplification of the input variables that are in absolute sense most
amplified by that hidden neuron, that they promote or dampen the
particular output node. The sign of the weight indicates promotion
(positive) or inhibition (negative).
the third part not explicitly represented in the parameters of the neural network is the multivariate distribution of the input variables. That is, how
often does it occur that the value $1$ is provided to input node $3$ - with the really large
weight to hidden node $2$ ?
a bias term is just a translation constant that shifts the average of a hidden (or output) neuron. It acts like the shift $\beta$, presented above.

Reasoning back from an output neuron: which hidden neurons have the highest absolute weight values, on their connections to the output neurons? How often does the activation of each hidden node become close to $1$ (assuming sigmoid activation functions). I'm talking about frequencies, measured over the training set. To be precise: what is the frequency with which the hidden nodes $i$ and $l$, with large weights to the input variables $t$ and $s$, that these hidden nodes $i$ and $l$ are close to $1$? Each hidden node propagates a weighted average of its input values, by definition. Which input variables does each hidden node primarily promote - or inhibit? Also the $\Delta_{j,k}=\mid w_{i,j} -  w_{i,k}\mid$ explains much, the absolute difference in weights between the weights that fan out from hidden node $i$ to the two output nodes $j$ and $k$.
The more important hidden nodes are for an output node (talking in frequencies, over the training set), which 'input weights times input frequencies' are most important? Then we close in on the significance of the parameters of feed-forward neural networks.
Probabilistic interpretation:
The probabilistic perspective means to regard a classification neural network as a Bayes classifier (the optimal classifier, with the theoretically defined lowest error-rate). Which input variables have influence on the outcome of the neural network - and how often? Regard this as a probabilistic sensitivithy analysis.
How often can varying one input variable lead to a different classification? How often does input neuron $x_{input}$ have potential influence on which classification outcome becomes the most likely, implying that the corresponding output neuron achieves the highest value?
Individual case - pattern
When varying a real-numbered input neuron $x_{input}$ can cause the most likely classification to change, we say that this variable has potential influence. When varying the outcome of an enumerated variable (changing weekday from monday $[1,0,0,0,0,0,0]$ to tuesday $[0,1,0,0,0,0,0]$, or any other weekday), and the most likely outcome changes, then that enumerated variable has potential influence on the outcome of the classification.
When we now take the likelihood of that change into account, then we talk out expected influence. What is the probability of observing a changing input variable $x_{input}$ such that a the input case changes outcome, given the values of all the other inputs? Expected influence refers to expected value, of $x_{input}$, namely $E(x_{input} \mid {\bf x}_{-input})$. Here ${\bf x}_{-input}$ is the vector of all input values, except from input $x_{input}$. Keep in mind that an enumerated variable is represented by a number of input neurons. These possible outcomes are here regarded as one variable.
Deep leaning - and the meaning of the NN parameters
When applied to computer vision, neural networks have shown remarkable progress in the last decade. The convolutional neural networks introduced by LeCunn in 1989 have turned out to eventually perform really well in terms of image recognition. It has been reported that they can outperform most other computer-based recognition approaches.
Interesting emergent properties appear when convolutional neural networks are being trained for object recognition. The first layer of hidden nodes represents low-level feature detectors, similar to the scale-space operators T. Lindeberg, Feature Detection with Automatic Scale Selection, 1998. These scale-space operators detect

lines,
corners,
T-junctions

and some other basic image features.
Even more interesting is the fact that perceptual neurons in mammal brains have been shown to resemble this way of working in the first steps of (biological) image processing. So with CNNs, the scientific community is closing in on what makes human perception so phenomenal. This makes it very worthwhile to pursue this line of research further.
Comment: This is interesting - doesn't sound like it would provide much interpretability in the case of correlated features?
Comment: The expected vallue E(.) is also known as the average of the conditional distribution, x_input given x_-input, all the other variables. Hence, correlations are fully incorporated into this expected influence concept. Note that probabilistic independence has a wider definition than 'correlation' - the latter primarily being defined for Gaussian distributed data.
Comment: Nice. Is this sort of a generalization of an interpretation of logistic regression to a set of stacked regression models, one feeding into the next?
Comment: A subset of hidden nodes can act as a logical 'OR' for an output neuron, or more like a logical 'AND'. OR occurs when one hidden node activation is enough to cause the output neuron to become close to 1. AND occurs when only a sum of hidden node activations can cause the output node activation to become close to 1. Whether more 'OR' or more 'AND', that depends on the trained weight vector of the 'fan in', into the output node.


QDA vs EM with Gaussian likelihoods

Question: QDA (quadratic discriminant analysis) assumes that the K different classes are generated by K different multivariate Gaussians, each with potentially different mean vector and covariance matrix.
If these assumptions hold, will QDA give the same classification results as using EM (expectation maximization) clustering with Gaussian likelihoods?
Answer: See my answer to a related question https://stats.stackexchange.com/a/253235/71228
Yes, both methods use essentially the same assumptions and are capable of learning the exact same kind of underlying data distribution (mixture of Gaussians); the difference is the use case.
Your question is somewhat ill-posed, since you want to compare a supervised learning method (classification given class labels) to an unsupervised learning method (clustering with no class labels).
If you're talking about classification performance given class labels, and assuming somehow you could identify the class labels with clusters learned from EM (and more importantly you somehow know how many true clusters there are!), I would guess QDA would give slightly better results, since no label information is used in clustering the data with GMM+EM.

QDA vs EM with Gaussian likelihoods


Find out the DNS server for a DHCP connection in Linux

Question: I know they're defined in /etc/resolv.conf, but what if it's not there? And more specifically, how do you find the DNS server returned by DHCP?
In GNOME you can use the NetworkManager applet to see the primary DNS for any connection, so how would you do the same from the command line?
Answer: Usually dhclient.leases file is located at /var/lib/dhcp3/dhclient.leases, type the following command:
less /var/lib/dhcp3/dhclient.leases
OR
cat /var/lib/dhcp3/dhclient.leases
OR
You can just use grep command to get DHCP server address, enter:
grep dhcp-server-identifier /var/lib/dhcp3/dhclient.leases
OR
dhclient eth0
Comment: I ran `ps aux | grep dhclient` and found that my leases file was set to `/var/run/dhclient.eth0.leases` with the `-lf` option.
Comment: Great, thanks! In case anybody looks for this, the IPCop file is found on /var/ipcop/dhcpc/dhcpcd-*.info
Comment: This is specific of one DHCP client but there are others (such as pump).
Comment: also, `less /var/lib/dhcp/dhclient.leases` works for **Ubuntu 22**, while the mentioned in the answer—do not
Answer: I recently had this problem where my dhcpcd was misconfigured as was not setting DNS servers.  I found out that I can query which nameservers are available by DHCP with the following command:
sudo dhcpcd -o domain_name_servers -T

The command will output a bunch of network connection information.  Look for the line beginning with new_domain_name_servers.
From here I was able to manually set the nameservers.
Comment: Perfect. This should be the accepted answer, hehe.
Comment: This was also the solution that worked for me.
Comment: why `-o` switch is necessary if it dumps all the same options with `-T` alone?
Answer: One more solution to query the DHCP server:
sudo nmap --script broadcast-dhcp-discover

The output tells you the name server offered by the DHCP server:
Starting Nmap 7.60 ( https://nmap.org ) at 2019-12-12 23:50 UTC
Pre-scan script results:
| broadcast-dhcp-discover:
|   Response 1 of 1:
|     IP Offered: 18.104.22.168
|     DHCP Message Type: DHCPOFFER
|     Server Identifier: 220.127.116.11
|     IP Address Lease Time: 5m00s
|     Subnet Mask: 255.255.0.0
|     Router: 220.127.116.11
|     Domain Name Server: 220.127.116.11
|_    Domain Name: localdomain

Answer: I found my DHCP lease info at /var/lib/dhclient/dhclient-eth0.leases in case anyone can't find it at /var/lib/dhcp3/dhclient.leases
Answer: well, why not simply dig that?
marcus@marcus ~ $ dig google.de

; <<>> DiG 9.4.3-P3 <<>> google.de
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27579
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 6, ADDITIONAL: 0

;; QUESTION SECTION:
;google.de.         IN  A

;; ANSWER SECTION:
google.de.      1785    IN  A   22.214.171.124
google.de.      1785    IN  A   184.108.40.206
google.de.      1785    IN  A   188.8.131.52

;; AUTHORITY SECTION:
de.         23431   IN  NS  L.DE.NET.
de.         23431   IN  NS  Z.NIC.de.
de.         23431   IN  NS  F.NIC.de.
de.         23431   IN  NS  C.DE.NET.
de.         23431   IN  NS  S.DE.NET.
de.         23431   IN  NS  A.NIC.de.

;; Query time: 1 msec
;; SERVER: 192.168.1.15#53(192.168.1.15)
;; WHEN: Thu Sep 17 19:33:39 2009
;; MSG SIZE  rcvd: 181

look at the last lines:
;; SERVER: 192.168.1.15#53(192.168.1.15)
Comment: When no DNS server responds, dig doesn't show any useful info, is it possible to make it spit out the DNS servers it tried to ask?
Comment: I was thinking the same thing originally, but I don't that that would show ALL of the assigned DNS servers, would it?  I think dig only returns the one it used for that particular lookup.  On second look, the poster didn't specify so if he's happy with it then your answer is definitely the easiest. :-)
Comment: yes you are right output only shows up the dns used by that query... so you can see what __is__ really happening. I found your solution fine to figure out what __should__ happen, thy for that :-)
Comment: That wasn't exactly what I was looking for, but thanks anyway.
Comment: So after RingTFM, I see it just uses whatever is on /etc/resolv.conf
Answer: If there is nothing in /etc/resolv.conf, DNS resolution (that is, resolution using dns as defined in /etc/nsswitch.conf) will not work. This is what dig, host and the libc routines use to do DNS resolution. If there is nothing specified in this file, DNS resolution will not work (though host resolution may work via other means, if another host resolution method is configured, such as NIS (this is unlikely)).
Answer: this command show the DNS server on your net
dig | grep SERVER: | awk -F# '{ print $1 }' | awk -F: '{ print $2 }'
172.17.0.1
Comment: This does answer the question. In fact it is less reliable than the approach the OP mentions in the question, which is simply to read it from `/etc/resolv.conf`. After all dig is going to read from `/etc/resolv.conf` as well and then (try to) communicate with the server before producing any output.
Answer: If you are using NetworkManager, to get ipv4 DNS servers for YourConnectionName connection, you can use something like
nmcli -g IP4.DNS connection show YourConnectionName

Use nmcli connection show YourConnectionName to get all possible options and values.
Answer: Read /etc/resolv.conf. That's all.
man resolv.conf is useful, too.
Comment: Yes, but I ignored this point, which makes no sense. Since the libc resolver uses resolv.conf, it has to be there.
Comment: except he said what to do if it's *not* there..
Comment: The question said specifically not to use resolve.conf and there is plenty of reasons where resolve.conf is not holding everything (e.g. in my case when a DHCP Server is used)
Answer: In modern distributions it is looks like
ps aux | grep dhcli | grep -v grep
root       367  0.0  0.0  16128  3076 ?        S    Apr30   0:00 /sbin/dhclient -d -q -sf /usr/lib/NetworkManager/nm-dhcp-helper -pf /var/run/dhclient-enp3s0.pid -lf /var/lib/NetworkManager/dhclient-324caec3-58f9-3f00-bff4-2f8bd7d2aafa-enp3s0.lease -cf /var/lib/NetworkManager/dhclient-enp3s0.conf enp3s0

tail -16 /var/lib/NetworkManager/dhclient-324caec3-58f9-3f00-bff4-2f8bd7d2aafa-enp3s0.lease
lease {
  interface "enp3s0";
  fixed-address 10.191.104.3;
  option subnet-mask 255.255.255.0;
  option routers 10.191.104.254;
  option dhcp-lease-time 86400;
  option dhcp-message-type 5;
  option domain-name-servers 220.127.116.11,18.104.22.168;
  option dhcp-server-identifier 22.214.171.124;
  option ntp-servers 220.127.116.11;
  option broadcast-address 10.191.104.255;
  option domain-name "myISP.ru";
  renew 1 2017/05/01 12:56:02;
  rebind 2 2017/05/02 00:27:11;
  expire 2 2017/05/02 03:27:11;
}

where 126.96.36.199 is my ISP's DNS server
nslookup serverfault.com 184.108.40.206
Server:     22.214.171.124
Address:    22.214.171.124#53

Non-authoritative answer:
Name:   serverfault.com
Address: 22.214.171.124
Name:   serverfault.com
Address: 126.96.36.199
Name:   serverfault.com
Address: 22.214.171.124
Name:   serverfault.com
Address: 22.214.171.124

Answer: if use systremd-resolve:
$ resolvectl status wlp0s20f3
//OR
$ systemd-resolve --status wlp0s20f3

outputs:
Link 3 (wlp0s20f3)
      Current Scopes: DNS
DefaultRoute setting: yes
       LLMNR setting: yes
MulticastDNS setting: no
  DNSOverTLS setting: opportunistic
      DNSSEC setting: no
    DNSSEC supported: no
  Current DNS Server: 192.168.27.1
         DNS Servers: 192.168.27.1
          DNS Domain: ~.
                      lan

//OR
$ resolvectl dns
Global:
Link 5 (tun0): 10.1.2.1
Link 3 (wlp0s20f3): 192.168.1.1


Function Convergence

Question: Suppose that $f,g : [a, b] \to \mathbb{R}$ are continuous functions with $g$ strictly increasing. Show that for every $\varepsilon > 0$ we can find some constants $a_1,\ldots, a_n \in \mathbb{R}$ such that
$$\left| f(x) - \sum_{k = 0}^n a_k (g(x))^k \right| < \varepsilon$$
for all $x \in [a, b]$.
I haven't really made any progress besides fooling around with the fact that $g$ is bounded.
I was wondering if I could get a hint.
Thanks!
Comment: I kind of like this wording of the problem:

Suppose $f,g : [a,b] \rightarrow \mathbb{R}$.  Then there is a polynomial p such that $|f(x) - p(g(x))| < \epsilon$ for all $x \in [a,b]$
Answer: Suppose $g(a)=c$ and $g(b)=d$.  Let $g^{-1}:[c,d]\to [a,b]$ be the inverse function of $g$.  You can approximate $f\circ g^{-1}:[c,d]\to\mathbb R$ with polynomial functions using the Weierstrass approximation theorem, then compose with $g$ to get the result stated above.  (Note that you will need to give a reason why $g^{-1}$ is continuous.)

Function Convergence


Bounds for the expectation of the max

Question: Consider the following expected value
$$
E(\max\{a_1+\epsilon_1, a_2+\epsilon_2,...,a_n+\epsilon_n\})
$$
where the expectation is taken with respect to the random variables $(\epsilon_1,...,\epsilon_n)$ and $a_1,...,a_n$ are real numbers.
Can we say whether $E(\max\{a_1+\epsilon_1, a_2+\epsilon_2,...,a_n+\epsilon_n\})$ is greater or smaller than $\max\{a_1,...,a_n\}$?
Can we characterise some upper or lower bound of  $E(\max\{a_1+\epsilon_1, a_2+\epsilon_2,...,a_n+\epsilon_n\})$ that does not depend on the distribution of $(\epsilon_1,...,\epsilon_n)$?
Answer: No, consider the easiest case $n=1$. $E[a+\epsilon]=a+E[\epsilon]$ and it can be either greater or smaller than $a$ based on the distribution of $\epsilon$.
Comment: @user3285148 for $a_i=a$, the answer would be YES. But, the other case didn't seem (to me) trivial to prove or give counter example. Would be a good question though.
Comment: Thanks. Suppose $E(\epsilon)=0$. The answer to my question is still NO?

Bounds for the expectation of the max


Transfer Aegir site to LAMP

Question: Can anyone please advice me on how to move a Drupal 7 site from Aegir to a LAMP stack?
I've been handed the emergency project of moving a site to a new hosting. Ordinarily, that would be easy enough until I realise the site is loosing it's hosting in a matter of days, settings.php is not in the sites/default directory nor is the files directory. The structure is not standard D7 at all.
I later realised the site is currently hosted on Aegir which admittedly I know nothing about but I wonder if Aegir created the non standard structure. I used the tree command to show the structure below:
tree project/sites/ -L 2
core-1.23/sites/
|-- all
|   |-- drush
|   |-- libraries
|   |-- modules
|   |-- README.txt
|   `-- themes
|-- default
|   `-- default.settings.php
|-- example.sites.php
|-- sites.php
`-- www.clientsite.com
    |-- drushrc.php
    |-- email
    |-- files
    |-- googlef1245ba6b3c38a3c.hmtl
    |-- libraries
    |-- local.settings.php
    |-- modules
    |-- pdf
    |-- private
    |-- README.md
    |-- settings_.php
    |-- settings.php
    |-- themes
    `-- vendor

As you can see, the settings.php is in a sub-directory named after the actual live site, very unlike a vanilla Drupal 7 code structure.
I admit I am stymied by this.
Can anyone tell me if this is how Aegir layout the Drupal structure and how I can move this to a more traditional D7 structure ???
Comment: Unless it changed since I used it a few years back, Aegir uses a standard multisite approach, which is the structure you're seeing. You don't really need to move it to a "traditional" structure, it'll work just fine as-is
Answer: You wrote:

settings.php is not in the sites/default directory nor is the files directory. The structure is not standard D7 at all.

That's incorrect.  From the Aegir documentation:

Aegir installs sites on a platform, which means it places the site directory in /var/aegir/platforms/<your_platform>/sites/<yoursite.com>. Many sites can all be installed alongside each other inside a platform's 'sites' directory. This is a standard, built-in Drupal feature known as 'multisite', and it is not unique to Aegir.

...and the official Drupal documentation on multisite:

It should be noted, however, that using the Aegir hosting system is preferred.  It does all of the heavy lifting for you using secure best practices including automatically configuring virtual hosts for both Apache and Nginx, adding HTTPS support, running Composer commands, etc.  See the documentation for setting up a "platform", Aegir-speak for a multisite codebase.

So when you ask, how I can move this to a more traditional D7 structure?, the correct answer is that you can't.  Besides following Drupal standards, Aegir additionally follows best practices for security (e.g. setting secure file permissions) and sustainability (point-and-click upgrades of multiple sites at once).  So you're far better off hosting Drupal sites with Aegir than the "traditional" way.
My recommendation would therefore be to set up a new Aegir instance elsewhere, and then remotely import the sites before the older instance is decommissioned.  If you're running a LAMP stack yourself, there are very few reasons why you shouldn't be running Aegir for multi-site Drupal.

Transfer Aegir site to LAMP


What is the definition of "center" or "kernel" of a set in a Banach space

Question: In the text I'm reading the author says:

We also define the kernel of $K\subseteq X$ ($X$ a Banach space) to be the set
$$
\overset{c}{K}:=\{x\in K:\forall y\in X\exists a>0\text{ with }\lambda y+(1-\lambda)x\in K,\text{ for all }0\leq\lambda\leq a\}.
$$
Sometimes it is called the "center" of $K$.

However, in numerous proofs in the text (chapter really), the author shows some point is in the center of a set and I don't follow why; for instance, the following part of a proof:
Consider the function $\mu\colon X\to[0,\infty)$ with the property $\mu(\lambda x)=\lambda\mu(x)$ for any $\lambda\geq0$. Consider the set $M:=\{x:\mu(x)\leq1\}$. The author claims that $0\in\overset{c}{M}$ since $\mu(\lambda x)=\lambda\mu(x)$. I don't see why. I mean, because of $\mu(\lambda x)=\lambda\mu(x)$ it follows that $\mu(0)=0\leq1$, so $0\in M$. Therefore take $x:=0\in M$ and any $y\in X$. By definition of the set $\overset{c}{M}$, our goal is to show there exists $a>0$ with $\lambda y+(1-\lambda)x\in M=\{x:\mu(x)\leq1\}$ for all $0\leq\lambda\leq a$.
Since $x=0$, this means we want to show there exists $a>0$ with $\lambda y\in\{x:\mu(x)\leq1\}$ for all $0\leq\lambda\leq a$. Or rather there exists $a>0$ with $\mu(\lambda y)=\lambda\mu(y)\leq 1$ for all $0\leq\lambda\leq a$. This doesn't need to hold for every $y\in X$, though right? I believe we need the restriction that $y\in M$, thus the definition for the kernel would change to
$$
\overset{c}{K}:=\{x\in K:\forall y\in K\exists a>0\text{ with }\lambda y+(1-\lambda)x\in K,\text{ for all }0\leq\lambda\leq a\}.
$$
Therefore if we knew that $y\in M$, then $\mu(y)\leq1$, and we could take $a=1$.
Answer: When you say $\forall \, y \, \exists \, a$ you are allowing $a$ to depend on $y$.  $\lambda \mu (y) \leq 1$ for $0\leq \lambda \leq a$ certainly holds for some $a$ depending on $y$.
Comment: I see. I forgot that by the order of the quantifiers the $a$ value may depend on the $y$ in $X$. Thus taking $a:=1/\mu(y)$ the result holds.


javascript for loop won't iterate

Question: novice programmer here, I'm trying an exercise where the objective is to print a string backwards. Here is my code:
function firstReverse(str) {
    for(var i=str.length - 1; i >= 0;i --) {
        return(str[i]);
    }
};
firstReverse("riyad");

I keep getting the return of letter "d", but not the entire string. Not sure what is wrong here? Please help!
Comment: After the first return, you left the function. When you execute `return` the method is abandoned, nothing else is executed after that.
Comment: As you're learning, this how you'd generally do it -> `"riyad".split('').reverse().join('')`
Comment: You need to look at what the `return` function does. That is kind of key to your issue.
Comment: Yeah your function returns in the first iteration so you would only get the last letter. You cannot have a return statement since it immediately exits the function. Save the answers in a variable and then after the for loop return the variable.
Answer: You could try this:
function firstReverse(str) {
    for(var i=str.length - 1; i >= 0;i --) {
        console.log(str[i]);
    }
};

I keep getting the return of letter "d", but not the entire string

It is reasonable that you take this output, because the first time that your code steps into the for statement it will exit from it due to the return statement. In other words, you for statement will be executed only for the first step.

function firstReverse(str) {
    for(var i=str.length - 1; i >= 0;i --) {
        document.write(str[i]);
    }
};

firstReverse("riyad");
Comment: Please don't encourage a beginner to use `document.write()`.  While it might be useful for a test program, it is rarely what you want to use in a real web page for a variety of reasons.
Comment: @jfriend00 of course. Just wanted to so it in action. Thanks
Answer: The problem is, that return statement breaks your loop and returns the first letter it gets.
If you want to reverse a string - you may try to convert it to the Array, reverse it and concat elements of the new array back to string. I think its a simple solution. Here is a piece of code:

function reverse(str) {
  return str.split('').reverse().join('');
}

console.log(reverse('abc'));

javascript for loop won't iterate


Como obter aws service com aws-sfk-php-zf2 v 2.0.*?

Question: Eu estava usando aws/aws-sdk-php-zf2 1.2.* e tive de dar update para 2.0.* e agora o AWS SDK é v3.
Antes, eu chamava o serviço com o código:
<code>$this->s3 = $serviceLocator->get('aws')->get('s3');
</code>
Mas agora retorna esse erro:

Zend\ServiceManager\ServiceManager::get was unable to fetch or create an instance for aws

Não consegui identificar nem encontrar a diferença na documentação de migração.
Alguém tem alguma ideia?
Answer: Achei uma forma que funcionou pra mim.
Agora o código está assim:
<code>use Aws\S3\S3Client;

$this->s3 = S3Client::factory(array(
            'credentials' => array(
               'key' => $this->config['aws']['key'],
               'secret' => $this->config['aws']['secret'],
            ),
            'region' => $this->config['aws']['region'],
            'version' => '2006-03-01'
        ));
</code>


Problem while solving question related to quadratic equations by some another method

Question: Consider following quadratic equation whose roots are $a$ and $b$.
$$x^2 +ax +b = 0 $$
We need to find $a$ and $b$ provided that $a,b\neq0$
So it seems a pretty easy question and it is,
From product of roots we get $a=1$
From sum of roots we get $b=-2$
Now I decided solve with an other way as follows-
Since $a$ and $b$ are roots of equation, therefore they will satisfy it as follows-
$ a^2 + a^2 + b = 0$ and hence $2a^2=-b$
$b^2 + ab +b=0$
Substituting value of $b$ we get
$4a^4-2a^3-2a^2=0$
Since $a\neq0$ solving for a gives two values $a=1$ and $a=-1/2$
and therefore corresponding values of $b$ as $-2,-1/2$ and hence we get following two quadratic equations
$x^2+x-2=0$ and $2x^2-x-1=0$
The first one has roots $(1,-2)=(a,b)$ , therefore we got one answer
but second one has roots $(1,-1/2)\neq(a,b)$
Why does this happens with second equations that it's roots are not equal to $a,b$
Comment: @lalittolani It's a one way implication, not an equivalence, and so can introduce extraneous solutions. In this case, you are losing the fact that $a,b$ must not be the *same* root of the original equation.
Comment: Is it $ab\neq0$ or $a,b\neq0$?
Comment: $a,b\neq0$ is correct condition I corrected it
Comment: "*therefore they will satisfy...*" This step is not reversible, so the solutions must be verified each.
Comment: what do you mean by reversible
Comment: Information seems to be missing: "From product of roots we get $a = 1$" : **How**?  All I see, prior to that statement is $x^2 + ax + b = 0$.  How do you conclude from *that equation* that $a=1$?
Comment: @user2661923 I guess OP used $ab=b$
Comment: @DatBoi then I should have got b=1 but I got a=1
Comment: @DatBoi Perhaps, but combining *off-beat* analysis with missing info is too much.  OP needs to respond directly, especially by editing the question.
Comment: @lalittolani that was a typo. fixed it now
Comment: @user2661923 agreed
Comment: @user2661923 what's the problem in question
Comment: Information seems to be missing: "From product of roots we get $a=1$" : **How**?  All I see, prior to that statement is $x^2 + ax + b = 0$.  How do you conclude from *that equation* that $a=1$?
Comment: everyone knows product of roots = constant term / highest coefficient so if I  have mentioned roots and equation clearly and used the term " product of roots" there must be no problems
Answer:
Since $a$ and $b$ are roots of equation, therefore they will satisfy it as follows-
$a^2 + a^2 + b = 0$
$b^2 + ab +b=0$

This system of equations does indeed follow from the problem statement, but it is not equivalent to it. Reason is that the problem assumes $a,b$ to be the roots of the quadratic, with emphasis on the plural of "roots".

The first solution based on Vieta's formulas preserves this assumption, and provides the correct result.

But the substitution that leads to the system of equations loses the distinction that $a,b$ must be the two roots of the quadratic, and allows for $a,b$ to be one and the same root.

In other words, extraneous solutions may be introduced among the solutions of the system, where $a$ is a root and $b=a$. Which is precisely what happened in this case with $a = b = -1/2$.
Therefore, the solutions to the system are not guaranteed to satisfy the original problem, and must be verified each. (Note: it is not safe to automatically reject any solution with $a=b$, because they could be equal in case a real solution existed which turned out to be a quadratic with a double root.)
Comment: thankyou for solving my problem fabulously in both your answers
Comment: @lalittolani Yours was a well asked question. It's good that you tried it, and even better that you noticed the inconsistency. The fallacy here was of a more subtle variety than the usual algebraic pitfalls.
Comment: Elaborated some more in my other answer [here](https://math.stackexchange.com/a/4150157/291201).


Determining independent linear systems of equations

Question: I am not if I am using the right terms. But I am looking for a method to determine if equations of a linear system are independent (decoupled) or not. For example consider the following system:
$
\begin{bmatrix}
a_{11} & a_{12} & 0 & 0 \\
a_{21} & a_{22} & 0 & 0 \\
0 & 0 & a_{33} & a_{34} \\
0 & 0 & a_{43} & a_{44}
\end{bmatrix}
\begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix}
=
\begin{bmatrix} b_1 \\ b_2 \\ b_3 \\ b_4 \end{bmatrix}
$.
The variables $x_1$ and $x_2$ in this system are completely independent of $x_3$ and $x_4$; i.e., if for example the value of $b_1$ changes, it only changes the values of $x_1$ and $x_2$ and has not any impact on the values of  $x_3$ and $x_4$.
The problem I am looking for, is not only about coefficient matrix itself, but it is about the whole system. For example consider this system:
$
\begin{bmatrix}
1 & 0 & 0 \\
1 & 1 & 1 \\
0 & 0 & 1
\end{bmatrix}
\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}
=
\begin{bmatrix} +1 \\ \alpha \\ -1 \end{bmatrix}
$.
Looking at the coefficient matrix, one may say it is a coupled system. But the RHS says that the value of $x_2$ depends only on $\alpha$.
Is there any method to determine if equations of a linear system are independent (decoupled)?
Answer: Yes: we use Gaussian elimination to place the matrix of coefficients in row echelon form. See this: http://en.m.wikipedia.org/wiki/Gaussian_elimination Now this is not an efficient way because we have to do it for all the $n!$ possible orderings of the variables. For the right ordering, when the system admits decoupling: you will see the blocks of zeros as in your first example.
Comment: What is the point of placing the matrix of coefficients in row echelon form? Please note that I don't mean linear dependency of equations.
Comment: You are right. My answer is wrong or incomplete at the best. The brute force way I see is to do Guaussian elimination for every possible ordering of the variables, $n!$ and then for the **right** ordering you would see the blocks/decoupled components as in your example.


Alpha correction in a family of tests

Question: Suppose that one runs 8 correlation analyses but in two separate batches: Would the alpha-correction apply to all 8 tests as a family or each of the 4 tests as a family?
For example, if there are two research questions and both require correlation analysis, the researcher conducts the first 4 tests (for research question 1) and then conducts the second 4 tests (for research question 2).
In this case, should the alpha be corrected for a family of 4 tests or for a family of 8 tests?
(Note that the same statistical test--Pearson's $r$--is used in both cases.)
I understand there isn't a consensus on this matter (as with many other matters in statistics as I am learning fast!) but some broad guidelines would be most helpful.
Comment: There was [similar question](http://stats.stackexchange.com/questions/19572/what-might-be-a-clear-practical-definition-for-a-family-of-hypotheses-with-r) on what might be considered a family of tests.
Comment: Thanks! The context above is slightly different, I believe, though.  As always, many thanks for the responses.
Comment: @AdheshJosh: I think you have to click some symbol (I don't remember what) to award the bounty to MånsT.
Comment: I did click "+50" but nothing happened!
Answer: I'd say that it depends on your research questions. Are the two research questions related to the same data set? Are they completely independent from eachother? Would a false negative be worse than a false positive for either of these questions? If you have jellybeans type questions, correct for all of them. If your first question is about the link between smoking and cancer and your second question is about the link between a father's length and his son's length, correct for 4 in each batch. If you're somewhere in between, I don't think that there is a "right answer" to this.
You could correct for all 8 tests. But then again, you could correct for all 400 tests that you run this year, to get at most a 5 % (say) risk of making a type I error in 2012. You have to draw the line somewhere...
Also, as you run multiple correlation tests, it sounds like you might have multivariate data. Have you considered using some multivariate generalization of Pearson's correlation coefficient, such as canonical correlations?
Comment: There can't be a better explanation than this! Many thanks. I am giving you the bounty points!
Comment: A very well-argued response.  For a similar but more detailed one, I recommend Geoffrey Keppel's Design and Analysis:  A Researcher's Handbook.  I'll find references to specific page no.s (1991 edition) if anyone would like that.

Alpha correction in a family of tests


Cant open a https website with php and fopen

Question: I am using fopen to retreive the contents of a URL.
It works on http URLs but not https URLs
Can anyone see why??
<?php

//this works fine
echo ("

url 1 is ".OutputURL("http://nuenergy.acornsoftware.com.au/staff/interface/index.php"));

//returns nothing
echo ("

url 2 is ".OutputURL("https://nuenergy.acornsoftware.com.au/staff/interface/index.php"));

function OutputURL($url)
{
  $handle = fopen($url, "r");
  $contents = stream_get_contents($handle);
  fclose($handle);
  return $contents;
}
//
?>

Comment: Is your php_openssl.dll enabled in apache?
Comment: I deleted my answer as I'm not to look further into an answer as of this moment. Hope you find some help!
Comment: See related answers: http://stackoverflow.com/questions/1975461/filegetcontents-with-https
Answer: You need to have OpenSSL installed and configured with PHP, I recommend you ask this on ServerFault.
Also, you can just use file_get_contents() instead of OutputURL().
Answer: The problem isn't with PHP not returning anything for HTTPS, it's your file not outputting anything for the HTTPS version of the file. When I view the HTTP version, it outputs a small XML diagram representing some sort of 'File Not Found' error. When I view the HTTPS form of the file, it returns a blank page, absolutely no content in the source code. You need to look at your Apache configurations and make sure it's pointing to all the files properly.
Answer: It'll either be that your apache/php setup does not support https via fopen or your server's firewall has blocked this type of connection.


Difference in "polar" command in Matlab from 2013 version to 2015

Question: I wanted to remove the numbers from the polar plot so I used the following:
myline = polar(txxx, pxxx); get handle of the data line
set(0,'Showhiddenhandles','on') % to show hidden handles
extrastuff = setdiff(get(gca,'children'),myline);
delete(extrastuff)

and it works in 2013b version but does not work in 2015 version. Can anyone help me to do the same in 2015 version.
Comment: This may be helpful: http://stackoverflow.com/questions/28853280/change-axis-in-polar-plots-in-matlab-to-radians. You can do something similar to avoid plotting the numbers
Comment: Thank you. It might work, let me try. Thanks.
Answer: The polar function creates those text objects with their 'HandleVisibility' set to 'off', so you can't see them from outside the function.
You can use a modified version of polar as follows:

Open file polar.m.
Find all lines that include text(. Those are the lines that create the text objects.
Comment out those lines, as well as their continuations.
Save the file with a different name, for example polar2.m.


markov chain with probability trends

Question: I have clients with debts that can pass from states own 1 bill, own 2 bills, own 3 bills, leave the service, new debtors and owe nothing.
So I could calculate the probabilities of being in state 1bill and going back to own nothing, and all the rest transition probabilities.
Until here I though I could you markov chains.
But then, I see that for example those "transition" probabilities have trend, because the company buys new technology for the call center, or invest in training. Also there are seasonal effects (like Christmas or holidays that make those rates change).
So my question is how can I represent the trend and seasonality in this case using a markov chain. If I fit the trend with a linear regression (just to be simple, with dummies for seasonality), how can I get the parameters considering the result of each trend must match the original transition probabilities?
At this point I've read about hidden markov models, but I'm not sure if this would be the proper model.
So, my question is what model would you use to solve this and how would you fit it?
Answer: Christmas and other holidays will probably affect all variables strongly, so you could just fit a model for these type of day or come up with ad-hoc rules. These are more anomalies than seasonal factors. It's hard to model these dates and probably pointless.
An idea to deal with this could be to compute the L1 trend filter of the yearly probabilities of each state and use it as a measure of local tendency. However, if your models are fit monthly you wouldn't have much data for this, it's just an idea of an "ad-hoc" limit/rule.
You mention having seasonal factors affecting your probabilities. I guess something reasonable to asume is there is less involuntary churn in january & february (summer holidays where I live) and things normalize in march.
Let's imagine thats the case. In that case, I would fit a model for these months (for each transition), that are somehow special using similar months to train the model.
There you would have a similar framework to the one you currently have, having a model per transition per season. I like this set up, there are some trivial non-linearities that a single model would find hard to find.
Regarding your questions:
I don't think the markov chain helps here, the problem is having good estimates of the probabilities (I believe).
I wouldn't go for the time series models. You have panel data, therefore you should be able to design features to capture seasonal effects o just fit models for different seasons in case this works with your data.
It's hard to discuss it withouth knowing the data or business anyway, hope this helps!
Answer: I think you need to just use multinomial regression to fit the transition probabilities based on your variables. see eg Statistical Inference for Discrete Time Stochastic Processes - 2.3 Extensions of Markov Chain Models
Models based on the Logistic Regression. ie for each starting state you predict the probability of going into the different next states.

markov chain with probability trends


How to detect non-intrusive phone call

Question: In iOS 14 you can now accept phone calls that do not take up the entire screen (the new non-intrusive phone call UI looks like a push notification). If you accept the call, but do not open up the full screen call UI, the app remains open. This is true even if you pick up the phone and put it to your ear the app remains active.
What would the best way be to detect an incoming call, or that the user began the call?
Answer: You can't detect it. If the user just takes the call direct from the banner, your app remains active and there is nothing to detect. That is the whole point of this feature: a call need not interrupt your app in any way.
Comment: Thanks for the quick response. Shouldn't there be a notification fired when the user brings the phone up to their ear to answer the call? Why does the app remain active still at that point?

How to detect non-intrusive phone call