how to remove module-created links from menus?

Question: I've ended up with two basic pages in the navigation menu and there seems no way to get them off.
The pages are just plain ol' pages; the pages are not created by a module. I am using the Drupal 7 core Book Module. One of the pages in question is a book; the other is a child page of the book. The book has other child pages which are not listed in the navigation menu. The Book navigation menu works fine.

admin/structure/menu/manage/navigation has no "Delete" Links by the pages.
editing the page, there is no "add page to menu" specified. I've even tried turning that on, creating a menu item, then turning it off again. The pages still list in Navigation.
I've tried drush cc all

I have the option to disable these pages, but clearly that's just wrong. Can anyone help?
Answer: Ok, just in case anyone else finds this useful: I got in this pickle because I installed a module (nodehierarchy) which generated menu items. Menu items created by modules are not deletable in the normal admin UI. When I removed the module it did not clean up after itself, leaving orphaned entries that drupal thought belong to a module that doesn't exist.
I don't know why drush cc menu didn't clean them out, perhaps a policy of not deleting data just in case.
To fix: start a mysql client on your drupal db and inspect the contents of menu_links, find the rows and delete. e.g.
mysql> delete from menu_links where module='nodehierarchy';

Then clear caches and fixed.
Answer: In my case I have two modules which have the ability to add links (that I know of...):

Views (/admin/structure/views -> Edit a View -> Page Settings ->
Menu)
Pages (/admin/structure/pages -> Edit a Custom Page -> Click on Menu)

In order to remove the menu links I had to do the following:

In the module that creates the menu item (Views, Pages, etc.) set the Menu option to "No Menu".
In the menu config (/admin/structure/menu/manage/), go to the menu item and click on "reset" to remove it from the menu.
Disable the Custom Page or View (the individual view or page, not the entire module)
Re-enable the Custom Page or View
The menu item should no longer be present in the menu_links table.

My issue was related to what seems to be a core problem (see:  https://www.drupal.org/node/1058672 OR https://www.drupal.org/node/1029022) which is that if a menu item is created and then removed, it can leave behind a stale record in the menu_links DB table.  Disabling the affected Custom Page or View is what actually removed it from the DB for me.
Answer: my problem was very close to the previous one, with some differences;
after an upgrade from D6 to D7, i suppressed some type of contents (better to do it before !!!);
but navigation links stayed under "create content";
i reinitialized every suppressed type of content link (which is not the right way), and it created each time a new "add content" line in the menu;
i went in menu_links, and suppressed each line with add/content (found in link_path and router_path) one (except one)


TCL array index - what is the spec

Question: I happen to run into this:
set a(()) 100

It is taken by TCL interpreter and added to array a an element whose index is () and value is 100.
I am perplexed. The official TCL spec does not say much about array index, it can take any character. That is fine. The problem is, how to retrieve it?
puts $a(()) does not work:
can't read "a(()": no such element in array

The workaround are:
set i ()
puts $a($i)

or
puts [set a(())]

I am beating myself to make sense out of why $a(()) does not work when "a(())" is happily taken, but cannot find any.
Can you help out? Or is this something we just recognize and live with it?
Comment: @MoDJ: nothing wrong with trying to figure out how a language works.
Comment: Why would you even do that? How is this even a question?
Comment: To my shame, I by mistake approved an edit to add the [array] tag to the question (I thought I was approving the deletion of the tag). Tcl arrays aren't in the same category as the kind of arrays the tag refers to. Please undo my mistake by rejecting the edit.
Answer: a(()) is a valid name for an array member, and as you say we can assign to it and retrieve its value using set:
set a(()) 100
set a(())

When invoking the set command, just like with all commands, the interpreter first splits the command line into words, performing substitutions as required, and then passes those words as arguments to the command. The first argument to set is the variable name: the set command uses the variable associated with the name or creates a new variable when assigning, alternatively raises an error if it can't find any such variable while retrieving.
$ syntax is mostly syntactic sugar for the single-argument set, but does not necessarily use exactly one command line word for variable name. When the interpreter finds an unescaped $ it attempts to find a sequence of characters that make up a variable name, and then substitutes the variable's value for the $ and the name. The word can contain more text after this, and the word does not have to start with a $.
The interpreter is careful not to overmatch, so only a limited set of characters are allowed, and as soon as it finds an illegal character it stops and attempts the substitution using what it has found. If an opening parenthesis is found, the matching is relaxed somewhat (because it can expect to find an ending marker in the form of a closing parenthesis), so that different kinds of substitution can be performed within the index (which is why set i () ; puts $a($i) works).
In this case the matching stops if the word ends (i.e. whitespace is found; the substitution will fail because of the malformed variable name) or a closing parenthesis is found. This is why $a(()) fails: the interpreter tries to use a(() as a variable name, followed by a ) as the next character in the word. $a((\)) does work, since escaping the first right parenthesis negates its syntactic meaning.
If the first character after the $ is an opening brace, all characters up to the matching closing brace are used as a variable name, meaning that there can be no matching close brace within the parenthesis, but otherwise almost anything goes. This means that ${a(())} works too.
Documentation: set, Tcl
Comment: It's also possible to use `upvar 0` to make an alias (with a simple name) to the array element.
Answer:
The problem is, how to retrieve it?

The issue is exactly that the $ syntax is restrictive. There are legal values for element names that you cannot write using the $ form (for example, ()}{() will hugely confuse the parser!) and in those cases, you need to use one of the alternate techniques you list. It's not usually a problem precisely because the places where these awkward cases come up are when you've not got elements under the control of the script — since why would you deliberately choose to make things awkward for yourself? — and then you're doing stuff like iterating over the set of names you've looked with array names, in which case you've already got the value in a variable and won't have the problem.
There are several ways to retrieve the value. Here are the ones I recommend (in no particular order):

set theValueIs [set a(())]
set theElementIs (); set theValueIs $a($theElementIs)
upvar 0 a(()) alias; set theValueIs $alias

There's also this, which works but isn't much to my taste:

set theValueIs ${a(())}

TCL array index - what is the spec


Setting Windows Forms sizes and maintaining proportion

Question: I am developing a windows form application. It now has like 10 forms and in every form I am setting its size at properties. I'm wondering, now that I am using a 19inch square screen, what if the user will have something like a 32inch rectangular monitor. Will the proportion of my app forms be maintained or it will look tiny?
Comment: That is automatic, thanks to the Form.AutoScaleMode property.  It would be wise to actually check if the layout of your form still looks okay on such a monitor and that you properly declared your app to be dpiAware.  You can test it by changing your monitor's DPI setting.  Or a very rough check is changing the form's Font property, make it bigger.
Answer: The form should be the same size (in inches) if the bigger screen has the same DPI (dots per inch) set for their screen resolution.  The form will take up less of the screen, but will be the same size.  However, the forms will look smaller if the user with the bigger screen has a greater DPI resolution.  This is because the forms are sized by pixels:

The AutoScaleMode property on a form is set to "font" by default, so there should not be too much of a problem with the user on the bigger screen reading the form.
If you still need to make controls "bigger", you can have the form detect the user's DPI (Dots per Inch) and resize the form controls, if you think it is necessary.  Here is a link on how to do that:
How to control the font DPI in .NET WinForms app


Como imprimir uma string em C?

Question: Estou estudando C e quero fazer um algoritmo que leia o nome de 30 alunos de uma turma e retorne a maior nota, menor nota e o nome desses alunos. Eu já pesquisei muito e ainda não consigo colocar o nome dos alunos com a maior e menor nota. Meu código até agora está assim (com erro):
#include<stdio.h>

main(){
char aluno[21],Maluno[21], maluno[21];
float nota, maiorNota=0, menorNota=10;
int i;

for(i=0; i<30; i++){
printf("Insira o nome do(a) aluno(a) %d:\n", i+1);
fflush(stdin);
scanf("%s", aluno);

printf("Insira a nota do(a) aluno(a) %s:\n", aluno);
fflush(stdin);
scanf("%f", & nota);

if(nota>=0 && nota <=10){

if(maiorNota<nota){
    maiorNota = nota;
    Maluno = aluno;
}

if(menorNota>nota){
    menorNota = nota;
    maluno = aluno;
}
}
}
printf("Aluno com a menor nota e o: %s\n que tirou%f\n" ,maluno,menorNota );
printf("Aluno com a maior nota e o: %s\n que tirou%f\n" ,Maluno,maiorNota );
}

Alguém saberia como fazer isso?
Answer: No caso de string você não consegue passar de uma variável pra outra desse jeito:
maluno = aluno;

Você precisa da função strcpy da biblioteca <string.h>.
#include <stdio.h>
#include <string.h>

int main()
{
    char aluno[21], Maluno[21], maluno[21];
    float nota, maiorNota = 0, menorNota = 10;
    int i;

    for (i = 0; i < 30; i++)
    {

        printf("Insira o nome do(a) aluno(a) %d:\n", i + 1);
        fflush(stdin);
        scanf("%20s", aluno);

        printf("Insira a nota do(a) aluno(a) %s:\n", aluno);
        fflush(stdin);
        scanf("%f", &nota);

        if (nota >= 0 && nota <= 10)
        {

            if (maiorNota < nota)
            {
                maiorNota = nota;
                strcpy(Maluno, aluno);
            }

            if (menorNota > nota)
            {
                menorNota = nota;
                strcpy(maluno, aluno);
            }
        }
    }
    printf("Aluno com a menor nota e o: %s\n que tirou %f\n", maluno, menorNota);
    printf("Aluno com a maior nota e o: %s\n que tirou %f\n", Maluno, maiorNota);

    return 0;
}

Pra mais infoemações sobre a função você pode dar uma olhada aqui
Comment: O `scanf()`já põe o `'\0'` na string. Não é necessário fazê-lo antes, mas é conveniente limitar o scanf para evitar *buffer overflow*: `scanf("%20s", aluno);`
Comment: Usei essa função da biblioteca de string mas continua dando erro. Não faço ideia o que estou fazendo errado
Comment: if(maiorNotanota){
  menorNota = nota;
  strcpy(menoraluno,aluno);
Comment: @Jackgba ajeitei a resposta, tinha esquecido a parte do caractere delimitador

Como imprimir uma string em C?


unfortunately Aplicacao has stopped

Question: Boa tarde.
Estou começando com o desenvolvimento Android e estou tentando fazer uma navegação entre várias telas porém ocorre o erro unfortunately TCC has stopped no momento da execução no emulador. Já pesquisei e não conseguir encontrar o problema. Alguém poderia me ajudar?
Obrigado
Minha Activity
public class MenuActivity extends Activity {

private Button btnLancamentos;
private Button btnObjetivos;
private Button btnSobre;
public  Button btnObjInserir;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_menu);

    btnLancamentos = (Button)findViewById(R.id.btnLancamentos);
    btnObjetivos   = (Button)findViewById(R.id.btnObjetivos);
    btnSobre       = (Button)findViewById(R.id.btnSobre);
    btnObjInserir  = (Button)findViewById(R.id.btnObjInserir);

    btnLancamentos.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            TelaLanc();
        }
    });

    btnObjetivos.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            TelaObj();
        }
    });

    btnSobre.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            TelaSobre();
        }
    });

    btnObjInserir.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            TelaObjInserir();
        }
    });

    if (savedInstanceState == null) {
        getFragmentManager().beginTransaction()
                .add(R.id.container, new PlaceholderFragment())
                .commit();
    }

}

protected void TelaLanc() {
    Intent intent = new Intent(this, TelaLancActivity.class);
    startActivity(intent);
}

protected void TelaObj() {
    Intent intent = new Intent(this, TelaObjActivity.class);
    startActivity(intent);
}

protected void TelaSobre() {
    Intent intent = new Intent(this, TelaSobreActivity.class);
    startActivity(intent);
}

protected void TelaObjInserir() {
    startActivity(new Intent (MenuActivity.this, TelaObjInserirActivity.class));

}

@Override
public boolean onCreateOptionsMenu(Menu menu) {

    // Inflate the menu; this adds items to the action bar if it is present.
    getMenuInflater().inflate(R.menu.menu, menu);
    return true;
}

@Override
public boolean onOptionsItemSelected(MenuItem item) {
    // Handle action bar item clicks here. The action bar will
    // automatically handle clicks on the Home/Up button, so long
    // as you specify a parent activity in AndroidManifest.xml.
    int id = item.getItemId();
    if (id == R.id.action_settings) {
        return true;
    }
    return super.onOptionsItemSelected(item);
}

/**
 * A placeholder fragment containing a simple view.
 */
public static class PlaceholderFragment extends Fragment {

    public PlaceholderFragment() {
    }

    @Override
    public View onCreateView(LayoutInflater inflater, ViewGroup container,
            Bundle savedInstanceState) {
        View rootView = inflater.inflate(R.layout.fragment_menu, container, false);
        return rootView;
    }
}

}
Classe de chamada de tela
package pacote.tcc;

public class TelaObjActivity extends Activity{

protected void onCreate(Bundle savedInstanceState){
    super.onCreate(savedInstanceState);
    setContentView(R.layout.tela_obj);
}

}
Classe de chamada de tela
package pacote.tcc;

public class TelaObjInserirActivity extends Activity{

protected void onCreate(Bundle savedInstanceState){
    super.onCreate(savedInstanceState);
    setContentView(R.layout.tela_obj_inserir);

    Button btVoltar = (Button) findViewById(R.id.btVoltar);
    btVoltar.setOnClickListener(new OnClickListener() {

            @Override
            public void onClick(View v) {
                startActivity(new Intent (TelaObjInserirActivity.this, MenuActivity.class));
                finish();
            }
});

}

}
Comment: Oi, João, bem vindo ao [pt.so]. Por favor, confira http://pt.stackoverflow.com/editing-help
Comment: João, você declarou suas activities no manifest?
Answer: Você tem um fragment na Activity, verifique se activity_menu.xml contém os botões btnLancamentos, btnObjetivos, btnSobre e btonObjInserir ou se eles estão em fragment_menu.xml. Se estiverem no fragment, as linhas
btnLancamentos = (Button)findViewById(R.id.btnLancamentos);
btnObjetivos   = (Button)findViewById(R.id.btnObjetivos);
btnSobre       = (Button)findViewById(R.id.btnSobre);
btnObjInserir  = (Button)findViewById(R.id.btnObjInserir);

retornarão null e a linha seguinte, btnLancamentos.setOnClickListener(…) lançará NullPointerException, causando o crash. Se for esse o caso, mova o código para dentro do PlaceholderFragment.
Comment: Obrigado Douglas. Ajudou muito. (:

unfortunately Aplicacao has stopped


Most complex "block" of sheet music notation

Question: Wondering how complex a single piece of sheet music can get, and what it looks like. For example, on the guitar, theoretically you could define the following:

a chord
bend from another chord
bend to another chord
hold for 3 beats (dot)
wiggle string

I don't have the music software to write the notation at the moment, but a chord might look like:

or

I'm wondering if you also took into account the following, how complex you can make it look while still being readable and realistic:

How hard you hit the notes.
Different sheet music notations for different stringed instruments
Guitar petal notations.
Temporary time signature changes.
Screaming if it is vocals
Other vocal features.
Etc.

Those are just some ideas to add to the complexity of the sheet music. But basically I'm wondering how complex sheet music can get when it tries to encode every feature possible.
By "complex" I just mean the visible look of the notation, rather than how hard it is to play. Doesn't matter the instrument, just whatever results in some of the more complex notations you can construct.
Comment: You might find the musicologist Don Byrd's work to be of interest:  [Extremes of Conventional Music Notation](http://homes.sice.indiana.edu/donbyrd/CMNExtremes.htm);  [More Counterexamples in Conventional Music Notation](http://homes.sice.indiana.edu/donbyrd/MoreCMNCounterexamples.htm);  [Gallery of Interesting Music Notation.](http://homes.sice.indiana.edu/donbyrd/InterestingMusicNotation.html)
Comment: Something that typically comes up in these discussions (but probably doesn't count as it was written as a joke) is "[Faerie's Aire and Death Waltz](http://socks-studio.com/2012/05/19/the-unplayable-score-faeries-aire-and-death-waltz-john-stump/)" by John Stump.
Comment: Perhaps a better question title: How dense can I make the sheet music before it gets unreadable?
Comment: For pure aesthetic delight, see: http://reddit.com/r/sheetmusicporn
Comment: Please explain ' bend to/from another chord'. Chords are not usually bent to or from another. A note played on a string often is.
Answer: It's interesting you use the word "complex." I was hesitant to answer for fear that this question was too opinion-based (and it might still be), but your use of this term made the answer very clear: consider the music (and notation) of the New Complexity movement.
To quote directly from an opening line of the Wikipedia article:

Though often atonal, highly abstract, and dissonant in sound, New Complexity music is most readily characterized by the use of techniques which require complex musical notation. (emphasis my own)

Take, for example, this excerpt from Brian Ferneyhough's "Unity Capsule," which only involves two instruments (played by one performer!):

Ferneyhough is one of the leaders of the New Complexity movement, and any score of his you find will be among the most complex out there.
Comment: Note that you can find this online. It sounds like a constipated budgie farting into a bucket of water.
Answer: Among other things, the sheet music might indicate

a tempo indication (in bold above everything else) e.g. Allegro
a mood indication e.g. espressivo
an absolute dynamic, e.g. mf
a relative dynamic, e.g. poco cresc.
a phrasing slur
an articulation mark e.g. . (staccato) or - (tenuto). You sometimes get . and - combined.
which finger to use
which string to play on
(for vocal music) the words


SignalR WPF Client can't reach hub deployed on IIS when IIS runs on a different system

Question: I just play a little bit with signalR. My application has only one simple hub which is stored in an ASP.NET Application and I wrote a WPF client, which interacts via the hubconnection and the created proxy with the ASP.NET Application. Everything works fine on my local PC. I deployed the ASP.NET Application on IIS.
Now I am getting to the point...
When I type the following into my browser on my own PC (pcthi-and)
http://pcthi-and:8080/signalr/hubs

I'll get what I want
When I type the same url into a browser of another pc I'll get the same response and everything looks fine.
But my Application only works on my pc and not on the other one. When I start the hubconnection on the other pc I don't get a connectionId.
I tried to change the url to my IP-Address without effect.
Browser call to hub works but the Application doesn't work.
The call looks like this:
private bool tryToConnectToCoffeService()
    {
        try
        {
            this.hubConnection = new HubConnection(ConfigurationManager.ConnectionStrings["coffeeConnection"].ConnectionString);

            this.hubConnection.Credentials = CredentialCache.DefaultNetworkCredentials;

            this.coffeeService = this.hubConnection.CreateHubProxy("coffee");

            this.hubConnection.Start();

            if (string.IsNullOrEmpty(hubConnection.ConnectionId))
            {
                return false;
            }

            return true;
        }
        catch(Exception ex)
        {
            return false;
        }
    }

The Global.asax:
public class Global : System.Web.HttpApplication
{

    protected void Application_Start(object sender, EventArgs e)
    {
        RouteTable.Routes.MapHubs();
    }

The hub like this
[HubName("coffee")]
public class CoffeeHub : Hub
{

My Hub Connection String is this:
"http://pcthi-and:8080/"

Or:
"http://My-Current-IP-Address:8080/"

I use SignalR 1.0 rc2.
Does anyone have an idea? Thanks for helping.
Cheers
Frank
Answer: I think you need to change
hubConnection.Start();

to
hubConnection.Start().Wait();

If you are running .NET 4.5 you could make the tryToConnectToCoffeService method async and then await when you start the hub connection.
await hubConnection.Start();

It likely works today on localhost because the client can finish connecting before if (string.IsNullOrEmpty(hubConnection.ConnectionId)) executes.
It is probably taking longer to connect from another machine which exposes the race condition present when you don't wait for HubConnection.Start() to complete.
Comment: thank you very much, yesterday I read your post and I was really looking forward to follow your advice at work, and it works ;-)! thanks a lot.


Positive correlation and negative regressor coefficient sign

Question: Is it possible to obtain a positive correlation between a regressor and a response (<code>+0,43</code>) and, after that, obtain a negative coefficient in the fitted regression model for this regressor?
I'm not talking about changes in the sign of the regressor among some models. The coefficient sign always remains.
Could  the remaining variables of the fitted model influence the changing of the sign?
Comment: You will find a similar question and answers here : http://stats.stackexchange.com/questions/33888/x-and-y-are-not-correlated-but-x-is-significant-predictor-of-y-in-multiple-regr/33897#33897
Answer: Both @Henry, and @JDav are pointing you in the right direction (+1 to each).  However, I'm very visual and it helps me if I can see how this works.  In that respect, here's a quick plot in which the first variable is confounded with group membership.  If the groups are ignored, the correlation coefficient is positive (as can be seen in the figure), but in a multiple regression, $\beta_{x1}=-1$, albeit with different intercepts for the three groups.

As further food for thought, when all variables are categorical (instead of continuous as in this case) the phenomenon of reversing the apparent relationship upon inclusion of other variables is known as Simpson's paradox.  Since it's ultimately quite similar, it may help to read about that as well.  It is discussed on CV here.
Comment: FWIW, I have a fuller treatment of essentially the same issue here: [Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?](http://stats.stackexchange.com/questions/78828//78830#78830)
Comment: @GuilhermeParreira, of course.
Comment: @GuilhermeParreira, comments are not the place to ask questions. 1st read the linked thread above to help better understand the phenomenon.  You will also want to read some of our threads on causal inference with observational data.  If you still have a question afterwards, you should ask a new question.
Comment: Could this same pattern occur in the context of causal inference?
Comment: I see. Now I understand why this occurs. But it is still challenging to present it for a business area. Ex.: I have a variable X (satisfaction score ) positively correlated to Y (NPS score), which makes sense. When I run a Lasso model, the beta is negative (due to other covariates). In either 'causation' or 'correlation', how could I explain that increasing satisfaction score would decrease NPS? (net promoting score, which is likely you would recommend my business to someone else?)
How could I communicate this? Tks!!
Answer: If the positively-correlated  regressor  is the only  regressor  in a linear model then its coefficient should be positive.
If there are several  regressors and they are not independent then you can see the effect you are asking about.  Read about confounding for some explanation


How to tick checkbox of listview checkbox listcell programmatically in javafx

Question: This is my app
I am using mongodb.
For other types which are string , int , etc i can easily store and read data from db. But for list view i have a document which has two keys: title and completed.
The completed stores boolean value of tick(checkbox listcell).
But i cannot find a way such that:
1 to save the tick(tick or unticked) in db.
[2] to manually set the tick during app startup(i use db to prevent data loss after closing app)
    asList.setCellFactory(CheckBoxListCell.forListView(new Callback<String, ObservableValue<Boolean>>() {

        @Override
        public ObservableValue<Boolean> call(String arg) {
            return new SimpleBooleanProperty();
        }

    }));

// Here asList is ListView
Answer: The selectedStateCallback maps each element in the list view to a BooleanProperty that is bidirectionally bound to the selected state of the check box. So to select or unselect the check box, you need to set that BooleanProperty to true or false, respectively.
In your code, you don't keep any reference at all to the BooleanProperty you create for this, so you have no opportunity to change its state.
The best approach is to create a class to represent your document, having title and completed properties. Represent the completed property with a BooleanProperty, and supply that property from the selectedStateCallback. Make the ListView a ListView<Document>, and then you just have to set the completed state of the Document and the check box will update accordingly.
SSCCE:
import javafx.application.Application;
import javafx.beans.property.BooleanProperty;
import javafx.beans.property.SimpleBooleanProperty;
import javafx.beans.property.SimpleStringProperty;
import javafx.beans.property.StringProperty;
import javafx.geometry.Insets;
import javafx.geometry.Pos;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.ListView;
import javafx.scene.control.cell.CheckBoxListCell;
import javafx.scene.layout.BorderPane;
import javafx.stage.Stage;
import javafx.util.StringConverter;

public class CheckBoxListCellExample extends Application {

    @Override
    public void start(Stage primaryStage) {
        ListView<Document> listView = new ListView<>();

        StringConverter<Document> converter = new StringConverter<Document>() {

            @Override
            public String toString(Document document) {
                return document.getTitle();
            }

            // not actually used by CheckBoxListCell
            @Override
            public Document fromString(String string) {
                return null;
            }

        };

        listView.setCellFactory(CheckBoxListCell.forListView(Document::completedProperty, converter));

        for (int i = 1 ; i <= 20 ; i++) {
            listView.getItems().add(new Document("Document "+i, i%2==0));
        }

        Button markAllCompleted = new Button("Mark All Completed");
        markAllCompleted.setOnAction(e -> {
            for (Document doc : listView.getItems()) {
                doc.setCompleted(true);
            }
        });

        BorderPane root = new BorderPane(listView);
        BorderPane.setAlignment(markAllCompleted, Pos.CENTER);
        BorderPane.setMargin(markAllCompleted, new Insets(10));
        root.setBottom(markAllCompleted);
        Scene scene = new Scene(root);
        primaryStage.setScene(scene);
        primaryStage.show();
    }

    public static class Document {
        private final StringProperty title = new SimpleStringProperty();
        private final BooleanProperty completed = new SimpleBooleanProperty();

        public Document(String title, boolean completed) {
            setTitle(title);
            setCompleted(completed);
        }

        public final StringProperty titleProperty() {
            return this.title;
        }

        public final String getTitle() {
            return this.titleProperty().get();
        }

        public final void setTitle(final String title) {
            this.titleProperty().set(title);
        }

        public final BooleanProperty completedProperty() {
            return this.completed;
        }

        public final boolean isCompleted() {
            return this.completedProperty().get();
        }

        public final void setCompleted(final boolean completed) {
            this.completedProperty().set(completed);
        }

    }
    public static void main(String[] args) {
        launch(args);
    }
}


Conectar microfone ao analyser para obter hertz com javascript

Question: Como faço para conectar a entrada de audio vinda do microfone ao analyser da API AudioContext para obter a frequencia em Hertz?
Javascript:
let display = document.getElementById("display");
let ctx = new AudioContext();
var analyser = ctx.createAnalyser();
analyser.connect(ctx.destination);

navigator.mediaDevices
    /* Solicita permissão de acesso */
    .getUserMedia({ audio: { echoCancellation: true } })
    /* Cria um fonte de stream */
    .then(stream => {
        let source = ctx.createMediaStreamSource(stream);
        source.connect(analyser);
        display.innerText = analyser.getFloatFrequencyData()[0];
    })
    .catch(err => {
        console.log("Microfone negado");
    });

HTML:
<!DOCTYPE html>

    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>API JS</title>

0

Não estou conseguindo fazer essa obra, se eu conectar a variavel stream do then() diretamente a um objeto de audio (com controle de bausa e mute...) consigo ouvir ao mesmo tempo que gravo, porém não consigo interceptar com o analyser para obter os Hertz o audio.
Answer: Dá para ligar um Analyser e um ScriptProcessor.
var display = document.getElementById("display");

navigator.mediaDevices.getUserMedia({audio: true})
.then(function(stream) {
    var ctx = new AudioContext();
    /* Cria um fonte de stream */
    var source = ctx.createMediaStreamSource(stream);
    var analyser = ctx.createAnalyser();
    var processor = ctx.createScriptProcessor(256, 1, 1);
    source.connect(processor);
    source.connect(analyser);
    processor.connect(ctx.destination);

    processor.onaudioprocess = (data) => {
        var dataArray = new Float32Array(analyser.frequencyBinCount);

        setInterval(() => {
            analyser.getFloatFrequencyData(dataArray);
            display.innerHTML = dataArray;
        }, 1000);
    }

})
.catch(err => {
  console.log(err.message);
});


Flutter Codemagic Static Pages failed without any error

Question: I try to build my flutter app for web with Codemagic. But the publish failed every time.
On my pc it works without any problems and Codemagic also worked till one week ago.
Here the output of Codemagic:

I see no reason why it failed.
Any ideas?
I disabled the Publish step and then it works
Answer: unfortunately there was a bug in gcloud utils which Codemagic uses internally to publish web apps.
The team prepared a fix and it was deployed. So the publishing should work as expected.
=== BEFORE UDATE ===
There is also a workaround you can switch to Xcode 13.2 virtual machine that not affected by the gcloud issue.
Comment: I'm afraid this still seems to happening as of today, using the Linux X2 VM.
Comment: @Maks thank you for reporting this, we have found another issue on Linux instances and it was fixed now. Can you give it another try?
Comment: thanks Mikhail! Yes the deploy step succeeds now but the deployed app doesn't seem to work correctly in comparison to the same build being deployed and working correctly on Firebase hosting. I'll look into it more and take up the discussion on CM's slack.


Reflect data quality in Kalman filter

Question: In a single stream of observations, I have some prior knowledge about the level of noiseness of the data that get feed into my Kalman filter. Some points are more noisy than the others.
To make use of this knowledge, I have two thoughts
(1) partition the data into two streams: low confidence and high confidence
(2) scale my confidence in data to a factor b/w 0 and 1, and multiply the kalman gain with this factor.
Can someone shed some light on this problem with some new thoughts or comment on my thoughts. Any comments would be appreciated.
Comment: @Memming it is stationary. The confidence level is more like a continuous spectrum. For example, if stock price is the measurement series, the associated volume could act as a confidence level. The higher the volume is, the higher confidence we have on this dat point. On the other hand, we may discretize the volume spectrum and separate the series based on that.
Comment: can you elaborate on this noise process you have prior knowledge about? Is it stationary? Is it binary (since you mention low and high regime)?
Answer: Kalman filter in principle supports variable noisiness of data, so you are good. Just encode the noisiness info as a sequence of observational (co)variance matrices, $\mathbf{R}_k$ in wikipedia notation.
It might be that the implementation of the Kalman filter that you use does not support it, then either hack it (like I did with pykalman for myself), or find one that does.
Also, you will have to translate the "level of noiseness" to variance, which might involve some thinking if you get that level in some crazy form (like the binary form that you were contemplating, bad idea btw).
Comment: Thank you. But I'm not sure this answers my question. Of course I could ignore the prior knowledge about the different confidence level and construct R matrix based on the **average** variance.
Comment: No, don't construct one R matrix based on average variance. Use different R matrices for each time step! That's why it is denoted $\mathbf{R}_k$, $k$ is the index for time step.
Comment: I see what you mean. The flexible scaling in R definitely makes sense.

Reflect data quality in Kalman filter


How to get a Clob using java

Question: I'm having some problem to get a Clob field using java. So this is what I've done:
    String
    sql = ("SELECT DBMS_LOB.substr(LDTEXT, 1000) AS TEXT "
        +  "FROM LONGDESCRIPTION ");

    try (Connection connection = BDOracle.abreConexaoDB2();
            Statement statement = connection.createStatement();
                ResultSet resultSet = statement.executeQuery(sql);
            ) {

        while (resultSet.next()) {

It breaks here:
            Clob clob = (Clob) resultSet.getClob("TEXT");

            long length = clob.length();
            String out = clob.getSubString(1, (int) length);
        }

    } catch (SQLException err) {
        System.out.println(err.getMessage());
    }

Exception returns:

ERRORCODE=-4461, SQLSTATE=42815
  [jcc][t4][1092][11644][3.53.70]
  invalid data conversion wrong result column type for requested conversion.

The query works fine on the database, I'm using DB2.
Answer: Assuming LDTEXT is a CLOB, DBMS_LOB.SUBSTR() will return a VARCHAR value, as indicated in the manual. Subsequently, you should use getString(), not getClob() to retrieve it (and use the correct column name -- you have TEXT0 while the query defines it as TEXT).
Comment: You were right! I changed the code and used "getString()" and it worked perfectly. Many thanks

How to get a Clob using java


In php, increment number is something wrong

Question: I made writer : comment pair programing.
So from $4 to $5 is the first pair of comments.
$arr[even] (>= 4) is username, $arr[odd] (>=5) is a comment of $arr[odd-1], that is a writer name.
So I am writing the php code.
 for ($i=4;$i<count($arr)-1;$i+=2) {
        $name = $arr[$i];

        $comment = $arr[$i++];
        echo $name." ".$comment."
";
                    }

But, that's results is not my thinking.

I want the below picture.

I am waiting for your kind advice.
Answer: You are using a post increment in the comment($i++), so this will use the value and then increment it.  So this means that they both use the same value of $i in...
   $name = $arr[$i];
   $comment = $arr[$i++];

As you are incrementing the value of $i in the loop, I would just do...
   $name = $arr[$i];
   $comment = $arr[$i+1];


GKE cluster and Steam master server

Question: I need to specify a list of IP ranges in Steamworks to mark them as the official servers of my game. How can I create a cluster that creates nodes only in a constant range of external IP addresses? Is there a better solution?
Answer: There is not enough information your infrastructure. But I can assume, that you should reserve IP addresses in Google Cloud console. Then you should use them in your GKE installation, as IP address for Ingress controller or for Service.

GKE cluster and Steam master server


Dynamic Outlier Detection in loop with Random Forest regression in R

Question: I'm using randomforest regressor to predict values. I'm trying to make a automatic learning model.
My database has let's say 100 rows. So, I train my model with only 10 rows.
And one by one I want to predict a value and see if with only 10 rows, my random Forest can predict the value correctly.
If not, then I will insert this new row in the trainset (with 10 values, now becoming 11) then try again with the next value and so forth until 100.
Is there a way to find out, before predicting the value with the model, if randomForest can precisely predict the value?
Would an outlier detector be useful?
I am looking for something that will say : Oh I have never seen a value like this, I can't predict it precisely.
I tried using the lofactor function from DMwR package but the outlier method just finds the outlier of the whole TrainSet and not wether or not the last value I added in the Trainset is an outlier or not.
I hope this is somewhat clear,
thank you all
Answer: Is there a way to find out, before predicting the value with the model, if randomForest can precisely predict the value?
Well, if the values of the predictors (especially the important ones) for the new data point(s) are outside the range of values that the model was trained on, it is much less likely to generate accurate predictions. This is because RFs do not extrapolate well, essentially predicting that the response beyond the most extreme points used for training is completely flat. But besides this, I do not see a way to generate such an expectation without actually using the model to make the prediction.


Keep track of user uploads within articles

Question: just can't get my head around this.
I just finished my own WYSIWYG editor. It allows my users to upload images to embed in their article they are writing.
But how can i keep track of those user uploads ? I want them to be attached to the article, so that when i remove an article all images will also be removed. (keep my disk clean)
My first bet was to add an hidden input field to the article form for every uploaded image containing the image name, then on submit move all attached images from the tmp dir to the article image dir.
But this way a user can attach 100 times a picture to an article, remove them 99 times from the WYSIWYG editor and use only 1. Now i would have to save all the unused images. This seems like waste to me.
Any best practices on this ?
Comment: How does your editor upload/attach images to begin with?
Comment: As said:
Upon (ajax) upload it appends an hidden input field to the article form. On submitting the article form, i will loop trough the hidden input fields en move the uploaded files.
Comment: any other suggestions on the upload/attaching system are appreciated
Answer: Don't worry about the edge case. Nobody is going to upload then deassociate 99 images.
You have to move the received files out of the tmp dir as soon as the (ajax?) upload sends them over. You cannot wait till the article submission, because the /tmp files would be gone by then.
Try to save your images with enumerated filenames like ArticleName.1.jpeg. And let your WYSIWIG editor inject uniqure placeholder tokens like {{img1}} on which you can see which images are factuall still in the article. (You can also try to deduce the file ids out from <img src="upl/articlename.1.jpeg"... since your WYSIWIG editor generates HTML.)
Anyway, on upload compare the list of enumerated {{img123}} tokens with the existing list of files.123.jpeg. Remove the unused.
Alternative: Do not remove unused images. Sell it as feature, since you now have more article history. And I guess the filesizes are negligible for most web sites.


What's the name in english of putting your voice over a leading voice in a song?

Question: What's the name in english of putting your voice over a leading voice in a song?
Suposse Adele just recorded a song, and they want another singer to record some vocals for the song in order to "harmonize" the song.
How do you call that process in english? In spanish we would say: 'Hacer segunda voz' which would translate to something like 'make second voice'.
Comment: "Harmonize" is a common word for this. As in, "I harmonized with the lead vocals" or "I recorded some harmonies" or "We recorded all the harmony vocals".
Answer: In general it's called 'backup vocals.'
However, I'm not clear what you mean by 'over.'  If you mean above (in pitch) I don't know a specific term for that.
Comment: "I'm recording backup vocals", or "I'm overdubbing backup vocals".  Or you could call them "harmony vocals" or "harmony parts".
Comment: 'Backing' rather than 'backup' is the usual, in English. Or 'doubling' if singing in unison - the OP doesn't state which, 'over' could mean alongside or maybe an upper harmony - more common than a lower harmony in pop.Even just 'a harmony'.
Comment: So, would I say? "I'm backing up some vocals."?
Comment: If you mean above in pitch then that is usually called a descant or descant line.
Comment: @Francisco Ochoa - "I'm backing up some vocals," while perfectly correct, doesn't sound idiomatic.  "I'm doing some backup vocals,"  or one of Bruce's versions would sound better, in my opinion.
Comment: @JimM - I know 'descant' is used in classical, particularly sacred classical.  Is it used in other genres as well?
Comment: L3B -  I've heard it used in folk as well.  However perhaps it is not the term most  commonly used, although I think it is still technically accurate.
Answer: Counter singing?  You'd use this term for the kind of singing Art Garfunkel did in the group Simon & Garfunkel, here in "The Boxer".
Answer: Harmonization. The process of singing in a scale above or below the main scale to create a satisfying audio.
Comment: @Shevliaskovic - harmonisation with s is the accepted spelling in English. American English prefers the z. So, both correct! - And the OP asked for English !!
Comment: @Tim The Oxford dictionary gives either -ise or -ize. A lot of the woo-woo about "British English always uses -ise" actually came from Microsoft's early spelling checkers applying a simple rule to convert American spellings into what they *thought* was British English. Many -ize spellings have been in British English for centuries before that.
Comment: @alephzero - I guess my comment was about why change something that is already correct for something else that is correct. If it ain't broke...


How can I tell if I'm on a taxonomy page in preprocess_page?

Question: If I'm on a node, inside preprocess_page(&$variables), I can check
if (isset($variables['node']) && $variables['node']->bundle() == 'my-type')
to see if I'm on a node, and a node of a certain type.
Then I can get field information with
$variables['node']->field_icon->entity->getFileUri()

How can I do the same thing for a taxonomy term page to know that I'm on a taxonomy term page and get the field values?
Answer: Unfortunately I don't think there is anything that the hook gives you directly that you can use in this instance. However you can always check the route name to determine if you are on a taxonomy term page.
// Will be "true" if you are currently on a taxonomy term page.
if (\Drupal::routeMatch()->getRouteName() == 'entity.taxonomy_term.canonical') {

  // Since this is a taxonomy term page, you may wish to get the term ID.
  $term_id = \Drupal::routeMatch()->getRawParameter('taxonomy_term');
}

Answer: To check term data in theme_preprocess_page for drupal 7. you can use below code.
if (arg(0) == 'taxonomy' && arg(1) == 'term') {
if (isset($variables['page']['content']['system_main']['term_heading']['term']['#term'])) {
  $term_data = $variables['page']['content']['system_main']['term_heading']['term']['#term'];
}

}
Hope this helps.
Answer: There is two ways to get the term in preprocess_page.

Get the tid from parameter and then load the term.
Get the Term directly from ParameterBag.
Her is code snippet

use Drupal\taxonomy\Entity\Term;
/**
 * Implements hook_preprocess_HOOK().
 */
function myModule_preprocess_page(&$variables) {
  // Get the tid.
  $tid = \Drupal::routeMatch()->getRawParameter('taxonomy_term');
  // Load the term.
  $term = Term::load($tid);

   // Get possible params.
   /** @var \Symfony\Component\HttpFoundation\ParameterBag $possible_parameters */
   $possible_parameters = \Drupal::routeMatch()->getParameters();
    // Get term
    $term = $possible_parameters->get('taxonomy_term');
}


Using template on subsite for certain node type

Question: I am using a custom Theme in a multisite setting. With the help of @leymannx I implemented
hook_theme_suggestion_HOOK_alter to insert a page Template with the following naming convention:
page--NAMEOFSUBSITE.html.twig
See here
If I try to do the same for the node of my content type, which is using a template named node--CONTENTTYPENAME.html.twig at the moment, I can insert a name suggestion node--NAMEOFSUBSITE.html.twig, which will then be used for ALL nodes. I'd like to insert a suggestion such as
node--CONTENTYPENAME--NAMEOFSUBSITE.html.twig, for only this one nodetype.
How can I do that?
Thanx in advance!!
Answer: Activate twig debug and use the browser inspector to see the template precise name available for your content-type
Do a cache rebuild when the temaplte is created to make it happen.
Answer: MYTHEME_theme_suggestions_node_alter() should look somewhat like this:
<code>function MYTHEME_theme_suggestions_node_alter(array &$suggestions, array $variables) {
  $site_path = \Drupal::service('site.path');
  $site_path = explode('/', $site_path);
  $site_name = $site_path[1];
  $type = $variables['elements']['#node']->getType();
  $suggestions[] = 'node__'.$type.'__' . $site_name;

}
</code>
I found this kint() inspecting the variables within the hook, then finding the methods defined on ['#node'].


Code::Blocks not compiling release for SFML project

Question: Whenever I try to build a release .exe for any SFML project, I get this message:

<code>-------------- Build: Release in SFMLPROJECT (compiler: GNU GCC Compiler) ---------------

mingw32-g++.exe -LC:\SFML-2.1\lib -o bin\Release\SFMLPROJECT.exe obj\Release\main.o
-s  -lsfml-graphics -lsfml-window -lsfml-system -mwindows

Process terminated with status 0 (0 minute(s), 0 second(s))
0 error(s), 0 warning(s) (0 minute(s), 0 second(s))
</code>

And when I go to check on the Release folder, It's empty. If I try to run the project from Code Blocks, it "builds" like about, says this:

<code>-------------- Run: Release in SFMLPROJECT (compiler: GNU GCC Compiler) ---------------

Checking for existence: C:\User\Desktop\C++\SFMLPROJECT\bin\Release\SFMLPROJECT.exe
</code>

...and then says: "It seems that this project has not been built yet. Do you want to build it now?". If I hit "yes", it just "builds" again without actually building anything. If I click "no" it just says the execution failed, and cancel of course does nothing.
I've tried looking for a solution but I just haven't come across any. I also tried using another compiler (Digital Mars) and I still encounter the same problem. Everything runs fine with Debug. Does anyone know what's happening here?
Answer: I've seen a few guys who couldn't even get their first project running with Code::Blocks, so I strongly recommend you to throw it away and switch to Visual C++.  Because I assume your final goal is not to become an expert of C++ compiler and software development toolkits, but to present your creativity to the world through your product.  (Maybe you can even throw away C++ in favor of Unity but I'm not an expert of it.)
If you are committed to using Code::Blocks, these articles might help:

http://www.youtube.com/watch?v=ZqjNOd1UYtE
http://en.sfml-dev.org/forums/index.php?topic=11652.0

(I recommend you to read them because of my general knowledge on trickiness of using MingW)
Comment: I think you might be right about Code::Blocks. That said, I'm definitely going to check out your links because I'm pretty used to the set-up already. Thanks!
Comment: I don't think that telling people to use another IDE does help them much at all. Especially when so many people recommend Code::Blocks to C++ newbies.
That said, I recommend using Visual Studio even when there is no (official) SFML2.1 for VS2013.


Comparing dictionaries containing actually equatable types, based on non-equatable data structures

Question: So I want to equate two dictionaries of the same type [AnyLanguage: SortedList<String>].
AnyLanguage conforms to Hashable, and therefore also Equatable, so it's not a problem. SortedList on the other hand does not conform to Equatable, as it is generic over Element, which has no restrictions:
public struct SortedList<Element> { ... }

Now, I have implemented the equality operator for SortedList, when its Elements conform to Equatable:
public func ==<T: Equatable>(left: SortedList<T>, right: SortedList<T>) -> Bool

Now, when comparing two of the aforementioned dictionaries:
let dictA: [AnyLanguage: SortedList<String>] = [:]
let dictB: [AnyLanguage: SortedList<String>] = [:]

dictA == dictB

I get the following error:

I assume that the compiler is complaining because SortedList does not conform to Equatable, even though I have implemented the comparison operator for equatable Elements (which SortedList<String> would be able to use, as String is equatable).
How can I compare the two dictionaries?
Comment: This is the same problem as in https://forums.developer.apple.com/thread/7172: "Why can't I make Array conform to Equatable?". – You cannot make `SortedList` conform to Equatable *only if* `Element` is Equatable.
Answer: The == operator for dictionaries requires that both Key and Value
type conform to Equatable.
You have implemented a (restricted) == operator for SortedList<Element>.
But implementing a == operator for a type does not make that type
Equatable automatically. The conformance must be declared explicitly.
Unfortunately, it is (currently) not possible to make SortedList<Element> conform to Equatable only if Element
is Equatable. The same problem exists for arrays, compare
Why can't I make Array conform to Equatable? for a discussion
in the Apple developer forum.
The only solutions – as far as I know – are to make SortedList<Element>
conform to  Equatable unconditionally (as in Cristik's answer), or to define
a custom comparison operator for dictionaries
func ==<Key : Equatable, T : Equatable>(lhs: [Key : SortedList<T>], rhs: [Key : SortedList<T>]) -> Bool {
    // ...
}

Answer: SortedList is not Equatable, so Dictionary can't compare the elements. You need to declare as Equatable, and implement == without the generic constraint. This will work:
public struct SortedList<Element>: Equatable { ... }

public func ==<T>(left: SortedList<T>, right: SortedList<T>) -> Bool

If you want to restrict your SortedList to Equatable elements, you can declare it to accept only stuff that conforms to Equatable
public struct SortedList<Element: Equatable>: Equatable

however for your SortedList I think Comparable would suit better.
Comment: @MarcusRossel that can also be done by adding a type constraint to `Element`, I updated my answer with this.
Comment: `SortedList` can't simply be made `Equatable`,as I can't compare the elements, if `Element` is not `Equatable`. That'S the whole problem.


REGEX match string include content inside optional delimiter

Question: I am trying to match a string using regex and I am very close to having it working the way I want.
Lets say I have a string 5A(test1),4B,3C(test2)
The first thing I do is break the string apart on the commas, so i end up with 3 strings in an array

5A(test1)
4B
3C(test2)

Now I want to pull the following information out; the digit, the letter and the content in the parentheses.  But the parentheses are optional.
Here is my pattern
([1-9][0-9]*)([AaBbCcIiFfPpSs]+)(\(.*\))?
This works except that it includes the parentheses.  so i get
5 A  (test1)
when what i want is
5 A test1
Ive also tried
([1-9][0-9]*)([AaBbCcIiFfPpSs]+)\(([^)]*)\)?
But this doesn't match on the strings without the parentheses so
5A(test1) and 3C(test2) match but 4B does not.
Any assistance would be appreciated.
Answer: You need an additional non capturing group around the optional last part
([1-9][0-9]*)([AaBbCcIiFfPpSs]+)(?:\(([^)]*)\))?

See it here on Regexr
Answer: Change your regex a bit:
([1-9][0-9]*)([AaBbCcIiFfPpSs]+)(\((.*)\))?

The content inside () will be in capturing group 4.
If your language supports non-capturing group (?:pattern):
([1-9][0-9]*)([AaBbCcIiFfPpSs]+)(?:\((.*)\))?

This will prevent unnecessary capturing (saves some memory), and the content inside () will be in capturing group 3.
Comment: Your second answer does seem to process faster on the server (running in Perl) SO thanks, much appreciated :)


Fourier transform either changes sign infinitely often far out or is continuous at $x=0$

Question: I am reading a book "Fourier Series and Integrals" by Dym & McKean.
There is an exercise (Page 106):

Exercise: Check that if $f$ is a real, even, summable function and
  if $f(0+)$ and $f(0-)$ exist, then either $f(0-) =f(0+)$ or $\hat f(\gamma)$ changes sign
  infinitely often as $|\gamma| \to \infty$.

Note that $\hat f(\gamma)$ is a real function, so its "sign" makes sense!
There is a hint for the exercis as follow:

Hint: The function $f$ is summable if it is of one sign far out, as you can see from
  $$\frac{f(0-) +f(0 + )}{2} = \lim_{t \to 0} \, (P_t * f) (0) = \lim_{t \to 0} \int_{-\infty}^{\infty} \exp(-2 \pi^2 \gamma^2 t) \hat f(\gamma) \, \mathrm{d}\gamma.$$
  Here $P_t=P_t(x)=\dfrac{\exp(-\dfrac{x^2}{2})}{\sqrt{2\pi t}}$ is the Gauss Kernel and $P_t * f$ means the convolution of $f$ with Gauss Kernel.

My try:
If f is of one sign far out, then by using
$$\frac{f(0-) +f(0 + )}{2} = \lim_{t \to 0} \int_{-\infty}^{\infty} \exp(-2 \pi^2 \gamma^2 t) \hat f(\gamma) \, \mathrm{d}\gamma,$$
and Monotone convergence Theorem we deuce that $\hat f \in L^1(\mathbb{R})$ so $$f(-x)=\hat {\hat f}, $$ is continuous and consequently $f(x)$ will be continuous at $x=0$ and therefore $f(0-) =f(0+)$.
I don't know how to handle the other half.
Thanks.
Screenshot of the exercise
Comment: Err, something seems to be amiss. If $f$ is even, we automatically have $f(0+) = f(0-)$ if one (and by evenness both) of the one-sided limits exists.
Comment: @ Daniel Fischer: It was odd for me too. I uploaded the screenshot of exercise.
Comment: http://mathoverflow.net/a/258997/11260
Answer: As Daniel Fischer noted, in the present form the exercise is trivial: if $f$ is even and one-sided limits at $0$ exist, they are automatically equal. I think what the authors really mean is this:

Exercise: Check that if $f$ is a real, even, summable function and
  if $f(0+)$ and $f(0-)$ exist, then either $f$ has a continuous representative or $\hat f(\gamma)$ changes sign infinitely often as $|\gamma| \to \infty$.

Having a continuous representative means there is a continuous function $g$ such that $f=g$ almost everywhere.
The proof then goes as hinted in the book. Since $f$ is even and real-valued, $\hat f$ is also even and real-valued. We suppose $\hat f$ has only finitely many changes of sign; so the goal is to prove that $f$ has a continuous representative.
Since $\hat f$ is continuous, the question of its integrability is one of behavior at infinity. At infinity, $\hat f$ is of constant sign (its evenness ensures it's the same sign at $-\infty$ as $+\infty$).  The existence of the limit
$$\lim_{t \to 0} \int_{-\infty}^{\infty} \exp(-2 \pi^2 \gamma^2 t) \hat f(\gamma) \, \mathrm{d}\gamma$$
then implies $\hat f\in L^1(\mathbb R)$. By the $L^1$ Fourier inversion, the inverse Fourier transform of $\hat f$ agrees with $f$ almost everywhere. Since this transform is continuous, the conclusion is proved.


MAPE vs R-squared in regression models

Question: Usually regression models are evaluated using $R^2$. I understand this metric can be misleading too at times but as far as I understand the first parameter we look at is $R^2$.
There is another parameter which is often used and it is $MAPE$.
Both are functions of errors between predicted and true value. I am just wondering if there are certain cases where one should be preferred above another?
$R^2$ can become negative if being used on test data (on which the model is not built). Is this a reason to not use $R^2$ in these cases?
Can anyone give a qualitative insight on where $MAPE$ should be used and where $R^2$ should be used.
Comment: Very much related: [What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?](https://stats.stackexchange.com/q/299712/1352)
Comment: Have your read the [Wikipedia article on MAPE](https://en.wikipedia.org/wiki/Mean_absolute_percentage_error)'s list of disadvantages with the metric?
Comment: @Alexis yes I have. I did not see any comparison with R^2 and more interested in comparing these two metrics. Does R^2 overcome any of the disadvantages which MAPE has?
Comment: Yes. For example, $R^{2}$ is, for a model with a single predictor, bounded by 0 and 1. Adjusted-$R^{2}$, which is a quantity one gets with more than one predictor variable breaks some of the interpretability of $R^{2}$. However, there are other measures, such as [*generalized* $R^{2}$](https://en.wikipedia.org/wiki/Coefficient_of_determination#Generalized_R2), which provides much of the interpretability, even for multiple predictors, and even for things like binomial-outcome link regressions (which don't have residuals in the same way as OLS regression).
Answer: Let's look at the definition of $R^2$: if $y_i$ are the actuals and $f_i$ the predictions, then we let $\overline{y}$ denote the mean of the actuals and $e_i = y_i-f_i$ the error. Then
$$ R^2 := 1-\frac{\sum e_i^2}{\sum (y_i-\overline{y})^2}. $$
If we want to maximize $R^2$, we note that we cannot influence the denominator in this formula. Thus, maximizing $R^2$ is equivalent to minimizing the sum of squared errors (or the Mean Squared Error, mse).
And this actually makes a lot of sense. The prediction that minimizes the expected MSE is the expected value of each $Y_i$ (the distribution from which we observe $y_i$). This is often what we want. Note that other error measures like the MAPE may be minimized by other quantities, so minimizing the MAPE may yield biased point predictions.
For the difference between the MAPE and $R^2$ (also known as MSE, as we have seen), see this earlier post: The difference between MSE and MAPE.

MAPE vs R-squared in regression models


Erro ao gerar gráfico de barras: 'height' must be a vector or a matrix

Question: Olá! Ao construir um gráfico de barras em R Language usando o RStudio me deparei com o seguinte erro:
Error in barplot.default(database$results, main = "EvasionChart", xlab = rotulo[1],  :
'height' must be a vector or a matrix

Pesquisei um pouco sobre, vi respostas de vários tipos mas que não resolveram meu problema, para auxiliar no contexto estou compartilhando o script. Segue Abaixo o código:
database <- data.frame("curso" = 1:10, "e_eolica" = 1:10)
database$curso <- c("ItemA", "ItemB", "ItemC", "ItemD", "ItemE", "ItemF", "ItemG", "ItemH", "ItemI", "sdois")
database$results <- c(1, 2, 3, 4, 5, 6, 7, 8)
rotulo <-  c(1, 2)

par(mgp=c(1,1,0))
png(filename = "C:/Users/BREWERTONTHIAGOOLIVE/Desktop/chart.png", width = 800, height = 800)
barplot(database$results, main = "EvasionChart", xlab = rotulo[1], ylab = rotulo[2], names.arg = database$curso, ylim = c(0, database$results[8]), cex.names = 0.8, xaxs = "i")
grid(nx=NA, ny=NULL)
barplot(database$results, main = "EvasionChart", xlab = rotulo[1], ylab = rotulo[2], names.arg = database$curso, ylim = c(0, database$results[8]), cex.names = 0.8, xaxs = "i", add = TRUE)

dev.off()

Answer: Analizando a linha abaixo veremos que o código de erro está apontando que o xlab e ylab que estão sendo populados com os dados contidos em database$curso e database$results não tem valor equivalente, sendo necessários serem preenchidos. Ou seja, o vetor contem mais elementos que valores:
barplot(database$results, main = "EvasionChart", xlab = rotulo[1], ylab = rotulo[2], names.arg = database$curso, ylim = c(0, database$results[8]), cex.names = 0.8, xaxs = "i")

O erro claro é que o database$curso aponta 10 elementos no vetor e o database$results preenche o vetor com 8 valores, deixando 2 indefinidos, tudo pode ser resolvido acrescentando 2 valores no database$results como no exemplo abaixo:
Antes:
database$results <- c(1, 2, 3, 4, 5, 6, 7, 8)

Depois:
database$results <- c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)


Spring security login for external links

Question: I am developing a web application and I have integrated spring security for authentication. I am using the default login page of spring. There are external links present for my application from other applications and in some documents. I have written controllers for these links to show the required data to the user after successful login. My problem is that when I access the links from other applications or documents I am taken to the login page (which is expected if the user is not logged in) and then I am redirected to home page of the application insted of previously requested URL. This happens all the time when I access from those links and asks for login even if the session is active and then then taken to home page after that. Where as if the URL is copied and pasted in the same browser it works fine. What could be the reason for this to happen?
Here is my spring config file
    <security:http auto-config="true">
        <security:intercept-url pattern="/**" access="ROLE_EDIT,ROLE_VIEW" />
        <security:session-management invalid-session-url="/">
        </security:session-management>
    </security:http>

    <security:authentication-manager>

    </security:authentication-manager>

Could anyone help me on this?
Answer: You can create your own AuthenticationSuccessHandler.  Something like this
<http auto-config="true">
<http-basic  />
<form-login authentication-success-handler-ref="customAuthenticationSuccessHandler"/>
</http>

Inside customAuthenticationSuccessHandler you can override onAuthenticationSuccess method and write your own code for identifying the request URL and redirecting accordingly to the same if it is from external app.
public class CustomAuthenticationSuccessHandler extends
    SavedRequestAwareAuthenticationSuccessHandler {

@Override
public void onAuthenticationSuccess(HttpServletRequest request,
        HttpServletResponse response, Authentication authentication)
        throws ServletException, IOException {
    UserDetails userDetails = (UserDetails) authentication
            .getPrincipal();
    // CODE TO IDENTIFY THE REQUEST URL AND REDIRECT TO SAME IF IT IS EXTERNAL URL(FROM OTHER APP).
    super.onAuthenticationSuccess(request, response, targetUrl);

}

Comment: I tried this earlier and the url which I get from the request is "app root"/login , Im unable to get the previously requested url with this method.
Comment: You can try appending a parameter in request such as clientId which will be specific to the application from where the request is coming. Also in your application maintain a map of each clientId  with a specific redirectURL. In the handler then set the redirectURL as per the clientID u get


How to run multiple input files one after another in C-program

Question: I have a C-Program, which need to be run for 10 different input files. Instead of running the program 10 different times, is it possible to run just 1 time allowing to run each file one after another.
Thank you.
Regards,
Kamal
Comment: Put the file-handling logic into a separate function, call that function for every file?
Answer: Of course, that's what the main arguments are for:
void process_file(char const * filename);

int main(int argc, char * argv[])
{
    for (int i = 1; i < argc; ++i)
    {
        process_file(argv[i]);
    }
}

Usage:
./a.out file1 file2 file3

Answer: int main(int argc, char * argv[])

argc is argument count, so when you pass arguments to your program this variable will tell your main how many arguments have been passed.
argv[0] -> your program's name
argv[1]..argv[n] -> the name of the argumets passed.
So in your main:
for (int i = 1; i < argc; i++)
{
   fopen(argv[i],....);
}

And you can pass the arguments either in the project properties (if you are using MSVS), of command line.


dnsjava library return "no records found"

Question: I'm using dnsjava in my android project "android studio" I have imported the library. But when I run the code I'm getting "no records found" does anyone face this? I found a post here that talks about similar problem but the solution didn't work. as I change the type of query to ANY or A or NS. It just doesn't work. Anyone know if this is due to malfunction of the library code? any suggestions welcome.
<code>package org.pctechtips.netdroid.async;

import android.os.AsyncTask;
import android.util.Log;

import org.pctechtips.netdroid.response.DnsAsyncResponse;

import org.xbill.DNS.Lookup;
import org.xbill.DNS.Record;
import org.xbill.DNS.TextParseException;
import org.xbill.DNS.Type;

import java.lang.ref.WeakReference;

public class DnsLookupAsyncTask extends AsyncTask<String, Void, String> {

    private final WeakReference<DnsAsyncResponse> delegate;

    /**
     * Constructor to set the delegate
     *
     * @param delegate Called when the DNS lookup finishes
     */
    public DnsLookupAsyncTask(DnsAsyncResponse delegate) {
        this.delegate = new WeakReference<>(delegate);
    }

    /**
     * Performs the appropriate lookup for specified record type
     *
     * @param params
     * @return DNS answer
     */
    @Override
    protected String doInBackground(String... params) {
        String domain = params[0];
        int recordType = Integer.parseInt(params[1]);
        Record[] records;

        try {
            records = new Lookup("google.com", Type.ANY).run();
            Log.d("DNS", records.toString());
            if (records == null) {
                return "No records found.";
            }

            StringBuilder answer = new StringBuilder();
            for (Record record : records) {
                String rClass = this.parseRecordClass(record.getDClass());
                answer.append(String.format("%s\t\t\t\t%s\t\t\t\t%s\t\t\t\t%s%n%n", record.getName(), record.getTTL(), rClass, record.rdataToString()));
            }

            return answer.toString();
        } catch (TextParseException e) {
            return "Error performing lookup!";
        }
    }

    /**
     * Determines the string representation of the DNS record class
     *
     * @param recordClass Numeric record class
     * @return Human readable record class
     */
    private String parseRecordClass(int recordClass) {
        switch (recordClass) {
            case 1:
                return "IN";
            case 2:
                return "CS";
            case 3:
                return "CH";
            case 4:
                return "HS";
            default:
                return "IN";
        }
    }

    /**
     * Calls the delegate when the DNS lookup has finished
     *
     * @param result DNS answer
     */
    @Override
    protected void onPostExecute(String result) {
        DnsAsyncResponse activity = delegate.get();
        if (activity != null) {
            activity.processFinish(result);
        }
    }
}
</code>
Answer: Android Oreo made a change that prevents looking up dns servers.  https://developer.android.com/about/versions/oreo/android-8.0-changes.html#o-pri
See: https://stackoverflow.com/a/48973823/5420880 for more details and workarounds.
As a specific workaround for dnsjava, you can supply a comma-separated list of dns servers via the dns.server system property.


Subcomplex of the standard $n$-simplex having nonzero $H_{n-1}$

Question: This is an additional exercise of Hatcher's Algebraic Topology book (link: https://pi.math.cornell.edu/~hatcher/AT/AT-exercises.pdf, section 2.1).
Let $X$ be the standard $n$-simplex with its natural $\Delta$-complex structure and $A$ a subcomplex with $H_{n-1}$ nonzero. How can we show that $A=\partial X$? Using the simplical chain complex of $A$, we see that $A$ must be $(n-1)$-dimensional. There are exactly $n$ $(n-1)$-simplices in $X$. How can we show that $A$ must contain these all? It will suffice to show that if $A$ is the union of $<(n-1)$ faces of $X$ then $H_{n-1}(A)=0$. Any hints?
Answer: Look at the boundary map from degree $n$ to degree $n-1$, and use the fact that $H_{n-1}(X)=0$.
Comment: Yes, but look at the boundary in $X$. Cycles in $A$ of degree $n-1$ are boundaries in $X$.
Comment: A cycle in $A$ is also a cycle in $X$. Since $X$ has $H_{n-1}=0$, such a cycle should be a  boundary in $X$. But every boundary in $X$ is of the form $k(\sigma_0+\cdots+\sigma_n$, where $\sigma_0,\dots,\sigma_n$ are the $(n-1)$-dimensional faces of $X$. This implies that $k=0$ and that there are no nontrivial cycles in $A$, right?
Comment: The boundary map from degree $n$ to degree $n-1$ is trivial of course, because $A$ must be $(n-1)$-dimensional, i.e. it has no $n$-cells.
Comment: @user302934 You got it.


Which Robust fit method to apply to exclude single outliers?

Question: I got some data samples (a,b) and I am trying to calculate correlation between a and b. As I am new in this type of analysis, I have calculated Rsquared with linear regression method and got 0.5 as the result. However, in my graph I could see one clear outlier and if I exclude it manually, Rsquared increases from 0.5 to 0.748 which is significant increase.
In order not to do it manually, is there some method that somebody would recommend, that does robust fit and identifies outlier and excludes them from final R square calculation?
Thanks!
Answer: Yes, to keep it simple, you can request that the standardized residuals be determined during the regression run.  Any standardized residual less than -1.96 or greater than 1.96 will be an outlier.  The question you are asking addresses the topic of Regression Diagnostics, which focuses on different types of residuals, and goodness-of-fit (e.g. $R^2$) of a regression model.
Residuals are basically observations that are "overly influential" on the regression model.
One of my favorite types of residuals are called leverage residuals, which are observations whose $x/y$ values "pull too much" on the fitted regression line.  This can occur with e.g. a cluster of observations that are far removed from the rest of the bulk of the data.  Cook's distance is another type of residual, as well as deletion residuals, studentized residuals, DFFITS, and DFBETAS.  But for starters, you should focus on standardized using the criterion of $\pm 2$.
Comment: Thanks! Ill look at it now. Does Regression diagnostics and standardized residuals have anything to do with RANSAC robust regression? I have found out that possibly I can try to use RANSAC but due to my low understanding of the matter I am not sure how to apply RANSAC on my original dataset as all the scripts I found are related to the training sets.
Comment: Don't know about RANSAC, but robust methods usually focus on methods to reduce inflated standard errors, s.e., of regression coefficients.  Statistical significance for each $j$th regression coefficient is equal to a t-statistic, where $t=\beta_j / s.e.(\beta_j)$ and robust methods usually try to reduce $s.e.(\beta_j)$.   Thus, not related to outliers.
Comment: Thanks! I will take a look now if I can apply it on my data. I have would out from here [https://email@example.com/got-outliers-ransac-them-f12b6b5f606e] that I could possibly also try RANSAC but need to find a away how to apply it on my data :)
Comment: @nxglogic, what you're describing is robust standard errors, but robust regression does aim to reduce the influence of outliers. An example is `MASS::rlm()`, asked about [here](https://stats.stackexchange.com/questions/13702/what-is-the-difference-between-lm-and-rlm/13703).


How to stream WebRTC to WebSocket?

Question: Hi I want to stream a WebRTC (MediaStream) video to a WebSocket (Twisted+Autobahn) and process the stream for OpenCV object detection and send it back to the client via FFServer (FFMPeg)?
I have try using only WebRTC, WebSocket and OpenCV, but the streaming is not real-time.
Comment: What do you think of [this](https://github.com/Computician/WebSocketWebRTCRecorders)? It does not send the stream directly, but sends images and audio packets in real time over a websocket that you provide. Disclaimer, I wrote it.
Comment: Thats nice, thanks. But my requirements is python.
Answer: It is not possible.
WebRTC has its own way of sending media, and it uses DTLS-SRTP as transport in a channel established with ICE.
WebSockets are based on a TCP connection from client to server in order to send raw data in both directions.
You can check this similar answer for more info.

How to stream WebRTC to WebSocket?


Can a positive definite matrix $S$ always be expressed in the form $S=A^TA$?

Question: Can a positive definite matrix $S$ always be expressed in the form $S=A^TA$?
If so, is there a simple proof of this?  I ask the question because another proof in the book Intro to Linear Algebra  by Strang uses this fact.
Comment: If it is Hermitian, yes. https://en.wikipedia.org/wiki/Cholesky_decomposition
Answer: Assuming the matrix is real and symmetric, then yes it can. One way of doing it is known as the Cholesky decomposition. In fact, here we may even assume that $A$ is upper triangular, with positive entries on the diagonal.
(Note that the Wikipedia article mostly assumes that $S$ has complex entries, and that it is Hermitian rather than symmetric. However, in the "Statement" paragraph, the article mentions that the decomposition works for real matrices too, and in that case, $A$ will also have real entries. The original result by Cholesky was also for real matrices, and it was only later generalized to the complex case. Also, Wikipedia transposes the second factor, not the first, so their matrix is lower triangular instead.)
Comment: Yes, it's better now.


Training a neural network for regression always predicts the mean

Question: I am training a simple convolutional neural network for regression, where the task is to predict the (x,y) location of a box in an image, e.g.:

The output of the network has two nodes, one for x, and one for y. The rest of the network is a standard convolutional neural network. The loss is a standard mean squared error between the predicted position of the box, and the ground truth position. I am training on 10000 of these images, and validating on 2000.
The problem I am having, is that even after significant training, the loss does not really decrease. After observing the output of the network, I notice that the network tends to output values close to zero, for both output nodes. As such, the prediction of the box's location is always the centre of the image. There is some deviation in the predictions, but always around zero. Below shows the loss:

I have run this for many more epochs than shown in this graph, and the loss still never decreases. Interestingly here, the loss actually increases at one point.
So, it seems that the network is just predicting the average of the training data, rather than learning a good fit. Any ideas on why this may be? I am using Adam as the optimizer, with an initial learning rate of 0.01, and relus as activations

If you are interested in some of my code (Keras), it is below:
<code># Create the model
model = Sequential()
model.add(Convolution2D(32, 5, 5, border_mode='same', subsample=(2, 2), activation='relu', input_shape=(3, image_width, image_height)))
model.add(Convolution2D(64, 5, 5, border_mode='same', subsample=(2, 2), activation='relu'))
model.add(Convolution2D(128, 5, 5, border_mode='same', subsample=(2, 2), activation='relu'))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(2, activation='linear'))

# Compile the model
adam = Adam(lr=0.01, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
model.compile(loss='mean_squared_error', optimizer=adam)

# Fit the model
model.fit(images, targets, batch_size=128, nb_epoch=1000, verbose=1, callbacks=[plot_callback], validation_split=0.2, shuffle=True)
</code>
Comment: Are the images on top examples of your actual samples?  Is that 5 separate samples? There appears to be no information in the images that would help generalize. I mean, you don't need a neural net to find the x,y location of the white square, you can just parse the image and look for a white pixel. Explain a bit more about your vision for this model. Is there some temporal pattern, whereby you are predicting the next location?
Comment: Hi, and yes, the images are 5 separate samples. I'm not sure how they are rendered for you, but they should be 5 individual square images (I've changed the layout a little to help...). Yes, I realise that you don't need a neural network for this task, but it is just a test experiment to help me learn how to do regression with a neural network. I don't understand what you mean by there being no information to help generalize.... Each training pair consists of a square image, and a two-dimensional vector of the (x, y) location of the square. Thanks :)
Comment: 1) Your input shape on the first conv layer is using 3 (rbg) channels, but your data are greyscale (1 channel)
2) You don't need that many conv layers and filters, in fact I think a single layer, and a handful of small kernels will be fine.
Comment: Like @photox says, you do not need the conv layers. Adding these make it more difficult for the optimizer to find a good solution. If you remove the 3 conv layers I suspect your "model" will work.
Comment: Convolutional layers help with translational invariance due to weight sharing. This doesn't help you at all. As other said before, you would get the result you expect without them.
Comment: Are you sure that images do indeed correspond the targets?
Comment: I know that I do not need 3-channels for this (the images I use are actually RGB), or so many layers, but I am just using this for a test case before applying it to more sophisticated images.
Comment: Yes, I have checked that the images and targets correspond, by drawing the target values on the images and displaying the images.
Comment: I can't see any image.
Comment: My guess would be a bug somewhere in the code we're not seeing.  I implemented more or less what you had above using newer tf.keras and it trains just fine https://colab.research.google.com/drive/1mrCL2m8y50kd3WQA9IvtDl3-GwVku1gX?usp=sharing
Comment: Perhaps obvious, but a big lurking variable is image size and coordinate system.  If I up from 10x10 images to 20x20 it takes way longer to train.  If you have large images you'll likely need to downsample with some pooling layers.
Answer: I am going to contradict @Pieter's answer and say that your problem is that you have too much bias and too little variance. In other words, your network is not complex enough for this task.
To see this, let $Y$ be the true and correct output that your network should return (the target), and let $\hat{Y}$ be the output that your network actually returns. Your loss-function is the mean squared-error, averaged over all examples in your training dataset $\mathcal{D}$ :
$$
\mathbb{E}_{\mathcal{D}}\left[(Y - \hat{Y})^2\right]
$$
In this loss-function, using your network, we are trying to adjust the probability distribution of $\hat{Y}$ such that it matches the probability distribution of $Y$. In other words, we are trying to make $Y=\hat{Y}$, such that the mean squared-error is $0$. This is the lowest possible value of the mean squared-error:
$$
\mathbb{E}_{\mathcal{D}}\left[(Y - \hat{Y})^2\right] \geq 0
$$
However, from the question How can I prove mathematically that the mean of a distribution is the measure that minimizes the variance?, we know that the mean squared-error actually has a tighter lower-bound, which is when $\hat{Y} = \mathbb{E}_{\mathcal{D}}[Y]$, such that the mean squared-error loss-function becomes
$$
\mathbb{E}_{\mathcal{D}}\left[(Y - \mathbb{E}_{\mathcal{D}}[Y])^2\right] = \text{Var}(Y)
$$
Since we know that the variance of $Y$ is non-negative, then the mean squared-error loss-function has the following lower-bounds
$$
\mathbb{E}_{\mathcal{D}}\left[(Y - \hat{Y})^2\right] \geq \text{Var}(Y) \geq 0
$$
In your case, you have reached the lower-bound $\text{Var}(Y)$, since you observe that $\hat{Y} = \mathbb{E}_{\mathcal{D}}[Y]$. This means that the bias (strictly speaking, this is not the correct definition of bias, but it gets the point across.) of $\hat{Y}$ is
$$
(Y - \hat{Y})^2 = (Y - \mathbb{E}_{\mathcal{D}}[Y])^2
$$
The variance of $\hat{Y}$ is
$$
\mathbb{E}_{\mathcal{D}}\left[\left(\hat{Y} - \mathbb{E}_{\mathcal{D}}\left[\hat{Y}\right]\right)^2\right] = \mathbb{E}_{\mathcal{D}}\left[\left(\mathbb{E}_{\mathcal{D}}[Y] - \mathbb{E}_{\mathcal{D}}[\mathbb{E}_{\mathcal{D}}[Y]]\right)^2\right] = 0
$$
Clearly, you have too much bias and too little variance.
So, how do we reach the lower-lower-bound of $0$? We need to increase the variance of $\hat{Y}$ by either adding more parameters to the network or adjusting the network architecture. As discussed in What should I do when my neural network doesn't learn? (highly recommended read), consider over-fitting and then testing your network on a single example by adding many more parameters or by adjusting the network architecture.
If the network no longer predicts the mean on a single example, then you can scale up slowly and start over-fitting and testing the network on two examples, then three examples, and so on. Otherwise, you need to keep adding more parameters/adjusting the network architecture until your network no longer predicts the mean.
Eventually, once you reach a dataset size of around 100 examples, you can start to split your data into training and testing to evaluate the generalization performance of your network. At this point, if it starts to predict the mean again, then make sure that the examples that you are adding to the dataset are similar to the examples that you already worked through in the smaller datasets. In other words, they are normalized and "look" similar. Also, keep in mind that as you add more data to the dataset, you will more likely need to add more parameters for better generalization performance.

Another helpful modification, but not as helpful as what I stated above, that helps in practice, is to slightly adjust the mean squared-error loss function itself. If your mean squared-error loss function is
$$
\mathcal{L}(y,\hat{y}) = \frac{1}{N} \sum_{i=1}^N (y_i-\hat{y}_i)^2
$$
where $N$ is the dataset size, then consider using the following loss function instead:
$$
\mathcal{L}(y,\hat{y}) = \left[\frac{1}{N} \sum_{i=1}^N (y_i-\hat{y}_i)^2\right] + \alpha \cdot \left[\frac{1}{N} \sum_{i=1}^N (\log(y_i)-\log(\hat{y}_i))^2\right]
$$
Where $\alpha$ is a hyperparameter that can be tuned via trial and error. A starting value for $\alpha$ could be $\alpha=5$. The advantage of this loss function over the plain mean squared-error loss function is that the $\log(\cdot)$ function stretches small values in the interval $[0,1]$ away from each other, which means that small differences between $y$ and $\hat{y}$ are amplified, leading to larger gradients. I have personally found this modified loss function to be very helpful in practice.
For this to work well, it is recommended (but not necessary) that $y$ and $\hat{y}$ are each scaled to have values in the interval $[0,1]$. Also, since $\log(0)=-\infty$, and since it is likely that $y$ and $\hat{y}$ will have values very close to $0$, then it is recommended to add a small value $\epsilon$, such as $\epsilon=10^{-9}$, to $y$ and $\hat{y}$ in the loss function as follows:
$$
\mathcal{L}(y,\hat{y}) = \left[\frac{1}{N} \sum_{i=1}^N (y_i-\hat{y}_i)^2\right] + \alpha \cdot \left[\frac{1}{N} \sum_{i=1}^N (\log(y_i + \epsilon)-\log(\hat{y}_i + \epsilon))^2\right]
$$
This loss function may be thought of as the Mean Squared Log-scaled Error Loss.
Answer: It means that your model decides to "do nothing" rather than prediction. And you should know that "doing nothing" will get a lower error than a random guess for the regression.
For example, if the true values are in the range [-10 10], then keep predicting 0 will return you about 2.5 mean absolute error; In contrast, if you try to guess values in the same range, you will get about 3.3 mean absolute error.
Therefore, if the model finds itself doesn't have the ability to make predictions better than random guesses, or maybe only a little bit better than random guesses (say MAE=2.8), then it will give up and choose to output zeros.
The problem may come from the model design (for example, capacity), or maybe just the data labels are too weak (for example, random labels).
Actually, this problem always appears in regression problems, as long as your data labels and/or models are not perfect, regardless of the data balance.
Answer: I was actually working on a very similar problem. Basically, I had a bunch of dots on a white background and I was training a NN to recognize the dot that was placed on the background first. The way I found to work was to just use one fully-connected layer of neurons (so a 1-layer NN). For example, for a 100x100 image, I would have 10,000 input neurons (the pixels) directly connected to 2 output neurons (the coordinates). In PyTorch, when I converted the pixel values to a tensor, it was normalizing my data automatically, by subtracting the mean and dividing by the standard deviation. In normal machine learning problems, this is fine, but not for an image where there might be a disparity in the number of colored pixels in an image (i.e. yours where there are only a few white pixels). So, I manually normalized by dividing all pixel intensity values by 255 (so they're now in the range of 0-1 without the typical normalization technique that tries to fit all the intensity values to a normal distribution). Then, I still had issues because it was predicting the average coordinate of the pixels in the training set. So, my solution was to set the learning rate very high, which goes against almost all ML instructors and tutorials. Instead of using 1e-3, 1e-4, 1e-5, like most people say, I was using a learning rate of 1 or 0.1 with stochastic gradient descent. This fixed my issues and my network finally learned to memorize my training set. It doesn't generalize to a testing set too well, but at least it somewhat works, which is a better solution than most everybody else suggested on your question.
Answer: Here is a more information-theoretic reason for why this is happening. Let $Y$ be the true and correct output that your network should return (the target), and let $\hat{Y}$ be the output that your network actually returns. The problem that you observe is that $\hat{Y}=K$, where $K$ is some constant, which means that $Y$ is independent of $\hat{Y}$.
If we compute the mutual information between $Y$ and $\hat{Y}$, we observe that:
$$
I(Y;\hat{Y}) = h(Y) - h(Y|\hat{Y}) = h(Y) - h(Y) = 0
$$
Where $h(\cdot)$ is the differential entropy and $h(Y|\hat{Y})=h(Y)$ since $Y$ and $\hat{Y}$ are independent. In other words, if we assume that $Y \rightarrow \hat{Y}$ is a communications channel, then there is no information flowing between the input of the channel $Y$ and the output of the channel $\hat{Y}$!
A more accurate channel would be the Markov chain $Y \rightarrow X \rightarrow \hat{Y}$, where $X$ is the input to the network. However, by the data processing inequality:
$$
I(Y;\hat{Y}) \leq I(Y;X) \\
I(Y;\hat{Y}) \leq I(X;\hat{Y})
$$
Which are just upper-bounds on the mutual information $I(Y;\hat{Y})$. Our objective is to increase the mutual information $I(Y;\hat{Y})$, which can be done by increasing a lower-bound on $I(Y;\hat{Y})$. So, we are not interested in $I(Y;X)$ and $I(X;\hat{Y})$ at this time.
Now let the mean-squared error between $Y$ and $\hat{Y}$ be:
$$
\mathbb{E}\left[(Y-\hat{Y})^2\right] = \delta
$$
Now, to find a lower-bound on $I(Y;\hat{Y})$, note that:
$$
\begin{align}
I(Y;\hat{Y})
&= h(Y) - h(Y|\hat{Y}) \\
&= h(Y) - h(Y-\hat{Y}|\hat{Y}) \tag{1} \\
&\geq h(Y) - h(Y-\hat{Y}) \tag{2} \\
&\geq h(Y) - \frac{1}{2} \log{(2\pi e \delta)} \tag{3}
\end{align}
$$
Where:

$(1)$ is because differential entropy is translation invariant.
$(2)$ is because conditioning reduces entropy.
$(3)$ is because random variable $Z=Y-\hat{Y}$ has variance $\mathbb{E}[(Z-\mathbb{E}[Z])^2]=\mathbb{E}\left[(Y-\hat{Y})^2\right] = \delta$, and so this is the Gaussian bound on differential entropy.

So:
$$
I(Y;\hat{Y}) \geq h(Y) - \frac{1}{2} \log{(2\pi e \delta)}
$$
If we further assume that the target $Y$ has mean $\mathbb{E}[Y]=0$ and variance $\text{Var}(Y)=\sigma^2$, then we get the tighter lower-bound:
$$
I(Y;\hat{Y}) \geq \frac{1}{2} \log{(2\pi e \sigma^2)} - \frac{1}{2} \log{(2\pi e \delta)} \geq h(Y) - \frac{1}{2} \log{(2\pi e \delta)}
$$
And so we can increase the mutual information by either increasing the variance $\sigma^2$ of $Y$ or decreasing the mean squared-error $\delta$. Unfortunately, we do not usually have the ability to adjust the variance of $Y$, so we would need to resort to decreasing $\delta$.
Interestingly, we have just shown that minimizing the mean squared-error $\delta$ corresponds to maximizing the mutual information between $Y$ and $\hat{Y}$, and vice-versa. At first glance, it seems like there is a contradiction here: you clearly can achieve small mean squared-error values, which means that the mutual information between $Y$ and $\hat{Y}$ is maximized. However, what you practically observe is that the mutual information between $Y$ and $\hat{Y}$ is $0$.
To resolve this contradiction, it is helpful to take a closer look at the mean squared-error between $Y$ and $\hat{Y}$. We know that:
$$
\mathbb{E}\left[(Y-\hat{Y})^2\right] = \text{Var}(\hat{Y}) + \text{Bias}(\hat{Y},Y)^2
$$
Since both of these terms are non-negative, then the variance and bias of $\hat{Y}$ each individually form lower-bounds for the mean squared-error:
$$
\mathbb{E}\left[(Y-\hat{Y})^2\right] \geq \text{Var}(\hat{Y}) \\
\mathbb{E}\left[(Y-\hat{Y})^2\right] \geq \text{Bias}(\hat{Y},Y)^2
$$
And therefore the true lower-bounds on the mutual information are:
$$
I(Y;\hat{Y}) \geq h(Y) - \frac{1}{2} \log{(2\pi e \cdot \text{Var}(\hat{Y}))} \geq h(Y) - \frac{1}{2} \log{(2\pi e \delta)} \\
I(Y;\hat{Y}) \geq h(Y) - \frac{1}{2} \log{(2\pi e \cdot \text{Bias}(\hat{Y},Y)^2)} \geq h(Y) - \frac{1}{2} \log{(2\pi e \delta)}
$$
These new lower-bounds indicate that although your mean squared-error values were small, they were not small enough, as they are upper-bounded by the variance and bias of $\hat{Y}$. Therefore, what we really want to do to maximize the mutual information between $Y$ and $\hat{Y}$ is to decrease the variance and bias of $\hat{Y}$. Unfortunately, we know that there is a trade-off between the variance and bias of $\hat{Y}$.
Answer: I am facing the same problem with my data set. It turns out that in my case the predictors are highly concentrated with a very small variance. You should check out the variance of your prediction variables and see how it is distributed.
However, some transformations on the output variable can be performed to modify or change its scale. This might result in a more uniform type distribution. For example, in image recognition tasks histogram equalization or contrast enhancement sometimes works in the favor of correct decision making.
Answer: It looks like a typical overfitting problem. Your data does not provide enough information to get the better result. You choose the complex NN with you train to remember all nuances of the train data. Loss can never be a zero, as it is on your graph. BTW It seems your validation has a bug or validation set is not a good for validation because the validation loss is also getting zero.
Comment: The question says the network almost always outputs zero. That would be a case of severe *underfitting*, not overfitting. There's also no gap between training and validation error on the learning curve, indicating that overfitting isn't the problem (the error isn't zero, the scale is logarithmic)


IIS 7 or ASP.NET security

Question: I'm building a custom ASP.NET 4 blog app. For the admin directory I could use ASP.NET forms authentication, but since I will be the only user I'm thinking of just using IIS 7 directory security to prompt for user name and password.  Is there any reason not to do this?  Is IIS 7 security good?
Answer: If you're the only user, it's your own machine, and you will be using NTLM then great, I'd favour it over forms any day.
I'd favour digest over forms too.
I'd favour forms over basic, but only without SSL. With SSL, I'd favour basic.
As well as preferring their security, they also don't force you to turn on sessions.

IIS 7 or ASP.NET security


Bigger almost-linear zone for tanh(x)

Question: The hyperbolic tangent is almost linear around the origin.
I'm looking for a way to modify the tanh function in order to have a bigger almost-linear zone while keeping the asymptote to y = 1.
In other words, the function should 'leave' later the y = x straight line and approach sooner the y = 1 straight line, coming closer to the (1, 1) point. (think about "sooner" and "later" as if x was a time variable).
I'd like this 'linearity control' to be a parameter.
If useful, feel free to ignore what happens for x < 0.
Thanks for any suggestion!
Comment: Something like $(\tanh x^n)^{\frac 1n}$ ?
Comment: Welcome to MSE. For some basic information about writing mathematics at this site see, *e.g.*, [basic help on mathjax notation](/help/notation), [mathjax tutorial and quick reference](//math.meta.stackexchange.com/q/5020), [main meta site math tutorial](//meta.stackexchange.com/a/70559) and [equation editing how-to](//math.meta.stackexchange.com/q/1773).
Answer: One function that works is $\sqrt[a]{tanh(x^a)}$ for a real parameter $a$. The higher the value of $a$, the wider the "linear-looking" area around $x=0$. Unfortunately for negative values of $x$ it is required that $a$ is a rational number in lowest terms, with both an odd numerator and denominator (that is, if you wish to have outputs in the real numbers).
Animation:
Comment: $a$ can be anything if you write $\text{sgn}(x) \sqrt[a]{\tanh \left(\left(x^2\right)^{a/2}\right)}$
Comment: That's perfect! Thanks a lot!
Answer: You need to increase the value of $a$ in $\tanh(ax)$.
Let $f(x) = \tanh(2x)$. This squishes $\tanh(ax)$ by a factor of $2$. Since $\tanh x \approx 1$ for $x < -1$ and $x > 1$, then the change in the $y$ values won't be too noticeable.
Here is a Desmos graph of the function.

Bigger almost-linear zone for tanh(x)