Optimizing regression coefficients to predict the largest outcomes

Question: What is a sound methodology to improve the efficiency of the regression coefficients when we are interested in predicting the larger values of the marginal distribution (tails)?
For example, we want to predict seismic waves based on a number of covariates recorded by our probes. Data can not be assumed to be strictly linear, given that most earthquakes develop abruptly after a certain covariates threshold is reached. The vast majority of observations are not considered harmful and should be down-weighted in our analysis. What we are really interested in estimating are the more extreme outcomes.
[thoughts...]
Weighted least square comes to mind, but how should the weights be calculated?
Is quantile regression with, say, $\tau = [0.2, 0,8]$ a better approach?
[/thoughts]
Comment: I would second quantile regression, but be careful to not go into the very extreme quantiles. If the larger values are qualitatively different, my suggestion would be to use a finite mixture or latent class model.
Comment: @FgNu Thanks. Why the warning on extreme quantiles? Which $\tau$ levels would you consider extreme?
Comment: Well, the mathematical principle is easy to state - assume that you are trying to estimate the $100p$th quantile, where $p\in(0, 1)$, and define $k_n =  \lfloor np_n \rfloor$ to be the $k$th sample order statistic ($n$ is the sample size). Then, the sample order statistic is called _extreme_ if $k_n - n = o(1)$.

In practice, I would be wary of modelling any above the 90th quantile with quantile regression for moderate sample sizes.
Answer: It seems that you either need to relax linearity assumptions for covariates (e.g., use restricted cubic splines, i.e., natural splines) or to define a new optimality criterion (other than maximum likelihood) and then to optimize the fit to that.    I'm glad you clarified that you don't want to discard observations.  The optimality criterion will define the effective weights; you don't have to.
Least squares will give automatic emphasis to fitting more extreme $Y$ values at the expense of higher mean absolute error or median absolute error.
Comment: Correct, but the "far out" points are effectively given much more weight because of minimizing the sum of squared residuals rather than, for example, the sum of absolute residuals.  With heavy-tailed $Y$ you will find accuracy for predicting in the middle sometimes sacrificed so that major errors are minimized.
Comment: Doesn't least square minimizes the overall sum of squared residuals? Let's say I have 1,000 observations < 2 and only 1 observation at 10: wouldn't least square optimize the coefficients to minimize the errors of the 1,000 points, rather than the single error of the larger point at 10?
Comment: Got it. Another subtle difference to quantile regression where, if I'm not mistaken, most regressors minimize the sum of absolute errors, rather than squared.
Comment: Also, should I choose to use a robust MM estimator I would loose the natural least square adjustment towards the larger observations, right?
Comment: @Frank Harrell, are there methods that deliberately weight or scale Y for heavy-tail estimation ?
Comment: There is a huge literature on robust regression models (think trimmed means and minimizing sum of absolute residuals or median absolute residual).  But if you correctly specify the conditional distribution of $Y$, weighting is automatic during maximum likelihood estimation.
Answer: What you have written suggests that logistic regression, with attention to diagnostics and residuals, will be your best bet.  It can help you take into account nonlinearity in the relationship between predictor and outcome  You'll want to test multiplicative effects (interactions), as the thresholds you talk about may be joint thresholds involving multiple predictors.  I am concerned, though, by your statement that "The vast majority of observations are not considered harmful and will be discarded from our analysis."  In order to explain what causes the events of interest, it is absolutely necessary to know something about the conditions that do not produce it, just as it is to know about the ones that do.
Comment: @Pardis You're right, that's what I had in mind when I posted the question: if robust regressors are used to down-weight observations as the residuals grow larger, why wouldn't there be a technique to down-weight the smaller to null observations. As Frank observed in his answer, least square naturally overweights the larger outcomes. But I'm still a bit puzzled on why I haven't found this problem stated more often. It seems to me that focusing on large outcomes is a common problem, like in the example I described.
Comment: Here we're not after what causes the event, but the outcome of a similar rare event. In this setting, aren't the majority of observations considered 'outliers'?
Comment: What I meant by "discarded" is actually down-weighted. In robust regression, the observations showing larger *residuals* are given lower weight to improve the efficiency of the coefficients. Here the focus is on *large outcomes*, so I was thinking to apply the same methodology (the larger the outcome, the larger the weight) to improve the accuracy of the coefficients.
Comment: @rolando2 I think the comment above makes what I was asking clear. In robust regression, we don't consider outliers to be "conditions that do not produce the desired outcome" but observations we'll rather have removed in a pre-processing step. What I was asking was that in this setting, should the observations with small outcomes be considered outliers?


How see test reports after maven build in Intelij Idea?

Question: I have run maven goal from Intellij Idea, build was been completed, tests were passed. How to browse with Intellij Idea all test reports which are stored in taget folders?
Answer: There is maven test support plugin for Intellij Idea. Just select maven subproject folder in project tree, press left mouse button and select "Show test results". :)


Correcting sample bias

Question: I am studying the correlation of two observed variables (call them $A$ and $B$). the underlying distribution for $A$ is symmetric around $0$ (for sure), however in my sample I have $411$ observations where it is negative and $470$ where it is positive due to sample bias.
Is it a correct way to remove the bias to simply filter out $470-411 = 59$ random positive observations from variable $A$ and the corresponding observations of variable $B$ before carrying on my analysis? This would rebalance the distribution of observations for $A$.
Ok, let's take an absurd example to make my point. Suppose you have a process following a perfect normal distribution centered at 0. A student draws 10000 samples one at the time and writes down the result. However, for reasons known only to her, she likes to write all the positive results on sheets of red paper and all the negatives on sheets of violet paper. If she can write 100 numbers on each sheet, at the end she will have roughly 50 red sheets and 50 violet sheets. She takes back all the sheet to her office desk to carry on the analysis but something terrible happens: a mutant paper-eating creature attacks her and eats only 23 of her violet sheets (the creature despises red paper). Now she has still around 50 red sheets, ie. around 5000 positive numbers, but only around 27 violet sheets, ie. 2700 negative numbers. She cannot repeat the experiment. What can she do? Can she choose randomly 23 red sheets and throw them away to rebalance the samples?
Comment: Are you sure that this is caused by sampling bias? If so, do you know anything about the confounding factor that causes it? The reason that I'm asking is that the inbalance between positive and negative values isn't all that big. If there was no bias, you would get inbalance at least this extreme about 5 % of the time...
Comment: You would need to have more information about the sampling bias first. For example, are positive numbers favored over negatives, or larger numbers are favored over lower ones? Both would produce an apparent excess of positives, but the adjustment would be different.
Answer: I will demonstrate that this effort at bias removal can, at least under certain (realistic) circumstances, introduce substantial bias.  The demonstration is done with a simulation.

The answer depends on how the sample was obtained, the structure of the data, and what analyses will be performed.  To make progress and to put this question into the usual framework, let's assume all $470+411$ observations are independently sampled from a common population (an iid sample).
First of all, the discrepancy between $411$ and $470$ could easily be due to chance.  For variable $A$, being positive is an event with a $50%$ chance, whence (due to the independence of the observations) its count follows a binomial distribution with parameters $411+470$ and $0.5$.  The chance of observing $470$ or more values of $A$ having a common sign is 5.06%: not terribly unlikely and not sufficient evidence to conclude the randomization failed.  Ergo, there is no sampling bias.
However, isn't it intuitively obvious that the proposal to remove the excess data randomly in order to equalize the numbers of positive and negative $A$'s will not bias the sample?  Although it makes sense, unfortunately it's not true.
As an example of what can go wrong, imagine yourself in the position of a consultant who recommends sampling procedures to scientists.  The one being recommended here seems to be the following:

Randomly and independently draw $411+470$ individuals from the population.
If there is an excess of positive values of $A$ in the sample, randomly remove any individuals with positive values of $A$ in order to make the counts of positive and negative values equal.  If there is an excess of negative values, follow a similar procedure to remove them.
Perform all subsequent analyses with the remaining data as if they formed an iid sample of the population.

Step 3 is a mistake.  The resulting sample is not iid.  Often it behaves very much like an iid sample, but not always.  Suppose, for instance, that $B$ is a measurement of an (unobserved) attribute closely correlated with $A$: in fact, the unobserved attribute is believed to be a multiple of $A$ plus some unknown constant.  However, the measurement is so crude that all it can indicate is whether that variable is positive or negative: this is what $B$ tells us.  It is desired to test whether this underlying attribute has zero mean.  The test will be based on the difference between the number of positive $B$'s and the number of negative $B$'s.  (This is about the best possible test under the circumstances.)
Hypothesis tests rely on computing the expectation of a test statistic and the amount by which the statistic may vary: its sampling variance.  If, for instance, $A$ has a normal distribution (of zero mean of course), then for an iid sample of size $411+470$, $B$ should range between $-100$ and $100$ and about $50%$ of the time it should lie between $-19$ and $19$.  However, applying the "bias removal" procedure changes this.  The reason is evident: in most random samples there will be an imbalance in the numbers of positive and negative values of $A$ by chance alone. If $B$ is strongly correlated with $A$, then it, too, will tend to have a similar imbalance.  The "bias removal" process of balancing $A$ thereby approximately balances $B$.  This causes $B$ to vary much less than one would expect.
Because this is a subtle phenomenon, detailed study is warranted.  Interested readers might want to experiment with a simulation.  Here is some R code.  First, a function to "remove the bias":
filter <- function(x,y) {
  #
  # Randomly remove some (x,y) values in order to
  # balance the numbers of positive and negative x's.
  #
  plus <- x > 0
  n <- length(x)
  n.plus <- length(x[plus])
  n.minus <- n - n.plus
  if (n.plus > n.minus) {
    omit <- sample(which(plus), n.plus-n.minus)
  }
  else {
    omit <- sample(which(x <= 0), n.minus-n.plus)
  }
  if (length(omit) > 0) {
    cbind(x,y)[-omit,]
  } else cbind(x,y)
}

Next, a function to convert values into a binary indicator of their sign so we can simulate $B$ exactly as described:
ind <- function(x) {
  y <- x*0
  y[x > 0] <- 1
  y
}

Now we can implement one iteration of the sampling process.  The last argument, control, determines whether the "bias removal" is not or is applied.
trial <- function(n, rho, control=FALSE) {
  x <- rnorm(n)                            # `A` has a normal distribution
  y <- rnorm(n) * sqrt(1-rho^2) + rho * x  # Correlation coefficient is `rho`
  if (control) cbind(x,ind(y)) else filter(x,ind(y))
}

The statistical analysis tracks the net number of positive values of $B$:
stat <- function(y) {
  2 * length(which(y>0)) - length(y)
}

Finally, we can replicate the sampling many times to understand what will happen in the long run:
rho <- 0.9       # Correlation between `A` and the variable underlying `B`
n.trials <- 5000
sample.size <- 470+411
set.seed(17)
sim <- replicate(n.trials, apply(trial(sample.size, rho), 2, stat))
sim.control <- replicate(n.trials, apply(trial(sample.size, rho, control=TRUE), 2, stat))

A plot of the output clearly shows how the test statistic for $B$ varies much less than expected:
plot(sort(sim.control[2,]+rnorm(n.trials, sd=0.25)), sort(sim[2,]+rnorm(n.trials, sd=0.25)),
     xlab="Control", ylab="Filtered")
abline(a=0, b=1, lwd=2, col="Gray", lty=2)

(The points were jittered slightly to resolve the thousands of overlaps: $B$ can have only integral values, after all.)
In this Q-Q plot, the control (on the horizontal coordinate) varies as expected: it's centered at 0 and stays generally between $-100$ and $100$. The statistic from the "filtered" dataset, using "bias removal," varies much less: only between about $-65$ and $65$.  This is shown by the sharp departure between the simulated data (black circles) and the line $y=x$ (dashed gray).
In short, this effort to remove bias actually can create bias.
For some analyses, the bias introduced by this "bias removal" procedure will be small and wholly undetectable.  Seeing the bias requires that $B$ and $A$ be correlated, probably strongly; and certain statistics will be more sensitive to it than others.  But if nothing else, this simulation demonstrates that the "bias removal" procedure can create erroneous results.
A theoretical analysis of this effect would be difficult to carry out in most cases, I suspect.  Simulations like this one can help determine the extent to which "bias removal" actually adds bias.  But why go to all that effort when the sample is perfectly fine to begin with?
Comment: Would a random removal process work better if finer groupings were made?   For example, rather than just positive and negative, we could group the observations into 10 groups that are each expected to contain equal proportions of data, say based on using a cumulative distribution function.  Then if a grouping is over weighted, it could be reduced in size randomly.
Comment: The main take-away message here, @Joel, is that *ad hoc* efforts to "improve" random samples create subtle, but sometimes important, errors. At the very least they call into question almost any method of analysis based on the assumption of random sampling, which includes all hypothesis tests, confidence intervals, prediction limits, and so on.
Comment: There seem to be conflicting goals here. The sample is clearly not representative, even though it was selected randomly.  What is the lesser evil: correcting for the known bias or risking the unknown bias?
Comment: @Joel You are implicitly raising interesting questions about what "bias" and ["representative"](http://stats.oecd.org/glossary/detail.asp?ID=3831) mean. It's impossible to address them in the space allowed for a comment, so I will limit my reply. First, how do you know "the sample is clearly not representative"? Second, please consider that except when a very small number of variables are involved, it's generally impossible for any non-census sample to be fully representative of a population. Third, you may be thinking of procedures such as stratified sampling.
Comment: Thanks for the in depth analysis. I get the point that 470 vs 411 samples is not enough of an imbalance to worry about as it is within reasonable probabilty of a proper random sampling. Let's make it more extreme. Say that -for whatever reason- I have 1400 positive data points and 410 negatives. I am still certain that $A$ is in itself symmetric (please don't question the assumption, it's not the point here). What would you do?
Comment: I would conclude, with certainty, that the data are not from a simple random sample. Until further information about how the sample was collected becomes available, I would advise people not to trust *any* inferences made from the sample under the assumption that it is random. Depending on how it was collected, inferences might be possible (based typically on procedures for weighting the data).
Comment: See my edit in the answer

Correcting sample bias


D3/React - Skip blanks in scatterplot not working

Question: I have a dataset with blank cells that I would like not to be drawn in my linechart with scatter dots. For the line I did manage to skip blanks, but for the line I can't get it to work.
My data looks like this:
DATE,VALUE
5/31/2022,12
6/1/2022,23
6/2/2022,24
6/3/2022,26
6/4/2022,32
6/5/2022,32
6/6/2022,35
6/6/2022,32
6/7/2022,31
6/8/2022,30
6/8/2022,21
6/10/2022,23
6/11/2022,13
6/12/2022,13
6/13/2022,14
6/14/2022,
6/15/2022,
6/16/2022,
6/17/2022,
6/18/2022,
6/19/2022,
6/20/2022,
6/21/2022,
6/22/2022,
6/23/2022,
6/24/2022,
6/25/2022,
6/26/2022,
6/27/2022,

I skip the blanks in the line like this:
const linevalue = d3.line()
  .defined(function(d) { return d.value; })
  .x(function(d) { return x(d.date) })
  .y(function(d) { return y(d.value) })

And try to do it in the scatter / dot part like this:
g.append("svg")
.selectAll("dotDemand")
.data(data)
.enter()
.append("circle")
.defined(function(d) { return d.value; })  // <<<< Here I try to do it in the same way
.attr("cy", function(d) { return y(d.value) } )
.attr("cx", function(d) { return x(d.date) } )
.attr("r", 5)
.attr("fill", "#7EE0FA")

But it doesn't work, this gives an error:
Uncaught TypeError: g.append(...).selectAll(...).data(...).enter(...).append(...).defined is not a function

I'm brand new to both React as D3 and I think I might also mess up dome things in the g.append("svg") parts but I can't figure it out how it should work.
I will upload the full code for the chart in a moment, stackoverflow is having issues at the moment with adding a lot multiple lines with code (saying to dent then indent, the code even if I try to add it as a javascript snippet)
import * as d3 from 'd3';
import {useEffect,useState} from 'react';
import csvFile from'../../data/TestData.csv';

const BasicLinechart=(props)=>{

const [data,setData]=useState([]);
console.log(csvFile)

const {width,height} =props;

useEffect(()=>{
if(data.length>0){
drawChart();
}else{
getCSVData();
}
},[data]);

const getCSVData=async()=>{
const tempData=[];

awaitd3.csv(
csvFile,
function(d){
tempData.push({
date:d3.timeParse("%m/%d/%Y")(d.DATE),
month:d3.timeParse("%m/%d/%Y")(d.DATE),
value:Number(d.VALUE),
});
}
);
setData(tempData);
console.log("1");

console.log(data.date)
};

const drawChart=()=>{

const parseDate=d3.timeParse("%m/%d/%Y"),
formatDay=d3.timeFormat("%d"),
formatMonth=d3.timeFormat("%b");

var div=d3.select("body").append("div")
.attr("class","tooltip")
.style("opacity",0);

//createthechartarea
const svg=d3.select('.svg-canvas')
svg.selectAll("*").remove();
var margin={top:20,left:30};

//AddXaxis-->itisadateformat
var x=d3.scaleTime()
.domain(d3.extent(data,function(d){returnd.date;}))
.range([0,width])

var x2=d3.scaleTime()
.domain(d3.extent(data,function(d){returnd.date;}))
.range([0,width]);

var g=svg.append('g')
.attr('transform','translate('+margin.left+','+margin.top+')')
.attr("class","legendOrdinal");

var xAxisDay=d3.axisBottom()
.scale(x)
.ticks(d3.timeDay.every(1))
.tickFormat(formatDay);

g.append("g")
.attr("transform","translate(0,"+height+")")
.attr("stroke","#595959")
.attr("font-family","cursive")
.call(xAxisDay)
.selectAll("text")
.style("font-size","12px")

g.selectAll("path")
.style("stroke","#595959")

g.append("g")
.select(".domain")
.attr("stroke","#595959")
.attr("stroke-width","6")
.attr("opacity",".6");

var xAxisMonth=d3.axisBottom()
.scale(x2)
.ticks(d3.timeMonth.every(1))
.tickFormat(formatMonth);

g.append("g")
.attr("transform","translate(0,"+(height+40)+")")
.call(xAxisMonth)
.selectAll("text")
.style("font-size","12px")
.attr("stroke","#595959");

//AddYaxis
var y=d3.scaleLinear()
.domain([0,d3.max(data,function(d){return+Math.max(d.value);})])
.range([height,10]);

g.append("g")
.call(d3.axisLeft(y))
.attr("stroke","#595959")
.selectAll("text")
.style("font-size","12px")
.attr("stroke","#595959");

//set lines coordinates
const linevalue=d3.line()
.defined(function(d){returnd.value;})
.x(function(d){returnx(d.date)})
.y(function(d){returny(d.value)})

//Addtheline&pointsforvalue
g.append("path")

.datum(data)
.attr("fill","none")
.attr("stroke","#7EE0FA")
.attr("stroke-width",2)
.attr("d",linevalue)

g.append("svg")
.selectAll("dotDemand")
.data(data)
.enter()
.append("circle")
//.defined("cy",function(d){returny(d.value);})
.attr("cy",function(d){returny(d.value)})

.attr("cx",function(d){returnx(d.date)})
.attr("r",5)
.attr("fill","#7EE0FA")
.on('mouseover',function(d,i){
d3.select(this).transition()
.duration('100')
.attr("r",10);
div.transition()
.duration('50')
.style("opacity",0);
div.html("$"+d3.format(".2f")(d.date))
.style("left",x(d.value)+10+"px")
.style("top",y(d.date)+"px");
})
.on('mouseout',function(d,i){
d3.select(this).transition()
.duration('200')
.attr("r",5);
//makesdivdisappear
div.transition()
.duration('200')
.style("opacity",0);
});
}

return(

<svgclassName="svg-canvas"width="800px"height="600px"/>

)
}

export default BasicLinechart;

Answer: Indeed the defined() function only exists on a d3 line generator, you cannot do that on a selection. Instead you can filter the data that you bind to the selection like this:
g.append("svg")
.selectAll("dotDemand")
.data(data.filter(d => d.value))
.enter()
.append("circle")
.attr("cy",function(d){return y(d.value)})
.attr("cx",function(d){return x(d.date)})
.attr("r",5)
.attr("fill","#7EE0FA")


Aurelia Binding Click Trigger in Nav Bar

Question: Given the following layout for app.html:
<template>
  <require from="nav-bar.html"></require>
  <require from="bootstrap/css/bootstrap.css"></require>

This is the sidebar.

    <router-view></router-view>

</template>

How do I bind to the toggleSidebar function (which is exported from app.js) in nav-bar.html?
<template bindable="router">

      ....

        ${router.title}

    ....

</template>

Currently I get the following error when I click on the toggleSidebar link:
"toggleSidebar is not a function".
Answer: In nav-bar.html, add another bindable property for the toggle method. Bindable properties are the way to share data/functions with a custom element.  This keeps custom elements encapsulated and portable.
<template bindable="router,toggleSidebar">

Then in app.html, bind the toggleSidebar method to the nav-bar element:

I think you may need to return true from your toggleSidebar method so that the default action (following the link) isn't canceled.
Comment: How is `toggleSidebar()` declared in `app.js`? This pattern isn't working for me. In my case, I have a table component within a view. The table has a `shuffle()` method that I need to call from the parent view.
Comment: That worked perfectly - thanks.  I'm just new to Aurelia, building my first app after being a long term Durandal user.


How to prove that $f(x,y) = xe^{−2x^2−y^2}$ has a global maximum and a global minimum

Question: We know that a function defined on a closed bounded domain must have a maximum and a minimum, But $f(x,y) = xe^{−2x^2−y^2}$ is defined on the domain $R^2$, which is not closed bounded, so how to show that it actually has a global max and a global min?
I am trying to prove that outside a disk with distance d, the absolute value of the function value can be as small as possible. That is, for all $\epsilon>0$, exist $d>0$ such that $||(x,y)-(0,0)||>d$ implies $||f(x,y)||<\epsilon$ But how may I  prove that? Am I on the right track?
Comment: Yes. This is indeed the correct way to approach this problem
Comment: Yes, you are on the right track. Notice that $|x|\leq\Vert (x,y)\Vert$, and $e^{-2x^2-y^2}\leq e^{-\Vert (x,y)\Vert^2}$.
Comment: Use the inequalities I mentioned, and the fact that the function $r\mapsto re^{-r^2}$ tends to $0$ as $r\to\infty$ to find exactly what you wanted to. You do not need an explicit control.
Comment: @zuggg Yes, but could you please extend your answer? I cannot find a proper way to control $xe^{-2x^2-y^2}$ so far.
Answer: Take $(x,y) \in R^2$ such that $r:=||(x,y)|| > 0$. Then we have $2x^2+y^2 \ge x^2+y^2=r^2$, thus $e^{-2x^2-y^2} \le e^{-r^2}$. This gives
$$|f(x,y)| \le \frac{r}{e^{r^2}}$$


I want to make a wrapped acces type for certain internals of one of classes and I have some performance questions

Question: I am writing an abstract matrix class (and some concrete subclasses) for use on very differing hardwares/architectures, etc. and I want to write a row and column type that provides a transparent reference to the rows and columns of the matrix.
However, I want to tune for performance, so I'd like this class to be essentially a compiler construct. In other words, I'm willing to sacrifice some dev time to making the overhead of these classes as small as possible.
I assume all (small) methods would want to be inline? Keep the structure small? Any other suggestions?
Comment: "I assume all (small) methods would want to be virtual?" - why would you assume that?
Comment: That sentence doesn't make any sense. Maybe he was thinking "inline"?
Comment: No inheritance, no RTTI, no virtual members, templates for every operation that could be done at compile time, headache pills.
Comment: I'm sorry, I meant to say inline... fixed!
Answer: Design for efficiency golden rule number 1 is to really properly fully completely totally understand and appreciate all of the nuances of the environment in which you are working.
Including, but not limited to:

Understanding the costs of each compiler construct you are using
Understand how the compiler uses memory
Do a proper design on the back of an envelope or napkin. If it doesn't fit you're in trouble.
Build small examples and test / disassemble
Revisit the design.

Then write some realistic test cases and test on the disparate hardware to identify if there are performance issues; I've done this a few times and unless you're writing in assembler the performance hit can be in the strangest places.
If it's too slow add hardware. If you can't add hardware optimise.
Answer: Don't do it. Write your code naturally. Make an elegant design and clear design. Make it work. Then, run it with a profiler and see if you have any performance holes.
Comment: Well the thing is that I want to pass these things around a lot. I need to be able to use rows and columns from arbitrary matrices and operate one them, once again without knowing knowing how the matrices are implemented.


Methods That Return Values Quiz

Question: I am super new to programming and I had a question on a quiz and the output answer was 36. I don't understand how that outcome came out of this code.
public class Method {

    public static int method(int number) {

        int result = 0;
        while ( number > 0) {
            result += number % 10;
            number = number / 10;
        }
        return result;
    }

    public static void main (String[] args) {
        System.out.println(method(9999));
    }
}

Comment: What was your answer, and how did you arrive at it?
Answer: Clearly within this code the part we need to look at is the while loop, this is the area of interest because it's where all of the calculation occurs.
while(number > 0){
    result += number % 10;
    number = number/10;
}

So the first line within the loop will add a value to the result:
result += number % 10;

The % operator in Java and many other languages can be thought of as a remainder of division, hence we are adding the remainder of division by 10 to the result.
9999 / 10 = 999 remainder 9.
So result has 9 added.
Then we call:
number = number / 10;

In Java when dividing an int we do not consider the remainder, so 9999/10 = 999.
And then we repeat. So essentially we are adding up the digits of the number.
9 + 9 + 9 + 9 = 36.
Answer: Even if you are new at programming, you could try to use an IDE like Eclipse.
Using an IDE you can use the breakpoints and follow the code line by line and the variable line by line. So you will understand what happens.
You could create a watch expression that will show on a tab the variable to you or use the inspect to do this.
A video: https://www.youtube.com/watch?v=drk_ldaRMaY

Methods That Return Values Quiz


Variável na consulta SQL

Question: Galera, estou tentando fazer uma consulta com uma variável que estou recebendo, porém não está funcionando. Alguma sugestão ?
$model = $_REQUEST['model'];
$customer = $_REQUEST['customer'];
SELECT * FROM sistema WHERE concluido <> 1 AND site='$site' AND $model='X'

Comment: Está muito confuso isso. Você não está a usar  o $customer, em vez disso está usando $site. Não entendi se a string sql esta assim mesmo e se você não sabe fazer consulta no banco ou se foi só de exemplo...
Comment: Espero que `$model` não seja o que parece...
Comment: Qual o erro? E estas variáveis estão vindas corretamente?
Comment: Tentou imprimir a consulta e executar direto no banco?
Answer: Elabore da seguinte forma:
$model = $_REQUEST['model'];
$customer = $_REQUEST['customer'];

$consulta = "SELECT * FROM sistema WHERE concluido <> 1 AND site = '{$site}' AND {$model} = 'X'";

Não esqueça de verificar se as variáveis estão vindas corretamente e se realmente o sinal de comparação é <>, talvez você posa usar = '1'.
Comment: Sensacional André, Obrigado amigo!!

Variável na consulta SQL


Time series analysis with known outliers

Question: Let's say I have a time series of site usage data that I would like to analyze. The data is for people visiting my site, with a known seasonality that I can easily calculate, but in addition, there are also events (say the day when a social media post gets a lot of shares, or I purchase ads) that are scheduled, but are not at any regular cadence. Is there a way to add this source of variability to my model in a way that allows me to make a better estimate of my baseline trend?
Answer: Well first test your data in an autocorrelation plot to double check that your popular social media post and ads are random over time and if it is random a Box-Jenkins model should still be able to account for the variability of your model and predict where your data will trend.
Comment: Say that the assumptions of randomness are met, can I use the model with known future events to generate a more accurate forecast?
Comment: Well generally the more data for your time-series the more accurate you can predict the future data point here is a good link on time-series and predictions. http://www.itl.nist.gov/div898/handbook/pmc/section4/pmc4.htm
Comment: Hope this site can help with any additional questions you have.
Comment: I'm actually fairly familiar with trends and seasonality in time series forecasting, but don't know what to do with events (such as increased site usage due to an upcoming event) that I know about ahead of time, but occurs randomly.


How to find complex numbers such that this list is linearly independent?

Question: Show that if we think of $\mathbb C$ as a vector space over $\mathbb C$, then the list $(1+i,1-i)$ is linearly dependent.
I need to find two complex numbers $a+bi$ and $c+di$, both not $0$ such that
$$(a+bi)(1+i) + (c+di)(1-i) = 0.$$
Now:
\begin{align*}
a+ai+bi-b+c-ci+di-d&=0
\\
(a+c)+i(a-c)+i(b+d)-(b+d)&=0
\\
(a+c)+i(a-c)&=(b+d)-i(b+d)
\end{align*}
I know the solution but I don't know what I need to do to get to it. Maybe I'm missing something really obvious. Any pointers or solutions is appreciated.
Comment: Let $a= 1+i$ and $b=1-i$.  Then note that $\frac{1}{a}\cdot a + \frac{-1}{b}\cdot b = 0$.
Comment: @bungo oh that's a good thought I forgot about it, so since (a+bi) spans c then any list > size 1 is dependant in C
Comment: You don't have to do any computation if you recognize that the dimension of $\mathbb C$ over $\mathbb C$ is $1$. This implies that any list of more than one "vector" must be linearly dependent.
Comment: Right, any nonzero $z \in \mathbb C$ spans $\mathbb C$, since any other $w \in \mathbb C$ can be written as a scalar multiple of $z$, namely $w = az$ where $a = z^{-1}w$.
Answer: When we multiply $1+i$ by $1-i$ we get the same number as when we multiply $1-i$ by $1+i$. Consequently,
$$
(1+i)(1-i) - (1-i)(1+i) = 0.
$$
Comparing to
$$
(1+i)(a+ib) + (1-i)(c+di) = 0
$$
we see that we can set $a=1$ and $b=c=d=-1$.
Answer: Let $\lambda=\frac{1-i}{1+i}=-i$. Then $\lambda(1+i)-(1-i)=0$, and therefore $1-i$ and $1+i$ are linearly dependent.
Answer: To solve $z_1(1 + i) + z_2(1-i) = 0$,
take $z_1 =$ any complex number such that
$z_1 \neq 0 \implies z_1(1 + i) \neq 0.$
Then, $z_2 = - \frac{z_1(1 + i)}{1-i}.$
Answer: Any field $K$ is $1$-dimensional when viewed as a $K$-vector space: let $x, y \in K$, then, if either of $x$ or $y$ is $0$, say $x$, then $1x + 0 y = 0$, otherwise they are both non-zero, and then $x^{-1}x + (-1)y^{-1}y = 1 - 1 = 0$. So two distinct elements of the field $\Bbb{C}$ do not form a linearly independent set.


Periodic solution existence using Poincaré-Bendixson

Question: Given the following system:
$\dot{x}=x-y-2x(x^2+y^2)$
$\dot{y}=x+y+xy-2y(x^2+y^2)$
Converting this to polar coordinates using $r\dot{r}=x\dot{x}+y\dot{y}$ and $\dot{\theta}=\frac{x\dot{y}-y\dot{x}}{r^2}$ I got:
$\dot{r}=r-2r^3+r^2\cos(\theta)\sin^2(\theta)$
and
$\dot{\theta}=1+r\cos^2(\theta)\sin(\theta)$
I'm trying to identify the regions where $r$ is expanding or shrinking by limiting the values of $\cos(\theta)\sin^2(\theta)$ I thought rewriting it to be $\cos(\theta)(1-\cos^2(\theta))=\cos(\theta)-\cos^3(\theta)$ would help seeing as how
$-0.4\leq \cos(\theta)-\cos^3(\theta)\leq 0.4$
But I don't know how to move forward from here. Most examples lead to a much simpler reduction of $\dot{r}$ am I doing something wrong here?
Answer: Indeed the maximum of $\cos(θ)\sin^2(θ)$ is $\frac2{3^{1.5}}=0.3849...<0.4$.
Now you know that the radius dynamic obeys the inequalities
$$
r(1-0.4r-2r^2)\le \dot r\le r(1+0.4r-2r^2)
$$
The lower bound is positive and thus the radius always increasing in time on the interval $0<r<\sqrt{0.51}-0.1=0.6141...$. The upper bound is negative and thus the radius always decreasing in time for $r>\sqrt{0.51}+0.1=0.8141...$. Thus the annulus $0.5\le r\le 1$ is a trapping region in forward time. As $\dot θ\ge 1-0.4=0.6$ on this annulus, there are no other equilibrium points in this region, thus implying the existence of a limit cycle.


java.lang.NoSuchMethodError: org.hibernate.cfg.Environment.verifyProperties(Ljava/util/Map;)V

Question: Trabalho em um projeto Java e ao mudar de máquina precisei informar novamente as bibliotecas do projeto, porém ao compilar esta gerando um erro na saída.
Saída:
<code>Exception in thread "main" java.lang.NoSuchMethodError: org.hibernate.cfg.Environment.verifyProperties(Ljava/util/Map;)V
at org.hibernate.boot.registry.StandardServiceRegistryBuilder.build(StandardServiceRegistryBuilder.java:245)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.buildServiceRegistry(EntityManagerFactoryBuilderImpl.java:1020)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:835)
at org.hibernate.jpa.HibernatePersistenceProvider.createEntityManagerFactory(HibernatePersistenceProvider.java:75)
at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:54)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:52)
</code>
Porém não sei onde procurar pelo problema.
Answer: eu já tive problemas semelhantes a este por usar versões diferentes de alguma biblioteca.
Confirma se as bibliotecas da máquina nova são as mesmas da antiga.
Comment: realmente alguns jar estavam na versão mais antiga.


Derivative of $f(x)=\int_{x}^{x^3} e^{t^2} dt$

Question: Find $f'(0)$ and $f'(\sqrt{3})$ when $f(x)=\int_{x}^{x^3} e^{t^2} dt$
I know that $\int_{}^{} e^{t^2} dt$ doesn't return elementary functions, but I am not sure how to solve this. Can anyone give a little hint?
Comment: reference: https://math.stackexchange.com/questions/2802614/how-to-find-the-derivative-of-this-function/2802626#2802626
Answer: Recall by the Fundamental Theorem of Calculus, $$\frac{d}{dx}\int_a^xB(t)dt=B(x),\space\text{for $a\in \mathbb{R}$}$$
So if we have $$I=\int_{g(x)}^{h(x)}B(t)\space dt=\int_{a}^{h(x)}B(t)\space dt+\int_{g(x)}^{a}B(t)\space dt=\int_{a}^{h(x)}B(t)\space dt-\int_{a}^{g(x)}B(t)\space dt$$
We want to differentiate $I$, namely $$\frac{dI}{dx}=\frac{d}{dx}\left(\int_{a}^{h(x)}B(t)\space dt-\int_{a}^{g(x)}B(t)\space dt
\right)=\underbrace{\frac{d}{dx}\left(\int_{a}^{h(x)}B(t)\space dt\right)}_{\frac{dI_1}{dx}}-\underbrace{\frac{d}{dx}\left(\int_{a}^{g(x)}B(t)\space dt\right)}_{\frac{dI_2}{dx}}$$
So $$\frac{dI}{dx}=\frac{dI_1}{dx}-\frac{dI_2}{dx}$$
Calculate separately:
$$\text{For $\frac{dI_1}{dx}$, let $u=h(x)$ so that $\frac{du}{dx}=\frac{dh}{dx}$}$$
Hence
$$\frac{dI_1}{dx}=\frac{d}{du}\left(\int_{h(a)}^u B(t)\space dt\right)\cdot\frac{du}{dx}=B(u)\cdot\frac{du}{dx}=B(h(x))\cdot h'(x)$$
And similarly, $$\text{For $\frac{dI_2}{dx}$, let $u=g(x)$ so that $\frac{du}{dx}=\frac{dg}{dx}$}$$
Hence $$\frac{dI_2}{dx}=\frac{d}{du}\left(\int_{g(a)}^u B(t)\space dt\right)\cdot\frac{du}{dx}=B(u)\cdot\frac{du}{dx}=B(g(x))\cdot g'(x)$$
$$$$
We conclude $$\frac{dI}{dx}=\frac{dI_1}{dx}-\frac{dI_2}{dx}=B(h(x))\cdot h'(x)-B(g(x))\cdot g'(x)$$ or that
$$$$

$$\frac{d}{dx}\left(f(x)\right)=\frac{d}{dx}\left(\int_{g(x)}^{h(x)}B(t)\space dt\right)=h'(x)B(h(x))-g'(x)B(g(x))$$

Note in your case,

$f(x)=\int_x^{x^3}e^{t^2}dt$
$h(x)=x^3$
$g(x)=x$
$B(t)=e^{t^2}$

So really, you're not trying to find an antiderivative of $y_1=e^{x^2}$, but a derivative of $y_2=\int_{g(x)}^{h(x)}e^{t^2}dt$.
Answer: Hint:
$\int_{u}^{v} f'(t)\mathrm dt=f(v)-f(u)$, we get that $\left(\int_{u}^{v}f'(t)\mathrm dt\right)'=f'(v)v'-f'(u)u'$. Here $u$ and $v$ are functions of $x$. Using this fact which is popularly called Leibniz Integral Rule, you can get the required result by setting $f(t)=e^{t^2}$, $u(x)=x^2$ and $v(x)=x^3$ and you'll be done in no time.


Is there any rate limit when using the whois command on ips?

Question: The data returned by whois ip-address appears to be very useful and more accurate than the data I get from the maxmind geoip databsae I am currently using.
Where does the data come from and are there any rate limits?
Of course I will cache information for an ip, but nevertheless I am expecting tens to hundrets of thousands of calls daily.
Please note that I am quering information on IP addresses, not domain names. Therefore the registrars are not involved. The data appears to come from RIPE and ARIN, however they do not write about any limits in their terms and there is also no information available whether the databases are somehow distributed in a DNS like fashion or come from a single server.
Answer: Yes, there is certainly a rate limit and it will be in the order of dozens or hundred calls a day, depending on the registrar providing the data for a given TLD.
The whois data and database has a completely different purpose and it certainly is not meant to be abused by people not wanting to pay for quality GeoIP database access.
Comment: As I pointed out I do not want to look up domains, I want to look up IPs. So there are no TLDs and no registrars involved. I am aware of the registrar limits.
Also I own licenses for the best database I could find (Maxmind). My problem is not the money, its the quality.
Comment: Read what `whois` data is and where it comes from. Also, start thinking: Do you honestly believe anyone would pay GeoIP providers a single cent if much better data would be available for free elsewhere?
Comment: The manpage of whois says that it comes from the ripe and arin databaes, however they do not write about any limits. Thats why i'm asking.
And I do not believe, I know. I am paying 25 USD every month for data that IS less accurate.


Mensagens de erro em português do SQLServer - (LocalDB)\v11.0

Question: Estou a desenvolver uma aplicação com o Entity Framework 6 utilizando a abordagem Code-First.
Quero configurar o SQLServer (LocalDB)\v11.0 para apresentar as mensagens de erro em português sem utilizar o SQL Server Management Studio.
Pela pesquisa que fiz, uma das possibilidades, seria utilizar o atributo Current Language no ConnectionString.
Foi o que tentei mas sem resultado.
<connectionStrings>
    <add name="AssociGestorDb"
        connectionString="Server=(LocalDB)\v11.0;
                          Database=AssociGestorDb;
                          Integrated Security=True;
                          Current Language=Portuguese;
                          Connect Timeout=30"
        providerName="System.Data.SqlClient" />
</connectionStrings>

Por outro lado se utilizar SQLExpress já funciona.
<connectionStrings>
    <add name="AssociGestorDb"
         providerName="System.Data.SqlClient"
         connectionString="Server=.\SQLExpress;
                           Trusted_Connection=true;
                           Current Language=Portuguese;
                           Database=AssociGestorDb"/>
</connectionStrings>

Qual é a modo correto para conseguir ter as mensagens de erro em português no (LocalDB)\v11.0?
Answer: Não sei como realizar a configuração do servidor, mas quando preciso saber o significado de uma mensagem de erro, executo o seguinte select:
select * from master.dbo.sysmessages
where msglangid = 1046
order by error

Onde msglangid 1046 faz referencia à mensagem no idioma português, e error é o código do erro que o servidor retorna.
Comment: Muito útil buscar as mensagens de erro por código do idioma! Estou tentando escrever uma biblioteca para extrair heuristicamente o máximo de informação dessas mensagens de erros de forma estruturada e comparar as mensagens nos diversos idiomas facilitou muito!


É possível fazer submit sem dar refresh, e sem jQuery?

Question: A minha pergunta é a seguinte:

É possível fazer submit de um formulário, sem ter que ir para o action desse formulário?

Ou seja, eu clico submit, fico na mesma página mas o formulário é validado, sem dar refresh.
Como posso fazer isso?
Comment: Sim, é possível. Você deve utilizar AJAX.
Comment: Apontar o action para a mesma página ?
Comment: Não, por exemplo, eu estou na página index.php, e o action="validar.php", eu clico submit e continuo na index.php sem ter que ir para a validar.php
Comment: E que tal ir para validar e lá colocar um `header('Location: index.php');` ou jogar o conteúdo de validar com um include e no action se deixa em branco ou aponta para si ?
Comment: Poderiam-me dar um exemplo?
Comment: Como é seu formulário, o que acontece entre o submit e o fim de validar ?
Comment: Bom é o seguinto, basicamente o que quero é clicar submit, sem mudar de pagina e o formulario ser enviado.
Comment: @Randrade tou em duvida, pq essa aqui não pede jquery.
Comment: @Bacco Eu pensei nisso. Porém, pede javascript. A outra também pede em javascript, mesmo não tendo resposta
Comment: Gonçalo, alguma das respostas resolveu o teu problema?
Answer: Para enviar o formulário sem fazer refresh tens de usar AJAX e parar o evento submit.
Parar o evento submit:
var form = document.querySelector('form');
form.addEventListener('submit', function(e){
    e.preventDefault(); // <--- isto pára o envio da form

Juntar os dados da form
Para enviar via ajax tens de saber o url para o qual queres enviar. Se for o mesmo do action do form podes fazer
var url = form.action;

Se não junta essa string. Depois juntas os dados da form:
var formData = new FormData(form);

e tudo junto, com o AJAX poderia ser assim:
Enviar via ajax:
var form = document.querySelector('form');
form.addEventListener('submit', function(e) {
  e.preventDefault(); // <--- isto pára o envio da form

  var url = this.action; // <--- o url que processa a form
  var formData = new FormData(this); // <--- os dados da form
  var ajax = new XMLHttpRequest();
  ajax.open("POST", url, true);
  ajax.onload = function() {
    if (ajax.status == 200) {
      var res = JSON.parse(ajax.responseText); // a resposta do servidor
      // fazer algo com a resposta do servidor
      alert(res);
    } else {
      alert('Algo falhou...');
    }
  };
  ajax.send(formData);
});

Exemplo: http://jsfiddle.net/t65u21xk/
Leitura na MDN: (link)
Answer: Este exemplo é puro javascript, a transação está sendo realizada através da API XMLHttpRequest e não pelo action convencional... Ou seja você nem precisaria da tag form com seus atributos.
O formulário:
A questão do onsubmit="return false;", é devido ao fato de que na ausência do mesmo, o usuário poderá submeter o form, teclando ENTER ou no caso de algum type="submit"...
<form  onsubmit="return false;">
    Entrada:<input type="text" id="ajaxTextbox" />
    <input type="button" id="ajaxButton" value="Enviar"/>

Dados Enviados :

</form>

O Ajax:
Nesse código eu estou colocando o retorno, para exemplificar que a página não será recarregada, como no seu caso é só o envio e não a recuperação aí você altera. Entretanto como o Ajax é muito usado tanto para envio como para recuperação, já deixei desse jeito. Por exemplo, no momento em que um novo usuário vá realizar o cadastro, pode verificar se o nome já existe e pedir para ele colocar outro sem recarregar a página.
Em resumo Ajax serve tanto para enviar como para recuperar dados...Ou os 2 como nesse caso do exemplo...
(
    function()
    {
        var httpRequest;
        document.getElementById("ajaxButton").onclick = function()
        {
            var entrada = document.getElementById("ajaxTextbox").value;
            requisita("validar.php", entrada);
        };
        function requisita(url, entrada)
        {
            httpRequest = new XMLHttpRequest();
            if(!httpRequest)
            {
                alert("Algo está errado verifique a compatibilidade do Navegador !");
                return false;
            }
            httpRequest.onreadystatechange = mostraconteudo;
            httpRequest.open("GET", "validar.php?entrada=" + entrada, true);
            httpRequest.send();
        }
        function mostraconteudo()
        {
            if(httpRequest.readyState === XMLHttpRequest.DONE)
            {
                if(httpRequest.status === 200)
                {
                    document.getElementById("envio").innerHTML = httpRequest.responseText;

                }
                else
                {
                    alert("Algum problema !");
                }
            }
        }
    }
)();

E o validar.php, onde você vai receber os dados... E colocar suas regras.
<?php
$name = $_GET["entrada"];
echo $name;
if(empty($_GET["entrada"]))
{
    $name = "Não Digitou !";
    echo $name;
}
?>

Espero ter ajudado...Qualquer coisa vai comentando...
Comment: Vou editar a minha pergunta.
Comment: Pronto a minha duvida, agora ta melhor explicada.
Comment: Ah sem dar `refresh`....
Comment: Bom é isso aí, embora fora marcada como duplicada sua questão, e a questão solicita Javascript e não Jquery, talvez agora que está editada e a dúvida está clara, vou solicitar que seja reaberta se assim os moderadores acharem proveitoso @bigown
Comment: A edição a que vc se refere não mudou a pergunta, só o termo usado. Agora eu editei para que fique realmente diferente, o "sem jQuery" no título talvez ajude ;) Já tinha pensado nisso ontem, quando deixei este comentário para o @Randrade: http://pt.stackoverflow.com/questions/163498/%C3%89-poss%C3%ADvel-fazer-submit-sem-dar-refresh#comment336302_163498


When can we choose a covering local basis at a point which does not contain the whole space?

Question: I was wondering if we can ensure by relatively mild seperation axioms that a local basis ,which is a cover of the space, for at a topology at a point $x_0\in X$ does not contain $X$?
I'm pretty sure that if $X$ is $T_1$ and has more, then $3$ points for any $y\in X$ distinct from $x_0$ we have an open neighbourhood of $x_0$ containing also $y$ which is not the whole space, by the following argument:
I need to find for all $y\in X$ an open neighbourhood $U_y$ such that $x_0,y\in U_y$ and $U_y\neq X$. Let $z\in X$ be a point distinct from both $x_0$ and $y$. Then by the Hausdorff property, there exists $U_{y,z}$ which contains $y$ but not $z$. Similarly we have a $U_{x_0,z}$ containing $x_0$ but not $z$. Then $U_y:=U_{y,z}\cup U_{x_0,z}$ is such a neighbourhood.
My question is first whether the argument seems valid, and second can one reduce the assumption on the seperation axiom to obtain the same argument? I would also like to know whether it seems that $T_0$ is sufficient for having a basis, not local, which does not contain $X$?
Edit:
I added an assumption on that local basis would also be a basis, since that's what I had in mind and it seems that my original question was simpler and not what I had intended to ask about.  I would appreciate if any one could adress my assumptions now.
Answer: A point p in a space S can have a local base without S
iff there is an open nhood of p that is not S.
The Serpenski space ({0,1}, {empty set, {0}, {0,1}})
is a T$_0$ space where the only local base for 1 is {{0,1}}.
Exercise.  Show if open U is nhood p, then
{ V : p in V, V open subset U } is a local base for p.
Comment: Yes @Keen-ameteur, that is so of a base but not a local base.  Yes a local base is a family of open sets.  As the exercise shows, if there is a non-trival nhood for p, then there is a local base for p without S.  I did not say just any non-trivial open set is a local base.
Comment: Perhaps I used the term incorrectly. I was trying to find a basis for the open sets containing a point $x_0$.
Comment: @Keen-ameteur.  A point by any other name is still a point.
Comment: Shouldn't a basis satisfy the property that each open set is the union of elements in the basis? In that sense I was thinking of a local basis, and then it needs more than just having a non-trivial neighbourhood of the point.
Answer: Let $(X,\mathcal{T})$ be a topological space. A trivial observation:

$p \in X$ has at least one open neighbourhood $O_p \neq X$ iff $p$ has a local neighbourhood base that does not contain $X$.

Proof:$\Rightarrow$: given $O_p$, define $\mathcal{B}_p=\{O \cap O_p: O \in \mathcal{T}, p \in O\}$ and this is a local base at $p$: if $O$ is open and contains $p$, then $p \in O \cap O_p \subseteq O$ and $O \cap O_p \in \mathcal{B}_p$ so the local base requirement is satisfied. And as $O \cap O_p \subseteq O_p \neq X$, no member of $\mathcal{B}_p$ can equal $X$ either.
$\Leftarrow$: trivial: take a local base element from the stated local base (there is at least one, for the open set $X$, e.g.)
And a $T_1$ space of at least two points clearly obeys the left hand premise: if $p \in X$, pick $q \neq p$ and let $O_p= X\setminus \{q\}$. (A space is $T_1$ iff all singleton sets are closed).
But in much weaker separated spaces we can do this: take $X=\{0,1,2,3\}$ with topology $\{\emptyset,X,\{0,1\},\{2,3\}\}$ which is not even $T_0$ but does satisfy the requirements above.


ListData.svc does not exist when connecting to web service

Question: I have recently been developing a suite of WCF/JSON web services. These all work fine when running in my development environment. In trying to get it running on the test environment I have been getting the following error.

The service '/_vti_bin/ListData.svc' does not exist.

I have added the (hosted) web service as a new service reference which appears to work fine and said web service does not connect to any other services, it simply performs data look ups using linq2sql.
This is the full event log message, when a request is made from a (local) WPF client:

WebHost failed to process a request.  Sender Information:
  System.ServiceModel.Activation.HostedHttpRequestAsyncResult/11454272
  Exception: System.Web.HttpException (0x80004005): The service
  '/_vti_bin/ListData.svc' does not exist. --->
  System.ServiceModel.EndpointNotFoundException: The service
  '/_vti_bin/ListData.svc' does not exist.    at
  System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String
  normalizedVirtualPath)    at
  System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String
  relativeVirtualPath)    at
  System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()
  at
  System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()
  at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
  at
  System.ServiceModel.Activation.HostedHttpRequestAsyncResult.End(IAsyncResult
  result)  Process Name: w3wp  Process ID: 3768
Answer: By enabling includeExceptionDetailInFaults I was managed to get a much more appropriate response, it was simply a database connection issue in the end.


What's the term for the intervals between notes within a scale that produces a chord

Question: I probably described that poorly in the title, and I'm not even sure there is a word for what I'm thinking.
Lets say you're playing on just the white keys on a keyboard. C E G is C major, and D F A is D minor, the next chord after that. The number of steps between the notes in each chord is different, but they're the same number of notes within the scale apart.
Is there a word that describes the number of notes apart within a scale like that?
Answer: Sure is!  These are called Intervals.  It has to do with the number of "Scale Degrees" that separate two notes - basically, the number of notes you have to move through the scale from the first note to reach the second note (including the note you started on).  These are worded "Second", "Third", "Fourth", "Fifth", "Sixth", "Seventh", "Octave" (Octave indicates an 8 note separation)
So, in your example:

C to E is a third (because you have to go from C (1) -> D (2) -> E
(3))
E to G would also be a third (E -> F -> G)
C to G is a fifth (C -> D -> E -> F -> G)

You get a bit more complicated if you consider more unusual notes.  For instance, C to Eb would be a minor third, because it's only three half steps.  A more exhaustive list of Interval names is located here, but these are the basics.
I hope this helps.  Disregard if I've misunderstood your question =)
EDIT:  Just to be more clear / complete here (in case wikipedia ever disappears ;)):

Number of half steps    Interval Name
0                       Perfect unison
1                       Minor second
2                       Major second
3                       Minor third
4                       Major third
5                       Perfect fourth
6                       Diminished fifth, or augmented fourth
7                       Perfect fifth
8                       Minor sixth
9                       Major sixth
10                      Minor seventh
11                      Major seventh
12                      Perfect octave
Comment: E -> G is a minor third.  A chord that spans a fifth (like all of the ones discussed so far) contains one minor third and one major third.  In CEG the major is first; in DFA it's second.  If you have two major thirds, your overall interval is a tritone, not a fifth.
Comment: I've updated the answer with a bit more detail, I should have included that in the first place!
Comment: @Monica +1 Good catch!  I can't believe I actually typed that E -> G was a major third haha.
Comment: @Monica: A tritone consists of three tones, which would be about two minor thirds (except for comma's). Two major thirds amounts to an augmented fifth or a minor sixth.
Comment: CEG and DFA both have the same interval then, despite being a major and a minor chord, is that correct?
Comment: You're on the track.  C -> E and D -> F are both "thirds", but (more specifically) C -> E is a *major* third, while D -> F is a *minor* third.  C -> G and D -> A are both *perfect* fifths.  E -> G and F -> A are both *major* thirds  Check out the table in the link, it gives some of those details
Comment: @Matthew good edit, I didn't consider the synonyms there.
Comment: @Raskolnikov , oops!  Now it's my turn to say: good catch!
Answer: In addition to jadarnel27's excellent answer, I think it's worth discussing diatonic intervals. A diatonic internal is one that is composed entirely of notes in a scale.
For example, in the key of C, a C major chord is made of the notes C E G. The interval between C and E is a major this. A d minor chord is spelled D F A and has a minor third between the D and the F. However, in the key of C, both intervals can be described as diatonic thirds because they are thirds in the C major scale, even though they are of different qualities.
Hence, most typical chords in a typical scale or mode can be said to be made of a diatonic third and a diatonic fifth above the chord's root.
The word "diatonic" can be used to describe other musical ideas when you're only interested in the notes in a particular key. For example, let's say you have a motif in the key of C that you want to transpose to the key of A. You can do this by transposing up a major sixth, moving all notes the same number of semitones. However, this will lead to some notes outside of the C major scale. For instance, an E would be transposed to a C#.
You could also transpose by a diatonic sixth. In this case, all notes are moved up six scale degrees in the key of C. The precise interval each note is moved will vary as a result. C will be transposed a major sixth to A and E will be transposed a minor sixth to C, but they will all stay in the key of C.
Comment: Here is a Wikipedia article about diatonic sounds: http://en.m.wikipedia.org/wiki/Diatonic_and_chromatic
Answer: If you are looking for a specific, standard term, I think it is "interval class."
This would be the more general name of the interval, like a "third", as compared to the specific minor third or major third.
https://en.wikipedia.org/wiki/Interval_class
Mike
Answer: Triads are the specific answer to your question. Triads are stacked thirds and are described with major, minor, diminished, augmented.
C E G is a major triad
D F A is a minor Triad
C E G# is a augmented triad
C Eb Gb is a diminished triad.
Comment: C E G# is a augmented triad
C Eb Gb is a diminished triad

What is C E Gb then?


Calcular o total de uma coluna table com base no checkbox pegando apenas os selecionados

Question: Tenho a seguinte tabela:

    <form class="form-inline" action="registrosVazios.php" method="GET">

              <input type="checkbox" name="chkAll" onClick="checkAll(this)" />

              DATA

              PDV

              CUPOM

              VALOR

          <?php
          if (mysqli_num_rows($resultImportaPdv) > 0) {
            while($row = mysqli_fetch_assoc($resultImportaPdv)) { ?>

                <input type='checkbox' id='<?php echo $row["ie_cupom"]; ?>' name='<?php echo $row["ie_cupom"]; ?>' value='<?php echo $row["ie_cupom"]; ?>' onClick="verCheck()" >

                <?php echo date_format (new DateTime($row["ie_data"]), 'd/m/Y'); ?>

                <?php echo $row["ie_pdv"]; ?>

                <?php echo $row["ie_cupom"]; ?>

                <?php echo round($row["ie_valor"],2); ?>

        <?php
      }
    } else {
      echo "0 registros";
    }
    ?>

Ela ta seguinte forma:

o que preciso, quando eu marcar com checkbox ele calcular o total da coluna valor

                    <?php echo round($row["ie_valor"],2); ?>

somente dos que estão checados.
Alguém sabe como posso fazer, lembrando que o nome do checkbox pode ser diferente, e preciso manter assim.
Resultado Final esperado:
Answer: Para facilitar o processo optei por acrescentar a class check na checkbox e uma classe total nas colunas de totais.
Para cada checkbox a lógica a aplicar é:

Definir o evento change
Nesse evento buscar todas as checkboxs selecionadas à custa do seletor .check:checked
Para cada checkbox encontrada navegar até ao total utilizando closest e querySelector
Interpretar o valor com parseFloat e acrescenta-lo ao total
Mostrar o total caso existam checkboxs selecionadas ou 0 caso contrário

Implementação:

const checks = document.querySelectorAll(".check");
const divtotal = document.getElementById("total");

for (let i = 0; i < checks.length; ++i){ //para cada checkbox
  checks[i].addEventListener("change", function(){//definir código para o evento change
    //buscar só os checks selecionados
    const checksSelecionados = document.querySelectorAll(".check:checked");
    let total = 0;

    checksSelecionados.forEach(c => { //para cada check selecionado
      let trValor = c.closest("tr").querySelector(".valor"); //achar o tr com o valor
      total += parseFloat(trValor.innerHTML); //transformar em numero e somar ao total
    });

    divtotal.innerHTML = "Total :" + total.toFixed(2); //mostrar o total
  });
}

    <form class="form-inline" action="registrosVazios.php" method="GET">

              <input type="checkbox" name="chkAll" />

              DATA

              PDV

              CUPOM

              VALOR

                <input type='checkbox' class="check" >

                data

                pdv

                cupom

                15.50

                <input type='checkbox' class="check" >

                data

                pdv

                cupom

                39.12

                <input type='checkbox' class="check" >

                data

                pdv

                cupom

                4.70

                <input type='checkbox' class="check" >

                data

                pdv

                cupom

                2.78

Documentação:

closest
querySelector
querySelectorAll


Drupal Commerce - Persistent Data Storage within the Cart

Question: My objective is to build an commerce site that attributes 'tokens' (integer values) to products and than keeps a running total of the complete token value within the cart. The idea is to check against how many tokens the user has before allowing a purchase, allowing a product to be added to cart, etc. etc. So far my strategy has been to do this:

Hook_entity_info_alter on the line item product type 'token-based-products' to add a persistent data field for the token value which is required and accessible within views. This seems pretty straight forward.
My Question. How do I keep persistent data storage about tokens within the cart for validation? The objective for this is to create a few hooks that check the running total token value of the cart before allowing/disallowing a product being added AND before allowing products to be checked out.
Answer: I discovered that the wise thing to do was to attribute tokens per user and do either a custom module validation or rules before the product being approved for checkout. Due to the fact that quantities can change in the cart, the validation would also trigger on deletes, or quantity changes in the cart.


Expected value and the standard simple regression model

Question: Given the standard simple regression model:
$y_i = β_0 + β_1 x_i + u_i$
What is the expected value of the estimator $\hat\beta_1$in terms of $x_i, \beta_0$ and $\beta_1$ when $\hat\beta_1=\sum x_i y_i/\sum x_i^2$?
Comment: What are your ideas?
Comment: You must substitute y_i into the expression to get
$β̂_1=∑x_i(β_0 + β_1 x_i +u_i )/∑x^2_1$ and then split up the equation into separate summations, but I'm unclear how to separate treat the expected value of $∑x_i u_i / ∑x_i^2$
Answer: \begin{align}
\text{E} \left ( \frac{\sum_{i=1}^{n} x_i Y_i}{\sum_{j=1}^{n} x_j^2} \right ) &= \left ( \sum_{j=1}^{n} x_j^2 \right )^{-1} \sum_{i=1}^{n} x_i \text{E}(Y_i) \\
&= \left ( \sum_{j=1}^{n} x_j^2 \right )^{-1} \sum_{i=1}^{n} x_i (\beta_0 + \beta_1 x_i) \\
&= \beta_0 n\bar{x} \left ( \sum_{j=1}^{n} x_j^2 \right )^{-1} + \beta_1 .
\end{align}
Comment: You used the right formula for $\hat \beta_1$, if there is no intercept. But here we have $\beta_0$.
Comment: Correct, which is what the poster asked about. $\beta_0$ is an assumed part of the true model, not necessarily the model being estimated.  The apparent point is that the estimator is unbiased if $\beta_0 = 0$ or $\bar{x} = 0$.
Comment: If the estimators of the parameters are unbiased, then it doesn´t imply that $\beta_0=0$-only $E(\hat \beta_0)=\beta_0$
Comment: I don´t understand your comment.
Answer: Denote $x=(x_1,\dots,x_n)$. Assuming that $\mathbb{E}[u_i|x]=0$
$$\mathbb{E}[\hat\beta_1|x]=\mathbb{E}\left[\frac{\sum x_i y_i}{\sum x_i^2}\mid x\right]=\mathbb{E}\left[\frac{\sum x_i (\beta_0 + \beta_1 x_i + u_i)}{\sum x_i^2}\mid x\right]=\frac{\beta_0\sum x_i+\beta_1\sum x_i^2+\sum x_i\mathbb{E}[u_i|x]}{\sum x_i^2}=\beta_1+\beta_0\frac{\sum x_i}{\sum x_i^2}$$
and
$$\mathbb{E}[\hat\beta_1]=\beta_1+\beta_0\mathbb{E}\left[\frac{\sum x_i}{\sum x_i^2}\right]$$
Comment: @thevbm $\hat{\beta}_1$ is biased in this case (i.e. it's expectation is not $\beta_1$).
Comment: I want to clear my concept in this topic. If $$\mathbb{E}[\hat{\beta_1}]=\beta_1$$ is true, then from the result (that you have proved nicely), how can I derive $\mathbb{E}\hat{\beta_1}=\beta_1$?


An exception of type 'System.InvalidCastException' occurred in EntityFramework.Core.dll but was not handled in user code

Question: I am trying to get the long value associated to an ActivityLogType it is associated to either a create, edit, and delete  record in the database.
It is used for an audit / activity log that is needed.
public enum ActivityLogType
{
    Create = 1,
    Update = 2,
    Delete = 3
}

My getter method:
public ActivityType GetType(ActivityLogType type)
{
    var id = (long)type;

    Console.WriteLine(id); // <---- this produces a 1 in the console. So the cast works?

    return _context.ActivityTypes.Where(x => x.Id == id).FirstOrDefault(); // <-- This line throws the error
}

UPDATE 1
un-lucky suggested using (long)ActivityLogType.Create to get the desired output. I tried this: (Still not working)
public ActivityType GetType(ActivityLogType type)
{

    switch (type)
    {
        case ActivityLogType.Create:
            return _context.ActivityTypes.Where(x => x.Id == (long)ActivityLogType.Create).FirstOrDefault();
        case ActivityLogType.Update:
            return _context.ActivityTypes.Where(x => x.Id == (long)ActivityLogType.Update).FirstOrDefault();
        case ActivityLogType.Delete:
            return _context.ActivityTypes.Where(x => x.Id == (long)ActivityLogType.Delete).FirstOrDefault();
        default:
        return null;
    }

}

UPDATE 2
Here is the ActivityType entity
public class ActivityType
{

    public ActivityType()
    {
        this.Activities = new HashSet<Activity>();
    }

    [Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public long Id { get; set; }

    [Display(Name = "Label")]
    public string Label { get; set; }

    [Display(Name = "Display Order")]
    public int Order { get; set; }

    [Display(Name = "Is Active")]
    public bool IsActive { get; set; }

    [Display(Name = "Activities")]
    public virtual ICollection<Activity> Activities { get; set; }
}

Comment: and what is `ActivityTypes` here
Comment: @un-lucky it is a `DbSet`
Comment: @un-lucky `public virtual DbSet ActivityTypes { get; set; }` in my `DbContext.cs`
Comment: Can you please update how the ActivityLogType coded in ActivityType entity and what is the Data Type in DB. The above code working fine for me.
Comment: Instead of using where and firstOrDefault you can just write FirstOrDefault(x => x.Id == (long)ActivityLogType.Create).
Comment: @Rijnhardt The updated code also works fine for me. One question here, you are going to store only three values in the ActivityType entity, but you have declared it as long. Is there any specific requirement?
Comment: @SelvaTS I think I found a potential reason why it is not working, I am quickly remigrating the database.

The reason I am storing it as longs is just compatability with the rest of the database. All my other tables are long. (It is an OCD thing -- I understand for performance an int would be better)
Comment: @SelvaTS It is fixed, thanks for the input. There was a mismatch between the db and the entity.
Answer: You can try this:
long id= Int64.Parse(Convert.ChangeType(type, Enum.GetUnderlyingType(type.GetType())).ToString());


Espaço entre imagem e conteudo Android studio

Question: Estou desenvolvendo uma aplicação mobile utilizando o android studio 2.2.2 e queria deixar minha aplicação com esse visual.

Mas não estou conseguindo , ao colocar a imagem ele fica com um espaço entre a imagem e os TextView de cima e do de baixo.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:orientation="vertical"
tools:context="com.example.tulio.exercicio3.MainActivity">

<TextView
    android:id="@+id/Lamborghini"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_weight="1"
    android:background="@android:color/holo_green_dark"
    android:text="Lamborghini"
    android:textAlignment="center"
    android:textColor="#ffffff"
    android:textSize="50sp" />

<ImageView
    android:id="@+id/imageView"
    android:layout_width="match_parent"
    android:layout_height="272dp"
    android:layout_weight="3"
    app:srcCompat="@drawable/lamborghini" />

<TextView
    android:id="@+id/aventador"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_weight="1"
    android:background="@android:color/darker_gray"
    android:text="Aventador"
    android:textAlignment="center"
    android:textColor="@android:color/holo_red_dark"
    android:textSize="36sp" />

<TextView
    android:id="@+id/preco"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_weight="1"
    android:background="@android:color/darker_gray"
    android:text="R$ 434,000"
    android:textAlignment="center"
    android:textColor="@android:color/holo_red_dark"
    android:textSize="36sp" />

    </LinearLayout>

Answer: Adicione
android:scaleType="centerCrop"

No seu imageview, para ficar do jeito que você quer, a imagem precisar ser cortada e centralizada


Why is cross validation better than training once with the whole data set?

Question: A form of cross validation takes multiple subsets of a data and trains the model on them, then tests on the remaining subsets. Eventually, all subsets will be trained on.
Although I know that training once on the whole dataset will more likely result in overfitting than using cross validation, the end result is both methods result in the whole dataset being trained on. Why is cross validation better?
Comment: This question doesn't make sense. You state that using the whole dataset "will more likely result in overfitting than using cross validation", but then ask why cross validation is better. Are you wondering if overfitting is a good thing?
Comment: This is not a duplicate. I even explained what cross validation is the details. I'm asking why it is better.
Comment: @Goldname I'm not sure I picked the best redirect target, but the answers there do cover the issue of what the point of cross-validation is, which is what this question is, so far as I can tell.
Comment: @gung To the extent that this question doesn't make sense, it does so due to a misunderstanding that might be the focus of an answer. The core question here seems to be "how come cross-validation is still effective, even though it ultimately uses the entire dataset?" which is arguably a distinct question in its own right. (Though I'd agree that several of the answers to the linked question really also answer this one, I do think the question itself is distinctive.)
Comment: @Silverfish, if it were edited to be more coherent (eg, like you suggest), I'd be willing to leave / re- open. At present, I think this should probably be closed.
Answer: Cross validation is a way to deal with overfitting. In a simpler setup, you have a training and hold out sample. You train on a training, then test the model on the holdout. The problem was that people started using the results on (out-of-sample) holdout samples to select the models. This made holdout samples similar to training sample in many ways. You look at in-sample and out-of-sample fit metrics, then pick the best model. This is almost the same as fitting to the whole sample once (in-sample).
So, the trend now is to split into a training, cross-validation and test samples. You have this one test sample which is not used for model selection at all. It's your last bullet: once you selected the best model in-sample and out-of-sample, you test this one last selection on the test sample (which was not used so far) to validate that the model works.
Comment: Do you understand the overfitting issue? That's where the answer is.
Comment: It's like why do we need seat belts if one can kill himself even with it? Well, for 10 idiots who overfit with cross validation, there'll be one who'll use it to avoid this trap. I came to cross validation on my own, we were not trained on this back in the days. Once I realized that my out-of-sample tests are not that different from in-sample if they're used for model selection. Then I saw what people were doing with cross validation, and adopted it in my practice.
Comment: You are asking to close this question, effectively
Comment: But why is cross validation better?
Comment: Once you're all done with the cross validation, you would have basically overfit to your dataset
Comment: You can overfit with cross-validation, especially, when using automated routines like R's autoarima (or whatever it's called). That's the reason you have the third (test) set. It helps. You can still overfit, if you're really hell bent on overfitting. I think the key here is that there's no pure statistical, mechanical algorithm to building a good model despite some will tell you otherwise on this board.
Comment: So one overfits whether or not there is cross validation. So why use cross validation?
Comment: I am asking why using seat belts reduce your likelihood of dying.
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/41205/discussion-between-aksakal-and-goldname).
Answer: I guess you are asking for something slightly different here: after you used cross validation (CV) for model training, evaluation, and selection, you  train the best performing model type and hyperparameter set once again, using all training data. This model is the one you will use on your held back test set on once. So if I understand correctly what you want to point out is that only using CV and getting some results does not change anything in the training process of the chosen model, hence conceptually does not prevent any overfitting - and this is correct so far. But:
Your CV results give you more and better information about how different model types and hyperparameterizations perform on data the model has not seen before. Imagine only doing one training and one cross validation partition: you could be lucky (having the partitions separated in a way that from each you can predict the other well). If you use e.g. a 10 fold CV with 20 repeats you train your model 200 times and each time evaluate it on a portion of data it has not seen before - so this will be much more robust, as being lucky 200 times is less likely. If you have 200 such evaluation results for multiple types of models and multiple hyperparameter sets you have better information on which to choose the model you actually want to train once using all training data. This process ensures you choose something that will less likely overfit, therefore is a big advantage over using a single training+test partition - and I guess this is the difference you asked for in your question.


First derivative of this secial function

Question: What is the derivative of the following function:
$$f(x) = \frac{a}{((\sqrt{b+bx})(d-\sqrt{e+gx}))^2}$$
Comment: [Wolfram Alpha](http://www.wolframalpha.com/) can tell you.
Answer: Hint
Rewrite $$f(x) = \frac{a}{\Big((\sqrt{b+bx})(d-\sqrt{e+gx})\Big)^2}= \frac{a}{b(1+x)\Big((d-\sqrt{e+gx})\Big)^2}$$ $$f(x) = \frac{a}{b~~ u(x)~~ v(x)}=\frac{a}{b}\frac{1}{u(x)~ v(x)}$$ using $$u(x)=1+x$$ $$v(x)=w^2(x)$$ $$w(x)=d-\sqrt{e+gx}$$ which will make life simpler.  Now $$u'(x)=1$$ $$v'(x)=2w(x)~w'(x)$$ $$w'(x)=-\frac{g}{2 \sqrt{e+g x}}$$
I am sure that you can take from here and finish.


Valid vs. sound (and if these differences matter in syntax vs. semantics, metalogic vs. logic)

Question: I am trying to keep the concepts of metalogic and logic as separate as I can to avoid confusion, but I still get a little lost when we talk about validity and soundness. I look at past answers and they seem a little hazy.
Are these metalogical or logical concepts? Is there such a thing as syntactical validity/soundness? Semantic validity/soundness? How are these terms used and defined? Why are they important exactly? Are we able to say anything useful even if something is invalid or unsound?
Do these definitions change depending on the logic we're talking about? (propositional calculus, predicate calculus, Hilbert-style, ND-style, etc). Does the concept of validity/soundness only apply to a 2-valued logic system with true/false?
I suppose in particular I am focusing on classical propositional calculus but it would be nice to know if things change outside of that context as well.
Answer: Definitions

An argument is valid if and only if it takes a form that makes it impossible for the premises to be true and the conclusion nevertheless to be false. It is not required that a valid argument have premises that are actually true, but to have premises that, if they were true, would guarantee the truth of the argument's conclusion.
A formula is valid if and only if it is true under every interpretation, and an argument form (or schema) is valid if and only if every argument of that logical form is valid.

In propositional logic, a valid formula is also caleld a tautology.

An argument is sound if it is valid and all the premises true.

A logical calculus (language + formation rules (defining well-formed formulas) + axioms + inference rules) with its semantics is sound (or has the soundness property)

if and only if every formula that can be proved in the system is logically valid with respect to the semantics of the system.

A logical calculus is complete

with respect to a particular property if every formula having the property can be derived using that system, i.e. is one of its theorems.
The term "complete" is also used without qualification, referring to the property of semantical validity. Intuitively, a system is called complete in this particular sense, if it can derive every formula that is true.

Logical systems different from the classical ones, like e.g Intuitionistic logic and Modal logic, are sound and complete wih respect to the relevant semantics : see Kripke semantics.

The study of classical propositional logic is useful because we can see how the basic "machinery" works in a very simple case.
We have the language made of propositional variables : $p_1, p_2, \ldots$, the usual truh-functional logical connectives and the rules for producing well-formed formulas.
We have to define the truth-valuation i.e. a function

$v : \text {Prop} → \{ \text T, \text F \}$,

where Prop is the set of propositional variables of the language.
Then we extend the valuations to all formulas of the language using the usual truth-tables for the propositional connectives.
Example : if formula $\alpha$ is $(p_1 \land p_2)$, we have that $v(\alpha)= \text T$ iff $v(p_1)= \text T$ and $v(p_2)= \text T$, and so on.
Thus, to check that our preferred calculus is sound is an easy task. We have to :

(i) verify that all axioms are tautologies (using truth table);
(ii) verify that the inference rules preserve truth. In the case of modus ponens, this is again a simple application of the truth table for $\to$.

The completeness of the calculus is a mathematical problem that is much harder to solve.
Comment: @user525966 This is the last I'll answer here (other things to do... also so as to not flood Mauro's inbox) Remember what I said about saying 'not valid' rather than 'invalid'. Yes, it is not a valid formula provided $p$ and $q$ are sentence variables. The fact that it is true under *some* interpretation is expressed by saying it is *satisfiable*. (And since you got it in on time: in mathematical logic, 'sound' is usually a term that describes a deductive system, not a statement.)
Comment: Are we defining premise and conclusion as $p \to q$ or is it instead $p \vdash q$, i.e. are we talking about validity within-the-system or outside-the-system?
Comment: What is "true under every interpretation" exactly? Is there an example of this?
Comment: Is it not possible for something to be sound, but invalid?
Comment: @user525966 - in classical propositional logic, "true under every interpretation" are tautologies; see [truth table](https://en.wikipedia.org/wiki/Truth_table).
Comment: Does "every interpretation" mean "all possible values across all possible inputs"? e.g. $a \land b$ has interpretations $(0 \land 0)$, $(0 \land 1)$, $(1 \land 0)$, $(1 \land 1)$?
Comment: @user525966 Yes, that's pretty much what it means in this semantics. If $a$ and $b$ are propositional variables, then for sentences just involving $a$ and $b,$ there will be four relevant classes of interpretation corresponding to $(a,b) = (0,0), (0,1), (1,0), (1,1)$ ('classes' because the other propositional variables aside from $a$ and $b$ can take various truth values, but who cares what they are). We see $a\land b$ takes the value $0\land 0 = 0$, $ 0\land 1 = 0$, $1\land0 =0$, and $1\land 1 = 1$ respectively. Thus we conclude $a\land b$ is not a valid formula, nor is its negation valid.
Comment: @spaceisdarkgreen But if $a \land b$ is invalid, why do we use it in practice? It seems like quite a fundamental operator?
Comment: @user525966 You can use something without asserting it is always true... I use the equation $3x + 4y =0$ even though it isn't true for all $x$ and $y.$
Comment: But then this would imply the only valid things are tautologies and everything else is invalid, no? That seems strange to me that only tautologies can be considered logically valid
Comment: @user525966 Valid means it is true by virtue of its form, i.e. its truth is not contingent on the circumstances. If I told asked you if "(proposition 97) and (proposition 12)" is true, you'd look at me strange and say, "well, that depend what proposition 97 and proposition 12 are, in particular, whether they're true or false." On the other hand if I asked you if " either (proposition 80) or the negation of (proposition 80)" is true, you would say 'yes' without needing to know what proposition 80 is.
Comment: @user525966 That is *exactly* what it implies. Why is that strange? (Perhaps the connotation of invalid? We generally say 'not valid' rather than 'invalid' as the negation of 'valid' in this context, presumably for this reason.)
Comment: @user525966 Sometimes other qualifiers are used to highlight the distinction between "valid" and the colloquially synonymous but mathematically very different "true". For instance you often hear "universally valid". (and it doesn't help that 'valid' has other non-mathematical meanings as well, such as pertaining to arguments rather than statements).
Comment: Does this mean true (i.e. just $\top$) is valid/a tautology? And false (i.e. $\bot$) is invalid / not a tautology?
Comment: @user525966 sure. But we can say something stronger about false. It is the negation of a valid formula... i.e. something that is false in all interpretations, also known as a contradiction or an unsatisfiable formula.
Comment: @spaceisdarkgreen So the $p \to q$ statement is also invalid because it is false when $T \to F$? Even though it isn't possible for it to be false when the premise is true?
Comment: So a sound statement is one that is a tautology and happens to have true premises?
Comment: Yes; the propositional *constant* $\top$ is a *tautology*, because we interpret it as denoting TRUE, and thus, as a formula, it is always true. The propo const $\bot$ is a *contradiction*, i.e. always false, for the same reason.
Comment: $p \to p$ as well as $\lnot p \lor p$ are tautologies. $\lnot p \land p$ is a contradicition. $p \to q$ is neither taut nor contra (someone call it : contingent). You have to try to use truth table: it is the best way to learn the machinery of the (elementary) semantics of prop logic.


Confusion about feature maps in CNN -- why don't they learn the same thing

Question: I have a theoretical question about CNN. As a reference I'm using: http://cs231n.github.io/convolutional-networks/#case
I'm pretty clear on the RELU layer and Pooling Layers. Those are deterministic and easy to figure out. Even the fully connected layers are fairly straightforward -- a gradient descent algorithm is used to reduce minimize the cross entropy. So far so good.
However, I'm confused on the convolution layer. I understand how a convolution works. However, I don't understand is why different convolutions are learned for different feature maps.
For example, suppose I start with a 32 x 32 x 3 image. Suppose I use a stride of 1 and a pad of 1. That means my first neuron is connected to 32x32x3=3072 pixels (plus +1 for a bias). Specifically it connects to [1:32, 1:32, 1:3]. My second neuron connects [2:33, 2:33, 1:3] and so forth.  So I ask it to train and it estimates 3073 total parameters. Great.
Now suppose I want a second feature map. The convolutions are calculated in parallel as they only take the original receiving image. So the first neuron again takes pixels [1:32, 1:32, 1:3]. The second neuron again takes pixels [2:33, 2:33, 1:3] etc. Again it estimates 3073 parameters.
Why won't it learn the exact same values for the 3073 parameters as it's just doing gradient descent?
In other words why might one feature learn parameters that activate on a "squiggly curve" and one feature learn parameters that activate a "90 degree angle". They both start with the same image and they both do gradient descent!
Thanks in advance for your help.
Answer: This is exactly the same reasoning to why parallel neurons in general not learn the same thing. By doing random initialization the neurons are by construction different and as the optimization progresses the neurons have no benefit of reproducing other neuron. So the gradients are not the same otherwise you are correct. Check out: http://cs231n.github.io/neural-networks-2/#init

Lets start with what we should not do. Note that we do not know what the final value of every weight should be in the trained network, but with proper data normalization it is reasonable to assume that approximately half of the weights will be positive and half of them will be negative. A reasonable-sounding idea then might be to set all the initial weights to zero, which we expect to be the "best guess" in expectation. This turns out to be a mistake, because if every neuron in the network computes the same output, then they will also all compute the same gradients during backpropagation and undergo the exact same parameter updates. In other words, there is no source of asymmetry between neurons if their weights are initialized to be the same.
Comment: I thought that might be it but I'm not completely convinced. Say my image has a very *strong* feature -- eg if there is a big horizontal black line it means a positive class (and let's say it's just 2 classes). Won't many feature maps of the CNN (if not all of them at that depth) then simply learn that big black line?
Comment: The gradient of well classified samples are smaller than the gradients from poorly classified samples. When the network have learned that black lines means positive class the samples which obeys this pattern will contribute very little to the gradient since they are well classified. So the gradient signal will be more about modeling the residual. So not modeling the black lines. If there are no residual sure you are correct. But then it is not a very interesting problem.


Ruby CSV.open need to remove quotes and null characters

Question: I am retrieving a large hash of results from a database query and writing them to a csv file. The code block below takes the results and creates the CSV. With the quote_char: option it will replace the quotes with NULL characters which I need to properly create the tab-delimited file.
However, the NULL characters are getting converted into "" when they are loaded into their destination so I would like to remove those. If I leave out quote_char: every field is double quoted which causes the same result.
How can I remove the NULL characters?
begin
    CSV.open("#{file_path}"'file.tab', "wb", Options = {col_sep: "\t", quote_char: "\0"}) do |csv|
        csv << ["Key","channel"]
        series_1_results.each_hash do |series_1|
         csv << ["#{series_1['key']}","#{series_1['channel']}"]
        end
    end
end

Comment: To provide more clarity on what I need for the output, it must match the format provided by the 'Windows Formatted Text.txt' in the Save As... menu. That is a tab-delimited format with no quotes in the file.
Answer: As it is stated in the csv documentation you have to the set quote_char to some character, and this character will always be used to quote empty fields.
It seems the only solution in this case is to remove used quote_chars from the created csv file. You can do it like this:
quotedFile = File.read("#{file_path}"'file.tab')
unquotedFile = quotedFile.gsub("\0", "")
File.open("#{file_path}"'unquoted_file.tab',"w") { |file| file.puts replace }

I assume here that NULL's are the only escaped fields. If that's not the case use default quote_char: '"' and gsub(',"",', '') which should handle almost all possible cases of fields containing special characters.
But as you note that the results of your query are large it might be more practical to prepare the csv file on your own and avoid processing the outputs twice. You could simply write:
File.open("#{file_path}"'unquoted_file.tab',"w") do |file|
    csv.puts ["Key","channel"]
    series_1_results.each_hash do |series_1|
        csv.puts ["#{series_1['key']},#{series_1['channel']}"]
    end
end

Once more, you might need to handle fields with special characters.
Answer: From the Ruby CSV docs, setting force_quotes: false in the options seems to work.
CSV.open("#{file_path}"'file.tab', "wb", { col_sep: "\t", force_quotes: false }) do |csv|

The above does the trick. I'd suggest against setting quote_char to \0 since that doesn't work as expected.
There is one thing to note though. If the field is an empty string "" - it will force the quote_char to be printed into the CSV. But strangely a nil value does not. I'd suggest that if at all you're expecting empty strings in the data, you somehow convert them into nil when writing to the CSV (maybe using the ActiveSupport presence method or anything similar).
Answer: First, a tab-separated file is "TSV", vs. a comma-separated file which is "CSV".
Wrapping quotes around fields is necessary any time there could be an occurrence of the field delimiter inside a field.
For instance, how are you going to embed this string in a tab-delimited file?
Foo\tbar

The \t is the representation of an embedded Tab.
The same problem occurs when writing a CSV file with a field containing commas. The field has to be wrapped in double-quotes to delimit the field itself.
Comment: There aren't any instances of the tab character in the data. I am ok with wrapping the fields that have some text quotes because you are right it is better to have it. But then how can I have the NULL fields left with no quotes? If I remove quote_char: all the fields have quotes. What do you suggest?
Answer: If your input contains any data that needs to be escaped (such as the column separator or the quote character), then you do need to quote your data. Otherwise it cannot be parsed correctly later.
CSV.open('test.csv', 'wb', col_sep: "\t") do |csv|
  csv << ["test", "'test'", '"test"', nil, "test\ttest"]
end

puts open('test.csv').read
#test    'test'  """test"""              "test   test"

The CSV class won't quote anything unnecessarily (as you can see above). So I'm not sure why you're saying all your fields are being quoted. It could be somehow force_quotes is getting set to true somewhere.
If you're absolutely certain your data will never contain \t or ", then the default quote_char (") should work just fine. Otherwise, if you want to avoid quoting anything, you'll need to pick another quote character that you're absolutely certain won't occur in your data.
CSV.open('test.csv', 'wb', col_sep: "\t", quote_char: "|") do |csv|
  csv << ["test", "'test'", nil, '"test"']
end

puts open('test.csv').read
#test    'test'          "test"


Comparador numérico

Question: print ("Insira dois valores para serem comparados:")

Valor1= float(input("Escreva um número:"))
Valor2= float(input("Escreva outro número:"))

print ("Os números que você inseriu foram esses?","\n", Valor1, "\n", Valor2)

sim = str
não = str

if (str(input())=="não") or (str(input())=="sim"):

if str(input(sim)):
    if (Valor1 > Valor2):
        print ("O primeiro valor é maior que o segundo")

    if (Valor2 > Valor1):
        print ("O segundo valor é maior que o primeiro")

    if (Valor2 == Valor1):
        print ("Os números são iguais")

if str(input(não)):
    print ("Então insira os números corretos")

Por que ele fica mostrando a classe da variável Sim depois de eu responder se digitei corretamente os valores?
E aparentemente ele pede a resposta mais de uma vez, não entendi o porquê.
Tentei fazer o código utilizando while em vez de if para começar a condicional e fechar as sequências com break, mas o mesmo problema aconteceu.
Comment: Por isso que eu perguntei, você não aceitou a resposta, você só votou, eu percebi que não entendeu a diferença, dá uma olhada de novo no [tour].
Answer: Este código tem uma enorme quantidade de erros. Alguns casos não são bem erros mas deixa o código não tão legível ou não é comum escrever assim:

Não faço ideia o que sejam essas variáveis sim enão, só as eliminei já que elas não tem utilidade
Não converti o input() para str() porque ele já é uma string
Apenas comparei essa entrada com sim e mais nada porque é só assim que deve fazer as comparações, se digitar qualquer outra coisa considerará como errado
Um laço poderia ser usado aí mas depende de requisito, nada indica na pergunta que tenha, isso pode ter tentando usar por chute
Preferi usar nomes de variáveis em minúsculo que é o normal, e para manter consistente com as outras
Preferi não tratar a exceção lançada se não for digitado um número adequado, mas o mais correto é fazer isto
Tem if demais aí e alguns desses if funcionarão melhor se forem elifs
Talvez o maior problema é pedir a confirmação várias vezes, isso não faz o menor sentido, é por você ter colocado no código pedindo para digitar várias vezes que está pedindo várias vezes, se quer só uma vez coloque só uma vez.
Eu recomendaria entender o que cada coisa no código faz antes de usar, colocar coisas aleatórios nele que parecem se encaixar porque viu em algum lugar alguém fazendo algo parecido não vai te fazer aprender programar. Você precisa mudar a forma de aprendizado para progredir.

Veja:
print("Insira dois valores para serem comparados:")
valor1 = float(input("\nEscreva um número:"))
valor2 = float(input("\nEscreva outro número:"))
print("\nConfirme os números que você inseriu digitando 'sim':\n", valor1, "\n", valor2)
if input() == "sim":
    if valor1 > valor2:
        print("O primeiro valor é maior que o segundo")
    elif valor2 > valor1:
        print("O segundo valor é maior que o primeiro")
    elif valor2 == valor1:
        print("Os números são iguais")
else:
    print("Então insira os números corretos")

Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura.
Comment: O motivo disso eu identifiquei que é porque está tentando aprender de forma não estruturada, isso não dá certo. Preciso construir o conhecimento um passo de cada vez.
Comment: Estou tentando afinar esse sentido, sou muito novato na programação e estou tirando bastante leite de pedra para aprender, estou achando muitas coisas confusas, principalmente na hora de escolher os problemas para resolver.

Muito obrigado pela ajuda e pelas dicas com o código :)
Comment: Sim, mas justamente a minha dúvida é não achar uma forma muito linear de como progredir dentro da programação, parece que tem tanta coisa a ser apreendida que pelo excesso de opções, torna tudo um pouco mais confuso.
Ainda mais que meu objetivo com a programação em si, é tornar algumas técnicas que uso em física, química e matemática algo mais palatável, porém transcrever isso para uma lógica que o computador entenda, está sendo meu desafio.

Comparador numérico