Math With Base 3 Truth Values

Question: I was wondering if there is any work towards a system of mathematics were truth values could take on 3 values. Traditionally, we  view every statement as either true or false. Computers are built in binary but in theory there could be a ternary computer https://hackaday.com/2016/12/16/building-the-first-ternary-microprocessor/. Could we do something similar in mathematics and what would it look like? Also, is there anyone working towards this? Thanks in advance!
Comment: There's also a three-valued Heyting algebra, which could form the basis for a three-truth-valued intuitionistic logic.
Comment: like true, false, and maybe?
Comment: See [Inconsistent Mathematics](https://plato.stanford.edu/entries/mathematics-inconsistent)
Answer: Sure, this can be and has been done.
A third truth value $u$ may be thought of in different ways: as "neither true nor false" (a truth value gap) or as "both true and false" (a truth value glut), or possibly something different altogether.
Two common examples of three-valued logics which assume a third truth value $u$ include strong Kleene 3-valued logic $K_3$, and $LP$ .
Both of these logics define the following truth tables for the standard connectives:

Note that the particular semantics defined here is a conservative extension of classical logic: Classical input values yield classical output values.
The interpretation of $u$ is where $K3$ and $LP$ differ: $K_3$ treats $u$ as a truth value gap; $LP$ treats $u$ as a truth value glut.  Although the semantics of the object language connectives are the same between the two logics, the different interpretations associated with the truth value $u$ give rise to a different meta logic, resulting in some logical inferences being valid in $K_3$ but not in $LP$ or vice versa. For instance, $\nvDash_{K_3} p ∧ ¬p → q$, and $p → q, p \nvDash_{LP} q$ (that is, $LP$ rejects modus ponens!).
A reasonable alternative to the above semantics is to change the truth tables for $\to$ and $\leftrightarrow$ such that in the interpretation where $φ$ and $ψ$ are both $u$ (middle cell), the implication and the biimplication is $1$ rather than $u$. This modification is known as Lukasiewicz logic, $L_3$.
And of course, there are a number of other possible three-valued logics with different truth tables and logical laws.
If you would like to read up more, I recommend

Priest, Graham. 2008. An introduction to non-classical logic: From if to is. Cambridge University Press.

This is a rather accessibly written overview of non-standard logics that includes a chapter on many-valued logics.

Math With Base 3 Truth Values


Mode from continuous distribution

Question: I've continuous distribution set of values.
For example:
0, 0.01, 0.012, 1.2, 5.33, 5.24, 5.38, 30.20, 30.21, 30.13, 30.12
I want to calculate most frequent value from this set. So I've found this question which says it's mode.
I've problem with spliting this set into clases.
I want to put in my algorithm some delta value and assign to one class values which fullfills x - delta < mean_class_value < x + delta. Of course I see the problem that I don't have a class and to create it I need it's mean value. Also solution which will make the same reasonable result will be ok.
Any solution in pseudocode will be great help.
My current solution its pseudocode mixed with c++ but hope understandable:
std::vector<std::pair<double, std::vector<elementtype> > > classes;
foreach(element)
{
    foreach(class in classes)
    {
        if(std::abs(element.value - class.first) //class first is value of class
        {
             //assign element to class here
             class.first = (class.first + element.value) / 2.0 //averaging class value
             break;
        }
        else
        {
             //create new class
        }

    }
}

Answer: There are many estimators for the mode of a continuous distribution.
The answer here links to the documentation of an R package.
The advantage at looking at that R package is that one gets to learn about different mode estimators (with the appropriate reference in the pdf document).
Also, since you are interested in code, here is the source code link for all the routines in there.
Comment: I've found this answer. But seems to be too complicated for my problem. Look at my attached code it is very simple. I'm looking for some easy estimation.
Answer: First of all you have discrete set of values . Depending on the purpose of your data analysis you can evaluate different statistics ( functions of the data). If you want to divide your data into classes you should calculate histogram. Assume $x_{(1)}=min_{i} x_i$ ( the smallest element) and $x_{(n)}=max_i x_i$ ( the largest element ), assume also that you want to divide your data set into $k$ disjoint classes. Proceed as follows: let $\delta=\frac{x_{(n)}-x_{(1)}}{k}$, $\forall i=0,...,k-1$ calculate amount of the elements in the interval $[x_{(1)}+i*\delta,x_{(1)} + (i+1)*\delta]$. This will be representation of your data as a histogram, nothing more. As for the most frequent value, I doubt that mode will be of any value ( if you have indeed continuous distribution it won't even have a meaning) instead you should calculate mean.

Mode from continuous distribution


How many combinations are there for the interior angles of a triangle?

Question: Suppose the interior angles of a triangle are all Natural numbers. How many combinations of angles are there without repeating similar triangles? So for instance, {1,1,178}, {1,2,177},...But without relisting similar triangles {178,1,1}, {177,1,2},...
Comment: I just came up with the problem for fun. I am currently trying to find a solution. Will post if I get somewhere.
Comment: An idea to maybe play with: how many solutions are there to $x+y+z=180$ where $0<x,y,z<179$?  I know that there are many similar problems to this that can be found. A common way to tackle this is with generating functions.
Answer: (I'm copying and updating an answer I gave two days ago.)
Your "combinations of angles" are called partitions of $180$ into three parts, or $3$-partitions in the sequel.
Since $a$, $b$, $c$ have to be $\geq1$, by "stars and bars" there are ${(180-3)+2\choose 2}=15\,931$ ordered admissible triples summing to $180$. Here  each partition of $180$ with three different parts occurs six times, the $88$ $3$-partitions with exactly two equal parts occur  three times, and the partition $(60,60,60)$ occurs one time. If we now form the number
$$15\,931+ 3\cdot 88+5\cdot 1=16\,200$$
then all $3$-partitions of $180$ have been counted exactly six times. It follows that the number of $3$-partitions of $180$  is ${16\,200\over6}=2700$.
Answer: Hint 1: Once you give the first two angles, third angle is given too.
Hint 2: To avoid repeating triangles, list the angles in increasing order.
Hint 3: If $\alpha\le\beta\le\gamma$ are the angles, then $\alpha\le60°$ and $\beta\le90°-\alpha/2$.
Answer: Supposing you are working with degrees, the question is that of finding triples $(a,b,c)$ with $a+b+c=180$, where $a,b,c$ are the interior angles. To avoid similar triangles, we may require that $180 \geq a\geq b\geq c \geq 0$ (resp. $178$, $1$ if you wish to avoid degenerate cases) . Suppose then we know $a$. $b,c$ sum to $b+c=180-a$ But since $b \geq c$, there are only $\lfloor \frac{180-a}{2}\rfloor$ possibilities for $(b,c)$, so the answer to the question is $\sum_{a=60}^{180} \lfloor \frac{180-a}{2}\rfloor$ (resp. $178$ for the upper bound)


Using grep to extract html from  container tags

Question: I have a page that has many posts by different authors. I want the posts from user A from that page of posts.
How can I set up grep to look at each post's html block in the page for the author, then print the content of the post to a file? The post structure is something like

[useless junk i'm not interested in here]

User A

[more junk]

                     Here's the text I want to extract

[more junk]

I think the structure is something like
grep /pattern/ output file

but do I need to explicitly tell it to hunt only between the

and

tags that bound the post, or is grep smart enough to do that automatically? I'm worried that when grep finds the pattern of User A, it will print all the post contents to a file instead of just that particular one.
Answer: If all the post text is on one line, then try
grep /(author vcard|


How to make a "copy-paste" macro run faster?

Question: I have written a macro in Excel VBA that basically copy-pastes 53 rows 1440 times, one under another, in order to populate two columns in a ~70000 row table. The macro works, but it takes about five minutes to run completely. This would be fine if I didn't have to run this on ~1000 other files. I am looking for any way to speed up this process so that it doesn't take 5 days to run.
I tried using the range copy method:
<code>    Set range1 = {the table I'm copying}
    Set range2 = {the cells I want to paste into}
    range1.Copy range2
</code>
but it took just as long, if not longer.
Here is my current code:
<code>    Windows("as_built_comp.xlsm").Activate
    Sheets(siteName).Activate
    j = Cells(Rows.Count, 1).End(xlUp).Row
    Range("C2").Select
    Range(Selection, Selection.End(xlToRight)).Select
    Range(Selection, Selection.End(xlDown)).Select
    Selection.Copy
    wb.Activate
    Range("I12").Select
    For i = 1 To 1440
        Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone,
              SkipBlanks _
        :=False, Transpose:=False
        ActiveWindow.SmallScroll Down:=56
        ActiveCell.Offset(j - 1, 0).Select
    Next i
</code>
I'm thinking the solution might have something to do with using sql in VBA, but I have yet to learn that syntax. Either way, any advice is greatly appreciated. Thank you for reading!
Comment: Remove all the '.Activate` and `.Select`. This is unnecessary steps that slows down your code. Qualify your objects with sheets - this will remove the need for your code to care what is selected or active. You can also try a value transfer instead of a copy/paste. This is much faster
Comment: See [this](https://stackoverflow.com/questions/10714251/how-to-avoid-using-select-in-excel-vba)
Answer: Load it all into an array and then output the entire array at the end.  Code refactored to avoid the use of activate/select
<code>Sub tgr()

    Dim wbDest As Workbook
    Dim wbData As Workbook
    Dim wsDest As Worksheet
    Dim wsData As Worksheet
    Dim aTemp() As Variant
    Dim aData() As Variant
    Dim SiteName As String
    Dim RepeatData As Long
    Dim ixTemp As Long
    Dim ixData As Long
    Dim ixCol As Long

    SiteName = "SiteName1"
    RepeatData = 1440

    Set wbDest = ThisWorkbook
    Set wbData = Workbooks("as_built_comp.xlsm")
    Set wsDest = wbDest.Worksheets(1)
    Set wsData = wbData.Worksheets(SiteName)

    With wsData.Range("C2:D" & wsData.Cells(wsData.Rows.Count, "C").End(xlUp).Row)
        If .Row < 2 Then Exit Sub   'No data
        aTemp = .Value
        ReDim aData(1 To .Rows.Count * RepeatData, 1 To .Columns.Count)
    End With

    For ixData = 1 To UBound(aData, 1)
        ixTemp = ((ixData - 1) Mod UBound(aTemp, 1)) + 1
        For ixCol = 1 To UBound(aTemp, 2)
            aData(ixData, ixCol) = aTemp(ixTemp, ixCol)
        Next ixCol
    Next ixData

    wsDest.Range("I12").Resize(UBound(aData, 1), UBound(aData, 2)).Value = aData

End Sub
</code>
Comment: This was perfect!!! Thank you so much for your help- this literally ran in an instant. You're a lifesaver!!


QMediaPlayer undefined-reference linker errors

Question: I installed Qt5 and since Phonon is not supported in Qt5 I'm forced to use something else, so I decided to use QtMultimedia.
.pro file:
QT       += core gui
CONFIG += mobility
MOBILITY += multimedia

.cpp code:
#include "mainwindow.h"
#include "ui_mainwindow.h"
#include <QtGui>
#include <QtCore>
#include <QtMultimedia/QMediaPlayer>

...

void MainWindow::on_pushButton_clicked()
{
    QMediaPlayer *player = new QMediaPlayer(this);
    player->setVolume(50);
    player->setMedia(QUrl::fromLocalFile("some_path"));
    player->play();
}

But I'm getting following errors:

How can I solve this. Thank you
Answer: Add multimedia module to QT in .pro file, run qmake and then build your project:
QT += core gui multimedia

In Qt 5 that QMediaPlayer class is in multimedia module.
And you might want the widgets module too (i see you have a mainwindow)
LE: Use the include without module folder:
#include <QMediaPlayer>


How to get first difference of count variable for poisson regression

Question: Poisson regression using Panel data requires the dependent variable ($y_{it}$) to be a non-negative count variable. I need to take the first difference of the dependent variable to deal with reverse causality of the regressors. Cameron and Trivedi's Microeconometrics using Stata outline a transformation
$\frac{\lambda_{i,t-1}}{\lambda_{i,t}}*y_{i,t}-y_{i,t-1}$
where
$\lambda_{i,t}=exp(x_{it}'\beta)$
which, they state, can be used to eliminate the FE and as the basis for estimation of dynamic panel count models. My question is how can one actually estimate $\lambda_{it}$ (ideally in Stata) so that it can then be used to create first differences?
Answer: It seems that I need to use the gmm program evaluator to take the differencing as required in the question. I wrote the following program but it's not running and giving the error

could not calculate numerical derivatives -- flat or discontinuous
  region encountered

the program that I wrote is as follows
capture program drop gmm_poifd
program gmm_poifd
    version 11
    syntax varlist if, at(name) myrhs(varlist) mylhs(varlist) myidvar(varlist)
    quietly {
    tempvar mu muLag yLag
    gen double `mu' = 0 `if'
    local j = 1
    foreach var of varlist `myrhs' {
        replace `mu' = `mu' + `var'*`at'[1,`j'] `if'
        local j = `j' + 1
    }
    replace `mu' = exp(`mu')
    gen double `muLag' = L.`mu' `if'
    gen double `yLag' = L.`mylhs' `if'
    replace `varlist' = `muLag'/`mu' * `mylhs' - `yLag' `if'
}
end

gmm gmm_poifd, mylhs(PostMile1) myrhs(Traffic $Messages $Time $Weather) ///
myidvar(MessageBoardID) nequations(1) parameters(Traffic $Messages $Time $Weather) ///
instruments(Traffic $Messages $Time $Weather) onestep

Can somebody please help me identify the problem and fix this answer?


how to use unicode character groups in javascript's regexs?

Question: there is a way to use patterns like "\p{L}" in javascript, natively?
(i suppose that is a perl-compatible syntax)
I'm interested firstly in firefox support, and webkit, possibly
Answer: No, \p{..} is not supported natively by any of the big browsers. However, it does work in JavaScript if you use the XRegExp library and it's Unicode plugins.
Answer: No, Javascript has slightly different syntax. To catch unicode you have to use character selector like \uXXXX. However, on practice if your page and files in UTF-8, setting non-ASCII characters in range [абвг] does work too.
http://www.javascriptkit.com/jsref/regexp.shtml
Answer: Unfortunately, no. You can only specify a set of characters in the usual syntax, writing characters and ranges in brackets, but this becomes awkward since e.g. letters are scattered all around the Unicode space, with other characters between them.
There's an inefficient workaround: fetch the UnicodeData.txt file from the Unicode site, put its content inside your JavaScript code as data, and parse it. And then you could have the data e.g. in an array of objects containing the Unicode properties, such as gc (General Category), which tells you whether the character is a letter or not. But even then, you would just have the data handy for simple testing, not as something you can use as a constituent of a regexp.
In theory, you could use the data to construct a regexp... but it would be rather large.
Comment: ok.. this sucks. and it is unbelievable. thanks for the answer btw :)
Answer: The library found here:
http://inimino.org/~inimino/blog/javascript_cset
seems to work for me and is fairly small and independent of other libraries.


Is Android Gradle Plugin test apk builder automatically multidexing my test apk? If so, why is this generating "NoSuchMethodError"

Question: This all started with this error when running instrumentation tests:
java.lang.NoSuchMethodError: No static method closeQuietly(Ljava/net/ServerSocket;)V in class Lokhttp3/internal/Util; or its super classes (declaration of 'okhttp3.internal.Util' appears in /data/app/com.example-vKdPJoTLl49ntRbZfsRBqQ==/base.apk!classes2.dex)
at okhttp3.mockwebserver.MockWebServer$2.execute(MockWebServer.java:333)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)

I went looking into the generated test.apk to see what was present/missing in dex files.
First thing I noticed is that the test apk has two .dex files. Why? I'm not using multidex (and IIRC multidex doesn't work on test apks anyways). I then added up the two "referenced method counts" and sure enough I'm over the 65k limit. So is AGP auto-multi-dexing my test apk?

Furthermore I see the "missing" method in the first dex.
Why is it listed as a "reference" not a "defined" method? It's not like there is an OkHttp Util class provided by the framework.
In the initial crash, it said it can't find method in classes2.dex. Why is it looking in classes2.dex? Why not look in both?
I'm keeping everything in my test apk (using proguard but keeping everything) (maybe this explains the +65k ref methods). So why is this getting stripped/screwed with in the first place.
UPDATE:
It turns out this method was needed in my app .apk (not the test .apk). When I updated my proguard rules for the app apk everything worked. I still don't know why this class is needed in-app?
Answer:
java.lang.NoSuchMethodError:
No static method closeQuietly(java.net.ServerSocket) in class okhttp3.internal.Util;

method closeQuietly(java.net.ServerSocket) might not be in use.
to explicitly keep it:
-keep class okhttp3.internal.Util {
    public static void closeQuietly(java.net.ServerSocket);
}

Comment: I don't understand. Doesn't the crash I'm seeing mean the method is used? If its not in use as you suggest, how would keeping them make a difference? Why would I keep all three overloads of the `closeQuietly` method?
Comment: @tir38 the constructor with the `ServerSocket` might not have been kept by `includedescriptorclasses` and that's why my example is a rule which should explicitly keep one more constructor.
Answer: I don't know how I didn't know this but apparently multidex is enabled by default on API 21+
https://developer.android.com/studio/build/multidex#mdex-on-l

Therefore, if your minSdkVersion is 21 or higher multidex is enabled
  by default, and you do not need the multidex support library.

So yeah, this is normal. What might be happening is that the class is just not in the first dex file.


If a piece (classical or otherwise) has multiple sections with different keys, is "this piece is in key X" a valid statement? Why?

Question: Now, this would be an easy enough question if:

The piece in question had a title containing the key, or
The piece would have a home key, modulate to another key and then come back to finish in the home key.

My question is concerned with all other instances regardless of genre.
Instead of saying "This piece is in key X", is saying "this piece has multiple keys, X and Y and Z" the more correct version?
If these pieces are purposefully reduced to a single key, what may be the motivations behind it?
Comment: Maybe compounded more so by 'Symphony in X'?
Answer: This answer covers key labels and descriptions with a heavier focus on popular music styles, as opposed to classical music:
I think the handiest way to lump all of that information together is usually "starts in X" rather than "is in X", when pieces of music have multiple keys throughout. Obviously if there is one main key (jazz standards which modulate in certain sections but return back, for example "Beyond the Sea", would fit this bill), then "this piece is in X" makes sense. But this question is asking specifically about songs and music where that is not the case.
If there are multiple keys and no clear primary key with which to label the piece, it is probably best practice to use a more descriptive statement. Here are some situational examples of what I mean by that, since every song will be different:

The Beatles' song "Let It Be" is in the key of C major. (This is not up for debate.)
"Enter Sandman" by Metallica is in the key of E minor. (The parts in F# minor lead strongly back down to E minor.)
Beyonce's "Love On Top" is in C major. (Many modulations up a half-step at the end, but still primarily C major - the ending is clearly a repeated variation of the main part of the song, so C has a convincing claim to being the key of the song.)
"Layla" by Eric Clapton is in D minor, but drops everything down a half-step to C# minor in the verses. (It makes sense to say that D minor is the primary key, since it is only the verses that temporarily shift.)
Frank Sinatra's recording of "Fly Me to the Moon" can be viewed in C major or in its relative key A minor. (There is a continuous blur between the two relative keys.)
"God Only Knows" by the Beach Boys has E major competing with A major to make a weak and fluid tonal center. (Though E is likely the most viable candidate, the song weakens the key by constantly flirting with other keys such as A major and avoiding strong resolutions to the E chords.)
"Bohemian Rhapsody" by Queen begins by tonicizing the key of Bb major and modulates to various other key centers. (It is not feasible to pick a single key to label the entire song with, but since it starts in one clear key, this can be used to communicate the necessary information.)
"Giant Steps" by John Coltrane cycles between the keys of B major, Eb major, and G major. (It is useless to single out any one of the three key centers.)
Schoenberg Piano Concerto Op. 42 is atonal. (Defies any tonal logic, and no key could make a claim to represent this composition.)
The concept of a key should not be applied to "4'33"" by John Cage. (Assigning a key signature to silence is pointless.)

Some might disagree with some of the descriptions I gave, but this is fairly subjective, and my point is that each case will be different and require different methods to communicate the necessary key-related information. There are many different ways that multiple keys could be present in a piece of music, and they must be described situationally as a result. It is not always going to be possible to state unequivocally that "X key is the key of this song".

Generally, the most important purpose for giving one single key for a song is to get every musician on the same page as to what to play, and in cases with multiple keys present, as long as one key is given for one agreed-upon moment in the music, every other part of the music can be deduced with some transposition. The trouble with that often lies in two places: either it is not obvious what the key at a particular moment is, or there is no obvious moment to select as representing the key of the entire piece, or potentially both. Thus, occasionally music may require a more detailed description of its varying tonalities in order to communicate unambiguously what to play.
Comment: @MorganKendal if you're doing any sort of "music automation" I'd recommend not using "songs" as the smallest "collection of sound". Songs can be far too complicated. I'd use sections of music that combined into larger ones. It makes sense for most music to talk about the key signature of one section of music, even in extreme cases like "Giant Steps". I'd say its very rare not to be able to break music up into discrete sections with describable characteristics, but common not to be able to apply those characteristics to the larger work.
Comment: Nice list. I'd add 'Unforgettable', seems firmly in G to start, ends up in C. Written with a C key sig., though. And I can't figure out where it modulates! 4'33" - just checked, and it's written in 7#, modulating to 6b, but there's a version in key C for poor sight-readers.
Comment: I originally asked this question from a "music automation" standpoint - for automatic key detection, the standard procedure is to come up with a single key, even if the piece contains many modulations, without a clear single key label. I was mainly questioning that standard, and it seems to me that it is indeed a problem yet to be solved...
Comment: @MorganKendal Unambigious automatic key detection: Reinterpret the key as "the key signature that minimizes the number of accidentals". Not quite the same as the traditional meaning, but it has an objective metric. Less so if the music has many modulations, since you can reduce the number of accidentals by switching keys more frequently. But that's akin to compression, there is probably a way to describe an optimal number of key signature changes. It needs a 'cost' of key changes so it doesn't produce a key change for every single accidental.
Comment: And strangely, we tend to consider the last chord as representing the key, rather than the first.
Comment: @MorganKendal Oooooh, working on key detection software, huh? That kind of thing can be very tricky, since even we humans sometimes disagree on the "correct" key! Not sure I know of any good method to tackle the issue of multiple keys being detected... would it be possible to have different keys turn up similarity scores based on how well they fit the music (*C major is a 96% fit for this section of the music, and F major is a 45%fit, G minor is a 4.9% fit, et cetera...*)? Just an idea off the top of my head, I don't know if it's feasible. Best of luck on your endeavor!
Comment: @user45266 That is something being done indeed! Usually, a piece is divided into segments (which can vary in length) and these are individually investigated based on their correlation with every possible key. Trouble is, that is never the end goal with these systems, but rather a means to create a singular key label. That end goal seems very unproductive to say the least!
Answer: It's a historical convention. Generally, multi-movement pieces are named using the main key of the first movement. The convention dates back (at least) to the suites for various instruments written as the idea of "keys" being important was gaining interest. Old Gregorian Chants were often named by appending the mode to the first few words of the chant. This practice has continued.
The question is also interesting in that some chants transposed modes (and there was a difference in B and Bb) but the naming convention has stuck for at least a millennium.
Comment: There also used to be a rule that a piece *had* to end in the same key in which it started.  This included multi-movement pieces, so the last movement *had* to be in the same key as the first.  The only common exception to this rule is that pieces in a minor key would often end in the (relative or absolute) major.  This key change might not happen until the very last note in some cases (J.S. Bach did that a lot).  Other exceptions existed, but were very rare until around the 20th century or so.
Answer: Is "this piece is in key X" a valid statement?
Sometimes, at least.

Baroque-era minor-key pieces frequently end with a major chord. Obviously this stretches the limits of the question, since it can't really be said there are multiple sections in different keys, but it's worth a mention insofar as the piece does technically end in a different key that it began and is still consider to be "in minor".

Chopin Ballades #2 (in F major, Op. 38) and #4 (in F minor, Op. 52) start in one key but end in another. In Ballade #2, the first 46 measures are in F major, but the remainder of the piece (158 measures) is in A minor. Ballade #4 begins in C major (7 measures), but is predominantly in F minor (232 measures).

George Gershwin's "Impromptu in Two Keys" again stretches the question, but is worth noting. In this case, the right-hand part is in D major while the left-hand part is in Eb major. Although the piece ends with a cadence in Eb major, it would be misleading to say it's "in the key of...." This might be a case where it would be appropriate to say "This piece is in the keys of D and Eb major."
Comment: I'm certain the F major initial material returns in the middle of Chopin's Ballade No. 2, and I've always interpreted the start of Chopin's Ballade No. 4 as a giant dominant prep. What's more interesting, IMO, is that the start of Chopin's Ballade No. 1 in G Minor can  easily be interpreted as being in *A flat major*.
Comment: @Dekkadeci Yes on both counts: In Ballade #2, the F Major material has a substantial return in the middle of the piece as well as a return, transformed to A minor, at the end. And in Ballade #4 it is an extended dominant, though that isn't clear until the entrance of the main theme. Nice call on the Ab major opening of Ballade #1!
Answer: The main key is normally the opening movement and the final section.
The other movements are usually in the relative keys or mediant keys.
E.g. Beethoven's Piano concerto Cm (2nd movement in E major!)
Also the different sections in the sonata form are modulating to the dominant or relative key.
In music there is much more convention than theory based. Bach's prelude for lute (bwv 999) transcription for harpsichord begins in Cm ending in G major.
The purpose of this labeling may be to avoid endless titles in the announcement for concerts and sheet music. (Speculation!)
Answer: As a short addition to the answers: In former times, it was not uncommon for composers themselves to designate a piece as, for example, "ex C", Latin for "out of C" meaning "starting out from C". So a piece might have been called "Prelude and Fugue ex g". Thereby the composer accounts for all possible modulations ensuing.
Answer: You've already considered the case of a classical multi-movement work with a well-defined tonal 'home' key, and a piece mostly based on a minor key but with a final section like a prolonged 'Tierce de Picardie' (Tchaikovsky 5th Symphony springs to mind).    These often have a key as part of the title.   Obviously this is valid.
Otherwise, well, it's valid if it's valid.   If there's a demonstrable 'home key' it's valid to say so.   If there isn't, well, don't!
Answer: It's a convention of structure and naming, not a description of how much material is in any particular key.
Tonal music conventionally starts and ends in the same key. That key is often used in the title, like "Sonata in C Major."
In terms of how much material is in various keys through out a piece of music, even a typical classical sonata might have, very roughly speaking, only half the material in the starting/ending key. A title like "Sonata in C Major" is not at all indicating how much material is C major, or other keys.


A way to put a unique ID on a node based on the taxonomy it has?

Question: I've been tasked with building a system that assigns job numbers categorized by client. Each job number is made up of a 3 letter abbreviation of the client, the current year, and a numerical number, starting from 0 and counting up for each job created.
For example a job # would look like this: ABC1400001
The 2nd job number under client ABC would be ABC1400002 etc etc
ABC = the abbreviated client name
14 = the current year
00001 = the first job created under client ABC
I'm trying to figure out a way to do something like this with drupal where each node would be a job and each client would be a taxonomy term. Grabbing the "ABC" is easy as it could just be the taxonomy term. 14 is easy as it would just be the year. The part that is tripping me up is how to get that numerical value of 00001, 00002, 00003 etc. Then lets say I had another client named DEF the first job under that client would need to start at 00001 as well. So its not as easy as just appending a node ID on to it unfortunately.
Any ideas on how I could accomplish this?
Answer: This looks very possible with the Serial Field module in combination with Rules and Token modules
Serial Field module page reads:

This module provides an auto-increment (serial) field.
Unlike Drupal's built-in auto-increment node ID, which is global and
  shared by nodes belonging to all content types, serial fields are
  managed per content type (D6) / entity (D7). For example, the serial
  field of an Invoice instance will generate a unique sequential number
  (starting at 1, then 2, etc.) exclusively for Invoice instances.
The allocation of serial numbers by this module is atomic. In other
  words, the serial values are unique even when multiple instances of
  the same content type / entity are created simultaneously.


Post-hoc test in two-way ANOVA

Question: I am doing a two-way independent ANOVA using type of organisational change (5 levels) and job level (7 levels) within the organisation as the factors (in SPSS). I followed Andy Field's procedure (Discovering Statistics Using SPSS, third edition) and I cannot manage to get the option of Post-Hoc tests for cases where equal variance is not assumed, i.e. Games-Howell.
I searched the syntax help files and somewhere there's a note saying that when using more than one factor Games-Howell cannot be performed. However, Andy Field did it! (please see page 434 of his book). Any ideas how can I do the same?
Comment: In the options of Post-Hoc, down there are some options that you can selected as Games-Howell.
Answer: I'm not sure whether there's a better way but perhaps you can run one-way anovas on factor 1 for each category group of factor 2 separately?
You would use split files ... for this.
Try it and see whether that duplicates Andy's results.
Comment: I see what you mean, the only thing I can think of is that Andy had one factor (gender) with only two categories.

Post-hoc test in two-way ANOVA


Error "Function or interface marked as restricted, or the function uses an Automation type not supported" assigning a form object

Question: I have two forms: Form1 and Form2
Form1 creates a new instance of Form2:
Dim NewFrm as new Form2
Newfrm.show

I need to update or call a sub from NewFrm (Form2) to the calling form, Form1.
I have tried creating a Property Let in Form2 as so:
Dim CalledFromFormInstanceFrm as Form1

Public Property Let CalledFromFormInstance(oValue As Form1)
    CalledFromFormInstanceFrm = oValue
End Property

So, in Form1 I'm now doing
Dim NewFrm as New Form2
NewFrm.CalledFromFormInstanceFrm=Me
NewFrm.Show

when it get to the Public Property Let in Form, it works - but the CalledFromFormInstanceFrm is failing with an error:

Function or interface marked as restricted, or the function uses an Automation type not supported in Visual Basic'.

Googling, the error seems to point to people using other 3rd party ActiveX's not Forms in VB6.
I have also tried change the Property Let to
Public Property Let CalledFromFormInstance(oValue As Form)

and changing the declaration to
Dim CalledFromFormInstanceFrm as Form

Just in case it didn't like specifics about a Form and just accepted the Form object.
I'm stuck - though the first Property Let , allows me to use it so I think it might be allowed in VB6, I'm just not sure of the syntax I should be using - the equivalent in .net works fine but .net is a million miles away from VB6 as far as an improved language.
Any idea?
Comment: Try changing your property to `Property Set` since you are working with objects.  Likewise, use the `Set` keyword inside the property itself and on the main form.
Comment: Not addressing the direct question, but another way to 'call' back to Form1 would be to define an event on Form2, add a matching event handller to Form1 for Form2, then raise the event in Form2. The event handler does what it needs to do and can even 'return' data via the event parameters/arguments, if need be.
Answer: Set is the thing you need when dealing with object references so it should be:
Dim CalledFromFormInstanceFrm As Form1

Public Property Set CalledFromFormInstance(oValue As Form1) '//set
    Set CalledFromFormInstanceFrm = oValue '//set
End Property

Called as:
Dim NewFrm As New Form2
Set NewFrm.CalledFromFormInstance = Me '// set
NewFrm.Show

Also note that you try to use the private variable: NewFrm.CalledFromFormInstanceFrm = Me instead of the public property NewFrm.CalledFromFormInstance.
Comment: Gordon Bennett - thats worked a treat ! I forgot I have to Set everything in VB6, to be honest, I never attempted to use OOP properly in VB6, I always ended up using Form1.whateveritis without having to create it, but since discovering it .NET ive had forms popping out all over the place lol - thats a life saver - I wished I had asked this 3 hours ago, would have saved myself some time! THank you Brian and thank you Alex - both stars! :)


Assume $f$ is analytic on $\left|z-1\right|<1$,$f'(z)=1/z$ and $f(1)=0$, Prove that on this region $f(z)=\text{Log}(z)$.

Question: Assume $f$ is analytic on $\left|z-1\right|<1$,$f'(z)=1/z$ and $f(1)=0$, Prove that on this region $f(z)=\text{Log}(z)$.

Since $f$ is analytic on $\left|z-1\right|<1$, so for every $z=x+iy$ in this region we have that $u_x(x,y)=v_y(x,y)$ and $u_y(x,y)=-v_x(x,y)$, on the other hand $f'(z)=1/z$ implies that $u_x(x,y)=\frac{x}{x^2+y^2}$ and $v_x(x,y)=\frac{-y}{x^2+y^2}$, So $u(x,y)=1/2 \ln(x^2+y^2)+c_1(y)$, Moreover $$\frac{y}{x^{2}+y^{2}}+c_1'\left(y\right)=\frac{y}{x^{2}+y^{2}}$$
From which we conclude that $c_1(y)=c$
$$v\left(x,y\right)=\arctan\left(\frac{y}{x}\right)+c_{2}\left(y\right)$$
$$\frac{1}{\sqrt{x^{2}+y^{2}}}+c'_{2}\left(y\right)=\frac{x}{x^{2}+y^{2}}$$
Is this the correct way to go?
Comment: After you ask a question here, if you get an acceptable answer, you should "accept" the answer by clicking the check mark $\checkmark$ next to it. This scores points for you and for the person who answered your question. You can find out more about accepting answers here: [How do I accept an answer?](http://meta.math.stackexchange.com/questions/3286/), [Why should we accept answers?](http://meta.math.stackexchange.com/questions/3399/), [What should I do if someone answers my question?](https://math.stackexchange.com/help/someone-answers).
Answer: $u_x=\dfrac{x}{x^2+y^2}$ implies $u(x,y)=\dfrac{1}{2}\ln(x^2+y^2)+c(y)$
whereas $u_y=\dfrac{y}{x^2+y^2}$ implies $u(x,y)=\dfrac{1}{2}\ln(x^2+y^2)+l(x)$
which forces $c(y)$ and $l(x)$ to be equal to the same constant function.
Also,
$v_y=\dfrac{x}{x^2+y^2}$ implies $v(x,y)=\arctan\left(\dfrac{y}{x}\right)+g(x)$
whereas $v_x=\dfrac{-y}{x^2+y^2}$ implies $v(x,y)=\arctan\left(\dfrac{y}{x}\right)+h(y)$, which you can verify by partial-differentiating $\arctan\left(\dfrac{y}{x}\right)$ with respect to $x$.
Comment: Why $v(x,y)=\arctan\left(\dfrac{y}{x}\right)+g\left(y\right)=\arctan\left(\dfrac{y}{x}\right)+h(x)$ and where does that $h(y)$ come from?
Comment: You say "using the formulae for $u_x$ and for $u_y$", What formulae do you mean?
Comment: @user1040538 answer edited
Comment: So $f(z)=1/2 \ln(x^2+y^2)+i\arctan\left(\frac{y}{x}\right)$, But the question asks to show that $f=\text{ Log}(z)$
Comment: @user1040538 that is $Log(z)$. see https://math.stackexchange.com/questions/2260028/find-the-real-and-imaginary-parts-of-lnz
Answer: For $z \in D_1(1) = \{z \in \mathbb{C} : |z - 1| < 1\}$, let $g(z) = f(z) - \log(z)$. Since $\log'(z) = 1/z$ and $\log(1) = 0$, we have $g'(z) = 0$ for all $z \in D_1(1)$ and $g(0) = 0$. Since $D_1(1)$ is connected, this implies that $g = 0$ on $D_1(1)$.


My navigation drawer doesn't show up and i get an Error inflating class fragment

Question: I am trying to implement a navigation drawer, i have an issue where the page loads and the hamburger icon shows up. When i click the icon however, it changes into an arrow like it always does, but my drawer does not show up. and when i try to force one of the fragments to show up by default i get an Error inflating class fragment error.
Below is my page layout that holds the drawer
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    tools:context=".app.index">

    <include layout="@layout/toolbar">
    </include>

    <android.support.v4.widget.DrawerLayout
        xmlns:android="http://schemas.android.com/apk/res/android"
        android:id="@+id/drawer_layout"
        android:layout_width="match_parent"
        android:layout_height="match_parent">

        <FrameLayout
            android:id="@+id/content_frame"
            android:layout_width="match_parent"
            android:layout_height="match_parent" />

        <fragment
            android:id="@+id/nav_drwr_fragment"
            class="com.mobile.map.mapversion20.helpers.NavDrawerManager"
            android:layout_width="280dp"
            android:layout_height="match_parent"
            android:layout_gravity="start"
            app:layout="@layout/nav_drawer_layout"
            tools:layout="@layout/nav_drawer_layout"/>

    </android.support.v4.widget.DrawerLayout>

</LinearLayout>

Below is my drawer layout
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="match_parent"
    android:layout_height="match_parent">
    <RelativeLayout
        android:id="@+id/nav_header_container"
        android:layout_width="match_parent"
        android:layout_height="160dp"
        android:background="@color/colorPrimary">

        <ImageView
            android:layout_width="70dp"
            android:layout_height="70dp"
            android:layout_centerInParent="true"
            android:scaleType="fitCenter"
            android:src="@drawable/ic_profile"/>

    </RelativeLayout>

    <android.support.v7.widget.RecyclerView
        android:id="@+id/drawerList"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_marginTop="16dp"/>
</LinearLayout>

Down here is my Fragment manager
public class NavDrawerManager extends Fragment
{
    private ActionBarDrawerToggle mDrawerToggle;
    private DrawerLayout mDrawerLayout;
    private static boolean isLaunch = true;

    @Override
    public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState)
    {
        View view = inflater.inflate(R.layout.nav_drawer_layout, container, false);

        setUpRecyclerView(view);

        return view;
    }
    private void setUpRecyclerView(View view) {

        RecyclerView recyclerView = (RecyclerView) view.findViewById(R.id.drawerList);

        NavDrawerAdapter adapter = new NavDrawerAdapter(getActivity(), drawerItem.getData());
        NavDrawerManager manager = new NavDrawerManager();
        recyclerView.setAdapter(adapter);
        recyclerView.setLayoutManager(new LinearLayoutManager(getActivity()));

        if(isLaunch)
        {
            isLaunch = false;
            manager.selectItem("DashBoard");
        }
    }

    public void setUpDrawer(int fragmentId, DrawerLayout drawerLayout, Toolbar toolbar) {
        mDrawerLayout = drawerLayout;
        mDrawerToggle = new ActionBarDrawerToggle(getActivity(), drawerLayout, toolbar, R.string.drawer_open, R.string.drawer_close) {
            @Override
            public void onDrawerOpened(View drawerView) {
                super.onDrawerOpened(drawerView);
//                getActivity().invalidateOptionsMenu();
            }

            @Override
            public void onDrawerClosed(View drawerView) {
                super.onDrawerClosed(drawerView);
//                getActivity().invalidateOptionsMenu();
            }

            @Override
            public void onDrawerSlide(View drawerView, float slideOffset) {
                super.onDrawerSlide(drawerView, slideOffset);
                // Do something of Slide of Drawer
            }
        };

        mDrawerLayout.setDrawerListener(mDrawerToggle);

        mDrawerLayout.post(new Runnable() {
            @Override
            public void run() {
                mDrawerToggle.syncState();
            }
        });
    }
    public void selectItem(String title)
    {
        FragmentManager fragmentManager = getChildFragmentManager();

        switch (title)
        {
            case "Dashboard":
                Fragment dashFragment = new dashboard();
                        fragmentManager.beginTransaction()
                        .replace(R.id.content_frame,dashFragment)
                        .commit();
                break;
            case "Profile":
                Fragment profileFragment = new profile();
                fragmentManager.beginTransaction()
                        .replace(R.id.content_frame,profileFragment)
                        .commit();
                break;
            case "Notifications":
                Fragment notificationsFragment = new notifications();
                fragmentManager.beginTransaction()
                        .replace(R.id.content_frame,notificationsFragment)
                        .commit();
                break;
            case "Contact":
                Fragment contactFragment = new contact();
                fragmentManager.beginTransaction()
                        .replace(R.id.content_frame,contactFragment)
                        .commit();
                break;
            default:
                Fragment defaultFrag = new dashboard();
                fragmentManager.beginTransaction()
                        .replace(R.id.content_frame,defaultFrag)
                        .commit();
                break;
        }
    }
}

Comment: can you post the error logcat here ?
Answer: The issue is you need to make the DrawerLayout as the parent of you layout file like below :
<?xml version="1.0" encoding="utf-8"?>
<android.support.v4.widget.DrawerLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:id="@+id/drawer_layout"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:orientation="vertical">

        <include layout="@layout/toolbar"/>

       <FrameLayout
           android:id="@+id/content_frame"
           android:layout_width="match_parent"
           android:layout_height="match_parent" />

    </LinearLayout>

    <fragment
        android:id="@+id/nav_drwr_fragment"
        class="com.mobile.map.mapversion20.helpers.NavDrawerManager"
        android:layout_width="280dp"
        android:layout_height="match_parent"
        android:layout_gravity="start"
        app:layout="@layout/nav_drawer_layout"
        tools:layout="@layout/nav_drawer_layout"/>

</android.support.v4.widget.DrawerLayout>


JavaScript new Date(dateStr) Giving Yesterday's Date in Certain Formats

Question: tl;dr - When I try to create a new Date object with a YYYY-MM-DD format date string, it gives me an incorrect date (yesterday). Why?
I've written the following test code to help me demonstrate the problem I'm perceiving:
var dateConfig = {weekday: "long", year: "numeric", month: "long", day: "numeric"},
    dates = [
        "01/21/2014",
        "01-21-2014",
        "2014/01/21",
        "2014-01-21"
    ];

for (var i = 0; i < dates.length; ++ i) {
    var date = new Date(dates[i]);
    console.log(date.toLocaleDateString("en-US", dateConfig));
}

Link to see for yourself: http://s.codepen.io/AdrianTP/fullpage/prKyf
Chrome 31.0.1650.63 m returns the following in the console:
Tuesday, January 21, 2014
Tuesday, January 21, 2014
Tuesday, January 21, 2014
Monday, January 20, 2014

Firefox 26.0 returns the following in the console:
"Tuesday, January 21, 2014"
"Invalid Date"
"Tuesday, January 21, 2014"
"Monday, January 20, 2014"

Even Internet Explorer 8 gets most of it right, returning the following in the console:
"Tuesday, January 21, 2014"
"Tuesday, January 21, 2014"
"Tuesday, January 21, 2014"
"NaN"

In short, I am aware that date handling between browsers is inconsistent (dates[2] in Chrome and Firefox differ, and dates[3] just outright breaks in IE 8), but that is not my question.
My question is:

Why would Chrome and Firefox return yesterday's date for a YYYY-MM-DD
  formatted date string specifying today's date, when it works fine with
  slashes?

Another question:

Is this a known issue?

I have not encountered it before, and was unable to find any documentation of the issue, nor documentation of the Date() object which would indicate to me that such string-transformation would occur so regularly-irregular. Does anyone out there have experience with this and maybe an explanation or a link to one that I haven't found? I could just be using the wrong search terms here...
Comment: other related topics:
https://stackoverflow.com/questions/35448343/why-isnt-2016-02-16-equal-to-2016-02-16-0000
https://stackoverflow.com/questions/2587345/why-does-date-parse-give-incorrect-results
Answer: The JavaScript standard stipulates that the "official" supported format is ISO 8601, which looks like YYYY-MM-DDTHH:mm:ss.sssZ. (Z    is the time zone offset specified as "Z" (for UTC) or either "+" or "-" followed by a time expression HH:mm.) Though the format does include a time zone offset, Firefox and Chrome (and I think IE) don't pay attention, and always interpret ISO 8601 dates as UTC.
Any implementation can accept other formats too, and the browsers aren't in sync at present. Firefox will accept RFC 2822 dates, other browsers don't.
Comment: Thanks for the ISO/RFC numbers. They spurred additional research on my part, which brought me to an understanding that it's assuming I'm entering the date in the expanded ISO-8601 format, which means when I enter `new Date("2014-01-23")` I'm actually entering `new Date("2014-01-23T00:00:00")` which gets converted to my computer's time zone (GMT-0600), thus causing me to actually be referring to yesterday's date. `2014-01-23T00:00:00` in GMT is equivalent to `2014-01-22T18:00:00` in CST (GMT-0600). Because no time zone was specified for the date string I entered, it assumed GMT.

Thank you!
Comment: @Adrian yup, that's exactly right!  I should have added that info but I was in a hurry when I was answering that :)
Comment: @Adrian—I think you assume too much. ;-) ECMA-262 as of ed 5.1 (2011) says that "2014-01-23" should be treated as UTC (contrary to ISO 8601), I think all the majors at least do that now. However, 2014-01-01T00:00:00 should be treated as local (except Safari treats it as UTC). Just don't use the built–in parser, it will get you into trouble.


Jinja2: how to check if input is hidden

Question: I have the following stardard render code:
{% macro render_field(field) %}

            {% if not field.hidden %}
                {{ field.label }}
            {% endif %}
            {{ field(**kwargs)|safe }}

{% endmacro %}

And for hidden field I do not want label to appear.
But the HTML result I got is:

            Foo   <-- why this did not go?
            <input id="foo" name="foo" type="hidden">

Am I wrong saying " if not field.hidden ", what is the right way of checking this?
And I am not talking here about CSRF token.
Comment: No, it's not about that special CSRF hidden field
Comment: The accepted answer contains a solution for all hidden fields, not just CSRF.
Answer: WTForm
foo = HiddenField('email',[])
on HTML page
Use field.foo.hidden
Comment: field.email.hidden does not seem to work I am afraid

Jinja2: how to check if input is hidden


Pygame Xbox One Controller

Question: I am trying to run some code that will allow the user to control with an Xbox Controller. I have it working with the Xbox 360 controller using Pygame. Then when I try to use the Xbox one controller and it is able to read the as "connected" but it doesn't read the actual button presses.
I tried running the joystick analyzer found on the Pygame website and it shows it connected again but no button input is taken.

The code for this can be found at the bottom of this documentation page:
https://www.pygame.org/docs/ref/joystick.html
<code>import pygame

# Define some colors.
BLACK = pygame.Color('black')
WHITE = pygame.Color('white')

# This is a simple class that will help us print to the screen.
# It has nothing to do with the joysticks, just outputting the
# information.
class TextPrint(object):
    def __init__(self):
        self.reset()
        self.font = pygame.font.Font(None, 20)

    def tprint(self, screen, textString):
        textBitmap = self.font.render(textString, True, BLACK)
        screen.blit(textBitmap, (self.x, self.y))
        self.y += self.line_height

    def reset(self):
        self.x = 10
        self.y = 10
        self.line_height = 15

    def indent(self):
        self.x += 10

    def unindent(self):
        self.x -= 10

pygame.init()

# Set the width and height of the screen (width, height).
screen = pygame.display.set_mode((500, 700))

pygame.display.set_caption("My Game")

# Loop until the user clicks the close button.
done = False

# Used to manage how fast the screen updates.
clock = pygame.time.Clock()

# Initialize the joysticks.
pygame.joystick.init()

# Get ready to print.
textPrint = TextPrint()

# -------- Main Program Loop -----------
while not done:
    #
    # EVENT PROCESSING STEP
    #
    # Possible joystick actions: JOYAXISMOTION, JOYBALLMOTION, JOYBUTTONDOWN,
    # JOYBUTTONUP, JOYHATMOTION
    for event in pygame.event.get(): # User did something.
        if event.type == pygame.QUIT: # If user clicked close.
            done = True # Flag that we are done so we exit this loop.
        elif event.type == pygame.JOYBUTTONDOWN:
            print("Joystick button pressed.")
        elif event.type == pygame.JOYBUTTONUP:
            print("Joystick button released.")

    #
    # DRAWING STEP
    #
    # First, clear the screen to white. Don't put other drawing commands
    # above this, or they will be erased with this command.
    screen.fill(WHITE)
    textPrint.reset()

    # Get count of joysticks.
    joystick_count = pygame.joystick.get_count()

    textPrint.tprint(screen, "Number of joysticks: {}".format(joystick_count))
    textPrint.indent()

    # For each joystick:
    for i in range(joystick_count):
        joystick = pygame.joystick.Joystick(i)
        joystick.init()

        try:
            jid = joystick.get_instance_id()
        except AttributeError:
            # get_instance_id() is an SDL2 method
            jid = joystick.get_id()
        textPrint.tprint(screen, "Joystick {}".format(jid))
        textPrint.indent()

        # Get the name from the OS for the controller/joystick.
        name = joystick.get_name()
        textPrint.tprint(screen, "Joystick name: {}".format(name))

        try:
            guid = joystick.get_guid()
        except AttributeError:
            # get_guid() is an SDL2 method
            pass
        else:
            textPrint.tprint(screen, "GUID: {}".format(guid))

        # Usually axis run in pairs, up/down for one, and left/right for
        # the other.
        axes = joystick.get_numaxes()
        textPrint.tprint(screen, "Number of axes: {}".format(axes))
        textPrint.indent()

        for i in range(axes):
            axis = joystick.get_axis(i)
            textPrint.tprint(screen, "Axis {} value: {:>6.3f}".format(i, axis))
        textPrint.unindent()

        buttons = joystick.get_numbuttons()
        textPrint.tprint(screen, "Number of buttons: {}".format(buttons))
        textPrint.indent()

        for i in range(buttons):
            button = joystick.get_button(i)
            textPrint.tprint(screen,
                             "Button {:>2} value: {}".format(i, button))
        textPrint.unindent()

        hats = joystick.get_numhats()
        textPrint.tprint(screen, "Number of hats: {}".format(hats))
        textPrint.indent()

        # Hat position. All or nothing for direction, not a float like
        # get_axis(). Position is a tuple of int values (x, y).
        for i in range(hats):
            hat = joystick.get_hat(i)
            textPrint.tprint(screen, "Hat {} value: {}".format(i, str(hat)))
        textPrint.unindent()

        textPrint.unindent()

    #
    # ALL CODE TO DRAW SHOULD GO ABOVE THIS COMMENT
    #

    # Go ahead and update the screen with what we've drawn.
    pygame.display.flip()

    # Limit to 20 frames per second.
    clock.tick(20)

# Close the window and quit.
# If you forget this line, the program will 'hang'
# on exit if running from IDLE.
pygame.quit()
</code>
Does anyone have any insight into why this is?
Comment: have you installed the controller drivers for x-box one?
Comment: @PythonProgrammer This isn't a pressing issue for me anymore. But in case others are interested in getting a solution I added the code. The code was just the example code from the link I provided previously.
Comment: @Pygasm This is the code found on the pygame documentation page. https://www.pygame.org/docs/ref/joystick.html. I thought I would start there because this test code has always worked for my Xbox 360 controller but now won't work for the Xbox One.
Comment: Well to me it seems that the joystick code was designed not for an xbox one controller, most likely pygame is trying to read from gates or pins that arent exactly there anymore and ends up reading the wrong input, you could try reading from serial manually and detecting the output of the controller, although that may a bit advanced for you.
Comment: @Pygasm That is what I was thinking too. The strange thing is that there are other posts about specific things with Xbox One controllers and Pygame out there. For Example: https://stackoverflow.com/questions/28132156/how-can-i-set-the-range-for-xbox-one-controller-triggers-using-pygame So it seems some people are getting it to work out there.
Comment: @Rjbeckwith From the question you mentioned I managed to track down the code he used to get this working at https://github.com/ryanvade/robotproject/blob/master/xboneControllerTest.py. I can't see what changes he has made for his to work and yours to not though. Try running his code on pygame 1.9.2 and see what you get???
Comment: @Rjbeckwith I have used the Xbox One Controller in pygame many times. Can you display the code?
Comment: 1) Does it work in any other context?
2) Is it USB or BT?
3) In what order do you connect the controller and start the program?
Comment: in [that code](https://gitlab.com/nicolalandro/tello_code_sample/-/blob/master/joystik_controller/joystick_and_video.py) the Xbox One joystick work, the class used is JoystickXONE at line 147, maybe it can be usefull.
Comment: Check out this answer - https://stackoverflow.com/a/53991551/2570277
Comment: For older linux kernel versions I had to use xboxdrv https://github.com/xboxdrv/xboxdrv
Comment: have you installed pygame2 ?
Answer: Bluetooth may not be enabled yet try enabling it or checking the controllers batteries!
Answer: I have tried this same script with an Xbox One Series 2 Controller and can confirm that it works flawlessly. I connected the controller to the PC with the charging cable, not wirelessly (because I have an Xbox in the same room).
You should try the same first and be able to open the Xbox Game Bar by holding down the  button, otherwise check your Xbox settings on the PC and update your drivers from the device manager (while wired).
Also, make sure that you have the last Pygame version, I tried with 2.1.2 on Windows 10.
Hope this give you a hint.
Answer: try running a controller testing tool if it doesn't work there then it is probably a problem with the controller also try reinstalling pygame and checking if you have drivers  correctly installed

Pygame Xbox One Controller


Apply multiple digital Signatures on PDF

Question: I have a project and i have to digitally sign a PDF more than once. I've tried to do this with TCPDF, but it seems that TCPDF allows only one Signature! because when I import the already signed PDF (with FPDI) and sign it, I have only the second sig. and the appearance (image) of the first one.
is that right?? or It's because FPDI didn't import the embedded signature?? and does TCPDF support incremental update??

is there a way that i can sign the same PDF twice with TCPDF??
or another Library that allows multiple signing with incremental updates??

Please any helpful Ideas or recommandations!
Best regards,
Answer: With FPDI you cannot edit an existing PDF document but you import the individual pages into a reusable structure. Content outside of a pages content stream (such as annotations or digital signatures) get lost.
Setasign (I'm related to that company), the authors of FPDI, also offers the SetaPDF-Signer component, which allows you to digital sign PDFs in pure PHP. The component is not free or open-source as FPDI.
See here for how to sign a PDF document several times.
Comment: Hi. what is your competitor that is free?


A problem about equilateral triangle

Question: In below equilateral triangle $ABC$, $AD=CE$, $DH=GH$. Prove that $BE=BG$
My thoughts: It looksl ike we need to prove that $ABGD$ are on the same circle but I am not sure how to use the condition $DH=GH$
Answer: I'll prove it the other way around.
Suppose we have $AD = CD$. We find the point $G$ on the line $AE$ such that $BE = BG$, and connect $D$ and $G$. Let $H$ be the intersection of $DG$ and $CF$. I'm going to show that $DH = GH$.
Since $\angle BGE = \angle BEG = \angle AEC = \angle ADB$, we have $\angle GBF = \angle FAD = \angle ABF$, which means that $BF$ is the bisector of $\angle ABG$. Therefore we have $\frac{AB}{BG} = \frac{AF}{FG}$.
But we have also $AB = AC$ and $BG = BE = CD$, and Menelaus theorem w.r.t. the triangle $AGD$ and the line $FHC$ says $\frac{AC}{CD} \cdot \frac{DH}{HG} \cdot \frac{GF}{FA} = 1$.
This then gives $DH = GH$.
Comment: The trick is that, there is a unique $G$ such that $DH = GH$, again by Menelaus. It then follows from the proof that the $G$ such that $BE = BG$ is exactly the (unique) $G$ such that $DH = GH$. Perhaps I should have used a different notation $G'$...
Comment: Impressive!  But proving from $DH=GH$ from $BE=BG$  is different from the other way around? I mean $A \Rightarrow B$ is different from $B \Rightarrow A$  unless $A \Leftrightarrow B$?
Answer: Let $A,B,C$ be vectors from $O(0,0)$ to $A,B,C$ resp, let $a=A-C,b=B-C$. And let $C$ be $(0,0)$ then.
Let's write all we have:
$D=(1-t)a, E=tb$,
$F$:$F=AE\cap BD$, $AE:ua+(1-u)tb$, $BD:vb+(1-v)(1-t)a$
$$\begin{cases}
u=(1-t)(1-v)\\\
(1-u)t=v
\end{cases}$$
$$u=(1-t)(1-t+ut)$$
$$u-t(1-t)u=(1-t)^2$$
$$u = \frac{(t - 1)^2}{t^2 - t + 1}, v = \frac{t^2}{t^2 - t + 1}$$
$$F=\frac{(t - 1)^2}{t^2 - t + 1}a+\frac{t^2}{t^2 - t + 1}b$$
$H=\frac{w}2F$
$G=xA+(1-x)E=xa+(1-x)tb$, and, finally $DH=HG$:
$2H=D+G=(1-t+x)a+(1-x)tb$
$$\begin{cases}
1-t+x=w\frac{(t - 1)^2}{t^2 - t + 1}\\\
(1-x)t=w\frac{t^2}{t^2 - t + 1}
\end{cases}$$
$$w = 2-t,x=\frac{2 t^2 - 3 t + 1}{t^2 - t + 1},
1-x=-\frac{t (t - 2)}{t^2 - t + 1}$$
$$\hbox{Thus }G=xa+(1-x)tb=\frac{2 t^2 - 3 t + 1}{t^2 - t + 1}a-\frac{t^2 (t - 2)}{t^2 - t + 1}b,$$
$$BE=E-B=(1-t)b, BG=G-B=\frac{2 t^2 - 3 t + 1}{t^2 - t + 1}a-\frac{(t - 1)^2 (t + 1)}{t^2 - t + 1}b$$
$$BG^2=(k_1a+k_2b)^2=k_1^2a^2+2k_1k_2ab+k_2^2b^2,$$
then recall $a^2=b^2$, $ab=|a|\cdot|b|\cos\angle ACB=\frac12a^2,$ thus $BG^2=\left(k_1^2+k_1k_2+k_2^2\right)a^2$ $=(1-t)^2a^2$ $=BE^2$, QED.

A problem about equilateral triangle


Sum of the series formula

Question: I need to figure out the sum of the series as quickly as possible in a program given n and k: $$f(n,k)= \frac{\sum_{i=0}^{n-1}f(i,k-1)}{n} $$ and $ f(i,0) =i $ for all $i$.
I did some paper work and brought the solution to $$f(n,k)=\frac{f(n-1,k-1)}{n} + \frac{(n-1)f(n-1,k)}{n}  $$ But I need a closed form solution. Is it possible to give results in O(1) or O( lg n) time (preprocessing is allowed but should not be of the order of O(n^2 ) or O( k^2)).
Answer: The solution by Marvis would be really nice if it worked, but the formula for $f(n,1)$ fails when $n=0$, and this propagates to subsequent computation. By the way: although this is implicit in the recursive formula, it would help to specify $f(0,k)=0$ explicitly in the statement of the problem.
Let's consider only $n\ge 1$ from now on. Then $f(n,1) = \frac12(n-1)$ holds, and
$$f(n,2) = \frac{1}{2n} \sum_{i=1}^{n-1}(i-1) =\frac{1}{2n} \left\{\frac{n(n-1)}{2}-n+1\right\}
= \frac{n-3}{4} +\frac{1}{2n} \tag1$$
Using the notation $H_n=1+\frac12+\dots+\frac1{n}$ for harmonic numbers, we find
$$\begin{align}f(n,3) & = \frac{1}{4n} \sum_{i=1}^{n-1}(i-3) + \frac{1}{2n}\sum_{i=1}^{n-1} \frac{1}{i} = \frac{1}{4n} \left\{ \frac{n(n-1)}{2} -3n+3\right\}+\frac{H_{n-1}}{2n}
\\ &= \frac{n-7}{8} +\frac{3}{4n}+\frac{H_{n-1}}{2n} \end{align} \tag2$$
This brings an unpleasant realization: if we could compute $f(n,3)$ in $O(1)$, then the same could be done for the harmonic numbers. I looked around for algorithms for computing $H_n$ and found  this discussion. Did not read carefully, but maybe it can be optimized to $O(\lg n)$.
Moving forward, we encounter the sums of $H_{n-1}/n$. A superficially(?) similar concept is hyperharmonic numbers, defined by $H_n^{(1)}=H_n$ and  $H_n^{(r)}=\sum_{i=1}^n H_n^{(r-1)}$ when $r\ge 2$. The formula by Conway and Guy (e.g., here) tells us that $r$ adds very little complexity:
$$H_n^{(r)} = \binom{n+r-1}{r-1}(H_{n+r-1}-H_{r-1}) \tag3$$
Unfortunately, $n$ in the denominator of $H_{n-1}/n$ does not allow the hyperharmonic numbers to appear, as far as I can tell.
$$\begin{align}f(n,4) & = \frac{1}{8n} \sum_{i=1}^{n-1}(i-7) + \frac{3}{4n}H_{n-1} + \frac{1}{2n}\sum_{i=1}^{n-1}\frac{H_{i-1}}{i}
\\ &= \frac{n-15}{16} +\frac{7}{8n} + \frac{3}{4n}H_{n-1} + \frac{1}{2n}\sum_{i=1}^{n-1}\frac{H_{i-1}}{i}  \end{align} \tag4$$
In general, if we define $$H_n^{[r]}=\sum_{i=1}^{n-1}\frac{H_{i}^{[r-1]}}{i},\quad H_n^{[0]}=1 \tag5$$
then
$$f(n,k) = \frac{n-(2^k-1)}{2^k} +\sum_{r=0}^{k-2} \frac{2^{k-r-1}-1}{2^{k-r-1}n}H_n^{[r]}    \tag6$$
where the corrector terms $H_n^{[r]}$ get increasingly expensive to compute.
Answer: This solution is wrong.
$$f(n,1) = \dfrac{\sum_{i=0}^{n-1} f(i,0)}n = \dfrac{\sum_{i=0}^{n-1} i}n = \dfrac{n(n-1)}{2n} = \dfrac{n-1}2$$
$$f(n,2) = \dfrac{\sum_{i=0}^{n-1} f(i,1)}n = \dfrac{\sum_{i=0}^{n-1} (i-1)}{2n} = \dfrac{n(n-1)/2 - n}{2n} = \dfrac{(n-1)/2-1}2 = \dfrac{n-3}4$$
$$f(n,3) = \dfrac{\sum_{i=0}^{n-1} f(i,2)}n = \dfrac{\sum_{i=0}^{n-1} (i-3)}{4n} = \dfrac{n(n-1)/2 - 3n}{4n} = \dfrac{(n-1)/2-3}4 = \dfrac{n-7}8$$
Hence, in general, we can guess
$$f(n,k) = \dfrac{n-(2^k-1)}{2^k} = -1 + \dfrac{n+1}{2^k}$$
Once we have this prove that our guess is indeed true by induction.
Comment: Your formula is incorrect. f(n,1) is ofcourse correct but not the others. I think this is because you have taken the value of (i-1) as -1 when i=0. But its should be 0. Taking that into account i have got the formula for f(n,2) as (n-1)(n-2)/4n but i cant go any further.
Comment: For eg. in f(4,2) that answer should be 3/8 but your solution gives 1/4

Sum of the series formula


Finding the standard matrix of linear operator

Question: This is similar to one of my other questions.
Question:
Let $T$ be a linear operator in $\mathbb{R}^4\rightarrow\mathbb{R}^4$ such that
$T\begin{pmatrix}2\\1\\0\\0\end{pmatrix} = \begin{pmatrix}2\\1\\0\\0\end{pmatrix},T\begin{pmatrix}3\\1\\0\\0\end{pmatrix} = 0,T\begin{pmatrix}1\\2\\1\\1\end{pmatrix} = \begin{pmatrix}1\\2\\1\\1\end{pmatrix},T\begin{pmatrix}2\\0\\-1\\-2\end{pmatrix} = \begin{pmatrix}-2\\0\\1\\2\end{pmatrix}$
Find the standard matrix of the linear operator T
I'm also not sure where to start with this one, any help appreciated. TIA.
Answer: Hint: The information you are given is the same as saying that $(2,1,0,0)^\top, (3,1,0,0)^\top, (1,2,1,1)^\top$ and $(2, 0, -1, -2)^\top$ are eigenvectors of $T$ with eigenvalues $1, 0, 1$ and $-1$ respectively. Then the standard matrix representation of $T$ is similar to the diagonal matrix $D = \text{diag}(1, 0, 1, -1)$.
Comment: Thank you. Is there a way to answer this without the use of eigenvectors and eigenvalues?
Comment: Definitely, but I think that is the most straight-forward way here. How much do you know about change of bases?
Comment: Not a lot I don't think...
Comment: Well, then this will perhaps be a lot of work, but what you can do is to set up a system of equations so that you based on the information you have, can determine what $T$ does to the standard basis vectors for $\mathbb{R}^4$ and then the matrix you are looking for has these vectors as its columns. To me this looks like a strange exercise to give if you have neither worked with diagonalization and eigenvectors nor change of bases.


Drupal module for download limit

Question: Is there any Drupal 7 module there, that can restrict any user, based on role/privilege, number of downloads per day. For example, 'x' type of user can download 10 files/day.
Comment: I haven't seen a module that does that (doesn't mean there isn't one of course). If writing your own is an option, it wouldn't be that much work. [`hook_file_download()`](https://api.drupal.org/api/drupal/modules!system!system.api.php/function/hook_file_download/7) is the key,  check out [Commerce File](https://drupal.org/project/commerce_file) as a starting point for the code, it has everything you need
Answer: I don't think such module exist. But you can write your own module.  Download Count may help you while writing your custom module.

Drupal module for download limit


Netcool Object Server api

Question: Running Netcool 7.3.1.  Looking for simple api to access Object Server Tables.  I've already done the run an SQL command from nco_sql, and scraped the output into a C# data table, but wondering if there was some type of api that I could use for either C# or Java to access table data?
Answer: If you can use a more up-to-date version of Omnibus, you can use the built-in HTTP / REST API.
http://www-01.ibm.com/support/knowledgecenter/SSSHTQ_8.1.0/com.ibm.netcool_OMNIbus.doc_8.1.0/omnibus/wip/api/concept/omn_api_http_overview.html?lang=en
Answer: You may need to use sybase database adapter so far I have used below three ways to query netcool object server:

Free TDS - This is free sybase client.
Jconn3 - this is paid sybase client, but if you are using WebGUI/Impact, this driver comes with TIP.
nco_sql - here you may need to create a file with query and then pass it to nco_sql. This require extra effort to parse column wise information as output will be on console.

I prefer jconn3, simple and similar to jdbc driver, you only need this jar in classpath.
Answer: You can write your own java program to connect to Objectserver by simply initiating
<code>//Load Sybase Driver
Class.forName("com.sybase.jdbc3.jdbc.SybDriver");
String url = "jdbc:sybase:Tds:" + host + ":" + port;
con = DriverManager.getConnection(url, user, pass);
</code>
Execute Statements
<code>Statement stat = conn.createStatement();
ResultSet result = stat.executeQuery("Select count(*) from alerts.status");;`
</code>

Netcool Object Server api


Como funcionam os operadores de bit?

Question: Eu tenho o seguinte código:
volatile long Hex_To_int(long Hex , char bits)
{

  long Hex_2_int;
  char byte;

  Hex_2_int = 0 ;

  for(byte = 0 ; byte < bits ; byte++)
  {

     if(Hex& (0x0001 << byte)) Hex_2_int += 1*(pow(2,byte));

     else Hex_2_int  += 0*(pow(2,byte));

  }

  return Hex_2_int;
}

Como é que funciona o if com operadores de bit nessa seguinte comparação?
if(Hex& (0x0001 << byte))

Comment: Repara que somar `0` (ou `0 * qualquer coisa`) é o mesmo que não somar: podes remover o `else` do código :-)
Comment: O `if` funciona igual, não importa o que tem nele, você quer saber como o operador de *shift* funciona? Seja mais específico.
Comment: Quero saber o contexto da operação . oque esta fazendo
Comment: Você pode aceitar uma resposta se ela resolveu seu problema. Você pode votar em todos os posts do site também. Alguma lhe ajudou mais? Precisa que algo seja melhorado?
Answer: O código é estranho e faz coisas desnecessárias.
A operação de shift (operador <<) desloca os bits para a esquerda. Como a posição determina o número representado, como no decimal, cada bit deslocado para a esquerda multiplica por 2 (a base do binário) o valor existente (assim como deslocar um dígito para a esquerda no decimal multiplica por 10, a base do decimal). Neste caso cada passada pelo laço produzirá 1, 2, 4, 8, etc.
O resultado será calculando com a variável Hex fazendo uma operação de and (operador &). O and funciona de forma semelhante à multiplicação, ou seja, cada bit que um operando é calculado com o bite correspondente do outro operando. O resultado será 1 sempre que ambos sejam 1 e 0 quando não houver esta coincidência.
O resultado final sendo um valor diferente de zero executa o if, já que, em C, isto é considerado verdadeiro. Se o resultado for 0, executa o else.
Saiba mais aqui. A linguagem é diferente mas o funcionamento é o mesmo.
Comment: Ya ... simplificando fica: `unsigned long Hex_To_int(unsigned long Hex, char bits) { return Hex & ((1 << bits) - 1); }`

Como funcionam os operadores de bit?


Variance estimates in k-fold cross-validation

Question: K-fold cross-validation can be used to estimate the generalization capability of a given classifier. Can I (or should I) also compute a pooled variance from all validation runs in order to obtain a better estimate of its variance?
If not, why?
I have found papers which do use the pooled standard deviation across cross-validation runs. I have also found papers explicitly stating there is no universal estimator for the validation variance. However, I have also found papers showing some variance estimators for the generalization error (I am still reading and trying to comprehend this one). What do people really do (or report) in practice?
EDIT: When CV is used to measure the crude classification error (i.e. either a sample has been labeled correctly or it hasn't; e.g. true or false) then it may not make sense to talk about a pooled variance. However, I am talking about the case in which the statistic we are estimating does have a variance defined. So, for a given fold, we can end up with both a value for the statistic and a variance estimate. It does not seems right to discard this information and consider only the average statistic. And while I am aware I could build a variance estimate using bootstrap methods, (if I am not very wrong) doing so would still ignore the fold variances and take only the statistic estimates into consideration (plus requiring much more computation power).
Comment: @Cesar: nice observation: your models are very unstable (high variance between iterations). In classification (unless the classifier is worse than guessing), unstable predictions will lead to wrong predictions. An illustrative thought for that is that a deviation from a correct prediction will always be in direction "wrong", there is no too high which would cancel with too low.
Comment: @cbeleites: wouldn't this be somewhat expected then, since the large variance changes occur mostly with models showing higher error rates? By the way, nice update on your answer. I still have to read it more carefully, but I am already very grateful. Thanks.
Comment: @Cesar: Thx. Sure it is expected, a less illustrative way to state that is the variance formula for proportions (see my answer): the more extreme the true error rate is, the lower the variance, the maximum variance is at error rate = 50%.
Comment: Did you calculate the variance considering the two possible ways to see if they differ greatly from each other?
Comment: Yes, I did. In some experiments there was a change of about an order of magnitude between the variance and the pooled variance for training samples. There wasn't much difference for validation samples. The larger changes seemed to be associated with less accurate models.
Answer: Very interesting question, I'll have to read the papers you give... But maybe this will start us in direction of an answer:
I usually tackle this problem in a very pragmatic way: I iterate the k-fold cross validation with new random splits and calculate performance just as usual for each iteration. The overall test samples are then the same for each iteration, and the differences come from different splits of the data.
This I report e.g. as the 5th to 95th percentile of observed performance wrt. exchanging up to $\frac{n}{k} - 1$ samples for new samples and discuss it as a measure for model instability.
Side note: I anyways cannot use formulas that need the sample size. As my data are clustered or hierarchical in structure (many similar but not repeated measurements of the same case, usually several [hundred] different locations of the same specimen) I don't know the effective sample size.
comparison to bootstrapping:

iterations use new random splits.
the main difference is resampling with (bootstrap) or without (cv) replacement.
computational cost is about the same, as I'd choose no of iterations of cv $\approx$ no of bootstrap iterations / k, i.e. calculate the same total no of models.
bootstrap has advantages over cv in terms of some statistical properties (asymptotically correct, possibly you need less iterations to obtain a good estimate)
however, with cv you have the advantage that you are guaranteed that

the number of distinct training samples is the same for all models (important if you want to calculate learning curves)
each sample is tested exactly once in each iteration

some classification methods will discard repeated samples, so bootstrapping does not make sense

Variance for the performance
short answer: yes it does make sense to speak of variance in situation where only {0,1} outcomes exist.
Have a look at the binomial distribution (k = successes, n = tests, p = true probability for success = average k / n):
$\sigma^2 (k) = np(1-p)$
The variance of proportions (such as hit rate, error rate, sensitivity, TPR,..., I'll use $p$ from now on and $\hat p$ for the observed value in a test) is a topic that fills whole books...

Fleiss: Statistical Methods for Rates and Proportions
Forthofer and Lee: Biostatistics has a nice introduction.

Now, $\hat p = \frac{k}{n}$ and therefore:
$\sigma^2 (\hat p) = \frac{p (1-p)}{n}$
This means that the uncertainty for measuring classifier performance depends only on the true performance p of the tested model and the number of test samples.
In cross validation you assume

that the k "surrogate" models have the same true performance as the "real" model you usually build from all samples. (The breakdown of this assumption is the well-known pessimistic bias).
that the k "surrogate" models have the same true performance (are equivalent, have stable predictions), so you are allowed to pool the results of the k tests.
Of course then not only the k "surrogate" models of one iteration of cv can be pooled but the ki models of i iterations of k-fold cv.

Why iterate?
The main thing the iterations tell you is the model (prediction) instability, i.e. variance of the predictions of different models for the same sample.
You can directly report instability as e.g. the variance in prediction of a given test case regardless whether the prediction is correct or a bit more indirectly as the variance of $\hat p$ for different cv iterations.
And yes, this is important information.
Now, if your models are perfectly stable, all $n_{bootstrap}$ or $k \cdot n_{iter.~cv}$ would produce exactly the same prediction for a given sample. In other words, all iterations would have the same outcome. The variance of the estimate would not be reduced by the iteration (assuming $n - 1 \approx n$). In that case, assumption 2 from above is met and you are subject only to $\sigma^2 (\hat p) = \frac{p (1-p)}{n}$ with n being the total number of samples tested in all k folds of the cv.
In that case, iterations are not needed (other than for demonstrating stability).
You can then construct confidence intervals for the true performance $p$ from the observed no of successes $k$ in the $n$ tests. So, strictly, there is no need to report the variance uncertainty if $\hat p$ and $n$ are reported. However, in my field, not many people are aware of that or even have an intuitive grip on how large the uncertainty is with what sample size. So I'd recommend to report it anyways.
If you observe model instability, the pooled average is a better estimate of the true performance. The variance between the iterations is an important information, and you could compare it to the expected minimal variance for a test set of size n with true performance average performance over all iterations.
Comment: The problem with estimating variance (and then with confidence intervals) is that the cross validation models and tests (over iterations) are not independent: in the end you only have $n$ independent cases. The models should not be independent at all - in fact, the usual evaluation of CV assumes them to be equal. But regardless of how many iterations you have, each case has been tested already in the first iteration. And there's always the question how representative your small data set is.
Comment: You iterate with new random splits, with replacement, like in bootstrap? Or you repeat k-fold cross-validation several times? This is interesting, because it doesn't seems like bootstrap but may work like so. But how many replications do you perform? This could get very costly easily.
Comment: @Cesar: it is very similar to bootstrap, see the expanded answer.
Comment: @cbeleites: In terms of variance estimates, what do you think it is more useful: confidence intervals of the average RMSE or the standard deviation of the average RMSE? Also, is it more useful to calculate variance estimates on the train or the test data? Thanks
Comment: @jpcgandre: At the moment I'd even go one step back behind that: I report observed variation (percentiles). The reason is that at least for classification I'm not sure how to really calculate confidence intervals. However, I'm usually in one of 2 situations: either the models are stable, then the variance across iterations is $\ll$ variance calculated for total number of test cases and doesn't matter. Or it is large, and then I  aggregate to get again into a situation where this part of the variance doesn't matter.
Comment: I've been calculating variance estimates on cross validation results only in very special regression situations: a) the models were stable and b) the cross valiation used the very unusual setup of training on one "case" (concentration series of one stock solution for calibrating a PLS regression model) and testing on 4 independent stock solutions. This I "rotated", so the set-up was basically an "the other way round" version of leave-one-out. Because each surrogate had enough test cases to calculate standard deviation, I was more comfortable in that situation.
Comment: How does CV leave the "same number of distinct training samples" for each model, but bootstrapping does not?  I don't follow, as the CV "replicate data sets" are a different combination of observations - how can they possibly provide the same number of distinct observations?  Perhaps you are assuming that each record is distinct in the original training set?
Comment: @probabilityislogic: the CV replicate data sets are *smaller* than the original data set. Thus, different such replicates can be produced even with resampling without replacement. Resampling with replacement you can draw the same record several times. Thus, the number of *unique* records can vary. Yes, I assume the original records to be distinct. In order to keep statistical independence which is crucial for many applications, resampling should be done at the highest level of the data hierarchy. (E.g. I work with patient data with hundreds of rows for each patient => resample patients)
Answer: Remember CV is an estimate only and can never represent the 'real' generalisation error.
Depending on your sample size (which will impact your number of folds or fold size) you can be severely limited in your ability to calculate any parameter estimates of the distribution of the generalisation error.
In my opinion (and I've seen it purported in various text books, 'Knowledge Discovery with Support Vector Machines'-Lutz Hamel)you can do some bootstrapping variant of CV to estimate the distribution of the generalisation error, but a standard 10-1 (for example) once off CV will not give you enough data points to make inferences about the true gen-error.
Bootstrapping requires you to take multiple samples with replacement from your training/test/val effectively doing multiple (say 1000 or so) 10-1 (or whatever) CV tests.
You then take your sample distribtion of averages for each CV test as an estimate of the sampling distribution of the mean for the population of CV errors and from this you can estimate distributional parameters i.e. mean, median, std min max Q1 Q3 etc...
It's a bit of work, and in my opinion only really required if your application is important/risky enough to warrant the extra work. i.e. perhaps in a marketing environment where the business is simply happy to be better than random then maybe not required.
BUT if you are trying to evaluate patient reactions to high risk drugs or predict income expectations for large investments you may well be prudent to carry it out.
Comment: It is an estimate, but so could be said about virtually any use of a statistic. However, when each fold already has a variance estimate, it doesn't seems correct to discard this information. I have updated the question with clarifications.
Comment: Perhaps I'm not getting it. I don't really understand why you are agonizing over the variance of a single fold?
Comment: Running a single 10-fold CV is not exactly cheap for the particular problem I am trying to tackle. You are correct I could estimate the variance by taking multiple samples with replacement as in bootstrap. But taking a large number of CVs, even a few hundred, could be very impractical in my case. I am looking for a way (if there is any) to combine individual fold variance estimates so I could at least reduce the number of bootstrap samples needed. And besides, curiosity.
Comment: ah okay. Perhaps then the variance of each of the 10 fold means from the from the overall fold mean will be a random variable selected from the sampling distribution for variances...(which I think is a chi sq dist or F dist)
Comment: @clancy: with 10-1 do you mean a leave-one-out cv for n = 10 samples? Note that for leave-one-out iterations do not make sense.
Comment: @clancy: Looking at iterations essentiall measures model (prediction) instability. This is quite different from estimating the binomial distribution behind testing a single (surrogate) model. OTOH, you are probably right with the practical conclusions: if you have enough test samples to prove that the model is better than some other approach (look at the uncertainty of the estimate as derived from binomial distribution), quite often you don't have problems with instability any longer...
Comment: Hi Cbeleites, I was meaning folds not leave one out...more like cut the test or val set into 10 equal sized portions and then run a test/val error on each to get an estimate of the mean test/val error...the variance of these from the mean will be from the sampling distribution of variances for the test/val error.
Comment: @clancy: yes, iff the surrogate models are stable. Otherwise, the instability adds also to the between-folds variance. Of course you can try to separate these two from one iteration and k folds, but a few iterations can make things much easier... ;-)
Comment: I was just reading a text book t his morning that covers CV error as an estimate for the actual error (which can never be observed unless you have a sample that actually represents the population)...and the authors said /showed that the CV error is an unbiased estimator for the actual error (or variance + bias error).Admittedly the authors used n fold cross validation.


Como realmente entender Streams?

Question: Trabalho faz um tempo com C# e .NET e várias vezes já vi o uso de streams por aí para leitura de arquivos, escrita de resposta HTML, upload de arquivos e etc. Acontece que eu até hoje não entendi o que realmente é um stream, quando eles devem ser usados e nem como funciona.
Por exemplo, pra ler um arquivo usamos o seguinte:
using (StreamReader streamReader = new StreamReader(caminhoArquivo))
{
    string conteudo = streamReader.ReadToEnd();
}

Outro exemplo é o upload de dados pra um blob do Microsoft Azure
using (Stream stream = File.OpenRead(caminhoArquivo))
{
    blockBlob.UploadFromStream(stream);
}

E um outro exemplo, que não envolve arquivos, é escrever o conteudo de uma resposta HTTP num middleware do OWIN. Pesquisando um pouco sobre streams achei o seguinte:

Temos uma classe abstrata que representa uma sequência de bytes na qual podemos realizar operações de leitura e escrita. Essa classe abstrata é chamada de Stream.

Dessa forma um stream é uma sequencia de bytes? Mesmo assim, quando isso deve ser usado? Por que nesses casos que eu citei usamos streams? No caso da resposta HTTP é até um pouco mais dificil entender, não poderiamos somente escrever o texto da resposta inves de um stream?
Comment: O stream é uma maneira de se tratar um conteúdo aos poucos, sem colocá-lo inteiro de uma vez na memória. É um fluxo de dados. Esse pelo menos é o meu entendimento, de maneira bem simplificada.
Comment: Então @bfavaretto, a ideia de usar streams é ao inves de colocar um volume muito grande de dados na memória de uma vez, carregar e tratar aos poucos sem deixar o conteúdo todo na memória?
Answer: Quase isso. Stream é uma sequência de dados, de elementos. Podem ser bytes, o mais comum, mas não necessariamente. Os elementos individuais podem ser objetos mais complexos ou mais simples, como o bit. Se formos traduzir a palavra, provavelmente chamaríamos de fluxo contínuo. É apenas um conceito.
Fluxo é a palavra-chave aí. O stream é criado justamente para evitar ter que lidar como o todo. Com ele é possível ir manipulando os dados desejados sob demanda.
Com o stream a aplicação trabalha com um modelo de conexão sempre. O código inicia a conexão abrindo o stream e pode usá-lo até que ele seja fechado pelo código, eventualmente decidido através de sinalização recebida durante o processo. A conexão não precisa ser um mecanismo externo, basta ter uma coleção de dados.
Vantagens
Esses dados podem estar disponíveis como um todo ou não. Ele pode vir/ir aos poucos resolvendo um potencial problema de sobrecarga de memória (de qualquer tipo), congestionamento e ausência temporária de transmissão. Enquanto o stream estiver aberto e não der nenhuma indicação que ele terminou ele pode fornecer dados novos mesmo que uma espera seja necessária.
Outra vantagem é que não precisa saber como a informação vem/vai, não importa a fonte ou alvo. Você não precisa saber que o stream funciona de uma determinada forma. Se a fonte/alvo de dados é um arquivo em disco, na memória, ou objeto na memória, uma transmissão da rede em qualquer protocolo, uma conexão de banco de dados, uma comunicação com outro processo, um algoritmo que gere estes dados sob demanda (números randômicos, por exemplo) ou outra forma, isto é problema da fonte/alvo de dados com o stream.
O programa que vai consumir o stream não precisa saber como os dados chegam até ele. Por isso ela é chamada de classe abstrata (nada a ver com abstract de OOP, ainda que este mecanismo seja usado em linguagens OO para implementá-lo). Você não precisa saber a implementação concreta da fonte/alvo de dados. Também não precisa saber qual é o modo usado para o fornecimento de dados. Diversos mecanismos podem ser usados dentro dele sem que o código consumidor do stream precise estar ciente.
Talvez um exemplo citado que caiba explicação é a geração de randômicos. Ao gerar números randômicos não significa que o gerador seja um stream. Assim como gerar uma string também não o é. O stream encapsula dados. Então um gerador de randômicos pode ser a fonte de dados do stream mas não o stream em si. Não podemos chamar qualquer sequência de elementos de stream, precisa atender ao protocolo estabelecido por ele.
Um stream pode inclusive manipular dados em certas situações se houver algum motivo para seu mecanismo prover algo assim. Ainda assim o programador não precisa saber internamente como e porque ele faz isto, só precisa entender como os dados vêm e vão através deles.
APIs que aceitam stream
Em tese se uma API quiser fornecer uma forma de trabalhar com dados sem o stream não há problema. Uma resposta HTTP poderia trabalhar direto com o texto, mas perderia todas estas vantagens citadas acima. Por exemplo, System.IO.File.WriteAllText() não trabalha com stream.
Obviamente que se a API só aceita comunicação através de streams você terá que fazê-lo desta forma. No geral só tem vantagens em usar desta forma.
Desvantagem
Talvez alguém possa colocar que há desvantagem por ter uma camada a mais, que aumenta ligeiramente a complexidade e consumo de processamento. Mas são coisas muito pequenas que não costumam ser relevantes perto das vantagens apresentadas pelo seu uso. Vou ressaltar que o custo extra do seu uso é mínimo em todos os sentidos.
Um exemplo real de implementação
Talvez analisar o código da classe Stream possa ajudar (ou atrapalhar :)) entender seu funcionamento. Note que ela é apenas a base para as classes que realmente fornecem streams concretos, que sabem como lidar em determinadas fontes/alvos. Talvez seja interessante analisar o código de algumas classes que herdam dela, como a FileStream.
Coloquei no GitHub para referência futura.

Como realmente entender Streams?


Symmetric but not so good looking arctan sum

Question: If
$a, b, c$ are real positive numbers and
$\theta=\tan ^{-1}\left[\frac{a(a+b+c)}{b c}\right]^{\frac{1}{2}}+\tan ^{-1}\left[\frac{b(a+b+c)}{c a}\right]^{\frac{1}{2}}+
\tan ^{-1}\left[\frac{c(a+b+c)}{a b}\right]^{\frac{1}{2}}$
then $\tan \theta$ equals?

I did this by adding the arctan terms twice, It took a lot of time and a ton of trials and error. I don't even want to type the solution in TeX
I know this is a symmetric equation,
which are usually easy to solve (Still learning that) but I'm not able to simplify it. How can I solve it in a more simple way?
(And help me with the tags too)
Answer: It would be nice to first the  simplify the argument of $\tan^{-1}$
Let $ \dfrac{a+b+c}{abc}=s^2$ so the sum looks simple
$$\theta=  \tan ^{-1}(as )+\tan ^{-1}(bs) +\tan ^{-1}(cs )$$
Now you can do the same, or simply use the identity:
If $x_1,x_2 \ldots x_n$ are real then
$$\tan ^{-1} x_{1}+\tan ^{-1} x_{2}+\ldots+\tan ^{-1} x_{n}=\tan ^{-1}\left(\frac{S_{1}-S_{3}+S_{5}-S_{7}+\ldots}{1-S_{2}+S_{4}-S_{6}+\ldots}\right)$$
Where $S_{k}$ denotes the sum of products of $x_{1}, x_{2}, \ldots x_{n}$ taken $k$ at a time.
And in this equation, $ \displaystyle \sum_{cyc} \tan^{-1}(s\cdot a_i) $
$$\theta = \tan ^{-1}\left(\frac{(as+bs+cs) - (abcs^3)}{1-(a bs^{2}+bcs^{2}+a cas^{2})} \right) \\ =  \tan ^{-1}\left(\frac{s\left((a+b+c)-a b c s^{2}\right)}{1-s^2(ab+b c+a c)}\right)$$
See that $ \left( a+b+c - abc \cdot s^2 \right) = 0$
Thus, $\tan (\theta) = 0$
Comment: Okay,  first of all, how did you type all that so quickly ? And second, how did you know what to substitute?
Comment: Remember  $$\sqrt{a}\ge0$$  for real positive $a$
Comment: @Ashley, I use a LaTeX editor, which is helpful in typing long equations. And .. well, you see the common thing in all the terms.. it comes with practice


Is it possible to get the timestamp from an existing picture in android studio?

Question: Does anyone know if it is possible to retrieve the time of an existing picture? So i basically want to use the time when the picture was taken?
Answer: Create an ExifInterface with the image. You can then grab any attribute that is available, including the timestamp of when the image was taken. Documentation
String timestamp = exifInterface.getAttribute(ExifInterface.TAG_DATETIME);

Comment: Thanks i will check later today!
Answer: you should use the ExifInterface , it provides information about image like orientation , date and time of creation if it's available
ExifInterface exif = new ExifInterface(pathToTheImage);
String datetime =  exif.getAttribute(ExifInterface.TAG_DATETIME);

for more attribute have a look here
Comment: Thanks i will check later today!


CentOS 6 with webmin and virtualmin - cant get POP / SMTP working?

Question: I've set up Webmin and Virtualmin on a CentOS 6 dedicated server.
Postifx / Dovecot config is untouched by me, and I havent tried to open or close any ports manually.
My problem is that after creating a new virtual host and email users for that host, the new email addresses don't seem to be working.
According to nmap, ports 25 and 110 are open, and the MX record and mail.domain.com A record on my domain are all correct, but incoming email isn't getting to the inbox and I can't connect to the SMTP server using a mail client - connection gets refused on port 25. I seem to be able to log in to the POP server but none of my test emails have come through.
Any ideas on what could be wrong?
Comment: ok... any explanation for the downvotes? am I being braindead-stupid or have I asked this question incorrectly?
Comment: alrighty then. *grumble bloody unconstructive downvoters grumble*
Comment: I think the general consensus is that Webmin and Virtualmin aren't meant for full-scale deployment as it can open up new attack surfaces, and also this could have probably been solved by reading some documentation. Don't worry about it too much.
Comment: @tombull89 thanks for clarifying; I did get through most of the virtualmin / webmin docs but they are thin on the ground regarding email setup. doesnt matter, sorted now, thanks.
Answer: Turns out my problem was solved by a couple of relatively simple measures:
first, I enabled the submission and smtps services via servers -> postfix -> server processes; these were both disabled by default.
then, I restarted postfix and dovecot from the command line
sudo /etc/init.d/postfix restart
sudo /etc/init.d/dovecot restart

and my SMTP was suddenly magically working, and the server began accepting external mail too.
This may have been required either because of my changes to the services or perhaps some config changes that were made by Virtualmin when the virtual host was created. Couldn't say for sure but it is working now. Keeping an eye on /var/log/maillog in any case.