Relation between $\operatorname{Proj} \, k[x_0,\cdots,x_n]$ and $\mathbb{P}^n$

Question: In Hartshorne's "Algebraic Geometry" p. 77, Example 2.5.1, it is mentioned that if "$k$ is an algebraically closed field, then the subspace of closed points of $\operatorname{Proj} \, k[x_0,\cdots,x_n]$ is naturally homeomorphic to the projective $n$-space $\mathbb{P}^n$. He refers to Ex. 2.14d, however I don't see the connection. Any insights?
Thanks.
P.S. Ex. 2.14(d) seems to me a little bit obscure at this point, this is why i am not reproducing it. Any argument relating $\operatorname{Proj} \, k[x_0,\cdots,x_n]$ and $\mathbb{P}^n$ is very welcome.
Comment: Do you understand the connection between $Spec(k[x_1,\ldots,x_n])$ and $\mathbb A_k^n$, when $k$ is an algebraically closed field?
Comment: @Berci: The Proj operator is described e.g. here : http://en.wikipedia.org/wiki/Proj_construction. I mention the reference to Exercise 2.14(d) for completeness, reproducing it here i think it would be confusing.
Comment: What does 'Proj' stand for? What is Ex.2.14d?
Comment: @MTurgeon: Very good question. No, i don't. I understand the connection between $\operatorname{Specm}(k[x_1,\cdots,x_n])$ and $\mathbb{A}^n_k$. Its a $1-1$ correspondence. If you can point to me the relevant theorems in Hartshorne i would appreciate it.
Comment: @Manos As noted below by acyrl, you could have a look at *The Geometry of Schemes* by Eisenbud-Harris; the relevant section is II.1.1. Also, this is discussed in Hartshorne. Have a look at Proposition 2.6 in Chapter 2.
Comment: @MTurgeon: Thanks. I've been studying proposition 2.6 since the day before yesterday, it takes some time to digest :)
Answer: Exercise $2.14~ d)$ states that for any projective variety with homogeneous coordinate ring $S$,
$$t(V) \simeq \operatorname{Proj}S$$
Which include $\mathbb{P}^{n}$, meaning $V$ could be $\mathbb{P}^{n}$.
Now by proposition 2.6, $V$ and $t(V)$ have homeomorphic closed points.
Comment: No worries. Well what is the homogeneous coordinate ring of the variety $\mathbb{P}^{n}$.
Comment: Do you mean $\mathbb{P}^{n}$ as in the variety sense or the scheme.
In the variety sense they are all closed, but not in the scheme $\operatorname{Proj} S$ sense.
Comment: $\Bbb P^n$ as a scheme has many non-closed points. The notation $\{P\}^{-}$ usually means the Zariski closure of the point $P.$
Comment: @Manos: you have to be careful by what you mean by "variety". $\operatorname{Proj}S$ is a scheme and has not the slightest chance to be a variety in the sense Hartshorne defines varieties in Chapter 1. However Proposition 2.6 shows that every such "old-fashioned" variety gives rise to a scheme in a canonical way. So the new definition of (abstract) variety should be that it is a scheme, which is isomorphic to a scheme, which is in the image of the functor of 2.6 (but see 4.10 for a more concrete characterization).
Comment: @Manos: 1) Yes, $\mathbb{P}^{n}$ is irreducible (as its coordinate ring is an integral domain). 2) I think I didn't make my point clear enough, thanks for being persistent. A variety in the sense of chapter one does **always** live in an **ambient space**, which is $\mathbb{P}^{n}$ for some $n$ depending on the variety. So there really is no relation between $\operatorname{Proj}k[x_0,...,x_n]$ (which is an "abstract" topological space) and $\mathbb{P}^{n}$ as a variety in the sense of Ch.1 a priori. Also $\operatorname{Proj}k[x_0,...,x_n]$ will always contain a non-closed point,...
Comment: ...corresponding to the zero ideal of $k[x_0,...,x_n]$, so your statement that $\operatorname{Proj}k[x_0,...,x_n]$ consists of closed points only is not true (except for $n=0$). As for your second comment, in a scheme-theoretic setting, ignoring "old-fashioned" varieties one usually defines $\mathbb{P}_{k}^{n}$ as $\operatorname{Proj}k[x_0,...,x_n]$. You may even replace $k$ by an arbitrary commutative ring, so with schemes at hand, it makes even sense to talk about, e.g. "projective $n$-space over $\mathbb{Z}$". But it's hard judging the content of your 2nd comment w/o knowing its source
Comment: Thanks for the answer. How does $k[x_0,\cdots,x_n]$ come into play?
Comment: It is $k[x_0,\cdots,x_n]$. So in general, $\mathbb{P}^n$ is different from $\operatorname{Proj} k[x_0,\cdots,x_n]$. Is there a sense (or conditions) in which we have equality?
Comment: In the sense above. That they have the "same" closed points, but $\operatorname{Proj} S$ has points which are not close. As mentioned above, you can gain some intuition by thinking about the affine case. Geometry of Schemes by Eisenbud has quite a few examples with pictures that can help you out.
Its worth noting that sometimes the notation $\mathbb{P}^{n}$ really means $\operatorname{Proj} S$, when you are exclusively talking of schemes.
Comment: Thanks. All points of $\mathbb{P}^n$ are closed?
Comment: Yes, i meant in the variety sense. So is it accurate to say that $\operatorname{Proj}S$ is a projective variety if and only if all of its points are closed? And one more short question regarding Proposition 2.6: in the proof of 2.6 what is the notation $\left\{P\right\}^-$ in defining the map $\alpha$ in the end of the first paragraph?
Comment: @NilsMatthes: Thanks for your comment. Hartshorne requires any variety to be irreducible. He also refers to $\mathbb{P}^n$ as projective variety. 1) Is $\mathbb{P}^n$ irreducible? 2) Since every point of $\mathbb{P}^n$ is closed in the variety sense, if all points of $\operatorname{k[x_0,\cdots,x_n]}$ are closed, then the latter is topologically isomorphic to $\mathbb{P}^n$. Correct?
Comment: @NilsMatthes: I also encountered in some notes the statement "$\operatorname{Proj} k[x_0,\cdots,x_n]$ is isomorphic to the algebraic $\mathbb{P}^n$." Any idea what that means?
Comment: @NilsMatthes: Thank you for your very enlightening comments. So, since $\operatorname{Proj} k[x_0,\cdots,x_n]$ always contains a non-closed point (for n>0), and any point of a projective variety is closed, we can say that $\operatorname{Proj} k[x_0,\cdots,x_n]$ is a projective variety if and only if $n=0$.
Comment: @Manos: You're right, $\operatorname{Proj}k[x_0]$ has only one point, and hence is isomorphic to a projective variety, namely to any one-point variety in $\mathbb{P}^{n}$. We should better stop discussing here, though, as discussions in the comments are commonly discouraged on this site. Maybe you want to digest the information you gained and ask some more questions later?
Comment: @NilsMatthes: Definitely, i was planning to do that :) Thanks again for your help.
Comment: Sorry, I had gone away from the computer and missed the follow up comments. Glad someone else helped. =)


How to show only my content?

Question: I want to make a view that only show the content that is made by the user itself. For example, I have a library and it contains all content that I flagged, but I also want to show all content that I made (even if I have not flagged it).
Answer: Use views CONTEXTUAL FILTERS, filter by content author is the current user.
Example:

How to show only my content?


Limit Superior And Limit Inferior with sequences

Question: Use the definition to calculate the limit superior and the limit inferior of the sequence $f:\mathbb N\to \mathbb R$ such that $f(n)=2+\frac{(-1)^n}{n}$ for all $n\in \mathbb N.$
I'm confused by what definition this means. Should I use $\delta$ and $\epsilon$ to show an interval that this is true?
Comment: Has your question been answered? If yes, you should accept an answer.
Answer: Note that
$$\sup\limits_{k\geq n} f(k) = 2 +\frac{1}{n}$$
$$\inf\limits_{k\geq n} f(k) = 2 -\frac{1}{n}$$
By definition, we have that
$$\limsup\limits_{n\to\infty} f(n) = \lim\limits_{n\to\infty} \sup\limits_{k\geq n} f(k)=\lim\limits_{n\to\infty}\left(2+\frac{1}{n}\right)$$
$$\liminf\limits_{n\to\infty} f(n) = \lim\limits_{n\to\infty} \inf\limits_{k\geq n} f(k)=\lim\limits_{n\to\infty}\left(2-\frac{1}{n}\right)$$
You can show that these two limits are 2 by the $\epsilon,\delta$ definition of a limit.
Answer: Your sequence tends to $2$. So you have
$$\limsup f(n) = \liminf f(n) = \lim f(n) =2$$
Comment: But wouldnt I have to justify that?
Comment: Like by proving that the sequence is going to converge?
Comment: Of course, you have to justifiy all that is not clear to you ! So indeed, first explain why your sequence converges ; then prove that in this case, $\liminf = \limsup = \lim$.
Comment: is this  a valid proof?
Comment: Who is $x$ ? who is $\epsilon$ ? who is $n$ ?... it's unreadable !


Error when importing View

Question: I am trying to import View from one Drupal 7 Environment (from local to
pantheon one) and got this error:
- Field handler views_entity_commerce_order.operations_dropbutton is not available
- Unable to import view.

Any ideas what is going on here ?
Comment: Where are you Exporting the view from? Check to see if the original site has additional views modules than the site you are trying to Import to.
Answer: Create a field with machine name 'views_entity_commerce_order' in the pantheon one and then import your view.
Answer: I was able to find a workaround by setting this option in View Import:
Bypass view validation
Answer: Arun has provided the correct answer above but may not be providing enough information for people who are new to Drupal, new comers simply may not know how to "create a machine field name" so hopefully this will help. The whole point of creating a View is to display some sort of list or collection of a given content type, this is usually a custom content type but it doesn't have to be. If you try to import the View before you have added all the fields to the content type you are creating the View for, you will get an error similar to the one Michal got. You should always create (or import) the View after you've finished creating your custom content type or finished modifying an existing one, and make absolutely sure you use the same field names you used in your development environment or the View Import will fail.

Error when importing View


C - How to make my data structure implementation synchronized?

Question: I have a data structure which I personally implemented that now needs to be used across multiple threads.
typedef struct
{
    void** array_of_elements;
    size_t size;
} myStruct;

For simplicity, let's say my data structure has these functions:
// Gets a data element from the structure.
void* get(myStruct *x);
// Prints out all the data elements.
void print(myStruct *x);
// Adds an element into the structure.
void add(myStruct *x, void *to_be_added);

It's not a problem whatsoever to call get while another thread is calling print since they are both accessors. However, get and print cannot work while add is currently being called. Vice versa, add cannot work if get and print are currently in-progress.
So I changed myStruct to look like the following:
typedef struct
{
    void** array_of_elements;
    size_t size;

    // True when a mutator is editing this struct.
    bool mutating;
    // The number of threads currently accessing this struct.
    int accessors;
} myStruct;

Now my functions look like the following:
void* get(myStruct *x)
{
    // Wait for mutating to end.
    while (x->mutating);
    // Indicate that another accessor is now using this struct.
    x->accessors++;

    // get algorithm goes here

    // Declare we are finished reading.
    x->accessors--;

    return ...
}

// Same as above...
void print(myStruct *x)
...

void add(myStruct *x)
{
    // Wait for any accessors or mutators to finish.
    while (x->mutating || x->accessors > 0);
    x->mutating = true;

    // add algorithm here

    x->mutating = false;
}

BUT, I think there are a lot of problems with this approach and I can't find a way to solve them:

One of my classmates told me using while loops like this slows the thread down immensely.
It has no sense of a queue. The first method that begins waiting for the myStruct to finish being used isn't necessarily the one that goes next.
Even IF I had a queue data structure for which thread goes next, that data structure would also need to be synchronized, which in itself is an infinite loop of needing a synchronized data structure to synchronize itself.
I think it's possible that in the same nano second one thread changes the accessors counter from 0 to 1 (meaning they want to start reading), it's possible for a mutator thread to see it's value is 0 and start mutating. Then, both a mutator thread and an accessor thread would be going at the same time.
I'm pretty sure this logic can cause grid-lock (threads waiting on each other infinitely).
I don't know how to make certain threads sleep and wake up right when they need to for this task, besides having it stuck in a while loop.
Comment: Look into read-write locks...
Comment: @Dmitri Is what I have for read/write locks not even remotely close of what the real implementation would look like?
Comment: If you're sharing a variable among threads, you should be using atomic accesses or some sort of synchronisation mechanism like a mutex.  A read-write lock is like a mutex, except that it allows multiple threads to read, but requires exclusive access for writing.  For posix threads, there's `pthread_rwlock_t`, and windows has "slim reader writer locks"
Comment: you need an atomic check and mutate, your version multiple threads could check mutating and it be false, and all move into setting the mutating variable all at the same time
Comment: @Dmitri I understand threads contain mutexes but isn't it my **structure itself** that needs a mutex or lock? Like, inside my `struct` there must be some variable of some sort that handles that right?
Comment: You acquire (lock) the mutex/rwlock/etc. before accessing a resource, and release (unlock) when you're done... as long as *all* code that accesses it does this, the resource is protected.  It doesn't matter whether it's part of the structure or not, as long as you use it when accessing the structure.
Comment: @KeithNicholas Yeah I definitely understand that. How does one implement an 'atomic check and mutate' ?
Answer: You have the right idea, just the wrong approach.  I'm not sure what OS you're programming on, but you want to look at the concepts of mutex or semaphore to do what you want to do.
On Linux/Unix that is POSIX compliant, you can look at pthreads:
http://www.cs.wm.edu/wmpthreads.html
On Windows, you can look at Critical Sections for something close to a mutex concept:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms682530(v=vs.85).aspx
Or WaitForMultipleObjects for something close to a semaphore:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms687025(v=vs.85).aspx
And, yes, using while loops are a bad idea.  In this case, you are using what is known as a busy loop.  More reading on it here:
What is a busy loop?
Using mutex or semaphore, no while loop is required.  Good luck!
Comment: So the implementation is library dependent? That doesn't seem right. I thought inside of my `struct`, I need to add some variables to lock my threads since my `struct` **is the mutex**?
Comment: Let me restate: is there any way to do this without use of any thread libraries? The developer that uses my structure may not be on the operating system I design it for. So instead, can I just have a variable inside my `struct` that dictates when it is or is not safe to use that `struct` ?
Comment: @Hatefiend if you don't want to tie your code to any particular thread API, it may be better to leave the locking up to the user rather than trying to implement your own one-off mutex or read-write lock (which may be more difficult to do correctly than you expect).  All they'd need to do is acquire/release the lock of their choice (presumably one that goes with whatever threading implementation they're using) before/after calling your access functions.
Comment: @Hatefiend *is there any way to do this without use of any thread libraries?*  Yes.  You can reimplement what the thread libraries already do.  There's a reason those libraries exist...
Comment: @AndrewHenle It's just that I really hate having to learn new libraries and I'll avoid it at all costs. External libraries require a ton of hassle to download, link with visual studio, lot of reading and after finally learning how it all works, there might be a better library out there or maybe your library isn't cross platform so now your amazing cross platform project is windows only, etc. I do like the idea of the user providing the locking/unlocking mechanism though.
Comment: @Hatefiend If you're on Windows and using Visual Studio, you likely won't have to link any new libraries.  You just have to add a #include to your source file.  Most C++ projects in visual studio already link the Windows API library.


Correct Method: Scale emprical CDF or best fit CDF

Question: I have some data, with max value $M$. I assume it is sampled from some distribution with infinite support. A best-fit CDF is calculated $F(y)$ and so is an empirical CDF $F_{N}(y)$.
My issue is that $1-F_{N}(M)=0$ whereas $1-F(M)>0$. My solution is to simply let
$F_{N}(y)=F_{N}(y)\times F(M)$. Is this best practice? The other option would be to let $F(y)=F(y)/F(M)$ so that all the weight is before $M$.
Why I care: I'm going to do some p-value analysis to determine whether the hypothesis of the data being drawn from the assumed distribution is plausible and so the empirical CDF and best-fit CDF have to be as close as possible.
Answer: The K-S test doesn't worry that $F_N(M)=1$ and, as a goodness-of-fit test, seems to be what you're looking for.  If that and similar tests are no good then could you tell us a bit more about the problem you're trying to solve?


Como passo valores de um CSV para um JSON em Python?

Question: Tentei de duas formas ler um arquivo CSV e passar os valores no parâmetro value do JSON, porém, não retorna resultado algum, se passo os valores direto, funciona.
Código 1:
import boto3
from datetime import datetime
import csv

client = boto3.client('cloudwatch', region_name='us-east-1')

with open('metrics_tcc.csv', mode='r') as csv_file:
csv_reader = csv.DictReader(csv_file)
for row in csv_reader:
    response = client.get_metric_data(
        MetricDataQueries=[
            {
                'Id': 'tcc',
                'MetricStat': {
                    'Metric': {
                        'Namespace': 'metrics.tcc',
                        'MetricName': 'responseTime',
                        "Dimensions": [
                    {
                        "Name": "isColdStart",
                        "Value": row['isColdStart']
                    },
                    {
                        "Name": "requestId",
                        "Value": row['requestId']
                    },
                    {
                        "Name": "interval",
                        "Value": row['interval']
                    },
                    {
                        "Name": "target",
                        "Value": row['target']
                    }
                        ]
                    },
                    'Period': 300,
                    'Stat': 'Maximum',
                    'Unit': 'Milliseconds'
                },
                    'ReturnData': True
            },
        ],
        StartTime=datetime(2018, 10, 8),
        EndTime=datetime(2018, 10, 9),
    )
    print(response['MetricDataResults'])

A saída é esse JSON:
[{u'Timestamps': [], u'StatusCode': 'Complete', u'Values': [], u'Id': 'tcc', u'Label': 'responseTime'}]

Em Values era pra ter vindo um valor, porém, é como se não entendesse o valor passado no JSON, daí retorna vazio.
Segunda tentativa:
import boto3
import numpy as np
from datetime import datetime

target, interval, isColdStart, requestId =
np.loadtxt('metrics_tcc.csv', delimiter = ',', unpack = True, dtype =
'str')
client = boto3.client('cloudwatch', region_name='us-east-1')

t = target[1]
i = interval[1]
bcs = isColdStart[1]
rq = requestId[1]

response = client.get_metric_data(
MetricDataQueries=[
    {
        'Id': 'tcc',
        'MetricStat': {
            'Metric': {
                'Namespace': 'metrics.tcc',
                'MetricName': 'responseTime',
                "Dimensions": [
            {
                "Name": "isColdStart",
                "Value": t
            },
            {
                "Name": "requestId",
                "Value": i
            },
            {
                "Name": "interval",
                "Value": bcs
            },
            {
                "Name": "target",
                "Value": rq
            }
                ]
            },
            'Period': 300,
            'Stat': 'Maximum',
            'Unit': 'Milliseconds'
        },
            'ReturnData': True
    },
],
        StartTime=datetime(2018, 10, 8),
        EndTime=datetime(2018, 10, 9),
)
print(response['MetricDataResults'])

A saída é a mesma:
[{u'Timestamps': [], u'StatusCode': 'Complete', u'Values': [], u'Id': 'tcc', u'Label': 'responseTime'}]

Mas caso eu passe o valor direto, funciona:
import boto3
from datetime import datetime

client = boto3.client('cloudwatch', region_name='us-east-1')

response = client.get_metric_data(
MetricDataQueries=[
    {
        'Id': 'tcc',
        'MetricStat': {
            'Metric': {
                'Namespace': 'metrics.tcc',
                'MetricName': 'responseTime',
                "Dimensions": [
            {
                "Name": "isColdStart",
                "Value": "false"
            },
            {
                "Name": "requestId",
                "Value": "0"
            },
            {
                "Name": "interval",
                "Value": "660"
            },
            {
                "Name": "target",
                "Value": "when-will-i-coldstart-dev-system-under-test-256"
            }

                ]
            },
            'Period': 300,
            'Stat': 'Maximum',
            'Unit': 'Milliseconds'
        },
            'ReturnData': True
    },
],
StartTime=datetime(2018, 10, 8),
EndTime=datetime(2018, 10, 9),
)

print(response['MetricDataResults'])

A saída é esse JSON:
[{u'Timestamps': [datetime.datetime(2018, 10, 8, 20, 0, tzinfo=tzutc())], u'StatusCode': 'Complete', u'Values': [141.89045000006445], u'Id': 'tcc', u'Label': 'responseTime'}]

Note que o Values agora vem certo. Poderiam me ajudar?
Answer: Não tenho certeza, mas parece que o problema é no seu csv; Ele não está sendo lido corretamente, talvez por estar vazio ou por não estar usando o delimitador correto. Se continuar com problemas coloque parte do arquivo csv na pergunta.


Copying multiple files by *.extension with xcopy

Question: I'm making a batch script to copy all .doc, .pdf, .xls etc from disk to a pendrive.
My current code below:
@echo off
Title ""

::All Docs
 XCOPY C:\*.doc W:\xdatabase /C /S /I /F /H > W:\database\XC\AllInf.txt

::All PDFs
 XCOPY C:\*.pdf W:\xdatabase /C /S /I /F /H >> W:\database\XC\AllInf.txt

::All WRI
 XCOPY C:\*.wri W:\xdatabase /C /S /I /F /H >> W:\database\XC\AllInf.txt

::All TXT
 XCOPY C:\*.txt W:\xdatabase /C /S /I /F /H >> W:\database\XC\AllInf.txt

::All PPT
 XCOPY C:\*.ppt W:\xdatabase /C /S /I /F /H >> W:\database\XC\AllInf.txt

::All XLS
 XCOPY C:\*.xls W:\xdatabase /C /S /I /F /H >> W:\database\XC\AllInf.txt

The question is: How can I add more extensions but avoid all that duplication in the code?
Answer: Always true, in all programming languages: If you have to do a thing multiple times in a row, use a loop.
@echo off
Title ""

for %%e in (doc pdf wri txt ppt xls) do (
    XCOPY "C:\*.%%e" W:\xdatabase /C /S /I /F /H > W:\database\XC\AllInf.txt
)

The for loop can be tricky in batch scripting. A handy guide is here: http://ss64.com/nt/for.html
Comment: @johnywhy That's a loop, too. That you wrote it on one line makes no difference.
Comment: Oh, im grateful, that exacly what I was asking thks a lot
Comment: @Derley You can also check out robocopy for ways to include multiple sources in one single command.
Comment: @Tomalak This for loop can't be done in one statement?
Comment: @johny I don't think so. `xcopy` only accepts only one source argument.
Comment: @Tomalak This works on one line: `for %f in (xlsm xlsb) do xcopy SourceDir\*.%f DestinationDir\ /S`
Comment: @Tomalak True, i never said it wasn't a loop.
Comment: @johnywhy It's also not one statement. But we really don't need to argue about this. Your and my approach are equivalent. Mine is formatted to be used in a batch file, yours is formatted as a one-off command. They're still the same thing. Use whatever is more convenient in any given situation.
Comment: @Tomalak It's my confusion about the difference between "one statement" vs "one command". Anyway, now i'm playing with robocopy.
Answer: This works on one line, no script-file needed. Needs single percent-sign, not double:
for %f in (xlsm xlsb) do xcopy SourceDir\*.%f DestinationDir\ /S

or in your case
for %e in (doc pdf xls wri txt ppt xls) do XCOPY C:\*.%e W:\xdatabase /C /S /I /F /H > W:\database\XC\AllInf.txt

according to my tests, robocopy supports multiple extensions without a loop.
ROBOCOPY C:\ W:\xdatabase /S *.doc *.pdf *.xls *.wri *.txt *.ppt *.xls

However, in my experience xcopy seems to beat robocopy on speed, even with multiple extensions, but i only tested with a very small batch of files.
Comment: +1 for including robocopy, which helped me as it's clearly an XY question and robocopy's a better fit than xcopy for multiple extensions.


Entropy of (a binary random variable plus gaussian random variable) increases with distance between binary values

Question: Let $X\in\{x_1,x_2\}$ be a real binary random variable with probability mass function $\{p_1,p_2=1-p_1\}$. Let $N$ be a real random variable with standard normal distribution (mean = $0$, std = $1$). Let $Y=X+N$. Let $H(Y)$ denote the differential entropy of $Y$.
It seems like it should be easy to prove that $H(Y)$ increases as $|x_1-x_2|$ increases, that is, as the distance between the two sample points of the binary random variable increases. This is motivated by the mutual information of the output of an additive Gaussian noise channel with binary input; for a given noise power, the mutual information is maximized by spreading the two points as far as possible (subject to whatever constraints one has).  The mutual information is
$$I(X;Y) = H(Y) - H(Y|X) = H(Y)-H(N)$$
Since the mutual information is increased by increasing the distance between $x_1$ and $x_2$, and since $H(N)$ is independent of $X$, $H(Y)$ must increase as $|x_1-x_2|$ increases.
I have tried to prove this from the definition of entropy and so far have been unsuccessful.  We have
$$H(y) = -\int{p(y)\log\left(p(y)\right)dy} = -\int{\sum_{i=1}^2{p_i p_N(y-x_i)\log{\left(\sum_{i=1}^2{p_i p_N(y-x_i)}\right
)}}dy}$$
I have tried using the fact that relative entropy is nonnegative and also the log sum inequality (Theorems 2.7.1 and 8.6.1 in Elements of Information Theory, Cover and Thomas, 2nd Edition). This seems like it should be simple, but I have had no luck. Do I need to use calculus of variations?  Thank you in advance.
Answer: I worked out the proof of this, which was straightforward when I took a more direct approach.
Let $x_2=x_1+\Delta$. Let $g(n)$ be the pdf of a standard normal random variable.
$$g(n)=\frac{1}{\sqrt{2\pi}}e^{-n^2/2}$$
The pdf of $Y=X+N$ is given by
$$\begin{aligned}
p(y) & = p_1 g(y-x1)+(1-p_1)g(y-x_2) \\
 & = p_1 g(y-x1)+(1-p_1)g(y-(x_1+\Delta))
\end{aligned}$$
The entropy of $Y$ is given by
$$H(Y)=-\int_{-\infty}^\infty{p(y)\log(p(y))dy}$$
So
$$
\begin{aligned}
\frac{d H(Y)}{d \Delta}& =
-\int_{-\infty}^\infty{
\frac{d p(y)}{d \Delta}(\log(p(y))+1)dy
}
\\&=-(1-p_1)\int_{-\infty}^\infty{\bigl(y-(x_1+\Delta)\bigr)g\bigl(y-(x_1+\Delta)\bigr)\bigl(1+\log(p(y))\bigr)dy} & (1)
\\&=-(1-p_1)\int_{-\infty}^\infty{yg(y)\bigl(1+\log(p(y+x_1+\Delta)\bigr)dy} & (2)
\\&=-(1-p_1)\int_{-\infty}^\infty{yg(y)\log\bigl(p(y+x_1+\Delta)\bigr)dy} & (3)
\\&=-(1-p_1)\int_{0}^\infty{yg(y)\log\bigl(p(y+x_1+\Delta)\bigr)-yg(-y)\log\bigl(p(-y+x_1+\Delta)\bigr)dy} & (4)
\\&=(1-p_1)\int_{0}^\infty{yg(y)\log\left(\frac{p(-y+x_1+\Delta)}{p(y+x_1+\Delta)}\right)dy} & (5)
\end{aligned}
$$
(2) is a change of variables, (3) follows because $yg(y)$ is odd, (4) follows by changing the limits of integration, and (5) utilizes the fact that $g(y)$ is even.
Now
$$\frac{p(-y+x_1+\Delta)}{p(y+x_1+\Delta)}= \frac{(1-p_1)g(y)+p_1 g(y-\Delta)}{(1-p_1)g(y)+p_1 g(y+\Delta)}$$
Further
$$g(y+\Delta)=g(y-\Delta)e^{-2y\Delta}$$
Hence
$$
\frac{d H(Y)}{d \Delta} = (1-p_1)\int_{0}^\infty{yg(y)\log\left(\frac{(1-p_1)g(y)+p_1 g(y-\Delta)}{(1-p_1)g(y)+p_1 g(y-\Delta)e^{-2y\Delta}}\right)dy}
$$
For $\Delta=0$ (i.e., $x_1=x_2$), the argument of the $\log$ function is $1$ and the derivative is $0$.  This is expected, since $H(y)=0$ (and therefore minimized) when $x_1=x_2$. Noting that the integral is only over nonnegative $y$, the argument of the log function is greater than $1$ for all $\Delta>0$, and it is less than $1$ for all $\Delta<0$. The integrand is therefore strictly positive for $\Delta>0$ and strictly negative for $\Delta<0$ over the range of integration for positive $y$. So the derivative has the same sign as $\Delta$, and $H(Y)$ increases monotonically as $|\Delta|$ increases.
Answer: I would argue that it is not the case actually. Indeed whether $X$ turns out to be $x_1$ or $x_2$ matters little since once it is fixed, you have that the output of the additive channel is a Normal random variable centered at $x_1$ or $x_2$. Moreover, since the entropy of a Normal random variable does not depend on its mean, this implies $H(Y|X) = H(N) = \frac12 \log(2\pi e)$.
More formally you have:
$\begin{align}
H(Y) &= \sum_{i=1,2} H(Y|X=x_i)P(X=x_i)\\&=p_1H(Y|X=x_1) + p_2H(Y|X=x_2)\\&=p_1H(N)+p_2H(N)\\&=H(N)\\&=\frac12 \log(2\pi e),
\end{align}$
which does not depend on $x_1$ or $x_2$.
Hope this helps!
Comment: Thank you for your reply. I am sure that what I was trying to prove is in fact correct, and I have since completed the proof.  I will post it shortly.
Comment: What you have derived above is the conditional entropy $H(Y|X)$, which is defined in Cover and Thomas eq (2.10), matching the first line of your derivation exactly. In fact, the mutual information $I(Y;X) = H(Y)-H(Y|X) = H(Y)-H(N)$.  As the separation between the points goes to infinity, $H(y)$ goes to $\log(2)+H(N)$, so $I(Y;X)$ goes to 1 bit using base 2.
Comment: When I said that $H(Y)$ approaches $H(N)+\log(2)$ as the separation of the points goes to $\infty$, I had in mind $p_1=p_2=1/2$.  I should have said that the entropy approaches $H(N)+H(X)$, which is of course $H(N)+\log(2)$ for the case I mentioned.


Bounds on determinants of Gram matrices of a certain form

Question: Let $A \in \text{Mat}_{n,(m+n)}(\mathbb R)$ be a $n \times (m+n)$ real matrix, of the form $A = [I_n \ | \ D]$, where $I_n$ is the identity matrix. We are interested in the determinant of the Gram matrix $\det(AA^T)$.
Question: Is it true that $\det(AA^T) \geq 1$, with the equality holds only when $D = 0$?
The motivation of the question is to understand the volume of the rank $n$ lattice in $\mathbb R^{m+n}$, with the basis being $\{[1,0,\cdots, 0, a_{11},\cdots a_{1m}], \cdots, [0,\cdots, 0, 1, a_{n_1}, \cdots a_{nm}]\}$. If we regard the last $m$ entries as "perturbations" and drop them off, clearly the lattice in $\mathbb R^n$ has volume 1. So the geometric meaning of the question is, perturbations is those extra dimension does not shrink the volume.
It seems intuitively trivial but I did not find a quick proof. Maybe I just missed something silly. Any references/hints are appreciated.
Some smaller cases can be checked immediately. For $n = 2$ and any $m$, the assertion is true by the Cauchy inequality. For $n = 3$ and $m = 1$, I also verified by checking determinants, but naive approach will be too complicated if we test larger $(m,n)$ values.
Answer: Your conjecture is true.
In fact, note that
$$
AA^{\top}=I_n+DD^{\top},
$$
and that $DD^{\top}$ is positive semi-definite, for which it yields an orthogonal diagonalization, i.e.,
$$
DD^{\top}=U\Lambda U^{\top},
$$
where $U$ is orthogonal, and $\Lambda$ is diagonal with each entry non-negative.
As a result, it follows that
$$
AA^{\top}=I_n+DD^{\top}=I_n+U\Lambda U^{\top}=U\left(I_n+\Lambda\right)U^{\top},
$$
with which it follows that
$$
\det\left(AA^{\top}\right)=\det\left(I_n+\Lambda\right)\ge 1.
$$
It is also obvious that the equality holds if and only if $\Lambda=O_n$, if and only if $DD^{\top}=O_n$, if and only if $D=O_{n\times m}$.


Custom module versioning

Question: I'm writing some custom modules (for a customer), but as expected their are showed in error in update dashboard (/admin/reports/updates): "No version available"
I was wondering if there's any way to tell Drupal that these module belong to another webservice to check for latest updates. So I can manage my module versioning and be aware quickly if some of my module are up to date or not.
I searched a lot about it but no result so far, if this is a duplicate feel free to point it out so I can check it out.
Answer: By default, Drupal assumes that any installed module or theme is hosted on drupal.org. This means that, if you are using a custom module (which is probably not hosted on drupal.org), Drupal will return the error you are seeing because it doesn't find any information about the module you are using, on drupal.org.
If you just want to stop Drupal from looking for a module updates, normally because you are using a different method to update it, or because the module is not hosted on drupal.org (for example, a custom module you have on your computer, on Git, Gitlab, or similar services) you can implement hook_update_projects_alter().
function custom_module_update_projects_alter(&$projects) {
  // Hide a site-specific module from the list.
  unset($projects['custom_module']);
}

With this example code, the custom_module module is hiding itself, but a module can hide any module. For example, if I had a custom_core module that is a dependency for every custom module I use on my site, I would implement the hook using code similar to this.
function custom_core_update_projects_alter(&$projects) {
  // Hide all the site-specific modules from the list.
  unset($projects['custom_user_data']);
  unset($projects['custom_entities']);
  unset($projects['custom_spam_prevention']);
}

If you want Drupal checks for updates on a site that is not drupal.org, then the .info.yml file for the module needs to contain the following line. (See the code of the UpdateFetcher class, which is the class that fetches project information from remote locations.)
project status url: http://example.org/project-history

In this case, for a module using my_module as machine name, Drupal will fetch the information about Drupal 8 releases of the module from http://example.org/project-history/my_module/8.x.
Notice also that:

If the .info.yml file also contains a project: my_custom_project line, Drupal will check for information about Drupal 8 releases of the module from
http://example.org/project-history/my_custom_project/8.x, not http://example.org/project-history/my_module/8.x (This is useful when a module is a sub-module of another module, and both are packaged in the same file.)
Drupal expects that http://example.org/project-history/my_module/8.x returns XML data that follows a specific format; you can see for example what drupal.org returns for the Date module (https://updates.drupal.org/release-history/date/8.x)
<?xml version="1.0" encoding="utf-8"?>
<project xmlns:dc="http://purl.org/dc/elements/1.1/">
  <title>Date</title>
  <short_name>date</short_name>
  <dc:creator>vijaycs85</dc:creator>
  <type>project_module</type>
  <api_version>8.x</api_version>
  <project_status>unsupported</project_status>
  <link>https://www.drupal.org/project/date</link>
  <terms>
    <term>
      <name>Projects</name>
      <value>Modules</value>
    </term>
    <term>
      <name>Maintenance status</name>
      <value>Actively maintained</value>
    </term>
    <term>
      <name>Development status</name>
      <value>Under active development</value>
    </term>
  </terms>
  <releases>
    <release>
      <name>date 8.x-1.x-dev</name>
      <version>8.x-1.x-dev</version>
      <tag>8.x-1.x</tag>
      <version_major>1</version_major>
      <version_extra>dev</version_extra>
      <status>published</status>
      <release_link>https://www.drupal.org/project/date/releases/8.x-1.x-dev</release_link>
      <download_link>https://ftp.drupal.org/files/projects/date-8.x-1.x-dev.tar.gz</download_link>
      <date>1380568703</date>
      <mdhash>f36524e70ed83fe7f2812d3768a337c0</mdhash>
     <filesize>164918</filesize>
     <files>
       <file>
         <url>https://ftp.drupal.org/files/projects/date-8.x-1.x-dev.tar.gz</url>
         <archive_type>tar.gz</archive_type>
         <md5>f36524e70ed83fe7f2812d3768a337c0</md5>
         <size>164918</size>
         <filedate>1380568703</filedate>
       </file>
       <file>
         <url>https://ftp.drupal.org/files/projects/date-8.x-1.x-dev.zip</url>
         <archive_type>zip</archive_type>
         <md5>781eb8cb238ed8649c93f0e8e2b9e778</md5>
         <size>232819</size>
         <filedate>1380568703</filedate>
       </file>
     </files>
     <security>Dev releases are not covered by Drupal security advisories.</security>
   </release>
 </releases>

Not all the information provided is required, but for each release there must be a URL for the archive containing the module code, which is used from Drupal to download and copy the module on the site server.

As side note, updating a module files with Composer would be the first step to update the module. Drupal still needs to invoke the update hooks the module provide, and update the module information it stores in its database tables (and cache).
Comment: great tips I did not know!
Answer: It's all in the docs. See Creating custom modules and then the chapter about Prepare a Module skeleton which will lead you to this example module on GitHub: https://github.com/sidharrell/D8HWexample
hello_world.info.yml
name: Hello World Module
description: Creates a page showing "Hello World".
package: Custom
type: module
version: 1.0
core: 8.x

I guess you left out version (which is optional). Simply add it and you should get rid of the error.

I was wondering if there's any way to tell Drupal that these module
  belong to another webservice to check for latest updates.

Well, yes, with Composer. You could start Using Composer to manage Drupal sites, most likely with this template: https://github.com/drupal-composer/drupal-project
You now can include GitHub-hosted repos pretty easily. Here comes an extremely simplified example composer.json that includes MYMODULE hosted on GitHub. The manually added type drupal-custom-module and the added directory take care of the rest. You may additionally add that module to your .gitignore now.
{
    "name": "drupal-composer/drupal-project",
    "description": "Project template for Drupal 8 projects with composer",
    "type": "project",
    "repositories": [
        {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        },
        {
            "type": "package",
            "package": {
                "name": "MYMODULE",
                "version": "1.2.3",
                "type": "drupal-custom-module",
                "source": {
                    "url": "https://github.com/john_doe/MYMODULE.git",
                    "type": "git",
                    "reference": "tags/1.2.3"
                }
            }
        }
    ],
    "require": {
        "composer/installers": "^1.2",
        "davidbarratt/custom-installer": "^1.0@alpha",
        "drupal-composer/drupal-scaffold": "^2.2",
        "drupal/core": "~8.4",
        "drush/drush": "~9.0",
        "MYMODULE": "^1.2.3"
    },
    "conflict": {
        "drupal/drupal": "*"
    },
    "minimum-stability": "alpha",
    "prefer-stable": true,
    "config": {
        "sort-packages": true,
        "bin-dir": "bin/"
    },
    "autoload": {
        "classmap": [
            "scripts/composer/ScriptHandler.php"
        ],
        "files": ["load.environment.php"]
    },
    "extra": {
        "installer-paths": {
            "web/core": ["type:drupal-core"],
            "web/libraries/{$name}": ["type:drupal-library"],
            "web/modules/contrib/{$name}": ["type:drupal-module"],
            "web/profiles/contrib/{$name}": ["type:drupal-profile"],
            "web/themes/contrib/{$name}": ["type:drupal-theme"],
            "drush/Commands/{$name}": ["type:drupal-drush"]
        },
        "custom-installer": {
            "drupal-custom-module": "web/modules/custom/{$name}"
        }
    }
}

Comment: There is a way to stop Drupal complaining with _Version available_ and make Drupal check for updates on a site that is not drupal.org. I will write an answer when I find any reference for Drupal 8.
Comment: "I guess you left out version (which is optional). Simply add it and you should get rid of the error."
The version was specified in the .info.yml file, I think it's just because Drupal cannot retrieve the version from Drupal's public repositories

"Well, yes, with Composer."

This could do the job except that will not show the update status in the Drupal admin update page like other modules does.

Actually those 5 custom modules are part of a custom Drupal profile I made for my customers and are bundled and shipped with this profile.

Custom module versioning


merge multiple data.frame by row in R

Question: I would like to merge multiple data.frame in R using row.names, doing a full outer join.  For this I was hoping to do the following:
x = as.data.frame(t(data.frame(a=10, b=13, c=14)))
y = as.data.frame(t(data.frame(a=1, b=2)))
z = as.data.frame(t(data.frame(a=3, b=4, c=3, d=11)))
res = Reduce(function(a,b) merge(a,b,by="row.names",all=T), list(x,y,z))

Warning message:
In merge.data.frame(a, b, by = "row.names", all = T) :
  column name 'Row.names' is duplicated in the result
> res
  Row.names Row.names V1.x V1.y V1
    1         1         a   10    1 NA
    2         2         b   13    2 NA
    3         3         c   14   NA NA
    4         a      <NA>   NA   NA  3
    5         b      <NA>   NA   NA  4
    6         c      <NA>   NA   NA  3
    7         d      <NA>   NA   NA 11

What I was hoping to get would be:
    V1 V2 V3
  a 10 1  3
  b 13 2  4
  c 14 NA 3
  d NA NA 11

Answer: The following works (up to some final column renaming):
res <- Reduce(function(a,b){
        ans <- merge(a,b,by="row.names",all=T)
        row.names(ans) <- ans[,"Row.names"]
        ans[,!names(ans) %in% "Row.names"]
        }, list(x,y,z))

Indeed:
> res
  V1.x V1.y V1
a   10    1  3
b   13    2  4
c   14   NA  3
d   NA   NA 11

What happens with a row join is that a column with the original rownames is added in the answer, which in turn does not contain row names:
> merge(x,y,by="row.names",all=T)
  Row.names V1.x V1.y
1         a   10    1
2         b   13    2
3         c   14   NA

This behavior is documented in ?merge (under Value)

If the matching involved row names, an extra character column called
  Row.names is added at the left, and in all cases the result has
  'automatic' row names.

When Reduce tries to merge again,  it doesn't find any match unless the names are cleaned up manually.
Answer: For continuity, this is not a clean solution but a workaround, I transform the list argument of 'Reduce' using sapply.
Reduce(function(a,b) merge(a,b,by=0,all=T),
                      sapply(list(x,y,z),rbind))[,-c(1,2)]
   x y.x y.y
1 10   1   3
2 13   2   4
3 14  NA   3
4 NA  NA  11
Warning message:
In merge.data.frame(a, b, by = 0, all = T) :
  column name 'Row.names' is duplicated in the result

Answer: For some reason I did not have much success with Reduce. given a list of data.frames (df.lst) and a list of suffixes (suff.lst) to change the names of identical columns, this is my solution (it's loop, I know it's ugly for R standards, but it works):
df.merg <- as.data.frame(df.lst[1])
colnames(df.merg)[-1] <- paste(colnames(df.merg)[-1],suff.lst[[1]],sep="")
for (i in 2:length(df.lst)) {
    df.i <- as.data.frame(df.lst[i])
    colnames(df.i)[-1] <- paste(colnames(df.i)[-1],suff.lst[[i]],sep="")
    df.merg <- merge(df.merg, df.i, by.x="",by.y="", all=T)
}

merge multiple data.frame by row in R


Difficult Second-Order ODE solution?

Question: I'm looking at the functional
$$T[y] = \int_a^b\sqrt{\frac{1+y'^2}{2g(y-\mu x)}}\ \mathrm dx\tag{1}$$
and trying to minimize it via the Euler-Lagrange-equations, which I can do, however, it seems that the simplification
$$(1+y'^2)(1+\mu y')+2(y-\mu x)y'' =0\tag{2}$$
is correct, and exactly how the rather nasty output of the E-L-equation simplifies to this is eluding me. I'm also rather interested in how one goes about solving such differential equation... Any help is appreciated.
Answer: FWIW, it is possible to obtain a first integral via the Beltrami identity. The trick is to introduce Cesareo's variable $$Y~:=~y-\mu x$$ to remove explicit $x$-dependence in OP's Lagrangian
$$ L~=~\sqrt{\frac{1+y^{\prime 2}}{y-\mu x}}
~=~ \sqrt{\frac{1+(Y^{\prime}+\mu)^2}{Y}}.$$
The momentum & energy read
$$ P~=~\frac{\partial L}{\partial Y^{\prime}}
~=~\frac{y^{\prime}}{\sqrt{Y}\sqrt{1+y^{\prime 2}}}$$
and
$$ E~=~PY^{\prime}-L~=~-\frac{1+\mu y^{\prime}}{\sqrt{Y}\sqrt{1+y^{\prime 2}}},$$
respectively. The energy is a first integral.
Answer: Hint.
You can get
$$(1+y'^2)(1+\mu y')+2(y-\mu x)y'' =0$$
keeping the numerator from the messy fraction. After that, making the transformation $Y = y-\mu x$ gives
$$
(1+(Y'+\mu)^2)(1+\mu(Y'+\mu))+YY''=0
$$
This nonlinear DE is solved with the contribution of MATHEMATICA
$$
y = \text{InverseFunction}\left[\frac{2 \left(\sqrt{\text{$\#$1} \left(e^{2 c_1} \left(\mu
   ^2+1\right)-\text{$\#$1}\right)}\pm\text{$\#$1} \mu \right)+e^{2 c_1} \left(\mu ^2+1\right) \tan ^{-1}\left(\frac{2
   \sqrt{\text{$\#$1} \left(e^{2 c_1} \left(\mu ^2+1\right)-\text{$\#$1}\right)}}{2 \text{$\#$1}-e^{2 c_1} \left(\mu
   ^2+1\right)}\right)}{2 \left(\mu ^2+1\right)}\&\right]\left[c_2+x\right]
$$

Difficult Second-Order ODE solution?


Iterative Deletion Algorithm to Test Robustness?

Question: Just had an idea, and was wondering if there were existing strategies to implement it:
Let's say I run some well-specified linear regression and discover that a coefficient of interest (say a treatment variable in an RCT specification) is significant.  Is there a strategy for discovering the fewest number of observations that would need to be deleted from your sample in order to negate the effect found for a particular dependent variable?
The brute force approach of leaving an observation out, and then every pair-wise observations out, and then every triplet, and so on, is obviously computationally prohibitive.  Is there a way to make the search more efficient, though? Looking for high leverage points, for example?
This is more than just looking for outliers.  Some combination of observations may collectively exert some kind of unique influence, but individually may blend in during visual inspections.
Answer: There are two separate ways of specifying 'the smallest number of records' to negate the effect.  The first is to remove a specific subset of records, and the second is to remove a random subset of size n.
You describe the way to find the first method in your question.  This will give you a specific subset of n records where removing them will negate the variable effect and there is no subset of n-1 records that will negate the effect.  To find this you need to do an exhaustive search of looking at every record, every pair, every triplet, etc.
If instead you want to know on average, what is the minimum number of randomly chosen records that you can remove to negate the effect, then you can randomly remove subsets of different sizes until you find the smallest generic subset without the effect.  Start by randomly removing 1/2 of your records and look for the effect (do this 10 times).  If the effect is not there then remove 1/4 of the records from your full dataset and look for the effect. If the effect was there then remove 3/4 of the records from the full dataset and try again.  Each iteration should cut the distance between unknown points in half.
EX:
Remove 1/2 - effect is there
Remove 3/4 - effect is not there
Remove 5/8 - effect is there
Remove 11/16 etc.
Comment: Thanks Bryan -- I have thought about this strategy.  The issue is that this random strategy does not resolve the specific observations that would need to be removed (just, on average, what proportion of the dataset would need to be deleted).  If there were 4 specific observations that, if deleted, would undo the effect -- how would I find these?  An exhaustive search isn't possible.  Are there heuristics to figure out ahead of time which four observations these are likely to be?
Comment: The only way I know that will give you a specific subset of 4 that fits your criteria, while being sure there are no subsets of 1, 2, or 3 that would fit your criteria is to do an exhaustive search of every single, double, and triple in the dataset.

I would guess that there might be heuristics that help you be reasonably sure that there are no subsets of 1, 2, or 3 that fit your criteria.  I would be surprised if there are any that let you be 100% sure


How do I calculate sample size so I can be confident that the sample mean approximates the population mean?

Question: Unfortunately, it's a long while since I did statistics and despite reading & research I'm not 'confident' as to how to calculate this correctly.
I would like to know the smallest sample size required in order to have a given confidence level that the sample mean would be with a given % of the population mean.
Whilst arbitrary, the following would be known (can be calculated):

population size
population mean
population standard deviation

Sampling would be without replacement.
Distribution is normal.
For example, for a population of 1,000,000 with a mean of 0.90 and a population standard deviation of 1.32 I would need a sample n to be 99% confident that the sample mean is within 1% of the population mean.
I'm interested in understanding the formula as I have to solve this many times for different populations, different confidence levels, and different margins of error. Thank you.
Comment: see [Chebyshev's inequality](http://en.wikipedia.org/wiki/Chebyshev%27s_inequality)
Answer:
For example, for a population of 1,000,000 with a mean of 0.90 and a population standard deviation of 1.32 I would need a sample n to be 99% confident that the sample mean is within 1% of the population mean.

Okay.

Sampling would be without replacement.

With a million in the population?
To a first approximation, it doesn't matter enough to be worth worrying about
Actually, turns out in this case it does. I'll do it both without replacement and with. With replacement is simpler, and I do it first.

Distribution is normal.

Don't need it. The sample size will be large enough that with the other assumptions, only really strongly non-normal distributions will have any impact.
Can we assume independence (apart from the effect of sampling without replacement)? e.g. sampling completely at random? I'll take it that we can.
$\mu = 0.90$
$\sigma = 1.32$
Want 'to be 99% confident that the sample mean is within 1% of the population mean'.
i.e. Find $n$ such that $P(|\bar{x}-\mu| < .01\mu) = 0.99$
$\bar{x}-\mu \sim N(0, \frac{\sigma^2}{n})$
99% of a normal distribution is within 2.576  s.d.'s of the population mean (this figure is gettable from normal tables, or using a function in a program. I used R)
`
Thus I need $2.576 \times \sigma/\sqrt{n} < 0.01 \mu = 0.009$
Hence $2.576^2 \sigma^2/n < 0.009^2$
Hence $2.576^2 \sigma^2 < n \times 0.009^2$
Or $n > (2.576 \times 1.32/0.009)^2 = 142742.9$
So if $n$ is about 142700, (the means and sd's and normal table values were only accurate to about the same number of figures - only the first 3-4 digits will be meaningful) then the required probability statement should hold.
If we allow for the 'without replacement' the sample size would reduce about 14% percent (google for finite population correction to the variance); other factors are likely to affect you by more than a couple of percent (like not having perfectly random sampling, for one example)

Let's look at the without replacement case using the finite population correction now.
The finite population correction multiplies the variance by a factor $f = \frac{N-n}{N-1} = 1-\frac{n-1}{N-1}$.
Some people approximate this by $1 -\, n/N$, which is easily accurate enough with the large numbers for $n$ and $N$ involved here. However, I'll try to do the first version there.
$2.576^2 \sigma^2 (N-n)/(N-1) < n \times 0.009^2$
$(2.576\sigma/0.009)^2 /(N-1) < n/(N-n) $
$(2.576\sigma/0.009)^2 /(N-1) < 1/[N/n\,\,\,  -1] $
$142743 \times 1000000/1142742 < n$
So (if I did that right), $n > 124912.7$
Or to the accuracy in the normal value, $n$ should be about $124900$.
(assuming the mean and s.d. are actually accurate to at least 4 figures, too)
Calculation check:
Interval half-width =
$(2.576\times 1.32/\sqrt{124900})\sqrt{(1000000-124900)/999999}$
$= 0.00900$
Comment: Thank you so much Glen. I think I follow most of that and will practise with some examples. Given that I told you I was very rusty, how do I represent this in the form 'n = ...."

The equation n = ... will help me test my understanding in Excel.

Thank you again.
Comment: Clair - I screwed up the calculation, sorry, leaving out the 2.576 in later calculation. Please look again. Because of that the finite population correction matters, too.
Comment: Thanks very much for the update & explanation Glen. That is much closer to the answer I was expecting. Just to confirm, this example allows the sample mean to be plus or minus 1% of the population mean?
Comment: Just saw your most recent edits.  The detail you have gone to is wonderful. Thank you.  I am impressed by how far the sample size falls for finite populations. Terrific. Thanks, Clair.
Comment: How far it falls depends on how big a fraction of the population you'd be sampling. If you're only sampling 1% of the population, it makes about 1% difference to the sample size. If you're sampling about 10% of the population, the FPC makes roughly 10% difference to the answer.
Comment: @Glen_b What is the mathematical basis behind of the calculation you're doing? Do you have some reference?
Comment: @RobertSmith Hi Robert, please bu assured that I do not mean to be rude to you or Glen but are you suggesting this is not the correct approach, or that there is another approach?
Comment: @Clair.Gibbon don't be concerned on my account. You should ask such questions.
Comment: @RobertSmith Which calculation - the with replacement or the without replacement?
Comment: @Glen_b With replacement. The part starting with $P(|\bar{x}-\mu| < .01\mu) = 0.99$. I have tried to find this type of problem in books or posts but usually authors don't take the total population into account. Therefore, they infer population samples based on a couple of simple formulas without much explanation depending on known variance or proportion of $\bar{x}$.
Comment: @RobertSmith In the case the OP described, the variance was known, and even a population mean was given. Using the CLT ($n$>100000 after all), $\bar{x}-\mu$ is normal with mean 0 and variance $\sigma^2/n$ when sampling with replacement. The calculation is straightforward manipulation of normal areas.
Comment: @RobertSmith Even without known $\sigma^2$, one could invoke either Slutsky or the convergence of the $t$ to the normal to produce a normal interval based off an $s$ based from a decently sized sample, or use a t-interval for a small one. Similarly, without a known $\mu$, one could base an interval off a sample there as well (taking into account the effect of uncertainty in the estimate), or find ways to solve (perhaps numerically) a more complex formula with $\mu$ occurring in two places.
Comment: @RobertSmith Lastly, one can always try a range of possible values for the parameters and proceed from a position of caution - such as picking the largest resulting sample size in the set of possibilities.
Comment: @Glen_b Yes, I mean that most of the procedures I have seen, don't take this approach with or without known variance and involve an explicit margin of error. Can you give me a reference to a formal explanation of what leads to manipulation of normal areas?
Comment: Hi @Glen_b, I have tried to reproduce your answer for the sample size in Excel, substituting into your formula (2.576 * 1.32 / 0.009 ) ^ 2. I get an answer 142,742.9 - and I just can't figure what I've done wrong. :( Thanks for your effort & time.
Comment: @RobertSmith I don't really know a reference. Well, *a reference* of sorts would be [this text](http://www.openintro.org/stat/textbook.php) (2nd Ed). If you combine eq 4.4 and 4.15, you get the form of CI I used, and the only other thing to do is add the condition that the interval half-width (margin of error, radius) is $\leq 0.01 \mu$.
Comment: @Clair.Gibbon You may not have done anything wrong; the z-value I typed in was rounded off to 4 significant figures, so the answer has *at best* 4 figure accuracy. Since I computed in R, the value in memory may not have been exactly that; or I may simply have typed the final answer wrong. I'll redo it from scratch using the numbers given (which might end up less accurate, but more reproducible).
Comment: @Clair.Gibbon calculations redone. Hopefully these reproduce to more than 3 or 4 significant figures
Comment: @Clair.Gibbon Thanks a lot. By the way, that book looks great.
Comment: @Glen_b Perfect Glen. It appears to have just been a rounding issue.
Comment: @RobertSmith Not sure which book you refer to?
Comment: @Clair.Gibbon The one you linked to: http://www.openintro.org/stat/textbook.php
Comment: @RobertSmith I'm pretty certain I didn't link to this book but I'm glad you did :) - it looks like a great way to improve my own understanding. Thanks.
Comment: @Clair.Gibbon Right!! I think I misread the name. I was referring to Glen_b. Thanks Glen!


Android Mediaplayer won't load

Question: I have a class which calls this service class to play the media player
<code> class MusicService extends Service implements OnCompletionListener {
 MediaPlayer mediaPlayer;

   @Override
  public IBinder onBind(Intent intent) {
 return null;
  }

  @Override
  public void onCreate() {
  mediaPlayer = MediaPlayer.create(this, R.raw.s);// raw/s.mp3
  mediaPlayer.setOnCompletionListener(this);
  }

 @Override
public int onStartCommand(Intent intent, int flags, int startId) {
  if (!mediaPlayer.isPlaying()) {
  mediaPlayer.start();
  }
return START_STICKY;
 }

public void onDestroy() {
   if (mediaPlayer.isPlaying()) {
   mediaPlayer.stop();
   }
   mediaPlayer.release();
 }

   public void onCompletion(MediaPlayer _mediaPlayer) {
 stopSelf();
 }

}
</code>
I don't want it to load the raw file i want it to load a path which a string in the class that calls this service.
<code> String path = "path is here"

 private void playAudio(String url) throws Exception{
 Intent music = new Intent(this,MusicService.class);
 intent.putExtra("paths",path)
 startService(music);
</code>
How would i go about doing this...
The music file comes from the sd card
Comment: where is your music file stored?
Comment: So what is your problem, music is not playing or getting any exception?
Answer: Simply use this---------
<code>MediaPlayer md=new MediaPlayer();
        md.prepare();
        md.setDataSource(path);
</code>
Comment: Yea except when they exit that screen pick a new song and it loads the screen again now both music player are playing over each other
Answer: I don't know what are you doing in your activity but, If your are just passing a music file path in service and then want to start media player in service then I think you have to try something like this,
<code>    MediaPlayer mediaPlayer;
     String musicFile;

    @Override
    public void onCreate() {
       mediaPlayer = new MediaPlayer();
       mediaPlayer.setOnCompletionListener(this);
     }
    @Override
    onStartCommand(Intent intent, int flags, int startId)        {
      musicFile=intent.getExtra("paths");
    try {

            mediaPlayer.reset();
            mediaPlayer.setDataSource(musicFile);
            mediaPlayer.prepare();

        } catch (IllegalArgumentException e) {
         // TODO Auto-generated catch block
          e.printStackTrace();
       }
     if (!mediaPlayer.isPlaying())
       {
        mediaPlayer.start();
        }
      return START_STICKY;
     }
</code>
Comment: after setDatacource you have to put prepare method.When you are going to play,check the player is already set path  set  and playing.

Android Mediaplayer won't load


Huge page buffer vs. multiple simultaneous processes

Question: One of our customer has a 35 Gb database with average active connections count about 70-80. Some tables in database have more than 10M records per table.
Now they have bought new server: 4 * 6 Core = 24 Cores CPU, 48 Gb RAM, 2 RAID controllers 256 Mb cache, with 8 SAS 15K HDD on each.
64bit OS.
I'm wondering, what would be a fastest configuration:
1) FB 2.5 SuperServer with huge buffer 8192 * 3500000 pages = 29 Gb
or
2) FB 2.5 Classic with small buffer of 1000 pages.
Maybe some one has tested such case before and will save me days of work :)
Thanks in advance.
Comment: If there is someone you will probably find them on serverfault.com.
Answer: Because there is many processor I would start by Classic.
But try all.
Perhaps soon 2.5 with superclassic can be great for you.
Answer: just to dig out the old thread for anyone who may need this.
We use fb classic 2.5 on 75GB db, machine almost the same as described one.
SuperServer was inefficient during tests. Buffers and page size changes only made performance a little bit less miserable.
Currently we use Classic with xinetd, page size = 16384, page buffers = 5000,
Answer: SuperServer will use ONLY ONE procesor.
Since you have 24 cores your best option is to use Clasic.
SuperClasic is not yet ready to scale well in a multi processor enviroment.
Answer: Definitely go with one of the 'classic' architectures.
If you're using Firebird 2.5, check out SuperClassic.
Answer: I am currently having a client who has similar requirements.
The best solution for that case was to install FirebirdSQL 2.5 SuperClassic and just leaving the default small caching settings, because if you have free memory (RAM), Windows and also Linux do better caching of the database then firebird does. The Caching feature of Firebird is not really fast, so let the OS do it.
Also depending on what backup-software you use - if it creates full backups of the firebird-database often, then you can deactivate forced writes on the databases. (just do it if you know what you are doing and if know what can happes by deactivating the forced writes).
Comment: I disagree with _Windows and also Linux do better caching of the database than firebird does_

I think you should elaborate on what exactly you mean with that.


Como autenticar um usuário quando o campo "username" recebe valores que podem referenciar colunas diferentes na tabela do banco?

Question: Tenho um formulário de login onde tenho os campos "username" e "password". A dificuldade que venho encontrando é a seguinte: Quero dar a opção do usuário se logar por CPF, email ou telefone (no campo "username"). Sendo que no método attempt() da classe Auth(), que uso pra autenticar, só consigo passar valores que referenciam colunas fixas. Veja como está o meu código no método que faz o login:
public function login(Request $request){
      $array = ["error" => ''];

      $username = $request->input('username'); //Será passado o email,cpf ou telefone
      $password = $request->input('password');

      if($user && $password){

        $token = Auth()->attempt(['email' => $username, 'password' => $password]);

        if(!$token){
          return $this->unauthorized();
        }

        $array['token'] = $token;
      }
  }

Acima, aquela variável $username pode ser cpf, email ou phone (que são colunas distintas da tabela users).
Resumindo, eu quero dar a oportunidade do usuário se logar com a opção que ele desejar entre essas 3 (cpf,email e phone).
A grosso modo, só para ilustrar minha necessidade, seria algo como isso:
$token = Auth()->attempt(['email/cpf/phone' => $username, 'password' => $password]);

Vocês poderiam me ajudar quanto a isso ? Se não fui claro, avisem nos comentários que eu edito e acrescento informações.
OBS: Estou usando Laravel 7.
Comment: olhando a documentação aqui não vejo outra forma de faze-lo a não ser retentando a chamada do attempt, ex if(Auth()->attempt(['email' => $username, 'password' => $password] || Auth()->attempt(['cpf' => $username, 'password' => $password]);
Comment: Se você tiver um meio **totalmente determinístico** para diferenciar `email` de `cpf` e de `phone` fornecido pelo usuário, você pode definir a primeira chave do array associativo dinamicamente. Caso contrário, o mais ideal é montar a _query_ manualmente e, depois de validar a autenticação, logar o usuário utilizando o método `login` (ou `setUser`), veja a [API](https://laravel.com/api/8.x/Illuminate/Contracts/Auth/StatefulGuard.html). Não conheço muito Laravel (nunca usei, mas acho que é esse método mesmo). Enfim, essa é a ideia, espero que tenha dado para entender. :) [...]
Comment: @LucasMiranda, puts, olhei a doc pra caramba também. Queria evitar ao máximo faz isso. Só faria assim se não houvesse outra alternativa.
Comment: @LuizFelipe , então, fiz a pergunta mais no intuito de saber se existe algo no laravel que eu usasse especificamente para isso, sem fazer possíveis "malabarismos". Se eu não encontrar, procurarei alternativas como as citadas por você e pelo Lucas.
Answer: Precisei personalizar uma consulta de login em um projeto do trabalho e descobri como fazer. O processo não está documentado e precisa ler bastante código para entender detalhadamente como funciona. A solução é simples: sobrescreva o método EloquentUserProvider::retrieveByCredentials, o método original tem uma $query em que você pode fazer sua regra de negócio com condições orWhere.
Para isso precisa seguir algumas etapas:

Crie uma classe que estenda \Illuminate\Auth\EloquentUserProvider (nesse exemplo MeuEloquentUserProvider;
Sobrescreva o método retrieveByCredentials de acordo com sua necessidade;
Informe ao framework para usar a classe personalizada MeuEloquentUserProvider no método register de algum service provider, recomendo o \App\Providers\AuthServiceProvider:

use \Illuminate\Auth\AuthManager;

public function register()
{
    $this->app->extend(AuthManager::class, function(AuthManager $manager) {
        return $manager->provider('eloquent', function($app, $config) {
            return new MeuEloquentUserProvider($app['hash'], $config['model']);
        });
    });
}

Os métodos relacionados são:

\Illuminate\Container\Container::extend
\Illuminate\Auth\AuthManager::provider
Comment: Funciona sim. Usei essa estrutura no 5.5, deve funcionar no 7 e 8 pois o código envolvido não mudou quase nada.
Comment: Pedro, apesar da sua resposta ser útil, eu consultei alguns programadores em Laravel (galera do grupo do Laravel Brasil do Telegram) e eles disseram que ela engessa um pouco o código. Eu vou aceitar sua resposta, pq ela é sim funcional, mas vou deixar como resposta a alternativa que encontrei para resolver o problema e que achei ficar mais legível no caso de futuros erros acontecerem. Obrigado!
Comment: Opa, Pedrão. Essa sua resposta foi se baseando em qual versão do Laravel ? Funciona na minha versão que mencionei na pergunta ? Pior que na sexta eu fiz maior engenharia, rs, pra fazer essa situação funcionar. Vou analisar a sua amanhã, pra ver se vale a pena substituir pela engenharia toda que eu fui obrigado a fazer por não ter visto nada que suprisse a minha necessidade.
Comment: tem toda razão, justamente porque essa solução modifica o comportamento de classes do framework, além de ficar "escondida" de quem for fazer manutenção do código no futuro
Answer: Fiz uma personalização (um método em separado) que verifica se o dado, que o usuário que usou para tentar logar, bate com uma das colunas do banco de dados (email, cpf ou phone). Veja esse método que foi criado:
public function verifyLogin($user){

   $u = User::where('email', $user)->orWhere('cpf', $user)->orWhere('phone',$user)->first()->toArray();

   if($u)
   {
     return  $u;
   }

   return false;
}

Se um dos dados baterem, é retornado os dados do usuário e no método attempt() do Auth, nós usamos o id desse usuário que foi encontrado para realizar o login. Veja como ficou o código no método login(), que é o método que faz a autenticação.
public function login(Request $request){

       $array = ["error" => ''];

       $username = $request->user; //Será passado o email,cpf ou telefone
       $password = $request->password;

    if($username && $password){

       //Chamando método para verificar o tipo de login
       $data = $this->verifyLogin($username);

       if(!$data)
       {
           $array["error"] = "Dados Inválidos";
           return $array;
       }

       $token = Auth()->attempt(['id' => $data['id'], 'password' => $password, 'is_active' => 1]);

       if(!$token){
          return $this->unauthorized();
       }

       $array['token'] = $token;

       $user = User::select('id', 'name', 'surname', 'email', 'internal_client_id', 'is_client','is_client_employee',
       'permission_id','is_active','is_spreadsheet')->where('id', '=', $data['id'])->get();

        $array['userData'] = $user;

        return $array;

   }else{
      $array = ["error" => 'Dados não enviados'];
   }

   return $array;
}

OBS: No meu caso, essa lógica foi aplicada levando em consideração que os campos são "unique" e que eles não são obrigatórios.
OBS-2: Esses dois métodos estão dentro da classe AuthController.
OBS-3: O Illuminate\Support\Facades\Auth, o Illuminate\Http\Request e o App\Models\User são importados no arquivo para os métodos dessas classes serem usadas na classe AuthController.


sql union displays duplicates

Question: image of what I want

I have tried the join on x or y and it didn't work, even the group by didn't work.
What almost gave me the result is the query below
SELECT A.Id ,A.AccNo ,A.Name ,B.Id ,B.AccNo1 ,B.AccNo2 ,B.Name
   from Table1  as A
   left outer  join Table2 as B on A.AccNo = B.AccNo1
union
 SELECT A.Id ,A.AccNo ,A.Name ,B.Id, B.AccNo1, B.AccNo2, B.Name,
   from Table1 as A
   left outer  join Table2 as B on A.AccNo = B.AccNo2

After getting the query correct I want to show only the exceptions where there was no link between the tables and its kind of difficult if the T1.ID is repeated
Comment: Dont post images.  Then post table structures is better than data exmaples
Comment: The posted SQL gives you a syntax error
Comment: Which RDBMS are you using?
Comment: It works but it gives duplicates which is what I don't want
Comment: Please explain what the logic is that builds up your required output
Comment: There are no duplicate _rows_ in your result!
Answer: You seem to want a left join:
select t1.*, t2.*
from table1 t1 left join
     table2 t2
     on t1.id in (t2.accno1, t2.accno2);

Answer: Try:
SELECT A.Id ,A.AccNo ,A.Name ,B.Id ,B.AccNo1 ,B.AccNo2 ,B.Name
from Table1  as A
left outer  join Table2 as B
ON A.AccNo = (CASE WHEN  A.AccNo = B.AccNo1 THEN B.AccNo1 ELSE B.AccNo2 END)

Answer: You may nest your original query, and then use max aggregate function with grouping :
SELECT Id ,AccNo ,Name, max(Id2) as Id2, max(Name2) as Name2,
       max(AccNo1) as AccNo1, max(AccNo2) as AccNo2
  FROM
(
  SELECT A.Id ,A.AccNo ,A.Name ,B.Id Id2 ,B.AccNo1 ,B.AccNo2 ,B.Name Name2
     from Table1  as A
     left outer  join Table2 as B on A.AccNo = B.AccNo1
  union
   SELECT A.Id ,A.AccNo ,A.Name ,B.Id Id2, B.AccNo1, B.AccNo2, B.Name Name2
     from Table1 as A
     left outer  join Table2 as B on A.AccNo = B.AccNo2
) q
GROUP BY Id ,AccNo ,Name;

SQL Fiddle Demo
Answer: Do a LEFT JOIN to return the table1 values along with matching table2 values (where t2.accno2 = t1.accno):
select t1.*, t2.*
from table1 t1
left join table2 t2
     on t1.accno = t2.accno2

Or, perhaps you want table2 values for matching accno1's as well?
select t1.*, t2.*
from table1 t1
left join table2 t2
     on t1.accno in (t2.accno1, t2.accno2)

Answer: It this way to resolve:
SELECT
 t1.id,
 t1.accno,
 t1.name,
 (
     SELECT DISTINCT
         id
     FROM
         table2
     WHERE
         accno2 = t1.accno
 ),
 (
     SELECT DISTINCT
         name
     FROM
         table2
     WHERE
         accno2 = t1.accno
 ),
 (
     SELECT DISTINCT
         accno1
     FROM
         table2
     WHERE
         accno2 = t1.accno
 ),
 (
     SELECT DISTINCT
         accno2
     FROM
         table2
     WHERE
         accno2 = t1.accno
 ) FROM
 table1 t1
 LEFT JOIN table2 t2 ON t1.accno = t2.accno1 OR t1.id = t2.id

Comment: Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
Comment: select t1.id,
    t1.accno,
    t1.name,
    (select distinct id from table2 where accno2=t1.accno),
    (select distinct name from table2 where accno2=t1.accno),
    (select distinct accno1 from table2 where accno2=t1.accno),
    (select distinct accno2 from table2 where accno2=t1.accno)
from table1 t1  left  join
     table2 t2
     on t1.accno = t2.accno1 or t1.id=t2.id
Comment: I change code in answer - please marked as good answer. Thanx

sql union displays duplicates


Include specific interaction terms

Question: I have a question regarding interactions in GLM.
I run a Poisson regression with the purpose of predicting claims in insurance.
I have the following problem :
<code>model 1 : CLAIMS ~ X1 + X2
model 2 : CLAIMS ~ X1 + X2 + X1:X2
</code>
X1 has 10 levels, X2 has 4 levels
When I do a deviance test (Chi Square), my result is that model 2 is significantly better compared to model 1 (at the 5% level).
On the other side the AIC of model 2 is worse. (higher)
When I look at the individual parameter estimates and their corresponding standard error, I see that only 3 out of 27 levels from X1:X2 differ significantly from the base level.
I am not sure how to interpret these observations. Can someone help me and say something meaningful about it? And more important, how to deal with it?
My attempt would be: "There are interactions in the model that are meaningful, however most of them are not. This explains the increasing AIC, although the deviance test says that model 2 is the better one." Should I try to construct a few dummy variables for these three interactions that seem significant? (However, I am not sure this is a correct approach, because it is only significant relative to the base level...)
Answer: You seem to have "meaningful" confused with "significant". A significant variable can be unmeaningful and a nonsignificant variable can be meaningful.
The decreasing deviance is because the new terms improve fit. The increasing AIC is because the interactions add a lot of complexity, so much that it is not "worth it" to include the terms.
However, prior to all this is whether the interaction is, in fact, meaningful and important. This is not really a statistical question - or not purely a statistical question. It is also substantive. For your purposes, this means "does it improve my prediction?" and the best way to answer this is probably by dividing the data into training, test and validation sets and then seeing what happens.
Comment: I plan to do the following then. Train three models : model 1 from my example, model 2 from my example and model 3 (which is model 1 + three dummy variables corresponding with the three interactions that are significant). And the I will see which one performes best... Does this seem reasonable to you?
Comment: The usual practice is to use the training set to develop models then test them on the test data set. Including interactions only at a few levels is probably not right.
Answer: One question is whether the 4 and 10 leveled variables $X_1$ and $X_2$ are ordinal or categorical. If they are categorical, then observing 3 out of the 27 to be positive findings strongly indicates that you should be correcting for multiple testing since such a number is consistent with a 0.05 Type I error rate under the null hypothesis.
The word "meaningful" is misleading as far as interpretation is concerned. It's important to recall that the parameters in Poisson GLMs are interpreted as relative rates comparing groups differing by 1 unit in the predictor. What, then, would a significant interaction actually mean about the relative rates for claims comparing units differing in $X_1$ and/or $X_2$?
Tests of dispersion are possibly worth considering here, however you seem to have a vague understanding of what it achieves. It is not necessarily a diagnostic test, nor are diagnostic tests requisite for confidently reporting model estimates. The dispersion parameter is a proportionality parameter that is estimated in quasilikelihood models which handle small, unobserved sources of covariation between observations. For an illustrative example, consult Alan Agresti's Categorical Data Analysis and the section on horseshoe crabs.
Alternately, AIC is a measure of predictive ability. It is used to compare two possibly non-nested models, however in this case you've chosen to compare nested models. If indeed the parameters are categorical, it explains why the addition of 27 parameters in this model dramatically increased the AIC and led to the conclusion that model 2 did not out perform model 1. However, testing for interaction is not best achieved with comparing AICs and you should consider a better approach.
Comment: It is categorical. Thanx for your response.

Include specific interaction terms


Proving reasoning by cases. $P(X \mid Y) = \sum_{z} P(X,z \mid Y)$

Question: I am trying to prove the following statement:

Let $X, Y , Z$ be random variables, then $P(X \mid Y) = \sum_{z} P(X,z \mid Y)$.

I have a sketch of the proof but I do not know if it is correct:
$$\sum_{z} P(X,z \mid Y) = \sum_{z} \frac{P(X,Y,z)}{P(Y)} =$$
$$\frac{\sum_{z}P(X,Y,z)}{P(Y)}$$
Then by the fact that the joint probability distribution must be consistent with the marginal probability:
$$\sum_{z}P(X,Y,z) = P(X,Y) $$
Completing the proof.
Comment: You need to be more specific about the Z's.
Comment: really? that is all the book I'm using says... You mean in the proof statement or my proof?
Comment: The proof seems ok if you are summing $z$ over all the support points of $Z$ (I assume you mean there is a random variable $Z$ in the context). It essentially says that the law of total probability also applicable on conditional probability (in fact a conditional probability share all the properties from the "unconditional" probability we learn)
Comment: When you say _support points_, do you mean all the possible values that the random variable $Z$ can take? And yeah it might be a good idea to specify that there is a random variable $Z$ in the proof.
Answer: Your proof is essentially correct except $X,Y,z$ are not events; they are two random variables and a value. $X=x, Y=y, Z=z$ are events.

For discrete random variables $X,Y,Z$, we use the Law of Total Probability to state:
$$\begin{align}
\mathsf P(X {=} x \mid Y{=}y) \; & =\; \mathsf P(X{=}x,Y{=}y)\,/\,\mathsf P(Y{=}y)
& \textsf{iff } \mathsf P(Y{=}y)\neq 0
\\[1ex] & = \; \sum\limits_{z\in \mathcal Z}\mathsf P(X {=} x, Z{=}z, Y{=}y)\,/\,\mathsf P(Y{=}y) &\textrm{where }\mathcal Z\textrm{ is the support of }Z
\\[2ex]\therefore \mathsf P(X {=} x \mid Y{=}y)  & =\; \sum\limits_{z\in \mathcal Z}\mathsf P(X {=} x, Z{=}z\mid Y{=}y)
& \Box
\end{align}$$


Iterated suspensions of a finite discrete space

Question: If you take iterated suspensions of a one-point space, you get $n$-balls. If you start from a two-point space, you get $n$-spheres.
Is there a natural interpretation to the iterated suspensions of a three-point space ? What about a general discrete finite space ?
Answer: It's easier than it looks. Consider a finite discrete space $X_{n+1}$ with $n+1$ points. This is a wedge sum of $n$ copies of $S^0$, the two-points space, $X_{n+1} = \bigvee^n S^0$. And if $\Sigma$ denotes the reduced suspension, then $\Sigma(X \vee Y) \cong \Sigma X \vee \Sigma Y$. Thus by induction:
$$\Sigma^d X_{n+1} \cong \bigvee^n \Sigma^d(S^0) = \bigvee^n S^d$$
is a wedge sum of $n$ copies of the $d$-sphere.

If you use the unreduced suspension (since you say that the suspension of a point is a ball I guess this is actually the case), the result is a bit weirder
The first suspension of $X_{n+1}$ is $n$ circles, which you can imagine are all arranged in a row from left to right, and they are then glued as follows: the right hemicircle of the first one is glued to the left hemicircle of the second one, the right hemicircle of the second one is glued to the left hemicircle of the third one, and so on. This is of course homotopy equivalent to the wedge sum of $n$ circles, by collapsing the left hemisphere of the first circle.
The pattern continues: $S^d X_{n+1}$ consists of $n+1$ copies of $k$-spheres, glued in a similar manner: the right hemisphere of the first one is glued to the left hemisphere of the second one, the right hemisphere of the second one is glued to the left hemisphere of the third one, and so on. (This is still homotopy equivalent to a wedge sum of $n$ copies of $S^d$). If it helps for visualization, you can also imagine a homeomorphic space that is a $d$-dimensional "rectangular" shape divided in $n$ "chambers" (when $d=1$, this looks like a row of a chessboard).


what does "===" imply in Javascript/Jquery?

Question:
Possible Duplicate:
JavaScript === vs == : Does it matter which "equal" operator I use?
The 3 different equals

I'm trying to understand what is happening here:
 data.toPage = $('div#someID');

if ( typeof data.toPage === "string" ) {
      // sth
      console.log("hello");
      }

So I'm checking for a string am I not? I'm curious because my console "helloes".
Thanks for some input!
Comment: [Here](http://stackoverflow.com/questions/523643/difference-between-and-in-javascript) is the answer.
Comment: Don't coerce types. This *has* to be a duplicate.
Comment: Forget all those silly wannabe's givin easy answers, @sp00m had the answer right to begin with. :P
Comment: http://stackoverflow.com/questions/523643/difference-between-and-in-javascript
Comment: Thanks for all the answers. Still - I was asking why my console says "Hello" = why $('div#someID') passes as === "string".
Answer: ==
This is the equal operator and returns a boolean true if both the operands are equal. JavaScript will attempt to convert different data types to the same type in order to make the comparison. Assuming 'a' to be 2 and 'b' to be 4, the following examples will return a value of true:
a == 2
a == "2"
2 == '2'

===
This is the strict equal operator and only returns a Boolean true if both the operands are equal and of the same type. These next examples return true:
a === 2
b === 4

Answer: The triple equals sign === compares both value and type, whereas the double == only compares value
for example "1" and 1 have the same value (so to speak) but are of different types. Therefore the following will occur:
"1" == 1 //true
"1" === 1 //false

This is a great read for some useful javascript knowledge, which includes the triple equals amongst other good-to-know stuff
Answer: The === comparison operator means that the two values won't have their types modified before the comparison is made, so they need to be of the same type as well as representing the same value for it to return true.
'1' == 1 // true
'1' === 1 // false