A/B tests: z-test vs t-test vs chi square vs fisher exact test

Question: I'm trying to understand the reasoning by choosing a specific test approach when dealing with a simple A/B test - (i.e. two variations/groups with a binary respone (converted or not). As an example I will be using the data below
Version  Visits  Conversions
A        2069     188
B        1826     220

The top answer here is great and talks about some of the underlying assumptions for z, t and chi square tests. But what I find confusing is that different online resources will cite different approaches, and you would think the assumptions for a basic A/B test should be pretty much the same?

For instance, this article uses z-score:
This article uses the following formula (which I'm not sure if it's different from the zscore calculation?):

This paper references the t test(p 152):

So what arguemnts can be made in favor of these different approaches? Why would one have a preference?
To throw in one more candidate, the table above can be rewritten as a 2x2 contingency table, where Fisher's exact test (p5) can be used
              Non converters  Converters  Row Total
Version A     1881            188         2069
Versions B    1606            220         1826
Column Total  3487            408         3895

But according to this thread fisher's exact test should only be used with smaller sample sizes (what's the cut off?)
And then there's paired t and z tests,f test (and logistic regression, but I want to leave that out for now)....I feel like I'm drowning in different test approaches, and I just want to be able to make some kind of argument for the different methods in this simple A/B test case.
Using the example data I'm getting the following p-values

https://vwo.com/ab-split-test-significance-calculator/ gives a
p-value of 0.001 (z-score)
http://www.evanmiller.org/ab-testing/chi-squared.html (using chi
square test) gives a p-value of 0.00259
And in R fisher.test(rbind(c(1881,188),c(1606,220)))$p.value gives
a p-value of  0.002785305

Which I guess are all pretty close...
Anyway - just hoping for some healthy discussion on what approaches to use in online testing where sample sizes are usually in the thousands, and response ratios are often 10% or less. My gut is telling me to use chi-square, but I want to be able to answer exactly why I'm choosing it over the other multitude of ways to do it.
Comment: I found this demonstration pretty helpful. Which shows that z test for proportions is essentially equivalent to chi-square test of homogeneity on the 2x2 contingency table.
http://rinterested.github.io/statistics/chi_square_same_as_z_test.html
Comment: As about $z$- and $t$-tests your question is already answered in here: http://stats.stackexchange.com/questions/85804/choosing-between-z-test-and-t-test
Answer: We use these tests for different reasons and under different circumstances.

$z$-test. A $z$-test assumes that our observations are independently drawn from a Normal distribution with unknown mean and known variance. A $z$-test is used primarily when we have quantitative data. (i.e. weights of rodents, ages of individuals, systolic blood pressure, etc.) However, $z$-tests can also be used when interested in proportions. (i.e. the proportion of people who get at least eight hours of sleep, etc.)

$t$-test. A $t$-test assumes that our observations are independently drawn from a Normal distribution with unknown mean and unknown variance. Note that with a $t$-test, we do not know the population variance. This is far more common than knowing the population variance, so a $t$-test is generally more appropriate than a $z$-test, but practically there will be little difference between the two if sample sizes are large.

With $z$- and $t$-tests, your alternative hypothesis will be that your population mean (or population proportion) of one group is either not equal, less than, or greater than the population mean (or proportion) of the other group. This will depend on the type of analysis you seek to do, but your null and alternative hypotheses directly compare the means/proportions of the two groups.

Chi-squared test. Whereas $z$- and $t$-tests concern quantitative data (or proportions in the case of $z$), chi-squared tests are appropriate for qualitative data. Again, the assumption is that observations are independent of one another. In this case, you aren't seeking a particular relationship. Your null hypothesis is that no relationship exists between variable one and variable two. Your alternative hypothesis is that a relationship does exist. This doesn't give you specifics as to how this relationship exists (i.e. in which direction the relationship goes) but it will provide evidence that a relationship does (or does not) exist between your independent variable and your groups.

Fisher's exact test. One drawback to the chi-squared test is that it is asymptotic. This means that the $p$-value is accurate for very large sample sizes. However, if your sample sizes are small, then the $p$-value may not be quite as accurate. As such, Fisher's exact test allows you to exactly calculate the $p$-value of your data and not rely on approximations that will be poor if your sample sizes are small.

I keep discussing sample sizes - different references will give you different metrics as to when your samples are large enough. I would just find a reputable source, look at their rule, and apply their rule to find the test you want. I would not "shop around", so to speak, until you find a rule that you "like".
Ultimately, the test you choose should be based on a) your sample size and b) what form you want your hypotheses to take. If you are looking for a specific effect from your A/B test (for example, my B group has higher test scores), then I would opt for a $z$-test or $t$-test, pending sample size and the knowledge of the population variance. If you want to show that a relationship merely exists (for example, my A group and B group are different based on the independent variable but I don't care which group has higher scores), then the chi-squared or Fisher's exact test is appropriate, depending on sample size.
Does this make sense? Hope this helps!
Comment: For clarity, chi-squared test and Fisher's exact test are doing the same thing but the p-value is calculated slightly differently. (It's an approximation under chi-squared and an exact calculation under Fisher's exact.) I'll address chi-squared and it will generalize to Fisher's. The issue here is the premise. "If all inferential statistics tests provides a confidence level around whether two samples are drawn from..." - that's not what the chi-squared test does. The null hypothesis for the chi-squared test is that there is no association and the alternative hypothesis...
Comment: ...is that there is some association between the two categorical variables. You're merely testing for the existence of an association and not pre-specifying a certain direction. (There are some lesser-known statistics out there that DO specify a certain relationship, so it is possible; however this is not what the chi-squared test is designed to do.) To infer afterward that there is a particular directional relationship based on a p-value that was calculated under a different set of hypotheses designed to just test for the existence of an association would be a mistake.
Comment: I have been strugling with this: What is the logical difference between a A/B test and a relationship (proportion) test? To me, they are intuitively the same: testing if one group associates with specific characteristics.
Comment: Thanks for the detailed answer! I'm going to go through it in detail - I'm sure I'll have a few questions!
Comment: Could you further explain how the chi-squared and Fisher exact test don't indicate the direction of an effect? If all inferential statistics tests provide a confidence level around whether two samples sets are drawn from different populations or the same population, then what is it about the mathematical theory that won't let you say the directional difference in mean values would hold (B group has higher score)?
Comment: As an example, consider the hypotheses $H_0: \mu = 0$ versus $H_A: \mu \neq 0$ and say you perform a $t$-test and get a $p$-value of 0.04. You would reject the null hypothesis and conclude there's a difference. If your estimate for $\mu$ was above 0, you might be tempted to conclude that the true average $\mu$ is above 0. However, if you had considered the hypotheses $H_0: \mu \leq 0$ versus $H_A: \mu > 0$ with the same data, your $p$-value would be 0.08 and you wouldn't reject the null, assuming $\alpha=0.05$, meaning that you could not conclude that $\mu$ is greater than 0.
Comment: OPs example data had two versions, A and B. In your last paragraph you suggested a Z or T test. Would having a 3 way test change things in anyway? If there were options A, B and C and we want to know if there is a "difference" in conversion rates?
Comment: Can you point to a "reputable source"?
Answer: For a 3 way test you usually use an ANOVA rather than 3 separate tests. Please also check on the Bonferroni correction before multiple testing.
Please use this
https://www.google.com/search?q=testing+multiple+means&rlz=1C1CHBD_enIN817IN817&oq=testing+multiple+means+&aqs=chrome..69i57j69i60l3j69i61j0.3564j0j7&sourceid=chrome&ie=UTF-8


Continuity of inverse using epsilon delta

Question:
Given a continuous function $f(x):[a,b] \to[c,d] $ and its inverse $g(x) : [c,d]\to [a,b]$ prove the continuity of $g(x)$

Since $f$ is continuous I thought I can write for $x=x_1$,$\lim_{x\to x_1} = f(x_1)$ due to continuity, and, $|x-x_1| \lt \delta \implies |f(x)-f(x_1)| \lt \varepsilon$.
But now for $g(x)$ I get $|x-x_1| = |g(f(x)) - g(f(x_1))| \lt \delta \implies |f(x)-f(x_1)| \lt \varepsilon$
But rather I understand what I need is $|f(x)-f(x_1)| \lt \delta \implies |g(f(x))-g(f(x_1))| \lt \varepsilon$ which I am not able to do
Also is it justified to take for granted $g(f(x)) = x$ for all $x\in[a,b]$ ?
Answer: Yes, we have $g(f(x)) = x$ for all $x \in [a,b]$. Furthermore $f(g(y))=y$ for all $y \in [c,d].$
With sequences: let $y_0 \in [c,d]$ and $(y_n))$ a sequence in $[c,d]$ such that $y_n \to y_0$. We have to show that $g(y_n) \to g(y_0)$.
To this end put $x_n:=g(y_n)$. Then $(x_n)$ is a sequence in $[a,b]$. Let $x_0$ be an  accumulation point of $(x_n)$. Then there is a subsequence $(x_{n_k})$ such that $x_{n_k} \to x_0$ as $k \to \infty$.
Since $f$ is continuous, we have $y_{n_k}=f(x_{n_k}) \to f(x_0)$.
This gives $f(x_0)=y_0$, hence $x_0=g(y_0)$.
Conclusion: the bounded sequence $(g(y_n))$ has exactly one accumulation point: $g(y_0)$.
This shows that $g(y_n) \to g(y_0)$.
Comment: Thank you but what is accumulation point
Comment: Let $(z_n)$ be a sequence. $z_0$ is an  accumulation point of this sequence, if there is a subsequence $(z_{n_k})$ with $z_{n_k} \to z_0$


Comparison of different covariance function for GP model?

Question: Currently I am using squared exponential covariance function for GP model construction for non linear model. It seems Squared exponential covariance work well however I do not have any idea which covariance would might work instead of squared exponential covariance functions. I am using matlab tool for GP model construction, I would be happy if I can get some covariance of similar characteristics of squared exponential.
Answer: Without knowing more about your problem, it's difficult to say which kernels might work better. The squared exponential kernel is often cited as being "too smooth" to be realistic.
In terms of other options, a good starting place might be the kernel cookbook, which has a nice list of other covariance kernels with illustrations and some intuitions. The author of the kernel cookbook also has code which automatically searches for a good kernel given your data (using Matlab and numpy). There are also a number of examples given in chapter 4 of the Gaussian process book.  If you are interested in implementations, the julia Gaussian Processes package has quite a few kernel implementations and generally similar syntax to Matlab.


NFS share: no space left on device

Question: My setup consists of a server, running Ubuntu 10.10, and a device, running a customized Linux. The device gets its kernel and mounts its root filesystem from the server using TFTP and NFS, respectively.
In my exports file I have:
/export/rootfs 192.168.1.0/24(rw,no_root_squash,no_subtree_check,sync)
I have a file containing the EXT2 filesystem which I mount onto /export/rootfs:
mount -t ext2 -o loop TargetFS.ext2 /export/rootfs
When I boot my device I get "no space left on device during initialization, although it has space. If I copy the contents of TargetFS.ext2 to /export/rootfs instead of mounting it, I don't have this problem.
Any thoughts?
Answer: You are mounting the TargetFS.ext2 file using the loopback driver.  Depending on the location & type of the file, you can mount it rw by using -o rw,loop instead of just -o loop. Despite the fact that you are mounting your file in the same place you are mounting your NFS share, you are still trying to write changes to the file.
Keep in mind that wherever the file resides will have to be writable, and have sufficient space to accommodate any/all changes.  Sometimes, files that you can mount as loopback, still can not be written-to due to compression or whatever... or you decide you don't want to write changes to the raw image...   The alternative is to look at a "fan-out" file-system which will let you mount one file-system as read-only... and a 2nd as read/write on-top as an "overlay".  mini_fo has done this quite well in many embedded devices like openwrt for a very long time.  (image is read-only and a ram-drive is overlay'd on top to allow logs & such to have a place to write to)
Comment: I had tried before mounting with the `rw` option without luck. Anyway, I went to check the permissions and I realized I missed running a script that fixes all the permission issues of `TargetFS.ext2` and now it works! Thanks.
Comment: About the mini_fo, does it do what I wanted in [this](http://serverfault.com/questions/258329/avoid-write-on-a-mounted-loopback-device-to-be-propagated) question? If so, answer me there too :) In the meanwhile I'm trying to compile the module to test in my setup.

NFS share: no space left on device


How do I build sqlcipher?

Question: First, I completed the build using mingw and created the sqlcipher.exe file.
By the way, there was a warning during the build.
So, I doubt if I built it properly.
Warning content:
$ make
./libtool --mode=compile --tag=CC gcc -DSQLITE_HAS_CODEC -DSQLCIPHER_CRYPTO_OPENSSL -I/c/opensslbuild32/include /c/sqlcipher/libeay32.dll -L/c/sqlcipher/ -static-libgcc -DSQLITE_OS_WIN=1 -I. -I/c/sqlcipher/src -I/c/sqlcipher/ext/rtree -I/c/sqlcipher/ext/icu -I/c/sqlcipher/ext/fts3 -I/c/sqlcipher/ext/async -I/c/sqlcipher/ext/session -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -DNDEBUG  -DSQLITE_THREADSAFE=1    -DSQLITE_TEMP_STORE=1 -c sqlite3.c
libtool: compile:  gcc -DSQLITE_HAS_CODEC -DSQLCIPHER_CRYPTO_OPENSSL -I/c/opensslbuild32/include /c/sqlcipher/libeay32.dll -L/c/sqlcipher/ -static-libgcc -DSQLITE_OS_WIN=1 -I. -I/c/sqlcipher/src -I/c/sqlcipher/ext/rtree -I/c/sqlcipher/ext/icu -I/c/sqlcipher/ext/fts3 -I/c/sqlcipher/ext/async -I/c/sqlcipher/ext/session -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -DNDEBUG -DSQLITE_THREADSAFE=1 -DSQLITE_TEMP_STORE=1 -c sqlite3.c  -DDLL_EXPORT -DPIC -o .libs/sqlite3.o
sqlite3.c: In function 'sqlcipher_cipher_profile':
sqlite3.c:21289:8: warning: implicit declaration of function 'fopen_s' [-Wimplicit-function-declaration]
 if(fopen_s(&f, destination, "a") != 0){
    ^~~~~~~
gcc.exe: warning: c:/sqlcipher/libeay32.dll: linker input file because linking not done

especially,I think the sentence is a problem.

Can I ignore the warning?
I am worried that the openssl library is properly linked and built.
Was it a good build?
Comment: The linker misses the implementation of `fopen_s()` called in file sqlite3.c at line 21289. So no, this build failed.
Comment: You say "I am worried that the `openssl` library *is* ...".  Do you mean "*is not*"?
Comment: I think I do not seem to be linked properly.
Answer:
Can I ignore the warning?

No, you cannot.
The linker misses the implementation of fopen_s() called in file sqlite3.c at line 21289:
sqlite3.c:21289:8: warning: implicit declaration of function 'fopen_s' [-Wimplicit-function-declaration]
 if(fopen_s(&f, destination, "a") != 0){
    ^~~~~~~

So no again, this build for libeay32.dll failed.
Comment: An executable cannot be built with a linker error, right? Is it possibly an old executable?
Comment: @FiddlingBits: err what? Which executable?
Comment: Thank you for answer.
May I know what the problem is?
Comment: OP: "First, I completed the build using `mingw` and created the `sqlcipher.exe` file."
Comment: @FiddlingBits: As I wrote: it's the build for a DDL that is failing.
Comment: @hoyo: As I wrote: "*The linker misses the implementation of `fopen_s()` called in file sqlite3.c at line 21289*". This function's implementation should be available either in a library or inside any of the sources belonging to the DLL.
Comment: [`fopen_s()`](https://msdn.microsoft.com/en-us/library/z5hh6ee9.aspx) should come with the [MSC run-time libraries](https://msdn.microsoft.com/en-us/library/abx4dbyh.aspx).
Comment: `/c/sqlcipher/libeay32.dll ` is missing an option, isn't?  Should use `-l`?
Comment: What is the MSC runtime library ..?
I built it by referring to the site
http://www.jerryrw.com/howtocompile.php
Comment: I entered it in migw as follows:
/ Configure --with-crypto-lib = none --disable-tcl CFLAGS = "- DSQLITE_HAS_CODEC -DSQLCIPHER_CRYPTO_OPENSSL -I / c / opensslbuild32 / include /c/sqlcipher/libeay32.dll -L /c/sqlcipher/static -libgcc "LDFLAGS =" - leay32 "
Comment: You better turn to the author of this "how to compile" page.
Comment: Okay..Thank you very much for taking the time to answer.

How do I build sqlcipher?


Python returns TypeError: slice indices must be integers or None or have an __index__ method when searching for a key from a dictionary in a string

Question: I have this code that imports strings from an Excel file. After importing every cell of that file, the data is stored into a 2d array named data[col][row]
I have to define a dictionary that contains keys that can be found in the cells.
One cell can contain multiple keys inside, and the code has to recognize all of them.
import xlrd
book = xlrd.open_workbook("c:/Users/uids3896/desktop/VWexport.xlsx")
print ("The number of worksheets is", book.nsheets)
print ("Worksheet name(s):", book.sheet_names())
sh = book.sheet_by_index(0)
print (sh.name, sh.nrows, sh.ncols)
print ("Cell D30 is", sh.cell_value(rowx=29, colx=3))
data = [[sh.cell_value(r, c) for c in range(sh.ncols)] for r in range(sh.nrows)]

def issa(s):
    try:
        float(s)
        return True
    except ValueError:
        return False

dict={ 'ID' : 'this would be the return for ID '}
dict['1. Press  BT1 button for T= T_SHORT'] =  'this would be the return for BT1SHORTPRESS'
for col in range (sh.ncols):
    for row in range(sh.nrows):
        for key in dict.keys():
            a=data[col][row]
            s=issa(data[col][row])
            if s:
                c=1
            else:
                if a.find(a, key):
                    print (key)

But when I run the code it gives me:
Traceback (most recent call last):
File "C:\Users\uids3896\Desktop\New Text Document.py", line 34, in <module>
if a.find(a, key):
TypeError: slice indices must be integers or None or have an __index__ method

How to properly search if a string contains a key, and not only one, how to check for all of them, and concatenate the coresponding string from the dictionary in the order they occur in the string?
Answer: "How to properly search if a key contains a key?"
If by that you mean "How to check if a dictionary key is present in a string:
for key in dict.keys():
...
    if key in string: <do stuff>

That's because both your keys are strings.
If you want to check that any element of the dictionary's keys is present in the string:
if any(x in str(a) for x in list(dict.keys())):

Example
test_string = 'This is a test string 1. Press  BT1 button for T= T_SHORT testeteditesttest ID blablah'
dict={ 'ID' : 'this would be the return for ID '}
dict['1. Press  BT1 button for T= T_SHORT'] =  'this would be the return for BT1SHORTPRESS'
any(x in test_string for x in list(dict.keys()))

After going over the q&a below, here's how I would do it:
for col in range (sh.ncols):
    for row in range(sh.nrows):
        # Moved reading the value fron col/row out of the for because it made no sense to read it for every key in the dict
        a=data[col][row]
        for key in dict.keys():
            s=issa(data[col][row])
            if s:
                c=1
            else:
                 if key in a:
                    a = a.replace(key, dict[key])
        data[col][row] = a
        <save and close the excel or do other things to it>

Comment: yes, that was what i meant, i edited now.
This is an example of a cell:
"
1. Send valid LF Go with TRUNK Telegram

2. Verify the start time for battery measurement

3. Save the oscilloscope diagrams and put them as remarks in TestRaport

4. Repeat steps 1 and 2 for Keyminder with Trunk Telegram

5. Repeat steps 1 and 2 for Easy Close Telegram
"
Some keywords are in the dictionary and have a corresponding "translation", I need to export for each cell, the coresponding "translations" for the substrings that are keys in the dictionary and ignore the rest.
Comment: So if I understand correctly your dictionary keys are substrings in your parsed string and you want to apply a transformation to that string so that you will replace your keys with their dictionary values in your resulting string. Say your string is '1 2 3 4 blabla 5 6 ' and your dictionary is {'bla': 'notBla'} the result of the transformation, as I understand it, should be the string '1 2 3 4 notBla 5 6'. Is that correct?
Comment: Yes that is absolutelly corect!
Comment: I'll try to modify your implementation and edit the answer. Please test it and update.
Comment: Yes, Thank You very much. It does replace the key if is contained, but Only the first one. If in a string are 2 keys, only the first one is proccessed, but i guess i cand fix that with a loop
Comment: It should replace every key it finds in a (which is the string you read from the cell) with it's translation value. If you only get one value replaced then something is not right. I tested it with the following code and for me it works: `>>> dict = {'1': 'cici', '2': 'dada'}
>>> test_string = '1 2 3 4 1 2 3 4 1 2 3 4'
>>> for key in dict.keys():
...     if key in test_string:
...             test_string = test_string.replace(key, dict[key])
...
>>> test_string
'cici dada 3 4 cici dada 3 4 cici dada 3 4'`


How to insert EntityFramework with circular references withouth Violation of PRIMARY KEY constraint

Question:
Error: Violation of PRIMARY KEY constraint 'PK_dbo.CompanyDtoes'.
  Cannot insert duplicate key in object 'dbo.CompanyDtoes'. The
  duplicate key value is (b20a140d-440b-4a41-b2c3-6763fa752246). The
  statement has been terminated.

PersonDto
public class PersonDto : PartnerDto, IPartner
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime BirthDate { get; set; }
    public string BirthPlace { get; set; }
    public string MothersName { get; set; }
    public string TaxId { get; set; }
    public List<CompanyDto> OwnedCompanies { get; set; }

    ///
    /// Partner címe(i)
    ///
    public List<PersonAddressDto> Addresses { get; set; }

    public PersonDto()
    {
        OwnedCompanies = new List<CompanyDto>();
        Addresses = new List<PersonAddressDto>();
    }
}

CompanyDto
public class CompanyDto : PartnerDto, IPartner
{
    public string TaxNumber { get; set; }
    public int CompanyValue { get; set; }
    public List<PersonDto> Owners { get; set; }

    ///
    /// Partner címe(i)
    ///
    public List<CompanyAddressDto> Addresses { get; set; }

    public CompanyDto()
    {
        Owners = new List<PersonDto>();
        Addresses = new List<CompanyAddressDto>();
    }
}

My DBContext:
public class PartnerDBContext : DbContext
{
    public DbSet<PersonDto> Persons { get; set; }

    public DbSet<CompanyDto> Companies { get; set; }

    public DbSet<AddressDto> Addresses { get; set; }

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        modelBuilder.Entity<PersonDto>()
            .HasKey(k => k.PartnerId);

        modelBuilder.Entity<CompanyDto>()
            .HasKey(k => k.PartnerId);

        modelBuilder.Entity<AddressDto>()
            .HasKey(k => k.ID);
    }
}

I try to insert a new person, that contains some references for already exists companies:
public bool InsertPerson(PersonDto personToInsert)
{
    try
    {
        using (var db = new PartnerDBContext())
        {
            db.Persons.Add(personToInsert);
            db.SaveChanges();
        }
    }
    catch (Exception ex)
    {
        return false;
    }

    return true;
}

My problem that, I can't insert it, cos it writes Violation of primary key for CompanyDto. I know it's already exists and I don't want to add a new one, but how should I add it? I use it in a WCF service, that called from UWP. Unfortunately can't use DataAnnonations using from UWP (it's a bug), so I use ModelBuilder...
Answer:     public bool InsertPerson(PersonDto personToInsert)
    {
        try
        {
            using (var db = new PartnerDBContext())
            {
                var companies = personToInsert.OwnedCompanies;

                personToInsert.OwnedCompanies = new List<CompanyDto>();

                foreach (var company in companies)
                {
                    var companyInDb = db.Companies.Find(company.PartnerId);
                    personToInsert.OwnedCompanies.Add(companyInDb);
                }

                db.Persons.Add(personToInsert);
                db.SaveChanges();
            }
        }
        catch (Exception ex)
        {
            return false;
        }

        return true;
    }

I found that solution if I get the companies from the db it saves everything as I wanted.
Answer: The problem is that when you add the person entity to the context, it also adds the related entities and marks them as Added (i.e. new). The SaveChanges() in turn tries to insert them in the database and you get the duplicate PK exception.
The solution you posted works, but involves unnecessary database trips for  retrieving the related objects. Since you know they are existing, you can avoid that by simply attaching them to the context in advance, which will mark them as Unchanged (i.e. existing). Then SaveChanges will insert only the person record and the links.
using (var db = new PartnerDBContext())
{
    foreach (var company in personToInsert.OwnedCompanies)
        db.Companies.Attach(company);
    db.Persons.Add(personToInsert);
    db.SaveChanges();
}

Alternatively you can mark them as Unchanged after adding the person to the context:
using (var db = new PartnerDBContext())
{
    db.Persons.Add(personToInsert);
    foreach (var company in personToInsert.OwnedCompanies)
        db.Entry(company).State = EntityState.Unchanged;
    db.SaveChanges();
}

In my test (latest EF6.1.3, short lived new DbContext as in the posted sample) both approaches work.
Comment: I tried this before, but it also wrote me the same error  when I tried to attach. :o ...
Comment: I've tested it before posting, and It works in my environment (latest EF6.1.3 if that matters).
Comment: First solution writes me: Attaching an entity of type 'andocdemo.Model.Dto.Partner.CompanyDto' failed because another entity of the same type already has the same primary key value. This can happen when using the 'Attach' method or setting the state of an entity to 'Unchanged' or 'Modified' if any entities in the graph have conflicting key values. This may be because some entities are new and have not yet received database-generated key values. In this case use the 'Add' method or the 'Added' entity state to track the graph and then set the state of non-new entities to 'Unchanged' o...
Comment: The second one writes me: Violation of PRIMARY KEY constraint 'PK_dbo.CompanyDtoes'. Cannot insert duplicate key in object 'dbo.CompanyDtoes'. The duplicate key value is (70fb08e5-8c4a-4dad-8cce-d8a378226921).
The statement has been terminated.
Comment: I believe you have tested, but than what can be the problem in my solution?
Comment: Hmm, the only logical reason this code to fail and your current solution to work is if you have companies with duplicate `PartnerId` inside `personToInsert.Compaines` list.
Comment: If you compare  `personToInsert.OwnedCompanies.Select(c => c.PartnerId).Distinct().Count()` with `personToInsert.OwnedCompanies.Count`, are they the different?
Comment: They have same count. I use this DBContext in a WCF project, that is called from an UWP project... It may can causes some problems...


Are there models of set theory where every real number is definable but not every set is definable.

Question: I know that Joel David Hamkins has constructed a model of set theory where every set and hence every real number is pointwise-definable. But, is there a model of set theory where every real number is definable, but not every set is definable?
Comment: Not sure why this got downvoted... perfectly reasonable question. Note although JDH co-wrote a very nice paper on this topic, this specific result doesn't originate with that paper but goes back much further. I think to Sheperdson in the 1950s, but not 100% sure.
Answer: Start with a pointwise definable model satisfying $V=L$, e.g. $L_\alpha$ where $\alpha$ is the least ordinal for which $L_\alpha\models\sf ZFC$.
Over this model force with $\operatorname{Add}(\omega_1,\omega_1)$. Namely, add $\omega_1$ subsets to $\omega_1$ using countable conditions. Let $x_\alpha$ denote the $\alpha$th subset added.
Since we did not add new reals, and since the ground model was $L_\alpha$, all the real numbers are definable still (if not by their original definition, then by relativising it to $L$). However, none of the $x_\alpha$ is definable (without parameters). To see why, simply note that if $p\Vdash\varphi(\dot x_\alpha)$, then there is some $q\leq p$ and $\beta\neq\alpha$ such that $q\Vdash\varphi(\dot x_\beta)$.
Why? Simply take $\beta$ which is not in the support of $p$, let $q$ be the extension of $p$ on which $q(\beta,\xi)=p(\alpha,\xi)$. Then the automorphism of the forcing, $\pi$, given by switching the $\alpha$ and $\beta$ coordinates satisfies that $\pi q=q$, so $\pi q\Vdash\varphi(\pi\dot x_\alpha)$ rewrites itself as $q\Vdash\varphi(\dot x_\beta)$.


Scientific number handling

Question: I am using package qvalue which has the following pre-processing step
<code> if(min(p)<0 || max(p)>1) {
    print("ERROR: p-values not in valid range"); return(0)
    }
    if(length(lambda)>1 && length(lambda)<4) {
    print("ERROR: If length of lambda greater than 1, you need at least 4 values."); return(0)
    }
    m <- length(p)
</code>
I have 6000 values like the following: I could show them due to character limit of the overflow
<code>[1] 0.3671178710 0.9360140780 0.2428194940 0.5185496048 0.3280692392
   [6] 0.4681617243 0.3654880576 0.3121728291 0.6357866236 0.2576397417
  [11] 0.7224965457 0.4020649535 0.1359902232 0.4313780172 0.7543924642
  [16] 0.1915732183 0.4359938726 0.2705147307 0.5155629834 0.4446611542
  [21] 0.6636414268 0.4666097886 0.5121170670 0.7911474854 0.0662147755
  [26] 0.8659648750 0.0307181717 0.0910574293 0.7331402271 0.3078877515
  [31] 0.8331673742 0.4992904913 0.0773711040 0.6134791426 0.6714738843
  [36] 0.6620106441 0.3944466498 0.3330491388 0.7726571107 0.5989331217
  [41] 0.4429749257 0.7650029317 0.7608016901 0.3642432987 0.1672484189
  [46] 0.4305554981 0.4308085746 0.7999056710 0.8117493501 0.3325086551
  [51] 0.7233274303 0.4939756680 0.7763859166 0.8281922847 0.5195117763
  [56] 0.6581468025 0.7082172344 0.7201224910 0.8420571108 0.3118079731
  [5996] 0.3066819785 0.6066206806 0.7524323861 0.6815655815 0.7105895186
</code>
Whenever I fun the function for this variable:
    fdrq = qvalue(as.numeric( nd[,3]), fdr.level = 0.05 )
I am getting error:
<code>[1] "ERROR: p-values not in valid range"
</code>
I tried to see the maximum and minimum, looks like it is in range.
<code>> max(nd[,3])
[1] 1
> min(nd[,3])
[1] 0.0001641695
</code>
Please provide suggestion on what might be wrong ! is this something special on handling of decimal numbers .....  or this is a bug ...
Thanks;
Comment: What do `min(nd[,3])<0` and `max(nd[,3])>1` print?
Comment: > min(nd[,3])<0
[1] FALSE
> max(nd[,3])>1
[1] TRUE
Answer: I suspect that you have a value that's a tiny bit greater than 1, but that gets printed as "1":
<code>> x <- 1.000000000000001
> x
[1] 1
> x > 1
[1] TRUE
</code>
You can configure R to display more digits:
<code>> options(digits=20)
> x
[1] 1.000000000000001
</code>
A simple workaround might be to force all values greater than 1 to exactly 1.0:
<code>> x <- c(1, 0.7, 1.000000000000001, 0.3, 0)
> x
[1] 1.000000000000000 0.700000000000000 1.000000000000001 0.300000000000000
[5] 0.000000000000000
> x[x>1] <- 1
> x
[1] 1.0 0.7 1.0 0.3 0.0
</code>
Whether this workaround is a sensible thing to do in your case, I don't know.
Comment: Your work-around defeats the purpose of the pre-processing step.  It would be better to investigate how that number got to be slightly larger than 1 and fix that issue.
Comment: @Joshua Ulrich: Perhaps. I don't think we have enough information to judge either way. I've added a note to that effect.
Comment: Yes, when I set digits 20, I got the following number: 1.0000000000000002220
Comment: How can I force every value that are larger than 1 to 1
Comment: @John: See the last example in my answer (there, `x` is basically `nd[,3]`).
Comment: I am not worried to make >1 to 1 as this p-value has been arising due to tranformation issues ...thanks
Comment: Isn't there a precision limit to consider as well?

Scientific number handling


App Crashes after Button is Pressed

Question: When my app loads, a microphone appears on the screen. When you tap the microphone a label that says "Recording" pops up along with a stop button. When you tap the stop button, the app crashes.
<code>import UIKit

class ViewController: UIViewController {

@IBOutlet weak var recordingInProgress: UILabel!
@IBOutlet weak var stopButton: UIButton!

override func viewDidLoad() {
    super.viewDidLoad()
    // Do any additional setup after loading the view, typically from a nib.
}

override func didReceiveMemoryWarning() {
    super.didReceiveMemoryWarning()
    // Dispose of any resources that can be recreated.
}

override func viewDidAppear(animated: Bool) {
    //Hides the stop button
    stopButton.hidden = true
}

@IBAction func recordAudio(sender: UIButton) {
    //Shows the stop button
    stopButton.hidden = false

    //Shows the recording label
    recordingInProgress.hidden = false

    //TODO: Record Audio
    print("Recording in Progress.")

    }
</code>
This is my crash report:
Recording in Progress.
Recording in Progress.
2015-10-20 15:44:48.247 Vex[1468:533404] -[Vex.ViewController stopButton:]: unrecognized selector sent to instance 0x14d5418a0
2015-10-20 15:44:48.253 Vex[1468:533404] * Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[Vex.ViewController stopButton:]: unrecognized selector sent to instance 0x14d5418a0'
* First throw call stack:
(0x183be4f5c 0x1987dbf80 0x183bebc6c 0x183be8c14 0x183aecdcc 0x1891723c8 0x189172344 0x18915ac6c 0x189171c5c 0x18917188c 0x18916aac0 0x18913ba10 0x189139efc 0x183b9c5a4 0x183b9c038 0x183b99d38 0x183ac8dc0 0x18ec1c088 0x1891a2f44 0x1000322b0 0x1990068b8)
libc++abi.dylib: terminating with uncaught exception of type NSException
Comment: Crashes with what? Crash message / signal? etc?
Comment: show your crash report
Comment: I added my crash report @LordZsolt
Comment: where is your `stopButton(_:)` function?
Answer: Make sure that
<code>@IBOutlet weak var stopButton: UIButton!
</code>
is connected on your storyboard to the actual button.
Answer: if you use storyBoard check the IBAoutlet connection , maybe you connect it twice to the class . please tell the result
Answer: you should check on storyboard/xib that your stop button Touch Up Inside Event is correctly linked with recordAudio action.
otherwise post your log.
Comment: I added the crash report.
Comment: @FelixDoe seems it's a problem on Storyboard. Could you past Connection inspector tab of your button?

App Crashes after Button is Pressed


for some $x$ approaching a maxima , given $f'(x), f''(x)$ can I approximate , in terms of $x$ , when it will reach the maxima?

Question: Assuming that:

$f'(x)$ tells me how far away $x$ is from the maxima; as $x$
approaches maxima then $f'(x)$ approaches $0$.
$f''(x)$ tells me how fast $x$ is approaching the maxima; the more
    negative the second derivative is the faster I'm approaching the
    maxima.

Can I make a "smart" guess on when, at what $x$, it's likely to reach maxima?
Answer: Suppose that you made the computations at $x=a$. So, using Taylor $$f(x)=f(a)+(x-a) f'(a)+\frac{1}{2} (x-a)^2 f''(a)+O\left((x-a)^3\right)$$ Ignoring higher order terms, then $$f(x)\approx f(a)+(x-a) f'(a)+\frac{1}{2} (x-a)^2 f''(a)$$ So, since you want the extremum, $$f'(x)\approx f'(a)+ (x-a) f''(a)=0$$ which gives $$x_*=a-\frac{f'(a)}{f''(a)}\qquad , \qquad f(x_*)=f(a)-\frac{f'(a)^2}{2 f''(a)}$$ For sure, $a$ must not be too far away.
For illustration purposed, let us consider function $f(x)=x^\pi-x^e$ and assume $a=1$. This would give $$f(a)=0\qquad , \qquad f'(a)=\pi-e\qquad , \qquad f''(a)=(\pi -1) \pi -(e-1) e$$ then $$x_*=1-\frac{1}{e+\pi-1 }\approx 0.794233$$  while the exact maximum takes place at $$x=e^{-\frac{\log (\pi )-1}{\pi -e}}\approx 0.710419$$
Let us repeat the process using $a=0.794233$; this will lead to $x_*=0.721358$.
Comment: This is, incidentally, applying [Newton's method](https://en.wikipedia.org/wiki/Newton%27s_method) to find the root of $f'$.
Comment: @WillieWong. It is exactly that ! Cheers :-)


rhel + parted failed after increasing OS VM disk and rescan

Question: we have VM rhel machine version 7.6
example from lsblk
lsblk
NAME             MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
fd0                2:0    1     4K  0 disk
sda                8:0    0   300G  0 disk
├─sda1             8:1    0   500M  0 part /boot
├─sda2             8:2    0  39.5G  0 part
│ ├─VG-RHEL_root 253:0    0  40.9G  0 lvm  /
│ ├─VG-RHEL_swap 253:1    0   7.7G  0 lvm  [SWAP]
│ └─VG-RHEL_var  253:2    0 100.9G  0 lvm  /var
└─sda3             8:3    0   110G  0 part
  ├─VG-RHEL_root 253:0    0  40.9G  0 lvm  /
  └─VG-RHEL_var  253:2    0 100.9G  0 lvm  /var

before using parted
we increase the OS disk from vsphare client from 200G to 300G
then we rescan the sda disk and we get from lsblk 300 instead of previous 200G
now we want to use parted
but we get the following
  parted /dev/sda  resizepart 2 100%
Usage: parted [OPTION]... [DEVICE [COMMAND [PARAMETERS]...]...]
Apply COMMANDs with PARAMETERS to DEVICE.  If no COMMAND(s) are given, run in
interactive mode.

OPTIONs:
  -h, --help                      displays this help message
  -l, --list                      lists partition layout on all block devices
  -m, --machine                   displays machine parseable output
  -s, --script                    never prompts for user intervention
  -v, --version                   displays the version
  -a, --align=[none|cyl|min|opt]  alignment for new partitions

what could be wrong here?
Comment: The `100%` is not supported. You need to use the new `end` position or omit it to maximize the size of the partition.
Answer: Try:
parted -s /dev/sda resizepart 2

Unfortunately the free space is after sda3 partition and you can increase sda2 after you move sda3 towards the end of the disk. If you can shut down the VM, boot it from a GParted Live iso image and resize it with gparted
The syntax is: resizepart partition end See the parted man page for details.
Alternatively, you can also add space the the LVM volume group by:

creating an extended partition
inside the extended partition, create a logical partition of type LVM
use pvcreate to create a LVM disk label on the logical partition. This will be your new LVM physical volume
use vgextend to add the LVM physical volume to the LVM volume group
use lvextend --resizefs to resize the logical volume and the file system that is on top of it.

I wrote a script to help me with resizing of the block devices: https://github.com/mircea-vutcovici/scripts/blob/master/vol_resize.sh
It is just preparing the commands to run, not doing any changes.
Comment: so do you think by replacing 100% to "end" it will succeeded ?
Comment: no. You just need to remove "100%", not replace it with anything.
Comment: we perform the parted -s /dev/sda resizepart 2 , and we get the same errors
Comment: You need to add the extended partition, then logical one, create PV, then add the PV to VG, then increase the LV. Or you can move sda3 to the end of the disk with GParted Live.
Comment: so you  mean to add sda4 ? , in that case when using fdisk can we automate the fdisk process with sda4 creation ?
Comment: Best is to create another disk that you attach to the VM and use only LVM disk label. No MS-DOS or GPT partition table.
Comment: You can add sda4 as extended partition and inside sda5 as logical partition


How to call AJAX on check-box of form API?

Question: I need to get radio buttons in a form, whenever a checkbox gets enabled/clicked.
Say, I have 5 items listed in a checkbox, named orange, apple, .....When I enable orange checkbox, a set of radio button comes that will ask color of the corresponding fruit. Any working example regarding this will be highlyy appreciated.
EDIT:
$form['toggle_me'] = array(
  '#type' => 'checkboxes',
  '#title' => t('Tick this box to type'),
   '#options' => array(0 => t('Orange'), 1 => t('Apple')),
);
$form['settings'] = array(
  '#type' => 'radios',
  '#states' => array(
    // Only show this field when the 'toggle_me' checkbox is enabled.
    'visible' => array(
      ':input[name="toggle_me"][#options][0]' => array(0 => t('red'), 1 => t('yellow')),
    ),
  ),
);

Comment: You may use '#states' attribute from Form API, more information you can find here
http://internetdevels.com/blog/drupal-states
Comment: https://www.drupal.org/project/conditional_fields module will help you to do this task
Comment: Thanks to understand my exact requirement. But I am looking to do it pro grammatically.
Comment: I edit the code, can you please tell what's wrong with this. It's not working.I follow this[https://api.drupal.org/api/drupal/includes%21common.inc/function/drupal_process_states/7] approach,
Comment: ':input[name="toggle_me[1]"]' => array('checked' => TRUE)
Comment: @SugandhKhanna You can use simple jquery too if it doesn't work for you.
Answer: Try This, may be helpful for you
$form['checkbox'] = array(
    '#title' => 'Select',
    '#type' => 'checkboxes',
    '#options' => drupal_map_assoc(array(t('Orange'), t('Apple'))),
    '#ajax' => array(
      'callback' => 'fruits_callback_form',
      'wrapper' => 'fruit-check',
      'method' => 'replace',
      'effect' => 'fade',
    ),
  );
  $checkbox = !empty($form_state['values']['checkbox']) ? $form_state['values']['checkbox'] : 0;
  $form['slots']['#prefix'] = '
';
  if ($checkbox == 'Orange') {
    $color = array(0 => t('Red'), 1 => t('Blue'));
  }
  $form['color'] = array(
    '#type' => 'radios',
    '#title' => t('Select Color'),
    '#options' => $color,
  );
  $form['slots']['#suffix'] = '
';

  //Callback function

  function fruits_callback_form($form, &$form_state) {
    return $form['color'];
  }

Comment: can I write radio form element in ajax callback function?Because I want radio buttons to get visible only if check-box gets enabled.
Comment: then use !empty($form_state['values']['checkbox']) in form.
If not empty then show your radio buttons.
Otherwise as you said, you can define in callback function also


How to prepare the object before request to be compatible with the requirements of the service?

Question: When the service requires an object like in method to change name:
{
  user: {
    data: {
      name: string
    }
  }
}

but, I have object like
{
  user: {
    age: 18,
    height: 150,
    weight: 80
    data: {
      name: 'John',
      surname: 'Juhn'
    }
  }
}

How do I prepare for an object to be compatible with service?
remove unwanted properties with a copy of an object in order not to lose data?
eg.
export function removeUnnecessaryProperty(object: Object, ...necessaryKeys: string[]) {

  Object.keys(object).forEach(key => {
    if (necessaryKeys.indexOf(key) < 0) { delete object[key]; };
  });
}

let objectToRequest = JSON.parse(JSON.stringify(object));
removeUnnecessaryProperty(objectToRequest.data, 'name');
removeUnnecessaryProperty(objectToRequest, 'data');

create a new object only with the required properties?
eg.
export function createObjectWithProperty(object: Object, ...Keys: string[]) {
  let newObject: Object = {};
  Keys.map(key => {
    newObject[key] = object[key];
  });
  return newObject
}
let objectToRequest = {data: undefined}
objectToRequest.data = createObjectWithProperty(object.data, 'name');

or send an object with additional properties?
Answer: Of course you need to pass the exact number of properties.
Why? Don't pass any data that your service doesn't require. Besides this if an hacker can catch your data, he/she will get only that data, not all data.
Think a case when I need to pass only user's name to my Web Api. But my object contains also the email. Why the hacker need also get the email? With name he can't do anything, but with mail he can do many other things.


Multithreading within entity framework context

Question: I have a fairly large query I want to execute in multiple threads to maximize performance.  Here is a generalization of my code:
public ActionResult ControllerAction()
{
    using(var context = new DataEntities())
    {
        var person = context.People.First(p => p.UserId == User.Identity.UserId);
        var model = new PersonViewModel(person);
        return View(model);
    }
}

// Inside the PersonViewModel class

public List<ApplicationViewModel> Applications { get; set; }
public List<MessageViewModel> Messages { get; set; }

public PersonViewModel(Person person)
{
    var applicationsThread = new Thread(PopulateApplications);
    applicationThread.Start(person);
    var messagesThread = new Thread(PopulateMessages);
    messagesThread.Start(person);
    // Other code to be done while application/messages threads run
    applicationThread.Join();
    messagesThread.Join();
}

private void PopulateApplications(object personObject)
{
    var person = personObject as Person;
    Applications = person.Applications.Where(/*logic*/);
}

private void PopulateMessages(object personObject)
{
    var person = personObject as Person;
    Messages = person.Messages.Where(/*logic*/);
}

However, when I hit that controller action, only Applications are populated.  I am guessing this has something to do with trying to access properties of the entity context object from multiple threads, but none of these threads should be colliding with each other.  The hypothesis is strengthened by the fact that switching the thread order causes Messages to be populated but Applications will remain unpopulated.
Edit, Here is the updated version using krillgar's answer:
private static async void GenerateAbilities(int personId)
{
using (var context = new BattleGameEntities())
{
    var personAbilities = await context
        .persons
        .First(p => p.personId == personId)
        .Messages
        .OrderBy(m => m.Date).Select(a => new MessageViewModel
        {
            //Logic
        }).ToListAsync();
}

}
Answer: Correct. The underlying cause of this is the fact that DbContext operates on DataReader. Only one of those can be open at a time.
If you're using .NET 4.5 and EF 6, then you can use the new Async() methods to automatically use multiple threads for all of that. Basically it's just a suffix on any of the methods. Here is an article on using these new methods.
var applications = context.Applications.ToListAsync();
var messages = context.Messages.ToListAsync();

If you're using .NET 4.0, then you'd have to create a new Context in each thread, and maintain them separately. At that point, you'll also run into issues where the objects are coming from separate contexts.
Yes, I know the article is in VB.NET, but the syntax is similar. For whatever reason, that's what all their articles are written in.
Comment: Note that a single context can only have one concurrent async action working against it as of EF 6. Async doesn't get you multiple threads, it gets you responsiveness. You will need multiple context instances for multiple concurrent reads.
Comment: I am using EF6.0 and am targeting the 4.5 framework (NuGet confirms that I have the latest version of EF referenced the project in question) but I don't see an exposed `ToListAsync()` method.  I'll update my post to indicate what is going on.
Comment: @AdamHouldsworth I did not know that. Thanks. I hadn't used that yet as we're still using VS2010 at work.


How to duplicate a line item in shopping cart programmatically

Question: At certain point in the checkout process I'd like to seperate line items with quantity more than one into individual line items with quantity 1.
So for example if I have a line item in cart with quantity 5, I'd like to create 4 new line items just like the original one, add them to the cart with quantity 1 individually.
I tried cloning the line item entity into a new variable and use commerce_line_item_save() with it, but it doesn't work.
Does anybody know how to achieve something like this from a module?
Thank you!
Answer: You can clone a line item and save it without any problems:
$cloned_line_item = clone($line_item);

But if what you want is to add it back to the order, you'll probably want to call commerce_cart_product_add to add your line item back to your order:
commerce_cart_product_add($uid, $line_item, FALSE);

Be careful with setting the combine options correctly if you don't want your line items to be aggregated.
$line_item->data['context']['add_to_cart_combine']

Update: Adding more details on the combine part.
In case you don't want to combine any line items for that product type:
Be aware that the combine options are at formatter level for the add to cart button, so you need to set them everywhere you're using this formatter, in manage display settings for all your view modes, in views... the configuration is not shared. This is very important as there might be hidden places you forgot to configure (if I had a penny every time I forget to configure this in "Teaser", i.e....)
In case you want to combine some times and others not:
The function commerce_cart_product_add above has a combine parameter to make Drupal Commerce try to combine or not and add a new element to the cart or just increase the quantity of an existing one.
On top of that, there's something called comparison properties, by default these are all the fields that the line item has:
foreach (field_info_instances('commerce_line_item', $line_item->type) as $info) {
  if (!empty($info['commerce_cart_settings']['field_access'])) {
    $comparison_properties[] = $info['field_name'];
  }
}

So one way you've got to hook into this process is hook_commerce_cart_product_comparison_properties_alter to define more comparison properties.
Comment: What I described is roughly the process of adding a product to the cart already, so the memory problems must be somewhere else, check your memory allowance and debug that part. I'm updating more details on the combine process in my answer.
Comment: Great answer - the add_to_cart_combine bit is especially important here
Comment: It doesn't seem to work. When I do this as you describe the memory runs out. I've also tried increasing the line item id of the new, cloned item. Quantity update in cart works okay then, but it doesn't add a new item. I've checked combine, it is turned off. Any ideas on what should I be doing differently?


Creating jar file directory across?

Question: At first I have to say, that I've just started learning java a short while ago, so I'm not familiar with the language at all. Due to this, I try to get things done without using an IDE, so I can understand how things work. However, it's not the language that drives me crazy, but the process of making a .jar file.
I have the directories E:\Java\MyLib\mylib.java, E:\Java\Test\PartA\parta.java and E:\Java\Test\PartB\partb.java, which contains my main. mylib.java is a package that gets imported by parta.java, parta.java gets imported by partb.java.
I created a .bat file consulting several tutorials as much as the official specification from oracle about how to use the jar.exe, I've created a valid manifest.txt, I told the programm where it could find the partb.class containing my main, everything gets compiled to its own .class file just fine what tells me that my code is correct, but trying to merge all the files together into one .jar file took me hours without a working result.
According to any instructions I was able to find I'm doing everything right, I tried many different spellings and options, but at the end, either the compiler does not even find the files in the subfolders, or the files are in the .jar, but the javab.class is not able to find them during runtime.
It's sickening. I think I'm missing something, doing something absolutely wrong, but I can't figure out what it is. Any advice would be highly appreciated.
Comment: I use Maven exactly because I don't want to have to deal with this stuff. IMHO, its much more important you understand the relationships between the manifest, dependencies, and your code (which is sounds like you do), than it is to know how to manually package a java app to a jar, because professionals use build tools. Making a jar manually only ensures that you understand how to make it virtually impossible for another developer to reproduce your build process.
Comment: Try including your batch file. You might also consider having a look at ant as an alternative make process
Comment: Including the batch file does not change anything, the imported files are still missing.
Comment: No, include in your post so we see what you're doing. Assuming rye doing it right, but not getting the results you expect suggest you're doing something wrong
Answer: Use an IDE, it makes life so much easier. But, if you insist on doing things the old fashion way, try turning off your antivirus. If that doesn't work, check to make sure your environment variables are set correctly. If all else fails, try reinstalling your JDK.
Comment: I'm going to use an IDE in the near future, but I'm curious about how to get things working "old school".
Comment: Been there, done that. I hate it when people don't give you answers, just tell you to not do it.

Creating jar file directory across?


Align (CSS Rotated) Element to left of its container - HTML / CSS

Question: I have a series of 3 SVG elements inside a wrapper element and I have rotated this wrapper by -90 deg with a CSS transform.
I'm trying to get it so the yellow containing block is pinned to the left of the screen, so I can then alter how far from the left (% or vw) this yellow wrapper sits.
I've reduced the size of the yellow wrapper element so it's easier to see the problem. The yellow wrapper must remain vertically centred like it is now.
I can't for the life of me get it stick to the left - the transform seems to be making everything a lot harder.  I need it so that even if the window is resized it stays a set % (or vw) from the far left - for arguments sake say 5% / 5vw.
P.S I'm using SVG text because it's part of a clip-path animation, but I have removed this code to make things easier to read.
Codepen: https://codepen.io/emilychews/pen/qojZae

body {
  margin: 0;
  display: flex;
  justify-content: center;
  align-items: center;
  width: 100%;
  height: 100vh;
  position: relative;
}

#wrapper {
  display: flex;
  width: 25%;
  justify-content: center;
  background: yellow;
  transform: rotate(-90deg);
  position: absolute;
}

#wrapper svg {margin: 1rem}
text {font-size: 1rem}

  <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="120" height="40" viewBox="0 0 120 40">
    <rect class="masker" x="0" y="0" width="120" height="40" fill="red" />
    <text x="16" y="25" fill="white">Some Text 1</text>
  </svg>

  <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="120" height="40" viewBox="0 0 120 40">
    <rect class="masker" x="0" y="0" width="120" height="40" fill="red" />
    <text x="16" y="25" fill="white">Some Text 2</text>
  </svg>

  <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="120" height="40" viewBox="0 0 120 40">
    <rect class="masker" x="0" y="0" width="120" height="40" fill="red" />
    <text x="16" y="25" fill="white">Some Text 3</text>
  </svg>

Comment: what your question ?
Answer: You can set a transform origin and then adjust the offset from the transform like this. I also removed the Flex properties on body to accomplish this correctly. You can adjust to your liking from this point.

body {
  margin: 0;
  /*display: flex;
  justify-content: center;
  align-items: center;*/
  width: 100%;
  height: 100vh;
  position: relative;
}

#wrapper {
  display: flex;
  justify-content: center;
  background: yellow;
  transform: rotate(-90deg) translate(-100%, 0);
  transform-origin: left top;
  position: absolute;
  top: 0px;
  left: 0px;
}

#wrapper svg {margin: 1rem}
text {font-size: 1rem}

  <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="120" height="40" viewBox="0 0 120 40">
    <rect class="masker" x="0" y="0" width="120" height="40" fill="red" />
    <text x="16" y="25" fill="white">Some Text 1</text>
  </svg>

  <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="120" height="40" viewBox="0 0 120 40">
    <rect class="masker" x="0" y="0" width="120" height="40" fill="red" />
    <text x="16" y="25" fill="white">Some Text 2</text>
  </svg>

  <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="120" height="40" viewBox="0 0 120 40">
    <rect class="masker" x="0" y="0" width="120" height="40" fill="red" />
    <text x="16" y="25" fill="white">Some Text 3</text>
  </svg>

Comment: Hi @bsk this nearly works but the when I reduce the window size the text elements disappear.  I removed the width: 25% that was causing this on the wrapper element but then the layout broke again.
Comment: Hi @TheChewy, I just updated my example. That should work for you now.
Comment: Thanks @BSK.  Really appreciate it
Comment: Glad I could help!
Answer: You can use left on your #wrapper like this:
#wrapper {
  display: flex;
  width: 25%;
  justify-content: center;
  background: yellow;
  transform: rotate(-90deg);
  position: absolute;
  left: 5%;
}

Comment: No that doesn't work I've tried that. I need to be able pin it to the far left, and then add the % value.  Doing that just means the left:0 value is not to the far left so 5% just adds extra to this.
Comment: yes I know but I just showed you that you can use left to move I think that you asked for a percentage too so it was obvious that 0 was the left
Comment: oh wow... does having a value of `left: 5%' move an element 5% to the left?  Wow. Thanks for clearing that up. As a user of the internet I am truly, truly grateful.
Answer: If I understand the question correctly it would be sufficient to remove the justify-content: center on the body element.
Why would you need it there anyway?
If the justify-content is removed, you can then use padding(-left) to move your wrapper more to the center of the screen


Venn representation of $(P \cup Q) \cap R$. What did I do wrong?

Question: Let's say you have a question of sets saying:
$$(P \cup Q) \cap R$$
I answer this way:

(The black part is the shaded)
But the correct answer looks like this:

(The black part is the shaded)
So why is it like this? Aren't you supposed to shade everything if the question wants union?
Comment: why are you ignoring the intercept R ?
Comment: No? I didn't? I shaded R too?
Comment: Why did you shade it *too*, when you should have shaded the intersection *only*?  You wanted (P or Q) *and* R.
Comment: P union Q, that's why
Comment: Then intersect R
Answer: You are supposed to shade everything within the $P$ and $Q$ regions to get their union. But, after that, you only want the part of that union which also belongs to $R$.
Comment: OMG thank you! Now I get it!
Answer: The question is $$(P \cup Q) \cap R$$
the region must intersect with $R$. You have shaded region that are outside $R$.
Comment: Ok, thanks for the tip!
Answer: Note that $(P\cup Q)\cap R$ and $P\cup(U\cap R)$ are different things, and just from $P\cup Q\cap R$ we won't know which one we're asked about. The given answer represents the former. Your answer represents just $P\cup Q$, however, and that is incorrect regardless of which of the two above interpretations is intended.
Comment: Ok then, thanks for the tip!


$A_i \cap A_j$ has $i + j$ elements

Question: I have the next problem:
Find sets $A_1,...,A_n$, such that $|A_i\cap A_j|=i+j$ for $i,j \leq n$ with $i \neq j$
I've tried to solve the problem but I just can't imagine the way to do it. I know the solution by an induction construction in the infinite case, but the finite case is not clear to me. I don't want just say something like "this works in the infinite case, so in the finite would do it too".
Comment: If your "infinite case" is an infinite sequence $A_1,A_2,\dots$ such that $|A_i\cup A_j|=i+j$ for $i\ne j$, then just take the first $n$ elements of that sequence.
Comment: Your closing remark makes me suspect that by "the finite case" you intended to require that each set $A_i, i = 1,\ldots,n$ is finite.  However the one-line problem statement doesn't include that restriction.
Answer: For each pair $(i,j)$ with $1\leq i<j\leq n$ let $B_{i,j}$ be a set such that $B_{i,j}$ has size $i+j$ so that all the $B_{i,j}$ are disjoint.
Make $A_l$ be the union of all the $B_{i,j}$ such that $l=i$ or $l=j$.
For $i < j$ we have $A_i \cap A_j = B_{i,j}$.
Comment: Is not necessary say that $ B_{ij} = B_{ji}$?
Comment: We only define it for pairs with $i<j$
Answer: A quick and dirty approach with induction assuming it works for $n$ you can extend it to $n+1$ by adding $n+1+i$ distinct, not in any $A_j, j\neq i$ elements into $A_i$ and make $A_{n+1}$ the union of all the elements you added.
Example $A_1=A_2=\{1,2,3\}$, then for $3$ we'd take $B_1=\{1,2,3\}\cup\{4,5,6,7\},B_2=\{1,2,3\}\cup\{8,9,10,11,12\},B_3=\{4,5,6,7\}\cup \{8,9,10,11,12\}$

$A_i \cap A_j$ has $i + j$ elements


search algorithms performance

Question: A company database has 10,000 customers sorted by last name, 20% of whom are known to be good customers. Under typical usage of this database, 60% of lookups are for the good customers. Two design options are considered to store the data in the database:

Put all the names in a single array and use binary search.
Put the good customers in one array and the rest of them in a second array. If we do not find the query name on a binary search of the first array, we do a binary search of the second array.

Given these options, answer the following.
i.  Calculate the expected worst-case performance for each of the two structures above, given typical usage. Which of the two structures is the best option?
My approach:
In the worst case for binary search, when the value is not in our array, the algorithm must continue iterating until the span has been made empty; this will have taken at most $log2(N)+1$ iterations. In our situation, it is $log2(10000)+1 = 13.28+1 = 14.28 = 15$? Not sure i'm on the right track. How does 20% of 10000 are good customers and 60% lookups are for good customers taking into account this problem?
ii. Suppose that over time the usage of the database changes, and so a greater and greater fraction of lookups are for good customers. At what point does the answer to part i change?
iii.    Under typical usage again, suppose that instead of binary search we had used linear search. What is the expected worst-case performance of each of the two structures and which is the better option? Where is the cross-over this time?
Answer: Let $X \equiv$ the number of iterations needed in the worst case for the second scenario. Notice that:
$$
X
= \begin{cases}
\lceil \log_2(0.2 \cdot 10000) \rceil + 1 = 12 &\text{if customer is good} \\
\lceil \log_2(0.8 \cdot 10000) \rceil + 1 = 14 &\text{otherwise} \\
\end{cases}
$$
and:
$$
P(X = x)
= \begin{cases}
0.6 &\text{if }x = 12 \\
0.4 &\text{otherwise} \\
\end{cases}
$$
Hence, the average number of iterations is:
$$
E(X) = \sum_x x \cdot P(X = x) = 12(0.6) + 14(0.4) = 12.8
$$
which is smaller than $15$, and so is more efficient than the second scenario.

search algorithms performance


Spearman's rank correlation coefficient calculation

Question: I tried to calculate Spearman's rank coefficient by hand.
Data:

But when I use python, it returns a different value.
rawdata = pd.DataFrame(
    [
        [3,4],
        [5,4],
        [6,2],
        [6,4],
        [8,9],
        [11,7]
    ],
    columns=['Set of A','Set of B'])
print(rawdata)
correlation, pval = spearmanr(rawdata)
print(f'correlation={correlation:.6f}, p-value={pval:.6f}')

It returns:
correlation=0.585239, p-value=0.222365

What am I doing wrong here?
Answer: The formula you're using is a simplified version for when there are no ties. Try the full version of the formula:
$r_s=\frac{\Sigma_i(a_i-\bar{a})(b_i-\bar{b})}{\sqrt{\Sigma_i(a_i-\bar{a})^{2}(b_i-\bar{b})^{2}}}$
Where $a_i$ and $b_i$ are the fractional ranks you've already found in your table
Comment: Thank you for modifying and explaining your notation.  Doing the calculation confirms your answer is a correct explanation of the discrepancy (+1).
Comment: You seem to be writing about the *Pearson* correlation coefficient.  The calculations presented in the question clearly deal with ties (in a standard manner).
Comment: I believe that Spearman correlation is equal to the Pearson correlation of the fractional ranks


Number of times you have to make a bet on a uniform distribution to expect to achieve a minimal result

Question: Edited for the sake of clarity:
If you have a random variable $Q$ distributed uniformly on some interval, say $[a,b]$, what is the function $f$ that describes how many times you have to draw on the distribution to expect to achieve an outcome of at least $c \in [a,b]$?
Comment: The probability of achieving a specific outcome $c$ is $0$.  Are you sure you don't want to know the probability of achieving an outcome that is $\leq c$?
Comment: @BillMance I mean the idea that, if you were to roll the dice a million times you would expect to see some extremes in there
Comment: So I think your question is a little contradictory then.  Do you have a uniform distribution on some set $\{1,2,\cdots,n\}$ or on the interval $[a,b]$?  Your analogy of rolling the dice only really makes sense in the former case.
Comment: I'm having trouble understanding OP's question, but maybe what he means by "rolling the dice" is a geometric distribution?
Comment: @BillMance ... right, I mean if I have a random number generator that generates numbers in $[a,b]$ I would expect my minimum and maximum results to get closer and closer to the extremes as I ran it more times
Comment: So do you mean: for some small $\epsilon>0$, you want to know how many samples you take before you pick a real number that is in $[a,a+\epsilon]$, then?
Comment: @BillMance apologies for being dense - I think I see what you are getting at - I mean that if I am generating real numbers in $[0,1]$, after one draw, I would expect (it would be equally like that it would happen that it would not, I am 50% confident) that the maximal outcome of all of my 1 draws is the midpoint...
Comment: Well, no.  There are uncountably many points in $[0,1]$ so none of them have a positive probability.  The probability of picking any particular real number is zero.  You can only speak of the probability of picking a real number in some set $S \subseteq [0,1]$.  In particular, this ends up being $b-a$ if $S=[a,b]$.  If not then you don't have a uniform distribution on $[0,1]$. See http://en.wikipedia.org/wiki/Uniform_distribution_%28continuous%29
Comment: @BillMance that's what I mean, I mean picking a number in $[\frac{1}{2},1]$
Comment: @BillMance I'm asking, I have a uniform distribution on $[a,b]$. How many times do I have to draw in order to expect (with a 50% probability) a maximum result in $[c,b]$?
Comment: Ok, gotcha.  These links might be helpful.

http://en.wikipedia.org/wiki/Bernoulli_distribution
http://en.wikipedia.org/wiki/Binomial_distribution

So you're looking at a "success" as choosing a real number in $[c,b]$.
Answer: "Throw the dice" $n$ times. Let the results be $X_1,X_2,\dots,X_n$. Let $Y=Y_n$ be the minimum. The probability that this is $\gt y$ is the probability that all the $X_i$ are greater than $y$.
Any particular $X_i$ is greater than $y$, where $a\le y\le b$, with probability $\frac{b-y}{b-a}$. So the probability they all are is $\left(\frac{b-y}{b-a}\right)^n$.
It follows that the cumulative distribution function of $Y$ is
$$1-\left(\frac{b-y}{b-a}\right)^n.\tag{1}$$
Now you can evaluate any probability you want that concerns the random variable $Y=Y_n$.
The expectation: At an earlier stage of your post, you seemed to be asking for the mean of the minimum. We now proceed to calculate that.
Formula (1) shows that the density function of $Y$ is $n(b-y)^{n-1}(b-a)^{-n}$ on our interval  and $0$ elsewhere.
Now we find the expectation of $Y$ in the usual way. So we want to integrate $y$ times the density from $a$ to $b$. Use $y=b-(b-y)$.  So we want
$$\int_a^b \left(bn(b-y)^{n-1}(b-a)^{-n}-n(b-y)^{n}(b-a)^{-n}\right)\,dy.$$
Both parts of the integral are easy to handle. We get
$$b-\frac{n}{n+1}(b-a),$$
which simplifies to
$$\frac{n}{n+1}a+\frac{1}{n+1}b.$$
Nice and simple! The mean of the minimum is $\frac{1}{n+1}$ of the way from $a$ to $b$.
The cdf found in Formula (1), and the mean of $Y$, will be I hope enough for you to solve your applied problem. If there are difficulties with that, just ask.
Remark: It is possible that you may need the maximum $Z$ of our $n$ random variables.
This has very nice expectation also. It is $\frac{1}{n+1}a +\frac{n}{n+1}b$.
The distribution of the maximum is marginally nicer  than the distribution of the minimum, For the probability that the maximum is $\le z$ is $\left(\frac{z-a}{b-a}\right)^n$.
If you want to find the $n$ for which the expectation of the maximum  is greater than $c$, then it is the distribution of the maximum that is relevant. The actual applied problem needs to be described in greater detail before we can see what the appropriate calculations are.
Comment: You then want the probability that the maximum is $\ge c$ to be $0.5$, or equivalently the probability that the maximu is $\le c$ to be $0.5$. So (if you look at my cdf for the max, in remark at end) the breakpoint is given by $\left(\frac{c-a}{b-a}\right)^n=0.5$. So solve for $n$, take the log of both sides. Here we are finding the **median** of the max, not the mean. The mean has a more attractive formula.
Comment: And from there do you just set $E(Y)=c$ and solve for $n$?
Comment: If that's what you want to find, sure. The cdf is quite a bit more informative, since it lets you calculate actual probabilities. It happens to have a simple functional form that one can comfortably do algebra on.
Comment: So you provided the reasoning for the minimum result of $n$ trials, but since I am asking about finding $n$ to achieve an expected maximum, I should use the CDF corresponding to the event that ALL the trials are less than some value $y$?
Comment: Yes, if it is the maximum you want the distribution of, I have added a comment about it, The derivation is about the same, as is the calculation of the mean.
Comment: The problem is then that I don't know what I'm looking for. I want to know how many draws I have to make in order to be 50% sure that at least one of them will meet or exceed $c$. So I think I want the probability that the event that all draws are less than $c$ to be 50% as well. So wouldn't I want to set the CDF for $Y$ from $a$ to $c$ to be 1/2 and solve for $n$?
Answer: "Having a maximum result in $[c,b]$" is one of those statements where the language clouds the issue a bit. This event is equivalent to having at least one result in $[c,b]$, since you couldn't have a result in $[c,b]$ unless one of them is the maximum.
The complement event is easily examined: on each pull you fall outside the interval with probability $\frac{c-a}{b-a}$, so the chance that all $n$ pulls fall outside the interval is $\left(\frac{c-a}{b-a}\right)^n$.
Therefore, we can create a function $g_c(n)=1-\left(\frac{c-a}{b-a}\right)^n$ that is the probability that you have a maximum in $[c,b]$. Remember that you want to input a $c$ and find an $n$. In order to do this, we have to describe what we mean by "expect", so I'll also allow a parameter $p$ to be the certainty that we want to have that the event has occurred.
The problem has been reduced to this: Given $c$, find $f(c )$, the smallest $n$ such that $g_c(n)\geq p$. Some simple algebra gets  $f(c )=\lceil\log_k(1-p)\rceil$ where $k=\frac{c-a}{b-a}$.


How to associate models in rails

Question: In my application there are two models User and Event. I want to associate two models such that one user can create only single event but can login to many events created by the others users.
I have tried the below association but its not working for me
class User < ActiveRecord::Base
  has_one :event
  belongs_to :event
end

class Event < ActiveRecord::Base
  has_many :users
  belongs_to :user
end

How do I associate two models in above condition.
Comment: You're probably not going to be able to do this with associations alone. You will need extra code to manipulate the associations....
Answer: Those relationships refer to the underlying database structure.
When you say belongs_to :event, Rails will look for a column called event_id on the users table. You're telling it that there is one event for every user and the user will store which event it is associated with.
When you write has_one :event, Rails will look for a column called user_id on the events table in the database. You're telling it that there is one event per user and that the event model is responsible for holding which user it is associate with.
If you want the user to only create one event and have each event belong to a user, then you would want the following:
class User < ActiveRecord::Base
  has_one :event
end

class Event < ActiveRecord::Base
  belongs_to :user
end

If you want users to be able to attend multiple events, you'll want a separate table that holds just that information.
class EventRegistration < ActiveRecord::Base
  belongs_to :event
  belongs_to :user
end

You would then add a new row to the event_registrations table for each event you wanted to connect with a given user. This table would be referred to as a join table, since its primary purpose is to create a relationship between records in other tables (which is what a SQL JOIN statement does.)
Comment: You should use has_and_belongs_to_many for attendee relationship. Checkout this docs for it: http://guides.rubyonrails.org/association_basics.html#the-has-and-belongs-to-many-association
Comment: @JakeKaad I avoid that in all production applications. HABTM abstracts away an entire table, which is almost never what I want. Eventually your join table ends up with logic or its own data. You want to deal with it directly.

How to associate models in rails


How to avoid two variables refering to the same data? #Pytorch

Question: During initializing, I tried to reduce the repeats in my code, so instead of:
output=  (torch.zeros(2, 3),
          torch.zeros(2, 3))

I wrote:
z = torch.zeros(2, 3)
output=  (z,z)

However, I find that the second method is wrong.
If I assign the data to variables h,c, any change on h would also be applied to c
h,c = output
print(h,c)
h +=torch.ones(2,3)
print('-----------------')
print(h,c)

results of the test above:
tensor([[0., 0., 0.],
        [0., 0., 0.]]) tensor([[0., 0., 0.],
        [0., 0., 0.]])
-----------------
tensor([[1., 1., 1.],
        [1., 1., 1.]]) tensor([[1., 1., 1.],
        [1., 1., 1.]])

Is there a more elegent way to initialize two indenpendent variables?
Comment: Your first attempt is perfectly fine. If you *really* want to avoid repeating yourself, you can try `output = [torch.zeros(2, 3) for _ in range(2)]` but that is less readable IMO.
Comment: Like lists, tensors in pytorch are mutable data types/objects. You can think of them as the reference to an object so when you use them they will pass the reference rather than values (It helps in speeding up the operations). What you can do is either use copy() or clone() depending on what library you use.
Answer: I agree that your initial line needs no modification but if you do want an alternative, consider:
z = torch.zeros(2, 3)
output=  (z,z.clone())

The reason the other one (output = (z,z)) doesn't work, as you've correctly discovered is that no copy is made. You're only passing the same reference in each entry of the tuple to z
Answer: Assign it in a single statement but use separate reference for each, as below:
h, c = output, output


How to design Mixture of Experts where we want only one active model at a time?

Question: I'm trying to design a Mixture of Experts where we want only one active neural network at a time. Suppose that we have 10 experts. I want  to train a MoE such that only one of the experts is active for a given feature vector.
How should I design this? Of the top of my head: perhaps one way is to have a normal gating mechanism -- use that gating mechanism to assign probabilities to each expert: then pick that expert.
However, the downfall to this approach is that the gating mechanism isn't trained to pick only 1 expert -- they were trained to work in cooperation with each other. So, if I used my proposed approach: I would make bad predictions.
TLDR: How do I design a Mixture of Experts that only has one expert active for a given input?
Comment: Take a look at these works: [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://arxiv.org/abs/2101.03961), [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538). There the authors describe ways to achieve sparsity in the gating weights.
Answer: You can approach this using sparsity-aware regularization.
Let's say your algorithm outputs $$h(x) =
\sum_i{\alpha_i f_i(x)}$$
You want to minimize the size of the set $\{\alpha_i : \alpha_i \neq 0\}$, which is $\|\alpha\|_0$
Your problem has the form of
Minimize $err(h(x), y)$ with $\|\alpha\|_0 \leq 1$
In dual form, for appropriate $\lambda$
Minimize $err(h(x), y) + \lambda \|\alpha\|_0$
Unfortunately that optimization is very hard in general, since $\|\alpha\|_0$ is very bad-behaved function. But there exist alternatives - for example since $\|\alpha\|_1$ can be thought of as convexification of $\|\alpha\|_0$ you might use LASSO (you'd need to find appropriate $\lambda$ yourself though). You could also try using some Matching Pursuit (it seems more in tune with what you want since it's a greedy method, and you'd need to only run one step).


How to sum two fields

Question: I have a content type with the field_watermelons and field_apples fields. I want to record the sum of these fields in the (existing) field_total_fruits field. How can I achieve this?
I do not want to use the Computed field module(because of PHP filter requirement) and the Math Field module is also out because it can't handle Field Collections at the moment (if it could, then I'd have my answer).
I tried hook_form_alter with a callback function attached to the $form['submit']. In that callback, I've tried a number of methods of inserting new values including adding values to $form_state's input, values, completed_form. I've also tried to use entity_metadata_wrapper() approaches in the callback, to no avail. I also tried form_set_value().
How to create a Rules action set to calculate a sum of integers in a Field Collection? looks interesting, but there's no answer.
Surely I'm over-thinking this. Can someone tell me if there's a simple way to sum two field values and insert that sum into the value of another field programmatically?
I suppose I could use the Drupal AJAX system too. I just didn't want that hassle, thinking that there's an easier way, but perhaps it's the best option.
Comment: Comment have been removed. Please don't use them for extended discussion; instead, add the relevant parts to either the question or any of the answers.
Answer: Ok, I've got a solution that will work for me. This d.o forum comment was very helpful.
Instead of changing the $form['submit'] array, prepend the callback to the submit callback stack! Like so:
array_unshift($form['actions']['submit']['#submit'], 'MODULE_my_custom_submit_handler');
Note that the code in the comment from d.o is a bit off, you don't need to assign the unshift'ed array, you just run array_unshift on the array itself.
Once that's added in your hook_form_alter you can do WHATEVER processing magic you want with form_set_value(). Cool! Thanks all.
EDIT: here's what the callback looks like. Also note that for some reason someone took out my fields in my original question. But these field names would make more sense if someone didn't do that.
function MODULE_my_custom_submit_handler($form, &$form_state) {
  $lang = $form['language']['#value'];

  $a = $form_state['values']['field_fruit_collection'][$lang][0]['field_watermelon']['und'][0]['value'];
  $b = $form_state['values']['field_fruit_collection'][$lang][0]['field_apples']['und'][0]['value'];

  $sum = $a + $b;
  form_set_value($form['field_total_fruit'][$lang][0]['value'], $sum, $form_state);
}

Comment: Yes this is, in hindsight, simpler to work with. The fields being stored are redundant, but so what? Now you have this data available as fields, which can be useful along the way.
Comment: However, I would suggest using hook_entity_presave() instead of hook_form_alter(). Consider that a node or entity can be updated outside the usual edit form! With hook_entity_presave() you can be sure that your logic always applies (unless someone writes directly with the database, skipping Drupal APIs)
Comment: I added a section to my own answer.
Comment: vote up, this is better and simpler solution.
Comment: @donquixote very good point, I will look into `hook_entity_presave()` thanks!
Answer: If the value of the field_total_fruits is always the sum of the other fields, then there is not really a point in storing it. Instead, you can compute the value when you want to display it, or when you want to do other things with it.
You now have a number of options, depending what you want to do with the result.
I personally do not like solutions like "computed field", from what I remember from last time I used it.
Actual fields + hook_entity_presave() (EDIT)
Yes, you can store the sum in a dedicated field added to the entity. This means having redundant data in the database, but it also means having the data available to functionality that only works with fields. It can make your life easier.
To do so, you can use hook_entity_presave() to calculate the sum and store it in the new field.
Notice that you probably need to load the field collection items, because the field items of $node->field_fruit_collection will only contain the field collection item ids.
You should pay attention to the array indices [$langcode][$delta] of field items. On a specific site it can be sufficient to always hardcode `[LANGUAGE_NONE][0], but this limits the reusability of the module a lot. And you'd still need to check if the keys are set to prevent PHP notices.
I think it is ok to hardcode some assumptions into your module. You are already hardcoding the field names, so you can also hardcode which of the fields are single-value. So in [$langcode][$delta], the $delta is always 0. Or use reset() to pick the first item. This makes your module unsuitable for publishing on drupal.org, but good enough for your site.
But for the language key you should be more careful. Normally I would say loop over the languages, then store results per language in the field_total_fruit. Maybe use some kind of fallback logic to use LANGUAGE_NONE if no language present.
What complicates things here, is that both the field collection and the field_apples and field_watermelons can have values per language. E.g. you could have $node->field_fruit_collection['fr'][0]['field_collection_entity']->field_apples['en'][0]['value'].
Things like this can go unnoticed for a while, and then you get strange bug reports that make you scratch your head.
Since all of this is numbers, I'd say configure these fields so that only LANGUAGE_NONE is used.
I would say do NOT use field_get_items() here. Or if you do, be careful what you are doing. The entity will be saved in all languages, and field_get_items() filters the result to the current language only. So you could have inconsistent behavior depending on the current user language.
Entity properties
You could use hook_entity_property_info() to register a property with a value callback. In this value callback you would use field_get_items() on both fields and calculate the sum.
Entity properties can be used in a number of places, but they do not naturally show up in field ui for entity view modes, or as views fields.
You would need to do more research to find a way to make them show up in views or elsewhere. Although the hook has some interesting fields like 'entity views field' which might help.
I think it would be worth it to do some more research in this direction, possibly even open a new question. Entity properties are quite a powerful concept, and really there has to be a way to make them show up where you need them.
DS custom field
You could declare a Display suite custom field where you calculate the value. This will only be available in DisplaySuite-enabled view modes, though.
Custom Views field handler
You could write you own views field handler. But these are only available in Views, not elsewhere.
I personally have created some of those, but never really enjoyed working with the API.
Custom "entity display plugin"
There is a collection of modules that I have created myself, that would make this specific task quite easy. The downside is that this is an entire new world to dive into, not something you want to do for just one little task. I am presenting this option here for completeness.
You start with EntDisP. It has a bunch of dependencies, which are listed on the module page. The basic getting-started is explained on the module page.
You would then create a custom module, e.g. "fruitsum" with a custom entity display plugin.
I could elaborate on this further, but this would give it unjust priority over the other options.
The advantage is that these plugins are easy to write and maintain once you get the hang of it, and they can be used in a lot of places.
The disadvantage is that, as said, this is a completely new world to dive into.
Conclusion
I personally would suggest to follow the "entity property" route, with some research of your own. If you find a way to make those things show up, you have a new tool at your disposal for future tasks.
I am even considering to create a new entdisp plugin for the purpose of displaying properties.
Maybe other answers can elaborate more on this direction. Or if you find out, you could write the answer yourself!
Comment: Thanks for the answer. I'm not sure any of these will be a good way forward for me, but I appreciate the ideas!
Answer: So you want to sum 2 fields of a field collection, and you think the Rules module may help you doing so. Indeed, that's a way to do so, have a look at my answer to "How to add up all items of a field collection field using Rules?" which includes a rule (in Rules export format). Here is how the rule looks like:
{ "rules_calculate_sum_of_prices_in_all_field_collection_items" : {
    "LABEL" : "Calculate sum of prices in all field collection items",
    "PLUGIN" : "reaction rule",
    "OWNER" : "rules",
    "REQUIRES" : [ "rules" ],
    "ON" : { "node_view--article" : { "bundle" : "article" } },
    "IF" : [
      { "entity_has_field" : { "entity" : [ "node" ], "field" : "field_article_details" } }
    ],
    "DO" : [
      { "drupal_message" : { "message" : "\u003Cstrong\u003EDrupal calculator\u003C\/strong\u003E started ..." } },
      { "variable_add" : {
          "USING" : { "type" : "decimal", "value" : "0" },
          "PROVIDE" : { "variable_added" : { "total_price" : "Price total" } }
        }
      },
      { "LOOP" : {
          "USING" : { "list" : [ "node:field-article-details" ] },
          "ITEM" : { "article_details_item" : "Article details item" },
          "DO" : [
            { "data_calc" : {
                "USING" : {
                  "input_1" : [ "total-price" ],
                  "op" : "+",
                  "input_2" : [ "article-details-item:field-price" ]
                },
                "PROVIDE" : { "result" : { "calculation_result" : "Calculation result" } }
              }
            },
            { "data_set" : { "data" : [ "total-price" ], "value" : [ "calculation-result" ] } },
            { "drupal_message" : { "message" : "After adding a price of \u003Cstrong\u003E[article-details-item:field-price]\u003C\/strong\u003E for field collection item with id \u003Cstrong\u003E[article-details-item:item-id]\u003C\/strong\u003E, subtotal is \u003Cstrong\u003E[calculation-result:value]\u003C\/strong\u003E." } }
          ]
        }
      },
      { "drupal_message" : { "message" : "The \u003Cstrong\u003ETotal price\u003C\/strong\u003E for all prices included as field collection items is \u003Cstrong\u003E[total-price:value]\u003C\/strong\u003E." } },
      { "drupal_message" : { "message" : "\u003Cstrong\u003EDrupal calculator\u003C\/strong\u003E ended ..." } }
    ]
  }
}

Refer to the related answer for "some" details, and a demo, about it.
To "insert that sum into the value of another field" (as in your question), just add another Rules action after the sum got calculated, and use a "Set a data value" Rules Action.
PS 1: I'm assuming this question is about D7.
PS 2: this answer applies to revision 2 of your question, and does not yet take into consideration your most recent edit you added (and which makes your question a chameleon  question, which IMO should have been a followup question ... instead of changing your original question).
Comment: So... this is interesting. I did tag this question `7` and another user, @kiamlaluno, removed it. And I think that user (or is it a group of users?) is a moderator. So, what's the way to indicate a D7 question? Confused.
Comment: will do. I must say I'm not real happy with the heavy editing of the body of my question either.
Comment: I am going too fast, sorry. Digesting your full answer in the other post now, it is really impressive. Something about this approach *does* seem more sustainable to me. Rules is probably more universal than a callback on a hook form alter. I'm hard up against a deadline at the moment, (when am I not come to think of it) but I will implement this and give it a shot soon.
Comment: I got this when I tried to import `Import failed.
Unable to parse the pasted export.` But I naively didn't change any of the field names... let me see if I that fixes it.
Comment: @cdmo my mistake, tiny copy/paste error from the related answer (let me fix it quickly ... at the very begiinning it's missing an open- parent) ...
Comment: @cdmo OK, do what works for you. Note that I just corrected my exported rule (I had pasted it twice, sorry). If you try an import again it should not cause that import failed anymore. But make sure the field names used match with your field names (otherwise you'll get some other import error). Keep me updated if you ever make progress on this (after you'll get it to work, and fully understand it, you might like what you learned).
Comment: @cdmo how is your deadline doing? I mean: did you have any chance already to qa-review my suggestion?

How to sum two fields


$\lim x \sin (1/x)$, when $x \to 0$

Question: Here's my solution: $$\lim x\sin (1/x) = \lim\, x \dfrac{\sin (1/x)}{x(1/x)} = \lim\, x/x = 1$$ when $x \to 0$
However on the internet I read that the solution of this equation is 0. How can this be? Where a I making a mistake?
Answer: Notice that:
$0\le|x\sin(\frac{1}{x})|\le|x|$
so the limit is $0$. In your solution you wrote:
$\lim_{x\to0}\frac{x\sin(\frac{1}{x})}{\frac{x}{x}}=\lim_{x\to0}\frac{x}{x}$.
I'm afraid I don't see why this is true.
Comment: That's because $|\sin(x)|\le 1$ for all $x\in\mathbb{R}$ (this is still true for $\sin(\frac{1}{x})$ when $x\neq0$ since we are evaluating $\sin$ at real values) and because the absolute value function is non-negative.
Comment: I think it's true because lim (sint / t), when t->0 is 1
Comment: It is true that $\lim_{x\to0}\frac{\sin(x)}{x}=1$ but notice that $\lim_{x\to0^{+}}\frac{\sin (\frac{1}{x})}{\frac{1}{x}}=\lim_{y\to\infty}\frac{\sin(y)}{y}$ by taking $y=\frac{1}{x}$ and noting that as $x$ tends to $0$ from the right then $y$ tends to $\infty$.
Comment: 0≤|xsin(1x)|≤|x|  Why?
Answer: I think you are trying to assume that $$\lim_{x \to 0}\dfrac{\sin(1/x)}{(1/x)} = 1$$ where as in reality we have $$\lim_{x \to 0}\frac{\sin x }{x} = 1$$ and this $x$ in $(\sin x) / x$ can be replaced by any other expression (like $x^{2}, \sin x$ etc) which tends to zero when $x \to 0$. Thus we may write $$\lim_{x \to 0}\frac{\sin(\sin x)}{\sin x} = 1$$
In your case you have committed an error in replacing $x$ by $1/x$. Note that when $1/x$ does not tend to $0$ as $x \to 0$.
Answer: Hint:
$$
\lim_{x\to0}x\sin\left(\frac{1}{x}\right) = \lim_{t\to\infty}\frac{\sin(t)}{t}
$$
Now, try to think why your solution is not true.
Comment: From where is this formula?
Comment: Which formula? I just have used a change of variables $t=1/x$..
Answer: Well see function $\sin\left(\frac{1}{x}\right)$ is bounded between $[-1,1]$  and as $$\lim_{x \to 0}x\sin\left(\frac{1}{x}\right)$$ the value of  $x$ will also go to zero hence value of limit is $0$.
Answer: This solution is not true because the formula is : $\lim_{x\to 0} \frac{\sin(x)}{x} =1$ whereas here there is $\sin(1/x)$ so this can not be applied , $\lim \frac{\sin(1/x)}{1/x}$ will be $1$ when $\frac{1}{x} \to 0$.

$\lim x \sin (1/x)$, when $x \to 0$