View diff contents of 1 specific stashed file in Git

Question: Let's say I have 2 stashed files:
$ git stash show

a.txt | 2 +-
b.txt | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

I can view the complete diff of the stash:
diff --git a/a.txt b/a.txt
index bc56c4d..a688182 100644
--- a/a.txt
+++ b/a.txt
@@ -1 +1 @@
-Foo
+Baz
diff --git a/b.txt b/b.txt
index ebd7525..4105321 100644
--- a/b.txt
+++ b/b.txt
@@ -1 +1 @@
-Bar
+Qux

More files would clutter the output, so I want to view only 1 specific file. However, the manual appears to say there isn't an option for using a filename argument:

NAME
       git-stash - Stash the changes in a dirty working directory away

SYNOPSIS
       ...
       git stash show [<stash>]
       ...

Is there a way to view the diff contents of only 1 specific stashed file?
Answer: Building on How would I extract a single file (or changes to a file) from a git stash? you could:
git diff stash@{0}^! -- a.txt
Comment: Perfect, exactly what I needed.


Printing all occurrences of mapped data from 2 data frames in pandas

Question: I have a dataframe: df1
  SAP_Name  SAP_Class  SAP_Sec
  Avi       5          Avi
  Rison     6          A
  Slesh     7          B
  San       8          C
  Sud       7          B

df2:
Name_Fi Class std
Avi     5     fgh
Rison   6     Rij
Slesh   7     jkh

I am trying to get the mapping of headers basis the values contained in the columns of the 2 dataframes.
When used the below code:
dfs1 = {col1: df1[col1].drop_duplicates() for col1 in df1.columns}
set2 = {col2: set(df2[col2]) for col2 in df2.columns}
d = {}
for col2, v2 in set2.items():
    for col1, v1 in dfs1.items():
        cond = set(v2).issubset(v1)
        if cond:
           d[col2] = col1
print(d)

I have a result:
{'Name_Fi': 'SAP_Name', 'Class': 'SAP_Class'}

Desired output:
{'Name_Fi': ['SAP_Name','SAP_Sec'], 'Class': 'SAP_Class'}

Basically trying to print every matching column header wherever there are matching values in df1. Can anypne pleae help?
Comment: But why should `SAP_Sec` be included? There's no column in `df2` whose values are a *subset* of columns values in `df1['SAP_Sec']`.
Comment: @jpp sorry for the late reply. But if you see the first row of sap sec, it's a subset as it's contains one of the names. Don't know how I missed this comment. But appreciate your solutions. :)
Answer: I think need intersection between sets, so for performance was in dict comprehension converted both values of dictionary to sets:
from collections import defaultdict

dfs1 = {col1: set(df1[col1].drop_duplicates()) for col1 in df1.columns}
set2 = {col2: set(df2[col2]) for col2 in df2.columns}

d = defaultdict(list)
for col2, v2 in set2.items():
    for col1, v1 in dfs1.items():
        cond = v2.intersection(v1)
        if cond:
           d[col2].append(col1)
print(d)
defaultdict(<class 'list'>, {'Name_Fi': ['SAP_Name', 'SAP_Sec'], 'Class': ['SAP_Class']})

Comment: Great..!! I have never worked with intersections. Thanks for the quick help @jezrael
Answer: You aren't looking for issubset here, since this will only work if values in one column are a subset of the values in another, e.g. all the values of df2['Name_Fi'] are contained in df1['SAP_Sec'].. Instead, you can use set.intersection, or the syntactic sugar & to check if there is any overlap.
You can combine this idea with collections.defaultdict extract your desired result:
from collections import defaultdict

d = defaultdict(list)
for col2, v2 in set2.items():
    for col1, v1 in dfs1.items():
        cond = set(v2) & set(v1)
        if cond:
            d[col2].append(col1)

print(d)

defaultdict(<class 'list'>, {'Name_Fi': ['SAP_Name', 'SAP_Sec'],
                             'Class': ['SAP_Class']})

More efficiently, you can calculate set objects in your comprehensions to avoid explicit conversion in your nested loop:
from collections import defaultdict

dfs1 = {col1: set(df1[col1]) for col1 in df1}
set2 = {col2: set(df2[col2]) for col2 in df2}

d = defaultdict(list)

for col2, v2 in set2.items():
    for col1, v1 in dfs1.items():
        if v2 & v1:
            d[col2].append(col1)

print(d)

For larger arrays, you may find using pd.Series.unique + any with a generator expression more efficient:
import pandas as pd
import numpy as np

np.random.seed(0)
A = pd.Series(np.random.randint(0, 1000, 10000)).unique()
B = pd.Series(np.random.randint(0, 100000, 10000)).unique()

%timeit np.in1d(B, A).any()     # 1.1 ms per loop
%timeit set(A) & set(B)         # 1.46 ms per loop
%timeit any(i in B for i in A)  # 102 µs per loop

Comment: @anky_91, For performance you can try `any` [see the benchmarking at the bottom of my answer].
Comment: Thanks @jpp,however the code runs for long when used against large files. My actual file has 112444 rows × 446 columns and the target file to be changed has 3 rows × 35 columns. But thanks a lot for the help, would be referring this for future references. :)


Como gerar um relatório a partir de uma macro gravada?

Question: Boa tarde a todos,
gravei uma macro para gerar um relatório a partir de uma planilha que estou alimentando com dados.
Basicamente a macro vai criar outra pasta de trabalho na mesma planilha e colar os dados na vertical (transpor). Porém, eu preciso voltar a planilha original e copiar outra tabela, aí ao voltar a nova planilha criada a macro dá erro por causa do nome da planilha criada.
ActiveWindow.SmallScroll Down:=-21
Sheets("MUNICÍPIO").Select
ActiveWindow.ScrollColumn = 21
ActiveWindow.ScrollColumn = 22
ActiveWindow.ScrollColumn = 23
ActiveWindow.ScrollColumn = 24
Range("AK5:AN24").Select
Selection.Copy
Sheets("Planilha15").Select

A macro dá um erro nessa última linha, pois o arquivo não vai selecionar "Planilha15" denovo. Gostaria de resolver isso fixando o nome da planilha para Relatório, ou mesmo um nome qualquer, só para não ocorrer este erro.
Obrigado.
Answer: Application.Workbooks("<nome da sua pasta de trabalho com extensão>").Activate

Você também pode apagar isso, é a gravação do scroll do mouse.
ActiveWindow.ScrollColumn = 21
ActiveWindow.ScrollColumn = 22
ActiveWindow.ScrollColumn = 23
ActiveWindow.ScrollColumn = 24


Invite users to a Google Apps domain

Question: I have a google apps domain which is associated with the google sites feature, google email and so on. How do you invite users to your google apps? According to the google apps help, there should be an "Invite Users" link on the dashboard. But since there's no such link, can someone tell me how to do it?
Note: I do not want to create new google accounts, I want to invite existing ones.
Answer: At least for our Google Apps for Education account, we cannot invite members outside of our domain.  We can only create new accounts.  The Google Apps accounts are separate from regular Google accounts.
I have seen documentation that says this is changing (as of a year ago for some!), but for us, the account space is still completely separate.
Comment: Wow, that kind of sucks.. But thanks for your answer!

Invite users to a Google Apps domain


populating an array from a text file using fgets

Question: I have this program for class where we just need to show that we can read, write and populate an array from both a text and binary file. I didnt have many issued with the binary file but the text file is giving big time brain pains.
I believe this little bit of code is to blame. userInputText[MAXSIZE]is a char array (MAXSIZE is defined at 100000) and both textCount and textCounter are initialized to zero when the program is at this point.
textPointer = fopen("textFile.txt", "r");
if (textPointer)
{
    while (fgets(userInputText, sizeof(userInputText), textPointer) != NULL)
    {
        sscanf(userInputText, "%d", &textCount);
        userInputText[textCounter] = textCount;
        textCounter += 1;
    }
}

When the user first runs the program, this part of the program is skipped and the user is asked to create and write to a text file. The user inputs integers and they are written to a text file and stored in userInputText and displayed to the screen at the end of the program. All of that works fine, the text file is appended correctly every time the program runs. I have tried about every combo I can think of in the above piece of code. I have tried using fopen(r+), sizeof(int and MAXSIZE), experimenting with the %d. In my fprintf part later in the program I have "\n" set as a delimeter, but I have also tried removing it. I switched out the fgets for  fread. Ive gotten to the point where I'm resembling a baboon throwing poo at the problem.
I don't want people to think I am lazy and looking for an easy answer, mentally handicapped maybe but not lazy.
Any suggestions?
Thanks,
Mike
Comment: What exactly are you trying to *do*?  This code attempts to read an integer from formatted text-input of a `FILE*` stream, then for some odd reason you update the *same text buffer* that you're using for line processing with the very value you just read as text, but now with the `int` converted result. If that is your intention (and I can't see how it could be), so be it, but you do realize you're blowing away the very value you just saved with each new line read, right?
Comment: @MichaelBrooks I understand. What I'm saying is you don't have to populate *your line-read buffer* as the array you're accumulating your `int`-conversions with. As written you store the `int`-converted value from the prior read back into the line buffer, then immediately throw that buffer back into `fgets()` where the data you just stored will promptly be overwritten. Use a *different* array for storing your converted integers.
Comment: What actually happens when you run the program?
Comment: When I run the program the first time. It asks the user to input numbers. I do a scanf(%d) in a while loop until the user enters -999. While user is in loop, numbers are stored to userInputText[] and counter is incremented for subscript. I then use fprintf(textPointer, "%d\n", textTemp); to write file to text file. The array and text file are fine up until this point. Its the next program run where I have issues, because before allowing the user to input more numbers, I have to populate an array based on the text file.
Comment: Thanks Craig, along with creating another array it also had to be an int array. When I first started with the program, I was trying to use an int array but I kept trying to use it as an argument in fgets which only takes char arrays. I appreciate it
Answer: Try while( !feof(textPointer) )
     { fgets(userInputText,MAXSIZE,textPointer); }
sizeof(userInputText) will only return the size of a char pointer.
Comment: This is not true if `userInputText` is declared as a fixed array, which the OP specifically states it *is* in the question.
Comment: I hope I'm answering everyone sufficiently here. My instructions are, to create an integer array to store ip to 5,000 numbers and allow the user to input numbers into an array and save the numbers to a file. When the user runs the code for the second time, I need to populate an array based on the numbers saved to a file. Later in the program the user should be able to enter in more numbers that will take over where the populated array left off.
Comment: The code needs to write and read from both a text file and a binary file. I did the same code with some varriation (fread, fwrite etc..) for my binary file and it worked great. The code I originally posted should (after the first run) get the file and populate an array. If there are 20 numbers in the file. 10,20,30etc.. I should go to the input portion of the program with userInputText[20] = textCount and than start adding to the array. I would post the whole program but it kind of lengthy.


HTTP GET Reads wrong values

Question: I have this object data

I parse the object to use in a HTTP GET
baseURL+ "/previewDocument?" + $.param(data)

and it generated a url like this:
http://localhost:8081/api/previewDocument?parameters%5BMAIL_BODY%5D=&parameters%5BMAIL_FROM%5D=&parameters%5BMAIL_RECEIVED_DATE%5D=&parameters%5BMAIL_SUBJECT%5D=&parameters%5BMAIL_TO%5D=&deliveryChannel=NONE&templateID=9093

My API controller is as such:
 @RequestMapping(value = "/previewDocument", method = RequestMethod.GET, produces = "application/pdf")
public byte[] previewDocument(@RequestParam Map<String, String> parameters, @RequestParam String templateID, @RequestParam String deliveryChannel) {
    // stuff
}

The problem is that, instead  of parameters having only the Map related to my data.parameters
it has this content:
{parameters[MAIL_BODY]=, parameters[MAIL_FROM]=, parameters[MAIL_RECEIVED_DATE]=, parameters[MAIL_SUBJECT]=, parameters[MAIL_TO]=, deliveryChannel=NONE, templateID=9093}

Which is wrong. I Should have something like:
 {MAIL_BODY=, MAIL_FROM=, MAIL_RECEIVED_DATE=, MAIL_SUBJECT=, MAIL_TO=}

This values (@RequestParam String templateID, @RequestParam String deliveryChannel) are correctly filled. My only issue is with Map parameters.
Can someone explain me how to fix this?
Answer: As per documentation:

If the method parameter is Map<String, String> or
  MultiValueMap<String, String> and a parameter name is not specified,
  then the map parameter is populated with all request parameter names
  and values.

You can manually remove excess parameters from map or send them as path params.
Updated:
Looks like you need to flatten your data object first, so you have all the params at the same level. Something like this (I'm not very familiar with JS):
function flatten(obj) {
    var result = {};

    function process(element) {
        $.each(element, function(key, value) {
            if (typeof value === 'object') process(value);
            else result[key] = value;
        });
    }
    process(obj);
    return result;
}

And then use it to generate URL:
baseURL+ "/previewDocument?" + $.param(flatten(data))

Comment: but then how can I get the values?

HTTP GET Reads wrong values


Limit number of parallel running methods

Question: In my class I have a download function. Now in order to not allow a too high number of concurrent downloads, I would like to block this function until a "download-spot" is free ;)
void Download(Uri uri)
{
    currentDownloads++;

    if (currentDownloads > MAX_DOWNLOADS)
    {
        //wait here
    }

    DoActualDownload(uri); // blocks long time

    currentDownloads--;
}

Is there a ready made programming pattern for this in C# / .NET?
edit: unfortunatelyt i cant use features from .net4.5 but only .net4.0
Comment: static variable to keep track of number of current downloads and make it thread safe
Comment: I don't see any parallel code. It's impossible to help if you don't show the actual code. There *are* constructs that will help but they depend on the API you use to perform the asynchronous action. For example, PLINQ and Dataflow have MaxDegreeOfParallelism options. Task.Run can be combined with a TaskScheduler that allows only a certain number of concurrent tasks
Comment: I'm assuming `Download` is an IO bound operation?
Comment: @YuvalItzchakov it downloads a file via http and then writes it to disk. so yes.
Comment: @PanagiotisKanavos this method can be called from many instances of this class. so there would be many downloads in paralell.
Comment: Can you install `Microsoft.Bcl.Async`? Would help you here.
Comment: @YuvalItzchakov i guess i can.
Answer: May be this
var parallelOptions = new ParallelOptions
{
    MaxDegreeOfParallelism = 3
};

Parallel.ForEach(downloadUri, parallelOptions, (uri, state, index) =>
{
    YourDownLoad(uri);
});

Comment: thanks! this sounds interesting! but in my case i want to make sure that the caller of YourDownload only continues once the call is actually finished.
Answer: You should use Semaphore for this concurrency problem, see more in the documentation:
https://msdn.microsoft.com/en-us/library/system.threading.semaphore(v=vs.110).aspx
Answer: For such cases, I create an own Queue<MyQuery> in a custom class like QueryManager, with some methods :

Each new query is enqueued in Queue<MyQuery> queries
After each "enqueue" AND in each query answer, I call checkIfQueryCouldBeSent()
The checkIfQueryCouldBeSent() method checks your conditions : number of concomitant queries, and so on. In your case you accept to launch a new query if global counter is less than 5. And you increment the counter
Decrement the counter in query answer

It works only if all your queries are asynchronous.
You have to store Callback in MyQuery class, and call it when query is over.
Comment: The .NET Framework already contains such a class, [ActionBlock](https://msdn.microsoft.com/en-us/library/hh194684(v=vs.110).aspx) with a MaxDegreeOfParallelism option
Comment: @PanagiotisKanavos Thank you for information, I read ActionBlock specifications. Nevertheless I think it is not adapted to asynchronous Action, which must be able to notify the ActionBlock to continue with next action. In the case of asynchronous actions, it seems the whole ActionBlock would be finished immediatly.
Comment: ActionBlock is purpose-built for asynchronous processing - it's a descendant of the Robotics framework. If your method returns a Task, you can either `await` inside the block delegate or return the Task itself. There is an ActionBlock constructor that processes functions returning Task. You can also wait for the ActionBlock to finish processing by calling `block.Complete(); await block.Completion`;
Answer: You're doing async IO bound work, there's no need to be using multiple threads with a call such as Parallel.ForEach.
You can simply use naturally async API's exposed in the BCL, such ones that make HTTP calls using HttpClient. Then, you can throttle your connections using SemaphoreSlim and it's WaitAsync method which asynchronously waits:
private readonly SemaphoreSlim semaphoreSlim = new SemaphoreSlim(3);
public async Task DownloadAsync(Uri uri)
{
    await semaphoreSlim.WaitAsync();
    try
    {
        string result = await DoActualDownloadAsync(uri);
    }
    finally
    {
        semaphoreSlim.Release();
    }
}

And your DoActualyDownloadAsync will use HttpClient to do it's work. Something along the lines of:
public Task<string> DoActualDownloadAsync(Uri uri)
{
    var httpClient = new HttpClient();
    return httpClient.GetStringAsync(uri);
}


Decode byte array to bitmap that has been compressed in Java

Question: I am compressing a bitmap in the following way
Bitmap bmpSig = getMyBitMap();
int size = bmpSig.getWidth() * bmpSig.getHeight();
ByteArrayOutputStream out = new ByteArrayOutputStream(size);
bmpSig.compress(Bitmap.CompressFormat.JPEG, 100, out);
byte[] bytSig = out.toByteArray();

I am then trying to display the image in an Android ImageView from the byte array. When I do this I get an image that is completely black image.
ImageView myImg = (ImageView) findViewById(R.id.img_view);
myImg.setImageBitmap(BitmapFactory.decodeByteArray(bytSig, 0, bytSig.length));

I'm assuming it's because I am missing a step before BitmapFactory.decodeByteArray() to reverse the jpeg compression. Or have I misunderstood how the compression works?
Answer: I didn't realise that the background of my bitmap (from a Canvas object) was transparent. Since this bitmap is just black lines on a white background the black image is due to compressing to JPEG giving the image a black background.
I have changed
bmpSig.compress(Bitmap.CompressFormat.JPEG, 100, out);

to
bmpSig.compress(Bitmap.CompressFormat.PNG, 100, out);

And it is working as expected.
Comment: Nah ... you just needed to use a darker shade of black.


Partition of the identity in a simple $C^*$-Algebra

Question: The following question is from $C^*$-Algebras by Examples written by Kenneth R. Davidson, Problem V.9. Given a unital simple $C^*$-Algebra $\mathfrak{A}$ and a positive element $A$, how to show there exists finitely many $\{X_i\}_{i \leq n}$ such that $\sum_{i \leq n} X_i^* A X_i = I$? Beside the original question, is it possible that all these $X_i$ are unitary?
The given hint is $X A Y + Y^* A X^* \leq X A X^* + Y^* A Y$ iff $(X - Y^*) A (X^* - Y) \geq 0$ which is true in this case. I have no idea where to come up with a $\{X_i\}_{i \leq n}$ such that $\sum_{i \leq n} X_i^* A X_i$ and how to show it is the identity. One of my attempt is to define $\mathcal{A} = \{X Z Y\,\vert\,Z \in \overline{A \mathfrak{A} A}, X, Y \in \mathfrak{A}\}$. This is an non-zero ideal and hence equal to the whole $\mathfrak{A}$.
I wish I can define $\mathcal{A}_n = \{ \sum_{i \leq n}X A X^*\,\vert\,X \in \mathfrak{A}\}$ and show that for some $n$ it would be a hereditary subalgebra. Then $0 \leq Y \leq X A X^*, \exists B \in \mathfrak{A}\,\implies\,\sqrt{Y} = B (X A X^*)^{\frac{1}{4}}\,\implies\,Y = B (X A X^*)^{\frac{1}{4}} B^* \in \mathcal{A}_n$. If this can be done, then according to the provided inequality $2 \vert\,X A Y\,\vert \leq \sum_{i \leq n} X_i^* A X_i\,\implies X A Y \in \mathcal{A}_n$ and I can replace $A$ by other elements in $\overline{A \mathfrak{A} A}$. However, it seems to me that $\mathcal{A}_n$ can hardly be an algebra... If I am on the right track where can I find such an algebra that contains finite sums of $X A X^*$?
Answer: It looks like you have a lot of the right ingredients, but you're looking in the wrong direction.
Let $\mathfrak I=\operatorname{span}\{XAY:X,Y\in\mathfrak A\}$.  Then $\mathfrak I$ is a nonzero ideal in $\mathfrak A$, hence is dense in $\mathfrak A$ as $\mathfrak A$ is simple.  But then there is some $\tilde X_k,Y_k\in\mathfrak A$ such that $\|I-\sum_{k=1}^n\tilde X_kAY_k\|<1$, so $Z=\sum_{k=1}^n \tilde X_kAY_k$ is invertible in $A$, and thus $I=Z^{-1}Z=\sum_{k=1}^n(Z^{-1}\tilde X_k)AY_k$.  Now write $X_k=\frac12Z^{-1}\tilde X_k$, so that $I=2\sum_{k=1}^nX_kAY_k$.   Thus
$$I=\frac{I+I^*}{2}=\sum_{k=1}^nX_kAY_k+Y_k^*AX_k^*\leq\sum_{k=1}^nX_kAX_k^*+Y_k^*AY_k.$$
Now let $B=\sum_{k=1}^nX_kAX_k^*+Y_k^*AY_k$, and put $W_k=B^{-1/2}X_k$ for $k=1,\ldots,n$ and $W_k=B^{-1/2}Y_k^*$ for $k=n+1,\ldots,2n$.  Then we have
$$I=B^{-1/2}BB^{-1/2}=\sum_{k=1}^{2n}W_kAW_k^*.$$
As far as the second question, this fails to work for the case $\mathfrak A=\mathbb C$, as pointed out in the comments.
Comment: Nice answer! Regarding if its possible to do with unitaries: Look at $A=5\,\Bbb 1$ as an example where its not possible.
Comment: Of course.  Thanks for pointing that out!
Comment: @Aweygan Very brilliant answer! Could you explain how you conclude $B$ is invertible? I tried to show $\|\,I - B\,\| \leq 2 \|\,\frac{I}{2} - \sum_{k = 1}^n X_k A Y_k\,\| < 1$ which will be equivalent to $\|\,I - \sum_{k = 1}^n Z^{-1} \tilde X_k A Y_k\,\| < 1$. But I need to adjust the norm of $\sum_{k = 1}^n \tilde X_k A Y_k$ before calling it $Z$.
Comment: @s.harp Thank you for your answers. If $A$ is just a big enough constant then my guess will be obviously wrong. Could that be true when $A$ is not a constant? If the identity can be partitioned in such way, maybe among those many invertible elements above we can build an arc to connect some of them to some unitaries.
Comment: $B$ is invertible because $B\geq I$.  This means $B-I\geq 0$, so $\sigma (B-I)\subset[0,\infty)$, so $\sigma(B)\subset[1,\infty)$, so in particular $0$ is not in the spectrum of $B$
Comment: @Aweygan Got it. I figure it out.


Does it make sense to fit linear regression on single-case studies?

Question: Just on principle, do you think it would make sense to make linear regression models for each individual in a study and treat replicates as data points? I don't mean reporting p-values or anything like that, just for visualization purposes.
Specifically, it'd be a comparison of subjects exposed to treatments, say treatment1 and treatment2 vs. their basal states. Could I show plots for each subject's "trajectory" from basal state to treatment1 to treatment2?
The actual analysis I would do is a linear mixed-effects model, but I think it could be useful to be able to show how each subject individually responded to the treatments as well.
Edit:
I was thinking something like this:

Do you think this is useful or is it misleading? An important problem is that not all subjects were exposed to all the treatments, so some of them only have 2 observations with 2 replicates each. I could maybe add the replicates as poits in the regression as well? Just to see how they varied.
Edit 2:
This is just made up data, but I just realized that if two treatments have opposite effects, the regression would not show that. So maybe my best bet is to just plot each treatment vs the basal state and forget about the regression?
Comment: Plotting the data is always a good idea. Please show us the plots and describe your data and research questions.
Comment: Do you have an example you could post using some data or simulated data? I'm struggling to understand what you want to plot.
Comment: As several have noted, of course this is not only possible but it is an approach at the heart of many iterative, approximating routines. The question remains *is it useful?* Iow, depending on the sample size, at what point does it become an impractical, overwhelming burden to review results from dozens, hundreds, much less thousands of unaggregated mini-regressions?
Answer: To answer your question: yes this is certainly an acceptable thing to do and can do a great job of showing individual differences in random intercepts/slopes. From the original lme4 paper, they construct an XY plot from the lattice package to show how reaction time shifted over time in each subject. I simplify and annotate the code from their R script to make it easier to read:
#### Load Libraries ####
library(lmerTest)
library(lattice)

#### Plot ####
xyplot(Reaction ~ Days | Subject,
       sleepstudy,
       aspect = "xy",
       type = c("g", "p", "r"),
       xlab = "Days of sleep deprivation",
       ylab = "Average reaction time (ms)")

Shown below, where you can see generally reaction time increases by day, but some subjects actually have either little change or even the opposite effect where the RT diminishes (Subject 335):

Of course in your case there are only two/three time points, but that isn't really an issue. Regarding the point you made here:

Do you think this is useful or is it misleading? An important problem
is that not all subjects were exposed to all the treatments, so some
of them only have 2 observations with 2 replicates each.

No, this is not misleading. You are in fact showing that information in your plot, hence this provides context about the variation in random effects. Keep in mind that imbalance in the random effects with small sample sizes can affect your inference about them as shown in some simulation studies, so you may need to report that as a limitation in your report. With respect to your other question:

This is just made up data, but I just realized that if two treatments
have opposite effects, the regression would not show that. So maybe my
best bet is to just plot each treatment vs the basal state and forget
about the regression?

The XY plot above shows that this isn't an issue and is in fact common (some slopes may be positive and some may be negative).


Spring Data JPA + Hibernate on Websphere 8.5.5

Question: While moving from Tomcat to WebSphere, I've encountered issues with my Spring 4 / Hibernate projects not loading correctly. I've already set the app to a PARENT_LAST classloader and reverted my Hibernate version back to 4.2.16 (due to WAS 8.5.5 only supporting JPA 2.0). I did have to now enclose the WAR inside an EAR to comply with IDE integrations as well.
I am now getting the following error upon startup. Any assistance would be appreciated. If anyone has run Spring + Hibernate (and CXF) inside of a web app on WebSphere 8.5.5, I'd also like to hear if you've had other issues.
                             org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [datasource.xml]: Invocation of init method failed; nested exception is java.lang.IncompatibleClassChangeError: org.hibernate.engine.spi.CascadeStyle
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1553)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:973)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:750)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:403)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:106)
at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:1686)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.initialize(WebAppImpl.java:414)
at com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWebApplication(WebGroupImpl.java:88)
at com.ibm.ws.webcontainer.VirtualHostImpl.addWebApplication(VirtualHostImpl.java:169)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApp(WSWebContainer.java:749)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApplication(WSWebContainer.java:634)
at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:426)
at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:718)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1177)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1370)
at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:968)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:776)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplicationDynamically(ApplicationMgrImpl.java:1379)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2189)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:446)
at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:389)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:117)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$1.run(CompositionUnitMgrImpl.java:664)
at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5384)
at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5600)
at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.startCompositionUnit(CompositionUnitMgrImpl.java:678)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.startCompositionUnit(CompositionUnitMgrImpl.java:622)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:1269)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:618)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:87)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:618)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:291)
at javax.management.modelmbean.RequiredModelMBean$4.run(RequiredModelMBean.java:1260)
at java.security.AccessController.doPrivileged(AccessController.java:300)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1254)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:1092)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:831)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:813)
at com.ibm.ws.management.AdminServiceImpl$1.run(AdminServiceImpl.java:1335)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.AdminServiceImpl.invoke(AdminServiceImpl.java:1228)
at com.ibm.ws.management.connector.AdminServiceDelegator.invoke(AdminServiceDelegator.java:181)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:618)
at com.ibm.ws.management.connector.soap.SOAPConnector.invoke(SOAPConnector.java:488)
at com.ibm.ws.management.connector.soap.SOAPConnector.service(SOAPConnector.java:324)
at com.ibm.ws.management.connector.soap.SOAPConnection.handleRequest(SOAPConnection.java:65)
at com.ibm.ws.http.HttpConnection.readAndHandleRequest(HttpConnection.java:733)
at com.ibm.ws.http.HttpConnection.run(HttpConnection.java:532)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1864)

Caused by: java.lang.IncompatibleClassChangeError: org.hibernate.engine.spi.CascadeStyle
at java.lang.ClassLoader.defineClassImpl(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:306)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:154)
at com.ibm.ws.classloader.CompoundClassLoader._defineClass(CompoundClassLoader.java:853)
at com.ibm.ws.classloader.CompoundClassLoader.localFindClass(CompoundClassLoader.java:763)
at com.ibm.ws.classloader.CompoundClassLoader.loadClass(CompoundClassLoader.java:586)
at java.lang.ClassLoader.loadClass(ClassLoader.java:707)
at java.lang.J9VMInternals.verifyImpl(Native Method)
at java.lang.J9VMInternals.verify(J9VMInternals.java:94)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:171)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.buildBootstrapServiceRegistry(EntityManagerFactoryBuilderImpl.java:484)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:206)
at org.hibernate.jpa.boot.spi.Bootstrap.getEntityManagerFactoryBuilder(Bootstrap.java:51)
at org.hibernate.jpa.HibernatePersistenceProvider.getEntityManagerFactoryBuilder(HibernatePersistenceProvider.java:182)
at org.hibernate.jpa.HibernatePersistenceProvider.getEntityManagerFactoryBuilder(HibernatePersistenceProvider.java:177)
at org.hibernate.jpa.HibernatePersistenceProvider.createContainerEntityManagerFactory(HibernatePersistenceProvider.java:152)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:336)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:318)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1612)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1549)
... 68 more

Here is the Maven dependency tree
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ test.mytestapp ---
[INFO] com.mycompany.service:test.mytestapp:war:1.0.0.CI-SNAPSHOT
[INFO] +- com.mycompany.base:base-transaction:pom:1.0.4.CI-SNAPSHOT:compile
[INFO] |  +- com.mycompany.base:base-spring:pom:1.0.4.CI-SNAPSHOT:compile
[INFO] |  |  +- log4j:log4j:jar:1.2.16:compile
[INFO] |  |  +- org.springframework:spring-context-support:jar:4.0.5.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-aop:jar:4.0.5.RELEASE:compile
[INFO] |  |  |  \- aopalliance:aopalliance:jar:1.0:compile
[INFO] |  |  \- org.springframework:spring-orm:jar:4.0.5.RELEASE:compile
[INFO] |  |     +- org.springframework:spring-jdbc:jar:4.0.5.RELEASE:compile
[INFO] |  |     \- org.springframework:spring-tx:jar:4.0.5.RELEASE:compile
[INFO] |  +- com.mycompany.base:base-testing:pom:1.0.4.CI-SNAPSHOT:compile
[INFO] |  |  +- junit:junit:jar:4.9:compile
[INFO] |  |  |  \- org.hamcrest:hamcrest-core:jar:1.1:compile
[INFO] |  |  \- org.springframework:spring-test:jar:4.0.5.RELEASE:compile
[INFO] |  +- com.mycompany.base:base-data-access:pom:1.0.4.CI-SNAPSHOT:compile
[INFO] |  |  +- org.hibernate:hibernate-entitymanager:jar:4.2.16.Final:compile
[INFO] |  |  |  +- org.jboss.logging:jboss-logging:jar:3.1.0.GA:compile
[INFO] |  |  |  +- dom4j:dom4j:jar:1.6.1:compile
[INFO] |  |  |  +- org.javassist:javassist:jar:3.18.1-GA:compile
[INFO] |  |  |  +- org.jboss.spec.javax.transaction:jboss-transaction-api_1.1_spec:jar:1.0.1.Final:co
[INFO] |  |  |  +- org.hibernate.javax.persistence:hibernate-jpa-2.0-api:jar:1.0.1.Final:compile
[INFO] |  |  |  +- org.hibernate.common:hibernate-commons-annotations:jar:4.0.2.Final:compile
[INFO] |  |  |  \- org.hibernate:hibernate-core:jar:4.2.16.Final:compile
[INFO] |  |  |     \- antlr:antlr:jar:2.7.7:compile
[INFO] |  |  \- org.springframework.data:spring-data-jpa:jar:1.5.2.RELEASE:compile
[INFO] |  |     +- org.springframework.data:spring-data-commons:jar:1.7.2.RELEASE:compile
[INFO] |  |     +- org.aspectj:aspectjrt:jar:1.7.4:compile
[INFO] |  |     \- org.slf4j:jcl-over-slf4j:jar:1.7.6:runtime
[INFO] |  +- org.springframework:spring-aspects:jar:4.0.2.RELEASE:compile
[INFO] |  |  \- org.aspectj:aspectjweaver:jar:1.7.4:compile
[INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.7.6:compile
[INFO] |  |  \- org.slf4j:slf4j-api:jar:1.7.6:compile
[INFO] |  \- com.mycompany.global:fed-spring-aop:jar:1.0.0.CI-SNAPSHOT:compile
[INFO] |     \- com.thoughtworks.xstream:xstream:jar:1.2.2:compile
[INFO] |        \- xpp3:xpp3_min:jar:220.127.116.11.O:compile
[INFO] +- com.mycompany.base:base-web:pom:1.0.4.CI-SNAPSHOT:compile
[INFO] |  +- org.springframework:spring-webmvc:jar:4.0.2.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-beans:jar:4.0.2.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-context:jar:4.0.2.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-core:jar:4.0.2.RELEASE:compile
[INFO] |  |  |  \- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] |  |  +- org.springframework:spring-expression:jar:4.0.2.RELEASE:compile
[INFO] |  |  \- org.springframework:spring-web:jar:4.0.2.RELEASE:compile
[INFO] |  +- org.apache.cxf:cxf-rt-frontend-jaxws:jar:2.7.11:compile
[INFO] |  |  +- xml-resolver:xml-resolver:jar:1.2:compile
[INFO] |  |  +- asm:asm:jar:3.3.1:compile
[INFO] |  |  +- org.apache.cxf:cxf-api:jar:2.7.11:compile
[INFO] |  |  |  +- org.codehaus.woodstox:woodstox-core-asl:jar:4.2.1:compile
[INFO] |  |  |  |  \- org.codehaus.woodstox:stax2-api:jar:3.1.4:compile
[INFO] |  |  |  +- org.apache.ws.xmlschema:xmlschema-core:jar:2.1.0:compile
[INFO] |  |  |  +- org.apache.geronimo.specs:geronimo-javamail_1.4_spec:jar:1.7.1:compile
[INFO] |  |  |  \- wsdl4j:wsdl4j:jar:1.6.3:compile
[INFO] |  |  +- org.apache.cxf:cxf-rt-core:jar:2.7.11:compile
[INFO] |  |  |  \- com.sun.xml.bind:jaxb-impl:jar:2.2.6:compile
[INFO] |  |  +- org.apache.cxf:cxf-rt-bindings-soap:jar:2.7.11:compile
[INFO] |  |  |  \- org.apache.cxf:cxf-rt-databinding-jaxb:jar:2.7.11:compile
[INFO] |  |  +- org.apache.cxf:cxf-rt-bindings-xml:jar:2.7.11:compile
[INFO] |  |  +- org.apache.cxf:cxf-rt-frontend-simple:jar:2.7.11:compile
[INFO] |  |  \- org.apache.cxf:cxf-rt-ws-addr:jar:2.7.11:compile
[INFO] |  |     \- org.apache.cxf:cxf-rt-ws-policy:jar:2.7.11:compile
[INFO] |  |        \- org.apache.neethi:neethi:jar:3.0.3:compile
[INFO] |  \- org.apache.cxf:cxf-rt-transports-http:jar:2.7.11:compile
[INFO] +- xerces:xercesImpl:jar:2.11.0:compile
[INFO] |  \- xml-apis:xml-apis:jar:1.4.01:compile
[INFO] \- com.mycompany.transaction:toolslang.test:jar:1.0.0.CI-SNAPSHOT:compile
[INFO]    \- com.mycompany.base:base-db2:pom:1.0.4.CI-SNAPSHOT:compile
[INFO]       +- jdbc:db2:jar:driver:1.6.2:compile
[INFO]       \- jdbc:db2:jar:license:1.6.2:compile

Answer: For using Hibernate in WAS 8.5.5 you need to modify persistence.xml and add the provider
<provider>org.hibernate.ejb.HibernatePersistence</provider>

For using CXF as Web service engine you need to set class loader to parent last and disable annotation scanning:

Set the DisableIBMJAXWSEngine property in the META-INF/MANIFEST.MF of
  a WAR file or EJB module to true

See documentation for more details:

Using third-party persistence providers
Using a third-party JAX-WS web services engine
Comment: I'd give you two accepts if I could. Thank you! For anyone else fighting this, in addition to the above on WAS 18.104.22.168, I had to exclude the neethi dependency of CXF as well (the only exclusion I had). I still have a bit of classloader errors on startup, but the application starts and functions properly w/ Spring Data + Hibernate + CXF after these changes.
Comment: I am getting CWWJP9991W: openjpa.Runtime: Warn: An error occurred while registering a ClassTransformer with JPAPUnitInfo
Comment: @svlada it looks like you didn't correctly override persistent provider. I'd suggest you to create new question and describe fully your issue there.
Comment: note that in newer versions you need org.hibernate.jpa.HibernatePersistenceProvider instead


Uso de Break no Python

Question: Eu fiz esse código que vai passando de fase, porém, quando se erra eu usei o Break para finalizar o jogo... mas ele apresenta a mensagem de Game Over e mesmo assim continua para a próxima fase... alguém pode me ajudar?
Segue o código:
<code>#menu
hotel = [['*', '*', '_', 'G'], ['R','_', '*', '*']]

print('Seja bem vindo ao hotel dos animais!')
nome = input('Qual o seu nome, para que possamos iniciar? ')

print('{}, vamos iniciar nossa jogatina!...:'.format(nome))
sleep(5)
regras= [['Regras: O rato não pode ficar ao lado do gato.'],['O cão não pode ficar ao lado do osso.'], ['O gato não pode ficar ao lado do cão.'], ['O queijo não pode ficar ao lado do gato.']]
for linha in regras:
  print(linha)
sleep(5)

#fase1
print('Na fase 1 você deve alocar o Rato e o Gato na seguinte matriz que representa os quartos')

sleep(5)

for linha in hotel:
  print(linha)

sleep(5)

posRato = int(input("Qual a posição que o Rato deve ficar? "))
posGato = int(input('Qual a posição que o gato deve ficar? '))
while True:
  if posRato == 5 and posGato == 4:
    print('Parabéns {}, você segue para a fase 2!'.format(nome))
  else:
    print('Ops {}, infelizmente você perdeu!'.format(nome))
    break
</code>
Answer: Parece que o trecho que cita na dúvida não está descrito no código, mas o break irá parar o loop do while, se o restante do código estiver com a mesma indentação não será executado, mas caso esteja fora do while irá ser executado mesmo com o break, pois ele está parando somente o loop.
Answer: Você não colocou o break na condição correta que você quer que ele pare o programa, por exemplo se você colocar o break depois do print do if, ele vai fazer com que o programa pare sem executar o else.

Uso de Break no Python


Spring-Data-SOLR: Stats result on SimpleQuery won't return me distinct calcs

Question: I'm trying to use a SimpleQuery to get distinct result from my solr collection, but even after setting my StatsOption with calcDistinct true, I can't get the result I want.
BTW I'm using spring-data-solr-2.1.4.RELEASE.
SampleCode:
Field field = new SimpleField("fieldName");
StatsOptions statsOptions = new StatsOptions().addField(field).setCalcDistinct(true);
SimpleQuery query = new SimpleQuery("*:*").setStatsOptions(statsOptions);
StatsPage<MyClass> statsPage = solrTemplate.queryForStatsPage(query, MyClass.class);
FieldStatsResult statsResult = statsPage.getFieldStatsResult(field);
Collection<Object> distinctValues = statsResult.getDistinctValues();
Set<String> result = distinctValues.stream().map((i) -> i.toString()).collect(Collectors.toSet());
return result;

After trying the above code, all I get is the max, min, count, but no results for distinct totals or distinct values.

What am I doing wrong in this sample?
Answer: Looks like your disctintValues collection is also an implementation of EmptyList which means that there are no values in the response.
Check if your query returns any result, first.
Comment: I've already tried the same query with stats=on in my solr API and it returns me results for distinct values.
Answer: StatsOptions statsOptions = new StatsOptions().addField(field).setSelectiveCalcDistinct(true).setCalcDistinct(true);

The processStatsOptions method in org.springframework.data.solr.core.DefaultQueryParser is described as follows :
Boolean selectiveCountDistincts = statsOptions.isSelectiveCalcDistincts(field);

if (selectiveCountDistincts != null) {

  solrQuery.add(selectiveCalcDistinctParam, String.valueOf(selectiveCountDistincts.booleanValue()));

}


One to many relationship in android sqlite

Question: I have a database with tables Teachers,Subjects and Teachers_Subjects in my android sqlite database.My tables have structure as shown in the image.
I need to query tables to get all subjects rows that are related to a single teacher. Initially I have the _id of the teacher.Using teachers _id I need to find the subjects.According to me first I need to find all the rows in Teachers_Subjects Table related to a Teacher and then make other query using the resulted rows and Subjects Table with JOIN statement to get all rows related to that teacher.I wanted to know is there any better way/query to accomplish this?If not then what should be the query for the solution mentioned above?
Comment: You can do it all in one step. Use a raw query with two inner joins.
Comment: @Leco Can you plz show me example query ...i would be very thankful to you
Answer: SELECT Subjects.*
FROM Teachers_Subjects JOIN Subjects
                       ON Teachers_Subjects.subject_id = Subjects._id
WHERE Teachers_Subjects.teacher_id = ?


How to upload multiple files and predefine filetypes?

Question: Does someone know how it is possible to upload multiple files with the dropbox client module for drupal 7?
It would be nice if it could be integrated with plupload.
Also i would like to know how i can predefine the filetypes users can upload?
I'm a n00b in php, can someone give me some tips or solutions?
Here is the part of the upload funktion in the dropbox client module.
<code>function dropbox_client_upload_form( $form, &$form_state ) {

  $form[ 'upload' ] = array(
    '#title' => t('Upload file'),
    '#type' => 'file',
    '#size' => 20,
    );
  $form[ 'submit' ] = array(
    '#type' => 'submit',
    '#value' => t( 'Upload' ),
    );
  return $form;
}

function dropbox_client_upload_form_submit( $form, &$form_state ) {

  global $user;

  $dropbox = dropbox_client_get_client();

  //Process files
  $uploads_dir = dirname( $_FILES[ 'files' ][ "tmp_name" ][ 'upload' ] );

  $tmp_name = $_FILES[ 'files' ][ "tmp_name" ][ 'upload' ];
  $name = $_FILES[ 'files' ][ "name" ][ 'upload' ];
  $filename = "$uploads_dir/$name";
  move_uploaded_file( $tmp_name, $filename );

  $dpath = (isset( $_GET[ 'dpath' ] )) ? $_GET[ 'dpath' ] : '/';
  $result = $dropbox->add( $dpath, $filename );

  if ( property_exists( $result, 'error' ) ) {
    drupal_set_message( $result->error );
  }
  else {
    drupal_goto( $_GET['q'], array( 'query' => array( 'dpath' => $dpath ) ) );
  }
}
</code>
Thank you!
Best regards,
meckl
Answer:
Also i would like to know how i can predefine the filetypes users can
  upload?

Specify the file types restriction on the file field with 'file_validate_extensions' in the #upload_validators array in the file field.
See the form api for reference. See the file_managed field.


Are these two definitions of differentiability equal?

Question:
A function $f$ is differentiable in $x$ iff

the limit $~\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h}~~\text{exists}$ ("normal" definition)
$|f(x+h)-f(x)|<C|h|~$ holds for small $h$ with $C>0$

Now for the sake of comparison, I called the first limit $C$:
$$\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h}=C \iff \frac{|f(x+h)-f(x)|}{|h|}<C$$
but maybe that's not the right approach to understanding it intuitively. Could someone explain the connection/equivalence? I'm not seeing through.
Comment: The second is usually called pointwise Lipschitz continuity, and is much weaker a condition than differentiability as is defined in the first.
Comment: The equivalence $\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h}=C \iff \frac{|f(x+h)-f(x)|}{|h|}<C$ is false.
Comment: @Vim Key infromation, thank you! Seems to be a mistake in the script.
Answer: Those conditions are not the same. To see this, try $f(x) = |x|$ at $0.$
Answer: Those two definitions of differentiability are not equivalent. Consider the famous function
$$f(x)=\begin{cases}
 x\sin\left(\frac 1x\right) & x\ne 0 \\
 0 & x=0
\end{cases}$$
It is easy to show that $|f(0+h)-f(0)|\le |h|$, so you can choose any value of $C$ greater than $1$ and the second definition holds. However, the first limit does not hold, since the value of the difference quotient oscillates between $-1$ and $1$, so there is no limit. There is not even a one-sided limit.
Comment: I think you want $|f(0+h) -f(0)|$ there.
Comment: @zhw.: Yes, you are right. Fixed now. Thanks for the correction!


Matching two towers of blocks in height.

Question: This is my first post in this stack, but I have posted on StackOverflow before.  Let me declare that I am a coder in search of an algorithm.  This is not homework.
Bart has two bags of kiddies wooden blocks, bag A and bag B.
All the blocks in both bags are random (and potentially different) heights.
Miss Hoover asks Bart to build a tower from bag A's blocks, and a tower from bag B's blocks, such that both towers are exactly the same height.
Bart is allowed to measure the height of each block before building.
All heights are integer values (ref: User24142).
Let's assume I cannot rotate the blocks to use their width, "height" is just used to illustrate a single dimension, an analogous property of the actual object in question.
To solve this for Bart, my first guess would be to combine all blocks in bag A and work out all possible total tower heights, repeat the process for bag B, and then try to find two values in both result sets that match.  This is computationally expensive and becomes untenable with large numbers of blocks.
Is this a known problem with a known solution, or do I have no alternative but to grind out the solution from all possible data?
Comment: this begs for dynamic programming (I believe). How many blocks in each bag? are you sure that a combination of blocks in $A$ could potentially match a height in $B$?
Comment: It seems to relate to the knapsack problem, which is known to be NP complete. https://en.wikipedia.org/wiki/Knapsack_problem I wouldn't calculate all possible heights for both sets though. I would iterate through all possible heights for bag A, and for each height, I would try to match it with a tower from B, remembering all the B-height values I get, including overflows, and then using this information in later iterations. If I knew all the heights were integers or something, I might use a different approach, but this should work.
Comment: @Chinny84 Blocks in each bag is unknown, 1 to any value, in either bag.  There does not have to be a solution, but you have to try to find a solution to know that there is no case where two towers can match in height.  On no solution, more blocks might be added or removed, and when they are, we iterate and try again.
Comment: @user24142 It does look like the knapsack problem, but doubled up.  All heights are integers, if that helps.  At least with the knapsack problem to go I have a way to look into the problem, so thanks very much for that pointer!
Comment: @Chinny84 For the dynamic programming aspect, I was thinking that factorising the integer block size would allow me to categorise block heights into 'multiples of 2', 'multiples of 3' etc, so that I could work from the large scale blocks first, and then look for suitable candidates to plug the small gaps.  Incidentally, if two possible solutions exist, the tallest solution is preferred.
Answer: This is exactly the subset sum problem, which is NP-complete. Bag A corresponds to the positive numbers, bag B corresponds to the negative numbers.
Comment: Yes, that looks like it exactly, spot on.  Very much obliged for your extremely valuable assistance.


What is the time signature of this piano rhythm?

Question:
I've attached a photo. I know the notation isn't correct on the treble clef and I'm really sorry for that. I just need to know the characteristics of this rhythm and whether 4/4 is a suitable time signature for it? What type of a rhythm is it?
Comment: what piece is this from?
Comment: Hi, it's a harmonic accompaniment for my own composition :)
Answer: It does work in 4/4.
If you look at the green voice, the first beat is marked as a triplet and contains a quarter rest and an eighth note so making three eighth notes altogether. The second beat isn't marked as a triplet and the triplet eighth-note from the first beat is tied over onto an ordinary eighth-note at the beginning of this beat. If you count up all the thirty-second notes in the thirty-second note rest plus the dotted sixteenth note, you get another ordinary eighth-note so you have a full quarter note for the second beat.
The red and blue voices aren't marked in triplets and they just seem to be doing half notes and quarter notes on the beat so I think they are OK.
The thirty-second note rest is so short at the marked tempo that you would get the effect of a grace note with the blue voice on the beat sounding just before the green voice dotted note after the beat.
As for what you call this rhythm, I don't know the correct name for it. Apart from the thirty-second note rest it's the sort of 'lazy dotted rhythm' found in jazz and I would identify it with the clip-clop of a horse crossing the prairie - this sort of thing The Dillards
Comment: I think I know that horse. Although the term 'shuffle' has come to mean all sorts of things, because we all march (or mosey) to the beat of a different drummer (or horse hoof), you could probably do worse than 'cowboy shuffle'.


Add product metadata using `add_product()` when cloning an order on WooCommerce

Question: I am creating a cloning functionality for WooCommerce Order.
What I want is to make sure that the product item meta data is cloned properly.
 /**
 * Get the original order items to be clone
 * Each product inside the new order must also contain the meta_data
 */
$order     = wc_get_order($post_id);
$new_order = wc_get_order($new_post_id);
foreach ($order->get_items() as $item_id => $item) {
    $product_id = $item->get_product_id();
    $quantity   = $item->get_quantity();

    // Get product's meta data values
    $_wwp_wholesale_priced = get_post_meta( $product_id, '_wwp_wholesale_priced', true);
    $_wwp_wholesale_role = get_post_meta( $product_id, '_wwp_wholesale_role', true);
    $_amount_requested = get_post_meta( $product_id, '_amount_requested', true);
    $_amount_colli = get_post_meta( $product_id, '_amount_colli', true);

    $args       = array(
      '_wwp_wholesale_priced' => $_wwp_wholesale_priced,
      '_wwp_wholesale_role'   => $_wwp_wholesale_role,
      '_amount_requested'     => $_amount_requested,
      '_amount_colli'         => $_amount_colli,
    );

    $new_order->add_product(wc_get_product($product_id), $quantity, $args);
}

On the documentation, it seems like passing the $args should do the trick but I don't see any meta being passed. I have also tried using $item->add_meta_data and $item->update_meta_data but they don't seem to work. What am I missing?
Goal:

Current:
Answer: There are some mistakes and missing things in your code, try the following instead:
 /**
 * Get the original order items to be clone
 * Each product inside the new order must also contain the meta_data
 */

$order     = wc_get_order($post_id);
$new_order = wc_get_order($new_post_id);

foreach ($order->get_items() as $item) {
    $product  = $item->get_product(); // get an instance of the WC_Product Object

    $new_item_id = $new_order->add_product($product, $item->get_quantity()); // Add the product
    $new_item    = $new_order->get_item( $new_item_id, false ); // Get the new item

    // Add custom meta data (afterwards)
    $new_item->add_meta_data( '_wwp_wholesale_priced', $item->get_meta('_wwp_wholesale_priced'), true );
    $new_item->add_meta_data( '_wwp_wholesale_role', $item->get_meta('_wwp_wholesale_role'), true );
    $new_item->add_meta_data( '_amount_requested', $item->get_meta('_amount_requested'), true );
    $new_item->add_meta_data( '_amount_colli', $item->get_meta('_amount_colli'), true );
    $new_item->save(); // Save item data
}
// Calculate totals and save order data
$new_order->calculate_totals();

I should work.


ESLint semi rule doesn't enforce on import statements before class declaration in react-native

Question: I am trying to integrate ESLint on our react-native project by defining the very basic rules. I started with defining semi rule. Everything seems ok so far except for the .js files which has a class declaration. semi rule doesn't throw warning or error if I don't use semi-colon for import statements.
.eslintrc.json
{
    "plugins":["react", "react-native"],
    "parserOptions": {
        "sourceType": "module",
        "ecmaFeatures": {
            "jsx": true
        }
    },
    "rules":{
        "jsx-quotes": [
            "error",
            "prefer-double"
        ],
        "semi": [
            "error",
            "always"]
    }
}

Example .js file.
import React, { Component } from 'react';
import { Easing, View, StyleSheet, Animated } from 'react-native'
import { connect } from 'react-redux'
import { NavigationActions } from 'react-navigation'
import { updateLicense } from '../actions'

class SomeContainer extends Component {
  somefunction() {}
}

On the other hand, if there is no class declaration (just import and export statements) it complains about lacking semi-colons.
Any ideas? Thanks a lot for your help!
Comment: try with this rule "semi": [2, "always"]
Comment: @ZeeshanAnsari I tried but it didn't work. I also figured out that none of the rules work if there is a class definition in any `.js` file
Comment: I think you should define render method to display something in Text then check because linter is not working on that file cause of syntax error
Comment: and if you are using the vscode so you can install eslint plugin for best practice.
Comment: This config+example works as expected for me in Sublime. You might want to make sure you have the latest versions installed.
Comment: @ZeeshanAnsari yeap I am using vscode and already installed it.
Comment: @DavidHellsing I checked the eslint version which is 5. Also, I have the latest version of vscode as well. It sounds weird that it works on Sublime. I might open an issue on vscode.
Comment: did you add "eslint.alwaysShowStatus": true, in workspace setting
Comment: actually vscode displays errors. but the problem is it doesn't work for .sj files which includes `Components`.
Comment: because there could be syntax error
Comment: solve the syntax error first
Answer: The problem gone after I started to extend configuration from airbnb style with babel-eslint option. So, I added;
"extends": "airbnb",
"parser": "babel-eslint",

lines into .eslintrc file.


Deploying Win10 with Windows Deployment Services

Question: I'm trying to deploy Windows 10 Pro with WDS on Server 2008 R2. I successfully uploaded a boot image (boot.wim) to WDS, but when I try to add the install image (install.wim) I get the following error: "The file does not contain a valid install image." Both .wim files have been extracted from an ISO file. Has the latter file corrupted, or is it possible that the 2008 R2 version of WDS doesn't support deploying Windows 10? Is there any workaround available? Thanks!
Answer: Solved it! Apparently install.wim used an incompatible compression, so what I needed to do was:
dism.exe /Export-Image /SourceImageFile:C:\install.wim /SourceIndex:1 /DestinationImageFile:C:\install2.wim /Compress:max

For this you need Deployment Tools ja Windows Preinstallation Environment which can be downloaded here. Then dism.exe can be found in C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools.


How do I programmatically "place" an order?

Question: When making a payment through the Commerce admin interface; the Orders do not get completed. I can see a Place Order button when viewing a cart Order; which seems to do the trick. I need to do this in code.
I know that i need to react to the the commerce_order.order.paid event but I am not sure what to do there.
I found this post: How do I programmatically update the order status? which suggests i need to "apply a transition". But the suggestion there doesn't work.
I opened a new question for this as I am specifically asking about "placing an order"; whereas that question was asking about simply changing Order status.
Answer: A little digging and I figured it out on my own:
Event handler for for this event: commerce_order.order.paid
and with this handler code:
<code>  public function paymentCompletedHandler($event) {
    $order = $event->getOrder();
    $order->getState()->applyTransitionById('place');
    return;
  }
</code>


Service discovery on docker-compose Service Fabric cluster running on Linux

Question: I have created a docker-compose cluster that I want to deploy to a Linux based SF cluster. All the apps are obviously created on docker and they talk to each other using DNS. Upon cloud development on the aforementioned Linux SF cluster, services cannot talk to each other via DNS although I have configured hostname and network aliases as shown below.
On my local machine, everything runs smooth and services can talk to each other. Does anybody know what I am doing wrong here?
identity:
    image: .....
    environment:
    ...
    ports:
      - "5020:5020"
    hostname: identity.dhcloud
    networks:
       default:
          aliases:
            - identity.dhcloud

Thanks,
Kind Regards,
José.
Answer: SF DNS doesn't work (yet) in Linux Clusters.

DNS service for Service Fabric services is not yet supported on Linux.

Only a small subset of docker compose operations is supported. Hostnames and aliases aren't in the list.

Services > Deploy > Replicas
Services > Deploy > Placement > Constraints
Services > Deploy > Resources > Limits

cpu-shares
memory
memory-swap

Services > Commands
Services > Environment
Services > Ports
Services > Image
Services > Isolation (only for Windows)
Services > Logging > Driver
Services > Logging > Driver > Options
Volume & Deploy > Volume

You can use manual resolution using Fabric Client/ServicePartitionResolver is the available alternative. See this answer for details
var resolver = ServicePartitionResolver.GetDefault();
var fabricClient = new FabricClient();
var services = await fabricClient.QueryManager.GetServiceListAsync(applicationName);
var resolved = await resolver.ResolveAsync(services[0].ServiceName, someKey, CancellationToken.None);

Instead of looping over all apps and services you can provide name filters directly.


Java SSL connection returns previous reply

Question: I'm using an SSL connection to connect with a server (which i have no control over and no access to it's code, could be it's fault, but i wanna be sure), when i send the data (a byte array) for the first time i get the correct response, but in subsequent sends i get the response expected by the previous send. for example, let's say if i send x, i expect the server to reply x, y to y, z to z, etc...
when the app starts, i call x, and get x. but then i call y and get x, call z and get y, call x and get z, etc...
here's the generic code implemented for each command to send and receive (bytes is initiated with a predetermined set of bytes to simulate, say, command x)
byte[] bytes =  new byte[6];
if(socket == null || !socket.isConnected() || socket.isClosed())
try {
    getSocket(localIp);
} catch (IOException e1) {
    e1.printStackTrace();
}

if (socket == null || !socket.isConnected()) {
    try {
        getSocket(globalIp);
    } catch (IOException e1) {
        e1.printStackTrace();
        return null;
    }
}

byte[] recievedBytes = null;

String sentBString = "sendGetConfig: ";
for (int i = 0; i < bytes.length; i++) {
    sentBString += String.valueOf(bytes[i]) + ", ";
}
System.out.println(sentBString);

if (socket != null){
    try {
        DataOutputStream os = new DataOutputStream(socket.getOutputStream());
        os.write(bytes);

        DataInputStream is = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
        int tries = 0;
        while (tries < 20 && (recievedBytes == null || recievedBytes.length == 0)) {
            if (is.markSupported()) {
                is.mark(2048);
            }

            ByteArrayOutputStream buffer = new ByteArrayOutputStream();
            int nRead;
            byte[] data = new byte[1024];

            try {
                nRead = is.read(data, 0, data.length);
                buffer.write(data, 0, nRead);
            } catch (Exception e) {
            }

            buffer.flush();

            recievedBytes = buffer.toByteArray();
            if (recievedBytes.length == 0)
                is.reset();
        }

        is.close();
        os.close();
        socket.close();
    }
}

i know the implementation of the read is not perfect, its the result of a workaround i did because the server does not send any end of stream indication and so any read command implemented in a loop results in a timeout exception
any help will be greatly appreciated
Comment: Test the server with `$ echo -e "GET / HTTP/1.0\r\n" | openssl s_client -connect www.google.com:443 -ign_eof`.
Answer:
the server does not send any end of stream indication

Of course the server sends an EOS indication. The problem is that you're completely ignoring it. When the peer has closed the connection and there is no more pending data to be read, read() returns -1.

and so any read command implemented in a loop results in a timeout exception

Nonsense.
The correct form of your loop is as follows:
while ((count = in.read(buffer)) > 0)
{
    out.write(buffer, 0, count);
}

substituting your own variable names as required.
The reason you keep reading the same data is because you keep resetting the stream to the same point. Just throw your mark() and reset() calls away.
Comment: no, still doesn't work... any bytearray (buffer) initiated to more than 10 length will resolve in a timeout exception. which coincidentally is the number of bytes supposed to be sent...


Non-RSA TLS1.2 Packet decryption

Question: I am trying to decrypt a pcap file. This pcap file contains a capture of an HLS encrypted video stream. The pcap contains TLSv1.2 packets.
Below are some information from the pcap file

Server Hello message Cipher Suite:
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384.

EC Diffie-Hellman server Params: pubkey (1)

The Certificate Status message:

Signature Hash Algorithm Hash: SHA256
Signature Hash Algorithm Signature: ECDSA
Client Key Exchange Message
EC Diffie-Hellman server Params: pubkey (2)

I tried to follow [this Wireshark SSL decryption tutorial][1]. But it seems that it works only for RSA encryptions.
I have been researching for a while and found [this discussion][2]. I am quoting an extract from this discussion:

There is an important parameter to mind: decryption of a passively
recorded session (with a copy of the server private key) works only if
the key exchange was of type RSA or static DH; with "DHE" and "ECDHE"
cipher suites, you won't be able to decrypt such a session, even with
knowledge of the server private key. In that case, you will need
either the negotiated "master secret", or to use the server private
key to actively intercept the connection

It's note worthy that I have the client private key. In my case, the client is FFmpeg video streamer (FFplay). I had a look also on the [TLS v1.2 RFC][3].
My question:
Is it possible to do a decryption in this scenario ? If yes, what do I need to have to do so?
Is the decryption done using the client's private key or using the pre_shared_master (i.e. Diffie-Hellman) ?
[1]: https://wiki.wireshark.org/SSL
[2]: https://security.stackexchange.com/questions/117778/how-to-decrypt-ssl-traffic-with-a-packet-sniffer-when-i-have-the-private-key
[3]: https://www.rfc-editor.org/rfc/rfc5246
Comment: Pre-shared secret would be needed. What is client key? Is it a certificate's private key? It's used only for authentication.
Comment: Thanks for your reply really appreciate it! If the pre-shared secret is needed then it's the Diffie-Hellman public keys ? It's not really clear in the RFC (https://tools.ietf.org/html/rfc5246#section-8.1.2). The Client private key (FFmpeg key) is found in (https://github.com/FFmpeg/FFmpeg/blob/415f907ce8dcca87c9e7cfdc954b92df399d3d80/libavformat/tests/rtmpdh.c)..It's a 256 long string static const char *private_key
Answer: First, the clients private or public key are not involved in the key exchange in any way but only used to authenticate the client (if requested by the server). What is used in the key exchange are the servers private and public key, but only if RSA key exchange is used. For DHE/ECDHE they are not used so private/public key are not sufficient. See it is possible to decrypt HTTPS with the (private, public) pair if it uses DHE? for the details why this is the case.
What you would need instead of the private key is actually the exchanged key which is unique for each TLS session even if the private key is the same. Some clients can store this key for later use and if your client can do it see Decrypting TLS Browser Traffic With Wireshark – The Easy Way! how to proceed then to decrypt the traffic.
Comment: @JosephWahba: ffplay is a TLS client, i.e. it establishes a TLS connection with the server which includes the key exchange. After this key exchange both server and client know the encryption key. But somebody just looking at the traffic does not know the key. I recommend you have a look at one of the various videos on youtube which playful explain diffie hellman key exchange to understand how this works.
Comment: @JosephWahba Why not just modify the client, compile it and have it dump all decrypted data to file?
Comment: Thanks for your reply really appreciate it! I had a look on (it is possible to decrypt HTTPS with the (private, public) pair if it uses DHE? ) before asking my question. I'm curious though...How come then the FFplay client is able to decrypt the encrypted video packets? How do you think I would be able to manually compute these pre-shared exchanged keys? The client is opensource code and I'm capturing all the packets so ideally I should be able to decrypt the video traffic manually
Comment: @Eugene Mayevski 'EldoS Thanks for your suggestion. My concern is how the encryption/decryption happens. Not concerned about the data itself.
Answer: No, it is not possible to decrypt in this scenario. That would involve breaking EC Diffie-Hellman.
Decryption is not directly performed using the pre_master_secret but it is performed by keys directly derived from the pre-master secret. That is: the client and server decryption keys that are derived from it by first deriving the master_secret and then performing the PRF and dividing the output to the session keys and IV's.
Comment: Thanks for your reply. This is what I thought from the beginning but wanted to double check with experts. Appreciate your help
Comment: Changed the wording somewhat, there is no such thing as a `pre_shared_master` (in the TLS RFC) and `pre_shared_key` is an entirely different way of establishing session keys.
Comment: No, decryption is not performed using the pre_master_secret. It is performed using the session key.
Comment: @EJP It was explained below that what I meant with it but you're right, changed wording. Other remarks welcome of course.

Non-RSA TLS1.2 Packet decryption


Combinatorial definition of Hall–Littlewood polynomials (sum over SSYT?)

Question: Hall–Littlewood polynomials $P_\lambda(x;t)$ is an important deformation of Schur polynomials
forming a basis in the ring of symmetric polynomials over $\mathbb Z[t]$.
There are various definitions, including quite explicit
$$
P_\lambda(x;t)=
\sum_{w\in S_n/\lambda} w\left(x^\lambda\prod_{\lambda_i>\lambda_j}\frac{x_i-tx_j}{x_i-x_j}\right)
$$
but I'm interested in a combinatorial description.
Schur polynomial $s_\lambda$ is a sum of monomials corresponding to semi-standard Young tableux of shape $\lambda$ —
so I'm expecting an answer in the form of a weight on SSYT.
P.S. I'm mostly interested in the principal specialization of H-L polynomials — maybe this $(q,t)$-weight on SSYT is easier to describe than the full answer.
Comment: Наткнулся на твой вопрос и решил уж было, что настал мой звездный час, но потом понял, что за Grigory M. Правильный ответ тебе Миша уже рассказал наверняка, да?
Comment: Pardon my French.
Comment: @IgorMakhlin О, привет. Well, yes and no: there is an explicit description of the $t$-weight $\psi$ in Macdonald's book — but it's complicated and not terribly satisfying. So if you have a better answer, please explain it (here or iRL).
Comment: @IgorMakhlin (И про t-версию Бриона и т.п. мы бы с М.Б. с интересом послушали в какой-то момент.)
Comment: Apparently, I do not have a better answer, although I've spent quite some time trying to come up with some more insightful perspective: http://mathoverflow.net/questions/199629
Answer: I'm not sure if this is exactly what you're looking for, but I recently worked on a paper regarding just this question (http://arxiv.org/abs/1403.8139). We were generalizing Tokuyama's deformation formula (http://projecteuclid.org/euclid.jmsj/1230129805) for the Schur polynomial, so we used Gelfand-Tsetlin patterns rather than Young tableaux. Tokuyama's original formula is in terms of strict GT patterns, which are in bijection with standard Young tableaux, but extending to the Hall-Littlewood polynomials required using nonstrict GT patterns (which are in bijection with SSYT).
Comment: To me GZ-patterns are as good as SSYT (the bijection is straightforward anyway) — thank you, I'll take a look!
Answer: It is also possible to get a combinatorial description using semi-standard augmented fillings, by specializing the combinatorial formula for Macdonald polynomials (see wikipedia on the Macdonald polynomials).
It is also possible to get a formula as sum over SSYT's and the cocharge statistic. This gives the expansion of HL-polys in terms of Schur polynomials.
Comment: Wikipedia page on Macdonald polynomials a) doesn't contain words 'semi-standard' or 'augmented'; b) doesn't give a combinatorial description of Macdonald polynomials — it mentions a description of _transformed_ Macdonald polynomials that uses '_certain_ combinatorial statistics' inv and maj. I guess details can be found in http://arxiv.org/abs/math/0409538
Comment: ...but extracting something _explicit_ for HL polynomials from that doesn't look like an easy task...
Comment: It depends what you mean by explicit; You can easily write this as a sum over a family of tableaux, with some not-so-difficult combinatorial statistics on said tableaux.


I can't get json.loads to work - don't understand error message

Question: New to development and python and trying to create a json file.
I'm trying to generate a list of people with a start time and onboarding time and incident time:
import datetime
import random
import math
import json
import numpy as np

# generate list of candidates

cand = input("Enter number of candidates: ")
cand = int(cand)
startRange = 0.06 # standard deviation (scale)
startTime = 9.30 #mean (loc)
onboardRange = 0.5
onboardTime = 4.5
incidenceRatio = [0,0,0,0,1] #1 in 5 chance an incident arises

def calCan(): # creates a dictionary of candidates with start times, onboarding, incident times.
    candattr = []
    candidates = {"candidates":candattr}
    a = 1
    while a <= cand:
        b = np.random.normal(loc=startTime, scale=startRange, size=(1, 1)) # startime
        c = np.random.normal(loc=onboardTime, scale=onboardRange, size=(1, 1)) # onboarding time
        d = random.choice(incidenceRatio) * np.random.normal(loc=onboardTime, scale=onboardRange, size=(1, 1)) # incident time - random.choice(incidenceRatio) - takes a random selection from incidenceRation list defined above.
        btime = datetime.time(int(b//1),int(math.ceil(100 * (59 * (b % 1))/100)), 0)#converts bnum into time
        ctime = datetime.time(0,int(c//1),int(math.ceil(100 * (59 * (c % 1))/100)))
        dtime = datetime.time(0,int(d//1) ,int(math.ceil(100 * (59 * (d % 1))/100)))
        candattr.append({
        "name": "candidate {}".format(a),
        "startTime": "{}".format(btime),#json doesn't recognize time so have to store as a string
        "onboardTime": "{}".format(ctime),
        "incidentTime": "{}".format(dtime)
        })
        a = a + 1
    return candidates

And then I'm trying to put this into a Json.
I can get the JSON to print just fine using
candidateList = calCan()

print(json.dumps(candidateList, indent=4))

it returns something like this:
{
    "candidates": [
        {
            "name": "candidate 1",
            "startTime": "09:17:00",
            "onboardTime": "00:03:37",
            "incidentTime": "00:04:04"
        },
        {
            "name": "candidate 2",
            "startTime": "09:16:00",
            "onboardTime": "00:04:19",
            "incidentTime": "00:00:00"
        },
        {
            "name": "candidate 3",
            "startTime": "09:15:00",
            "onboardTime": "00:04:13",
            "incidentTime": "00:03:25"
        }
    ]
}

but when i try to manipulate it using json.loads with
candidateList = calCan()
candidateList = str(candidateList)
data = json.loads((candidateList)

I get all sorts of horrible errors that I can't make heads nor tail of:
Traceback (most recent call last):
  File "queue.py", line 56, in <module>
    data = json.loads(candidateList)
  File "C:\Users\Gordon Rose\AppData\Local\Programs\Python\Python38\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "C:\Users\Gordon Rose\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\Gordon Rose\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

If I try and add double quotes to (candidateList):
data = json.loads("candidateList")

It still doesn't get me anywhere
Traceback (most recent call last):
  File "queue.py", line 56, in <module>
    data = json.loads(("candidateList"))
  File "C:\Users\Gordon Rose\AppData\Local\Programs\Python\Python38\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "C:\Users\Gordon Rose\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\Gordon Rose\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Following some advice below - i have stopped converting candidateList into a string and am using json.load instead.
candidateList = calCan()
data = json.load(candidateList)

this returns:
Traceback (most recent call last):
  File "queue.py", line 56, in <module>
    data = json.load(candidateList)
  File "C:\Users\Gordon Rose\AppData\Local\Programs\Python\Python38\lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
AttributeError: 'dict' object has no attribute 'read'

Comment: `("candidateList")` is a literal string, not a json. I don't see how that makes sense. Perhaps you mean `candidateList`?
Comment: @JohnColeman - if i remove the double quotes i get a different error set again:
Comment: You probably need `json.load()`. `json.loads()` is for loading `string` objects into JSON. Your `candidateList` is already JSON.
Comment: thanks guys - have updated the code. still not there yet unfortunately. I am -very- new to this so appreciate the help.
Answer: In a properly formatted JSON, strings need to be double-quoted but in a string literal for a dictionary, string keys and values are given as single-quoted strings. This is the key difference between json.dumps(candidateList) and str(candidateList) (where candidateList = calCan()).
The first is
'{"candidates": [{"name": "candidate 1", "startTime": "09:18:00", "onboardTime": "00:04:10", "incidentTime": "00:00:00"}, {"name": "candidate 2", "startTime": "09:18:00", "onboardTime": "00:04:23", "incidentTime": "00:00:00"}, {"name": "candidate 3", "startTime": "09:19:00", "onboardTime": "00:05:08", "incidentTime": "00:00:00"}, {"name": "candidate 4", "startTime": "09:14:00", "onboardTime": "00:05:17", "incidentTime": "00:00:00"}, {"name": "candidate 5", "startTime": "09:12:00", "onboardTime": "00:04:38", "incidentTime": "00:00:00"}]}'

and the second is
"{'candidates': [{'name': 'candidate 1', 'startTime': '09:18:00', 'onboardTime': '00:04:10', 'incidentTime': '00:00:00'}, {'name': 'candidate 2', 'startTime': '09:18:00', 'onboardTime': '00:04:23', 'incidentTime': '00:00:00'}, {'name': 'candidate 3', 'startTime': '09:19:00', 'onboardTime': '00:05:08', 'incidentTime': '00:00:00'}, {'name': 'candidate 4', 'startTime': '09:14:00', 'onboardTime': '00:05:17', 'incidentTime': '00:00:00'}, {'name': 'candidate 5', 'startTime': '09:12:00', 'onboardTime': '00:04:38', 'incidentTime': '00:00:00'}]}"

both
data = json.loads(json.dumps(candidateList))

as well as
data = json.dumps(str(candidateList).replace("'",'"'))

work as expected.
Comment: data = json.loads(json.dumps(candidateList)) - works - no errors. Thanks John


Unit root tests ambiguous - is time series stationary?

Question: I am testing a time series (quarterly) for stationarity. However, using the KPSS test, the ADF test and PP test, I get different results (ADF and PP reject non-stationarity, KPSS rejects stationarity, all of them at a 95% significance level). I am not sure how to handle this now. Can I just go for stationarity, because two of the tests resulted in stationarity?
My time series looks as follows:
Comment: What results did you get from your tests?
Comment: p was always smaller than 0.01
Answer: ADF and PP tests address a specific form of nonstationarity, i.e. unit-root nonstationarity. Apparently, you reject that form. However, unit-root nonstationarity it is not the only possible form of nostationarity. Hence, you do not conclude that the data is stationary (you only conclude that there is not enough evidence for unit-root nonstationarity). Combine this with an indication of nonstationarity by the KPSS test, and you are even more convinced the data is nonstationary.
Your data might have some level shift (the first half of the sample does not have quite the same level as the second half) and perhaps some heteroskedasticity (the oscilations in the first half of the sample seem a little larger than those in the second half).
So no, I would not feel comfortable "going for stationarity".
Answer: I am not surprised by these results.  I got them very often.  The KPSS test for some reason is very sensitive, if not overly so, as it rejects the vast majority of variables as stationary.  In other words, it diagnoses almost everything as non-stationary.  Because of that, I have stopped using the KPSS test for stationary diagnostics.  And, I rely on the other tests that seem fairer and more accurate on this issue.  The two other tests you use (PP and ADF) generate far more reliable results on this count.  Also, visually you can tell that your variable appears pretty stationary.
I am revising my answer from 2017.  The KPSS test after all does not reject stationarity as often as I thought it did.  If you run the test in R, it almost always gives you a p-value of 0.1 making it a bit difficult to accept the null hypothesis that the variable is stationary.  But, underneath the calculation you see a red written warning that states "... p-value is greater than the printed value shown."  It means that whether the p-value is 0.11 or 0.99, the test result will show it as 0.1.  In other words, the KPSS does not reject that a variable is stationary nearly as often as I thought it did.  And, now I pretty much use it again all the time in such circumstance.  It is nice to run a stationarity test that runs in the opposite direction of the others as its null is that the variable is stationary instead of non-stationary as in all the other tests.
This does not detract that different tests will give you often contradicting results.  I typically run three tests, and if the variable passes 2 out of 3 tests, I deem it as adequate evidence that the variable is stationary.
Last but not least, when I look at your time series graph your variable appears stationary enough so that it should not render any model that you build using it "spurious."
Answer: There are six different unit root test available in Eviews:

The Augmented Dickey-Fuller (ADF) Test
Dickey-Fuller Test with GLS Detrending (DFGLS)
The Phillips-Perron (PP) Test
The Kwiatkowski, Phillips, Schmidt, and Shin (KPSS) Test
Elliot, Rothenberg, and Stock Point Optimal (ERS) Test
Ng and Perron (NP) Tests

Each test has some assumptions, for example, ADF test should be used is series has auto-correlation but it is homoskedastic.
and PP test should be used if series has both issues suto-correlation and presence of heteroskedasticity.
It is better to select the suitable test for stationarity, instead of applying all.
You can read more about these test at
http://www.eviews.com/help/helpintro.html#page/content/advtimeser-Unit_Root_Testing.html
All the best!


proper interpretation for these notations

Question: we have this 2 equations listed below
<code>eq 1. δ[n] = u[n] - u[n-1]
eq 2. y[n] = x[n] - x[n+1]
</code>
Now the question is, 1.) which of the above equation corresponds to a time shift? (a sequence and then we simply shift them forward/backward with respect to the original sequence)?
2.) how is x[n+1] in equation 2 properly interpreted? is it the whole x[n] sequence shifted to the left, or we simply pick a value for n and add 1 to it(so basically we are referring to the value right after x[n])?
Answer: Neither is a time shift. A backward time shift of the $u$ sequence, for instance, would be $v_n=u_{n-1}$.
The $\delta$ sequence is the backward difference of the $u$ sequence; the $y$ sequence is the negative of the forward difference of the $x$ sequence.
Both equations are to be understood as holding for each possible value of $n$, which probably means either for all $n\in\Bbb N$, for all $n\in\Bbb Z^+$, or possibly for all $n\in\Bbb Z$. If, for instance $u_n=n^2$ for $n\in\Bbb N$, so that $u_0=0,u_1=1,u_2=4$, and so on, then $\delta_1=u_1-u_0=1$, $\delta_2=u_2-u_1=3$, and in general
$$\delta_n=u_n-u_{n-1}=n^2-(n-1)^2=2n-1\;.$$


ArrayIndexOutOfBoundsException error on two-dimensional array

Question: I want to check the 8th positions adjacent to each of the positions of my two-dimensional array but the firsts ones give me error because there aren't positions there.
How can I avoid that error? The same happened to me in the past and I couldn't solve it neither.
Example image of what I want to do:

Code of my method:
 private static void evoluciona(char[][] tauler2, char[][] tauler22) {

        for(int i = 0; i < files; i++) {
            for(int j = 0; j < columnes; j++) {
                tauler2[i][j] = tauler[i][j];
            }
        }

        for(int i = 0; i < files; i++) {
            for(int j = 0; j < columnes; j++) {
                if(tauler2[i][j] == 'x') {

                    int cont = 0;

                    if(tauler2[i-1][j] == '*') {
                        cont++;
                    }

                    if(tauler2[i-1][j-1] == '*') {
                        cont++;
                    }

                    if(tauler2[i-1][j+1] == '*') {
                        cont++;
                    }

                    if(tauler2[i][j-1] == '*') {
                        cont++;
                    }

                    if(tauler2[i+1][j-1] == '*') {
                        cont++;
                    }

                    if(tauler2[i+1][j] == '*') {
                        cont++;
                    }

                    if(tauler2[i+1][j+1] == '*') {
                        cont++;
                    }

                    if(tauler2[i][j+1] == '*') {
                        cont++;
                    }

                    if(cont == 2 || cont == 3) {
                        tauler2[i][j] = '*';
                    }
                }

                else if(tauler2[i][j] == '*') {

                    int cont = 0;

                    if(tauler2[i-1][j] == '*') {
                        cont++;
                    }

                    if(tauler2[i-1][j-1] == '*') {
                        cont++;
                    }

                    if(tauler2[i-1][j+1] == '*') {
                        cont++;
                    }

                    if(tauler2[i][j-1] == '*') {
                        cont++;
                    }

                    if(tauler2[i+1][j-1] == '*') {
                        cont++;
                    }

                    if(tauler2[i+1][j] == '*') {
                        cont++;
                    }

                    if(tauler2[i+1][j+1] == '*') {
                        cont++;
                    }

                    if(tauler2[i][j+1] == '*') {
                        cont++;
                    }

                    if(cont == 2 || cont == 3) {
                        tauler2[i][j] = '*';
                    }

                    else {
                        tauler2[i][j] = 'x';
                    }
                }
            }
        }
    }

Comment: Ignoring the exception doesn't solve the problem.
Comment: It sure does...
Comment: @Idos You mistake a matter of education for a matter of code. Hiding the exception will allow the code to run for in-bounds cases, but it doesn't help OP understand what his code is doing. As he explicitly mentioned, he's run into this type of problem before without understanding how to resolve it. Stack Overflow is about learning, not "fix my code for me."
Answer: I had a similar problem when programming my own little Minesweeper clone around christmas.
I ended up with a field class that knows its neighbors.
class Field {
    char value;
    Field[] neighbors;
}

Whenever initializing a new board those 'connections' were sort of precompiled.
class FieldManager {

    void initializeFields(int firstDimension, int secondDimension) {
        fields = new Field[firstDimension][secondDimension];
        for(int i = 0; i < fields.length; i++) {
            for(int j = 0; j < fields[i].length; j++) {
                fields[i][j] = new Field();
                fields[i][j].neighbors = neighborsFor(i, j);
            }
        }
    }

    private Field[] neighborsFor(int iIn, int jIn) {
        List<Field> result = new ArrayList<>();
        for(int i = iIn - 1; i <= iIn + 1; i++) {
            for(int j = jIn - 1; j < jIn + 1; j++)
                // do some range checks and fill result
            }
        }
        return result.toArray(new Field[result.size()]);
    }

    Field[][] fields;
}

Instead of array you can as well use Collections, such as ArrayList. Especially the newer java versions have a lot of useful utility built around those interfaces.
Answer: I'd recommend explicitly checking against boundaries!
Modify your if statements, so that whenever you have i+1, j+1 you check it's not too large ( i < files and j < columnes), and i-1, j-1 that they are not 0 ( i > 0 and j > 0).
As you are counting neighboring stars, this method will not add extra stars if you are on the border.
Alternatively, the "lazy way", you can try{} catch(ArrayOutOfBoundsException e) { continue; } inside the inner loops!
Comment: There are situations where ignoring exceptions is an appropriate thing to do, but I'd say that's rare (or possibly non-existent for out-of-bounds exceptions). Ignoring exceptions is generally a bad practice which new developers should avoid in favour of focusing on understanding the reason for the exception and learning how to write code to *properly* avoid it. @Óscar73
Comment: That "try{} catch(ArrayOutOfBoundsException e) { continue; }" made the trick! Many thanks!
Answer: In all the places that you have an index of i-1 or j-1, on the first iteration of either cycle (i or j) you can have an index of -1 as per your exception in the stack trace, which causes the ArrayIndexOutOfBoundsException.
Instead of catching it and ignoring it, place an if around the areas that can cause this exception before doing anything.
// your code
...

if (i - 1 >= 0) {
    if(tauler2[i-1][j] == '*') {
        cont++;
    }
}

...

and likewise in the case of tauler2[i][j-1], a condition check on j - 1 >= 0.
Answer: you are accessing such an index that is not in array. As you are starting forloops i-e i=0 and j=0,  and in if conditions u r using indexes that are [i-1] and [j-1]. here u r getting exception of index out of bounds.  eliminate or change these conditions.


Sequence of product of Rademacher variables

Question: I am trying to figure out what a sequence of product of Radenmacher variables looks like.
The sequence is defined as: Let $\{\nu_j\}$ be IID fair ±1 coin-flips (Rademacher variables). Let $Y_1 = \nu_1$, let $Y_2 = \nu_1 \nu_2$
and for $m = 2^{n−1} + j$ with $0 < j ≤2^{n−1}$ let $Y_m = \nu_{n+1}Y_j$.
Given the definition above, I can't figure out what is the form of $Y_3, Y_4, Y_5$ and so on. Would appreciate any help.
The question asks me to prove that $Y_i$ are independent which seems to be true since it is the product of coin flips. But I am finding the notation for how $Y_i$ is defined very confusing.
Answer: Let's find a few more:

$m=3$: Since $3 = 2^{2-1} + 1$, we conclude $Y_3 = \nu_3Y_1$.
$m=4$: Since $4 = 2^{2-1} + 2$, we conclude $Y_4 = \nu_3Y_2$.
$m=5$: Since $5 = 2^{3-1} + 1$, we conclude $Y_5 = \nu_4Y_1$.
$m=6$: Since $6 = 2^{3-1} + 2$, we conclude $Y_6 = \nu_4Y_2$.

From this, we see $Y_6 = Y_3Y_4Y_5$, so in particular, the $Y_i$ aren't independent.
Comment: The phrasing of the question implies that the $\mathsf{Y}_i$'s are simultaneously independent, which I agree they are not. However I think maybe OP is asked to show they are pairwise independent, which seems to be true since any pair $\mathsf{Y}_i, \mathsf{Y}_j$ has at least one $\nu_k$ not in common.
Answer: The procedure for writing $Y_i$ in terms of the $\nu_j$ is to take $i-1$ in binary, add a $1$ to the end of the binary string, and take the product of the $\nu_j$ over all $j$ where bit $j$ (counting right to left, starting with $1$) is a $1$.
For example, to find $Y_6$, note that $6-1 = 5$, the binary representation of $5$ is $101$, adding a $1$ to the end gives $1011$, which has the first, second, and fourth bits from the right equal to $1$. So $Y_6 = \nu_1 \nu_2 \nu_4$.
The proof for this is just an induction; the recurrence relation makes it clear that the recurrence can be defined in terms of the binary expansion and all you need to do is verify the base cases.
The other answer and comment tell you how to show that these variables are not independent, but are pairwise independent.
(I am curious about the context for why the $Y_i$ are defined that way. Was it to enumerate all possible products of the $\nu_i$ that contain $\nu_1$?)