memory pools: will they improve cache usage for structs larger than the cache line size?

Question: As I understand it, memory pools should improve cache performance for objects commonly accessed together, if the objects are smaller than the cache line size - because then adjacent objects will likely be fetched into the cache at the same time.
But what about objects larger than the cache line size?  Is there any benefit to pooling such data into the same region of memory?
(Assuming that allocation/deallocation times are insignificant, it's access I'm worried about...)
Thanks!
Comment: You're talking about _the_ cache line size. Most systems have multi-level caches with different cache line sizes. Also, main memory is organized in pages, and a page which isn't actively accessed is powered down a bit. I.e. even main memory benefits from locality of reference.
Answer: One important reason for using pools is that they make for a much simpler allocation scheme than a general-purpose allocator. Since all objects have the same size, there's no fragmentation, and you just need to maintain a free list. For a new allocation, you try to pop off the top of the free list, or if the list is empty you increment the high watermark, done. (You can implement the free list in O(1) space inside the pool memory itself.)
However, the use of pools is highly situational, and whether there's any benefit depends very much on your actual code path and allocation requirements. The modern standard allocator is already very good with many short-lived fixed-size allocations, so you really need to profile and check.
Answer: Memory pooling makes sense if your app uses a huge amount of memory and starts to swap. Then, if the objects lie adjacent to each other, they will be paged in and out together.
Comment: I'm marking this definitive - while Kerrick's answer is true, it is really about allocation/deallocation which doesn't bother me.  MSalters comment is relevant too.  But in general I concluded that access will only be significantly faster in the case you highlight here.
Comment: As a footnote, I did find another use for pools: they mean that pointers tend to be allocated in strictly increasing order of address.  This greatly speeds up a routine I have which builds a large stl::set of objects indexed by their pointers, as a heap insert hint can then be used.  (The objects are also indexed by another field, but that happens to be equal for all of them at heap creation time).
Answer: My previous project was an embedded app with built-in webserver for ARM SAM9x platform. It has only 64K heap, and does not have any console/display or filesystem, so there's no way to just printf() an error to stderr or log it to file. Altough, it must work 7/24 there's it should not stop with "out of memory" error, it must run without error. If it's started once, it would never stop. Out of memory is not a recoverable error, it's the complete fail of the system.
So, I decided not to use new. I've been used object arrays: ringbuffers, fixed-size pools, etc. - and it just works. Java (and C# etc.) makes us wrong, these modern languages say, that memory is a big ocean, which anyone can dip from. Yep, it's true, if you have plenty, but the cost is high, just as you brought up in this post.
Try it! Use as few new (and malloc(), of course), just as possible. A nice side effect: you have not to use delete (and free()), there would be no memory leak problems.
Comment: In C++, you can implement `operator new` yourself, even on a class by class basis. Therefore, the use of "fixed-size pools" does not imply that you have to forego `new`. Besides, you _do_ have to use `delete` or an equivalent, even with fixed-size pools. After all, you must return unused memory to the pool or it will exhaust quickly.
Comment: this is not an answer to his question.
Comment: it is not about memory allocation optimization, it is about memory _access_ optimization, which is an entirely different concern, even if it can be obtained by particular allocation disciplines.
Comment: The question is about mem alloc optimizing, I just added another approach.
Comment: It's a kind of programming *style* not using new/delete in C++. You allocate all objects compile-time. This is a kind of **round** the problems of pool. There are wide range of problems where this method works - and there're also several, where you can't live without allocating objects runtime. *Before* you thinking on pools or own allocator, check this option.


Showing that $\{z\in\mathbb{C}:|z-1|<|z+i|\}$ is an open set

Question: Got stuck on some homework (from H. A. Priestley, Complex Analysis). My topology ain't quite up to speed yet.
So, I want to show that $S=\{z\in\mathbb{C}:|z-1|<|z+i|\}$ is open. Geometrically it's the points above the line through $1-i$ and the origin. (Eh?)
So, essentially what I want to do is to prove that for every $z\in S$, there's a $r>0$ such that $D(z;r)\subset S$. In more concrete words, I want to figure out an $r$ such that the implication $|w-z|<r \Rightarrow |w-1|<|w+i|$ holds.
I started to fiddle around with the triangle inequality
$$|w-1|=|w-z+z-1|\leq |w-z|+|z-1|$$
and then figured I could set $0<r<|z+i|-|z-1|$, which then would yield
$$|w-1|\leq |z+i|$$
but that isn't quite it. I tried to use more terms in the "triangle inequality trick" but I can't seem to get it the way I want it. Help me oh math.stackexchange! (If you want, of course.)
Comment: Complex numbers, yeppers, of course. Been working with discrete maths a lot recently
Answer: Regardless of the form of the set $S,$ this is actually fairly straightforward to prove, using basic topology results.
Consider the function $f:\Bbb C\to\Bbb R$ given by $$f(z)=|z+i|-|z-1|.$$ One can show that $f$ is continuous, which I leave to you, and that $$S=\{z\in\Bbb C:f(z)>0\},$$ which again I leave to you. Put another way, $$S=f^{-1}\bigl[(0,\infty)\bigr]:=\bigl\{z\in\Bbb C:f(z)\in(0,\infty)\bigr\}.$$ Since $f:\Bbb C\to\Bbb R$ is continuous and $(0,\infty)$ is open in $\Bbb R,$ what can we then say about $S$?
If you have no earthly idea what I'm talking about, let me know, and I'll try to clear things up for you.

Added: Let's explore this a bit further, using the familiar (but a bit tedious) definition of continuity:

If $z_0\in\Bbb C$ and $g:\Bbb C\to\Bbb C,$ we say that $g$ is continuous at $z_0$ if $$\forall\epsilon>0,\exists\delta>0:\bigl(|z-z_0|<\delta\bigr)\implies\bigl(|g(z)-g(z_0)|<\epsilon\bigr).$$ We say that such a $g$ is a continuous function if it is continuous at all $z_0\in E.$ Put another way: $$\forall z\in \Bbb C,\forall\epsilon>0,\exists\delta>0:\bigl(|w-z|<\delta\bigr)\implies\bigl(|g(w)-g(z)|<\epsilon\bigr).$$

Well, that's all pretty messy looking. How does it help? There are a few steps to take.
First, we prove that the function $f$ described above is continuous. (The proof is basically just several applications of the triangle inequality, and is a very good exercise. Let me know if you get stuck, or if you just want to bounce your thoughts off somebody.)
Next, we note/prove that $S$ is precisely the set of all $z\in\Bbb C$ for which $f(z)>0.$
Next, we want to use these fact to show that $S$ is open, but it may not seem clear how this is possible! The kicker is to translate continuity into terms of open sets. Well, since $f:\Bbb C\to\Bbb R$ is continuous, that means $$\forall z\in\Bbb C,\forall\epsilon>0,\exists\delta>0:\bigl(|w-z|<\delta\bigr)\implies\bigl(|f(w)-f(z)|<\epsilon\bigr),$$ which can instead be put as $$\forall z\in\Bbb C,\forall\epsilon>0,\exists r>0:\bigl(w\in D(z;r)\bigr)\implies\bigl(|f(w)-f(z)|<\epsilon\bigr).$$ (Do you see why?) From this, it follows (since $S\subseteq\Bbb C$ that $$\forall z\in S,\forall\epsilon>0,\exists r>0:\bigl(w\in D(z;r)\bigr)\implies\bigl(|f(w)-f(z)|<\epsilon\bigr).$$ Note moreover that for all $z\in S,$ we have $f(z)>0,$ so it follows that $$\forall z\in S,\exists r>0:\bigl(w\in D(z;r)\bigr)\implies\bigl(|f(w)-f(z)|<f(z)\bigr).$$
Now, you should be able to show that if $|f(w)-f(z)|<f(z),$ then $f(w)>0.$ Hence, we have $$\forall z\in S,\exists r>0:\bigl(w\in D(z;r)\bigr)\implies(f(w)>0).$$ Can you justify the above claims and take it from there?
Comment: wow,  it is excellent
Comment: Hello, I have no earthly idea what you're talking about.
Comment: @Tobias: What definition of continuity are you familiar with?
Comment: The one back from calculus, which isn't particularly helpful here. $lim_{x\rightarrow a}f(x)=f(a)$
Comment: Gotcha. What definition of $L=\lim_{x\to a}f(x)$ are you familiar with?
Comment: The good ole epsilon delta one. $\forall \epsilon > 0.\exists \delta. |x-a|<\delta\Rightarrow |f(x)-L|<\epsilon$.
Comment: Excellent! I can work with that.
Comment: @Tobias: I have expanded my answer to give you an idea how we may use continuity to our advantage. See what you can do with it, and let me know if you have any questions.
Comment: @CameronBuie: All very nice. I get it, but I must be able to reproduce such an argument. Yep, all very nice. Pretty. Thanks!
Comment: @Tobias: Following through with the proof will not only make you more familiar with methods that you'll be using more later. It will also show you how to choose $r$ and use triangle inequality to prove it (as you originally intended).
Comment: @CameronBuie: But I don't quite see how this quite formal approach can help me get to a short solution like Litho's? However, I'd probably be way better at reproducing an argument like this, than the "out of a hat" argument (albeit very pretty) that Litho used.
Comment: @Tobias: To prove that $f$ is continuous, you'll be applying the same sort of triangle inequality approach you were already taking, and after you've gone as far as you can with those, a "natural choice" for $r$ (for $\delta$) should become self-evident. See if you can get that, at least, and let me know if you get stuck. I'll help you get "unstuck" as soon as I can.
Comment: Ok, I'll check the continuity a bit more. If I do get more stuck I at least have to spend a day or so to resolve it myself. You sure gave me a lot to work with. Thanks!
Answer: Choose $r = \frac{|z+i|-|z-1|}2$. Then, if $|w-z|<r$,
$$
|w-1|<|z-1|+r= |z+i|-r<|w+i|.
$$
Comment: Why $|z+i|-r<|w+i|$?
Comment: @MarioG : Because by triangle inequality $|z+i|<|w+i|+|(z+i)-(w+i)|=|w+i|+|z-w|<|w+i|+r$.
Comment: Heck yeah! Nice, just what I was looking for.
Comment: @Litho: How did you figure that out? Any particular strategy?
Comment: @TobiasShuxueLaoshi : Well, you can look at it like this: we have the function $f(z)=|z+i|-|z-1|$. It consists of two summands such that, when the argument $z$ is changed by $a$, each summand changes by at most $|a|$. It means that $|f(z+a)-f(z)|\leq 2|a|$. So, if $f(z)>0$ and $a0$. Note, though, that it's easy to find an estimate on $r$ in this case only because the function $f$ is nice enough. Cameron Buie's answer contains a much more general method applicable for an arbitrary continuous $f$.
Comment: @Litho: Yes, after fully digesting it, I see the power of his method.


Flutter Firestorage getDownloadUrl() returns NoSuchMethodError

Question: inside my Flutter app I'm trying to upload an image to Firebase Storage and get the URL afterwards. The image is getting uploaded and I can see it in the web interface, but when I call getDownloadUrl() on a valid reference (I can see the relative path with debugger) I get type 'NoSuchMethodError' is not a subtype of type 'Exception' and the actual error is happening inside method_channel_reference.dart where for some reason, where storage is null and as such storage.app throws the Error.
 @override
  Future<String /*!*/ > getDownloadURL() async {
    try {
      Map<String, dynamic> data = await MethodChannelFirebaseStorage.channel
          .invokeMapMethod<String, dynamic>(
              'Reference#getDownloadURL', <String, dynamic>{
        'appName': storage.app.name,
        'maxOperationRetryTime': storage.maxOperationRetryTime,
        'maxUploadRetryTime': storage.maxUploadRetryTime,
        'maxDownloadRetryTime': storage.maxDownloadRetryTime,
        'bucket': storage.bucket,
        'path': fullPath,
      });

      return data['downloadURL'];
    } catch (e) {
      throw convertPlatformException(e);
    }
  }

The upload code is this:
final uploadTask = _firebaseStorage.ref().child(storagePath).putFile(recipeImage);

    try {
      await uploadTask.whenComplete(() {});
      final url = await uploadTask.snapshot.ref.getDownloadURL();
      return right<RecipeFailure, String>(url);
    } catch (e) {
      return left<RecipeFailure, String>(RecipeFailure.imageUploadFailed());
    }

I'm using
firebase_core: ^0.7.0
firebase_auth: ^0.20.0+1
firebase_storage: ^7.0.0
cloud_firestore: ^0.16.0

I tried cleaning, rebuilding and downgrading dependencies. At this point I don't how where else to look. I guess the issue is with the storage instance being removed or not initialized at all, but why does the upload work then? Any help is welcomed.
Answer: I hope this can help you, but you can store the link in each upload in the document for each product or user wherever it is.
In this method, for example, you can retrieve the link file from the same document, and you do not need another way to retrieve it.
In this code you can upload the file and restore the downloadUrl when you have finished uploading the file.
 static Future<dynamic> uploadFile(
      {@required File imageFile, @required String folderPath}) async {
    String fileName = DateTime.now().millisecondsSinceEpoch.toString();

    Reference reference =
    FirebaseStorage.instance.ref().child(folderPath).child(fileName);

    TaskSnapshot storageTaskSnapshot = await reference.putFile(imageFile);
    // TaskSnapshot storageTaskSnapshot =  uploadTask.snapshot;

    print(storageTaskSnapshot.ref.getDownloadURL());

    var dounloadUrl = await storageTaskSnapshot.ref.getDownloadURL();

    return dounloadUrl;
  }

Answer: When I changed from
@lazySingleton
FirebaseStorage get firebaseStorage => FirebaseStorage.instanceFor(bucket: "bucket-url");

to this
@lazySingleton
FirebaseStorage get firebaseStorage => FirebaseStorage.instance;

the error disappeared and everything was working properly.
The main reason I have the bucket URLs there was because I was working with project flavors and two Firebase projects. The seperate google-services.json seem to be enough for Android (haven't tested iOS until now). However, why the upload was working, but download not is still a big mistery to me.


Outliers and possible dispersion in neg. binomial glmm residuals (DHARMa package)

Question: I need help fixing the model I landed on through backwards step-wise elimination. I chose a negative binomial model because my variance seems much larger than the mean, with random intercepts from the variable "Site" because my sampling scheme has a repeated measures design (fish counts recorded at the same sites each season/year). I wasn't sure how to make a reproducible dataframe as there are 1000+ rows of data - I attached a subsample of 10 rows to give an idea.
Prior to modelling question: Should SAV (% cover = 0->100) be arcsine transformed for this model?
Main question:
I'm not sure what to do about outliers and potential dispersion (p=0.064). After fitting the model, I tried adding "~Site" to the dispformula, but this resulted in model convergence problems. I'm also not sure why my model summaries don't show the random effects section, was this entered correctly?
Zero-inflation doesn't seem to be a problem.
My data (CYR = calendar year, num = count of fish):
> str(toad)
'data.frame':   1262 obs. of  16 variables:
 $ use_for_analysis: chr  "Standard" "Standard" "Standard" "Standard" ...
 $ CYR             : int  2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 ...
 $ Season          : chr  "DRY" "DRY" "DRY" "DRY" ...
 $ Month           : int  1 1 1 1 1 1 1 1 1 1 ...
 $ Site            : Factor w/ 47 levels "1","2","3","4",..: 10 11 12 13 14 15 16 17 9 1 ...
 $ area_sampled    : int  3 3 3 3 3 3 3 3 3 3 ...
 $ common_name     : chr  "Gulf Toadfish" "Gulf Toadfish" "Gulf Toadfish" "Gulf Toadfish" ...
 $ num             : int  0 0 0 0 0 1 0 0 0 0 ...
 $ den             : num  0 0 0 0 0 ...
 $ occur           : int  0 0 0 0 0 1 0 0 0 0 ...
 $ temp            : num  23.3 23.7 22.8 22.6 22.7 ...
 $ sal             : num  30.7 30.9 30.2 30 29.8 ...
 $ DO              : num  6.18 7.56 5.62 5.85 6.49 6.29 5.21 5.36 6.66 5.9 ...
 $ sed_depth       : num  23 48 12 32 43 45 49 32 23 90 ...
 $ water_depth     : num  80 72 98 95 97 85 78 90 58 74 ...
 $ SAV             : num  85.6 53.8 81.6 71.8 91.5 ...

# Variance not equal to the mean
> var(toad$num)
[1] 1.97994
> mean(toad$num)
[1] 0.3716323

toad_example <- toad %>% dplyr::select(CYR, Season, Month, Site, area_sampled, num, temp,
                               sal, DO, sed_depth, water_depth, SAV)

# Randomly selected rows for display purposes (N = 1262 rows)
toad_example <- toad_example[sample(nrow(toad_example), 10), ]

toad_example:
     CYR Season Month Site area_sampled num  temp   sal   DO sed_depth water_depth  SAV
1476 2020    WET     9    8            3   0 30.30 20.55 7.83         8          65 93.5
1527 2021    WET     9   40            3   0 28.80 24.28 4.83        37         117 53.5
577  2011    DRY     1   16            3   0 23.43 27.32 9.05        44          78 98.0
668  2012    DRY     1   43            3   0 20.45 25.74 7.17        37          58 87.0
864  2014    DRY     3   36            3   0 22.40 17.31 8.26        61          80 34.0
507  2010    DRY     1   24            3   0 20.41 23.67 8.65        51          20 45.5
1180 2017    DRY     3   45            3   1 23.20 19.87 7.42        95          80 94.0
861  2014    DRY     3   33            3   1 22.40 18.50 9.98        14          75 52.0
506  2010    DRY     1   23            3   0 20.08 23.41 9.55        31          35 49.0
808  2013    WET     9   47            3   1 30.70 24.88 6.45        79          71 79.5

Original model:
full <- glmmTMB(num ~ temp + sal + DO + sed_depth + water_depth +
               SAV + CYR*Season + (1|Site) + offset(area_sampled),
               family=nbinom1, toad)

# Warning message: In (function (start, objective, gradient = NULL, hessian = NULL,:
# NA/NaN function evaluation

> summary(full) # AIC 1599.9
 Family: nbinom1  ( log )
Formula:          num ~ temp + sal + DO + sed_depth + water_depth + SAV + CYR *
    Season + (1 | Site) + offset(area_sampled)
Data: toad

     AIC      BIC   logLik deviance df.resid
  1599.9   1661.6   -788.0   1575.9     1250

Random effects:

Conditional model:
 Groups Name        Variance Std.Dev.
 Site   (Intercept) 0.3403   0.5834
Number of obs: 1262, groups:  Site, 47

Dispersion parameter for nbinom1 family (): 2.18

Conditional model:
                Estimate Std. Error z value Pr(>|z|)
(Intercept)   -3.324e+02  7.549e+01  -4.403 1.07e-05 ***
temp          -3.557e-02  4.003e-02  -0.889   0.3742
sal            1.027e-02  1.041e-02   0.987   0.3238
DO             7.336e-02  3.667e-02   2.001   0.0454 *
sed_depth      6.846e-03  2.976e-03   2.300   0.0214 *
water_depth    1.725e-03  4.120e-03   0.419   0.6755
SAV            2.800e-02  4.150e-03   6.748 1.50e-11 ***
CYR            1.615e-01  3.762e-02   4.293 1.76e-05 ***
SeasonWET      4.182e+02  8.526e+01   4.904 9.38e-07 ***
CYR:SeasonWET -2.070e-01  4.224e-02  -4.899 9.62e-07 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Step-wise elimintion:
reduced <- glmmTMB(num ~ sal + DO + sed_depth + water_depth +
                  SAV + CYR*Season + (1|Site) + offset(area_sampled),
                  family=nbinom1, toad)

summary(reduced) # AIC 1598.7

reduced2 <- glmmTMB(num ~ DO + sed_depth + water_depth + SAV +
                   CYR*Season + (1|Site) + offset(area_sampled),
                   family=nbinom1, toad)

summary(reduced2) # AIC 1597.3

reduced3 <- glmmTMB(num ~ DO + sed_depth + SAV + CYR*Season + (1|Site) +
                    offset(area_sampled), family=nbinom1, toad)

summary(reduced3) # 1595.8

reduced4 <- glmmTMB(num ~ sed_depth + SAV + CYR*Season + (1|Site) +
                    offset(area_sampled), family=nbinom1, toad)

summary(reduced4) # AIC 1596.9 (reduced3 is slightly better)

Final model:
final <- glmmTMB(num ~ DO + sed_depth + SAV + CYR*Season +
                (1|Site) + offset(area_sampled), family=nbinom1, toad)

> summary(final)
 Family: nbinom1  ( log )
Formula:          num ~ DO + sed_depth + SAV + CYR * Season + (1 | Site) + offset(area_sampled)
Data: toad

     AIC      BIC   logLik deviance df.resid
  1595.8   1642.1   -788.9   1577.8     1253

Random effects:

Conditional model:
 Groups Name        Variance Std.Dev.
 Site   (Intercept) 0.3549   0.5958
Number of obs: 1262, groups:  Site, 47

Dispersion parameter for nbinom1 family ():  2.2

Conditional model:
                Estimate Std. Error z value Pr(>|z|)
(Intercept)   -3.054e+02  6.641e+01  -4.599 4.24e-06 ***
DO             6.258e-02  3.528e-02   1.774   0.0761 .
sed_depth      6.814e-03  2.992e-03   2.277   0.0228 *
SAV            2.803e-02  4.150e-03   6.754 1.44e-11 ***
CYR            1.479e-01  3.286e-02   4.501 6.77e-06 ***
SeasonWET      3.803e+02  7.735e+01   4.916 8.84e-07 ***
CYR:SeasonWET -1.883e-01  3.837e-02  -4.907 9.23e-07 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residuals:
simulationOutput <- simulateResiduals(final)
plotResiduals(simulationOutput, main=NULL)

testUniformity(simulationOutput)

testOutliers(simulationOutput, type = 'bootstrap')

> testDispersion(simulationOutput, type = "DHARMa")

    DHARMa nonparametric dispersion test via sd of residuals fitted vs. simulated

data:  simulationOutput
dispersion = 1.4687, p-value = 0.064
alternative hypothesis: two.sided

Zero-inflation:
> testZeroInflation(simulationOutput, plot = T)

    DHARMa zero-inflation test via comparison to expected zeros with simulation under H0 = fitted
    model

data:  simulationOutput
ratioObsSim = 0.99281, p-value = 0.728
alternative hypothesis: two.sided

This model worked better. I had to include Year (categorical) as a random effect to capture the relatedness of observations within years, and use Year (continuous) as a fixed effect to capture any trends in abundance over time.
Additionally, even though sample effort doesn't differ by year, I needed an offset term, because area needs to be logged (negative binomial uses a log-link function and this keeps the units consistent with what is happening to the count).
*Not sure if I need to "center" the variables (grand mean or cluster) or include an optimizer though..
reduced4 <- glmmTMB(num ~ DO + SAV + CYR*Season + (1|Site) +
                      (1|as.factor(CYR)) + offset(log(area_sampled)),
                    family=nbinom1, toad)

> summary(reduced4) # AIC 1568.0 (reduced3 is slightly better)
 Family: nbinom1  ( log )
Formula:          num ~ DO + SAV + CYR * Season + (1 | Site) + (1 | as.factor(CYR)) +
    offset(log(area_sampled))
Data: toad

     AIC      BIC   logLik deviance df.resid
  1568.0   1614.2   -775.0   1550.0     1253

Random effects:

Conditional model:
 Groups         Name        Variance Std.Dev.
 Site           (Intercept) 0.5014   0.7081
 as.factor(CYR) (Intercept) 0.2309   0.4805
Number of obs: 1262, groups:  Site, 47; as.factor(CYR), 15

Dispersion parameter for nbinom1 family (): 1.88

Conditional model:
                Estimate Std. Error z value Pr(>|z|)
(Intercept)   -2.521e+02  9.091e+01  -2.773  0.00555 **
DO             7.934e-02  3.800e-02   2.088  0.03683 *
SAV            3.047e-02  4.515e-03   6.748 1.49e-11 ***
CYR            1.223e-01  4.506e-02   2.714  0.00664 **
SeasonWET      3.568e+02  8.365e+01   4.265 2.00e-05 ***
CYR:SeasonWET -1.766e-01  4.150e-02  -4.256 2.08e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Comment: Re: Arcsine transformation: https://esajournals.onlinelibrary.wiley.com/doi/full/10.1890/10-0340.1
Answer: Actually, based on what I see here alone, it doesn't look like your model is in some serious danger. The DHARMa vignette explains that the visual indicators of the residuals are a lot more informative than the p-values, especially when sample sizes are large (there is a fair amount of literature on how normality tests are actually fairly problematic for this very reason alone). Your qq plots and quantile plots don't look seriously bad, as the qq plot is pretty linear and your quantiles dont have any bizarre curves. Your test of dispersion is also showing your model is underdispersed...the same vignette explains that mixed models are biased towards underdispersion and it is usually better to set the alternative argument in the function to "greater" for overdispersion tests.
Here is the vignette if you haven't read it already. I find it to be a good explanation of this package and trouble-shooting common issues. It also has a lot of examples of way more problematic visuals of DHARMa residuals:
https://cran.r-project.org/web/packages/DHARMa/vignettes/DHARMa.html


List of images with checkbox (user registration)

Question: This is what I would like to do:
When a user registers on the website they have to give in if some options like for example if they are politically engaged, professionally engaged, an expert, ... .
I also want a field that contains 3 images. And based on the chosen options of the user I would like to select 1,2 or 3 of them. So the user can contain image 1 & 2 or ... .
So I was thinking of 3 images with 3 checkboxes and cross the checkboxes next to the images (programmatically). But is this a good way to do this? And if yes, how can I implement this?
Answer: You can do that by the following.
Go to Configuration » People » Account settings
Add a Field type : List (text) and WIDGET: Check boxes/radio buttons
In Allowed values list  textarea add key|
Enter one  per line and set Number of values as Unlimited (to appear as checkboxes)
That's it. I tested it and working fine.
Add key whatever you want.You may have to make minor CSS changes in order to align checkbox and image.
Hope this helps.
Comment: @nielsv Glad to hear that. :)


References/clarifications for the Mittag-Leffler argument

Question: While reading the paper MC elements in pronilpotent dg Lie algebras by Yekutieli, I stumbled across the folowing lines:

According to lemma 3.10(1) the complexes $M_j$ are acyclic. Threfore, using the Mittag-Leffler argument, the complex $M$ is also acyclic.
(page 15, in the proof of lemma 4.1(1))

Here, $M,M_j$ are complexes of $R$-modules, $R$ a ring, and $M\cong \lim_{\leftarrow j}M_j$. I would like some clarification about this Mittag-Leffler argument. I think I can reverse-engineer something close enough to the correct statement in this case, but what is the precise statement? What assumptions do we need, and how to prove it?
I tried looking it up on google, but I couldn't find any (elementary) exposition of it. Any insight or reference is welcome.
Answer: A. Grothendieck, Éléments de Géométrie Algébrique, $0_{III}$ Proposition 13.2.3 (on page 66 of the first volume of EGA III), since in your case $\{M_*\}$ and $\{H(M_*)\}$ are Mittag-Leffler.
Comment: Thank you. I added the link to the book to your answer.


Chords between i and an augmented-sixth chord

Question: I am wondering about chords that can go between i and different augmented-sixth chords. Are there any resources on this? I've googled and searched past questions and didn't find anything...
edit: in the context of a second-semester college harmony course
Comment: Sorry, by A6, do you literally mean a chord consisting of A, C#, E, and F#? You're mixing two sets of chord notations, so I don't quite know what you mean.
Comment: Any chord can go before or after or between any other chords.
Comment: see edit, I mean in the context of a second-semester college harmony course, as in contrapunctally
Comment: I mean an Augmented 6th chord
Answer: When we think of the bassline, we see that we'll have scale-degree 1 in the bass of the tonic moving to scale-degree 6 in the bass of the augmented sixth. What's especially common connecting those two chords is having a lowered scale-degree 7 in the bass.
As such, if you're moving directly from tonic to the augmented sixth, I'd recommend v6 as the passing chord between them. Notice that this will be a minor chord on account of having the subtonic in the bass as opposed to the leading tone. (It won't be the leading tone, because then we'd have an augmented second between it and the lowered scale-degree 6 of the augmented-sixth chord.)
But if we can insert two chords in between, it's especially common to have a V42/iv after tonic which then resolves to a iv6. Then it's just a slight change from the iv6 to the augmented sixth. This is especially nice because of the chromatic motion in an upper voice from scale-degree 3 of the tonic chord up to the raised scale-degree 4 of the augmented sixth.
Answer: Based on the classical music I've listened to, you don't even need to insert any chords in between i and an augmented 6th chord.
Chords I have heard inserted between i and an augmented 6th chord and put immediately before the augmented 6th chord include V (J. C. Heed's "In Storm and Sunshine"), ♭II a.k.a. the Neapolitan (Franz Schubert's Moments Musicaux No. 5 in F Minor), and (♭)VI (Moments Musicaux No. 5 again).
I'd imagine that vii°6/5 of V would be a good chord to put between i and an augmented 6th chord, though, as it's one note away from a German Augmented 6th, and the root of vii°6/5 of V moves nicely down one semitone to the root of the German Augmented 6th (e.g., in F minor, D-F-A♭-B to D♭-F-A♭-B).
Often, there is more than one chord between i and an augmented 6th chord.


Reference request for Prüfer groups

Question: It will be very helpful for me if I get a reference(notes/book e.t.c) where I can get details about Prüfer groups.
Answer: One of the very good references would be
An Introduction to the Theory of Groups by J. J. Rotman. In Chapter $10$ which is about abelian groups, you  will see some valuable remarks and strong facts about free abelian groups and after that as a very good and practical theorem you'll see that.

Theorem $10.13$: There is an infinite $p-$primary group $G=\mathbb Z(p^{\infty})$ each of whose proper subgroups is finite (and cyclic).

Moreover, you'll find very interesting facts (at least for me) stating that:

$\mathbb Q/\mathbb Z\cong\sum_p\mathbb Z(p^{\infty})$.
$\mathbb R/\mathbb Z\cong\prod_p\mathbb Z(p^{\infty})\cong\bf{T}\cong \mathbb R\oplus(\mathbb Q/\mathbb Z)$.
And if $G$ is an infinite group such that every proper subgroup is finite then for a proper prime $p$, $G\cong\mathbb Z(p^{\infty})$.
Comment: Nice reference. You like your Rotman! ;-) +1
Comment: @amWhy: Somehow, it becomes my Bible.
Comment: @BabakS.: I agree, this is a very nice reference +1

Reference request for Prüfer groups


how could we store other values in list partition in postgresql

Question: how could we store other values in list partition in postgresql?
Sample: How can i add a partition for different values than (1,2,3,4) in table below.
CREATE TABLE countrymeasurements
(
  countrycode int NOT NULL,
  countryname character varying(30) NOT NULL,
  languagename character varying (30) NOT NULL,
  daysofoperation character varying(30) NOT NULL,
  salesparts    bigint,
  replaceparts  bigint
)
PARTITION BY LIST(countrycode);
Define the partitions:

create table india
  partition of countrymeasurements
  for values in (1);

create table japan
  partition of countrymeasurements
  for values in (2);

create table china
  partition of countrymeasurements
  for values in (3);

create table malaysia
  partition of countrymeasurements
  for values in (4);

Answer: Found out the solution now:
create table dwh_user.countrymeasurements_def
partition of countrymeasurements
default;

Comment: Beware. While this will work as you expect, it may prevent you from adding new partitions later: you cannot add a partition if there is already a value in the default partition that would belong in the new partition.


Run vba module with specify of username

Question: is it possible to run excel module when workbook opened by certain user?
The idea is very simple - refreshing all data source when excel workbook opened by certain user, save and close. Another way it just opens.
Comment: Welcome to Stackoverflow. This is probably not the answer you want to see, but why don't you try it first. And then, come to this board with a specific question. Ideally, you could paste some code and / or error messages.
Answer: I'm not sure how "secure" this is, but it worked for me.  Note that you need to put this in the "ThisWorkbook" Object (not a Module):
Private Sub Workbook_Open()
If Environ$("Username") = "bWayne" Then
    MsgBox ("Hi Bruce")
   ' Do whatever
End If
End Sub

Save as .xlsm and when you open it, if the username is bWayne, the message box will show...otherwise, nothing happens.
Answer: Also it is possible to check Excel user's name by using
If Application.UserName = "bWayne" Then
…
End If

in Workbook_Open() procedure.

Run vba module with specify of username


Python or WPF reporting application

Question: I have an existing VS-2008 Windows application with back-end MySQL Server 5.5. The existing application uses Crystal Reports for reporting. I want to get rid of Crystal Reports and want to use another tool that seamlessly integrates with VS-2010 Express. I want to remove reporting options from my existing application and want to write a new WPF reporting application using VS-2010 Express. I want a free tool but as I am using MySQL, tools like SSRS are not useful for me.
What are the options? I am planning the other way round by writing a Python reporting application powered by any open-source Python reporting tool.
Answer: For WPF you can use http://wpfreports.codeplex.com/ for simple reports or follow this article instructions to make you own: http://janrep.blog.codeplant.net/post/WPF-Multipage-Reports-Part-I.aspx
I've used SSRS, but only the RDLC part (you can use it with any datasource), but not very happy with the results and WPF does not have a native RDLC ReportViewer.
Also check this related question What's the best approach to printing/reporting from WPF?
Answer: Why not have both? IronPython is a python implementation on top of .NET. It allows you to write code that is totally python, but gives you access to all of .NET, including WPF.
For a quick look at what this looks like, here is a basic WPF project using ironpython.
Comment: But which reporting tool to use? I don't want to use Crystal Reports from IronPython.

Python or WPF reporting application


Help evaluating this limit

Question:
Evaluate the limit $$\lim_{\alpha\to\omega}-\frac{\alpha r_0}{\omega(\omega^{2}-\alpha^{2})}\sin(\omega t)+\frac{r_0}{\omega^{2}-\alpha^{2}}\sin(\alpha t)=\frac{r_0}{2\omega^{2}}sin(\omega t)+\frac{r_0}{2\omega}t\sin(\omega t)$$

This limit came up in my introduction differential equations class when going over resonance. I just don't know how they were able to solve this limit. Thanks for any help.
Comment: I changed a minus sign in the front shouldn't be an equal sign does that help? That was a typo on my part sorry.
Comment: Are you sure you have written the formulas correctly? Maybe there should be some minus at some place in the left-hand side. And a cosine instead of a sine in the last term (depending on what you change of course).
Answer: The limit we want to calculate is

$$\lim_{\alpha\to\omega}-\frac{\alpha r_0}{\omega(\omega^{2}-\alpha^{2})}\sin(\omega t)+\frac{r_0}{\omega^{2}-\alpha^{2}}\sin(\alpha t)
$$

Here is a hint:
Do partial fraction decomposition on
$$
\frac{\alpha r_0}{\omega(\omega^2-\alpha^2)}\quad\text{and}\quad
\frac{r_0}{\omega^2-\alpha^2}.
$$
You will get, among other things, terms that looks like difference quotients for sine. If you know that
$$
\frac{d}{d\omega}\sin(\omega t)=t\cos(\omega t)
$$
then you are almost done. Can you take it from here?
Added
Let me add the correct formula. It is

$$\lim_{\alpha\to\omega}-\frac{\alpha r_0}{\omega(\omega^{2}-\alpha^{2})}\sin(\omega t)+\frac{r_0}{\omega^{2}-\alpha^{2}}\sin(\alpha t)
=\frac{r_0}{2\omega^2}\sin(\omega t)-\frac{r_0}{2\omega}t\cos(\omega t).
$$
Comment: Thanks will work on when home if you have the time could you show steps blacked out to reference later.

Help evaluating this limit


Undefined offset when using array_chunk

Question: I have the following array:
$people = array(
            array(  'name'=>'Sarah',
                    'gender'=>'F'),
            array(  'name'=>'Darren',
                    'gender'=>'M'),
            array(  'name'=>'John',
                    'gender'=>'M'),
            array(  'name'=>'Phil',
                    'gender'=>'M'),
            array(  'name'=>'Alice',
                    'gender'=>'F'),
            array(  'name'=>'Sam',
                    'gender'=>'M'),
            );

I would like to get it to display in a 2 column structure as follows:
Sarah | Darren
John | Phil
Alice | Sam

I am using array_chunk and looping through as follows:
foreach(array_chunk($people, 2, true) as $array)
{
    ?>

<?php echo $array[0]['name']; ?>

<?php echo $array[1]['name']; ?>

    <?php
}

The above does not work because it says: Undefined offset: 0
The value of print_r($array) is:
Array ( [0] => Array ( [name] => Sarah [gender] => F ) [1] => Array ( [name] => Darren [gender] => M ) )

Comment: That array is already chunked. Look at array_column if you have php 5.5
Answer: You passed true as the 3rd arg to array_chunk(), that preserves keys.
0 doesnt exist on the 2nd loop because the next available index would be 2.
Remove the 3rd argument from array_chunk and you should have what you need.
Answer: I achieved the output like this:
$count = 0;
foreach($people as $person) {
if($count % 2 == 0) {
    echo $person['name'] . ' | ';
} else {
    echo $person['name'] . '

';
}
$count++;
}

Undefined offset when using array_chunk


Probability of two exponential random variables take the same value

Question: Let $A$ and $B$ be independent random variables drawn from the exponential distribution with parameters $\lambda_A$  and  $\lambda_B$. What is the probability that $A=B$?
Comment: For independent continuous r.v.'s $A$ and $B$ P(A=B)=0$.
Comment: Could you post a link to the proof and make it an answer so I can accept it?
Comment: Welcome to MSE. Your question is phrased as an isolated problem, without any further information or context. This does not match [many users' quality standards](http://goo.gl/mLWc8), so it may attract downvotes, or closed. To prevent that, please [edit] the question. [This](http://goo.gl/PlJyVQ) will help you recognise and resolve the issues. Concretely: please provide context, and include your work and thoughts on the problem. These changes can help in formulating more appropriate answers.
Answer: If $X$ and $Y$ are independent continuous random variables, then
$$
\mathsf{P}(X=Y)=\mathsf{E}[\mathsf{P}(X=Y\mid Y)]=\int \mathsf{P}(X=y)f_Y(y)\, dy=\int 0 f_Y(y)\, dy=0,
$$
where $f_Y$ is the pdf of $Y$.


How to solve $\lim_{x\to1}=\frac{x^2+x-2}{1-\sqrt{x}}$?

Question: let $f(x)=\dfrac{x^2+x-2}{1-\sqrt{x}}$
How do I solve this limit?
$$\lim_{x\to1}f(x)$$
I can replace the function with its content
$$\lim_{x\to1}\dfrac{x^2+x-2}{1-\sqrt{x}}$$
Then rationalizing the denominator
$$\lim_{x\to1}\dfrac{x^2+x-2}{1-\sqrt{x}}\cdot\dfrac{1+\sqrt{x}}{1+\sqrt{x}}$$
With $(a + b)(a - b) = a^2 - b^2$, I can remove the irrational denominator.
$$\lim_{x\to1}\dfrac{(x^2+x-2)(1+\sqrt{x})}{1-x}$$
I the multiply the two parenthesis
$$\lim_{x\to1}\dfrac{\sqrt{x} \cdot x^2+\sqrt{x}\cdot x- \sqrt{x} \cdot 2 + x^2 + x - 2}{1-x}$$
I'm not sure where to continue to solve this limit.
Comment: Alternatively, try to factor $(x^2 + x - 2)$.
Comment: What about L'Hopital?
Comment: I'm self learning limits, I don't know about that yet. Is that the only way to solve this?
Answer: Observe that
$$ x^2 + x - 2 = (x-1)(x+2). $$
Therefore
\begin{align} \lim_{x\to 1}\frac{(x^2+x-2)(1+\sqrt{x})}{1-x}
&= \lim_{x\to 1} \frac{(x-1)(x+2)(1+\sqrt{x})}{1-x} \\
&= \lim_{x\to 1} \frac{-(1-x)(x+2)(1+\sqrt{x})}{1-x} \\
&= -\lim_{x\to 1} (x+2)(1+\sqrt{x}) \\
&= -(1+2)(1+\sqrt{1}) && \text{($\ast$)}\\
&= -6. \end{align}
At step ($\ast$), we are using the fact that both factors are continuous at $x=1$ and that the product of continuous functions is continuous.
Answer: Alternatively you can alwasy factor the hard way:
$\frac {x^2 + x - 2}{-\sqrt x + 1}=$
$\frac {x^{\frac 32}(\sqrt{x} - 1) + x^{\frac 32} + x - 2}{-\sqrt x + 1}=$
$-x^{\frac 32} + \frac {x(\sqrt x-1) + 2x - 2}{-\sqrt x+ 1}=$
$-x^{\frac 32} -x + \frac {2\sqrt x(\sqrt x -1) +2\sqrt x - 2}{-\sqrt x +1}=$
$-x^{\frac 32} - x - 2\sqrt{x}-2$
So $\lim\limits_{x\to 1} \frac {x^2 + x - 2}{-\sqrt x + 1}=\lim\limits_{x\to 1}(-x^{\frac 32} - x - 2\sqrt{x}-2) = -(1^{\frac 32}) - 1 - 2\sqrt 1 - 2= -1-1-2-2 = -6$.
Which is kind of difficult and hard and one ought to have seen and made use of $x^2 +x -2= (x+2)(x-1)$ and somehow noticed that $(x -1) = (\sqrt x+1)(\sqrt x -1)$ to make it easier.
But sometimes we don't see things and it's nice to know that brute force can work when we don't.
........
Or if we just let $y = \sqrt x$ then $\lim_{x\to 1} \frac {x^2 + x - 2}{1-\sqrt x }= \lim\limits_{y^2 \to 1;y > 0}\frac {y^4 + y^2 - 2}{1-y}$ and psychologically that seems easier.  We just factor $y^4 + y^2 -2 = (y^2 + 2)(y^2 - 1) = (y^2+2)(y+1)(y-1)$ and the $y-1$s cancel out.
Answer: $$\lim_{x\to1}\dfrac{(x^2+x-2)(1+\sqrt{x})}{1-x}=\\
\lim_{x\to1}(1+\sqrt{x})\lim_{x\to1}\dfrac{(x^2+x-2)}{1-x}\\
=2\lim_{x\to1}\frac{(x+2)(x-1)}{1-x}=-6
$$
Answer: $$\lim_{x\to 1}\frac{x^2+x-2}{1-\sqrt{x}}=\lim_{y\to 1}\frac{y^4+y^2-2}{1-y}=-\lim_{y\to 1}\frac{y^4+y^2-2}{y-1}$$
And that last limit is, by definition, the derivative $\frac{d}{dy}(y^4+y^2)=4y^3+2y$ at $y=1.$
So the result is $-6.$
Or you can just factor:
$$\frac{y^4+y^2-2}{y-1}=\frac{y^4-1}{y-1}+\frac{y^2-1}{y-1}=1+y+y^2+y^3 + 1+y.$$


Inequality of vector norms with projections

Question: Let $W$ be a vector subspace of $V$, a space with a dot product; $v\in V$. Let $p_W(v)$ be the orthogonal projection of $v$ onto $W$ and $w\in W, w\neq p_W(v)$.
How can i prove that $||v-w|| > ||v-p_W(v)||$?
Comment: My bad, I meant $w$
Comment: Pray tell, what is $w'$?
Comment: Thanks for the correction!
Answer: Let $v = v_\| + v_\bot $, where $v_\| = p_W(v)$, $v_\bot \in W^\bot$.
$ \|v-w\|^2 = (v-w)^2 = v^2 + w^2 - 2vw = v_\bot^2 + v_\|^2 + w^2 - 2v_\|w = v_\bot^2 + (v_\|-w)^2$
$ \|v-v_\|\|^2 = v_\bot^2 < v_\bot^2 + (v_\|-w)^2 $, QED.
Answer: Since $W \subset V$ is a fixed subspace, I am going to drop the subscript "$W"$ from $P$; in this answer, $P = P_W$.
Recall that an orthogonal projection $P$ satisfies
$P^2 = P = P^T; \tag 1$
thus for any
$x, y \in V \tag 2$
we have
$\langle x, Py \rangle = \langle P^Tx, y \rangle = \langle Px, y \rangle; \tag 3$
now consider the three vectors $v$, $Pv$, and $w$;
$Pv, w \in W \Longrightarrow Pv - w \in W, \tag 4$
$\langle v - Pv, w \rangle = \langle v, w \rangle - \langle Pv, w \rangle = \langle v, w \rangle - \langle v, Pw \rangle = \langle v, w \rangle - \langle v, w \rangle = 0, \tag 5$
where we have used (3) and the fact that $Pw = w$ for $w \in W$; since this holds for every $w \in W$, we have established that
$v - Pv \in W^\bot; \tag 6$
we write
$v - w = (v - Pv) + (Pv - w), \tag 7$
and by virtue of (4) and (6)
$\langle v - Pv, Pv - w \rangle = 0, \tag 8$
whence
$\Vert v - w \Vert^2 = \langle v - w, v - w \rangle = \langle (v - Pv) + (Pv - w), (v - Pv) + (Pv - w) \rangle$
$= \langle v - Pv, v - Pv \rangle -2\langle v - Pv, Pv - w \rangle + \langle Pv - w, Pv - w \rangle$
$= \Vert v - Pv \Vert^2 + \Vert Pv - w \Vert^2;\tag 9$
since each term on the right is non-negative, we find that
$\Vert v - w \Vert^2 \ge \Vert v - Pv \Vert^2, \tag{10}$
whence
$\Vert v - w \Vert \ge \Vert v - Pv \Vert, \; \forall w \in W, \tag{11}$
with equality holding precisely when
$\Vert w - Pv \Vert = 0 \Longleftrightarrow w = Pv; \tag{12}$
thus
$w \ne Pv \Longrightarrow \Vert v - w \Vert > \Vert v - Pv \Vert. \tag{13}$
$OE\Delta$.
Inequality of vector norms with projections


Add a project to a plugin

Question: I have a ordinary java project (P1). The project uses another java project (P2) which is added to the build path of P1. I create this dependency as I need to change both projects. I want if I change P2 then see changes in P1.
I recently convert my project as a plugin. However, when I run the program it can not find the second project, and shows NoClassDefFoundError compiler error while from class path file I can see the project is added in class path.
<code>classpathentry combineaccessrules="false" kind="src" path="/P2"/
</code>
I know how to create a jar file from P2 and add that to the plugin (it can solve my problem), but I do not know how to add another eclipse project (as source code) to a plugin. As I said I want to change P2 as well and it is difficult to create a jar file from P2 after each change and add that to the plugin.
Please let me know how can I add a source project to a plugin.
Is it possible to do that? I also read this page, but could not help me.
Comment: Did you try Maven or any other dependency mgmt tool?
Comment: I am not familiar with Maven, but if it can help please let me know then I can try that
Answer: You can try converting the second project to a plugin project and add it in the manifest of the first(Dependencies>Required Plugins).

Add a project to a plugin


Divisibility of numbers between $n^3$ and $n^3+n$

Question: Let $n$ be a positive integer. Given are numbers $n^3,n^3+1,\ldots,n^3+n$. Of them, $a$ are colored red, and $b$ others blue. The sum of the red numbers divides the sum of the blue numbers. Prove that $a$ divides $ b$.
Some small cases. For $n=2$, we have $8,9,10$. the only way is to color $9$ red and $8,10$ blue. For $n=3$ we have $27,28,29,30$. The only ways are to color $28$ red and $27,29$ blue, or $29$ red and $28,30$ blue. In all cases, $a$ divides $b$.
My intuition is that the numbers are in a small enough range that the average of the red numbers must be the same as the blue numbers, which implies that $a$ divides $b$.
Comment: Where did you come across this? This is a nice problem. What have you tried with it so far?
Comment: @StevenStadnicki I got it from a math contest training. I've added some small cases and my intuition.
Answer: You're on the right general track with your intuition, but proving that the averages are the same is going to be tricky in its own right.  Here's an outline of a related but different approach to the problem:
Note that the sum of the numbers from $1$ to $n$ is approximately $\frac{n^2}{2}$; in particular, it's less than $n^2$ for all $n\gt 1$.  This means that the sum of the red numbers — call it $\sum A$ — is between $an^3$ and $an^3+n^2$, and in particular that it's bracketed between $an^3$ and $(a+1)n^3$.  Likewise, the sum of the blue numbers ($\sum B$) is between $bn^3$ and $bn^3+n^2$.  Now, what do the multiples of $\sum B$ look like?  You know that the ratio of $\sum B$ to $\sum A$ has to be less than $n$ (why?), so you should be able to bracket each of the multiples of $\sum B$, $m\cdot\sum B$, between $mbn^3$ and $(mb+1)n^3$ (why?).  Now, do you see why this implies your result?


Illegal choice error in views exposed filter select list

Question: My question is related to Drupal 7 Views Exposed Filter - How to Select all Taxonomy terms Upon adding new terms? however I have a different problem. I want to filter a view by users of specific roles and for that i have added the user field in views filter and exposed it.
The problem is that if i check "select all" in "options" and leave "Limit list to selected items" unchecked, the select shows "--Any--" as default selection and also includes any users added later but throws "An illegal choice has been detected. Please contact the site administrator" and does not load any views.

On the other hand, if i check "Limit list to selected items" in options it shows no error but then it will only load the selected users in options and not any user added later. The field "check_user" has a field collection relationship. How to solve this? Is this a bug or misconfiguration?
Using views 3.8 and Drupal 7.32
[EDIT]
I managed to remove the error by deselecting the "select all" option and checking all the users, this way the select list filter include the checked users in exposed filter and also any user added later. The select filter also provided "--Any--" option but when I first load the view, the first user in the list is automatically select instead of "--Any--". How to default the select filter to "--Any--"?
Comment: Is it a user reference field?
Comment: Yes, included in field collection.
Answer: Don't choose select all option (it is a simple javascript effect to choose the values) or don't select any user from list. If you just choose expose the filter option it will display all the users list automatically.
Comment: Actually by not checking any option (neither select all nor any users/items), the view is perfectly loading with providing "--Any--" option at the first load. Thanks again for the hint.
Answer: That error is documented in the following thread https://www.drupal.org/node/1177882#comment-8891105. This solution worked for me:
$form['your-views-exposed-filter-field-name-here']['#validated'] = true;


High quality "self promotion" allowable?

Question: Today I am seeing a lot of Late Answers coming in for review, all from the same user and referencing articles on https://www.mageplaza.com/
Usually those answers are very short and tell me to read more on the user's own blog. Normally I will flag these as "self promotion", but this time the answers are actually quite good, so I am marking these as "Looks OK".
Also I leave the reference (URL) in there as that is the original place of the content and could also help others with similar issues in their search.
Anyone care to share their opinion on this?
Answer: Links to personal blogs have a high chance to 404 in the future.
So, in my opinion, every answer should be read as if the links are already 404. Does the answer still answers the question?
This means that the link should be treated only as an extension to the answer, maybe an evironment to go deeper into the issue.
If the answer, by itself, without checking the link, is not an answer to your question, it just provides some quotes and blocks of text with missing info, then it is a bad answer and downvotable.
Of course, one should differentiate between links to personal blogs and links to the magento2 github repo.
Answer: Quote from: https://magento.stackexchange.com/help/how-to-answer

Provide context for links
Links to external resources are encouraged, but please add context around the link so your fellow users will have some idea what it is and why it's there. Always quote the most relevant part of an important link, in case the target site is unreachable or goes permanently offline.

Conclusion.
Single link answers are not the best answer.
Write a comment below one of the answers explaining why the OP should add more content to the answer.
If nothing happens by tomorrow, start flagging answers.
[EDIT]
If the answer is high quality, has context and not just a link and most probably solves the problem I see no problem with letting it be.
Although, if the answers keep recommending paid extensions over and over again I might need to summon the other mods.
As a general practice: Flag everything that looks fishy to you.
Comment: ok. I've edited my answer.
Comment: These are not single link answers. The discussion is about long and high quality answers which reference the source at the bottom. The source is an extension provider's blog and the rate at which they are posting this today, makes it a bit doubtful...
Answer: If the answers are still valuable without the link and just include it for further reference or as source, it's perfectly fine IMHO. If not, at least relevant parts of the source should be quoted.
(I've done that myself so who am I to judge)


How to pass variable to a views template?

Question: I have this in my template.php:
function mytheme_preprocess_views_view(&$vars) {
$vars['test'] = 'Filtertest';
}

And this in my views-view-responsive-grid--taxonomy-term.tpl.php:
<?php print $test; ?>

... this does not work though. The template is called and works but my $test variable does not get printed.
What am I missing here? Any help is greatly appreciated.
Comment: it's better to check first on view name like `if($vars['view']->name == 'your_view_name') { $vars['test'] = 'Filtertest';  }` , and make sure that  `views-view-responsive-grid--taxonomy-term.tpl.php` the right tpl for this view, and flush all caches
Comment: Yes, you are right, I have changed that now, The template is called and works, so i suppose it should be possible to pass variables to it?
Comment: To variable to view tpl `$vars['test'] = 'Filtertest';` should works, just be sure that you in the right tpl .
Comment: berramou: I finally got it to work by changing the template filename to a shorter name and using the exact same name in the preprocess function. See my answer below. Thank's for helping.
Comment: great, you are welcome.
Answer: Replace this, with your function
public static function mytheme_preprocess_views_view(&$vars) {
  $vars['test'] = 'Filtertest';
  return $vars['test'] ;
}

Then use this short code on your twig view:
{{ test }}

or if wanna use it in something.php in your views:
dump(test )

Answer: In case anyone else have the same problem. Changing the name of the template file to views-view-responsive-grid.tpl.php and then using that same name in the preprocess function in template.php, like this:
function mytheme_preprocess_views_view_responsive_grid(&$vars) {
  if($vars['view']->name == 'taxonomy_term') {
    $vars['test'] = 'Filtertest';
  }
}

... fixed the problem. Now the $test variable is printed. Apparently the template's filename can't be too specific.


Is there a way to clear the table (listObject) content and resizing it one column?

Question: I wish to reset a table (listObject) via VBA. I tried it in two ways:
Idea 1: Clear table content and resize the table to one column
 With ActiveWorkbook.Worksheets("Sheet1").ListObjects("tblExample")
        If Not .DataBodyRange Is Nothing Then .DataBodyRange.ClearContents
        Dim numColumns As Long: numColumns = .ListColumns.Count
        If numColumns > 1 Then
               .DataBodyRange.Offset(1).Resize(, .Columns.Count - 1).Delete 'Last statement
        End If
End With

The problem: When evaluating the last statement I get the runtime '438' error stating something like: 'Object does not support this Property or Method.'
Idea 2: Delete the table and make a new one
With ActiveWorkbook.Worksheets("Sheet1")
If .ListObjects.Count = 1 Then
           .ListObjects(1).Delete
           .ListObjects.Add(Destination:=.Range("I1:I105")).Name = "tblExample"
End If
End With

The problem: The new table is placed at a random location (= selected cell).
Answer: With ActiveWorkbook.Worksheets("Sheet1").ListObjects("tblExample")
    .Resize .Range.Resize(ColumnSize:=1)  ' resize entire table to only first column
End With

ListObjects("tblExample").Resize expects the new range of the table.
ListObjects("tblExample").Range gives the old range of the table and .Resize(ColumnSize:=1) resizes the old range to only one column.

Are you sure you mean ActiveWorkbook?

ActiveWorkbook the workbook that is on top (has focus).
can be any workbook the user clicks on while the macro is running.
ThisWorkbook the workbook the VBA code is written in.
is a defined workbook that never changes.

In most cases you should avoid ActiveWorkbook as it can change by a single mouse click and use ThisWorkbook.
Comment: Great! I wasn't aware of the subtelty `ThisWorkbook` vs `ActiveWorkbook` yet... I'm gonna fix this throughout my Workbook...


Dealing with 0 in cell count for Fisher's exact test

Question: <code>    Control Cases
  A     0       2
  B  1000     298
</code>
I am dealing with 0 in my 2x2 tables, and I was wondering what are some of my options when it comes to adjusting for these 0s. In class I did learn that it is not unheard of to just add 1 to the 0 counts, however, since the counts in category A are so low, I don't know how doing so would affect the analysis. I have read a similar post, and part of the answer involves thinking more about the experiment or obtaining more samples. If I cannot obtain more samples at this moment, what are some possible methods that I can use to adjust for the 0 counts?
Answer: Fisher's exact test does deal with zero cells without any problem.
Adding 0.5 to all cells is a commonly done to remove improve the small sample properties from the Chi-square test (and it asymptotically removes the first-order bias from the estimate of the log-odds ratio). This is (for a simple 2x2 table) equivalent to maximum-a-posteriori estimation using the Jefferys prior (=a Beta(0.5,0.5) prior for each proportion), as well as Firth's penalized likelihood logistic regression (for stratified tables these two approaches no longer match to adding 0.5 to each cell). Doing so penalizes the effect size (and test decision) towards no effect.
If you have prior information, a Bayesian approach with informative priors may be another option.
Comment: Good answer.  Just would like to add Fisher's exact test is pretty conservative, so would suggest Barnard's exact test for 2x2 tables.  Assuming only the column margins are fixed, Barnard's exact test is 0.01 compared with 0.053 for Fisher's exact test.  A Bayesian approach may also yield less conservative tests.


Permissão de arquivos GIT

Question: Todos os meus sistemas que tenho estão dando problema de permissão após eu dar um git pull, de forma que por exemplo eu tenho uma pasta onde são gerados os meus arquivos de js e css mimificado. Pasta assets/min/, após eu dar um pull eu preciso sempre de rodar chmod -R 777 assets/min para que eu possa alterar esses arquivos min, gera-los novamente no caso, qual seria a solução nesse caso para que eu não precise dar permissão toda vez que fizer um pull?
Obs.: Meu sistema operacional é macOS High Sierra
Comment: Você pode alterar  o umask ... Segue URL sobre http://www.itnerante.com.br/group/colunadoboechat/forum/topics/alterando-permiss-es-padr-es-no-linux-com-umask
Comment: Relacionada: https://pt.stackoverflow.com/questions/131475/como-desabilitar-modo-permiss%c3%a3o-do-arquivo-no-git
Answer: Tive esse problema há um tempo atrás. Resolvi da seguinte maneira:

Abra a pasta do seu repositório local com o Finder. Clique na pasta e pressione  ⌘I. Isso deve abrir as informações da pasta. Se aparecer um cadeado na janela, clique e digite a senha do seu Mac para dar permissões de escrita.
No final dessa janela, procure por "Sharing & Permissions". Lá tem uma tabela de permissões e usuários. No seu usuário, onde indica "(Me)", selecione "Read and Write".
Clique no ícone de configuração (engrenagem) e selecione "Apply to enclosed items".

Isso vai colocar permissões de escrita e leitura para todas as pastas, subpastas e arquivos no seu usuário para a pasta do seu repositório local. Se tiver dificuldades, usa esse print como referência.
Se por ventura você compartilha essa pasta com alguém na rede, é um pouco mais chato de resolver. Aí você pode tentar adicionar o seguinte no seu .gitconfig:
[core]
  repositoryformatversion = 0
  filemode = true
  bare = false
  logallrefupdates = true
  sharedRepository = group

Tem um blog mais detalhado sobre aqui.

Permissão de arquivos GIT


Questions about ObjectPooling

Question: I am currently working on a simple UDP server for a game. I have already implemented multithreading with the help of a ThreadPoolExecutor. When reading about how to maximize performance I stumbled across ObjectPooling. After some reading I have been left with 3 questions:
1) Will a server program be able to take advantage of an ObjectPool?
2) Which "Objects" should be pooled and which should not?
3) How to create a simple ObjectPool?
Answer:
Will a server program be able to take advantage of an ObjectPool?

This doesn't depend on the type of application. The usefulness of object pooling depends on the use-case.

Which "Objects" should be pooled and which should not?

Usually, objects which are expensive (in memory or CPU terms) to create should be pooled (still opinable).
Avoid pooling any other type, as pooling can get quite expensive just by itself.

How to create a simple ObjectPool?

Don't. It's a complex subject.
Instead, use something such as Apache Commons Pool.

And I'll add my own

Should I care about object pooling?

Not until you benchmarked performance without using it.
Pooling is a risky pattern, especially in multi-threaded environments.
The pool implementation should provide the required synchronization, but still, adding your own is not bad either, if that object maintains a state.
Answer: Here are answers to your questions,

1) Will a server program be able to take advantage of an ObjectPool?

Yes, any object can be pooled. Usually objectpools are used if object creation is heavy operation.

2) Which "Objects" should be pooled and which should not?

Any object can be pooled.

3) How to create a simple ObjectPool?

You can implement simple pool using apache commons pool. For more information, refer https://commons.apache.org/proper/commons-pool/examples.html
EDIT: FYI, we implemented objectpool using apache commons pool for connecting to CORBA clients. Connecting to CORBA and creating object was taking more time so we implemented CORBA client pool.
Comment: anyobject can be pooled, but as long as your object is being cached somewhere in your code, it does not make sense, unless your object is immutable.
Answer:
1) Will a server program be able to take advantage of an ObjectPool?

irrespective of type of application, it depends upon use case, specially when performance of your program is memory and CPU sensitive.

2) Which "Objects" should be pooled and which should not?

IMO, what scenarios make a pool for certain class objects useful:

when system needs only fixed or limited type (states) of objects at a time
when you need to impose an upper limit on maximum objects of certain class
when object creation is memory and / or cpu intensive and objects can be reused as is, or with minimal number of operations
when object itself is memory intensive and / or used in high frequency

in addition to above criteria, object must not be cached OR must be immutable

3) How to create a simple ObjectPool?

Its a complex topic altogether, but in essence you need to build an object creator (sort of Factory) which should decide whether to create new objects or return existing with needed change in their states.

Questions about ObjectPooling


How to Pass a String or Variable with HTTP Redirect in Laravel

Question: I'm developing a quiz and want to check if a question is answered already. Then I'd like to print something and/or disable the form after submitting. The page will be redirected back to show the new option if the user has any observation or reservation. If the form redirects back how can I check or disable the form so a user can't submit the form again? I am using the following method to redirect back.
<?php

return redirect()->back();

If I pass a variable using the with() method I get no success. The system doesn't recognize my variable. I also tried...
<?php

return redirect()->back()->withDisable('disable');

Answer: you can use flash data in laravel for this kind of task.
What is Flash Data?
Flash data helps you to store items in the session only for the next request.Data stored in the session using this method will only be available during the subsequent HTTP request, and then will be deleted.
$request->session()->flash('disabled', 'true');

However, If you need to keep your flash data around for several requests
$request->session()->keep(['disabled', 'true']);

to access your session/flash data you can simply do
$request->session()->all() //all flash session data

$request->session()->get('disabled') // for specific

hope that helps
Answer: Just with should do it.  For example:
return redirect()->back()->with('status', 'disable');

This is saved in the session data, so you could check for it with:
@if (session('status'))
...

From: https://laravel.com/docs/5.5/redirects#redirecting-with-flashed-session-data
Comment: thanks, i resolved my issue with you help but i cant accept your answer because stackflow allow me within 2 mins to accept this question.


Reverse causality

Question: I have 4 time series, $y, x_1, x_2$ and $x_3$, and I'm running the following reduced form VAR:
$ \Delta y_{t} = A_1 \,x_{1,\,t-1} + A_2 \,x_{2,\,t-1} + A_3 \,x_{3,\,t-1} + \varepsilon_t$
Is there any chance the regression estimates may suffer from reverse causality?
My guess would be no, since it's unlikely that the future can affect the past, right?
Comment: Staying outside the realm of exotic physics, it's safe to assume that the future doesn't cause the past. But, common causes and spurious correlations are still possible, of course. So, non-zero coefficients wouldn't imply that $x_{t}$ causes $y_{t+1}$.
Answer: If people have accurate expectations about the future, then that can certainly happen.
For example, if I know that I am going to go on vacation and eat out a lot at time $t$ (so $y_t$ is restaurant expenditure, so $(y_t - y_{t-1})>0$), my expenditures on wine, meat, and cheese at $t-1$ may reflect that ($x_{1,t-1},x_{2,t-1},x_{3,t-1}$ would be lower). That does not mean that lowered grocery expenditures cause the restaurant expenditure.
Comment: What if x_1, x_2 and x_3 are the volumes at time t-1 of a particular stock that is traded in three different exchanges, and y is the price of that stock in one of the three exchanges? Can the A coefficients be regarded as price impacts? Is this model any similar to the Kyle (1985) model?
Comment: I am not a finance person, so I can't really say, and I have not even glanced at this paper. I expect all the coefficients to be one, so that would not be a a very interesting impact to interpret.

Reverse causality


Mount shared folder (vbox) as another user

Question: I'm trying to mount my vbox shared folder every time my ubuntu starts.
So, I added an entry on /etc/init with this:
description     "mount vboxsf Desktop"

start on startup

task
exec mount -t vboxsf Desktop /var/www/shared

Seems to work, except by the fact that all the files are owned by "root", and I don't have permission to write on the folder (neither chmod nor chown seems to be working).
So, how can I make all the files under this shared folder to be owned by www-data user/group?
Thanks

ps.: The main reason for me to have an automatic shared folder, is so I can create/edit files from the HOST on the GUEST www folder.
If you have a better idea for that, instead of sharing the folder, fell free to say.
Answer: [Same answer as in: StackOverflow]
Well, while I was having another issue related with my shared folder, I ended up getting to this stackoverflow question: https://stackoverflow.com/questions/6298933/shared-folder-in-virtualbox-for-apache
It helped me in 2 ways, and seems that what I need are those uid and gid options.
So, to mount a shared folder as another user, I would run:
mount -t vboxsf SHARE_NAME /some/dir -o uid=48,gid=48

Also, to see what are your www-data's gid and uid, just run id www-data.
If you also need to change the permissions on the mounted files, just add "dmode" to the options, as:
sudo mount -t vboxsf SHARE_NAME -o rw,dmode=777,gid=GROUP_ID,uid=USER_ID /path/on/guest

The available options are (from mount's help):
rw         mount read write (default)
ro         mount read only
uid       =<arg> default file owner user id
gid       =<arg> default file owner group id
ttl       =<arg> time to live for dentry
iocharset =<arg> i/o charset (default utf8)
convertcp =<arg> convert share name from given charset to utf8
dmode     =<arg> mode of all directories
fmode     =<arg> mode of all regular files
umask     =<arg> umask of directories and regular files
dmask     =<arg> umask of directories
fmask     =<arg> umask of regular files

And if you need it to run during the system init, just create a file on /etc/init/SOMETHING.conf, with something like this:
description     "SOME DESCRIPTION"

start on startup

task
exec mount -t vboxsf SHARE_NAME /path/on/guest -o uid=1000,gid=33

Comment: Goodness, I have been looking for well over an hour how to get the owner to anything else than `root` and I was going crazy: you saved my sanity!
Comment: excellent, excellent answer. I did not realise the idea is to mount it as another user. I answered my own question http://superuser.com/questions/527963/how-to-allow-nginx-in-guest-vm-access-shared-folders-in-host-os/527991 with your help. Thank you!
Comment: Thanks for pointing that out, @MartinCleaver. I've edited the answer.
Comment: I'm glad that it helped (If you could, just up vote my answer please) :)
Comment: Thanks! And of course SHARE_NAME-o is actually SHARE_NAME -o
Answer: If you want to share your files between your host and your guest, you can easily run a Web Server instead of hard sharing of a folder.
Please note that host and client is Ubuntu 14.04 in my commands:
In the host:
mkdir /home/SomeDirectory
cd /home/SomeDirectory
python -m SimpleHTTPServer

Supposing host ip (10.1.0.110), now in the guest, open the directory  using the followings:
xdg-open http://10.1.0.110:8000

Note that your port above may be different than 8000.
xdg-open is in xdg-utils package. If you don't install it before, run this in the guest:
sudo apt-get install xdg-utils


Is there any problem which can be solved only by fuzzy theory but not by statistics?

Question: When statistics is solving all the problems, why does one study the fuzzy theory? Is there any problem which can be solved only by fuzzy theory but not by statistics?  Please kindly answer with examples.
Comment: Where did you learn that statistics sole all the problems ??
Comment: Statistics show that not all problems can be solved using statistics.
Comment: Come on, it is like asking why there are chefs if all they do is just a mixture of physics and chemistry.
Answer: Instead of hearing it from me, why don't you read about it from the man[1] himself. In this paper, as the title suggests, the author points out some of the shortcomings of the probability theory and suggests the Fuzzy Theory can complement probability theory in overcoming those. In fact, he goes further and publishes [2]. You can go through these and see if his arguments are convincing enough for you. If they are not, then there is nothing I can say that is going to convince you anyways :-).
[1] Discussion: Probability Theory and Fuzzy Logic Are Complementary Rather Than Competitive By Lotfi Zadeh
[2] Toward a perception-based theory of probabilistic reasoning
with imprecise probabilities By Lotfi Zadeh
=======================================================================
Personal Opinion (written on request of the OP, others may ignore this content):
A couple of years back, I was working on an Extraction Management System [3]. This system was to work along with a plethora of Information Extraction operators and extracts several entity relationships from the web. Now, for example, let us say, an operator extracts the unary relationship "Flat(Earth)". There are two things we need here : a] Correctness of extraction [What is the probability that the extraction has an error?] b] Correctness of the extracted relationship [What is the probability that the statement made in the extraction is incorrect?]. To clarify, let us say the webpage from which the relationship was extracted actually said, " ... made the trip to earth. Flat model was used ...". Clearly, the webpage is not saying, "earth is flat", so the extraction made an error (type a above). On the other hand, if the webpage actually said, "earth is flat", then the statement being made is incorrect (type b above). You can easily see cases where there is an correct extraction with an incorrect statement and incorrect extraction with an incorrect statement (an incorrect extraction with a correct statement is somewhat unlikely I presume). Downstream, we wanted to combine these two probabilities in a meaningful way and we did come up with a bayesian model to do it (along the lines of [4]), however, what I would like to point out here is that we did look at other possibilities at the time (like the imprecise probability stuff in [2]). Now, I would like to have given you numbers and so on and conclude by saying that systemA did better than systemB. But, I can't because I never got to doing those experiments :-). However, the takeaway here is that problems like these do exist in the real world applications and many different approaches to handling such cases in a generalized way is always nice. So, I welcome fuzzy models as much equally as Bayesian models.
[3] Purple SOX Extraction Management System
[4] Truth Discovery with Multiple Conflicting Information
Providers on the Web
Comment: Raman! Sir, fine. If possible please tell me your own opinion.
Comment: @KRRG-BITS I have updated the answer with my personal opinion, although I doubt I have done any justice whatsoever and not sure how much of help it would be to you.
Answer: There is a theory which seems to combine statistics and fuzzy logic. See the book Statistical methods for non-precise data by Reinhard Viertl (see also this pdf). Unfortunately I can't say more about this. I took a quick look at this book but it does not give some examples of applications (except if I missed them). I would be interested if somebody could tell more about Viertl's works.
Comment: I have already tell my opinion: I can't say more about this...
Comment: @Sir, please tell me your own opinion and thank you for providing the file
Answer: I think fuzzy logic and statistics are very different things.  Fuzzy logic has to do with set membership where an element may belong to a set with a probability less than 1 associated with it.  Probability uses measure theory and ordinary set theory to develop its theory which is the basis for statistical inference.
Comment: I thought I had heard that the set membership values used in Fuzzy Set theory weren't probabilities (At least, that's what I remember of Zadeh's statement regarding it).
Comment: Maybe probability is the wrong term to use to characterize the membership properties.  The point is that membership inclusion is not known with certainty for at least some elements that could belong to the set.  That would not affect the point I am making about the methods not being compatible in the sense that one approach could be substituted for the other.
Comment: You most likely have more experience with FST than I do, so that's fair enough.
Answer: For years I had no idea what people were talking about when they dropped the term "fuzzy logic". No one taught me about it; it was completely skipped in all my mathematics and artificial intelligence courses. I believe this is because it was part of a wave of attempts to recategorize pieces of probability theory, most of which no one today has heard of because academics realized they were kind of silly and stopped passing them on.
For some of that context, I recommend this vibrantly, hilariously scathing article from the 80s in which the author calls attention to a proliferation of (then) new theories for reasoning with uncertainty, calls them all balderdash, and shows why you can just use ordinary probability theory instead. (He actually calls out Zadeh specifically.)
At the end he states Probability is the proper generalization of Logic, not "fuzzy logic", and I tend to agree. So to answer your question, Probability (Bayes Theorem, marginals, conditionals, etc.) isn't exactly the same thing as Statistics (populations, distributions, etc.), so, yes, there are questions Probability theory answers that Statistics doesn't, like "70% of the people on the bus are men. 50% of men have bags. 80% of women have bags. If I choose a person at random, what is the probability they have a bag?"


How can I stay informed of new art music?

Question: As a composer, it's vitally important that I'm aware of what's happening in the contemporary music world. I find it very difficult to keep myself informed of a field that often flies below the radar of popular culture.
Are there any resources for keeping up-to-date on new art music, especially those that are validated by the academic community?
Comment: I reopened this question given its edits.  Please consider the question with a fresh mind. (:
Comment: @NReilingh: Your edit is totally in line with my original intent. I think your wording might help make the question less controversial.
Comment: Introducing subjectivity in that way will cause arguements, which is why your question would be considered [bad subjective](http://blog.stackoverflow.com/2010/09/good-subjective-bad-subjective/).  See [this meta answer](http://meta.music.stackexchange.com/questions/52/the-innovative-contemporary-music-question-is-not-subjective-and-argumentative/54#54) for more detail.
Comment: This looks like a niche question, and I would suggest that it be reopened as seen in [this question from Jeff](http://meta.superuser.com/questions/2324/in-defense-of-obscure-niche-hardware-recommendation-questions/2339#2339).
Answer: Sequenza 21 is probably the best for keeping current with living composers - it's run by a whole bunch of contemporary composers, and it rarely discusses any composers or musicians who are not currently making music today.
http://www.sequenza21.com/
On the right side bar is an entire list of contemporary classical music blogs.  I believe Nico Muhly and Kyle Gann (PostClassic) have the most widely-read contemporary classical blogs, but they are a bit more specialized in focus.
Answer: i don't know of a single "reliable source for this sort of list" but here are some resources that could help.

nonesuch records hosts nonesuchradio, which showcases their 20th-century-classical releases.
alex ross, the music critic for the new yorker magazine and the author of the rest is noise frequently covers 20th-century-classical eventss for the magazine. (i recommend his book, too, if you haven't read it.
wikipedia has a list of 21st-century composers who have made a significant impact on the world of classical music since 2001, which may not clear anything up for you but certainly is an interesting browse.
Answer: Find some musical blogs.  Here is one that is primarily about Boston-area live performances, but you can always find opinions on many composers: http://classical-scene.com/.  This site is very strongly art for art's sake rather than about the business of music.
I don't know if Musical America is still around.  The web site (www.musicalamerica.com) is not responding to me.  If it is, this is the place to look for news on the business of classical music, i.e. whose music is getting performed where.
I have a couple of Facebook friends who are always posting links to contemporary composers, famous and obscure.  Start your own Facebook page and post your own links to videos and recordings that you like, and before long you'll be noticed, and others who do the same will "friend" you.
The place to find classical-style composers whose music is heard and loved, though not necessarily innovating, and who are highly-regarded in the sense that they can always be counted on to deliver the goods, is in the video-game and movie industries -- not just here but also in Europe, Japan, China and India.
Answer: As was already mentioned I don't think there is a single reliable source for this information, but from my Google Reader here are my 2 cents:

NPR Music's Classical RSS Feed
William C. White's Blog
LA Times Culture Monster, Not just classical music, but very interesting
Tom Service on Classical from the UK's The Guardian
Alex Ross: The Rest Is Noise, The New Yorker's music critic
Unquiet Thoughts, Alex Ross' column on The New Yorker's website
Greg Sandow's Blog

Hope it helps at least as a starting point. Check out the content and see if you might be interested in some of what they have to say. Sometimes you find other blogs or sources when the ones you read link to them.


Struggling parse key value pairs into array with space in key names

Question: I'm attempting to parse part out a windows event message into key/value pairs but I am struggling to figure out how to do this. It comes in from the WinCollect agent so I am not really able to do much to format the origiThe main issue is that the key names have spaces in them (I.E. SID  Account Name, Security ID, Logon ID) and I need to PascalCase them removing the space. Complicating matters, the value's often have a space after the equals and often times there is no value for the key. The messages often vary and I can't for the life me figure out a simple JavaScript function to tackle this. You'll notice after the LastKey Value pair (Logon Guid) it's just a giant message string I'd like to discard.
Here's the sample string:
Subject=  Security ID=   SID  Account Name=  -  Account Domain=  -  Logon ID=  0x0  Logon Information=  Logon Type=  3  Restricted Admin Mode= -  Virtual Account=  No  Elevated Token=  Yes  Impersonation Level=  Impersonation  New Logon=  Security ID=  MDITMonSvc  Account Name=  MonSvc  Account Domain=  MD3T.CE-IT.NET  Logon ID=  0xAE5D3D4  Linked Logon ID=  0x0  Network Account Name= -  Network Account Domain= -  Logon GUID=  {e2fe21c6-1533-f673-af2c-1e289f01739f}  Process Information=  Process ID=  0x0  Process Name=  -  Network Information=  Workstation Name= -  Source Network Address= 11.300.2.35  Source Port=  49357  Detailed Authentication Information=  Logon Process=  Kerberos  Authentication Package= Kerberos  Transited Services= -  Package Name (NTLM only)= -  Key Length=  0  This event is generated when a logon session is created. It is generated on the computer that was accessed.  The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.  The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).  The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.  The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.  The impersonation level field indicates the extent to which a process in the logon session can impersonate.  The authentication information fields provide detailed information about this specific logon request.  - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.  - Transited services indicate which intermediate services have participated in this logon request.  - Package name indicates which sub-protocol was used among the NTLM protocols.  - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

From the above string I'd expect to have the following values:
keyValueArr[0] = "Subject=";
keyValueArr[1] = "SecurityID=";
keyValueArr[2] = "SIDAccountName=-";
keyValueArr[3] = "AccountDomain=-";
keyValueArr[4] = "LogonID=0xAE5D3D4";
keyValueArr[5] = "LogonInformation=";
keyValueArr[6] = "LogonType=3";
keyValueArr[7] = "RestrictedAdminMode=-";
//etc

Can't figure out how to do this! Any ideas?
EDIT:
Pavel Savva provided a fantastic answer below for my original string:
const regex = /(?:\S+(?:\s\S+)*|\s{2,})=/g

const keys = input.match(regex)
const values = input.split(regex)

const keysAndValues = []

for (let i = 0; i < keys.length; i++) {
  const key = keys[i].replace(/\s+/g, '')
  let value = values[i + 1].replace(/\s+/g, ' ').trim()

  if (i === keys.length - 1) {
        value = value.split(/\s{2}/).shift()
  }

  const kv = [key, value].join('')

  keysAndValues.push(kv)
}

But sometimes the event comes in as this:
Dns query= RuleName= - UtcTime= 2023-03-14 17=29=56.244 ProcessGuid= {FBE8C53C-AF2D-6410-401C-000000004100} ProcessId= 3892 QueryName= kbrunette.chi.NET QueryStatus= 0 QueryResults= ==ffff=10.58.2.45; Image= C=Windowssystem32wbemwmiprvse.exe

Any ideas on how to make Pavel's solution work with this type of format as well?
Comment: Can the values of each field contain spaces or `-`? Parsing questions tend to be received poorly because you basically need to give us the entire spec of the input for an answer. (also, show what you've tried)
Comment: So what distinguishes between `Subject=;SecurityID=;SIDAccountName=-;` and `Subject=Security;ID=SID;AccountName=-;`?
Comment: Are Windows events not better structured? Why do you get them as a string?
Comment: @MisterJojo "*You'll notice after the LastKey Value pair (Logon Guid) it's just a giant message string I'd like to discard.*" - I'm pretty certain that refers to the unreadable part that your edit moved from data to text
Comment: @MisterJojo yup Bergi is right that last portion should be a part of the data string, not the post itself. Check the examples here: https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4624
Answer: For sure not the most elegant solution, but I hope it helps...
Looking around the Internet (and specifically here) it looks like the second and third values should be:
eyValueArr[1] = "SecurityID=SID";
keyValueArr[2] = "AccountName=-";

With that assumption in mind, (?:\S+(?:\s\S+)*|\s{2,})= (shamelessly stolen from here with a non-capturing group added to account for =) seems to work reasonably well to match all the keys:

We can then split on the regex and build our key-value pairs:
const regex = /(?:\S+(?:\s\S+)*|\s{2,})=/g

const keys = input.match(regex)
const values = input.split(regex)

const keysAndValues = []

for (let i = 0; i < keys.length; i++) {
  const key = keys[i].replace(/\s+/g, '')
  let value = values[i + 1].replace(/\s+/g, ' ').trim()

  if (i === keys.length - 1) {
        value = value.split(/\s{2}/).shift()
  }

  const kv = [key, value].join('')

  keysAndValues.push(kv)
}

Above I'm making the assumption that the last value ends with two spaces in a row.
And this results in:
"Subject="
"SecurityID=SID"
"AccountName=-"
"AccountDomain=-"
"LogonID=0x0"
"LogonInformation="
"LogonType=3"
"RestrictedAdminMode=-"
"VirtualAccount=No"
"ElevatedToken=Yes"
"ImpersonationLevel=Impersonation"
"NewLogon="
"SecurityID=MDITMonSvc"
"AccountName=MonSvc"
"AccountDomain=MD3T.CE-IT.NET"
"LogonID=0xAE5D3D4"
"LinkedLogonID=0x0"
"NetworkAccountName=-"
"NetworkAccountDomain=-"
"LogonGUID={e2fe21c6-1533-f673-af2c-1e289f01739f}"
"ProcessInformation="
"ProcessID=0x0"
"ProcessName=-"
"NetworkInformation="
"WorkstationName=-"
"SourceNetworkAddress=11.300.2.35"
"SourcePort=49357"
"DetailedAuthenticationInformation="
"LogonProcess=Kerberos"
"AuthenticationPackage=Kerberos"
"TransitedServices=-"
"PackageName(NTLMonly)=-"
"KeyLength=0"

Comment: This works for the string in my post but the problem is it doesn't seem to work when the text is in another format: "Dns query= RuleName= - UtcTime= 2023-03-14 17=29=56.244 ProcessGuid= {FBA8C56C-AF1D-6410-401C-000000004100} ProcessId= 3892 QueryName= MD2-DC-P3.Churchill.CSG-IT.NET QueryStatus= 0 QueryResults= ==ffff=10.58.2.45; Image= C=Windowssystem32wbemwmiprvse.exe".
Comment: @Kevin I see, I'd say this one would be a slightly different regex since it looks like keys are already in Pascal case (except for `Dns query`), but I don't think there would be a one-size-fits-all solution. Are these the only two event formats you have or are there more?


In lsof output what are those 'sock' lines?

Question: In lsof command under Ubuntu/Debian, I'm seeing a lot of lines like below:
PROGRAM 829 root  140u     sock        0,8       0t0 244182 protocol: TCP

What are these lines?
Can they be results of failed TCP connection attempts?
Answer: This line is displayed when lsof cannot find additional informations on the TCP socket: it knows it's a TCP socket but not more.
There are two reasons I can think of:

Unlikely: the socket is still not listening nor connecting: ie a server or client used for example socket(AF_INET, SOCK_STREAM, 0) to create a TCP socket but didn't call yet listen(2) or connect(2). This could be caused by lack of resources or a buggy software.

Most likely today: the process seen by lsof runs in an other network namespace, typically in a container (Docker, LXC, LXD ...) and thus lsof doesn't have access to the relevant information and doesn't display it.
You should then run lsof from the same network namespace as the process. The lsns and nsenter commands can greatly help for this. For your case this would then probably work:
  nsenter -t 829 --net lsof -n -p 829

In normal cases lsof would display IPv4 or IPv6 instead of sock and would have additional informations, like listening port or addresses involved. Even a connection still ongoing would be displayed with the addresses involved and for example SYN_SENT.


When removing the StemEngraver, how can I ensure Lilypond properly places adjacent notes?

Question: The following code compiles without a hitch:
\version "2.20.0"

left = \relative c' {

}

\layout {
  \context {
    \Voice
      % When I uncomment below, the
      % note spacing goes awry
      %\remove "Stem_engraver"
  }
}

<<
    \new Staff \left
>>

However, when I \remove the Stem_engraver, the B and C are no longer properly spaced, but instead are stacked right on top of each other:

Is it possible to remove the Stem_engraver without sacrificing this note spacing? Because of the larger project of which this is a part, I'm hesitant to simply use \hide Stem, since some staves will have these stems and others will not.
Answer: Personally I'd simply do \override Stem.stencil = ##f or \omit Stem. The problem here is that the NoteHead spacing relies on the direction of the stem. Of course it would be possible to implement a stemless version of this, but I do not think there would be much of a benefit (and definitely a benefit worth the effort).
Of course you could manually set NoteHead.X-offset.
Note that \override Stem.stencil = ##f and \revert Stem.stencil does offer the benefit that you can switch between Stem and no Stem within a Voice.


Pandas Pivot - "TypeError: 'NoneType' object is not iterable"

Question: Pandas read_SQL is not working, I am trying to access a SQL pivot function (Example 1). I have tried reading all the data but Pandas does not also process the data.
EXAMPLE 1
sql_status_20 = """
SET NOCOUNT ON
USE [HilltopSamplerFix]

DECLARE @DynamicPivotQuery AS NVARCHAR(MAX)
DECLARE @ColumnName AS NVARCHAR(MAX)

select @ColumnName = STUFF((SELECT distinct ',' +
QUOTENAME(Name)
FROM [Hilltopsamplerfix].[dbo].[tempHilltopWaterQualityExtractAll]
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '');

SET @DynamicPivotQuery =
N'SELECT
SiteID,SiteName,RunDate,SampleTime,SampleID,' + @ColumnName + '
FROM [Hilltopsamplerfix].[dbo].[tempHilltopWaterQualityExtractAll]
PIVOT(max(Value)
FOR Name IN (' + @ColumnName + ')) AS PVTTable'

EXEC sp_executesql @DynamicPivotQuery
"""
connection_string = '...works...'
cnxn = pyodbc.connect(connection_string)
test = pd.read_sql(sql_status_20, cnxn)
cnxn.close()

ERROR
columns = [col_desc[0] for col_desc in cur.description]
TypeError: 'NoneType' object is not iterable
Comment: Please limit it to one question, eg split the second question on pivot_table to another question.
Comment: Regarding the first question, can you show the output of `cnxn.execute(sql_status_20).fetchall()` ? (this will test the query itself, without putting the data in a dataframe)
Comment: Thanks joris. The result from the fetchall() "pyodbc.ProgrammingError: No results.  Previous SQL was not a query." It appears python is not able to process advanced queries. The query works in Studio manager.
Answer: Old question, but I had the same issue as mentioned above. The problem lies in the USE [HilltopSamplerFix]. The database name is already mentioned in the connectionstring, so this part can be removed.
Answer: I had a similar issue, and removing the USE [DatabaseName] clause fixed the issue. I also had the Database specified in my connection string.


Variables are not expanded in smb.conf when using Samba 4

Question: I'm using Samba 4.0.0beta2 on Debian Wheezy as Active Directory domain controller. I also use winbind in my /etc/nsswitch.conf so getent passwd returns domain users.
I've set
template homedir = /home/%U

in my smb.conf file but, instead of substituting %U for username, getent passwd returns:
HOME\Administrator:*:0:100::/home/%U:/bin/bash
HOME\Guest:*:3000000:3000001::/home/%U:/bin/bash

Why is that?
Answer: The value has change since Samba4,
New value for the username is %ACCOUNTNAME% use as this :
template homedir = /home/%ACCOUNTNAME%

Comment: I wrote a [similar QA](http://unix.stackexchange.com/a/156449/9259) on the Unix and Linux site:


How do I set a variable inside a bash for loop?

Question: I need to set a variable inside of a bash for loop, which for some reason, is not working for me. Here is an excerpt of my script:
function unlockBoxAll
{
appdir=$(grep -i "CutTheRope.app" /tmp/App_list.tmp)
for lvl in {0..24}
key="UNLOCKED_$box_$lvl"
plutil -key "$key" -value "1" "$appdir/../Library/Preferences/com.chillingo.cuttherope.plist" 2>&1> /dev/null
successCheck=$(plutil -key "$key" "$appdir/../Library/Preferences/com.chillingo.cuttherope.plist")
if [ "$successCheck" -eq "1" ]; then
 echo "Success! "
    else
 echo "Failed: Key is $successCheck "
fi
done
}

As you can see, I try to write to a variable inside the loop with:
key="UNLOCKED_$box_$lvl"

But when I do that, I get this:
/usr/bin/cutTheRope.sh: line 23: syntax error near unexpected token `key="UNLOCKED_$box_$lvl"'
/usr/bin/cutTheRope.sh: line 23: `key="UNLOCKED_$box_$lvl"'

What am I not doing right? Is there another way to do this?
Please help, thanks.
Answer: Use
for lvl in 1 2 3 4
do
    key="UNLOCKED_${box}_$lvl"
done

You were missing "do"/"done" keywords wrapping around the loop body
$box_$lvl is treated by bash as a variable with the name box_ followed by variable with the name lvl. This is because _ is a valid character in a variable name. To separate the variable name from following _, use ${varname} syntax as shown above
{0..24} does not work in bash v2 (which our servers have here) though it works as a range shortcut on modern bash so that should not cause youproblems.


NSThread, UITableViewCell and Image

Question: I am having some problem with my UITableView. I am downloading the data and setting up the cells like usual, but this time with the NSThread added.
First I will call ASIHTTPRequest in my view did load
Then in - (void)requestFinished:(ASIHTTPRequest *)request I will add the url of the thumbnail into an NSMutableArray
 NSString *photoURLString = [NSString stringWithFormat:@"http://some.url.com/img/%@",[thearray objectForKey:@"tn_url"]];
[thumbNailURL addObject:(photoURLString)];

And in the cellForRowAtIndexPath: where I am setting up other data to display, I will call my NSThread
[NSThread detachNewThreadSelector:@selector(loadImage:) toTarget:self withObject:indexPath];

And in my loadImage thread
- (void) loadImage: (NSIndexPath*) indexPath{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSString *iconURL = [thumbNailURL objectAtIndex:indexPath.row];
NSURL *url = [NSURL URLWithString:iconURL];
UIImage *icon = [UIImage imageWithData:[NSData dataWithContentsOfURL:url]];
cellImageView = [[UIImageView alloc] initWithFrame:CGRectMake(11.0, 6.0, 61.0, 50.0)];
[cellImageView setImage:icon];
UITableViewCell *cell = [self.tableView cellForRowAtIndexPath:indexPath];
[cell.contentView performSelectorOnMainThread:@selector(addSubview:) withObject:cellImageView waitUntilDone:NO];
[pool drain];

}
All the data and image loaded just fine, but if I scroll faster up and down , the application will hang with the following error
[1997:7a1f] *** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[__NSArrayM objectAtIndex:]: index 6 beyond bounds [0 .. 5]'    (0x31312d2f 0x345f50e3 0x3127d1b1 0x35aefc1d 0xf4dd 0x333ffb51 0x3347c67b 0x366af589 0x366b2310) terminate called throwing an exceptionProgram received signal:  "SIGABRT".

It would be great if anyone could show me the correct configuration if I am wrong, as I have been trying different approaches for the past 3 days already.
Answer: A couple things.
First, I would recommend you consider using something like SDWebImage for async image downloading (looks like Olivier Poitrey is the original author). Github repository here: https://github.com/rs/SDWebImage.  I use it for this very purpose and it's great.
That said, I see a few things here.
1) You don't seem to be checking if the cell image is already set... no need to re-download the image if you've already got it.  You might get away with this since dataWithContentsOfUrl might offer some caching, but I'm not sure.
2) I can't tell from the code here, but you need to be sure you don't allow the table view to load it's data until after requestFinished is called.  Otherwise you'll have a race condition that could cause the type of error you are seeing.  Consider setting the tableView's data source in the requestFinished method and then calling reloadData.  Otherwise, do as downed suggested and just prefill the URL array with [NSNull null] objects.
Comment: First of all, thank you so much for your great help, SDWebImage works so great! And so easy to integrate.

And to cover what I have missed in my question, yes, i didn't check if the image is existed, and I guess it download everytime when the cell resuse and caused the crash (though i am not sure where should i integrate the check); I actually have a placeholder cell setup if the async hasn't finish and once it is finished, it will reloadData. So everything is fine up to the point of the NSThread.
Answer: Have you tried having the array "thumbNailURL" contain [NSNull null] objects for the number of items that you are trying to display, then replacing them after you load them async, that way you can't be faster than the array.
thumbNailURL = [[NSMutableArray alloc] init];
for (unsigned i = 0; i < [items count]; i++) {
    [thumbNailURL addObject:[NSNull null]];
}

Then simply:
[thumbNailURL replaceObjectAtIndex:index withObject:thumbnail];

Comment: thanks, but I believe it's the NSThread which caused the problem, not the Array, I have a blank loading cell configured if the async request hasn't finished or returns no result.  Without the NSThread loading image part, everything is just fine. So, there must be some problem to it which I couldn't figure out >,<

NSThread, UITableViewCell and Image