How to get indexOf to check for one or multiple values

Question: I'm using google scripts and as a result, I can't use ES6 or ES5 in my HTML code. I'd like to check if a string variable might include one or multiple of the words below. The example below doesn't work. How is this suppose to write this?
string.indexOf("Word1" || "Word2" || "Word3") > -1
Answer: One option is to use a regular expression instead:
/Word1|Word2|Word3/.test(string)

Or iterate over an array of words to search for:
var found = false;
var arr = ['Word1', 'Word2', 'Word3']
for (var i = 0; i < arr.length; i++) {
  if (string.indexOf(arr[i]) !== -1) {
    found = true;
    break;
  }
}
// use `found` variable

If you want to ensure that the string doesn't contain any of those words with a regex, then continually match characters from the beginning of the string to the end while using negative lookahead for the alternated pattern:
^(?:(?!Word1|Word2|Word3)[\s\S])+$

https://regex101.com/r/FsChvB/1
But that's strange, it'd be a lot easier just to use the same test as above, and invert it.
var stringContainsForbiddenWords = !/Word1|Word2|Word3/.test(string)

Comment: Thanks. How would that look if you wanted it 'not' to include the words using regular expressions?
Answer: Just make it a for loop to check each array element.
var array = ["test234", "test9495", "test234", "test93992", "test234"];

for (i=0;i<array.length;i++) {
  if (array[i] == "test234") {
    document.write(i + "
");
  }
}

Answer:

// Polyfill
String.prototype.includes = String.prototype.includes || function(s) {
  return this.indexOf(s) >= 0
}

// ES6:
// const strIncludes = (string, words) =>
//   words.some(w => string.includes(w))

function strIncludes(string, words) {
  return words.some(function(s) {
    return string.includes(s)
  })
}

console.log(strIncludes('hello world and goodbye', ['hi', 'hello']))


how to get a variable by number javascript

Question: so, I have these 4 varibles:
var hi1 = "Hello!";
var hi2 = "Hello to you too!";
var hi3 = "I'm good, thanks!";
var i = 0;

And depending on what a user does, the variable i goes one up.
My question is, how can I get hi1, or hi2, just by i?
for example I want to do this:
document.getElementById("idol").innerHTML = hi + number;

When I do something like this, I get the error 'hi is not defined' (or something of this sort)
Comment: You should definitely drop this and try jQuery. :D BTW, please accept my downvote, which I give you for not doing enough research.
Comment: Use an array to store your data.
Comment: What I wrote was an example. The stuff are actually functions, and I do not want to store them in array. Any alternatives? (except using if / elseif, I don't want to use em)
Comment: "I don't want to use them" ... Then what about `eval("hi" + number)` ? (Thats the worst thing you can do, but well if you don't like the other options)
Comment: How about abusing global scope and using `window["hi" + number]`?
Comment: @Mxm why do you not want to store them in an array? At any rate, you should be storing them *somewhere* - could be an array, it could be an object. Trying to fetch a *variable* by name is the wrong approach.
Answer: You could use the eval syntax "eval()" to get your property eval("hi" + 2) == (variable)Hi2.
Please have a look at the documentation to learn more about eval.

Note: Eval is probably the worst thing you could do in javascript. I would strongly advise against using the eval-syntax anywhere in your code.
But anyway. Here is some example code matching to your question:

var hi1 = "Hello!";
var hi2 = "Hello to you too!";
var hi3 = "I'm good, thanks!";
var i = 0;

let number = 2
document.getElementById("idol").innerHTML = eval("hi" + number);

Another possibility would be to use the window object using this syntax:
let content = window["hi" + number];

Or you could also use an object to store data.
let hiObject = {
   1: "Hello!",
   2: "Hello to you too!",
   3: "I'm good, thanks!"
}
let content = hiObject[number]

... or even an array:
let hiObject = [
   "Hello!",
   "Hello to you too!",
   "I'm good, thanks!"
]
let content = hiObject[number]

Comment: eval is also way slower as it is parsed on execution, not just once.
Comment: Why is eval() bad?
Comment: I would recommend you to read https://stackoverflow.com/a/86580/3596335
Comment: @Mxm as a simple example, let's say you want the user to type `1` or `2` - if you do `eval("hi" + userInput)`, then if the user types in `1; alert('you can execute any JavaScript here!')` you'd get that alert. Or any other arbitrary javascript. From there, an attacker can get a victim to execute some code that will steal their account info. Sure you can say "but there is no sensitive information" or "I'm not passing raw input", yet would that be true a year down the line? If you or somebody else changes something, it's suddenly a security hole.
Comment: And all you needed was to fetch something by name, for which you never even needed to use `eval`. Yes, you can be careful using it but it's literally easier to use something else to get the result you want.
Comment: I don't think eval is bad for this, but I'm gonna go with the array method down below.
Answer: In this case the best approach would be using an array. I will replicate your case with following code.

var i = 0;
var hi = ["Hello!","Hello to you too!","I'm good, thanks!"];
for(var i=0; i<hi.length; i++){
  document.getElementById("idol").innerHTML += hi[i]+"
";
}

Answer: You mentioned that those hi1, hi2 are functions. But you can still use arrays to store them, even if they are functions:
var hi1 = () => "Hello!";
// hi1 is a function that returns "Hello!"
var hi2 = () => "Hello to you too!";
// hi2 is a function that returns "Hello to you too!"
var array = [hi1, hi2];

To access the functions:
array[0]()
=> "Hello!"
array[1]()
"Hello to you too!"
document.getElementById("idol").innerHTML = array[i-1]();


How to debug leak in native memory on JVM?

Question: We have a java application running on Mule.  We have the XMX value configured for 6144M, but are routinely seeing the overall memory usage climb and climb. It was getting close to 20 GB the other day before we proactively restarted it.
Thu Jun 30 03:05:57 CDT 2016
top - 03:05:58 up 149 days,  6:19,  0 users,  load average: 0.04, 0.04, 0.00
Tasks: 164 total,   1 running, 163 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.2%us,  1.7%sy,  0.0%ni, 93.9%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  24600552k total, 21654876k used,  2945676k free,   440828k buffers
Swap:  2097144k total,    84256k used,  2012888k free,  1047316k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 3840 myuser  20   0 23.9g  18g  53m S  0.0 79.9 375:30.02 java

The jps command shows:
10671 Jps
3840 MuleContainerBootstrap

The jstat command shows:
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       PC     PU    YGC     YGCT    FGC    FGCT     GCT
37376.0 36864.0 16160.0  0.0   2022912.0 1941418.4 4194304.0   445432.2  78336.0 66776.7    232    7.044  17     17.403   24.447

The startup arguments are (sensitive bits have been changed):
3840 MuleContainerBootstrap -Dmule.home=/mule -Dmule.base=/mule -Djava.net.preferIPv4Stack=TRUE -XX:MaxPermSize=256m -Djava.endorsed.dirs=/mule/lib/endorsed -XX:+HeapDumpOnOutOfMemoryError -Dmyapp.lib.path=/datalake/app/ext_lib/ -DTARGET_ENV=prod -Djava.library.path=/opt/mapr/lib -DksPass=mypass -DsecretKey=aeskey -DencryptMode=AES -Dkeystore=/mule/myStore -DkeystoreInstance=JCEKS -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf -Dmule.mmc.bind.port=1521 -Xms6144m -Xmx6144m -Djava.library.path=%LD_LIBRARY_PATH%:/mule/lib/boot -Dwrapper.key=a_guid -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.disable_console_input=TRUE -Dwrapper.pid=10744 -Dwrapper.version=3.5.19-st -Dwrapper.native_library=wrapper -Dwrapper.arch=x86 -Dwrapper.service=TRUE -Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1 -Dwrapper.lang.domain=wrapper -Dwrapper.lang.folder=../lang

Adding up the "capacity" items from jps shows that only my 6144m is being used for java heap.  Where the heck is the rest of the memory being used? Stack memory?  Native heap? I'm not even sure how to proceed.
If left to continue growing, it will consume all memory on the system and we will eventually see the system freeze up throwing swap space errors.
I have another process that is starting to grow.  Currently at about 11g resident memory.
pmap 10746 > pmap_10746.txt
cat pmap_10746.txt | grep anon | cut -c18-25 | sort -h | uniq -c | sort -rn | less

Top 10 entries by count:
    119     12K
    112   1016K
     56      4K
     38 131072K
     20  65532K
     15 131068K
     14  65536K
     10    132K
      8  65404K
      7    128K

Top 10 entries by allocation size:
     1 6291456K
      1 205816K
      1 155648K
     38 131072K
     15 131068K
      1 108772K
      1  71680K
     14  65536K
     20  65532K
      1  65512K

And top 10 by total size:
Count   Size    Aggregate
1   6291456K    6291456K
38  131072K 4980736K
15  131068K 1966020K
20  65532K  1310640K
14  65536K  917504K
8   65404K  523232K
1   205816K 205816K
1   155648K 155648K
112 1016K   113792K

This seems to be telling me that because the Xmx and Xms are set to the same value, there is a single allocation of 6291456K for the java heap.  Other allocations are NOT java heap memory.  What are they? They are getting allocated in rather large chunks.
Comment: Check direct ByteBuffer memory (available through MXBean `java.nio:type=BufferPool`). Try also [Native Memory Tracking](https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html) feature.
Comment: Not a proper answer, but I've had that happen with network buffers. Apparently if you subscribe to multicast and don't read it (or read it slowly) the inbound buffers can grow to huge sizes (I've observed 10's of GB). Maybe it's true for other network. I've not found a way to effectively analyze a process memory dump, not even a rough view unfortunately (and would be very interested in an actual answer).
Comment: I'm still running Java 7, so it looks like the Native Memory Tracking is not an options for me.  Looks really useful though.
Answer: Expanding a bit more details on Peter's answer.
You can take a binary heap dump from within VisualVM (right click on the process in the left-hand side list, and then on heap dump - it'll appear right below shortly after).  If you can't attach VisualVM to your JVM, you can also generate the dump with this:
jmap -dump:format=b,file=heap.hprof $PID

Then copy the file and open it with Visual VM (File, Load, select type heap dump, find the file.)
As Peter notes, a likely cause for the leak may be non collected DirectByteBuffers (e.g.: some instance of another class is not properly de-referencing buffers, so they are never GC'd).
To identify where are these references coming from, you can use Visual VM to examine the heap and find all instances of DirectByteByffer in the "Classes" tab.  Find the DBB class, right click, go to instances view.
This will give you a list of instances.  You can click on one and see who's keeping a reference each one:

Note the bottom pane, we have "referent" of type Cleaner and 2 "mybuffer".  These would be properties in other classes that are referencing the instance of DirectByteBuffer we drilled into (it should be ok if you ignore the Cleaner and focus on the others).
From this point on you need to proceed based on your application.
Another equivalent way to get the list of DBB instances is from the OQL tab.  This query:
select x from java.nio.DirectByteBuffer x

Gives us the same list as before.  The benefit of using OQL is that you can execute more more complex queries.  For example, this gets all the instances that are keeping a reference to a DirectByteBuffer:
select referrers(x) from java.nio.DirectByteBuffer x

Comment: Thanks for the thorough explanation.  VisualVM was not able to load my heap dump, but MAT was.  It showed 1.3 million DirectByteBuffer objects, even after a forced gc.  The majority of them are associated with com.mapr.fs.jni.Page.  I've contacted their support.
Answer: What you can do is take a heap dump and look for object which are storing data off heap such as ByteBuffers.  Those objects will appear small but are a proxy for larger off heap memory areas. See if you can determine why lots of those might be retained.


estimate on a convolution

Question: Let $\psi$ be a non-negative Schwartz function on $\mathbb{R}$ such that supp$\hat{\psi}$ is contained in $[-0.1, 0.1]$ and $\hat{\psi}(0)=1$. Define $\psi_k(x)=2^k\psi(2^kx)$ for any integer $k$. Let $I_k$ be an interval of length $2^{-k}$ and $1_{I_k}$ be the indicator function  of $I_k$. Define $f(x)=1_{I_k}*\psi_k(x)$. Prove that $$|f(x)|\le\frac{C_N}{(1+2^kdist(x, I_k))^N}$$
I can prove this if I assume $\psi$ is radial and decreasing (in this case no information of Fourier transform is needed), but I do not know how to use $\hat{\psi_k}$ or its support (which is an interval of length $0.2\times2^k$) in the proof of this estimate. Any help is greatly appreciated.
Answer: Perhaps I am missing something and have made an error, but I don't see why you need to the information on the support of $\hat{\psi}$ to prove the estimate for $f$; that $\psi(x)\lesssim_{N} (1+\left|x\right|)^{-N}$, for any $N>0$, together with the information on $I_{k}$ is sufficient.
Write $I_{k}=[a-2^{-k-1},a+2^{-k-1}]$. First, I claim that it suffices to consider the case $a=0$. Write $I_{k}'=[-2^{-k-1},2^{-k-1}]$. Making the change of variable $z+a=y$, we see that
$$f(x)=\int_{\mathbb{R}}\mathrm{1}_{I_{k}}(y)\psi_{k}(x-y)dy=\int_{\mathbb{R}}\mathrm{1}_{I_{k}}(z+a)\psi_{k}(x-(z+a))dz=\int_{\mathbb{R}}\mathrm{1}_{I_{k}'}(z)\psi_{k}((x-a)-z)dz$$
So if we have the estimate for the case $a=0$, then using the fact that $\text{dist}(x-a,I_{k}')=\text{dist}(x,I_{k})$ (you can check this), we have
$$\left|f(x)\right|\lesssim_{N}(1+2^{k}\text{dist}(x-a,I_{k}'))^{-N}=(1+2^{k}\text{dist}(x,I_{k}))^{-N}$$
So it suffices to assume that $a=0$. Observe that
$$\int_{\mathbb{R}}\mathrm{1}_{I_{k}}(y)\psi_{k}(x-y)dy=\int_{\mathbb{R}}\mathrm{1}_{I_{k}}(y)2^{k}\psi(2^{k}(x-y))dy=\int_{\mathbb{R}}\mathrm{1}_{I_{k}}(2^{-k}z)\psi(2^{k}x-z)dz=\int_{-1/2}^{1/2}\psi(2^{k}x-z)dz, \tag{1}$$
where we make the change of variable $z=2^{k}y$. Since $\psi$ is Schwartz, the RHS in (1) is
$$\leq C_{N}\int_{-1/2}^{1/2}(1+\left|2^{k}x-z\right|)^{-N}dz \tag{2}$$
for any $N>0$. We consider now three case.
First case $x\geq 2^{-k-1}$. Then the integrand in (2) is majorized by $(1+\left|2^{k}x-1/2\right|)^{-N}$, and since $\text{dist}(x,I_{k})=\left|x-2^{-k-1}\right|$, we get
$$\left|f(x)\right|\leq C_{N}(1+2^{k}\text{dist}(x,I_{k}))^{-N}$$
Second case $x\leq -2^{-k-1}$. The integrand is majorized by $(1+\left|2^{k}x+1/2\right|)^{-N}$, and since $\text{dist}(x,I_{k})=\left|x+2^{-k-1}\right|$, we get
$$\left|f(x)\right|\leq C_{N}(1+2^{k}\text{dist}(x,I_{k}))^{-N}$$
Third case $-2^{-k-1}<x<2^{-k-1}$. The integrand is majorized by $1$, and since $\text{dist}(x,I_{k})=0$, we get
$$\left|f(x)\right|\leq 1=(1+2^{k}\text{dist}(x,I_{k}))^{-N}$$
Taking $C_{N}'=\max\left\{1,C_{N}\right\}$ completes the proof.

estimate on a convolution


In Angular 5, How do I prevent a component from invoking its subscription twice when the components has been contained twice within another component?

Question: In Angular 5, How do I prevent a component from invoking its subscription twice when the components has been contained twice within another component?
For example, I have a NavMenuComponent, and within the template, I have a cart contained twice

And, Contained within the NavMenuCartComponent, the ctor has a subscription as follows
    this.subscription = cartConnectorService.cartItemAdded$.subscribe(
  newCartItem => {
    console.log("in cartConnectorService.cartItemAdded$.subscribe");
    this.saveCartWithNewItem(newCartItem);
  });

saveCartWithNewItem ultimately invokes the CartService.saveCart(cart)
The problem is that the saveCartWithNewItem function is invoked twice, whereas it needs to be invoked only once. (log screenshot below showing its been invoked twice)

Someone may ask, Why is your NavMenuCartComponent responsible for making the call to saveCartWithNewItem? Why not have the data contained within say, a CartService, and simply refresh the data in the NavMenuCartComponent in the subscription instead? The reason is that the CartService is only responsible for HTTP calls, not storing data, at this time.
Also, the component initially responsible for invoking the Observable is ProductsGridComponent. Which uses the connector service CartConnectorService.
Fundamentally, what I want to happen is, I want to allow the ProductsGridComponent to allow adding items to the cart. And, given that the NavMenuComponent has two NavMenuCartComponent's, I need the data to be updated in those, as well as in persistance layer (through the service). My implementation was to use the  CartConnectorService to ensure that this happens, but now I have this duplicate invokation issue. Any help is appreciated.
Comment: The only way I can see around it is creating an `@Input` on your component specifying whether or not to invoke `saveCartWithNewItem`, so that you can tell one of them to save and not the other. Feels a bit messy though.
Comment: Can you provide some code to support this possibility?
Answer: You can create an @Input on the NavMenuCartComponent, that tells it whether or not to saveCartWithNewItem.
In the NavMenuCartComponent:
@Input callSaveCart: boolean;

//...

ngOnInit() {
    // ...

    if(callSaveCart) {
        this.subscription = cartConnectorService.cartItemAdded$.subscribe(newCartItem => {
            console.log("in cartConnectorService.cartItemAdded$.subscribe");
            this.saveCartWithNewItem(newCartItem);
        });
    }
}

Then, in your HTML, specify the one you want to save:

Comment: @joey If the two components need to share state, then that shared state belongs in a service
Comment: almost works. So I upvoted. Only issue now is that the NavMenuCartComponent has   navMenuCart: ICart = {
    buyerId: '',
    items: []
  }; and this actually needs to be updated after the save, any ideas here? With the update you mentioned above, only one of the templates gets its data updated.
Comment: I appreciate the help. Ill take a look into that since that changes things quite a bit. Thank you.
Answer: Simple solution I can think of:
if (!this.subscription) {
  this.subscription = cartConnectorService.cartItemAdded$.subscribe(
    newCartItem => {
      console.log("in cartConnectorService.cartItemAdded$.subscribe");
      this.saveCartWithNewItem(newCartItem);
    });
}

Comment: this wont work, the ctor is invoked twice, and the subscription is not set until below your conditional, therefore this.subscription.unsubscribe(); will never happen. I just tested this.


Why my taxonomy term list in content is not using node languages prefererence

Question: I have a multilingual website, in which I activated the i18n module and every related modules.
I have a taxonomy vocabulary Colors.
When I create or edit a content, all my colors are displayed. What I mean is, even if my node are in English, I saw every colors (English and French).
Is there a way to displayed my color based on node languages preference?
Comment: I have answered a similar question http://drupal.stackexchange.com/a/120763/17003, check whether it helps you.
Comment: Thanks a lot for your answer. I'm using the second method (Translate). My concern by using the Localize is that I not only have the name and description to translate, I have more fields. Other things, I want to be able to specify the url alias for each languages and localize do not permit that.
Comment: Can you make sure that language is selected for each term, and it is not `Language neutral`
Comment: Thanks wolverine, it was the problem. But for a reason that I didn't understand, when I try to translate my term with an image, the language field return back to Language Neutral. I opened an new question for that http://drupal.stackexchange.com/questions/121119/taxonomy-with-image-field-is-not-translating
Answer: Check the "translation mode" that you've specified within admin/structure/taxonomy/colors/edit (whereas "colors" is the name of your specific category): by default that is set to "Localize", though it should be set to "translate".
After changing this value, be aware that you then also have to:

enter a translated value for each of the tags and
revisit any of the existing nodes that are already using the specific taxonomy, so that they use the (translated) term that corresponds to the language of the node.


Estimate the number of sides of the die

Question: Suppose you have a fair die with an unknown number of sides, $T$. Each side is distinct, marked with a complex symbol. Given enough rolls, can you estimate $T$?
My intuition is that can do this via a generalization of the mark and recapture procedure. Roll the die some large number of times, keeping track of all the sides that you have rolled and the number of times you have rolled them. Since the probability of rolling a previously rolled side in $N$ rolls is completely determined by $N$ and $T$, then expected proportion of unique to total sides $p$ should also be determined by $N$ and $T$. What is the relationship between $p$, $N$, and $T$?
As pointed out by Don Thousand, you can save yourself a lot of overhead and just track one arbitrarily picked side. Since the probability of rolling the chosen side on any one roll is $T^{-1}$, you expect that you would see the side repeated $NT^{-1}$ times. You can use standard statistical techniques to determine how precise this estimate likely is. If you have reasonably strong priors about the likely size of the population, within an order of magnitude or so, you can probably figure out how big of an $N$ you need to satisfy your requirements.
However, this procedure is inefficient, since we will likely have re-rolls of sides other than the one we have picked. The frequency of those re-rolls is almost equally as informative. How can we combine all the information we have to arrive at an estimate for $T$?
Comment: My intuition is that can do this via a generalization of the mark and recapture procedure [1].
Roll the die some large number of times, keeping track of all the sides that you have rolled and the number of times you have rolled them. Since the probability of rolling a previously rolled side in $N$ rolls is completely determined by $N$ and $T$, then expected proportion of unique to total sides $p$ should also be determined by $N$ and $T$. What is the relationship between $p$, $N$, and $T$?

  [1]: http://en.wikipedia.org/wiki/Mark_and_recapture
Comment: Is the die fair? If it is, this is as trivial as recording the first symbol, and then rolling the die an obscene number of times and seeing the frequency of that recorded symbol, and a good estimate is $\frac1{\text{frequency}}$
Comment: You should [edit] that info into your question. I agree that Mark & Recapture is relevant, but even more relevant: https://en.wikipedia.org/wiki/German_tank_problem
Comment: On 2nd thoughts, maybe the strategy used in the German tank problem isn't so helpful here, since we have arbitrary symbols, not sequential serial numbers.
Comment: @DonThousand: If you're happy with rolling the die an obscene number of times, why don't you just roll the die a rather large number of times instead, and count the different outcones?
Comment: Hmmm. That's a *big* die. ;) If you need to take $\sqrt T$ samples to get a decent estimate, and you can generate a million samples / second, it'll take over 3 million years to generate $10^{20}$ samples. And the storage requirements are rather hefty as well. The next comment contains a Python script that crudely estimates $T$ from $i$, the number of samples, and $k$, the number of duplicates. It takes samples until $k^2$ exceeds the number of unique samples. Then $T\approx i*k/2$. This is pretty useless for your large $T$, though.
Comment: For a bit of context: This is motivated by trying to come up with an empirical way of determining the number of sensible chess games. Because the number of possible games is large, the game you play is usually unique, but occasionally you will repeat a game exactly by chance. (Controversially) assume that all sensible chess games are equally probable. Then the above model gives us a way of estimating the total number of sensible games.
Comment: A conservative analytic estimate of $T$ in this context is $10^{40}$. It is determined by considering the average number of moves made in a game, and the average number of sensible options at each move. However, chess games are frequently characterized by long sequences of forced moves, where there is only one sensible option. On the other hand, there are often more than the average number of choices. Thus, such methods are likely to overestimate the true number of games on one hand, and underestimate them on the other. An empirical approach would complement an analytic one.
Comment: Of course, working with such numbers it might not be feasible to get a large enough sample. Additionally, the assumption that all sensible games are equally likely is probably radically inaccurate. So I am only expecting so much.
Comment: [Python script](https://sagecell.sagemath.org/?z=eJxtksFugzAMhu95Ct8a2milO1Zi2mlPwL1Kh9my0IBCkPr4_U3XlhYsBYPj74_5lTq2J4o2VEju1LUxjV9YP6ziEHoqaJfnuVKfLiSO9jupims66KYtsGHo10nGS5-4u5bskNrD0FU2cfFlm56zvSJE3UYqyQUa5aEg9Ea4K_3fJpEcDk4ej_xeE_zvgct0E0KiZw6GhOo56cxM4JuAewjsDJUvAhJn4HcPdJnNGlyNHqjIaXNcwtMGti1uAfZrTx8FNRy0SCyMcItjZOtnuwxHlxmRe7NVpc_PU8NNDOSea-OQfur4Fj8OU9Wk5bXURVwDXRpa7VcGDJaXvE5--55dAEDSitQ=⟨=python)
Comment: @TonyK You could do that too. I just think that my method is easier to write down.
Answer: Here is one way you could do it. Let $D_n$ be the number of distinct symbols observed after seeing $n$ rolls. Let $S$ be the number of symbols, and I'm assuming that we have prior distribution for $\mathbb{P}(S = s) = \pi(s)$ before we observe any dice rolls.
This answer shows that if $d \leq s$,
$$\mathbb{P}(D_n = d \mid S = s) = \frac{d!\binom{s}{d} \def\stir#1#2{\left\{{#1\atop#2}\right\}}\stir nd}{s^n},$$
where $\stir nd$ is a Stirling number of the second kind. Using Bayes' Theorem we can compute
$$\mathbb{P}(S = s \mid D_n = d) = \frac{\mathbb{P}(D_n = d \mid S = s) \mathbb{P}(S = s)}{\sum_s \mathbb{P}(D_n = d \mid S = s) \mathbb{P}(S = s)}.$$
To give an example, suppose I know that $S$ is in the set $\{6, 7, 8, 9, 10\}$ and that my prior is $\pi(6) = \pi(10) = 0.1$, $\pi(7) = \pi(9) = 0.2$, and so $\pi(8) = 0.4$. Now suppose we observe the following 15 rolls:

K K Q H X G A X X Q C C C G X

We have $D_{15} = 7$ because there are 7 distinct observed symbols: $\{A, C, G, H, K, Q, X\}$. We now have $\mathbb{P}(S = 6 \mid D_{15} = 7) = 0$ because we know there are at least 7 symbols. We now calculate the likelihood $\times$ prior terms:

$\mathbb{P}(D_{15} = 7 \mid S = 6) \mathbb{P}(S = 6) = 0$,
$\mathbb{P}(D_{15} = 7 \mid S = 7) \mathbb{P}(S = 7) = 0.433\ \cdot \ 0.2 = 0.087$,
$\mathbb{P}(D_{15} = 7 \mid S = 8) \mathbb{P}(S = 8) = 0.468\ \cdot \ 0.4 = 0.187$,
$\mathbb{P}(D_{15} = 7 \mid S = 9) \mathbb{P}(S = 9) = 0.360\ \cdot \ 0.2 = 0.072$,
$\mathbb{P}(D_{15} = 7 \mid S = 10) \mathbb{P}(S = 10) = 0.247\ \cdot \ 0.1 = 0.025$.

Then the posterior probabilities are

$\mathbb{P}(S = 6 \mid D_{15} = 7) = 0$,
$\mathbb{P}(S = 7 \mid D_{15} = 7) = 0.087/0.371 = 0.23$,
$\mathbb{P}(S = 8 \mid D_{15} = 7) = 0.187/0.371 = 0.50$,
$\mathbb{P}(S = 9 \mid D_{15} = 7) = 0.072/0.371 = 0.19$,
$\mathbb{P}(S = 10 \mid D_{15} = 7) = 0.025/0.371 = 0.07$.

So we are now even more convinced that $S=8$ than before (and I did actually use $S=8$ to generate the sequence of rolls).
You could also look into using Approximate Bayesian Computation (ABC) to solve this problem. There is a really nice example application of ABC here.
A related problem that you might find interesting if you haven't seen it before is the coupon collector's problem.

Estimate the number of sides of the die


Problem with HTTP traffic being 'hijacked' by firewall

Question: I'm experiencing a major head-ache at the moment with our company firewall, we're currently changing ISPs (new lease line and new block of static IPs) and I'm setting up firewall rules (we're using a NETGEAR ProSafe VPN Firewall FVX538).
The firewall is visible internally on 184.108.40.206 (all IPs for example only) and we can configure it through the web admin on ports 80 and 443. All traffic from our building goes out on 126.96.36.199, but when you go to the 220.127.116.11 IP (from within or outside the network), the firewall admin page is shown. This isn't really what we want!
I've tried adding firewall rules for HTTP traffic on 188.8.131.52 to be routed to one of our web servers at 18.104.22.168, but when I add this rule, requests to 18.104.22.168 just hang. If I change my rule to 126.96.36.199, it works fine but the NETGEAR admin's still visible on 188.8.131.52 - problem not solved.
The real problem is that this means we can't point our domain example.com to 188.8.131.52, because the NETGEAR admin would show rather than our site, but we need this domain to resolve to 188.8.131.52 because all mail out of our building will come from this IP (our client's spam filters won't let our mail through - reverse DNS problems).
I'm banging my head against my desk: this MUST be possible, or how would a company that has a single IP address be able to resolve web traffic? Can anyone help? Sorry if the questions doesn't make a lot of sense - I always find questions like this difficult to express!
Update / Things I've tried

Ensure remote management is disabled (it's never been enabled anyway) screen
Rebooted firewall
Checked for firmware updates
Tried adding DMZ WAN rule in place of a LAN WAN rule
Answer: There's 2 reasons why the router admin page would show up on port 80.

you connected the WAN (your ISP's connection) to a LAN port (probably unlikely since you would have had to set the public IP on the LAN interface)
the remote management port is set to port 80 (check under Administration -> Remote Management)
Comment: Point one may be the issue, point two I've already tried - remote management is completely disabled! As you may have guessed, I'm not *really* a 'server admin', I've been tasked with this because 'web developer' is the closest we've got... Could you explain point one a little more? I'm not 100% sure how I'd actually go about checking/solving that.. Thanks :)
Comment: Well first of all check where the WAN cable (the one from the ISP's device) is plugged in to. As far as I can see it should be plugged into one of the WAN ports on the left side. Can you check that?
Comment: Yep, that's plugged into WAN1 so not a problem
Comment: OK, that means that your Internet connection is good. You can access the internet through it normally, right?
Comment: Yep, there's no problem connecting to the internet at all
Comment: Next thing I'd try doing is just check what port the Remote Administration is set to. If its set to port 80 ... just change it to 8080 and apply settings. And if possible reboot the router afterwards. Just in case turning it off didn't work.
Comment: Can you telnet to the device and view comprehensive logs to paste here by chance?
Comment: Remote management is off, and the port setting is 8080 currently - here's a screenshot of the Administration/Remote Managment page: http://img638.imageshack.us/img638/9590/screenshot20110415at103.png
Comment: Did you reboot the router?
Comment: Just rebooted, still having the same problems, will look into logs
Answer: Turn off remote management or move it to a different port. Beat it over the head then offer it chocolate. Repeat cycle until it actually turns off remote management. Then complain to your vendor about the buggy firmware.
Edit, new answer: Remove all special "forwarding" style rules and put 22.214.171.124 in a DMZ zone so that all outside calls get routed to it instead.
Edit, right answer: Connect to the right device. (see comments)
Comment: Ballsy balls! OK, we have two firewalls running in parallel, the new one on 126.96.36.199 and the old one on 18.104.22.168. When we changed 18.104.22.168 to my machine's IP (126.96.36.199) it just worked! Then we realised that 126.96.36.199 is connected to the internetz via 188.8.131.52, but my machine's connected via 18.104.22.168, doh! moving my machine to use 18.104.22.168 causes the same hanging behaviour... I feel like such a noob!
Comment: Um. Ya. So forwarding to a device inside the network that is talking to the outside world via a different router will definitively screw up your network!
Comment: Thanks for being my [Rubber Duck](http://en.wikipedia.org/wiki/Rubber_duck_debugging)! I think you probably contributed most, although the real answer is: "Rowan, you shouldn't be configuring a firewall, stick to the websites..."
Comment: Hi, thanks for the answer but remote management is completely off.
Comment: @Rowan: Huh. Well that' dumb. It sounds like it's ignoring you. Please tell me you rebooted it :) See my edited answer for another idea.
Comment: Yep, rebooted, see comments in @iserko's answer
Comment: Excuse my ignorance, but what would the DMZ thing do? I'm not 100% sure what you mean, sorry!
Comment: DMZ (Acronym borrowed from the military meaning Demilitarized Zone) is a common configuration option on firewall and router appliances that allow you to expose one (or in advanced systems, more) computers on the "inside" of a protected network as if they were really on the outside. They still have full access to the other inside resources, but they themselves live outside the fortified walls and this are easy to run as servers interfacing with the outside world.
Comment: OK, I think I understand what that is now, but what will adding 184.108.40.206 to a DMZ zone do? Can you guess I'm not a *real* sysadmin? :)
Comment: I'm sorry, I write that address from memory, but the example from your post should have been `220.127.116.11`.
Comment: ah I see! give me a mo, cheers
Comment: OK, adding a DMZ WAN rule has *exactly* the same effect as adding a LAN WAN rule, the 22.214.171.124 just hangs
Comment: I looked up specs on that thing and it looks like it is known to over-promise and under-deliver. I see it actually has hardware DMZ ports. Did you try putting the mentioned dot-9 webserver into that port? Which WAN port are you using? You aren't using both WAN ports are you?
Comment: Nope, we're only using WAN 1, WAN 2 has no cable plugged in and nothing configured for it. I can't plug dot-9 into that at the moment (Boss won't be happy - mail will go down), but I have a webserver running on my personal machine so we'll try plugging me (my machine, not me personally) directly into the firewall. Keep you updated, thanks so much!
Answer: It sounds as if the firewall's firmware is buggy. Have you checked to see whether you're running the latest version of the firmware?
You may want to try enabling remote management, restricting the access to some bogus IP address (127.0.0.1 should do just fine), and changing the port number. At that point, it may be possible to install the inbound forwarding rules you want.
Note that you can have reverse DNS entries that point 126.96.36.199 to example.com so that your mail works, while still having forward DNS entries that point example.com and www.example.com somewhere else...but I would endeavor to avoid this situation if possible.
Comment: Firmware is completely up-to-date, I'll try setting remote management with a bogus IP temporarily
Comment: That had no effect I'm afraid :(


application blocked for security reason.

Question: While trying to self teach java I have stumbled upon a problem.
I have a simple GUI applet which I am trying to run.
I am using java 8 version 66. At runtime I get an 'application blocked by Java security' when I try to preview it in a browser using Dreamweaver.
The reason given is that my security settings have blocked a local application from running.
When clicking on the more information link I get info about the java control panel. They instructed to add a url in the exception site list.
When I want to add my dir in this case C:\dwtest\applet\www\html, it won't add to the exception site list.
I have read the documentation for controling an untrusted app running in my browser on https://www.java.com/en/download/help/jcp_security.xml and https://www.java.com/en/download/faq/exception_sitelist.xml.
Here they are only discussing about adding an URL and nothing about a directory for where the html and .class files are stored.

Is this really a security issue or am I overlooking something?
I know applets are not used much but I am trying to grasp the concept of it. I think it would increase my over-all knowledge of Java.
Any help would be grealty apreciated.
This is what I know/think I am getting correct:
*1- my html code
<title>HTML Test Page</title>

SimpleGui4b will appear in a java enabled browser.

<applet aligned="middle" vspace="0" hspace="0" height="300" width="400"
name="SimpleGui4b" code="SimpleGui4b.class" codebase="."> wordt niet geladen
</applet>

]

*2- I have the html file and two .class files in the same dir which is
C:\dwtest\applet\www\html
*3- my class files are working fine as an application in java but not when I made it in to an applet.
*4-dreamweaver has access to C:\dwtest\applet\www\html
Thanks,
Comment: Your applet must be a signed jar file. Have a look here: https://docs.oracle.com/javase/tutorial/deployment/applet/deployingApplet.html
Comment: When you open local files in a browser, there is usually a `file:///` in the beginning instead of the protocol. Try adding `file:///C:\dwtest\applet\www\html` to the whitelist and/or use `/` instead of \.
Comment: I add                                               **file:///C|/dwtest/applet/www/html/applet.html**    and **file:///C|/dwtest/applet/www/html/applet.html\**    and **file:///C|/dwtest/applet/www/html**    to Exception site list (just to be sure) . The first path is the pathname in dreamweaver . Now I get an explorer content can't be displayed error and browser crashed. When reading https://docs.oracle.com/javase/tutorial/deployment/applet/deployingApplet.html  it's stated that I have to get an certificate. but how is this even needed if I add the pathname in the Exception site list?
Comment: the pathenames without the ** though.many thanks
Comment: *"While trying to self teach java I have stumbled upon a problem."*  Applets and problems will always be associated.  Skip the applet stuff, and you'll avoid moist of those problems.
Comment: Yeah you guys are right. Stomped my head too long on this applet stuff. No one seems to know the answer.
Answer: I tried looking for a solution to this myself. So far the only solution is to install and use a previous version of java (prior to 8u20). For some reason oracle took away the option to run applets which aren't signed by a trusted authority.
The better question is what is a good alternative to applets now?
Comment: I figure that this security issue will always be present. So I just ran the applet in in eclipse. This way I can see if the applet is running without any errors.
Comment: I tried v.7u 79 but to no avail. So running applet in eclipse can still be a good way to practice applets. searching for applet alternative myself.


Which Server for operations with big MySQL databases (milion rows in the table)

Question: What kind of server should I use in order to work with database with table with milion rows in order to save time while operating on the database. Now operation like optimize table takes few hours which is quite bad. Other simple operations also takes many hours. What kind of server will be suitable for my needs? Now I use 2.4 Intel Core 2 Duo and 2 GB of SDRAM.
Comment: Asking for hardware recommendations for a vague software situation is not something somebody can provide a right answer for. You should consider bench marking your application against several different hardware configurations. If you have specific questions like "when would x hardware option effect y software situation", feel free to ask.
Answer: While more and faster processing cores will help in general for best database performance you need to focus on memory and IO capabilities.
If you can buy as much memory as you can afford, ensure it's configured to run at its peak performance (usually by buying the right type and number of modules to match your server/CPUs).
Then try to optimise your IO subsystem by using low-latency disks (such as SSDs/2.5" SAS) in a hardware RAID 10 arrays, if possible with a dedicated array just for your logs.
I'm not aware of your budget, perhaps come back and tell us about what you're looking to spend and we can help.
Comment: The budget is about 2000-6000$ at the beginning. Operating system might be Linux.
Comment: Well in that case there's no reason for you not to go for something along the lines of a decent manufacturer (Dell, HP, IBM etc.) with one or two current model (56xx-series) Xeons with 4/6 cores plus hyperthreading each, 12-24GB of fast memory and 4-to-6 fast (15/10krpm SAS) disks in a RAID 10 array. Take a look at some models by these manufacturers and come back with a list of kit that you're interested in and I'll take a look and see if there are any problems or areas where you can improve ok :)
Comment: I am close to choose PowerEdge T710 Tower Chassis for Up to 8x 3.5" HDDs (6Gbps) and Intel 5500/5600 Series Processors with Intel Xeon E5506, 4C, 2.13GHz, 4M Cache, 4.80GT/s, 80W TDP, DDR3-800MHz and 12GB Memory for 1 CPU, DDR3, 1333Mhz (3x4GB Dual Ranked RDIMMs). Is this fine?
Comment: The T710 is fine but go for the 56xx series and don't go 'lower' than the 5620 as those before don't have hyperthreading. The rest of the spec is fine but make sure you get a hardware RAID controller and don't just buy those large, cheap 7200rpm disks, go for SAS ones if you have the budget, try for at least 4 disks in RAID 10 ok.
Comment: Thanks a lot, don't you think 12 GB is a bit too much for my needs? For this spec I can add 1 more processor in the future if needed? Aha meybe I wasn't clear before but this won't be production server but just the serwer to generate the website/counts/cache etc., so the spec is right here?
Comment: No, 12GB is just fine (3x4GB right?) and yes you can add another identical CPU but you'll have to add the same memory then too ok.
Comment: and how about operating system to this machine. Will Ubuntu 10.04 LTS be fine?
Comment: Personally I prefer one of the RHEL variants (i.e. RHEL, Centos, OEL) but it all comes down to what you're most comfortable with and therefore what's going to get the best support for you, so have no fear with Ubuntu if it's your kind of thing :)
Answer: You're very sparse on your details (operating system? version? budget?) but some general points that might help:

use a 64 bit OS on your 64 bit processor (32 bit will be slower for MySQL)
add as much memory as possible (machine and budget wise)
use the fastest disk possible (SSD is nice but they tend to fail / are expensive)
optimize your MySQL my.cnf for your specific needs (see mysqltuner.pl)
use current versions of your software

Other considerations:

linux tends to be the quickest OS for MySQL according to various benchmarks
Consider replacing MySQL with MariaDB, a drop-in replacement by original author of MySQL
Comment: With the mentioned budget of $2K to $6K I would look into clustering multiple database servers. That is if the setup you are using this for allows for it(!). Basic idea is you spread the READ (select queries) over multiple machines, lowering the load. The WRITE queries (UPDATE/INSERT/DELETE et al) are performed on the master(s) which are then synced by the slaves.


Given a 32 bit number, what is an efficient way to scale each byte by a certain factor?

Question: Given a uint32 number 0x12345678 (e.g., an RGBW color value), how could I efficiently and dynamically scale each byte in it (given a scaling factor 0 <= f <= 1 (or an equivalent integer divisor)?
I know I could do this a longer way (break the number into its components, perhaps via a struct, and loop to manipulate each in turn), but is there a way you to do it faster, without looping? (Static value mapping would be another way, but a dynamic method is preferable.)
Edit: C++ (C ideas are interesting as well), embedded, hundreds or thousands of pixels (not millions). Specifically scaling RGBW leds.
One other thing that came up - this is with gcc, so type punning is allowed (I already use it for similar things - I just wanted to see if there's a better way than that).
Edit again: This is for embedded platforms (microcontrollers). While I'm all for answers that help a wider audience, I purposely asked about this in the context of the language(s) and algorithms rather than optimizations for specific platforms and instruction sets, as platform-specific optimizations can vary if present at all.
Comment: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackoverflow.com/rooms/196190/discussion-on-question-by-martymacgyver-given-a-32-bit-number-what-is-an-effici).
Answer: The number of multiplications can be reduced by using the multiplications more effectively, on more "full" bits at once, not wasting as many bits on emptiness. Some padding bits are still needed to ensure that the product for one channel doesn't corrupt the result for an other channel. Using a 8bit fixed-point scale, and since there are 8 bits per channel, the output is 16 bits per channel, so two of them fit into the uint32_t side by side. That needs 8 bits of padding. So R and B (with 8 zeroes between them) can be scaled with one multiplication together, same for G and W. The result is the high 8 bits of the 16 bit result per channel. So something like this (not tested):
uint32_t RB = RGBW & 0x00FF00FF;
uint32_t GW = (RGBW >> 8) & 0x00FF00FF;
RB *= scale;
GW *= scale;
uint32_t out = ((RB >> 8) & 0x00FF00FF) | (GW & 0xFF00FF00);

The scale is a number from 0..256 which is interpreted as 0..1, in steps of 1/256. So scale = 128 corresponds to halving the channel values and so on.
It's possible to add a rounding step, just by adding a suitable bias after multiplying.
The multiplication does this, where the x results are not used:

Here is a quickbench to compare various scaling methods, from Timo in comments.
Comment: I see now, didn't notice you were using the MSB of the result instead of the LSB. [Here](https://godbolt.org/z/Nrsd17) is a little comparison of different methods. Your algorithm (_scale6_) wins by far (13 instructions on clang) even without any SIMD.
Comment: @4386427 because it's an integer between 0 and 255 now (well or 256 I suppose for "scale by 1")
Comment: If the scale is 0..1 shouldn't your method do a division instead of a multiplication? To quote OP: _"given a scaling factor 0 <= f <= 1 (or an equivalent integer divisor)"_
Comment: @Timo it doesn't work right. Using `dump` like that is not sufficient to force re-evaluation in a loop. Even with `benchmark::DoNotOptimize`, what ends up happening is that the result is computed once, and then stored to a variable a million times. Not sure what to do here. Though in any case, OP mentioned using embedded, so benchmarks on x64 may not be super relevant..
Comment: @Timo ok how's [this](http://quick-bench.com/16vXRyY9u4dDVf67vKBwP-LgvLQ)? (I put the `factor * 256` outside the loop which was my intent)
Comment: Here's what I would do: http://quick-bench.com/n_rG8b1OgWT5EHFPwH8UCHu-4SU
Comment: ...and even without type-punning... ;-)
Comment: Seems to me that there is something wrong with the calculation of the final result. Shifting RB will destroy the "result" bits. Further GW was shifted when initialized but not shifted when calculating the result.
Comment: @4386427 that's supposed to happen, the result bits are the high 8 bits of the 16 bit results, so in a way they implicitly got shifted left by 8. E: I'll sketch a diagram maybe that'll help
Comment: @harold hmm... how can the result be in the 8 high bits when the scaling factor is between zero and 1 ?
Comment: @Timo I took that as not knowing about this option (so OP tried to not limit the options, but accidentally did so anyway, depending on the interpretation), but scale is just `f * 256` so it's not any harder to use than if there was a divisor `1 / f`
Comment: @harold okay, I see.... I think the answer would be better if it included the calculation of `scale` from `f`. At least I got confused as I thought `scale` was in range [0..1].
Comment: @Timo adding `volatile` qualifiers to `dump` variables in benchmakrs functions, like [here](http://quick-bench.com/fK2HmGaiOg78gWFtF3fo4rfIe4w) changes drastically the execution time, so you're definitely right about function being evaluated only once. The result is that all 6 benchmarks perform equivalently now
Comment: @MaxLanghof ok thanks, so that's how that `DoNotOptimize` is supposed to be used then. I see it's re-evaluating the `f * 256` too now, so which option is a more accurate reflection of reality depends on how OP is actually using this, with the same scale for all pixels vs a changing scale
Comment: @Scheff: Type punning could improve performance if it were necessary to multiply pairs of pixels by the same value on a 64-bit processor.
Comment: You could do a single multiplication by expanding the input into one `uint64_t`, if the target has native 64 bit arithmetic that could be an improvement (or equally it might not - from what I can tell that would require an additional shift).
Comment: To be clear, as @harold and others understood, this means transforming the original range [0-255] to some smaller integer range starting at zero (it's easy enough to offset if needed). So, a factor of 50%, 0.5, 1/2, 128/256 or such scales [0-255] to [0-127]. Trivial scalings are easy - I was curious if there's a more general algorithm and as this is embedded, processor- and compiler-specific hacks wouldn't be very portable (gcc punning being a handy exception: it seems to be present on all the platforms I'd use). Native 64-bit processing on such platforms is uncommon.
Answer: You can directly calculate the power-of-two fractions of the input values with shifts and masks:
unsigned long src_2 = ((src >> 1) & 0x7f7f7f7fUL) + (src & 0x01010101UL);
unsigned long src_4 = ((src >> 2) & 0x3f3f3f3fUL) + ((src >> 1) & 0x01010101UL);
unsigned long src_8 = ((src >> 3) & 0x1f1f1f1fUL) + ((src >> 2) & 0x01010101UL);
unsigned long src_16 = ((src >> 4) & 0x0f0f0f0fUL) + ((src >> 3) & 0x01010101UL);
unsigned long src_32 = ((src >> 5) & 0x07070707UL) + ((src >> 4) & 0x01010101UL);
unsigned long src_64 = ((src >> 6) & 0x03030303UL) + ((src >> 5) & 0x01010101UL);
unsigned long src_128 = ((src >> 7) & 0x01010101UL) + ((src >> 6) & 0x01010101UL);
unsigned long src_256 = ((src >> 7) & 0x01010101UL);

(Here src_2 is src with each field individually divided by 2, src_4 is src with each field individually divided by 4 and so on).
Any of the other fractions from 0/256 through 255/256 can be made through optionally adding each of these values (eg 0.75 is src_2 + src_4).  This might be useful if your embedded system does not have a fast multiplier (you can precalculate the necessary masks from the scaling factor once before processing all pixels), or if you only really need a limited set of scaling factors (you can just hardcode the combinations of power-of-two-fractions you need into a set of specialised scaling functions).
For example a specialised scale-by-0.75 function in its inner loop would just do:
dest = ((src >> 1) & 0x7f7f7f7fUL) + (src & 0x01010101UL) +
    ((src >> 2) & 0x3f3f3f3fUL) + ((src >> 1) & 0x01010101UL);

Although not applicable to your use case, this method can also be used to precalculate masks that apply different scaling factors to each component of the vector as well.
Answer: It has been mentioned in the discussion that the optimal solution can be
architecture specific. Someone also suggested to code it in assembly.
Assembly has a cost in terms of portability, but it also begs the
question of whether (and by how much) you can beat the compiler's
optimizer.
I did an experiment on an Arduino, which is based on an AVR
microcontroller. This is a very limited 8-bit, Harvard, RISC MCU, with
an 8 × 8 → 16-bit hardware multiplier.
Here is the straightforward implementation, using type-punning to
multiply the individual bytes:
static inline uint32_t scale_pixel(uint32_t rgbw, uint16_t scale)
{
    union {
        uint32_t value;
        uint8_t bytes[4];
    } x = { .value = rgbw };
    x.bytes[0] = x.bytes[0] * scale >> 8;
    x.bytes[1] = x.bytes[1] * scale >> 8;
    x.bytes[2] = x.bytes[2] * scale >> 8;
    x.bytes[3] = x.bytes[3] * scale >> 8;
    return x.value;
}

Compiled with gcc at -Os (typical in these memory-constrained devices)
this takes 28 CPU cycles to execute, i.e. 7 cycles per byte.
The compiler is smart enough to allocate rgbw and x to the same CPU
registers and thus avoid a copy.
Here is the version based on harold's answer:
static inline uint32_t scale_pixel(uint32_t rgbw, uint16_t scale)
{
    uint32_t rb = rgbw & 0x00FF00FF;
    uint32_t gw = (rgbw >> 8) & 0x00FF00FF;
    rb *= scale;
    gw *= scale;
    uint32_t out = ((rb >> 8) & 0x00FF00FF) | (gw & 0xFF00FF00);
    return out;
}

This is a very smart optimization that is likely to pay off on a 32-bit
MCU. However, on this small 8-bitter, it took 176 CPU cycles to
execute! The generated assembly features two calls to a library function
that implements a full 32-bit multiplication, along with many moving and
clearing registers.
Lastly, here is my inline assembly version:
static inline uint32_t scale_pixel(uint32_t rgbw, uint16_t scale)
{
    asm(
        "tst %B[scale]           \n\t"  // test high byte of scale
        "brne 0f                 \n\t"  // if non zero, we are done
        "mul %A[rgbw], %A[scale] \n\t"  // multiply LSB
        "mov %A[rgbw], r1        \n\t"  // move result into place
        "mul %B[rgbw], %A[scale] \n\t"  // same with three other bytes
        "mov %B[rgbw], r1        \n\t"  // ...
        "mul %C[rgbw], %A[scale] \n\t"
        "mov %C[rgbw], r1        \n\t"
        "mul %D[rgbw], %A[scale] \n\t"
        "mov %D[rgbw], r1        \n"
        "0:"
        : [rgbw] "+r" (rgbw)   // output
        : [scale] "r" (scale)  // input
        : "r0", "r1"  // clobbers
    );
    return rgbw;
}

This one uses the fact that the scale factor can be no larger than 256.
In fact, any factor greater than 256 is treated as 256, which could be
considered a feature. The execution takes 14 cycles, and only
3 cycles if the scale is 256.
Summary:

176 cycles for the version optimized for a 32-bit core
28 cycles for the naive type-punning version
14 cycles for the assembly version

My conclusion from this experiment is that you are looking here at the
kind of micro-optimization where architecture really matters. You can't
seriously try to optimize this at the C level without any assumption
about the architecture it will run on. Also, if a factor 2 in the speed
matters to you, it is worth trying an implementation in assembly. Use
conditional compilation to enable the asm implementation in the
targeted architecture, and fall back to a generic C implementation in
any other architecture.


How to modify a parameter's value in sass files from typescript?

Question: I am trying to change a variable's value in "../myStyle.scss" file from myComponent.ts . I read that it is impossible to achieve it because sass file are compiled into css by the sass pre-processor and the variable disapears. Isn't there a way to work this around.
My variable is called $theme.
And I want to change it's value.
$theme: #5A352A;
and I want it to become $theme: #ffffff; when the user clicks
Comment: Possible duplicate of [How to control Sass Variable with javascript](https://stackoverflow.com/questions/17787845/how-to-control-sass-variable-with-javascript)
Comment: Refer to this post : https://stackoverflow.com/questions/17787845/how-to-control-sass-variable-with-javascript
Answer: You have to work in a different way.
Basically when you compile the angular app , it will generate a css file where it substitute the variable with the value , wherever you used it.
So if you want to achieve a color change you have to create a other variable and other classes and swipe it in your class attributes (this is one way, check also ngStyle and ngClass in angular reference).
For example white-theme/dark-theme (the most common case).
variables ->   $black: #00000; $white: #ffffff
Example classes:
.body-dark {
 background-color: $black;
}

.body-white {
 background-color: $white;
}

and swipe the classes in the html elements.
setDark(){
document.getElementById("bodyId").setAttribute("class","body-dark ")
}

the same for white.
Comment: Thank you. I already know this way but i find it very heavy everything that you want to change on your screen must have this attribute. And it gets heavier when your theme has many colors
Comment: maybe you can work with ngStyle and ngClass , could help to find a solution that fit your needs https://codecraft.tv/courses/angular/built-in-directives/ngstyle-and-ngclass/


Manipulating popover to contain text and html in different div inside data-content

Question: I am using twitter bootstrap to implement popover. I know that I can pass html: true during invocation if I need to have data-content title rendered as html. But if I have different div inside data-content or title and I want different behaviors for those div(some to be rendered as html and some as text), then how should I implement it.
eg. :
data-content=
"

Rendering As Text

Rendering As Html

"

And result should look like below:
Rendering As Text

Rendering As Html
Answer: Use event bootstrap popover shown and jquery function text to render content as text:
$(function () {
    $("[data-toggle=popover]").popover().on("shown.bs.popover", function () {
        var html = $('#RenderAsText').html();
        $('#RenderAsText').text(html);
    });
});

JSFIDDLE


How to build conditional formattings with script?

Question: I'm using the google script to manage the google admin. Below is the script to list all google users in the domain.
If I create conditional formatting directly on the sheet, the rules are deleted whenever the script is launched. That's why I'd like to include the conditional formatting within the script itself, but it's not working?.
Basically what I want are :

if the value of the columns 'is Super Admin?' or 'is Delegated Admin?' are TRUE, then I'd like the rows to be colored in bold red.
And I'm trying to find out how to convert the results of lastLoginTime and creationTime (which are in string ISO8601 to datevalue, I know how to do it using formula, but I don't know how to using the script)

I'd really appreciate it if anyone can guide me.
Many many thanks in advance.

function listAllUser() {
  var sheet11 = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("SheetNAME");
  var sheet11range = sheet11.getRange("I3:Q")
  sheet11range.clear()

  var data = [];// array to store values
  data.push(['Last Name','First Name','Email','orgUnitPath', 'is Suspended?','is Super Admin?','is Delegated Admin?', 'LastLoginTime','Creation']);// store headers
  var pageToken, page;

  do {
    page = AdminDirectory.Users.list({
      domain: 'DomainNAME',
      pageToken: pageToken,
    });
    var users = page.users;
    if (users) {
      for (var i = 0; i < users.length; i++) {
        try{
        var user = users[i];
        data.push([user.name.familyName,user.name.givenName,user.primaryEmail,user.orgUnitPath, user.suspended,user.isAdmin,user.isDelegatedAdmin, user.lastLoginTime , user.creationTime  ]);//store in an array of arrays (one for each row)}
    }catch (e){}
       }}
    pageToken = page.nextPageToken;
  } while (pageToken);
  sheet11.getRange(2,9,data.length,data[0].length).setValues(data).setVerticalAlignment("middle").setWrap(true);
  var t1 = sheet11.getRange("I1:P1").merge();
  var t1bis = t1.setValue("List of all USERS");
  var dated = sheet11.getRange("Q1")
  dated.setValue("Last updated: " + Utilities.formatDate(new Date(),Session.getScriptTimeZone(),'dd-MM-yy hh:mm'));
  var range = sheet11.getRange("I1:Q2");
  range.setFontWeight("Bold").setFontColor("Blue").setHorizontalAlignment("center").setVerticalAlignment("middle").setBackground("#beedda").setWrap(true);

  var isSuperAdm = sheet11.getRange("N3:N").getValue();
  var isDelegatAdm = sheet11.getRange("O3:O").getValue();
  if(isSuperAdm==true || isDelegatAdm==true){
    range.setFontWeight('bold').setFontColor('red')
  }

}
Answer: After looking around for some tutorials, I finally succeeded to have what I wanted, formating the cell :

when the lastLoginTime was before the specific date
when the value of 'is Super Admin?' and 'is Delegated Admin?' are true

function cellsFormatting(){
  var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("SheetName");
  var range1 = sheet.getRange("A1:Q2");
  range1.setFontWeight("Bold").setFontColor("Blue").setHorizontalAlignment("center").setVerticalAlignment("middle").setBackground("#beedda").setWrap(true);
  var range2 = sheet.getRange ("A3:Q");
  range2.setVerticalAlignment('middle').setWrap(true);
}

function setRangeToRedBoldFont(cellRange){
  var sheet = SpreadsheetApp.getActiveSheet();
  var range = sheet.getRange(cellRange);
  var rule = SpreadsheetApp.newConditionalFormatRule()
    .whenTextEqualTo("TRUE")
    .setFontColor("#FF0000")
    .setBold(true)
    .setRanges([range])
    .build();
var rules = sheet.getConditionalFormatRules();
rules.push(rule);
sheet.setConditionalFormatRules(rules);
}

function setRangeToGrayItalicFont(cellRange){
  var sheet = SpreadsheetApp.getActiveSheet();
  var range = sheet.getRange(cellRange);
  var rule = SpreadsheetApp.newConditionalFormatRule()
    .whenFormulaSatisfied("=DATEVALUE(MID($P3;1;10)) + TIMEVALUE(MID($P3;12;8))<date(2018;8;31)")
    .setFontColor("#A9A9A9")
    .setItalic(true)
    .setRanges([range])
    .build();
var rules = sheet.getConditionalFormatRules();
rules.push(rule);
sheet.setConditionalFormatRules(rules);
}

function listAllUser() {
  var sheet11 = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("SheetName");
  var sheet11range = sheet11.getRange("I3:Q")
  sheet11range.clear();
  cellsFormatting();

  var t1 = sheet11.getRange("I1:P1").merge();
  var t1bis = t1.setValue("List of all USERS");
  var dated = sheet11.getRange("Q1")
  dated.setValue("Last updated: " + Utilities.formatDate(new Date(),Session.getScriptTimeZone(),'dd-MM-yy hh:mm'));  //will input the date & time when the script was launched

  var data = [];// array to store values
  data.push(['Last Name','First Name','Email','orgUnitPath', 'is Suspended?','is Super Admin?','is Delegated Admin?', 'LastLoginTime','Creation']);// store headers
  var pageToken, page;

  do {
    page = AdminDirectory.Users.list({
      domain: 'DomainNAME',
      pageToken: pageToken,
    });
    var users = page.users;
    if (users) {
      for (var i = 0; i < users.length; i++) {
        try{
        var user = users[i];
        data.push([user.name.familyName,user.name.givenName,user.primaryEmail,user.orgUnitPath, user.suspended,user.isAdmin,user.isDelegatedAdmin, user.lastLoginTime , user.creationTime  ]);//store in an array of arrays (one for each row)}
    }catch (e){}
       }}
    pageToken = page.nextPageToken;
  } while (pageToken);
  sheet11.getRange(2,9,data.length,data[0].length).setValues(data);

  var formatRange = "I3:Q";
  setRangeToRedBoldFont(formatRange);
  setRangeToGrayItalicFont (formatRange);
}
Answer: For the row color:
For user.isAdmin & user.isDelegatedAdmin
function colorLine() {
  const ss=SpreadsheetApp.getActive();
  const sh=ss.getSheetByName('SheetName');
  const rg=sh.getRange(3,9,sh.getLastRow(),9);
  const vs=rg.getValues();
  vs.forEach((r,i)=>{
    if(r[5]||r[6]) {
      sh.getRange(i+3,1,1,sh.getLastColumn()).setFontWeight('bold').setFontColor('red');
    }
  });

}


Quadratic Variation of Brownian Motion

Question: Quadratic Variation of a Brownian motion $B$ over the interval $[0,t]$ is defined as the limit in probability of any sequence of partitions $\Pi_n([0,t])=\{0=t^n_0<\cdots<t^n_{k(n)}=t\}$ of the interval $[0,t]$ such that $\lim_{n\to \infty}\max_{i=1,\cdots,k(n)}|t^n_i-t^n_{i-1}|=0$, of the functional
$$V([0,t],\Pi_n)(B_.)=\sum_{i=1}^{k(n)}(B_{t_{i-1}}-B_{t_i})^2.$$
And for any such sequence of partition we have then $[B]_t=P-\lim_{n\to \infty} V([0,t],\Pi_n)(B_.)=t$.
Nevertheless when you take the sup over all finite partitions of $[0,t]$ then it is a known fact that almost surely $\sup_{\Pi\in \mathrm{partition}([0,t])} V([0,t],\Pi)(B_.)=+\infty$.
I have never been able to derive this fact properly and in every details.
I'd be really gratefull if anyone could take the time to provide a detailed proof of this fact.
Comment: In addition to the answer below, these MIT lecture notes [here](https://ocw.mit.edu/courses/sloan-school-of-management/15-070j-advanced-stochastic-processes-fall-2013/lecture-notes/MIT15_070JF13_Lec8.pdf) are also pretty decent.
Answer: You can find a short proof of this fact (actually in the more general case of Fractional Brownian Motion) in the paper :
M. Prattelli :  A remark on the 1/H-variation of the Fractional Brownian Motion.  Probability Seminar Vol. XLIII (pdf)
Comment: IMO, this should be added to every stochastic calculus book when talking about quadratic variation of Brownian Motion.
Comment: @TheBridge : Yes, I agree. Also if you are interested in a more precise result for BM, i.e. the "largest" $\varphi$ s.t. BM has finite $\varphi$-variation a.s., you can find in the paper "Exact asymptotic estimates of Brownian path variation",SJ Taylor, Duke Math. J. Volume 39, Number 2 (1972), 219-241.
Comment: @ pgassiat : Thank's this looks promising let me take a look at it.
Comment: Excellent proof !!! And note that for BM lemma 1 simplifies further as it is only Strong Law of Large Numbers (Brownian increments are i.i.d. in this context). Really thank you so much, I have been looking for such a detailed proof for years !!!
Comment: @The Bridge : You're welcome.
Comment: @thebridge thanks for the link. I can't agree with you strongly enough that it should be in every stochastic calculus book. It was so confusing at first. Thank you so much.
Comment: The link seems to have changed - I believe https://people.dm.unipi.it/pratelli/research/FBM.pdf is the same thing.
Answer: The book Brownian Motion - An Introduction to Stochastic Processes by René Schilling & Lothar Partzsch contains a (detailed) proof of this fact.

Quadratic Variation of Brownian Motion


Sorting using two arrays

Question: There are two arrays:
The first one contain numbers, and the second one contains "weight" of the first array values.
It works like this:
arr1 = [56,65,100,89,180,90];

"Weight" of the numbers are calculated in this way:
 56 = 5+6 = 11;
 65 = 6+5 = 11;
 100 = 1+0+0 = 1; and so on..

So, arr2 = [11,11,1,17,9,9];
My question is how can I sort the values of the arr1 according to values of arr2?
I tried to modify simple bubble sort for this problem, but nothing changed.

function bubble(arr1, arr2) {
  var len = arr1.length;
  for (var i = 0; i < len; i++) {
    for (var j = 0; j < len - i - 1; j++) {
      if (arr2[j] > arr2[j + 1]) {
        var temp = arr1[j];
        arr1[j] = arr1[j + 1];
        arr1[j + 1] = temp;
      }
    }
  }
  return arr1;
}

arr1 = [56, 65, 100, 89, 180, 90];

arr2 = [11, 11, 1, 17, 9, 9];

console.log(bubble(arr1, arr2));

I expect the output of the bubble function to be [100,180,90,56,65,89].
This is why:
FirstArray -          [56,65,100,89,180,90] - arr1
"Weight of the values"[11,11, 1, 17, 9, 9 ] - arr2
Output                [100,180,90,56,65,89]
                      [1,  9,  9 ,11,11,17]

Comment: Possible duplicate of [Javascript - sort array based on another array](https://stackoverflow.com/questions/13304543/javascript-sort-array-based-on-another-array)
Comment: You have to sort *both arrays*, not just one. Otherwise the arrays start to missmatch and you get chaos
Comment: @JonasWilms if second array is derived from first ,  then sorting arr1 alone works -  https://codepen.io/nagasai/pen/WNeoaEv?editors=1010
Comment: Why not just supply a comparison function and compute the "weight" during the sort?
Answer: You can just calculate the weight on the fly while sorting:

const arr = [56,65,100,89,180,90];

arr.sort( (a,b) =>
  (a + '').split( '' ).reduce( (sum,x) => sum + +x, 0 ) -
  (b + '').split( '' ).reduce( (sum,x) => sum + +x, 0 )
);

console.log( arr );
Answer: To achieve expected result, use below option of sorrting arr1 with logic mentioned in question

Sort arr1 bycomparing sum of digits
Convert number to string and split it to array
Use reduce sum all the digits and compare with the array elements

var arr1 = [56, 65, 100, 89, 180, 90];
var arr2 = [11, 11, 1, 17, 9, 9];

console.log(
  arr1.sort(
    (a, b) =>
      a
        .toString()
        .split("")
        .reduce((acc, v) => parseInt(acc) + parseInt(v)) -
      b
        .toString()
        .split("")
        .reduce((acc, v) => parseInt(acc) + parseInt(v))
  )
);

codepen - https://codepen.io/nagasai/pen/WNeoaEv?editors=1010
Comment: My thanks! I just feel a little bit weak in arrow functions, so I need more practice.

Sorting using two arrays


How do I configure Araxis Merge for use with Visual Studio Git?

Question: I have to use Git with Visual Studio 2017 now, and there doesn't seem to be a way to link Araxis Merge with Visual Studio / Git.
This explains how to link Visual Studio TFS with Araxis.
This explains how to link Git with Araxis.
But how to link Visual Studio 2017 Plug-in Selection (with Git) to Araxis?
Answer: If you're running Visual Studio 2019 Enterprise (as I am--this might work for other versions of Visual Studio that use built-in Git for source control):

Navigate to the folder that contains the global .gitconfig file, which is at:
%HOMEDRIVE%%HOMEPATH% (your user folder).
Find .gitconfig there and edit it with Notepad.exe, or some other text editor.

You're going to be looking for four key sections: [diff], [difftool], [merge], and [mergetool].

[diff]: Tells Visual Studio Git which diff tool to launch when you
compare one version of a file with another.
[difftool]: Specifies the name of the diff tool as well as the commandline to execute to launch that tool.
[merge]: Tells Visual Studio Git which merge tool to launch when you merge one file with another.
[mergetool]: Specifies the name of the merge tool as well as the commandline to
execute to launch that tool.

I suggest removing your existing [diff], [difftool], [merge], and [mergetool] sections and replacing them so that your gitconfig file looks something like this:
[user]
    name = <your name>
    email = <your email address>
[diff]
    tool = araxisdiff
[difftool "araxisdiff"]
    cmd = \"C:\\Program Files\\Araxis\\Araxis Merge\\Merge.exe\" \"$LOCAL\" \"$REMOTE\"
[merge]
    tool = araxismerge
[mergetool "araxismerge"]
    cmd = \"C:\\Program Files\\Araxis\\Araxis Merge\\Merge.exe\" \"$REMOTE\" \"$LOCAL\" \"$BASE\" \"$MERGED\"
[difftool "vsdiffmerge"]
    cmd = \"C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\Common7\\IDE\\CommonExtensions\\Microsoft\\TeamFoundation\\Team Explorer\\vsdiffmerge.exe\" \"$LOCAL\" \"$REMOTE\" //t
    keepBackup = false

What's going on is rather straightforward: The [diff] section specifies which of the [difftool] entries will be used for file compares; the [merge] section specifies which of the [mergetool] entries will be used for file merges. In the above case, I specified araxisdiff as my diff tool and araxismerge as my merge tool. If I wanted to use the built-in Visual Studio diff/merge tool, I could change the tool =  statement for each so that it specifies vsdiffmerge instead.
Comment: I strongly felt that we needed some better-quality help posted here for this important topic. I happen to think Araxis Merge is the best diff program on the market, and shame on Araxis for not having a section on its website to specifically address the configuration needs of us Visual Studio enthusiasts and professionals. So glad I could help!
Answer: In the current source control plug-in drop-down, select Visual Studio Team Foundation.

You can follow the link that you mentioned in the question to setup araxis.
If you still can't, it's because your source is being controlled by Git instread for Team Foundation. You need to update git config use araxis as diff and merge too. You can find your git config file in ur repo under '.git/config'.
Sample below.

[diff]
    tool = araxis
[merge]
    tool = araxis
[mergetool "araxis"]
        path = C:/Program Files/Araxis/Araxis Merge/Compare.exe
[difftool "araxis"]
        path = C:/Program Files/Araxis/Araxis Merge/Compare.exe
Comment: How will that link it to Araxis??
Comment: the link you mentioned in your question shows that.
However, it still doesnt work, it's very likely your repo git config file has some different diff and merge tool. In my case it was vsdiffmerge. I had the got config file diff and merge to araxis
Answer: In Visual Studio 2019 you need to edit at minimum your global .gitconfig and if needed the local to the project/solution .gitconfig.
Also my latest version of of araxis installed itself into a location other than Program Files but in appdata. Uggg

Here is what I had to use in both configs:
[diff]
tool = araxis
[difftool "araxis"]
path = C:\\Users\\{my UserName}\\AppData\\Local\\Apps\\Araxis\\Araxis Merge\\compare.exe

[merge]
tool = araxis
[mergetool "araxis"]
path = C:\\Users\\{my UserName}\\AppData\\Local\\Apps\\Araxis\\Araxis Merge\\compare.exe

If it works then you see a grayed out araxis and the Use Visual Studio is selectable which at first glance is counter intuitive (IMHO).
This is what you see in config:


The Activity image field is required in CodeIgniter

Question: I'm trying to upload image in my code project. In the image field I'm getting

"The Activity image field is required."

The below code is my Activity_controller
public function create_activity(){
//...Upload the image
//... path to upload image
  $config['upload_path'] = './assets/images/uploads'; // The path to upload the image
  $config['allowed_types'] = 'gif|jpg|jpeg|png|pdf|doc';
  $config['max_size'] = '2048';
  $config['max_width'] = '500';
  $config['max_height'] = '500';

  $this->load->library('upload',$config);

  if(!$this->upload->do_upload()) {
    $error = array('error' => $this->upload->display_errors());
    $activity_image = 'noimage.jpg';
  } else {
    $data = array('upload_data' => $this->upload->data());
    $activity_image = $_FILES['userfile']['name'];
  }

  $this->activity_model->save_activity($activity_image);
  redirect('activity_controller');
}

Activity model, insert activity in the database    Here is only the function to insert data in the database.
public function save_activity($activity_image){
//... code to insert others data in the database

  'activity_image' => $activity_image
                            );
                            return $this->db->insert('activity', $data);
                        }

Below code is create_activities.php file in the view folder:
                    <?php echo validation_errors(); ?>
                    <?php echo form_open_multipart('activity_controller/create_activity');  ?>

                        Upload image
                        <input name="userfile" type="file">

I have loaded the model in the autoload.php file.
I also used <input name="activity_image type="file">
"image_name" is the name of the field in the database table
Please someone can help me to solve this problem? I will appreciate your help
Thank you in advance
Answer: I think you forgot to add (enctype="multipart/form-data") in your view file in form Tag.
Here, Usen this code. I hope this one is help full to you.
<form action="/action_page_binary.asp" method="post" enctype="multipart/form-data">
  First name: <input type="file" name="name">

  <input type="submit" value="Submit">
</form>


Is ANOVA good for non-normally distributed series?

Question: I just read Finding Seasonal Spreads, by Paul Teetor.
As far as I understand how ANOVA works, it should be accurate for NORMAL distributed series. Financial series are not normally distributed, or maybe am I wrong?
Could someone explain why he used ANOVA in this case?
Comment: [CLT](http://en.wikipedia.org/wiki/Central_limit_theorem): large sample sizes make the means normally distributed.
Comment: If your sample sizes are large enough, the CLT will alleviate the need for normality of your variables themselves: it is really only the means of samples that need to be distributed normally.
Comment: @NickSabbe what do you mean with CLT? large sample size means that sereis are normally distributed? could you exmplain this point? thanks....
Comment: @NickSabbe wait, I have a doubt, using ANOVA i have to subdivide the series in groups, so those groups doesnt have a very large size. what do you think about it?
Comment: Yes, you do need normality of the means _per group_. Note that one typically accepts sample sizes of 30 as "large" enough for CLT (though opinions differ).
Comment: let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/1750/discussion-between-dail-and-nick-sabbe)
Comment: @NickSabbe one question about it... if i have a series with 800 numbers and i create 4 groups with 200 numbers each. Does it mean that i could "assume" are normal distributed because 200 > 30 (where 30 is a number that we can take to consider a serie, large or not) Right?
Answer: He used ANOVA as "one way to identify seasonal spreads using legitimate statistical techniques."
However, you must consider the limitation of the analysis (See limitation on http://quanttrader.info/public/findingSeasonalSpreads.html).
As is known, ANOVA is robust enough to deal with non normal data on large sample sizes (CLT). Nevertheless, ANOVA will give analysts information about the difference on the "levels" of the curves if the data have a seasonal behavior. Then, there are other tools that give analysts more information than ANOVA in such cases: ARIMA models, bootstraping, and so on.
Comment: @Dail ANOVA doesn't test, despite its name, difference of variances. It tests for differences of means of groups. Fligner-Kileen will test for differences of variances of the groups.
Comment: Should Fligner.test be better then ANOVA in testing difference of variance?
Comment: Could you give me an example using ARIMA model (with R if you know) to check if the variance is constant?


1-d extension of 2-d variance?

Question: A random variable X={X1..Xm} and Var(X) is the mXm variance-covariance matrix.  Is there an accepted 1-d statistic like variance that may be extracted from the sample variance of data representing a 2-d random variable?
(explanation of the 2d equivalent of a sample variance)
How to find variance between multidimensional points?
Comment: The thing which is comparable is the $2\times 2$ variance/covariance matrix.  You could take the mean of the square of the Euclidean distances between values and their mean, but that is unlikely to be as informative in most circumstances.
Comment: The thrust of our comments, Will, is that because there are myriad ways to summarize an $n$ by $n$ matrix with a single number, then in order to answer this question we need to know *why* you need just a single number and how you intend to interpret it.  Please edit your question to clarify.
Comment: What do you want this statistic to do or represent?
Comment: Something comparable to the 1-d variance.  In other words.. if we have n individuals with m data each, what is the total variance of that sample?
Comment: It's not clear what you want. Are you after something like the variance of the first principal component?
Comment: Sorry, I'm not being clear.  The n measurements would determine the variance of each of m PC's, if you like.  So X=[X1...Xm] and I believe that Var(X) is the mXm (Variance-Covariance Matrix).. as seen in the answers to the other question above.
I want to know if there is a standard, accepted way way of collapsing that mXm matrix into a single value in R.  For example would it be legitimate to take a weighted mean of the separate variances for each of the Xi's i.e. trace(Var(X))/m.  Or is there a better way?
Comment: @whuber, the three meaningful summaries that I can think of are all based on the spectrum of this matrix, and are commonly used in multivariate analysis: $l_\infty$, the largest eigenvalue; $l_1$, trace of the matrix; and an analogue of the entropy, the determinant of the matrix. If you reopen the question, I'd be happy to post this as an answer; I don't think it is an unreasonable questions per se.
Comment: @StasK It might be a reasonable question on [math.se], where nobody cares what anything means or how it is applied, but on this site we need more information, because the question asks for a *statistic* and statistics have a *purpose*: they are used to compare things in particular ways.  In the absence of any context, we are left having to guess what comparison is intended.  The OP only needs to give us a glimmer of their intention for this question to become answerable; surely there is no harm in waiting for that information.
Comment: The variance is the expected variation about the mean.  My question is if there is a 2-d equivalent of the variance.  Implicitly I want to know if there is something similar to the first central moment as well.
In response to the question about application:  I have m PC vectors of length n (one for each of n individuals).
Answer: I don't believe there is an answer to your question. You can't scale down a variance-covariance matrix. If you did (say by taking the determinant) you will lose all the usefull information -- it would no longer be explaining the variance.
As suggested in the comments, you can run a Principal Component Analysis to reduce the dimension of your data to one. Then the inertia (not the variance) of the first component can provide a variance-like summary of the full data.
However, this is different from the variance because you will be calculating the variation with respect to a new calculated principal component and not the original variables. It is therefore no longer the variance, but the inertia.
Comment: Thanks Drew and userZ/6. I'd suspected something like that.
Answer: I guess it really depends on what you are doing. If your are investigating the relationship between the two variables, the answer would be the covariance. (off-diagonal element in your 2x2 matrix.)
In case you are trying to track the uncertainty of estimates from 2 different systems using their covariance matrices, you could use:
$det(\Sigma)$, or $tr(\Sigma)$, where $\Sigma$ is the covariance matrix.
i.e. the determinant of the covariance matrix is a 1D measure to track uncertainty, and there is some theory about how and why it might make sense. Intuitively, the det tells you by how much you would the cov matrix scale a space if applied as a linear transformation.
Comment: Just read Drew's answer. He is right on the information loss.Having said that, in the end, what is "accepted" as measure really depends on the field you work in. I would refrain from re-inventing the wheel. Look at what papers in your field are using and do that. For SLAM research in robotics, the det and trace are being used as optimality criteria for a long time with varying success and Google returns many papers to prove this.
Comment: Very interesting information. I'

1-d extension of 2-d variance?


How to access the value of a variable of a specific instance of GameViewController in Swift

Question: Here's my relevant code:
from my GameScene class:
    func resetCircleAndScore(scale: CGFloat)
{
    circleIsStationary = true
    currScore = 0
    CIRCLE.speed = 1
    CIRCLE.physicsBody!.velocity = CGVector(dx: 0,dy: 0)
    CIRCLE.position = CGPointMake(midX, midY)
    circleScale = scale
    circleSpeed = CONSTSPEED
    CIRCLE.setScale(circleScale)
    currScoreLabel.text = "SCORE: " + Int(currScore).description
    //need to update modeSelected here
}

from my GameViewController class:
public func pickerView(pickerView: UIPickerView, didSelectRow row: Int, inComponent component: Int)
{
    modeCopy = modes[row]

    println(modeCopy)
}

I tried to add a getter for modeCopy, as follows:
public func getMode() -> String
{
    return modeCopy
}

The problem is that when I try to call this function from another class, it needs a parameter called "self" of type GameViewController. What I really want to do is access the instance of GameViewController that created my GameScene. I could also access the instance of GameScene the GameViewController created, but I'm not sure how to do that. My ViewDidLoad in GameViewController is below.
override public func viewDidLoad()
{
    self.pickerView.dataSource = self
    self.pickerView.delegate = self

    println("got to beginning of viewDidLoad")
    super.viewDidLoad()

    if let scene = GameScene.unarchiveFromFile("GameScene") as? GameScene
    {
        // Configure the view.
        let skView = self.view as SKView
        let showStats: Bool = true
        skView.showsFPS = showStats
        skView.showsNodeCount = showStats

        /* Sprite Kit applies additional optimizations to improve rendering performance */
        skView.ignoresSiblingOrder = true

        /* Set the scale mode to scale to fit the window */
        scene.scaleMode = .AspectFill

        skView.presentScene(scene)
    }
}

Comment: Is GameScene your own class? Or do you have source code for it?
Comment: It is my class, and I already put the relevant code.
Answer: If GameScene is your own class you could add one optional parameter of type GameViewController.
class GameScene {
    weak var parentController: GameViewController?

....
}

In viewDidLoad() method when you unarchive GameView do this
scene.parentController = self

Now you could use it in the GameScene


Steroid for IBD banned by WADA?

Question: I have IBD (inflammatory bowel disease), and so my colon as well as cecum get inflamed, and to keep me functional, my doctor has advised me to take prednisolone steroid to reduce swelling of my intestine and cecum. Also, I am on some NSAIDs (Nonsteroidal anti-inflammatory drug).
Are these against WADA rules? Will I be banned from WADA-approved sports events?
Answer: Talk to your governing body, not the Internet. They will be able to give you accurate advice as to exactly what is and isn't a prohibited substance and the possibility of you obtaining a therapeutic use exemption (TUE).
Answer: It is on the WADA prohibited list as being prohibited "in competition", that doesn't mean you couldn't get a Theraputic Use Exemption (TUE) for it. TUE's for corticosteriods used to treat IBD are definitely a thing.
In general terms there's 4 criteria which must all be met for a TUE:

The athlete would experience significant health problems without taking the prohibited
substance or method;
The therapeutic use of the substance would not produce significant enhancement of
performance;
There is no reasonable therapeutic alternative to the use of the otherwise prohibited
substance or method;
The requirement to use that substance or method is not due to the prior use of the
substance or method without a TUE, which was prohibited at the time of use.

You don't submit a request for a TUE to WADA yourself - it's done through which ever Anti-Doping Organisation (ADO) that governs your region/sport. Although your first port of call (as @Philip Kendall says) should probably be the appropriate governing body for the sport

Steroid for IBD banned by WADA?


SQL counts with left join

Question: On my Microsoft SQL Server 2016 database I'm trying to determine how many labs (Lab_Space table) have had an assessment (EHS_Assessment_Audit table) done within the last year, grouped by location (Locations table).  It's common to have more than one assessment done per year per lab.
Everything I've tried so far results in more "done" counts than labs.  For example:
WITH cte AS
(
    SELECT DISTINCT
        Lab_Space_Id
    FROM
        EHS_Assessment_Audit
    WHERE
        Audit_Date >= DATEADD(year, -1, GETDATE())
)
SELECT
    l.Site_Name, l.Campus_Name,
    COUNT(DISTINCT s.id) Total,
    SUM(CASE WHEN a.Lab_Space_ID IS NOT NULL THEN 1 ELSE 0 END) Audited
FROM
    Lab_Space s
LEFT OUTER JOIN
    cte a ON s.id = a.Lab_Space_Id
JOIN
    Locations l ON l.Building_Code = s.Building_Code
GROUP BY
    l.Site_Name, l.Campus_Name
ORDER BY
    l.Site_Name, l.Campus_Name

The cte there should get me a unique list of labs that have had an assessment done, and then I'm trying to count that grouped by location.  I'm ending up with output though where it'll say there are 178 total and 1080 audited for a single site/campus combo.
Comment: You have enough rep around here to know that you need to post sample data. http://spaghettidba.com/2015/04/24/how-to-post-a-t-sql-question-on-a-public-forum/
Comment: Posting a broken query without any other information will not help us help you.
Answer: I think using a CTE in this case is going to be more trouble than it's worth.  A subquery is going to be easier to read and modify.  For example:
SELECT
    l.Site_Name,
    l.Campus_Name,
    COALESCE(b.NumAudits, 0) as NumTotalAudits,
    COALESCE(b.NumLabs, 0) as AuditedLabs
FROM Locations l
LEFT JOIN (
    SELECT s.Building_Code, COUNT(*) as NumAudits, COUNT(DISTINCT s.Lab_Space_Id) as NumLabs
    FROM Lab_Space s
    INNER JOIN EHS_Assessment_Audit a ON s.Lab_Space_Id = a.Lab_Space_Id
    WHERE a.Audit_Date >= DATEADD(year, -1, GETDATE())
    GROUP BY s.Building_Code
) as b ON l.Building_Code = b.Building_Code

With overly simplistic temp tables and example data:
CREATE TABLE #EHS_Assessment_Audit (Lab_Space_Id int, Audit_Date datetime)
CREATE TABLE #Lab_Space (Lab_Space_Id int, Building_Code int)
CREATE TABLE #Locations (Location_Id int, Building_Code int, Site_Name nvarchar(30), Campus_Name nvarchar(30))

INSERT INTO #Locations VALUES (1, 1, 'Location1', 'Campus1'), (2, 2, 'Location2', 'Campus2')
INSERT INTO #Lab_Space VALUES (1, 1), (2, 1), (3, 2), (4, 2)
INSERT INTO #EHS_Assessment_Audit VALUES (1, '2018-10-11'), (1, '2018-09-11'), (2, '2018-10-11'), (3, '2015-10-11')

SELECT * FROM #Locations
SELECT * FROM #Lab_Space
SELECT * FROM #EHS_Assessment_Audit

SELECT
    l.Site_Name,
    l.Campus_Name,
    COALESCE(b.NumAudits, 0) as NumTotalAudits,
    COALESCE(b.NumLabs, 0) as AuditedLabs
FROM #Locations l
LEFT JOIN (
    SELECT s.Building_Code, COUNT(*) as NumAudits, COUNT(DISTINCT s.Lab_Space_Id) as NumLabs
    FROM #Lab_Space s
    INNER JOIN #EHS_Assessment_Audit a ON s.Lab_Space_Id = a.Lab_Space_Id
    WHERE a.Audit_Date >= DATEADD(year, -1, GETDATE())
    GROUP BY s.Building_Code
) as b ON l.Building_Code = b.Building_Code

SQL counts with left join


Change tempo and stretch pattern

Question: I have a drum pattern in FL studio 20. I want to multiply the tempo by factor of 2 and stretch the drum pattern by factor of two as well (so that it doesn't become 2x faster and keeps the pace).
Is it possible?
I don't want to rewrite the drum pattern.
Comment: Is this drum pattern a sequence of MIDI notes corresponding to each drum hit, or is it an audio recording of a drum pattern (or a combination of both)?
Comment: I can answer this, but I need clarification (see my above comment)
Answer: Yes. The magic is called editing.
It's simple. View the drum pattern. When it repeats after 1 bar, you'll double it to 2, 2 bars to 4 and so on.
Next, recreate the pattern AND make each duration twice as long. So durations needing 1 field become 2 fields (beats), 2 become 4 in time and so on.
Repeat for all voices.
It's easier to do than to note down here ;-)
Answer: If I understand correctly, you want to double the tempo while keeping the drum pattern at the same pace. FL Studio already does this by default, but to make sure, you can open the sampler, on the right side near the top, there is a selection next to the Time knob. Make sure it says resample, and the speed wont change as the tempo does.

Change tempo and stretch pattern


Errors with Spark 1.4.2

Question: 8 node virtual-metal cluster, with 4 nodes are used for analytics. DSE version 4.8.6, Spark version 1.4.2.  ... Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77)
Getting these errors ( repeatedly ) when running dse pyspark or dse spark:
<code>org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend: Asked to remove non-existent executor
</code>
I think this only happens when an interactive shell is used: a job seems to return results when submitted like this:
<code>$ dse spark-submit ./test.py
WARN  2016-05-05 19:21:51,614 org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
+---------+---------+-------+---------------+----------+
( results )
</code>
Answer: This was apparently a firewall issue.
I was pretty sure that I opened up all the ports here: https://docs.datastax.com/en/datastax_enterprise/4.5/datastax_enterprise/sec/secConfFirePort.html ...
.. at the time, it looked like communication was being attempted to a random port.  That wouldn't be an issue for the server sending the request, but it would be an issue for the server receiving the request ...
So I turned the firewall off, and everything worked.

Errors with Spark 1.4.2


Alternating integral convergence

Question: This test exist for alternating series:
https://en.wikipedia.org/wiki/Alternating_series_test
I want to know if there is an equivalent for integrals of this form
$$
\int\limits_a^\infty \mathrm{d}x\ f(x) \sin{x}
$$
Where $f$ is monotonically decreasing, positive, does not diverge and when x goes to $\infty$, $f$ goes to zero.
I do not have much experience with real analysis so I have no idea how to proceed. I thought of using something like Riemann integral to be able to use the test above, but had no success.
Comment: Is [this](https://math.stackexchange.com/q/141048/97236) what you are after?
Comment: I believe it does solve the problem as well, thanks.
Answer: Yes: such an integral always converges under the conditions you described.
For clarity of the argument, I'll assume that $a = 0$, but it can be adapted for any $a$. Break up the interval $[0, \infty)$ into a union:
$$[0, \infty) = [0, \pi] \cup [\pi, 2 \pi] \cup [2 \pi, 3 \pi] \cup \dots$$
Then, compare the value of the integrals on adjacent regions. Note that
\begin{align*}
\left|\int_{\pi}^{2 \pi} f(x) \sin(x) \, \textrm{d} x \right| &= \int_{\pi}^{2 \pi} \left| f(x) \sin(x) \right| \, \textrm{d} x\\
&= \int_0^{\pi} \left|f(u + \pi) [\sin(u + \pi)] \right| \, \textrm{d} u \\
&= \int_0^{\pi} \left|f(u + \pi) [-\sin(u)] \right| \, \textrm{d} u \\
&= \int_0^{\pi} f(u+ \pi) \sin(u) \, \textrm{d} u \\
& \leq \int_0^{\pi} f(u) \sin(u) \, \textrm{d} u \\
&= \left| \int_0^{\pi} f(x) \sin(x) \, \textrm{d} x  \right|
\end{align*}
and the same argument can be applied to any adjacent intervals. Thus, the integral terms $a_n := \int_{n \pi}^{(n+1)\pi} f(x) \, \textrm{d} x$ for $n \geq 0$ form a sequence that satisfies the alternating series test hypotheses.
One caveat: these integrals will converge if considered as Riemann integrals -- but as Lebesgue integrals, they may not. In order for the Lebesgue integrals to converge, you also have to guarantee that $a_0 + a_2 + a_4 + \dots$ and $a_1 + a_3 + a_5 + \dots$ each converge to a finite value.
Comment: Thanks a lot for the answer, it was very clear and only used the resources I provided (the convergence test). Cheers. I wish I could upvote, but I don't have reputation for that.

Alternating integral convergence